On the Sentence Embeddings from Pre-trained Language Models Pre-trained contextual representations like BERT have achieved great success in natural language processing However, the sentence embeddings from the pre-trained language models without fine-tuning have been found to poorly capture semantic meaning of sentences
On the Sentence Embeddings from Pre-trained Language Models Abstract Pre-trained contextual representations like BERT have achieved great success in natural language processing However, the sentence embeddings from the pre-trained language models without fine-tuning have been found to poorly capture semantic meaning of sentences
On the Sentence Embeddings from Pre-trained Language Models In this paper, we show how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised
Extracting Sentence Embeddings from Pretrained Transformer Models Abstract Background introduction: Pre-trained transformer models shine in many natural language processing tasks and therefore are expected to bear the representation of the input sentence or text meaning These sentence-level embeddings are also important in retrieval-augmented generation
On the Sentence Embeddings from Pre-trained Language Models Pre-trained contextual representations like BERT have achieved great success in natural language processing However, the sentence embeddings from the pre-trained language models without fine-tuning have been found to poorly capture semantic meaning of sentences
ACL Anthology - ACL Anthology %0 Conference Proceedings %T On the Sentence Embeddings from Pre-trained Language Models %A Li, Bohan %A Zhou, Hao %A He, Junxian %A Wang, Mingxuan %A Yang, Yiming %A Li, Lei %Y Webber, Bonnie %Y Cohn, Trevor %Y He, Yulan %Y Liu, Yang %S Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) %D 2020 %8
On the Sentence Embeddings from Pre-Trained Language Models A key enabler of deep learning for natural language processing has been the development of word embeddings One reason for this is that deep learning intrinsically involves the use of neural network models and these models only work with numeric inputs