【NLP】ELMo

Reason is the light and the light of life.

Jerry Su Mar 24, 2020 Mar 24, 2020 1 mins
nlp deep learning nlp elmo

ELMo: Embeddings from Language models, BiLSTM vector concat，weighed hidden layers stacked

  • 与GloVe embedding最大区别，引入了上下文，contextualized word-embeddings (BERT, ELMo)

  • ELMo不会为每个单词使用固定的embedding向量，而是会在为每个单词分配embedding之前先查看整个句子(上下文)。

elmo1

elmo2

http://jalammar.github.io/illustrated-bert


Read more:
Related posts: