NLP Attention

http://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention

Context attention - LSTM

Encoder and Decoder

class Attention(nn.Module):
    def __init__(self, method, hidden_size):
        super(Attention, self).__init__()
        self.method = method
        self.hidden_size = hidden_size

        if self.method == 'general':
            self.attention = nn.Linear(self.hidden_size, self.hidden_size)
        elif self.method …
more ...


【NLP】ELMo

content

    ELMo: Embeddings from Language models, BiLSTM vector concat,weighed hidden layers stacked

    • 与GloVe embedding最大区别,引入了上下文,contextualized word-embeddings (BERT, ELMo)

    • ELMo不会为每个单词使用固定的embedding向量,而是会在 …

    more ...

    【RL】Policy Gradient

    1. Reinforcement Learning

    • Actor(Policy)

      Neural Network as Actor (Deep). vs lookup Table(Q Learning).

    使用神经网络作为Actor比查表的优势?

    查表无法穷举输入,e.g.图像画面或者语言输入 …
    more ...


    CNN

    content

      (batch_size, height, width, channel) = (200, 32, 32, 256)

      卷积核,用于提取特定特征,由卷积核决定。网络学习的参数也就是卷积核的参数,所以相比于全连 …

      more ...

      Deploy my blog quickly

      Date Tags Blog

      1. Clone Blog

      $ git clone git@github.com:jerrylsu/blog.git

      2. Create virtual environment

      $ conda create -n blog python=3.6.8
      
      $ conda activate blog
      
      $ pip install pelican -i https://pypi.doubanio.com/simple
      
      $ pip install bs4 markdown webassets cssmin -i https://pypi.doubanio.com/simple
      

      3. Install themes …

      more ...


      【NLP】RASA Featurizer

      • 训练集stories如何构建状态state作为训练输入数据?

      • 构建的状态state作为输入X如何编码?

      • 输出y是什么?如何编码?

      https://rasa.com/docs/rasa/api …

      more ...

      【NLP】Embedding

      content
        embedding_dim=16
        
        model = keras.Sequential([
          layers.Embedding(vocab_size, embedding_dim, input_length=maxlen),
          layers.GlobalAveragePooling1D(),
          layers.Dense(16, activation='relu'),
          layers.Dense(1, activation='sigmoid')
        ])
        

        Embedding: This layer takes the integer-encoded vocabulary and looks up the embedding vector for each word-index. These vectors are learned as the …

        more ...