cmrc2018 dataset
Convert pytorch checkpoint to tensorflow2
https://github.com/huggingface/transformers/issues/6124
from transformers import TFBertModel
model = TFBertModel.from_pretrained("./rubert-base-cased-pt", from_pt=True)
model.save("./rubert-base-cased") # this adds a TF model file (tf_model.h5) to your directory
CMRC Metric
F1
# find longest common string
def find_lcs(s1, s2):
m = [[0 for i in range(len(s2)+1)] for j in range(len(s1)+1)]
mmax = 0
p = 0
for i in range(len(s1)):
for j in range(len(s2)):
if s1[i] == s2[j]:
m[i+1 …
JDMDC2020
2020京东多模态对话JDMDC2020第二名解决方案
CCL2020: 中国计算语言学大会(CCL 2020)技术评测结果
JDMDC2020 RANK:https://jddc.jd.com/rank
Code: JDMDC2020-Solution-2nd
Hibot团队成绩:初赛 …
more ...Deploy model in production
Install wsgi using code
git clone https://github.com/GrahamDumpleton/mod_wsgi
apt-get install apache2-dev
apt-get install python-dev
cd mod_wsgi/
./configure
make
make install
Install wsgi using pip
apt-get install libapache2-mod-wsgi-py3
instead of libapache2-mod-wsgi
for python3
Load wsgi module in Ubuntu
How do I use a …
more ...NLP Resources
Pytorch Transformer
content
import math
import torch
import torch.nn as nn
from torch.nn import TransformerEncoder, TransformerEncoderLayer, TransformerDecoder, TransformerDecoderLayer
from layers.encoder import CustomEmbedding
import layers
class PositionalEncoding(nn.Module):
"""
Args:
d_model: the number of expected features in the input (required).
dropout: the dropout value (default …
Pytorch contiguous
【NLP】Teacher Forcing
content
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Teacher-Forcing 技术在训练前期的确是能够很大的加速模型收敛的:
模型在训练过程中的每一个时间步steps,有 …
more ...Pytorch distributed train
主卡线程暴涨
异常:
正常:
def to_var(x, on_cpu=False, gpu_id=None):
"""Tensor => Variable"""
if torch.cuda.is_available() and not on_cpu:
x = x.cuda(gpu_id, non_blocking=True)
# x = Variable(x)
return x
def normal_kl_div(mu1, var1 …