cooscao / bert-bilstm-crf-pytorch Goto Github PK

View Code? Open in Web Editor NEW

269.0 5.0 54.0 3.67 MB

bert-bilstm-crf implemented in pytorch for named entity recognition.

License: MIT License

Python 9.74% Jupyter Notebook 90.26%

pytorch named-entity-recognition bert crf

bert-bilstm-crf-pytorch's Introduction

Bert-BiLSTM-CRF-pytorch

bert-bilstm-crf implemented in pytorch for named entity recognition.

python == 3.6
pytorch == 0.4.1
pytorch_pretrained_bert == 0.6.1

Data

首先将数据处理成BIO格式，processed文件夹下存放的是医疗命名实体识别的数据，代码可参考data_process.ipynb
下载中文BERT预训练模型,来自pytorch-pretrained-bert

模型训练

python main.py -- n_epochs 100 --finetuning --top_rnns

模型预测

python crf_predict.py

bert-bilstm-crf-pytorch's People

Contributors

Stargazers

Watchers

bert-bilstm-crf-pytorch's Issues

CUDA error

First, thank u for having shared ur code with us
I tried to execute ur code on google colab but it gave me this error even if CUDA is activated

do u have any ideas plz?
thank u

TypeError: 'NoneType' object is not callable?

您好，感谢您的代码，在crf.py中的_forward_alg函数里面有一行代码不太明白：
log_alpha = (log_sum_exp_batch(self.transitions + log_alpha, axis=-1) + feats[:, t]).unsqueeze(1)
在_forward_alg函数中的log_alpha处，为什么在函数log_sum_exp_batch的外面加feats[:, t]，感觉在pytorch官方非batch版本中，是先加feats再进行log_sum_exp的啊，batch版本有什么不同吗？谢谢您的回复~

预训练模型载入失败

请问一下作者，我已经下载了chinese_L-12_H-768_A-12的压缩包，但是解压之后里面模型名为bert_model.ckpt.data-00000-of-00001，我是应该用链接使用这个文件么？之前我指向整个chinese_L-12_H-768_A-12文件夹的时候，终端报错提示找不到model，还是说我需要重命名模型文件么

more elaborations on the whole pipeline?

Hi! :)

Your work attracted my attention, and I want to follow your approach for my use case

If you do not mind, I have questions:

Do you have any paper/article/repo/blog you depended on to create this work?
Can you breifly elaborate on your work flow, what is your pipeline, how did you order the layers and models?

Thanks

model.py文件问题

好像在整个训练过程中并没有用到model.py文件，然而finetuning的逻辑是写在model.py文件中的，所以整个训练过程并没有实现finetuning功能。不知这样理解对不对？

P,R,F值一直没变化是怎么回事？我训练到了epoch=7的时候一直都没变化，一直都是0.81，请问有跟我遇到一样问题的小伙伴吗？或者作者能给解答一下吗？

请问sent_vocab.json是根据训练集生成的，还是通用的？

bert一直报错

Model name 'model/bert-base-chinese' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese). We assumed 'model/bert-base-chinese\vocab.txt' was a path or url but couldn't find any file associated to this path or url.

感觉_score_sentence方法错了

score = score +
batch_transitions.gather(-1, (label_ids[:, t] * self.tagset_size + label_ids[:, t - 1]).view(-1, 1))
+ feats[:, t].gather(-1, label_ids[:, t].view(-1, 1)).view(-1, 1)
是否应该变为
score = score +
batch_transitions.gather(-1, (label_ids[:, t-1] * self.tagset_size + label_ids[:, t ]).view(-1, 1))
+ feats[:, t].gather(-1, label_ids[:, t].view(-1, 1)).view(-1, 1)
真实分数不应该是前一个到后一个的lable的分数吗是不是做反了

struggle with '/checkpoints/finetuning/100.pt'

How can I find '/checkpoints/finetuning/100.pt'?
Can you share model checkpoints?