Codes for our paper Uncertainty-Aware Sequence Labeling and Leveraging Document-Level Label Consistency for Named Entity Recognition.
Python: 3.6 or higher.
PyTorch 1.0 or higher.
Download Glove embedding from here.
We use standard CoNLL format with each character and its label split by a whitespace in a line. The "BMES" tag scheme is prefered.
Make sure to use -DOCSTART- to indicate the begining of a document.
A example from CoNLL2003 (additional pos/chunk features are not used in our experiments):
-DOCSTART- -X- -X- O
EU NNP B-NP S-ORG
rejects VBZ B-VP O
German JJ B-NP S-MISC
call NN I-NP O
to TO B-VP O
boycott VB I-VP O
British JJ B-NP S-MISC
lamb NN I-NP O
. . O O
Peter NNP B-NP B-PER
Blackburn NNP I-NP E-PER
Run without document-level memory:
CUDA_VISIBLE_DEVICES=0 python main.py --train_dir 'data/conll2003/train.txt' --dev_dir 'data/conll2003/dev.txt' --test_dir 'data/conll2003/test.txt' --model_dir 'outs' --word_emb_dir 'data/glove.6B.100d.txt --use_memory False'
Run with document-level memory:
CUDA_VISIBLE_DEVICES=0 python main.py --train_dir 'data/conll2003/train.txt' --dev_dir 'data/conll2003/dev.txt' --test_dir 'data/conll2003/test.txt' --model_dir 'outs' --word_emb_dir 'data/glove.6B.100d.txt'
Run without document-level memory:
CUDA_VISIBLE_DEVICES=0 python main.py --status decode --model_dir <model dir> --raw_dir <file to be predicted> --use_memory False
Run with document-level memory:
CUDA_VISIBLE_DEVICES=0 python main.py --status decode --model_dir <model dir> --raw_dir <file to be predicted>
We upload a model trained on CoNLL2003 dataset here.
If you use our code, please cite our paper as follows:
@inproceedings{gui2020leveraging,
author = {Gui, Tao and Ye, Jiacheng and Zhang, Qi and Zhou, Yaqian and Gong, Yeyun and Huang, Xuanjing},
title = {{Leveraging Document-Level Label Consistency for Named Entity Recognition}},
publisher = {International Joint Conferences on Artificial Intelligence Organization},
year = {2020}
}
@article{gui2021uncertainty,
title={Uncertainty-Aware Sequence Labeling},
author={Gui, Tao and Ye, Jiacheng and Zhou, Xiang and Zheng, Xiaoqing and Zhang, Qi},
journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
year={2021},
publisher={IEEE}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent