Chinese clinical named entity recognition (CNER) using pre-trained BERT model

Introduction

We pre-trained BERT model to improve the performance of Chinese CNER. Different layers such as Long Short-Term Memory (LSTM) and Conditional Random Field (CRF) were used to extract the text features and decode the predicted tags respectively. And we also proposed a new strategy to incorporate dictionary features into the model. Radical features of Chinese characters were also used to improve the model performance.

Model structure

Usage

Examples

To replicate the result of CCKS-2018 dataset

python main.py \
--data_dir=data/ccks_2018 \
--bert_model=model/  \
--output_dir=./output  \
--terminology_dicts_path="{'medicine':'data/ccks_2018/drug_dict.txt','surgery':'data/ccks_2018/surgery_dict.txt'}" \
--radical_dict_path data/radical_dict.txt \
--constant=0 \
--add_radical_or_not=True \
--radical_one_hot=False \
--radical_emb_dim=20 \
--max_seq_length=480 \
--do_train=True \
--do_eval=True \
--train_batch_size=6 \
--eval_batch_size=4 \
--hidden_dim=64 \
--learning_rate=5e-5 \
--num_train_epochs=5 \
--gpu_id=3 \

Results

CCKS-2018 dataset

Method	P	R	F1
FT-BERT+BiLSTM+CRF	88.57	89.02	88.80
+dictionary	88.58	89.17	88.87
+radical(one-hot encoding)	88.51	89.39	88.95
+radical(random embedding)	89.24	89.11	89.17
+dictionary +radical	89.42	89.22	89.32
ensemble	89.59	89.54	89.56

Team Name	Method	F1
Yang and Huang (2018)	CRF(feature-rich + rule)	89.26
heiheihahei	LSTM-CRF(ensemble)	88.92
Luo et al.(2018)	LSTM-CRF(ensemble)	88.63
dous12	-	88.37
chengachengcheng	-	88.30
NUBT-IBDL	-	87.62
Our	FT-BERT+BiLSTM +CRF+Dictionary(ensemble)	89.56

CCKS-2017 dataset

Method	P	R	F1
FT-BERT+BiLSTM+CRF	91.64	90.98	91.31
+dictionary	91.49	90.97	91.23
+radical(one-hot encoding)	91.83	90.80	91.35
+radical(random embedding)	92.07	90.77	91.42
+dictionary+radical	91.76	90.88	91.32
ensemble	92.06	91.15	91.60

Team Name	Method	F1
Qiu et al. (2018b)	RD-CNN-CRF	91.32
Wang et al. (2019)	BiLSTM-CRF+Dictionary	91.24
Hu et al. (2017)	BiLSTM-FEA(ensemble)	91.03
Zhang et al. (2018)	BiLSTM-CRF(mt+att+ms)	90.52
Xia and Wang (2017)	BiLSTM-CRF(ensemble)	89.88
Ouyang et al. (2017)	BiRNN-CRF	88.85
Li et al. (2017)	BiLSTM-CRF(specialized +lexicons)	87.95
Our	FT-BERT+BiLSTM +CRF+Dictionary(ensemble)	91.60

yovoss / bertcner Goto Github PK

bertcner's Introduction

Chinese clinical named entity recognition (CNER) using pre-trained BERT model

Introduction

Model structure

Usage

Examples

Results

CCKS-2018 dataset

CCKS-2017 dataset

bertcner's People

Contributors

Watchers

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent