Codes and corpora for paper "GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge" (EMNLP 2019)
--> Now the checkpoint of GlossBERT (Sent_CLS_WS) is available at: https://drive.google.com/file/d/1iq_h3zLTflraEU_7tVLnPcVQTeyGDNKE/view?usp=sharing.
Recently we have received several requests for the checkpoint of our best model in the paper. Thanks for their attention to our work. As they mentioned in the email, it is quite expensive to train the model. However, the original checkpoint implemented more than half a year ago was lost during the upgrade of the server. Thus, we spend some time to train a new one under the same hyperparameters recent days.
Note: the the GPUs on the server were changed from Tesla V100-PCIE to Tesla V100-SXM2, so the results of this checkpoint on the evaluation datasets might be slightly different (but certainly comparable) from the original version in our paper:
| SE07 | SE2 | SE3 | SE13 | SE15 | ALL (4 test sets) | |
|---|---|---|---|---|---|---|
| original version | 72.5 | 77.7 | 75.2 | 76.1 | 80.4 | 77.0 |
| this checkpoint | 72.1 | 77.7 | 75.9 | 76.8 | 79.3 | 77.2 |
- pytorch: 1.0.0
- python: 3.7.1
- tensorflow: 1.13.1 (only needed for converting BERT-tensorflow-model to pytorch-model)
- numpy: 1.15.4
We generate datasets for GlossBERT based on the evaluation framework of Raganato et al. and WordNet 3.0.
Run following commands to prepare datasets for tasks and extract vocabulary information from ./wordnet/index.sense (if you only need the processed datasets, download here):
bash preparation.sh
Then for each dataset, there are 6 files in the directory. Take Semcor as an example:
./Training_Corpora/SemCor:
semcor.csv
semcor.data.xml
semcor.gold.key.txt
semcor_train_sent_cls.csv
semcor_train_sent_cls_ws.csv
semcor_train_token_cls.csv
semcor.data.xmlandsemcor.gold.key.txtcome from the evaluation framework of Raganato et al.semcor.csvis generated fromsemcor.data.xmlby us, which is used to generate other files and is the dataset forexp-BERT(Token-CLS).semcor_train_sent_cls.csv,semcor_train_sent_cls_ws.csvandsemcor_train_token_cls.csvare datasets forexp-GlossBERT(Sent-CLS),exp-GlossBERT(Sent-CLS-WS)andexp-GlossBERT(Token-CLS)respectively.
Besides, four .pkl files have been generated in directory:./wordnet/ , which are need in exp-BERT(Token-CLS).
Download BERT-Base-uncased model and then run following commands to convert a tensorflow checkpoint to a pytorch model:
python convert_tf_checkpoint_to_pytorch.py \
--tf_checkpoint_path bert-model/uncased_L-12_H-768_A-12/bert_model.ckpt \
--bert_config_file bert-model/uncased_L-12_H-768_A-12/bert_config.json \
--pytorch_dump_path bert-model/uncased_L-12_H-768_A-12/pytorch_model.bin
For example, for exp-GlossBERT(Sent-CLS-WS):
CUDA_VISIBLE_DEVICES=0,1,2,3 python run_classifier_WSD_sent.py \
--task_name WSD \
--train_data_dir ./Training_Corpora/SemCor/semcor_train_sent_cls_ws.csv \
--eval_data_dir ./Evaluation_Datasets/semeval2007/semeval2007_test_sent_cls_ws.csv \
--output_dir results/sent_cls_ws/1314 \
--bert_model ./bert-model/uncased_L-12_H-768_A-12/ \
--do_train \
--do_eval \
--do_lower_case \
--max_seq_length 512 \
--train_batch_size 64 \
--eval_batch_size 128 \
--learning_rate 2e-5 \
--num_train_epochs 6.0 \
--seed 1314
See more examples for other experiments in commands.txt.
For example, for exp-GlossBERT(Sent-CLS-WS):
CUDA_VISIBLE_DEVICES=0,1,2,3 python run_classifier_WSD_sent.py \
--task_name WSD \
--eval_data_dir ./Evaluation_Datasets/senseval3/senseval3_test_sent_cls_ws.csv \
--output_dir results/sent_cls_ws/1314/4 \
--bert_model results/sent_cls_ws/1314/4 \
--do_test \
--do_lower_case \
--max_seq_length 512 \
--train_batch_size 64 \
--eval_batch_size 128 \
--learning_rate 2e-5 \
--num_train_epochs 6.0 \
--seed 1314
See more examples for other experiments in commands.txt.
Refer to ./Evaluation_Datasets/README provided by Raganato et al. .
First, you need to convert the output file to make sure its format is the same as the gold key file. You can use code like:
# GlossBERT_sent_cls or GlossBERT_sent_cls_ws or GlossBERT_token_cls
python convert_result_token_sent.py \
--dataset semeval2007 \
--input_file ./results/results.txt \
--output_dir ./results/
# BERT_baseline
python convert_result_baseline.py \
--dataset semeval2007 \
--input_file ./results/results.txt \
--output_dir ./results/
Then, you can use the Scorer provided by Raganato et al. to do evaluation.
Example of usage:
$ javac Scorer.java
$ java Scorer ./Evaluation_Datasets/semeval2007/semeval2007.gold.key.txt ./results/final_result_semeval2007.txt
@inproceedings{huang-etal-2019-glossbert,
title = "{G}loss{BERT}: {BERT} for Word Sense Disambiguation with Gloss Knowledge",
author = "Huang, Luyao and
Sun, Chi and
Qiu, Xipeng and
Huang, Xuanjing",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)",
month = nov,
year = "2019",
address = "Hong Kong, China",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/D19-1355",
doi = "10.18653/v1/D19-1355",
pages = "3507--3512"
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent