This repository contains the source code of the paper A Co-Attentive Cross-Lingual Neural Model for Dialogue Breakdown Detection.

If you use the source code or models from this work, please cite our paper:

@inproceedings{lin-etal-2020-cxm,
  author    = "Lin, Qian and Kundu, Souvik and Ng, Hwee Tou",
  title     = "A Co-Attentive Cross-Lingual Neural Model for Dialogue Breakdown Detection",
  booktitle = "Proceedings of COLING",
  year      = "2020",
}

Install the packages listed in the requirements.txt file.

pip install -r requirements.txt

Install allennlp

bash install_allennlp.sh

Refer to data/README.md for instructions of data downloading and preprocessing.

The processed data files will be located at data/en and data/jp for English track and Japanese track, respectively.

We provide training configuration files in training_configs. Modify the paths to data inside the configuration files.

For English track:

allennlp train -s models/en_cxm_d --include-package cxm training_configs/en_cxm_d.json

Similarly for Japanese track:

allennlp train -s models/jp_cxm_d --include-package cxm training_configs/jp_cxm_d.json

We provide trained models. They can be downloaded by running bash download_trained_models.sh.

model_dir = "en_cxm_d"
allennlp predict models/$model_dir/model.tar.gz data/en/eval.jsonl \
                    --output-file models/$model_dir/eval_pred.jsonl \
                    --batch-size 2 \
                    --cuda-device 0 \
                    --predictor cxm_predictor \
                    --include-package cxm \
                    --silent

The evaluation script will be downloaded during the process of data downloading and preprocessing.

First, convert prediction file to seperate json files:

cd evaluation
model_dir = "en_cxm_d"
python convert_predictions_to_files.py --eval_file ../models/$model_dir/eval_pred.jsonl

Then run the evaluation script:

python2 eval_script/eval.py -t 0.0 -p ../data/en/eval_all/ -o pred_label_files/labels_$model_dir

The code and models in this repository are licensed under the GNU General Public License Version 3. For commercial use of this code and models, separate commercial licensing is also available. Please contact:

• Qian Lin ([email protected])
• Souvik Kundu ([email protected])
• Hwee Tou Ng ([email protected])