This repository contains the code for the paper: "Investigating Pretrained Language Models for Graph-to-Text Generation".

This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.

This project is implemented using the framework HuggingFace. Please, refer to their websites for further details on the installation and dependencies.

• python 3.6
• transformers 3.3.1
• pytorch-lightning 0.9.0
• torch 1.4.0
• parsimonious 0.8.1

In our experiments, we use the following datasets: AMR17, WebNLG and AGENDA.

First, convert the dataset into the format required for the model.

For the AMR17, run:

./preprocess_AMR.sh <dataset_folder>

For the WebNLG, run:

./preprocess_WEBNLG.sh <dataset_folder>

For the AGENDA, run:

./preprocess_AGENDA.sh <dataset_folder>

For finetuning the models using the AMR dataset, execute:

./finetune_AMR.sh <model> <gpu_id>

For the WebNLG dataset, execute:

./finetune_WEBNLG.sh <model> <gpu_id>

For the AGENDA dataset, execute:

./finetune_AGENDA.sh <model> <gpu_id>

Options for <model> are t5-small, t5-base, t5-large, facebook/bart-base or facebook/bart-large.

Example:

./finetune_AGENDA.sh t5-small 0

For decoding, run:

./decode_AMR.sh <model> <checkpoint> <gpu_id>
./decode_WEBNLG.sh <model> <checkpoint> <gpu_id>
./decode_AGENDA.sh <model> <checkpoint> <gpu_id>

Example:

./decode_WEBNLG.sh t5-base webnlg-t5-base.ckpt 0

* BLEU values for AMR17 are calculated using sacreBLEU in detok outputs. BLEU values for WebNLG are calculated using tok outputs using the challange's script, that uses multi-bleu.perl.

For more details regading hyperparameters, please refer to HuggingFace.

Contact person: Leonardo Ribeiro, [email protected]

@misc{ribeiro2020investigating,
      title={Investigating Pretrained Language Models for Graph-to-Text Generation}, 
      author={Leonardo F. R. Ribeiro and Martin Schmitt and Hinrich Schütze and Iryna Gurevych},
      year={2020},
      eprint={2007.08426},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}