A high-accuracy incremental parser implemented in Python. Based on Learned Incremental Representations for Parsing from ACL 2022, which is built upon the works of Constituency Parsing with a Self-Attentive Encoder from ACL 2018, and with additional changes described in Multilingual Constituency Parsing with Self-Attention and Pre-Training.

This notebook gives an example of inference using the model we trained for our paper.

1. Training
2. Reproducing Experiments
3. Citation
4. Credits

If you are primarily interested in training your own parsing models, skip to the Training section of this README.

Training requires cloning this repository from GitHub. While the model code in src/benepar is based on the benepar package on PyPI, much of the related code and training and evaluation scripts directly under src/ are not.

• Python 3.7 or higher.
• PyTorch 1.6.0, or any compatible version.
• All dependencies required by the benepar package, including: NLTK 3.2, torch-struct 0.4, transformers 4.3.0, or compatible.
• pytokenizations 0.7.2 or compatible.
• EVALB. Before starting, run make inside the EVALB/ directory to compile an evalb executable. This will be called from Python for evaluation. If training on the SPMRL datasets, you will need to run make inside the EVALB_SPMRL/ directory instead.
• clusopt
• torch_struct

A new model can be trained using the command python src/main.py train .... Some of the available arguments are:

Additional arguments are available for other hyperparameters; see make_hparams() in src/main.py. These can be specified on the command line, such as --num-layers 2 (for numerical parameters), --predict-tags (for boolean parameters that default to False), or --no-XXX (for boolean parameters that default to True).

For each development evaluation, the F-score on the development set is computed and compared to the previous best. If the current model is better, the previous model will be deleted and the current model will be saved. The new filename will be derived from the provided model path base and the development F-score.

Prior to training the parser, you will first need to obtain appropriate training data. We provide instructions on how to process standard datasets like PTB, CTB, and the SMPRL 2013/2014 Shared Task data. After following the instructions for the English WSJ data, you can use the following command to train an English parser using the default hyperparameters:

python src/main.py train --use-pretrained --model-path-base models/en_bert_base

See EXPERIMENTS.md for more examples of good hyperparameter choices.

A saved model can be evaluated on a test corpus using the command python src/main.py test ... with the following arguments:

As an example, you can evaluate a trained model using the following command:

python src/main.py test --model-path models/en_bert_base_dev=*.pt

See EXPERIMENTS.md for instructions on how to reproduce experiments reported in our ACL 2022 paper.

Our paper can be cited as follows:

@inproceedings{kitaev-etal-2022-learned,
    title = "Learned Incremental Representations for Parsing",
    author = "Kitaev, Nikita  and
      Lu, Thomas  and
      Klein, Dan",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.220",
    pages = "3086--3095",
    abstract = "We present an incremental syntactic representation that consists of assigning a single discrete label to each word in a sentence, where the label is predicted using strictly incremental processing of a prefix of the sentence, and the sequence of labels for a sentence fully determines a parse tree. Our goal is to induce a syntactic representation that commits to syntactic choices only as they are incrementally revealed by the input, in contrast with standard representations that must make output choices such as attachments speculatively and later throw out conflicting analyses. Our learned representations achieve 93.72 F1 on the Penn Treebank with as few as 5 bits per word, and at 8 bits per word they achieve 94.97 F1, which is comparable with other state of the art parsing models when using the same pre-trained embeddings. We also provide an analysis of the representations learned by our system, investigating properties such as the interpretable syntactic features captured by the system and mechanisms for deferred resolution of syntactic ambiguities.",
}

The code in this repository and portions of this README are based on https://github.com/nikitakit/self-attentive-parser