OpinionDigest
This repository contains the codebase of our paper OpinionDigest A Simple Framework for Opinion Summarization accepted at ACL 2020.
OpinionDigest is an unsupervised opinion summarization framework that generates a summary from multiple review documents without requiring any gold-standard summary. OpinionDigest relies on an aspect-based sentiment classification model, which extracts opinions from input reviews, to train a seq2seq model that generates a summary from a set of opinions. This framework enables the user to easily control the output by filtering input opinions using aspects and/or sentiment polarity.
Please see our paper for details. Please also try our online demo.
This project is a collaboration with the Natural Language Processing Group at the University of Edinburgh (EdinburghNLP).
Installation
$ git clone https://github.com/megagonlabs/opiniondigest.git
$ cd opiniondigest
$ pip install -r requirements.txt
(Optional) To download data and pre-trained model files, run the command below.
$ ./download.sh
Preprocessing
To run our framework, you need to preprocess your dataset and extract opinion phrases from the input sentences. You can use any ABSA models to perform the extraction task.
For our experiment on Yelp dataset, we used this extractor (Snippext) to extract opinions from the reviews.
Please follow our example to format your extraction files.
Running
The workflow has following 4 steps. You can configure the settings of each step by creating JSON file.
- Step 1. Data preparation
- Step 2. Training
- Step 3. Aggregation
- Step 4. Generation
You can skip Steps 1-3 by downloading our pre-trained model and dataset.
Step 1. Data preparation
$ python src/prepare.py \
config/prepare_{p_name}.json
The script will create training/development/test datasets.
$ ls data/{p_name}
train.csv
dev.csv
test.csv
Step 2. Training
To train a model, run the following script with configurations files for preparation and training.
$ python src/train.py \
config/prepare_{p_name}.json \
config/train_{t_name}.json
The training script saves following model files.
$ ls model
{p_name}_op2text_{t_name}.model
{p_name}_op2text_{t_name}_IN_TEXT.field
{p_name}_op2text_{t_name}_OUT_TEXT.field
{p_name}_op2text_{t_name}_ID.field
Step 3. Aggregation
$ python src/aggregate.py \
config/aggregate_{a_name}.json \
config/prepare_{p_name}.json \
config/train_{t_name}.json
The script generates following three files.
$ ls data/{p_name}
train_{a_name}.csv
dev_{a_name}.csv
test_{a_name}.csv
Here p_name
needs to be specified in the configuration file aggregate_{a_name}.json
.
a_name
is generated from the parameters _n_k_att_pol
:
n
is the number of reviews;k
is the top-k frequent extractions;att
is the attribute of the extractions;pol
is the sentiment polarity of the extractions.
Step 4. Generation
To generate summaries using OpinionDigest model, run the command below:
$ python src/generate.py \
config/prepare_{p_name}.json \
config/train_{t_name}.json \
config/aggregate_{a_name}.json \
config/generate_{g_name}.json
This will creates following output and log files.
$ ls output
{p_name}_op2text_{t_name}_{g_name}.csv
{p_name}_op2text_{t_name}_{g_name}.log
Evaluation
$ python src/evaluate.py \
config/prepare_yelp-default.json \
config/train_tiny.json \
config/aggregate_default.json \
config/generate_beam.json
$ cat output/default_op2text_default_greedy.eval
bleu,0.06273896942347468
rouge_1,0.4166751301989266
rouge_2,0.1566288226261539
rouge_l,0.2938877832779797
Citation
@misc{suhara2020opiniondigest,
title={{OpinionDigest}: A Simple Framework for Opinion Summarization},
author={Yoshihiko Suhara and Xiaolan Wang and Stefanos Angelidis and Wang-Chiew Tan},
year={2020},
eprint={2005.01901},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
*The first two authors contributed equally.