This repository contains the official Pytorch implementation of the paper

Contrastive Losses Are Natural Criteria for Unsupervised Video Summarization

Zongshang Pang, Yuta Nakashima, Mayu Otani, Hajime Nagahara

In WACV2023

git clone https://github.com/pangzss/pytorch-CTVSUM.git
cd pytorch-CTVSUM
conda env create -f environment.yml
conda activate ctvsum

We use three datasets in our paper: TVSum, SumMe, and a random subset of Youtube8M.

TVSum and SumMe are used for training and evaluation, and Youtube8M is only used for training.

To prepare the datasets,

1. Download raw videos from TVSum and SumMe and put them in ./data/raw
2. Download the extracted features from GoogleDrive (GoogLeNet features for TVSum and SumMe, kindly provided by the authors of DRDSN, and quantized Inception features for Youtube8M).
3. Put eccv_* files in ./data/interim, and unzip selected_features.zip in ./data/interim/youtube8M/

1. In ./configs/aln_unif_config.yml, modify the dataset and evaluation settings, e.g.

data:
  name: tvsum # summe
  setting: Canonical # Augmented/Transfer

2. Set

is_raw: True

3. Set use Global Consistency or not

use_unif: True # False

4. For Youtube8M features (quantized Inception), in ./configs/aln_unif_y8_config.yml, set

is_raw: True
hparams:
  use_unif: True # False

5. Run

./run_ablation.sh
./run_y8.sh

1. For TVSum and SumMe, set training/evaluation setting in ./configs/aln_unif_config.yml, and decide whether to use global consistency or uniqueness filter

is_raw: False
use_unif: True # False
use_unq: True # False

The code will run 5-fold cross validation by default.

2. For Youtube8M, similarly in ./configs/aln_unif_y8_config.yml,

is_raw: False
hparams:
  use_unif: True # False
  use_unq: True # False

3. Run

./run_ablation.sh
./run_y8.sh

We would like to thank DRDSN, which provides the extracted features and the evaluation code for TVSum and SumMe. Moreover, we are thankful to the insightful work Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere, which inspired our work.

@inproceedings{pang2023contrastive,
  title={Contrastive Losses Are Natural Criteria for Unsupervised Video Summarization},
  author={Pang, Zongshang and Nakashima, Yuta and Otani, Mayu and Nagahara, Hajime},
  booktitle={WACV},
  year={2023}
}