Trajectron++ with Social-NCE
This is an official implementation of the Social-NCE applied to the Trajectron++ forecasting model.
Social NCE: Contrastive Learning of Socially-aware Motion Representations
by
Yuejiang Liu,
Qi Yan,
Alexandre Alahi at
EPFL
to appear at ICCV 2021
TL;DR: Contrastive Representation Learning + Negative Data Augmentations ๐กฒ Robust Neural Motion Models
- Rank in 1st place on the Trajnet++ challenge since November 2020 to present
- Significantly reduce the collision rate of SOTA human trajectroy forecasting models
- SOTA on imitation / reinforcement learning for autonomous navigation in crowds
Please check out our code for experiments on different models as follows:
Social NCE + Trajectron | Social NCE + STGCNN | Social NCE + CrowdNav
Preparation
Setup environments follwoing the SETUP.md
Training & Evaluation
To train a model on the ETH / UCY Pedestrian datasets, the dataset needs to be specified, e.g.,
DATASET=univ
Baseline
The vanilla Trajectron++ model can be trained and evaluated as follows:
bash scripts/run_train.sh ${DATASET} 0.0 && bash scripts/run_eval.sh ${DATASET} 0.0
Contrastive
To train the Trajectron++ with Social-NCE, run the following command:
bash scripts/run_train.sh ${DATASET} && bash bash scripts/run_eval.sh ${DATASET}
Comparison
To search for hyper-parameters on different datasets, run the following bash scripts:
bash scripts/run_<dataset>.sh
Our pre-trained models can be downloaded as follows:
gdown https://drive.google.com/uc?id=1APAIlgJS9BDZHFCvwMrfzj9z_9DSS6LB
unzip pretrained_trajectron++.zip -d experiments/pedestrians/models
To compare different models, run the following command:
python benchmark.py --dataset ${DATASET}
Basic Results
The scripts above yield the following results (on GeForce RTX 3090). The result may subject to mild variance on different GPU devices.
On average, our method reduces the collision rate of the Trajectron++ by over 45%, without degrading its performance in terms of prediction accuracy and diversity.
Epoch | Trajectron++ w/o Ours | Trajectron++ w/ Ours | ||||
---|---|---|---|---|---|---|
ADE | FDE | COL | ADE | FDE | COL | |
ETH | 0.388 | 0.810 | 1.156 | 0.386 | 0.791 | 0.000 |
HOTEL | 0.110 | 0.184 | 0.837 | 0.107 | 0.177 | 0.381 |
UNIV | 0.199 | 0.450 | 3.378 | 0.195 | 0.435 | 3.079 |
ZARA1 | 0.148 | 0.320 | 0.462 | 0.150 | 0.330 | 0.178 |
ZARA2 | 0.114 | 0.250 | 1.027 | 0.114 | 0.255 | 0.993 |
Average | 0.192 | 0.403 | 1.372 | 0.191 | 0.398 | 0.926 |
Citation
If you find this code useful for your research, please cite our paper:
@article{liu2020snce,
title = {Social NCE: Contrastive Learning of Socially-aware Motion Representations},
author = {Yuejiang Liu and Qi Yan and Alexandre Alahi},
journal = {arXiv preprint arXiv:2012.11717},
year = {2020}
}
Acknowledgement
Our code is developed upon the official implementation of Trajectron++.