Pytorch implementation of our method for adapting semantic segmentation from the synthetic dataset (source domain) to the real dataset (target domain). Based on this implementation, our result is ranked 3rd in the VisDA Challenge.
Contact: Yi-Hsuan Tsai (wasidennis at gmail dot com) and Wei-Chih Hung (whung8 at ucmerced dot edu)
Learning to Adapt Structured Output Space for Semantic Segmentation
Yi-Hsuan Tsai*, Wei-Chih Hung*, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang and Manmohan Chandraker
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018 (spotlight) (* indicates equal contribution).
Please cite our paper if you find it useful for your research.
@inproceedings{Tsai_adaptseg_2018,
author = {Y.-H. Tsai and W.-C. Hung and S. Schulter and K. Sohn and M.-H. Yang and M. Chandraker},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
title = {Learning to Adapt Structured Output Space for Semantic Segmentation},
year = {2018}
}
-
Install PyTorch from http://pytorch.org with Python2 and CUDA 8.0
- Note: This repo was trained with PyTorch0.2, the training code is currently not compatible with PyTorch0.4 (testing/evaluation is fine). We will migrate the code to PyTorch0.4 soon.
-
Clone this repo
git clone https://github.com/wasidennis/AdaptSegNet
cd AdaptSegNet
-
Download the GTA5 Dataset as the source domain, and put it in the
data/GTA5
folder -
Download the Cityscapes Dataset as the target domain, and put it in the
data/Cityscapes
folder
-
Download the pre-trained multi-level GTA5-to-Cityscapes model and put it in the
model
folder -
Test the model and results will be saved in the
result
folder
python evaluate_cityscapes.py --restore-from ./model/GTA2Cityscapes_multi-ed35151c.pth
- Or, test the VGG-16 based model Model Link
python evaluate_cityscapes.py --model DeeplabVGG --restore-from ./model/GTA2Cityscapes_vgg-ac4ac9f6.pth
- Compute the IoU on Cityscapes (thanks to the code from VisDA Challenge)
python compute_iou.py ./data/Cityscapes/data/gtFine/val result/cityscapes
- Train the GTA5-to-Cityscapes model (multi-level)
python train_gta2cityscapes_multi.py --snapshot-dir ./snapshots/GTA2Cityscapes_multi \
--lambda-seg 0.1 \
--lambda-adv-target1 0.0002 --lambda-adv-target2 0.001
- Train the GTA5-to-Cityscapes model (single-level)
python train_gta2cityscapes_multi.py --snapshot-dir ./snapshots/GTA2Cityscapes_single \
--lambda-seg 0.0 \
--lambda-adv-target1 0.0 --lambda-adv-target2 0.001
- W.-C. Hung, Y.-H Tsai, Y.-T. Liou, Y.-Y. Lin, and M.-H. Yang. Adversarial Learning for Semi-supervised Semantic Segmentation. In ArXiv, 2018. [paper] [code]
- Y.-H. Chen, W.-Y. Chen, Y.-T. Chen, B.-C. Tsai, Y.-C. Frank Wang, and M. Sun. No More Discrimination: Cross City Adaptation of Road Scene Segmenters. In ICCV 2017. [paper] [project]
This code is heavily borrowed from Pytorch-Deeplab.
The model and code are available for non-commercial research purposes only.
- 07/23/2018: update evaluation code for PyTorch 0.4
- 06/04/2018: update pretrained VGG-16 model
- 02/2018: code released