This repository implements the training and testing of T2Net for "T2Net: Synthetic-to-Realistic Translation for Depth Estimation Tasks" by Chuanxia Zheng, Tat-Jen Cham and Jianfei Cai at NTU. A video is available on YouTube. The repository offers the original implementation of the paper in Pytoch.

• Outdoor Translation

• Indoor Translation

This repository can be used for training and testing of

• Unpaired image-to-image Translation
• Single depth Estimation

This code was tested with Pytoch 0.4.0, CUDA 8.0, Python 3.6 and Ubuntu 16.04

• Install Pytoch 0.4, torchvision, and other dependencies from http://pytorch.org
• Install python libraries visdom and dominate for visualization

pip install visdom dominate

• Clone this repo:

git clone https://github.com/lyndonzheng/Synthetic2Realistic
cd Synthetic2Realistic

The indoor Synthetic Dataset renders from SUNCG and indoor Realistic Dataset comes from NYUv2. The outdooe Synthetic Dataset is vKITTI and outdoor Realistic dataset is KITTI

Warning: The input sizes need to be muliples of 64. The feature GAN model needs to be change for different scale

• Train a model with multi-domain datasets:

python train.py train.py --name Outdoor_nyu_wsupervised --model wsupervised
--img_source_file /dataset/Image2Depth31_KITTI/trainA_SYN80.txt
--img_target_file /dataset/Image2Depth31_KITTI/trainA.txt
--lab_source_file /dataset/Image2Depth31_KITTI/trainB_SYN80.txt
--lab_target_file /dataset/Image2Depth31_KITTI/trainB.txt
--shuffle --flip --rotation

• To view training results and loss plots, run python -m visdom.server and copy the URL http://localhost:8097.
• Training results will be saved under the checkpoints folder. The more training options can be found in options.

• Test the model

python test.py --name Outdoor_nyu_wsupervised --model test
--img_source_file /dataset/Image2Depth31_KITTI/testA_SYN80
--img_target_file /dataset/Image2Depth31_KITTI/testA

More trained models will be released

If you use this code for your research, please cite our papers.

@article{zheng2018t2net,
  title={T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks},
  author={Zheng, Chuanxia and Cham, Tat-Jen and Cai, Jianfei},
  journal={arXiv preprint arXiv:1808.01454},
  year={2018}
}

Code is inspired by Pytorch-CycleGAN