- 2024.07.06:
The code will be uploaded in a few weeks. - 2024.07.17: The paper and supplementary materials are available now.
- 2024.07.22: The code is available now. Video demos are coming soon.
Our code is based on CUDA 11.7 and PyTorch 2.0.1. We recommend using Anaconda to create a new environment:
conda create -n tcstereo python=3.8
conda activate tcstereo
Then, install the dependencies:
pip install -r requirements.txt
We use the TartanAir, SceneFlow, and Raw data of KITTI datasets in our experiments. Please download the datasets and organize them as follows:
datasets
├── TartanAir
│ ├── abandonedfactory
│ │ ├── abandonedfactory
│ │ │ ├── Easy
│ │ │ │ ├── P000
│ │ │ │ │ ├── depth_left
│ │ │ │ │ ├── image_left
│ │ │ │ │ ├── image_right
│ │ │ │ │ ├── pose_left.txt
│ │ │ │ ├── ...
│ │ │ ├── Hard
│ │ │ │ ├── ...
│ ├── ...
├── Driving
│ ├── frames_cleanpass
│ ├── frames_finalpass
│ ├── disparity
│ ├── pose
├── FlyingThings3D
│ ├── frames_cleanpass
│ ├── frames_finalpass
│ ├── disparity
│ ├── pose
├── Monkaa
│ ├── frames_cleanpass
│ ├── frames_finalpass
│ ├── disparity
│ ├── pose
├── kitti_raw # training sequences & pesudo labels, the pose files are generated by core/utils/preprocess_kitti_raw_pose.py
│ ├── 2011_09_26
│ ├── ...
├── KITTI # testing sequences
│ ├── kitti_seq
│ │ ├── kitti2015_testings
│ │ │ ├──000000
│ │ │ ├── ...
We provide the trained models on TartanAir, SceneFlow, and KITTI_raw datasets. Please download the checkpoints from Dropbox.
Before evaluation, please download the checkpoints and put them in the ./checkpoints
directory.
You can evaluate the pre-trained models on TartanAir, SceneFlow, and KITTI_raw datasets by running the following scripts:
bash tartanair_evaluate.sh
bash sceneflow_evaluate.sh
bash submit_kitti.sh
For TartanAir dataset, you can train the model by running the following script:
bash tartanair_ddp_train.sh
For SceneFlow dataset, you can train the model by running the following script:
bash sceneflow_ddp_train.sh
For KITTI_raw dataset, you can train the model based on the pre-trained model on TartanAir dataset by running the following script:
bash KITTI_ddp_train.sh
Our code is based on RAFT-Stereo, TemporalStereo, IGEV-Stereo and HITNet. We thank the authors for their great works.
If you find our work useful in your research, please consider citing:
@article{zeng2024temporally,
title={Temporally Consistent Stereo Matching},
author={Zeng, Jiaxi and Yao, Chengtang and Wu, Yuwei and Jia, Yunde},
journal={arXiv preprint arXiv:2407.11950},
year={2024}
}