Official code of our WACV paper "ECSIC: Epipolar Cross Attention for Stereo Image Compression" by Matthias Wödlinger, Jan Kotera, Manuel Keglevic, Jan Xu and Robert Sablatnig.

Check out the paper on arxiv.

Install the necessary packages from the requirements.txt file with pip:

pip install -r requirements.txt

Train a new model with train.py. Example:

python train.py gpu_idx exp_name --config configs/rd_cs_ecsic_m48.json --log_dir log_dir [--options]

gpu_idx and exp_name need to be specified. The model weights are saved under log_dir/experiments/HASH_DATE_TIME (where HASH is added to prevent collisons for experiments with the same EXP_NAME).

Test a model with test.py. Example:

python test.py gpu_idx exp_name --resume_dir experiments/RD_curves/cs/0.01/

where gpu_idx and exp_name need to be specified and resume_dir can be set to any path with a config file config.json and weights file model.pt in it. Weights of trained models are available for download here. If the folder is copied to the project root the command above should replicate the results from our paper on Cityscapes for lambda=0.01 (bpp=0.089, psnr=38.56).

The data can be located anywhere you want, however the train and test scripts expect text files with paths to the data. data/load_cityscapes.py and data/load_instereo2k.py automatically create such text files for the Cityscapes and InStereo2k datasets. In case something does not work you can also create them by hand by creating train.txt, eval.txt and test.txt where each line is left_image, right_image. See data/DATASET_NAME for an example.

Use generate_rd_curve.py to generate rate distortion curves. For this specifiy a list of lambda values in the command line as a string. E.g.:

python generate_rd_curve.py gpu_dix exp_name --config configs/rd_cs_ecsic_m48.json --lmda "0.001, 0.01, 0.1"

The program will then perform full train/test runs for all specified lmda values and store the results in a json file in experiments/RD_curves/exp_name/results.json and the model weights for the run with lambda=lmda in experiments/RD_curves/exp_name/lmda.

Weights to reproduce these curves are available via google drive.

If you use this project please cite our work

@inproceedings{wodlinger2024ecsic,
  title={ECSIC: Epipolar Cross Attention for Stereo Image Compression},
  author={W{\"o}dlinger, Matthias and Kotera, Jan and Keglevic, Manuel and Xu, Jan and Sablatnig, Robert},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={3436--3445},
  year={2024}
}