Video Demoireing with Relation-Based Temporal Consistency (CVPR 2022)
Peng Dai, Xin Yu, Lan Ma, Baoheng Zhang, Jia Li, Wenbo Li, Jiajun Shen, Xiaojuan Qi.
Paper, Project_page, Video

We constrcut the first video demoireing dataset, containing 290 videos (720p, 60 frames for each video), and the equipments we adpoted are 'TCL20 pro mobile phone + Huipu v270 monitor' and 'iPhoneXR + MackBook Pro'.

In data_v1, image pairs are aligned using the homography. In data_v2, aligned image pairs are refined using the estimated optical flow. Besides, we also provide the original moire videos. The Baidu_disk_link, and the password: b28v

Please download and unzip the dataset you need, then change the data paths in 'config/video_demoire_temporal_mbr_scratch_v2.yaml' line 7 and 8. Note that the results in our paper are based on the data_v1.


To train a model better suited for your use or device, you can follow the 'dataset_prepare/data_prepare.py' step by step to prepare your own training data.

Create the conda virtual environment (python 3.7) and activate it.
conda create -n vdm python=3.7
conda activate vdm

Intsall pytorch 1.6.0 and other libaries in the 'requirements.txt'.
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch
pip install -r requirements.txt

Install the DCNv2.
cd model/DCNv2
python3 setup.py build develop

Our implementation has been tested on one NVIDIA 2080Ti GPU with cuda 10.2.

To train the model, just run:
python train.py --config config/video_demoire_temporal_mbr_scratch_v2.yaml

Note: if too large offsets appear while training the PCD model, please restart training. Alternatively, you can use this model (train two epoches) to initialize the network.

To test the model, just run:
python test.py --config config/video_demoire_temporal_mbr_scratch_v2.yaml

We also provide pre-trained models for both TCL and iPhone. The 'checkpoint_000049.tar' means no realtion-based loss, and 'checkpoint_000059.tar' means with realtion-based loss. Please download these models into the corresponding folder (line 29) and change the test epoch (line 54) as descirbed in 'config/video_demoire_temporal_mbr_scratch_v2.yaml'.

If you have any questions, you can email me ([email protected]).

If you are interested in our code or method, please consider citing the following:

@inproceedings{dai2022video,
  title={Video Demoireing with Relation-Based Temporal Consistency},
  author={Dai, Peng and Yu, Xin and Ma, Lan and Zhang, Baoheng and Li, Jia and Li, Wenbo and Shen, Jiajun and Qi, Xiaojuan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}