Paper

Haopeng Li, Yuan Yuan, Qi Wang

IEEE Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020

1. Introduction
2. Requirements and Dependencies
3. Installation
4. Pre-trained Model
5. Testing Demo
6. Downloading Results

We propose Video Frame Interpolation via Residue Refinement (RRIN) that leverages residue refinement and adaptive weight to synthesize in-between frames.

Residue refinement is used for optical flow and image generation for higher accuracy and better visual appearance, while the adaptive weight map combines the forward and backward warped frames to reduce the artifacts.

All sub-modules in our method are implemented by U-Net with various depths.

Experiments on public datasets demonstrate the effectiveness and superiority of our method over the state-of-the-art approaches.

• Python = 3.6.8 in Anaconda3 = 4.7.5
• CUDA = 9.2 & cuDNN = 7.0
• PyTorch = 1.0

Download the repository:

$ git clone https://github.com/HopLee6/RRIN.git

We provide the pre-trained model of "RRIN" at OneDrive, which achieves the same results as reported in the paper. Download the pre-trained model to /RRIN.

Test the model using frames in /RRIN/data:

$ python demo.py

and get the interpolated frame /RRIN/data/im_interp.png.

Our RRIN model achieves the state-of-the-art performance on Vimeo90K, and comparable performance on UCF101. Download our interpolated results:

• Vimeo90K

• UCF101

Haopeng Li

The use of this code is RESTRICTED to non-commercial research and educational purposes.

@INPROCEEDINGS{RRIN, 
author={Haopeng, Li and Yuan, Yuan and Qi, Wang}, 
booktitle={ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
title={Video Frame Interpolation Via Residue Refinement}, 
year={2020}, 
pages={2613-2617}
}