This repository contains an op-for-op PyTorch reimplementation of Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining.

• CSNLN-PyTorch
  • Overview
  • Table of contents
  • About Image Super-Resolution Using Very Deep Residual Channel Attention Networks
  • Download weights
  • Download datasets
  • How Test and Train
    • Test
    • Train model
    • Resume train model
  • Result
  • Credit
    • Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining

If you're new to CSNLN, here's an abstract straight from the paper:

Deep convolution-based single image super-resolution (SISR) networks embrace the benefits of learning from large-scale external image resources for local recovery, yet most existing works have ignored the long-range feature-wise similarities in natural images. Some recent works have successfully leveraged this intrinsic feature correlation by exploring non-local attention modules. However, none of the current deep models have studied another inherent property of images: cross-scale feature correlation. In this paper, we propose the first Cross-Scale Non-Local (CS-NL) attention module with integration into a recurrent neural network. By combining the new CS-NL prior with local and in-scale non-local priors in a powerful recurrent fusion cell, we can find more cross-scale feature correlations within a single low-resolution (LR) image. The performance of SISR is significantly improved by exhaustively integrating all possible priors. Extensive experiments demonstrate the effectiveness of the proposed CS-NL module by setting new state-of-the-arts on multiple SISR benchmarks.

• Google Driver
• Baidu Driver

Contains DIV2K, DIV8K, Flickr2K, OST, T91, Set5, Set14, BSDS100 and BSDS200, etc.

• Google Driver
• Baidu Driver

Both training and testing only need to modify the config.py file.

• line 31: upscale_factor change to 2.
• line 33: mode change to valid.
• line 71: model_path change to results/pretrained_models/CSNLN_x2-DIV2K-xxxxxxxx.pth.tar.

• line 31: upscale_factor change to 2.
• line 33: mode change to train.
• line 35: exp_name change to CSNLN_x2.

• line 31: upscale_factor change to 2.
• line 33: mode change to train.
• line 35: exp_name change to CSNLN_x2.
• line 49: resume change to samples/CSNLN_x2/epoch_xxx.pth.tar.

Source of original paper results: https://arxiv.org/pdf/1807.02758.pdf

In the following table, the value in () indicates the result of the project, and - indicates no test.

Low Resolution / Super Resolution / High Resolution

Yiqun Mei, Yuchen Fan, Yuqian Zhou, Lichao Huang, Thomas S. Huang, Humphrey Shi

Abstract
Deep convolution-based single image super-resolution (SISR) networks embrace the benefits of learning from large-scale external image resources for local recovery, yet most existing works have ignored the long-range feature-wise similarities in natural images. Some recent works have successfully leveraged this intrinsic feature correlation by exploring non-local attention modules. However, none of the current deep models have studied another inherent property of images: cross-scale feature correlation. In this paper, we propose the first Cross-Scale Non-Local (CS-NL) attention module with integration into a recurrent neural network. By combining the new CS-NL prior with local and in-scale non-local priors in a powerful recurrent fusion cell, we can find more cross-scale feature correlations within a single low-resolution (LR) image. The performance of SISR is significantly improved by exhaustively integrating all possible priors. Extensive experiments demonstrate the effectiveness of the proposed CS-NL module by setting new state-of-the-arts on multiple SISR benchmarks.

[Code(PyTorch)] [Paper]

@inproceedings{Mei2020image,
  title={Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining},
  author={Mei, Yiqun and Fan, Yuchen and Zhou, Yuqian and Huang, Lichao and Huang, Thomas S and Shi, Humphrey},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020}
@InProceedings{Lim_2017_CVPR_Workshops,
  author = {Lim, Bee and Son, Sanghyun and Kim, Heewon and Nah, Seungjun and Lee, Kyoung Mu},
  title = {Enhanced Deep Residual Networks for Single Image Super-Resolution},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
  month = {July},
  year = {2017}
}