[PDF] [pixelshift200]
TENetTrinity of Pixel Enhancement: a Joint Solution for Demosaicing, Denoising and Super-Resolution
By Guocheng Qian, Jinjin Gu, Jimmy S. Ren, Chao Dong, Furong Zhao, Juan Lin
Citation
Please cite the following paper if you feel TENet is useful to your research
@article{qian2019trinity,
title={Trinity of Pixel Enhancement: a Joint Solution for Demosaicking, Denoising and Super-Resolution},
author={Qian, Guocheng and Gu, Jinjin and Ren, Jimmy S and Dong, Chao and Zhao, Furong and Lin, Juan},
journal={arXiv preprint arXiv:1905.02538},
year={2019}
}
Resources Collection
Pretrained models
Test data
PixelShift200 dataset
Quick Test
Dependencies
- Python >= 3
- PyTorch 0.4.1 (CUDA version >= 7.5 if installing with CUDA. More details)
- Tensorflow (cpu version is enough, only used for visualization in training)
- Python packages:
pip install opencv-python scipy scikit-image
conda create --name pytorch04
conda activate pytorch04
conda install pytorch=0.4.1 cuda90 torchvision tensorflow -c pytorch
pip install opencv-python scipy scikit-image
Test Models
-
Clone this github repo.
git clone https://github.com/guochengqian/TENet cd TENet
-
Place your own input images in
$YourInputPath
folder. You will save output in$YourSavePath
folder. Input images should be Bayer Raw images (bayer pattern is rggb). -
Run test.
-
test model trained by synthesis datasets
sh ./script/test_tennet2-dn-df2k.sh
-
test model trained by PixelShift200 datasets
sh ./script/test_tenet2-dn-ps200.sh
Don't forget to change $YourInputPath and $YourSavePath in .sh file.
-
How to Train
We train our model both on synthesis datasets(DF2k) and our proposed full color sampled real word 4k dataset PixelShift200 (to be released soon).
-
Data preparation
-
Synthesis data preparation
- Download (DF2k) dataset, which is a combination dataset of DIV2K and Flickr2K
- Crop images into resolution 256*256, using followed code:
python ./dataset/crop_images.py
- generate txt file used for training
python ./dataset/generate_train_df2k.py
-
PixelShift200 data preparation
- Download Pixelshift200. They are .mat format, having 4 channels (R, Gr, Gb, B).
- Crop images into 512*512, using followed code:
python ./dataset/crop_mats.py
- generate txt file used for training
python ./dataset/generate_train_mat.py
-
-
Train model on synthesis dataset
sh script/run_tenet2-dn-df2k.sh
-
Train model on PixelShift200 dataset
sh script/run_tenet2-dn-ps200.sh
TENet
Our approach can be divided into two parts, the first part is a mapping of joint denoising and SR, and the second part converts the SR mosaic image into a full color image. The two parts can be trained and performed jointly. The network structure is illustrated as follows.
PixelShift200 dataset
We employ advanced pixel shift technology to perform a full color sampling of the image. Pixel shift technology takes four samples of the same image, and physically controls the camera sensor to move one pixel horizontally or vertically at each sampling to capture all color information at each pixel. The pixel shift technology ensures that the sampled images follow the distribution of natural images sampled by the camera, and the full information of the color is completely obtained. In this way, the collected images are artifacts-free, which leads to better training results for demosaicing related tasks.
Download dataset from pxielshift200 website.