Giter VIP home page Giter VIP logo

syn2real's Introduction

Syn2Real

Syn2Real Transfer Learning for Image Deraining using Gaussian Processes

Rajeev Yasarla*, Vishwanath A. Sindagi*, Vishal M. Patel

Paper Link(CVPR '20)

Oral video Link

@InProceedings{Yasarla_2020_CVPR,
author = {Yasarla, Rajeev and Sindagi, Vishwanath A. and Patel, Vishal M.},
title = {Syn2Real Transfer Learning for Image Deraining Using Gaussian Processes},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

We propose a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset while generalizing better using unlabeled real-world images. Through extensive experiments and ablations on several challenging datasets (such as Rain800, Rain200H and DDN-SIRR), we show that the proposed method, when trained on limited labeled data, achieves on-par performance with fully-labeled training. Additionally, we demonstrate that using unlabeled real-world images in the proposed GP-based framework results in superior performance as compared to existing methods.

Journal extension:

Semi-Supervised Image Deraining using Gaussian Processes

Paper Link

Prerequisites:

  1. Linux
  2. Python 2 or 3
  3. Pytorch version >=1.9
  4. CPU or NVIDIA GPU + CUDA CuDNN (CUDA 10.2)

Dataset structure

  1. download the rain datasets and arrange the rainy images and clean images in the following order
  2. Save the image names into text file (dataset_filename.txt)
   .
    ├── data 
    |   ├── train # Training  
    |   |   ├── derain        
    |   |   |   ├── <dataset_name>   
    |   |   |   |   ├── rain              # rain images 
    |   |   |   |   └── norain            # clean images
    |   |   |   └── dataset_filename.txt
    |   └── test  # Testing
    |   |   ├── derain         
    |   |   |   ├── <dataset_name>          
    |   |   |   |   ├── rain              # rain images 
    |   |   |   |   └── norain            # clean images
    |   |   |   └── dataset_filename.txt

To test Syn2Real:

  1. mention test dataset text file in the line 57 of test.py, for example
    val_filename = 'SIRR_test.txt'
  1. Run the following command
    python test.py -category derain -exp_name DDN_SIRR_withGP

To train Syn2Real:

  1. mention the labeled, unlabeled, and validation dataset in lines 119-121 of train.py, for example
    labeled_name = 'DDN_100_split1.txt'
    unlabeled_name = 'real_input_split1.txt'
    val_filename = 'SIRR_test.txt'
  1. Run the following command to train the base network without Gaussian processes
    python train.py  -train_batch_size 2  -category derain -exp_name DDN_SIRR_withoutGP  -lambda_GP 0.00 -epoch_start 0
  1. Run the following command to train Syn2Real (CVPR'20) model
    python train.py  -train_batch_size 2  -category derain -exp_name DDN_SIRR_withGP  -lambda_GP 0.0015 -epoch_start 0 -version version1
  1. Run the following command to train Syn2Real++ (journal submission GP modellig at feature map level)
    python train.py  -train_batch_size 2  -category derain -exp_name DDN_SIRR_withGP  -lambda_GP 0.0015 -epoch_start 0 -version version2
    

Cross-domain experiments and Gaussian kernels

cross domain experiments are performed using DIDMDN dataset as source dataset, and other datasets like Rain800, JORDER_200L, DDN as target datasets.

----------------------------------------------------
Source datasets   | Target datasets                | 
----------------------------------------------------
DIDMDN            | Rain800, JORDER_200L, DDN      |
----------------------------------------------------

Gaussian processes can be modelled using different kernels like Linear or Squared_exponential or Rational_quadratic. the updated code provides an option to choose the kernel type

-kernel_type <Linear or Squared_exponential or Rational_quadratic>

Fast version of GP

use GP_new_fast.py file for faster version of GP.

To use this GP_new_fast.py :
    comment line 14 in train.py
    and uncomment line 15 in train.py

Additionally you can use "train_new_comb.py" instead of "train.py".

In "train_new_comb.py" does iterative training of the network, i.e. each iteration contains one labeled train step and one unlabeled train step.

Run the following command to train Syn2Real (CVPR'20) model using "train_new_comb.py".

    python train_new_comb.py  -train_batch_size 2  -category derain -exp_name DDN_SIRR_withGP  -lambda_GP 0.0015 -epoch_start 0 -version version1

syn2real's People

Contributors

rajeevyasarla avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.