We propose a novel approach of CNN and Random Forests for 8x facial image super-resolution. We claim the following points:

• A Siamese Network for Low-Resolution facial image segmentation.

• Random Forests based model for facial feature refinement.

Please cite our work if you use our code or dataset as,

    @InProceedings{Liu2021refvae,
        author = {Zhi-Song Liu, Wan-Chi Siu and Yui-Lam Chan},
        title = {Features Guided Face Super-Resolution via Hybrid Model of Deep Learning and Random Forests},
        booktitle = {IEEE Transactions on Image Processing},
        volume = {30},
        page = {4157-4170},
        year = {2021}
    }

Python 2.XXX<3.0
OpenCV 3.4.0
Caffe 
NVIDIA GPU + CUDA
Jupyter Notebook
Visual Studio 2015 X64

1. Face Coarse SR and Face segmentation

run HCRF_main.ipynb on Jupyter Notebook. Modify the directories of files based on your working environment.

2. Random Forest refinement

download the random forests models from 

https://drive.google.com/open?id=1eiQDcYh-s1PAlFmr8-PHEQDaqRTKaJiF

https://drive.google.com/open?id=1sLDjUh8kSbTQFgj01KKrGpBB3Qmlr7yX

to stage1 and stage2 folders and run HCRF_s3_RF_s1.exe and HCRF_s3_RF_s2.exe

3. Testing resulrs on Helen, CelebA and IJCAI dataset can be downloaded from

https://drive.google.com/open?id=1MYXyk0J_P3u6NV-x6m4Q73BG3wzVm2T2

https://drive.google.com/open?id=15uCll3pwgOV9wkQYP-ag2RQypESnA6rQ

1. We compared our proposed approach with state-of-the-arts face image SR approaches on objective quality by using PSNR and SSIM as follow

We also calculate the Cosine Similarity on HELEN dataset as follow

2. We also compared different approaches on subjective quality by using OpenFace toolbox to measure similarity of facial features. The lower the better. We tested the results on Helen testing datasets

3. Visual Comparison