This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, which is an extension version of our previous conference paper. Compared with the previous one, this version has more powerful performances.

✨ DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition

• Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, and Ran He. IEEE TPAMI 2021

✨ Dual Variational Generation for Low Shot Heterogeneous Face Recognition

• Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, and Ran He. NeurIPS 2019

✨ Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition

• Chaoyou Fu, Xiaoqiang Zhou, Weizan He, and Ran He. IEEE TPAMI 2022

✨ Cross-Spectral Face Hallucination via Disentangling Independent Factors

• Boyan Duan, Chaoyou Fu, Yi Li, Xingguang Song, and Ran He. CVPR 2020

• Python 3.7.0 & PyTorch 1.5.0 & Torchvision 0.6.0
• Download LightCNN-29 [Google Drive] pretrained on MS-Celeb-1M.
• Download Identity Sampler [Google Drive] pretrained on MS-Celeb-1M.
• Put the above two models in ./pre_train

train_generator.py: Fill out options of '--img_root' and '--train_list', which are the image root and training list of the heterogeneous data, respectively. An example of the training list:

NIR/s2_NIR_10039_001.jpg 232
VIS/s1_VIS_00134_010.jpg 133
NIR/s1_NIR_00118_011.jpg 117

Here we use 'NIR' and 'VIS' in the training list to distinguish the modalities of images. If your list has other distinguishable marks, please change them correspondingly in ./data/dataset.py (lines 28, 38, 66, and 68).

python train_generator.py --gpu_ids 0

gen_samples.py: Fill out options of '--img_root' and '--train_list' that are the same as the above options.

python gen_samples.py --gpu_ids 0

The generated images will be saved in ./gen_images

train_lightcnn.py: Fill out options of 'num_classes', '--img_root_A', and '--train_list_A', where the last two options are the same as the above options.

python train_ligthcnn.py --gpu_ids 0,1

If you use our code for your research, please cite the following papers:

@article{fu2021dvg,
  title={DVG-face: Dual variational generation for heterogeneous face recognition},
  author={Fu, Chaoyou and Wu, Xiang and Hu, Yibo and Huang, Huaibo and He, Ran},
  journal={IEEE TPAMI},
  year={2021}
}

@inproceedings{fu2019dual,
  title={Dual Variational Generation for Low-Shot Heterogeneous Face Recognition},
  author={Fu, Chaoyou and Wu, Xiang and Hu, Yibo and Huang, Huaibo and He, Ran},
  booktitle={NeurIPS},
  year={2019}
}

@article{fu2022towards,
  title={Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition},
  author={Fu, Chaoyou and Zhou, Xiaoqiang and He, Weizan and He, Ran},
  journal={IEEE TPAMI},
  year={2022}
}

@inproceedings{duan2020cross,
  title={Cross-spectral face hallucination via disentangling independent factors},
  author={Duan, Boyan and Fu, Chaoyou and Li, Yi and Song, Xingguang and He, Ran},
  booktitle={CVPR},
  year={2020}
}