Giter VIP home page Giter VIP logo

gafar's Introduction

GaFaR: Geometry-aware Face Reconstruction

Project page

Installation

You can use the following command to create and activate your Python environment:

conda env create -f environment.yml
conda activate gafar

Training face reconstruction model

We use EG3D as a pretrained face generator network based on generative neural radiance fields (GNeRF). Therefore, you need to clone its git repository and download available pretrained model:

$ git clone https://github.com/NVlabs/eg3d.git

We use ffhqrebalanced512-128.pkl checkpoint in our experiments.

To train the face reconstruction model, you can use train.py. For example, for blackbox attack against ElasticFace using ArcFace in loss function, you can use the following commands:

python train.py --path_eg3d_repo <path_eg3d_repo>  --path_eg3d_checkpoint <path_eg3d_checkpoint>       \
                --FR_system ElasticFace   --FR_loss  ArcFace  --path_ffhq_dataset <path_ffhq_dataset>  \

Pre-trained models (GaFaR Mapping Network)

Checkpoints of trained models of the mapping network for whitebox and blackbox attacks are available in the project page.

Evaluation

For evaluation script and also access to the dataset of presentation attack using the reconstructed face images, please check the project page.

Citation

  @article{tpami2023ti3d,
    author    = {Hatef Otroshi Shahreza and S{\'e}bastien Marcel},
    title     = {Comprehensive Vulnerability Evaluation of Face Recognition Systems to Template Inversion Attacks Via 3D Face Reconstruction},
    journal   = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
    year      = {2023},
    volume    = {45},
    number    = {12},
    pages     = {14248-14265},
    doi       = {10.1109/TPAMI.2023.3312123}
  }

  @inproceedings{iccv2023ti3d,
    author    = {Hatef Otroshi Shahreza and S{\'e}bastien Marcel},
    title     = {Template Inversion Attack against Face Recognition Systems using 3D Face Reconstruction},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    pages     = {19662--19672},
    month     = {October},
    year      = {2023}
  }

gafar's People

Contributors

otroshi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

gafar's Issues

the reconstructed 3d face

Thank you for your excellent work. I am a novice learner. Where can I check the reconstructed 3d face results?

The results of training.

Hello, after I completed the training, I felt that the training results were not very good. The reconstructed images in each round look similar and are quite different from the original images.
So I would like to ask, do FR_system ElasticFace and FR_loss ArcFace need to be downloaded additionally, or are they already included in the environment setup in the README?
image

new_mapping checkpoint

Thank you very much for your outstanding work, I would like to ask you about the checkpoints of new_mapping, I found that you have given checkpoints after using black-box white-box attacks, is the checkpoint used to generate the reconstructed 3D face? Why did I use the checkpoint model you gave and the reconstructed face is very different from the original image? Thank you for your help and look forward to your reply.

SAR of evaluation

Dear author, thank you for your excellent work. I would like to ask why there is a big gap between the SAR of my evaluation results and the one you give. I have tried the evaluation of each model, and the following are some examples of my evaluation results:

For ElasticFace-ElasticFace:
FMR: 0.01 threshold: -0.7965051331730502 TMR: 0.9683333333333334, SAR: 0.631590181430096
FMR: 0.001 threshold: -0.7072022021466418 TMR: 0.947, SAR: 0.32689434364994663

training problem

Thank you for your excellent work! I encountered the following error while executing the training instructions. Could you please advise on how to resolve it?
image

train.pyproblem

Hello, may I ask why this error occurred while running the train.py file


NOTE: The torch device is: cuda
Traceback (most recent call last):
File "/mnt/data/hangyicheng/face-test/bob.paper.iccv2023_face_ti/train.py", line 58, in
EG3D = pickle.load(f)['G_ema']
ModuleNotFoundError: No module named 'torch_utils.persistence'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.