Giter VIP home page Giter VIP logo

soubhiksanyal / flame_pytorch Goto Github PK

View Code? Open in Web Editor NEW
597.0 597.0 85.0 2.01 MB

This is a implementation of the 3D FLAME model in PyTorch

License: MIT License

Python 100.00%
3d 3d-graphics 3d-landmarks 3d-mesh 3d-models 3d-reconstruction computer-graphics computer-vision dynamic-conture face face-model face-reconstruction fitting flame flame-model morphable-model pose-dependent-conture python3 pytorch statistical-models

flame_pytorch's Introduction

FLAME: Articulated Expressive 3D Head Model (PyTorch)

This is an implementation of the FLAME 3D head model in PyTorch.

We also provide Tensorflow FLAME, a Chumpy-based FLAME-fitting repository, and code to convert from Basel Face Model to FLAME.

FLAME is a lightweight and expressive generic head model learned from over 33,000 of accurately aligned 3D scans. FLAME combines a linear identity shape space (trained from head scans of 3800 subjects) with an articulated neck, jaw, and eyeballs, pose-dependent corrective blendshapes, and additional global expression blendshapes. For details please see the following scientific publication

Learning a model of facial shape and expression from 4D scans
Tianye Li*, Timo Bolkart*, Michael J. Black, Hao Li, and Javier Romero
ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 2017

and the supplementary video.

Installation

The code uses Python 3.7 and it is tested on PyTorch 1.4.

Setup FLAME PyTorch Virtual Environment

python3.7 -m venv <your_home_dir>/.virtualenvs/FLAME_PyTorch
source <your_home_dir>/.virtualenvs/FLAME_PyTorch/bin/activate

Clone the project and install requirements

git clone https://github.com/soubhiksanyal/FLAME_PyTorch
cd FLAME_PyTorch
python setup.py install
mkdir model

Download models

  • Download FLAME model from here. You need to sign up and agree to the model license for access to the model. Copy the downloaded model inside the model folder.
  • Download Landmark embedings from RingNet Project. Copy it inside the model folder.

Demo

Loading FLAME and visualising the 3D landmarks on the face

Please note we used the pose dependent conture for the face as introduced by RingNet Project.

Run the following command from the terminal

python main.py

License

FLAME is available under Creative Commons Attribution license. By using the model or the code code, you acknowledge that you have read the license terms (https://flame.is.tue.mpg.de/modellicense.html), understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not use the code.

Referencing FLAME

When using this code in a scientific publication, please cite

@article{FLAME:SiggraphAsia2017,
  title = {Learning a model of facial shape and expression from {4D} scans},
  author = {Li, Tianye and Bolkart, Timo and Black, Michael. J. and Li, Hao and Romero, Javier},
  journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)},
  volume = {36},
  number = {6},
  year = {2017},
  url = {https://doi.org/10.1145/3130800.3130813}
}

Additionally if you use the pose dependent dynamic landmarks from this codebase, please cite

@inproceedings{RingNet:CVPR:2019,
title = {Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision},
author = {Sanyal, Soubhik and Bolkart, Timo and Feng, Haiwen and Black, Michael},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
month = jun,
year = {2019},
month_numeric = {6}
}

Supported Projects

FLAME supports several projects such as

FLAME is part of SMPL-X: : A new joint 3D model of the human body, face and hands together

Contact

If you have any questions regarding the PyTorch implementation then you can contact us at [email protected] and [email protected].

Acknowledgements

This repository is build with modifications from SMPLX.

flame_pytorch's People

Contributors

neelays avatar soubhiksanyal avatar timobolkart avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

flame_pytorch's Issues

OpenGL.error.NullFunctionError: Attempt to call an undefined function glGenVertexArrays, check for bool(glGenVertexArrays) before calling

OpenGL.error.NullFunctionError: Attempt to call an undefined function glGenVertexArrays, check for bool(glGenVertexArrays) before calling
free(): invalid pointer

  • ubuntu 22
  • pyglet 2.0.14
  • PyOpenGL 3.1.0
  • PyOpenGL-accelerate 3.1.7
  • pyrender 0.1.45
  • gpu 1080ti
  • nvidia driver Version 550.67

I've been stuck with this problem for a long time and if anyone could give me some advice I'd be very grateful

Flame Model Details

Hi,
Does the Flame model include ear details?
About whether the ear part is unified or can be adjusted?

Some really honest questions

The current code can produce random faces, but there is no texture only grid, can you provide a face project with texture output, thank you very much, I am a graduate student in a Chinese university, your project has helped me a lot, I saw your download link on flame official website, there is a texture space

Can we retrieve the FLAME parameters for COMA?

Hi,

Thank you for this amazing model! I was wondering if there is any way to download the FLAME parameters (pose, shape, expression vectors) for COMA dataset somewhere. I can only find the registered mesh on the different websites, but not the model parameters (as opposite to AMASS where you only have SMPL model parameters but no meshes)

TypeError: lbs() got an unexpected keyword argument 'dtype'

Hello~
Well, Thanks for the project that helps me firstly.
But when I followed what the requirement.txt said, and configured my environment
I run "python main.py"
But it gets this error, I couldn't fix the bug.
Your help will be very appreciated.

How do I set the internal and external parameters?

  # pyrender.Viewer(scene, use_raymond_lighting=True)
    
  renderer = pyrender.OffscreenRenderer(viewport_width=1024, viewport_height=1024)
  image, _ = renderer.render(scene)

  # 保存图片到文件
  filename = f'scene_{i:02}.png'
  import cv2
  cv2.imwrite(filename, image)
  print(f"Scene saved to {os.path.abspath(filename)}")

I want to render it into an image. How do I set the internal and external parameters? I set it to the unit matrix, and the rendering result is empty.
@TimoBolkart @soubhiksanyal @NeelayS

Python version

The readme says that the code uses python 3.7 but the setup.py requires >=3.8.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.