Comments (7)
I tried various 4x4 matrices from these values (focal=1015, znear=5, zfar=15), but none of them match the same result as your code.
reduce unnecessary calculations
for the processor it's nanoseconds, for the programmer and those who will use the repo it's a headache.
I believe you can directly refer to /util/nv_diffrast.py to get the result you want. The camera parameters have also been verified by using PyTorch3D. Let me explain further:
Starting from line 442 of model/recon.py, we assume the homogenous 3D point v = (x,y,z,1)
is one of the points in v3d. First, we calculate the perspective projection matrix:
Invert the z direction of v
to correspond to the camera's coordinate system. (In /util/nv_diffrast.py, you might also find that the y direction of v
is inverted as well, which is done to adapt to the renderer.) So we get (x, y, -z, 1), and then perform the perspective projection to obtain:
Homogenize the coordinates:
Finally, the coordinate in ndc space is converted to the image plane:
You will find that this result is consistent with the result obtained using self.persc_proj
and homogenizing in model/recon.py.
I believe the above process is detailed enough and hope it helps you. The cause of your incorrect results could be due to some axis inversions (such as the common y-flip in images). This is likely because of different coordinate system definitions used in different rendering methods. Usually, you just need to visualize the results and check the steps where inversion is needed.
from 3ddfa-v3.
Thank you for your support, this issue is valuable.
The parameters of the perspective projection camera we use are: focal=1015, znear=5, zfar=15
. The camera is located at (0, 0, 10)
and faces the negative direction of the z-axis. The rendered image size is 224×224
.
I completely agree with what you said about the 4x4 projection matrix being fundamental to some rendering calculations. I believe you can find the process you mentioned in /util/nv_diffrast.py, which is the calculation method for transforming homogenous 3D points (x, y, z, 1)
.
However, in model/recon.py, the purpose of self.persc_proj
is merely to obtain the x and y coordinates without involving the rendering process, so z is not needed (otherwise the calculation would be redundant). You can verify that the v2d
obtained using self.persc_proj
are consistent with the first two dimensions of screen coordinates obtained by transforming the vertex_ndc
using /util/nv_diffrast.py. In short, self.persc_proj
is just a way to slightly reduce unnecessary calculations, a similar approach is common in HRN, Deep3D, etc. Hope this helps.
from 3ddfa-v3.
I tried various 4x4 matrices from these values (focal=1015, znear=5, zfar=15), but none of them match the same result as your code.
reduce unnecessary calculations
for the processor it's nanoseconds, for the programmer and those who will use the repo it's a headache.
from 3ddfa-v3.
@wang-zidu Hi, thanks for your excellent work. I wonder if this work supports CPU inference. I'm trying to use cpu device but get an error with nvdiffrast (RasterizeCudaContext could not use cpu device)
from 3ddfa-v3.
@wang-zidu Hi, thanks for your excellent work. I wonder if this work supports CPU inference. I'm trying to use cpu device but get an error with nvdiffrast (RasterizeCudaContext could not use cpu device)
Thank you for your feedback. nvdiffrast can be replaced with a simpler renderer such as face3d. You can try replacing it, or if you only need the mesh results, you can simply remove the corresponding nvdiffrast content. If I have time, I will address this request as soon as possible and let you know.
from 3ddfa-v3.
@wang-zidu Hi, thanks for your excellent work. I wonder if this work supports CPU inference. I'm trying to use cpu device but get an error with nvdiffrast (RasterizeCudaContext could not use cpu device)
Thank you for your support and patience. We have updated a new fast CPU renderer (based on face3d). Now the entire work supports CPU inference (using RetinaFace for face box).
from 3ddfa-v3.
I tried various 4x4 matrices from these values (focal=1015, znear=5, zfar=15), but none of them match the same result as your code.
reduce unnecessary calculations
for the processor it's nanoseconds, for the programmer and those who will use the repo it's a headache.
Did you figure out how to ‘export’ the camera in a way that can be imported to a 3D DCC?
from 3ddfa-v3.
Related Issues (17)
- When to release codes? HOT 11
- Is there any plan to release the training code? HOT 1
- labels HOT 1
- pos+rot entangled in id+exp HOT 1
- Question about the symbol $C_p$ in the paper HOT 2
- Error landmarks in large base landmark HOT 2
- Video Demo HOT 1
- i test on video the result is a bit jittor, any method to optim the pose? HOT 2
- NoW Challenge HOT 5
- AssertionError("Torch not compiled with CUDA enabled") HOT 2
- BFM with Neck HOT 7
- How to Export Materials(OBJ) HOT 1
- 使用opencv部署3D人脸重建3DDFA-V3,包含C++和Python两个版本的程序,只依赖opencv库就能运行 HOT 1
- Question on Camera Position / Focal Length HOT 1
- Question on Inference Speed
- Choosing the anchors
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 3ddfa-v3.