Comments (1)
Dear woo1,
So sorry for late reply.
CCS_3D_info.txt
is created using OpenFace, but it could be created with other libraries or methods as well. There is one row per frame. You can follow the header to know which features are needed:
- OF_head_pos_X,Y,Z: 3D head position wrt camera coordinate system (X,Y,Z).
- OF_head_rot_X,Y,Z: euler angles for head rotation (X,Y,Z).
- 3Dcam_landmarks_<n_landmark>_X,Y,Z: 3d face landmarks wrt camera coordinate system. 1 value per coordinate and landmark (64 landmarks).
landmarks.txt
is created using Bulat et al. face alingment code (link on ReadMe file). There is one row per frame, and 64 landmarks (X,Y,Z) per row, such that, for 1 frame:
landmark1_X landmark1_Y landmark1_Z landmark2_X landmark2_Y .......
One may argue why landmarks are computed twice, that is, once using the former file and another time with the latter. At the time this code was implemented, Bulat et al was showing better performance than OpenFace for close-to-profile faces. However, it only provides 3d landmarks wrt to pixels, that is, not real 3D landmarks in the 3D space, which is needed to compute real-world head position and rotation. As head position and rotation was already provided by EYEDIAP for the paper experiments, we just used Bulat et al. face alignment code to extract the "3D" landmarks info which is sufficient to have the geometry info needed by the network, and thus the network is trained with that "3d" landmarks format. However, to apply the gaze estimation code to images outside EYEDIAP, we do need to compute the real 3D landmarks to compute the head position and rotation wrt camera coordinate system, but we still need the Bulat et al landmarks format to comply with how the network was trained. Therefore, we use OpenFace to compute head position and rotation, and Bulat et al for the geometry face info. The network could in fact be retrained using OpenFace's landmarks format, to get rid of Bulat et al.'s dependency, but due to time constraints it is not in my current roadmap.
And finally, calibration.txt
refers to the camera calibration:
- resolution: image resolution
- intrinsic: Camera matrix (intrinsic parameters)
- distortion: distortion parameters.
If you have a calibrated camera, you can replace the file values with your values. If you don't know your camera parameters, you could use a dummy matrix. The results won't be as accurate as with a calibrated camera, but it will be an approximation. You can use the following dummy matrix, assuming a 720x576 image as in the example file, and assuming that the principal point is in the middle of the image:
1.2*720.0, 0.0, 360.0
0.0,1.2*720.0, 288.0
0.0, 0.0,1.0
from recurrentgaze.
Related Issues (9)
- when will put the model and code on the github? HOT 3
- test: detection time per frame? HOT 1
- one frame HOT 2
- Testonline Issue OpenFaceDLL.so file HOT 2
- memory access error SIGSEGV
- AttributeError: type object 'LandmarksEstimation' has no attribute 'landmarks_model' HOT 1
- Ask for pretrained model HOT 2
- generate training files HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from recurrentgaze.