tadasbaltrusaitis / openface Goto Github PK

Hey Tadas,

very impressive work! I'm currently writing my thesis and I'm tending to make use of your library in it. I've got like a million questions after reading through your papers and code for days, but I'll try to limit them to a few generic ones here first :-)

1. Would you like to share with us what your next steps/improvements on OpenFace might look like?

2. Do you plan to add something like 'Gender' or 'Age' estimation at some point?

3. Are you looking for contributions? Which are the aspects of OpenFace that you would like to see improved, but don't have enough time to do so by yourself?

4. The version 0.2 suggests the lib is at an early development stage. Do you plan to do any major API-breaking rewriting/refactoring within the next couple of months?

5. Just some feedback: In my tests, the AU45 ('Blink') classifier detects closed eyes well, but unfortunately also shows a lot of false-positives (e.g. when moving/rotating the head around with eyes open). However, at least the AU45 regressor value nicely peaks for closed eyes and the face position and orientation estimations are just awesome anyways!

Kind regards,
Clint

Anyone who run the experiment? My result is terrible....

Have anyone ported this amazing api to the web through Emscripten for example!!!

• Any place for me to configure building dynamic link?
• Any place for me to specify building into /usr/local/lib ?

Cheers
Pei

I was building OpenFace on Ubuntu 16.04. Wiki says to use Clang for 15.10, so I did. I built again using gcc as well to see the difference and noticed the speed was about 10-15 fps worse when using gcc (both in release mode).

I was wondering if OpenFace can be built with gcc on Ubuntu 16.04 without any loss in speed. I was trying to use OpenCV and boost for a different library that requires compilation with gcc. and I am unable to use both OpenFace and the other library in my project.

Hi
I just downloaded whole file on github (https://github.com/TadasBaltrusaitis/OpenFace) and found that there is a file which is not mentioned in readme.
(located in matlab_version\demo\face_image_demo_eyes.m)

I found that in this file at line 19, the first parameter of function Load_Patch_Experts() which should be the file direction is point to the author's c drive and I can not find the file name “svr_patches_*_synth.mat” in the whole open face folder.Therefore I can not run this file.

I would like to know that if this file can label landmark only on eyes without the whole face. If so, would you release the new version of the code? If not, what would this code do?

Thanks.

Is there a built-in way to run FeatureExtraction in real-time? According to the wiki there seems like no way to specify a webcam from which to read the images.

I was install openface
and try to run it:
bin/FaceLandmarkImg -f ./image_sequence/005.jpg

dyld: Library not loaded: /usr/local/opt/webp/lib/libwebp.6.dylib
  Referenced from: /usr/local/opt/opencv3/lib/libopencv_imgcodecs.3.1.dylib
  Reason: image not found
[1]    17647 trace trap  bin/FaceLandmarkImg -f ./image_sequence/005.jpg`

I tried to install the webp, but it ships without libwebp.6.dylib

Hi, I've found a bit strange in the 2d landmarks format when float fraction part and a separator between values share the same symbol ','
For example: 0,1,0,981,275,12,268,09,271,85,289,85,328,27,... etc.
here x1 is 981,275, etc.
P.S. I've tried the OpenFace application from the main page - http://www.cl.cam.ac.uk/research/rainbow/projects/openface/
In the source I've found that FeatureExtraction has a different file store code.

Hi tadas,
I saw that you stored the modules in .txt format, but I don't know what every row data means, can you tell me what the data structure of the modules is?

When I run this line: bin/FaceLandmarkImg -f ./image_sequence/001.jpg -oi ./output3, the program loads the model but then I get this error:
OpenCV Error: Unspecified error (could not find a writer for the specified extension) in imwrite_, file /tmp/opencv3-20160729-19925-gxxi24/opencv-3.1.0/modules/imgcodecs/src/loadsave.cpp, line 459
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /tmp/opencv3-20160729-19925-gxxi24/opencv-3.1.0/modules/imgcodecs/src/loadsave.cpp:459: error: (-2) could not find a writer for the specified extension in function imwrite_

I have experienced problems with opencv on this program in the past. I did this as a test, what I really want to run is FeatureExtraction, but when I run that, it says "Starting tracking", and then nothing else. The output file is always just an empty .mov. Any ideas?

I transplanted the project to the IOS, https://github.com/FaceAR/OpenFaceIOS, But the cpu usage is so high, Can you give me some advice about it.

I used this program like this : FaceLandmarkImg.exe -f face.jpg
but crashed!

Hello @TadasBaltrusaitis ,

I am currently using your CLM and FaceAnalyser libraries from your previous CLM code (i.e. this https://github.com/TadasBaltrusaitis/CLM-framework ), and I have been planning to move to the new OpenFace repository.

Now, I have seen that you at least changed the name of your library from CLM to LandmarkDetector. Is there anything else I should be aware of? Is the transition going to be smooth? Are there going to be significant changes in the API?

Thank you in advance for your help!

Hello, I downloaded zip.You said that we don't need to download anything additional to run the project, just open "OpenFace.sln" using Visual Studio 2015 and compile the code.I did it but failed.I wonder if there exists some details that i don't know.Cause when i open the "OpenFace.sln" ,VS tells that the project ''LandmarkDetector''comes from an untrusted location and when compile it, the VS cannot
start the program
''D:\OpencvProject\OpenFace-master\OpenFace-master\x64\Release'LandmarkDetector.lib''
Can you tell me which step appears the problem?

Hi,
It turns out that for low-resolution videos this package is unable to detect faces with high recall. I am wondering if there exists some sort of threshold for Haar face detection that I can decrease, and later on remove false alarms through some other approach?
Thanks

can i used it by C#?

Hi, I installed a tested the library with command calls and works perfectly. Now I'm trying to use it in a qt project and I'm having trouble with the API calls, i'm trying the code on the wiki section for landmark detector and I get the error
"OpenFace/build/lib/local/LandmarkDetector/libLandmarkDetector.a(LandmarkDetectionValidator.cpp.o): undefined reference to symbol"

What I'm missing? This is my qt .pro confing

opencv

INCLUDEPATH += /usr/local/include/opencv
LIBS += -L/usr/local/lib ##-lopencv_core -lopencv_imgcodecs -lopencv_highgui
LIBS += pkg-config opencv --libs

landmark detector

INCLUDEPATH +=/home/pp/OpenFace/lib/local/LandmarkDetector/include
LIBS += -L/home/pp/OpenFace/build/lib/local/LandmarkDetector -lLandmarkDetector

INCLUDEPATH +=/home/pp/OpenFace/lib/3rdParty/dlib/include
LIBS+= -L/home/pp/OpenFace/build/lib/3rdParty/dlib -ldlib
LIBS += -pthread
CONFIG += link_pkgconfig
PKGCONFIG += x11

Thanks

Hi, I'm getting this strange exception when deploying FaceLandmarkVid using Release x64 configuration on VS2015 Enterprise. It takes me to line 649 of interpolation.h ad precisely to const double x_scale = (in_img.nc()-1)/(double)std::max<long>((out_img.nc()-1),1);

Really strange behavior since using Release configuration with Win32 is OK (though not reaching the same FPS as it does on the same machine with linux: Win goes around 20fps and Linux goes around 30fps - I know that the docs says it's worth running in x64 for better performance...) >> Makes me think there's something with 64bit addressing and so on...

Trying to change SDK to UWP (Windows 10 SDK) does not solve the problem.

Anyone experiencing the same issues?

I am running into an issue when I run "make" after I have run "cmake -D CMAKE_BUILD_TYPE=RELEASE ." Seems like the program is not finding 'opencv2/imgcodecs.hpp', even though I believe opencv2 is there. I have tried running it with the tweaks to CMake and without, and I end up with the same result. Any help would be appreciated.

Linking CXX executable ../../bin/FaceLandmarkVidMulti
/usr/bin/ld: cannot find -lvtkRenderingOpenGL
/usr/bin/ld: cannot find -lvtkImagingHybrid
/usr/bin/ld: cannot find -lvtkIOImage
/usr/bin/ld: cannot find -lvtkCommonDataModel
/usr/bin/ld: cannot find -lvtkCommonMath
/usr/bin/ld: cannot find -lvtkCommonCore
/usr/bin/ld: cannot find -lvtksys
/usr/bin/ld: cannot find -lvtkCommonMisc
/usr/bin/ld: cannot find -lvtkCommonSystem
/usr/bin/ld: cannot find -lvtkCommonTransforms
/usr/bin/ld: cannot find -lvtkCommonExecutionModel
/usr/bin/ld: cannot find -lvtkDICOMParser
/usr/bin/ld: cannot find -lvtkIOCore
/usr/bin/ld: cannot find -lvtkmetaio
/usr/bin/ld: cannot find -lvtkImagingCore
/usr/bin/ld: cannot find -lvtkRenderingCore
/usr/bin/ld: cannot find -lvtkFiltersExtraction
/usr/bin/ld: cannot find -lvtkFiltersCore
/usr/bin/ld: cannot find -lvtkFiltersGeneral
/usr/bin/ld: cannot find -lvtkCommonComputationalGeometry
/usr/bin/ld: cannot find -lvtkFiltersStatistics
/usr/bin/ld: cannot find -lvtkImagingFourier
/usr/bin/ld: cannot find -lvtkalglib
/usr/bin/ld: cannot find -lvtkFiltersGeometry
/usr/bin/ld: cannot find -lvtkFiltersSources
/usr/bin/ld: cannot find -lvtkIOXMLParser
/usr/bin/ld: cannot find -lvtkInteractionStyle
/usr/bin/ld: cannot find -lvtkRenderingLOD
/usr/bin/ld: cannot find -lvtkFiltersModeling
/usr/bin/ld: cannot find -lvtkIOPLY
/usr/bin/ld: cannot find -lvtkIOGeometry
/usr/bin/ld: cannot find -lvtkFiltersTexture
/usr/bin/ld: cannot find -lvtkRenderingFreeType
/usr/bin/ld: cannot find -lvtkftgl
/usr/bin/ld: cannot find -lvtkIOExport
/usr/bin/ld: cannot find -lvtkRenderingContext2D
/usr/bin/ld: cannot find -lvtkRenderingGL2PS
collect2: error: ld returned 1 exit status
make[2]: *** [bin/FaceLandmarkVid] Error 1
make[1]: *** [exe/FaceLandmarkVid/CMakeFiles/FaceLandmarkVid.dir/all] Error 2

and so on

Hi Tadas,
Hello from ICT!

OpenFace runs perfectly on my windows machine, but when I tried to install on Mac, it cannot find TBB;

cmake -D CMAKE_BUILD_TYPE=RELEASE .
OpenCV information:
OpenCV_INCLUDE_DIRS: /usr/local/Cellar/opencv3/3.1.0_3/include/opencv;/usr/local/Cellar/opencv3/3.1.0_3/include
OpenCV_LIBRARIES: opencv_videostab;opencv_videoio;opencv_video;opencv_superres;opencv_stitching;opencv_shape;opencv_photo;opencv_objdetect;opencv_ml;opencv_imgproc;opencv_imgcodecs;opencv_highgui;opencv_flann;opencv_features2d;opencv_core;opencv_calib3d
OpenCV_LIBRARY_DIRS:
-- Boost version: 1.60.0
-- Found the following Boost libraries:
-- filesystem
-- system
Boost information:
Boost_INCLUDE_DIRS: /usr/local/include
Boost_LIBRARIES: /usr/local/lib/libboost_filesystem-mt.dylib;/usr/local/lib/libboost_system-mt.dylib
Boost_LIBRARY_DIRS: /usr/local/lib
X11 information:
X11_INCLUDE_DIR: /usr/X11R6/include
X11_LIBRARIES: /usr/X11R6/lib/libSM.dylib;/usr/X11R6/lib/libICE.dylib;/usr/X11R6/lib/libX11.dylib;/usr/X11R6/lib/libXext.dylib
X11_LIBRARY_DIRS:
CMake Error at /Applications/CMake.app/Contents/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find TBB (missing: TBB_ROOT_DIR TBB_INCLUDE_DIR TBB_LIBRARY)
Call Stack (most recent call first):
/Applications/CMake.app/Contents/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
cmake/modules/FindTBB.cmake:291 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:32 (find_package)

I also tried to add something to the CMakeLists.txt similar to what you suggested in Wiki for finding OpenCV, something like, find_package( TBB HINTS ~/OpenFace/lib/3rdParty/tbb/... But it didn't work..

hi :)
can i run the code with depth images obtained by a Kinect ?

Excuse me , I have followed the steps of compiling in WiKi,,,but it produces a lot of errors
such as " error MSB8020: The build tools for v140 (Platform Toolset = 'v140') ........."
" error MSB3073: The command "xcopy /I /E /Y /D /C .........."
I compile the code in VS2013, but I have installed the "64-bit Visual C++ redistributable package",
can you help me ？Thank you ver much!!

Hi @TadasBaltrusaitis, thanks for this useful toolkit to the CV/multimodal research community! I had followed the steps in your Wiki but was stumped by the following errors during the late stages at 95% of the build process for Mac. Is this a known issue, and if so, can you please shed some light on how to resolve it?

$cmake -D CMAKE_BUILD_TYPE=RELEASE .
-- The C compiler identification is AppleClang 7.3.0.7030031
-- The CXX compiler identification is AppleClang 7.3.0.7030031
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
OpenCV information:
  OpenCV_INCLUDE_DIRS: /usr/local/Cellar/opencv3/3.1.0_3/include/opencv;/usr/local/Cellar/opencv3/3.1.0_3/include
  OpenCV_LIBRARIES: opencv_videostab;opencv_videoio;opencv_video;opencv_superres;opencv_stitching;opencv_shape;opencv_photo;opencv_objdetect;opencv_ml;opencv_imgproc;opencv_imgcodecs;opencv_highgui;opencv_flann;opencv_features2d;opencv_core;opencv_calib3d
  OpenCV_LIBRARY_DIRS:
-- Boost version: 1.57.0
-- Found the following Boost libraries:
--   filesystem
--   system
Boost information:
  Boost_INCLUDE_DIRS: /usr/local/include
  Boost_LIBRARIES: /usr/local/lib/libboost_filesystem-mt.dylib;/usr/local/lib/libboost_system-mt.dylib
  Boost_LIBRARY_DIRS: /usr/local/lib
-- Looking for XOpenDisplay in /usr/X11R6/lib/libX11.dylib;/usr/X11R6/lib/libXext.dylib
-- Looking for XOpenDisplay in /usr/X11R6/lib/libX11.dylib;/usr/X11R6/lib/libXext.dylib - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found X11: /usr/X11R6/lib/libX11.dylib
X11 information:
  X11_INCLUDE_DIR: /usr/X11R6/include
  X11_LIBRARIES: /usr/X11R6/lib/libSM.dylib;/usr/X11R6/lib/libICE.dylib;/usr/X11R6/lib/libX11.dylib;/usr/X11R6/lib/libXext.dylib
  X11_LIBRARY_DIRS:
-- Found TBB: /usr/local (found version "4.4")
-- Found JPEG: /usr/local/lib/libjpeg.dylib
-- Searching for BLAS and LAPACK
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found LAPACK library
-- Found CBLAS library
-- Looking for cblas_ddot
-- Looking for cblas_ddot - found
-- Check for STD namespace
-- Check for STD namespace - found
-- Looking for C++ include iostream
-- Looking for C++ include iostream - found
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/cleong/OpenFace
(vamp)cleong@M-AH552347~/OpenFace$make
[ 69%] Built target dlib
[ 84%] Built target LandmarkDetector
[ 94%] Built target FaceAnalyser
[ 95%] Building CXX object exe/FaceLandmarkImg/CMakeFiles/FaceLandmarkImg.dir/FaceLandmarkImg.cpp.o
In file included from /Users/cleong/OpenFace/exe/FaceLandmarkImg/FaceLandmarkImg.cpp:60:
In file included from /Users/cleong/OpenFace/exe/FaceLandmarkImg/../../lib/local/LandmarkDetector/include/LandmarkCoreIncludes.h:63:
In file included from /Users/cleong/OpenFace/exe/FaceLandmarkImg/../../lib/local/LandmarkDetector/include/LandmarkDetectorModel.h:64:
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/objdetect.hpp:166:1: error: extraneous 'template<>' in declaration of variable 'DefaultDeleter'
template<> CV_EXPORTS void DefaultDeleter<CvHaarClassifierCascade>::operator ()(CvHaarClassifierCascade* obj) const;
^~~~~~~~~~
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/objdetect.hpp:166:28: error: variable has incomplete type 'void'
template<> CV_EXPORTS void DefaultDeleter<CvHaarClassifierCascade>::operator ()(CvHaarClassifierCascade* obj) const;
                           ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/objdetect.hpp:166:42: error: expected ';' at end of declaration
template<> CV_EXPORTS void DefaultDeleter<CvHaarClassifierCascade>::operator ()(CvHaarClassifierCascade* obj) const;
                                         ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/objdetect.hpp:166:42: error: expected unqualified-id
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/objdetect.hpp:432:5: error: unknown type name 'UMat'; did you mean 'Mat'?
    UMat oclSvmDetector;
    ^
/usr/local/include/opencv2/core/core.hpp:1711:18: note: 'Mat' declared here
class CV_EXPORTS Mat
                 ^
In file included from /Users/cleong/OpenFace/exe/FaceLandmarkImg/FaceLandmarkImg.cpp:68:
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1163:50: error: use of undeclared identifier 'BORDER_DEFAULT'
                                int borderType = BORDER_DEFAULT );
                                                 ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1195:53: error: use of undeclared identifier 'BORDER_DEFAULT'
                                   int borderType = BORDER_DEFAULT );
                                                    ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1224:47: error: use of undeclared identifier 'BORDER_DEFAULT'
                             int borderType = BORDER_DEFAULT );
                                              ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1247:50: error: use of undeclared identifier 'BORDER_DEFAULT'
                                int borderType = BORDER_DEFAULT );
                                                 ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1269:42: error: use of undeclared identifier 'BORDER_DEFAULT'
                        int borderType = BORDER_DEFAULT );
                                         ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1303:64: error: use of undeclared identifier 'BORDER_DEFAULT'
                            double delta = 0, int borderType = BORDER_DEFAULT );
                                                               ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1325:67: error: use of undeclared identifier 'BORDER_DEFAULT'
                               double delta = 0, int borderType = BORDER_DEFAULT );
                                                                  ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1372:43: error: use of undeclared identifier 'BORDER_DEFAULT'
                         int borderType = BORDER_DEFAULT );
                                          ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1394:53: error: use of undeclared identifier 'BORDER_DEFAULT'
                                   int borderType = BORDER_DEFAULT );
                                                    ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1420:44: error: use of undeclared identifier 'BORDER_DEFAULT'
                          int borderType = BORDER_DEFAULT );
                                           ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1451:47: error: use of undeclared identifier 'BORDER_DEFAULT'
                             int borderType = BORDER_DEFAULT );
                                              ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1498:55: error: use of undeclared identifier 'BORDER_DEFAULT'
                                     int borderType = BORDER_DEFAULT );
                                                      ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1521:50: error: use of undeclared identifier 'BORDER_DEFAULT'
                                int borderType = BORDER_DEFAULT );
                                                 ^
/usr/local/Cellar/opencv3/3.1.0_3/include/opencv2/imgproc.hpp:1551:60: error: use of undeclared identifier 'BORDER_DEFAULT'
                                          int borderType = BORDER_DEFAULT );
                                                           ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[2]: *** [exe/FaceLandmarkImg/CMakeFiles/FaceLandmarkImg.dir/FaceLandmarkImg.cpp.o] Error 1
make[1]: *** [exe/FaceLandmarkImg/CMakeFiles/FaceLandmarkImg.dir/all] Error 2
make: *** [all] Error 2

Firstly, I would like to thank you for a great multi-threaded framework. I just wanted to clarify how to measure accuracy of the gaze estimation:

One way is to take the tan inverse of the error_distance/distance from screen. But in your paper titled "Learning an appearance-based gaze estimator from one million synthesised images", you mention that "The eye gaze error was computed as median angle between ground truth and estimated
eye gaze vectors in degrees." Can you please explain this to me? (Does this mean the accuracy is calculated in terms of difference of the yaw and pitch from original ground truth)

Hi.

There only exist the script for generating PDM using wild dataset.
It generates the model file named 'pdm_68_aligned_wild.mat' well.
But, for more accurate head orientation, 'pdm_68_multi_pie.mat' model file should be required as you comment in the source file.
So, how can I generate the pdm model file with Multi-PIE dataset (pdm_68_multi_pie.mat)?
(I have Multi-PIE dataset and its labels)

thank you in advance

Facial landmark detection is performed using Dlib?

Hello, I downloaded the entire folder but I have no way to run gaze_extraction_demo_vid.m this file, because I want to learn the direction of line of sight detection algorithm. I come from a high school student in China, please help me to solve this problem.
Error：
Error using dlmread (line 120)
The file '?_gaze.txt' could not be opened because: Invalid argument

Error in gaze_extraction_demo_vid (line 40)
gaze = dlmread([2015-10-15-15-14, '_gaze.txt'], ',', 1, 0);

Hello.
I downloaded the code, and compile it successfully. But it can not run normally using command line, VS debug window tells "facelandmarkvid.exe have some unprocessed win32 exception".
Then I downloaded the zip, and use the compiled facelandmarkvid.exe in this packet. The problem is same.
What is wrong? Thanks.

Could the code run by opencv in linux?

Hello!
I'm curious that if we track with less facial points, can face tracking be faster?
Also, I want to know if there's a way to fasten the speed up even if we lose some accuracy.
Can anyone give me some idea?

Thanks in advance.

hi,
can i run the FeatureExtraction and the see the right window of the picture (https://github.com/TadasBaltrusaitis/OpenFace/blob/master/imgs/au_sample.png), it's the "Facial Action Unit Recognition", it looks very good, i want to test it.

Hi
I found that bounding box of DLib‘s HOG Face detector and opencv‘s Haar Face detector need to compensate.
I want to try some other face detector,like NPD Detector,so what's your standard bounding box of LFPW ,Helen and so on.
or I have to transform it through the two offered scaling?

Thanks in advance.

Is this code the same code used in the paper "Rendering of Eyes for Eye-Shape Registration and Gaze Estimation" for testing on the Max Planck Database? If not, what changes were made.

Also why is offset taken as (0,-3.5,0) in GazeEstimation.cpp? I.e. why this particular value and should it not be along the Z-direction?

Hey Tadas,

The project you did It's very impressive!

I find out you modify the dlib source code in scan_fhog_pyramid.h on both CLM-framework and OpenFace project.

It seems like you want to make create_fhog_pyramid and detect_from_fhog_pyramid run in parallel by TBB.

And I try to calculate the processing time in object_detector.h:430 in fuction void object_detector<image_scanner_type>::operator().

I got puzzled by the unexpected results.

The code without parallel is 2 times faster than code with TBB parallel.

So I'm not sure is there anything I missed or I use the wrong settings?

Thanks,
Frank

I realize many of the videos I would like to run through FeatureExtraction are .wmv files. It looks like the software is open and run them, I think because I am trying to run .wmv files on a mac. I do have software that can play the videos (mplayerx), is there a way I could implement this?

I have tried converting them to a different format, but the converted videos seemed to be lower quality, and when OpenFace ran them the visualization was a small window, rather than the whole screen. I am afraid this is affecting the software's ability to detect facial features, am I correct?

Thanks for any help

How can I train my facelandmark model ? Do you have guide to train a landmark detection model for facial landmark detection ?

I did not see this information anywhere in the paper. What is the coordinate system used by the estimategaze(), My initial tests show gaze.X and gaze.Y varies between -0.50 to +0.50 in normal laptop use scenario (as seen through built in camera of the laptop). What is the origin of the coordinate system (camera origin or egocentric) ?

Also, is there any built in way to translate this to screen coordinates ?

Is it possible to detect eye and/or iris landmarks, given only the image of the eye as follows:

or a much more cropped eye image:

If so, could you please provide a pseudo-code on how to do so given any such image.

Image taken from http://efdreams.com/data_images/dreams/face/face-03.jpg.

Hello !
I was thinking to integrate OpenFace with Android but I saw it uses about 100mb ram on ubuntu and 200 mb ram on windows which is too much for a device but seeing this difference in os I'm thinking weather it will work on Android where 50/100 mb is about the maximum you can get. Do you think it can handle it ? Also, do you have any suggestions on how one should proceed with the integration ? I'm not sure if I should generate .a or .so files and if it will work out of the box like this.

With reference to the Max Planck Gaze in the wild database (https://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/gaze-based-human-computer-interaction/appearance-based-gaze-estimation-in-the-wild-mpiigaze/), how do you convert the 3D gaze direction vector to the 2D Screen Gaze Target Coordinate?

I don’t think I’m running this correctly. On Unix installation file you said to test it with videos with the following code: ./bin/FaceLandmarkVid -f "../videos/changeLighting.wmv"
However, I don’t see the “bin" directory. I see that the FaceLandmarkVid file is in the “exe” directory. On the linux terminal, I’m running (From within the exe directory) this: FaceLandmarkVid -f “../videos/changeLighting.wmv”
And I get an error saying “FaceLandmarkVid: Command not found”. And if I run the cpp file, I get a permission error.
Sorry if this is a trivial problem. What am I doing wrong?
Thanks

Under normal circumstances, it should show a good photo processing,
but what are not, am I not set argc and argv normally?

Hi @TadasBaltrusaitis , is there a place in which I can find the reference frames used by CLM and OpenFace? I assume you're using pretty standard reference frames, but I would like to have a definitive reference on this.

Thank you!

When I run the face_image_demo.m in matlab_version, it returns me error in Load_Patches_Experts at line 5 , which is colourPatchFiles = dir([col_patch_dir col_patch_file]);

"output argument "patches" (and maybe others) not assigned during call to "Load_Patch_Experts".

Hi

I don't understand how to set the focal lengths.

I have a Logitech C930e with a FOV of 90° and is 16/9.
Searching a bit it seems that HFOV is ~52° and VFOV is ~82°.
Not easy to find more info about the specs but i could find that the big brother C920 as a focal length of 3.67 mm

Values in the code are about 500 (mm ?) (based on 640*480 résolution) so I realy don't understand what is the unit and how to compte the value.

Thank you

Hi, i install in AWS GPU with CUDA
https://gist.github.com/fortunto2/530a2b096d497f01dedabd4789fb84ce

and final error in make

[ 94%] Built target FaceAnalyser
Linking CXX executable ../../bin/FaceLandmarkImg
/usr/bin/ld: cannot find -lQt5::Core
/usr/bin/ld: cannot find -lQt5::Gui
/usr/bin/ld: cannot find -lQt5::Widgets
/usr/bin/ld: cannot find -lQt5::Test
/usr/bin/ld: cannot find -lQt5::Concurrent
/usr/bin/ld: cannot find -lQt5::OpenGL
collect2: error: ld returned 1 exit status
make[2]: *** [bin/FaceLandmarkImg] Error 1
make[1]: *** [exe/FaceLandmarkImg/CMakeFiles/FaceLandmarkImg.dir/all] Error 2

but Qt in system

ubuntu@ip-172-31-1-31:~/OpenFace/build$ apt-file search Qt5CoreConfig.cmake
qtbase5-dev: /usr/lib/x86_64-linux-gnu/cmake/Qt5Core/Qt5CoreConfig.cmake
ubuntu@ip-172-31-1-31:~/OpenFace/build$ qmake --version
QMake version 3.0
Using Qt version 5.2.1 in /usr/lib/x86_64-linux-gnu

qtchooser -print-env
QT_SELECT="default"
QTTOOLDIR="/usr/lib/x86_64-linux-gnu/qt5/bin"
QTLIBDIR="/usr/lib/x86_64-linux-gnu"