Comments (8)
Hi,
What you're trying to do with initialising using the previous points was also my first intuition, and similar to you, it didn't work for the reason you mentioned. I think what's happening is the following:
- Initialising with the mean landmarks in each frame acts as a kind of regularisation, we go back near the space of valid faces. If you're curious, what you could try is train a model and train it with a high regularisation value, so the training error would be quite large - and then see if the landmarks also get scattered in your testing. I guess this would help analyse this behaviour.
- If you're initialising with the previous frame, the landmarks will be fairly close to the true location already (depending on how much the subject moves obviously). This is something that the regressor (or at least the first) actually has not learned from the training data. In the training, we initialise from the face-boxes, and thus quite far away from the true location. So if we're already close, the regressor in a sense can't really know the right thing to do. What I would do here is re-train a model with a different initialisation strategy, i.e. perturb around the ground-truth location of the training landmarks, and not around the face box. I've started this because I think it's a neat idea but never finished it so far.
I hope this helps, let me know how it goes and I'm glad to help more, I'm quite curious as well. (It's unfortunately the stuff that never gets mentioned in papers, for example Xiong & De La Torre don't disclose this information in their paper. But I'm actually pretty sure my idea should work (and might also be exactly how they do it)).
from superviseddescent.
Thank you for your detailed response. I'm working to implement your ideas. I'll post here if I manage to make any headway. Thanks again!
from superviseddescent.
Cool to hear that! Just let me know if you need any help.
from superviseddescent.
I have ever implement the idea "perturb around the ground-truth location of the training landmarks", however the tracking for rotation was week. So, did you have some better idea for simulating the perturbation of rotation.
from superviseddescent.
You can correct for rotation by in-plane rotating the input image. That way you don't need to train it. I think there's a discussion about it in some of the (possibly closed) issues here.
Alternatively you can try perturbing in the training with a 2D rotation.
I'm closing this since the original question is resolved, and the new question is kind of off-topic.
from superviseddescent.
from superviseddescent.
I see.
The tracking works well enough for my purposes. Landmark detection isn't my main topic. I am focussing my effort on eos, the 3D fitting library.
from superviseddescent.
from superviseddescent.
Related Issues (20)
- Parallelise CalculateHogDescriptor HOT 2
- eigen function runs very slow HOT 3
- save trained model with non-C++11 syntax HOT 10
- question: CPU time needed for hog feature in SDM code HOT 2
- Confidence measure of landmark detection? HOT 2
- Question: about pose estimation HOT 5
- Questions: how to optimize the speed of landmark detection code HOT 4
- Run rcr-train on my own data HOT 16
- Error Building in Visual Studio RC2015 on Windows 10 HOT 3
- Crashing after a face is detected with rcr_track.cpp HOT 5
- Way to detect whether the detected landmarks are actual landmarks on a face HOT 3
- landmark detection.cpp trained result looks like mean shape. HOT 2
- Pose estimation for an input image HOT 4
- 68 landmark points pre-trained model HOT 3
- How can i get mean_ibug_lfpw_68.txt in new dataset? HOT 1
- Inter-eye Distance Normalization HOT 3
- Pre trained model for 2d landmarking tracking HOT 1
- Training with 10K images HOT 1
- Bug suspicion HOT 3
- Learning the bias B_k HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from superviseddescent.