Some labels are displaced in ELFW dataset about elfw HOT 10 CLOSED

First time I see it.
Do you have a list of cases so we can check?

from elfw.

No, I just noticed by manual inspection. But I named the two faces from the baseline directory. I might try realigning them, but I don't know yet whether it is worth the effort. In case I do, I will generate a list of displaced faces in the process which I can share. But for now: Could you quickly verify that maybe I simply got an invalid version of the dataset? (I requested the dataset using this form.) Because if a correct version of the dataset exists, it could save me some time :-)

from elfw.

Your are right! There's a shifting in those two cases.

Some of the faces in the original LFW dataset have only skin or hair, so they were not included in the upgrade-refinement process. Note that the ELFW version (left) and the original LFW version (right) are exactly the same (a part from the color codes).

So those are shiftings from the source. I'll send the LFW team a request on this regard.

In the meantime, since you already have that script ready, could you please go over the faces and print a list of names to see the extent of the tragedy?

from elfw.

I did some investigation and as you said, the problem lies with the labels that only contain [SKIN, BG, HAIR] and were hence not refined. I looped through the ELFW dataset and checked all labels that contained some of [HEAD_WEARABLE, BEARD] as this indicated that these labels were refined labels. They seem to be fine.

I checked the different versions of the original LFW faces and I think I have found the issue. The labels in the ELFW dataset seem to correspond to the funneled-LFW faces. However, ELFW uses the deep-funneled-LFW faces. I combined the masks from ELFW with the corresponding faces of funneled-LFW for Aaron_Sorkin_0001.jpg and Abdoulaye_Wade_0004.jpg and now they are correctly aligned:

The question is: How easy is it to re-generate the ELFW dataset based on the funneled-LFW dataset? (Because all augmentations with hands/sunglasses/facemasks that are based on the mismatched labels are wrong as well.)

from elfw.

Thank you for your insight @zenoli. That seems like a probable explanation.
Curiously, we used superpixels to relabel the other cases and we didn't notice any misalignment.

If this is true, a simple way to go is to replace only misaligned faces with the funneled versions. The resulting dataset would have both, funneled and deep-funneled faces. But it shouldn't be a problem as far as they can be tracked back, with a log file for instance.

As you said any augmentation on the misaligned labels are wrong as well, since eyes and mouth would lay in a different location.

Re-generating the EFLW, I mean replacing misalignments and augmenting, is super easy with these scripts. It takes minutes. However, it's worth making the effort to preserve the same statistics shown in the report.

I'll do it anytime soon so a second version of ELFW can be released.

from elfw.

No problem :-)
Could you give a rough estimate of "anytime soon"? Not wanting to stress, but I would have to know whether I need to do it myself or whether I can wait :-)

from elfw.

Roughly a week.

from elfw.

Done. The ELFW has been corrected and will be released as ELFW v2.0.

Unfortunately the same face-object pairs couldn't be exactly regenerated because faces are not equally detected after being aligned. See further explanations at the Project Site and in the attached document -included a list of the funneled faces- in the ELFW2 dataset.

All users who already requested the dataset have been provided a new download link.

Thanks @zenoli for reporting this issue.
While no other news, I'll proceed to close this issue.

from elfw.

Thank you very much!
So in ELFW 2.0, all you do is replacing the problematic deep-funneled LFW-faces with the funneled LFW-faces and keep the labels from ELFW 1.0? It's not the other way round, correct? (By "the other way round" I mean refining the problematic labels so that they match the deep-funneled images again).

from elfw.

All faces should be plain funneled because labels and superpixels are aligned just with them. And that's why I said: "Curiously, we used superpixels to relabel the other cases [deep-funneled] and we didn't notice any misalignment." Either way, extended faces in ELFW 1.0 are apparently correct and the funneled ones -as you pointed out- did the trick for the remaining skin-hair-only LFW-faces. Please have a look and if any bug is still persistent in ELFW2, please report it.

from elfw.