Similarity is above 50% with 2 different persons about human HOT 5 CLOSED

@Donymak Thanks for your amazing explanation.

I was doing more internal tests I decide to try vlad's face-api package and for my surprise, similarity level lowered to 20% with all default settings and exactly same images.

I don't know if this happens because of models used on face-api or if is because it uses only 68-point face mesh while human uses 468-point but comparing both results face-api seems to be more precise on comparing two faces.

from human.

i believe insightface is most precise, but also most sensitive to cropping and preprocessing in general.

from human.

In my comment, I want to share what I found out about face matching in Humans and in general.

To be honest the faces are similar, but the persons are different. Usually models do not take into account gender, hair and anything else exept the FACE itself. Getting high match scores is ok in this case. Taking into count that the model for creating face embeddings is not the strongest and the most precise in the world)

The face matching can be affected by:

• Face angle
• Emotions
• Image scale and resolution(dimensions)

The last factor in your case can slightly affect the creation of face embeddings but not more than 0.05 as it happened usually during my tests on different images.

Also I noticed that cropping an image to the bounding box of the face on the image and using that image to get embeddings can sometimes increase accuracy although the library should do it under the hood.

Human:
Have a look at the different options for face comparasing using Human on extended code sandbox fork.

As you can see there, removing the order option from the match function changes the similarity score to 0.45 and it is not a match anymore)

For my project, I considered using cosine similarity to compare face embeddings instead of Euclidean or Manhattan Distance based similarity metrics. In the code sandbox I installed the package with this funciton.

In general:
Also, it was interesting for me to test images using different tech, here is what I got.

AWS Rekognition:
Uncompressed: "Similarity": 0.7741245627403259,
Compressed: "Similarity": 0.8348438739776611,
In both cases, Amazon does not match the result, but the similarity value is pretty high.

Python FaceNet with vggface2 model:

Cosine similarity between img1 and img2: 0.12417466938495636
Cosine similarity between img1 and img3: 0.12017473578453064

The similarity methics from this model are percise as it was trained on 3.3M images and model is the best i found as free open source.

The most "percise" hovewer is the Regula Face SDK Demo with score of
"similarity": 0.0084031
But it may use additional normalisation to move all the results to extra range as 0-0.1 for not match and 0.9-0.999 for match based on the different similarity value that comes out of private model.

from human.

@Donymak thanks for the write-up, its pretty much spot on.

one correction - number of face-points in mesh (68 vs 468) is not related to face recognition at all, those are separate models.

one thing to add is that different face recognition models have different levels of sensitivity to crop factor and face rotation angle and also prefer a very different values to start with. in some cases, those can be quite significant.

@nonam4 did you try using different face regnonition modules in human? there are several supported. default one is just default because its light enough to run without major impact, but there are better ones - its always a tradeoff between performance and precision.

from human.

@vladmandic No I didn't tested with other models. Based on your knowledge, do you have any suggestions on which model I should test first?

from human.