Hi @melgor - thanks for reporting this!
The negDist < alpha is indeed inconsistent with the FaceNet paper.

I've updated the code to your second suggestion and added some output info about how many random
negatives are selected in the offline processing. Also posted this to the mailing list at https://groups.google.com/forum/#!topic/cmu-openface/4yk9dO1Z0ww

Interestingly the conditions with negDist < alpha is satisfied about half the time in the early phases of training, which is what I looked at when I validated this portion of code. For the same random seed and images, Suggestion 1 gives 513 random triplets out of 1006 total triplets, Suggestion 2 gives 490, and the bug (negDist < alpha) gives 637.

I tested this for later iterations with the nn4.v1 model and the bug is much closer to random as you mention. Suggestion 1 gives 498 random triplets out of 996 total, Suggestion 2 also gives 498, and the bug gives 974!

I've been training a new model with suggestion 1 over the past few days, interested to see how much this improves the accuracy.

-Brandon.

from openface.

Re-opening, I'm having some strange accuracy issues with this. @melgor - have you successfully trained a model with your Suggestion 2?

In the following, I switched from your Suggestion 1 to suggestion 2 at about iteration 45. I don't yet understand why suggestion 2 causes such bad performance. Any ideas?

Since suggestion 1's model was starting to perform better, I've made this the default version for now.

Iter 40

• 0.73 avg LFW accuracy

Latest Iter

• 0.51 avg LFW accuracy.

from openface.

It it really strange. I have one idea why it may work so bad, but still it
will not explain the random guess. I must look it closer.

About my tests, I do not have 12 GB on my card, so I only test 2-step
training (like in Oxford Face paper). And here the result using all three
different sampling strategy are similar and does not produce better feature
representstion than original net. So sth is wrong here.

I think that you should stay at Suggestion 1, it may be better is some
cases.

9:58 PM pon., 09.11.2015 Brandon Amos użytkownik [email protected]
napisał:

Re-opening, I'm having some strange accuracy issues with this. @melgor
https://github.com/melgor - have you successfully trained a model with
your Suggestion 2?

In the following, I switched from your Suggestion 1 to suggestion 2 at
about iteration 45. I don't yet understand why suggestion 2 causes such bad
performance. Any ideas?

Since suggestion 1's model was starting to perform better, I've made this
the default version for now.

[image: loss]
https://cloud.githubusercontent.com/assets/707462/11046106/631a61e6-86f9-11e5-8735-ed6f87b00f15.png
Iter 40

• 0.73 avg LFW accuracy

[image: roc]
https://cloud.githubusercontent.com/assets/707462/11046189/ecfe92b0-86f9-11e5-8d99-ef577e36bb37.png
Latest Iter

• 0.51 avg LFW accuracy.

[image: roc]
https://cloud.githubusercontent.com/assets/707462/11046103/5f4e8286-86f9-11e5-83e2-9987d73330ff.png

—
Reply to this email directly or view it on GitHub
#48 (comment)
.

from openface.

@melgor - suggestion 1 is working well so far. Averaging ~300 random triplets out of 1000 in the offline sampling and almost at the previous net's accuracy.

Epoch 55

• accuracy: 0.7773
  

Epoch 54

• accuracy: 0.7915
  

from openface.

Training with suggestion 1 is clearly faster as the experiment from the past week shows, closing this issue. The accuracy hasn't surpassed the existing models, but I'm going to keep the training going.

LFW Experiment, Epoch 142

• Accuracy: 0.8390

from openface.