The need for maintaining standard Market1501 evaluation set splits? about open-reid HOT 7 OPEN

Maybe I did not understand your requirements. You want to reproduce the results of paper In Defense of the Triplet Loss for Person Re-Identification? If it's true, you have to keep those important details the same as in the paper, e.g.

• Two cascaded embedding layers, with dimension 1024 and 128 respectively
• base lr set to 3e−4
• The number of training iterations

from open-reid.

Thank you very much for pointing out this issue! For Market-1501 and DukeMTMC, our current code choose query according to the person IDs, rather than the images. This could have more query images than originally defined, and indeed causes a mismatch with the official evaluation settings. However, the difference between our evaluation setting and the official one is usually less than 1% mAP according to my test. And arguably splitting by person IDs is better IMHO.

Nevertheless, your solution is really good, but I think we might need to do a bit more for a new split protocol that works for all the datasets. I think one way is to store the filenames directly to split.json. I will soon figure out a way to use exactly the official settings on these two datasets. Thanks again for the suggestion!

from open-reid.

Nice explanation, now it makes sense to me that more queries means better averaging and leads to more stable evaluation results.

And my quick and rough modification to the Market1501 class is only for my own verification (I used the open-reid API functions in another experimental project with standard Market1501 eval splits), so it's not universal at all :)

Thanks for your kind help!

from open-reid.

@Cysu @huanghoujing Hi, I maintain standard Market1501 evaluation set splits and train the model with the setting of TriNet(i.e. one GPU, 18 identities x 4 instances/id), but I get rank1 78.5%. Could you give me some suggestions on reproducing the results of TriNet with open reid?

from open-reid.

• You can try to increase number of identities to 32 (~9600MB GPU) or 64 (two 12GB GPUs).
• Set the margin to 0.3 and then train for longer time, e.g. base learning rate 2e-4 for 150 epochs then decrease it exponentially (as in open-reid) till 300 epochs.

from open-reid.

I have tried to increase the number of identities and got a better rank1, but I have the doubt that if I want to utilize the reproduced result for comparison, Is it proper to use more identities?

from open-reid.

Yes, I want to reproduce the results of the paper you mentioned. Seems that a longer training time help sometimes. Thanks a lot!

from open-reid.