networks weight decay about sfmlearner HOT 7 CLOSED

Cool! Thanks for pointing it out. I think you are right that the weight decay is not applied properly here. Have you tried playing with various weight decay values in your implementation?

from sfmlearner.

Btw, would you like me to include a pointer to your pytorch implementation in the README? I think it would be helpful for people who prefer pytorch to find it.

from sfmlearner.

I'm still not able to reach your score with my implementation. The weird thing is that when I set a smooth loss of 0.5, everything goes to 0 pretty quickly and never come back. I have a similar output pattern than your hyper parameters when I select a smooth loss of 0.1 and convergence is nice, but results are still not as good as yours. I testes different values of weight decay, without much success for the moment, the search is still ongoing :)

And for the pointer, that would be super cool, thanks ! As code is still moving, I'd be happy to have people test it and try some hyperparameters to see what converges best :)

from sfmlearner.

Cool. Just added a pointer. I would love to be posted on your re-implementation effort as well :)

from sfmlearner.

Done !

from sfmlearner.

@ClementPinard About the weight decay configuration, I experimented a bit and found no big difference. A weight regularizer more than 1e-4 deteriorates the performance, while smaller choices give no improvement.

from sfmlearner.

@yzcjtr , thanks for sharing your finding!

from sfmlearner.