Default config (emoji.yaml) fails to converge about im2vec HOT 7 CLOSED

Hey @evancasey
This tends to happen with high learning rates, could you try decreasing the learning rate https://github.com/preddy5/Im2Vec/blob/master/configs/emoji.yaml#L23 to 0.0001 and see if the changes anything.

from im2vec.

I'm starting to see better results after reducing LR to 0.0001, however it takes much longer to train than your logged results. Would you suggest tuning LR some more or are there any other hyperparameters that come to mind?

from im2vec.

After 500 epochs, LR=0.0001

from im2vec.

After 2000 epochs it still does not converge

from im2vec.

Hey @evancasey I think you might have to tune the LR a bit, could you try 0.0002 maybe, also increasing https://github.com/preddy5/Im2Vec/blob/master/configs/emoji.yaml#L8 the radius helps with the initial part of the optimization maybe try 20. Hopefully, this helps with the optimization.

from im2vec.

Same here. Tried everything. Almost 3000 epochs and it still does not converge.
There are 3 logical possibilities:
1 - The code in this repository is not the one used in the paper (i.e. notebooks?)
2 - There is a secret way to tune those parameters with a separate tool/script
3 - The results in the paper are made up
Excluding the third, both the 1st and the 2nd explanations are easy to solve.
You just need to merge/add the source code of the notebook or of the tool with this repo.
Are you still under deadlines? I hope you will find a moment to fix this. This is an important work.
Thanks. 👍

from im2vec.

Hey @Emasoft
There are many reasons why the network might not have converged when you train it. 
The repo includes an example training run please use that for debugging your training. For each new training run, the code also creates a copy of the training code in the log directory, so the example training run also contains a backup of the code used for that specific training and trained weights, you can use them for debugging as well. 

Hope that helps.

from im2vec.