How to choose the best iter? about transferable-interactiveness-network HOT 4 CLOSED

In fact, we cannot know the exact 'best' iteration, just use the experience and the loss curve to choose an experiential iteration.
According to our experience, when using cosine lr decay, a good iteration can be found at about 3~5 complete annealing decay. Usually, we set the first decay as 2 epochs of the training set. More details can be found in the paper: SGDR: Stochastic Gradient Descent with Warm Restarts.

from transferable-interactiveness-network.

Thanks!
What about training v-coco, I see that your just train 20000 iteration and the result of 6000 iteration preforms a good result. However, when I training the vcoco dataset, I found the loss continued to go down when the iteration reached 20000 and the testing result was not good.

from transferable-interactiveness-network.

A more detailed analysis of the classes may provide you more clues about the fitting state of the model, some classes' performances begin to degrade after xx iterations, meanwhile, some other classes are still increasing. V-COCO is quite smaller, it exactly needs much fewer iterations to achieve a good point and has a very early stopping in our experiment.

from transferable-interactiveness-network.

Thanks a lot!

from transferable-interactiveness-network.