Evaluation interval in MT10/MT50 benchmarks for multi-task RL about cagrad HOT 4 CLOSED

Hi, thanks! I think you can modify the eval frequency in the mtrl config file here .
But I think the default eval frequency is indeed 10,000 training steps (Maybe you can make sure you are on the right commit).

from cagrad.

Thank you for the quick response!
The eval frequency is set to 10,000 in my local repo, similar to your reference. But still log files have evaluations at 30,000 steps. I noticed that there are 150 steps per episode, which suggest that some evaluations episodes have to be run before finishing a training episode (since 10,000 is not a multiple of 150). I am wandering whether this is the issue. Is this the correct number of steps used in the original implementation? I could not see a config option to set the steps per episode.

from cagrad.

Hi, I started a quick a run and it does seem that what you mentioned is the issue. So even if we set eva_freq to 10,000, since here the evaluation happens inside an if-statement where step has to be a multiple of max_episode_steps, the actual evaluation interval will be 30,000. Sorry that in the paper we just report the config parameter we set, but in actual implementation it is indeed 30,000 steps per evaluation because of mtrl's implementation. You could set the max_episode_steps to 100, but that might influence the final evaluation, so I would suggest not doing that.

from cagrad.

Hi, thanks again for the quick response, and thanks for the clarification!
Yes, I think that makes a lot of sense. And yes, changing the per episode number of steps will affect the evaluation. I will look into further whether I can avoid this while having an evaluation frequency of 10,000. But I will close this issue, since my original concern was clarified. Thanks again for your time!

from cagrad.