Value of Epsilon Decay Period about dopamine HOT 3 OPEN

Also, for the JAX Full Rainbow agent (which has Noisy Nets), and when using Noisy Nets, epsilon greedy is disabled (as in paper snippet above, as well as some other implementations like Kaixhin Rainbow here and here). However, I still see the epsilon_train set to 0.01 in JAX Full Rainbow (here) and if Noisy is true, the identity_epsilon function is called which just returns the epsilon value (but doesn't uses 0).

from dopamine.

thank you for pointing this out! this has been fixed here: ed92c57

from dopamine.

Thanks! As for

Is there a Is there a discrepancy here (Rainbow should anneal within 62k steps and not 250k steps), or am I misunderstanding something (or perhaps it really doesn't matter?)

Should the epsilon_decay_period value for TF Rainbow (which does not use Noisy Nets) be 250k frames as in the Rainbow paper (which makes it 62500 steps with frame_skip=4) or 250k steps (as in current implementation) or perhaps it does not matter)? I have rarely seen a value as low as 62500 steps for epsilon decay, for example RLlib also uses 200k for its DQN variant and epislon greedy exploration is off when using Noisy Nets.

from dopamine.