[Question] Why is ortho_init=False during hyperparameter searches? about rl-baselines-zoo HOT 2 CLOSED

Hello,

basically universally slightly helpful

in practice, it mostly depends on the other hyperparameters (in my own experience, when doing hyperparameter optimization with ortho_init as an hyperparam, it was not an important one).

There are also other reasons:

• ortho_init does not only do orthogonal initialization, there is also some scaling for the value function last layer, setting it to False removes one trick (the less tricks the better, especially if it does not affect performance), see https://github.com/DLR-RM/stable-baselines3/blob/master/stable_baselines3/common/policies.py#L514
• ortho_init scaling is fixed (and I suspect that this likely not work well for wider/smaller architectures compared to the defaults)
• setting to ortho_init=False reduces the search space

from rl-baselines-zoo.

Thanks a ton

from rl-baselines-zoo.