Can't tune hyperparameters with CustomSACPolicy - multiple values for keyword argument 'layers' about rl-baselines-zoo HOT 5 CLOSED

Hello,
This is normal. If you do hyperparameter tuning, you should set policy='MlpPolicy' otherwise you will get the mentioned error, as the CustomSACPolicy is already custom in term of number of layers, would be nice to change CustomSACPolicy to MlpPolicy but with policy_kwargs="dict(layers=[256,256])"

from rl-baselines-zoo.

Ok thanks for your very quick reply.

from rl-baselines-zoo.

Just a doubt, is it OK then to tune the hyperparameters with policy='MlpPolicy' and then to train the model with CustomSACPolicy? Does it not defeat the purpose of tuning in the first place? i.e. would hyperparameters optimised with one policy be also optimal for another policy?

from rl-baselines-zoo.

Does it not defeat the purpose of tuning in the first place? i.e. would hyperparameters optimised with one policy be also optimal for another policy?

If in your hyperparameter optimization you allow architecture search:

then it does make sense to have policy='MlpPolicy'.
However, if you fix the architecture (by commenting the lines above), then you can use CustomSACPolicy (or in a equivalent way, MlpPolicy + policy_kwargs="dict(layers=[256,256])")

from rl-baselines-zoo.

Ok thanks a lot for your help, I'm closing this issue now.

from rl-baselines-zoo.