kzl / lifelong_rl Goto Github PK

Hi,
Thanks so much for the open-sourced work. I fail to run the experiment without REPO_DIR/aws_config/config.ini. Could you please share the config files?

Hi,

Thanks for providing this wonderful package !

I notice that in line 114 of lifelong_rl/lifelong_rl/trainers/mbrl/mbrl.py you have

self.ensemble.elites = np.argsort(holdout_losses)

AFAIK this line of code will not achieve the goal of selecting the elite models based on the holdout_losses since all models will participate in the 'fake' rollouts, as shown in the function sample in the base class of model lifelong_rl/lifelong_rl/models/networks.py/ParallelizedEnsemble.

Can we adjust this line of code to the following ?

self.ensemble.elites = np.argsort(holdout_losses)[:self.num_elites] 

I think this will modify the list of elites in the base class to include, say, top 5 models out of total 7.

I haven't test this thought yet and please let me know if I miss anything.

Best,
Shentao