In the proposed algorithm, called Coupling Policy Gradient with Population-based Search (PGPS), a single TD3 agent, which learns by a gradient from all experiences generated by population, leads a population by providing its critic function Q as a surrogate to select better performing next generation population from candidates. On the other hand, if the TD3 agent falls behind the CEM population, then the TD3 agent is updated toward the elite member of CEM population using loss function augmented with the distance between the TD3 and the CEM elite.

1. Create and activate conda environment

cd PGPS-master # Download or clone whole files
conda env create -f environment.yml
source activate pgps

2. Main packages' versions

'gym==0.16.0' 
'mujoco-py==2.0.2.9' 
'torch==1.2.0+cu92' 

environment_rllab.yml 
'gym==0.10.0' 
'mujoco-py==1.50.0.1' 
'torch==1.2.0+cu92' 

The environment should be ready to run. See examples section for examples of how to train an agent and play the pretrained one.

'Hopper-v2'
'HalfCheetah-v2'
'Swimmer-v2'
'Ant-v2'
'Walker2d-v2'

python Main.py -env_name "HalfCheetah-v2" -seed=1 -ada_steps -guided -q_surr

'learning_curves' Folder

python Play_Model.py -env_name "HalfCheetah-v2" -seed=1 -render=True