This repository contains PyTorch implementations of deep reinforcement learning algorithms. This implementation uses PyTorch. For a TensorFlow implementation of algorithms, take a look at tsallis_actor_critic_mujoco.

1. Deep Q-Network (DQN) _{^{(V. Mnih et al. 2015)}}
2. Double DQN (DDQN) _{^{(H. Van Hasselt et al. 2015)}}
3. Advantage Actor Critic (A2C)
4. Vanilla Policy Gradient (VPG)
5. Natural Policy Gradient (NPG) _{^{(S. Kakade et al. 2002)}}
6. Trust Region Policy Optimization (TRPO) _{^{(J. Schulman et al. 2015)}}
7. Proximal Policy Optimization (PPO) _{^{(J. Schulman et al. 2017)}}
8. Deep Deterministic Policy Gradient (DDPG) _{^{(T. Lillicrap et al. 2015)}}
9. Twin Delayed DDPG (TD3) _{^{(S. Fujimoto et al. 2018)}}
10. Soft Actor-Critic (SAC) _{^{(T. Haarnoja et al. 2018)}}
11. Automating entropy adjustment on SAC (ASAC) _{^{(T. Haarnoja et al. 2018)}}
12. Tsallis Actor-Critic (TAC) _{^{(K. Lee et al. 2019)}}
13. Automating entropy adjustment on TAC (ATAC)

1. CartPole-v1 _{^{(as described in here)}}
2. Pendulum-v0 _{^{(as described in here)}}
3. MuJoCo environments (HalfCheetah-v2, Ant-v2, Humanoid-v2, etc.) _{^{(as described in here)}}

• Observation space: 4
• Action space: 2

• Observation space: 3
• Action space: 1

• Observation space: 17
• Action space: 6

• Observation space: 111
• Action space: 8

• Observation space: 376
• Action space: 17

• PyTorch
• TensorBoard
• gym
• mujoco-py

The repository's high-level structure is:

├── agents                    
    └── common 
├── results  
    ├── data 
    └── graphs        
├── tests
    └── save_model

To train all the different agents on MuJoCo environments, follow these steps:

git clone https://github.com/dongminlee94/deep_rl.git
cd deep_rl
python run_mujoco.py

For other environments, change the last line to run_cartpole.py, run_pendulum.py.

If you want to change configurations of the agents, follow this step:

python run_mujoco.py \
    --env=Humanoid-v2 \
    --algo=atac \
    --seed=0 \
    --iterations=200 \
    --steps_per_iter=5000 \
    --max_step=1000

To watch all the learned agents on MuJoCo environments, follow these steps:

cd tests
python mujoco_test.py --load=envname_algoname_...

You should copy the saved model name in tests/save_model/envname_algoname_... and paste the copied name in envname_algoname_.... So the saved model will be load.