This repository is forked from the original OpenAI spinningup repository to implement changes required for the State Augmented Contstrained RL, as well as the more robust adaptive entropy penalty for SAC.

SpinningUp contains a code repo of the implementation of key Reinforcement Learning algorithms including Soft Actor-Critic, Proximal Policy Optimization and Twin Delayed DDPG used in Augmented Contstrained RL project. Visit spinningup.openai.com for more information on Spinning Up.

First Install OpenMPI:

sudo apt-get update && sudo apt-get install libopenmpi-dev

Now install the spinningup repo:

git clone https://github.com/flodorner/spinningup.git
cd spinningup
pip install -e .

The following changes were implemented in this fork:

• Added GPU support for PyTorch to SAC, PPO and TD3.

• Implemented adaptive entropy penalty for SAC as an alternative to the fixed entropy regularization parameter. More information about the entropy constraint can be found in this paper.

• Implemented data augmentation using the known cost dynamics to boost sample efficiency.

• Added adaptive sampling data augmentation to prevent bias in data augmentation.