This repository contains the work I did during my Reinforcement Learning internship from September 2017 to February 2018.

During these 6 months, I reimplemented the main deep-RL algorithms that have been developped since 2013, using only Tensorflow and numpy. This repository contains implementations of :

• A3C : the 2016 algorithm that uses asynchronous gradient descent for optimization on multi-CPU instead of a single GPU
• C51 : the 2017 algorithm that explores the idea of predicting not only the value of a state, but instead the value distribution
• DDPG : the 2015 algorithm that tackles the problem of continuous control using an actor-critic architecture
• Rainbow : the 2017 algorithm that combines six classical extensions to DQN
• D4PG : the 2018 algorithm that applies the distributional approach to a DDPG with an asynchronous architecture

The general architecture of these algorithm is always the same :

• the main.py file initialize the agent and run it
• the Model.py file implements the Neural Network (actor-critic or not, with convolution or not)
• the QNetwork.py file instantiates a Network and build the tensorflow operations to perform the gradient descent to train it
• the Agent.py file implements the agent class that interacts with the environment in order to get experiences
• the settings.py file is used to change the hyperparameters of the algorithm and the network

Others directories include :

• utils : a set of classes and functions used in other algorithms
• BlogFiles : a jupyter notebook that tries to explain the idea behind A3C, DDPG and Rainbow
• Environment Test : copies from the main algorithms set up to run in specific environments
• GIF : a set of GIF saved after having trained different agents on many environments