Repository from the paper A review on Deep Reinforcement Learning for Fluid Mechanics.
List of other repo. with CFD + DRL (if your code is not here, please feel free to make a pull request)
(Explanatations on how to use the code are below)
| Fenis + DRL | https://github.com/DonsetPG/fenics-DRL |
|---|---|
| Flow Control of the 2D Kármán Vortex Street with Deep Reinforcement Learning | https://github.com/jerabaul29/Cylinder2DFlowControlDRL |
| Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach | https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel |
| Deep Reinforcement Learning control of the unstable falling liquid film | https://github.com/vbelus/falling-liquid-film-drl |
| Direct shape optimization through deep reinforcement learning | https://github.com/jviquerat/drl_shape_optimization |
| Fluid directed rigid ball balancing using Deep Reinforcement Learning | https://github.com/sahilgupta2105/Deep-Reinforcement-Learning |
| Efficient collective swimming by harnessing vortices through deep reinforcement learning | https://github.com/cselab/smarties |
| Training an RL agent to swim at low Reynolds Number | https://github.com/RpDp-git/LearningToSwim-DQN |
We used Fenics for this project. The easiest way to install it is by using Docker. Then :
docker run -ti -v $(pwd):/home/fenics/shared -w /home/fenics/shared quay.io/fenicsproject/stable:current
should install Fenics.
We are using both Gym - OpenAI and Stable-Baselines. They both can installed with :
pip install gym
pip install stable-baselines
More generally, everything you need can be installed with :
pip install --user tensorflow keras gym stable-baselines sklearn
An experiment consists of an Environement (based on Gym - OpenAI & Fenics), and an Algorithm from Stable-Baselines. They can be launched with test_main.py. You will only have to precise a few parameters :
*nb_cpu*: the amount of CPU you want to use (e.g. 16)
*agents*: an array of the algorithms you want to use (e.g. ['PPO2','A2C'])
*name_env*: The name of the environment (e.g. 'Control-cylinder-v0')
*total_timesteps*: the amount of timesteps the training will last (e.g. 100000)
*text*: Some precisions you wanna add to you experiment (e.g. '1_step_1_episode_2CPU')You can find examples of such environments in example 1 : Control Cylinder or example 2 : Flow Control Cylinder. They always share the same architecture :
class FluidMechanicsEnv_(gym.Env):
metadata = {'render.modes': ['human']}
def __init__(self,
**kwargs):
...
self.problem = self._build_problem()
self.reward_range = (-1,1)
self.observation_space = spaces.Box(low=np.array([]), high=np.array([]), dtype=np.float16)
self.action_space = spaces.Box(low=np.array([]), high=np.array([]), dtype=np.float16)
def _build_problem(self,main_drag):
...
return problem
def _next_observation(self):
...
def step(self, action):
...
return obs, reward, done, {}
def reset(self):
...Here, most of these functions are DRL related, and more informations can be found at this paper (for applications of DRL on fluid mechanics) or here (for more general informations about DRL). The only link with Fenics is made with the
def _build_problem(self,main_drag):
...
return problemfunction, where you will be using functions from Fenics.
We built several functions to help you use Fenics and build DRL environment with it. Three main classes exist :
- class Channel
- class Obstacles
- class Problem
Allows you to create the 'box' where your simulation will take place.
Allows you to add forms and obstacles (Circle, Square and Polygons) to your environment.
Build the simulation with Channl and Obstacles. Also get parameters for the mesh and the solver. Finally, this is a Problem object you will return in the Gym.env class.
We built this repository in order to get a code as clean as possible for fluid mechanics with DRL. However, Fenics is not the best solver, especially with very demanding problem. The goal is to keep the same philosophy in mind (DRL and Fluid mechanics coupled easily) but with other (and faster) libraries. Since most of these libraries are C++ based, and using powerful clusters, the architecture will be completely different. We are still working on it and doing our best to release an alpha version as soon as possible.
This repository will be updated when such library finally comes out. Until then, we hope that with this paper and this repository combined, some Fluid Mechanics researcher might want to try to apply Deep Reinforcement Learning to their experiments.
- Paul Garnier : MINES Paristech - PSL Research University
- Jonathan Viquerat : MINES Paristech - PSL Research University - CEMEF
- Aurélien Larcher : MINES Paristech - PSL Research University - CEMEF
- Elie Hachem : MINES Paristech - PSL Research University - CEMEF
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent