author : Samuel Berrien

The final goal of this project is to implement any kind of optimisation algorithm in order to teach creature to walk, run etc.

This project requires multiple libraries.

• physic engine stuff : Bullet
• graphic stuff : OpenGL, GLM, GLEW, GLFW
• reinforcement learning : LibTorch CXX 11 ABI (not pre-cxx 11 ABI) according to your CUDA version. Put libtorch folder in /opt/, (see cmake files)
• utils stuff : p-ranav argparse, p-ranav indicators, nlohmann json

Install dependencies with pacman :

$ # as root
$ pacman -Syu cmake bullet glm glew glfw

Download libtorch from torch official website and then copy the extracted folder in /opt/ (here version 2.3.0 with CUDA 12.1) :

$ # as root
$ unzip /path/to/libtorch-cxx11-abi-shared-with-deps-2.3.0+cu121.zip -d /opt/

Any attempt to build this project on other OS will be appreciated to complete this section.

To build this project you need a decent C++ compiler with cmake and make. An internet connexion is also required in order to download dependencies inside cmake file.

1. Clone the project

   $ git clone https://github.com/Ipsedo/EvoMotion.git

2. Then build the project:

   $ cd /path/to/EvoMotion
   $ mkdir build && cd build && cmake .. && make

3. Ready for training ! You can choose the environment on which train the agent you want.

   Run training on 3D cartpole

    $ cd /path/to/EvoMotion/build
    $ evo_motion cartpole3d actor_critic --seed 1234 --cuda --hidden_size 32 train ./out/cartpole3d_a2c --episodes 1024 --nb_saves 1024 --learning_rate 1e-3

   Run training on creature muscles

   $ cd /path/to/EvoMotion/build
   $ evo_motion muscles actor_critic_liquid --seed 1234 --cuda --hidden_size 32 train ./out/muscles_a2c_liquid --episodes 1024 --nb_saves 1024 --learning_rate 1e-3

4. After the first save (here after 1024 episodes), you can now evaluate your trained agent.

   Evaluate agent on 3D cartpole (here the first model save) with GLFW window of 1920 * 1024 pixels

   $ evo_motion cartpole3d actor_critic --seed 1234 --hidden_size 32 --cuda run ./out/cartpole3d_a2c/save_0 -w 1920 -h 1024

   Evaluate agent on creature muscles (here the first model save) with GLFW window of 1920 * 1024 pixels

   $ evo_motion muscles actor_critic_liquid --seed 1234 --hidden_size 32 --cuda run ./out/muscles_a2c_liquid/save_0 -w 1920 -h 1024

[1] Reinforcement Learning: An Introduction - Richard S. Sutton and Andrew G. Barto - Second Edition 2018

[2] Liquid Time-constant Networks - Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, Radu Grosu - 8 Jun 2020