GOD AI

• Make the goal of agents harder to reach by modifying environment in order to generalize the AI and avoid overfitting later on real environment
• Tweak the paramaters like life loss per actions, rewards etc. in order to improve the agents performances

I propose to use a Finite State Machine (FSM) code architecture for the heuristic AI
An AI is always in a state, a state has:

• Actions (what to do)
• Transitions which includes decisions (conditions to change state)

Which looks like this

• Living beings with animations
• Environment (Universim-like design would be nice ?) like this:
  

TODO list by order of importance

• Character (model, rig, animations ...)
• Wolf (rig, animations ...)
• Tree boss (model, rig, animations ...)
• Level design
• Base camp
• UI #24

https://circleci.com/docs/2.0/building-docker-images/

to make it faster to deploy Evol and to make sure every commit builds

There are many ways to implement evolution but here is a simple one :
For every kind of agent (herbivorous, carnivorous) :
Each worker is an individual with carecteristics (speed, size,... )
Every N (= 10000 for example or 1000) steps a selection+reproduction is done : it select the 5 individual with the best reward, do the crossover and produce 50 new individual, one by worker.
Reinforcement learning procedure stay the same.

On death and reset of an individual during an evolution step, take the same carecteristics (the mean reward is considered at the end of the evolution step)

Make a script to train on cloud (aws, gc ...)

• clone ml agents
• install everything
• scp the Evol binary + chmod +x Evol.x86_64
• train

tensorboard ?

we are using transform.Rotate wrong
see https://docs.unity3d.com/ScriptReference/Transform.Rotate.html
and https://github.com/Unity-Technologies/ml-agents/search?q=transform.Rotate&unscoped_q=transform.Rotate

first parameter is the axis, the size of the vector representing it doesn't matter
the second parameter is the number of degree to turn

To allow slowing mobs / players and improving navmesh nice link
http://www.theappguruz.com/blog/navmesh-techniques-to-improve-unity-navigation

Evol\UnityProject\Assets\packages\Google.Protobuf\ is generated sometimes when you build, delete it manually if it happens

First player to join the server can control the char perfectly, when another player join, he loses control and control the other player (the other player can control his own gameobject).

Both players are perfectly owning their own player gameobject, maybe script enable/disable issue ...

Cameras are ok

We are mixing 3 axes here :

• The game : with features, graphism, Android compilation,...
• Reinforcement learning : make animals do the right action in a smart way, using the right reward, algorithm, actions and observations
• Evolution : make animals characteristics change based on survival in the environment

It is hard to explore the 3 axes in a single scene/game.
I propose to instead make 3 scenes with a single axe in each scene :

• One with only the game : with interesting features and graphism but no rl and no evolution, behavior of animals will be a basic heuristic (go to nearest food)
• One with only rl : with a fast converging algo producing interesting behaviors but no complex game features and no evolution
• One with only evolution : be able to quickly see animals evolve but with no complex game features and no rl (no training time)

We could then have a few other scenes mixing 2 of the axes or the 3 of them.

That organization will make it possible to make quick progress on these mostly independent problematics and get interesting results faster.

Setup account system with
https://doc.photonengine.com/en-us/realtime/current/reference/custom-authentication
for example ?
With nodejs + mongodb backend ? (shouldn't take too much time)

It could be interesting to implement either from the python side or the C# side communication between agents to make them collaborate to acheive a common goal

Interesting paper about that

Test files are lacking assembly reference
https://stackoverflow.com/questions/50223866/how-to-set-up-unit-tests-in-unity-and-fix-missing-assembly-reference-error

Problem is when following this, we lose ref to the ML-Agent ...

• how to train (what command ?)
• how to install nugget

Follow these steps
Unity-Technologies/ml-agents#1507

Another way to do evolution is to implement it as it works in real life nature : make individual go to each other to reproduce when their life is high enough. Produce a child when they reproduce. No selection phase, only death is the selection.

It is realistic and work well but it can result in herbivorous or carnivorous winning.

What kind of game would it be ?

TODO list dev by order of importance:

• Forest
• Game loop
• Player controller
• Improve network
• Add spells / balance
• AI

or even better, add a script to download it and add it automatically at the right place