The duckietown-RL-transformer repository provides solution for lane following in the Duckietown environment. It includes train, evaluation and test script for a hybrid CNN - Transformer (GTrXL) machine learning model, which is supposed to control duckiebot. The solution builds upon the Ray RLlib framework for reinforcement learning (RL) and Weights & Biases framework for logging.

Simulation	Real world

Steps for Native (or in virtual env) setup

0.1 (Optional) Make your venv.

sudo apt update
sudo apt install python3.8-venv
cd to/your/project/path
python -m venv .venv_duckietown/  #create venv
source .venv_duckietown/bin/activate  #activate it

0.2. Get the repository and step into it.

git clone https://github.com/TothAron/duckietown-RL-transformer-thesis.git
cd duckietown-RL-transformer-thesis/

0.3. Install dependecies

pip install -r requirements.txt

1.0. (Optional) Make sure you activate venv and got to the right path, if not:

cd to/your/project/path
source .venv_duckietown/bin/activate  #activate it
cd duckietown-RL-transformer-thesis

1.1. Login to your Weights and Biases account (if do not have one, create at: https://wandb.ai)

wandb login # paste your token from wandb.ai/authorize

1.2. Set your settings and hyperparameters in train config file at:

dev_and_test/config/default.py

1.3. Start training. ✈️

CUDA_VISIBLE_DEVICES=<gpu_idx> xvfb-run -a -s "-screen 0 1400x900x24" taskset --cpu-list <start_cpu_idx>-<end_cpu_idx> python dev_and_test/run_train.py

NOTE: <gpu_idx> and <start_cpu_idx> can be selected with the help of nvidia-smi and htop unix commands relatively.

1.4. The only thing is left to check your training logs live at wandb.ai

2.1. Setup your testing configuration at:

dev_and_test/config/test_config.py

2.2. Start testing 👓

CUDA_VISIBLE_DEVICES=<gpu_idx> xvfb-run -a -s "-screen 0 1400x900x24" taskset --cpu-list <start_cpu_idx>-<end_cpu_idx> python dev_and_test/run_test.py

2.3. Check evaluation results at:

evaluation_results/ 

The used environment wrapper modules and the evaluator for testing are the modified versions of:

https://github.com/kaland313/Duckietown-RL
MIT License Copyright (c) 2019 András Kalapos