Deep Reinforcement Learning AI controller that plays the SuperTuxKart race using only inputs available to the user (images, velocity, and rotation). The controller is based on a Decision Transformer. The SuperTuxKart race game has been wrapped using a gym environment.

Full report pdf

5-minute summary video (also available youtube):

• Results
• Training Process
• Get Data
• CNN Autoencoder
• Full Model
• Steer Model
• Drift Model
• Evaluate Models
• Environment

The results obtained by the different models can be found in evaluate.xlsx. Some sample videos and a chart summarizing the results can be found in the evaluate folder.

Full model (steer, drift, and acceleration):

Steer and drift model:

The following steps have to be done to train a model:

• Choose the tracks (Environment)
• Gather data (Get Data)
• Train CNN autoencoder (CNN Autoencoder)
• Train model

Once the model has been trained and saved, we can evaluate the models by specifying the models to be evaluated in run.py.

It runs the baseline model (with and without noise) on the environment to obtain a dataset of episodes that are used for training our models.

To change the way data is gathered or more information check run.py.

python -m run -t

The CNN autoencoder that is later used by the controller models can be found in agents/cnn.py.

To train the CNN autoencoder on the available data:

python -m agents.cnn

Learns to steer, drift, and accelerate.

It can be trained from scratch. However, we have first pretrained a CNN autoencoder and used the encoder for the image embedding layer. To modify this behavior or more information check agents/decision_transformer.py.

The following command trains this model with the autoencoder and data already available:

python -m agents.decision_transformer 

Learns to steer and has fixed acceleration and drift disabled.

It can be trained from scratch. However, we have first pretrained a CNN autoencoder and used the encoder for the image embedding layer. To modify this behavior or more information check agents/decision_transformer.py.

The following command trains this model with the autoencoder and data already available:

python -m agents.decision_transformer --no_drift 

Learns to drift and steer, and has fixed acceleration.

It can be trained from scratch. However, we have used the steer model with the best validation loss as our starting model for training. To modify this behavior or more information check agents/decision_transformer.py.

The following command trains this model with the steer model and data already available:

python -m agents.decision_transformer --fixed_velocity 

It runs the set of given models, the baseline, and the game AI for a certain number of episodes to evaluate their performance.

The models being evaluated are preset and need to be unziped before being used (the models are available in the saved folder). To change the models being evaluated or more information check run.py.

To evaluate the models on the lighthouse track:

python -m run --evaluate_lighthouse

To evaluate the models on all selected tracks (models given have not been trained in other tracks than lighthouse):

python -m run --evaluate

Gym environment that wraps the PySuperTuxKart game environment.

To check the environment options or select which tracks to use check environments/pytux.py.