Welcome to the gym-duckietown-agent, the default template for machine learning AI-DO submissions. It runs PyTorch, numpy, scipy and some other useful libraries for machine learning, however you can customize it to your liking.

There are two versions of the gym-duckietown-agent, both built from this repository and available on Docker Hub. The default version, duckietown/gym-duckietown-agent, runs on x86 and ARM devices. The GPU enabled version, duckietown/gym-duckietown-agent:gpu, runs on x86 or GPU-powered machines with nvidia-docker. Both of these depend on the duckietown/gym-duckietown simulator, or a compatible messaging interface. Below, you will find instructions for installing and running a gym-duckietown-agent container.

To enable fully reproducible software environments, we use a container tool called Docker. For your convenience, we have prepared some reading material about Docker.

• git
• Docker CE or nvidia-docker (for GPU acceleration)
• docker-compose - this allows you to "compose" or "orchestrate" multiple Docker containers into a functional system.

To get started, fork or clone this git repository and cd into the project directory:

git clone https://github.com/duckietown/gym-duckietown-agent.git && cd gym-duckietown-agent

Now, start the Docker containers (always make sure to have the latest simulator). On first start this will download the ~2GB duckietown/gym-duckietown-server container. Due to layer caching, successive starts should be significantly faster.

Several competition challenges are available to run.

docker-compose -f docker-compose-lf.yml pull && \
docker-compose -f docker-compose-lf.yml up 

docker-compose -f docker-compose-lfv.yml pull && \
docker-compose -f docker-compose-lfv.yml up

This will give you a reward output like:

gym-duckietown-agent_1   | starting sub socket on 8902
gym-duckietown-agent_1   | listening for camera images
gym-duckietown-agent_1   | [Challenge: LF] The average reward of 10 episodes was -315.7005. Best episode: -283.9803, worst episode: -368.1122
gym-duckietown-agent_gym-duckietown-agent_1 exited with code 0

(The important bit is the second-to-last line above)

You can terminate the run at any time by pressing CTRL+c.

By default, Docker will use the ARM-emulator when running on x86, however you can switch to the GPU version by replacing duckietown/gym-duckietown-agent with duckietown/gym-duckietown-agent:gpu in your docker-compose*.yml file.

To write your own agent, first fork this repository, and edit the file agent.py. Then run the following command from the root directory of this project on your local machine to evaluate the agent's performance:

docker build -t duckietown/gym-duckietown-agent . && \
docker-compose -f docker-compose-lf.yml up`

Check the average reward and try to improve your score. Good luck!

The duckietown/gym-duckietown-agent[:gpu] Docker image is based on PyTorch. You can use this image as a template for implementing your own agent. It runs on both x86 and ARM platforms.

docker run -it duckietown/gym-duckietown-agent python -c "import torch; print(torch.randn(10))"

docker run -it duckietown/gym-duckietown-agent:gpu python -c "import torch; print(torch.randn(10))"

Docker images for x86 and ARM are automatically rebuilt from this GitHub repository, however you should rebuild them locally when you want to test changes to the code.

To do so, first cd to the project directory on your local machine. Then, depending on whether you are targeting ARM/x86 or GPU, run one of the following commands:

docker build -t duckietown/gym-duckietown-agent .

docker build --file=docker/gpu/Dockerfile -t duckietown/gym-duckietown-agent:gpu .

When the competition opens, you will submit your Docker image using dt, the Duckietown Shell like so:

dt aido18 submit <YOUR_DOCKER_HUB_USERNAME>/<YOUR_IMAGE_BASED_ON_GYM_DUCKIETOWN_AGENT>

The referenced Docker image will contain a pretrained model which you have trained locally. If you send us a GPU model, we will run it on the simulator only. If you send us an ARM image, we will evaluate the submission and run it on a physical robot, then send you a link to the corresponding log. Should you wish to participate in the robotarium challenge, you must submit an ARM-based image.

The leaderboard evaluation will occur in some randomized environments - so be careful not to overfit!

By default, we provide a working PyTorch installation, however you can customize the duckietown/gym-duckietown-agent Dockerfile to use your favorite machine learning libraries. The only hard requirement is duckietown-slimremote to interact with the simulator. Here are some other popular machine learning libraries:

• TensorFlow (works on RPi)
• mxnet (also works on RPi)

To customize the default distribution, modify the Dockerfile (or docker/gpu/Dockerfile if you are using a GPU) and rebuild the image using the build instructions.