Code for AI-QMIX: Attention and Imagination for Dynamic Multi-Agent Reinforcement Learning (Iqbal et al., arXiv 2006.04222)
This codebase is built on top of the PyMARL framework for multi-agent reinforcement learning algorithms.
- Docker
- NVIDIA-Docker (if you want to use GPUs)
Build the Dockerfile using
cd docker
./build.sh
Set up StarCraft II.
./install_sc2.sh
Run an ALGORITHM from the folder src/config/algs
in an ENVIRONMENT from the folder src/config/envs
on a specific GPU using some PARAMETERS:
./run.sh <GPU> python3 src/main.py --env-config=<ENVIRONMENT> --config=<ALGORITHM> with <PARAMETERS>
Add the --no-mongo
flag before the "with" statement if you do not have a MongoDB database set up to log results (the results will still be logged in a local file).
Possible environments are:
firefighters
: SaveTheCity environment from the papersc2custom
: StarCraft environment from the paper
For each environment you can specify the set of scenarios to train/test on by including the parameter with scenario=<scenario_set_name>
.
Here are the possible scenario sets for each environment:
- SaveTheCity
- '2-8a_2-8b_05': train on 5% of the possible scenarios
- '2-8a_2-8b_25': train on 25% of the possible scenarios
- '2-8a_2-8b_45': train on 45% of the possible scenarios
- '2-8a_2-8b_65': train on 65% of the possible scenarios
- '2-8a_2-8b_85': train on 85% of the possible scenarios
- '2-8a_2-8b_sim_Q1': train on the quartile of scenarios least similar to the testing ones
- '2-8a_2-8b_sim_Q2': train on the quartile of scenarios 2nd least similar to the testing ones
- '2-8a_2-8b_sim_Q3': train on the quartile of scenarios 2nd most similar to the testing ones
- '2-8a_2-8b_sim_Q4': train on the quartile of scenarios most similar to the testing ones
- StarCraft
- '3-8sz_symmetric'
- '3-8MMM_symmetric'
The testing scenarios are the same on all SaveTheCity scenario set
Possible algorithms are:
- 'imagine_qmix': AI-QMIX (our method)
- 'atten_qmix': A-QMIX
- 'imagine_vdn': AI-VDN
- 'atten_vdn': A-VDN
If you use this repo in your work, please consider citing the corresponding paper:
@article{iqbal2020ai,
title={AI-QMIX: Attention and Imagination for Dynamic Multi-Agent Reinforcement Learning},
author={Iqbal, Shariq and de Witt, Christian A Schroeder and Peng, Bei and B{\"o}hmer, Wendelin and Whiteson, Shimon and Sha, Fei},
journal={arXiv preprint arXiv:2006.04222},
year={2020}
}