r-ceph / bail Goto Github PK

Code for the paper "BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning"(link: https://arxiv.org/abs/1910.12179), published at NeurIPS 2020.

The BAIL implementation can be found under spinup/algos/BAIL/. Code runs with pytorch >= 1.2, mujoco 150

Create a folder of the path "spinup/algos/BAIL/buffers" and put the RL batches there.

Then you run BAIL in the following procedure:

Under the BAIL folder, first do the returns calculation with python main_get_mcret.py, which stores the calculated returns in "./results".

Then run BAIL with python main_static_bail.py or run Progressive BAIL with python main_prog_bail.py. We use the logger of Spinup to record algorithmic performances. You may consult Spinup documentation for output and plotting: (https://spinningup.openai.com/en/latest/user/saving_and_loading.html, https://spinningup.openai.com/en/latest/user/plotting.html). Or you could modify the code and save it in your style.

Implementation of the BCQ algorithm: https://github.com/sfujim/BCQ

If you use our code, please cite our paper:

@inproceedings{chen2020bail,
 author = {Chen, Xinyue and Zhou, Zijian and Wang, Zheng and Wang, Che and Wu, Yanqiu and Ross, Keith},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin},
 pages = {18353--18363},
 publisher = {Curran Associates, Inc.},
 title = {BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning},
 url = {https://proceedings.neurips.cc/paper/2020/file/d55cbf210f175f4a37916eafe6c04f0d-Paper.pdf},
 volume = {33},
 year = {2020}
}