This repository contains implementations for the work "Distributionally Robust Policy and Lyapunov-Certificate Learning".
If you find our work useful, please consider citing our paper:
@article{long2024distributionally,
title={Distributionally Robust Policy and {L}yapunov-Certificate Learning},
author={Long, Kehan and Cortes, Jorge and Atanasov, Nikolay},
journal={arXiv preprint arXiv:2404.03017},
year={2024}
}
git clone https://github.com/KehanLong/DR_Stabilizing_Policy
Install the required dependcies by using a anaconda virtual environment:
conda env create -f environment.yml
Activate the environment:
conda activate DR_Stabilizing_Policy
Finally, go to the project directory
cd DR_LF_Learning
This project uses a modified version of the Gymnasium library, which is an open-source library for developing and comparing reinforcement learning algorithms. We would like to acknowledge the original work by the Farama Foundation and the contributors of the Gymnasium repository. The original repository can be found at https://github.com/Farama-Foundation/Gymnasium.
The Gymnasium_modified directory in this project contains a few minor modifications to the system dynamics of the Mountain Car and Inverted Pendulum environments.
To re-train the provided examples, simply run
python Inverted_pendulum_learning.py
or python mountain_car_learning.py
Pre-trained models are available in the saved_models/joint_clf_controller_models directory.
To visualize the performance of the learned controllers, refer to the files Inverted_pendulum_evaluate
; Mountain_car_evaluate
.
The results of the learned DR Lyapunov-stable policy are demonstrated through the following GIFs:
Baseline | Distributionally Robust |
---|---|
![]() |
![]() |
Baseline | Distributionally Robust |
---|---|
![]() |
![]() |
Baseline | Distributionally Robust |
---|---|
![]() |
![]() |