This repository contains implementations for the work "Distributionally Robust Policy and Lyapunov-Certificate Learning".

If you find our work useful, please consider citing our paper:

@article{long2024distributionally,
  title={Distributionally Robust Policy and {L}yapunov-Certificate Learning},
  author={Long, Kehan and Cortes, Jorge and Atanasov, Nikolay},
  journal={arXiv preprint arXiv:2404.03017},
  year={2024}
}

git clone https://github.com/KehanLong/DR_Stabilizing_Policy

Install the required dependcies by using a anaconda virtual environment:

conda env create -f environment.yml

Activate the environment:

conda activate DR_Stabilizing_Policy

Finally, go to the project directory

cd DR_LF_Learning

This project uses a modified version of the Gymnasium library, which is an open-source library for developing and comparing reinforcement learning algorithms. We would like to acknowledge the original work by the Farama Foundation and the contributors of the Gymnasium repository. The original repository can be found at https://github.com/Farama-Foundation/Gymnasium.

The Gymnasium_modified directory in this project contains a few minor modifications to the system dynamics of the Mountain Car and Inverted Pendulum environments.

To re-train the provided examples, simply run python Inverted_pendulum_learning.py or python mountain_car_learning.py

Pre-trained models are available in the saved_models/joint_clf_controller_models directory.

To visualize the performance of the learned controllers, refer to the files Inverted_pendulum_evaluate; Mountain_car_evaluate.

The results of the learned DR Lyapunov-stable policy are demonstrated through the following GIFs:

Baseline	Distributionally Robust

Baseline	Distributionally Robust

Baseline	Distributionally Robust