Webpage: https://serl-robot.github.io/

SERL provides a set of libraries, env wrappers, and examples to train RL policies for robotic manipulation tasks. The following sections describe how to use SERL. We will illustrate the usage with examples.

🎬: SERL video, additional video on sample efficient RL.

Table of Contents

• SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
  • Installation
  • Overview and Code Structure
  • Quick Start with SERL in Sim
  • Run with Franka Arm on Real Robot
  • Contribution
  • Citation

1. Setup Conda Environment: create an environment with

   conda create -n serl python=3.10

2. Install Jax as follows:

   • For CPU (not recommended):

     pip install --upgrade "jax[cpu]"

   • For GPU: (change cuda12 to cuda11 if you are using older driver versions)

     pip install --upgrade "jax[cuda12_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

   • For TPU

     pip install --upgrade "jax[tpu]" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html

   • See the Jax Github page for more details on installing Jax.

3. Install the serl_launcher

   cd serl_launcher
   pip install -e .
   pip install -r requirements.txt

SERL provides a set of common libraries for users to train RL policies for robotic manipulation tasks. The main structure of running the RL experiments involves having an actor node and a learner node, both of which interact with the robot gym environment. Both nodes run asynchronously, with data being sent from the actor to the learner node via the network using agentlace. The learner will periodically synchronize the policy with the actor. This design provides flexibility for parallel training and inference.

Table for code structure

We provide a simulated environment for trying out SERL with a franka robot.

Check out the Quick Start with SERL in Sim

• Training from state observation example
• Training from image observation example
• Training from image observation with 20 demo trajectories example

We provide a step-by-step guide to run RL policies with SERL on the real Franka robot.

Check out the Run with Franka Arm on Real Robot

• Peg Insertion 📍
• PCB Component Insertion 🖥️
• Cable Routing 🔌
• Object Relocation 🗑️

We welcome contributions to this repository! Fork and submit a PR if you have any improvements to the codebase. Before submitting a PR, please run pre-commit run --all-files to ensure that the codebase is formatted correctly.

If you use this code for your research, please cite our paper:

@misc{luo2024serl,
      title={SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning},
      author={Jianlan Luo and Zheyuan Hu and Charles Xu and You Liang Tan and Jacob Berg and Archit Sharma and Stefan Schaal and Chelsea Finn and Abhishek Gupta and Sergey Levine},
      year={2024},
      eprint={2401.16013},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}