A gym environment for xArm

Create a virtual environment with Python 3.10 and activate it, e.g. with miniconda:

conda create -y -n xarm python=3.10 && conda activate xarm

Install gym-xarm:

pip install gym-xarm

# example.py
import gymnasium as gym
import gym_xarm

env = gym.make("gym_xarm/XarmLift-v0", render_mode="human")
observation, info = env.reset()

for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)
    image = env.render()

    if terminated or truncated:
        observation, info = env.reset()

env.close()

To use this example with render_mode="human", you should set the environment variable export MUJOCO_GL=glfw or simply run

MUJOCO_GL=glfw python example.py

The goal of the agent is to lift the block above a height threshold. The agent is an xArm robot arm and the block is a cube.

The action space is continuous and consists of four values [x, y, z, w]:

• [x, y, z] represent the position of the end effector
• [w] represents the gripper control

Observation space is dependent on the value set to obs_type:

• "state": observations contain agent and object state vectors only (no rendering)
• "pixels": observations contains rendered image only (no state vectors)
• "pixels_agent_pos": contains rendered image and agent state vector

Instead of using pip directly, we use poetry for development purposes to easily track our dependencies. If you don't have it already, follow the instructions to install it.

Install the project with dev dependencies:

poetry install --with dev

The equivalent of pip install some-package would just be:

poetry add some-package

# install pre-commit hooks
pre-commit install

# apply style and linter checks on staged files
pre-commit

gym-xarm is adapted from FOWM and is based on work by Nicklas Hansen, Yanjie Ze, Rishabh Jangir, Mohit Jain, and Sambaran Ghosal as part of the following publications:

• Self-Supervised Policy Adaptation During Deployment
• Generalization in Reinforcement Learning by Soft Data Augmentation
• Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
• Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulation
• Visual Reinforcement Learning with Self-Supervised 3D Representations