med-air / surrol Goto Github PK

[IROS'21] SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

Home Page: https://med-air.github.io/SurRoL/

License: MIT License

Shell 0.42% Python 91.32% Jupyter Notebook 8.26%

medical-robotics reinforcement-learning simulation dvrk

surrol's Introduction

SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

[Project Website]

IEEE RA-L'23 Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learning
ICRA'23 Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot
ISMR'22 Integrating artificial intelligence and augmented reality in robotic surgery: An initial dVRK study using a surgical education scenario
IROS'21 SurRoL: An open-source reinforcement learning centered and dVRK compatible platform for surgical robot learning

Features

dVRK compatible robots.
Gym style API for reinforcement learning.
Ten surgical-related tasks.
Various object assets.
Based on PyBullet for physics simulation.

Installation

The project is built on Ubuntu with Python 3.7, PyBullet, Gym 0.15.6, and evaluated with Baselines, TensorFlow 1.14.

Prepare environment

Create a conda virtual environment and activate it.

conda create -n surrol python=3.7 -y
conda activate surrol

Install gym (slightly modified), tensorflow-gpu==1.14, baselines (modified).

Install SurRoL

git clone https://github.com/med-air/SurRoL.git
cd SurRoL
pip install -e .

Get started

The robot control API follows dVRK (before "crtk"), which is compatible with the real-world dVRK robots.

You may have a look at the jupyter notebooks in tests. There are some test files for PSM and ECM, that contains the basic procedures to start the environment, load the robot, and test the kinematics.

We also provide some run files to evaluate the environments using baselines.

Citation

If you find the paper or the code helpful to your research, please cite the project.

@inproceedings{xu2021surrol,
  title={SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning},
  author={Xu, Jiaqi and Li, Bin and Lu, Bo and Liu, Yun-Hui and Dou, Qi and Heng, Pheng-Ann},
  booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year={2021},
  organization={IEEE}
}

License

SurRoL is released under the MIT license.

Acknowledgement

The code is built with the reference of dVRK, AMBF, dVRL, RLBench, Decentralized-MultiArm, Ravens, etc.

Contact

For any questions, please feel free to email [email protected]

surrol's People

Contributors

Stargazers

Watchers

Forkers

springxiacj cladett wellbeinglwb kingmatty314 bogatovam andreykurenkov choirong teng4 taohuang13 maryam-zare gavinchen1314 ramstorage rndaorg linhongbin-ws qiguaiderenmen shikejianalan

surrol's Issues

Raising EOF Error.

First off, I am new to github so I apologize if the format of the issue is wrong.

The issue I am having is that the program raises an EOF Error when I run the bash files using ./bashfilename.

I was able to debug the program and figure out that the issue only occurs when n == 0 and remaining is equal to size, but remaining and size being equal is not possible unless size is zero which leads me to believe that the data file being passed is an empty datafile.

def _recv(self, size, read=_read):
buf = io.BytesIO()
handle = self._handle
remaining = size
print("Start of the while loop")
while remaining > 0:
chunk = read(handle, remaining)
n = len(chunk)
print("handle is:", handle)
print("n is:",n)
print("remaining is:",remaining)
if n == 0:
print("Code is here")
if remaining == size:
print("Remaining is equal to size")
raise EOFError
else:
raise OSError("got end of file during message")

That is the code I have in my connections.py in the conda folder of my linux dist. I modified it a little for debugging purposes. Is there any fix or something I need to do differently, thanks for the advice.

error: Cannot load URDF file.

Dear Sir
I had tried the test_psm, but it showed "error: Cannot load URDF file." in 3rd cell.
I think there is lack of urdf_file. What could I solved this problem. Would you kindly show it? Thanks a lot

HOW to test training of environment

Hello,

thank you for sharing your repo.
I wanted to test it and replicate your results.
I have downloaded and installed it.
I run the test scripts and everything seems fine.
I then checked inside /run where there are three bash script.

I tried to run ddpg_activetrack.sh with the following env id -env=ActiveTrack-v0.

I get the following error:

Traceback (most recent call last):

File "/home/claudia/.pyenv/versions/3.7.12/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/claudia/.pyenv/versions/3.7.12/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/claudia/data/virtual_environments/surrol/lib/python3.7/site-packages/baselines-0.1.6-py3.7.egg/baselines/run.py", line 250, in
main(sys.argv)
File "/home/claudia/data/virtual_environments/surrol/lib/python3.7/site-packages/baselines-0.1.6-py3.7.egg/baselines/run.py", line 216, in main
model, env = train(args, extra_args)
File "/home/claudia/data/virtual_environments/surrol/lib/python3.7/site-packages/baselines-0.1.6-py3.7.egg/baselines/run.py", line 54, in train
env_type, env_id = get_env_type(args)
File "/home/claudia/data/virtual_environments/surrol/lib/python3.7/site-packages/baselines-0.1.6-py3.7.egg/baselines/run.py", line 143, in get_env_type
assert env_type is not None, 'env_id {} is not recognized in env types'.format(env_id, _game_envs.keys())
AssertionError: env_id ActiveTrack-v0 is not recognized in env types

Is there a problem with the environment registration in gym?

Following the Issue #662 from Baslines i have add the following line in baselines/run.py at line 52:
_game_envs['custom_type'] = { 'ActiveTrack-v0'}

But again i get the following error:
gym.error.UnregisteredEnv: No registered env with id: ActiveTrack-v0

Thank you,
Cheers
Claudia

Error while running script

Hi, how can i use step function correctly? when i wrote:
`import gym
import numpy as np
from surrol.const import ROOT_DIR_PATH
import surrol.gym

env = gym.make("NeedlePick-v0")
env.reset()
while True:
obs = env.step(np.random.randn(5))
env.render()`
it gave me an error:

Traceback (most recent call last): File "dato1.py", line 12, in <module> obs = env.step(np.random.randn(5)) File "/home/dato/src/my_projects/robotics/venv/lib/python3.8/site-packages/gym/wrappers/time_limit.py", line 50, in step observation, reward, terminated, truncated, info = self.env.step(action) ValueError: not enough values to unpack (expected 5, got 4) Destroy EGL OpenGL window.
how can i fix this?

cpu based evaluations

Can the environments be evaluated without GPU support ?

Unexpected Aborted (core dumped) when running test_multiple_scenes_keyboard.py

I met an Aborted (core dumped) issue when running test_multiple_scenes_keyboard.py, there is no error message shown.

All output info is shown as below(I erased the loaded modules part):

[INFO ] [Logger ] Record log in /home/hao/.kivy/logs/kivy_24-05-08_10.txt
[INFO ] [Kivy ] v2.3.0
[INFO ] [Kivy ] Installed at "/home/hao/anaconda3/envs/surrolv2/lib/python3.7/site-packages/kivy/init.py"
[INFO ] [Python ] v3.7.16 (default, Jan 17 2023, 22:20:44)
[GCC 11.2.0]
[INFO ] [Python ] Interpreter at "/home/hao/anaconda3/envs/surrolv2/bin/python"
[INFO ] [Logger ] Purge log fired. Processing...
[INFO ] [Logger ] Purge finished!
[INFO ] [Factory ] 195 symbols loaded
pybullet build time: Nov 28 2023 23:50:19
[INFO ] [Image ] Providers: img_tex, img_dds, img_sdl2, img_pil (img_ffpyplayer ignored)
[DEBUG ] [falling back to hashlib hashing] pip install xxhashfor 50x faster cache checks
[DEBUG ] [matplotlib data path] /home/hao/anaconda3/envs/surrolv2/lib/python3.7/site-packages/matplotlib/mpl-data
[DEBUG ] CONFIGDIR=/home/hao/.config/matplotlib
[DEBUG ] interactive is False
[DEBUG ] platform is linux

[DEBUG ] CACHEDIR=/home/hao/.cache/matplotlib
[DEBUG ] Using fontManager instance from /home/hao/.cache/matplotlib/fontlist-v330.json
Known pipe types:
glxGraphicsPipe
(1 aux display modules not yet loaded.)
[INFO ] [GL ] Using the "OpenGL" graphics system
[INFO ] [GL ] Backend used
[INFO ] [GL ] OpenGL version <b'4.6.0 NVIDIA 550.54.15'>
[INFO ] [GL ] OpenGL vendor <b'NVIDIA Corporation'>
[INFO ] [GL ] OpenGL renderer <b'NVIDIA GeForce GTX 1070/PCIe/SSE2'>
[INFO ] [GL ] OpenGL parsed version: 4, 6
[INFO ] [GL ] Shading version <b'4.60 NVIDIA'>
[INFO ] [GL ] Texture max size <32768>
[INFO ] [GL ] Texture max units <32>
[INFO ] [Window ] virtual keyboard not allowed, single mode, not docked
[INFO ] [KivyMD ] 1.2.0, git-Unknown, 2024-05-08 (installed at "/home/hao/anaconda3/envs/surrolv2/lib/python3.7/site-packages/kivymd/init.py")
[WARNING] [KivyMD ] Version 1.2.0 is deprecated and is no longer supported. Use KivyMD version 2.0.0 from the master branch (pip install https://github.com/kivymd/KivyMD/archive/master.zip)
[INFO ] [Text ] Provider: sdl2
[INFO ] [Clipboard ] Provider: xclip
[INFO ] [CutBuffer ] cut buffer support enabled
[INFO ] [GL ] NPOT texture support is available
[INFO ] [ProbeSysfs ] device match: /dev/input/event6
[INFO ] [MTD ] Read event from </dev/input/event6>
[INFO ] [Base ] Start application main loop
[INFO ] [MTD ] </dev/input/event6> range position X is 1472 - 5706
[INFO ] [MTD ] </dev/input/event6> range position Y is 1408 - 4800
[INFO ] [MTD ] </dev/input/event6> range touch major is 0 - 0
[INFO ] [MTD ] </dev/input/event6> range touch minor is 0 - 0
[INFO ] [MTD ] </dev/input/event6> range pressure is 0 - 255
[INFO ] [MTD ] </dev/input/event6> axes inversion: X is 0, Y is 0
[INFO ] [MTD ] </dev/input/event6> rotation set to 0
[INFO ] [Loader ] using a thread pool of 2 workers
[INFO ] [Base ] Leaving application in progress...
[INFO ] [GL ] Using the "OpenGL" graphics system
[INFO ] [GL ] Backend used
[INFO ] [GL ] OpenGL version <b'4.6.0 NVIDIA 550.54.15'>
[INFO ] [GL ] OpenGL vendor <b'NVIDIA Corporation'>
[INFO ] [GL ] OpenGL renderer <b'NVIDIA GeForce GTX 1070/PCIe/SSE2'>
[INFO ] [GL ] OpenGL parsed version: 4, 6
[INFO ] [GL ] Shading version <b'4.60 NVIDIA'>
[INFO ] [GL ] Texture max size <32768>
[INFO ] [GL ] Texture max units <32>
[INFO ] [Window ] virtual keyboard not allowed, single mode, not docked
[INFO ] [ProbeSysfs ] device match: /dev/input/event6
[INFO ] [MTD ] Read event from </dev/input/event6>
[INFO ] [Base ] Start application main loop
[INFO ] [MTD ] </dev/input/event6> range position X is 1472 - 5706
[INFO ] [MTD ] </dev/input/event6> range position Y is 1408 - 4800
[INFO ] [MTD ] </dev/input/event6> range touch major is 0 - 0
[INFO ] [MTD ] </dev/input/event6> range touch minor is 0 - 0
[INFO ] [MTD ] </dev/input/event6> range pressure is 0 - 255
[INFO ] [MTD ] </dev/input/event6> axes inversion: X is 0, Y is 0
[INFO ] [MTD ] </dev/input/event6> rotation set to 0
[INFO ] [Base ] Leaving application in progress...
Aborted (core dumped)

her_ecmreach Joint Position out of valid range

When I was running her_ecmreach.sh in SurRoL/run, I received the error below.

From stderr

EGL device choice: -1 of 10.
EGL device choice: -1 of 10.
pybullet build time: Nov 28 2023 23:50:19
EGL device choice: -1 of 10.
/bd_targaryen/users/kleelakunwet/miniconda3/envs/surrol/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: ESC[33mWARN: Box bound precision lowered by casting to float32ESC[0m
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))

From stdout

Joint position out of valid range!
Set joint: [ 1.03259074e-01 -7.87232390e-01  9.36983488e-02 -6.75628442e-11]
Joint position out of valid range!
Set joint: [ 9.70008124e-02 -7.94572073e-01  8.63137182e-02 -8.20556956e-11]
Joint position out of valid range!
Set joint: [ 2.00471853e-01 -8.64014449e-01  8.36032845e-02 -8.20747914e-11]
Joint position out of valid range!
Set joint: [ 2.12226728e-01 -8.49095129e-01  8.90256233e-02 -8.09095013e-11]
Joint position out of valid range!
Set joint: [-4.06831465e-01 -8.12679394e-01  9.07127260e-02  6.82343071e-11]
Joint position out of valid range!
Set joint: [-5.96498305e-02 -8.03887682e-01  9.62620034e-02  2.87383450e-11]
Joint position out of valid range!
Set joint: [-1.21999198e-01 -8.26476681e-01  9.64618835e-02  2.17572627e-11]
Joint position out of valid range!
Set joint: [ 6.38048726e-02 -8.21610594e-01  9.38262929e-02 -5.43147749e-11]
Joint position out of valid range!
Set joint: [-4.35025623e-01 -8.63535838e-01  9.02180260e-02  5.20916643e-13]
Loaded EGL 1.5 after reload.
GL_VENDOR=NVIDIA Corporation
GL_RENDERER=NVIDIA GeForce RTX 4090/PCIe/SSE2
GL_VERSION=3.3.0 NVIDIA 545.23.08
GL_SHADING_LANGUAGE_VERSION=3.30 NVIDIA via Cg compiler
Version = 3.3.0 NVIDIA 545.23.08
Vendor = NVIDIA Corporation
Renderer = NVIDIA GeForce RTX 4090/PCIe/SSE2
Destroy EGL OpenGL window.
Loaded EGL 1.5 after reload.
GL_VENDOR=NVIDIA Corporation
GL_RENDERER=NVIDIA GeForce RTX 4090/PCIe/SSE2
GL_VERSION=3.3.0 NVIDIA 545.23.08
GL_SHADING_LANGUAGE_VERSION=3.30 NVIDIA via Cg compiler
Version = 3.3.0 NVIDIA 545.23.08
Vendor = NVIDIA Corporation
Renderer = NVIDIA GeForce RTX 4090/PCIe/SSE2
Destroy EGL OpenGL window.

Do you know what might be the problem? Thank you in advance!

Wrong InverseKinematics results with new dVRK model

Hi,

I am trying to integrate a new dVRK PSM Si model (different with dVRK classic) into SurRoL. I basically wrote the new URDF file and re-define the psm environment. For URDF I tried to make the link order and joint arrangement similar to what is defined in the original psm classic URDF file.

However, when testing, I get stuck in the wrong or unstable solution of InverseKinematics from pybullet. I was wondering if you have faced the same issue before? Here is an example:

joint_input = psm.move_joint([0., 0., 0.2, 0., 0., 0.])
step(0.5)
pose_world = forward_kinematics(psm.body, eef_link=psm.EEF_LINK_INDEX) 
target_world = get_matrix_from_pose_2d(pose_world)
print('pose_world: ')
print(target_world)

joints_inv = p.calculateInverseKinematics(
    bodyUniqueId=psm.body,
    endEffectorLinkIndex=psm.EEF_LINK_INDEX,
    targetPosition=pose_world[0],  # inertial pose, not joint pose
    targetOrientation=pose_world[1],
    lowerLimits=psm.limits['lower'][:psm.DoF],
    upperLimits=psm.limits['upper'][:psm.DoF],
    jointRanges=psm.limits['upper'][:psm.DoF] - psm.limits['lower'][:psm.DoF],
    restPoses=[0.] * psm.DoF,
    # jointDamping = [0.5] * psm.DoF,
    # solver = p.IK_SDLS,
    residualThreshold=1e-12,  # can tune
    maxNumIterations=1000
    )
joints_inv = np.array(joints_inv)
for i in range(psm.DoF):
    if psm.JOINT_TYPES[i] == 'P':
        joints_inv[i] /= psm.scaling
joints_sol = wrap_angle(joints_inv[:psm.DoF])
print('***TEST*** joints_inv', joints_sol)

psm.move_joint(joints_sol)
step(0.5)
pose_world = forward_kinematics(psm.body, eef_link=psm.EEF_LINK_INDEX) 
print('pose_world sol: ')
print(get_matrix_from_pose_2d(pose_world))

And the output is:

pose_world: 
[[-1.96549771e-05 -8.12493362e-01 -5.82970443e-01  4.59036201e-01]
 [ 1.00000000e+00 -2.41588265e-05 -4.47552144e-08 -6.61911059e-08]
 [-1.40475185e-05 -5.82970442e-01  8.12493362e-01  4.68582630e-01]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]
***TEST*** joints_inv [ 9.49841379e-07  2.40407970e-07  1.99999986e-01 -1.57079623e+00
  1.77197320e-05 -2.59519567e-06]
pose_world sol: 
[[-5.82970651e-01 -8.12493212e-01  1.63319895e-05  4.59036201e-01]
 [ 5.60809389e-06 -2.41249322e-05 -1.00000000e+00 -6.93162292e-08]
 [ 8.12493212e-01 -5.82970651e-01  1.86206656e-05  4.68582630e-01]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]

The IK solved joint result joints_inv is [0 0 0.2 -1.57 0 0] while the provided input is [0 0 0.2 0 0 0]. From testing so far, I found that the position tends to be accurate but for orientation, there is always a -1.57079623(-pi/2) offset of the 4rd joint(tool roll). However, in some cases such offset is gone. So the stability of IK solver is not reliable. I cannot find any problem with either the URDF file or the environment. I feel it is more of the problem with bullet itself.

Could you please provide any hint to this problem? I am attaching the necessary scripts for your reference. I am more than happy to have a further discussion on adding more robot modules into SurRoL. Thanks.
https://drive.google.com/drive/folders/12SArtA-CraVkCvT5lst_lGiUnBx_vpZp?usp=sharing

SurRol RL setting merge

Hello,

I've got trouble to make SurRol RL env setting.
First of all, I was deeply impressed with your SurRol projects.

Here is my problem.
I can execute 'needle_pick' and 'needle_regrasp' tasks easily.
Then, I want to merge them and learn with Reinforcement Learning.

I divided needle to 18 parts to grab.(This is pybullet env and I pressed 'w' and 'a' key when it execute)

And I want to make,

PSM1 grasp needle random point of 18 parts
PSM2 regrasp needle random point with RL with stable_baseline3(Cause baseline in github service has been terminated.)

I appreciate if you give me an advice for me.
Also, I sent you an email for details.

Best regards,
Donghyeon Choi

GPU memory overflow when execute reseting the environment for multiple times

When running the following commands:

import gym

env = gym.make('NeedlePick-v0')

for _ in range(100000000):

env.reset()

My GPU occupied memory will gradually overflow. Would you please help me solve the problem? Thank you very much

Cannot connect to X server

I was trying to run herdemo_needlepick.sh to test the environment after installing, the error suggested that the file ../surrol/data/demo/data_NeedlePick-v0_random_100.npz does not exist. Therefore, I ran python data_generation.py --env NeedlePick-v0 in surrol/data to get training data, and it showed the following error.

pybullet build time: Nov 28 2023 23:50:19
startThreads creating 1 threads.
starting thread 0
started thread 0 
argc=2
argv[0] = --unused
argv[1] = --start_demo_name=Physics Server
ExampleBrowserThreadFunc started
X11 functions dynamically loaded using dlopen/dlsym OK!

	cannot connect to X server