Giter VIP home page Giter VIP logo

franka_grasp_baseline's Introduction

franka_grasp_baseline

Table top manipulation based on franka fr3 robot arm and graspnet, including calibration, 3d reconstruction and grasping.

Install

For caclibration:

  • Please install numpy, open3d, opencv-python, opencv-contrib-python.
  • The code is tested on python 3.8 and python 3.10, you may try other version.

For grasp:

  • Please refer to graspnet and lerftogo.
  • The code is tested on python 3.8, you may try other version.

harwares

  • The code is tested on realsense 405/435(tested), other realsense cameras may also work.
  • The code is tested on franka emika research 3, the code should also work on franka panda.

Usage

Overall pipeline:

  1. Use "hand_in_eye_shooting.ipynb" in "examples/hand_in_eye" to collect wrist arm data.
    • The calibration board is genereted by calib.io.
  2. Use "hand_in_eye_calib.ipynb" in "examples/hand_in_eye" to do calibration and generate pointcloud with tsdf.
  3. Put the pointcloud in "./examples/graspnet/pointclouds" and run gen_grasp.py to generate grasp.
    1. Click "Load Pointcloud" to visualize pcd.
    2. Click "Generate Grasp" to get the grasps.
    3. Select best grasp with "Grasp Number".
    4. Click "Grasp" to do IK and save the joints angle.
  4. Use "grasp_test.ipynb" to control the robot arm do real world task.

Additional notes:

  • The Pipeine is based on only wrist camera, and the intrinsics are from realsense. You may further calibrate them but the inner params work well.
  • Hand-to-eye is based on eye-to-eye and hand-in-eye, which is not a standard method.
  • You may check other examples about realsense usage, eye-to-eye/hand-to-eye calibration and intrinsic calibration. The data fromat is shown in the folder "eye_to_eye" and "hand_in_eye".

Applications

ui
sim real

TODO

  • Hand to eye calibration with a mark on the arm.
  • Automatic data collection with spherical view points.

franka_grasp_baseline's People

Contributors

jimazeyu avatar yuelei0428 avatar

Stargazers

张志诚 avatar

Watchers

Roger Q. avatar  avatar

franka_grasp_baseline's Issues

How to obtain the transformation matrix of the gripper relative to the Franka base coordinate system

Sorry to bother you, may I ask how to obtain the transformation matrix of the gripper relative to the Franka base coordinate system. I have used Realsense and Franka robots for hand eye calibration to obtain the transformation matrix from the camera to the end axis of the robotic arm. How to obtain the coordinate system from the end axis to the Franka base, and how to obtain the Franka gripper relative to the base coordinate system?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.