Hey there,

I'm currently working on automatic API documentation for nimble (see this branch) with sphinx.

The documentation from the python modules (e.g. in python/nimblephysics) is parsed and displayed nicely, however the pybind-bound code from python/_nimblephysics needs to be compiled into an extension module first (related issue) before sphinx can extract the API documentation.

However, I have troubles locally installing nimble with either pip install [-e] . or python setup.py install/develop. It seems as if there are several install scripts in the repository, but it's not clear to me which scripts to use or which steps to take. I have tried running pip install -e . and fixing each CMake Error by installing the respectively missing package from source, but to no avail.

It would be nice to provide a small snippet on how to install nimblephysics from source (or more generally, how to install nimblephysics when one would like to contribute).

For example, the test fails when building the package in <diffdart>/build/gcc with the following errors:

[==========] Running 6 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 6 tests from UniversalLoader
[ RUN      ] UniversalLoader.LOAD_SDF_ATLAS_FROM_DART
Msg [NameManager::issueNewName] (Skeleton::N4dart8dynamics9ShapeNodeE | drc_skeleton) The name [utorso - visual shape] is a duplicate, so it has been renamed to [utorso - visual shape(1)]
Msg [NameManager::issueNewName] (Skeleton::N4dart8dynamics9ShapeNodeE | drc_skeleton) The name [utorso - visual shape] is a duplicate, so it has been renamed to [utorso - visual shape(2)]
Msg [NameManager::issueNewName] (Skeleton::N4dart8dynamics9ShapeNodeE | drc_skeleton) The name [utorso - visual shape] is a duplicate, so it has been renamed to [utorso - visual shape(3)]
Msg [NameManager::issueNewName] (Skeleton::N4dart8dynamics9ShapeNodeE | drc_skeleton) The name [utorso - visual shape] is a duplicate, so it has been renamed to [utorso - visual shape(4)]
Msg [NameManager::issueNewName] (Skeleton::N4dart8dynamics9ShapeNodeE | drc_skeleton) The name [utorso - collision shape] is a duplicate, so it has been renamed to [utorso - collision shape(1)]
Warning [World.cpp:722] [World::addSkeleton] Attempting to add a Skeleton "drc_skeleton" to the world with non-zero damping coefficient! This version of DiffDART doesn't support damping coefficients. It will be automatically set to zero.
[       OK ] UniversalLoader.LOAD_SDF_ATLAS_FROM_DART (78 ms)
[ RUN      ] UniversalLoader.LOAD_SDF_ATLAS_FROM_RELATIVE_PATH
Warning [LocalResource.cpp:48] [LocalResource::constructor] Failed opening file '/home/js/dev/keenon/working/build/cmd/unittests/unit/../../../data/sdf/atlas/atlas_v3_no_head.sdf' for reading: No such file or directory
Warning [CompositeResourceRetriever.cpp:96] [CompositeResourceRetriever::retrieve] All ResourceRetrievers registered for this schema failed to retrieve the URI 'file:///home/js/dev/keenon/working/build/cmd/unittests/unit/../../../data/sdf/atlas/atlas_v3_no_head.sdf' (tried 1).
Warning [SdfParser.cpp:322] [SdfParser::readSkeleton] Loading file [file:///home/js/dev/keenon/working/build/cmd/unittests/unit/../../../data/sdf/atlas/atlas_v3_no_head.sdf] failed: Failed retrieving a resource from 'file:///home/js/dev/keenon/working/build/cmd/unittests/unit/../../../data/sdf/atlas/atlas_v3_no_head.sdf'.
Error [UniversalLoader.cpp:92] [UniversalLoader] Error when ettempting to load a file [/home/js/dev/keenon/working/build/cmd/unittests/unit/../../../data/sdf/atlas/atlas_v3_no_head.sdf]. Underlying loader returned a NULL skeleton. Please double check that the file being loaded is valid.
/home/js/dev/keenon/working/unittests/unit/test_UniversalLoader.cpp:34: Failure
Value of: skel != nullptr
  Actual: false
Expected: true
[  FAILED  ] UniversalLoader.LOAD_SDF_ATLAS_FROM_RELATIVE_PATH (0 ms)

This is totally not necessary, but would be nice. I'd like to be able to call a method from Python, and have it save a video that was rendered on the web GUI. Possibly even on a headless browser, if for example you're doing experiments on a beefy AWS machine and want to generate videos async. Here's an interesting proof of concept: http://www.smartjava.org/content/capture-canvas-and-webgl-output-video-using-websockets/

Hello, thank you for your awesome work and this repository. I am currently having issues with Mesh - Sphere collisions. When I initialize and drop a sphere initialized by a URDF file, it does not collide with a URDF terrain and passes through.

I don't think this was the intended behavior because a quick search through the source code has revealed mesh-sphere collisions. Is this an implemented feature?

I am attaching a .gif below.

?

Hi,

I am loading a humanoid from a urdf file. It seems that we can not directly define a ball joint in urdf. So I wonder if we can load a urdf with 1 DOF and then change the DOF to 3 after loading the model into the nimble?

Thanks

verifyJacobiansWrt(mass) in verifyWrtMass() is currently disabled because the wrt-vel jacobian fails on GRADIENTS.ARM_3_LINK_30_DEG_MIDDLE_ATTACH. Fix whatever's wrong to be able to re-enable those tests.

Currently, in order to run the web GUI from C++, you need to boot the websocket server using code that looks like this:

    server::GUIWebsocketServer server;
    server.renderWorld(world);
    server.renderBasis();
    server.serve(8070);

    while (server.isServing())
    {
    }

If you then attempt to directly connect to localhost:8070 in your web browser, you'll get an error, because that's only a websocket server. You also need to go into our javascript folder and call npm run dev to host an HTTP server, then visit localhost:9000 in your browser. It'd be nice to bundle the server on the C++ side, so it hosts the HTTP server too.

If you call gui.stateMachine().renderTrajectoryLines(...) multiple times with the same input, you'll get different results visually. This indicates to me some kind of data corruption or race conditions hidden in here that we need to ferret out.

Thanks for the great work.
After installation via pip install, the error occurs while importing nimblephysics

We need to pass unittests/comprehensive/test_HalfCheetahTrajectory.cpp without errors.

A sequence of 120 fps is actually visualized with 12 fps in NimbleGUI.

^This ticker is sending 10x slower ticks (not realtime). Removing the multiplication with 10 works but it chokes up the browser with frequent calls and GUI starts lagging.

We can have the GUIServer in C++ silently drop frames that exceed some framerate threshold.

The method ContactConstraint::getTangentBasisMatrixODE(Eigen::Vector3d normal) produces the pair of vectors that, when combined with the vector normal, form an orthonormal basis of R^3. These two vectors become our two frictional directions.

This method experiences a discontinuity when normal == (0,0,1), and very large gradients near that point. This is obnoxious, and we'd like to rewrite it to avoid that. This is not mathematically possible in the general case: https://scicomp.stackexchange.com/a/27843. However, I'm sure we can somehow make our frictional basis "path dependent", and in so doing avoid the non-differentiable points by moving it to an unlikely spot given this timestep's collision information. This will require some thought.

Hi,

The object is described via generalized coordinate. I wonder how to get the joint DOF of each link? The question may be very basic... So I wonder if there is any documents besides the examples in the current repository that introduce the basic functions?

Thanks

Hi,

I wonder if the simulation function "nimble.timestep(world, state, action)" supports batch and GPU processing?

Thanks

Hello,

It looks like the recent RGBA update to the GUI broke the python examples since the new color format is a 4-array (as opposed to 3-array as it was before).

The error is something like this for the initial_conditions.py:

TypeError: createSphere(): incompatible function arguments. The following argument types are supported:
    1. (self: nimblephysics_libs._nimblephysics.server.GUIStateMachine, key: str, radius: float = 0.5, pos: numpy.ndarray[numpy.float64[3, 1]] = array([0., 0., 0.]), color: numpy.ndarray[numpy.float64[4, 1]] = array([0.5, 0.5, 0.5]), castShadows: bool = True, receiveShadows: bool = False) -> None

updating the color to something like color=[255, 0, 0, 255] solves this issue.

When custom loss functions for IPOPT optimization problems are specified in Python, and throw an exception in the method body, we currently print the supremely unhelpful error:

Exception of type: IpoptException in file "Unknown File" at line -1:
 Exception message: Unknown Exception caught in Ipopt

It'd be great if we instead printed a more useful error message.

Hi,

I am trying to load a .skel file defined by a collision detector of "bullet" and a body link of multi_sphere. The skeleton can be loaded but I the following error message pop up

Warning [Factory-impl.hpp:168] [Factory] Failed to create an object of 'N4dart9collision17CollisionDetectorE' class with the key (type: 'Ss'). Returning nullptr instead.
Warning [SkelParser.cpp:726] Unknown collision detector[bullet]. Default collision detector[dart] will be loaded.
Error [DARTCollisionDetector.cpp:304] [DARTCollisionDetector] Attempting to create shape type [MultiSphereConvexHullShape] that is not supported by DARTCollisionDetector. Currently, only BoxShape and EllipsoidShape (only when all the radii are equal) and SphereShape and MeshShape and CapsuleShape are supported. This shape will always get penetrated by other objects.
Error [DARTCollisionDetector.cpp:304] [DARTCollisionDetector] Attempting to create shape type [MultiSphereConvexHullShape] that is not supported by DARTCollisionDetector. Currently, only BoxShape and EllipsoidShape (only when all the radii are equal) and SphereShape and MeshShape and CapsuleShape are supported. This shape will always get penetrated by other objects.

Is this a bug or we can not define the .skel file like that?

In dart/dynamics/skeleton.cpp

Eigen::VectorXs Skeleton::getLinkCOMs()
{
  Eigen::VectorXs inertias = Eigen::VectorXs::Zero(getLinkCOMDims());
  std::size_t cursor = 0;
  for (std::size_t i = 0; i < getNumBodyNodes(); i++)
  {
    const Inertia& inertia = getBodyNode(i)->getInertia();
    inertias(cursor++) = inertia.COM_X;
    inertias(cursor++) = inertia.COM_Y;
    inertias(cursor++) = inertia.COM_Z;
  }
  return inertias;
}

Uses enumerated id (1,2,3) as different COM value; which should be inertia.getParameters(dynamics::Inertia::Param::COM_X)

Hello,

Does the project support mimic joints? It looks like the loader is able to load an .urdf model with mimic joints but the backpropagation terminates without raising error if the .urdf model contains a mimic joint after I run loss.backward().

The following is the behavior that I noticed in Python:

    world = nimble.simulation.World()
    skel: nimble.dynamics.Skeleton = world.loadSkeleton(
        "data/sdf/atlas/ground.urdf"
    )
    skel.getJoints()

We're seeing penetration in Yannis's demos: https://github.com/iexarchos/DiffDART_DDP-iLQR_opt

Specifically, HalfCheetah and Reacher

This is probably an issue with capsule-box collisions.

Hi!

Thanks a lot for the amazing work! When I try to run the rajagopal.py example, I encountered the error below:

❯ python rajagopal.py
Msg [NameManager::issueNewName] (default) The name [Joint_rot_z] is a duplicate, so it has been renamed to [Joint_rot_z(1)]
Msg [NameManager::issueNewName] (default) The name [hip_r_z] is a duplicate, so it has been renamed to [hip_r_z(1)]
Msg [NameManager::issueNewName] (default) The name [hip_l_z] is a duplicate, so it has been renamed to [hip_l_z(1)]
Msg [NameManager::issueNewName] (default) The name [back_z] is a duplicate, so it has been renamed to [back_z(1)]
Msg [NameManager::issueNewName] (default) The name [acromial_r_z] is a duplicate, so it has been renamed to [acromial_r_z(1)]
Msg [NameManager::issueNewName] (default) The name [acromial_l_z] is a duplicate, so it has been renamed to [acromial_l_z(1)]
Traceback (most recent call last):
  File "rajagopal.py", line 12, in <module>
    world.addSkeleton(skel)
TypeError: addSkeleton(): incompatible function arguments. The following argument types are supported:
    1. (self: nimblephysics_libs._nimblephysics.simulation.World, skeleton: nimblephysics_libs._nimblephysics.dynamics.Skeleton) -> str

Invoked with: <nimblephysics_libs._nimblephysics.simulation.World object at 0x7f95df2df880>, <nimblephysics_libs._nimblephysics.biomechanics.OpenSimFile object at 0x7f95df2df8b8>

I have the version 0.6.1 installed via pip. How can I solve this problem?

Yannis reports extremely sluggish Web GUI displaying Atlas on Firefox on Ubuntu.

Here's his output from cat /proc/cpuinfo:
out.txt

Environment

• Ubuntu 18.04
• python 3.8
• Installed with pip install nimblephysics

Current Behavior

In the .Rajagopal example, the skeleton model loading fails with the following errors (for each skeleton part):

Warning [MeshShape.cpp:491] [MeshShape::loadMesh] Failed loading mesh 'file:///home/mkeller2/.pyenv/versions/impproj_env/lib/python3.8/site-packages/nimblephysics/models/rajagopal_data/Geometry/metacarpal5_lvs.vtp.ply' with ASSIMP error 'Unable to open file "file:///home/mkeller2/.pyenv/versions/impproj_env/lib/python3.8/site-packages/nimblephysics/models/rajagopal_data/Geometry/metacarpal5_lvs.vtp.ply".'.
Warning [LocalResource.cpp:48] [LocalResource::constructor] Failed opening file '/home/mkeller2/.pyenv/versions/impproj_env/lib/python3.8/site-packages/nimblephysics/models/rajagopal_data/Geometry/little_proximal_lvs.vtp.ply' for reading: No such file or directory
Warning [CompositeResourceRetriever.cpp:96] [CompositeResourceRetriever::retrieve] All ResourceRetrievers registered for this schema failed to retrieve the URI 'file:///home/mkeller2/.pyenv/versions/impproj_env/lib/python3.8/site-packages/nimblephysics/models/rajagopal_data/Geometry/little_proximal_lvs.vtp.ply' (tried 1).

The model .osim file do list the proper .vtp paths, and those files exist, but somehow the loader tries to load the files with the extension '.ply' .

Code to Reproduce

import nimblephysics as nimble
skel: nimble.biomechanics.OpenSimFile = nimble.models.RajagopalHumanBodyModel()

Hi! I am having troubles with the convertion between skeleton types. After creating two skeletons with different number (and order) of joints, I followed the docs creating a converter between the two skeletons. However, when Python executes the code's row
converter.rescaleAndPrepTarget(), it crashes and gives me the error "segmentation fault (core dumped)".

gui.displayState(world.getState()) report numpy.ndarray has no attribute detach error

If we try to display a simulation from 1000fps at real time in the browser, right now the C++ will blindly attempt to send JSON packets to the browser at 1000fps. That's too much for the browser to handle, and slows everything down. So we should really have the C++ rate-limit itself to 100fps, and silently drop/batch updates that come in faster than that on the C++ side.

To actually tease out the gravity forces, one could do getCoriolisAndGravityForces() - getCoriolisForces().

The helper method DARTCollide::createFaceFaceContacts() returns a list of contact points given a pair of convex hulls colliding at a witness plane. The method receives all the "witness points" (vertices within epsilon of the witness plane) from each convex hull. Then the method uses 2D geometry, mapping all the "witness points" from both objects into the witness plane. In 2D "witness plane space" the witness points themselves form 2D convex shapes (any slice of a convex set is also convex). To generate and classify contact points, we take:

• any points from object A that lie within object B are vertex-face collisions from A to B.
• any points from object B that lie within object A are face-vertex collisions from A to B.
• any edges that intersect between A and B become an edge-edge collision from A to B.

This ticket is about the fact that the edge-edge collisions do not currently have a deterministic order. We'd like there to always be consistent ordering of collisions, for cacheing and finite differencing elsewhere relies on this fact.

Hi,

Thanks for building this useful software! I have several question towards using the AMASS MoCap data in nimblephysics.

Example code of the skeleton mapping between AMASS and Rajagopal is provided in the documents. But I am still not sure how the mapping is realized and how I can exploit the AMASS the data. Specifically, (1) AMASS is based on parametric body mesh body. Do we need to construct a correponding urdf file to realize the mapping? (2) The example code map the links between two skeletons. If I want to incorprate the AMASS pose data (the joint angles) into the Rajagopal, how should I define the "dataSourceSkeletonPoses"? What would be the dimension of "dataSourceSkeletonPoses" for one frame? Also, is that possible to provide more detailed example code of using the AMASS data?

Best,
Yufei

File "/home/yannis/anaconda3/envs/pytorch/lib/python3.7/site-packages/diffdart/dart_layer.py", line 96, in dart_layer
return DartLayer.apply(world, pos, vel, torque, pointer) # type: ignore
File "/home/yannis/anaconda3/envs/pytorch/lib/python3.7/site-packages/diffdart/dart_layer.py", line 35, in forward
world.setForces(torque.detach().numpy())
AttributeError: 'diffdart_libs._diffdart.simulation.World' object has no attribute 'setForces'

with joint.setPositionLimitEnforced(True), setting position limits using skeleton.setPositionLowerLimit() and skeleton.setPositionUpperLimit() for BallJoint seems to cause segfaults when the skeleton strays outside of the valid region during simulation

Describe the solution you'd like
We'd like to have a way of creating shapeNodes given an array of shape (N, 3) (3D vectors for vertex positions) and an array of shape (T, 3) (defining T triangluar faces with the respective vertex indices).

Additional context
The format above corresponds to the standard triangle mesh format, so supporting this kind of initialization could help in a lot of scenarios (e.g. we're dealing with non-convex objects, so we obtain the triangle mesh data by using approximate convex decomposition with V-HACD before passing it to a physics engine).

Keep up the good work, I'd be interested in helping out by improving the python side of documentation/API.

Currently, we have separate complete methods for finite differencing every function.

This creates a whole lot of almost-duplicated code, which drastically bloats the codebase and makes it harder to debug issues due to slight variations in the implementations.

We should have one finite differencing implementation which works on all of our functions, with options for central differencing and ridders method. This is complicated by the fact that we perturb one element of the input vector at a time, and also that different functions have different criteria for a "small-enough" step size, such as the number of clamping indices staying constant.

Our LCP's are only guaranteed solvable if A is positive-semidefinite and there are no force bounds. To increase stability of our LCPs, we can do a trick called "Constraint Force Mixing". In practice, this means multiplying the elements on the diagonal of A by 1.0 + eps, where eps is some small positive value. This has the effect of ensuring A isn't singular, and in general reducing "singular-ness" of A.

You can turn constraint force mixing on and off with void World::setConstraintForceMixingEnabled(bool enable). Currently, by default CFM is turned off, because our Jacobians don't support it.

This ticket is about supporting it.

The matrix A is computed in dart/constrant/BoxedLcpConstraintSolver.cpp in the method solveConstrainedGroup(). The A matrix it computes is in Open Dynamics Engine format, which means row-major order, where each row's length is rounded up to the nearest multiple of 4 (to allow vectorization) and any padding entries are ignored. This calls out to individual constraints to populate A. For each constraint, it applies a unit impulse, and then measures the change in relative velocity at the constraint. The method that applies the CFM is ContactConstraint::getVelocityChange(), towards the bottom.

Supporting this in our differentiation means tracking all the CFM constants for each element of the diagonal of A, and storing them for later. These are constants wrt differentation, but we need to ensure that we scale A's diagonals, and the gradient of A's diagonals, by these constants wherever it is computed.

Hi,

Thanks a lot for the awesome work! I am running the arm_ik.py example, but I noticed that the state directly is optimized for that sample. However I would like to optimize the action directly. When I did it in a straightforward way by replacing state -= learning_rate * state.grad with action it doesn't work. Could you help me with that?

Running DDP optimization (main.py) for 2d reacher env works in the forward calculation of the cost but during the backward operation yields the following error:

RuntimeError: Function DartMapPositionBackward returned an invalid gradient at index 0 - got [14] but expected shape compatible with [2]

See the description here. It's possible to get much, much more accurate finite difference estimates of gradients with careful computation.

Currently, there are spots where our tests have to have larger tolerances than 1e-8 that we like to use, because the finite differencing introduces error into computing the ground truth. This is undesirable, because it means we could be missing bugs in our analytical gradients.

Double-cartpole, accessible by loading the attached skel file, used to vibrate uncontrollably on pip3 install diffdart==0.2.5. Now, somehow, on pip3 install diffdart>=0.2.6 the problem seems to have gone away... We should investigate to figure out what changed.
inverted_double_pendulum.zip

Creating a PR triggers a new build in the Azure pipeline system, which always fails to clone the repo with "fatal: reference is not a tree"

I've been trying to set up a simple pool table simulation, and haven't been able to correctly configure Nimble to simulate the collision events properly.

I have two balls set up on a flat plane (mcs_coll_test.skel file included in zip). The dark blue ball has an initial velocity such that it will collide with the light blue ball, and I'm trying to optimize that initial velocity such that the light blue ball rests on a goal position (nimble_skel_coll_test.py included in zip).

When assembling the skel file I tried to model it after the example 'mesh_collision.skel' (included in zip). I also saw what I perceived to be "weirdness" in this world when object masses are equivalent, with the bottom-most box shifting in response to the dropped block falling to the side despite an equal or greater mass.

So all in all, how do we allow for the interaction and transfer of forces between objects (collisions)?

nimble_examples.zip

We don't have good tests for MappedBackpropSnapshot::backprop, and that's a problem. This could be a source of bugs for IK trajectories, for example.

This is all around unittests/comprehensive/test_AtlasTrajectory.cpp:

We'd like to have Atlas train a trajectory without tripping our World::setSlowDebugResultsAgainstFD(true) flag (see Finding and fixing bugs in the wiki). Currently, it crashes quite quickly.

To hunt for new bugs, use the TEST(ATLAS, FULL_TEST) test towards the bottom the file. That attempts to train a trajectory involving Atlas for 500 iterations, and then display the resulting trajectory in a web GUI.

Currently, the full test crashes after less than 1 iteration of training. I've copied the replication instructions from that failure out in TEST(ATLAS, BROKEN_2). The root cause appears to be that the Jacobian of the vector b that feeds into our LCP(A,b) solver is slightly wrong. As context, the b vector that goes into the LCP solver is a vector of the relative velocities at each of the contact points before the LCP solve (see the wiki for more details). I'm not sure why this Jacobian is wrong in this case, and it's not wrong by much, but the Jacobian needs to be broken down into its sub-components and tested to figure out where the error is creeping in. You can use BackpropSnapshot::scratch() to help explore which pieces of b are introducing our errors.

Once whatever it is is fixed, it's back to TEST(ATLAS, FULL_TEST) to mine for more bugs, until Atlas passes with a clean bill of health :)

In traditional DART, time integration uses v_{t+1} to compute p_{t+1} = p_t + dt * v_{t+1}. This introduces problematic dependencies for backpropagation, so we've changed it to p_{t+1} = p_t + dt * v_t. This behavior can be turned on and off with World::setParallelVelocityAndPositionUpdates(bool enabled).

When we turn this on, it creates vibrations in double-cartpole. Unfortunately, our gradients currently assume that we always have this flag turned on, and so produce incorrect Jacobians if we turn the flag off. Ideally, we'd like to be able to handle backprop even when we're using the old time integration scheme.

We've got a test, unittests/comprehensive/test_ParallelPosAndVel.cpp to produce this vibration behavior with double cartpole.

Fixing this will require some thought.

In DARTCollide::dBoxBox(), there's an edge case at the end where we have face-face contact that can generate up to 8 contact points. These contact points are redundant for a simulation, and may reduce the stability of the LCP solver. But the current method of reducing as many as 8 contacts to 4, by just taking the 4 contacts with the greatest penetration depth, introduces discontinuities when finite differencing. Because of that, we currently return all contacts, even if there are more than 4.

We could introduce a way to reduce redundant contacts by some smooth process, like a weighted average, if we get around to it.

We'd like to support damping and springs in our Jacobians. Currently, they're automatically disabled by World::addSkeleton(), which also prints warning messages. That code should be deleted from World::addSkeleton(), we should add tests to unittests/comprehensive/test_Gradients.cpp for a skeleton with springs, one with damping, and one with both combined.

Need to show an informative error when you call getXxJacobian() with anything other than a world (like a robot)
Bonus: make an overloaded version of each getXxJacobian() that works with robots instead of worlds

This grows out of Yannis attempting to call getXxJacobian(robot), hoping to get a Jacobian for just that robot. It's a totally intuitive thing to do that you would expect to work, but instead it segfaults. We should print more helpful error messages when this happens.

Currently, neural.IKMapping doesn't respect joint limits when solving forward IK. It seems like it really should, and it shouldn't be too hard to do.

Current naming of the function is misleading. getBodyNodeIndex() sounds like it returns the index of a BodyNode. Rename to getBodyNodeByIndex() to better reflect what this function does.

Will DiffDART be like PyBullet but with DART?

Or are there plans for Gazebo integration? Since Gazebo doesn't have good Python API like PyBullet, I'd imagine the system would be Gazebo Plugins-based, with Gazebo ROS publisher nodes calling backward() after physics time step, and some agent subscriber nodes receiving the signal to start collecting grad for updating weights.