How to use Stereo DNN with trailnet and px4_controller? about redtail HOT 1 OPEN

Our work on PX4 for the rover is blocked on PX4 stack properly supporting rovers. We are following up with PX4 team on this right now. Hopefully, if they support offboard for the rovers, then we will update PX4_controller with waypoints. You are right, rc_overrides are not optimal there.
As for stereo DNN - you can host our runtime for it with the provided models as say ROS node. It can consume images from ZED camera (with ZED SDK) and estimate disparity/depth maps. We did this internally and it works.

We were not planning to add this code to GitHub, since thought that stereo runtime is enough. We can do it if we have enough bandwidth due to other projects (at NVIDIA). For now, you can try implementing a ROS node that hosts stereo DNN and contribute it to the project.

A note on performance: our smallest stereo DNN model runs at only 3 FPS on Jetson TX2 (due to use of 3D convolutions). We have a different/less accurate model that can run at 10-30 FPS on TX2. We are thinking to add that binary model to GitHub as well (around June). It was trained specifically for ZED camera. Then the only thing missing will be the ROS node to wrap stereo DNN runtime.

Thanks!

from redtail.