Comments (8)
Hello,
That sounds like a good plan. We haven't uploaded the raw drone FPV video, let me check if that can be done quickly.
For now, you can first try playing our test images using image_pub node to make sure your build produces expected results on most basic scenarios. Essentially you need to make sure that if you play, say, "right" image continuously then your drone makes something that vaguely resembles a circle.
Next, you can feed videos from IDSIA Forest Trails dataset using the same image_pub
node.
I would recommend doing both steps in simulator first before going outside.
Let us know how it works and don't forget to record and share the videos! :)
from redtail.
Let me ensure the drone can position hold with px4low first, today I find a wide and open place to test optical flow. Unfortunately, the drone always drifts from right to left, but altitude looks keeping good, I can see the led is green. Before that, I checked px4flow were connected okay, I hand it with 1 meter distance to ground, the led can turn green.
I have few experience on px4flow, could you help me analysis what happened with it according your experience. Some parameters are set incorrectly?
btw, I have calibrate the sensor with 1m distance.
here is the parameters I referred:
Name value
BFLOW_F_THLD 5000
BFLOW_V_THLD 30
BFLOW_HIST_FIL 0
BFLOW_GYRO_COM 1
BFLOW_LP_FIL 0
BFLOW_W_NEW 0.3
DEBUG 0
GYRO_SENS_DPS 250
GYRO_COMP_THR 0.01
IMAGE_WIDTH 64
IMAGE_HEIGHT 64
IMAGE_L_LIGHT 0
IMAGE_NOISE_C 1
IMAGE_TEST_PAT 0
LENS_FOCAL_LEN 16
POSITION 0
SONAR_FILTERED 0
SONAR_KAL_L1 0.8461
SONAR_KAL_L2 6.2034
SYS_ID 81
SYS_COMP_ID 50
SYS_SENSOR_ID 77
SYS_TYPE 0
SYS_AP_TYPE 1
SYS_SW_VER 13XX
SYS_SEND_STATE 1
USART_2_BAUD 115200
USART_3_BAUD 115200
USB_SEND_VIDEO 1
USB_SEND_FLOW 1
USB_SEND_GYRO 1
USB_SEND_FWD 0
USB_SEND_DEBUG 1
VIDEO_RATE 150
VIDEO_ONLY 0
from redtail.
Making PX4FLOW work can sometimes be a bit tricky indeed. Please make sure that you read and followed our PX4FLOW instructions. That is, check the firmware, lenses and camera video quality. If everything is set up correctly then PX4FLOW should initialize when you lift the drone above the ground about 20-30 cm (as described here).
It might be also useful to see the output of PX4FLOW module in QGroundControl: connect PX4FLOW to your computer via USB, run QGC and select Widgets -> Analyze
menu. Add PX4FlOW topics (I don't remember the names but it should be clear from topic names) and check that x,y coordinates are stable.
If PX4FLOW is configured properly and there is enough texture on the ground then the drone should not drift while flying.
P.S. I'm re-assigning the issue to Nikolai as I'm at the conference now.
from redtail.
Thanks, I will try.
btw, image_pub node looks not built in ws.
Can you make changes as below, it works well on my tx1.
diff --git a/ros/scripts/jetson_ros_install.sh b/ros/scripts/jetson_ros_install.sh
index 1d036bc..e0c207d 100755
--- a/ros/scripts/jetson_ros_install.sh
+++ b/ros/scripts/jetson_ros_install.sh
@@ -143,6 +143,7 @@ if [ ! -L "$CATKIN_WS/src/caffe_ros" ]; then
# Create symlinks to catkin workspace.
ln -s $HOME/redtail/ros/packages/caffe_ros $CATKIN_WS/src/
ln -s $HOME/redtail/ros/packages/px4_controller $CATKIN_WS/src/
+ ln -s $HOME/redtail/ros/packages/image_pub $CATKIN_WS/src/
fi
echo "Installing dependencies..."
@@ -162,6 +163,8 @@ echo "Building caffe_ros package..."
catkin_make caffe_ros_node
echo "Building px4_controller package..."
catkin_make px4_controller_node
+echo "Building image_pub package..."
+catkin_make image_pub_node
# Environment setup.
echo "source $CATKIN_WS/devel/setup.bash" >> $HOME/.bashrc
diff --git a/ros/packages/image_pub/CMakeLists.txt b/ros/packages/image_pub/CMakeLists.txt
index a2f31ff..05c3704 100644
--- a/ros/packages/image_pub/CMakeLists.txt
+++ b/ros/packages/image_pub/CMakeLists.txt
@@ -6,6 +6,14 @@ find_package(catkin REQUIRED COMPONENTS
std_msgs
)
+catkin_package(
+### INCLUDE_DIRS include
+ LIBRARIES image_pub_node
+ CATKIN_DEPENDS roscpp std_msgs
+#### DEPENDS system_lib
+)
+
+
###########
## Build ##
###########
from redtail.
Hi Alexey, Nikolai
My px4flow is not ready now.
So, I use gazebo simulation on x64 system to simulate image pub and offboard flight control.
The configuration is a bit different with yours, my configuration is real tx1(image pub, mavros, trailnet_dnn, yolo_dnn, px4-controller) + x64 PC(gazebo, qground), that can workaround your nvidia graphics card on PC and also without installing docker.io. I think the principle is the same.
As you mentioned before, when pub a jpg image(redtail/ros/packages/caffe_ros/tests/data/rot_c.jpg) with repeatable mode, the drone circle and circle....
I think it's the reasonable logic, the same frame/image input, the same angle output; so, circle...circle.., but it's not the expected behavior, if the previous one frame was complete same as current one, the dnn network should ignore it, and keep the drone stop and stay in current position. Another test, when I ctrl+c image pub node, there was no image data input, the drone stopped and stay in current position, it looks safe to us.
from redtail.
Your setup is another good option to test/develop the components. Basically, since all our components are implemented as ROS nodes, you can mix and match workstations, Jetsons and Dockers in any way you like :) We decided to provide and support just 2 scenarios, one based on real Jetson and another - Docker. Such scenarios are relatively easily reproducible and repeatable (especially with Docker).
As for the image_pub
node and what to do when the same image is published repeatedly: you can change the code to check for image similarity and do something else (like stopping) in case your metric reaches certain threshold. There is currently no safety mechanisms other than very basic ones like when controller dies, PX4 turns off the offboard mode and switches back to position hold.
from redtail.
@xuming-shanghai: Hello, for PX4FLOW floating around -- what lens did you use on PX4FLOW unit? The parameters you posted list: LENS_FOCAL_LEN 16 which is 16mm focal length and it is for the lens that comes default with the unit. If you replaced it with 6mm lens like we suggest, then you need to recompile PX4FLOW firmware after changing the focal length in the settings .h file. It is listed in PX4FLOW instructions.
from redtail.
@xuming-shanghai: just to explain what's going on if the focal length is wrong - PX4FLOW cannot estimate velocity properly (due to wrong camera model) and so this leads to overcontrol in Pixhawk. It will stabilize, but it will "float" around (typically side to side)
from redtail.
Related Issues (20)
- camera image not received for gscam kind camera in trailnet_debug_gscam.launch/trailnet_debug_zed_gscam.launch
- stereoDNN resnet18 test fails HOT 2
- gscam problem with no video shown, even not shown in rqt-image window, not even gst-launch on host machine HOT 1
- tx2 onboard connection with px4 flushed by apm not connected even with mavros HOT 11
- gscam donot stream on nano,but do stream on tx2
- lateral offset dataset
- Failed with error code 1 in /tmp/pip-build-U_9LbZ/numpy/
- Extended Redtail implementation for Arducopter HOT 7
- Training Resnet18_2D
- Error Loading TensorRT plan for ResNet-18_2D HOT 1
- Testing in Simulator
- Regarding tensorRT of TrailNet_SResNet-18 HOT 1
- Takeoff Mode Initiated Mid-air when starting px4_controller node in Navigate Mode HOT 2
- Unable to complete build_redtail_image.sh script HOT 3
- clarification on difference between visionworks and redtail? HOT 1
- Dataset used for training HOT 2
- The segmentation effect in your video HOT 1
- Jetpack 4.4 caffe_ros node Assertion failed HOT 3
- 2.4 GHz range cut due to Jetson RF noise HOT 6
- Could not find a configuration file for package "OpenCV"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redtail.