mgonzs13 / yolov8_ros Goto Github PK

Ultralytics YOLOv8, YOLOv9, YOLOv10 for ROS 2

License: GNU General Public License v3.0

CMake 1.57% Python 98.43%

object-detection ros2 ultralytics yolov8 object-tracking 3d-human-pose-estimation 3d-object-detection human-pose-estimation instance-segmentation yolov9

yolov8_ros's Introduction

yolov8_ros

ROS 2 wrap for Ultralytics YOLOv8 to perform object detection and tracking, instance segmentation and human pose estimation. There are also 3D versions of object detection and human pose estimation based on depth images.

Installation

$ cd ~/ros2_ws/src
$ git clone https://github.com/mgonzs13/yolov8_ros.git
$ pip3 install -r yolov8_ros/requirements.txt
$ cd ~/ros2_ws
$ rosdep install --from-paths src --ignore-src -r -y
$ colcon build

Usage

YOLOv8 / YOLOv9

$ ros2 launch yolov8_bringup yolov8.launch.py

$ ros2 launch yolov8_bringup yolov9.launch.py

Topics

/yolo/detections: Objects detected by YOLO using the RGB images. Each object contains a bounding boxes and a class name. It may also include a mak or a list of keypoints.
/yolo/tracking: Objects detected and tracked from YOLO results. Each object is assigned a tracking ID.
/yolo/debug_image: Debug images showing the detected and tracked objects. They can be visualized with rviz2.

Parameters

model: YOLOv8 model (default: yolov8m.pt)
tracker: Tracker file (default: bytetrack.yaml)
device: GPU/CUDA (default: cuda:0)
enable: Wether to start YOLOv8 enabled (default: True)
threshold: Detection threshold (default: 0.5)
input_image_topic: Camera topic of RGB images (default: /camera/rgb/image_raw)
image_reliability: Reliability for the image topic: 0=system default, 1=Reliable, 2=Best Effort (default: 2)

YOLOv8 3D

$ ros2 launch yolov8_bringup yolov8_3d.launch.py

Topics

/yolo/detections: Objects detected by YOLO using the RGB images. Each object contains a bounding boxes and a class name. It may also include a mak or a list of keypoints.
/yolo/tracking: Objects detected and tracked from YOLO results. Each object is assigned a tracking ID.
/yolo/detections_3d: 3D objects detected. YOLO results are used to crop the depth images to create the 3D bounding boxes and 3D keypoints.
/yolo/debug_image: Debug images showing the detected and tracked objects. They can be visualized with rviz2.

Parameters

model: YOLOv8 model (default: yolov8m.pt)
tracker: tracker file (default: bytetrack.yaml)
device: GPU/CUDA (default: cuda:0)
enable: wether to start YOLOv8 enabled (default: True)
threshold: detection threshold (default: 0.5)
input_image_topic: camera topic of RGB images (default: /camera/rgb/image_raw)
image_reliability: reliability for the image topic: 0=system default, 1=Reliable, 2=Best Effort (default: 2)
input_depth_topic: camera topic of depth images (default: /camera/depth/image_raw)
depth_image_reliability: reliability for the depth image topic: 0=system default, 1=Reliable, 2=Best Effort (default: 2)
input_depth_info_topic: camera topic for info data (default: /camera/depth/camera_info)
depth_info_reliability: reliability for the depth info topic: 0=system default, 1=Reliable, 2=Best Effort (default: 2)
depth_image_units_divisor: divisor to convert the depth image into metres (default: 1000)
target_frame: frame to transform the 3D boxes (default: base_link)
maximum_detection_threshold: maximum detection threshold in the z axis (default: 0.3)

Lifecycle nodes

Previous updates add Lifecycle Nodes support to all the nodes available in the package. This implementation tries to reduce the workload in the unconfigured and inactive states by only loading the models and activating the subscriber on the active state.

These are some resource comparisons using the default yolov8m.pt model on a 30fps video stream.

State	CPU Usage (i7 12th Gen)	VRAM Usage	Bandwidth Usage
Active	40-50% in one core	628 MB	Up to 200 Mbps
Inactive	~5-7% in one core	338 MB	0-20 Kbps

Demos

Object Detection

This is the standard behavior of YOLOv8, which includes object tracking.

$ ros2 launch yolov8_bringup yolov8.launch.py

Instance Segmentation

Instance masks are the borders of the detected objects, not the all the pixels inside the masks.

$ ros2 launch yolov8_bringup yolov8.launch.py model:=yolov8m-seg.pt

Human Pose

Online persons are detected along with their keypoints.

$ ros2 launch yolov8_bringup yolov8.launch.py model:=yolov8m-pose.pt

3D Object Detection

The 3D bounding boxes are calculated filtering the depth image data from an RGB-D camera using the 2D bounding box. Only objects with a 3D bounding box are visualized in the 2D image.

$ ros2 launch yolov8_bringup yolov8_3d.launch.py

3D Object Detection (Using Instance Segmentation Masks)

In this, the depth image data is filtered using the max and min values obtained from the instance masks. Only objects with a 3D bounding box are visualized in the 2D image.

$ ros2 launch yolov8_bringup yolov8_3d.launch.py model:=yolov8m-seg.pt

3D Human Pose

Each keypoint is projected in the depth image and visualized using purple spheres. Only objects with a 3D bounding box are visualized in the 2D image.

$ ros2 launch yolov8_bringup yolov8_3d.launch.py model:=yolov8m-pose.pt

yolov8_ros's People

Contributors

Stargazers

Watchers

yolov8_ros's Issues

how to set the live stream for detection?

I have started the camera and publish image, the default
value 'input_image_topic' in launch is changed to correct path.

however, no message published in /yolo/detections ?

How can I get the image?

how can i convert 3d coordinate to real word coordinate?

thanks for this pretty project, i can get 3d coordinate with launch file:yolov8_3d.launch.py, if i want to get the real world coordinate, how can i do it?

Issue when trying to launch yolov8 - using realsense camera

Hello,

I'm trying to launch the yolov8, in 2D and 3D based on the instructions in your readme file. However, I keep facing an issue when I update the "input_image_topic" variable to "/camera/camera/color/image_raw" which matches my realsense topic, can be seen in rviz2.

Please see error below:

~/ros2_ws$ ros2 launch yolov8_bringup yolov8.launch.py
[INFO] [launch]: All log files can be found below /home/myaccount/.ros/log/2024-02-04-05-15-45-946799-myaccount-OS-114383
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [114384]
[INFO] [tracking_node-2]: process started with pid [114386]
[INFO] [debug_node-3]: process started with pid [114388]
[yolov8_node-1] Traceback (most recent call last):
[yolov8_node-1] File "/home/myaccount/ros2_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 33, in
[yolov8_node-1] sys.exit(load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'yolov8_node')())
[yolov8_node-1] File "/home/myaccount/ros2_ws/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/yolov8_node.py", line 245, in main
[yolov8_node-1] rclpy.spin(node)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/init.py", line 222, in spin
[yolov8_node-1] executor.spin_once()
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 713, in spin_once
[yolov8_node-1] raise handler.exception()
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/task.py", line 239, in call
[yolov8_node-1] self._handler.send(None)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 418, in handler
[yolov8_node-1] await call_coroutine(entity, arg)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 343, in _execute_subscription
[yolov8_node-1] await await_or_execute(sub.callback, msg)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 107, in await_or_execute
[yolov8_node-1] return callback(*args)
[yolov8_node-1] File "/home/myaccount/ros2_ws/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/yolov8_node.py", line 196, in image_cb
[yolov8_node-1] results = self.yolo.predict(
[yolov8_node-1] File "/home/myaccount/.local/lib/python3.10/site-packages/ultralytics/engine/model.py", line 250, in predict
[yolov8_node-1] self.predictor.setup_model(model=self.model, verbose=is_cli)
[yolov8_node-1] File "/home/myaccount/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py", line 321, in setup_model
[yolov8_node-1] device=select_device(self.args.device, verbose=verbose),
[yolov8_node-1] File "/home/myaccount/.local/lib/python3.10/site-packages/ultralytics/utils/torch_utils.py", line 119, in select_device
[yolov8_node-1] raise ValueError(f"Invalid CUDA 'device={device}' requested."
[yolov8_node-1] ValueError: Invalid CUDA 'device=0' requested. Use 'device=cpu' or pass valid CUDA device(s) if available, i.e. 'device=0' or 'device=0,1,2,3' for Multi-GPU.
[yolov8_node-1]
[yolov8_node-1] torch.cuda.is_available(): False
[yolov8_node-1] torch.cuda.device_count(): 0
[yolov8_node-1] os.environ['CUDA_VISIBLE_DEVICES']: None
[yolov8_node-1] See https://pytorch.org/get-started/locally/ for up-to-date torch install instructions if no CUDA devices are seen by torch.
[yolov8_node-1]
[ERROR] [yolov8_node-1]: process has died [pid 114384, exit code 1, cmd '/home/myaccount/ros2_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node --ros-args -r __node:=yolov8_node -r __ns:=/yolo --params-file /tmp/launch_params_315kfwu6 -r image_raw:=/camera/camera/color/image_raw'].

Would appreciate any help. I tried to adjust the QoS but still no Image or data is being shown.

base_link passed to lookupTransform argument target_frame does not exist

Hello there,

Trying to use a DepthAi Luxonis Pro Wide POE camera with a ROS setup. I am getting the disparity frame topic published to /oak/stereo/image_raw, but this error gets created which I do not understand.

ros2 launch yolov8_bringup yolov8_3d.launch.py input_image_topic:=/oak/rgb/image_raw input_depth_topic:=/oak/stereo/image_raw input_depth_info_topic:=/oak/stereo/camera_info
[INFO] [launch]: All log files can be found below /home/foo/.ros/log/2024-02-13-18-45-10-639973-foo-desktop-4676
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [4677]
[INFO] [tracking_node-2]: process started with pid [4679]
[INFO] [detect_3d_node-3]: process started with pid [4681]
[INFO] [debug_node-4]: process started with pid [4683]
[yolov8_node-1] /home/foo/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[yolov8_node-1] warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
[tracking_node-2] /home/foo/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[tracking_node-2] warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
[debug_node-4] /home/foo/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[debug_node-4] warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
[detect_3d_node-3] [ERROR] [1707867925.587148194] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867925.805917716] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867925.877574412] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.300230844] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.489552873] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.595309086] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.655508926] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.697220906] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.741311144] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867926.820498681] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.
[detect_3d_node-3] [ERROR] [1707867927.058484653] [yolo.detect_3d_node]: Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist.

I also get this package error, though I am not sure how to solve it:

/home/foo/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[yolov8_node-1] warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
[tracking_node-2] /home/foo/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[tracking_node-2] warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
[debug_node-4] /home/foo/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[debug_node-4] warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "

Please let me know any steps I could take in order to get this working. Thank you.

Encoding error by ultralytics prediction

Hi @mgonzs13,

thank you for your great work!!!

I have an error when when running the yolov8_node the self.yolo.predict cause an error (tried on 2 different cameras):

[yolov8_node-1] RuntimeError: Given groups=1, weight of size [48, 3, 3, 3], expected input[1, 4, 384, 640] to have 3 channels, but got 4 channels instead

I could fix this after i changed the desired encoding on the cv2_bridge:
from:

self.cv_bridge.imgmsg_to_cv2(msg)

self.cv_bridge.imgmsg_to_cv2(msg, desired_encoding='bgr8')

should we maybe add this permanent or is there some reason this encoding is empty?

Policy CMP0148

Hi,
When I try to build the package I meet this warning:

--- stderr: yolov8_ros
/home/ntnguyen/.local/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

    ********************************************************************************
    Please avoid running ``setup.py`` directly.
    Instead, use pypa/build, pypa/installer or other
    standards-based tools.

    See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
    ********************************************************************************

!!
self.initialize_options()

Finished <<< yolov8_ros [1.08s]
Starting >>> yolov8_bringup
--- stderr: yolov8_bringup
CMake Warning (dev) at /opt/ros/foxy/share/ament_cmake_core/cmake/core/python.cmake:22 (find_package):
Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
are removed. Run "cmake --help-policy CMP0148" for policy details. Use
the cmake_policy command to set the policy and suppress this warning.

Call Stack (most recent call first):
/opt/ros/foxy/share/ament_cmake_core/cmake/core/all.cmake:53 (include)
/opt/ros/foxy/share/ament_cmake_core/cmake/ament_cmake_core-extras.cmake:17 (include)
/opt/ros/foxy/share/ament_cmake_core/cmake/ament_cmake_coreConfig.cmake:41 (include)
/opt/ros/foxy/share/ament_cmake/cmake/ament_cmake_export_dependencies-extras.cmake:15 (find_package)
/opt/ros/foxy/share/ament_cmake/cmake/ament_cmakeConfig.cmake:41 (include)
CMakeLists.txt:9 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

When I launched the package I met this error:
/opt/ros/foxy/bin/ros2:6: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pkg_resources import load_entry_point
[INFO] [launch]: All log files can be found below /home/ntnguyen/.ros/log/2023-08-17-11-59-28-750244-trunkngin-9360
[INFO] [launch]: Default logging verbosity is set to INFO
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<LaunchService._process_one_event() done, defined at /opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py:226> exception=InvalidLaunchFileError('py')>
Traceback (most recent call last):
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_description_sources/any_launch_file_utilities.py", line 53, in get_launch_description_from_any_launch_file
return loader(launch_file_path)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_description_sources/python_launch_file_utilities.py", line 62, in get_launch_description_from_python_launch_file
launch_file_module = load_python_launch_file_as_module(python_launch_file_path)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_description_sources/python_launch_file_utilities.py", line 37, in load_python_launch_file_as_module
loader.exec_module(mod)
File "", line 844, in exec_module
File "", line 980, in get_code
File "", line 1037, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/home/ntnguyen/gcamp_ros2_ws/install/yolov8_bringup/share/yolov8_bringup/launch/yolov8.launch.py'

The above exception was the direct cause of the following exception:

Run YOlo8 in ROS

Hi,

Can we run this wrapper in ROS?

Thanks,
Golnaz

How to test this project for ROS noetic

Hi,
is it project compatible with ROS noetic?

AttributeError: type object 'SetBool' has no attribute 'Request'

~/v8_ros2/src/yolov8_ros$  ros2 launch yolov8_bringup yolov8.launch.py device:=cpu
[INFO] [launch]: All log files can be found below /home/jtw/.ros/log/2023-06-27-09-51-51-294832-home-5348
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [5350]
[yolov8_node-1] Traceback (most recent call last):
[yolov8_node-1]   File "/home/jtw/v8_ros2/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 33, in <module>
[yolov8_node-1]     sys.exit(load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'yolov8_node')())
[yolov8_node-1]   File "/home/jtw/v8_ros2/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 25, in importlib_load_entry_point
[yolov8_node-1]     return next(matches).load()
[yolov8_node-1]   File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
[yolov8_node-1]     module = import_module(match.group('module'))
[yolov8_node-1]   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
[yolov8_node-1]     return _bootstrap._gcd_import(name[level:], package, level)
[yolov8_node-1]   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
[yolov8_node-1]   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
[yolov8_node-1]   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
[yolov8_node-1]   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
[yolov8_node-1]   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
[yolov8_node-1]   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
[yolov8_node-1]   File "/home/jtw/v8_ros2/install/yolov8_ros/lib/python3.8/site-packages/yolov8_ros/yolov8_node.py", line 40, in <module>
[yolov8_node-1]     class Yolov8Node(Node):
[yolov8_node-1]   File "/home/jtw/v8_ros2/install/yolov8_ros/lib/python3.8/site-packages/yolov8_ros/yolov8_node.py", line 98, in Yolov8Node
[yolov8_node-1]     req: SetBool.Request,
[yolov8_node-1] AttributeError: type object 'SetBool' has no attribute 'Request'
[ERROR] [yolov8_node-1]: process has died [pid 5350, exit code 1, cmd '/home/jtw/v8_ros2/install/yolov8_ros/lib/yolov8_ros/yolov8_node --ros-args -r __node:=yolov8_node -r __ns:=/yolo --params-file /tmp/launch_params_rqr5qbe3 -r image_raw:=/camera/rgb/image_raw'].

what I should do to resolve the error

distance to object

how to get distance to an object and publish it with object name and it's score?
for box_data in results.boxes:
hypothesis = {
"class_id": int(box_data.cls),
"class_name": self.yolo.names[int(box_data.cls)],
"score": float(box_data.conf)
}
for example in this part add distance line

custom yolo pretrained problem.

It has problem when using the custom model, it is a yolov8s-seg model trained by me with the pre-trained models, and it does not le me use it, however yolov9m-seg allowed me to use it.

I tryed ros2 launch yolov8_bringup yolov8.launch.py model:=yolov8s-seg.pt
And show below infos in terminal.

[INFO] [launch]: All log files can be found below /home/xiao/.ros/log/2024-05-07-00-15-06-915019-xiao-laptop-3712
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [3713]
[INFO] [tracking_node-2]: process started with pid [3715]
[INFO] [debug_node-3]: process started with pid [3717]
[tracking_node-2] /usr/lib/python3/dist-packages/scipy/init.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.4
[tracking_node-2] warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
[yolov8_node-1] [INFO] [1715033711.300011187] [yolo.yolov8_node]: Yolov8Node created
[yolov8_node-1] [INFO] [1715033711.300812167] [yolo.yolov8_node]: Configuring yolov8_node
[tracking_node-2] [INFO] [1715033711.372381450] [yolo.tracking_node]: Configuring tracking_node
[yolov8_node-1] [INFO] [1715033711.388569432] [yolo.yolov8_node]: Activating yolov8_node
[tracking_node-2] [INFO] [1715033711.464118411] [yolo.tracking_node]: Activating tracking_node
[debug_node-3] [INFO] [1715033711.501457219] [yolo.debug_node]: Debug node created
[debug_node-3] [INFO] [1715033711.502009061] [yolo.debug_node]: Configuring debug_node
[debug_node-3] [INFO] [1715033711.592135822] [yolo.debug_node]: Activating debug_node
[yolov8_node-1] Traceback (most recent call last):
[yolov8_node-1] File "/home/xiao/ros2_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 33, in
[yolov8_node-1] sys.exit(load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'yolov8_node')())
[yolov8_node-1] File "/home/xiao/ros2_ws/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/yolov8_node.py", line 293, in main
[yolov8_node-1] rclpy.spin(node)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/init.py", line 222, in spin
[yolov8_node-1] executor.spin_once()
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 739, in spin_once
[yolov8_node-1] self._spin_once_impl(timeout_sec)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 736, in _spin_once_impl
[yolov8_node-1] raise handler.exception()
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/task.py", line 239, in call
[yolov8_node-1] self._handler.send(None)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 437, in handler
[yolov8_node-1] await call_coroutine(entity, arg)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 362, in _execute_subscription
[yolov8_node-1] await await_or_execute(sub.callback, msg)
[yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 107, in await_or_execute
[yolov8_node-1] return callback(*args)
[yolov8_node-1] File "/home/xiao/ros2_ws/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/yolov8_node.py", line 239, in image_cb
[yolov8_node-1] results = self.yolo.predict(
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/engine/model.py", line 439, in predict
[yolov8_node-1] return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py", line 168, in call
[yolov8_node-1] return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 35, in generator_context
[yolov8_node-1] response = gen.send(None)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py", line 228, in stream_inference
[yolov8_node-1] self.model.warmup(imgsz=(1 if self.model.pt or self.model.triton else self.dataset.bs, 3, *self.imgsz))
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/nn/autobackend.py", line 596, in warmup
[yolov8_node-1] self.forward(im) # warmup
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/nn/autobackend.py", line 425, in forward
[yolov8_node-1] y = self.model(im, augment=augment, visualize=visualize, embed=embed)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
[yolov8_node-1] return self._call_impl(*args, **kwargs)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
[yolov8_node-1] return forward_call(*args, **kwargs)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/nn/tasks.py", line 89, in forward
[yolov8_node-1] return self.predict(x, *args, **kwargs)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/nn/tasks.py", line 107, in predict
[yolov8_node-1] return self._predict_once(x, profile, visualize, embed)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/nn/tasks.py", line 128, in _predict_once
[yolov8_node-1] x = m(x) # run
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
[yolov8_node-1] return self._call_impl(*args, **kwargs)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
[yolov8_node-1] return forward_call(*args, **kwargs)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/ultralytics/nn/modules/head.py", line 111, in forward
[yolov8_node-1] x = self.detect(self, x)
[yolov8_node-1] File "/home/xiao/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1709, in getattr
[yolov8_node-1] raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
[yolov8_node-1] AttributeError: 'Segment' object has no attribute 'detect'
[ERROR] [yolov8_node-1]: process has died [pid 3713, exit code 1, cmd '/home/xiao/ros2_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node --ros-args -r __node:=yolov8_node -r __ns:=/yolo --params-file /tmp/launch_params_zms6h5zx -r image_raw:=/camera/rgb/image_raw'].

[debug_node-3] AttributeError: module 'cv2' has no attribute 'Mat'

Hi @mgonzs13
Thanks for the great repo.

I am using your latest main branch inside a docker container on Jetson Orin Nx with Ros2 container.

My Opncv version is 4.5.0

The debug_node throws the following error.

[debug_node-3] AttributeError: module 'cv2' has no attribute 'Mat'

Any idea what could be the issue ?

Train

Is this version of code unable to modify network structure and train?

How to Deploy a Single Instance of yolov8_ros Node for Multiple Camera Detections

I am currently utilizing the yolov8_ros node for object detections and tracking on camera streams. However, the current setup involves deploying a separate yolov8_ros node for each camera, leading to performance issues. I am interested in exploring the possibility of deploying a single instance of the yolov8_ros node to handle detections on multiple cameras concurrently.

Here is a brief overview of the current vs desired process:

Current System:
Desired System:

missing pip module lap

The following failed with missing module lap ... needed to run pip install lap

ros2 launch yolov8_bringup yolov8.launch.py device:=CPU input_image_topic:=/base_cam/image_raw

Missing yolov8_msgs dependency

When one builds using colcon build --packages-select yolov8_bringup and then attempt to launch via ros2 launch yolov8_bring yolov8.launch.py you get an error about missing yolov8_msgs dependency. This happens because in yolov8_ros/package.xml it is missing <depend>yolov8_msgs</depend>.

3D object detection Transformation issue

Hi,
I am trying to run the stack with only D435 camera. As I dont have the base_link, I change the target_frame into camera_link. However, I am still getting the following error even though the camera_link frame exists.

"[detect_3d_node-3] [ERROR] [1709019322.086835673] [yolo.detect_3d_node]: Could not transform: "camera_link" passed to lookupTransform argument target_frame does not exist.
"

size error

[INFO] [launch]: All log files can be found below /home/turtlebot/.ros/log/2023-10-12-13-43-33-963493-ubuntu-24081
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [24083]
[INFO] [tracking_node-2]: process started with pid [24085]
[INFO] [debug_node-3]: process started with pid [24087]
[yolov8_node-1] YOLOv8m summary (fused): 218 layers, 25886080 parameters, 0 gradients, 78.9 GFLOPs
[yolov8_node-1] Traceback (most recent call last):
[yolov8_node-1] File "/home/turtlebot/yolo_deepsort/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 33, in
[yolov8_node-1] sys.exit(load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'yolov8_node')())
[yolov8_node-1] File "/home/turtlebot/yolo_deepsort/install/yolov8_ros/lib/python3.8/site-packages/yolov8_ros/yolov8_node.py", line 231, in main
[yolov8_node-1] rclpy.spin(node)
[yolov8_node-1] File "/opt/ros/galactic/lib/python3.8/site-packages/rclpy/init.py", line 196, in spin
[yolov8_node-1] executor.spin_once()
[yolov8_node-1] File "/opt/ros/galactic/lib/python3.8/site-packages/rclpy/executors.py", line 713, in spin_once
[yolov8_node-1] raise handler.exception()
[yolov8_node-1] File "/opt/ros/galactic/lib/python3.8/site-packages/rclpy/task.py", line 239, in call
[yolov8_node-1] self._handler.send(None)
[yolov8_node-1] File "/opt/ros/galactic/lib/python3.8/site-packages/rclpy/executors.py", line 418, in handler
[yolov8_node-1] await call_coroutine(entity, arg)
[yolov8_node-1] File "/opt/ros/galactic/lib/python3.8/site-packages/rclpy/executors.py", line 343, in _execute_subscription
[yolov8_node-1] await await_or_execute(sub.callback, msg)
[yolov8_node-1] File "/opt/ros/galactic/lib/python3.8/site-packages/rclpy/executors.py", line 107, in await_or_execute
[yolov8_node-1] return callback(*args)
[yolov8_node-1] File "/home/turtlebot/yolo_deepsort/install/yolov8_ros/lib/python3.8/site-packages/yolov8_ros/yolov8_node.py", line 182, in image_cb
[yolov8_node-1] results = self.yolo.predict(
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/engine/model.py", line 248, in predict
[yolov8_node-1] return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/engine/predictor.py", line 194, in call
[yolov8_node-1] return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 43, in generator_context
[yolov8_node-1] response = gen.send(None)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/engine/predictor.py", line 253, in stream_inference
[yolov8_node-1] preds = self.inference(im, *args, **kwargs)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/engine/predictor.py", line 133, in inference
[yolov8_node-1] return self.model(im, augment=self.args.augment, visualize=visualize)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
[yolov8_node-1] return forward_call(*input, **kwargs)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/nn/autobackend.py", line 334, in forward
[yolov8_node-1] y = self.model(im, augment=augment, visualize=visualize) if augment or visualize else self.model(im)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
[yolov8_node-1] return forward_call(*input, **kwargs)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/nn/tasks.py", line 45, in forward
[yolov8_node-1] return self.predict(x, *args, **kwargs)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/nn/tasks.py", line 62, in predict
[yolov8_node-1] return self._predict_once(x, profile, visualize)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/nn/tasks.py", line 82, in _predict_once
[yolov8_node-1] x = m(x) # run
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
[yolov8_node-1] return forward_call(*input, **kwargs)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/ultralytics/nn/modules/conv.py", line 42, in forward_fuse
[yolov8_node-1] return self.act(self.conv(x))
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
[yolov8_node-1] return forward_call(*input, **kwargs)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 447, in forward
[yolov8_node-1] return self._conv_forward(input, self.weight, self.bias)
[yolov8_node-1] File "/home/turtlebot/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 443, in _conv_forward
[yolov8_node-1] return F.conv2d(input, weight, bias, self.stride,
[yolov8_node-1] RuntimeError: Given groups=1, weight of size [48, 3, 3, 3], expected input[1, 4, 384, 640] to have 3 channels, but got 4 channels instead

Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist

Every things is ok when I launch with command below:
ros2 launch yolov8_bringup yolov8.launch.py input_image_topic:=/camera/color/image_raw

But it could not work when I launch with command below:
ros2 launch yolov8_bringup yolov8_3d.launch.py input_image_topic:=/camera/color/image_raw input_depth_topic:=/camera/depth/image_raw

When I view yolo image in rvit ，I got error message:
Could not transform: "base_link" passed to lookupTransform argument target_frame does not exist

ModuleNotFoundError: No module named 'ultralytics.tracker'

Launching with

ros2 launch yolov8_bringup yolov8.launch.py

gives me an import error:

[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [24716]
[yolov8_node-1] Traceback (most recent call last):
[yolov8_node-1]   File "/home/aaeon/interbotix_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 11, in <module>
[yolov8_node-1]     load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'yolov8_node')()
[yolov8_node-1]   File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
[yolov8_node-1]     return get_distribution(dist).load_entry_point(group, name)
[yolov8_node-1]   File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
[yolov8_node-1]     return ep.load()
[yolov8_node-1]   File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
[yolov8_node-1]     return self.resolve()
[yolov8_node-1]   File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
[yolov8_node-1]     module = __import__(self.module_name, fromlist=['__name__'], level=0)
[yolov8_node-1]   File "/home/aaeon/interbotix_ws/install/yolov8_ros/lib/python3.8/site-packages/yolov8_ros/yolov8_node.py", line 13, in <module>
[yolov8_node-1]     from ultralytics.tracker.trackers.basetrack import BaseTrack
[yolov8_node-1] ModuleNotFoundError: No module named 'ultralytics.tracker'
[ERROR] [yolov8_node-1]: process has died [pid 24716, exit code 1, cmd '/home/aaeon/interbotix_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node --ros-args -r __node:=yolov8_node -r __ns:=/yolo --params-file /tmp/launch_params_mijvcv3d -r image_raw:=/camera/rgb/image_raw'].

Any help would be appreciated.

How do you include ultralytics library in ROS2?

I'm trying more or less to replicate your node but adding some other things and I have problems including the ultralytics library. Could you tell me how you did it?

UBUNUT & ROS VERSION

Can you please tell which Ubuntu and ROS version you have used ?

Detect_3d_node: tuple index out of range

Hi,
using Azure Kinect RGB-D to do 3d human pose detection but I am getting the following error this time.

Thank you

ros2 launch yolov8_bringup yolov8_3d.launch.py input_image_topic:=/rgb/image_formatted model:=yolov8m-pose.pt input_depth_info_topic:=/depth/camera_info_formatted input_depth_topic:=/depth/image_formatted
[INFO] [launch]: All log files can be found below /home/nam017/.ros/log/2023-12-08-15-58-06-741606-drax-bm-59509
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [yolov8_node-1]: process started with pid [59510]
[INFO] [tracking_node-2]: process started with pid [59512]
[INFO] [detect_3d_node-3]: process started with pid [59514]
[INFO] [debug_node-4]: process started with pid [59516]
[detect_3d_node-3] Traceback (most recent call last):
[detect_3d_node-3]   File "/home/nam017/ws_yolov8/install/yolov8_ros/lib/yolov8_ros/detect_3d_node", line 33, in <module>
[detect_3d_node-3]     sys.exit(load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'detect_3d_node')())
[detect_3d_node-3]   File "/home/nam017/ws_yolov8/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/detect_3d_node.py", line 317, in main
[detect_3d_node-3]     node = Detect3DNode()
[detect_3d_node-3]   File "/home/nam017/ws_yolov8/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/detect_3d_node.py", line 64, in __init__
[detect_3d_node-3]     self.depth_sub = message_filters.Subscriber(
[detect_3d_node-3]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/message_filters/__init__.py", line 78, in __init__
[detect_3d_node-3]     self.topic = args[2]
[detect_3d_node-3] IndexError: tuple index out of range
[ERROR] [detect_3d_node-3]: process has died [pid 59514, exit code 1, cmd '/home/nam017/ws_yolov8/install/yolov8_ros/lib/yolov8_ros/detect_3d_node --ros-args -r __node:=detect_3d_node -r __ns:=/yolo --params-file /tmp/launch_params_orrjs_90 -r depth_image:=/depth/image_formatted -r depth_info:=/depth/camera_info_formatted -r detections:=tracking'].
[yolov8_node-1] YOLOv8m-pose summary (fused): 237 layers, 26447596 parameters, 0 gradients, 81.0 GFLOPs

Operating system

Hi, I recently installed ROS2 Humble on windows 10 operating system.
I'd like to know if your YOLOV8 X ROS2 package can be installed and run on windows 10 operating system.

Actually I tried to install them today as your instruction, but was not really successful:
Git clone and pip install was done well, but rosdep command said there is no available package for windows OS.I ignored the error from rosdep step, and proceeded to colcon build.

The /yolo/detections topic was created well, but the debugged image was not visualized in RVIZ, and there was no data coming from ros2 topic echo /yolo/detections.

If you think your code can be run on windows 10, then I would like to ask some questions about the error I've encountered.

Thank you so much for your code!

Lifecycle nodes

Hi, I would like to measure the CPU usage per core as described in the readme. Could you please provide detailed guidance on how to do this? Additionally, is there a way to determine how many CPU cores are being used?

IndexError/ out of bounds in detect_3d_node using RealSense

Hi there,

Firstly thanks for the great repo.

I'm having some issues while trying to use the detect_3d_node.py where the convert_bb_to_3d function always gives me an IndexError.

  File "/workspaces/yolov8_ws/src/yolov8_ros/yolov8_ros/yolov8_ros/detect_3d_node.py", line 91, in on_detections
    bbox3d = self.convert_bb_to_3d(points, detection)
  File "/workspaces/yolov8_ws/src/yolov8_ros/yolov8_ros/yolov8_ros/detect_3d_node.py", line 153, in convert_bb_to_3d
    center_point = points[int(center_y)][int(center_x)]
IndexError: index 263 is out of bounds for axis 0 with size 1

Looking at line center_point = points[int(center_y)][int(center_x)] and also line 83 where the points array is constructed, it seems like it expects the point_msg.height and point_msg.width to correspond to the input image size used in the detection node. Unfortunately the depth camera/ ros2 driver I'm using (Realsense D435) constructs the PointCloud2 message with a height of 1, meaning the points array ends up with a shape of [1, num_points, 3].

I'm happy to work on a PR to fix this if you would like? I feel that assuming the PointCloud2 message width and height matches those of the colour image isn't the most robust solution. One approach I've used in the past has been to calculate the 3D bounding box by applying the segmentation mask to the depth image rather than the pointcloud, but I'm also happy to try something else if you have any suggestions.

Adam

Issues with launching the launch file

Executed the below command when in the directory where this file is present:
ros2 launch yolov8.launch.py

Error:
[INFO] [launch]: All log files can be found below /home/arunvenkat/.ros/log/2023-09-28-23-26-26-077037
[INFO] [launch]: Default logging verbosity is set to INFO
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<LaunchService._process_one_event() done, defined at /opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py:226> exception=PackageNotFoundError("package 'yolov8_ros' not found, searching: ['/opt/ros/foxy']")>
Traceback (most recent call last):
File "/opt/ros/foxy/lib/python3.8/site-packages/ament_index_python/packages.py", line 50, in get_package_prefix
content, package_prefix = get_resource('packages', package_name)
File "/opt/ros/foxy/lib/python3.8/site-packages/ament_index_python/resources.py", line 48, in get_resource
raise LookupError(
LookupError: Could not find the resource 'yolov8_ros' of type 'packages'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py", line 228, in _process_one_event
await self.__process_event(next_event)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py", line 248, in __process_event
visit_all_entities_and_collect_futures(entity, self.__context))
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
[Previous line repeated 1 more time]
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 38, in visit_all_entities_and_collect_futures
sub_entities = entity.visit(context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/action.py", line 108, in visit
return self.execute(context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch_ros/actions/node.py", line 453, in execute
ret = super().execute(context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 823, in execute
self.__expand_substitutions(context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 668, in __expand_substitutions
cmd = [perform_substitutions(context, x) for x in self.__cmd]
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 668, in
cmd = [perform_substitutions(context, x) for x in self.__cmd]
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/perform_substitutions_impl.py", line 26, in perform_substitutions
return ''.join([context.perform_substitution(sub) for sub in subs])
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/perform_substitutions_impl.py", line 26, in
return ''.join([context.perform_substitution(sub) for sub in subs])
File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_context.py", line 232, in perform_substitution
return substitution.perform(self)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch_ros/substitutions/executable_in_package.py", line 76, in perform
package_prefix = super().perform(context)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch_ros/substitutions/find_package.py", line 79, in perform
result = self.find(package)
File "/opt/ros/foxy/lib/python3.8/site-packages/launch_ros/substitutions/find_package.py", line 96, in find
return get_package_prefix(package_name)
File "/opt/ros/foxy/lib/python3.8/site-packages/ament_index_python/packages.py", line 52, in get_package_prefix
raise PackageNotFoundError(
ament_index_python.packages.PackageNotFoundError: "package 'yolov8_ros' not found, searching: ['/opt/ros/foxy']"

Could you please help with this? At the same time when using the "ros2 launch yolov8_bringup yolov8.launch.py" statement did not produce any output except for an error (Package 'yolov8_bringup' not found: "package 'yolov8_bringup' not found, searching: ['/opt/ros/foxy']")

ZeroDivision error

Hi,
I'm trying to use Webots camera with ROS2 and yolov8.
I created node as a Webots controller, taking image and publish it in camera/rgb/image
img = Image() img.height = self.__cam.getHeight() img.width = self.__cam.getWidth() img.encoding = 'bgr8' img.data = self.__cam.getImage() # print() self.image_pub.publish(img)
But when I launch yolov8 node it gives me zero division error. When I commented publishing line (so camera/rgb/image_raw is empty), I have no errors so its probably issues with image I take, but I dont get it

[yolov8_node-1] File "/home/geraltrivskiy/.local/lib/python3.10/site-packages/ultralytics/data/augment.py", line 668, in __call__ [yolov8_node-1] r = min(new_shape[0] / shape[0], new_shape[1] / shape[1]) [yolov8_node-1] ZeroDivisionError: division by zero

Is there any problem if I run this code on a jetson xavier?

Hello. Thanks to the code you posted, I succeeded in detecting real-time objects with a webcam in a local environment. Using this code, I also tried to detect real-time objects with a real sense camera in Jetson Xavier. However, the camera image is experiencing a problem that it passes through the YOLO model well and does not detect any objects. (The "results" value is good without problems, but the "results[0].boxes" value is empty)

Is there any solution to this problem?

I'm using the ROS2 foxy version, so I installed the vision_msgs as a foxy version and changed the parameters to fit. (I succeeded in detecting objects in the same environment locally.)

Torch and Torchvision were also installed well for the jetpack version that is being used.

I would appreciate your advice if you know anything.

How can I get image? part2

Hi,

I have a similar problem. I have the input_image_topic remapped to /camera/color/image_raw. Ros topic list shows 1 publisher and 3 subscribers. I have attempted to visualise with rviz2. The /yolo/dbg_image streams fine, but with no detections. Changing to /yolo/detections shows a 'No Image' error.

I'm assuming that upon launching within terminal with 'ros2 launch yolov8_bringup yolov8.launch.py model:=best4.pt input_image_topic:=/camera/color/image_raw' the stream should start up by itself with the bounding box detections, but I only see 'Model summary (fused): 168 layers , 1125971 parameters, 0 gradients, 28.4 GFLOPS'

Device: Nvidia Jetson Orin Nano
Camera: Realsense D455
ROS-distro: Foxy
model: custom trained (best.pt)
input_image_topic: /camera/color/image_raw

How can I get the image?

Thanks

No image in Rviz2 using realsense camera!

Hello,

I've tried to launch 'yolov8_bringup/yolov8.launch.py' but I have encountered an issue on Rviz2 yolo topic vizualization, see image below:

There is not any camera image from topic /yolo/dbg_image...You can see below a screenshoot with log terminal information:

This is a list with all my ROS2-humble available topics:

And some info's about my Jetson AGX Xavier that I'm using, with Jetpack 5.1.3:

I'm using a Intel RealSense camera, model D415.

I don't know what I'm missing? Do you have any suggestions?

detect_3d_node.py: on_detections not being called!

Hello,

I am debugging the repo on my Jetson Orin (with ROS2 Foxy) and I can't get the 3d detections.
I debugged the nodes and it seems the callback on_detections() is not being called, even though the topics corresponding to all the 3 subscribers are being published (depth_image, depth_info, and detections).
What might be the reasons?

Here is the command I'm running:

ros2 launch yolov8_bringup yolov8_3d.launch.py input_image_topic:=/camera/color/image_raw input_depth_topic:=/camera/depth/image_rect_raw input_camera_info_topic:=/camera/color/camera_info input_depth_info_topic:=/camera/depth/camera_info model:=yolov8n-pose.pt threshold:=0.2

Many thanks.

Suggestion regarding further steps for speed estimation

I have tried to setup the stack with Intel realsense D435 and I have checked that topics are susbcribed by the yolo nodes and I can view the camera rgb feed and depth feed in rviz2.

What I want now, is after having the 3d data about the objects, I want to do speed estimation of the detected and tracked objects (let's say person's speed). For this, using yolo default trackers byte track/botosrt, they don't assign re-d to the object which leaves the frame. I want to take some suggestion.

Suppose, if the object is getting detected and tracked with track_id = 1 and it leaves the frame and comes back, tracker will agin assign a new id but will this affect the speed estimation because as soon as the object reappears the speed estimation will start from fresh but will report the correct speed data of that object.

How to create and train the dataset

Hello, may I ask how to use this model to train the dataset you have created? What should be the format and production of the RGB dataset and the RGBD dataset? Because what I want to do is to create a dataset of transparent objects in the laboratory. Thank you!

ros2 version

I had a question about the version of ROS2 which is compatible with this repository.
can I use foxy and humble for example ?
doesn't matter which version is installed?

mgonzs13 / yolov8_ros Goto Github PK

yolov8_ros's Introduction

yolov8_ros

Installation

Usage

YOLOv8 / YOLOv9

Topics

Parameters

YOLOv8 3D

Topics

Parameters

Lifecycle nodes

Demos

Object Detection

Instance Segmentation

Human Pose

3D Object Detection

3D Object Detection (Using Instance Segmentation Masks)

3D Human Pose

yolov8_ros's People

Contributors

Stargazers

Watchers

Forkers

yolov8_ros's Issues

!! self.initialize_options()

Recommend Projects

Recommend Topics

Recommend Org

!!
self.initialize_options()