The YOLOv8 segmentation model with batching option doesn't run on the GPU ? about ultralytics HOT 3 CLOSED

Hello! It looks like you're encountering an issue with running the YOLOv8 segmentation model with batching on a GPU using ONNX Runtime. The error message suggests that some operations are being assigned to the CPU instead of the GPU, which might be affecting performance.

Here are a couple of suggestions to potentially resolve this issue:

1. Ensure GPU Support: Make sure that your ONNX Runtime installation supports GPU execution. You might need to install the GPU-specific package for ONNX Runtime if not already done.

2. Execution Providers: In your script, you're specifying "webgpu" and "cpu" as execution providers. If you're aiming to run the model on an NVIDIA GPU, you should use "cuda" instead of "webgpu". Modify your script as follows:

   let model = await ort.InferenceSession.create(modelName, { executionProviders: ["cuda","cpu"] });

3. Verbose Logging: As the warning suggests, enabling verbose logging might provide more insights into why certain nodes are not being assigned to the GPU. This can help in diagnosing the issue further.

4. Dynamic Batching: You've mentioned using dynamic=True during export, which is great for flexibility in input sizes but can sometimes complicate execution provider optimizations. Ensure that your model and the ONNX Runtime version are compatible with dynamic batching.

If these steps don't resolve the issue, consider providing more details or the verbose logs as suggested by the error message. This could help in pinpointing the exact problem. Hope this helps! 😊

from ultralytics.

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

from ultralytics.

webgpu to cuda ??

no it under web env.

i met same problem, yolo v8-pose pt model to onnx cannot running under onnx/webgpu

from ultralytics.