Tensors misshaped when loading yolo8X.engine files about ultralytics HOT 6 OPEN

👋 Hello @MrYanMYN, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

from ultralytics.

Hello @MrYanMYN,

Thank you for reporting this issue and providing detailed information. It looks like you're encountering an error when loading and using the TensorRT version of the YOLOv8 model on your Jetson Orin AGX.

To help us investigate this further, could you please confirm the following:

1. Reproducible Code Example: The provided code snippet is helpful, but we need a minimal reproducible example that we can run to replicate the issue. Please refer to our Minimum Reproducible Example guide for more details on how to structure this.

2. Package Versions: Ensure you are using the latest versions of torch and ultralytics. You can update your packages using:

   pip install --upgrade torch ultralytics

3. Model Export and Inference: Verify that the model export and inference steps are correctly implemented. Here is a concise example for exporting and loading a TensorRT model:

   from ultralytics import YOLO
   
   # Load the YOLOv8 model
   model = YOLO("yolov8n.pt")
   
   # Export the model to TensorRT format
   model.export(format="engine")  # creates 'yolov8n.engine'
   
   # Load the exported TensorRT model
   tensorrt_model = YOLO("yolov8n.engine")
   
   # Run inference
   results = tensorrt_model("https://ultralytics.com/images/bus.jpg")

If the issue persists after ensuring the above steps, please provide the updated code and any additional error messages you encounter. This will help us diagnose and resolve the problem more effectively.

Looking forward to your response!

from ultralytics.

So the minimal amount of code for reproduction of the issue is just the normal snippet your provided for export and inference.

For some reason even when running it purely alone I get the same error

from ultralytics.

Hello @MrYanMYN,

Thank you for your prompt response and for confirming that the issue persists with the minimal code snippet provided.

To help us investigate further, could you please ensure the following:

1. Minimum Reproducible Example: Please provide a minimal reproducible example that we can run to replicate the issue. This is crucial for us to diagnose the problem effectively. You can refer to our Minimum Reproducible Example guide for more details.

2. Package Versions: Make sure you are using the latest versions of torch and ultralytics. You can update your packages using:

   pip install --upgrade torch ultralytics

3. Model Export and Inference: Verify that the model export and inference steps are correctly implemented. Here is a concise example for exporting and loading a TensorRT model:

   from ultralytics import YOLO
   
   # Load the YOLOv8 model
   model = YOLO("yolov8n.pt")
   
   # Export the model to TensorRT format
   model.export(format="engine")  # creates 'yolov8n.engine'
   
   # Load the exported TensorRT model
   tensorrt_model = YOLO("yolov8n.engine")
   
   # Run inference
   results = tensorrt_model("https://ultralytics.com/images/bus.jpg")

If the issue persists after ensuring the above steps, please provide the updated code and any additional error messages you encounter. This will help us diagnose and resolve the problem more effectively.

Looking forward to your response! 😊

from ultralytics.

Hey , as I said. I am running the most up to date version of torch and ultralytics while using the provided minimum viable code on a Jetson Orin AGX.

from ultralytics.

Hello @MrYanMYN,

Thank you for confirming that you're using the latest versions of torch and ultralytics and for providing the minimal code example. Given that the issue persists, let's delve a bit deeper.

It appears that the error might be related to the TensorRT model handling on the Jetson Orin AGX. Here are a few steps to help troubleshoot and potentially resolve the issue:

1. TensorRT Version: Ensure that the TensorRT version installed on your Jetson Orin AGX is compatible with the versions of torch and ultralytics you are using. Sometimes, mismatched versions can cause unexpected behavior.

2. Model Export Verification: Double-check the export process to ensure the model is correctly exported to TensorRT format. You can try running the export and inference on a different machine with an NVIDIA GPU to see if the issue is specific to the Jetson Orin AGX.

3. Debugging Tensor Shapes: The error message indicates a tensor shape mismatch. You can add some debug prints to check the shapes of the tensors at different stages of the inference process. This might help pinpoint where the shape mismatch occurs.

Here's an example of how you might add debug prints:

from ultralytics import YOLO

# Load the YOLOv8 model
model = YOLO("yolov8n.pt")

# Export the model to TensorRT format
model.export(format="engine")  # creates 'yolov8n.engine'

# Load the exported TensorRT model
tensorrt_model = YOLO("yolov8n.engine")

# Run inference with debug prints
image = "https://ultralytics.com/images/bus.jpg"
results = tensorrt_model(image)
print("Inference results:", results)

4. Ultralytics Documentation: For additional guidance, you can refer to the Ultralytics TensorRT Integration Guide.

If the issue persists, please provide any additional error messages or observations you encounter. This will help us assist you more effectively.

Thank you for your patience and cooperation! 😊

from ultralytics.