Comments (5)
The error you are encountering when trying to export your YOLOv8 model to INT8 format is related to the calibration images used for quantization. The error message indicates that there are not enough images for proper calibration and that there is a mismatch in the dimensions of the tensor during the permutation step.
Here are steps you can follow to resolve this issue:
-
Provide Adequate Calibration Data:
The warning message suggests that more than 300 images are recommended for INT8 calibration, but only 4 images were found. Make sure you provide a dataset with a sufficient number of images for calibration.You can specify the calibration dataset using the
data
argument in themodel.export()
function. Ensure this dataset has enough images and is correctly formatted.model.export(format='tflite', int8=True, data='path/to/your/calibration_dataset.yaml')
-
Verify the Calibration Dataset:
Ensure that the calibration dataset specified in the YAML file is correctly formatted and accessible. The YAML file should contain the paths to the images and annotations. -
Check Tensor Dimensions:
The errorpermute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions
suggests a mismatch in tensor dimensions. Ensure that your calibration images and labels are in the correct format and dimensions expected by the YOLOv8 model. -
Update Ultralytics Package:
Make sure you are using the latest version of the Ultralytics package, as there may have been bug fixes or improvements related to the export functionality.pip install --upgrade ultralytics
-
Debug Tensor Permutation:
If the problem persists, you may need to debug the tensor permutation step in the export code. Check the shape and dimensions of the tensors being permuted and ensure they match the expected format.You can add some print statements in the
exporter.py
file to inspect the tensor shapes before the permutation step:# Add these lines before the permute step in exporter.py print("Tensor shape before permute:", batch["img"].shape)
By following these steps, you should be able to resolve the issues related to exporting your model to INT8 format. If the problem persists, providing additional details about your dataset and calibration process will help in diagnosing the issue further.
from ultralytics.
Hi Glenn thanks for answering.
Two questions from your answer.
- Do you have a guide/example of how to introduce the calibration images? What format needs? File structure? Right now I did a YAML like this:
yaml_content = {
'train': "full_path/images",
'val': "full_path/images",
'names': {
0: "Object"
}
}
- The Debug Tensor Permutation print shows:
Tensor shape before permute: 0 torch.Size([2, 3, 640, 640])
In addition if you !pip install ultralytics
in google collab and run your export example, same error appears.
!pip install ultralytics
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
model.export(format="tflite", int8=True)
Error:
Downloading https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov8n.pt to 'yolov8n.pt'...
100%|██████████| 6.23M/6.23M [00:00<00:00, 77.1MB/s]
Ultralytics YOLOv8.2.20 🚀 Python-3.10.12 torch-2.3.0+cu121 CPU (Intel Xeon 2.20GHz)
WARNING ⚠️ INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'.
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs
PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)
requirements: Ultralytics requirements ['onnx>=1.12.0', 'onnx2tf>=1.15.4,<=1.17.5', 'sng4onnx>=1.0.1', 'onnxsim>=0.4.33', 'onnx_graphsurgeon>=0.3.26', 'tflite_support', 'onnxruntime'] not found, attempting AutoUpdate...
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com/
Collecting onnx>=1.12.0
Downloading onnx-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.9/15.9 MB 48.7 MB/s eta 0:00:00
Collecting onnx2tf<=1.17.5,>=1.15.4
Downloading onnx2tf-1.17.5-py3-none-any.whl (400 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 400.4/400.4 kB 276.3 MB/s eta 0:00:00
Collecting sng4onnx>=1.0.1
Downloading sng4onnx-1.0.4-py3-none-any.whl (5.9 kB)
Collecting onnxsim>=0.4.33
Downloading onnxsim-0.4.36-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 94.7 MB/s eta 0:00:00
Collecting onnx_graphsurgeon>=0.3.26
Downloading onnx_graphsurgeon-0.5.2-py2.py3-none-any.whl (56 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.4/56.4 kB 168.6 MB/s eta 0:00:00
Collecting tflite_support
Downloading tflite_support-0.4.4-cp310-cp310-manylinux2014_x86_64.whl (60.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 MB 111.6 MB/s eta 0:00:00
Collecting onnxruntime
Downloading onnxruntime-1.18.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 116.6 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.12.0) (1.25.2)
Requirement already satisfied: protobuf>=3.20.2 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.12.0) (3.20.3)
Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from onnxsim>=0.4.33) (13.7.1)
Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tflite_support) (1.4.0)
Requirement already satisfied: flatbuffers>=2.0 in /usr/local/lib/python3.10/dist-packages (from tflite_support) (24.3.25)
Collecting sounddevice>=0.4.4 (from tflite_support)
Downloading sounddevice-0.4.6-py3-none-any.whl (31 kB)
Collecting pybind11>=2.6.0 (from tflite_support)
Downloading pybind11-2.12.0-py3-none-any.whl (234 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 235.0/235.0 kB 201.2 MB/s eta 0:00:00
Collecting coloredlogs (from onnxruntime)
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 112.9 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from onnxruntime) (24.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime) (1.12)
Requirement already satisfied: CFFI>=1.0 in /usr/local/lib/python3.10/dist-packages (from sounddevice>=0.4.4->tflite_support) (1.16.0)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime)
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 242.6 MB/s eta 0:00:00
Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->onnxsim>=0.4.33) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->onnxsim>=0.4.33) (2.16.1)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime) (1.3.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from CFFI>=1.0->sounddevice>=0.4.4->tflite_support) (2.22)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich->onnxsim>=0.4.33) (0.1.2)
Installing collected packages: sng4onnx, pybind11, onnx2tf, onnx, humanfriendly, sounddevice, onnx_graphsurgeon, coloredlogs, tflite_support, onnxsim, onnxruntime
Successfully installed coloredlogs-15.0.1 humanfriendly-10.0 onnx-1.16.1 onnx2tf-1.17.5 onnx_graphsurgeon-0.5.2 onnxruntime-1.18.0 onnxsim-0.4.36 pybind11-2.12.0 sng4onnx-1.0.4 sounddevice-0.4.6 tflite_support-0.4.4
requirements: AutoUpdate success ✅ 22.4s, installed 7 packages: ['onnx>=1.12.0', 'onnx2tf>=1.15.4,<=1.17.5', 'sng4onnx>=1.0.1', 'onnxsim>=0.4.33', 'onnx_graphsurgeon>=0.3.26', 'tflite_support', 'onnxruntime']
requirements: ⚠️ Restart runtime or rerun command for updates to take effect
TensorFlow SavedModel: starting export with tensorflow 2.15.0...
WARNING ⚠️ tensorflow<=2.13.1 is required, but tensorflow==2.15.0 is currently installed https://github.com/ultralytics/ultralytics/issues/5161
Downloading https://github.com/ultralytics/assets/releases/download/v8.2.0/calibration_image_sample_data_20x128x128x3_float32.npy.zip to 'calibration_image_sample_data_20x128x128x3_float32.npy.zip'...
100%|██████████| 1.11M/1.11M [00:00<00:00, 19.3MB/s]
Unzipping calibration_image_sample_data_20x128x128x3_float32.npy.zip to /content/calibration_image_sample_data_20x128x128x3_float32.npy...: 100%|██████████| 1/1 [00:00<00:00, 34.65file/s]
ONNX: starting export with onnx 1.16.1 opset 17...
ONNX: simplifying with onnxsim 0.4.36...
ONNX: export success ✅ 3.6s, saved as 'yolov8n.onnx' (12.3 MB)
TensorFlow SavedModel: collecting INT8 calibration images from 'data=coco8.yaml'
Dataset 'coco8.yaml' images not found ⚠️, missing path '/content/datasets/coco8/images/val'
Downloading https://ultralytics.com/assets/coco8.zip to '/content/datasets/coco8.zip'...
100%|██████████| 433k/433k [00:00<00:00, 9.75MB/s]
Unzipping /content/datasets/coco8.zip to /content/datasets/coco8...: 100%|██████████| 25/25 [00:00<00:00, 2407.53file/s]Dataset download success ✅ (0.9s), saved to /content/datasets
Downloading https://ultralytics.com/assets/Arial.ttf to '/root/.config/Ultralytics/Arial.ttf'...
100%|██████████| 755k/755k [00:00<00:00, 14.1MB/s]
Scanning /content/datasets/coco8/labels/val... 4 images, 0 backgrounds, 0 corrupt: 100%|██████████| 4/4 [00:00<00:00, 160.31it/s]New cache created: /content/datasets/coco8/labels/val.cache
TensorFlow SavedModel: WARNING ⚠️ >300 images recommended for INT8 calibration, found 4 images.
TensorFlow SavedModel: export failure ❌ 35.0s: permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 4 is not equal to len(dims) = 3
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
[<ipython-input-2-ed7a5d72c261>](https://localhost:8080/#) in <cell line: 4>()
2
3 model = YOLO("yolov8n.pt")
----> 4 model.export(format="tflite", int8=True)
5 frames
[/usr/local/lib/python3.10/dist-packages/ultralytics/engine/exporter.py](https://localhost:8080/#) in export_saved_model(self, prefix)
867 if i >= 100: # maximum number of calibration images
868 break
--> 869 im = batch["img"].permute(1, 2, 0)[None] # list to nparray, CHW to BHWC
870 images.append(im)
871 f.mkdir()
RuntimeError: permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 4 is not equal to len(dims) = 3
from ultralytics.
Hi there! Thanks for reaching out with your questions. Let's address them:
-
Calibration Images Format: Your YAML format looks mostly correct. However, ensure that under each key (
train
,val
), you specify the full path to the images and their corresponding labels. Here's a refined example:train: path/to/train/images val: path/to/val/images names: 0: "Object"
Make sure the paths are accessible and the images are properly labeled.
-
Debug Tensor Permutation: The tensor shape before permutation (
[2, 3, 640, 640]
) indicates a batch size of 2, which should be fine. The error you're encountering in Google Colab suggests there might be an issue with the tensor operations specific to the INT8 conversion. This could be due to version compatibility issues between PyTorch, TensorFlow, and the additional libraries used during the export process.As for the error in Google Colab, it seems like there might be a compatibility issue or a specific bug with the INT8 conversion process in the Ultralytics export function. I recommend ensuring all dependencies are up to date and compatible with each other. If the issue persists, consider raising an issue on the Ultralytics GitHub page with the details of the error and your environment setup.
Hope this helps! Let me know if you have any more questions. 😊
from ultralytics.
Thanks for answering, finally I downgrade to 8.1.47.
from ultralytics.
Hi there! Great to hear that you found a solution by downgrading to version 8.1.47. If you encounter any further issues or have more questions, feel free to reach out. Happy coding! 😊
from ultralytics.
Related Issues (20)
- YOLOv8 is jointly trained with other models HOT 1
- Optimizer='auto' problem HOT 2
- Docker run yolov8 report error:Killed, OOM HOT 3
- Is there any other way to get faster YOLOv8n results without using GPU HOT 1
- Default training parameters for yolov8n? HOT 12
- Exporting a YOLO model fails when current directory is in a different filesystem HOT 4
- YOLOv8 resizes input images differently when training for classification? HOT 2
- FedAvg with YOLO HOT 4
- YOLOv8, v10, RT-DETR albumentation do not apply HOT 4
- How can i train better my project ? YOLOV8 HOT 11
- Codebase for running YoloV10 with ONNX HOT 6
- xywh returns wrong result while xyxy returns right result HOT 5
- Support distributed evaluation during training process
- Is there an example of yolov8n-segn Android split HOT 2
- @glenn-jocher tracker is not working for custom trained models,
- multi input video to YOLOv8 and using bytetrack.yaml return same ID to different object and keep increasing HOT 6
- The engine model RTX3060 exported by RTX4070 cannot be inferred HOT 2
- YOLO(model_yaml).load(model.pt) not work. HOT 4
- Exporting after training on YoloV10 raise a ValueError with MultiGPU HOT 7
- Yolov8 classifier training: impossible to disable some augmentation options HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ultralytics.