TensorFlow SavedModel: export failure ❌ 1.2s: permute(sparse_coo) about ultralytics HOT 5 CLOSED

mmr689 commented on June 25, 2024

TensorFlow SavedModel: export failure ❌ 1.2s: permute(sparse_coo)

from ultralytics.

Comments (5)

glenn-jocher commented on June 25, 2024 1

The error you are encountering when trying to export your YOLOv8 model to INT8 format is related to the calibration images used for quantization. The error message indicates that there are not enough images for proper calibration and that there is a mismatch in the dimensions of the tensor during the permutation step.

Here are steps you can follow to resolve this issue:

Provide Adequate Calibration Data:
The warning message suggests that more than 300 images are recommended for INT8 calibration, but only 4 images were found. Make sure you provide a dataset with a sufficient number of images for calibration.

You can specify the calibration dataset using the data argument in the model.export() function. Ensure this dataset has enough images and is correctly formatted.
```
model.export(format='tflite', int8=True, data='path/to/your/calibration_dataset.yaml')
```
Verify the Calibration Dataset:
Ensure that the calibration dataset specified in the YAML file is correctly formatted and accessible. The YAML file should contain the paths to the images and annotations.
Check Tensor Dimensions:
The error permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions suggests a mismatch in tensor dimensions. Ensure that your calibration images and labels are in the correct format and dimensions expected by the YOLOv8 model.
Update Ultralytics Package:
Make sure you are using the latest version of the Ultralytics package, as there may have been bug fixes or improvements related to the export functionality.
```
pip install --upgrade ultralytics
```
Debug Tensor Permutation:
If the problem persists, you may need to debug the tensor permutation step in the export code. Check the shape and dimensions of the tensors being permuted and ensure they match the expected format.

You can add some print statements in the exporter.py file to inspect the tensor shapes before the permutation step:
```
# Add these lines before the permute step in exporter.py
print("Tensor shape before permute:", batch["img"].shape)
```

By following these steps, you should be able to resolve the issues related to exporting your model to INT8 format. If the problem persists, providing additional details about your dataset and calibration process will help in diagnosing the issue further.

from ultralytics.

mmr689 commented on June 25, 2024

Hi Glenn thanks for answering.

Two questions from your answer.

Do you have a guide/example of how to introduce the calibration images? What format needs? File structure? Right now I did a YAML like this:

yaml_content = {
    'train': "full_path/images",
    'val': "full_path/images",
    'names': {
        0: "Object"
    }
}

The Debug Tensor Permutation print shows:

Tensor shape before permute: 0 torch.Size([2, 3, 640, 640])

In addition if you !pip install ultralytics in google collab and run your export example, same error appears.

!pip install ultralytics

from ultralytics import YOLO

model = YOLO("yolov8n.pt")
model.export(format="tflite", int8=True)

Error:

Downloading https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov8n.pt to 'yolov8n.pt'...
100%|██████████| 6.23M/6.23M [00:00<00:00, 77.1MB/s]
Ultralytics YOLOv8.2.20 🚀 Python-3.10.12 torch-2.3.0+cu121 CPU (Intel Xeon 2.20GHz)
WARNING ⚠️ INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'.
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs

PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)
requirements: Ultralytics requirements ['onnx>=1.12.0', 'onnx2tf>=1.15.4,<=1.17.5', 'sng4onnx>=1.0.1', 'onnxsim>=0.4.33', 'onnx_graphsurgeon>=0.3.26', 'tflite_support', 'onnxruntime'] not found, attempting AutoUpdate...
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com/
Collecting onnx>=1.12.0
  Downloading onnx-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.9/15.9 MB 48.7 MB/s eta 0:00:00
Collecting onnx2tf<=1.17.5,>=1.15.4
  Downloading onnx2tf-1.17.5-py3-none-any.whl (400 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 400.4/400.4 kB 276.3 MB/s eta 0:00:00
Collecting sng4onnx>=1.0.1
  Downloading sng4onnx-1.0.4-py3-none-any.whl (5.9 kB)
Collecting onnxsim>=0.4.33
  Downloading onnxsim-0.4.36-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 94.7 MB/s eta 0:00:00
Collecting onnx_graphsurgeon>=0.3.26
  Downloading onnx_graphsurgeon-0.5.2-py2.py3-none-any.whl (56 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.4/56.4 kB 168.6 MB/s eta 0:00:00
Collecting tflite_support
  Downloading tflite_support-0.4.4-cp310-cp310-manylinux2014_x86_64.whl (60.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 MB 111.6 MB/s eta 0:00:00
Collecting onnxruntime
  Downloading onnxruntime-1.18.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 116.6 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.12.0) (1.25.2)
Requirement already satisfied: protobuf>=3.20.2 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.12.0) (3.20.3)
Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from onnxsim>=0.4.33) (13.7.1)
Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tflite_support) (1.4.0)
Requirement already satisfied: flatbuffers>=2.0 in /usr/local/lib/python3.10/dist-packages (from tflite_support) (24.3.25)
Collecting sounddevice>=0.4.4 (from tflite_support)
  Downloading sounddevice-0.4.6-py3-none-any.whl (31 kB)
Collecting pybind11>=2.6.0 (from tflite_support)
  Downloading pybind11-2.12.0-py3-none-any.whl (234 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 235.0/235.0 kB 201.2 MB/s eta 0:00:00
Collecting coloredlogs (from onnxruntime)
  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 112.9 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from onnxruntime) (24.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime) (1.12)
Requirement already satisfied: CFFI>=1.0 in /usr/local/lib/python3.10/dist-packages (from sounddevice>=0.4.4->tflite_support) (1.16.0)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime)
  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 242.6 MB/s eta 0:00:00
Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->onnxsim>=0.4.33) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->onnxsim>=0.4.33) (2.16.1)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime) (1.3.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from CFFI>=1.0->sounddevice>=0.4.4->tflite_support) (2.22)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich->onnxsim>=0.4.33) (0.1.2)
Installing collected packages: sng4onnx, pybind11, onnx2tf, onnx, humanfriendly, sounddevice, onnx_graphsurgeon, coloredlogs, tflite_support, onnxsim, onnxruntime
Successfully installed coloredlogs-15.0.1 humanfriendly-10.0 onnx-1.16.1 onnx2tf-1.17.5 onnx_graphsurgeon-0.5.2 onnxruntime-1.18.0 onnxsim-0.4.36 pybind11-2.12.0 sng4onnx-1.0.4 sounddevice-0.4.6 tflite_support-0.4.4

requirements: AutoUpdate success ✅ 22.4s, installed 7 packages: ['onnx>=1.12.0', 'onnx2tf>=1.15.4,<=1.17.5', 'sng4onnx>=1.0.1', 'onnxsim>=0.4.33', 'onnx_graphsurgeon>=0.3.26', 'tflite_support', 'onnxruntime']
requirements: ⚠️ Restart runtime or rerun command for updates to take effect


TensorFlow SavedModel: starting export with tensorflow 2.15.0...
WARNING ⚠️ tensorflow<=2.13.1 is required, but tensorflow==2.15.0 is currently installed https://github.com/ultralytics/ultralytics/issues/5161
Downloading https://github.com/ultralytics/assets/releases/download/v8.2.0/calibration_image_sample_data_20x128x128x3_float32.npy.zip to 'calibration_image_sample_data_20x128x128x3_float32.npy.zip'...
100%|██████████| 1.11M/1.11M [00:00<00:00, 19.3MB/s]
Unzipping calibration_image_sample_data_20x128x128x3_float32.npy.zip to /content/calibration_image_sample_data_20x128x128x3_float32.npy...: 100%|██████████| 1/1 [00:00<00:00, 34.65file/s]

ONNX: starting export with onnx 1.16.1 opset 17...
ONNX: simplifying with onnxsim 0.4.36...
ONNX: export success ✅ 3.6s, saved as 'yolov8n.onnx' (12.3 MB)
TensorFlow SavedModel: collecting INT8 calibration images from 'data=coco8.yaml'

Dataset 'coco8.yaml' images not found ⚠️, missing path '/content/datasets/coco8/images/val'
Downloading https://ultralytics.com/assets/coco8.zip to '/content/datasets/coco8.zip'...
100%|██████████| 433k/433k [00:00<00:00, 9.75MB/s]
Unzipping /content/datasets/coco8.zip to /content/datasets/coco8...: 100%|██████████| 25/25 [00:00<00:00, 2407.53file/s]Dataset download success ✅ (0.9s), saved to /content/datasets


Downloading https://ultralytics.com/assets/Arial.ttf to '/root/.config/Ultralytics/Arial.ttf'...
100%|██████████| 755k/755k [00:00<00:00, 14.1MB/s]
Scanning /content/datasets/coco8/labels/val... 4 images, 0 backgrounds, 0 corrupt: 100%|██████████| 4/4 [00:00<00:00, 160.31it/s]New cache created: /content/datasets/coco8/labels/val.cache
TensorFlow SavedModel: WARNING ⚠️ >300 images recommended for INT8 calibration, found 4 images.

TensorFlow SavedModel: export failure ❌ 35.0s: permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 4 is not equal to len(dims) = 3
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[<ipython-input-2-ed7a5d72c261>](https://localhost:8080/#) in <cell line: 4>()
      2 
      3 model = YOLO("yolov8n.pt")
----> 4 model.export(format="tflite", int8=True)

5 frames
[/usr/local/lib/python3.10/dist-packages/ultralytics/engine/exporter.py](https://localhost:8080/#) in export_saved_model(self, prefix)
    867                     if i >= 100:  # maximum number of calibration images
    868                         break
--> 869                     im = batch["img"].permute(1, 2, 0)[None]  # list to nparray, CHW to BHWC
    870                     images.append(im)
    871                 f.mkdir()

RuntimeError: permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 4 is not equal to len(dims) = 3

from ultralytics.

glenn-jocher commented on June 25, 2024

Hi there! Thanks for reaching out with your questions. Let's address them:

Calibration Images Format: Your YAML format looks mostly correct. However, ensure that under each key (train, val), you specify the full path to the images and their corresponding labels. Here's a refined example:
```
train: path/to/train/images
val: path/to/val/images
names:
  0: "Object"
```
Make sure the paths are accessible and the images are properly labeled.
Debug Tensor Permutation: The tensor shape before permutation ([2, 3, 640, 640]) indicates a batch size of 2, which should be fine. The error you're encountering in Google Colab suggests there might be an issue with the tensor operations specific to the INT8 conversion. This could be due to version compatibility issues between PyTorch, TensorFlow, and the additional libraries used during the export process.

As for the error in Google Colab, it seems like there might be a compatibility issue or a specific bug with the INT8 conversion process in the Ultralytics export function. I recommend ensuring all dependencies are up to date and compatible with each other. If the issue persists, consider raising an issue on the Ultralytics GitHub page with the details of the error and your environment setup.

Hope this helps! Let me know if you have any more questions. 😊

from ultralytics.

mmr689 commented on June 25, 2024

Thanks for answering, finally I downgrade to 8.1.47.

from ultralytics.

glenn-jocher commented on June 25, 2024

Hi there! Great to hear that you found a solution by downgrading to version 8.1.47. If you encounter any further issues or have more questions, feel free to reach out. Happy coding! 😊

from ultralytics.

TensorFlow SavedModel: export failure ❌ 1.2s: permute(sparse_coo) about ultralytics HOT 5 CLOSED

Comments (5)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent