Unable to Export RTDETR Large Model(best.pt) to TFLite or NCNN for Raspberry Pi 4 Deployment about ultralytics HOT 5 OPEN

👋 Hello @Youssef-100, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

from ultralytics.

Hello! Thank you very much for your fast reply. Unfortunately the issue has not been resolved ,It shows the same error in both cases

from ultralytics.

Hi there! Sorry to hear the issue persists. Let's try a couple more things:

1. For TFLite, try using the dynamic=True option:

   model.export(format='tflite', dynamic=True)

2. For NCNN, ensure all dependencies are up-to-date and try reducing the input size further:

   model.export(format='ncnn', half=True, imgsz=224)

If these don't work, please share more details or logs. We're here to help! 😊

from ultralytics.

Hi Glenn! I am really sorry to bother you!
The NCNN shows the error below:
(force batch axis 233 for operand 241
force batch axis 233 for operand 242
binaryop tensor0 with rank 6 is not supported yet!
binaryop tensor1 with rank 6 is not supported yet!
binaryop tensor0 with rank 6 is not supported yet!
munmap_chunk(): invalid pointer
NCNN: export failure ❌ 62.9s
subprocess.CalledProcessError: Command '['/home/youssef/project/project/lib/python3.11/site-packages/ultralytics/pnnx', 'best.torchscript', 'ncnnparam=best_ncnn_model/model.ncnn.param', 'ncnnbin=best_ncnn_model/model.ncnn.bin', 'ncnnpy=best_ncnn_model/model_ncnn.py', 'pnnxparam=best_ncnn_model/model.pnnx.param', 'pnnxbin=best_ncnn_model/model.pnnx.bin', 'pnnxpy=best_ncnn_model/model_pnnx.py', 'pnnxonnx=best_ncnn_model/model.pnnx.onnx', 'fp16=1', 'device=cpu', 'inputshape="[1, 3, 224, 224]"']' died with <Signals.SIGABRT: 6>.)

While for the TFLITE ERROR:
ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces keras.layers and keras.operations). You are likely doing something like:
input_onnx_file_path: best.onnx
ERROR: onnx_op_name: /model.0/stem1/conv/Conv
ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.

Thank You so much for your help!

from ultralytics.

Hi there!

No bother at all! 😊 Thank you for providing the detailed error messages.

For the NCNN export issue, it seems like the model's tensor rank is causing problems. This might be due to the complexity of the RTDETR model. Unfortunately, NCNN might not fully support all operations used in this model.

For the TFLite export, the error suggests a problem with the ONNX to TensorFlow conversion. You might want to try the suggested parameter replacements or static shape rewrites as mentioned in the error message. Additionally, using the -kt or -kat options could help if the input OP has an irregular channel arrangement.

If these solutions don't work, consider simplifying the model or using a different model architecture that is known to be compatible with TFLite and NCNN.

Feel free to reach out if you need further assistance!

from ultralytics.