mpolaris / onnx2tflite Goto Github PK
View Code? Open in Web Editor NEWTool for onnx->keras or onnx->tflite. If tool is useful for you, please star it.
License: Apache License 2.0
Tool for onnx->keras or onnx->tflite. If tool is useful for you, please star it.
License: Apache License 2.0
Hi,
In the transposed convolution source code, the output shape is not the same as Pytorch, for same parameters. In order to solve this, I have moved the padding after self.conv call. This is for a tranposed convolution with kernel size [4, 4] and strides [2, 2]. Can you please recheck your conversion code on other tranposed convolutions?
https://github.com/MPolaris/onnx2tflite/blob/main/layers/conv_layers.py#L37
HI,MPolaris,我观察到保存的keras模型所占空间相比原始onnx大了不少,例如mobilenet_v2的h5模型接近88多M,原始模型大约13.9M,原因定位到是 Conv、Dense、Conv2DTranspose等 在导入权重时使用的是
kernel_initializer=keras.initializers.Constant(weights),
bias_initializer='zeros' if bias is None else keras.initializers.Constant(bias),
这种方式,这样使用会造成大量字符保存在h5模型中,占用空间大,netron打开也较慢
建议都采用https://github.com/MPolaris/onnx2tflite/blob/main/layers/conv_layers.py#L166 深度可分离卷积实现方式:
weights=[weights] 有bias则 weights=[weights, bias]
进行权重导入,感觉这样会更好些,实测修改后确实h5可以和原始onnx基本一样大,以上只是一定小小的建议
I'm trying to convert a torchvision faster_rcnn model from pytorch to tflite, and found two onnx_converter crashing issues.
With onnx opset_version 11 or 12, onnx_converter hits a tf.math.subtract error, [1024,1024,3] doesn't fit [3,1,1], crash.
With onnx opset_version 13, there's a KeyError: 'split', crash.
The code is pretty much the onnx2tflite demo code with the model switched over.
Thanks in advance for looking into this!
Checking 0/1...
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
Traceback (most recent call last):
File "D:\python\onnx2tflite\converter.py", line 108, in
run()
File "D:\python\onnx2tflite\converter.py", line 92, in run
onnx_converter(
File "D:\python\onnx2tflite\converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
File "D:\python\onnx2tflite\utils\builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "D:\python\onnx2tflite\layers\deformation_layers.py", line 97, in call
inputs = tf.reshape(inputs, shape=self.out_shape)
File "D:\python\onnx2tflite\venv\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "D:\python\onnx2tflite\venv\lib\site-packages\keras\src\layers\core\tf_op_layer.py", line 119, in handle
return TFOpLambda(op)(*args, **kwargs)
File "D:\python\onnx2tflite\venv\lib\site-packages\keras\src\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
ValueError: Exception encountered when calling layer "tf.reshape" (type TFOpLambda).
Cannot reshape a tensor with 8388608 elements to shape [0,1,?] (0 elements) for '{{node tf.reshape/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT64](Placeholder, tf.reshape/Reshape/shape)' with input shapes: [1,32,512,512], [3] and with input tensors computed as partial shapes: input[1] = [0,1,?].
Call arguments received by layer "tf.reshape" (type TFOpLambda):
• tensor=tf.Tensor(shape=(1, 32, 512, 512), dtype=float32)
• shape=array([ 0, 1, -1], dtype=int64)
• name=None
Failed to initialize detector. Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
Hi, nice work,
is there a method to remove pad before conv?
it may be fast without padding.
Hi, thanks for the great work, really cool repository.
I have a question about the grouped convolution implementation. I've noticed in TFGroupConv
class that when group != 1 and out_channels
, the groups are explicitly splitted and later concatenated which might be less efficient in runtime. What is the reason to this approach? Is there something problematic with group-conv conversion otherwise?
env as below:
python 3.7
numpy 1.21.6
onnx 1.12.0
onnx-simplifier 0.4.19
onnxruntime 1.14.1
opencv-python 4.7.0.72
protobuf 3.19.0
pytest 7.2.2
tensorboard 2.11.2
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.11.0
tensorflow-estimator 2.11.0
tensorflow-intel 2.11.0
tensorflow-io-gcs-filesystem 0.31.0
torch 1.9.0
torchvision 0.10.0+cu102
fail to run test_deeplabv3, and got the fllowing error msg:
`
torchvison_test.py::test_deeplabv3 FAILED [100%]
torchvison_test.py:31 (test_deeplabv3)
@pytest.mark.filterwarnings('ignore::UserWarning')
@pytest.mark.filterwarnings('ignore::DeprecationWarning')
def test_deeplabv3():
model = torchvision.models.segmentation.deeplabv3_resnet50(False)
onnx_model_path = os.path.join(MODEL_ROOT, "deeplabv3_resnet50.onnx")
torch.onnx.export(model, torch.randn(1, 3, 512, 1024), onnx_model_path, opset_version=13)
error = onnx_converter(onnx_model_path, need_simplify = False, output_path = MODEL_ROOT, target_formats = ['tflite'])
torchvison_test.py:38:
converter.py:21: in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
onnx_model = ir_version: 6
producer_name: "pytorch"
producer_version: "1.9"
graph {
node {
input: "input.1"
output: "363"...im {
dim_param: "Resize600_dim_3"
}
}
}
}
}
}
opset_import {
version: 13
}
native_groupconv = False
def keras_builder(onnx_model, native_groupconv:bool=False):
conv_layers.USE_NATIVE_GROUP_CONV = native_groupconv
model_graph = onnx_model.graph
'''
init onnx model's build-in tensors
'''
onnx_weights = dict()
for initializer in model_graph.initializer:
onnx_weights[initializer.name] = numpy_helper.to_array(initializer)
'''
build input nodes
'''
tf_tensor, input_shape = {}, []
for inp in model_graph.input:
input_shape = [x.dim_value for x in inp.type.tensor_type.shape.dim]
if input_shape == []:
continue
batch_size = 1 if input_shape[0] <= 0 else input_shape[0]
input_shape = input_shape[2:] + input_shape[1:2]
tf_tensor[inp.name] = keras.Input(shape=input_shape, batch_size=batch_size)
'''
build model inline node by iterate onnx nodes.
'''
for node in model_graph.node:
op_name, node_inputs, node_outputs = node.op_type, node.input, node.output
op_attr = decode_node_attribute(node)
tf_operator = OPERATOR.get(op_name)
if tf_operator is None:
raise KeyError(f"{op_name} not implemented yet")
E KeyError: 'Shape not implemented yet'
utils\builder.py:75: KeyError
`
Traceback (most recent call last):
File "converter.py", line 108, in
run()
File "converter.py", line 92, in run
onnx_converter(
File "converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
File "/home/liusuyue/3080ti/face/onnx2tflite/utils/builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "/home/liusuyue/3080ti/face/onnx2tflite/layers/activations_layers.py", line 78, in call
return self.PRelu(inputs)
File "/home/liusuyue/anaconda3/envs/dev/lib/python3.8/site-packages/keras/engine/base_layer.py", line 976, in call
return self._functional_construction_call(inputs, args, kwargs,
File "/home/liusuyue/anaconda3/envs/dev/lib/python3.8/site-packages/keras/engine/base_layer.py", line 1114, in _functional_construction_call
outputs = self._keras_tensor_symbolic_call(
File "/home/liusuyue/anaconda3/envs/dev/lib/python3.8/site-packages/keras/engine/base_layer.py", line 848, in _keras_tensor_symbolic_call
return self._infer_output_signature(inputs, args, kwargs, input_masks)
File "/home/liusuyue/anaconda3/envs/dev/lib/python3.8/site-packages/keras/engine/base_layer.py", line 886, in _infer_output_signature
self._maybe_build(inputs)
File "/home/liusuyue/anaconda3/envs/dev/lib/python3.8/site-packages/keras/engine/base_layer.py", line 2670, in _maybe_build
self.set_weights(self._initial_weights)
File "/home/liusuyue/anaconda3/envs/dev/lib/python3.8/site-packages/keras/engine/base_layer.py", line 1799, in set_weights
raise ValueError(
ValueError: Layer weight shape (1, 1, 128) not compatible with provided weight shape (128, 1, 1)
I got this error when converting onnx model to tflite, any idea to solve this?
Traceback (most recent call last):
File "/mnt/sda/AI/onnx2tflite-main/converter.py", line 70, in
run()
File "/mnt/sda/AI/onnx2tflite-main/converter.py", line 66, in run
image_root=opt.imgroot
File "/mnt/sda/AI/onnx2tflite-main/converter.py", line 18, in onnx_converter
keras_model = keras_builder(model_proto, input_node_names, output_node_names)
File "/mnt/sda/AI/onnx2tflite-main/utils/builder.py", line 81, in keras_builder
raise KeyError(f"{op_name} not implemented yet")
KeyError: 'Shape not implemented yet'
Hello,
I have an ONNX model that needs to be converted to TFLITE, Here is the command I use:
python3 converter.py --weights "./model.onnx" --outpath "./save_path" --formats "tflite"
but the error message is as follows when converting:
Dimensions must be equal, but are 128 and 3 for '{{node tf.math.multiply_1/Mul}} = Mul[T=DT_FLOAT](Placeholder, tf.math.multiply_1/Mul/y)' with input shapes: [1,128,128,3], [3,128,128].
It seems that the input of the multiplication operator has been transformed, but the second operand has not been dimensionally transformed, resulting in inconsistent dimensions?
Looking forward to your reply.
输入模型转换失败,以下是错误提示
引入onnxsim.simplify失败
模型优化失败, 从./end2end.onnx加载
2022-10-05 15:49:45.947888: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-05 15:49:45.949148: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Transpose 操作将会回到NCHW形式进行
Reshape 操作将会回到NCHW形式进行
Transpose 操作将会回到NCHW形式进行
Reshape 操作将会回到NCHW形式进行
Traceback (most recent call last):
File "/data/jdx/code/onnx2tflite/converter.py", line 70, in
run()
File "/data/jdx/code/onnx2tflite/converter.py", line 55, in run
onnx_converter(
File "/data/jdx/code/onnx2tflite/converter.py", line 18, in onnx_converter
keras_model = keras_builder(model_proto, input_node_names, output_node_names)
File "/data/jdx/code/onnx2tflite/utils/builder.py", line 88, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "/data/jdx/code/onnx2tflite/layers/deformation_layers.py", line 61, in call
indices = tf.keras.backend.arange(self.starts, self.ends, step=self.steps)
File "/data/jdx/miniconda3/envs/onnx/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/data/jdx/miniconda3/envs/onnx/lib/python3.9/site-packages/keras/backend.py", line 3601, in arange
result = tf.range(start, limit=stop, delta=step, name='arange')
tensorflow.python.framework.errors_impl.InvalidArgumentError: Requires start <= limit when delta > 0: 0/-1 [Op:Range]
At ImageLoader class cv2.resize is used, but opencv resize will resize image not in the shape assumed. For example, if you use 144x256 shape, it will return 256x144. For this reason, it leads to errors of mismatched shapes. Need to change this method, or swap axeses
I'm trying to run yolov7 using tflite.
First I export with:
python export.py --weights ../models/yolov7/yolov7-tiny.pt --grid --simplify
Then I run
python converter.py --weights ../models/yolov7/yolov7-tiny.onnx --outpath ../models/yolov7/ --int8
Finaly I run (using https://github.com/ultralytics/yolov5/blob/master/detect.py)
python detect.py --weights ../models/yolov7/yolov7-tiny.tflite --conf 0.25 --img-size 640 --source data/images/zidane.jpg
Do you have a example detect.py that works with the converted tflite models so I can figure out what I'm doing wrong?
Hi,考虑一下把clip_by_value如果最大值是6则改成激活函数relu6感觉会更好些
I am running into this error. It would be great if somebody could help.
python3.10 converter.py --weights "./model.onnx" --formats "tflite" --int8
Checking 0/1...
WARNING:onnx_loader running::onnxsim is failed, maybe make convert fails.
Traceback (most recent call last):
File "/home/dockerpaddle/Desktop/onnx2tflite/converter.py", line 108, in
run()
File "/home/dockerpaddle/Desktop/onnx2tflite/converter.py", line 92, in run
onnx_converter(
File "/home/dockerpaddle/Desktop/onnx2tflite/converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
File "/home/dockerpaddle/Desktop/onnx2tflite/utils/builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "/home/dockerpaddle/Desktop/onnx2tflite/layers/conv_layers.py", line 57, in init
out_channel, in_channel = node_weights[node_inputs[1]].shape[:2]
KeyError: 'conv2d_0.w_0'
hi,
when I try to convert my onnx model which consists of a maxpool layer, I found the maxpool layer operates wrong, due to the "pool size" and "strides" params. Please confirm whether the following codes in Class TFAveragePool() and TFMaxPool() of "layers/common_layers.py" needs to be modified from:
self.avg_pool = keras.layers.AveragePooling2D(pool_size=node_attribute.get("kernel_shape", [2])[0],
strides=node_attribute.get("strides", [1])[0], padding=pad_mode)
self.max_pool = keras.layers.MaxPool2D(pool_size=node_attribute.get("kernel_shape", [2])[0],
strides=node_attribute.get("strides", [1])[0], padding=pad_mode)
to :
self.avg_pool = keras.layers.AveragePooling2D(pool_size=node_attribute.get("kernel_shape", [2]),
strides=node_attribute.get("strides", [1]), padding=pad_mode)
self.max_pool = keras.layers.MaxPool2D(pool_size=node_attribute.get("kernel_shape", [2])[0],
strides=node_attribute.get("strides", [1]), padding=pad_mode)
Hi,
I'm trying to convert YoloV7-tiny.onnx to keras by using cli command below
python converter.py --weights "yolov7-tiny.onnx" --formats "keras"
but raise exception:
Traceback (most recent call last):
File "converter.py", line 73, in <module>
run()
File "converter.py", line 57, in run
onnx_converter(
File "converter.py", line 19, in onnx_converter
keras_model = keras_builder(model_proto, input_node_names, output_node_names, native_groupconv)
File "/outerDisk1/onnx2tflite/utils/builder.py", line 101, in keras_builder
raise KeyError(f"{op_name} not implemented yet")
KeyError: 'NonMaxSuppression not implemented yet'
anyone know how to fix it?
i am now trying to convert yolov5-6.1's model to tflite int8 model.
I use yolov5-6.1's official method (set opset=11) to export pytorch model to onnx model
then i use your method to convert the model from onnx to tflite as:
python converter.py \
--weights .../best.onnx \
--outpath "./test01" --formats "tflite" --int8 --imgroot "../val2017/" \
--int8mean 0 0 0 --int8std 255 255 255
during the process, i got this WARNING message:
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 56). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: /tmp/tmp408fv3h7/assets
INFO:tensorflow:Assets written to: /tmp/tmp408fv3h7/assets
/home/tino/venv/onnx2tflite/lib/python3.8/site-packages/tensorflow/lite/python/convert.py:766: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
warnings.warn("Statistics for quantized inputs were expected, but not "
the version of installed packages is:
python: 3.8
onnx: 1.13.0
onnxruntime: 1.14.0
numpy: 1.24.2
tensorflow: 2.9.0
opencv-python: 4.7.0
onnx-simplifier: 0.4.17
Hi
Trying to convert Yolov7 but getting following error
Traceback (most recent call last):
File "converter.py", line 64, in
run()
File "converter.py", line 60, in run
image_root=opt.imgroot
File "converter.py", line 16, in onnx_converter
keras_model = keras_builder(model_proto)
File "/yolo/v7/onnx2tflite-main/utils/builder.py", line 84, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "/yolo/v7/onnx2tflite-main/layers/common_layers.py", line 290, in call
inputs[i] = tf.cast(inputs, dtype=self.tf_cast_map[self.cast_to])
UnboundLocalError: local variable 'i' referenced before assignment
something error when i use it
2023-05-27 16:38:48.242776: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-05-27 16:38:48.288173: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-05-27 16:38:48.288582: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-05-27 16:38:48.962344: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
WARNING: The argument dynamic_input_shape=True
is not needed any more, onnxsim can now support dynamic input shapes natively, please refer to the
latest documentation. An error will be raised in the future.
Checking 0/1...
shape[0] of input "input_0" is dynamic, we assume it presents batch size and set it as 1 when testing. If it is not wanted, please set the it manually by --test-input-shape (see onnxsim -h
for the details).
2023-05-27 16:38:51.395079: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1956] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
WARNING:convolution_layers ::ConvTranspose with pad will lead output error to bigger, please check it out.
WARNING:convolution_layers ::ConvTranspose with pad will lead output error to bigger, please check it out.
Traceback (most recent call last):
File "/home/duyangfan/nxy/onnx2tflite-main/converter.py", line 108, in
run()
File "/home/duyangfan/nxy/onnx2tflite-main/converter.py", line 92, in run
onnx_converter(
File "/home/duyangfan/nxy/onnx2tflite-main/converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
File "/home/duyangfan/nxy/onnx2tflite-main/utils/builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "/home/duyangfan/nxy/onnx2tflite-main/layers/mathematics_layers.py", line 100, in call
out = tf.matmul(self.first_operand, self.second_operand)
File "/home/duyangfan/miniconda3/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/duyangfan/miniconda3/lib/python3.10/site-packages/keras/layers/core/tf_op_layer.py", line 119, in handle
return TFOpLambda(op)(*args, **kwargs)
File "/home/duyangfan/miniconda3/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
ValueError: Exception encountered when calling layer "tf.linalg.matmul" (type TFOpLambda).
Dimensions must be equal, but are 58 and 64 for '{{node tf.linalg.matmul/MatMul}} = BatchMatMulV2[T=DT_FLOAT, adj_x=false, adj_y=false](Placeholder, tf.linalg.matmul/MatMul/b)' with input shapes: [1,1,58,58], [64,64].
Call arguments received by layer "tf.linalg.matmul" (type TFOpLambda):
• a=tf.Tensor(shape=(1, 1, 58, 58), dtype=float32)
• b=array([[ 0.04544717, 0.07029884, 0.1148618 , ..., -0.11733958,
-0.05653606, 0.00598602],
[ 0.0400255 , 0.09087554, -0.03369874, ..., 0.11601257,
-0.11816581, -0.02627973],
[-0.05443765, -0.06790046, 0.07890876, ..., 0.01277761,
-0.03936964, -0.06176169],
...,
[ 0.02922536, 0.05941173, -0.11838005, ..., -0.07970631,
-0.02553718, 0.10650459],
[ 0.00736488, -0.11999485, 0.02790602, ..., 0.11965565,
0.11500892, 0.01097947],
[-0.12261999, -0.00388932, 0.04968777, ..., 0.05339718,
0.10601966, -0.11772572]], dtype=float32)
• transpose_a=False
• transpose_b=False
• adjoint_a=False
• adjoint_b=False
• a_is_sparse=False
• b_is_sparse=False
• output_type=None
• name=None
This was strange because my model didn't show any dimensional errors during training
FastestDet链接[https://github.com/dog-qiuqiu/FastestDet]
onnxsim is failed, maybe make convert fails.
[0] redundant input nodes are removed.
nodes name :
Traceback (most recent call last):
File "C:\Users\Ziniu\Desktop\onnx2tflite-main\converter.py", line 70, in
run()
File "C:\Users\Ziniu\Desktop\onnx2tflite-main\converter.py", line 55, in run
onnx_converter(
File "C:\Users\Ziniu\Desktop\onnx2tflite-main\converter.py", line 18, in onnx_converter
keras_model = keras_builder(model_proto, input_node_names, output_node_names)
File "C:\Users\Ziniu\Desktop\onnx2tflite-main\utils\builder.py", line 103, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "C:\Users\Ziniu\Desktop\onnx2tflite-main\layers\conv_layers.py", line 80, in call
return self.conv(inputs)
File "C:\Users\Ziniu\Desktop\onnx2tflite-main\layers\conv_layers.py", line 119, in call
return self.conv(inputs)
File "C:\Users\Ziniu\AppData\Roaming\Python\Python39\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Ziniu\AppData\Roaming\Python\Python39\site-packages\keras\initializers\initializers_v2.py", line 265, in call
return tf.constant(self.value, dtype=_get_dtype(dtype), shape=shape)
附onnx文件:
FastestDet.zip
And there is also this error with opset=11
KeyError: Identity not yet implemented
我使用yolov5的转换方式将yolov5s.pt转换成onnx,再通过onnx2tflite的转换命令转换成keras和tflite。
PS E:\pycharm_proj\yolov5_switf_test\onnx2tflite> python .\converter.py --weights ./yolov5s.onnx --formats tflite keras --int8 --imgroot .\datasets\coco128\images\train2017\ --int8mean 0 0 0 --int8std 255 255 255
Checking 0/1...
[0] redundant input nodes are removed.
nodes name :
E:\yolov5-master\venv\lib\site-packages\tensorflow\lite\python\convert.py:746: UserWarning: Statistics for quantized inputs were expected, but not specified; cont
inuing anyway.
warnings.warn("Statistics for quantized inputs were expected, but not "
Estimated count of arithmetic ops: 18.144 G ops, equivalently 9.072 G MACs
fully_quantize: 0, inference_type: 6, input_inference_type: 3, output_inference_type: 3
Estimated count of arithmetic ops: 18.144 G ops, equivalently 9.072 G MACs
WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics
will be empty until you train or evaluate the model.
WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics
will be empty until you train or evaluate the model.
但首先我使用netron打不开h5,其次我使用yolov5的验证命令对tflite进行测试,先是shape的问题会报错,onnx2tflite输出的shape是[1, 85, 25200],而yolov5中输出是[1. 25200, 85],所以我修改了yolov5 val.py,line206
out = out.permute(0, 2, 1) # onnx2tflite: [1, 85, 25200] to [1, 25200, 85]
,再测试得到的输出显示为0是:
val: Scanning 'E:\datasets\coco128\labels\train2017.cache' images and labels... 128 found, 0 missing, 2 empty, 0 corrupt: 100%|██████████| 128/128 [00:00<?, ?it/s]
Class Images Labels P R [email protected] [email protected]:.95: 100%|██████████| 128/128 [1:18:54<00:00, 36.99s/it]
all 128 0 0 0 0 0
Speed: 0.9ms pre-process, 36981.4ms inference, 0.6ms NMS per image at shape (1, 3, 640, 640)
请问,我的测试流程是否有问题?作者是用什么方式进行验证测试yolov5的?另外这里得到的tflite与yolov5导出的tflite相比,会多出许多transpose和gather操作,这是什么问题导致的?
附:onnx2tflite导出的tflite:
yolov5s_onnx2tflite.zip
yolov5的tflite:
yolov5s-int8.zip
Hi, I'm using this package to convert this onnx model to tflite (model) but I'm not able to explore result and extract score and class_id
command used to convert model :
python converter.py --weights "./model.onnx" --outpath "./save_path"
full code :
#!/usr/bin/env python
import traceback
import cv2
import numpy as np
import tensorflow as tf
def run_model():
try:
interpreter = tf.lite.Interpreter(model_path="model.tflite")
interpreter.allocate_tensors()
image = cv2.imread('image.PNG')
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
image = cv2.resize(image, (input_shape[1], input_shape[2]))
test_image = np.expand_dims(image, axis=0)
input_data_type = input_details[0]['dtype']
padded_image = np.ascontiguousarray(test_image, dtype=np.float32)
input_tensor_index = input_details[0]['index']
interpreter.set_tensor(input_tensor_index, padded_image.astype(input_data_type))
interpreter.invoke()
tflite_model_predictions1 = interpreter.get_tensor(output_details[0]['index'])
print(tflite_model_predictions1)
except Exception:
traceback.print_exc()
return "There was an error processing the file"
if __name__ == "__main__":
run_model()
the output of interpreter.get_tensor is something like this
index 0 : [1.4157819747924805, 2.1544673442840576, ...... ] : size = 7098
index 1 : [1.805556058883667, 1.946408987045288, ........ ] : size = 7098
index 2 : [1.0974576473236084, 1.980888843536377, ....... ] : size = 7098
..
...
index 20 : [0.0024664357770234, 0.007526415400207, ...... ] : size = 7098
Question :
if someone can help how I can extract the class_id and score from the tflite_model_predictions output ?
NOTE : in original git (onnx model ) for given image example score is 0.89389664 and class_id is 7
Thanks for helping !!
I couldn't run it successfully, even running the example you provided got errors. Maybe something is wrong with my environment? Can you provide your specific environment?
The environment I use is:
torch=1.7.1
torchvision=0.8.2
tensorflow=1.15.2
onnx=1.8.0
onnx-simplifier=0.3.7
onnxrumtime=1.8.1
numpy=1.17.4
opencv-python=3.14.1.5
I have a problem with reproducing the example you provided. And the error is as follows:
Hello, I used my onnx model to convert into your tflite model, but after the conversion, the model can use CPU but not GPU, and there is a warning in the conversion process.
Checking 0/1...
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:tensorflow:Assets written to: C:\Users\Admin\AppData\Local\Temp\tmpae__80la\assets
INFO:tensorflow:Assets written to: C:\Users\Admin\AppData\Local\Temp\tmpae__80la\assets
WARNING:converter running::convert is successed, but model running is failed, please check carefully!
Hello,
I tried to convert YOLOv7 ONNX model to tflite.
When I tried in Intel Macbook, I got some error message.
$ python converter.py --weights "../yolov7-tiny.onnx" --outpath "./" --formats "tflite"
2023-05-08 19:48:50.472025: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Checking 0/1...
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
INFO:deformation_layers ::Transpose will process tensor after change back to NCHW format.
Traceback (most recent call last):
File "/onnx2tflite/converter.py", line 108, in /onnx2tflite/converter.py", line 92, in run
run()
File "
onnx_converter(
File "/onnx2tflite/converter.py", line 21, in onnx_converter/onnx2tflite/utils/builder.py", line 75, in keras_builder
keras_model = keras_builder(model_proto, native_groupconv)
File "
raise KeyError(f"{op_name} not implemented yet")
KeyError: 'NonMaxSuppression not implemented yet'
Is there something I can do to fix this?
Looking forward to your reply.
这是我的转换代码
from converter import onnx_converter
onnx_converter(
onnx_model_path = "./yolov5s.onnx",
need_simplify = True,
output_path = "./",
target_formats = ['tflite'], # or ['keras'], ['keras', 'tflite']
weight_quant = False,
int8_model = False,
int8_mean = None,
int8_std = None,
image_root = None
)
这是报错内容
Checking 0/1...
INFO:deformation_layers ::Reshape will process tensor after change back to NCHW format.
Traceback (most recent call last):
File "e:\nncase\onnx2tflite-main\onnx2tflite-main\onnx2tflite.py", line 2, in
onnx_converter(
File "e:\nncase\onnx2tflite-main\onnx2tflite-main\converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
File "e:\nncase\onnx2tflite-main\onnx2tflite-main\utils\builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "e:\nncase\onnx2tflite-main\onnx2tflite-main\layers\deformation_layers.py", line 140, in init
end = start + node_attribute['split'][index]
KeyError: 'split'
能麻烦看一下吗
Hi!
I have been using your repo lately and managed to implement dynamic resize, could we do a PR about this and included in main
Greetings!
Hi 博主,
感谢你提供的工具,我在转换mobilefacenet 模型的时候,遇到以下几个问题,不知能否帮忙解答:
【问题描述】
1.onnx2tflite之后,模型输出由128变成1000
//转换前:
name: fc1
type: float32[1,128]
//转换后:
name: StatefulPartitionedCall:0
type: float32[1,1000]
location: 177
2.将生成的模型导入APK验证,脸部检测功能能正常工作,但是但是人脸识别功能失效
此处暂未提供LOG,优先确认问题1
2.生成的tflite模型尺寸几乎和onnx大小相当,未见尺寸优化
(base) rsi@DESKTOP-1NGTB52:~/onnx2tflite/onnx2tflite/input$ ll
total 62388
drwxr-xr-x 2 rsi rsi 4096 Oct 31 11:39 ./
drwxr-xr-x 8 rsi rsi 4096 Oct 31 11:34 ../
-rw-r--r-- 1 rsi rsi 13965850 Oct 31 11:00 face_recognition_sface_2021dec-act_int8-wt_int8-quantized.onnx
-rw-r--r-- 1 rsi rsi 13979876 Oct 31 11:00 face_recognition_sface_2021dec-act_int8-wt_int8-quantized.tflite
-rw-r--r-- 1 rsi rsi 13965850 Oct 31 11:33 face_recognition_sface_2021dec.onnx
-rw-r--r-- 1 rsi rsi 13979876 Oct 31 11:34 face_recognition_sface_2021dec.tflite
2210311357更新:tflite模型偏大的问题不知道是不是该文档提到大的,onnx转tflite需要去冗余操作:https://zhuanlan.zhihu.com/p/363317178
【所使用转换脚本】
使用博主的参考py,改成如下onnx2tflite.py
import torch
import torchvision
_input = torch.randn(1, 3, 112, 112)
model = torchvision.models.mobilenet_v2(True)
# use default settings is ok
torch.onnx.export(model, _input, './input/face_recognition_sface_2021dec.onnx', opset_version=11)# or opset_version=13
from converter import onnx_converter
onnx_converter(
onnx_model_path = "./input/face_recognition_sface_2021dec.onnx",
need_simplify = True,
output_path = "./",
target_formats = ['tflite'], # or ['keras'], ['keras', 'tflite']
weight_quant = False,
int8_model = False,
int8_mean = None,
int8_std = None,
image_root = None
)
[相关模型]:
1.我所转换的模型为opencv_zoo/models/face_recognition_sface/
1.face_recognition_sface_2021dec-act_int8-wt_int8-quantized.onnx
2.face_recognition_sface_2021dec.onnx
原始模型链接为:https://github.com/opencv/opencv_zoo/tree/master/models/face_recognition_sface
谢谢解答
模型优化失败, 从./models/arcface.onnx加载
2023-03-04 14:26:09.484399: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /mnt/sda/AI/miniconda3/envs/yolov5/lib/python3.7/site-packages/cv2/../../lib64:
2023-03-04 14:26:09.484419: W tensorflow/stream_executor/cuda/cuda_driver.cc:263] failed call to cuInit: UNKNOWN ERROR (303)
2023-03-04 14:26:09.484429: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ubuntu): /proc/driver/nvidia/version does not exist
2023-03-04 14:26:09.484577: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
File "/mnt/sda/AI/onnx2tflite-main/converter.py", line 70, in
run()
File "/mnt/sda/AI/onnx2tflite-main/converter.py", line 66, in run
image_root=opt.imgroot
File "/mnt/sda/AI/onnx2tflite-main/converter.py", line 18, in onnx_converter
keras_model = keras_builder(model_proto, input_node_names, output_node_names)
File "/mnt/sda/AI/onnx2tflite-main/utils/builder.py", line 88, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "/mnt/sda/AI/onnx2tflite-main/layers/activations_layers.py", line 65, in init
self.slope = node_weights[node_inputs[1]].transpose(1, 2, 0)
ValueError: axes don't match array
进程已结束,退出代码1
WARNING: The argument dynamic_input_shape=True
is not needed any more,
onnxsim can now support dynamic input shapes natively, please refer to the
latest documentation. An error will be raised in the future.
WARNING:onnx_loader running::onnxsim is failed, maybe make convert fails.
Traceback (most recent call last):
File "E:\work\project\onnx2tf\onnx2tflite\test.py", line 6, in
onnx_converter(
File "E:\work\project\onnx2tf\onnx2tflite\converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
File "E:\work\project\onnx2tf\onnx2tflite\utils\builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
File "E:\work\project\onnx2tf\onnx2tflite\layers\conv_layers.py", line 87, in call
return self.conv(inputs)
File "E:\work\project\onnx2tf\onnx2tflite\layers\conv_layers.py", line 155, in call
return self.conv(inputs)
File "C:\Users\tdf\AppData\Local\miniconda3\envs\onnx-tf\lib\site-packages\keras\src\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\tdf\AppData\Local\miniconda3\envs\onnx-tf\lib\site-packages\tensorflow\python\framework\ops.py", line 1751, in _create_c_op
raise ValueError(e.message)
ValueError: Exception encountered when calling layer "conv2d" (type Conv2D).
Negative dimension size caused by subtracting 6 from 4 for '{{node conv2d/Conv2D}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true](Placeholder, conv2d/Conv2D/ReadVariableOp)' with input shapes: [1,4,4,3], [6,6,3,32].
Call arguments received by layer "conv2d" (type Conv2D):
• inputs=tf.Tensor(shape=(1, 4, 4, 3), dtype=float32)
Process finished with exit code 1
my onnx model is yolov5s
PyTorch code:
def forward(self, x):
## x.shape = [B, T, F]
B, _, _ = x.shape
x = x.unsqueeze(1).transpose(2, 3)
x = self.layers(x)
x = torch.mean(x, dim=2, keepdim=True)
x = x.view((B, self.last_channel, -1))
x = x.transpose(1, 2)
return x
when convert onnx model to tflite, builder.py throw a KeyError, maybe we need to implement Shape op.
Hi author,
Following error rises up during conversion from an onnx to tflite:
ERROR:converter running::tflite model elements' max error has reached 1.4117E+01, but convert is done, please check xx.tflite carefully!
The onnx itself works normally for python. Is there any way to do further debug? Any suggestion is appreciated.
Thank you for this great job, im trying to add qat scheme with keras_model
in converter.py, but encounter this exception
keras_model = keras_builder(model_proto, input_node_names, output_node_names, native_groupconv)
# start qat
#! add by dongz
import keras
import tensorflow as tf
import tensorflow_model_optimization as tfmot
quantize_model = tfmot.quantization.keras.quantize_model(keras_model)
Error: Shape must be rank 4 but is rank 5 for '{{node depthwise_conv2d/depthwise}} = DepthwiseConv2dNative[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1]](Placeholder, depthwise_conv2d/depthwise/ReadVariableOp)' with input shapes: [1,1,80,80,24], [3,3,24,1]
It seems like keras builder add batch dimension, and quantizer add batch dimension too. So how can i let keras builder return a non-batch fixed model?
Hi, thanks for your work, I got a problem when converting onnx (pytorch) model to tflite model. Before adding SE module, the difference between onnx model and tflite model is reasonable, but after adding SE module to the original network, the difference became larger, for example, the mean of difference is 3 or even larger
I use the simple SE module
class SEBlock(nn.Module): # Squeeze and Excitation block
def __init__(self, channels, ratio=16):
super(SEBlock, self).__init__()
channels = channels # 输入的feature map通道数
hidden_channels = channels // ratio # 中间过程的通道数,原文reduction ratio设为16
self.attn = nn.Sequential(
nn.AdaptiveAvgPool2d(1), # avgpool
nn.Conv2d(channels, hidden_channels, 1, 1, 0), # 1x1conv,替代linear
nn.ReLU(), # relu
nn.Conv2d(hidden_channels, channels, 1, 1, 0), # 1x1conv,替代linear
nn.Sigmoid() # sigmoid,将输出压缩到(0,1)
)
def forward(self, x):
weights = self.attn(x) # feature map每个通道的重要性权重(0,1),对应原文的sc
return weights * x # 将计算得到的weights与输入的feature map相乘
Checking 0/1...
shape[0] of input "input" is dynamic, we assume it presents batch size and set it as 1 when testing. If it is not wanted, please set the it manually by --test-input-shape (see onnxsim -h
for the details).
Traceback (most recent call last):
File "G:\onnx2tflite-main\converter.py", line 108, in
run()
File "G:\onnx2tflite-main\converter.py", line 92, in run
onnx_converter(
File "G:\onnx2tflite-main\converter.py", line 21, in onnx_converter
keras_model = keras_builder(model_proto, native_groupconv)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\onnx2tflite-main\utils\builder.py", line 82, in keras_builder
tf_tensor[node_outputs[index]] = tf_operator(tf_tensor, onnx_weights, node_inputs, op_attr, index=index)(_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "G:\onnx2tflite-main\layers\common_layers.py", line 58, in call
return tf.pad(inputs, self.pad, mode=self.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Lee\AppData\Roaming\Python\Python311\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Lee\AppData\Roaming\Python\Python311\site-packages\keras\src\layers\core\tf_op_layer.py", line 119, in handle
return TFOpLambda(op)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Lee\AppData\Roaming\Python\Python311\site-packages\keras\src\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
^^^^^^^^^^^^^^^
ValueError: Exception encountered when calling layer "tf.pad" (type TFOpLambda).
Value of argument mode
expected to be one of "CONSTANT", "REFLECT", or "SYMMETRIC". Received mode
= EDGE
Call arguments received by layer "tf.pad" (type TFOpLambda):
• tensor=tf.Tensor(shape=(1, 64, 64, 3), dtype=float32)
• paddings=[['0', '0'], ['1', '1'], ['1', '1'], ['0', '0']]
• mode='EDGE'
• constant_values=0
• name=None
Hi! I'm getting an error in Google Colab on attempting to convert an ONNX model to TFLite using this tool. I've attached the ONNX model causing the bug, and also provided full code to reproduce below. I tried YOLOv5n models exported to ONNX using opset 11, 12 and 13, and saw problems in all of them.
!git clone https://github.com/ultralytics/yolov5
!git clone https://github.com/MPolaris/onnx2tflite
%pip install -qr yolov5/requirements.txt # install
!python yolov5/export.py --weights ./yolov5n.pt --include onnx
%cd onnx2tflite
from converter import onnx_converter
onnx_converter(
onnx_model_path = "../yolov5n.onnx",
need_simplify = True,
output_path = "./",
target_formats = ['tflite'], # or ['keras'], ['keras', 'tflite']
weight_quant = False,
int8_model = False,
int8_mean = None,
int8_std = None,
image_root = None
)
WARNING:onnx_loader running::模型优化失败, 从../yolov5n.onnx加载
/content/onnx2tflite/onnx2tflite
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
[<ipython-input-6-5f2991b979cc>](https://localhost:8080/#) in <module>
16 int8_mean = None,
17 int8_std = None,
---> 18 image_root = None
19 )
2 frames
[/content/onnx2tflite/layers/common_layers.py](https://localhost:8080/#) in __init__(self, tensor_grap, node_weights, node_inputs, node_attribute, *args, **kwargs)
199 else:
200 # 从scales取
--> 201 _, _, nh, nw = node_weights[node_inputs[2]]
202 _, h, w, _ = tensor_grap[node_inputs[0]].shape
203 nh, nw = int(h*nh), int(w*nw)
KeyError: 'onnx::Resize_445'
Hello Author.
Your repository implementation has helped me a lot. Perhaps the repository mentioned in the README refers to the tool I created, but I have already included the quoted comment in the source code a few days ago as a thank you to you.
https://github.com/PINTO0309/onnx2tf/blob/71fa4e112440a5a604b1e99a794a77699bfcded4/onnx2tf/ops/MaxPool.py#L127-L137
I was unaware that your useful tools existed and accidentally started creating my own tools.
I am sorry if I have caused you any discomfort. I do not intend to interfere with your activities at all. I myself find your tool very easy to handle.
My tool is very different in concept from onnx2tflite because I am very conscious of implementing as many means of avoiding conversion errors as possible, instead of allowing for code complexity. I am aware that my code is too complex and probably very difficult for other engineers, except you, to understand. Therefore, I have no intention of attracting many people to my side of the tool.
If a model could not be converted by your tool, there were several times when an issue was posted on my side of the tool, and I would try to politely respond to the issue. Since your repository is by far the more popular, I don't feel any desire to be a counter-rival now. I also do not expect any more Star in my repository.
I have not been able to find a way to contact you by DM or other means, so I am posting this issue to express my gratitude to you. If even this issue offends you, you may delete it.
Short sentences do not do a good job of conveying to you how I usually think.
Thank you again.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.