Undefined shape value in paddle generated onnx model about paddle2onnx HOT 13 OPEN

@varunarora Yes, property shape can be set to None in the latest master branch, while in the release 1.0.1 it cannot be, see here.

In principle, the shape of all the mediate tensors can be infered from inputs and parameters automatically in runtime. But unfortunately some backends may do dimension checking for each operator when importing models. For instance, I tried to assign some random values to the shape property here

https://github.com/PaddlePaddle/paddle-onnx/blob/02a9ffdd07bc5a95a8b17065e7c05a4308842f0f/fluid_onnx/variables.py#L25-L26

and failed to run the converted fit_a_line model by using onnx-tensorrt backend, and got the error like:

/xxx/onnx-tensorrt/builtin_op_importers.cpp:693 In function importMatMul:
[6] Assertion failed: weights.shape.d[1] == ninput

Seems that we have no choice but to set the right shape here in models' converting. But the first dimension is an exception, which is interpreted as the batch size, and it can be initialized by zero or any positive intergers and infered correctly in runtime.

@postrational @baojun-nervana We chose dim_value 0 instead of dim_paramto indicate the unknown batch size because it is more close to our convention（-1 in Fluid but caffe2 backend doesn't support). And we checked the models in onnx/models, they also use 0 for the batch size (not sure it is true for all models or any update after that). So can this dimension be inferred automatically in ngraph-onnx even we can set it to 1?

from paddle2onnx.

#54 is a simple workaround for the 0 dimension in shape, which forces any 0 to a 1.
You may want to parametrise the 1 value instead of hard-coding it or find another fix.

from paddle2onnx.

I wonder how to define unknown dimension in ngraph-onnx. The converted models have been tested on TensorRT and caffe2 ONNX backend. It is the problem of ngraph-onnx if it cannot important models following the ONNX specifications. You'd better consider to fix ngraph-onnx to support dim 0.

from paddle2onnx.

@kuke Could you point us to the part of the ONNX specification, which talks about how dim 0 should be supported?

from paddle2onnx.

@kuke It sounds some FWs can handle undefined dimensions (TensorRT), while others may not (like nGraph). Very likely another FW besides nGraph can encounter the similar issue. In the case we are discussing, it refers to batch size and we can enforce it to some other value. If the dim0 refer to anything else, it can be hard to work around. Paddle-ONNX will be one that knows what those dim_value represent and the best place to handle it. Is there any problem to have a default/defined value other than 0?

from paddle2onnx.

@postrational https://github.com/onnx/onnx/blob/master/docs/IR.md#tensor-shapes

from paddle2onnx.

@kuke On the basis of the specification, I guess the shape property should be undefined from the tensor record? I know the make_tensor requires a value to be passed for dimension, but make_tensor_value_info allows us to pass a None - I think.

https://github.com/PaddlePaddle/paddle-onnx/blob/02a9ffdd07bc5a95a8b17065e7c05a4308842f0f/fluid_onnx/variables.py#L38-L45

from paddle2onnx.

@varunarora
If I'm reading these docs correctly, they do not allow for a 0 dim value.

Each size in the list MUST be expressed as:

• an integral value
• or as a "dimension variable," a string denoting that the actual size of the dimension is not statically constrained to a particular number. This is useful for declaring interfaces that care about the number of dimensions, but not the exact size of each dimension.

The emptry string "", when used as a dimension name, denotes a single dimension of any cardinality.

As far as I can tell, if you want to express an unknown dim value in ONNX, you should not use dim_value=0, but instead use dim_param=''.

This will not help us however, as nGraph will still need to set this to some value to build a static graph.

from paddle2onnx.

I fully understand @postrational. Have you read the part that says:

A zero-dimension tensor is distinct from a tensor of unknown dimensionality, which is indicated by an absent 'shape' property in the Tensor record. When the shape property is absent for an input, a tensor value of any shape may be passed from the caller

As @kuke pointed out, it would only be right to make these changes at the nGraph-ONNX project level, because unknown dimensionality is allowed from a spec perspective, whichever way it is set. Makes lesser sense to change it in this convertor.

from paddle2onnx.

@varunarora If I understand the docs correctly, a "zero-dimension tensor" is a tensor with no dimensions denoting a scalar value (not one with a dimension set to 0).

The empty list of dimension sizes, [], is a valid tensor shape, denoting a zero-dimension (scalar) value. A zero-dimension tensor is distinct from a tensor of unknown dimensionality...

@kuke Could you tell us which model in https://github.com/onnx/models uses a 0 shape?

from paddle2onnx.

Yep @postrational agreed, we need to eliminate the shape property from tensors of unknown dimensionality. We don't have it right. The purpose to show you that tensors of dimensionality 0 are allowed, even if we don't use them fully correctly.

from paddle2onnx.

@postrational It was several months ago that I checked one or two binary models, now they have detailed READMEs and seems that those models take one image as input. But this is not the point. The important thing is whether an ONNX backend can infer this dimension correctly from inputs. Now that dim 0/1 or some other value is just a placeholder, we can use dim_value==1 for the unknown batch size. Hope that it will help to solve your problem. And @varunarora , what is your opinion?

from paddle2onnx.

I have made some preliminary support for undefined shapes at #104

from paddle2onnx.