Mt5 model loading fails about fastt5 HOT 11 OPEN

it looks like the issue is in onnxruntime itself, I suggest you to create an issue there.

from fastt5.

are you using it for GPU?

from fastt5.

yes, but i tried with CPU. Still it fails

from fastt5.

sorry, the library does not support GPU yet, but the issue is similar to microsoft/onnxruntime#3113

for CPU are you facing the same issue?

from fastt5.

Yes on cpu too same issue

from fastt5.

i found some more logs as,
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: SystemError : 2

from fastt5.

could you please provide the reproducible code and full log of error

from fastt5.

I am not sure about code but you can use following model, i am using example mentioned in main readme page
https://huggingface.co/google/mt5-large

from fastt5.

Also i was able to get soultion for this by onnx github, there is new tool for converting model from mt5 models
https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/transformers/models/t5.
just to give more idea, fastt5 exporter creates file size of 8.5 GB for https://huggingface.co/google/mt5-large model. But onnx tool creates around 9.2 GB so i think some additional files may be missing while exporting the model hence it was failing to create seesion

from fastt5.

I am facing input name issue as
[ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:pkv_95, is there any way to pass original name of inputs.
I could see in encoder module the inputs are passed as pkv_0 to pkv_95, which doesnt match with original input names

from fastt5.

i tried different approch, now it gives out
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Deserialize tensor onnx::MatMul_4622 failed.tensorprotoutils.cc:637 TensorProtoToTensor External initializer: onnx::MatMul_4622 offset: 0 size to read: 11534336 given file_length: 4194304 are out of bounds or can not be read in full.

from fastt5.