Comments (2)
@zirui Thank you for using ONNX with Triton. This is likely an issue with your model and ORT. Could you try to load and evaluate the model directly with ORT? If the issue repros, please close out this issue and ask for help over on ORT. If not, report back and we'll try to help you here.
@deadeyegoodwin I'm going to try to add this step to the issue template, but I might need your help.
from onnxruntime_backend.
@DavidLangworthy , I loaded the model directly with ORT, and it failed,
and i found another onnx model without optimize by onnxruntime(onnxruntime_tools.optimizer_cli) works , so it seems that the onnx model optimized by onnxruntime could not be loaded by ORT,
i' m close this issue, and ask for help over the ORT/onnxruntime
from onnxruntime_backend.
Related Issues (20)
- Possible to enable dynamic batch dimension only on one some input tensors?
- Expose `session.use_device_allocator_for_initializers` in onnxruntime_backend to completely shrink arena
- Add option to enable CUDA Graphs in CUDA EP
- Update onnxruntime to 1.14.0 or 1.14.1 to fix TensorRT issue
- Can I build the Onnxruntime backend for Windows without Docker??
- Onnxruntime Error
- Fatal error: TRT:EfficientNMS_TRT(-1) is not a registered function/op HOT 2
- InvalidArgumentError: The tensor Input (Input) of Slice op is not initialized.
- How to create onnx model for ragged batching?
- Add `enable_dynamic_shapes` To Model Config To Resolve CNN Memory Leaks With OpenVino EP
- GPU memory leak with high load for ONNX model HOT 3
- Onnxruntime backend error when workload is high since Triton uses CUDA 12 HOT 4
- how to use onnxruntime profiling in triton
- Error while Loading YOLOv8 Model with EfficientNMS_TRT Plugin in TRITON HOT 2
- Openvino doesn't work, it degrades inference performance instead HOT 4
- Support arbitrary options for execution providers
- Model failed to create because of output dimensions
- Question: Does ONNX-RT silently fallbacks to CPU? HOT 1
- Request for Supporting minShapes/optShapes/maxShapes for TensorRT HOT 1
- Will onxxruntime backend support INT8 on cpu ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime_backend.