Comments (2)
https://github.com/huggingface/optimum-neuron is probably what you are looking for!
See as well: https://huggingface.co/docs/optimum-neuron/index
from optimum.
Also neuronx, which is for the newer inf2/trn1. Unlike inf1, these are well suited to generative transformers.
from optimum.
Related Issues (20)
- GPTQ Quantization Need `use_marlin` HOT 1
- Latest Optimum library does not compatible with latest Transformers HOT 1
- Convert to onnx missing safety_checker
- Whisper-large-v3 transcript is trimmed HOT 4
- UMT5 & ByT5 Support
- Unexpected arguments `trust_remote_code` when exporting model to onnx with option `--library sentence_transformers` HOT 2
- attributeError: 'str' object has no attribute 'impl' HOT 3
- BetterTransformer support for VisionEncoderDecoder models like TrOCR
- Issue converting moss-moon-003-sft-int4 model to ONNX format
- [GPTQQuantizer] How to use multi-GPU for GPTQQuantizer? HOT 2
- SentenceTransformer to tflite export failure HOT 2
- Error while optimizing seq2seq model using optimum HOT 1
- Correct example to use TensorRT? HOT 2
- RuntimeError: Failed to import optimum.onnxruntime.modeling_ort because of the following error HOT 2
- qwen2 onnx model attention_mask && output_past_kv shape is wrong
- ORTModelForCustomTasks lacks attributes HOT 1
- Support Transformers v4.44 HOT 1
- AttributeError: FLOAT8E4M3FN HOT 3
- BetterTransformer for florence2 HOT 2
- NameError: name '_SENTENCE_TRANSFORMERS_TASKS_TO_MODEL_LOADERS' is not defined HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from optimum.