Comments (5)
Hi @jambayk
Thanks for the response & hard work.
I will test whisper multilingual model in a few days.
And will report the result.
thanks again for your hard work.
from olive.
Thanks for bringing this up! Our whisper example currently doesn't support multi-language support since the changes related to it were introduced in a newer unreleased version of onnxruntime. However, we have plans to add support for ort > 1.15 (nightly versions) and multi-language.
I will link this issue when the corresponding PR is made.
from olive.
Hi. @jambayk
Thanks for the response.
I hope that the work will go smoothly.
from olive.
@gyujin-kim The linked PR above added multilingual support. Please refer to the updated Whisper example readme for details on how to use it.
from olive.
Hi @jambayk
I have succeeded in creating a whisper onnx model and testing it.
- olive for convert : commit e559816
- onnxruntime for convert & inference : ort-nightly 1.16.0.dev20230625001
- inference test : use test_transcription.py
And I succeed to convert the other whisper model(ex. whisper-small) but recognized text quality is poor than torch model and onnx encoder-decoder model.
(https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/notebooks/whisper_onnxruntime.ipynb)
Testing several languages by tweaking 'decoder_input_ids' in the 'whisper_dataset.py' file, the quality loss was worse in non-English languages.
I can't figure out what could be the problem.
Any suggesions to check this problem?
Edit1:
I have test with int8 model. I will test another precision and re-checking.
Edit2:
I have test with fp32 model and got better result.
But still worse than original model output.
I did't test lot of input, I will be update the status later after more test.
thanks.
from olive.
Related Issues (20)
- Failed to run symbolic shape inference when doing LLM Optimization with DirectML HOT 8
- Error on the Generate an ONNX model and optimize step HOT 5
- status.IsOK() was false. Tensor shape cannot contain any negative value HOT 1
- Vitis quantization is broken with ORT 1.18 HOT 2
- Enabling openai/whisper-large-v3 using olive-ai-0.6.0 [onnxruntime-gpu: 1.17.1] on Intel CPU/GPU is not supporting HOT 2
- Llava-7b model Conversion to ONNX and Latency Optimization - OOM error (even after setting paging file size) HOT 2
- safetensor model
- onnx
- huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.'
- "num_images" doesn't work for the example of directml stable_diffusion_xl.
- Get Error while optimizing SDXL of DirectML example
- [Bug]: Optimization of Unet fails - AMD RDNA3.5 Strix Point Processor HOT 1
- [FR]: Finetuning QLoRA adapters and swapping HOT 1
- [Question]: How do you use a finetuned model for mobile HOT 1
- data_config: data_name is not None but olive always said it is None HOT 3
- Failed on optimization step of Unet - Intel Meteor Lake Processor
- Failed to optimize Whisper tiny by gpu (CUDA)
- Using Whisper for Chinese ASR in iOS may occasionally output illegal UTF-8 strings. HOT 2
- vae_encoder_gpu-dml_footprints.json file not found when converting stable diffusion xl base model HOT 5
- [Whisper] convert with FP32 running failed HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from olive.