Comments (1)
Describe the bug
When generating higher-pitched female voices after fine-tuning the xtts-v2 model, there is a noticeable hoarseness, resembling the strain one might experience when trying to reach high musical notes.
abnormal example: https://mork.ro/NQjFi
normal example: https://mork.ro/3iZ8Q#
Two voices generated from the same model, using different audio prompts.
To Reproduce
infer
Expected behavior
No response
Logs
No response
Environment
{ "CUDA": { "GPU": [ "NVIDIA GeForce RTX 4090" ], "available": true, "version": "12.1" }, "Packages": { "PyTorch_debug": false, "PyTorch_version": "2.1.1+cu121", "TTS": "0.22.0", "numpy": "1.22.0" }, "System": { "OS": "Linux", "architecture": [ "64bit", "ELF" ], "processor": "x86_64", "python": "3.10.13", "version": "#202310061235~1697396945~22.04~9283e32 SMP PREEMPT_DYNAMIC Sun O" } }Additional context
No response
Describe the bug
When generating higher-pitched female voices after fine-tuning the xtts-v2 model, there is a noticeable hoarseness, resembling the strain one might experience when trying to reach high musical notes.
abnormal example: https://mork.ro/NQjFi
normal example: https://mork.ro/3iZ8Q#
Two voices generated from the same model, using different audio prompts.
To Reproduce
infer
Expected behavior
No response
Logs
No response
Environment
{ "CUDA": { "GPU": [ "NVIDIA GeForce RTX 4090" ], "available": true, "version": "12.1" }, "Packages": { "PyTorch_debug": false, "PyTorch_version": "2.1.1+cu121", "TTS": "0.22.0", "numpy": "1.22.0" }, "System": { "OS": "Linux", "architecture": [ "64bit", "ELF" ], "processor": "x86_64", "python": "3.10.13", "version": "#202310061235~1697396945~22.04~9283e32 SMP PREEMPT_DYNAMIC Sun O" } }Additional context
No response
I'm experiencing the same thing with 900 hours of Chinese data fine tuning, 40,000 STEP is prone to this. What is your data? Which languages? How many steps?
from tts.
Related Issues (20)
- [Bug] Anyway to run this as docker-compose ? HOT 2
- [Bug] DDP not actually working HOT 1
- [Bug] 'tts_models/ben/fairseq/vits' model didn't found Character 'য়' not found in the vocabulary. HOT 1
- [Bug] Anyway to run this as docker-compose ? HOT 2
- [Bug] ValueError: Can't infer missing attention mask on `mps` device. Please provide an `attention_mask` or use a different device. HOT 5
- Add Support for laughter annotation in Fine-Tuning with a special token [Feature request]
- [Feature request] pronounciation, cadence and nuances in XTTS v2... HOT 3
- [Bug] Armenian Language model training fail HOT 3
- [Feature request] is it possible to initialize speaker_wav 1 time?
- Error: Could not install packages due to an OSError [Bug] HOT 5
- [Feature request] Can we add the batch inference or batch decoding for XTTS
- [Feature request] I just want the timbre feature embedding of the wav file. How to design the API to get it?
- Coqui is shutting down????
- [Bug] Official linked XTTS_v2 Google Colab throws: ibcublas.so.11 is not found HOT 1
- nvm it work fine just dll fixed work fine
- is their any docs for using onnx with it ?
- [Bug] torch.isin(elements=inputs, test_elements=pad_token_id).any() TypeError: isin() received an invalid combination of arguments - got (elements=Tensor, test_elements=int, ) HOT 2
- [Bug] tts_with_vc_to_file uses cuda even though "cpu" is specified as device
- Gradio Live , Create Dataset gives an error : ValueError: Requested float16 compute type, but the target device or backend do not support efficient float16 computation.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tts.