Comments (7)
Where are you seeing this behavior?
Note that during training we set the validation data loader with the speakers from the training data loader.
https://github.com/NVIDIA/mellotron/blob/master/train.py#L44
from mellotron.
It's in inference.ipynb
when you calculate the male and female speakers, you only pass in the evaluation filelist
which only contains 68 of the 123 speakers.
I had added code to retrieve a sample of the chosen speaker and found in most cases they were wildly different from generated voice.
I wonder if this has anything role in the men sounding like women.
from mellotron.
Note that the first line defining speaker_ids
assumes the model was trained with the file list we provided by default, which only includes speakers with at least 5 minutes.
If you trained with the entire LibriTTS dataset, replace the first line with your training data.
from mellotron.
That makes sense - so far I haven't added (in this environment) any additional speakers, I've only used the included files. It shouldn't make any difference - this should be true for the pre-trained model? The problem comes from how it's calculating the mellotron_id in the inference jupyter page.
I just replaced the reference to the evaluation filelist with the training filelist and saw an immediate improvement in the correlation between the generated speaker and samples of the actual speaker. I'm not talking about the speaker used for the reference mel and rhythm, only the voice used for inference.
from mellotron.
I finally see your point, thank you.
It's a matter of substituting the filepath with filelists/libritts_train_clean_100_audiopath_text_sid_shorterthan10s_atleast5min_train_filelist.txt
Please pull from master for the fix and let us know if you still find errors.
from mellotron.
Does this also apply to the LJS model? Inferencing requires the Define Speakers Set block to be executed, however LJS only has 1 speaker.
from mellotron.
@camjac251
TLDR:This only applies to the inference.ipynb jupyter page.
If you have switched to a different model such as LJS, you would need to adjust the speaker id's so that it matched, otherwise your speakers will not match, or in all likelihood with LJS you would get "index out of range".
Basically this comes down to you need to use the same speaker Ids you used for training as you do for inference. Rafael's post references how this is done.
from mellotron.
Related Issues (20)
- NoneType' object is not iterable
- Mismatch model volume
- Training on a different language HOT 1
- Inference without rhythm and pitch
- parse_output error with Blizzard2013 data
- Training on EmovDB HOT 2
- Voice synthesis by model is not the same as the voice with speaker ID HOT 1
- Try to train some new words
- inference speed on CPU
- Training time
- Two key points of training multispeaker mellotron
- how to train?
- colab demo for inferenece
- How to generate .musicxml files like the examples in `/data`? HOT 1
- Synthesize own text without style transfer gives poor audio results HOT 1
- Here's some code to start mellotron inference by calling a .py file from CLI [Docs]
- Is there a way to extract character durations?
- Something wrong with text padding HOT 5
- Can I use TensorRT to speed up model inference?
- colab error HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mellotron.