Comments (2)
I have no empirical evidence, but I think the difference comes from whether to capture dependencies between output mel-spectrogram frames or not. The probabilistic modeling of each model is quite different, and therefore their factorization levels of output distribution are different.
For brevity, I'll not mention some conditions explicitly. For example, I'll use p(mel-frames), not p(mel-frames | text).
The post-net can be used to refine the output mel-frames after sampling procedure is over.
It makes up for the lack of in-channel or in-frame dependencies of mel-frames.
Now, look at the difference of all models:
Tacotron 2: no future-frame and in-channel info -> p(mel) = product of p(mel[i,j] | mel[:i])
FastSpeech: no in-frame and in-channel info -> p(mel) = product of p(mel[i,j])
Glow-TTS*: some degree of all-frames and all-channels info -> p(mel) = product of p(latent_representation[i,j]) * jacobian determinant
*In Glow-TTS, an 1x1 invertible convolution captures in-channel dependencies, and an affine coupling layer captures in-frame dependencies.
Therefore, although Glow-TTS samples all mel-frames in parallel, it can use some degree of information of all previous and next channels as well as all previous and next frames, to make the current channel of current mel-frame, without the need of post-net.
from glow-tts.
Thanks for your reply. As you explained, the results benefit from the coupling layer and the invertible convolution design. It's a nice work of normalizing flows! Congratulations and I will follow your paper and research!
from glow-tts.
Related Issues (20)
- Runtime Error: Multi speaker HOT 1
- GPU required or CPU-compatible? HOT 1
- Different Languages us different amount of GPU memory
- multi speaker
- Output compared to Fastspeech2
- Models for finetuning
- Could not create monotonic_align HOT 3
- Glowtts melspectrogram to fine tune hifigan HOT 2
- RuntimeError: CUDA error: invalid device function
- ImportError: /glow-tts/monotonic_align/monotonic_align/core.cpython-38-x86_64-linux-gnu.so: failed to map segment from shared object HOT 1
- Error using mel generated from glow-tts for hifi-gan training HOT 1
- Can I apply MAS method to other model ? HOT 1
- Query : How is the Model training different from the Model training of wave glow
- Multi speaker training error HOT 11
- With out Training DDI
- An explanation for the source code of finding the alignment path in GlowTTS? HOT 2
- DDI training compared to not DDI training HOT 1
- [Question] How many iterations for the available pretrained model?
- [Question] about `intersperse` function. HOT 2
- [CONTRIBUTION] Speech Dataset Generator
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from glow-tts.