GravesAttention with Tacotron 1 yields empty alignment plots during training and throwns no attribute error during inference about tts HOT 11 CLOSED

Good to hear that but I am personally not sure if the implementation is right comparing to this paper https://arxiv.org/abs/1910.10288

AFAIK this is the most robust Graves attention so far proposed for TTS. It may be wrong.

Itd be nice if you could double check.

from tts.

So, if I comment out the line self.attention.init_win_idx() in tacotron.py, I guess inference won't be complaining and in case OriginalAttention() is used, the self.attention.init_win_idx() method is called inside the OriginalAttention() module (line 226). I'm gonna launch a training to see if the empty alignment plots are still there, reporting back later today.

from tts.

I'll take a look at Graves attention but in the meantime, you can try DDC or DCA models for solving attention.

from tts.

Thanks, yeah just checked and the attention plots are still empty, moving onto DDC/DCA now.

from tts.

Just a side note. DCA is faster and uses less memory but DDC conspires better quality.

from tts.

@erogol thanks, if I wanted to use DDC, I understand I set "double_decoder_consistency": true, but then which attention_type should I choose?

from tts.

it should be like

    // TACOTRON ATTENTION
    "attention_type": "original",  // 'original' , 'graves', 'dynamic_convolution'
    "attention_heads": 4,          // number of attention heads (only for 'graves')
    "attention_norm": "sigmoid",   // softmax or sigmoid.
    "windowing": false,            // Enables attention windowing. Used only in eval mode.
    "use_forward_attn": false,     // if it uses forward attention. In general, it aligns faster.
    "forward_attn_mask": false,    // Additional masking forcing monotonicity only in eval mode.
    "transition_agent": false,     // enable/disable transition agent of forward attention.
    "location_attn": true,         // enable_disable location sensitive attention. It is enabled for TACOTRON by default.
    "bidirectional_decoder": false,  // use https://arxiv.org/abs/1907.09006. Use it, if attention does not work well with your dataset.
    "double_decoder_consistency": true,  // use DDC explained here https://erogol.com/solving-attention-problems-of-tts-models-with-double-decoder-consistency-draft/
    "ddc_r": 6,                           // reduction rate for coarse decoder.

from tts.

Thank you!

from tts.

@erogol Graves is working, something was off in my dataset and or config that's been solved and the training is yielding alignments after 5-10K steps. The inference issue is still there, I'll open a PR just deleting that one line mentioned above.

from tts.

After 43K steps the alignments are also still a bit wonky but not empty.

from tts.

Closing this because the no attribute bug was fixed in #479 and GMM (Graves) Attention will be looked at in a separate discussion.

from tts.