Model training too long without alignment. about flowtron HOT 4 OPEN

• can the second flow be on same data. If so whats the advantage of doing more flows if the first produces good alignment?
  Yes and they should be trained on the same data. Doing more flows gives the model more capacity and can produce better speech/

• in how many iterations can I expect the model to produce good alignment?
  In less than 24 hours on a NVIDIA V100 with batch size 1.

• if the loss plateus for a long time and the aligment is not good what to do?
  If you're using a single step of flow, use it to warm start a 2-step of flow model

• is validation loss representative of the aligment? and what validation loss did you achieve?
  Not necessarily, look at the alignment itself.

• if you are trying to get good on a single speaker only is it better to use tacotron2 instead, or flowtron adds some stability with more speakers? (Please answer this one)
  Flowtron is better than Tacotron 2.

from flowtron.

Is this the correct config.json/command for second flow?

{
"train_config": {
"output_directory": "outdir",
"epochs": 10000000,
"learning_rate": 1e-4,
"weight_decay": 1e-6,
"sigma": 1.0,
"iters_per_checkpoint": 5000,
"batch_size": 1,
"seed": 1234,
"checkpoint_path": "",
"ignore_layers": [],
"include_layers": [],
"warmstart_checkpoint_path": "",
"with_tensorboard": true,
"fp16_run": false
},
"data_config": {
"training_files": "data/processed/combined/dataset.train",
"validation_files": "data/processed/combined/dataset.test",
"text_cleaners": ["flowtron_cleaners"],
"p_arpabet": 0.5,
"cmudict_path": "data/cmudict_dictionary",
"sampling_rate": 22050,
"filter_length": 1024,
"hop_length": 256,
"win_length": 1024,
"mel_fmin": 0.0,
"mel_fmax": 8000.0,
"max_wav_value": 32768.0
},
"dist_config": {
"dist_backend": "nccl",
"dist_url": "tcp://localhost:54321"
},
"model_config": {
"n_speakers": 3,
"n_speaker_dim": 128,
"n_text": 185,
"n_text_dim": 512,
"n_flows": 2,
"n_mel_channels": 80,
"n_attn_channels": 640,
"n_hidden": 1024,
"n_lstm_layers": 2,
"mel_encoder_n_hidden": 512,
"n_components": 0,
"mean_scale": 0.0,
"fixed_gaussian": true,
"dummy_speaker_embedding": false,
"use_gate_layer": true
},
}

python -m torch.distributed.launch --use_env --nproc_per_node=4 train.py -c config.json -p train_config.output_directory=outdir train_config.warm_checkpoint_path="/workspace/models/model_1605000.pt" train_config.fp16=true

from flowtron.

Yes, this should work.

from flowtron.

Here are the results after 560,000 iterations for flow 2, no alignment

from flowtron.