necessary of adversarial duration predictor about vits2_pytorch HOT 14 CLOSED

https://arxiv.org/pdf/2206.12132.pdf
Section 2.5

from vits2_pytorch.

Added adversarial duration predictor, try and let me know if any training errors.

from vits2_pytorch.

Yes, also I think we can modify code a little to train both dp and sdp together and compare performance simultaneously to save time.

from vits2_pytorch.

In my experiments, all instantance with DPD (Duration Predictor Discriminator) are failed to continue enough steps, they are stucked. And their syntheiszed results with the stucked checkpoint are no better then that without DPD.

like in the vits2 paper, our training pipeline is: keep training without DPD to about 700K or 800K steps, then continue training with DPD to about 30K steps [mostly stucked after thousands of steps]. we did not make special change to the learning rate when continue, that means, the DPD is trained with initial lr=0.0002, while other parts are continue with schedular-decayed lr on that steps.

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

"The dpd is very naive in my implementation" is that the reason why i training with DurationDiscriminator and got stuck at the first steps. And should we use it.

In my experience use_sdp didn’t stuck, maybe try lowering batch size? There also might be some other issues causing it

Are you training from scratch? or fine-tune based on a pretrained?

Yes. I'm training from scratch and got stuck. And should I apply training strategy as you mention above?

No, nothing difference. The paper did not tell the detail about the DPD at all. you can train without it.

from vits2_pytorch.

It may be that each has its advantages and disadvantages, and the DDP in VITS is slightly worse and may be better trained。

（from vits paper）

from vits2_pytorch.

Yes yes, we have the parts ready, I just have to put in train.py. Will do it by weekend. You can still train a nosdp model then transfer learn it into sdp. We can check the performance by our experiments. Also, with papers, the problem is they just give metrics for that particular dataset, sdp might not work good on other datasets; so it is quite variable.

from vits2_pytorch.

@p0p4k hey I have one question about the "transfer learn it into sdp..." if you don't mind 😃 do you mean just continue training with SDP using the checkpoints trained with DP?

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

"The dpd is very naive in my implementation" is that the reason why i training with DurationDiscriminator and got stuck at the first steps. And should we use it.

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

"The dpd is very naive in my implementation" is that the reason why i training with DurationDiscriminator and got stuck at the first steps. And should we use it.

In my experience use_sdp didn’t stuck, maybe try lowering batch size? There also might be some other issues causing it

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

"The dpd is very naive in my implementation" is that the reason why i training with DurationDiscriminator and got stuck at the first steps. And should we use it.

In my experience use_sdp didn’t stuck, maybe try lowering batch size? There also might be some other issues causing it

Are you training from scratch? or fine-tune based on a pretrained?

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

"The dpd is very naive in my implementation" is that the reason why i training with DurationDiscriminator and got stuck at the first steps. And should we use it.

In my experience use_sdp didn’t stuck, maybe try lowering batch size? There also might be some other issues causing it

Are you training from scratch? or fine-tune based on a pretrained?

Yes. I'm training from scratch and got stuck. And should I apply training strategy as you mention above?

from vits2_pytorch.

The dpd is very naive in my implementation and could use some more sophisticated model.

"The dpd is very naive in my implementation" is that the reason why i training with DurationDiscriminator and got stuck at the first steps. And should we use it.

In my experience use_sdp didn’t stuck, maybe try lowering batch size? There also might be some other issues causing it

Are you training from scratch? or fine-tune based on a pretrained?

Yes. I'm training from scratch and got stuck. And should I apply training strategy as you mention above?

No, nothing difference. The paper did not tell the detail about the DPD at all. you can train without it.

from vits2_pytorch.