How to shut it up? about wav2lip HOT 6 CLOSED

Did you try once with the non-GAN model? It performs better on silences

from wav2lip.

Did you try once with the non-GAN model? It performs better on silences

I'm not familiar with that. How can I do that in colab?

from wav2lip.

Just download the corresponding checkpoint and pass it as an argument in the same inference command.

from wav2lip.

Just a suggestion...perhaps to improve performance on silence you can add some simulated data to the training data set, where:

1. The audio inputs would be silence or add some low DB noise to it (to avoid Mel bands being NaN)
2. The corresponding video could be created with an image of any random person with a neutral expression (of course the length's of the audio and video should match)
3. Train the model with this extra data along with the LSR2 dataset specified in the README. Or you could fine tune the model with the pre-trained weights.
4. Run inference with new model.

PS: this is not something I have tried yet, but if this is a limitation of the existing model, this could be one way of improving it, I believe
Would you agree @prajwalkr ?

from wav2lip.

Maybe it can help. Maybe using some short range temporal info in the generator might help even better. We are yet to experiment more on this.

from wav2lip.

Re-open if needed.

from wav2lip.