Questions for align_68.py about talking-face_pc-avs HOT 4 OPEN

Thanks for your prompt reply, it's really helpful.
Sorry for bring up another question. Is it possible to share the training hyperparameters and the schedulers (I only found the optimizers in the repo) of all the models? (Batch size, learning rate,.. etc). The hyperparameters and the schedulers would be super helpful if I want to train the model based on the repo. Thanks!

from talking-face_pc-avs.

Hi, thank you for looking into the details of the code! Most previous work for facial image generation normally align each face according to the three points (As I have done in my previous paper https://github.com/Hangz-nju-cuhk/Talking-Face-Generation-DAVS). However, this will lead to a zoom-out-and-in artifact due to the affine transformation.

Thus in this work, I do not align the samples provided in VoxCeleb2 and choose to align the faces according to the average of all key points in other videos. The bias is to ensure that the face is almost at the center of the cropped frame but certain misalignment is allowed.

from talking-face_pc-avs.

Thank you for your reply! Your reply answered my question!

Sorry for another question about the training details. Since no preprocessed code is released. I have a question regarding it.
During the training stage, I'm wondering did you train the entire clip or did you sample a few number of frames in a clip and each time you random sample a pair of frames as the training data? Thanks.

from talking-face_pc-avs.

Hi, for each epoch we sample like 12 continuous frames for contrastive learning and among them, 4 are used for reconstruction training.

from talking-face_pc-avs.