endo-yuki-t / animating-landscape Goto Github PK

Hello,

the site that's referenced by the paper is inaccesable anymore.

Thanks you for this very interesting repo.
I got some issues with training though: the parameters (codebook values for motion and total_loss for appearance) drop to NaN within few epochs (5-20) when trained with default hyperparameters.
Increase in learning rate to 0.5~1e-4 eliminates this behaviour, but doesn't look like a solution.
Did you encounter such issues in your practice and do you have any advice to sort this out?

During training of the appearance part I've encountered rather long delays (15-20 min) with calculation of the codebook values - obviously because of the amount of frames in the training set.
In the published code the codebook is calculated for every frame (in my case 30 videos for 30004000 frames each). On the contrary, the codebook_a in the published model counts for just 125 arrays, only dozens in length each - despite you mentioned ~2000 videos with hundreds of frames in each in your dataset.
Could you please clarify how should this codebook be properly constructed?

VIDEOIO ERROR: V4L: can't open camera by index 0

As the paper said, The dataset was divided into 1,825 video clips for training and 224 clips for testing at the resolution of 640 × 360. and The resultant video clips contain 227 frames on average.

That means 1825 video clips for training and the average frames is 227 about these 1825 clips, and each video clip the frames is consecutive, right？