Investigate the temporal consistency of Sora model

1.⁠ ⁠Data scouting: Finding >=15 seconds video including text input. Deadline: March 12

2.⁠ ⁠⁠Feature Extraction: Using 16x16 patching to extract video features to input DIT. Deadline: March 20th

3.⁠ ⁠⁠DIT training starting in March 20th.

Existing Repo we can use: open-sora, and they are using MSR-VTT dataset.

Additionally we can use DiT for the model architecture.

A simple method is to use same cropped sized videos to test the time consistency

[GD] For evaluation metrics: How to check consistency quantitatively.