Thanks for the project ❤, I made a Colab. I hope you like it. about text2video-zero HOT 11 CLOSED

thank you for the contribution

from text2video-zero.

seeds are random oops, Hi @kolabearafk 👋 Hi @MaayanBogin 👋 Hi @toughyear 👋 thanks for the info ❤ maybe it's fixed now, please try it again.

from text2video-zero.

Thank you for the Colab version. But why aren't the results from this repo and the results from the colab not matching? The examples from the Colab version seem to be lacking the temporal consistency? Please see below for the example with the prompt "a horse is galloping on a street".

from text2video-zero.

Thank you for the Colab version. But why aren't the results from this repo and the results from the colab not matching? The examples from the Colab version seem to be lacking the temporal consistency? Please see below for the example with the prompt "a horse is galloping on a street".

same here

from text2video-zero.

+1 to @kolabearafk. There is no temporal consistency. I also see a Picsart watermark at the bottom right.

PROMPT: Pixel art of a dancing bearded fat farmer

from text2video-zero.

@camenduru trying. Will update.

from text2video-zero.

Slightly better results. The pose and depth seem to stay consistent. The art style keeps changing.

PROMPT: Pixel art of a dancing bearded fat farmer
Time on a Python 3 Google Colab Compute Engine backend (GPU): ~ 173 seconds

I assumed it might help to prompt a common, easily repeatable character. There are still temporal consistency issues -

PROMPT: Pixel art of a santa claus dancing, white background
Time on a Python 3 Google Colab Compute Engine backend (GPU): ~ X seconds

Not using Pixel art in prompt produces slightly better results -
I would attribute this to mainly having a popular character that has a single source of truth for its looks.

PROMPT: iron man dancing, white background
Time on a Python 3 Google Colab Compute Engine backend (GPU): ~ 89 seconds

Obviously, these are the current shortcomings of the model only. Your colab works great. ZeroShot text-to-video models will get better with time.

Let me know if I can help with anything @camenduru

from text2video-zero.

🔥 thanks @toughyear ❤ please join our discord server https://discord.gg/k5BwmmvJJU maybe we can make some little improvements together

from text2video-zero.

temporal consistency problem coming from xformers 😭 without xformers it is working, but then the model does not fit in the free T4 GPU 😭

from text2video-zero.

with custom model https://huggingface.co/plasmo/woolitize-768sd1-5

from text2video-zero.

joined @camenduru

from text2video-zero.