![](assets/logo_long.png)
Jinbo Xing, Menghan Xia*, Yong Zhang, Haoxin Chen, Wangbo Yu,
Hanyuan Liu, Xintao Wang, Tien-Tsin Wong*, Ying Shan
(* corresponding authors)
From CUHK and Tencent AI Lab.
🤗 DynamiCrafter can animate open-domain still images based on text prompt by leveraging the pre-trained video diffusion priors. Please check our project page and paper for more information.
😀 We will continue to improve the model's performance, which includes offering higher resolution, eliminating watermarks, and enhancing stability.
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Input starting frame | Input ending frame | Generated video |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
- [2023.12.02]: 🔥🔥 Launch the local Gradio demo.
- [2023.11.29]: 🔥🔥 Release the main model at a resolution of 256x256.
- [2023.11.27]: 🔥🔥 Launch the project page and update the arXiv preprint.
Model | Resolution | Checkpoint |
---|---|---|
DynamiCrafter256 | 256x256 | Hugging Face |
It takes approximately 10 seconds and requires a peak GPU memory of 20 GB to animate an image using a single NVIDIA A100 (40G) GPU.
conda create -n dynamicrafter python=3.8.5
conda activate dynamicrafter
pip install -r requirements.txt
- Download pretrained models via Hugging Face, and put the
model.ckpt
incheckpoints/dynamicrafter_256_v1/model.ckpt
. - Run the commands based on your devices and needs in terminal.
# Run on a single GPU:
sh scripts/run.sh
# Run on multiple GPUs for parallel inference:
sh scripts/run_mp.sh
- Download the pretrained models and put them in the corresponding directory according to the previous guidelines.
- Input the following commands in terminal.
python gradio_app.py
VideoCrafter1: Framework for high-quality video generation.
ScaleCrafter: Tuning-free method for high-resolution image/video generation.
TaleCrafter: An interactive story visualization tool that supports multiple characters.
LongerCrafter: Tuning-free method for longer high-quality video generation.
MakeYourVideo, might be a Crafter:): Video generation/editing with textual and structural guidance.
StyleCrafter: Stylized-image-guided text-to-image and text-to-video generation.
@article{xing2023dynamicrafter,
title={DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors},
author={Xing, Jinbo and Xia, Menghan and Zhang, Yong and Chen, Haoxin and Yu, Wangbo and Liu, Hanyuan and Wang, Xintao and Wong, Tien-Tsin and Shan, Ying},
journal={arXiv preprint arXiv:2310.12190},
year={2023}
}
We would like to thank AK(@_akhaliq) for the help of setting up online demo.
We develop this repository for RESEARCH purposes, so it can only be used for personal/research/non-commercial purposes.