loading dreambooth model fails. about text2video-zero HOT 4 CLOSED

oh. my bad. I'll fix and reopen if it breaks again. thanks

from text2video-zero.

HI @johndpope,

we have updated the README and code, regarding your issue.

from text2video-zero.

from pathlib import Path
import torch
from model import Model

model = Model(device = "cuda", dtype = torch.float16)
prompt = 'sexy asian gta-5 style'
video_path = Path('./crop.mp4')
dreambooth_model_path = Path("GTA-5 DB")
out_path = Path(f'./{prompt}.gif')
model.process_controlnet_canny_db(dreambooth_model_path.as_posix(), video_path.as_posix(), prompt=prompt, save_path=out_path.as_posix())%      

still blows up

/home/oem/Documents/gitWorkspace/Text2Video-Zero/annotator/openpose/body.py:5: DeprecationWarning: Please use gaussian_filter from the scipy.ndimage namespace, the scipy.ndimage.filters namespace is deprecated.
from scipy.ndimage.filters import gaussian_filter
/home/oem/Documents/gitWorkspace/Text2Video-Zero/annotator/openpose/hand.py:6: DeprecationWarning: Please use gaussian_filter from the scipy.ndimage namespace, the scipy.ndimage.filters namespace is deprecated.
from scipy.ndimage.filters import gaussian_filter
/home/oem/miniconda3/envs/ldm/lib/python3.10/site-packages/skimage/util/dtype.py:27: DeprecationWarning: np.bool8 is a deprecated alias for np.bool_. (Deprecated NumPy 1.24)
np.bool8: (False, True),
cuda
cuda
Traceback (most recent call last):
File "/home/oem/Documents/gitWorkspace/Text2Video-Zero/test.py", line 10, in
model.process_controlnet_canny_db(dreambooth_model_path.as_posix(), video_path.as_posix(), prompt=prompt, save_path=out_path.as_posix())
File "/home/oem/Documents/gitWorkspace/Text2Video-Zero/model.py", line 247, in process_controlnet_canny_db
video_path = gradio_utils.get_video_from_canny_selection(video_path)
File "/home/oem/Documents/gitWorkspace/Text2Video-Zero/gradio_utils.py", line 53, in get_video_from_canny_selection
raise Exception
Exception

the readme says
dreambooth_model_path = "GTA-5 DB"
but it won't run / crashes with as_posix error
dreambooth_model_path = Path("GTA-5 DB")
this gets around this crash.

from text2video-zero.

Hi @johndpope,

from the error message I can see that you did not update the code.
With the latest code you can do as written in the documentation:

import torch
from model import Model

model = Model(device = "cuda", dtype = torch.float16)
prompt = 'woman gta-5 style'
video_path = "woman1"
dreambooth_model_path = "GTA-5 DB"
out_path = f'./{prompt}.gif'
model.process_controlnet_canny_db(dreambooth_model_path, video_path, prompt=prompt, save_path=out_path)%   

You can replace video_path = "woman1" with your custom video, e.g. video_path = "./crop.mp4"

from text2video-zero.