levihsu / ootdiffusion Goto Github PK
View Code? Open in Web Editor NEWOfficial implementation of OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on
License: Other
Official implementation of OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on
License: Other
This is great job, do you have plans to release model's checkpoints soon?
(ootd) root@autodl-container-cf29119c3c-0ecee78c:~/autodl-tmp/OOTDiffusion/run# python gradio_ootd.py
Traceback (most recent call last):
File "/root/autodl-tmp/OOTDiffusion/run/gradio_ootd.py", line 21, in
from ootd.inference_ootd_hd import OOTDiffusionHD
File "/root/autodl-tmp/OOTDiffusion/ootd/inference_ootd_hd.py", line 24, in
from pipelines_ootd.pipeline_ootd import OotdPipeline
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/pipeline_ootd.py", line 27, in
from .unet_vton_2d_condition import UNetVton2DConditionModel
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_condition.py", line 23, in
from .unet_vton_2d_blocks import (
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_blocks.py", line 23, in
from .transformer_vton_2d import Transformer2DModel
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/transformer_vton_2d.py", line 29, in
from diffusers.models.embeddings import CaptionProjection, PatchEmbed
ImportError: cannot import name 'CaptionProjection' from 'diffusers.models.embeddings' (/root/miniconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/embeddings.py)
This project is really impressive. where is the DC model ?
将"preprocess\humanparsing\modules\functions.py"中的内容替换如下,可以跑起来。
from os import path
import torch.distributed as dist
import torch.autograd as autograd
_src_path = path.join(path.dirname(path.abspath(__file__)), "src")
# Activation names
ACT_RELU = "relu"
ACT_LEAKY_RELU = "leaky_relu"
ACT_ELU = "elu"
ACT_NONE = "none"
def normalize_shape(x):
if x.ndimension() == 1:
return x.view(1, -1, 1)
else:
return x.view(x.size(0), x.size(1), -1)
import torch
def broadcast_to(v, x):
if x.ndimension() == 2:
return v
else:
broadcast_size = [1, -1]
for i in range(2, x.ndimension()):
broadcast_size.append(1)
return v.view(broadcast_size)
def forward_cpu(x, mean, var, weight, bias, affine, eps):
gamma = torch.abs(weight) + eps if affine else torch.ones_like(var)
mul = torch.rsqrt(var + eps) * gamma
x.sub_(broadcast_to(mean, x))
x.mul_(broadcast_to(mul, x))
if affine:
x.add_(broadcast_to(bias, x))
return x
def _check(fn, *args, **kwargs):
success = fn(*args, **kwargs)
if not success:
raise RuntimeError("CUDA Error encountered in {}".format(fn))
def _reduce(x):
if len(x.size()) == 2:
return x.sum(dim=0)
else:
n, c = x.size()[0:2]
return x.contiguous().view((n, c, -1)).sum(2).sum(0)
def _count_samples(x):
count = 1
for i, s in enumerate(x.size()):
if i != 1:
count *= s
return count
def leaky_relu_forward(z, slope):
return torch.nn.functional.leaky_relu(z, negative_slope=slope)
def _act_forward(ctx, x):
if ctx.activation == ACT_LEAKY_RELU:
leaky_relu_forward(x, ctx.slope)
class InPlaceABN(autograd.Function):
@staticmethod
def forward(ctx, x, weight, bias, running_mean, running_var,
training=False, momentum=0.1, eps=1e-05, activation=ACT_LEAKY_RELU, slope=0.01):
# Save context
ctx.training = training
ctx.momentum = momentum
ctx.eps = eps
ctx.activation = activation
ctx.slope = slope
ctx.affine = weight is not None and bias is not None
# Prepare inputs
count = _count_samples(x)
x = x.contiguous()
weight = weight.contiguous() if ctx.affine else x.new_empty(0)
bias = bias.contiguous() if ctx.affine else x.new_empty(0)
mean, var = running_mean.contiguous(), running_var.contiguous()
ctx.mark_dirty(x)
# BN forward + activation
forward_cpu(x, mean, var, weight, bias, ctx.affine,ctx.eps)
_act_forward(ctx, x)
# Output
ctx.var = var
ctx.save_for_backward(x, var, weight, bias)
ctx.mark_non_differentiable(running_mean, running_var)
return x, running_mean, running_var
class InPlaceABNSync(autograd.Function):
@classmethod
def forward(cls, ctx, x, weight, bias, running_mean, running_var,
training=False, momentum=0.1, eps=1e-05, activation=ACT_LEAKY_RELU, slope=0.01, equal_batches=True):
# Save context
ctx.training = training
ctx.momentum = momentum
ctx.eps = eps
ctx.activation = activation
ctx.slope = slope
ctx.affine = weight is not None and bias is not None
# Prepare inputs
ctx.world_size = dist.get_world_size() if dist.is_initialized() else 1
# count = _count_samples(x)
batch_size = x.new_tensor([x.shape[0]], dtype=torch.long)
x = x.contiguous()
weight = weight.contiguous() if ctx.affine else x.new_empty(0)
bias = bias.contiguous() if ctx.affine else x.new_empty(0)
mean, var = running_mean.contiguous(), running_var.contiguous()
ctx.mark_dirty(x)
# BN forward + activation
forward_cpu(x, mean, var, weight, bias, ctx.affine,ctx.eps)
_act_forward(ctx, x)
# Output
ctx.var = var
ctx.save_for_backward(x, var, weight, bias)
ctx.mark_non_differentiable(running_mean, running_var)
return x, running_mean, running_var
inplace_abn = InPlaceABN.apply
inplace_abn_sync = InPlaceABNSync.apply
__all__ = ["inplace_abn", "inplace_abn_sync", "ACT_RELU", "ACT_LEAKY_RELU", "ACT_ELU", "ACT_NONE"]
但,衣服覆盖的不全,这是为什么呢。
forward_cpu代码是"\preprocess\humanparsing\modules\src\inplace_abn_cpu.cpp"中的C++代码转写成python的
系统版本:Win10
显卡:4090
报错一:
subprocess.CalledProcessError: Command ‘[where, cl]‘ returned non-zero exit status 1
解决方案:https://zhuanlan.zhihu.com/p/582856277
报错二:
ninja: build stopped: subcommand failed.
解决方案:
Not run in Windows
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1'
I encountered this issue saying skimage cannot be found.
python run_ootd.py --model_path <examples/model> --cloth_path <examples/garment> --model_type dc --category 2 --scale 2.0 --sample 4
cd: no such file or directory: OOTDiffusion/run
Traceback (most recent call last):
File "/Users/it6/ootd/run/run_ootd.py", line 9, in <module>
from preprocess.openpose.run_openpose import OpenPose
File "/Users/it6/ootd/preprocess/openpose/run_openpose.py", line 20, in <module>
from preprocess.openpose.annotator.openpose import OpenposeDetector
File "/Users/it6/ootd/preprocess/openpose/annotator/openpose/__init__.py", line 16, in <module>
from .hand import Hand
File "/Users/it6/ootd/preprocess/openpose/annotator/openpose/hand.py", line 10, in <module>
from skimage.measure import label
ModuleNotFoundError: No module named 'skimage'
So I use pip install skimage
and received below errors:
Collecting skimage
Downloading skimage-0.0.tar.gz (757 bytes)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [3 lines of output]
*** Please install the `scikit-image` package (instead of `skimage`) ***
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
From here I'm in a deadloop.
Traceback (most recent call last):
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
subprocess.run(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Data2/wyw/OOTDiffusion/run/run_ootd.py", line 10, in
from preprocess.humanparsing.aigc_run_parsing import Parsing
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/aigc_run_parsing.py", line 7, in
from parsing_api import load_atr_model, load_lip_model, inference
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/parsing_api.py", line 11, in
import networks
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/networks/init.py", line 2, in
from networks.AugmentCE2P import resnet101
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/networks/AugmentCE2P.py", line 22, in
from modules import InPlaceABNSync
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/init.py", line 1, in
from .bn import ABN, InPlaceABN, InPlaceABNSync
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/bn.py", line 10, in
from .functions import *
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/functions.py", line 11, in
backend = load(name="inplace_abn",
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1284, in load
return jit_compile(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1509, in jit_compile
write_ninja_file_and_build_library(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1624, in write_ninja_file_and_build_library
run_ninja_build(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'inplace_abn': [1/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda.cu -o inplace_abn_cuda.cuda.o
FAILED: inplace_abn_cuda.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda.cu -o inplace_abn_cuda.cuda.o
nvcc fatal : Value 'c++17' is not defined for option 'std'
[2/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda_half.cu -o inplace_abn_cuda_half.cuda.o
FAILED: inplace_abn_cuda_half.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda_half.cu -o inplace_abn_cuda_half.cuda.o
nvcc fatal : Value 'c++17' is not defined for option 'std'
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/mnt/e/project/80297283/OOTDiffusion/run/gradio_ootd.py", line 29, in <module>
openpose_model_dc = OpenPose(1)
File "/mnt/e/project/80297283/OOTDiffusion/preprocess/openpose/run_openpose.py", line 32, in __init__
torch.cuda.set_device(gpu_id)
File "/home/pub_gpu/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/cuda/__init__.py", line 350, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
how do i fix it?
ps: who run the gradio demo successfully?
(ootd) E:\software\OOTDiffusion\run>python run_ootd.py --model_path E:\software\OOTDiffusion\run\examples\model --cloth_path E:\software\OOTDiffusion\run\examples\garment --scale 2.0 --sample 4
Traceback (most recent call last):
File "E:\software\OOTDiffusion\run\run_ootd.py", line 9, in
from preprocess.openpose.run_openpose import OpenPose
File "E:\software\OOTDiffusion\preprocess\openpose\run_openpose.py", line 3, in
import config
ModuleNotFoundError: No module named 'config'
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
File "/root/autodl-tmp/OOTDiffusion/run/run_ootd.py", line 47, in
model = OOTDiffusionHD(args.gpu_id)
File "/root/autodl-tmp/OOTDiffusion/ootd/inference_ootd_hd.py", line 61, in init
unet_vton = UNetVton2DConditionModel.from_pretrained(
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/root/autodl-tmp/OOTDiffusion/diffusers/src/diffusers/models/modeling_utils.py", line 569, in from_pretrained
config, unused_kwargs, commit_hash = cls.load_config(
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/root/autodl-tmp/OOTDiffusion/diffusers/src/diffusers/configuration_utils.py", line 370, in load_config
raise EnvironmentError(
OSError: Error no file named config.json found in directory ../checkpoints/ootd/ootd_hd/checkpoint-36000.
Comfyui workflow?
伟大的项目,但目前环境搭建困难,所以研读修改代码,搞出来了整合包
不用搭建环境(包括CUDA、ninja编译、VS C++编译等都不用),只要英伟达显卡就行,解压即用!
https://github.com/zhaoyun0071/OOTDiffusion-windows-GUI
ValueError: Trying to set a tensor of shape torch.Size([49408, 768]) in "weight" (which has shape torch.Size([49408, 512])), this look incorrect.
请问大佬这个是什么问题呢
Traceback (most recent call last):
File "E:\PyCharmProject\OOTDiffusion\run\run_ootd.py", line 10, in
from preprocess.humanparsing.aigc_run_parsing import Parsing
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\aigc_run_parsing.py", line 7, in
from parsing_api import load_atr_model, load_lip_model, inference
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\parsing_api.py", line 11, in
import networks
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\networks_init_.py", line 2, in
from networks.AugmentCE2P import resnet101
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\networks\AugmentCE2P.py", line 22, in
from modules import InPlaceABNSync
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\modules_init_.py", line 1, in
from .bn import ABN, InPlaceABN, InPlaceABNSync
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\modules\bn.py", line 10, in
from .functions import *
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\modules\functions.py", line 11, in
_backend = load(name="inplace_abn",
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1284, in load
return _jit_compile(
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1509, in _jit_compile
_write_ninja_file_and_build_library(
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1593, in _write_ninja_file_and_build_library
verify_ninja_availability()
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1649, in verify_ninja_availability
raise RuntimeError("Ninja is required to load C++ extensions")
RuntimeError: Ninja is required to load C++ extensions
File "/root/OOTDiffusion/run/run_ootd.py", line 47, in
model = OOTDiffusionHD(args.gpu_id)
File "/root/OOTDiffusion/ootd/inference_ootd_hd.py", line 57, in init
unet_vton = UNetVton2DConditionModel.from_pretrained(
File "/root/anaconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 812, in from_pretrained
unexpected_keys = load_model_dict_into_meta(
File "/root/anaconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 155, in load_model_dict_into_meta
raise ValueError(
ValueError: Cannot load ../checkpoints/ootd/ootd_hd/checkpoint-36000 because conv_in.weight expected shape tensor(..., device='meta', size=(320, 8, 3, 3)), but got torch.Size([320, 4, 3, 3]). If you want to instead overwrite randomly initialized weights, please make sure to pass both low_cpu_mem_usage=False
and ignore_mismatched_sizes=True
.
(ootd) E:\software\OOTDiffusion\run>python run_ootd.py --model_path E:\software\OOTDiffusion\run\examples\model\ --cloth_path E:\software\OOTDiffusion\run\examples\garment\ --scale 2.0 --sample 4
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3'
C:\Users\Administrator\pinokio\bin\miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py:359: UserWarning: Error checking compiler version for cl: [WinError 2] 系统找不到指定的文件。
where to get this file?
您好,我想要把这个ootdfusion给迁移到comfyui里,有一些问题想要咨询您,为了方便联系,请问能否方便先留个邮箱的联系方式给我呢,或者可以通过[email protected] 或者[email protected]电邮我~ 辛苦老师
python run_ootd.py --model_path /root/OOTDiffusion/run/examples/model/01008_00.jpg --cloth_path /root/OOTDiffusion/run/examples/garment/00055_00.jpg --scale 2.0 --sample 4
-:
Traceback (most recent call last):
File "/root/OOTDiffusion/run/run_ootd.py", line 29, in
openpose_model = OpenPose(args.gpu_id)
File "/root/OOTDiffusion/preprocess/openpose/run_openpose.py", line 33, in init
self.preprocessor = OpenposeDetector()
File "/root/OOTDiffusion/preprocess/openpose/annotator/openpose/init.py", line 63, in init
self.body_estimation = Body(body_modelpath)
File "/root/OOTDiffusion/preprocess/openpose/annotator/openpose/body.py", line 26, in init
model_dict = util.transfer(self.model, torch.load(model_path))
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/serialization.py", line 815, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x05'.
thanks for your amazing work! could you please open dc model and train code? thanks again!
(ootd) root@autodl-container-cf29119c3c-0ecee78c:~/autodl-tmp/OOTDiffusion/run# python gradio_ootd.py
Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00, 6.08it/s]
text_config_dict
is provided which will be used to initialize CLIPTextConfig
. The value text_config["id2label"]
will be overriden.
text_config_dict
is provided which will be used to initialize CLIPTextConfig
. The value text_config["bos_token_id"]
will be overriden.
text_config_dict
is provided which will be used to initialize CLIPTextConfig
. The value text_config["eos_token_id"]
will be overriden.
Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 11.18it/s]
Traceback (most recent call last):
File "/root/autodl-tmp/OOTDiffusion/run/gradio_ootd.py", line 31, in
ootd_model_dc = OOTDiffusionDC(1)
File "/root/autodl-tmp/OOTDiffusion/ootd/inference_ootd_dc.py", line 74, in init
).to(self.gpu_id)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 864, in to
module.to(device, dtype)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1145, in to
return self._apply(convert)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 820, in _apply
param_applied = fn(param)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA
to enable device-side assertions.
Should update the following.
The following files are required in checkpoints/humanparsing
exp-schp-201908301523-atr.pth
exp-schp-201908261155-lip.pth
exp-schp-201908270938-pascal-person-part.pth
Program modification of degradations.py is required
・Instead of from torchvision.transforms.functional_tensor import rgb_to_grayscale
・from torchvision.transforms.functional import rgb_to_grayscale
Otherwise, Module not found will occur.
Does that mean Silicon M2 is not supported yet? Just gone through all the way here.
(ootd) D:\OOTDiffusion\OOTDiffusion\run>python run_ootd.py --model_path D:\OOTDiffusion\checkpoints\ootd --cloth_path D:\OOTDiffusion\images\demo.png --scale 2.0 --sample 4
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00, 16.35it/s]
Traceback (most recent call last):
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\utils\hub.py", line 389, in cached_file
resolved_file = hf_hub_download(
File "D:\miniconda38\envs\ootd\lib\site-packages\huggingface_hub\utils_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "D:\miniconda38\envs\ootd\lib\site-packages\huggingface_hub\utils_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '../checkpoints/clip-vit-large-patch14'. Use repo_type
argument if needed.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\OOTDiffusion\OOTDiffusion\run\run_ootd.py", line 47, in
model = OOTDiffusionHD(args.gpu_id)
File "D:\OOTDiffusion\OOTDiffusion\ootd\inference_ootd_hd.py", line 78, in init
self.auto_processor = AutoProcessor.from_pretrained(VIT_PATH)
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\models\auto\processing_auto.py", line 234, in from_pretrained
preprocessor_config_file = get_file_from_repo(
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\utils\hub.py", line 541, in get_file_from_repo
return cached_file(
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\utils\hub.py", line 454, in cached_file
raise EnvironmentError(
OSError: Incorrect path_or_model_id: '../checkpoints/clip-vit-large-patch14'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
(ootd) D:\OOTDiffusion\OOTDiffusion\run>
I can't find any about clip-vit-large-patch14 model in your huggingface file. So I download the openai file. but it does't work.
How can I solve this problem?
File "/root/OOTDiffusion/run/run_ootd.py", line 11, in
from ootd.inference_ootd_hd import OOTDiffusionHD
File "/root/OOTDiffusion/ootd/inference_ootd_hd.py", line 24, in
from pipelines_ootd.pipeline_ootd import OotdPipeline
File "/root/OOTDiffusion/ootd/pipelines_ootd/pipeline_ootd.py", line 27, in
from .unet_vton_2d_condition import UNetVton2DConditionModel
File "/root/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_condition.py", line 23, in
from .unet_vton_2d_blocks import (
File "/root/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_blocks.py", line 23, in
from .transformer_vton_2d import Transformer2DModel
File "/root/OOTDiffusion/ootd/pipelines_ootd/transformer_vton_2d.py", line 29, in
from diffusers.models.embeddings import CaptionProjection, PatchEmbed
ImportError: cannot import name 'CaptionProjection' from 'diffusers.models.embeddings' (/root/anaconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/embeddings.py)
how to solve it?
Hello! I hope everyone is well! Firstly, I would like to thank you for making this incredible work available.
I'm doing some tests, and I've noticed some hallucinations. Apparently, the model reconstructs the clothing print, causing some differences in the final result.
I tried adjusting the parameters, but I couldn't resolve it.
Is there any way to get around the problem?
I know that the model must be in the early stages, and there is still a lot to be done, but I would like to know if I can do anything about it right now.
Thank you for your attention!
i try below both failed, anyone point right way
python run_ootd.py --model_path "/root/autodl-tmp/OOTDiffusion/run/examples/model/" --cloth_path "/root/autodl-tmp/OOTDiffusion/run/examples/garment/" --scale 2.0 --sample 4
python run_ootd.py --model_path "./examples/model" --cloth_path "./examples/garment" --scale 2.0 --sample 4
(ootd) E:\software\OOTDiffusion\run>python run_ootd.py --model_path E:\software\OOTDiffusion\run\examples\model\model_1.png --cloth_path E:\software\OOTDiffusion\run\examples\garment\02305_00.jpg --scale 2.0 --sample 4
Traceback (most recent call last):
File "E:\software\OOTDiffusion\run\run_ootd.py", line 9, in
from preprocess.openpose.run_openpose import OpenPose
File "E:\software\OOTDiffusion\preprocess\openpose\run_openpose.py", line 20, in
from preprocess.openpose.annotator.openpose import OpenposeDetector
File "E:\software\OOTDiffusion\preprocess\openpose\annotator\openpose_init_.py", line 15, in
from .body import Body
File "E:\software\OOTDiffusion\preprocess\openpose\annotator\openpose\body.py", line 14, in
from torchvision import transforms
File "C:\Users\Administrator\pinokio\bin\miniconda\envs\ootd\lib\site-packages\torchvision_init_.py", line 6, in
from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
File "C:\Users\Administrator\pinokio\bin\miniconda\envs\ootd\lib\site-packages\torchvision_meta_registrations.py", line 4, in
import torch._custom_ops
ModuleNotFoundError: No module named 'torch._custom_ops'
(ootd) E:\software\OOTDiffusion\run>
![image](https://github.com/levihsu/OOTDiffusion/assets/33121696/63351d0e-1f47-40fb-9bfe-9a1cfa4182e2)
I also downloaded the model of openai, my directory structure is like this, execute the following command 'python run_ootd.py --model_path /home/ysm/tmmt/OOTDiffusion/run/examples/model/01008_00.jpg --cloth_path /home/ysm/tmmt/OOTDiffusion/run/examples/garment/00055_00.jpg --scale 2.0 --sample 4', and it works fine
Originally posted by @nkTmmT in #29 (comment)
I want to create my own dataset to train a model, could you provide the training code?
python run_ootd.py --model_path ../assets/m1.png --cloth_path ../assets/t1.png --scale 2.0 --sample 4
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.75 GiB (GPU 0; 23.68 GiB total capacity; 20.97 GiB already allocated; 315.50 MiB free; 21.17 GiB reserved in total by PyTorch)
Can't run with a 24gb memory device?
Getting below error:
Traceback (most recent call last):
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/run/run_ootd.py", line 10, in
from preprocess.humanparsing.aigc_run_parsing import Parsing
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/aigc_run_parsing.py", line 7, in
from parsing_api import load_atr_model, load_lip_model, inference
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/parsing_api.py", line 11, in
import networks
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/networks/init.py", line 2, in
from networks.AugmentCE2P import resnet101
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/networks/AugmentCE2P.py", line 22, in
from modules import InPlaceABNSync
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/modules/init.py", line 1, in
from .bn import ABN, InPlaceABN, InPlaceABNSync
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/modules/bn.py", line 10, in
from .functions import *
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/modules/functions.py", line 8, in
from torch.utils.cpp_extension import load
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 19, in
from .hipify import hipify_python
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/site-packages/torch/utils/hipify/hipify_python.py", line 34, in
from .cuda_to_hip_mappings import CUDA_TO_HIP_MAPPINGS
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/site-packages/torch/utils/hipify/cuda_to_hip_mappings.py", line 34, in
rocm_path = subprocess.check_output(["hipconfig", "--rocmpath"]).decode("utf-8")
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 966, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
NotADirectoryError: [Errno 20] Not a directory: 'hipconfig'
找不到config文件
(Outfit) D:\Python310\Outfit\OOTDiffusion\run>python run_ootd.py --model_path examples\model --cloth_path examples\garment --scale 2.0 --sample 4
Traceback (most recent call last):
File "D:\Python310\Outfit\OOTDiffusion\run\run_ootd.py", line 9, in
from preprocess.openpose.run_openpose import OpenPose
File "D:\Python310\Outfit\OOTDiffusion\preprocess\openpose\run_openpose.py", line 3, in
import config
ModuleNotFoundError: No module named 'config'
Not sure where to put it, but it's a bit of a hassle to download the models correctly.
You can use pip install git+https://github.com/viktorfa/oot_diffusion.git
and use OOTD in any python code.
it seems like only the ootd_hd upper body model has been released,when and how can i get the full body ootd_dc model?i appreciate for your hard work
Really impressive results, thanks for sharing.
I compared the weights of the unet with SD1.5 and if I am not mistaken the whole net was fine-tuned. What training tricks were used to avoid overfiting on such a small dataset like viton-hd?
找不到说明
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.