ianyeung / mgld-vsr Goto Github PK

View Code? Open in Web Editor NEW

55.0 55.0 0.0 21.4 MB

Code for Arxiv Paper "Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution"

License: Other

Python 100.00%

mgld-vsr's People

Contributors

Stargazers

Watchers

Forkers

zhang-jing-xuan

mgld-vsr's Issues

test error

test cmd:
python scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py --config configs/mgldvsr/mgldvsr_512_realbasicvsr_deg.yaml --ckpt experiment/MGLD-VSR-Pretrained-Models/mgldvsr_unet.ckpt --vqgan_ckpt experiment/MGLD-VSR-Pretrained-Models/video_vae_cfw.ckpt --seqs-path /test/SR/png/val/ --outdir results --ddpm_steps 50 --dec_w 1.0 --colorfix_type adain --select_idx 0 --n_gpus 1

----------------------- Error1
Traceback (most recent call last):
File "scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 20, in
from ldm.models.diffusion.ddim import DDIMSampler
ModuleNotFoundError: No module named 'ldm.models'

----------------------- Error2
New error message appears after modifying the code
Traceback (most recent call last):
File "scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 28, in
from basicsr.archs.arch_util import resize_flow
ModuleNotFoundError: No module named 'basicsr'

I guess the environment was not set up properly, but I don't know how to modify it. Do you have any suggestions

where download flownets/raft-things.pth ?

AssertionError: An object named 'GANLoss' was already registered in 'loss' registry!

OS  : win11-wsl2  cuda-12.1
GPU : RTX4070ti  12GB

AssertionError: An object named 'GANLoss' was already registered in 'loss' registry!

(cuda121) root@LZH5:/mnt/d/AI/A0J/231220/wsl_cuda121# python scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py \
  --config configs/mgldvsr/mgldvsr_512_realbasicvsr_deg.yaml \
  --ckpt       /mnt/d/Data/ckpt_pth/mgldvsr_unet.ckpt \
  --vqgan_ckpt /mnt/d/Data/ckpt_pth/video_vae_cfw.ckpt \
  --seqs-path input/TaiDi \
  --outdir output/TaiDi \
  --ddpm_steps 50 \
  --dec_w 1.0 \
  --colorfix_type adain \
  --select_idx 0 \
  --n_gpus 1
/opt/conda/envs/cuda121/lib/python3.9/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
Global seed set to 42
>>>>>>>>>>color correction>>>>>>>>>>>
Use adain color correction
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Loading model from /mnt/d/Data/ckpt_pth/mgldvsr_unet.ckpt
Global Step: 42000
/opt/conda/envs/cuda121/lib/python3.9/site-packages/pytorch_lightning/utilities/distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
LatentDiffusionVSRTextWT: Running in eps-prediction mode

Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads.
...
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads.

DiffusionWrapper has 935.32 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 64, 64) = 16384 dimensions.
making attention of type 'vanilla' with 512 in_channels
Encoder Restored from /mnt/d/Data/ckpt_pth/v2-1_512-ema-pruned.ckpt with 0 missing and 1242 unexpected keys
Traceback (most recent call last):
  File "/mnt/d/AI/A0J/231220/wsl_cuda121/scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 554, in <module>
    main()
  File "/mnt/d/AI/A0J/231220/wsl_cuda121/scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 297, in main
    model = load_model_from_config(config, f"{opt.ckpt}")
  File "/mnt/d/AI/A0J/231220/wsl_cuda121/scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 97, in load_model_from_config
    model = instantiate_from_config(config.model)
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/ldm/models/diffusion/ddpm.py", line 3226, in __init__
    self.instantiate_flownet_stage(flownet_config)  # load flownet path
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/ldm/models/diffusion/ddpm.py", line 3364, in instantiate_flownet_stage
    model = instantiate_from_config(config)
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/ldm/util.py", line 103, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
  File "/opt/conda/envs/cuda121/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/basicsr/__init__.py", line 5, in <module>
    from .losses import *
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/basicsr/losses/__init__.py", line 7, in <module>
    from .gan_loss import g_path_regularize, gradient_penalty_loss, r1_penalty
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/basicsr/losses/gan_loss.py", line 11, in <module>
    class GANLoss(nn.Module):
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/basicsr/utils/registry.py", line 56, in deco
    self._do_register(name, func_or_class, suffix)
  File "/opt/conda/envs/cuda121/lib/python3.9/site-packages/A0J/basicsr/utils/registry.py", line 42, in _do_register
    assert (name not in self._obj_map), (f"An object named '{name}' was already registered "
AssertionError: An object named 'GANLoss' was already registered in 'loss' registry!

Consultation on data degradation issues

basicsr.data.realbasicvsr_dataset has two types of data degradation in the file, RealVSRRecurrentDataset and RealESRGANRecurrentDataset. Have you ever experimented with their differences.

Installation Issue with Xformers

On trying to install this repo, it seems there is a problem with the dependency

conda install xformers -c xformers/label/dev

Specifically, there does not seem to be xformers development binaries currently available for Pytorch < 2.1

conda search xformers -c xformers/label/dev
Loading channels: done
# Name                       Version           Build  Channel             
xformers               0.0.24.dev721 py310_cu11.8.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev721 py310_cu12.1.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev721 py39_cu11.8.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev721 py39_cu12.1.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev722 py310_cu11.8.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev722 py310_cu12.1.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev722 py39_cu11.8.0_pyt2.1.2  xformers/label/dev  
xformers               0.0.24.dev722 py39_cu12.1.0_pyt2.1.2  xformers/label/dev

I'm attempting to install from source, but not sure if this will contain the development binaries

# (Optional) Makes the build much faster
pip install ninja
# Set TORCH_CUDA_ARCH_LIST if running and building on different GPU types
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
# (this can take dozens of minutes)

python scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py  --config configs/mgldvsr/mgldvsr_512_realbasicvsr_deg.yaml  --ckpt "C:\Users\miki\MGLD-VSR\mgldvsr_unet.ckpt"  --vqgan_ckpt "C:\Users\miki\MGLD-VSR\video_vae_cfw.ckpt"  --seqs-path "C:\Users\miki\MGLD-VSR\input"   --outdir "C:\Users\miki\MGLD-VSR\out"  --ddpm_steps 50  --dec_w 1.0  --colorfix_type adain  --select_idx 0  --n_gpus 1
Traceback (most recent call last):
  File "C:\Users\miki\MGLD-VSR\scripts\vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 19, in <module>
    from ldm.util import instantiate_from_config
ModuleNotFoundError: **No module named 'ldm'**

Same error on the Ubuntu and Windows.
installing ldm through pypi doesnt help at all cause latest version of ldm is for python v2...

Can you share the code about caculating warp error?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

ianyeung / mgld-vsr Goto Github PK

mgld-vsr's People

Contributors

Stargazers

Watchers

Forkers

mgld-vsr's Issues

test error

where download flownets/raft-things.pth ?

AssertionError: An object named 'GANLoss' was already registered in 'loss' registry!

Consultation on data degradation issues

Installation Issue with Xformers

Test Error

Code release date

ModuleNotFoundError: No module named 'ldm'

Can you share the code about caculating warp error?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent