Comments (7)
Same promblem.
'mat1 and mat2 shapes cannot be multiplied (24576x256 and 512x512)'
from text-to-video-finetuning.
Hello. Is this a LoRA trained using this finetuning repository or are you using another one?
from text-to-video-finetuning.
I had the same problem using the latest version of this repo.
from text-to-video-finetuning.
If either of you @ChawDoe @zacaikido could provide a LoRa checkpoint, the settings it was trained with, and the settings you're running inference with, I can take a look.
from text-to-video-finetuning.
Pointing to a folder with only the xxx_text_encoder.pt in it will solve the problem.
Seems to be a problem with the xxx_unet.pt or multiple files situation?
from text-to-video-finetuning.
me too, RuntimeError: mat1 and mat2 shapes cannot be multiplied (12288x256 and 512x512)
from text-to-video-finetuning.
When trying to run inference using --lora_path parameter, getting :
LoRA rank 64 is too large. setting to: 4 list index out of range Couldn't inject LoRA's due to an error. 0%| | 0/50 [00:00<?, ?it/s] 0%| | 0/50 [00:00<?, ?it/s] Traceback (most recent call last): File "/content/drive/MyDrive/Text-To-Video-Finetuning/inference.py", line 194, in <module> videos = inference(**args) File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/content/drive/MyDrive/Text-To-Video-Finetuning/inference.py", line 141, in inference videos = pipeline( File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth.py", line 646, in __call__ noise_pred = self.unet( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/content/drive/MyDrive/Text-To-Video-Finetuning/models/unet_3d_condition.py", line 399, in forward emb = self.time_embedding(t_emb, timestep_cond) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/diffusers/models/embeddings.py", line 192, in forward sample = self.linear_1(sample) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/content/drive/MyDrive/Text-To-Video-Finetuning/utils/lora.py", line 60, in forward + self.dropout(self.lora_up(self.selector(self.lora_down(input)))) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (6x320 and 1280x16)
I'm running it on a Colab
in /utils/lora.py
here,
And here
from text-to-video-finetuning.
Related Issues (20)
- webui Lora Might be causing errors in checkpoint models. HOT 3
- How to train with folder video HOT 1
- Which paper? HOT 1
- RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1, 512] because the unspecified dimension size -1 can be any value and is ambiguous HOT 3
- Does this code support native finetune for damo text to video model? HOT 2
- AttributeError: 'Tensor' object has no attribute 'config' HOT 5
- How can I run the fine-tuning on a GPU with <= 16GB of VRAM? HOT 3
- I have some doubts about the framework HOT 4
- A typo
- TypeError: Linear.forward() got an unexpected keyword argument 'scale' HOT 6
- wrong norm method HOT 1
- issues on train.py HOT 1
- [inference] latents_window index error HOT 1
- Two forward passes in finetune_unet HOT 1
- Lora on ResnetBlock2D in modelscope model HOT 1
- Can the videocomposer model be adapted to this training framework?
- Normal finetuning instead of LoRA
- ControlNet
- init_video problem
- finetune train error of "UnboundLocalError: local variable 'use_offset_noise' referenced before assignment" HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from text-to-video-finetuning.