Comments (17)
I made my own as below
--- reference_only.py 2023-07-26 22:24:24.000000000 +0300
+++ reference_only2.py 2023-08-22 19:42:10.985318300 +0300
@@ -1,10 +1,11 @@
import torch
-class ReferenceOnlySimple:
+class ReferenceOnlySimple2:
@classmethod
def INPUT_TYPES(s):
return {"required": { "model": ("MODEL",),
"reference": ("LATENT",),
+ "input2": ("LATENT",),
"batch_size": ("INT", {"default": 1, "min": 1, "max": 64})
}}
@@ -13,12 +14,12 @@
CATEGORY = "custom_node_experiments"
- def reference_only(self, model, reference, batch_size):
+ def reference_only(self, model, reference, input2, batch_size):
model_reference = model.clone()
size_latent = list(reference["samples"].shape)
size_latent[0] = batch_size
- latent = {}
- latent["samples"] = torch.zeros(size_latent)
+ latent = input2
+ #latent["samples"] = torch.zeros(size_latent)
batch = latent["samples"].shape[0] + reference["samples"].shape[0]
def reference_apply(q, k, v, extra_options):
@@ -50,5 +51,5 @@
return (model_reference, {"samples": out_latent, "noise_mask": torch.cat((out_mask, mask))})
NODE_CLASS_MAPPINGS = {
- "ReferenceOnlySimple": ReferenceOnlySimple,
+ "ReferenceOnlySimple2": ReferenceOnlySimple2,
}
from comfyui_experiments.
@addddd2
this pretty looks like my experiment ;-)
But as already said - The output looks more like the input image instead of the reference image.
import torch
class Inp_ReferenceOnlySimple:
@classmethod
def INPUT_TYPES(s):
return {"required": { "model": ("MODEL",),
"reference": ("LATENT",),
"latent": ("LATENT",),
"batch_size": ("INT", {"default": 1, "min": 1, "max": 64})
}}
RETURN_TYPES = ("MODEL", "LATENT")
FUNCTION = "inp_reference_only"
CATEGORY = "custom_node_experiments"
def inp_reference_only(self, model, reference, latent, batch_size):
model_reference = model.clone()
size_latent = list(reference["samples"].shape)
size_latent[0] = batch_size
# latent = {}
# latent["samples"] = torch.zeros(size_latent)
batch = latent["samples"].shape[0] + reference["samples"].shape[0]
# batch = reference["samples"].shape[0]
def reference_apply(q, k, v, extra_options):
k = k.clone().repeat(1, 2, 1)
offset = 0
if q.shape[0] > batch:
offset = batch
for o in range(0, q.shape[0], batch):
for x in range(1, batch):
k[x + o, q.shape[1]:] = q[o,:]
return q, k, k
model_reference.set_model_attn1_patch(reference_apply)
out_latent = torch.cat((reference["samples"], latent["samples"]))
if "noise_mask" in latent:
mask = latent["noise_mask"]
else:
mask = torch.ones((64,64), dtype=torch.float32, device="cpu")
if len(mask.shape) < 3:
mask = mask.unsqueeze(0)
if mask.shape[0] < latent["samples"].shape[0]:
print(latent["samples"].shape, mask.shape)
mask = mask.repeat(latent["samples"].shape[0], 1, 1)
out_mask = torch.zeros((1,mask.shape[1],mask.shape[2]), dtype=torch.float32, device="cpu")
return (model_reference, {"samples": out_latent, "noise_mask": torch.cat((out_mask, mask))})
NODE_CLASS_MAPPINGS = {
"Inpaint ReferenceOnlySimple": Inp_ReferenceOnlySimple,
}
from comfyui_experiments.
@MoonMoon82
Yes, it's essentially the same thing. But it works well for me. I only change the level of noise removal. Values around 0.5 give the original image. Values closer to 0.9 give a reference image
from comfyui_experiments.
@addddd2
That just made me curious! I never recognized any similarities to the reference image! So a few minutes ago I tried to reproduce it again. All of a sudden, I realised that all *ancestral-Sampler create the result you mentioned! All non-*ancestral-Sampler just recreate the input image, instead of the reference image at high denoise factor!
Can you confirm my experiences or do you have any additional advices?
from comfyui_experiments.
@MoonMoon82 I dont know what is ancestral Sampler, I just use default KSampler
Wokrflows
from comfyui_experiments.
@addddd2 I meant the sampler name:
In my experiments, it makes a huge difference! Every other sampler (without "ancestral"-ending of the sampler name) results towards input image instead of the reference image.
Could you please try this combination of sampler+scheduler?
I get error-looking images if I'm using this combination.
from comfyui_experiments.
@MoonMoon82
dpmpp_3m_sde_gpu got error
and result
from comfyui_experiments.
Traceback (most recent call last):
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\reference_only.py", line 52, in inp_reference_only
return (model_reference, {"samples": out_latent, "noise_mask": torch.cat((out_mask, mask))})
RuntimeError: Tensors must have same number of dimensions: got 3 and 4
from comfyui_experiments.
@addddd2 The additional latent input works quite well, but I see at least two issues to solve (which I couldn't solve on my own yet):
- The "batch" number isn't working anymore after our changes. I don't know how to copy latent data n-times.
- I tried to inpaint using "SetLatentNoiseMask", but it ended in an error message. I guess the noise mask tensor shape isn't in the right format. ( Inpainting using Controllnet-Inpaint works fine ;-) )
Do you have an idea how to solve these issues?
from comfyui_experiments.
@comfyanonymous You're not going to support this, isn't it? :'(
from comfyui_experiments.
Sorry for the long time
inpaint does not work due to the difference in masks sizes
I fixed it like this
--- reference_only.py 2023-07-26 22:24:24.000000000 +0300
+++ reference_only2.py 2023-09-23 15:11:48.232217100 +0300
@@ -1,10 +1,11 @@
import torch
-class ReferenceOnlySimple:
+class ReferenceOnlySimple2:
@classmethod
def INPUT_TYPES(s):
return {"required": { "model": ("MODEL",),
"reference": ("LATENT",),
+ "input2": ("LATENT",),
"batch_size": ("INT", {"default": 1, "min": 1, "max": 64})
}}
@@ -13,12 +14,12 @@
CATEGORY = "custom_node_experiments"
- def reference_only(self, model, reference, batch_size):
+ def reference_only(self, model, reference, input2, batch_size):
model_reference = model.clone()
size_latent = list(reference["samples"].shape)
size_latent[0] = batch_size
- latent = {}
- latent["samples"] = torch.zeros(size_latent)
+ latent = input2
+ #latent["samples"] = torch.zeros(size_latent)
batch = latent["samples"].shape[0] + reference["samples"].shape[0]
def reference_apply(q, k, v, extra_options):
@@ -39,16 +40,22 @@
mask = latent["noise_mask"]
else:
mask = torch.ones((64,64), dtype=torch.float32, device="cpu")
+
if len(mask.shape) < 3:
mask = mask.unsqueeze(0)
if mask.shape[0] < latent["samples"].shape[0]:
print(latent["samples"].shape, mask.shape)
mask = mask.repeat(latent["samples"].shape[0], 1, 1)
-
- out_mask = torch.zeros((1,mask.shape[1],mask.shape[2]), dtype=torch.float32, device="cpu")
+
+
+ if len(mask.shape) < 4:
+ out_mask = torch.zeros((1,mask.shape[1],mask.shape[2]), dtype=torch.float32, device="cpu")
+ else:
+ out_mask = torch.zeros((1,1,mask.shape[2],mask.shape[3]), dtype=torch.float32, device="cpu")
+
return (model_reference, {"samples": out_latent, "noise_mask": torch.cat((out_mask, mask))})
NODE_CLASS_MAPPINGS = {
- "ReferenceOnlySimple": ReferenceOnlySimple,
+ "ReferenceOnlySimple2": ReferenceOnlySimple2,
}
for the batch to work, I suppose some fixes are required in the loops, but I don’t know which ones
And workflow for inpaint I used
from comfyui_experiments.
HI, I'm new to this stuff and very far away from programming, but I came across this workflows, installed ComfyUI_experiments, but I cant get it - how to install reference_only2.py. Can someone point me in the right direcction?
from comfyui_experiments.
I suspect it's defunct now. I have it installed but can't add more inputs.
from comfyui_experiments.
I suspect it's defunct now. I have it installed but can't add more inputs.
How do you install the node “reference_only2.py” in experiments I can find only reference_only.py which has only one image input node. Is there a link to file or code to it. Can you share the node and workflow files? Would be very thankfull 😉
Sorry I’m very new to all this but trying to get my way around.
from comfyui_experiments.
It is just a script. It goes in the root of the custom_nodes folder. But when you install the ComfyUI experiments through the Manager within ComfyUI it goes there too. I did not have to copy the file to custom_nodes manually.
from comfyui_experiments.
It is just a script. It goes in the root of the custom_nodes folder. But when you install the ComfyUI experiments through the Manager within ComfyUI it goes there too. I did not have to copy the file to custom_nodes manually.
The one which comes with experiments has only one image input node, I was wondering how to get one with two image input (reference and input2) like on workflow screenshots above.
from comfyui_experiments.
It is just a script. It goes in the root of the custom_nodes folder. But when you install the ComfyUI experiments through the Manager within ComfyUI it goes there too. I did not have to copy the file to custom_nodes manually.
The one which comes with experiments has only one image input node, I was wondering how to get one with two image input (reference and input2) like on workflow screenshots above.
addddd2 changed the code ... just see those codes..
from comfyui_experiments.
Related Issues (15)
- sampler_tonemap.py HOT 2
- reference_only behavior
- Reference only also saves original reference image HOT 4
- Rescale CFG nodes not working. HOT 2
- Reference Only might be the missing Key to transfer animation but can't handle animation HOT 3
- I am confused about the node named 'reference_only' in batch processing
- reference_only is not work!
- Seeking Advice on This Error HOT 1
- it's not working anymore?? HOT 1
- feat: sd-dynamic-thresholding-rcfg HOT 3
- Issue with reference_only installation
- 缺少文件么? HOT 1
- Plz make ModelMergeBolocks-Adv node HOT 1
- Plz make LoraMergeBolocks node
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comfyui_experiments.