Feature Request: IPAdapter and Latent Upscale (Neural Network) about fooocus-mre HOT 12 OPEN

@Ptbird1 Simplest way to create variations of existing image(s) are img2img and Revision modes. If you want output images to look more like original you can just lower denoise in Image-2-Image settings. In latest version you can also use Canny Edge Control-LoRA (works as ControlNet - see CN tab), which is also great way to produce variants of original image :).

Example:

->

from fooocus-mre.

@vcm07 I like those results, but problem is I don't see IPAdapter in base ComfyUI code - and I'd rather not pull any custom nodes not maintained by comfy. I can get back to it when IPAdapter will be available in comfy backend - I've added feature request for that on ComfyUI side (ComfyUI issue #1392).

No problem. I agree. Watching more closely, I see that most of these workflows and custom nodes with Ipadapter are very volatile. It happened a few days ago when Comfy updated recently; it broke everything, even the workflows, and these people (third parties) do add lots of new stuff (new custom nodes), change a lot of stuff, and this could create nightmarish upgrade problems, so you are right, it would be better to wait for an official implementation. Thanks for adding the feature request on the ComfyUI side.

from fooocus-mre.

Upscaling is now possible in v2.0.12 via Image-2-Image - see example in Wiki.

from fooocus-mre.

@vcm07 Those new features are still being stabilised in vanilla, but I will look into merging it within next couple of days.

from fooocus-mre.

@MoonRide303 Thanks for the prompt (hehe) reply! I have been AFK for a week so didn't know you implemented canny Cn!! That's so cool and I will definitely be using that once I'm back at my PC! I've used img2img a bunch and love it but what I was referring to was mixing two pre-existing images into a new generated image that contains the features of both originals. See photo

I'm unsure if this would be feasible in SDXL (at least yet) as after looking at op's link for IPadaptrer that has that feature it uses sd 1.5. Thanks again for a fantastic fork of a fantastic program! Keep up the good work and I'll be keeping a close eye on any updates, this stuff is just too fun!

from fooocus-mre.

I'm unsure if this would be feasible in SDXL (at least yet) as after looking at op's link for IPadaptrer that has that feature it uses sd 1.5. Thanks again for a fantastic fork of a fantastic program! Keep up the good work and I'll be keeping a close eye on any updates, this stuff is just too fun!

It is feasible; actually, it is already happening; people are already using it for SDXL on comfyui (only works there). All the links I've posted are for SDXL (check ip_adapter_sdxl_demo.ipynb on the tencent link). Ipadapter for SDXL is one of the hottest things happening in the scene right now. And there is a need for an upscaler on Foocus/MRE, or at least something similar to the "highresfix" (from Auto1111), to correct eyes, etc., and then upscale.

but what I was referring to was mixing two pre-existing images into a new generated image that contains the features of both originals. See photo

Yep, that is exactly it. Your baby yoda-cat example illustrates it nicely. Here is another example:

from fooocus-mre.

@vcm07 I like those results, but problem is I don't see IPAdapter in base ComfyUI code - and I'd rather not pull any custom nodes not maintained by comfy. I can get back to it when IPAdapter will be available in comfy backend - I've added feature request for that on ComfyUI side (ComfyUI issue #1392).

from fooocus-mre.

So, we are almost there, Fooocus with IPAdapter: lllyasviel#557

MRE with this = perfection;

from fooocus-mre.

Adding some form of image (re)mixing, iterations and upscaling would be a HUGE boon, one of my favorite features of MJ is the image remixing and iterations, give it an image you like and ask it to make similar ones or mix someone with a blobfish for hilarious results. Upscaling is just the icing on the cake. Would be very interested to see if this can be implemented.

from fooocus-mre.

@aimarxjg It should automatically pull update from github when you start it via run-mre.bat. Alternatively - if you have git installed in your system - you can just use git pull command in Fooocus-MRE folder. New version should automatically download any additional model files necessary for new features (like Canny) to work.

from fooocus-mre.

@aimarxjg If MRE will find those files in proper folders then he won't try to download them, again. So if you already have those files, or can download it in a different way faster - feel free to speed up the process, and just put them in proper places :).

from fooocus-mre.

Fantastic! Thanks for all the work you're doing

from fooocus-mre.