Use lama cleaner before inpainting inside stable-diffusion-webui

Python 96.20% JavaScript 3.80%

inpaint inpainting lama stable-diffusion stable-diffusion-webui content-aware-fill

sd-webui-lama-cleaner-masked-content's Introduction

Lama cleaner as masked content

This extenstion for AUTOMATIC1111/stable-diffusion-webui adds new value of "Masked content" field in img2img -> inpaint tab. Lama is a NN model useful for removing objects from pictures

This option means how to preprocess masked content before pass it into stable diffusion. It useful when you want to remove object in photo. Use inpainting model and denoising straight +-0.4

It also supports my other extension: sd-webui-replacer

Mask:

lama cleaner:

fill:

Others

original:

latent noise:

latent nothing:

Also you can use Lama cleaner in extras tab, if you want to use it without stable diffusion:

If you have installed this extensions, they appear as models here:

https://github.com/light-and-ray/sd-webui-resynthesizer-masked-content
https://github.com/light-and-ray/sd-webui-yandere-inpaint-masked-content
https://github.com/light-and-ray/sd-webui-manga-inpainting
+ MAT - another NN inpaint model
+ inpainting from OpenCV as bonus
"Insert background" inserts image into target image. Useful for massive compositing in batch

Options

You can adjust few settings:

Go to Settings -> Extras Inpaint:

Native lama's dataset resolution is 256p, but it shows good result for highers with little quality of content reduction. 512p is optimal

sd-webui-lama-cleaner-masked-content's People

Contributors

Stargazers

Watchers

Forkers

oreml

sd-webui-lama-cleaner-masked-content's Issues

Bug: cannot find supported_preprocessor

Hi author, I tried to use your extention in sd-webui. However, it seems that I cannot import the supported_preprocessor.

now do sd hijack: we add clip here Applying attention optimization: xformers... done. Model loaded in 237.6s (load weights from disk: 22.3s, create model: 0.7s, apply weights to model: 213.8s, load textual inversion embeddings: 0.3s, calculate empty prompt: 0.2s). *** Error running before_process: /mnt/nfs/file_server/public/stable-diffusion-webui/extensions/sd-webui-lama-cleaner-masked-content/scripts/lama_cleaner_masked_content_sctipt.py Traceback (most recent call last): File "/mnt/nfs/file_server/public/stable-diffusion-webui/modules/scripts.py", line 610, in before_process script.before_process(p, *script_args) File "/mnt/nfs/file_server/public/stable-diffusion-webui/extensions/sd-webui-lama-cleaner-masked-content/scripts/lama_cleaner_masked_content_sctipt.py", line 35, in before_process p.init_images[0] = lamaInpaint(p.init_images[0], p.image_mask, File "/mnt/nfs/file_server/public/stable-diffusion-webui/extensions/sd-webui-lama-cleaner-masked-content/lama_cleaner_masked_content/inpaint.py", line 122, in lamaInpaint tmpImage = lamaCNInpaint(convertIntoCNMaskedImageFromat(image256, mask256)) File "/mnt/nfs/file_server/public/stable-diffusion-webui/extensions/sd-webui-lama-cleaner-masked-content/lama_cleaner_masked_content/inpaint.py", line 51, in lamaCNInpaint from scripts import supported_preprocessor ImportError: cannot import name 'supported_preprocessor' from 'scripts' (unknown location)

object has no attribute 'cpu'

Today I updated sd-webui-lama-cleaner-masked-content and I got next an error:

*** Error running before_process: C:\SD\extensions\sd-webui-lama-cleaner-masked-content\scripts\lama_cleaner_masked_content_sctipt.py
    Traceback (most recent call last):
      File "C:\SD\modules\scripts.py", line 776, in before_process
        script.before_process(p, *script_args)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\scripts\lama_cleaner_masked_content_sctipt.py", line 38, in before_process
        p.init_images[0] = lamaInpaint(p.init_images[0], p.image_mask,
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\inpaint.py", line 82, in lamaInpaint
        tmpImage = processModel(imageRes, maskRes, model)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\inpaint.py", line 42, in processModel
        return lama_model(image, mask)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\model.py", line 41, in __call__
        return processModel(self.model, image, mask)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\model.py", line 22, in processModel
        model.cpu()
    AttributeError: 'MaskedImageModelDescriptor' object has no attribute 'cpu'

---
100%|██████████████████████████████████████████████████████████████████████████████████| 18/18 [00:04<00:00,  3.61it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 18/18 [00:06<00:00,  2.64it/s]

Can you fix this issue? I used an old version that I installed 4 months ago and it worked!

forge compatability?

great extension i like it a lot but it doesnt work with forge because of different controlnet integrations
https://github.com/lllyasviel/stable-diffusion-webui-forge

issue with using on AMD + ZLUDA

I would very much like to use this extension, however when attempting to run it the following error occurs:

*** Error running before_process: T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-lama-cleaner-masked-content\scripts\lama_cleaner_masked_content_sctipt.py
    Traceback (most recent call last):
      File "T:\auto1111\stable-diffusion-webui-directml\modules\scripts.py", line 817, in before_process
        script.before_process(p, *script_args)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-lama-cleaner-masked-content\scripts\lama_cleaner_masked_content_sctipt.py", line 35, in before_process
        p.init_images[0] = lamaInpaint(p.init_images[0], p.image_mask,
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\inpaint.py", line 122, in lamaInpaint
        tmpImage = lamaCNInpaint(convertIntoCNMaskedImageFromat(image256, mask256))
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\inpaint.py", line 53, in lamaCNInpaint
        return lama(image, None).value
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\preprocessor\lama_inpaint.py", line 42, in __call__
        prd_color = self.model(img_res)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\__init__.py", line 54, in __call__
        result = self.model(image_feed)[0]
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 419, in forward
        return self.model(input)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\container.py", line 217, in forward
        input = module(input)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 337, in forward
        x_l, x_g = self.conv1((x_l, x_g))
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 304, in forward
        x_l, x_g = self.ffc(x)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 275, in forward
        out_xg = self.convl2g(x_l) * l2g_gate + self.convg2g(x_g)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 198, in forward
        output = self.fu(x)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "T:\auto1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 86, in forward
        ffted = torch.fft.rfftn(x, dim=fft_dim, norm=self.fft_norm)
    RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR

---

i understand if this is not something that can be helped, but i wanted to ask and/or document the issue.

console error while starting webui

i see the following in my console window:

*** Extension "sd-webui-lama-cleaner-masked-content" requires "sd-webui-controlnet" which is disabled.

however, controlnet is installed, enabled, and working properly.

Request API support

The webui effect is great, I hope it can support APIs

light-and-ray / sd-webui-lama-cleaner-masked-content Goto Github PK

sd-webui-lama-cleaner-masked-content's Introduction

Lama cleaner as masked content

Options

sd-webui-lama-cleaner-masked-content's People

Contributors

Stargazers

Watchers

Forkers

sd-webui-lama-cleaner-masked-content's Issues

Bug: cannot find supported_preprocessor

object has no attribute 'cpu'

forge compatability?

issue with using on AMD + ZLUDA

console error while starting webui

Request API support

OSError: cannot write mode RGBA as JPEG

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent