Inpaint Anything performs stable diffusion inpainting on a browser UI using any mask selected from the output of Segment Anything.
Using Segment Anything enables users to specify masks by simply pointing to the desired areas, instead of manually filling them in. This can increase the efficiency and accuracy of the mask creation process, leading to potentially higher-quality inpainting results while saving time and effort.
Extension version for AUTOMATIC1111's Web UI
Please follow these steps to install the software:
- Create a new conda environment:
conda create -n inpaint python=3.10
conda activate inpaint- Clone the software repository:
git clone https://github.com/Uminosachi/inpaint-anything.git
cd inpaint-anything- For the CUDA environment, install the following packages:
pip install -r requirements.txt- If you are using macOS, please install the package from the following file instead:
pip install -r requirements_mac.txtpython iasam_app.py- Open http://127.0.0.1:7860/ in your browser.
- Note: If you have a privacy protection extension enabled in your web browser, such as DuckDuckGo, you may not be able to retrieve the mask from your sketch.
--save-seg: Save the segmentation image generated by SAM.--offline: Execute inpainting using an offline network.--sam-cpu: Perform the Segment Anything operation on CPU.
- Launch this application.
- Click on the
Download modelbutton, located next to the Segment Anything Model ID. This includes the Segment Anything in High Quality Model ID, Fast Segment Anything, and Faster Segment Anything (MobileSAM).- Please note that the SAM is available in three sizes: Base, Large, and Huge. Remember, larger sizes consume more VRAM.
- Wait for the download to complete.
- The downloaded model file will be stored in the
modelsdirectory of this application's repository.
- Drag and drop your image onto the input image area.
- Outpainting can be achieved by the
Padding options, configuring the scale and balance, and then clicking on theRun Paddingbutton. - The
Anime Stylecheckbox enhances segmentation mask detection, particularly in anime style images, at the expense of a slight reduction in mask quality.
- Outpainting can be achieved by the
- Click on the
Run Segment Anythingbutton. - Use sketching to point the area you want to inpaint. You can undo and adjust the pen size.
- Hover over either the SAM image or the mask image and press the
Skey for Fullscreen mode, or theRkey to Reset zoom.
- Hover over either the SAM image or the mask image and press the
- Click on the
Create maskbutton. The mask will appear in the selected mask image area.
Expand mask regionbutton: Use this to slightly expand the area of the mask for broader coverage.Trim mask by sketchbutton: Clicking this will exclude the sketched area from the mask.Add mask by sketchbutton: Clicking this will add the sketched area to the mask.
- Enter your desired Prompt and Negative Prompt, then choose the Inpainting Model ID.
- Click on the
Run Inpaintingbutton (Please note that it may take some time to download the model for the first time).- In the Advanced options, you can adjust the Sampler, Sampling Steps, Guidance Scale, and Seed.
- If you enable the
Mask area Onlyoption, modifications will be confined to the designated mask area only.
- Adjust the iteration slider to perform inpainting multiple times with different seeds.
- The inpainting process is powered by diffusers.
- You can directly drag and drop the inpainted image into the input image field on the Web UI. (useful with Chrome and Edge browsers)
- The inpainting model, which is saved in HuggingFace's cache and includes
inpaint(case-insensitive) in its repo_id, will also be added to the Inpainting Model ID dropdown list.- If there's a specific model you'd like to use, you can cache it in advance using the following Python commands:
pythonfrom diffusers import StableDiffusionInpaintPipeline
pipe = StableDiffusionInpaintPipeline.from_pretrained("Uminosachi/dreamshaper_5-inpainting")
exit()- The model diffusers downloaded is typically stored in your home directory. You can find it at
/home/username/.cache/huggingface/hubfor Linux and MacOS users, or atC:\Users\username\.cache\huggingface\hubfor Windows users.
- Choose the Cleaner Model ID.
- Click on the
Run Cleanerbutton (Please note that it may take some time to download the model for the first time). - Cleaner process is performed using Lama Cleaner.
- Gives ability to just save mask without any other processing, so it's then possible to use the mask in other graphic applications.
Get mask as alpha of imagebutton: Save the mask as RGBA image, with the mask put into the alpha channel of the input image.Get maskbutton: Save the mask as RGB image.
- The inpainted image will be automatically saved in the folder that matches the current date within the
outputsdirectory.
With the Inpaint Anything library, you can perform segmentation and create masks using sketches from other applications.
The source code is licensed under the Apache 2.0 license.
- Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W-Y., Dollár, P., & Girshick, R. (2023). Segment Anything. arXiv:2304.02643.
- Ke, L., Ye, M., Danelljan, M., Liu, Y., Tai, Y-W., Tang, C-K., & Yu, F. (2023). Segment Anything in High Quality. arXiv:2306.01567.
- Zhao, X., Ding, W., An, Y., Du, Y., Yu, T., Li, M., Tang, M., & Wang, J. (2023). Fast Segment Anything. arXiv:2306.12156 [cs.CV].
- Zhang, C., Han, D., Qiao, Y., Kim, J. U., Bae, S-H., Lee, S., & Hong, C. S. (2023). Faster Segment Anything: Towards Lightweight SAM for Mobile Applications. arXiv:2306.14289.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent