AMD gpu users can't run inpaint anything about sd-webui-inpaint-anything HOT 9 OPEN

Either ROCm's PyTorch needs to support the Segment Anything model, or the developers of the Segment Anything model need to provide support.

from sd-webui-inpaint-anything.

The Segment Anything model is written in PyTorch. Therefore, if it does not run on PyTorch for ROCm on AMD GPUs, it may be necessary to review and revise the model's code. However, making such adjustments could be challenging.

from sd-webui-inpaint-anything.

@Uminosachi
Will you try to do something about it ?

from sd-webui-inpaint-anything.

I don't have an AMD GPU card, so I can't verify the error.

from sd-webui-inpaint-anything.

i will be here for the test if you want to

from sd-webui-inpaint-anything.

If you could record the error you encounter when running Segment Anything with PyTorch for ROCm, I'd be willing to look into it.

from sd-webui-inpaint-anything.

@Uminosachi Here you go

Startup time: 19.6s (prepare environment: 6.5s, import torch: 3.8s, import gradio: 1.1s, setup paths: 1.0s, initialize shared: 1.5s, other imports: 0.3s, load scripts: 3.3s, refresh VAE: 0.4s, create ui: 0.9s, gradio launch: 0.4s, app_started_callback: 0.2s).
Creating model from config: C:\a1111\stable-diffusion-webui-directml\configs\v1-inpainting-inference.yaml
Applying attention optimization: sub-quadratic... done.
Model loaded in 6.8s (load weights from disk: 2.2s, create model: 0.4s, apply weights to model: 2.6s, apply float(): 0.9s, calculate empty prompt: 0.7s).
2024-02-09 17:56:20,376 - Inpaint Anything - INFO - input_image: (253, 450, 3) uint8
2024-02-09 17:56:21,693 - Inpaint Anything - INFO - SamAutomaticMaskGenerator sam_hq_vit_b.pth
Traceback (most recent call last):
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\scripts\inpaint_anything.py", line 187, in run_sam
sam_masks = inpalib.generate_sam_masks(input_image, sam_model_id, anime_style_chk)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\inpalib\samlib.py", line 139, in generate_sam_masks
sam_masks = sam_mask_generator.generate(input_image)
File "C:\a1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 151, in generate
mask_data = self._generate_masks(image, multimask_output)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 194, in _generate_masks
crop_data = self._process_crop(image, crop_box, layer_idx, orig_size, multimask_output)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 245, in _process_crop
batch_data = self._process_batch(points, cropped_im_size, crop_box, orig_size, multimask_output)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 289, in _process_batch
masks, iou_preds, _ = self.predictor.predict_torch(
File "C:\a1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\predictor.py", line 227, in predict_torch
sparse_embeddings, dense_embeddings = self.model.prompt_encoder(
File "C:\a1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\modeling\prompt_encoder.py", line 155, in forward
point_embeddings = self._embed_points(coords, labels, pad=(boxes is None))
File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\modeling\prompt_encoder.py", line 89, in _embed_points
point_embedding[labels == 0] += self.point_embeddings[0].weight
RuntimeError: The size of tensor a (0) must match the size of tensor b (256) at non-singleton dimension 1

2024-02-09 17:56:25,944 - Inpaint Anything - ERROR - The size of tensor a (0) must match the size of tensor b (256) at non-singleton dimension 1

from sd-webui-inpaint-anything.

The size of tensor a (0) must match the size of tensor b (256) at non-singleton dimension 1

I have seen this error before and believe it is due to ROCm's PyTorch not supporting higher dimensional shapes (e.g., length 5). Modifying the model could lead to an inability to load the standard weights, resulting in a significant overhaul that encompasses two models and carries risks.

from sd-webui-inpaint-anything.

So it's impossible to do something about it ?

from sd-webui-inpaint-anything.