runpod serverless permission error about comfyui HOT 11 CLOSED

Ok, apologies for this. The issue has come about due to a change in the underlying images that ComfyUI is inheriting from so builds are currently broken until I push the changes up to ComfyUI but still needs some testing.

The cause is that storage monitor is now being run as a system user (user) so it doesn't have permission to write to the root owned workspace (runpod-volume). An interim fix may be to override the storagemonitor supervisor file to ensure it still runs as root, or change the permissions in the workspace. You should be able to achieve this by running your images in GPU cloud with the volume attached.

Should be resolved by Monday but hopefully earlier.

from comfyui.

Both are bundled in my image. They are useful here but will be used more extensively in an upcoming image I have planned.

If you have no volume mounted you can just chown -R 1000:1000 /runpod-volume. Add it in preflight.sh on layer0. This is a temporary fix - I wouldn't normally suggest it but it may get you running for now.

That's as much help as I can offer currently as I need to test this myself. I'll update when I know more

from comfyui.

The new issue is related. It's all about permissions but fortunately I expect this to be resolved when the update is pushed.

from comfyui.

Source tree has been updated and builds should now work as expected. Base bumped to PyTorch 2.2.0 on nvidia-runtime base image.

from comfyui.

I've noticed recent unusual behaviour with the runpod network volumes. I'm not sure what, if anything, they have changed but I'll investigate and try to work around.

from comfyui.

Im running into the same issue as well. The changes I've made are adding some models/nodes to layer1/init.sh as well as these lines

    build_extra_get_models \
        "/opt/storage/stable_diffusion/models/ipadapter" \
        "${IPADAPTER_MODELS[@]}"
    build_extra_get_models \
        "/opt/storage/stable_diffusion/models/clip_vision" \
        "${CLIP_VISION_MODELS[@]}"

Adding to mappings.sh

storage_map["stable_diffusion/models/ipadapter"]="/opt/ComfyUI/models/ipadapter"
storage_map["stable_diffusion/models/clip"]="/opt/ComfyUI/models/clip"
storage_map["stable_diffusion/models/clip_vision"]="/opt/ComfyUI/models/clip_vision"

Changed the tags/image to my own ghcr repo

version: "3.8"
services:
  supervisor:
    build:
      context: ./build
      args:
        IMAGE_BASE: ${IMAGE_BASE:-ghcr.io/ai-dock/jupyter-pytorch:2.1.1-py3.10-cuda-11.8.0-base-22.04}
      tags:
        - "ghcr.io/bkunbargi/comfybrev:jupyter-pytorch:2.1.1-py3.10-cuda-11.8.0-base-22.04"
        
    image: ghcr.io/bkunbargi/comfybrev:jupyter-pytorch:2.1.1-py3.10-cuda-11.8.0-base-22.04

Update: I added a network volume and the issue changed to

[01qu4hfco957dp]
[info]
ln: failed to create symbolic link '/opt/ComfyUI/models/checkpoints/v1-5-pruned-emaonly.ckpt': Permission denied
2024-02-09 15:41:30.091
[01qu4hfco957dp]
[info]
ln: failed to create symbolic link '/opt/ComfyUI/models/controlnet/control_canny-fp16.safetensors': Permission denied
2024-02-09 15:41:30.091
[01qu4hfco957dp]
[info]
ln: failed to create symbolic link '/opt/ComfyUI/models/controlnet/diff_control_sd15_depth_fp16.safetensors': Permission denied
2024-02-09 15:41:30.091
[01qu4hfco957dp]
[info]
mkdir: cannot create directory ‘/opt/ComfyUI/models/ipadapter’: Permission denied```

from comfyui.

Ok, apologies for this. The issue has come about due to a change in the underlying images that ComfyUI is inheriting from so builds are currently broken until I push the changes up to ComfyUI but still needs some testing.

The cause is that storage monitor is now being run as a system user (user) so it doesn't have permission to write to the root owned workspace (runpod-volume). An interim fix may be to override the storagemonitor supervisor file to ensure it still runs as root, or change the permissions in the workspace. You should be able to achieve this by running your images in GPU cloud with the volume attached.

Should be resolved by Monday but hopefully earlier.

I see, thank you for the investigation. I have question. Is storage monitor included in this image(ai-dock/comfyui) or in runpod? And is it same with the inotifywait mentioned in README ? I want to know what are those two.

And I am deploying on runpod serverless, so I can't edit in cloud. Do I need to edit init.sh?

from comfyui.

Both are bundled in my image. They are useful here but will be used more extensively in an upcoming image I have planned.

If you have no volume mounted you can just chown -R 1000:1000 /runpod-volume. Add it in preflight.sh on layer0. This is a temporary fix - I wouldn't normally suggest it but it may get you running for now.

That's as much help as I can offer currently as I need to test this myself. I'll update when I know more

gotta try.
Thank you so much, I really appreciate for your work🙏

from comfyui.

@hongminpark did it work for you?

from comfyui.

@hongminpark did it work for you?

Nope, so I tried move custom files directly inside Comfy directory at init.sh

function build_extra_start() {
    build_extra_get_nodes
//...

    # Copy models directly
    mv /opt/storage/stable_diffusion/models/ckpt/* /opt/ComfyUI/models/checkpoints
    mv /opt/storage/stable_diffusion/models/controlnet/* /opt/ComfyUI/models/controlnet
    mv /opt/storage/stable_diffusion/models/embeddings/* /opt/ComfyUI/models/embeddings
}

Then the model files are correctly included but now I'm facing new problem : runpod serverless handler is not responding my api request, I am getting only 'queued' and comfyui isn't getting any request.

from comfyui.

Fixed. Issues relating to permissions are solved by building against the latest PyTorch/Jupyter PyTorch images.

from comfyui.