...
comfyui-supervisor-1  | ==> /var/log/supervisor/comfyui.log <==
comfyui-supervisor-1  | Starting ComfyUI...
comfyui-supervisor-1  |
comfyui-supervisor-1  | ==> /var/log/supervisor/supervisor.log <==
comfyui-supervisor-1  | 2023-11-13 13:49:38,998 INFO spawned: 'comfyui' with pid 514
comfyui-supervisor-1  | 2023-11-13 13:49:39,021 INFO success: comfyui entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
comfyui-supervisor-1  | 2023-11-13 13:50:40,388 INFO exited: comfyui (exit status 1; not expected)
comfyui-supervisor-1  |
comfyui-supervisor-1  | ==> /var/log/supervisor/comfyui.log <==
comfyui-supervisor-1  | Starting ComfyUI...
comfyui-supervisor-1  | Success
comfyui-supervisor-1  |
comfyui-supervisor-1  | ==> /var/log/supervisor/supervisor.log <==
comfyui-supervisor-1  | 2023-11-13 13:50:40,388 INFO exited: comfyui (exit status 1; not expected)
comfyui-supervisor-1  | 2023-11-13 13:50:41,004 INFO spawned: 'comfyui' with pid 525
comfyui-supervisor-1  | 2023-11-13 13:50:41,036 INFO success: comfyui entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
comfyui-supervisor-1  |
comfyui-supervisor-1  | ==> /var/log/supervisor/comfyui.log <==
comfyui-supervisor-1  | Starting ComfyUI...
comfyui-supervisor-1  |
comfyui-supervisor-1  | ==> /var/log/supervisor/supervisor.log <==
comfyui-supervisor-1  | 2023-11-13 13:50:41,004 INFO spawned: 'comfyui' with pid 525
comfyui-supervisor-1  | 2023-11-13 13:50:41,036 INFO success: comfyui entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
...

comfyui-supervisor-1  | {"level":"error","ts":1699883724.371419,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"ACCESSMACHINE","remote_port":"48944","client_ip":"ACCESSMACHINE","proto":"HTTP/1.1","method":"GET","host":"DOCKERMACHINE:8188","uri":"/","headers":{"Authorization":[],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.5"],"Accept-Encoding":["gzip, deflate"],"Dnt":["1"],"Connection":["keep-alive"]}},"duration":1.203366855,"status":502,"err_id":"admr5db0z","err_trace":"reverseproxy.statusError (reverseproxy.go:1265)"}

$ docker compose up -d
[+] Running 2/2
 ✔ Network comfyui_default         Created                                                                                                                                   0.1s
 ✔ Container comfyui-supervisor-1  Created                                                                                                                                   0.1s
Error response from daemon: path /home/ubuntu/comfyui/workspace is mounted on / but it is not a shared mount

AMD R4750G APU × 16GB UMA
Artix Linux
Latest ComfyUI dock Git trees
system/linux 6.7.4.artix1-1 [installed]
system/python 3.11.7-1 [installed]
world/comgr 6.0.0-1 [installed]
world/hsa-rocr 6.0.0-2 [installed]
world/rocm-device-libs 6.0.0-1 [installed]
extra/comgr 6.0.0-1 [installed]
extra/hsa-rocr 6.0.0-2 [installed]
extra/rocm-core 6.0.0-2 [installed]
extra/rocm-device-libs 6.0.0-1 [installed]
extra/rocm-language-runtime 6.0.0-1 [installed]
extra/rocm-opencl-runtime 6.0.0-1 [installed]
extra/rocm-opencl-sdk 6.0.0-1 [installed]

...
subprocess.CalledProcessError: Command '['/opt/micromamba/envs/comfyui/bin/ffmpeg'
...
Exception: An error occured in the ffmpeg subprocess:
Unrecognized option 'crf'
...

supervisor-1  | Downloading diffusers-0.26.3-py3-none-any.whl (1.9 MB)
supervisor-1  |    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB _16.4 kB/s eta_ 0:00:00

    def get_result(self, job_id):
        result = requests.get(self.ENDPOINT_HISTORY).json()[self.comfyui_job_id]

        prompt = result["prompt"]
        outputs = result["outputs"]

        self.result = {
            "images": [],
            "timings": {}
        }
        
        custom_output_dir = f"{self.OUTPUT_DIR}{self.request_id}"
        os.makedirs(custom_output_dir, exist_ok = True)
        for item in outputs:
            if "images" in outputs[item]:
                for image in outputs[item]["images"]:
                    original_path = f"{self.OUTPUT_DIR}{image['subfolder']}/{image['filename']}"
                    new_path = f"{custom_output_dir}/{image['filename']}"
                    # Handle duplicated request where output file in not re-generated
                    if os.path.islink(original_path):
                        shutil.copyfile(os.path.realpath(original_path), new_path)
                    else:
                        os.rename(original_path, new_path)
                        os.symlink(new_path, original_path)
                    key = f"{self.request_id}/{image['filename']}"
                    self.result["images"].append({
                        "local_path": new_path,
                        "base64": self.image_to_base64(new_path),
                        # make this work first, then threads
                        # "url": self.s3utils.file_upload(new_path, key)
                    })
        
        self.job_time_completed = datetime.datetime.now()
        self.result["timings"] = {
            "job_time_received": self.job_time_received.ctime(),
            "job_time_queued": self.job_time_queued.ctime(),
            "job_time_processed": self.job_time_processed.ctime(),
            "job_time_completed": self.job_time_completed.ctime(),
            "job_time_total": (self.job_time_completed - self.job_time_received).seconds
        }

        return self.result

    def image_to_base64(self, path):
        with open(path, "rb") as f:
            b64 = (base64.b64encode(f.read()))
        return "data:image/png;charset=utf-8;base64, " + b64

34.45
34.45 Transaction finished
34.45
34.45 To activate this environment, use:
34.45
34.45     micromamba activate comfyui
34.45
34.45 Or to execute a single command in this environment, use:
34.45
34.45     micromamba run -n comfyui mycommand
34.45
34.48 Installing jupyter kernels...
34.48 Cloning ComfyUI...
34.48 Cloning into 'ComfyUI'...
35.32 Completed common.sh processes.
35.32 INIT::Running nvidia script...
35.32 NVIDIA::Installing ComfyUI...
35.32 PIP_INSTALL = pip install --no-cache-dir
35.32 PYTORCH_VERSION = 2.1.0
35.33 Success
XXXXX <--- I should see an echo for "Running update script..." here if it got through the micromamba command
------
Dockerfile:13
--------------------
  11 |
  12 |     ARG IMAGE_BASE
  13 | >>> RUN /opt/ai-dock/bin/build/layer0/init.sh
  14 |
  15 |     ENV OPT_SYNC=ComfyUI:serverless:$OPT_SYNC
--------------------
ERROR: failed to solve: process "/bin/bash -c /opt/ai-dock/bin/build/layer0/init.sh" did not complete successfully: exit code: 1```

024-02-29T07:51:19.551713395Z cp: not writing through dangling symlink '/workspace/home/user/.ssh'
2024-02-29T07:51:19.559099834Z chown: changing ownership of '/workspace/home/user/.ssh': Operation not permitted
2024-02-29T07:51:19.566694095Z chmod: cannot access '/workspace/home/user/.ssh/authorized_keys': No such file or directory

ComfyUI RP API
This service is available on port 8188 and is used to test the [RunPod serverless](https://link.ai-dock.org/runpod-serverless) API.

You can access the api directly at /rp-api/runsync or you can use the Swager/openAPI playground at /rp-api.

There are several [example payloads](https://github.com/ai-dock/comfyui/tree/main/build/COPY_ROOT/opt/serverless/docs/example_payloads) included in this repository.

    # INPUT_DIR=f"{os.environ.get('WORKSPACE')}/ComfyUI/input/"
    # OUTPUT_DIR=f"{os.environ.get('WORKSPACE')}/ComfyUI/output/"
    INPUT_DIR=f"{os.environ.get('WORKSPACE')}ComfyUI/input"
    OUTPUT_DIR=f"{os.environ.get('WORKSPACE')}ComfyUI/output"

Main system  : ASRock DeskMini X300 AMD
APU          : AMD R4750G Zen2 with 16GB UMA (graphical memories)
AMDGPU driver: 23.0.0-1 (X11)
ROCm         : 5.7.1
Python       : 3.11.6-1
Gross RAM's  : 48GB DDR4
OS           : Artix Linux amd64 with Linux kernel 6.1.70
Docker       : 24.0.7-1 / Compose 2.24.3-1
UI browser   : Brave 1.63.131-1

<Error>
  <Code>AuthorizationQueryParametersError</Code>
  <Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-central-1'</Message>
  <Region>eu-central-1</Region>
  <RequestId>9G9706QZWPMWC3ZY</RequestId>
  <HostId>XWklCWgRIqcY8Xl7S7aSwZtXi97/KZfWUW6bJLfafVU3K9/tjmtVidH3SktNlz6ZmHxcVjIBQAQ=</HostId>
</Error>

2023-10-27 22:08:21 /opt/micromamba/envs/comfyui/bin/ffmpeg: error while loading shared libraries: libopenh264.so.5: cannot open shared object file: No such file or directory
2023-10-27 22:08:21 ERROR:root:!!! Exception during processing !!!
2023-10-27 22:08:21 ERROR:root:Traceback (most recent call last):
2023-10-27 22:08:21   File "/workspace/ComfyUI/execution.py", line 153, in recursive_execute
2023-10-27 22:08:21     output_data, output_ui = get_output_data(obj, input_data_all)
2023-10-27 22:08:21   File "/workspace/ComfyUI/execution.py", line 83, in get_output_data
2023-10-27 22:08:21     return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
2023-10-27 22:08:21   File "/workspace/ComfyUI/execution.py", line 76, in map_node_over_list
2023-10-27 22:08:21     results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
2023-10-27 22:08:21   File "/workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite/videohelpersuite/nodes.py", line 195, in combine_video
2023-10-27 22:08:21     proc.stdin.write(frame.tobytes())
2023-10-27 22:08:21 BrokenPipeError: [Errno 32] Broken pipe

version: "3.8"
# Compose file build variables set in .env
services:
  supervisor:
    restart: always
    platform: linux/amd64
    build:
      context: ./build
      args:
        IMAGE_BASE: ${IMAGE_BASE:-ghcr.io/ai-dock/jupyter-pytorch:2.2.0-py3.10-cpu-22.04}
      #tags:
        #- "ghcr.io/ai-dock/comfyui:${IMAGE_TAG:-jupyter-pytorch-2.2.0-py3.10-cpu-22.04}"
        #- "IMAGE_TAG:-jupyter-pytorch-2.2.0-py3.10-cpu-22.04"
    image: ghcr.io/ai-dock/comfyui:${IMAGE_TAG:-jupyter-pytorch-2.2.0-py3.10-cpu-22.04}
    ## For Nvidia GPU's - You probably want to uncomment this
    #deploy:
    #  resources:
    #    reservations:
    #      devices:
    #        - driver: nvidia
    #          count: all
    #          capabilities: [gpu]
    devices:
      - "/dev/dri:/dev/dri"
      ## For AMD GPU
      #- "/dev/kfd:/dev/kfd"
    volumes:
      ## Workspace
      - ./workspace:${WORKSPACE:-/workspace/}:rshared
      # You can share /workspace/storage with other non-ComfyUI containers. See README
      #- /path/to/common_storage:${WORKSPACE:-/workspace/}storage/:rshared
      # Will echo to root-owned authorized_keys file;
      # Avoids changing local file owner
      - ./config/authorized_keys:/root/.ssh/authorized_keys_mount
      - ./config/provisioning/default.sh:/opt/ai-dock/bin/provisioning.sh
    ports:
        # SSH available on host machine port 2222 to avoid conflict. Change to suit
        - ${SSH_PORT_HOST:-2222}:22
        # Caddy port for service portal
        - ${SERVICEPORTAL_PORT_HOST:-1111}:${SERVICEPORTAL_PORT_HOST:-1111}
        # ComfyUI web interface
        - ${COMFYUI_PORT_HOST:-8188}:${COMFYUI_PORT_HOST:-8188}
        # Jupyter server
        - ${JUPYTER_PORT_HOST:-8888}:${JUPYTER_PORT_HOST:-8888}
        # Rclone webserver for interactive configuration
        - ${RCLONE_PORT_HOST:-53682}:${RCLONE_PORT_HOST:-53682}
    environment:
        # Don't enclose values in quotes
        - DIRECT_ADDRESS=${DIRECT_ADDRESS:-192.168.1.101}
        - DIRECT_ADDRESS_GET_WAN=${DIRECT_ADDRESS_GET_WAN:-false}
        - WORKSPACE=${WORKSPACE:-/workspace}
        - WORKSPACE_SYNC=${WORKSPACE_SYNC:-false}
        #- CF_TUNNEL_TOKEN=${CF_TUNNEL_TOKEN:-}
        - CF_QUICK_TUNNELS=${CF_QUICK_TUNNELS:-false}
        - WEB_ENABLE_AUTH=${WEB_ENABLE_AUTH:-true}
        - WEB_USER=${WEB_USER:-user}
        - WEB_PASSWORD=${WEB_PASSWORD:-password}
        - SSH_PORT_HOST=${SSH_PORT_HOST:-2222}
        - SERVICEPORTAL_PORT_HOST=${SERVICEPORTAL_PORT_HOST:-1111}
        - SERVICEPORTAL_METRICS_PORT=${SERVICEPORTAL_METRICS_PORT:-21111}
        - COMFYUI_FLAGS=${COMFYUI_FLAGS:-}
        - COMFYUI_PORT_HOST=${COMFYUI_PORT_HOST:-8188}
        - COMFYUI_METRICS_PORT=${COMFYUI_METRICS_PORT:-28188}
        - JUPYTER_PORT_HOST=${JUPYTER_PORT_HOST:-8888}
        - JUPYTER_METRICS_PORT=${JUPYTER_METRICS_PORT:-28888}
        - SERVERLESS=${SERVERLESS:-false}

  GNU nano 6.2                                                                                                                                            Dockerfile                                                                                                                                                      # For build automation - Allows building from any ai-dock base image
# Use a *cuda*base* image as default because pytorch brings the libs
ARG IMAGE_BASE="ghcr.io/ai-dock/jupyter-pytorch:2.2.0-py3.10-cpu-22.04"
FROM ${IMAGE_BASE}

LABEL org.opencontainers.image.source https://github.com/ai-dock/comfyui
LABEL org.opencontainers.image.description "ComfyUI Stable Diffusion backend and GUI"
LABEL maintainer="Rob Ballantyne <[email protected]>"

ENV IMAGE_SLUG="comfyui"
ENV OPT_SYNC=ComfyUI:serverless

# Copy early so we can use scripts in the build - Changes to these files will invalidate the cache and cause a rebuild.
COPY --chown=0:1111 ./COPY_ROOT/ /

# Use build scripts to ensure we can build all targets from one Dockerfile in a single layer.
# Don't put anything heavy in here - We can use multi-stage building above if necessary.

ARG IMAGE_BASE
RUN set -eo pipefail && /opt/ai-dock/bin/build/layer0/init.sh | tee /var/log/build.log

# Must be set after layer0
ENV MAMBA_DEFAULT_ENV=comfyui
ENV MAMBA_DEFAULT_RUN="micromamba run -n $MAMBA_DEFAULT_ENV"

# Copy overrides and models into later layers for fast rebuilds
COPY --chown=0:1111 ./COPY_ROOT_EXTRA/ /
RUN set -eo pipefail && /opt/ai-dock/bin/build/layer1/init.sh | tee -a /var/log/build.log

# Keep init.sh as-is and place additional logic in /opt/ai-dock/bin/preflight.sh
CMD ["init.sh"]

supervisor_1  | got prompt
supervisor_1  | model_type EPS
supervisor_1  | Using split attention in VAE
supervisor_1  | Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
supervisor_1  | Using split attention in VAE
supervisor_1  | 2024-03-10 17:02:44,181 INFO exited: comfyui (exit status 137; not expected)
supervisor_1  | 2024-03-10 17:02:44,632 INFO spawned: 'comfyui' with pid 768
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/caddy.log <==
supervisor_1  | {"level":"error","ts":1710090163.5710387,"logger":"http.log.error","msg":"dial tcp: lookup localhost: i/o timeout","request":{"remote_ip":"192.168.2.101","remote_port":"53571","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"POST","host":"192.168.1.101:1111","uri":"/ajax/logs","headers":{"Dnt":["1"],"Accept-Encoding":["gzip, deflate"],"Hx-Target":["page"],"Referer":["http://192.168.1.101:1111/"],"Hx-Current-Url":["http://192.168.1.101:1111/"],"Content-Length":["0"],"Content-Type":["application/x-www-form-urlencoded"],"Cookie":[],"Sec-Gpc":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Hx-Request":["true"],"Origin":["http://192.168.1.101:1111"],"Connection":["keep-alive"],"Accept":["*/*"],"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"]}},"duration":81.496884408,"status":502,"err_id":"y5ew3hics","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  | {"level":"error","ts":1710090163.9898791,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53602","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/queue","headers":{"Dnt":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Referer":["http://192.168.1.101:8188/"],"Connection":["keep-alive"],"Cookie":[],"Sec-Gpc":["1"],"Accept":["*/*"],"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"],"Accept-Encoding":["gzip, deflate"],"Comfy-User":["undefined"]}},"duration":0.024025992,"status":502,"err_id":"47kp5rvq8","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  | {"level":"error","ts":1710090163.991298,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53603","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/history?max_items=200","headers":{"Accept-Encoding":["gzip, deflate"],"Comfy-User":["undefined"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Referer":["http://192.168.1.101:8188/"],"Connection":["keep-alive"],"Cookie":[],"Dnt":["1"],"Sec-Gpc":["1"],"Accept":["*/*"],"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"]}},"duration":0.000925198,"status":502,"err_id":"pdq6nmh3q","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  | {"level":"error","ts":1710090164.604226,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53604","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/ws?clientId=5015f3e3d12b455f90d1aa312ca028ac","headers":{"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"],"Sec-Websocket-Key":["dwWr+fR6Q71khkaZvhXXSg=="],"Cookie":[],"Cache-Control":["no-cache"],"Upgrade":["websocket"],"Accept":["*/*"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Pragma":["no-cache"],"Sec-Websocket-Version":["13"],"Origin":["http://192.168.1.101:8188"],"Sec-Websocket-Extensions":["permessage-deflate"],"Connection":["keep-alive, Upgrade"],"Accept-Encoding":["gzip, deflate"]}},"duration":0.00243078,"status":502,"err_id":"4ezxns4f4","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/comfyui.log <==
supervisor_1  | Starting ComfyUI...
supervisor_1  | Starting ComfyUI...
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/supervisor.log <==
supervisor_1  | 2024-03-10 17:02:44,181 INFO exited: comfyui (exit status 137; not expected)
supervisor_1  | 2024-03-10 17:02:44,632 INFO spawned: 'comfyui' with pid 768
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/caddy.log <==
supervisor_1  | {"level":"error","ts":1710090165.2220185,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53607","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/ws?clientId=b083dc43d61044c6ab65b32f1cdde643","headers":{"Pragma":["no-cache"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Accept":["*/*"],"Origin":["http://192.168.1.101:8188"],"Cookie":[],"Upgrade":["websocket"],"Sec-Websocket-Version":["13"],"Sec-Websocket-Extensions":["permessage-deflate"],"Accept-Encoding":["gzip, deflate"],"Cache-Control":["no-cache"],"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"],"Sec-Websocket-Key":["kNwEmADTHsl/vJ/0d/P9Fg=="],"Connection":["keep-alive, Upgrade"]}},"duration":0.001117775,"status":502,"err_id":"sm5g3mwcy","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/comfyui.log <==
supervisor_1  | ** ComfyUI startup time: 2024-03-10 17:02:45.025688
supervisor_1  | ** Platform: Linux
supervisor_1  | ** Python version: 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0]
supervisor_1  | ** Python executable: /opt/micromamba/envs/comfyui/bin/python
supervisor_1  | ** Log path: /workspace/ComfyUI/comfyui.log
supervisor_1  |
supervisor_1  | Prestartup times for custom nodes:
supervisor_1  |    0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
supervisor_1  |
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/caddy.log <==
supervisor_1  | {"level":"error","ts":1710090166.064027,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53608","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/ws?clientId=5015f3e3d12b455f90d1aa312ca028ac","headers":{"Sec-Websocket-Version":["13"],"Sec-Websocket-Extensions":["permessage-deflate"],"Sec-Websocket-Key":["R2uKGn4klgocfuTu+fSF+A=="],"Connection":["keep-alive, Upgrade"],"Accept":["*/*"],"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"],"Upgrade":["websocket"],"Cookie":[],"Pragma":["no-cache"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Accept-Encoding":["gzip, deflate"],"Cache-Control":["no-cache"],"Origin":["http://192.168.1.101:8188"]}},"duration":0.000765085,"status":502,"err_id":"fkc66rk5b","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  | {"level":"error","ts":1710090167.3226793,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53609","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/ws?clientId=b083dc43d61044c6ab65b32f1cdde643","headers":{"Sec-Websocket-Key":["/7oTZHXZkskiKv9SMGrrew=="],"Accept":["*/*"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"],"Sec-Websocket-Version":["13"],"Pragma":["no-cache"],"Sec-Websocket-Extensions":["permessage-deflate"],"Connection":["keep-alive, Upgrade"],"Cache-Control":["no-cache"],"Upgrade":["websocket"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Origin":["http://192.168.1.101:8188"],"Cookie":[]}},"duration":0.001707567,"status":502,"err_id":"i6zh30sn6","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/comfyui.log <==
supervisor_1  | Total VRAM 7937 MB, total RAM 7937 MB
supervisor_1  | Set vram state to: DISABLED
supervisor_1  | Device: cpu
supervisor_1  | VAE dtype: torch.float32
supervisor_1  | Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --use-split-cross-attention
supervisor_1  | 2024-03-10 17:02:49,846 INFO success: comfyui entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/caddy.log <==
supervisor_1  | {"level":"error","ts":1710090169.2041223,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:18188: connect: connection refused","request":{"remote_ip":"192.168.2.101","remote_port":"53610","client_ip":"192.168.2.101","proto":"HTTP/1.1","method":"GET","host":"192.168.1.101:8188","uri":"/ws?clientId=5015f3e3d12b455f90d1aa312ca028ac","headers":{"Accept-Language":["en-US,en;q=0.7,de-AT;q=0.3"],"Connection":["keep-alive, Upgrade"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"],"Accept-Encoding":["gzip, deflate"],"Sec-Websocket-Extensions":["permessage-deflate"],"Accept":["*/*"],"Pragma":["no-cache"],"Cache-Control":["no-cache"],"Upgrade":["websocket"],"Sec-Websocket-Version":["13"],"Origin":["http://192.168.1.101:8188"],"Sec-Websocket-Key":["k72s8QRipsT6yoKJQwv9MA=="],"Cookie":[]}},"duration":0.001642951,"status":502,"err_id":"tp6ne2h6p","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/comfyui.log <==
supervisor_1  | ### Loading: ComfyUI-Manager (V2.9)
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/supervisor.log <==
supervisor_1  | 2024-03-10 17:02:49,846 INFO success: comfyui entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
supervisor_1  |
supervisor_1  | ==> /var/log/supervisor/comfyui.log <==
supervisor_1  | ### ComfyUI Revision: 2057 [65397ce6] | Released on '2024-03-10'
supervisor_1  |
supervisor_1  | Import times for custom nodes:
supervisor_1  |    0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
supervisor_1  |
supervisor_1  | Starting server
supervisor_1  |
supervisor_1  | To see the GUI go to: http://127.0.0.1:18188
supervisor_1  | [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
supervisor_1  | [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
supervisor_1  | [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
supervisor_1  | [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json

[ERROR] An error occurred while retrieving information for the 'CR Select Font' node.
supervisor-1  | Traceback (most recent call last):
supervisor-1  |   File "/opt/ComfyUI/server.py", line 420, in get_object_info
supervisor-1  |     out[x] = node_info(x)
supervisor-1  |   File "/opt/ComfyUI/server.py", line 398, in node_info
supervisor-1  |     info['input'] = obj_class.INPUT_TYPES()
supervisor-1  |   File "/opt/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes/nodes/nodes_graphics_text.py", line 467, in INPUT_TYPES
supervisor-1  |     file_list = [f for f in os.listdir(font_dir) if os.path.isfile(os.path.join(font_dir, f)) and f.lower().endswith(".ttf")]
supervisor-1  | FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/fonts/truetype'

docker cp /usr/share/fonts/truetype/dejavu containerID:/usr/share/fonts/truetype

Traceback (most recent call last):
  File "/opt/ComfyUI/main.py", line 72, in <module>
    import execution
  File "/opt/ComfyUI/execution.py", line 12, in <module>
    import nodes
  File "/opt/ComfyUI/nodes.py", line 20, in <module>
    import comfy.diffusers_load
  File "/opt/ComfyUI/comfy/diffusers_load.py", line 4, in <module>
    import comfy.sd
  File "/opt/ComfyUI/comfy/sd.py", line 5, in <module>
    from comfy import model_management
  File "/opt/ComfyUI/comfy/model_management.py", line 114, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
  File "/opt/ComfyUI/comfy/model_management.py", line 83, in get_torch_device
    return torch.device(torch.cuda.current_device())
  File "/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/torch/cuda/__init__.py", line 674, in current_device
    _lazy_init()
  File "/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/torch/cuda/__init__.py", line 247, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
Starting ComfyUI...
** ComfyUI start up time: 2023-10-31 10:39:32.162144

Prestartup times for custom nodes:
   0.0 seconds: /opt/ComfyUI/custom_nodes/ComfyUI-Manager

Traceback (most recent call last):
  File "/opt/ComfyUI/main.py", line 72, in <module>
    import execution
  File "/opt/ComfyUI/execution.py", line 12, in <module>
    import nodes
  File "/opt/ComfyUI/nodes.py", line 20, in <module>
    import comfy.diffusers_load
  File "/opt/ComfyUI/comfy/diffusers_load.py", line 4, in <module>
    import comfy.sd
  File "/opt/ComfyUI/comfy/sd.py", line 5, in <module>
    from comfy import model_management
  File "/opt/ComfyUI/comfy/model_management.py", line 114, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
  File "/opt/ComfyUI/comfy/model_management.py", line 83, in get_torch_device
    return torch.device(torch.cuda.current_device())
  File "/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/torch/cuda/__init__.py", line 674, in current_device
    _lazy_init()
  File "/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/torch/cuda/__init__.py", line 247, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
Starting ComfyUI...

Starting ComfyUI...
WARNING: No ICDs were found. Either,
- Install a conda package providing a OpenCL implementation (pocl, oclgrind, intel-compute-runtime, beignet) or
- Make your system-wide implementation visible by installing ocl-icd-system conda package.
** ComfyUI startup time: 2024-03-02 21:15:23.498445
** Platform: Linux
** Python version: 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0]
** Python executable: /opt/micromamba/envs/comfyui/bin/python
** Log path: /opt/ComfyUI/comfyui.log
Prestartup times for custom nodes:
   0.0 seconds: /opt/ComfyUI/custom_nodes/ComfyUI-Manager
Traceback (most recent call last):
  File "/opt/ComfyUI/main.py", line 76, in
    import execution
  File "/opt/ComfyUI/execution.py", line 11, in
    import nodes
  File "/opt/ComfyUI/nodes.py", line 20, in
    import comfy.diffusers_load
  File "/opt/ComfyUI/comfy/diffusers_load.py", line 3, in
    import comfy.sd
  File "/opt/ComfyUI/comfy/sd.py", line 4, in
    from comfy import model_management
  File "/opt/ComfyUI/comfy/model_management.py", line 118, in
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
  File "/opt/ComfyUI/comfy/model_management.py", line 87, in get_torch_device
    return torch.device(torch.cuda.current_device())
  File "/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/torch/cuda/__init__.py", line 787, in current_device
    _lazy_init()
  File "/opt/micromamba/envs/comfyui/lib/python3.10/site-packages/torch/cuda/__init__.py", line 302, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW
/opt/ai-dock/bin/supervisor-comfyui.sh: line 17:   831 Killed                  /usr/bin/python3 /opt/ai-dock/fastapi/logviewer/main.py -p $LISTEN_PORT -r 5 -s "${SERVICE_NAME}" -t "Preparing ${SERVICE_NAME}"  (wd: /root)

ghcr.io/ai-dock/comfyui:latest
without PROVISIONING_SCRIPT
WORKSPACE_SYNC first time true , after true/false result down

2024-02-29T07:57:13Z create pod network
2024-02-29T07:57:13Z create container ghcr.io/ai-dock/comfyui:latest
2024-02-29T07:57:14Z latest Pulling from ai-dock/comfyui
2024-02-29T07:57:14Z Digest: sha256:47f033dddfba20771da60cfc3bc403812f88cb5b3b03ce6e17de7fa9272e3a50
2024-02-29T07:57:14Z Status: Image is up to date for ghcr.io/ai-dock/comfyui:latest
2024-02-29T07:57:14Z start container

...
2024-02-29T07:51:18.335204123Z 
2024-02-29T07:51:18.626160221Z chown: changing ownership of '/workspace/': Operation not permitted
2024-02-29T07:51:18.653602734Z chown: changing ownership of '/workspace//.ai-dock-permissions-test': Operation not permitted
2024-02-29T07:51:19.416355191Z chown: changing ownership of '/workspace/home/user': Operation not permitted
2024-02-29T07:51:19.451593327Z useradd: warning: the home directory /workspace/home/user already exists.
2024-02-29T07:51:19.451656302Z useradd: Not copying any file from skel directory into it.
2024-02-29T07:51:19.529256483Z usermod: group 'render' does not exist
2024-02-29T07:51:19.534188717Z usermod: group 'sgx' does not exist
2024-02-29T07:51:19.544032612Z mkdir: cannot create directory ‘/workspace/home/user/.ssh’: File exists
2024-02-29T07:51:19.551713395Z cp: not writing through dangling symlink '/workspace/home/user/.ssh'
2024-02-29T07:51:19.559099834Z chown: changing ownership of '/workspace/home/user/.ssh': Operation not permitted
2024-02-29T07:51:19.566694095Z chmod: cannot access '/workspace/home/user/.ssh/authorized_keys': No such file or directory
2024-02-29T07:51:19.589076075Z chmod: cannot access '/home/user-linux/.ssh/authorized_keys': Too many levels of symbolic links

2024-02-09T13:31:26.016742103Z ==> /var/log/supervisor/storagemonitor.log <==
2024-02-09T13:31:26.016748663Z error: could not lock config file /home/user/.gitconfig: Permission denied
2024-02-09T13:31:26.016753303Z Starting storage monitor..
2024-02-09T13:31:26.016757913Z ln: failed to create symbolic link '/runpod-volume/storage/README': Permission denied
2024-02-09T13:31:26.016763463Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016777443Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/controlnet/control_v11p_sd15_openpose.pth': No such file or directory
2024-02-09T13:31:26.016781692Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016785932Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/embeddings/bad_prompt_version2-neg.pt': No such file or directory
2024-02-09T13:31:26.016790592Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016795122Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/embeddings/easynegative.safetensors': No such file or directory
2024-02-09T13:31:26.016798732Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016802422Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/embeddings/ng_deepnegative_v1_75t.pt': No such file or directory
2024-02-09T13:31:26.016805872Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016811022Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/ultralytics/bbox/face_yolov8m.pt': No such file or directory
2024-02-09T13:31:26.016816342Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016821142Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/ultralytics/segm/sam_vit_b_01ec64.pth': No such file or directory
2024-02-09T13:31:26.016825712Z mkdir: cannot create directory ‘/runpod-volume/storage/stable_diffusion’: Permission denied
2024-02-09T13:31:26.016830732Z ln: failed to create symbolic link '/runpod-volume/storage/stable_diffusion/models/vae/vae-ft-mse-840000-ema-pruned.safetensors': No such file or directory
2024-02-09T13:31:26.016841892Z Setting up watches.  Beware: since -r was given, this may take a while!
2024-02-09T13:31:26.016846572Z Watches established.