"Event not found in queue" when deploying Gradio behind a proxy about gradio HOT 18 OPEN

you can try export no_proxy="localhost,127.0.0.1" before running the gradio app

from gradio.

I am also encountering this issue, running Gradio on Kubernetes with Ray Serve and a single replica:

(ProxyActor pid=7460)     await app(scope, receive, sender)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
(ProxyActor pid=7460)     response = await func(request)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
(ProxyActor pid=7460)     raw_response = await run_endpoint_function(
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
(ProxyActor pid=7460)     return await dependant.call(**values)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/gradio/routes.py", line 747, in queue_join
(ProxyActor pid=7460)     return await queue_join_helper(body, request, username)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/gradio/routes.py", line 765, in queue_join_helper
(ProxyActor pid=7460)     success, event_id = await blocks._queue.push(
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/gradio/queueing.py", line 225, in push
(ProxyActor pid=7460)     raise KeyError(
(ProxyActor pid=7460) KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

Here is the serve config.yaml:

proxy_location: EveryNode

http_options:
  host: 0.0.0.0
  port: 8000

applications:

- name: ChatBotGroup
  route_prefix: /generate
  import_path: demo:bot
  runtime_env: {
    "working_dir": "file:///root/app/app.zip",
    "env_vars": {
      "HF_HUB_ENABLE_HF_TRANSFER": "1",
    }
  }
  deployments:
  - name: ChatBot
    user_config:
      repo_id: "TheBloke/dolphin-2.5-mixtral-8x7b-GGUF"
      model_file: "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf"
      tensor_split: [0.6, 1, 1, 1]
    ray_actor_options:
      num_gpus: 4.0
    num_replicas: 2

- name: ChatClient
  route_prefix: /
  import_path: demo:app
  runtime_env: {
    "working_dir": "file:///root/app/app.zip",
    "env_vars": {
      "CONCURRENCY_LIMIT": "2",

      "TITLE": "Chat - Dolphin 2.5 Mixtral 8x7b",
      "CHAT_LABEL": "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf",
    }
  }
  deployments:
  - name: ChatIngress
    num_replicas: 1

Downgrading with pip install gradio==4.24.0 fixed the issue.

from gradio.

@shaojun (or anybody else who faced this issue), if you can provide more details about the proxy you're using and how it's configured, we might be able to provide a solution for this. （或遇到此问题的任何其他人），如果您可以提供有关您正在使用的代理及其配置方式的更多详细信息，我们也许能够为此提供解决方案。

Environment

• Operating System: Ubuntu 22.04
• Python Version: 3.10.14
• Gradio Version: 4.29.0

Issue Description

I encountered a KeyError: 'Event not found in queue' when running a Python program that uses Gradio on an Ubuntu 22.04 server, connected via SSH from a Windows machine(Same LAN). This setup involves a reverse proxy using the following environment variables:

export http_proxy=http://<windows-ip>:<port>
export https_proxy=http://<windows-ip>:<port>

where and are the IP and port of the proxy software on Windows.

Steps to Reproduce
Set up a reverse proxy using the above environment variables.
Run a Python program with Gradio on the server.

Error Message

KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

Temporary Solution
Adding export no_proxy="localhost,127.0.0.1" to the environment variables before running the Gradio app resolves the issue.

from gradio.

I have unset https_proxy and unset http_proxy, and then re-run and refresh the app web page, it works. so its confirmed caused by http proxy? but it does work in old version of gradio.

I have updated to gradio 4.31.1 and still see this error with http_proxy and https_proxy set.
And by unset http_proxy and unset https_proxy can fix this issue.

from gradio.

Hi @shaojun are you installing Gradio on multiple replicas? Or can you tell us more about how this Gradio app is deployed?

from gradio.

Hi @shaojun are you installing Gradio on multiple replicas? Or can you tell us more about how this Gradio app is deployed?

I just run with vscode debug as started with single python file, in my laptop.
I don't know what is multiple replicas.

from gradio.

This is super strange, I would not have expected this error. Can you try a fresh installation of gradio using pip instead of conda

cc @aliabid94 if you have any ideas

from gradio.

I do install the gradio by pip though i am using miniconda.
My ubuntu has a http and https proxy set, not sure it's related or not.

from gradio.

Ah yeah is it possible for you test without the proxy set?

from gradio.

I have unset https_proxy and unset http_proxy, and then re-run and refresh the app web page, it works.
so its confirmed caused by http proxy? but it does work in old version of gradio.

from gradio.

Downgrading with pip install gradio==4.24.0 fixed the issue.

This didn't work for me, but

you can try export no_proxy="localhost,127.0.0.1" before running the gradio app

This worked.

from gradio.

None of the above suggestion are working for me.

from gradio.

I have the same error, deploying gradio 4.27.0 on docker compose under FastAPI 0.110.3. I can access the Gradio chat example fine under localhost:8000/gradio on the host, but once I try to send a message, the same error occurs: KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

I run FastAPI using uvicorn: uvicorn gradio_app:app --host 0.0.0.0

In gradio_app.py:

from fastapi import FastAPI, Request
import gradio as gr
import requests
import datetime
CUSTOM_PATH = "/gradio"

app = FastAPI()

def greet(name):
    return "Hello " + name + "!"

io = gr.Interface(fn=greet, inputs="textbox", outputs="textbox")
gradio_app = gr.routes.App.create_app(io)
app.mount(CUSTOM_PATH, gradio_app)

I'm setting the following environment variables in the dockerfile (from python:3-slim):

ENV GRADIO_SERVER_NAME="0.0.0.0"
ENV COMMANDLINE_ARGS="--no-gradio-queue"

I can provide a full minimal example if needed.

from gradio.

you can try export no_proxy="localhost,127.0.0.1" before running the gradio app

Thanks, this works for me!

from gradio.

@shaojun (or anybody else who faced this issue), if you can provide more details about the proxy you're using and how it's configured, we might be able to provide a solution for this.

from gradio.

Which reverse proxy library /software are you using?

from gradio.

I'm using clash for windows(CFW), can this help?

from gradio.

Thanks @iwannabewater I think that should be enough for us to repro this. I'll take a look!

from gradio.