Timed out waiting for UP message from ForkProcess about querybook HOT 7 CLOSED

+1
faced the same issue in prod deployment, but on the local machine with dev build it works as expected.

there is a different behaviour in worker startup script depending on the presense of production flag.
https://github.com/pinterest/querybook/blob/master/querybook/server/tasks/all_tasks.py

so as workaround I've added production=false to the worker env variables which helped. not sure what side effects it may have added.

from querybook.

Thx. I have prod environment too.

from querybook.

I experienced the same issue. Adding kombu==5.3.1 in requirements/base.txt fixed the issue in my production deployment.

from querybook.

@mlivirov @adamstruck thanks for sharing findings.

for the flag of production=false in all_tasks.py, the only difference is it will run the query execution cleanup when production is true. This reminds me of what @baumandm mentioned in https://querybook.slack.com/archives/CHCNR2Y5B/p1695153621351919.

for the package of kombu, it seems to be a dep package of celery, and we do did a celery version upgrade. Does changing kombu to 5.3.1 work for other people?

from querybook.

I ran into this issue running locally (via make) only with production=true set, while I was investigating the worker startup issue.

Fortunately we haven't seen it in our production instance, but we are building our own Docker image and it's possible that the requirements are slightly different.

from querybook.

Hey all, we faced a similar issue to this. Turns out using celery with -P gevent would resolve this. I am defaulting to use gevent for workers to avoid this issue in future

from querybook.

turns out this is should be the issue celery/kombu#1785
will fix to 5.3.1 for now

from querybook.