RedisDynoQueue.pop() not returning HTTP tasks that are in the queue about dyno-queues HOT 6 CLOSED

we have configured the redis DB to the netflix docker containers. And a simple workflow with HTTP task is struck in scheduled status for long time. Any ideas?

from dyno-queues.

@rickfish why was this closed out? We're seeing tasks being stuck in a SCHEDULED state

from dyno-queues.

I believe I closed this because I was asked to open it on the Redis issues list. I honestly don't remember but nothing was ever resolved. Once we started using the SQL database vs memory we didn't need to pursue further.

from dyno-queues.

Did anybody find out how to solve this?
We're running Conductor on GKE with a setup like this docker-compose (albeit on Kubernetes pods/services):
https://github.com/flaviostutz/conductor-server

In the Conductor UI we can clearly see that SIMPLE tasks get scheduled, and they appear in the "Task Queue" - however when polling from a worker (like this:) it always returns "204 No Content" even if there are items in the queue so the tasks never get run:

curl -X GET http://conductor-server:8080/api/tasks/poll/task_name_here

@rickfish would you be able to give us some pointers on how you got this fixed? Ie. what would be the way to configure Conductor/Dynomite/dyno-queues to run on SQL instead (if this is the easiest way to fix the issue)

from dyno-queues.

Right, we never got this addressed as it is a problem with the redis dyno-queues, not conductor itself if I remember correctly. We instead decided that for anything that has to work well we would use MySQL (and since then we are now using Postgres).

Our setup for Postgres is this:

db=postgres
jdbc.url=jdbc:postgresql://<host>:<port>/<dbname>?ssl=false&currentSchema=<schema-name>
jdbc.username=<username>
jdbc.password=<password>

from dyno-queues.

Thanks @rickfish. I actually tried switching to MySQL as a provider but it was a nightmare due to migrations in Conductor's source code seemingly being incompatible with all the mysql distros I could find.
I got the polling to work when upgrading to newest version of Conductor (v2.25.5), although it has other issues (they seem to be transitioning away from the ack endpoint, but right now it means that more than one worker can receive the same task for processing). This was using dyno-queues with v2.25.5.
It's good to know that Postgres can be used - I'll try that out if this isn't fixed:
Netflix/conductor#1534 (comment)

from dyno-queues.