c45tr0 / agenda-db-sequelize Goto Github PK

In high-demand scenarios many jobs run more than once. I have observed that jobs that run more than once tend to be locked many times (in quick succession with MS between locks) and run nearly at the same time, again with MS between duplicate runs.

This is likely a Postgres issue: in some race conditions, jobs begin processing before the UPDATE transaction to set the job lockedAt has completed. It seems like with job queues in PG, there are a few issues to account for in terms of atomicity and isolation.

I believe that all SELECTS and UPDATES on lockedAt rows need a FOR UPDATE so that a row level lock is made in PG to ensure any updates to lockedAt are fully committed before continuing processing.