Comments (4)
@sylvainlap Thank you for reporting this issue.
I believe that retryDelay
should be also respected. But from your experience, that you reported, it sounds to me that something has definitely went wrong regarding the retryDelay
option.
I will investigate the issue ASAP.
Further updates will be posted later, so keep tuned.
from redis-smq.
This is not a bug.
After a quick checkup, I came to the conclusion that issue you reported is actually a normal behavior and it is not a bug. Let me make it more clear to you.
Here the script I used for testing:
import { Consumer, Producer, Message, QueueManager } from 'redis-smq';
Message.setDefaultConsumeOptions({
ttl: 0,
retryThreshold: 5,
retryDelay: 2000,
});
const queue = `queue_${Date.now()}`;
const consume = () => {
const consumer = new Consumer();
consumer.consume(
queue,
(msg) => {
console.log(`Received message ID ${msg.getId()} at ${Date.now()}`);
throw new Error('Explicit error');
},
(err) => {
if (err) console.log(err);
},
);
consumer.run();
};
const produce = () => {
const producer = new Producer();
producer.run((err) => {
if (err) console.log(err);
else {
const msg = new Message().setBody('test message').setQueue(queue);
producer.produce(msg, (err) => {
if (err) console.log(err);
else
console.log(
`Message ID ${msg.getId()} has been published at ${Date.now()}.`,
);
});
}
});
};
QueueManager.createInstance({}, (err, queueManager) => {
if (err) console.log(err);
else {
queueManager?.queue.create(queue, false, (err) => {
if (err) console.log(err);
else {
produce();
consume();
}
});
}
});
You should first understand the way RedisSMQ works.
RedisSMQ uses workers for various background tasks and one of those workers is the DelayWorker
which responsibility is to handle unacknowledged messages that need to be re-queued with a certain delay, given that retryThreshold
is > 1
and retryDelay > 0
.
Another crucial moment that needs to be taken into account is that the workers are allowed to run only from a single consumer instance, given many consumer instances. So for each consumer instance, before running background workers, it tries to indefinitely acquire a global lock. Once the lock is acquired, the workers are launched. An acquired lock is set to expire after 60 seconds.
When you run the script above for the first time, a fresh lock is acquired and the DelayWorker
is immediately launched. As a result the retryDelay
is respected. If you terminate the running script by ^C (control+c)
from the console, the consumer instance crashes and the lock is not released.
After that if you run the script for the second time, the lock is not acquired immediately because the previous lock, which was not released, has not yet expired. Once the lock is expired, the consumer acquires a new lock and from then the message is re-queued with respect to retryDelay
.
Additionally, it is worth to mention that JavaScript clocks are not precise and are allowed to drift randomly. So when you set the retryDelay
to some value like 2000
, it is not guaranteed that the message will be re-queued exactly after 2 secs of delay.
from redis-smq.
Closing as resolved.
from redis-smq.
Great response ! Thanks !
from redis-smq.
Related Issues (20)
- LockManagerAcquireError: could not acquire a lock. HOT 3
- consumer for all queues HOT 2
- Managed redis connection keeps throwing error HOT 3
- what is difference between setConsumeTimeout and setTTL HOT 1
- Configure retry/consume parameters on consumer side HOT 4
- Config is ignored HOT 2
- Question about Features which BullMQ Pro has (Batching, Concurrency, Sandboxed Worker and Message Handlers) HOT 3
- SyntaxError: Invalid or unexpected token on server start HOT 1
- Delete a message/process in a queue HOT 1
- Fanout to multiple subscribers question HOT 7
- Implement Reliable Pub/Sub pattern HOT 1
- Configuration is not present in the redis-smq HOT 2
- Not working with docker compose HOT 5
- MaxListenersExceededWarning: Possible EventEmitter memory leak detected HOT 15
- [Question] How to avoid duplicated tasks? HOT 6
- LockManagerExtendError: Acquired lock could not be extended HOT 3
- [Feature] Support option for a missing interval in the scheduler. HOT 4
- How to delete message in consumer HOT 4
- Error: ENOENT: no such file or directory, open '../lua/zpophgetrpush.lua HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redis-smq.