Comments (13)
I confirm that this fixes it for me as well, for both the exclusive and non-exclusive queue case.
I tried with weights of 34 and 100, and saw no issues.
Thanks for the quick response!
from rabbitmq-consistent-hash-exchange.
Thank you for your time.
Team RabbitMQ uses GitHub issues for specific actionable items engineers can work on. GitHub issues are not used for questions, investigations, root cause analysis, discussions of potential issues, etc (as defined by this team).
We get at least a dozen of questions through various venues every single day, often light on details.
At that rate GitHub issues can very quickly turn into a something impossible to navigate and make sense of even for our team. Because GitHub is a tool our team uses heavily nearly every day, the signal/noise ratio of issues is something we care about a lot.
Please post this to rabbitmq-users.
Thank you.
from rabbitmq-consistent-hash-exchange.
The latest revision of this plugin has been pretty extensively tested. I will try to reproduce but as most questions around this plugin, they likely scenario-specific or stem from misunderstanding around how it works.
Weights of 34 an 100 are rarely necessary, as the docs explain.
RabbitMQ tutorials are intentionally very basic and most depend on application startup order (consumers must be started first). We do have a couple of scripts here but none of the rest of the information.
Please move this to the mailing list.
from rabbitmq-consistent-hash-exchange.
The script above creates an exclusive queue and when it is stopped, the queue is deleted. This scenario was originally problematic in #37 due to transaction retries in the core that left this plugin no chance of updating its ring state. There are tests (in the integration suite and other) that do exactly that and they pass.
I managed to reproduce the warning with exclusive queues but it expectedly never happens with non-exclusive durable ones since there is no binding deletion concurrency. Weights or the implementation of this plugin as of #39 are not problematic. This comment explains what's really going on. We'll have to investigate first and 3.7.10 won't be delayed because of this.
from rabbitmq-consistent-hash-exchange.
This test seems to do exactly what shutting down the consumer above does.
I'm adding a variation of that test that uses a higher weight (say, 50). In case @dcorbacho and I uncover something we will reopen this or file a new issue.
from rabbitmq-consistent-hash-exchange.
I tried with non-exclusive queue (removed the 'exclusive' when declaring the queue on line 12 of receive_logs_topic.py). The same problem occurs if you delete the queue manually.
The instruction to reproduce are now:
python receive_logs_topic.py 34
<Ctrl-C>
-> manually delete the queue using the RabbitMQ admin website.
python receive_logs_topic.py 34
A weight of 50 works fine, please do the tests with the numbers I have identified as problematic (34 or 100).
I understand that high weights are (no longer?) necessary, but I thought it was a good idea to report so that other people don't hit the same issue.
from rabbitmq-consistent-hash-exchange.
Take a look at how the ring state is managed when a binding is removed. I find it hard to believe that there are magic weight values. We will add more cases.
from rabbitmq-consistent-hash-exchange.
@lukas-phaf we have a candidate fix. The tests with weights of 34, 50, 100 and greater numbers seem to pass with it. If I upload a plugin build, would you be able to give it a shot to confirm? Simply replace the plugin's archive (.ez) in your RabbitMQ distribution (I suggest using a one-off binary build).
from rabbitmq-consistent-hash-exchange.
I am willing to give it a try, but not clear on what you are asking me to do. So far, I have only installed RabbitMQ from RPM. Can you point me to instruction how to do a one-off binary build?
from rabbitmq-consistent-hash-exchange.
- Download a binary build
- Start it with
./sbin/rabbitmq-server
in the foreground - Reproduce the issue
- Delete
plugins/rabbitmq_consistent_hash_exchange-3.7.9.ez
, copy a file I will upload shortly to the same directory (thus replacing the plugin). - Stop the node, start it back
- Try to reproduce the issue
from rabbitmq-consistent-hash-exchange.
Below is a zip-compressed .ez file (GitHub doesn't support .ez file uploads). Please give it a shot, @dcorbacho and I have concluded that it's a sufficient fix. Thank you!
rabbitmq_consistent_hash_exchange-40.g392d964.ez.zip
from rabbitmq-consistent-hash-exchange.
@lukas-phaf OK, it will be included into 3.7.10 (due any time now) after we do some test suite refactoring. Thank you for reporting this with enough details to quickly reproduce.
from rabbitmq-consistent-hash-exchange.
3.7.10 is out and includes a fix.
from rabbitmq-consistent-hash-exchange.
Related Issues (20)
- Documentation Lacking around Routing Key Implications HOT 4
- Choosing the seed for generated points in the hash space HOT 6
- Binaries HOT 2
- Move test suite to Common Test
- Handling orphan queues HOT 2
- same message with same routing key goes to different consumer HOT 1
- Routing keys greater than 10 million when the system will lead to memory overflow HOT 3
- Use retries when waiting for Mnesia tables to load HOT 2
- Consistent hash exchange should keep same distribution after server restart HOT 15
- Exchange perfomance HOT 5
- routing to another exchanges HOT 1
- Message routing distribution among queues is not uniformly random HOT 23
- Is header hashing supported? HOT 3
- Highly non-uniform distribution of messages over queue when using UUID as routing key HOT 6
- Message Distribution is not uniform (at all) with weights less than 4 HOT 1
- Durable consistent hashing exchange cannot recover their previous state after node restart (3.8.4-specific) HOT 1
- Provide examples in more programming languages HOT 2
- Negative routing key should result in a channel error HOT 3
- Hash on AMQP Message Properties HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rabbitmq-consistent-hash-exchange.