rabbitmq / rabbitmq-prometheus Goto Github PK

View Code? Open in Web Editor NEW

145.0 27.0 109.0 35.83 MB

A minimalistic Prometheus exporter of core RabbitMQ metrics

License: Other

Makefile 21.80% Erlang 47.27% Dockerfile 11.44% Shell 10.31% Vim Snippet 9.19%

rabbitmq plugin prometheus metrics monitoring

rabbitmq-prometheus's Introduction

This was migrated to https://github.com/rabbitmq/rabbitmq-server

All issues have been transferred, archiving.

Prometheus Exporter of Core RabbitMQ Metrics

Getting Started

This is a Prometheus exporter of core RabbitMQ metrics, developed by the RabbitMQ core team. It is largely a "clean room" design that reuses some prior work from Prometheus exporters done by the community.

Project Maturity

This plugin is new as of RabbitMQ 3.8.0.

Documentation

See Monitoring RabbitMQ with Prometheus and Grafana.

Installation

This plugin is included into RabbitMQ 3.8.x releases. Like all plugins, it has to be enabled before it can be used:

To enable it with rabbitmq-plugins:

rabbitmq-plugins enable rabbitmq_prometheus

Usage

See the documentation guide.

Default port used by the plugin is 15692 and the endpoint path is at /metrics. To try it with curl:

curl -v -H "Accept:text/plain" "http://localhost:15692/metrics"

In most environments there would be no configuration necessary.

See the entire list of metrics exposed via the default port.

Configuration

This exporter supports the following options via a set of prometheus.* configuration keys:

prometheus.return_per_object_metrics returns individual (per object) metrics that are not aggregated (default is false).
prometheus.path defines a scrape endpoint (default is "/metrics").
prometheus.tcp.* controls HTTP listener settings that match those used by the RabbitMQ HTTP API
prometheus.ssl.* controls TLS (HTTPS) listener settings that match those used by the RabbitMQ HTTP API

Sample configuration snippet:

# these values are defaults
prometheus.return_per_object_metrics = false
prometheus.path = /metrics
prometheus.tcp.port =  15692

When metrics are returned per object, nodes with 80k queues have been measured to take 58 seconds to return 1.9 million metrics in a 98MB response payload. In order to not put unnecessary pressure on your metrics system, metrics are aggregated by default.

When debugging, it may be useful to return metrics per object (unaggregated). This can be enabled on-the-fly, without restarting or configuring RabbitMQ, using the following command:

rabbitmqctl eval 'application:set_env(rabbitmq_prometheus, return_per_object_metrics, true).'

To go back to aggregated metrics on-the-fly, run the following command:

rabbitmqctl eval 'application:set_env(rabbitmq_prometheus, return_per_object_metrics, false).'

Contributing

See CONTRIBUTING.md.

Makefile

This project uses erlang.mk, running make help will return erlang.mk help.

To see all custom targets that have been documented, run make h.

For Bash shell autocompletion, run eval "$(make autocomplete)", then type make a<TAB> to see all Make targets starting with the letter a, e.g.:

$ make a<TAB
ac               all.coverdata    app-build        apps             apps-eunit       asciidoc-guide   autocomplete
all              app              app-c_src        apps-ct          asciidoc         asciidoc-manual

Copyright

rabbitmq-prometheus's People

Contributors

Stargazers

Watchers

Forkers

lechen26 eaglewa naveenmykaarma jaylevin skycyan sanderegg jborza marcialrosales yanghaiji vveliev-tc 5p1ke roni75020 charli117 caliahub guozilong110 papr8ka humanteq margaale arnaspet starboyate stu233win obsessionsys xaclincoln leetal sdaras kazimunal tteat idanmel rindanikrutarth cdudevops clivern pwbprogramer xiechuanj tmatias sadjunky weetime pixelrobots sventschui ransoor2 laashub-soa aakcht itsx yuvalavidor xuxanwan jianzhiunique normanzhang maduradissanayake tungtv202 lweiiii diegorodrigues-elo mescu74 samarius nicolaiev kirilllk tanshion srinivreddy fgruntjes pathcl ironykins evenchan86 running-g sanditriana devops-man cbankston kaoanaa tharlesson xuyuanlian isabella232 vczyh xuhaiqing arianium cookie993 togas maciejbandykex7 scorpion ise7en kudosqujo abadjanier d-floyd ops-mayi truyenhv iamabhishek-dubey 95jinhong sergeytailoredio 51n15t9r kabathla liuwenzhi0318 spetek amarkantgupta1 10088 cat-rat silinaz lurenjie1 alexisduf a1198457636 falconertc richard-chivers tryweirdier ponponon gaodong0563

rabbitmq-prometheus's Issues

Expose a metric that reports federated exchanges and if they are in sync

Motivation

With more complex setups the federation plugin is used in many setups.
As the external exporter from https://github.com/kbudde/rabbitmq_exporter I would like the possibility to expose metrics for federations.

So i would like support for the rabbitmq_prometheus to serve metrics for the federation plugin if the plugin is enabled.

Failed to retrieve metrics when Accept header specifies extension

RabbitMQ prometheus endpoint returns status code 406 when request Accept header specifies additional parameters (version):

curl -v -s http://localhost:15692/metrics -H "Accept: text/plain;version=0.0.4" > /dev/null

* TCP_NODELAY set
> GET /metrics HTTP/1.1
> Host: localhost:15692
> User-Agent: curl/7.54.0
> Accept: text/plain;version=0.0.4
> 
< HTTP/1.1 406 Not Acceptable
< content-length: 0
< date: Thu, 24 Oct 2019 14:21:53 GMT
< server: Cowboy
< 
...

Other Accept parameters are processed by the prometheus endpoint without problems (including multiple values):

curl -v -s http://localhost:15692/metrics -H "Accept: */*,text/plain;q=0.9" > /dev/null

* TCP_NODELAY set
> GET /metrics HTTP/1.1
> Host: localhost:15692
> User-Agent: curl/7.54.0
> Accept: */*,text/plain;q=0.9
> 
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 1443970
< content-type: text/plain; version=0.0.4
< date: Thu, 24 Oct 2019 14:26:18 GMT
< server: Cowboy
< 
{ [1080 bytes data]
...

Authentication

Hi,

I don't see anywhere where we can protect the /metric endpoint with some authentication (basic, cert, ...).

It that even possible ?

Thanks

Get timestamp of queue using rabbitmq_management

I want to get timestamp of queue using rabbitmq-prometheus plugin and rabbitmq_message_timestamp.

Bcz I want know old message..

If I use rabbitmq_message_timestamp plugin, I could not get the metric(queue timestamp).
But When I use external exporter- kbudde/rabbitmq_exporter- , I can get the metric.

I want to use rabbitmq_message_timestamp plugin and rabbitmq-prometheus plugin together..

plz help me..

Metrics missing for distribution buffer busy limit - zdbbl

Now that erlang/otp#2270 & deadtrickster/prometheus.erl#94 have been merged and released, we want to expose and visualise the distribution buffer busy limit (a.k.a. zdbbl).

The goal is to be able to measure how busy a particular distribution link is, so that users of RabbitMQ would know when increasing the inter-node communication buffer size would be beneficial. When implemented, this feature will be a good companion to RabbitMQ Runtime Tuning - Inter-node Communication Buffer Size.

This was created part of rabbitmq/tgir#9. The expectation is that one of the viewers will implement this feature, by following the video. Hopefully, this will inspire others to contribute features & fixes to RabbitMQ 🙌🏻

Thanks Matt E for reminding us about this outstanding issue

Fix references in queue_metrics ETS table

The core metrics collector reads properties incorrectly from the queue_metrics ETS table, by prepending queue_ to properties that don't have this: e.g. queue_consumers instead of consumers.

Review all queue_metrics properties and fix.

Use the old version of service support v3.6.1

Currently don't want to by the upgrade version to solve the plug-in support, what better way?
I have put the plugins copy the contents to the old version but not properly installed.

No exchange stats (published in/published out)

rabbitmq-prometheus plugin currently doesn't return exchange stats (published in/published out).

Is it possible to add this feature?

Requests fail with an exception on Erlang 23

Facts

Upgrade done today on Erlang. With a following reboot. RabbitMQ is clustered, cluster status OK.

# grep "May 15" /var/log/yum.log
May 15 11:18:21 Updated: erlang-23.0-1.el7.x86_64

Installed active and inactive plugins

sudo rabbitmq-plugins list | grep -i "\[e"
[E*] rabbitmq_management               3.8.3
[e*] rabbitmq_management_agent         3.8.3
[E*] rabbitmq_prometheus               3.8.3
[e*] rabbitmq_web_dispatch             3.8.3

Installed packages

erlang.x86_64 23.0-1.el7 @rabbitmq_erlang
rabbitmq-server.noarch 3.8.3-1.el7 @/rabbitmq-server-3.8.3-1.el7.noarch

Plugin listening on 15692

netstat -lnpt | grep 156
tcp        0      0 0.0.0.0:15692           0.0.0.0:*               LISTEN      1150/beam.smp
tcp        0      0 0.0.0.0:15672           0.0.0.0:*               LISTEN      1150/beam.smp

Using curl, causes a crash.

$ curl -v -H "Accept:text/plain" "http://localhost:15692/metrics"
* About to connect() to localhost port 15692 (#0)
*   Trying ::1...
* Förbindelse vägras
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 15692 (#0)
> GET /metrics HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:15692
> Accept:text/plain
>
< HTTP/1.1 500 Internal Server Error
< content-length: 0
<
* Connection #0 to host localhost left intact

Log segment from the crash


2020-05-15 18:03:16.573 [error] <0.10987.3> CRASH REPORT Process <0.10987.3> with 0 neighbours crashed with reason: {badarg,[]}
2020-05-15 18:03:16.573 [error] <0.10986.3> Ranch listener rabbit_web_dispatch_sup_15692, connection process <0.10986.3>, stream 1 had its request process <0.10987.3> exit with reason badarg and stacktrace []

New style config support

We've just soft deprecated classic config files but this plugin does not support new style configuration for any settings.

Scraping endpoing: 169K queues causes /metrics not to load

We are using rabbit 3.8.2 with about 169k queues and when hitting /metrics it times out. Maybe a way to ignore queue metrics for large setups? I assume that is why it is not returned.

Any thoughts would be appropriated.

Is it possible to disable some metrics from being exposed?

Monitor queues using wildcard

Now I have this alert for Prometheus:

- alert: RabbitmqTooManyMessagesInQueue
    expr: rabbitmq_queue_messages_ready{queue="my-queue"} > 1000
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "Rabbitmq too many messages in queue (instance {{ $labels.instance }})"
      description: "Queue is filling up (> 1000 msgs)\n  VALUE = {{ $value }}\n  LABELS: {{ $labels }}"

Notice than I have:
{queue="my-queue"}

To monitor all queues I can use this line, right?
rabbitmq_queue_messages_ready

But I want to monitor queues using wildcard.
For example:
{queue="guest_*"}

Now I need to add each queues separately, but it is a lot of them and new queues becomes often.
This feature will help a lot.
Thank you.

A way to reduce the number of metric lines emitted

See #24 and #25 for the background.

The scraping endpoint serves a good dozen of metrics per stats-emitting entity and not all of them may be used. For example, with 100s of thousands of queues and connections, the number of lines in the response goes into many millions and data transfer goes into hundreds of MiBs, which is above what default settings of this plugin and Prometheus can realistically handle.

One way would be to allow excluding groups, something like this:

prometheus.collectors.opt-out.groups.1 = channel_exchange_metrics
prometheus.collectors.opt-out.groups.2 = channel_queue_metrics

but perhaps a more practical way would be to just delist specific metrics:

prometheus.collectors.opt-out.metrics.1 = queue_messages_paged_out_bytes
prometheus.collectors.opt-out.metrics.2 = queue_messages_ready_bytes

Worth noting that such fine-grained approach would certainly mess up some Grafana dashboard assumptions 🤷‍♂🤷‍♀ but allow some key metrics to be kept while halving the amount of data that has to be rendered, compressed and transferred.

Telegraf prometheus input fails with 406 Not Acceptable from Prometheus exporter endpoint

Hi,

I am trying to scrape the prometheus metrics from a RMQ 3.8.0 cluster using telegraf's prometheus input plugin.

When telegraf attempts to connect to the metrics endpoint on RMQ, RMQ responds with 406 Not Acceptable.

Below is a trace of the request/response from the client/server:

Request (Telegraf Prometheus Input Plugin):

T 2019/10/14 12:25:02.854993 10.6.0.227:38254 -> 10.6.0.249:15692 [AP]
GET /metrics HTTP/1.1
Host: 10.6.0.249:15692
User-Agent: Go-http-client/1.1
Accept: application/vnd.google.protobuf;proto=io.prometheus.client.MetricFamily;encoding=delimited;q=0.7,text/plain;version=0.0.4;q=0.3
Accept-Encoding: gzip
Connection: close

Response (RMQ /metrics endpoint)

T 2019/10/14 12:25:02.856160 10.6.0.249:15692 -> 10.6.0.227:38254 [AP]
HTTP/1.1 406 Not Acceptable
connection: close
content-length: 0
date: Mon, 14 Oct 2019 11:25:02 GMT
server: Cowboy

The logs from the telegraf client are as follows:

2019-10-14T11:25:02Z E! [inputs.prometheus]: Error in plugin: http://10.6.0.249:15692/metrics returned HTTP status 406 Not Acceptable
2019-10-14T11:25:10Z D! [outputs.file] buffer fullness: 0 / 10000 metrics. 
2019-10-14T11:25:14Z E! [inputs.prometheus]: Error in plugin: http://10.6.0.249:15692/metrics returned HTTP status 406 Not Acceptable
2019-10-14T11:25:20Z D! [outputs.file] buffer fullness: 0 / 10000 metrics. 
2019-10-14T11:25:20Z E! [inputs.prometheus]: Error in plugin: http://10.6.0.249:15692/metrics returned HTTP status 406 Not Acceptable
2019-10-14T11:25:30Z D! [outputs.file] buffer fullness: 0 / 10000 metrics.

Would this have something to do with the "Accept:" header in the initial request?
Would it be possible to add support for the prometheus plugin in telegraf?

Thanks for adding the metrics endpoint for monitoring!! If you require any additional information please respond with what you need from me.

Thanks!

EDIT: Link to Telegraf Prometheus Input Plugin

Submit grafana dashboards to Grafana.com

Hi,

Would it be possible to submit the Grafana dashboards in the repository to Grafana.com? That way management and versioning are simpler and easier to automate.

Metrics for individual queues are no longer returned

I saw the issue #9
Can we used to have the queue information exposed few months ago.
But now we cannot see the queue information again. (we are now using the rabbimq 3.8.3)
Is the new version turned off the output or do I need to change any settings?

PS: I also tried installing the 3 node Rabbitmq-Prometheus from docker following the official documentation. I cannot see the individual queue information either.

Consumer utilisation is reported as NaN by this plugin but not CLI tools

I have 3 rabbitmq hosts which are united to the cluster. НА has following settings:

ha-mode:	all
ha-sync-mode:	automatic

When I turn on the setting of rabbitmqctl eval 'application:set_env(rabbitmq_prometheus, return_per_object_metrics, true).' I get the result by each queue, but the r_abbitmq_queue_consumer_utilisation_ parameter in all queues is visible in NaN. There are near 800 queues in the cluster and the collect_statistics_interval parameter is used in 15000.

Upon the request to queues via rabbitmqctl, I get the correct data:

$ rabbitmqctl --vhost=test list_queues name messages messages_ready state consumer_utilisation --formatter json | column -tx -c "name, messages, messages_ready, state,consumer_utilisation"| grep -e TestRequestQueue

{"name":"TestRequestQueue","messages":0,"messages_ready":0,"state":"running","consumer_utilisation":1.0}

But via a rabbitmqctl --vhost test eval 'io:format("QUEUE STATS:npnnQUEUE METRICS:npnn", [ets:tab2list(queue_stats), ets:tab2list(queue_metrics)]).' request I get the following:

 {{resource,<<"test">>,queue,<<"TestRequestQueue">>},
  [{idle_since,<<"2020-04-09 11:10:38">>},
   {consumer_utilisation,''},
   {policy,<<"ha-all">>},
   {operator_policy,''},
   {effective_policy_definition,[{<<"ha-mode">>,<<"all">>},
                                 {<<"ha-sync-mode">>,<<"automatic">>}]},
   {exclusive_consumer_tag,''},
   {single_active_consumer_tag,''},
   {consumers,6},
   {memory,58020},
   {slave_nodes,['rabbit@rabbit-02','rabbit@rabbit-03']},
   {synchronised_slave_nodes,['rabbit@rabbit-03',
                              'rabbit@rabbit-02']},
   {recoverable_slaves,''},
   {state,running},
   {garbage_collection,[{max_heap_size,0},
                        {min_bin_vheap_size,46422},
                        {min_heap_size,233},
                        {fullsweep_after,65535},
                        {minor_gcs,226}]},
   {messages_ram,0},
   {messages_ready_ram,0},
   {messages_unacknowledged_ram,0},
   {messages_persistent,0},
   {message_bytes,0},
   {message_bytes_ready,0},
   {message_bytes_unacknowledged,0},
   {message_bytes_ram,0},
   {message_bytes_persistent,0},
   {head_message_timestamp,''},
   {backing_queue_status,#{avg_ack_egress_rate => 0.08002020176251025,
                           avg_ack_ingress_rate => 0.08002020176251025,
                           avg_egress_rate => 0.08002020176251025,
                           avg_ingress_rate => 0.08002020176251025,
                           delta => [delta,undefined,0,0,undefined],
                           len => 0,mirror_seen => 0,mirror_senders => 6,
                           mode => default,next_seq_id => 70,q1 => 0,q2 => 0,
                           q3 => 0,q4 => 0,target_ram_count => infinity}},
   {messages_paged_out,0},
   {message_bytes_paged_out,0}]},

Upon seen via a rabbitmq_prometheus plugin I get the following:

$ curl -s localhost:15692/metrics | grep TestRequestQueue
rabbitmq_queue_messages_published_total{channel="<0.13860.0>",queue_vhost="test",queue="TestRequestQueue",exchange_vhost="test",exchange="TestExchange"} 11
rabbitmq_queue_messages_published_total{channel="<0.13864.0>",queue_vhost="test",queue="TestRequestQueue",exchange_vhost="test",exchange="TestExchange"} 10
rabbitmq_queue_messages_ready{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_unacked{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_process_reductions_total{vhost="test",queue="TestRequestQueue"} 287651
rabbitmq_queue_consumers{vhost="test",queue="TestRequestQueue"} 6
rabbitmq_queue_consumer_utilisation{vhost="test",queue="TestRequestQueue"} NaN
rabbitmq_queue_process_memory_bytes{vhost="test",queue="TestRequestQueue"} 57956
rabbitmq_queue_messages_ram{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_ram_bytes{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_ready_ram{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_unacked_ram{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_persistent{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_persistent_bytes{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_bytes{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_ready_bytes{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_unacked_bytes{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_paged_out{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_messages_paged_out_bytes{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_disk_reads_total{vhost="test",queue="TestRequestQueue"} 0
rabbitmq_queue_disk_writes_total{vhost="test",queue="TestRequestQueue"} 0

NaN value is displayed on each node of our cluster, accordingly - this value is transferred to the prometheus too. Could you, please, advise what could be the problem in?

Expose metric for number of publishers when metrics are aggregated

Since #28 we aggregate metrics by default, which means that we can no longer use the rabbitmq_channel_messages_published_total metric to count the number of publishers.

This is the metrics that we are returning when aggregating - we cannot count all publishers, the total count is 3, one per node:

This is what we return when metrics are enabled per object - we can correctly count 10 publishers:

We don't have the same problem with consumers as every node emits the rabbitmq_channel_consumers metric. @dcorbacho would it make sense to have a similar metric for publishing channels?

Add cluster label to metrics

When you may be running multiple rabbitmq clusters in the same environment in combination with prometheus service discovery mechanisms it become necessary to differentiate the metrics belonging to multiple clusters.

Doing it as https://github.com/kbudde/rabbitmq_exporter does, this is, exposing the cluster name as a label would be enough.

Expose individual queue metrics

Hello!

It would be nice to have more queue metrics exported, like in separate rabbitmq-exporter: https://github.com/kbudde/rabbitmq_exporter#queues---counter

Scrape requests fail with an exception on Erlang 23

When using the latest 3.8.4 build with OTP23, RabbitMQ starts up correctly but every request to /metrics results in the following error in the log:

2020-05-15 15:58:13.203 [error] <0.979.0> CRASH REPORT Process <0.979.0> with 0 neighbours crashed with reason: {badarg,[]}
2020-05-15 15:58:13.203 [error] <0.978.0> Ranch listener rabbit_web_dispatch_sup_15692, connection process <0.978.0>, stream 1 had its request process <0.979.0> exit with reason badarg and stacktrace []

For those with access to this image, you can reproduce the issue the following way:

$ docker run -p 127.0.0.1:15692:15692/tcp eu.gcr.io/cf-rabbitmq-core/rabbitmq-tanzu@sha256:968fbd203b1b06f537ac5ffddca8a39601115126a560f135510b1416622b49b9
$ curl localhost:15692/metrics # from another terminal of course

Exposed metrics filtering

Hello,
there are too many high cardinality metrics exposed:

docker run -v enabled_plugins:/etc/rabbitmq/enabled_plugins -d --rm -p 15692:15692 rabbitmq:3.8.3-management
curl -s localhost:15692/metrics | egrep -v '^#' | sed -r 's/([a-z_]+).*/\1/' | sort | uniq -c | sort -n
...
      1 telemetry_scrape_encoded_size_bytes_sum
      1 telemetry_scrape_size_bytes_count
      1 telemetry_scrape_size_bytes_sum
      2 erlang_vm_memory_atom_bytes_total
      2 erlang_vm_memory_bytes_total
      2 erlang_vm_memory_processes_bytes_total
      5 erlang_vm_memory_system_bytes_total
    156 erlang_vm_msacc_alloc_seconds_total
    156 erlang_vm_msacc_aux_seconds_total
    156 erlang_vm_msacc_bif_seconds_total
    156 erlang_vm_msacc_busy_wait_seconds_total
    156 erlang_vm_msacc_check_io_seconds_total
    156 erlang_vm_msacc_emulator_seconds_total
    156 erlang_vm_msacc_ets_seconds_total
    156 erlang_vm_msacc_gc_full_seconds_total
    156 erlang_vm_msacc_gc_seconds_total
    156 erlang_vm_msacc_nif_seconds_total
    156 erlang_vm_msacc_other_seconds_total
    156 erlang_vm_msacc_port_seconds_total
    156 erlang_vm_msacc_send_seconds_total
    156 erlang_vm_msacc_sleep_seconds_total
    156 erlang_vm_msacc_timers_seconds_total
    912 erlang_vm_allocators

Metrics erlang_vm_msacc* are not even in the list of https://github.com/rabbitmq/rabbitmq-prometheus/blob/master/metrics.md
Metric erlang_vm_allocators should be aggregated by instance_no at least.
Could you please implement ability to filter out not needed metrics?

Example:
https://github.com/kbudde/rabbitmq_exporter#settings
see EXCLUDE_METRICS and also nice SKIP_* regexes

Populate queue names for queue metrics as a label

Hi All
I tried to integrate the rabbitmq metrics using the pivotal image here

image: pivotalrabbitmq/rabbitmq-prometheus:3.9.0-alpha.314-2020.04.22

I am able to see the metrics. Firstly Thanks for making it super easy to integrate, however, I can't seem to find queue name as one of the labels being populated.

Is this by design or am I missing something?

Default RabbitMQ node/color labelling regex

Hello, the documentation at https://www.rabbitmq.com/prometheus.html says:

Colour Labelling in Graphs
All metrics on all graphs are associated with specific node names. For example, all metrics drawn in green are for the node that contains 0 in its name, e.g. rabbit@rmq0. This makes is easy to correlate metrics of a specific node across graphs. Metrics for the first node, which is assumed to contain 0 in its name, will always appear as green across all graphs.
It is important to remember this aspect when using the RabbitMQ Overview dashboard. If a different node naming convention is used, the colours will appear inconsistent across graphs: green may represent e.g. rabbit@foo in one graph, and e.g. rabbit@bar in another graph.
When this is the case, the panels must be updated to use a different node naming scheme.

According to the documentation, node naming scheme used in K8S deployment example should work. The nodename there is rabbit@$(MY_POD_NAME).$(K8S_SERVICE_NAME).$(MY_POD_NAMESPACE).svc.cluster.local , so assuming $MY_POD_NAMESPACE doesn't have any numbers in it , the colour labelling should work fine. But actually it doesn't work, because the regex used in the dashboards is /^rabbit@\w+0/ for zero node. Maybe it's possible to change the regex to something like /^rabbit@.*?0.*/, /^rabbit@.*?1.*/, etc ? With the change color labelling in the dashboards should work for with kubernetes deployments without additional changes in the dashboards.