deadtrickster / prometheus_process_collector Goto Github PK

Prometheus.io process collector in Erlang

License: MIT License

Makefile 7.14% C 4.11% Erlang 29.23% Shell 6.09% Emacs Lisp 1.85% C++ 51.58%

prometheus_process_collector's Introduction

Prometheus.io Process Collector

Collector which exports the current state of process metrics including cpu, memory, file descriptor usage and native threads count as well as the process start and up times.

FreeBSD;
Linux - uses /proc;
MacOS X (expiremental).

Installation

Hex package note: Because OTP strictly validates NIF version c_src was compiled with, I removed precompiled binaries from the Hex package. This means that OTP source code is needed to build this library.

Also collector needs to be registered (for example in default registry):

prometheus_registry:register_collector(prometheus_process_collector)

require Prometheus.Registry
Prometheus.Registry.register_collector(:prometheus_process_collector)

Example scrape:

# TYPE process_open_fds gauge
# HELP process_open_fds Number of open file descriptors.
process_open_fds 13
# TYPE process_max_fds gauge
# HELP process_max_fds Maximum number of open file descriptors.
process_max_fds 20000
# TYPE process_start_time_seconds gauge
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
process_start_time_seconds 1508230997
# TYPE process_uptime_seconds counter
# HELP process_uptime_seconds Process uptime in seconds.
process_uptime_seconds 76
# TYPE process_threads_total gauge
# HELP process_threads_total Process Threads count.
process_threads_total 22
# TYPE process_virtual_memory_bytes gauge
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
process_virtual_memory_bytes 3015249920
# TYPE process_resident_memory_bytes gauge
# HELP process_resident_memory_bytes Resident memory size in bytes.
process_resident_memory_bytes 1224400
# TYPE process_cpu_seconds_total counter
# HELP process_cpu_seconds_total Process CPU seconds total.
process_cpu_seconds_total{kind="utime"} 0.768
process_cpu_seconds_total{kind="stime"} 0.368
# TYPE process_max_resident_memory_bytes gauge
# HELP process_max_resident_memory_bytes Maximum resident set size used.
process_max_resident_memory_bytes 50192384
# TYPE process_noio_pagefaults_total counter
# HELP process_noio_pagefaults_total Number of page faules serviced without any I/O activity.
process_noio_pagefaults_total 16015
# TYPE process_io_pagefaults_total counter
# HELP process_io_pagefaults_total Number of page faults serviced that required I/O activity.
process_io_pagefaults_total 0
# TYPE process_swaps_total counter
# HELP process_swaps_total Number of times a process was "swapped" out of main memory.
process_swaps_total 0
# TYPE process_disk_reads_total counter
# HELP process_disk_reads_total Number of times the file system had to perform input.
process_disk_reads_total 0
# TYPE process_disk_writes_total counter
# HELP process_disk_writes_total Number of times the file system had to perform output.
process_disk_writes_total 80
# TYPE process_signals_delivered_total counter
# HELP process_signals_delivered_total Number of signals delivered.
process_signals_delivered_total 0
# TYPE process_voluntary_context_switches_total counter
# HELP process_voluntary_context_switches_total Number of times a context switch resulted due to a process voluntarily giving up the processor.
process_voluntary_context_switches_total 1131
# TYPE process_involuntary_context_switches_total counter
# HELP process_involuntary_context_switches_total Number of times a context switch resulted due to a higher priority process becoming runnable or because the current process exceeded its time slice.
process_involuntary_context_switches_total 208

Usage

You can register this collector manually using prometheus_process_collector/0,1 or use default_collectors env entry for prometheus.

With Prometheus Plugs - just add prometheus_process_collector dependency to top-level project (i.e. like here).

Integrations / Collectors / Instrumenters

Build

$ rebar3 compile

License

FreeBSD-specific part uses copy-modified code from standard utils (limits and procstat) or standard API in some places.

MIT

prometheus_process_collector's People

Contributors

Stargazers

Watchers

prometheus_process_collector's Issues

{on_load_function_failed,prometheus_process_collector} error

I'm building a Distillery release in a Docker container running Alpine Linux, and am getting an {on_load_function_failed,prometheus_process_collector} error.

Here's some environment information:

The bug happens in prometheus_process_collector version 1.4.0. Version 1.3.1 runs without any problems.
I've seen this happen with both Elixir 1.6.5 with OTP 20, and Elixir 1.7.3 with OTP 21
The bug happens on Alpine Linux in a Docker container
I'm trying to run the app in a Distillery release. I'm using Distillery 2.0.10.
I installed the g++ library in the Docker image so prometheus_process_collector would compile
prometheus_process_collector is a dependency of one application in an umbrella project.

The Docker container crashes when I try running docker run <image ID>. The entrypoint for the Docker container is the Distillery binary, and it's set to run in the foreground. I'm building my dependencies with mix do deps.get, compile.

The app runs fine when I run it outside of a Docker container in a macOS 10.13.6 environment.

Here's the crash report:

2018-10-13 19:50:41.388715 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{supervisor,kernel,['Argument__1']}},{pid,<0.1197.0>},{registered_name,[]},{error_info,{exit,{on_load_function_failed,prometheus_process_collector},[{init,run_on_load_handlers,0,[]},{kernel,init,1,[{file,"kernel.erl"},{line,212}]},{supervisor,init,1,[{file,"supervisor.erl"},{line,295}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}},{ancestors,[kernel_sup,<0.1171.0>]},{message_queue_len,0},{messages,[]},{links,[<0.1173.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,273}],[]]}
2018-10-13 19:50:41.388764 supervisor_report   #{label=>{supervisor,start_error},report=>[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{on_load_function_failed,prometheus_process_collector}},{offender,[{pid,undefined},{id,kernel_safe_sup},{mfargs,{supervisor,start_link,[{local,kernel_safe_sup},kernel,safe]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
2018-10-13 19:50:42.392873 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.1170.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,prometheus_process_collector}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,138}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}},{ancestors,[<0.1169.0>]},{message_queue_len,1},{messages,[{'EXIT',<0.1171.0>,normal}]},{links,[<0.1169.0>,<0.1167.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,193}],[]]}
2018-10-13 19:50:42.393766 std_info            #{label=>{application_controller,exit},report=>[{application,kernel},{exited,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,prometheus_process_collector}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,prometheus_process_collector}}},{kernel,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,prometheus_process_collector}}},{kernel,sta

I poked around in erl_crash.dump for a bit, but didn't know exactly what to look for. What can I do to help troubleshoot this?

Compiling issues with Linux Ubuntu x86_64 Linux docker

Hello there, Im having issues compiling the dependencies in the CI (a docker image)

Linux d5cc9b5c9cfc 4.4.0-134-generic #160-Ubuntu SMP Wed Aug 15 14:58:00 UTC 2018 x86_64 Linux

I'm using the following dependencies

 prometheus 4.2.2
  prometheus_ecto 1.4.1
  prometheus_ex 3.0.5
  prometheus_phoenix 1.2.1
  prometheus_plugs 1.1.5
  prometheus_process_collector 1.3.1

( I tried others without success)

it fail with

===> Compiling prometheus_process_collector
sh: exec: line 1: make: not found
===> Hook for compile failed!

==> MyApp
** (Mix) Could not compile dependency :prometheus_process_collector, "/root/.mix/rebar3 bare compile --paths "/my_app/_build/test/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector

In my local env (Mac) it works excellent.

Do you have any idea in how to fix ? (or things to try)

mix does not compile prometheas_process_collector 1.4.0 correctly

Using Elixirs project management tool mix I get an error make: g++: Command not found make: *** [Makefile:69: prometheus_process_collector_nif.o] Error 127 on compilation.

Here is the build output:

===> Compiling prometheus_process_collector
make: Entering directory '/app/deps/prometheus_process_collector/c_src'
g++ -O3 -finline-functions -fPIC -I /usr/lib/erlang/erts-9.3/include/ -I /usr/lib/erlang/lib/erl_interface-3.10.2/include -std=c++11 -Wall  -c -o prometheus_process_collector_nif.o prometheus_process_collector_nif.cc
make: g++: Command not found
make: *** [Makefile:69: prometheus_process_collector_nif.o] Error 127
make: Leaving directory '/app/deps/prometheus_process_collector/c_src'
===> Hook for compile failed!

==> my_app
** (Mix) Could not compile dependency :prometheus_process_collector, "/root/.mix/rebar3 bare compile --paths "/app/_build/prod/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"

This error only occurs since Version 1.4.0 using Elixir 1.7.3 and Erlang/OTP 20

Process metrics not available

I've included many prometheus integrations:

{:prometheus_ex, "~> 3.0"},
{:prometheus_ecto, "~> 1.4.0"},
{:prometheus_phoenix, "~> 1.2.0"},
{:prometheus_plugs, "~> 1.1.1"},
{:prometheus_process_collector, "~> 1.4.0"}

however I don't see the process metrics, is that expected?

I've tried also to add:

Prometheus.Registry.register_collector(:prometheus_process_collector)

to my app start function but I get

warning: function Prometheus.Registry.register_collector/1 is undefined or private. Did you mean one of:

      * "MACRO-deregister_collector"/2
      * "MACRO-register_collector"/2
      * "MACRO-register_collector"/3
      * "MACRO-register_collectors"/2
      * "MACRO-register_collectors"/3

  lib/iot/application.ex:39

[info] [swarm on nonode@nohost] [tracker:init] started
[info] Already up
** (Mix) Could not start application iot: exited in: Iot.Application.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (UndefinedFunctionError) function Prometheus.Registry.register_collector/1 is undefined or private
            (prometheus_ex) Prometheus.Registry.register_collector(:prometheus_process_collector)
            (iot) lib/iot/application.ex:39: Iot.Application.start/2

`erlang:load_nif` fails with `Library version (2.11) not compatible (with 2.10)`

For some reason we can't load compiled library on apline 3.4.0

bash-4.3# mix deps.clean prometheus_process_collector
* Cleaning prometheus_process_collector
bash-4.3# mix deps.get prometheus_process_collector
Running dependency resolution
* Getting prometheus_process_collector (Hex package)
  Checking package (https://repo.hex.pm/tarballs/prometheus_process_collector-1.0.0.tar)
  Using locally cached package
bash-4.3# mix deps.compile prometheus_process_collector
===> Compiling prometheus_process_collector
make: Entering directory '/build/app/deps/prometheus_process_collector/c_src'
make: '/build/app/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so' is up to date.
make: Leaving directory '/build/app/deps/prometheus_process_collector/c_src'
src/prometheus_process_collector.erl:15: Warning: behaviour prometheus_collector undefined

bash-4.3# erl
Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]

Eshell V7.3.1  (abort with ^G)
1> erlang:load_nif("deps/prometheus_process_collector/priv/prometheus_process_collector",0).
{error,{bad_lib,"Library version (2.11) not compatible (with 2.10)."}}

Do you have any idea what could be the issue here?

OTP 23 has removed -lerl_interface

Hi,

OTP 23 has removed -lerl_interface from erl_interface-4.0, only -lei remains.

As a result, builds fail like this:

210 ===> Compiling prometheus_process_collector
211 make: Entering directory '/builds/deps/prometheus_process_collector/c_src'
212 g++ -O3 -finline-functions -fPIC -I /usr/lib/erlang/erts-11.0/include/ -I /usr/lib/erlang/lib/erl_interface-4.0/include -std=c++11 -Wall  -c -o prometheus_process_collector_nif.o prometheus_process_collector_nif.cc
213 g++ -O3 -finline-functions -fPIC -I /usr/lib/erlang/erts-11.0/include/ -I /usr/lib/erlang/lib/erl_interface-4.0/include -std=c++11 -Wall  -c -o prometheus_process_info_linux.o prometheus_process_info_linux.cc
214 cc prometheus_process_collector_nif.o prometheus_process_info_linux.o -shared -L /usr/lib/erlang/lib/erl_interface-4.0/lib -lerl_interface -lei -lstdc++ -o /builds/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
215 /usr/bin/ld: cannot find -lerl_interface
216 collect2: error: ld returned 1 exit status

Thanks for maintaining this library!

Conflicting dependencies with `prometheus_ex`

Forgive me if there is a reason for this, but at the moment there is a dependency conflict for prometheus versions between prometheus_ex and prometheus_process_collector.

Following deps:
{:prometheus_ex, "~> 3.0.2"}
{:prometheus_process_collector, "~> 1.3.1"}

Results in:

Resolving Hex dependencies...

Failed to use "prometheus" (versions 4.0.0 to 4.2.0) because
prometheus_ex (version 3.0.2) requires ~> 4.0
prometheus_process_collector (version 1.3.1) requires ~> 3.4

For the time being i'll try overriding it to 4.0 and see if that works.

module :prometheus_process_collector is not available

I also tried to use elixir official image, which is based on Debian:

PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

but also, I got that error:

(UndefinedFunctionError) function :prometheus_process_collector.collect_mf/2 is undefined (module :prometheus_process_collector is not available)

Missing metrics from Process Collector - Rabbit 3.7.7-1

Hello, there are no metrics from process collector and no errors related to plugins in logfile.

This should be enough to describe the situation

rabbit version
Installed plugins - *.ez files
comamand output - rabbitmq-plugins enable prometheus_process_collector
comamand output - return of command: rabbitmq-plugins list
rabbitmq.config
get and check metrics

You can check bellow..

Version

Ubuntu 16.04.4 LTS
ii  rabbitmq-server                    3.7.7-1

Installed *.ez

ls /usr/lib/rabbitmq/lib/rabbitmq_server-3.7.7/plugins | grep prometheus
prometheus-3.5.1.ez
prometheus_cowboy-0.1.4.ez
prometheus_httpd-2.1.8.ez
prometheus_process_collector-1.3.1.ez
prometheus_rabbitmq_exporter-3.7.2.3.ez

Enable plugin cmd

rabbitmq-plugins enable prometheus_process_collector
The following plugins have been configured:
  prometheus_rabbitmq_exporter
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@rabbit5...
Plugin configuration unchanged.

rabbitmq-plugins list

|/
[E*] prometheus_rabbitmq_exporter      3.7.2.3
[  ] rabbitmq_amqp1_0                  3.7.7
[  ] rabbitmq_auth_backend_cache       3.7.7
[  ] rabbitmq_auth_backend_http        3.7.7
[  ] rabbitmq_auth_backend_ldap        3.7.7
[  ] rabbitmq_auth_mechanism_ssl       3.7.7
[  ] rabbitmq_consistent_hash_exchange 3.7.7
[  ] rabbitmq_event_exchange           3.7.7
[  ] rabbitmq_federation               3.7.7
[  ] rabbitmq_federation_management    3.7.7
[  ] rabbitmq_jms_topic_exchange       3.7.7
[E*] rabbitmq_management               3.7.7
[e*] rabbitmq_management_agent         3.7.7
[  ] rabbitmq_mqtt                     3.7.7
[  ] rabbitmq_peer_discovery_aws       3.7.7
[  ] rabbitmq_peer_discovery_common    3.7.7
[  ] rabbitmq_peer_discovery_consul    3.7.7
[  ] rabbitmq_peer_discovery_etcd      3.7.7
[  ] rabbitmq_peer_discovery_k8s       3.7.7
[  ] rabbitmq_random_exchange          3.7.7
[  ] rabbitmq_recent_history_exchange  3.7.7
[  ] rabbitmq_sharding                 3.7.7
[  ] rabbitmq_shovel                   3.7.7
[  ] rabbitmq_shovel_management        3.7.7
[  ] rabbitmq_stomp                    3.7.7
[  ] rabbitmq_top                      3.7.7
[  ] rabbitmq_tracing                  3.7.7
[  ] rabbitmq_trust_store              3.7.7
[e*] rabbitmq_web_dispatch             3.7.7
[  ] rabbitmq_web_mqtt                 3.7.7
[  ] rabbitmq_web_mqtt_examples        3.7.7
[  ] rabbitmq_web_stomp                3.7.7
[  ] rabbitmq_web_stomp_examples       3.7.7

rabbitmq.conf

[
  {rabbit, [
     {tcp_listeners, []},
     {ssl_listeners, [5671,5672]},
     {ssl_options, [{cacertfile,"/etc/secrets/ca-certificates/ca.pem"},
                    {certfile,"/etc/secrets/rabbit5.local.net/cert"},
                    {keyfile,"/etc/secrets/rabbit5.local.net/key"},
                    {verify,verify_peer},
                    {fail_if_no_peer_cert,false}]}
   ]},
 {prometheus, [
   {rabbitmq_exporter, [
     {path, "/metrics"},
     {memory_stat_enabled, ture},
     {connections_total_enabled, false},
     {queue_messages_stat, []},
     {exchange_messages_stat, []}
   ]},
   {collectors, [

     %% Standard prometheus collectors
     prometheus_vm_statistics_collector,
     prometheus_vm_system_info_collector,
     prometheus_vm_memory_collector,
     prometheus_mnesia_collector,

     %% Rabbitmq collectors
     prometheus_rabbitmq_overview_collector,
     prometheus_rabbitmq_exchanges_collector,
     prometheus_rabbitmq_mnesia_tables_collector,
     prometheus_rabbitmq_nodes_collector
     %%prometheus_rabbitmq_queues_collector
   ]}
 ]}
].

Get metrics and check - There are 3008 metrics but NO PROCESS METRICS !!!

# wget -O /tmp/test.txt http://localhost:15672/api/metrics
# wc -l /tmp/test.txt
3008 /tmp/test.txt
# cat /tmp/test.txt | grep ^process

Compiling problems with Elixir 1.8, OTP21 and OSX 10.14.2

Can't compile the dependency

System:

Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]

Elixir 1.8.0 (compiled with Erlang/OTP 21)

Output:

===> Compiling prometheus_process_collector
cc prometheus_process_collector_nif.o prometheus_process_info_macos.o -L/usr/local/opt/openssl/lib -shared -L /usr/local/Cellar/erlang/21.2.2/lib/erlang/lib/erl_interface-3.10.4/lib -lerl_interface -lei -lstdc++ -o /Users/andresouza/arvore/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
Undefined symbols for architecture x86_64:
  "_enif_make_atom", referenced from:
      on_load(enif_environment_t*, void**, unsigned long) in prometheus_process_collector_nif.o
  "_enif_make_double", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_int", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_list_from_array", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_long", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_tuple", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_ulong", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/andresouza/arvore/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so] Error 1
===> Hook for compile failed!

Doesn't automatically capture metrics

@deadtrickster, this is probably the coolest underappreciated collection of Elixir packages I've come across so far. Super awesome, thanks a million!

Unfortunately, this particular one doesn't work for me out of the box – neither in the example app, nor in my own project. All other metrics are shown beautifully, and :prometheus_process_collector is loaded as a module, but I can't seem to get access to my system info. Could you help me figure this out?

The on_load function for module prometheus_process_collector returned :error

[info] GET /metrics
[warn] The on_load function for module prometheus_process_collector returned {:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): no suitable image found.  Did find:\n\t/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00\''}}

[warn] The on_load function for module prometheus_process_collector returned {:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): no suitable image found.  Did find:\n\t/Users/cba28/projects/superaggro/_build/dev/lib/prometheus_process_collector/priv/prometheus_process_collector.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00\''}}

[error] #PID<0.457.0> running Superaggro.Endpoint terminated
Server: 10.23.195.203:4000 (http)
Request: GET /metrics
** (exit) an exception was raised:
    ** (UndefinedFunctionError) function :prometheus_process_collector.collect_mf/2 is undefined (module :prometheus_process_collector is not available)
        (prometheus_process_collector) :prometheus_process_collector.collect_mf(:default, #Function<1.49806622/1 in :prometheus_text_format.registry_collect_callback/3>)
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/prometheus_collector.erl:157: :prometheus_collector.collect_mf/3
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/prometheus_registry.erl:58: :prometheus_registry."-collect/2-lc$^0/1-0-"/3
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/prometheus_registry.erl:58: :prometheus_registry.collect/2
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/formats/prometheus_text_format.erl:74: :prometheus_text_format.format/1
        (prometheus) /Users/cba28/projects/superaggro/deps/prometheus/src/metrics/prometheus_summary.erl:231: :prometheus_summary.observe_duration/4
        (superaggro) lib/prometheus_exporter.ex:2: Superaggro.PrometheusExporter.scrape_data/0
        (superaggro) lib/prometheus_exporter.ex:2: Superaggro.PrometheusExporter.call/2
        (superaggro) lib/superaggro/endpoint.ex:1: Superaggro.Endpoint.phoenix_pipeline/1
        (superaggro) lib/plug/debugger.ex:123: Superaggro.Endpoint."call (overridable 3)"/2
        (superaggro) lib/superaggro/endpoint.ex:1: Superaggro.Endpoint.call/2
        (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
        (cowboy) src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

This is from an Elixir 1.3.2 application, calling Prometheus.Registry.register_collector(:prometheus_process_collector) inside the application start function definition.

Mac m1 support

Getting and error like this on Mac M1

[warning] The on_load function for module prometheus_process_collector returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: 
...
prometheus_process_collector.so\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64\'))\''}}

I was able to build this library with rebar3 on my M1 and copy and pasted it over the Mac x86_64 files in my elixir project to test if it works at all. It appears to work.

Using rebar3_auto plugin (to auto-reload code) gives error `Upgrade not supported by this NIF library.`

Using either the command r3:do(compile). or the rebar3_auto plugin to auto-compile and reload the core, I get:

19:13:40.126 [warning] The on_load function for module prometheus_process_collector returned:
{error,{upgrade,"Upgrade not supported by this NIF library."}}

I'm on MacOS and Erlang 24.3.4 and other than this, the library works perfectly.

Document how to build with Docker

Hi @deadtrickster , can you document how to build the plugin so that it can be installed with a specific version of the RabbitMQ docker image? I need to build the plugin because of your last comment deadtrickster/prometheus_rabbitmq_exporter#12 (comment)

I tried to run ./rebar3 build as explained in the README.md, but I think I am missing some dependencies. Read on for how to reproduce.

I have written this Dockerfile:

# lock rabbitma image version to same we use in our compile image
FROM rabbitmq:3.6.8

ENV REVISION=v1.0.2

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y git && \
    apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN git clone https://github.com/deadtrickster/prometheus_process_collector.git
/usr/src/prometheus_process_collector && cd /usr/src/prometheus_process_collector && git checkout ${REVISION}

WORKDIR /usr/src/prometheus_process_collector

# copied from https://github.com/deadtrickster/prometheus_process_collector/blob/master/.travis.yml
RUN ./elvis rock && ./rebar3 compile

# Reset the rabbitmq image entrypoint so that we can debug
ENTRYPOINT /bin/bash

When I build the image, the "rebar3 compile" step fails with the following messages:

Unable to run pre hooks for 'compile', command 'compile' in namespace 'protobuf' not found

The full docker build trace is below:

Sending build context to Docker daemon 15.36 kB
Step 1/7 : FROM rabbitmq:3.6.8
 ---> 8cdcbee37f62
Step 2/7 : ENV REVISION v1.0.2
 ---> Using cache
 ---> 047355fe35cd
Step 3/7 : RUN apt-get update &&     DEBIAN_FRONTEND=noninteractive apt-get install -y git &&     apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 ---> Using cache
 ---> bc2531a58382
Step 4/7 : RUN git clone https://github.com/deadtrickster/prometheus_process_collector.git /usr/src/prometheus_process_collector && cd /usr/src/prometheus_process_collector && git checkout ${REVISION}
 ---> Using cache
 ---> d75fc7f36e66
Step 5/7 : WORKDIR /usr/src/prometheus_process_collector
 ---> Using cache
 ---> b793622e7bd0
Step 6/7 : RUN ./elvis rock && ./rebar3 compile
 ---> Running in cc5ef474a9e6
Loading files...
Loading src/prometheus_process_collector.erl
Applying rules...
# src/prometheus_process_collector.erl [OK]
Loading files...
Applying rules...
Loading files...
Loading rebar.config
Applying rules...
# rebar.config [OK]
Loading files...
Loading elvis.config
Applying rules...
# elvis.config [OK]
===> Package prometheus-3.1.0 not found. Fetching registry updates and trying again...
===> Updating package registry...
===> Writing registry to /var/lib/rabbitmq/.cache/rebar3/hex/default/registry
===> Generating package index...
===> [ldb:0.0.3] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.7] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ishikawa:0.0.1] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.5] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.6] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [partisan:0.2.0] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [cloudi_service_oauth1:1.5.1] Only existing version of cloudi_service_db_riak is 1.3.3 which does not match constraint ~> 1.5.1. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.2] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [partisan:0.1.1] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.1] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.9] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.8] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.4] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> [ldb:0.0.10] Only existing version of acceptor_pool is 1.0.0-rc.0 which does not match constraint ~> 1.0.0-rc.0. Using anyway, but it is not guaranteed to work.
===> Writing index to /var/lib/rabbitmq/.cache/rebar3/hex/default/packages.idx
===> Verifying dependencies...
===> Fetching prometheus ({pkg,<<"prometheus">>,<<"3.1.0">>})
===> Downloaded package, caching at /var/lib/rabbitmq/.cache/rebar3/hex/default/packages/prometheus-3.1.0.tar
===> Fetching rebar3_gpb_plugin ({pkg,<<"rebar3_gpb_plugin">>,<<"1.7.3">>})
===> Downloaded package, caching at /var/lib/rabbitmq/.cache/rebar3/hex/default/packages/rebar3_gpb_plugin-1.7.3.tar
===> Fetching gpb ({pkg,<<"gpb">>,<<"3.24.4">>})
===> Downloaded package, caching at /var/lib/rabbitmq/.cache/rebar3/hex/default/packages/gpb-3.24.4.tar
===> Compiling gpb
/usr/lib/erlang/lib/parsetools-2.1.4/include/leexinc.hrl: no such file or directory
===> Compiling /usr/src/prometheus_process_collector/_build/default/plugins/gpb/src/gpb_scan.xrl failed
/usr/lib/erlang/lib/parsetools-2.1.4/include/leexinc.hrl:none: no such file or directory

===> Plugin {rebar3_gpb_plugin,"1.7.3"} not available. It will not be used.
===> Fetching coveralls ({git,"https://github.com/markusn/coveralls-erl",
                              "master"})
===> WARNING: It is recommended to use {branch, Name}, {tag, Tag} or {ref, Ref}, otherwise updating the dep may not work as expected.
===> Compiling coveralls
===> Compiling prometheus
===> Unable to run pre hooks for 'compile', command 'compile' in namespace 'protobuf' not found.
The command '/bin/sh -c ./elvis rock && ./rebar3 compile' returned a non-zero code: 1

Does prometheus_process_collector support debian:stretch(ARM)

Erlang VM segfaults in `get_process_info () from prometheus_process_collector-1.3.1/priv/prometheus_process_collector.so` at startup

System information

--------------- System Information ---------------
OTP release: 20
ERTS version: 9.3
Compile date: Tue Apr  3 08:53:44 2018
Arch: x86_64-unknown-linux-gnu
Endianness: Little
Word size: 64-bit
HiPE support: yes
SMP support: yes
Thread support: yes
Kernel poll: Supported and used
Debug compiled: no
Lock checking: no
Lock counting: no
Node name: 'rabbit@rmq0-memory-alloc-a'
Number of schedulers: 2
Number of async-threads: 64

Backtrace

#0  0x0000000000000cd6 in ?? ()
#1  0x00007f43ce87b157 in get_process_info () from /var/vcap/store/rabbitmq-server/mnesia/rabbit@rmq0-memory-alloc-a-plugins-expand/prometheus_process_collector-1.3.1/priv/prometheus_process_collector.so
#2  0x000000000044b5c4 in process_main (x_reg_array=0x7f440f1a3df0, f_reg_array=0x0) at beam/beam_emu.c:3601
#3  0x00000000004f5489 in sched_thread_func (vesdp=0x7f4410dc2100) at beam/erl_process.c:8906
#4  0x000000000067806f in thr_wrapper (vtwd=<optimized out>) at pthread/ethread.c:118
#5  0x00007f4452c61184 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00007f445278603d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Artefacts

Version bump and release?

#10 updated the dependencies, but the package version wasn’t updated and a new release was not made, still requiring , override: true until this is done.

Always builds x86_64 binary on Darwin

Hi,

prometheus_process_collector.so is always built for -arch x86_64 on Darwin (=macOS) hosts. Now that Apple Silicon hardware exists, this makes the library fail to load on such hardware.

I have not researched why c_src/Makefile lines 24 and 25 hardcode x86_64, but that seems to be the source of this problem:

20:22:34.861 [warn]  The on_load function for module prometheus_process_collector returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/ch/Source/.../_build/test/lib/prometheus_process_collector/priv/prometheus_process_collector.so, 2): no suitable image found.  Did find:\n\t/Users/ch/Source/.../_build/test/lib/prometheus_process_collector/priv/prometheus_process_collector.so: mach-o, but wrong architecture\n\t/Users/ch/Source/.../_build/test/lib/prometheus_process_collector/priv/prometheus_process_collector.so: stat() failed with errno=35\''}}

Indeed, lipo agrees:

$ lipo -archs /Users/ch/Source/.../_build/test/lib/prometheus_process_collector/priv/prometheus_process_collector.so
x86_64

Version: prometheus_process_collector 1.6.0

Best,
Chris

Update version

I ran into the issue with the M1 compilation that was fixed #24 - thank you!
Would it be possible to tag a new version and/or release an updated package on hex?

Compiling problems on elixir:1.9.1-alpine docker image

Cannot compile the dependency

System:
docker elixir:1.9.1-alpine docker image

Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Interactive Elixir (1.9.1) - press Ctrl+C to exit (type h() ENTER for help)

Output

server_1  | ===> Compiling prometheus_process_collector
server_1  | make: Entering directory '/app/deps/prometheus_process_collector/c_src'
server_1  | g++ -O3 -finline-functions -fPIC -I /usr/local/lib/erlang/erts-10.4.4/include/ -I /usr/local/lib/erlang/lib/erl_interface-3.12/include -std=c++11 -Wall  -c -o prometheus_process_info_linux.o prometheus_process_info_linux.cc
server_1  | cc prometheus_process_collector_nif.o prometheus_process_info_linux.o -shared -L /usr/local/lib/erlang/lib/erl_interface-3.12/lib -lerl_interface -lei -lstdc++ -o /app/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
server_1  | prometheus_process_collector_nif.o: file not recognized: file format not recognized
server_1  | collect2: error: ld returned 1 exit status
server_1  | make: *** [Makefile:66: /app/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so] Error 1
server_1  | make: Leaving directory '/app/deps/prometheus_process_collector/c_src'
server_1  | ===> Hook for compile failed!
server_1  |
server_1  | ==> metric_service
server_1  | ** (Mix) Could not compile dependency :prometheus_process_collector, "/root/.mix/rebar3 bare compile --paths="/app/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"

This works completely fine on my macOS and I can see the metrics, but when running on docker it is not compiling.