Monitoring remote host with nrpe about docker_shinken HOT 9 CLOSED

Yes. You need to install: nagios-nrpe-server & nagios-plugins on every host/client -- you want to monitor using shinken. For example, it will be 'my-remote-host.com' from the example above.

It is available for install via standard package managers, like apt, yum, etc. Sample deb package link: https://packages.debian.org/squeeze/nagios-nrpe-server

Once installed, start nagios-nrpe-server service. It will listen on TCP port 5666. The configuration file for NRPE is: /etc/nagios/nrpe.cfg. All NRPE commands are written here. It is the "-c $ARG1$ " part from your shinken command definition. The default installation directory for NRPE plugins is: /usr/lib/nagios/plugins/ (for most systems).

For adding your own commands, you can modify /etc/nagios/nrpe_local.cfg (there is a include statement for this file in the main nrpe.cfg file). This will keep your main configuration clean and help you track changes better.

from docker_shinken.

This is NRPE based monitoring, just like Nagios does (by installing an agent on all clients). Agentless monitoring is also possible using ssh. If you want to explore all these, visit:
http://shinkenlab.io/
It contains a bunch of cool online courses (screencast tutorials). You may be interested in this one: http://shinkenlab.io/online-course-5-monitoring-plugins/

Please note: https://www.monitoring-plugins.org/ is a fork of nagios NRPE plugins. It contains latest versions of plugins. However, easy to install linux packages are not available. You need to compile from source code.

from docker_shinken.

First, thanks for your answer and your advices,

I have not mentioned that I had already installed nagios-nrpe-server & nagios-plugins on client, sorry.

it seems that my mistake was that I named the command check_nrpe, the "native" check_nrpe command (locate at /etc/shinken/commands/check_nrpe.cfg on the docker image) was called instead of mine. this explains the "not found" error.

I update my nrpe command like this :

define command{
  command_name     cmd_check_nrpe
  command_line     $NAGIOSPLUGINSDIR$/check_nrpe -H $HOSTADDRESS$ -n -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$
  module_type      nrpe_poller
}

now my server send command to my remote host (note I have to disable ssl with -n parameter, else nothing is happening, I do not know why yet), but I have new error : Empty response from the NRPE server

On me remote host syslog file, I have :

Host address is in allowed_hosts
Handling the connection...
Error: Could not complete SSL handshake. 1
Connection from xxx.xxx.xxx.xxx closed.

I saw on some forums that was a known problem, I have to look at it more closely.

I'm just wondering if this error is not caused by the -n parameter. do you have any idea about this ?

Thanks

from docker_shinken.

Is the IP of shinken server whitelisted at nagios-nrpe-server configuration: nrpe.cfg or nrpe_local.cfg?
Check for allowed_hosts parameter. By default it is configured to allow only localhost.
allowed_hosts=127.0.0.1

disable ssl with -n

I don't think this is required. I never used this.

from docker_shinken.

Oops, didn't saw that you mentioned 'Host address is in allowed_hosts'.

Try dropping 'module_type nrpe_poller' from command definition. It is used to enable booster-nrpe. You can try adding it later, once it works. Example configuration:

define command{
  command_name     cmd_check_nrpe
  command_line     $NAGIOSPLUGINSDIR$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$
}

from docker_shinken.

I have [Errno 2] No such file or directory again on the Thruk interface.

it seemed normal to me because there is no check_nrpe plugin file in docker image, am I wrong ?

This is why I have tried to configured the booster-nrpe.

Thanks for help

from docker_shinken.

Humm... Added check_nrpe plugin in the docker image:
320f25f
Build triggered!

from docker_shinken.

Did a full test and found the issue. Possible reasons:

1. You are running docker on the same host which you want to monitor. The source IP of the docker container is private and dynamic. It reaches the nrpe host via docker0 network (check ifconfig output). Solution - Allow private source ip addresses. Something like this:

   allowed_hosts=127.0.0.1,172.16.0.0/12

2. Your sample configuration contains NRPE arguments. This is by default disabled in NRPE server. Solution - enable arguments:

   dont_blame_nrpe=1

On the server side, your shinken configuration can be something like this:

$ cat custom_configs/my_host_and_service.cfg

define command{
    command_name               cmd_check_nrpe
    command_line               $NAGIOSPLUGINSDIR$/check_nrpe -H '$HOSTADDRESS$' -c '$ARG1$' -a '$ARG2$' '$ARG3$' '$ARG4$'
}

define host {
    host_name                  myhost.example.com
    use                        generic-host
    alias                      myhost.example.com
    address                    myhost.example.com
}

define service {
    service_description        check_users
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_users
}
define service {
    service_description        check_load
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_load
}
define service {
    service_description        check_hda1
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_hda1
}
define service {
    service_description        check_zombie_procs
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_zombie_procs
}
define service {
    service_description        check_total_procs
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_total_procs
}

Client side (myhost.example.com) changes:

$ sudo apt-get install nagios-nrpe-server 
$ sudo echo 'allowed_hosts=127.0.0.1,172.16.0.0/12' > /etc/nagios/nrpe_local.cfg
$ sudo echo 'dont_blame_nrpe=1' >> /etc/nagios/nrpe_local.cfg
$ sudo service nagios-nrpe-server restart

Let me know, if this solved the issue. Happy monitoring :)

from docker_shinken.

It's perfect now,

after pulling the docker image, my command with 4 args worked correctly, but not the others. after removing args parameters, all works fine. I will define commands args on the remote host.

Now, It remains for me to learn more about Shinken / Nagios monitoring.

Thank you very much for your time.

from docker_shinken.