madrisan / hashicorp-vault-monitor Goto Github PK

Hi,
thanks for the great work! I came across your project while setting up a vault cluster.
We are using nagios and want to monitor every aspect of the cluster.
So while setting up the checks, I saw that a "connection refused" is leading to the UNKNOWN status in the nagios output.

> service vault stop
> VAULT_CACERT=/etc/vault.d/vault-chain.cert.pem /usr/local/share/icinga/plugins/check_vault status -output=nagios
> vault UNDEFINED - error checking seal status: Get "https://127.0.0.1:8200/v1/sys/seal-status": dial tcp 127.0.0.1:8200: connect: connection refused

One can argue if one want's this to be critical, as this could simply mean a firewall is wrongly configured, while the cluster is healthy.
Although I have no experience in GO I am happy to look into it and hand in a MR, but I wanted to discuss the topic first, if you and possibly others even want this to be critical.

I would otherwise simply fork your project and adapt for my needs. But it's obviously easier to work together.

Hi, thanks for this check.
I'm trying to use it right now. Unfortunately it is not possible to add the parameter -address. It always takes the 127.0.0.1 default:

./hashicorp-vault-monitor status -output=nagios -address=https://vault.testenv.net:8200 vault UNDEFINED - error checking seal status: Get https://127.0.0.1:8200/v1/sys/seal-status: dial tcp 127.0.0.1:8200: connect: connection refused

Since we use the Icinga Director we can't set any environment variable and therefore need the parameter.

We have nagios3 in docker (~3400 checks). Added vault monitoring in nagios:

...
define service {
    use                     http-service
    host_name               some_server
    service_description     vault: expire token accessor token-for-nagios
    servicegroups           vault
    check_command           check_vault!token-lookup -address=some_server -token=s.O...5CK -token-accessor=ZQ...TA -output=nagios -warning=768h -critical=72h
}

cat /etc/nagios-plugins/config/vault.cfg
define command {
   command_name     check_vault
   command_line     /usr/lib/nagios/plugins/hashicorp-vault-monitor $ARG1$ 
}

When I try run check in CLI, everything is working:

 /usr/lib/nagios/plugins/hashicorp-vault-monitor token-lookup -address=some_server -token=s.O...5CK -token-accessor=ZQ...TA -output=nagios -warning=768h -critical=72h
vault WARNING - This (renewable) token will expire on Sat, 09 May 2020 05:32:23 UTC (1 week 5 days 1 hour 19 minutes 36 seconds left)
echo $?
1

Some other token:

/usr/lib/nagios/plugins/hashicorp-vault-monitor token-lookup -address=some_server -token="s.O...K" -token-accessor="hd...8" -output=nagios -warning=168h -critical=72h
vault OK - This (renewable) token will expire on Mon, 21 Jan 2030 08:52:06 UTC (9 years 38 weeks 5 days 4 hours 23 minutes 13 seconds left)
echo $?
0

In nagios web interface I see that - https://ibb.co/1K76F7p.
I try in my check "vault: expire token accessor token-for-nagios" add single and double quotes, but problem with null output is remain.
When check is generate null in web interface, I see her output in docker logs:

docker logs --tail 20 -f nagios3
vault WARNING - This (renewable) token will expire on Sat, 09 May 2020 05:32:23 UTC (1 week 5 days 49 minutes 28 seconds left)

I don't think that problem with nagios in docker ( we now run 3000 different checks without any trouble) + when hashicorp-vault-monitor generate "OK" - I see that in web interface. Trouble only when plugin generate "Warning" or "Critical" (also null outputs will be in other checks - I check that).

usr/lib/nagios/plugins/hashicorp-vault-monitor --version
HashiCorp Vault Monitor v0.8.4 ('7b2326ea73281891139e077aa39f2d91f83c493c+CHANGES')

For example, I found thread - https://www.linuxquestions.org/questions/linux-software-2/nagios-interprets-perl-plugin-output-as-null-948605/, where similar problem (plugin written on perl)

My issue is resolved. My plugin does file IO and wasn't opening a file for reading. Works from the command line possibly because I ran the script from the same directory as the script. Nagios runs the script from absolute path in another working directory.

May be same problem (or similar) in that plugin?

We're using this to monitor vault servers. Can it also monitor vault agent processes on client machines?

Just thought I'd check here before asking IT to try it.

Is it possible monitoring vault token accessors?
I'm planning monitoring important tokens in nagios, but I don't want show this tokens anywhere.
Expiration date of the Vault tokens I can receive from their accessors. In the utility documentation I don't find any examples about this opportunity.

in my case I was getting a laconic: "failed to read environment" and I could not understand that my user didn't have permission to read the certificate file.

fixed by #11

The readme uses the root token of a dev setup. In prod, what are the minimum permissions needed for this script to run?