Improve health checks for bouncer about sysadmin HOT 4 CLOSED

Any interest in adding non-blackbox monitoring? 😄 Checks like this are always going to be error-prone. Getting more direct instrumentation would be more reliable as you can get metrics on every function in the bouncer. Even adding nginx-module-vts would be useful to find out how many requests are being served.

from sysadmin.

@SuperQ yeah in the long term I think we want to have our services speak prometheus and publish more application specific metrics.

For example I have been rewriting the collector component in golang and are using there github.com/zsais/go-gin-prometheus to publish the number of submitted measurements by country and by platform: https://github.com/ooni/collector/blob/mvp-kiss/collector/metrics.go.

The nginx module is a good one and I think we should look into that, but I think the ideal scenario is that the metrics are published using some prometheus library by the application itself. Is this correct?

from sysadmin.

Yes, it's best to have the code itself do the tracking. It's pretty easy to use the official Go and Python libraries.

That's a very strange wrapper, wouldn't it be easier to use the Prometheus client_golang directly?

The nginx module idea helps because it lets you know what's going on inside nginx as well.

from sysadmin.

That's a very strange wrapper, wouldn't it be easier to use the Prometheus client_golang directly?

go-gin-prometheus is basically just exposing client_golang as a gin middleware (which is the routing library we use in OONI golang tooling). The nice thing is that they already figured out how to log the request size, request duration and request count for gin, so we don't have to re-implement this ourselves.
It effectively though is still reliant most on the client_golang to expose the metrics: https://github.com/zsais/go-gin-prometheus/blob/master/middleware.go#L78.

from sysadmin.