canaryio / canary Goto Github PK

Is it possible to pass headers with canary?

My use case is passing the Host header so I can monitor individual webs as well as balancers.

https://github.com/canaryio/canary/tree/master/cmd/canaryd#publishers is missing docs for the new statsd publisher

Would it be in scope to support metadata in a Target that would be available in a Publisher as it consumes a Measurement? For our external site monitoring project it would be useful to have tags and other metadata be available to the systems that consume these metrics. If so, it looks like I can probably add that pretty easily.

Noticed this while troubleshooting #46 it would make a lot of sense if canary had a flag to enable "debug mode". In this world, I'd like to see the request body emitted as well as the response info.

It would certainly make it easier to see what exactly is going wrong w/o having to resort to a series of log statements.

I wanted to sketch out a well scoped roadmap for this project. This includes V2 - V5.

Each version can include bugfixes, but we should hold off on new features until these have been knocked out.

V2

Introduce canaryd executable. Capable of retrieving a configuration manifest from a URL and reliably measuring a set of targets. Result output will simply flow to stdout. The application will not automatically reconfigure itself when the manifest changes.

V3

Add a Librato publisher and make it configurable via the manfiest.

V4

The application will automatically add and remove targets as the manifest changes.

V5

The application will automatically add and remove publishers as the manifest changes.

There are a lot of cases where I would like to monitor basic connectivity via ICMP (I am currently using smokeping with fping).

Are other protocols something you're considering doing?

For experimental purposes, adding an expvar publisher might be interesting. In this model, we'd expect canaryd to simply expose a URL, allowing for other things to poll and aggregate the values.

https://golang.org/pkg/expvar/

For the sake of simplifying the codebase, I'm interested in exploring the following:

• moving to a flags based setup for both canary and canaryd
• pushing most of canaryd's config to the manifest

Before I can do that, though, I'd like to get some feedback from how others are using canary / canaryd today. (cc @jmeichle @blalor and anyone else who wants to comment). I don't want to ruin anyone's day, but I def want to make config easier to reason about.

I've been looking at the code and the Go docs trying to figure out how to implement this and I'm coming up short. I don't see a way with net/http to get the IP address of the server that sent the response. Sampler does provide its own Transport which actually creates the connection, but I don't see a clean way to share that information with Sampler.Sample.

Another approach would be to do the address resolution in Sampler.Sample, but the code for doing so is sufficiently complex (ipv4 vs ipv6, etc) that it seems like a liability to reproduce that in Canary.

Thoughts? I'd like to have this information so I can identify which CDN is failing to serve content, without having to track that in the manifest.

While working on #42 I came across several instances where we're using int data types when we should just be using time.Duration. This makes things harder to reason about, as you have to continually ask "is this time in milliseconds? seconds?", etc.

Unifying these values around time.Duration would be a big win.

Someday I will live in a world where the HTTP status code of a service response will be sufficient for determining the proper status of that response. Still trying to find that place, however. It would be useful to return the body of the response as a property of the sample for further analysis outside of the core canary library.

Should be fairly straightforward; the body's already being read:

Pretty self-explanatory. Was testing against a site that I know does a 301 to an HTTPS version of the URL, and it seems to only report the 3xx and not the ultimate 2xx that I expected.

$ SAMPLE_INTERVAL=5 $GOPATH/bin/canary http://jgoin.net
2016-10-08T23:33:59-07:00 http://jgoin.net 301 35.747582 true 1 
2016-10-08T23:34:04-07:00 http://jgoin.net 301 32.764245 true 2