StatsD HTTP proxy with REST interface for using in browsers
StatsD uses UDP connections, and can not be used directly from browser. This server is a HTTP proxy to StatsD, useful for sending metrics to StatsD from frontend by AJAX.
Requests may be optionally authenticated using JWT tokens.
- Options can be set via environment variables
- Enhanced docker examples
- Installation
- Requirements
- Proxy client for browser
- Nginx config
- Usage
- Authentication
- Rest resources
- Response
- Testing
- Benchmark
- Useful resources
git clone [email protected]:GoMetric/statsd-http-proxy.git
make build
Build your own docker image: https://github.com/GoMetric/statsd-http-proxy-docker
Use Docker image:
Run by Docker with insecure connection:
docker run -p 80:80 gometric/statsd-http-proxy:latest --verbose
Run by Docker with secure connection:
docker run -p 4433:4433 -v "$(pwd)":/certs/ gometric/statsd-http-proxy:latest --verbose --http-port=4433 --tls-cert=/certs/cert.pem --tls-key=/certs/key.pem
- GoMetric/go-statsd-client - StatsD client library for Go
- dgrijalva/jwt-go - JSON Web Tokens builder and parser
- gorilla/mux - URL router and dispatcher
Basic implementation of proxy client may be found at https://github.com/GoMetric/statsd-http-proxy-client.
Configuration of Nginx balancer:
server {
listen 443 http2;
server_name statsd-proxy.example.com;
ssl on;
ssl_certificate /etc/pki/nginx/ssl.crt;
ssl_certificate_key /etc/pki/nginx/ssl.key;
upstream statsd_proxy {
keepalive 100;
server statsd-proxy-1:8825 max_fails=0;
server statsd-proxy-2:8825 max_fails=0;
}
location / {
proxy_pass http://statsd_proxy;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Connection "keep-alive";
}
}
- Run server (HTTP):
statsd-http-proxy \
--verbose \
--http-host=127.0.0.1 \
--http-port=8080 \
--statsd-host=127.0.0.1 \
--statsd-port=8125 \
--jwt-secret=somesecret \
--metric-prefix=prefix.subprefix
- Run server (HTTPS):
statsd-http-proxy \
--verbose \
--http-host=127.0.0.1 \
--http-port=433 \
--tls-cert=cert.pem \
--tls-key=key.pem \
--statsd-host=127.0.0.1 \
--statsd-port=8125 \
--jwt-secret=somesecret \
--metric-prefix=prefix.subprefix
Print server version and exit:
statsd-http-proxy --version
Command line arguments:
Parameter | Description | Default value |
---|---|---|
verbose | Print debug info to stderr | Optional. Default false |
http-host | Host of HTTP server | Optional. Default 127.0.0.1. To accept connections on any interface, set to "" |
http-port | Port of HTTP server | Optional. Default 80 |
http-timeout-read | The maximum duration in seconds for reading the entire request, including the body | Optional. Defaults to 1 second |
http-timeout-write | The maximum duration in seconds before timing out writes of the response | Optional. Defaults to 1 second |
http-timeout-idle | The maximum amount of time in seconds to wait for the next request when keep-alives are enabled | Optional. Defaults to 1 second |
tls-cert | TLS certificate for the HTTPS | Optional. Default "" to use HTTP. If both tls-cert and tls-key set, HTTPS is used |
tls-key | TLS private key for the HTTPS | Optional. Default "" to use HTTP. If both tls-cert and tls-key set, HTTPS is used |
statsd-host | Host of StatsD instance | Optional. Default 127.0.0.1 |
statsd-port | Port of StatsD instance | Optional. Default 8125 |
jwt-secret | JWT token secret | Optional. If not set, server accepts all connections |
metric-prefix | Prefix, added to any metric name | Optional. If not set, do not add prefix |
version | Print version of server and exit | Optional |
Sample code to send metric in browser with JWT token in header:
$.ajax({
url: 'http://127.0.0.1:8080/count/some.key.name',
method: 'POST',
headers: {
'X-JWT-Token': 'some-jwt-token'
},
data: {
value: 100500
}
});
Authentication is optional. It based on passing JWT token to server, encrypted with secret, specified in jwt-secret
command line argument. If secret not configured in jwt-secret
, then requests to server accepted without authentication.
Token sends to server in X-JWT-Token
header or in token
query parameter.
We recommend to use JWT tokens to prevent flood requests: you need to setup JWT token expiration time, and update JWT token in browser each time you get 403 in response.
See statsd ductmentation about supported types.
GET /heartbeat
If server working, it responds with OK
POST /count/{key}
X-JWT-Token: {tokenString}
value=1&sampleRate=1
Parameter | Description | Default value |
---|---|---|
value | Value. Negative to decrease | Optional. Default 1 |
sampleRate | Sample rate to skip metrics | Optional. Default to 1: accept all |
Gauge is an arbitrary value.
Only the last value during a flush interval is flushed to the backend. If the gauge is not updated at the next flush, it will send the previous value.
Gauge also may be set relatively to previously stored value.
Is shift
not passed, then value
used.
If value
not passed, used default value equals 1.
To set a gauge to a negative number you need first set it to 0.
Absolute value:
POST /gauge/{key}
X-JWT-Token: {tokenString}
value=1
Shift of previous value:
POST /gauge/{key}
X-JWT-Token: {tokenString}
shift=-1
Parameter | Description | Default value |
---|---|---|
value | Integer value | Optional. Default 1 |
shift | Signed int, relative to previously stored value | Optional |
POST /timing/{key}
X-JWT-Token: {tokenString}
time=1234567&sampleRate=1
Parameter | Description | Default value |
---|---|---|
time | Time in milliseconds | Required |
sampleRate | Float sample rate to skip metrics from 0 to 1 | Optional. Default to 1: accept all |
POST /set/{key}
X-JWT-Token: {tokenString}
value=1
Parameter | Description | Default value |
---|---|---|
value | Integer value | Optional. Default 1 |
Server sends 200 OK
if send success, even StatsD server is down.
Other HTTP status codes:
CODE | Description |
---|---|
400 Bad Request | Invalid parameters specified |
401 Unauthorized | Token not sent |
403 Forbidden | Token invalid/expired |
404 Not found | Invalid url requested |
405 Wrong method | Request method not allowed for resource |
It is useful for testing to start netcat
UDP server,
listening for connections and watch incoming metrics.
To start server run:
nc -kluv localhost 8125
To send command to statsd, run:
echo "counters" | nc localhost 8125
Machine for benchmarking:
Intel(R) Core(TM) i5-2450M CPU @ 2.50GHz Dual Core / 8 GB RAM
Os:
Linux hp 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Sysctl:
sudo sysctl net.ipv4.ip_local_port_range="15000 61000"
sudo sysctl net.ipv4.tcp_fin_timeout=30
Benchmarked by siege v. 4.0.4
Without JWT token:
$ GOMAXPROCS=1 ./bin/statsd-http-proxy --http-host=127.0.0.1 --http-port=8080 --statsd-host=127.0.0.1 --statsd-port=8125
With JWT token:
$ GOMAXPROCS=1 ./bin/statsd-http-proxy --http-host=127.0.0.1 --http-port=8080 --statsd-host=127.0.0.1 --statsd-port=8125 --jwt-secret=somesecret
siege -R <(echo connection = close) -c 255 -r 2000 "http://127.0.0.1:8080/count/a.b.c.d POST value=42"
$ siege -R <(echo connection = keep-alive) -c 255 -r 2000 "http://127.0.0.1:8080/count/a.b.c.d POST value=42"
$ siege -R <(echo connection = close) -c 255 -r 2000 -H 'X-JWT-Token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJzdGF0c2QtcmVzdC1zZXJ2ZXIiLCJpYXQiOjE1MDY5NzI1ODAsImV4cCI6MTg4NTY2Mzc4MCwiYXVkIjoiaHR0cHM6Ly9naXRodWIuY29tL3Nva2lsL3N0YXRzZC1yZXN0LXNlcnZlciIsInN1YiI6InNva2lsIn0.sOb0ccRBnN1u9IP2jhJrcNod14G5t-jMHNb_fsWov5c' "http://127.0.0.1:8080/count/a.b.c.d POST value=42"
$ siege -R <(echo connection = keep-alive) -c 255 -r 2000 -H 'X-JWT-Token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJzdGF0c2QtcmVzdC1zZXJ2ZXIiLCJpYXQiOjE1MDY5NzI1ODAsImV4cCI6MTg4NTY2Mzc4MCwiYXVkIjoiaHR0cHM6Ly9naXRodWIuY29tL3Nva2lsL3N0YXRzZC1yZXN0LXNlcnZlciIsInN1YiI6InNva2lsIn0.sOb0ccRBnN1u9IP2jhJrcNod14G5t-jMHNb_fsWov5c' "http://127.0.0.1:8080/count/a.b.c.d POST value=42"
Concurrent 255 users made 2000 requests each. Total request count: 510000
Router | Keep alive | JWT | Elapsed time | Transaction rate | Concurrency |
---|---|---|---|---|---|
GorillaMux 1.7.3 | disabled | disabled | 94.73 secs | 5383.72 trans/sec | 244.02 |
GorillaMux 1.7.3 | enabled | disabled | 55.70 secs | 9156.19 trans/sec | 252.27 |
GorillaMux 1.7.3 | disabled | enabled | 117.80 secs | 4329.37 trans/sec | 245.98 |
GorillaMux 1.7.3 | enabled | enabled | 77.97 secs | 6540.98 trans/sec | 253.70 |
HttpRouter 1.3.0 | disabled | disabled | 92.93 secs | 5487.99 trans/sec | 244.09 |
HttpRouter 1.3.0 | enabled | disabled | 54.87 secs | 9294.70 trans/sec | 252.65 |
HttpRouter 1.3.0 | disabled | enabled | 115.35 secs | 4421.33 trans/sec | 245.48 |
HttpRouter 1.3.0 | enabled | enabled | 75.14 secs | 6787.33 trans/sec | 253.25 |