Giter VIP home page Giter VIP logo

beeping's Introduction

DEPRECATED

This code is unmaintained anymore. Blackbox exporter is the right way to monitor your http services.

BeePing v0.5.0

Build Status

previously named pingmeback

It forages the servers and brings the metrics back to the hive

BeePing is a distant http check as a Service. Call the very simple API, BeePing will measure website for you.

πŸ‡ΊπŸ‡ΈπŸ’š 200 OK - 119 ms - (http_status_code, http_request_time)

πŸ“– DNS - 9 ms - (dns_lookup)

πŸ”„ TCP - 6 ms - (tcp_connection)

πŸ”’ TLS - 52 ms - (tls_handshake)

πŸ–₯️ Server Processing - 43 ms - (server_processing)

πŸ”½ Transfer - 6 ms - (content_transfer)

Other attributes :

  • cert_expiry_days_left
  • http_body_pattern

Features:

  • Very simple JSON API
  • Lot of metrics
  • Timeline of HTTP request
  • SSL Expiration check
  • Server SSL/TLS version and Ciphers
  • Pattern check (search for text in response)
  • GeoIP resolution
  • Single binary

Big hugs to :

  • Dave Cheney for his inspirational work on httpstat
  • Taichi Nakashima for his work on httpstat lib go-httpstat

Install

Download latest version on releases page

  • chmod +x beeping
  • sudo mv beeping /usr/bin
  • beeping
$ ./beeping -h
Usage of ./beeping:
  -geodatfile string
        geoIP database path (default "/opt/GeoIP/GeoLite2-City.mmdb")
  -instance string
        beeping instance name (default hostname)
  -listen string
        The host to bind the server to (default "127.0.0.1")
  -port string
        The port to bind the server to (default "8080")
  -tlsmode
        Activate SSL/TLS versions and Cipher support checks (slow)
  -validatetarget
          Perform some security checks on the target provided (default true)

Notes

  • If no GeoIP database is found, BeePing omit geo response silently
  • TLSMode returns more infos on SSL object. It tries the more ciphers and TLS version Golang can test but the checks can be way slower.

Optional

You can plug MaxMind GeoIP file to know on which country the pings goes.

See: http://dev.maxmind.com/geoip/geoip2/geolite2/

Build

Beeping is known to only compile with Golang 1.8.x + (see #14 )

go get -u github.com/golang/dep
go get -u github.com/yanc0/beeping
cd $GOPATH/src/github.com/yanc0/beeping
dep ensure
go build

API Usage

$ curl -XPOST http://localhost:8080/check -d '{"url": "https://google.fr", "pattern": "find me", "header": "Server:GitHub.com", "insecure": false, "timeout": 20}'
{
  "http_status": "200 OK",
  "http_status_code": 200,
  "http_body_pattern": true,
  "http_header": true,
  "http_request_time": 716,
  "instance_name": "X250",
  "dns_lookup": 14,
  "tcp_connection": 101,
  "tls_handshake": 228,
  "server_processing": 168,
  "content_transfer": 203,
  "timeline": {
    "name_lookup": 14,
    "connect": 115,
    "pretransfer": 344,
    "starttransfer": 512
  },
  "geo": {
    "country": "US",
    "ip": "192.30.253.112"
  },
  "ssl": {
    "ciphers": [
      "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
      "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
      "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
      "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305",
      "TLS_ECDHE_ECDSA_WITH_RC4_128_SHA",
      "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
      "TLS_RSA_WITH_RC4_128_SHA",
      "TLS_RSA_WITH_3DES_EDE_CBC_SHA",
      "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
      "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
      "TLS_ECDHE_RSA_WITH_RC4_128_SHA",
      "TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA"
    ],
    "protocol_versions": [
      "TLS12",
      "TLS10",
      "TLS11"
    ],
    "cert_expiry_date": "2018-05-17T12:00:00Z",
    "cert_expiry_days_left": 374,
    "cert_signature": "SHA256-RSA"
  }
}
  • If pattern is not filled http_body_pattern is always true
  • If header is not filled http_header is always true
  • ssl is omitted when http://. The same for the tls_handshake field
  • geo is omitted if geoip is not set

Beeping Clients

Error Handling

beeping returns HTTP 500 when check fail. The body contains the reason of the failure.

{
  "message": "Get https://mysite.com/health: net/http: request canceled (Client Timeout exceeded while awaiting headers)"
}

HTTP Basic Auth

Just add the 'auth' option in your JSON.

$ curl -XPOST http://localhost:8080/check -d '{"url":"http://127.0.0.1:3000","auth":"john:secret"}'

Changelog

0.6.0 - UNRELEASED

0.5.0 - 2017-05-07

  • Add TLS Mode, now show server supported ciphers and SSL/TLS versions
  • Add listen / Port options (breaking change)
  • Modify JSON response structure (breaking change)
  • Add proper logging
  • Set proper User-Agent
  • Add header check

0.4.0 - 2017-04-24

  • Pingmeback is now BeePing
  • Add BeePing logo
  • Adapt documentation
  • Change Travis build info

0.3.0 - 2017-04-21

  • pingmeback now returns geoip informations
  • Add Instance name in results
  • Add vendoring system

To Do

  • Add HTTP Auth
  • Add tests
  • More metrics
  • Packaging

Contributing

Feel free to make a pull request.

Contributors

  • Aaron Hnatiw
  • Aimof
  • Brice Colucci
  • Yann Coleu

Licence

The MIT License (MIT)

Copyright (c) 2016 Yann Coleu

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

beeping's People

Contributors

aimof avatar bcolucci avatar jimen0 avatar olpia avatar tpoindessous avatar yanc0 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

beeping's Issues

Check sha-1 SSL certificate

Hi,

as Chrome and Firefox last versions doesn't allow connection to SHA-1 HTTPS websites, it would be great to add an optional test to check if SSL certificate is not a SHA-1 (and other older algo which are also forbidden by Chrome or Firefox)

Thanks.

Implement rate limiting

As detailed in #16, the application's design makes it vulnerable to server-side request forgery (SSRF). While there have already been some mitigations put into place, it would also benefit from one more- rate limiting.

Attackers can currently use an open BeePing instance to launch an anonymous DoS attack by routing a large amount of traffic through the BeePing system to the target server. While it is likely that the single BeePing instance would crash before the targeted system, it may not always be the case (if BeePing was running on a beefy, auto-scaling AWS instance, for example). By implementing rate limiting, we can limit the amount of traffic that can go through BeePing from one source, which would severely limit any types of DoS attacks through a BeePing instance.

I would recommend adding a configuration option in the future and a command-line flag now to address this. The rate can be user-customizable, but a good default would be something like 10 requests per second (per source IP). That wouldn't be enough to bring down most web servers, and it would give victims and BeePing instance operators a chance to block the source IP of the attack with a firewall rule.

You would have to track the source IPs that BeePing sees for a short period of time (only a few seconds, if the metric is requests per second) in order to check the request rate in the programming logic. A small database (in-memory or local filesystem-based) would be perfect for this use case; I've heard good things about Hashicorp's memdb and boltdb. Just be sure to properly wipe IP addresses from the database at the end of their life, as it could cause privacy concerns for users to have them long-lived.

Cheers,
Aaron (insp3ctre)

New feature: support of regex in pattern check

Hi

we use beeping with pattern check and we would like to use regex pattern.

For example, we check Hashicorp Vault via an API call and we need to verify that multiple parameter are set.

For example, we call Vault on this URL ( https://vault/v1/sys/health ) and we get this response :

{"initialized":true,"sealed":false,"standby":false,"server_time_utc":1505381889,"version":"0.6.2","cluster_name":"vault-cluster-YYYYYY","cluster_id":"XXXXXX"}

We would like to use this regexp : .*\"sealed\":\"false\".*\"cluster_name\":\"vault-cluster.*

Thanks !

De-anonymize requests

Attackers commonly use proxy services (which is essentially what BeePing is) to anonymize their attack traffic. In the case of BeePing, an attacker could enumerate internet-facing web hosts or launch a DoS attack via a BeePing host, and their originating IP would not be disclosed to the target system. The "Forwarded" header has been standardized for use in these very instances, and would allow the target system to identify the true source of any attacks and respond accordingly.

I could put together a pull request to fix this issue if you'd like. Basically, I would be adding the appropriate header into each outbound request (somewhere around here. The header would look something like this: Forwarded: For=<IP>.

Cheers,
Aaron (insp3ctre)

External listener

BeePing listens on port 8080 on all interfaces (including the external interface) by default.

beeping global listener

I can submit a pull request if you'd like for the fix, but it's simply a matter of providing a specific IP and port combination to the router.Run() function. For example: router.Run("127.0.0.1:8080"). At the very least, people should be aware of it, so they aren't unknowingly opening up they internal network to enumeration by attackers ( #10 ) whenever they run a BeePing instance.

It's totally up to you, but I would suggest one of three fixes:

  • Add a command-line option to accept local connections only. This is probably the most user-friendly, and depending on how you are using it at @skale-5, may fit your use case the best. This is my recommended solution.
  • Set the router to listen locally automatically. This may not be how you wanted the program designed though, as it won't allow BeePing to run as an external-facing service.
  • Notify users in the README.md file that BeePing listens on all interfaces by default. This way they are at least informed, and can adapt their firewall rules accordingly, if needed.

Cheers,
Aaron

Local IP blacklist bypass

Issue

It appears that we can bypass the local IP blacklist (implemented in #16) by replacing decimal characters with hex. I have tried with octal, and that didn't appear to work. Furthermore, it seems to pass some type of malformed request through when I use hex as well; this may be an issue with Gin, but I'm not 100% sure. EDIT: Turns out this was because I was trying to use HTTPS with a listener that didn't support it

BeePing listener:
beeping service

HTTP listening service:
http listening service

Curl requests:
curl requests

Fix

My suggested fix is to cast the destination IP to an integer in the validateTarget() function, before parsing the IP with net.ParseIP(), because that function is unable to parse hex values. However, I want to be sure that we catch all test cases before doing so.

Other Notes

It's also worth tracking down what's happening with the data as it's passed through, as it appears to be corrupted or malformed somehow. When I debugged the request, the req value seemed fine (in the CheckHTTP() function), but there were two other weird values that probably shouldn't have been so off: EDIT: Ignore this, see edit above
weird variables

Thanks to @jimen0 for bringing this to my attention. He may be able to chime in here as well.

Cheers,
Aaron (insp3ctre)

HTTP Header check

It would be useful to have a header check :

  • header name
  • header value (regexp ?)

Thanks !

beeping doesn't seem to handle 3XX error codes

Hi

beeping doesn't seem to handle 3XX error codes. I tried with http://www.google.com :

Request :
{"url": "http://www.google.com/", "insecure": false, "timeout": 20}

Response :
{"http_status":"200 OK","http_status_code":200,"http_body_pattern":true,"http_request_time":41,"dns_lookup":3,"tcp_connection":5,"server_processing":30,"content_transfer":0,"timeline":{"name_lookup":3,"connect":9,"pretransfer":9,"starttransfer":40},"ssl":false}

but with curl :

$ curl -v http://www.google.com
* Rebuilt URL to: http://www.google.com/
*   Trying 173.194.79.99...
* TCP_NODELAY set
* Connected to www.google.com (173.194.79.99) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=UTF-8
< Referrer-Policy: no-referrer
< Location: http://www.google.fr/?gfe_rd=cr&dcr=0&ei=Qr1DWs_uCsmD1gK11IT4Bw
< Content-Length: 268
< Date: Wed, 27 Dec 2017 15:33:22 GMT

Thanks !

Can't build due to undefined ciphers

I believe that the TLS ciphers being used are dependent on version of openSSL being used on the operating system. In my case, it appears that I don't have support for a number of the ciphers, so it won't build or run properly without them. I'm running OSX v10.12.4 (latest of Sierra). Others may have issue depending on their particular operating system and version of openSSL (if that's indeed the library that is being used by crypto/tls).

undefined ciphers on build

Missing error message when port is already used

Hi

I launched beeping as a simple user with this command ./beeping-v0.5.0 -listen 0.0.0.0 but port 8080 was already used.

Beeping didn't warn me with an error message and exit code was 0.

Thanks !

Json compatibility

When I send a json in python it look like this :
{"url": "url.example.com", "insecure": "true", "timeout": "20"}
and I get "invalid json sent" from pingmeback.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.