Giter VIP home page Giter VIP logo

planb's Introduction

PlanB: a distributed HTTP and websocket proxy

Build Status

What Is It?

PlanB is a HTTP and websocket proxy backed by Redis and inspired by Hipache.

It aims to be fully compatible with Hipache when Redis is used as a backend. The same format is used for all keys stored in Redis so migrating from Hipache to PlanB should be completely seamless. The process should be as simple as replacing Hipache's executable for PlanB.

Start-up flags

The following flags are available for configuring PlanB on start-up:

  • --listen value, -l value: Address to listen (default: "0.0.0.0:8989")
  • --tls-listen value: Address to listen with tls
  • --tls-preset value: Preset containing supported TLS versions and cyphers, according to https://wiki.mozilla.org/Security/Server_Side_TLS. Possible values are [modern, intermediate, old] (default: "modern")
  • --metrics-address value: Address to expose prometheus /metrics
  • --load-certificates-from value: Path where certificate will found. If value equals 'redis' certificate will be loaded from redis service. (default: "redis")
  • --read-redis-network value: Redis address network, possible values are "tcp" for tcp connection and "unix" for connecting using unix sockets (default: "tcp")
  • --read-redis-host value: Redis host address for tcp connections or socket path for unix sockets (default: "127.0.0.1")
  • --read-redis-port value: Redis port (default: 6379)
  • --read-redis-sentinel-addrs value: Comma separated list of redis sentinel addresses
  • --read-redis-sentinel-name value: Redis sentinel name
  • --read-redis-password value: Redis password
  • --read-redis-db value: Redis database number (default: 0)
  • --write-redis-network value: Redis address network, possible values are "tcp" for tcp connection and "unix" for connecting using unix sockets (default: "tcp")
  • --write-redis-host value: Redis host address for tcp connections or socket path for unix sockets (default: "127.0.0.1")
  • --write-redis-port value: Redis port (default: 6379)
  • --write-redis-sentinel-addrs value: Comma separated list of redis sentinel addresses
  • --write-redis-sentinel-name value: Redis sentinel name
  • --write-redis-password value: Redis password
  • --write-redis-db value: Redis database number (default: 0)
  • --access-log value: File path where access log will be written. If value equals 'syslog' log will be sent to local syslog. The value 'none' can be used to disable access logs. (default: "./access.log")
  • --request-timeout value: Total backend request timeout in seconds (default: 30)
  • --dial-timeout value: Dial backend request timeout in seconds (default: 10)
  • --client-read-timeout value: Maximum duration for reading the entire request, including the body (default: 0s)
  • --client-read-header-timeout value: Amount of time allowed to read request headers (default: 0s)
  • --client-write-timeout value: Maximum duration before timing out writes of the response (default: 0s)
  • --client-idle-timeout value: Maximum amount of time to wait for the next request when keep-alives are enabled (default: 0s)
  • --dead-backend-time value: Time in seconds a backend will remain disabled after a network failure (default: 30)
  • --flush-interval value: Time in milliseconds to flush the proxied request (default: 10)
  • --request-id-header value: Header to enable message tracking
  • --active-healthcheck: Enable active healthcheck on dead backends once they are marked as dead. Enabling this flag will result in dead backends only being enabled again once the active healthcheck routine is able to reach them.
  • --engine value: Reverse proxy engine, options are 'native', 'sni' and 'fasthttp'. Using 'sni' and 'fasthttp' is highly experimental and not recommended for production environments. (default: "native")
  • --backend-cache: Enable caching backend results for 2 seconds. This may cause temporary inconsistencies.
  • --help, -h: show help
  • --version, -v: print the version

The --read-redis-* flags refer to the Redis server used for read-only operations (reading the backends for each frontend).

The --write-redis-* flags refer to the Redis server used for write operations (marking and publishing dead backends).

Separating the read and write servers is not mandatory but is useful for improving performance. A common scenario is having a slave Redis server on localhost configured as --read-redis and a remote Redis master configured as --write-redis.

Features

  • Load-Balancing
  • Dead Backend Detection
  • Dynamic Configuration
  • WebSocket
  • TLS

Install

The easiest way to install PlanB is to pull the trusted build from the hub.docker.com and launch it in the container:

# run Redis
docker run -d -p 6379:6379 redis

# run PlanB
docker run -d --net=host tsuru/planb:v1 --listen ":80"

VHOST Configuration

The configuration is managed by Redis that makes possible to update the configuration dynamically and gracefully while the server is running, and have that state shared across workers and even across instances.

Let's take an example to proxify requests to 2 backends for the hostname www.tsuru.io. The 2 backends IP are 192.168.0.42 and 192.168.0.43 and they serve the HTTP traffic on the port 80.

redis-cli is the standard client tool to talk to Redis from the terminal.

Follow these steps:

Create the frontend:

$ redis-cli rpush frontend:www.tsuru.io mywebsite
(integer) 1

The frontend identifer is mywebsite, it could be anything.

Add the 2 backends:

$ redis-cli rpush frontend:www.tsuru.io http://192.168.0.42:80
(integer) 2
$ redis-cli rpush frontend:www.tsuru.io http://192.168.0.43:80
(integer) 3

Review the configuration:

$ redis-cli lrange frontend:www.tsuru.io 0 -1
1) "mywebsite"
2) "http://192.168.0.42:80"
3) "http://192.168.0.43:80"

TLS Configuration using redis (optional)

$ redis-cli -x hmset tls:www.tsuru.io certificate < server.crt
$ redis-cli -x hmset tls:www.tsuru.io key < server.key

$ redis-cli -x hmset tls:*.tsuru.com certificate < wildcard.crt
$ redis-cli -x hmset tls:*.tsuru.io key < wildcard.key

TLS Configuration using FS (optional)

create directory following this structure

cd certficates
ls
*.domain-wildcard.com.key
*.domain-wildcard.com.crt
absolute-domain.key
absolute-domain.crt

While the server is running, any of these steps can be re-run without messing up with the traffic.

Debbugging and Troubleshooting

One way to debug/toubleshoot planb is by analyzing the running goroutines.

Planb is able to handle the USR1 signal to dump goroutines in its execution screen:

$ kill -s USR1 <planb-PID>

planb's People

Contributors

andrestc avatar andrewsmedina avatar caarlos0 avatar cezarsa avatar fsouza avatar ggarnier avatar guilhermef avatar gwmoura avatar mdjunior avatar morpheu avatar raulgbrmf avatar readmecritic avatar tarsisazevedo avatar wpjunior avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

planb's Issues

Add a flag to planb to facilitate tracing requests

If any request message had a header with an unique ID, tracking its the tracking of requests would be much easier
So, it would be very helpful if planb had a way to add an unique ID to the requests that travel through him and do not have an ID yet.
Add a flag called --request-id-header responsible for doing the issues above would solve this issue.

need to modify for detection of wildcard subdomain.

i need wildcards. I will modify the planb code myself. pls drop a hint where that file line is so i can work on it.

i will also add letencrypt ssl to it if u can point me to where i should modify the code. thx

log real client ip address

Hi
we are using planb behind a haproxy on tcp mode for handling ssl certificates with planb. We have more than 15000 domains with its certificates on redis and works fine.

Now we receive the x-forwarder-proto correctly, but at the planb logs only get the haproxy ipaddress, this may be caused because you do not receive the header, being haproxy on tcpmode.

Do you support the proxy-protocol from haproxy ?

http://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

planb log line

:ffff:172.16.70.41 - - [05/Nov/2017:20:53:39 +0000] "GET /es/ HTTP/1.1" 200 -1 "https://www.deltacartarragona.es/es/" "Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 BingPreview/1.0b" "Swplanb:77adf0d5-0489-4d4a-7887-a3f9ea06f798" "www.deltacartarragona.es" "" 0.325 0.325

MySQL backend

Is possible to add MySQL backend?
Google Cloud offers a fully-managed HA MySQL server, so having support for MySQL in PlanB would allow horizontal scaling with no issue, just point each planb instance to the Google SQL server and you are ready to go.

The same with Redis is much more complicated

Should planb ignore ports on hosts when looking for back-ends?

Currently, this is an inconsistency between Hipache and planb: if the Host header contains a port, Hipache ignores it, but planb doesn't.

For example:

% redis-cli lrange frontend:app.tsuru.example.com 0 -1
1) "http://localhost:9090"
% grep app.tsuru.example.com /etc/hosts
127.0.0.1 app.tsuru.example.com
% curl app.tsuru.example.com:8080
# hipache forwards this request to the back-end
% curl app.tsuru.example.com:8989
# planb responds with "no such route"

Should this be considered a bug in planb or we don't care?

Configuration to load enviroment variables with systemd

For Linux distro (in this case Ubuntu 16.04) that use the systemd, it worked when the file was written like this (Example: /etc/defaulft/planb)

PLANB_LISTEN=0.0.0.0:80
PLANB_READ_REDIS_HOST=redis.yourcompany.com
PLANB_READ_REDIS_PORT=6379
PLANB_WRITE_REDIS_HOST=redis.yourcompany.com
PLANB_WRITE_REDIS_PORT=6379

Instead of using in upstart:

PLANB_OPTS="--listen 0.0.0.0:80 --read-redis-host redis.yourcompany.com --read-redis-port 6379 --write-redis-host redis.yourcompany.com --write-redis-port 6379"

In the /etc/systemd/system/planb.service file, the line ExecStart=/usr/bin/planb, has been changed to:

ExecStart=/usr/bin/planb --listen ${PLANB_LISTEN} --read-redis-host "${PLANB_READ_REDIS_HOST}" --read-redis-port ${PLANB_READ_REDIS_PORT} --write-redis-host "${PLANB_WRITE_REDIS_HOST}" --write-redis-port ${PLANB_WRITE_REDIS_PORT}

Complete file:

[Unit]
Description=planb

[Service]
Type=simple
User=root
Group=root
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/planb
ExecStart=/usr/bin/planb --listen ${PLANB_LISTEN} --read-redis-host "${PLANB_READ_REDIS_HOST}" --read-redis-port ${PLANB_READ_REDIS_PORT} --write-redis-host "${PLANB_WRITE_REDIS_HOST}" --write-redis-port ${PLANB_WRITE_REDIS_PORT}
Restart=always
WorkingDirectory=/

[Install]
WantedBy=multi-user.target

Support forcing HTTPS traffic based on X-Forwarded-Proto?

Hello hello!

Would it make sense to have an option in planb to redirect HTTP traffic to HTTPS traffic based on X-Forwarded-Proto? i.e. planb would send a 301 redirect to the client even though it's not terminating the SSL connection.

HTTPS support

PlanB should support TLS using per front-end certificates, or a global certificate.

Open question: should it automatically handle wildcard certificates? If I have myapp.env1.company.com and myapp.env2.company.com and I have both *.env1.company.com and *.env2.company.com certificates, could I just configure them and then PlanB could use the proper certificate for each app?

X-Forwarded-Proto header

Hi, we are planing to use planb for a new service using https and need the X-Forwarded-Proto header.

Is planb sending this header ? I see at your code X-Forwarder-For

Automatically try another backend if available after an error

If an error caused a backend to be marked as dead it would be nice if planb could be configured to automatically try another backend without dropping the request. Only permanent network errors cause a backend to be marked as dead and it should be safe to replay the request after such errors because nothing arrived at the destination.

If we want to be extra safe we could automatically retry only GET/HEAD requests.

http: TLS handshake error from 172.16.70.41:19289: tls: client offered an unsupported, maximum protocol version of 301

Hello.

We are running planb on production with 18000 domains with tls and its certificates and keys on redis for more of 3 months and are very satisfied.
But in the last weeks, I am receiving issues from customers about problems seeing our webs from some versions of Internet Explorer.

At the syslog I can see a lot of messages regarding unsupported protocols from clients like these with versions protocol from 300 to 302

http: TLS handshake error from 172.16.70.41:19289: tls: client offered an unsupported, maximum protocol version of 301
http: TLS handshake error from 172.16.70.42:49109: tls: client offered an unsupported, maximum protocol version of 300
http: TLS handshake error from 172.16.70.41:19201: tls: client offered an unsupported, maximum protocol version of 302

at first, I thought it was a problem with the letsencrypt certificates that we are using, but testing the same certificate with haproxy don't fail

I know that the problem is at the client browser but need to ask you if there is any possible solution for this before moving to haproxy, because I prefer planb so it is very flexible and scalable.

Sites that fail with planb and IE - letsencrypt cert
https://www.bardetapassansebastian.com/
https://www.taller-pintura-moto-vigo.com

Sites working with haproxy and IE - letsencrypt cert
https://www.qdqplus.es/MP/petit-marc/
https://www.infoqdq.es/MP/rincon-luis-mari/

Prometheus metrics

I Opened the same issue on Galeb. I copy paste the same :P

Prometheus is one of the new metrics/monitoring/alerting system, has been developed by Soundcloud. This system has gained a lot of traction and popularity, and it's being used in some of the big companies like Google, Soundcloud, DigitalOcean...

The community is growing and it supports a lot of languages for its metric client including Java

Would be nice to add support for prometheus along with statsD on planB metrics.

Better error message when all backends are dead

If all backends are marked as dead, planb will return the error message no such route which is also returned when no frontend:<host> entry is found on redis. This is confusing and could lead users to think there's a problem with stored entries in redis when in fact the problem is in the backend.

The message no such route should ONLY be used if there is no frontend:<host> entry in Redis.

ETCd backend

I think using etcd watches should be a more performant than Redis Queries, may be an experiment or what do you think about this new backend?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.