Giter VIP home page Giter VIP logo

befw's Introduction

BEFW project

Dynamic Host-Based Firewall

Intro

BEFW is a couple of golang-written tools that are eligible to maintain dynamic firewall rules across the datacenter(s). The approach is based on the service discovery concept: if we know there's a service, we can protect it. If we know there's a client for the service, we can allow him to connect to. BEFW is using consul service catalog to store services ( with their tcp/udp ports ) and is using consul key-value storage to store services clients. It's just a bit smarter than curl localhost:8200/v1/agent/services|perl -e '<script>'|iptables-restore, but 'a bit' is significant here.

Requirements

  • Any modern Linux distro with iptables & ipset support.
  • You must have at least one consul cluster with acl support enabled.
  • You must have consul agent on every node that runs befw.
  • We provide the only PuppetDB orchestration provider. You may need to develop yourth.

How to build

go install github.com/wgnet/befw/cmd/befw-cli
go install github.com/wgnet/befw/cmd/befw-firewalld
go install github.com/wgnet/befw/cmd/befw-sync

Deployment scheme

BEFW is not only tools but the approach too. If you build dynamic firewall from scratch, you may use our deployment guide.

Consul cluster

  1. Install regular consul cluster ( or federated cluster ) with N ( N % 2 == 1) servers and M agents. Each server must have consul agent running.
  2. Push some data and enjoy

ACLs

To protect your firewall configuration from unauthorized changes you need to create at least 3 acl sets on your consul cluster. To enable ACL support please read this article.

anonymous ACL

Allows anyone to read data from firewall ( usable for monitoring, metrics & UI for end users). Example.

node/agent ACL

Allows node/agent to read firewall configuration and register services. Example.

Commit/Master ACL

Allows 3pc software to write firewall configuration and purge expired data. Example.

Configuration

We use single consul datacenter to store firewall KV and we call it dc. You should specify this datacenter into configuration file. We use multiple consul datacenters ( one per site ) to store services & local KVs. You may use single-dc configuration, specify it's name as dc though.

Default firewall rules

  1. Change default iptables template

Service

  1. Install & launch befw-firewalld service on every agent node
  2. Edit sample configuration and place it to /etc/befw.conf
  3. Place sample service to services dir and see what will happen

Puppet/Hiera/PuppetDB

We use puppet to do items 3-6 and below as we have a huge puppet installation. Skip this if you don't. Puppet provides a great way to enumerate services on every node. You can just add something like that in every role/application-definition class:

file { "/etc/befw.service.d/${title}.json":
    ensure  => file,
    content => "<-- json here -->",
}

This will provide your services in consul automagically. Just a few patches once will save your time forever. See samples for more information.

befw-sync

Here, in Wargaming, we also use hiera & puppetdb to provide both services & clients for our services. The idea is that if we can collect & commit something to puppetdb ( like 'i_need_this_service_to_work'), that we can just grab it from puppetdb and store into consul. befw-sync does this job, looking into puppetdb for a corresponding resource and grabbing parameters to push to consul KV storage. See sample config to get into, but we doubt it is useful if you don't have such huge puppet installation on your world.

Managing rules

Adding new service

  • Place a service file to services directory or
  • Post a http request to local consul agent ( v1/agent/register ) or
  • Allow consul-supported software ( like vault ) to manage it on its own

N.B. use tag 'befw' to generate rules for this service. Use tags 'port/protocol' ( like '80/tcp' ) to specify additional ports for the service.

Adding new rules

  • Place ip/network and expiry value to KV storage on dc datacenter or
  • Place alias and expiry value to KV storage on dc datacenter or
  • Place a new data to alias definition to KV storage on dc datacenter

N.B. You can specify world, dc or node level while placing the rules.

Examples

consul.KV().Put("befw/$service$/service_tcp_443/192.168.1.1/30", time.now()+2*week)
consul.KV().Put("befw/$service$/dc/service_tcp_443/192.168.1.1/30", time.now()+2*week)
consul.KV().Put("befw/$service$/dc/nodename/service_tcp_443/192.168.1.1/30", time.now()+2*week)
consul.KV().Put("befw/$service$/service_tcp_443/$trusted$", -1) // <0 never expires
consul.KV().Put("befw/$alias$/$trusted$/192.168.1.1.30", time.now()+1*hour)

Tools

pusher is a very primitive python (2.7) tool to manage befw manually. Please avoid using it in production. Usage: ./pusher.py list|add|rm|clist|cadd|crm [opts] Config file: ~/.pusher.conf

host=localhost
port=8500
dc=consul
token=2375a28a-75ec-4b5f-a30f-3e68f8239a0a

Example usage:

# no clients for service
$ ./pusher.py clist myown_tcp_5672
[+] DC= dc , Node= node
# adding new one for level 3 ( node )
$ ./pusher.py cadd myown_tcp_5672 127.0.0.1 3
[+] DC= dc , Node= node
Added 127.0.0.1 to befw/dc/node/myown_tcp_5672/127.0.0.1 with expiry=1554896188
# yes we have it on KV now
$ ./pusher.py clist myown_tcp_5672
[+] DC= dc , Node= node
befw/dc/node/myown_tcp_5672/
 *127.0.0.1
# deleteting 127.0.0.1 from all levels
$ ./pusher.py crm myown_tcp_5672 127.0.0.1
[+] DC= dc , Node= node
Deleting befw/myown_tcp_5672/127.0.0.1
Deleting befw/dc/myown_tcp_5672/127.0.0.1
Deleting befw/dc/node/myown_tcp_5672/127.0.0.1
# no clients found
$ ./pusher.py clist myown_tcp_5672
[+] DC= dc , Node= node
$

Changelog

See CHANGELOG.

Known issues

See ISSUES.

Contributing

See CONTRIB.

befw's People

Contributors

annmuor avatar difrex avatar hypunk avatar kodachi128 avatar nurmukhamed avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

befw's Issues

can't create static ipsets when starting befw-firewalld

Hi there,

I'm playing with your tool and while running through the demo I receive following error. Could you please help me with that?

agorovoy@net-hbfw01:~/go/src/github.com/wgnet/befw/demo$  befw-firewalld -noroot -debug -config befw.conf
2020/12/28 13:14:30 Opening queue
2020/12/28 13:14:30 Unbinding to selected family
2020/12/28 13:14:30 nflog_unbind_pf failed
2020/12/28 13:14:30 Binding to selected family
2020/12/28 13:14:30 Creating queue
2020/12/28 13:14:30 nflog_bind_group failed
NFLogger started, you can get information from /var/run/befw/*
[API]: Can't run local socket:  listen unix /var/run/befw/api.sock: bind: address already in use
[Consist] can't create static ipsets
[Rules] IPSet refresh error: 
-------
create rules_allow hash:net
create tmp-wODtb1Rj7sOBmGq0NRTr134JX hash:net
flush tmp-wODtb1Rj7sOBmGq0NRTr134JX
swap tmp-wODtb1Rj7sOBmGq0NRTr134JX rules_allow
destroy tmp-wODtb1Rj7sOBmGq0NRTr134JX

-------
Error while creating ipset rules_allow 
Can't apply state:  
Error while refresh():  ; sleeping for  40  seconds
^C2020/12/28 13:14:38 Closing queue

errors during compilation

Hi, not sure if the project is alive. I've got multiple errors while running make:
./build.sh

/home/xxx/befw/src/befw [/home/xxx/befw/src/befw.test]

befw/struct.go:26:6: befwServiceProto redeclared in this block
previous declaration at befw/config.go:37:6
befw/struct.go:28:6: RefreshMethod redeclared in this block
previous declaration at befw/config.go:39:6
....

compilation worked fine after removing befw/struct.go. Looks like the file is a previous copy of config.go.

client prefix not added to ipset members list

Hi folks,

I'm seeking for your support for the following.

I'm able to successfully register a service

curl -X PUT localhost:8500/v1/agent/service/register -H "Content-Type: application/json" -d "{ \"Name\": \"rdp\", \"Port\": 3389, \"Tags\": [ \"befw\" ]}"

so I can see it in Consul UI and I also have the corresponding ipset and Iptables rule created automatically

agorovoy@m1-net-hbfw01:~$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain BEFW (0 references)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             match-set rules_allow src
REJECT     all  --  anywhere             anywhere             match-set rules_deny src reject-with icmp-port-unreachable
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 8300,8301,8302,8500,8501,8600
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 3389 match-set rdp_tcp_3389 src
NFLOG      tcp  --  anywhere             anywhere             multiport dports 3389 nflog-group 402
REJECT     tcp  --  anywhere             anywhere             multiport dports 3389 reject-with icmp-port-unreachable
agorovoy@m1-net-hbfw01:~$ sudo ipset list
Name: rules_allow
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 344
References: 1
Number of entries: 0
Members:

Name: rules_deny
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 344
References: 1
Number of entries: 0
Members:

Name: rdp_tcp_3389
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 344
References: 1
Number of entries: 0
Members:

Unfortunately I don't get how to add clients there, so that client prefixes populate the ipset. Doing something like this does not help
consul kv put befw/$ipset$/rdp_tcp_3389/192.168.0.0/16 -1
curl -X PUT localhost:8500/v1/kv/befw/consul/rdp_tcp_3389/192.168.0.0/16 -H "Content-Type: application/json" -d "-1"

What is the correct format of storing key/values in Consul KV store so that they be populated to local ipsets via befw automatically. Could you please provide an examples via both curl and cli.

Thanks,
Alex

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.