Giter VIP home page Giter VIP logo

nflog-zmq-pcap-pipe's Introduction

nflog-zmq-pcap-pipe

Set of scripts to allow selective dumping of packets with netfilter NFLOG module and sending of these over zeromq channel to remote host (producing pcap stream there) for analysis.

Use-case is sending traffic to Snort IDS on a remote machine with some pre-filtering (with iptables, since it's generally faster, simplier and more flexible than BPF or userspace filters) to exclude encrypted and irrelevant traffic (like raw VPN/IPSec packets and p2p).

Usage

Simple example for sending outgoing traffic to some random IP address for analysis from gateway.host to ids.host.

gateway.host:

iptables -I OUTPUT -d google.com -j NFLOG --nflog-group 0 --nflog-range 65535
ip6tables -I OUTPUT -d ipv6.google.com -j NFLOG --nflog-group 1 --nflog-range 65535
nflog-zmq-send 0,1 tcp://ids.host:1234

ids.host:

mkfifo /run/snort.pcap
nflog-pcap-recv tcp://0.0.0.0:1234 /run/snort.pcap &
snort --treat-drop-as-alert -r /run/snort.pcap

("--treat-drop-as-alert" option is useful because snort can't really "drop" or otherwise control real traffic in this scenario)

ZeroMQ endpoints

All the source/destination sockets are ZeroMQ endpoints. For syntax of these, see zeromq docs.

If nothing receives the flow on the other side of the pipe (or has any kind of temporary network problems), packets are buffered up to "--zmq-buffer" (ZMQ_SNDHWM) count and just dropped afterwards - overall goal is to make the channel as robust and easy-to-maintain as possible, and zeromq helps a lot here.

Multiple senders (possibly from multiple hosts) can be connected to one receiver.

Rate control

Throughput rate can be controlled either via "--rate-control" option (available in nflog-zmq-send and nflog-pcap-recv) or by piping traffic through separate binaries - nflog-zmq-compress and nflog-zmq-decompress.

Packets get squashed+compressed (zlib) upon reaching configurable "low watermark" ("--lwm" option) and dropped upon reaching "high watermark" ("--hwm" option), otherwise sent out as soon as possible.

Both "--rate-control" option and separate binaries use the same underlying code (and have same CLI options), but the advantage is in offloading compression cpu cost to a separate thread at the cost of associated ipc overhead.

Note that nflog-pcap-recv do not have any detection of whether received data is compressed, so if "--rate-control" is enabled on the sending side (or nflog-zmq-compress is used), same flag (or nflog-zmq-decompress) has to be used on (or setup before) the receiver, otherwise traffic in the resulting dump will be corrupted (i.e. packets batched together and fully compressed).

libnetfilter_log controls

nflog-zmq-send binary has the options to control parameters of netlink socket which it creates.

See libnetfilter_log documentation for more verbose description of these.

nflog-pcap-recv buffer interface

Flag "--buffer-interface" enables the receiver to keep up to "--buffer-window" MiB of last traffic, without delaying pcap throughput though.

Contents of this buffer can be easily retreived (in pcap format) for later inspection via nflog-pcap-query binary (or by sending any request to specified socket from anywhere).

Idea is to have much more complete picture of what's happening on the wire at the moment of some event, not just the single packet or flow which was matched. Generated pcap dump can be inspected by generic tools like wireshark or tcpdump.

Metrics (statsd interface)

Packet counter metrics on both ends can be send to statsd (think etsy/statsd or any of these). Disabled by default, see --statsd-* options.

Probably a bit broken at the moment, fixes are welcome.

libnflog settings, warnings

Various important libnflog settings are exposed in nflog_zmq_send.py cli and can/should be adjusted for particular use-case.

For instance, if something like this appears in the logs occasionally:

WARNING:nflog:nlbufsiz seem to be insufficient to hold unprocessed packets,
  consider raising it via corresponding function keyword

Option --libnflog-nlbufsiz should probably be increased or maybe qthresh/timeout adjusted, if it's a bursts in traffic, otherwise it might be the case that script just can't keep up with traffic (i.e. hovering at 100% cpu core usage).

Requirements

  • Python 2.7 with ctypes support and zlib if "--rate-control" or nflog-zmq-compressor is used
  • libnetfilter_log.so.1 on the sending side
  • pyzmq and zeromq, version 2.2.0 or higher.

Note that pyzmq detects zeromq version at build-time, so even though it will work after non-major zeromq lib updates (like 2.1.X -> 2.2.X), it won't have support for any of the new features and has to be rebuilt.

Advantages of nflog vs low-level traffic capture options (e.g. via libpcap)

  • Extensive filtering capabilities - you have all the netfilter modules and techniques at your disposal, so non-interesting high-volume traffic - tor, p2p, etc - can be skipped.
  • Ability to capture tunneled packets after decryption (traffic coming from ipsec, pptp, openvpn, ssh, etc) or transformation (stripping of ipip wrapping, netlink re-injection, etc).
  • Runtime reconfiguration (via iptables, for example).
  • Superior performance.

Why ctypes (and not, say, nflog-bindings)?

  • I'm more comfortable writing python than C or cython.
  • nflog-bindings leaks RAM like titanic, uses printf() in the code (and for each. captured packet, no less), horribly incomplete and buggy (there is an nflog-bindings-based implementation in git-log).
  • No extra deps, consistency.
  • Better support in non-cPython.

There's a very similar bindings module in scapy-nflog-capture, based on cffi instead of ctypes that should be much more segfault-free and future-proof than the one used here. If you experience any issues with the current module (like segfault right on start), try swapping nflog.py for that one.

nflog-zmq-pcap-pipe's People

Contributors

mk-fg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.