florianl / go-conntrack Goto Github PK
View Code? Open in Web Editor NEWc-binding free API for golang to communicate with the conntrack subsystem
License: MIT License
c-binding free API for golang to communicate with the conntrack subsystem
License: MIT License
An issue has been reported, that ICMPv6 sessions can not be altered.
Currently, this project has three different git tags
All three of them are not reflected in https://github.com/florianl/go-conntrack/blob/master/go.mod and vice versa. In addtion, this leads to confusion for some users - see #25.
As most users of this pkg are using go modules, I will delete these git tags to reduce confusion in the future.
Therefore, on September 1st 2020 I will delete the git tags.
If there are concerns with the removal of these git tags, please speak up.
I have an application using this library, and when on a heavily stressed system, I get a
'error: netlink receive: recvmsg: no buffer space available' error.
The library can be configured to log it, but I want to receive notification so I can act upon this
information.
I've appended the patch I'm using to give me what I'm after. I allow an error channel to be passed in with the Config structure.
diff --git a/conntrack.go b/conntrack.go
index febf035..3cb0d13 100644
--- a/conntrack.go
+++ b/conntrack.go
@@ -341,6 +341,9 @@ func (nfct *Nfct) register(ctx context.Context, t Table, groups NetlinkGroup, fi
}
}
nfct.logger.Printf("receiving error: %v", err)
if nfct.errorChan != nil {
nfct.errorChan <- err
}
return
}
diff --git a/conntrack_gteq_1.12.go b/conntrack_gteq_1.12.go
index f5bdba4..6d098a3 100644
--- a/conntrack_gteq_1.12.go
+++ b/conntrack_gteq_1.12.go
@@ -44,5 +44,9 @@ func Open(config *Config) (*Nfct, error) {
nfct.setWriteTimeout = func() error { return nil }
}
if config.Error != nil {
nfct.errorChan = config.Error
}
return &nfct, nil
}
diff --git a/conntrack_lt_1.12.go b/conntrack_lt_1.12.go
index 37f3b3a..5e30728 100644
--- a/conntrack_lt_1.12.go
+++ b/conntrack_lt_1.12.go
@@ -28,5 +28,9 @@ func Open(config *Config) (*Nfct, error) {
nfct.setReadTimeout = func() error { return nil }
nfct.setWriteTimeout = func() error { return nil }
if config.Error != nil {
nfct.errorChan = config.Error
}
return &nfct, nil
}
diff --git a/types.go b/types.go
index de46107..69b51ab 100644
--- a/types.go
+++ b/types.go
@@ -24,6 +24,9 @@ type Config struct {
// Time till a write action times out - only available for Go >= 1.12
WriteTimeout time.Duration
// If there is an error while using a handler (Register), the error
// that closed the reader routine will be sent on this channel
Error chan error
// Interface to log internals.
Logger *log.Logger
}
@@ -35,6 +38,10 @@ type Nfct struct {
logger *log.Logger
// If a fatal error occurs, the error will be written to this channel
// before the reader go-routine exits
errorChan chan error
setReadTimeout func() error
setWriteTimeout func() error
}
I am playing with the library, using the example code provided, but using RegisterFiltered
var filterNoIPv6Loopback = []ct.ConnAttr{
{ct.AttrOrigIPv6Src,
net.IPv6loopback
net.CIDRMask(128, net.IPv6len*8),
true,
},
}
func ExampleNfctRegister(ctx context.Context) {
nfct, err := ct.Open(&ct.Config{AddConntrackInformation: true})
if err != nil {
log.Fatal(err)
return
}
go func() {
<-ctx.Done()
nfct.Close()
}()
if err := nfct.RegisterFiltered(ctx, ct.Conntrack, ct.NetlinkCtNew, filterNoIPv6Loopback, printer); err != nil {
fmt.Println("could not register callback:", err)
return
}
fmt.Println("registered")
}
func printer(c ct.Con) int {
jsonOut, _ := json.MarshalIndent(c, "", " ")
fmt.Println(string(jsonOut))
fmt.Println("-------------------------")
return 0
}
func main() {
ctx, ctxCancel := context.WithCancel(context.Background())
ExampleNfctRegister(ctx)
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
go func() {
for sig := range c {
_ = sig
ctxCancel()
return
}
}()
fmt.Println("waiting for SIGINT signal")
<-ctx.Done()
}
Seems that by registering a filter for an IPv6 IP, only IPv6 events are received, but no IPv4 event.
The opposite is also true: registering a IPv4 filter will exclude all IPv6 events.
By registering no filter, both IPv4 and IPv6 events are received.
Is this expected?
Hi,
I really like this library and I would like to use it in my project, but I found a problem which is kind of show stopper for me.
I have a server with a lot of traffic, but I need to analyse small portion of it.
So I wanted to use RegisterFiltered method for filtration of traffic which is irrelevant for me. But I can't make it work.
I have created filters to remove traffic going from/to localhost in this example:
package main
import (
"context"
"github.com/florianl/go-conntrack"
"log"
"os"
"time"
)
func main() {
var err error
logger := log.Logger{}
logger.SetOutput(os.Stdout)
nfct, err := conntrack.Open(&conntrack.Config{
Logger: &logger,
AddConntrackInformation: true,
})
if err != nil {
logger.Fatalln("could not create nfct: %v", err)
}
var processor = func(c conntrack.Con) int {
if c.Origin.Proto.DstPort == nil {
logger.Printf("Port for %s is nil\n", c.Origin.Dst.String())
return 0
}
logger.Printf("Conntrack connection: src: %s - dest: %s:%d\n", c.Origin.Src.String(), c.Origin.Dst.String(), *c.Origin.Proto.DstPort)
return 0
}
filters := make([]conntrack.ConnAttr, 0)
filterLocalhostSrc := conntrack.ConnAttr{Type: conntrack.AttrOrigIPv4Src, Data: []byte{0x7f, 0x0, 0x0, 0x1}, Mask: []byte{0xff, 0xff, 0xff, 0xff}, Negate: true}
filterLocalhostDst := conntrack.ConnAttr{Type: conntrack.AttrOrigIPv4Dst, Data: []byte{0x7f, 0x0, 0x0, 0x1}, Mask: []byte{0xff, 0xff, 0xff, 0xff}, Negate: true}
filters = append(filters, filterLocalhostSrc, filterLocalhostDst)
err = nfct.RegisterFiltered(context.Background(), conntrack.Conntrack, conntrack.NetlinkCtNew|conntrack.NetlinkCtUpdate|conntrack.NetlinkCtDestroy, filters, processor)
if err != nil {
logger.Printf("could not register callback: %v\n", err)
}
time.Sleep(2 * time.Second)
err = nfct.Close()
if err != nil {
logger.Printf("nfct close error: %v\n", err)
}
time.Sleep(1 * time.Second)
logger.Println("closing")
}
But it doesn't work for me. This traffic is still forwarder to the userspace. Example from console:
Conntrack connection: src: 127.0.0.1 - dest: 127.0.0.1:8125
Conntrack connection: src: 127.0.0.1 - dest: 127.0.0.1:8125
Conntrack connection: src: 127.0.0.1 - dest: 127.0.0.1:55586
I noticed that following error is produced when I call the Close method:
receiving error: netlink receive: use of closed file
could not remove filter: netlink remove-bpf: setsockopt: bad file descriptor
could not unsubscribe from group: netlink leave-group: setsockopt: bad file descriptor
But I'm not sure if it is relevant.
I have also noticed that if I use positive filter (for example to get just traffic from localhost}, it seems that it works:
filterLocalhostSrc := conntrack.ConnAttr{Type: conntrack.AttrOrigIPv4Src, Data: []byte{0x7f, 0x0, 0x0, 0x1}, Mask: []byte{0xff, 0xff, 0xff, 0xff}, Negate: false}
So probably just negative filters don't work.
Any suggestion would be appreciated.
Thank you,
Martin
I've already responded on the other issue as to why Delete events were not registering, and I believe I have to solution as to why there is zero data.
In conntrack.go,
func parseConnectionMsg(msg netlink.Message, reqType int) (Conn, error) {
if msg.Header.Type&netlink.Error == netlink.Error {
should be
if msg.Header.Type == netlink.Error {
from the relevant linux header
#define NLMSG_NOOP 0x1 /* Nothing. */
#define NLMSG_ERROR 0x2 /* Error */
#define NLMSG_DONE 0x3 /* End of a dump */
#define NLMSG_OVERRUN 0x4 /* Data lost */
The current code does a bitmask on the Header.Type (uint16) with netlink.Error.
When a normal event comes in, it is 0x0100, so all is good. For delete, it is 0x0102, which is causing the code to bail and not decode. My reading of the headers is that it is a full 16bits, of numbers, not bitmap. Note that LMSG_DONE would also trigger your error condition.
#define NLMSG_MIN_TYPE 0x10 /* < 0x10: reserved control messages */
So anything above that should be ok. In a few places, you mask the reqType before making the call to parseConnectionMsg().
From netfilter/nfnetlink.h
/* netfilter netlink message types are split in two pieces:
* 8 bit subsystem, 8bit operation.
*/
#define NFNL_SUBSYS_ID(x) ((x & 0xff00) >> 8)
#define NFNL_MSG_TYPE(x) (x & 0x00ff)
#define NFNL_SUBSYS_CTNETLINK 1
The SUBSYS is NFNL_SUBSYS_CTNETLINK, but I'm not sure what the MSG_TYPES are otherwise. I'm rather new to netlink/conntrack, but I'm reasonably sure this is why the events are not being decoded when they should be. I'm also not sure what would get broken with these changes
In might be my go mod skills are deficient, but
$ go list -m -versions github.com/florianl/go-conntrack
github.com/florianl/go-conntrack v0.1.0
$ git tag
v0.1.0
v2.0.0
v3.0.0
but I can't build with v3.0.0.
$ mkdir new
$ cd new
$ cp ~/git/go-conntrack/example_DumpCPUStats_test.go d.go
$ go mod init
$ go build d.go
go: finding module for package github.com/florianl/go-conntrack
go: found github.com/florianl/go-conntrack in github.com/florianl/go-conntrack v0.1.0
../../pkg/mod/github.com/florianl/[email protected]/conntrack_nonlinux.go:55:9: undefined: extractAttributes
Edit d.go to ct "github.com/florianl/go-conntrack/v3"
$ go build d.go
go: finding module for package github.com/florianl/go-conntrack/v3
d.go:6:2: module github.com/florianl/go-conntrack@latest found (v0.1.0), but does not contain package github.com/florianl/go-conntrack/v3
Any ideas?
I'm continuing to replicate conntrack(1) with your library and have hit another issue regarding ICMP type/code values. What seems to be happening is that inside extractTuple(), while it gets called twice for the two IP directions, in both cases it contains an ICMP protocol Tuple which gets decoded. The first (for a ping) has a type of 8, then the second has a type of 0, which overwrites the 8. My understanding of ICMP and how conntrack handles it is a little light, but the linux header files indicates that it makes no sense to have both a Orig/Repl for these fields. So I suspect the correct solution is to only update them when 'dir' is 0.
It seems to me that there should not even be a second embedded protocol tuple for ICMP.
This patch replicates the output from conntrack(1).
diff --git a/attribute.go b/attribute.go
index 15d4841..15effef 100644
--- a/attribute.go
+++ b/attribute.go
@@ -224,17 +224,29 @@ func extractProtocolTuple(conn Conn, dir int, data []byte) error {
eleType := map[int]ConnAttrType{dirOrig: AttrOrigPortDst, dirReply: AttrReplPortDst, dirMaster: AttrMasterPortDst}[dir]
conn[eleType] = attr.Data
case ctaProtoIcmpID:
- conn[AttrIcmpID] = attr.Data
+ if dir == 0 {
+ conn[AttrIcmpID] = attr.Data
+ }
case ctaProtoIcmpType:
- conn[AttrIcmpType] = attr.Data
+ if dir == 0 {
+ conn[AttrIcmpType] = attr.Data
+ }
case ctaProtoIcmpCode:
- conn[AttrIcmpCode] = attr.Data
+ if dir == 0 {
+ conn[AttrIcmpCode] = attr.Data
+ }
case ctaProtoIcmpv6ID:
- conn[AttrIcmpID] = attr.Data
+ if dir == 0 {
+ conn[AttrIcmpID] = attr.Data
+ }
case ctaProtoIcmpv6Type:
- conn[AttrIcmpType] = attr.Data
+ if dir == 0 {
+ conn[AttrIcmpType] = attr.Data
+ }
case ctaProtoIcmpv6Code:
- conn[AttrIcmpCode] = attr.Data
+ if dir == 0 {
+ conn[AttrIcmpCode] = attr.Data
+ }
default:
fmt.Printf("Tuple %d is not yet implemented: %v\n", attr.Type, attr.Data)
}
Please does anyone knows how I can use this library to delete IPv4 and IPv6 conntrack entry?
Please help.
Thanks
I continue to develop my monitoring program using your library, and my rather pathological test case, which is the current release of the
edgex package, fires up about 18 docker containers that then proceed to furiously talk to each other (50,000 active
network connections).
I hit an interesting situation where my program would sit on about 40% CPU while monitoring; a large amount of the
processes CPU time being in runtime.futex (35%) and runtime.findrunnable (%18), which, is to my understanding are locking/scheduling related. I have 7-8 quite active go-routines running, on a quad core box.
I shutdown the docker containers, which normally triggered a failure on go-conntrack, which I then restarted.
I then restart the containers. My program now sat on about %20 CPU and those functions would drop to nearly nothing.
If I started my program before starting edgex, it would also be well behaved.
Looking at the different run profiles, I found that after restarting go-conntrack, I was not shutting down the worker go-routine so I had two sitting around.
To me I suspect this is related to the underlying netlink library thread locking its go-routine to clear up issues around
the setNS system call operating on a thread basis. I'm using setNS (and sock_diag) elsewhere, and I can appreciate
why you would want to lock your go-routine to a real thread. In my case I setNS, open a socket, then setNS back.
For me this seems to work (the socket is in the alternative namespace and I can keep on talking to it).
When I modified the arguments to netlink to set DisableNSLockThread to true, my problems seemed to go away.
I know what I'm doing elsewhere with setNS and am will to take the chance with turning this option on.
My problem is the go-conntrack library does not expose this option. Could I request you add it to you
conntrack.Config structure and pass it through.
I believe I'm seeing a 'semi-random' go-routine scheduling issue because a very active go-routine is being thread locked.
thanks
Hi!
First, thanks for your work on this tool @florianl!
I have an use case where I'll be deleting like 50~100 rules per second, do you think this lib can do the trick? (if not, any other way of doing that?)
I've tried it and sometimes it's giving me a lot of delay to delete some rules, maybe I'm using it the wrong way... I'm trying it on an ubuntu 18 VM.
Thanks!
The current API design makes the use of go-conntrack fairly hard to use, without deeper knowledge of conntrack. E.g. the returned ConnAttr
s are from type []byte
with variable size instead of proper Go types. So the returned values for AttrOrigIPv4Src
could be an net.IP
instead of []byte
.
So in the future it could look something like:
type IPTuple struct {
Src net.IP
Dst net.IP
...
}
type Con struct {
Origin *IPTuple
Reply *IPTuple
ID *uint32
...
}
As not all attributes, which are possible, are provided every time, the returned Con
struct contains pointers.These pointers can be nil
, if the given attribute is not provided.
use RegisterFiltered with NetlinkCtDestroy, no message will receive.
Hi!
Would it me possible to build a tool like this: https://github.com/hiveco/conntrack_exporter using this library?
I haven't fully checked this project but the hiveco one does everything in this file: https://github.com/hiveco/conntrack_exporter/blob/master/src/connection_table.cc
Hi, have a problem:
What is the minimum support Linux kernel version?
when I test func ExampleNfct_RegisterFiltered, the program print
Unknown message type: 0x00
map[]
is there something wrong with Func RegisterFiltered?
Hi 👋
I'm not sure if this behavior is intentional or not so apologize in advance for the confusion. 😅
I noticed users of this library using Go >1.12 can set a read timeout for the underlying Netlink socket via the ReadTimeout
configuration option.
However if a socket read operation times out, the polling Go routine will simply terminate and the registered callback will no longer receive data.
I believe these errors should be handled somehow as in
if opError, ok := err.(*netlink.OpError); ok {
if opError.Timeout() || opError.Temporary() {
// some backoff logic
continue
}
}
around
Lines 310 to 313 in f722615
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.