Giter VIP home page Giter VIP logo

ipfix's Introduction

ipfix

Package ipfix implements an IPFIX (RFC 5101) parser and interpreter.

Build & Test API Documentation MIT License

Input data from an io.Reader or a []byte is parsed and chunked into messages. Template management and the standard IPFIX types are implemented so a fully parsed data set can be produced. Vendor fields can be added at runtime.

Example

To read an IPFIX stream, create a Session and then use ParseBuffer to parse data coming from a single UDP packet or similar.

var conn net.PacketConn // from somewhere
buf := make([]byte, 65507) // maximum UDP payload length
s := ipfix.NewSession()
for {
    n, _, err := conn.ReadFrom(buf)
    // handle err
    msg, err := s.ParseBuffer(buf[:n])
    // handle msg and err
}

To interpret records for correct data types and field names, use an interpreter:

i := ipfix.NewInterpreter(s)
var fieldList []ipfix.InterpretedField
for _, rec := range msg.DataRecords {
    fieldList = i.InterpretInto(rec, fieldList[:cap(fieldList)])
    // handle the field list
}

To add a vendor field to the dictionary so that it will be resolved by Interpret, create a DictionaryEntry and call AddDictionaryEntry.

e := ipfix.DictionaryEntry{
    Name: "someVendorField",
    FieldId: 42,
    EnterpriseId: 123456,
    Type: ipfix.Int32
}
i.AddDictionaryEntry(e)

License

The MIT license.

Usage

See the documentation.

ipfix's People

Contributors

calmh avatar floren avatar kris-watts-gravwell avatar john-floren-gravwell avatar georgmu avatar available-username avatar ashnwade avatar asenci avatar lebauce avatar

Stargazers

Entrapta Jones avatar avatar Sigurd Høgsbro avatar

Watchers

avatar James Cloos avatar avatar

Forkers

traetox floren lyckety david-fritz-gravwell ashnwade

ipfix's Issues

slice bounds check failed in ipfix packet parser

What are the steps to reproduce this issue?

  1. unknown

What happens?

panic in packet parser

panic: runtime error: slice bounds out of range [:516] with capacity 508

goroutine 23 [running]:
github.com/gravwell/ipfix.Message.marshalRecords({{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, {0xc000160a80, ...}}, ...)
    github.com/gravwell/[email protected]/parser.go:863 +0x55a
github.com/gravwell/ipfix.(*Session).Marshal(0xc0000b62d0, {{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, ...})
    github.com/gravwell/[email protected]/parser.go:737 +0x1ca
main.(*IpfixHandler).routine(0xc0001235c0, 0x0?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:301 +0xb13
created by main.(*IpfixHandler).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:186 +0xfb

What were you expecting to happen?

Not panic

Any logs, error output, etc?

panic: runtime error: slice bounds out of range [:516] with capacity 508

goroutine 23 [running]:
github.com/gravwell/ipfix.Message.marshalRecords({{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, {0xc000160a80, ...}}, ...)
    github.com/gravwell/[email protected]/parser.go:863 +0x55a
github.com/gravwell/ipfix.(*Session).Marshal(0xc0000b62d0, {{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, ...})
    github.com/gravwell/[email protected]/parser.go:737 +0x1ca
main.(*IpfixHandler).routine(0xc0001235c0, 0x0?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:301 +0xb13
created by main.(*IpfixHandler).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:186 +0xfb

goroutine 1 [chan receive, 81958 minutes]:
github.com/gravwell/gravwell/v3/ingesters/utils.WaitForQuit()
    github.com/gravwell/gravwell/v3/ingesters/utils/signals.go:25 +0x128
main.main()
    github.com/gravwell/gravwell/v3/ingesters/netflow/main.go:142 +0x1845

goroutine 18 [select]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).cacheHandler(0xc000121170)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:292 +0x41b
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:196 +0xa6d

goroutine 19 [chan receive, 29910 minutes]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).run(0xc000121170)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:207 +0x45
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:198 +0xaaf

goroutine 20 [select]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).cacheHandler(0xc000098000)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:292 +0x41b
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:196 +0xa6d

goroutine 21 [chan receive]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).run(0xc000098000)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:207 +0x45
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:198 +0xaaf

goroutine 5 [chan receive, 29910 minutes]:
github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).connRoutine(0xc000004300, 0x0)
    github.com/gravwell/gravwell/v3/ingest/muxer.go:1306 +0x3b7
created by github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingest/muxer.go:483 +0x1a8

goroutine 6 [sleep]:
time.Sleep(0x12a05f200)
    runtime/time.go:195 +0x125
github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).stateReportRoutine(0xc000004300)
    github.com/gravwell/gravwell/v3/ingest/muxer.go:551 +0x3f
created by github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingest/muxer.go:488 +0x278

goroutine 22 [select]:
github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).writeRelayRoutine(0xc000004300, 0xc0001234a0, 0xc0002280e0)
    github.com/gravwell/gravwell/v3/ingest/muxer.go:1122 +0x357
created by github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).connRoutine in goroutine 5
    github.com/gravwell/gravwell/v3/ingest/muxer.go:1299 +0x1f9

goroutine 24 [IO wait, 81958 minutes]:
internal/poll.runtime_pollWait(0x7fd0e8ae69a8, 0x72)
    runtime/netpoll.go:343 +0x85
internal/poll.(*pollDesc).wait(0xc000215300?, 0xc0001079b8?, 0x0)
    internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
    internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).ReadFromInet6(0xc000215300, {0xc0001079b8, 0x5b8, 0x5b8}, 0x0?)
    internal/poll/fd_unix.go:274 +0x22b
net.(*netFD).readFromInet6(0xc000215300, {0xc0001079b8?, 0x0?, 0x0?}, 0x0?)
    net/fd_posix.go:72 +0x25
net.(*UDPConn).readFrom(0x30?, {0xc0001079b8?, 0xc000034d80?, 0x0?}, 0xc000034d80)
    net/udpsock_posix.go:59 +0x79
net.(*UDPConn).readFromUDP(0xc000116658, {0xc0001079b8?, 0x0?, 0x0?}, 0x0?)
    net/udpsock.go:149 +0x30
net.(*UDPConn).ReadFromUDP(...)
    net/udpsock.go:141
main.(*NetflowV5Handler).routine(0xc000123620, 0x2?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:104 +0xec
created by main.(*NetflowV5Handler).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:90 +0xfb

goroutine 25 [select]:
main.relay(0xc000123560, 0x0?, {0x0, 0x0, 0x0}, 0x0?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/main.go:186 +0x13b
created by main.main in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/main.go:137 +0x1809

goroutine 7 [syscall, 81958 minutes]:
os/signal.signal_recv()
    runtime/sigqueue.go:152 +0x29
os/signal.loop()
    os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 1
    os/signal/signal.go:151 +0x1f

Any other comments?

Seems like a pretty standard lack of a bounds check: https://github.com/gravwell/ipfix/blob/master/parser.go#L863

What versions of software are you using?

5.4.1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.