SpoolEventReader packet data about py-idstools HOT 6 CLOSED

There are 2 different data fields.. SpoolEventReader.Next returns an Event which can have a list of "packets" and "extra-data" records attached to it. Each of this have a data field. In the case of the packet, the data is the raw binary data of the packet found in the unified log file.

Likewise for extra data, the data is just the raw data from that unified record. To interpret it you need to look at the data-type field and process it accordingly. Referring to the unified2 documentation in the Snort distribution should help here.

from py-idstools.

Sorry for not specifying the field. I am looking into Events and its packets.
I saw the doc https://www.snort.org/faq/readme-unified2 . It recommends U2Spewfoo to read logs, and that's what I was using until now that I need aggregation when events appears.
So, you said data in event's packets is in raw binary, but I don't know how to process it. I have tried with some different online converters to hexadecimal without success. I have seen that this format is also used for ip's in 'destination-ip.raw' and 'source-ip.raw' fields. But I don't find the transformation relation between this format and any other that I may know. Since U2Spewfoo provides an hexadecimal packet data, there should be a direct transformation between the binary raw and hex, but I can neither find it in the source code of U2Spewfoo.

from py-idstools.

Look at https://github.com/jasonish/py-idstools/blob/master/idstools/scripts/u2spewfoo.py#L71, this is the function that prints the raw data as hex, with a printable section for the idstools implementation of u2spewfoo.

Basically you have to do this yourself and format how you want it. In Python its basically:

for b in raw_data:
    print("0x%02x", b)

which will print the hex value of each byte on a new line.

from py-idstools.

I think one can just use some of the code included in py-idstools, the following works for me with f6b039e:

from idstools import maps
from idstools.scripts.u2json import Formatter
from idstools import unified2

msgmap = maps.SignatureMap()
classmap = maps.ClassificationMap()
formatter = Formatter(msgmap=msgmap, classmap=classmap)

reader = unified2.SpoolRecordReader(directory='/var/log/snort',
                                   prefix='unified2.log',
                                   follow=True)

for record in reader:
        formatted = dict(formatter.format(record))
        if 'packet' in formatted:
                print(formatted['packet']['data'])

from py-idstools.

I wonder if it would be useful to expose some formatting/printing functions in the library.

By that I mean pulling the formatting function out of the u2spefoo script and putting it in the library for easier access from user scripts.

from py-idstools.

There is also now util.format_printable. Closing. Use u2json and u2eve as a reference for printing the raw data.

from py-idstools.