Giter VIP home page Giter VIP logo

go-libaudit's Introduction

go-libaudit

Go Documentation

go-libaudit is a library for Go (golang) for communicating with the Linux Audit Framework. The Linux Audit Framework provides system call auditing in the kernel and logs the events to user-space using netlink sockets. This library facilitates user-space applications that want to receive audit events.

Installation and Usage

Package documentation can be found on GoDoc.

Installation can be done with a normal go get:

$ go get github.com/elastic/go-libaudit

audit example

go-libaudit has two example applications that you can use to try the library. The first is audit which registers to receive audit events from the kernel and outputs the data it receives to stdout. The system's auditd process should be stopped first.

$ go install github.com/elastic/go-libaudit/cmd/audit@main
$ sudo $GOPATH/bin/audit -d

auparse example

The second is auparse which parses the log files from the Linux auditd process or the output of the audit example command. It combines related log messages that are a part of the same event.

$ go install github.com/elastic/go-libaudit/cmd/auparse@main
$ sudo cat /var/log/audit/audit.log | auparse
---
type=CRED_ACQ msg=audit(1481077334.302:545): pid=1444 uid=0 auid=1000 ses=4 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/1 res=success'
---
type=USER_START msg=audit(1481077334.303:546): pid=1444 uid=0 auid=1000 ses=4 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/1 res=success'
---
type=SYSCALL msg=audit(1481077334.304:547): arch=c000003e syscall=59 success=yes exit=0 a0=7f683953a5d8 a1=7f683953fd38 a2=7f6839543a90 a3=6 items=2 ppid=1444 pid=1445 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=4 comm="su" exe="/usr/bin/su" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
type=EXECVE msg=audit(1481077334.304:547): argc=1 a0="su"
type=CWD msg=audit(1481077334.304:547):  cwd="/home/andrew_kroh"
type=PATH msg=audit(1481077334.304:547): item=0 name="/bin/su" inode=5026 dev=08:01 mode=0104755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:su_exec_t:s0 objtype=NORMAL
type=PATH msg=audit(1481077334.304:547): item=1 name="/lib64/ld-linux-x86-64.so.2" inode=16778495 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0 objtype=NORMAL

It supports outputting the messages as plain text (default), JSON, or YAML by using -format=yaml for example.

$ sudo cat /var/log/audit/audit.log | auparse -format=json
---
{"@timestamp":"2016-12-07 02:22:14.302 +0000 UTC","acct":"root","auid":"1000","exe":"/usr/bin/sudo","grantors":"pam_env,pam_unix","op":"PAM:setcred","pid":"1444","raw_msg":"audit(1481077334.302:545): pid=1444 uid=0 auid=1000 ses=4 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct=\"root\" exe=\"/usr/bin/sudo\" hostname=? addr=? terminal=/dev/pts/1 res=success'","record_type":"CRED_ACQ","result":"success","sequence":"545","ses":"4","subj_category":"c0.c1023","subj_domain":"unconfined_t","subj_level":"s0-s0","subj_role":"unconfined_r","subj_user":"unconfined_u","terminal":"/dev/pts/1","uid":"0"}
---
{"@timestamp":"2016-12-07 02:22:14.303 +0000 UTC","acct":"root","auid":"1000","exe":"/usr/bin/sudo","grantors":"pam_keyinit,pam_limits","op":"PAM:session_open","pid":"1444","raw_msg":"audit(1481077334.303:546): pid=1444 uid=0 auid=1000 ses=4 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits acct=\"root\" exe=\"/usr/bin/sudo\" hostname=? addr=? terminal=/dev/pts/1 res=success'","record_type":"USER_START","result":"success","sequence":"546","ses":"4","subj_category":"c0.c1023","subj_domain":"unconfined_t","subj_level":"s0-s0","subj_role":"unconfined_r","subj_user":"unconfined_u","terminal":"/dev/pts/1","uid":"0"}
---
{"@timestamp":"2016-12-07 02:22:14.304 +0000 UTC","a0":"7f683953a5d8","a1":"7f683953fd38","a2":"7f6839543a90","a3":"6","arch":"x86_64","auid":"1000","comm":"su","egid":"0","euid":"0","exe":"/usr/bin/su","exit":"0","fsgid":"0","fsuid":"0","gid":"0","items":"2","pid":"1445","ppid":"1444","raw_msg":"audit(1481077334.304:547): arch=c000003e syscall=59 success=yes exit=0 a0=7f683953a5d8 a1=7f683953fd38 a2=7f6839543a90 a3=6 items=2 ppid=1444 pid=1445 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=4 comm=\"su\" exe=\"/usr/bin/su\" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)","record_type":"SYSCALL","result":"success","sequence":"547","ses":"4","sgid":"0","subj_category":"c0.c1023","subj_domain":"unconfined_t","subj_level":"s0-s0","subj_role":"unconfined_r","subj_user":"unconfined_u","suid":"0","syscall":"execve","tty":"pts1","uid":"0"}
{"@timestamp":"2016-12-07 02:22:14.304 +0000 UTC","a0":"su","argc":"1","raw_msg":"audit(1481077334.304:547): argc=1 a0=\"su\"","record_type":"EXECVE","sequence":"547"}
{"@timestamp":"2016-12-07 02:22:14.304 +0000 UTC","cwd":"/home/andrew_kroh","raw_msg":"audit(1481077334.304:547):  cwd=\"/home/andrew_kroh\"","record_type":"CWD","sequence":"547"}
{"@timestamp":"2016-12-07 02:22:14.304 +0000 UTC","dev":"08:01","inode":"5026","item":"0","mode":"0104755","name":"/bin/su","obj_domain":"su_exec_t","obj_level":"s0","obj_role":"object_r","obj_user":"system_u","objtype":"NORMAL","ogid":"0","ouid":"0","raw_msg":"audit(1481077334.304:547): item=0 name=\"/bin/su\" inode=5026 dev=08:01 mode=0104755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:su_exec_t:s0 objtype=NORMAL","rdev":"00:00","record_type":"PATH","sequence":"547"}
{"@timestamp":"2016-12-07 02:22:14.304 +0000 UTC","dev":"08:01","inode":"16778495","item":"1","mode":"0100755","name":"/lib64/ld-linux-x86-64.so.2","obj_domain":"ld_so_t","obj_level":"s0","obj_role":"object_r","obj_user":"system_u","objtype":"NORMAL","ogid":"0","ouid":"0","raw_msg":"audit(1481077334.304:547): item=1 name=\"/lib64/ld-linux-x86-64.so.2\" inode=16778495 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0 objtype=NORMAL","rdev":"00:00","record_type":"PATH","sequence":"547"}

To normalize and interpret the messages, use the -i flag for "interpret". This adds a category to the event and creates the actor, action, thing, and how fields based on data from the event. By default it will resolve UID and GID values to their names (use -id=false to disable this).

$ sudo cat /var/log/audit/audit.log | auparse -format=yaml -i
---
timestamp: 2016-12-07T02:22:14.302Z
sequence: 545
category: user-login
record_type: cred_acq
result: success
session: "4"
summary:
  actor:
    primary: vagrant
    secondary: root
  action: acquired-credentials
  object:
    type: user-session
    primary: /dev/pts/1
  how: /usr/bin/sudo
user:
  ids:
    auid: "1000"
    uid: "0"
  names:
    auid: vagrant
    uid: root
  selinux:
    category: c0.c1023
    domain: unconfined_t
    level: s0-s0
    role: unconfined_r
    user: unconfined_u
process:
  pid: "1444"
  exe: /usr/bin/sudo
data:
  acct: root
  grantors: pam_env,pam_unix
  op: PAM:setcred
  terminal: /dev/pts/1
ecs:
  event:
    category:
    - authentication
    type:
    - info
  user:
    name: vagrant
    id: "1000"
    effective:
      name: root
      id: "0"
    target: {}
    changes: {}
  group: {}

---
timestamp: 2016-12-07T02:22:14.303Z
sequence: 546
category: user-login
record_type: user_start
result: success
session: "4"
summary:
  actor:
    primary: vagrant
    secondary: root
  action: started-session
  object:
    type: user-session
    primary: /dev/pts/1
  how: /usr/bin/sudo
user:
  ids:
    auid: "1000"
    uid: "0"
  names:
    auid: vagrant
    uid: root
  selinux:
    category: c0.c1023
    domain: unconfined_t
    level: s0-s0
    role: unconfined_r
    user: unconfined_u
process:
  pid: "1444"
  exe: /usr/bin/sudo
data:
  acct: root
  grantors: pam_keyinit,pam_limits
  op: PAM:session_open
  terminal: /dev/pts/1
ecs:
  event:
    category:
    - authentication
    type:
    - info
  user:
    name: vagrant
    id: "1000"
    effective:
      name: root
      id: "0"
    target: {}
    changes: {}
  group: {}

---
timestamp: 2016-12-07T02:22:14.304Z
sequence: 547
category: audit-rule
record_type: syscall
result: success
session: "4"
summary:
  actor:
    primary: vagrant
    secondary: root
  action: executed
  object:
    type: file
    primary: /bin/su
  how: /usr/bin/su
user:
  ids:
    auid: "1000"
    egid: "0"
    euid: "0"
    fsgid: "0"
    fsuid: "0"
    gid: "0"
    sgid: "0"
    suid: "0"
    uid: "0"
  names:
    auid: vagrant
    egid: root
    euid: root
    fsgid: root
    fsuid: root
    gid: root
    sgid: root
    suid: root
    uid: root
  selinux:
    category: c0.c1023
    domain: unconfined_t
    level: s0-s0
    role: unconfined_r
    user: unconfined_u
process:
  pid: "1445"
  ppid: "1444"
  name: su
  exe: /usr/bin/su
  cwd: /home/andrew_kroh
  args:
  - su
file:
  path: /bin/su
  device: "00:00"
  inode: "5026"
  mode: "0755"
  uid: "0"
  gid: "0"
  owner: root
  group: root
  selinux:
    domain: su_exec_t
    level: s0
    role: object_r
    user: system_u
data:
  a0: 7f683953a5d8
  a1: 7f683953fd38
  a2: 7f6839543a90
  a3: "6"
  arch: x86_64
  argc: "1"
  exit: "0"
  syscall: execve
  tty: pts1
paths:
- dev: "08:01"
  inode: "5026"
  item: "0"
  mode: "0104755"
  name: /bin/su
  obj_domain: su_exec_t
  obj_level: s0
  obj_role: object_r
  obj_user: system_u
  objtype: NORMAL
  ogid: "0"
  ouid: "0"
  rdev: "00:00"
- dev: "08:01"
  inode: "16778495"
  item: "1"
  mode: "0100755"
  name: /lib64/ld-linux-x86-64.so.2
  obj_domain: ld_so_t
  obj_level: s0
  obj_role: object_r
  obj_user: system_u
  objtype: NORMAL
  ogid: "0"
  ouid: "0"
  rdev: "00:00"
ecs:
  event:
    category:
    - process
    type:
    - start
  user:
    effective: {}
    target: {}
    changes: {}
  group: {}

ECS compatibility

This currently provides Elastic Common Schema (ECS) 1.8 categorization support for some of the more prominent or meaningful auditd events and syscalls.

go-libaudit's People

Contributors

adriansr avatar andrewkroh avatar andrewstucki avatar efd6 avatar florianl avatar ilyaglow avatar inge4pres avatar jlind23 avatar jongwooo avatar leehinman avatar leo-ri avatar maxenglander avatar mjwolf avatar moaddib666 avatar monicasarbu avatar mrodm avatar v1v avatar vitt-bagal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-libaudit's Issues

auparse -i doens't create multiple netfilter_cfg events as a list

By testing the auparse command I saw that it doesn't summarize the values correctly or as expected. I wanted to know if this is done by purpose or if this needs additional effort for the interpreter.

Here the example log:

type=NETFILTER_CFG msg=audit(1593786232.477:31): table=filter family=2 entries=0
type=NETFILTER_CFG msg=audit(1593786232.477:31): table=filter family=10 entries=0
type=SYSCALL msg=audit(1593786232.477:31): arch=c000003e syscall=272 success=yes exit=0 a0=40000000 a1=7fffd2436fb0 a2=40000040 a3=22 items=0 ppid=1 pid=769 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="(ostnamed)" exe="/usr/lib/systemd/systemd" subj=system_u:system_r:init_t:s0 key=(null)
type=PROCTITLE msg=audit(1593786232.477:31): proctitle="(ostnamed)"

Here the command
auparse -i -format yaml

And the result

timestamp: 2020-07-03T14:23:52.477Z
sequence: 31
category: configuration
record_type: netfilter_cfg
result: success
session: unset
summary:
  actor:
    primary: unset
    secondary: root
  action: loaded-firewall-rule-to
  object:
    type: firewall
    primary: filter
  how: /usr/lib/systemd/systemd
user:
  ids:
    auid: unset
    egid: "0"
    euid: "0"
    fsgid: "0"
    fsuid: "0"
    gid: "0"
    sgid: "0"
    suid: "0"
    uid: "0"
  names:
    egid: root
    euid: root
    fsgid: root
    fsuid: root
    gid: root
    sgid: root
    suid: root
    uid: root
  selinux:
    domain: init_t
    level: s0
    role: system_r
    user: system_u
process:
  pid: "769"
  ppid: "1"
  title: (ostnamed)
  name: (ostnamed)
  exe: /usr/lib/systemd/systemd
data:
  a0: "40000000"
  a1: 7fffd2436fb0
  a2: "40000040"
  a3: "22"
  arch: x86_64
  entries: "0"
  exit: "0"
  family: "2"
  syscall: unshare
  table: filter
  tty: (none)
ecs:
  event:
    category:
    - process
    type:
    - info

As you can see in the output for the section data, there is no second entry for the second event with "family 10". Shouldn't that be part of the output, similar to your example with the list entries for type: PATH?
Thank you and regards.

PATH name is not being hex decoded

The name field inside of PATH type messages is not being decoded from hex.

type=PATH msg=audit(1521758453.536:1428931): item=0 name=2F73686172652F67656E6572616C2F706174685F7265646163746564 inode=1442434 dev=fc:01 mode=042775 ouid=10067 ogid=7003 rdev=00:00 nametype=NORMAL

Parse more than one key

If a rule has more than one key this isn't parsed correctly by go-libaudit. The code should be updated to decode the hex value and split on 0x01.

-a always,exit -F arch=b64 -S execve,execveat -F key=exec -F key=64bit

type=SYSCALL msg=audit(1515619721.392:106081): arch=c000003e syscall=59 success=yes exit=0 a0=55df1048ec88 a1=55df1048ec38 a2=55df1048ec58 a3=1 items=2 ppid=27851 pid=27854 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="grep" exe="/bin/grep" key=65786563013634626974"

For a parsing example, see https://play.golang.org/p/Oo2N_XVgWYc.

Apply special handling to the "unset" auid

auid=4294967295 is the same as auid=-1, and it means that no login UID has been assigned. This is used in events like failed logins. I think it would be more clear if we just changed auid to "unset" in these cases.

Help aucoalesce.Event -> source fields is nil.

Hello, thanks for providing github.com/elastic/go-libaudit, it is very helpful to me!

However, I encountered a situation where the โ€˜sourceโ€™ information in โ€˜aucoalesce.Eventโ€™ was empty, which bothered me a lot.

  {
    "@timestamp": "2022-01-06T06:39:31.575Z",
    "sequence": 159878,
    "category": "audit-rule",
    "record_type": "syscall",
    "result": "success",
    "session": "unset",
    "tags": [
      "k99"
    ],
    "summary": {
      "actor": {
        "primary": "unset",
        "secondary": "root"
      },
      "action": "connected-to",
      "object": {
        "type": "socket",
        "primary": "::",
        "secondary": "22"
      },
      "how": "/usr/sbin/sshd"
    },
    "user": {
      "ids": {
        "egid": "0",
        "suid": "0",
        "euid": "0",
        "fsuid": "0",
        "gid": "0",
        "sgid": "0",
        "uid": "0",
        "fsgid": "0",
        "auid": "unset"
      },
      "names": {
        "fsuid": "root",
        "sgid": "root",
        "uid": "root",
        "fsgid": "root",
        "gid": "root",
        "egid": "root",
        "suid": "root",
        "euid": "root"
      },
      "selinux": {
        "user": "unconfined"
      }
    },
    "process": {
      "pid": "31071",
      "ppid": "27522",
      "title": "/usr/sbin/sshd -D -R",
      "name": "sshd",
      "exe": "/usr/sbin/sshd"
    },
    "destination": {
      "ip": "10.11.41.140",
      "port": "22"
    },
    // not "source": {}
    "network": {
      "direction": "egress"
    },
    "data": {
      "arch": "x86_64",
      "a3": "0",
      "syscall": "connect",
      "a0": "4",
      "socket_port": "22",
      "exit": "0",
      "a2": "1c",
      "a1": "55f514565950",
      "socket_addr": "::",
      "tty": "(none)",
      "socket_family": "ipv6"
    },
    "ecs": {
      "event": {
        "category": [
          "network"
        ],
        "type": [
          "connection",
          "start"
        ]
      },
      "user": {
        "effective": {},
        "target": {},
        "changes": {}
      },
      "group": {}
    }
  }

No support for setting enabled = 2, rules immutable

SetEnabled forces the value to be 1, however the audit spec permits the value of enabled to be set to 2, which prevents the audit rules from being changed. This is usually enabled by calling auditctl -e 2, or adding -e 2 to the audit rules on the system. Because the rule parser in this library, and the library itself have no support for making the rules immutable, auditbeat is unable to parse any ruleset that makes the rules immutable (elastic/beats#8352)

go-libaudit/audit.go

Lines 354 to 365 in db7d499

func (c *AuditClient) SetEnabled(enabled bool, wm WaitMode) error {
var e uint32
if enabled {
e = 1
}
status := AuditStatus{
Mask: AuditStatusEnabled,
Enabled: e,
}
return c.set(status, wm)
}

Allow asynchronous status requests

In order to perform status requests while the application is already running an event-receive loop (i.e. to query for lost packets), it is necessary to perform an async AUDIT_GET request so that the response can be processed in the event loop.

Create library to coalesce events

Create a utility to combine related message into a single event. Related messages have the same sequence number. Events can arrive out of order. And events can be dropped if the receiver isn't fast enough.

The library should be able to coalesce events even if they arrive out-of-order and it should provide notification of dropped events.

[BUG] auditd_manager file.mode does not capture the correct file mode

Issue

I opened this issue earlier at #integrations, so more context on this issue can be found issue #6525. The issue includes test data, and additional research. I got asked to open this issue here.

The issue:

I am working on a detection rule to detect the execution of SUID binaries, for which I wanted to use the auditd_manager file.mode field. I have one document with a SUID bit set, showing the following auditd.paths:

[
  {
    "item": "0",
    "nametype": "NORMAL",
    "ogid": "0",
    "cap_fi": "0",
    "cap_fp": "0",
    "cap_frootid": "0",
    "cap_fver": "0",
    "inode": "135805",
    "mode": "0102770",
    "dev": "08:01",
    "ouid": "0",
    "rdev": "00:00",
    "cap_fe": "0",
    "name": "/usr/bin/nmap"
  } ]

The document above with a mode of 2770 displays a mode of 0770 in the file.mode field, which would indicate a non SUID binary.

And I have another document that does not have a SUID bit set, showing the following auditd.paths :

[
  {
    "item": "0",
    "nametype": "NORMAL",
    "ogid": "0",
    "cap_fi": "0",
    "cap_fp": "0",
    "cap_frootid": "0",
    "cap_fver": "0",
    "mode": "0100755",
    "inode": "135805",
    "dev": "08:01",
    "ouid": "0",
    "rdev": "00:00",
    "cap_fe": "0",
    "name": "/usr/bin/nmap"
  } ]

This document with a mode of 0755 displays a mode of 0755 in the file.mode field, which would also indicate a non SUID binary.

The actual auditd.paths flattend object does have the correct value in there, so I think it should be possible have the file.mode field display the correct file mode.

It does seem to capture it correctly in both the original message and the auditd.paths. It would also be an option to add the suid, fsuid, sgid and fsgid fields.

apparmor messages not fully parsed

The fields from the apparmor AVC message are not being parsed out. We need to create a new test case from this data and fix the parser.

{
  "@timestamp": "2018-04-25T13:28:53.080Z",
  "@metadata": {
    "beat": "auditbeat",
    "type": "doc",
    "version": "6.2.4"
  },
  "process": {
    "pid": "5571",
    "ppid": "5552",
    "title": "metricbeat -e -system.hostfs=/hostfs -strict.perms=false",
    "name": "metricbeat",
    "exe": "/usr/share/metricbeat/metricbeat"
  },
  "auditd": {
    "result": "success",
    "session": "unset",
    "data": {
      "a0": "3",
      "arch": "x86_64",
      "a1": "c4206cb400",
      "a3": "0",
      "a2": "200",
      "exit": "193",
      "tty": "(none)",
      "syscall": "read"
    },
    "summary": {
      "actor": {
        "secondary": "ubuntu",
        "primary": "unset"
      },
      "how": "/usr/share/metricbeat/metricbeat"
    },
    "warnings": [
      "failed to parse message: failed to parse audit message",
      "failed to set subject primary using keys=[scontext] because they were not found",
      "failed to set object primary using keys=[tcontext] because they were not found"
    ],
    "messages": [
      "type=AVC msg=audit(1524662933.080:61207): apparmor=\"DENIED\" operation=\"ptrace\" profile=\"docker-default\" pid=5571 comm=\"metricbeat\" requested_mask=\"trace\" denied_mask=\"trace\" peer=\"unconfined\"",
      "type=SYSCALL msg=audit(1524662933.080:61207): arch=c000003e syscall=0 success=yes exit=193 a0=3 a1=c4206cb400 a2=200 a3=0 items=0 ppid=5552 pid=5571 auid=4294967295 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=(none) ses=4294967295 comm=\"metricbeat\" exe=\"/usr/share/metricbeat/metricbeat\" key=(null)",
      "type=PROCTITLE msg=audit(1524662933.080:61207): proctitle=6D657472696362656174002D65002D73797374656D2E686F737466733D2F686F73746673002D7374726963742E7065726D733D66616C7365"
    ],
    "sequence": 61207
  },
  "event": {
    "category": "mac-decision",
    "type": "avc",
    "action": "violated-selinux-policy",
    "module": "auditd"
  },
  "user": {
    "name_map": {
      "gid": "ubuntu",
      "sgid": "ubuntu",
      "suid": "ubuntu",
      "uid": "ubuntu",
      "egid": "ubuntu",
      "euid": "ubuntu",
      "fsgid": "ubuntu",
      "fsuid": "ubuntu"
    },
    "euid": "1000",
    "suid": "1000",
    "sgid": "1000",
    "gid": "1000",
    "egid": "1000",
    "fsgid": "1000",
    "uid": "1000",
    "fsuid": "1000",
    "auid": "unset"
  },
  "beat": {
    "name": "gce-ubuntu1604",
    "hostname": "gce-ubuntu1604",
    "version": "6.2.4"
  }
}

proctitle value is truncated

The proctitle value is usually hex encoded and can contain multiple null-terminated strings. But currently only the first null-terminated value is returned.

For example currently it returns /usr/sbin/ntpd, but the real value is /usr/sbin/ntpd -p /var/run/ntpd.pid -g -c /var/lib/ntp/ntp.conf.dhcp -u 112:116.

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -31,3 +31,3 @@
                            (string) (len=4) "ppid": (string) (len=1) "1",
                        -   (string) (len=9) "proctitle": (string) (len=14) "/usr/sbin/ntpd",
                        +   (string) (len=9) "proctitle": (string) (len=79) "/usr/sbin/ntpd -p /var/run/ntpd.pid -g -c /var/lib/ntp/ntp.conf.dhcp -u 112:116",
                            (string) (len=7) "syscall": (string) (len=8) "adjtimex",

Can you release a new version?

Hi, can you release a new version, beause I need the method to convert kernel rules to display format, but v0.3.0 does not haveใ€‚ Thank you very much!

Reassembler perfomance improvement

Hello!

I found, that on every new event reassembler sort sequenceNumSlice, which leads perfomance issues on high audit rate limits. I think it can be improved if make sort only if difference between new sequence and last:

l.seqs = append(l.seqs, seq)

if len(l.seqs) > 1 {
	lastSeq := l.seqs[len(l.seqs)-2]

	diff := abs(int64(seq) - int64(lastSeq))
	if diff > maxSortRange || diff < 0 {
		l.seqs.Sort()
	}
}

What do you think about such improvement?

aucoalesce: CoalesceMessages fails with "missing syscall message in compound event"

Hello,

While experimenting with this library on flatcar Linux, I noticed that the aucoalesce.CoalesceMessages function returns the following error when it encounters certain audit events: missing syscall message in compound event

If I supply the same events to the aureport tool (installed as a part of auditd package on Ubuntu), it appears to successfully parse those events - at least in the sense it does not produce error messages or exit with a non-zero status. It looks like commit 666ff1c introduced the formerly-mentioned error - but it does not reference any Linux audit documentation or code that contextualizes the check.

tl;dr - I am not sure if this check is required - or if the audit logs I happened to encounter are just broken. Any kind of guidance would be appreciated :)

Below are the three audit events and the aureport stderr/out for them. For reference, the --debug argument should result in error messages if any events cannot be parsed and -e generates a report about events:

Failure 1

root@x:~# cat avc-failure-audit.log
type=AVC msg=audit(1668179838.476:649407): avc:  denied  { search } for  pid=4059486 comm="cephcsi" name="crypto" dev="proc" ino=475090959 scontext=system_u:system_r:svirt_lxc_net_t:s0:c222,c955 tcontext=system_u:object_r:sysctl_crypto_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1668179838.476:649407): avc:  denied  { read } for  pid=4059486 comm="cephcsi" name="fips_enabled" dev="proc" ino=475090960 scontext=system_u:system_r:svirt_lxc_net_t:s0:c222,c955 tcontext=system_u:object_r:sysctl_crypto_t:s0 tclass=file permissive=1
type=AVC msg=audit(1668179838.476:649407): avc:  denied  { open } for  pid=4059486 comm="cephcsi" path="/proc/sys/crypto/fips_enabled" dev="proc" ino=475090960 scontext=system_u:system_r:svirt_lxc_net_t:s0:c222,c955 tcontext=system_u:object_r:sysctl_crypto_t:s0 tclass=file permissive=1
root@x:~# aureport -if avc-failure-audit.log --debug -e 

Event Report
===================================
# date time event type auid success
===================================
1. 11/11/2022 15:17:18 649407 AVC -2 no
root@x:~# echo $?
0

Failure 2

root@x:~# cat execve-uname-failure-audit.log
type=EXECVE msg=audit(1671230062.742:657491): argc=2 a0="uname" a1="-p"
type=CWD msg=audit(1671230062.742:657491): cwd="/root"
type=PATH msg=audit(1671230062.742:657491): item=0 name="/usr/bin/uname" inode=76040 dev=fe:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:unlabeled_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=PATH msg=audit(1671230062.742:657491): item=1 name="/lib64/ld-linux-x86-64.so.2" inode=98548 dev=fe:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:unlabeled_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=PROCTITLE msg=audit(1671230062.742:657491): proctitle=756E616D65002D70
root@x:~# aureport -if execve-uname-failure-audit.log --debug -e

Event Report
===================================
# date time event type auid success
===================================
1. 12/16/2022 22:34:22 657491 EXECVE -2 unset
root@x:~# echo $?
0

Failure 3

root@x:~# cat execve-ethtool-failure-audit.log
type=EXECVE msg=audit(1671230063.745:657579): argc=3 a0="/usr/sbin/ethtool" a1="-T" a2="lxc61be96845005"
type=CWD msg=audit(1671230063.745:657579): cwd="/root"
type=PATH msg=audit(1671230063.745:657579): item=0 name="/usr/sbin/ethtool" inode=162594 dev=fe:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:unlabeled_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=PATH msg=audit(1671230063.745:657579): item=1 name="/lib64/ld-linux-x86-64.so.2" inode=98548 dev=fe:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:unlabeled_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=PROCTITLE msg=audit(1671230063.745:657579): proctitle=2F7573722F7362696E2F657468746F6F6C002D54006C7863363162653936383435303035
root@x:~# aureport -if execve-ethtool-failure-audit.log --debug -e

Event Report
===================================
# date time event type auid success
===================================
1. 12/16/2022 22:34:23 657579 EXECVE -2 unset
root@x:~# echo $?
0

Other notes

I commented out the code that returns the error and tried running the project's Go tests with go test ./.... The tests ran successfully. Based on the test result and the error being generated with errors.New - it appears to be both untested and not programmatically checkable (e.g., errors.As).

Event where event.action is an empty string

Here is an event where event.action is an empty string when running this agg on servers:

GET /*/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "event.action": [
              ""
            ]
          }
        },
        {
          "terms": {
            "event.module": [
              "auditd"
            ]
          }
        }
      ]
    }
  },
  "aggs": {
    "module": {
      "terms": {
        "field": "process.title",
        "size": 1000
      }
    }
  }
}

Answers coming back for process title are:

{
  "took" : 552,
  "timed_out" : false,
  "_shards" : {
    "total" : 389,
    "successful" : 389,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "module" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/usr/bin/dpkg --status-fd 10 --no-triggers --unpack --auto-deconfigure /var/cache/apt/archives/openssh-sftp-server_1%3a7.4p1-10+",
          "doc_count" : 9
        },
        {
          "key" : "/usr/bin/dpkg --status-fd 23 --no-triggers --unpack --auto-deconfigure /var/cache/apt/archives/unzip_6.0-21_amd64.deb",
          "doc_count" : 1
        }
      ]
    }
  }
}

Create a stress test

There should be a stress test framework to find the limits of the library with and without the parser in the loop.

failed to set audit PID: file exists

I have been using go-libaudit/v2 to monitor network and file events on GitHub Actions runners.

The project is at https://github.com/step-security/harden-runner and https://github.com/step-security/agent and the code that uses go-libaudit/v2 is here: https://github.com/step-security/agent/blob/9d3f70b3123baaf6b4b2372ca7ba7109c0aebfaf/procmon_linux.go#L19

The code was working fine until a day back when GitHub Actions Ubuntu 20.04.4 VM was updated to use kernel 5.15.

Now the receive method at https://github.com/step-security/agent/blob/9d3f70b3123baaf6b4b2372ca7ba7109c0aebfaf/procmon_linux.go#L113 does not receive any events. The MonitorProcesses method runs without any errors, just no events are received after it.

Interestingly, using the Ubuntu 22.04 VM with kernel 5.15 works. This table shows the environments where the code works and where it doesn't work

Kernel Ubuntu Version Works?
5.13 20.04.4 Yes
5.15 20.04.4 No
5.15 22.04 Yes

Please let me know if you have ideas on how I can troubleshoot and fix this issue?

aucoalesce: Make warning capturing configurable

The current interface to aucoalesce is func CoalesceMessages(msgs []*auparse.AuditMessage) (*Event, error).

Given that the warnings produced by this function are mainly for debugging and development we could change the function to accept a variable number of options like func CoalesceMessages(msgs []*auparse.AuditMessage, opts ...Option) (*Event, error). Then we could pass an option to enable the inclusion of warnings with the event. Like

    evt, err := aucoalesce.CoalesceMessages(msgs, aucoalesce.IncludeWarnings)

Future options might be to make ECS enrichment configurable.

rule - File paths with spaces fail in rules

The following rules all resulted in errors:

-w /tmp/folder with space -p r -k test1
-w "/tmp/folder with space" -p r -k test2
-w '/tmp/folder with space' -p r -k test3
-w /tmp/folder\ with\ space -p r -k test4
-w "/tmp/folder\ with\ space" -p r -k test5
-w '/tmp/folder\ with\ space' -p r -k test6

This was logged downstream in Auditbeat. elastic/beats#27196

[rule] Add support for saddr_fam filters

Add support for filtering events based on saddr_fam. This is the address family number as found in /usr/include/bits/socket.h. For example, IPv4 would be 2 and IPv6 would be 10.

-a always,exit -F arch=b64 -S connect,accept,listen,bind -F a2=16 -F saddr_fam=2 -F success=1 -F key=network-v4
-a always,exit -F arch=b64 -S connect,accept,listen,bind -F a2=24 -F saddr_fam=10 -F success=1 -F key=network-v6

linux-audit/audit-kernel@61fc577
linux-audit/audit-userspace@fef4978

Fix dash separation around log-in (should be login)

From the file:
https://github.com/elastic/go-libaudit/blob/master/aucoalesce/normalizations.yaml

Fix dash separation of

failed-log-in-too-many-times-to
attempted-log-in-from-unusual-place-to
attempted-log-in-during-unusual-hour-to

To be:

failed-login-too-many-times-to
attempted-login-from-unusual-place-to
attempted-login-during-unusual-hour-to

Change past tense:

crypto-officer-logged-in
crypto-officer-logged-out
logged-out

to present tense:

crypto-officer-login
crypto-officer-logout
logout

final one to change would be when the event.action is:

logged-in

it would be better if that could change to

login

But don't see that one in the above normalization file.

Reason being four fold.

  • Users are more likely going to type login and not log-in such as event.category: user-login
  • Consistency. Within the file it has changed-login-id-to in which login is one word and we have the word login everywhere else such as event.category. Otherwise users are using login, log-in, and logged-in
  • It is grammar correct since Login (one word) can be a noun or an adjective and in our own docs for this file we state action - what happened and from the grammar above it looks like it is being treated as a noun rather an a verb.
  • It is weird to see an event action in past tense such as logged-in when we get a result of false indicating that they did not login, they attempted a login.

Add support for AUDIT_TIME_ADJNTPVAL

There are several new message types that should be handled in go-libaudit at:

AUDIT_FANOTIFY AuditMessageType = 1331

Additionally it appears that one of the assumptions within aucoalesce is broken. It assumes that only type=SYSCALL messages can have related messages with the same record ID. But the follow kernel code sends multiple message related to the NTP time adjustements (AUDIT_TIME_ADJNTPVAL).

https://elixir.bootlin.com/linux/v5.2/source/kernel/auditsc.c#L2534

When go-libaudit receives these messages it fails. See this test case.

diff --git a/aucoalesce/testdata/random-internet.yaml b/aucoalesce/testdata/random-internet.yaml
index d86b910..3b3d1cd 100644
--- a/aucoalesce/testdata/random-internet.yaml
+++ b/aucoalesce/testdata/random-internet.yaml
@@ -2,3 +2,8 @@
 tests:
   ANOM_ABEND SIGSEGV seg fault: |
     type=ANOM_ABEND msg=audit(1423234994.398:911150): auid=4294967295 uid=48 gid=48 ses=4294967295 subj=system_u:system_r:httpd_t:s0 pid=31242 comm="httpd" reason="memory violation" sig=11
+
+  AUDIT_TIME_ADJNTPVAL unknown type: |
+    type=UNKNOWN[1333] msg=audit(1642008700.569:2424985): op=offset old=-120656636899 new=-49254684950
+    type=UNKNOWN[1333] msg=audit(1642008700.569:2424985): op=freq old=190270475413120 new=190270282200256
+    type=UNKNOWN[1333] msg=audit(1642008700.569:2424985): op=status old=24577 new=8193
\ No newline at end of file
diff --git a/auparse/testdata/test3.log b/auparse/testdata/test3.log
index 99a4d31..83e585a 100644
--- a/auparse/testdata/test3.log
+++ b/auparse/testdata/test3.log
@@ -14,4 +14,4 @@ type=PROCTITLE msg=audit(1451781471.394:194438): proctitle="bash"
 type=PROCTITLE msg=audit(1451781471.394:194440): proctitle=737368643A206275726E205B707269765D

This fails with:

$ go test .
--- FAIL: TestCoalesceMessages (0.01s)
    --- FAIL: TestCoalesceMessages/testdata/random-internet.yaml (0.00s)
        coalesce_test.go:79: missing syscall message in compound event

Help .... When I add one audit rule I got Error .

Add Rule Code:

oneRule := "-a exit,always -F auid!=-1 -F arch=b64 -S execve -S bind -k wildcat "
ruleExec, err := flags.Parse(oneRule)
if err != nil {
    return errors.Wrap(err, "failed to build audit rule")
}
binaryRule, err := rule.Build(ruleExec)
if err := client.AddRule(binaryRule); err != nil {
   log.Warnf("add audit rule %+v", err)
}

Error Shows :

add audit rule unexpected sequence number for reply (expected 3 but got 2)
bbaudit/vendor/github.com/elastic/go-libaudit.(*AuditClient).getReply
	/Users/xingyue/go/src/bbaudit/vendor/github.com/elastic/go-libaudit/audit.go:488
bbaudit/vendor/github.com/elastic/go-libaudit.(*AuditClient).AddRule
	/Users/xingyue/go/src/bbaudit/vendor/github.com/elastic/go-libaudit/audit.go:290
main.initAudit
	/Users/xingyue/go/src/bbaudit/bbaudit.go:77
main.InitAuditPlugin
	/Users/xingyue/go/src/bbaudit/bbaudit.go:166
main.main
	/Users/xingyue/go/src/bbaudit/bbaudit.go:217
runtime.main
	/usr/local/go/src/runtime/proc.go:200
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337
failed to get ACK to rule delete request
bbaudit/vendor/github.com/elastic/go-libaudit.(*AuditClient).AddRule
	/Users/xingyue/go/src/bbaudit/vendor/github.com/elastic/go-libaudit/audit.go:292
main.initAudit
	/Users/xingyue/go/src/bbaudit/bbaudit.go:77
main.InitAuditPlugin
	/Users/xingyue/go/src/bbaudit/bbaudit.go:166
main.main
	/Users/xingyue/go/src/bbaudit/bbaudit.go:217
runtime.main
	/usr/local/go/src/runtime/proc.go:200
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1337 

"protocol not supported" error when CONFIG_AUDIT is disabled

libaudit.NewAuditClient() returns EPROTONOSUPPORT (protocol not supported) if the kernel does not have audit supported compiled in (CONFIG_AUDIT=y). For example, CONFIG_AUDIT is disabled in the Arch Linux kernel packages so a custom kernel is required to use auditing, and anyone who tries to use this library will see "protocol not supported" errors.

When creating the netlink audit socket we can wrap the EPROTONOSUPPORT error in a more informative message to help users.

Consistently use audit rather than auditd in the actions

Within the file:
https://github.com/elastic/go-libaudit/blob/master/aucoalesce/normalizations.yaml

It has these definitions which have the word auditd rather than audit. This should all be consistent for users of the system so they don't have to think about if they have to type in audit vs auditd

Change:

aborted-auditd-startup
changed-auditd-configuration
reconfigured-auditd

to be:

aborted-audit-startup
changed-audit-configuration
reconfigured-audit

Reason being in all those other places such as started-audit, rotated-audit-logs, initialized-audit-subsystem it has the word audit and not auditd in them.

Support of < 3.0 kernel

We need to use outdated kernel due to vendor lock, and binary response of such kernel on get audit status is smaller than expected one, please check my pr for details. #119

execveCmdLine improperly hex-decodes values which are not hex encoded

Given the following input:
type=EXECVE msg=audit(1507304439.922:1865): argc=5 a0="useradd" a1="-u" a2="3005" a3="-m" a4="trainer5"

The execveCmdLine func decodes a2 as 0\x05 which is invalid: 3005 is, in fact the value that should be returned.

execveCmdLine should check to see if the returned value is in the printable range or in the following character classes: digit, alpha, punct, or space

Add support for out-of-order audit messages

I've made a custom audit.Assembler to handle auditd messages at the end of a log pipeline that doesn't guarantee ordered delivery. This seems like a generally useful thing, so would you accept some PRs to get this added in?

I think this would mean refactoring libaudit.Reassembler to an interface and implementing a UnorderedReassembler struct that emits a slice of messages when it gets a complete set. FWIW, our NewUnorderedReassembler() has the same args as NewReassembler() and is used basically the same way.

I promise to also commit a bunch of tests at the same time. :)

Add ability to set backlog wait time

Add the ability to set audit_backlog_wait_time and detect if the kernel supports it (based on the audit_status version or feature bitmap). The feature was added to linux v3.14.

One use case is to be able to set it to 0 to ensure that nothing ever blocks in the kernel.

Add a tuning option to audit_backlog_wait_time so that if auditd can't keep up,
or gets blocked, the callers won't be blocked.

auditbeat ERROR: get status request failed:failed to get audit status reply: no reply received

original issue: elastic/beats#33258

long story short: we run auditbeat as DaemonSet on GKE clusters with slightly different versions, some nodes run docker, other nodes run containerd.

it runs with all permissions it needs, journald already unregistered by an initContainer so auditbeat can get audit events.
Problem is that some random auditbeat pods keep outputting this error until we restart them:

ERROR: get status request failed:failed to get audit status reply: no reply received

and if we restart a totally fine auditbeat pod, it might start outputting that error too.

it doesn't however stop writing audit logs to elasticsearch. we get audit logs from the pods that are outputting the error as much as the other pods.

I traced down the error to this block of code:

go-libaudit/audit.go

Lines 496 to 498 in 6fba496

if len(msgs) == 0 {
return nil, errors.New("no reply received")
}

Wouldn't it be okay if msgs was empty? At this point we already got through this without any error:

go-libaudit/audit.go

Lines 480 to 494 in 6fba496

for i := 0; i < 10; i++ {
msgs, err = c.Netlink.Receive(true, parseNetlinkAuditMessage)
if err != nil {
switch {
case errors.Is(err, syscall.EINTR):
continue
case errors.Is(err, syscall.EAGAIN):
time.Sleep(50 * time.Millisecond)
continue
default:
return nil, fmt.Errorf("error receiving audit reply: %w", err)
}
}
break
}

and func (c *NetlinkClient) Receive() already got the appropriate error checks here:

go-libaudit/netlink.go

Lines 152 to 190 in 6fba496

func (c *NetlinkClient) Receive(nonBlocking bool, p NetlinkParser) ([]syscall.NetlinkMessage, error) {
var flags int
if nonBlocking {
flags |= syscall.MSG_DONTWAIT
}
// XXX (akroh): A possible enhancement is to use the MSG_PEEK flag to
// check the message size and increase the buffer size to handle it all.
nr, from, err := syscall.Recvfrom(c.fd, c.readBuf, flags)
if err != nil {
// EAGAIN or EWOULDBLOCK will be returned for non-blocking reads where
// the read would normally have blocked.
return nil, err
}
if nr < syscall.NLMSG_HDRLEN {
return nil, fmt.Errorf("not enough bytes (%v) received to form a netlink header", nr)
}
fromNetlink, ok := from.(*syscall.SockaddrNetlink)
if !ok || fromNetlink.Pid != 0 {
// Spoofed packet received on audit netlink socket.
return nil, errors.New("message received was not from the kernel")
}
buf := c.readBuf[:nr]
// Dump raw data for inspection purposes.
if c.respWriter != nil {
if _, err = c.respWriter.Write(buf); err != nil {
return nil, err
}
}
msgs, err := p(buf)
if err != nil {
return nil, fmt.Errorf("failed to parse netlink messages (bytes_received=%v): %w", nr, err)
}
return msgs, nil
}

Shouldn't len(msgs) == 0 be reported as a warning instead of an error?

Add the ability to enrich events with UID and GID names

The library should be able to enrich events with UID and GID information. Ideally it would work on events irregardless of whether or not they have been parsed with auparse. It could just do a simple scan looking for uid= and gid= and do the lookups.

We should test the performance impact of UID/GID lookup and possibly add a cache.

cwd field is not hex decoded

The following audit event (as logged by systemd-journald):

Jun 18 00:39:21 xyz audit[2482]: USER_CMD pid=2482 uid=0 auid=0 ses=5 msg='cwd=2F746D702F6120622063 cmd="ls" terminal=pts/1 res=success'

gets logged by Metricbeat as follows:

{
  "_index": "metricbeat-6.0.0-alpha2-2017.06.17",
  "_type": "doc",
  "_id": "AVy4NfMKOMPbjbboyXKQ",
  "_score": 1,
  "_source": {
    "@timestamp": "2017-06-17T22:39:21.629Z",
    "audit": {
      "kernel": {
        "action": "ran-command",
        "actor": {
          "attrs": {
            "auid": "root",
            "uid": "root"
          },
          "primary": "root",
          "secondary": "root"
        },
        "category": "user-space",
        "data": {
          "cmd": "ls",
          "cwd": "2F746D702F6120622063",
          "pid": "2482",
          "terminal": "pts/1"
        },
        "record_type": "user_cmd",
        "result": "success",
        "sequence": 121,
        "session": "5",
        "thing": {
          "primary": "ls",
          "what": "process"
        }
      }
    },
    "beat": {
      "hostname": "xyz",
      "name": "xyz",
      "version": "6.0.0-alpha2"
    },
    "metricset": {
      "module": "audit",
      "name": "kernel"
    }
  },
  "fields": {
    "@timestamp": [
      "2017-06-17T22:39:21.629Z"
    ]
  }
}

In the case above, the cwd contains spaces (/tmp/a b c) which causes it to be hex encoded.
Apparently there is no code to hex decode the cwd field, so the field appears in Elasticsearch in its escaped form. This is against expectations.

If I'm not mistaken, any field will be hex encoded by the audit subsystem if it contains spaces or other special characters, so it might be worthwhile to implement hex decoding for more fields.

Remove usage of github.com/pkg/errors

github.com/pkg/errors is no longer maintained and has been archived. This library should no longer depend on that package. If required the minimum version of Go can be incremented as well.

A secondary benefit will be performance. The warnings in aucoalesce take a lot of time due to the stack trace capturing. On a related note I'm proposing to make warnings configurable in #100.

func BenchmarkCoalesceMessages(b *testing.B) {
	events := readEventsFromYAML(b, "testdata/rhel-7-linux-3.10.0.yaml")

	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, err := CoalesceMessages(events[i%len(events)].messages)
		if err != nil {
			b.Fatal(err)
		}
	}
}
$ benchcmp before.txt after.txt
benchmark old ns/op new ns/op delta
BenchmarkCoalesceMessages-12 4956 3444 -30.51%

benchmark old allocs new allocs delta
BenchmarkCoalesceMessages-12 17 14 -17.65%

benchmark old bytes new bytes delta
BenchmarkCoalesceMessages-12 2943 2487 -15.49%

Normalize success and failure field and value

Success and failure indicators are inconsistent in events and we should normalize them. Both res and success fields are used and their values vary (success, failed, yes, no, 0, 1, granted, denied, unknown). There are possibly other fields too.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.