linux-audit / audit-testsuite Goto Github PK

Add tests to for event filtering based on inode numbering; for obvious reasons this should also include the device major/minor filtering.

The lost_reset test continues to be the most fragile test in this test suite. What was once happening occasionally appears to happening with more frequency. Example test failure:

Running as   user    root
        with context unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
        on   system  Fedora

backlog_wait_time_actual_reset/test .. ok
exec_execve/test ..................... ok
exec_name/test ....................... ok
file_create/test ..................... ok
file_delete/test ..................... ok
file_rename/test ..................... ok
filter_exclude/test .................. ok
filter_saddr_fam/test ................ ok
filter_sessionid/test ................ ok
login_tty/test ....................... ok
# Test 4 got: "0" (lost_reset/test at line 166)
#   Expected: "1"
#  lost_reset/test line 166 is: ok( $found_msg, 1 );              # Was the message well-formed?
# Failed test 5 in lost_reset/test at line 167
#  lost_reset/test line 167 is: ok( $reset_rc == $reset_msg );    # Do the two lost values agree?
lost_reset/test ...................... 
Failed 2/5 subtests 
netfilter_pkt/test ................... ok
syscalls_file/test ................... ok
syscall_module/test .................. ok
time_change/test ..................... ok
user_msg/test ........................ ok
fanotify/test ........................ ok
bpf/test ............................. ok

Test Summary Report
-------------------
lost_reset/test                    (Wstat: 0 Tests: 5 Failed: 2)
  Failed tests:  4-5
Files=18, Tests=202, 53 wallclock secs ( 0.17 usr  0.04 sys +  8.67 cusr  3.32 csys = 12.20 CPU)
Result: FAIL
Failed 1/18 test programs. 2/202 subtests failed.

We need to attempt a test run on a RHEL-6 system.

system("auditctl -a always,exit -S all -F exe=$exec -k $key");
-F unknown field: exe

Maybe should system("auditctl -a always,exit -S all -F path=$exec -k $key"); ??

Add tests to filter on different UID/GID values, the list currently appears to be:

• auid
• uid
• euid
• suid
• fsuid
• obj_uid
• gid
• egid
• sgid
• fsgid
• obj_gid

A recent audit-userspace change (linux-audit/audit-userspace@958b1ce) has made the test suite output very noisy, for example:

% make test
make -C tests test
...
chmod +x */test
Running as   user    root
        with context unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
        on   system  Fedora

backlog_wait_time_actual_reset/test .. 1/5 q_depth should be larger than 512 for safety margin
backlog_wait_time_actual_reset/test .. ok   
exec_execve/test ..................... q_depth should be larger than 512 for safety margin
exec_execve/test ..................... ok
...

Test the ability to filter events based on PID and PPID.

Add tests to filter based on syscall argument values; this may be difficult due to platform differences.

On Ubuntu 16.04, the instructions in README.md lead to:

cc -g -O0 -Wall -D_GNU_SOURCE -m32 -o conn conn.c
In file included from /usr/include/unistd.h:25:0,
                 from conn.c:1:
/usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or directory
compilation terminated.
Makefile:8: recipe for target 'conn' failed

Fixed with:

diff --git a/README.md b/README.md
index 625a684..c6124c7 100644
--- a/README.md
+++ b/README.md
@@ -53,6 +53,7 @@ please follow the instructions below.
        # apt-get install auditd \
                          build-essential \
                          libc6-i386 \
+                         libc6-dev-i386 \
                          perl-modules \
                          netcat

For a first attempt it should be sufficient to audit a single syscall and verify things are working correctly, we can grow the list over time if necessary. The important part will be to ensure that the chosen syscall is supported on all platforms, e.g. open(...).

It would be good to have a test for at least basic event validation. Sometimes events are malformed even though there is a guidance for creating audit events. Since we have fields dictionary and quire clear requirements for events now it might be possible to create a test detecting malformed events.

We should create a test to ensure that audit records are written to the kernel ring buffer when the audit daemon is not connected and the hold buffer starts to overflow.

Test for the presence of 32-bit socketcall arguments on 64-bit systems as appears to be missing in linux-audit/audit-kernel#14

Note: This would be more useful if it tested the presence of correctly formed socketcall args in all systems.

Commit c69e8d9c01db ("CRED: Use RCU to access another task's creds and to
release a task's own creds") introduced a cred struct memory leak when the second of the three return conditions in the audit_filter_rules() didn't call put_cred() to match the initial get_task_cred(). This second exit condition tests if a subsequent rule has a higher priority and if it does not it simply exits, but should first call put_cred().

Test for this condition by looking for a matching number of cred_jar entries in the slab.

Reuse the key generation code from the other tests.

See the thread below for the test, and a comment from @rgbriggs:

• https://www.redhat.com/archives/linux-audit/2018-September/msg00012.html

Several tests have embedded references to 64-bit architecture rules when they may be run on 32-bit architecture machines. Convert them from "-F arch=b64" to "-F arch=b$ENV{MODE}".

tests/filter_sessionid/test:
"auditctl -a always,exit -F arch=b64 -F path=/tmp/$key -F sessionid=$sessionid -k $key"
tests/syscall_module/test:
"auditctl -a always,exit -F arch=b64 -S init_module -S finit_module -k $key-load"
tests/syscall_module/test:
system("auditctl -a always,exit -F arch=b64 -S delete_module -k $key-unload");
tests_manual/syscall_module_path_filter/setup:
echo "-a always,exit -F arch=b64 -S init_module -S finit_module -k ghak8-mod-load" ...

Some recent systems appear to be so robust that provoking lost messages is proving difficult if not impossible. The offending commit appears to be:
5a6f30a ("lost_reset: spin until msg lost")

@pmoore's method of deleting the "-F pid=$ping_pid" argument to the "auditctl -a ..." line of tests/lost_reset/test succeeds on these stubborn systems to trigger lost messages, but is pretty saddistic on any kernel, worse on older ones.

If we attempt to run the multiple instances of the testsuite simultaneously we hit a number of failures. While this was never an explicit goal of the testsuite, it can be handy at times for use as a crude stress test for the audit system (lock verification, etc.). There is nothing inherent in the design of the tests that should prevent multiple simultaneous instances so it would be good to understand why this is a problem in practice.

Lately I've been seeing sporadic test failures in the lost_reset test. I haven't had an opportunity to debug these failures to the level of determining the root cause, so I'm creating this issue to track the history/progress of the bug.

@The-Mule I believe you have been seeing these failures too in some automated test runs, any chance you have looked into this at all?

@rgbriggs This test has been pretty reliable for me up until recently, have you noticed any failures?

Create a mechanism to toggle a global test debug flag which tests can use to decide if any additional output should be displayed. I imagine an environment variable is probably the easiest approach, but I'm open to suggestions.

From an email with @rgbriggs:

We have 3 file_* tests in the test suite. There are 6 operations that
need testing. There are tests required when the file exists and when it
does not. That makes 12 tests (which could be 6 compound tests, or even
all covered in one test).

Test for kernel audit container id functionality:

• prohibit unsetting
• prohibit self-setting
• prohibit setting again
• prohibit without CAP_AUDIT_CONTROL
• verify AUDIT_CONTAINER record
• verify auditctl containerid filter
• verify kernel AUDIT_CONTAINERID filter functionality
• verify AUDIT_CONTAINER_INFO record

See: linux-audit/audit-kernel#32
See: linux-audit/audit-kernel#90
See: linux-audit/audit-kernel#91
See: linux-audit/audit-kernel#92
See: linux-audit/audit-userspace#40
See: https://github.com/linux-audit/audit-kernel/wiki/RFE-Audit-Container-ID

Test the ability to filter events based on file, dir, socket, link, char, block, and fifo files.

Create tests to filter based on the associated device. The use of loopback devices could be helpful.

SElinux context is parsed in filter_exclude test incorrectly:

if($subj =~ /([^:]+):([^:]+):([^:]+):([^-]+)-([^-]+)/) {
        ($subj_user, $subj_role, $subj_type, $subj_sen, $subj_clr) = ($1, $2, $3, $4, $5);
}

When there is no clearance (eg. system_u:unconfined_r:unconfined_t:s0 [*]) parsing will not work. The following regex works: ([^:]+):([^:]+):([^:]+):([^-]+)-?([^-]+)?.

Please notice that clearance is used in several places of the test and hence we need to fix those parts as well.

[*] FYI, when I am running testsuite via Beaker, tests are executed with this context.

Reproducer: runcon system_u:unconfined_r:unconfined_t:s0 make tests

Compilation of tests/syscall_socketcall fails with the following error:

make[2]: Entering directory '/mnt/tests/CoreOS/audit/Sanity/audit-rules-smoke/audit-testsuite/tests/syscall_socketcall'
cc -g -O0 -Wall -D_GNU_SOURCE -m32 -o conn conn.c
In file included from /usr/include/features.h:527,
                 from /usr/include/unistd.h:25,
                 from conn.c:1:
/usr/include/gnu/stubs.h:6:11: fatal error: gnu/stubs-32.h: No such file or directory
    6 | # include <gnu/stubs-32.h>
      |           ^~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:8: conn] Error 1
make[2]: Leaving directory '/mnt/tests/CoreOS/audit/Sanity/audit-rules-smoke/audit-testsuite/tests/syscall_socketcall'

In gnu/stubs.h, I have:

/* This file is automatically generated.
   This file selects the right generated file of `__stub_FUNCTION' macros
   based on the architecture being compiled for.  */

#if !defined __x86_64__
# include <gnu/stubs-32.h>
#endif
#if defined __x86_64__ && defined __LP64__
# include <gnu/stubs-64.h>
#endif
#if defined __x86_64__ && defined __ILP32__
# include <gnu/stubs-x32.h>
#endif

The <gnu/stubs-32.h> lib is included if __x86_64__ is not defined. However, this is not available. The reason why x86_64 is undefined, is because we are passing -m32 to the compiler in https://github.com/linux-audit/audit-testsuite/blob/main/tests/syscall_socketcall/Makefile#L8 .

Removing -m32 helps with the compilation, but I think it's there for a reason. Introduced in e276b30 . Any suggestion is appreciated.

Test the ability to filter events based on the different SELinux attributes for both subjects and objects.

Add tests to filter events based on file permissions.

Linux 4.3 will have the ability to audit by executable name, we should add tests for this.

Test the exclusion filters; at first we don't need a full set of "exclude" filters, but we may want to grow this over time.

Based on its settings fanotify subsystem can produce type=FANOTIFY events. We need a test that audit records are created only when it is needed and that they are associated with other records (PROCTITLE, SYSCALL, CWD, PATH).

There are tests that exist for issues (bugs) that are known but for which there does not yet exist a solution (fix, mitigation, patch).

How do we accommodate those tests into the repo while allowing the testsuite to pass?

Potential solutions:

• I originally wanted to have them in their intended target location ("tests") but simply omit the test name from "tests/Makefile" so it gets skipped over, a bit like we do for different vintages of kernels or distros.

• Having them in a separate branch and merging them when a fix is upstream leaves them out of the way, but makes things inconvenient for developers switching back and forth and adding more branches than necessary, one per test.

• They could go in a directory similar to "tests-manual" called "tests-pending" and when ready its sub-directory gets "git mv" into "tests" and update "tests/Makefile".

• Don't merge the test at all until the fix is upstream.

I'd prefer the first, third, then second, fourth, in that order.

The testsuite fails on RHEL7, likely due to changes between RHEL7 and upstream. Older RHEL releases are likely also affected.

Add a test to verify basic functionality of the ptrace and signal of the audit daemon.

When ptrace is run on a task, verify an AUDIT_OBJ_PID record is recorded on syscall exit of the ptrace task.

When a signal (TERM|HUP|USR1|USR2) is delivered to the audit daemon, test that a request for AUDIT_SIGNAL_INFO is correctly returned.

When signal logging is configured in audit, verify all signals delivered to a task are recorded on syscall exit of that target task.

So like "auditctl -d exclude,always -F pid=$pid" will fail.
I have use repo http://mirror.centos.org/centos/7/updates/x86_64/Packages/ yum update to audit-2.6.5.

Am I need update kernel from rhel7.2 to rhel7.3 ?

Thank you~

We don't want to include stress tests and/or potentially destructive tests in the standard audit-testsuite, but this doesn't mean we don't want to track these tests for occasional, manual testing. This issue has been created to discuss how best to handle these tests.

We should have a SUBMITTING_PATCHES file which documents how to submit patches to this repository.

An example file from libseccomp:

• https://github.com/seccomp/libseccomp/blob/master/SUBMITTING_PATCHES

The bpf tests fail on aarch64 because the test's bpf program fails with EPERM, even run as root on a system with SELinux in permissive mode. The failure does not occur on a similarly configured x86_64 system.
Here is a strace of the bpf program on a Fedora Rawhide aarch64 system:

# strace ./bpf
execve("./bpf", ["./bpf"], 0xffffd43251e0 /* 28 vars */) = 0
brk(NULL)                               = 0xeb27000
faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=35846, ...}) = 0
mmap(NULL, 35846, PROT_READ, MAP_PRIVATE, 3, 0) = 0xffffb37a9000
close(3)                                = 0
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\20C\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=3036328, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffffb37a7000
mmap(NULL, 1530744, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffffb35ff000
mprotect(0xffffb375a000, 73728, PROT_NONE) = 0
mmap(0xffffb376c000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15d000) = 0xffffb376c000
mmap(0xffffb3772000, 11128, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffffb3772000
close(3)                                = 0
mprotect(0xffffb376c000, 12288, PROT_READ) = 0
mprotect(0x41f000, 4096, PROT_READ)     = 0
mprotect(0xffffb37b4000, 4096, PROT_READ) = 0
munmap(0xffffb37a9000, 35846)           = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0xffffe2d64bc0, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0}, 120) = -1 EPERM (Operation not permitted)
exit_group(-1)                          = ?
+++ exited with 255 +++

One of the goals for this testsuite was to be able to use individual tests as reproducers in bug reports (here and on the various distribution bug trackers). While it is possible to simply point bug assignees to the entire testsuite, it is sometimes very helpful to include an individual test script in the bug tracker. However, in order to include the test script, the test script must have minimal dependencies on the audit-testsuite test infrastructure; this includes the use of $ENV{...} in the scripts.

Currently affected tests include:

# grep -l "ENV" $(find . -name "test")
./tests/filter_exclude/test
./tests/syscalls_file/test
./tests/exec_execve/test

In each of these cases it is the MODE variable which is referenced in the test scripts. We should investigate the possibility of doing the arch/ABI detection in the scripts themselves.

Different from filesystem watches, use the directory as an event filter.

filter_exclude test expected an selinux context of the form:
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
while failing to parse the valid form:
system_u:unconfined_r:unconfined_t:s0

Recommend using the previously parsed values $selinux_* from tests/runtests.pl

We need to make sure that we test the following:

• large numbers of arguments (force multiple EXECVE records)
• large argument sizes (split across multiple EXECVE records)
• "bad" characters which force hex encoding
• "bad" characters which force hex encoding and cause the argument to be recorded in multiple EXECVE records when it wouldn't normally

NOTE: "bad" characters are currently defined as ASCII 0x21, 0x22, and 0x7e.

There are several patterns used in tests which can be moved to our testing module to avoid duplication and speed up the process of writing new tests. Each test works more or less as follows:

1. Setup: add new audit rules and temporary files.
2. Trigger: generate audit events.
3. Verification: verify that correct events were generated.
4. Clean-up: restore audit rules and remove temporary files.

At the moment, I see the following candidates of "common subroutines" for this testing module:

1. Setup: unique key generation, audit rules backup, auditd conf backup, create temporary files
2. Trigger: none, this is usually very specific to tested functionality
3. Verification: traversing audit event to check it meets specification
4. Clean-up: restore rules and configuration, remove temporary files

As I mentioned somewhere else, clean-up should be handled as a list of callbacks so that it can be called by a signal handler (to make sure it always happens).

What do you think? I will prepare some example in a separate branch to have some code to discuss.

Not related to filesystem watches, test the ability to filter audit events based on individual files.

Test for kernel audit container id routing:

• verify initial and container policy loads
• verify messages routed to correct daemon(s)

See: linux-audit/audit-kernel#75
See: linux-audit/audit-userspace#41

^^^ I just spent the last hour figuring out why the testsuite doesn't work with locally installed audit userspace binaries until I found this line... Does it serve any particular purpose? Can we please remove it and let the user of the testsuite set the PATH variable as it suits them?

The problem is the following line in the syscalls_file/test file:

$result = system("auditctl -a always,exit -F arch=b64 -S open -k $key");

... which results in the following error on 32-bit x86:

# auditctl -a always,exit -F arch=b64 -S open
arch elf mapping not found

Right now the README isn't very complete when it comes documentation about installing and running the tests. At the very least we should have instructions for the different supported RHEL versions and upstream/Rawhide; ideally we would have instructions for non Red Hat distributions too, e.g. Debian/Ubuntu.

Sometimes the file_create tests fail on the first run but succeed on subsequent runs.