Describe the bug
In closing-off issue #114, an additional check was added to search for the string [vV]ulnerable
in files beneath /sys/devices/system/cpu/vulnerabilities
- however, to be an effective test more nuance is needed.
The current checks are all based on the content of files which are present - but there appears to be no validation of files which must be present for the issue to be mitigated, but which aren't. It could be argued that it's unreasonable to assess an older image against flaws which weren't known when it was created - but that's exactly what's currently being done by reporting failures for hardware issues which can't be (entirely) fixed in software, such as the outstanding issues with Intel's SMT implementation.
See the Additional context section for further discussion.
Expected behavior
Failures are flagged (only) when a software mitigation is not present or not correctly deployed.
Actual behavior
Unfixable hardware errors are reported as failures, the opportunity exists for missing software mitigations to go unreported.
Amazon Linux 2, Ubuntu 20.04, Ubuntu 18.04:
amazon-ebs: × os-12: Detect vulnerabilities in the cpu-vulnerability-directory (3 failed)
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/ is expected to be directory
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/spectre_v2 content is expected not to match "vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/spectre_v2 content is expected not to match "Vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/itlb_multihit content is expected not to match "vulnerable"
amazon-ebs: × File /sys/devices/system/cpu/vulnerabilities/itlb_multihit content is expected not to match "Vulnerable"
amazon-ebs: expected "KVM: Vulnerable\n" not to match "Vulnerable"
amazon-ebs: Diff:
amazon-ebs: @@ -1 +1 @@
amazon-ebs: -Vulnerable
amazon-ebs: +KVM: Vulnerable
amazon-ebs:
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/mds content is expected not to match "vulnerable"
amazon-ebs: × File /sys/devices/system/cpu/vulnerabilities/mds content is expected not to match "Vulnerable"
amazon-ebs: expected "Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\n" not to match "Vulnerable"
amazon-ebs: Diff:
amazon-ebs: @@ -1 +1 @@
amazon-ebs: -Vulnerable
amazon-ebs: +Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
amazon-ebs:
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/l1tf content is expected not to match "vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/l1tf content is expected not to match "Vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/spec_store_bypass content is expected not to match "vulnerable"
amazon-ebs: × File /sys/devices/system/cpu/vulnerabilities/spec_store_bypass content is expected not to match "Vulnerable"
amazon-ebs: expected "Vulnerable\n" not to match "Vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/tsx_async_abort content is expected not to match "vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/tsx_async_abort content is expected not to match "Vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/spectre_v1 content is expected not to match "vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/spectre_v1 content is expected not to match "Vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/srbds content is expected not to match "vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/srbds content is expected not to match "Vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/meltdown content is expected not to match "vulnerable"
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/meltdown content is expected not to match "Vulnerable"
Ubuntu 16.04 also reports:
amazon-ebs: ✔ File /sys/devices/system/cpu/vulnerabilities/tsx_async_abort content is expected not to match "vulnerable"
amazon-ebs: × File /sys/devices/system/cpu/vulnerabilities/tsx_async_abort content is expected not to match "Vulnerable"
amazon-ebs: expected "Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\n" not to match "Vulnerable"
amazon-ebs: Diff:
amazon-ebs: @@ -1 +1 @@
amazon-ebs: -Vulnerable
amazon-ebs: +Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
amazon-ebs:
OS / Environment
Amazon EC2 executing roles against Ubuntu 16.04LTS, 18.04LTS, 20.04LTS, Amazon Linux 2, CentOS 7 stock images - all of which fail since #138 was committed recently.
Inspec Version
Baseline Version
amazon-ebs: Profile: DevSec Linux Security Baseline (linux-baseline)
amazon-ebs: Version: 2.6.0
(from https://github.com/dev-sec/linux-baseline/archive/master.tar.gz
as-of 20201208
)
Additional context
Especially when validating a system image which might be deployed to a range of hardware, I would suggest that the tests performed should limit themselves to reporting failure cases where a mitigation exists (or make this an option) - the current checks will universally fail on certain (cloud) hardware, simply because there is no possible fix for that hardware (other than not to use it!). I would suggest that this security profile should try to limit itself to ensuring that all possible software work-arounds are deployed, where they exist.
Even this could be improved - for example, https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/Variant4 states that a system is vulnerable if /sys/devices/system/cpu/vulnerabilities/spec_store_bypass
doesn't exist - the current checks are only examining those files which do exist. In this case it seems that there a significant performance benefits to allowing Speculative Store, and so EC2 appears to continue to offer this facility to guest OS' - in which case, possibly this should be flagged as a warning rather than a failure? There doesn't seem to be any possible workaround a guest is able to employ, in any case.
It also appears that the mds
error cannot be fixed within a VM - suggest passing the test if SMT Host state unknown
is present (or operation within a virtualised environment by other means can be confirmed).
The itlb_multihit
mitigation (https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/multihit.html#itlb-multihit-system-information) appears to be solely focused on KVM - so if that module is not present (or not loaded?) on the system, then it appears that this test should also pass.
Finally, comparing the output from different images, older kernels lack vulnerability
nodes for itlb_multihit
, srbds
, tsx_async_abort
- which I take to mean that these systems have no mitigations in place, but too old a kernel to report this.
I would suggest that the CPU Vulnerability checks should have a list of expected files and should alert if any of these is missing - but should also account for scenarios where the test is being executed on hardware not under the control of the user, and not treat mitigations that cloud providers have chosen not to apply as fatal errors. It feels as if there is the need for a 'warning' category of feedback here which isn't as strong as an error, but which is still notified to the user.