plus3it / amigen7 Goto Github PK
View Code? Open in Web Editor NEWSet of tools to provide automation of tasks for creating STIG-partitioned EL7 AMIs
License: Apache License 2.0
Set of tools to provide automation of tasks for creating STIG-partitioned EL7 AMIs
License: Apache License 2.0
GetAmznLx.sh fails on RHEL 7.6.
Script should detect RHEL version as el7 and get the proper rpms.
ELVERSION returns "el7.6". This causes the case evaluation on line 29 to fail since it is looking for "el7".
Run the script on RHEL 7.6.
Insert a "cut" at the end of the command to strip off the .6.
Once new EBS Volume is mounted as boot volume for ec2 instance, unable to login via ec2-user or any other user. Existing Private Key refused.
Existing private key for ec2-user should be replicated to newly built LVM EBS volume
Unable to login to instance with new LVM EBS volume
Perform the public network processes.
Swap boot EBS with LVM ebs, so that LVM EBS is new boot device (/dev/sda1)
Power on instance
Attempt to login via ec2-user with initial private key or any other user.
Ensure existing users and ec2-user keys are replicated on new EBS volume.
Nature of documentation error
Document Location
Most of the contents in the Docs
folder is YEARS old and no longer accurately reflects the content they're supposed to document the usage of.
Relate: the disk setup script probably needs to be fully-documented to make usage of the -p
/--partitioning
option more obvious.
Proposed Fix
Due to NVMe devices using different naming conventions for disks/partition (/dev/nvmeXnYpN
vice /dev/xvdXN
), script fails when using ${CHROOTDEV}N
as targeting-method. See:
The DiskSetup.sh
script should work on both NVMe- and non NVME-enabled instance-types
Change targeting method used for the above-enumerated lines.
Is your feature request related to a problem? Please describe.
When creating non-AWS build-types – particularly those that don't typically include execution of userData, it will be necessary for the root
account to be enabled for direct-login to VMs spawned from resultant templates
Describe the solution you'd like
Is your feature request related to a problem? Please describe.
AMIGen7's current configuration of cloud-init
cause the default-user to be created with the SELinux user-context, unconfined_u
. With newer security guidelines applied, this context will make it so that the default-user cannot actually escalate privileges, as needed, due to a bad, combined SELinux label – seen when doing sudo -i
as:
$ sudo -i
sudo: unconfined_u:sysadm_r:sysadm_t:s0-s0:c0.c1023 is not a valid context
Describe the solution you'd like
Change the default-user's SELinux user-context to staff_u
Describe alternatives you've considered
Additional context
Describe the bug
AMIgen will (effectively) silently fail if the @Core
RPM group is not available in the source yum repository.
Severity
FATAL: script appears to work as expected but the error is hidden. This can result in what appear on the face to be successful AMI-creation runs that, instead, produce broken AMIs
Fix Suggestions
Update ChrootBuild.sh
script to add an error-exit immediately after Line 170 if the value of INCLUDE_PKGS
is null.
Describe the bug
When using to generate VMware and other image-types, /var/run
does not get created as a symlink to /run
Severity
To Reproduce
busctl
, timedatectl
, etc.)Expected behavior
All commands that interact with systemd should function properly
Deviance Description
Resulting-images do not have /var/run
as a symlink to /run
Describe the bug
AMIgen assumes that resultant instances are relatively short-lived, such that the kernel will either never be updated (absent a rebuild) or will only live long enough to be updated a couple times. Thus, no change is made to the value of installonly_limit
in the /etc/yum.conf
file's default value of 5
. If a system-owner persists a system long enough to install more than 2-4 kernel updates, subsequent updates will fail due to lack of free space on the /boot
partition.
Severity
To Reproduce
Steps to reproduce the behavior:
yum udate kernel-<VERSION>
(where <VERSION>
is some version between what's installed and whatever latest available is)rpm -q kernel
and find that there are 3+ kernels already installed and /boot
has less free space than the size of the to-be-installed kernel RPMExpected behavior
The kernel update RPMs should never fail to install due to space-limitations.
Fix Suggestions
Update AMIgen's ChrootCfg.sh
(or later) script to modify the installonly_limit
value in the /etc/yum.conf
file to 2
Ensure that resultant AMIs have FIPS mode available and enabled.
Note: will be necessary to ensure that boot=
kernel parameter must be defined to ensure that kernel will boot.
Currently, AMIs are all created with EXT4 filesystems. Some users want XFS "since that's what the vendor default is"
Selectable fstype for root-filesystems ...with XFS as an option
EXT4 is hard-coded
Add new flag to appropriate script(s) to select fstype for root filesystems. Default to prior behavior so as not to break dependant projects.
Enablement of FIPS mode can break a number of things - which can be quite deleterious if the breakage impacts launch-time tasks. Need to make generation of FIPS-enabled images selectable. Due to hardening requirements, FIPS should default to enabled but be overridable.
Given that more than one build-module handles FIPS configuration, might want to make the override happen in the form of an environment variable (FIPSDISABLE
?)
Describe the bug
When running the ChrootBuild.sh script the /mnt/ec2-user/bin folder/contents and any associated libraries are not available. This is causing failure for anything trying to execute from the /bin folder in the chroot'd environment. For example, /bin/yum-config-manage fails to run.
Severity
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Significantly less errors. Preferably none. All packages to be installed. /mnt/ec2-user/bin to exist as well as required libraries for yum-config-manager to run successfully.
Deviance Description
Errors cause failure to create new AMI.
Additional context
Fix Suggestions
Lines 82-83 have a silent failure if named EPEL repository is already present in CHROOT. Probably want to implement a pre-flight so that anything invoking script with stop-on-error doesn't abort.
Some install-defaults for sshd service have UsePrivilegeSeparation
enabled. Either:
sshd
user existssshd
group existsssh_keys
group existsUsePrivilegeSeparation
to no
Describe the bug
Systemd-managed services (identified in spel Issue plus3it/spel/issues/317) are not enabled/started in instances created from this project's automation.
Severity
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Each of the following services should be preset-enabled and running:
autotune.service
)amazon-ssm-agent.service
)hibinit-agent.service
)ec2-instance-connect.service
)Fix Suggestions
Update the AWScliSetup.sh
script to perform a chroot()
ed enablement of the impacted services.
Is your feature request related to a problem? Please describe.
By default, cloud-init
creates the provisioning-user with the first available uid
value higher than the SYS_UID_MAX
value declared in /etc/login.defs
. Further, this account (rightly) gets created with no password-aging attributes. Security scanners treat this as an interactive user and will alert on any tests around password-aging.
Describe the solution you'd like
Have default-user created with a uid
value less than the SYS_UID_MAX
value declared in /etc/login.defs
.
Describe alternatives you've considered
Could force password-aging attributes onto account, but that will risk the "break glass" use-case for the account once expiry has been reached.
Could also use the uid
parameter to hard-code a uidvalue less than the
SYS_UID_MAXvalue declared in
/etc/login.defs`, but don't know if that could create other problems, later, for applications using the resultant AMI.
Additional context
cloud-init
provides the system: true
attribute that results in creating a user with a uid
value less than the SYS_UID_MAX
value declared in /etc/login.defs
. However, such users get created with no ${HOME}
which means no ability to create a ${HOME}/.ssh/authorized_keys
file. While cloud-init
also provides the notional ability to override this behavior by explicitly declaring the no_create_home
parameter and setting its value to false
, trying to use this as an override-attempt seems to be ignored.
EC2 validator-template goes to CREATE_COMPLETE long before S3 contents have actually been updated. Add signaling so that impatient test-user(s) has better idea of when/whether the S3 artifact-write has actually completed.
/etc/systemd/journald.conf
currently has Storage="persistent"
value set. Need to remove quotation marks around persistent
.
Graceful shutdown via AWS GUI/CLI not currently possible
Issue an instance-shutdown via the AWS software and have the OS undertake a graceful shutdown procedure.
(Nitro HVM) Instance OSes have no "hook" to receive the graceful shutdown requests. Ultimately, the cloud-layer has to initiate a forcible shutdown.
Issue a shutdown request via AWS cloud-layer tools
Add the ACPID package to the AMI. Per the AWS FAQ:
Q. How are instance reboot and termination EC2 API requests implemented by the Nitro Hypervisor?
The Nitro Hypervisor signals the operating system running in the instance that it should shut down cleanly by industry standard ACPI methods. For Linux instances, this requires that acpid be installed and functioning correctly. If acpid is not functioning in the instance, termination events will be delayed by multiple minutes and will then execute as a hard reset or power off.
Describe the bug
Some groupnames passable via the -g
flag are actually a group of groups + individual RPMs. Failure to expand the sub-groups can cause build failures.
Severity
May be worked around using the -m
flag and a sufficiently fully-specified manifest if needing other than the core
package-group.
Expected behavior
Specifying any given yum group via -g
should act the same as doing yum install <GROUPNAME>
.
Fix Suggestions
If optional-group is not core
, look for and iterate over/expand subgroups as part of the current package-enumeration routine.
On instances launched RHEL 7 AMIs /var/run
is not a link to /run
. This breaks DBUS upgrades
/var/run
is a link to /run
in all created AMIs
/var/run
is not a link to /run
in all created AMIs
Build new RHEL 7 AMIs from current scripts
Add an "end-of-build" type of task to ensure that /var/run
is a link to /run
.
Looks like EL7's disk geometry related dracut modules don't grok LVM. Need to locate and patch requisite scripts to be more like the AMIgen6-patched versions.
Is your feature request related to a problem? Please describe.
Make the GRUB_TIMEOUT
value settable
Describe the solution you'd like
Currently, the GRUB_TIMEOUT
value is set to ONE second. As AMIgen gains use on more platforms – and as CSPs begin to provide access to virtual consoles – the default (1s) value will be too short for users seeking to be able to access the GRUB menu for diagnostic and other reasons.
This is fixable by changing line-37 of the GrubSetup.sh script to make use of a parameter-set value.
Describe alternatives you've considered
Continue with static value but set to a much higher value: however, this will too-broadly slow boot processes and change expected behaviors
Additional context
Beta AMIs don't have swapVol
configured into the swap system
Is your feature request related to a problem? Please describe.
Some users of AMIs created from this automation find the static partitioning-scheme problematic on long-lived systems or systems with specialty monitoring and/or tooling needs
Describe the solution you'd like
Add a flag that allows the passing of a partitioning-map to DiskSetup.sh
(replacing or overriding the static values in its CarvLVM function. Probably want to pass the partitioning scheme as JSON-type data or possibly delimited triplets (MountLocation, VolumeName, VolumeSize).
Describe alternatives you've considered
Continue leaving storage relayout as a launch-time task for AMI owners...
Additional context
Other tooling that inherits this will need to be updated to take advantage and flag/option will need to be wholly optional so as not to break reliant tooling.
Is your feature request related to a problem? Please describe.
After some security profiles are applied to a spel AMI, the provisioning user may no longer be able to execute sudo
.
Describe the solution you'd like
Update CleanChroot.sh's
default-user setup section to add:
selinux_user: unconfined_u
Further, also ensure that:
sudoers: files
Is present in the /etc/nsswitch.conf
file.
The above will, effectively, implement the fixes described in the Red Hat Solution #3916301 prior to any problematic hardening.
Describe the bug
When using build-scripts to facilitate a chroot()
ed build for VMware, current logic can result in /
and/or /boot
being overlay-mounted on top of the desired /
and /boot
in the chroot()
target
Severity
Manual edit of MkChrootTree.sh
required to surmount
To Reproduce
Steps to reproduce the behavior:
MkChrootTree.sh
chroot()
target are "missing" (not present in df
output but present in /proc/mounts
)Expected behavior
Neither the /
nor the /boot
from the boot-OS should be mounted into the chroot()
Fix Suggestions
Add /\/ /d
and /\/boot /d
to partition-setup sed
block for the MkChrootTree.sh
script
When generating new "recovery" AMIs, the GrubSetup.sh
currently sets the root=
token to the chroot-built device's real devnode rather than properly referencing its LABEL
value
The root=
tokens in the chroot()
ed /boot/grub2/grub.cfg
should be set to root=LABEL=<LABEL>
The root=
tokens in the chroot()
ed /boot/grub2/grub.cfg
are set to root=/chroot/devnode
Run GrubSetup.sh
to create non-LVM'ed AMI
Fix as necessary...
Is your feature request related to a problem? Please describe.
With the merging of fixes for #120 via PR #121, sudo
behavior for the default-user becomes sub-optimal: users currently have to specifically request an SELinux role and type to get a useful permissions-set when elevating privileges. Casual sudo
users won't understand the need for this and will complain about permission-errors when elevating privileges.
Describe the solution you'd like
Ensure that the default-user's sudoers mapping contains an appropriate privilege-elevation rights-mapping. Something like:
TYPE=sysadm_t ROLE=sysadm_r
Or (less optimally):
TYPE=unconfined_t ROLE=unconfined_r
Added to the cloud.cfg
stanza for the default-user is recommended.
Describe alternatives you've considered
Additional context
Yum exits zero if any of the specified packages install successfully. This leads to false successes, when a specified package is unavailable.
If all of the specified packages are unavailable, then yum exits non-zero.
Expected the build to fail if any specified packages did not get installed into the AMI.
Build "succeeds" even when specified packages do not get installed.
Specify an available package and an unavailable package in the EXTRARPMS options.
Loop over all packages, to install them individually. This will cause yum to exit non-zero if any of the packages are unavailable.
Test that all the specified packages actually are all installed after each yum install
.
Parse the yum history
.
Tee the yum output and parse that.
Also, would like to apply the fix to both AMIgen7 and AMIgen6...
The grub2
RPM-manifest lists /etc/sysconfig/grub
as a component-file. However, this file has proven to be absent in the 7.2.1511 RPMs. Previous workaround was to copy build-host's file to chroot. However, this has unreliable results. Need to change method.
Is your feature request related to a problem? Please describe.
Two scenarios:
@Core
Describe the solution you'd like
Add a switch-argument to the ChrootBuild.sh script to allow the passing of an RPM manifest. Ideally, the mainifest can be readable from a variety of locations (e.g., local file, HTTP URL or cloud-storage URL(s))
Hi, just found this repo and giving it a try. I was prompted to run GetAmznLx.sh, but it wasn't present. I pulled this from your AMIgen6 repo and seemed to work fine though. Thanks
Need to validate that the chroot-based generation-process isn't causing /var/lock
to fail to be a symlink of ../run/lock
:
@lorengordon one to add to (link into) the task-board
After applying recommended hardening, the users created by cloud-init
will no longer be able to use sudo
if they've not been granted an appropriate SEL login user context:
$ sudo -i
sudo: PERM_SUDOERS: setresuid(-1, 1, -1): Operation not permitted
Ensure that the default provisioning user be given an appropriate SEL login user context to prevent the above.
Describe the bug
Severity
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Deviance Description
Screenshots
Additional context
Fix Suggestions
Appears that when instances launch on 10Gbps-enabled nodes, networking isn't getting properly set up. Fix for EL6 doesn't seem to translate to EL7
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.