need for apptainer 1.0.0

side by side package install

e2e tests need to be updated to verify correctness of this for various supported platforms

see example on PR #3

dist/rpm/apptainer.spec.in
%attr(4755, root, root) %{_libexecdir}/singularity/bin/starter-suid
%{_bindir}/singularity
%attr(4755, root, root) %{_libexecdir}/apptainer/bin/starter-suid
%{_bindir}/apptainer
  
Contributor
@DrDaveD DrDaveD 14 hours ago
Need a %{_bindir}/singularity as well as %{_bindir}/apptainer.

===

noting: need some indications of administrator intent when "Singularity" is installed on top of Apptainer, or Apptainer is installed over Singularity

reapply changes from #30 after PR #3 is merged

This is a continuation of apptainer/singularity#468.

Many changes in #3 ended up with "a apptainer" or "a Apptainer" where it should be "an apptainer" or "an Apptainer".

update all example go.mod to refer to apptainer packages

prerequisite for v1.0.0

The technical charter requires the CONTRIBUTING.md file in the repository to contain a list of TSC members/committers, or to document an alternative method of designation.

Re: https://apptainer.org/technical-charter section 2(b)

add "corrupted container" to registry and change e2e test for corrupted container to use it.

re: library://sylabs/tests/verify_corrupted:1.0.1 -> oras://ghcr.io/apptainer/verify_corrupted:1.0.1

Originally posted by @PaulCharlton in #54 (comment)

where we are at

current test suites need to run as privileged user, with mixed privileges between root and sudo access
to drivers and other resources on an Ubuntu Host in the GitHub CI process.

desired state

developers need to be able to run each element of the test suite locally, regardless of their host OS, and
preferably without managing their own guest OS via virtualization

add formal barrier for parallel e2e tests

1. allow a test to request "Wait for previous test[by_name] to complete"
2. if previous test[by_name] has a good result, execute
3. otherwise, do not execute and report precedent failed

need to ensure that the barriers form a di-graph for execution (no cycles)

After #53 removed remote building, I still see some references to "remote build" in the code including "Usage" strings and also in man apptainer-build.

Attn: @PaulCharlton

there are instances in golang code where the string length is hard coded and then used in slice notation.

replace these with len(target_str)

evaluate changes to examples/legacy and perhaps revert to original contents after PR #3

defer remove dependencies on remote commercial services (key server) for compliance with LF open-source license

remote key server

target release: Apptainer v1.0.0

Originally posted by @PaulCharlton in #53 (comment)

Import /etc/singularity/singularity.conf if it has been changed and copy into /etc/apptainer/apptainer.conf.

@DrDaveD for the inbound environment variables, they go through a single function for the most part (and the remainder should be re-routed via that function)

Also, this issue should be broken into two separate issues.

1. inbound environment variables
2. published environment variables for child/exec'd processes

Originally posted by @PaulCharlton in #7 (comment)

feature request: add search functionality for OCI and ORAS registries

key search doesn't work with keyserver https://keys.openpgp.org. For example with singularity-3.8.5

$ sudo singularity remote add-keyserver https://keys.openpgp.org
$ sudo singularity remote remove-keyserver https://keys.sylabs.io
$  singularity key search [email protected]
Showing -1 results

ERROR:   search failed: unable to check key number

current code executes container's actions scripts with globbing, as follows:

for __script__ in /.singularity.d/env/*.sh; do

The order of script execution is dependent on how the shell glob operator scans the underlying directory.

Make the sort order strictly numeric, and independent of locale and default collation on the filesystem by using:

for __script__ in $(ls /.singularity.d/env/*.sh |sort -n); do

The legacy singularity supports environment variables named "SINGULARITY_" and "SINGULARITYENV_"

Apptainer supports environment variables named "APPTAINER_" and "APPTAINERENV_"

Proposed:

1. When the inherited environment has APPTAINER variable of same name suffix as a set SINGULARITY environment variable, the APPTAINER value shall have precedence.

2. When a SINGULARITY environment variable is set, and the APPTAINER environment with the same suffix is not set, copy the value of the SINGULARITY environment variable to a newly created APPTAINER variable with same name suffix.

3. When creating the environment for a child process, enumerate all APPTAINER variable values from (1, 2) above, and create a new set of SINGULARITY environment variables with the same value and name suffix as the APPTAINER variable.

ensure e2e tests exist for both cgroup and cgroup2 enabled kernels

This will require the creation of customer runner servers that can boot with cgroup2 enabled. (GH runners do not, as yet, have cgroup2 capability enabled)

remote container registries should be accessed via internal "interface", not hard-coded switch per function; plugins should be allowed to provide such an interface

Originally posted by @PaulCharlton in #54 (comment)

Continuation of apptainer/singularity#6302

This claim is made on the apptainer site.

It is designed to execute applications at bare-metal performance while being secure, portable, and 100% reproducible.

Singularity had some very annoying default behaviors that destroyed any hope of reproducibility:

• Mount the $HOME directory of the user. Because every user has their own .config files that set application behavior, this means that application behavior is not guaranteed to be reproducible.
• Import the user's environment. I don't have to explain this one: it is a reproducibility atrocity.

As a result we have been using singularity with the --containall flag all the time. Because it is the only viable way to run singularity in a reproducible manner.

I hope that apptainer sets this by default. Requiring explicit user interference if non-reproducible liabilities are introduced.

#5 removed everything CHANGELOG.md including unreleased changes. Those need to be restored.

add ability to checkpoint container to storage device as first class option.

At the moment, checkpoint capability is available via the plugin module from lancium
at checkpoint plugin

Continuation of apptainer/singularity#6030.

Code in #3 changed ~/.singularity and $HOME/.singularity to ~/.apptainer and $HOME/.apptainer. If that's going to stay that way, provide some kind of migration path or error exit if ~/.singularity exists but ~/.apptainer does not.

...

Originally posted by @PaulCharlton in #99 (review)

I noticed there's a PACKAGE_BUGREPORT defined pointing to sylabs.io in builddir/config.h. It doesn't appear to be used anywhere, but should be removed.

Apptainer v1.0.0 must build able to build SIF containers which are runnable via legacy Singularity.

Unresolved: should this be default, or enabled by command line switch.

Need: deprecation warnings when legacy features are used

Add E2E tests for validation

Scope of compatibility includes internal filesystem naming structure and the names of injected environment variables

moving embedded shell scripts out of golang strings into separate shell script files.

This will allow CI listing of all of the shell scripts, as well as allow easier maintenance of copyright and other attribution in those shell scripts.

The separate files can be interpolated into the golang sources via various golang preprocessors

As discussed on the mailing list, support for the library protocol that was removed in #69 can be restored.

feature request: add support for decorating bare image name for Oras and OCI compatible registries

Originally posted by @PaulCharlton in #61 (comment)

add IDE plugins for linting and syntax highlighting of .def files

remove dependencies on remote commercial services for compliance with LF open-source license

• remote library
• remote builder
• remote key server

target release: Apptainer v1.0.0

Fix these problems noticed in pr #3:

1. References to apptainer with versions 3.x should either be singularity 3.x if referring to a specific older version or apptainer 1.x if referring to a future or example version. Likewise apptainer 2.x should be singularity 2.x. Search for regexp [Aa]pptainer.*[23].
2. Fix URLs with sylabs.io/apptainer and github.com/sylabs/apptainer to be singularity instead of apptainer.
3. Fix Apptainer Image Format to be Singularity Image Format.

Comments captured from #3 for the follow up regarding image build:

By leveraging nfpm

The CI process currently uses container images hosted by sylab on AWS S3 and Docker Hub.

To avoid shifting costs and responsibilities to a third party, build those container images and host them on sites paid for by the appropriate Linux Foundation entity.

provide MD file documentation for all public and private environment variables recognized by apptainer and/or published b y apptainer when launching a new container/instance, complete with behavioral description

remove singularity desktop from codebase

As it is no longer maintained

feature request: add delete functionality for OCI and ORAS registries

Originally posted by @PaulCharlton in #61 (comment)

Planned for replacing pr apptainer/singularity#6206

Continuation of apptainer/singularity#6180

Continuation of apptainer/singularity#6259. This may be taken care of as part of #8.

the following containers need to be built and hosted in an OCI compatible registry (or not, as appropriate for negative tests) as they are used in both unit tests and e2e tests

library://alpine
library://alpine:3.11.5
library://alpine:3.11.5 /bin/true
library://alpine:latest
library://alpine:sha256.03883ca565b32e58fa0a496316d69de35741f2ef34b5b4658a6fec04ed8149a8
library://busybox
library://busybox:1.31.1
library://centos cat /etc/os-release`
library://collection/image
library://debian:latest
library://image
library://library.example.com/test/default/test:v0.0.3
library://library.sylabs.io/library/default/busybox:1.31.1
library://lolcow
library://notlibrary.sylabs.io/library/default/busybox:1.31.1
library://sylabs/tests/does_not_exist:0
library://sylabs/tests/signed:1.0.0
library://sylabs/tests/unsigned:1.0.0
library://sylabs/tests/verify_corrupted:1.0.1
library://sylabs/tests/verify_success:1.0.2
library://test/default/test:v0.0.3
library://user/collection/container[:tag]
library://user/collection/my.sif:latest
library://username/project/image:1.0`
library://valid_uri_but_not_real_image

_Originally posted by @PaulCharlton in https://github.com/apptainer/apptainer/issues/53#issuecomment-990586842_

Version of Apptainer:

We currently reach out to https://keys.sylabs.io in the e2e tests for pulling public keys dynamically, we should consider spawning and using a local keyserver just like we do with a local registry.

remove dependencies on remote commercial services (library) for compliance with LF open-source license

remote library

target release: Apptainer v1.0.0

Originally posted by @PaulCharlton in #53 (comment)

remove dead or unreachable code from repo