Giter VIP home page Giter VIP logo

os's People

Contributors

ajayk avatar chainguardian avatar cpanato avatar debasishbsws avatar dentrax avatar developer-guy avatar dlorenc avatar eyecantcu avatar found-it avatar hectorj2f avatar imjasonh avatar jdolitsky avatar jonjohnsonjr avatar joshrwolf avatar k4leung4 avatar kaniini avatar luhring avatar mamccorm avatar mattmoor avatar mauricio-dc-chainguard avatar mritunjaysharma394 avatar octo-sts[bot] avatar pdeslaur avatar rawlingsj avatar sergio-chainguard avatar tcnghia avatar tuananh avatar vaikas avatar wolfi-bot avatar xnox avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

os's Issues

[Wolfi Package Request]: Ruby Bundler

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

Ruby Bundler (https://bundler.io/)

which versions of the software should we include?

v2.4.3 (latest)

Add some justification for why this specific package and versions are important.

This package is needed to create Ruby APKs (see Alpine based Ruby example).

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

Support multiple python 3.x streams

It would be incredibly useful to have some way of having packages for each currently supported python 3 stream, so currently I think

  • 3.11
  • 3.10
  • 3.9
  • 3.8
  • 3.7 (though I think this just has 7 months or so left)

I saw #75 and wondered if a similar approach is possible; however, python seems a bit more complicated since there are also a bunch of py3-* packages that'd probably have to be split out as well since they currently have runtime dependencies on python3

Missing auto update PRs for some packages

wolfi auto updater isn't creating PRs for some package updates. Checking the logs I can see

2023/01/14 21:05:00 wolfictl update: there is a new stable version available s6, current wolfi version 2.11.1.2, new 2.11.2.0
2023/01/14 21:05:51 wolfictl update: there is a new stable version available llvm-libunwind, current wolfi version 15.0.6, new 15.0.7
2023/01/14 21:06:06 wolfictl update: there is a new stable version available libsm, current wolfi version 1.2.3, new 1.2.4
2023/01/14 21:06:18 wolfictl update: there is a new stable version available execline, current wolfi version 2.9.0.1, new 2.9.1.0
2023/01/14 21:06:51 wolfictl update: there is a new stable version available libice, current wolfi version 1.0.10, new 1.1.1
2023/01/14 21:07:30 wolfictl update: there is a new stable version available binutils, current wolfi version 2.39, new 2.40
2023/01/14 21:07:54 wolfictl update: there is a new stable version available nghttp2, current wolfi version 1.49.0, new 1.51.0
2023/01/14 21:07:57 wolfictl update: there is a new stable version available libarchive, current wolfi version 3.6.1, new 3.6.2
2023/01/14 21:08:57 wolfictl update: there is a new stable version available py3-packaging, current wolfi version 21.3, new 23.0
2023/01/14 21:09:12 wolfictl update: there is a new stable version available libpaper, current wolfi version 1.1.28, new 2.0.4
2023/01/14 21:09:18 wolfictl update: there is a new stable version available skalibs, current wolfi version 2.12.0.1, new 2.12.1.0
2023/01/14 21:10:30 wolfictl update: there is a new stable version available py3-installer, current wolfi version 0.5.1, new 0.6.0
2023/01/14 21:11:06 wolfictl update: there is a new stable version available xz, current wolfi version 5.4.0, new 5.4.1

Taking skalibs as an example, release monitor has a newer version available https://release-monitoring.org/project/5486/

question regarding memcached official alpine image

memcached official image is compiled a bit different with the alpine package

In official image, memcached is configured with

# official image
&& ./configure \
		--build="$gnuArch" \
		--enable-extstore \
		--enable-sasl \
		--enable-sasl-pwdb \
		--enable-tls \

vs the configure flags in alpine pkgs

./configure \
		--build=$CBUILD \
		--host=$CHOST \
		--prefix=/usr \
		--enable-sasl \
		--enable-sasl-pwdb \
		--enable-seccomp

I'm not sure how difference is it gonna be in this case. but what if we have this situtation in other packages, what should be the convention here?

  • Should we follow the official image so that switching over to Chainguard images is more seamless?
  • Or should we follow the package from alpine?

Package go 1.18

Upstream Go supports the last two releases, which right now are 1.18 and 1.19. We only package 1.19 today.

I think we should follow the example set by go-1.20.yaml (an unreleased rc) and package go-1.18.yaml and go-1.19.yaml, with a go package in go.yaml that simply has a runtime dependency on the latest stable go release (currently go-1.19).

When go-1.20 is officially released (expected "Feb 2023"), go.yaml can update to point to go-1.20, go-1.18.yaml can be deleted as it's no longer supported upstream (the go-1.18 package would continue to exist ~forever), and folks who want 1.19 can still get it by depending on the go-1.19 package.

We'd follow this same process with go-1.21 has an rc, and later the full release.

Does this sound right @kaniini ?

[Wolfi Package Request]: dotnet7-sdk

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

project repo: https://github.com/dotnet/sdk
related request in images repo: chainguard-images/images#170

which versions of the software should we include?

7

Add some justification for why this specific package and versions are important.

chainguard-images/images#170

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

py3-pip package broken

The py3-pip package currently seems broken. See chainguard-images/images#149 for details.

$ docker run -it cgr.dev/chainguard/python:latest-dev /bin/sh
/ $ pip --help
Traceback (most recent call last):
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/pip", line 33, in <module>
    sys.exit(load_entry_point('pip==22.3.1', 'console_scripts', 'pip')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/pip", line 22, in importlib_load_entry_point
    for entry_point in distribution(dist_name).entry_points
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 981, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 565, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for pip

We should split adaptation-specific files into a subpackage, and provide hardened defaults when no specific adaptation is utilized (e.g. in the chainguard images product)

related to :
Erase contents (but leave file) of /etc/securetty

Harden/clearnup /etc/inittab

/etc/inittab

::sysinit:/sbin/openrc sysinit
::sysinit:/sbin/openrc boot
::wait:/sbin/openrc default

Set up a couple of getty's

tty1::respawn:/sbin/getty 38400 tty1
tty2::respawn:/sbin/getty 38400 tty2
tty3::respawn:/sbin/getty 38400 tty3
tty4::respawn:/sbin/getty 38400 tty4
tty5::respawn:/sbin/getty 38400 tty5
tty6::respawn:/sbin/getty 38400 tty6

Put a getty on the serial port

#ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100

Stuff to do for the 3-finger salute

::ctrlaltdel:/sbin/reboot

Stuff to do before rebooting

::shutdown:/sbin/openrc shutdown

New package(s): docbook-xsl, libxml2-utils, libxslt, docbook-xml

docbook-xsl is required for building manpages for the bubblewrap package

This can be explicitly enabled by setting -Dman=enabled on the meson build, but results in:

2022/10/18 15:21:23 melange (bubblewrap/x86_64): Program xsltproc found: NO
2022/10/18 15:21:23 melange (bubblewrap/x86_64): 
2022/10/18 15:21:23 melange (bubblewrap/x86_64): meson.build:130:0: ERROR: Program 'xsltproc' not found or not executable

Here is the related Alpine APKBUILD: https://git.alpinelinux.org/aports/tree/main/docbook-xsl/APKBUILD

It appears to require also: libxml2-utils, libxslt, docbook-xml (so this may be a rabbit hole)

[Wolfi Package Request]: php-fpm

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

php-fpm: https://www.php.net/
git: https://github.com/php/php-src
source: https://www.php.net/distributions/php-8.0.26.tar.gz

PHP with fpm enabled
https://www.php.net/manual/en/install.fpm.install.php

which versions of the software should we include?

8.0.26

Add some justification for why this specific package and versions are important.

php-fpm is a common way to run the php application so would be nice if it could be added as a sub package to php. Also we are still on 8.0 so would be nice to support all currently supported php versions: 8.2, 8.1, 8.0

This should be nice if there could be a chainguard-images of php-fpm later with the different php versions as well.

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

py3 sqlite3 module broken

Hi, it seems that sqlite3 module doesn't work properly for both new variants chainguard-images/python available

steps to reproduce

$ podman run --pull always --rm -it cgr.dev/chainguard/python:latest python
Trying to pull cgr.dev/chainguard/python:latest...
Getting image source signatures
Copying blob 1b44cf38bce8 skipped: already exists
Copying config e7d64953c8 done
Writing manifest to image destination
Storing signatures
Python 3.11.1 (main, Jan  1 1970, 00:00:00) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.11/sqlite3/__init__.py", line 57, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/lib/python3.11/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'

some feedbacks

From build log relative to commit d9a27c9 during python3 ./configure

2023-01-28T02:07:24.3723812Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for LIBSQLITE3... yes
2023-01-28T02:07:24.4141281Z 2023/01/28 02:07:24 melange (python3/x86_64): checking sqlite3.h usability... yes
2023-01-28T02:07:24.4273190Z 2023/01/28 02:07:24 melange (python3/x86_64): checking sqlite3.h presence... yes
2023-01-28T02:07:24.4273664Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3.h... yes
2023-01-28T02:07:24.5099067Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_bind_double in -lsqlite3... no
2023-01-28T02:07:24.5642524Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_column_decltype in -lsqlite3... no
2023-01-28T02:07:24.6206648Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_column_double in -lsqlite3... no
2023-01-28T02:07:24.6758472Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_complete in -lsqlite3... no
2023-01-28T02:07:24.7314337Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_enable_shared_cache in -lsqlite3... no
2023-01-28T02:07:24.7873187Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_progress_handler in -lsqlite3... no
2023-01-28T02:07:24.8405633Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_result_double in -lsqlite3... no
2023-01-28T02:07:24.8922784Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_set_authorizer in -lsqlite3... no
2023-01-28T02:07:24.9462918Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_trace_v2 in -lsqlite3... no
2023-01-28T02:07:24.9998660Z 2023/01/28 02:07:24 melange (python3/x86_64): checking for sqlite3_trace in -lsqlite3... no
2023-01-28T02:07:25.0544294Z 2023/01/28 02:07:25 melange (python3/x86_64): checking for sqlite3_value_double in -lsqlite3... no
2023-01-28T02:07:25.1113163Z 2023/01/28 02:07:25 melange (python3/x86_64): checking for sqlite3_load_extension in -lsqlite3... no
2023-01-28T02:07:25.1659504Z 2023/01/28 02:07:25 melange (python3/x86_64): checking for sqlite3_serialize in -lsqlite3... no
2023-01-28T02:07:25.1668991Z 2023/01/28 02:07:25 melange (python3/x86_64): checking for --enable-loadable-sqlite-extensions... n/a
2023-01-28T02:07:25.1669623Z 2023/01/28 02:07:25 melange (python3/x86_64): configure: WARNING: Your version of SQLite does not support loadable extensions

Making a rapid comparison between sqlite package shipped within wolfi-base:latest or alpine-base:latest, .dbconfig return same output (load_extension on). But on alpine sqlite3 python3 module works fine.

sqlite> .dbconfig
          defensive off
            dqs_ddl on
            dqs_dml on
        enable_fkey off
        enable_qpsg off
     enable_trigger on
        enable_view on
     fts3_tokenizer off
 legacy_alter_table off
 legacy_file_format off
     load_extension on
   no_ckpt_on_close off
     reset_database off
        trigger_eqp off
     trusted_schema on
    writable_schema off

Thank you in advance for your support

libjpeg-turbo vs. libjpeg

We should sort this out eventually (see #325 for some of the confusion).

TLDR: The package we ship as libjpeg is actually libjpeg-turbo. Some packages ask for libjpeg-turbo directly as a dep, which measn it won't be found.

We could make libjpeg "provide" libjpeg-turbo to help here, or some other clever tricks I'm probably not thinking of.

[Wolfi Package Request]: argocd

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

As GitOps user I would like to install argocd CLI.

which versions of the software should we include?

v2.6.0

Add some justification for why this specific package and versions are important.

Argo CD CLI is one of the popularly used GitOps platform and this CD helps interacting Argo CD clusters.

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

[Wolfi Package Request]: haproxy

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

HAProxy: www.haproxy.org
git: https://github.com/haproxy/haproxy/ (mirror of https://git.haproxy.org)
source: https://www.haproxy.org/download/2.6/src/haproxy-2.6.7.tar.gz

which versions of the software should we include?

2.6.7

Add some justification for why this specific package and versions are important.

HAProxy is a high performance reverse proxy that is widely used

the 2.6.x releases is the latest LTS release with long support which I think is nice to stick to.

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

Fill in copyright attestations

...or remove them?

We have a bunch of attestation: TODOs in our configs.

$ git grep "attestation: TODO" | wc -l
     312

@kaniini says:

the attestation should ideally be a collection of the copyright lines plus the "this software is licensed under blabla" text. similar to debian/copyright, but we are still kinda playing with the format, so it's not so important right now.

We should figure out what we want these to be and fill them in, or maybe just remove them until we figure out what to do with them. In the meantime they're sort of just cargo-culted around every new config. We don't have any linting in place that it's set or valid, so it's just kind of ...there.

Remove unused users/groups and associated directories

see examples below:

root:x:0:0:root:/root:/bin/ash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/mail:/sbin/nologin
news:x:9:13:news:/usr/lib/news:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucppublic:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
man:x:13:15:man:/usr/man:/sbin/nologin
postmaster:x:14:12:postmaster:/var/mail:/sbin/nologin
cron:x:16:16:cron:/var/spool/cron:/sbin/nologin
ftp:x:21:21::/var/lib/ftp:/sbin/nologin
sshd:x:22:22:sshd:/dev/null:/sbin/nologin
at:x:25:25:at:/var/spool/cron/atjobs:/sbin/nologin
squid:x:31:31:Squid:/var/cache/squid:/sbin/nologin
xfs:x:33:33:X Font Server:/etc/X11/fs:/sbin/nologin
games:x:35:35:games:/usr/games:/sbin/nologin
cyrus:x:85:12::/usr/cyrus:/sbin/nologin
vpopmail:x:89:89::/var/vpopmail:/sbin/nologin
ntp:x:123:123:NTP:/var/empty:/sbin/nologin
smmsp:x:209:209:smmsp:/var/spool/mqueue:/sbin/nologin
guest:x:405:100:guest:/dev/null:/sbin/nologin
nobody:x:65534:65534:nobody:/:/sbin/nologin

rabbitmq-server image missing tools/access

Using the current rabbitmq-server image, the rabbitmq cluster operator encounter the following errors when operating on the rabbitmq cluster-server statefulset

First error

"command":"rabbitmq-queues rebalance all", "controller":"rabbitmqcluster", "controllerGroup":"rabbitmq.com", "controllerKind":"RabbitmqCluster", "error":"command terminated with exit code 127", "level":"error", "msg":"failed to run queue rebalance on pod", "name":"cluster", "namespace":"rabbitmq", "pod":"cluster-server-0", "reconcileID":"849456a3-362d-4e1c-9a19-6e64a6d31884", "stacktrace":"github.com/rabbitmq/cluster-operator/controllers.(*RabbitmqClusterReconciler).runQueueRebalanceCommand
	/workspace/controllers/reconcile_cli.go:112
github.com/rabbitmq/cluster-operator/controllers.(*RabbitmqClusterReconciler).runRabbitmqCLICommandsIfAnnotated
	/workspace/controllers/reconcile_cli.go:60
github.com/rabbitmq/cluster-operator/controllers.(*RabbitmqClusterReconciler).Reconcile
	/workspace/controllers/rabbitmqcluster_controller.go:229
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234", "stderr":"sh: rabbitmq-queues: not found
", "stdout":"", "ts":1.6746083575837216E9}

Second error

"command":"rabbitmqctl enable_feature_flag all", "controller":"rabbitmqcluster", "controllerGroup":"rabbitmq.com", "controllerKind":"RabbitmqCluster", "error":"command terminated with exit code 69", "level":"error", "msg":"failed to enable all feature flags on pod", "name":"cluster", "namespace":"rabbitmq", "pod":"cluster-server-0", "reconcileID":"55e3e2cc-2500-43cc-9b3a-9ee497cd083f", "stacktrace":"github.com/rabbitmq/cluster-operator/controllers.(*RabbitmqClusterReconciler).runEnableFeatureFlagsCommand
	/workspace/controllers/reconcile_cli.go:75
github.com/rabbitmq/cluster-operator/controllers.(*RabbitmqClusterReconciler).runRabbitmqCLICommandsIfAnnotated
	/workspace/controllers/reconcile_cli.go:53
github.com/rabbitmq/cluster-operator/controllers.(*RabbitmqClusterReconciler).Reconcile
	/workspace/controllers/rabbitmqcluster_controller.go:229
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234", "stderr":"warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell)
Error: unable to perform an operation on node '[email protected]'. Please see diagnostics information and suggestions below.

Most common reasons for this are:

 * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
 * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
 * Target node is not running

In addition to the diagnostics info below:

 * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more
 * Consult server logs on node [email protected]
 * If target node is configured to use long node names, don't forget to use --longnames with CLI tools

DIAGNOSTICS
===========

attempted to contact: ['[email protected]']

[email protected]:
  * connected to epmd (port 4369) on cluster-server-0.cluster-nodes.rabbitmq
  * epmd reports node 'rabbit' uses port 25672 for inter-node and CLI tool traffic 
  * TCP connection succeeded but Erlang distribution failed 
  * suggestion: check if the Erlang cookie is identical for all server nodes and CLI tools
  * suggestion: check if all server nodes and CLI tools use consistent hostnames when addressing each other
  * suggestion: check if inter-node connections may be configured to use TLS. If so, all nodes and CLI tools must do that
   * suggestion: see the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more


Current node details:
 * node name: '[email protected]'
 * effective user's home directory: /home/rabbitmq
 * Erlang cookie hash: mcXiZOYMMv4h8OBr8flQ3Q==

", "stdout":"", "ts":1.6746093515409944E9}

FYI @dlorenc

python3: weak binary security

Hello! I have been evaluating chainguard images lately and noticed that the python binary does not feature Stack canaries, Full RELRO and FORTIFY_SOURCE.

Steps to reproduce

$ cat <<EOF | docker build -
FROM cgr.dev/chainguard/python:latest-dev as cgr

FROM alpine:3.17 as alpine

RUN apk add python3

FROM ubuntu:jammy

COPY --from=cgr /usr/bin/python3.11 /check/python3.11-chainguard

COPY --from=alpine /usr/bin/python3.10 /check/python3.10-alpine

RUN apt-get update &&\
  apt-get install -y file checksec python3

RUN checksec --file=/check/python3.11-chainguard &&\
  file /check/python3.11-chainguard &&\
  ldd /check/python3.11-chainguard

RUN checksec --file=/check/python3.10-alpine &&\
  file /check/python3.10-alpine &&\
  ldd /check/python3.10-alpine

RUN checksec --file=/usr/bin/python3.10 &&\
  file /usr/bin/python3.10 &&\
  ldd /usr/bin/python3.10
EOF

Resulting output

for cgr.dev/chainguard/python

Step 8/10 : RUN checksec --file=/check/python3.11-chainguard &&  file /check/python3.11-chainguard &&  ldd /check/python3.11-chainguard
 ---> Running in 3bc694638b95
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Partial RELRO   No canary found   NX enabled    PIE enabled     No RPATH   No RUNPATH   No Symbols	  No	0		0		/check/python3.11-chainguard
/check/python3.11-chainguard: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 4.9.0, stripped
	linux-vdso.so.1 (0x0000ffffa5703000)
	libpython3.11.so.1.0 => not found
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffa54f0000)
	/lib/ld-linux-aarch64.so.1 (0x0000ffffa56ca000)

Results for reference and comparison

for Alpine Linux 3.17

Step 9/10 : RUN checksec --file=/check/python3.10-alpine &&  file /check/python3.10-alpine &&  ldd /check/python3.10-alpine
 ---> Running in 06182b5238fd
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Full RELRO      No canary found   NX enabled    PIE enabled     No RPATH   No RUNPATH   No Symbols	  No	0		0		/check/python3.10-alpine
/check/python3.10-alpine: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, stripped
	linux-vdso.so.1 (0x0000ffff8e321000)
	libpython3.10.so.1.0 => not found
	libc.musl-aarch64.so.1 => not found

for Ubuntu Jammy

Step 10/10 : RUN checksec --file=/usr/bin/python3.10 &&  file /usr/bin/python3.10 &&  ldd /usr/bin/python3.10
 ---> Running in 6bdbd9cb571e
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable	FILE
Full RELRO      Canary found      NX enabled    PIE enabled     No RPATH   No RUNPATH   No Symbols	  Yes	18		43		/usr/bin/python3.10
/usr/bin/python3.10: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=a1841de4f4ec9445a10bff638afa4c72deace9e0, for GNU/Linux 3.7.0, stripped
	linux-vdso.so.1 (0x0000ffffaca8d000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffffac3f0000)
	libexpat.so.1 => /lib/aarch64-linux-gnu/libexpat.so.1 (0x0000ffffac3b0000)
	libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000ffffac380000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffac1d0000)
	/lib/ld-linux-aarch64.so.1 (0x0000ffffaca54000)

Expected output

Aligned with the results from Ubuntu >=Jammy.

Automatically kickoff investigations of potential vulnerabilities

In order to improve the quality and utility of our advisory data, we should constantly monitor for potential new vulnerabilities in Wolfi packages.

If we detect that a Wolfi package might be affected by a vulnerability, we should have a way to automatically create an advisory, with an entry status of under_investigation. This should prompt a Wolfi team member to investigate the potential vulnerability and make a determination on whether we're affected or not. If we are affected, we should patch our package ASAP.

All of this should be as easy as possible for the Wolfi team, while still providing a transparent record to our users of our vulnerability statuses along the way.

Implementation considerations:

The existing secfixes-tracker deployment would be an ideal service to poll, since it's proactively tracking vulnerability matches in NVD on our behalf.

Repo guidelines

It would be nice to add some rules and guidance for what should go in this repo.

In other words what is considered non "os"? Would nginx be allowed here? Wordpress? Why?

[Wolfi Package Request]: add openjdk jre headless package

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

need openjdk-11-jre-headless and openjdk-17-jre-headless subpackages.

which versions of the software should we include?

No response

Add some justification for why this specific package and versions are important.

There's a request for it here chainguard-images/images#91

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

[Wolfi Package Request]: Ruby 3.2

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

https://www.ruby-lang.org/en/news/2022/12/25/ruby-3-2-0-released/

which versions of the software should we include?

3.2

Add some justification for why this specific package and versions are important.

No response

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

Move all packages to `./packages/` folder

Currently, the root dir . contains more than 300 YAML files; and increasing over time. This would have some pros/cons:

  • [+] A cleaner repo
  • [+] Make more visible non-packages files (contributing.md, build-scripts, other GitHub specific files, etc.)
  • [+] A low-hanging fruit task (edit Makefile and GitHub workflows for new dir)
  • [-] I'm not sure, but probably would not be a backward-compatible move?

I was wondering if it would be reasonable for creating a new ./packages/ folder and put the all packages into it. Thoughts?

Audit licensing and copyright entries in the packages

In the SBOM you can add a custom license ref, but we're not sure how to handle it in the apk, which will then flow back to the SBOM but without the context.

We ought to do an audit of all licensing and copyright entries in the packages to ensure we are keeping the notices and expressions correct.

[Wolfi Package Request]: yq

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

As YAML user I would like to install yq allowing me to manipulate YAMLs with ease

which versions of the software should we include?

4.30.8

Add some justification for why this specific package and versions are important.

yq is a very handy YAML utility that allows manipulating of YAML with ease. As Kubernetes developer I find it very handy with kube manifests e.g. patching.

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

Enforce best practices for melange YAMLs

  • checked-in packages shouldn't depend on packages.wolfi.dev/os or include its keyring; packages are fetched from that repo and depended on locally during the build
  • every melange YAML should have a corresponding entry in Makefile (would have caught #84)
  • maybe harder to catch without involving dag: no duplicate package or subpackage names (would have caught #85)
  • less importantly, consistently format YAML (mostly to avoid formatting arguments)
  • more generally, melange could fail if it encounters an unknown field -- this would have caught the epcho bug fixed in #84

These checks should be enforced at CI time to prevent bugs from being committed.

This is complicated somewhat by the fact that melange YAML files aren't actually parseable YAML files, they're templates that get Go-templated into parseable YAML. Maybe we could templateize them then YAML-format them, or something else smart we haven't thought of yet.

[Wolfi Package Request]: Dart

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

Dart: https://github.com/dart-lang/

which versions of the software should we include?

No response

Add some justification for why this specific package and versions are important.

No response

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

Split mariadb into subpackages

The current mariadb package is quite large (469M/1.2 GB unpacked). There are some very large binaries in it which can potentially be split out into a subpackage. Just these 4 are over 100 MB:

132M	local/mysql/bin/mariadb-ldb
132M	local/mysql/bin/sst_dump
216M	local/mysql/bin/mariadbd
219M	local/mysql/bin/mariadb-backup

The current apk still packs all manpages and development headers.

Patch patches with provenance/attribution header

After chatting with @kaniini about patches, authors and their origin, we think should improve the way we handle patch files.

When patching sources using patches from upstream or from a third party, we should first copy them to the wolfi repo (see this commit for an example).

After copying them, we should add an attribution header stating their origin.

The header should serve two purposes:

  1. Pay attribution to the author/project that worked to create them
  2. Collect the data we will need to build provenance attestations describing how we built our packages (what data TBD).

In the end, we should write a presubmit to ensure patch files includes the header to enforce it on all PRs.

Deno package!

Description

Deno just released and has full NPM compatibility now. Should we package that?

[Wolfi Package Request]: fluent-bit

What software would you like us to add to wolfi-os. Ideally include a URL to the project and its source.

Fluent Bit: https://github.com/fluent/fluent-bit

https://docs.fluentbit.io/

which versions of the software should we include?

2.0.8+

Add some justification for why this specific package and versions are important.

Most recent supported version, includes logging, metrics and OTEL output now as well.

Please check all that apply

  • This package has an un-restrictive license
  • I am interested in adding this package to Wolfi OS myself.
  • I am willing to help maintain this package.

Refactor APKINDEX generation

Today we generate the APKINDEX like so:

  1. GitHub CI builds and publishes new apks to the source bucket
  2. A Cloud Function watches for new files in the source bucket, which when triggered:
    1. generates an APKINDEX from all the apks in the destination bucket, plus the new file in the source bucket that triggered the function
    2. writes the new apk and the updated APKINDEX to the destination bucket where users consume it
    3. deletes the new apk from the source bucket

The result is that the destination bucket contains all the apks we build, plus an APKINDEX containing all the apks in the bucket.

This has some problems:

  1. When we publish many apks at once, the function triggers for each new apk, regenerating a new APKINDEX repeatedly and repeatedly pulling apks from the bucket to do so. Either the function has to run sequentially for each new apk, or it has to fail and retry when the APKINDEX it's replacing is different from the one it expected (effectively running the process in sequence, with more wasteage)
  2. The source->destination dance is not actually needed, since it's perfectly fine for the destination bucket to contain apks that aren't expressed in the APKINDEX
  3. Having a Cloud Function means maintaining more infrastructure, which is currently private and not transparent to the community
  4. To purge a package from the index today we delete the offending apk in the bucket, which triggers an APKINDEX rebuild, a process which is also not transparent to the community. Purging packages is a destructive action, how and when it happens should be more visible to package consumers.
  5. To rebuild the APKINDEX on-demand (as sometimes we need to do, if there's a bug in APKINDEX generation for example) we delete the APKINDEX in the bucket, which triggers the Cloud Function. This leads to downtime while the APKINDEX is unavailable, which is exacerbated by...
  6. generating the APKINDEX takes longer and longer the more packages we have (chainguard-dev/melange#279), and we're starting to get enough packages that it's becoming painful.

I think we can simplify the above to make it require less infrastructure, make what infrastructure it requires more transparent and auditable, and save time and reduce waste.

  1. Add a method to wolfictl to generate an APKINDEX from the list of apks in a GCS bucket, and sign and publish that APKINDEX back to the bucket
  2. Run this wolfictl command after each build completes and new packages are published. This will add N new packages to the APKINDEX at once, instead of running N times to add one package each.
  3. Run this wolfictl command in GitHub Actions hourly, and on-demand. When the command runs its logs will be publicly visible.

This will let us turndown the source bucket and Cloud Function, and the code that generates the APKINDEX will be public and when it runs its logs will be public. When an APKINDEX is refreshed, it will be replaced in-place, rather than deleted and regenerated.

There are also some optimizations we can make when consuming apks to generate the APKIDNEX, so that index generation doesn't take as long.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.