threefoldtech / 0-initramfs Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 1.0 1.05 MB

Zero-OS Initramfs Builder

License: Apache License 2.0

Shell 100.00%

0-initramfs's People

Contributors

Stargazers

Watchers

Forkers

majjihari

0-initramfs's Issues

static build

@maxux please fill in first comment.

EDIT (maxux):
My idea was compiling every subsoftware we use completely static to not depend on anything on runtime and just update the binary when we want to upgrade.

Compared to current build system, I'm investigating to use portage from Gentoo, which was the base idea few years ago. They have a musl overlay we can reuse to get working code compiling against musl and try to get static build out of it.

build: investigate to use portage as build mechanism

Checking how portage could be used or not in order to support dependencies tracking and package maintainers from Gentoo.

Support ARM build

Define required changes and supported target but support ARM target.
More globally, build should support multiple targets.

kernel: only 128 cpu detected

Kernel from Zero-OS only detect 128 CPU even if there are more.

Update software suite

Check the full software list we use and update what needs to be updated

please add sg_utils and hdparm to 0-OS (no flist)

these are small tools, and do not warrant a full flist

http://sg.danny.cz/sg/sg3_utils.html

https://sourceforge.net/projects/hdparm/files/hdparm/hdparm-9.58.tar.gz/download

in light of being able to unmount disks and spin them down.
For SATA disks there is a go tool that Dylan is integrating, but for scsi disks there is another tool necessary

from sg3_utils, at least sg_start

create a new release as the latest one is outdated

please add dhcpcd and bcache tools

include `bcachefs` and utilities in base image

We are switching to using bcachefs in zos instead of btrfs. We would need both to co-exist on the base image.

autobuild: github action ubuntu version deprecated

GitHub Actions is going to deprecate the ubuntu version we use to build the production initramfs.

To make things easier, the build process rely on couple ubuntu's libraries shipped. Actions will deprecated that version but we can't just move to the next version blindly. Some external binaries shipped by flist for the main OS rely on a specific libc version and are build using same ubuntu version to ensure compatibility.

A better system to keep coherence between build environment and runtime system needs to be used/developped.

In a first step, a workaround to still be able to build the image with the current target version on GitHub Actions will be put in place.

[docs] readme is out of date

The readme is completely out of date and need to be updated to match current state

Review release image

With all news changes, we need to review how release mode works:

Disable automatic debug features
Disable SSH access and keys
etc. (to be improved)

Can't create a vm inside a zos-vm on the top of zos-host

Scenario

1- Create a zos-vm on the top of a zos-host
2- Create a vm on the top of the zos-vm

Actual Result:

zeroos.core0.client.client.ResultError: "failed to create machine: virError(Code=8, Domain=44, Message='invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm ')"

Notes:
Amzy was debugging in this, and here are some notes

1- Nested kvm is not activated on the zos-host

In [22]: cl.bash('cat /sys/module/kvm_intel/parameters/nested').get()
Out[22]: 
STATE: 0 SUCCESS
STDOUT:
N

STDERR:

DATA:

2- Also flag -enable-kvm might need to be enabled

Update debugfs

Seems debugfs image is not compatible with latest version of 0-os/0-fs, update is required.
This could fix #34 btw

Kernel update to latest LTS

I had a look at node 29 and the current kernel is linux 5.10 which released on 2020-12-13
5.10 is LTS and usually only important bugfixes are applied to such kernels and they don't usually see very frequent releases, especially for older LTS tress.

There are three releases under long-term maintenance newer than the one we use, versions 6.6, 6.1 and 5.15.
I suggest to get the most stable and most secure kernel into zos. latest LTS is 6.6 which released on 2023-10-29.

After checking the changes (for example 5.15 have a lot of networking fixes), I expect more stable networking and performance improvements for zos bumping the kernel.

Build using initramfs.sh: GOPATH undefined

Following the readme:

. autobuild/tf-build-deps.sh
. autobuild/tf-build-settings.sh
bash -x initramfs.s

Gives the following output:

+ set -e
+ BUILDMODE=debug
+ DISTFILES=/home/toto/0-initramfs/archives
+ WORKDIR=/home/toto/0-initramfs/staging
+ CONFDIR=/home/toto/0-initramfs/config
+ ROOTDIR=/home/toto/0-initramfs/root
+ TMPDIR=/home/toto/0-initramfs/tmp
+ PKGDIR=/home/toto/0-initramfs/packages
+ EXTENDIR=/home/toto/0-initramfs/extensions
+ PATCHESDIR=/home/toto/0-initramfs/patches
+ TOOLSDIR=/home/toto/0-initramfs/tools
+ MUSLWORKDIR=/home/toto/0-initramfs/staging/musl
+ MUSLROOTDIR=/home/toto/0-initramfs/staging/musl/root
+ MIRRORSRC=https://download.grid.tf/initramfs-mirror/
++ grep -c bogomips /proc/cpuinfo
+ JOBS=9
+ MAKEOPTS='-j 9'
++ getopt -o adbtckMeolmnzrh --long all,download,busybox,tools,cores,kernel,modules,extensions,ork,clean,mrproper,nomirror,compact,release,help -n parse-options --
+ OPTS=' --'
+ '[' 0 '!=' 0 ']'
+ DO_ALL=1
+ USE_MIRROR=1
+ '[' ' --' '!=' ' --' ']'
+ true
+ case "$1" in
+ break
+ modules=0
+ DOWNLOADERS=()
+ EXTRACTORS=()
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/0-fs.sh
++ ZFS_VERSION=0.2.6
++ ZFS_HASH=8d576939d75b12613abdfc1bd18204f6
++ ZFS_BINARY=https://github.com/threefoldtech/rfs/releases/download/v0.2.6/rfs
++ registrar_zfs
++ DOWNLOADERS+=(download_zfs)
+ modules=1
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/bcache.sh
++ BCACHE_VERSION=494f8d187c74f557dfebbb5dc3591453436b507b
++ BCACHE_CHECKSUM=a698ee9ecd6f481070adfd5acfc881b5
++ BCACHE_LINK=https://github.com/koverstreet/bcache-tools/archive/494f8d187c74f557dfebbb5dc3591453436b507b.tar.gz
++ registrar_bcache
++ DOWNLOADERS+=(download_bcache)
++ EXTRACTORS+=(extract_bcache)
+ modules=2
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/btrfs-progs.sh
++ BTRFS_VERSION=4.20.2
++ BTRFS_CHECKSUM=f5487352c734a73c7b1ccded3b126715
++ BTRFS_LINK=https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/btrfs-progs-v4.20.2.tar.xz
++ registrar_btrfs
++ DOWNLOADERS+=(download_btrfs)
++ EXTRACTORS+=(extract_btrfs)
+ modules=3
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/busybox.sh
++ BUSYBOX_VERSION=1.31.0
++ BUSYBOX_CHECKSUM=cdba5d4458f944ceec5cdcf7c4914b69
++ BUSYBOX_LINK=https://www.busybox.net/downloads/busybox-1.31.0.tar.bz2
++ registrar_busybox
++ DOWNLOADERS+=(download_busybox)
++ EXTRACTORS+=(extract_busybox)
+ modules=4
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/ca-certificates.sh
++ CERTS_VERSION=20190110
++ CERTS_CHECKSUM=e19d56b1ec337f0841c8af944b509537
++ CERTS_LINK=http://ftp.fr.debian.org/debian/pool/main/c/ca-certificates/ca-certificates_20190110_all.deb
++ registrar_certs
++ DOWNLOADERS+=(download_certs)
++ EXTRACTORS+=(extract_certs)
+ modules=5
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/corex-musl.sh
++ COREX_MUSL_PKGNAME=corex
++ COREX_MUSL_VERSION=2.1.0
++ COREX_MUSL_CHECKSUM=2062aaca7609a6f50a67d4ef7d6221a7
++ COREX_MUSL_LINK=https://github.com/threefoldtech/corex/archive/2.1.0.tar.gz
++ registrar_corex_musl
++ DOWNLOADERS+=(download_corex_musl)
++ EXTRACTORS+=(extract_corex_musl)
+ modules=6
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/curl.sh
++ CURL_VERSION=7_65_1
++ CURL_CHECKSUM=651706b87c501317030ec317dd84f3ef
++ CURL_LINK=https://github.com/curl/curl/archive/curl-7_65_1.tar.gz
++ registrar_curl
++ DOWNLOADERS+=(download_curl)
++ EXTRACTORS+=(extract_curl)
+ modules=7
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/dhcpcd.sh
++ DHCPCD_VERSION=7.2.2
++ DHCPCD_CHECKSUM=2f17034432ea10415ee84a97ef131128
++ DHCPCD_LINK=https://roy.marples.name/downloads/dhcpcd/dhcpcd-7.2.2.tar.xz
++ registrar_dhcpcd
++ DOWNLOADERS+=(download_dhcpcd)
++ EXTRACTORS+=(extract_dhcpcd)
+ modules=8
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/dmidecode.sh
++ DMIDECODE_VERSION=3.2
++ DMIDECODE_CHECKSUM=9cc2e27e74ade740a25b1aaf0412461b
++ DMIDECODE_LINK=http://ftp.igh.cnrs.fr/pub/nongnu/dmidecode/dmidecode-3.2.tar.xz
++ registrar_dmidecode
++ DOWNLOADERS+=(download_dmidecode)
++ EXTRACTORS+=(extract_dmidecode)
+ modules=9
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/dnsmasq.sh
++ DNSMASQ_VERSION=2.80
++ DNSMASQ_CHECKSUM=e040e72e6f377a784493c36f9e788502
++ DNSMASQ_LINK=http://www.thekelleys.org.uk/dnsmasq/dnsmasq-2.80.tar.xz
++ registrar_dnsmasq
++ DOWNLOADERS+=(download_dnsmasq)
++ EXTRACTORS+=(extract_dnsmasq)
+ modules=10
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/e2fsprogs.sh
++ E2FSPROGS_VERSION=1.45.2
++ E2FSPROGS_CHECKSUM=d15898253dda2e5bce85593022e82432
++ E2FSPROGS_LINK=https://mirrors.edge.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.45.2/e2fsprogs-1.45.2.tar.xz
++ registrar_e2fsprogs
++ DOWNLOADERS+=(download_e2fsprogs)
++ EXTRACTORS+=(extract_e2fsprogs)
+ modules=11
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/ethtool.sh
++ ETHTOOL_VERSION=5.1
++ ETHTOOL_CHECKSUM=fe774357084027e3739f17ad76cbab4d
++ ETHTOOL_LINK=https://mirrors.edge.kernel.org/pub/software/network/ethtool/ethtool-5.1.tar.xz
++ registrar_ethtool
++ DOWNLOADERS+=(download_ethtool)
++ EXTRACTORS+=(extract_ethtool)
+ modules=12
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/eudev.sh
++ EUDEV_VERSION=3.2.9
++ EUDEV_CHECKSUM=e575ef39f66be11a6a5b6c8a169d3c7e
++ EUDEV_LINK=https://github.com/gentoo/eudev/archive/v3.2.9.tar.gz
++ registrar_eudev
++ DOWNLOADERS+=(download_eudev)
++ EXTRACTORS+=(extract_eudev)
+ modules=13
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/haveged.sh
++ HAVEGED_VERSION=1.9.4
++ HAVEGED_CHECKSUM=95867032bb3f2abd36179f92e328e651
++ HAVEGED_LINK=https://github.com/jirka-h/haveged/archive/1.9.4.tar.gz
++ registrar_haveged
++ DOWNLOADERS+=(download_haveged)
++ EXTRACTORS+=(extract_haveged)
+ modules=14
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/iproute2.sh
++ IPROUTE2_VERSION=5.4.0
++ IPROUTE2_CHECKSUM=54d86cadb4cd1d19fd7114b4e53adf51
++ IPROUTE2_LINK=https://www.kernel.org/pub/linux/utils/net/iproute2/iproute2-5.4.0.tar.xz
++ registrar_iproute2
++ DOWNLOADERS+=(download_iproute2)
++ EXTRACTORS+=(extract_iproute2)
+ modules=15
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/kmod.sh
++ KMOD_VERSION=26
++ KMOD_CHECKSUM=1129c243199bdd7db01b55a61aa19601
++ KMOD_LINK=https://www.kernel.org/pub/linux/utils/kernel/kmod/kmod-26.tar.xz
++ registrar_kmod
++ DOWNLOADERS+=(download_kmod)
++ EXTRACTORS+=(extract_kmod)
+ modules=16
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/libcap-musl.sh
++ LIBCAP_MUSL_PKGNAME=libcap
++ LIBCAP_MUSL_VERSION=2.33
++ LIBCAP_MUSL_CHECKSUM=c23bbc02b13d10c3889ef2b1bed34071
++ LIBCAP_MUSL_LINK=https://git.kernel.org/pub/scm/linux/kernel/git/morgan/libcap.git/snapshot/libcap-2.33.tar.gz
++ registrar_libcap_musl
++ DOWNLOADERS+=(download_libcap_musl)
++ EXTRACTORS+=(extract_libcap_musl)
+ modules=17
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/libfuse.sh
++ FUSE_VERSION=2.9.7
++ FUSE_CHECKSUM=91c97e5ae0a40312115dfecc4887bd9d
++ FUSE_LINK=https://github.com/libfuse/libfuse/archive/fuse-2.9.7.tar.gz
++ registrar_fuse
++ DOWNLOADERS+=(download_fuse)
++ EXTRACTORS+=(extract_fuse)
+ modules=18
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/libjsonc-musl.sh
++ JSONC_MUSL_PKGNAME=json-c
++ JSONC_MUSL_VERSION=0.13.1-20180305
++ JSONC_MUSL_CHECKSUM=20dba7bf773599a0842745a2fe5b7cd3
++ JSONC_MUSL_LINK=https://github.com/json-c/json-c/archive/json-c-0.13.1-20180305.tar.gz
++ registrar_jsonc_musl
++ DOWNLOADERS+=(download_jsonc_musl)
++ EXTRACTORS+=(extract_jsonc_musl)
+ modules=19
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/libwebsockets-musl.sh
++ LIBWEBSOCKETS_MUSL_PKGNAME=libwebsockets
++ LIBWEBSOCKETS_MUSL_VERSION=3.2.0
++ LIBWEBSOCKETS_MUSL_CHECKSUM=1d06f5602604e67e6f50cef9857c6b0c
++ LIBWEBSOCKETS_MUSL_LINK=https://github.com/warmcat/libwebsockets/archive/v3.2.0.tar.gz
++ registrar_libwebsockets_musl
++ DOWNLOADERS+=(download_libwebsockets_musl)
++ EXTRACTORS+=(extract_libwebsockets_musl)
+ modules=20
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/linux-firmware.sh
++ FIRMWARE_VERSION=20210716
++ FIRMWARE_CHECKSUM=d27c306e4a15756c7fc9f55250c53295
++ FIRMWARE_LINK=https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/snapshot/linux-firmware-20210716.tar.gz
++ registrar_firmware
++ DOWNLOADERS+=(download_firmware)
++ EXTRACTORS+=(extract_firmware)
+ modules=21
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/linux-kernel.sh
++ KERNEL_VERSION=5.10.55
++ KERNEL_CHECKSUM=10d3f47caac28cf126e6d98ad977b2bb
++ KERNEL_LINK=https://www.kernel.org/pub/linux/kernel/v5.x/linux-5.10.55.tar.xz
++ registrar_kernel
++ DOWNLOADERS+=(download_kernel)
++ EXTRACTORS+=(extract_kernel)
+ modules=22
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/modules.sh
++ MODULES_REPOSITORY=https://github.com/threefoldtech/zos
++ MODULES_VERSION=main
++ registrar_modules
++ DOWNLOADERS+=(download_modules)
++ EXTRACTORS+=(extract_modules)
+ modules=23
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/netcat.sh
++ NETCAT_MAJOR=110
++ NETCAT_VERSION=20180111
++ NETCAT_CHECKSUM=691e734b398bbbe2225feffdd21f63b7
++ NETCAT_LINK=http://gentoo.mirrors.ovh.net/gentoo-distfiles/distfiles/nc110.20180111.tar.xz
++ registrar_netcat
++ DOWNLOADERS+=(download_netcat)
++ EXTRACTORS+=(extract_netcat)
+ modules=24
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/nftables.sh
++ NFTABLES_VERSION=0.9.1
++ NFTABLES_CHECKSUM=e2facbcad6c5d9bd87a0bf5081a31522
++ NFTABLES_LINK=https://www.netfilter.org/projects/nftables/files/nftables-0.9.1.tar.bz2
++ LIBNFTNL_VERSION=1.1.3
++ LIBNFTNL_CHECKSUM=e2a7af0a85c283b2cc837c09635b6bca
++ LIBNFTNL_LINK=http://www.iptables.org/projects/libnftnl/files/libnftnl-1.1.3.tar.bz2
++ LIBMNL_VERSION=1.0.4
++ LIBMNL_CHECKSUM=be9b4b5328c6da1bda565ac5dffadb2d
++ LIBMNL_LINK=https://netfilter.org/projects/libmnl/files/libmnl-1.0.4.tar.bz2
++ registrar_nftables
++ DOWNLOADERS+=(download_nftables)
++ EXTRACTORS+=(extract_nftables)
+ modules=25
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/openssh.sh
++ OPENSSH_VERSION=8.0p1
++ OPENSSH_CHECKSUM=bf050f002fe510e1daecd39044e1122d
++ OPENSSH_LINK=https://ftp.fr.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-8.0p1.tar.gz
++ registrar_openssh
++ DOWNLOADERS+=(download_openssh)
++ EXTRACTORS+=(extract_openssh)
+ modules=26
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/openssl-musl.sh
++ OPENSSL_MUSL_PKGNAME=openssl
++ OPENSSL_MUSL_VERSION=1.1.1d
++ OPENSSL_MUSL_CHECKSUM=3be209000dbc7e1b95bcdf47980a3baa
++ OPENSSL_MUSL_LINK=https://www.openssl.org/source/openssl-1.1.1d.tar.gz
++ registrar_openssl_musl
++ DOWNLOADERS+=(download_openssl_musl)
++ EXTRACTORS+=(extract_openssl_musl)
+ modules=27
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/openssl.sh
++ OPENSSL_VERSION=1.1.1d
++ OPENSSL_CHECKSUM=3be209000dbc7e1b95bcdf47980a3baa
++ OPENSSL_LINK=https://www.openssl.org/source/openssl-1.1.1d.tar.gz
++ registrar_openssl
++ DOWNLOADERS+=(download_openssl)
++ EXTRACTORS+=(extract_openssl)
+ modules=28
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/ork.sh
++ ORK_VERSION=master
++ ORK_LINK=github.com/threefoldtech/0-ork
+ modules=29
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/parted.sh
++ PARTED_VERSION=3.2
++ PARTED_CHECKSUM=0247b6a7b314f8edeb618159fa95f9cb
++ PARTED_LINK=http://ftp.gnu.org/gnu/parted/parted-3.2.tar.xz
++ registrar_parted
++ DOWNLOADERS+=(download_parted)
++ EXTRACTORS+=(extract_parted)
+ modules=30
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/qemu.sh
++ QEMU_REPOSITORY=https://github.com/gigforks/qemu
++ QEMU_BRANCH=zerodb
++ registrar_qemu
++ DOWNLOADERS+=(download_qemu)
++ EXTRACTORS+=(extract_qemu)
+ modules=31
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/redis.sh
++ REDIS_VERSION=5.0.5
++ REDIS_CHECKSUM=2d2c8142baf72e6543174fc7beccaaa1
++ REDIS_LINK=http://download.redis.io/releases/redis-5.0.5.tar.gz
++ registrar_redis
++ DOWNLOADERS+=(download_redis)
++ EXTRACTORS+=(extract_redis)
+ modules=32
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/restic.sh
++ RESTIC_VERSION=0.9.2
++ RESTIC_CHECKSUM=dd08f71f2df5901d6b3e3faf3b4f6d2a
++ RESTIC_LINK=https://github.com/restic/restic/archive/v0.9.2.tar.gz
++ registrar_restic
++ DOWNLOADERS+=(download_restic)
++ EXTRACTORS+=(extract_restic)
+ modules=33
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/rscoreutils.sh
++ RSCOREUTILS_VERSION=91899b34b96da40797846f343f399ca420777c6a
++ RSCOREUTILS_CHECKSUM=b85c3e1328d6469b2a3b02baed7d2a05
++ RSCOREUTILS_LINK=https://github.com/uutils/coreutils/archive/91899b34b96da40797846f343f399ca420777c6a.tar.gz
++ registrar_rscoreutils
++ DOWNLOADERS+=(download_rscoreutils)
++ EXTRACTORS+=(extract_rscoreutils)
+ modules=34
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/rtinfo.sh
++ RTINFO_VERSION=7be022c00d28f4a73da488fa1f7bf8e36f87ea7e
++ RTINFO_CHECKSUM=eff05b3883ad9cdc3a34ee26d8bb7809
++ RTINFO_LINK=https://github.com/maxux/rtinfo/archive/7be022c00d28f4a73da488fa1f7bf8e36f87ea7e.tar.gz
++ LIBRTINFO_VERSION=473fb821e6d474e9fc85043a3778b074e1d79adc
++ LIBRTINFO_CHECKSUM=bd10525b42fe2311c1e1f2c4086dc0a6
++ LIBRTINFO_LINK=https://github.com/maxux/librtinfo/archive/473fb821e6d474e9fc85043a3778b074e1d79adc.tar.gz
++ registrar_rtinfo
++ DOWNLOADERS+=(download_rtinfo)
++ EXTRACTORS+=(extract_rtinfo)
+ modules=35
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/seektime.sh
++ SEEKTIME_VERSION=0.1
++ SEEKTIME_CHECKSUM=f16c0d67e9539219261a406bcd395729
++ SEEKTIME_LINK=https://github.com/threefoldtech/seektime/archive/v0.1.tar.gz
++ registrar_seektime
++ DOWNLOADERS+=(download_seektime)
++ EXTRACTORS+=(extract_seektime)
+ modules=36
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/smartmontools.sh
++ SMARTMON_VERSION=7.0
++ SMARTMON_CHECKSUM=b2a80e4789af23d67dfe1e88a997abbf
++ SMARTMON_LINK=https://netcologne.dl.sourceforge.net/project/smartmontools/smartmontools/7.0/smartmontools-7.0.tar.gz
++ registrar_smartmon
++ DOWNLOADERS+=(download_smartmon)
++ EXTRACTORS+=(extract_smartmon)
+ modules=37
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/tcpdump.sh
++ TCPDUMP_VERSION=4.9.2
++ TCPDUMP_CHECKSUM=9bbc1ee33dab61302411b02dd0515576
++ TCPDUMP_LINK=https://www.tcpdump.org/release/tcpdump-4.9.2.tar.gz
++ LIBPCAP_VERSION=1.9.0
++ LIBPCAP_CHECKSUM=dffd65cb14406ab9841f421732eb0f33
++ LIBPCAP_LINK=https://www.tcpdump.org/release/libpcap-1.9.0.tar.gz
++ registrar_tcpdump
++ DOWNLOADERS+=(download_tcpdump)
++ EXTRACTORS+=(extract_tcpdump)
+ modules=38
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/unionfs-fuse.sh
++ UNIONFS_VERSION=2.0
++ UNIONFS_CHECKSUM=40411d156ea7fa0e7cd0a8ec6fe60e70
++ UNIONFS_LINK=https://github.com/rpodgorny/unionfs-fuse/archive/v2.0.tar.gz
++ registrar_unionfs
++ DOWNLOADERS+=(download_unionfs)
++ EXTRACTORS+=(extract_unionfs)
+ modules=39
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/util-linux.sh
++ LINUXUTILS_VERSION=2.34
++ LINUXUTILS_CHECKSUM=a78cbeaed9c39094b96a48ba8f891d50
++ LINUXUTILS_LINK=https://www.kernel.org/pub/linux/utils/util-linux/v2.34/util-linux-2.34.tar.xz
++ registrar_linuxutil
++ DOWNLOADERS+=(download_linuxutil)
++ EXTRACTORS+=(extract_linuxutil)
+ modules=40
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/wireguard.sh
++ WIREGUARD_TOOLS_VERSION=1.0.20200102
++ WIREGUARD_TOOLS_CHECKSUM=611eb05e295550f8267092bbf2731fd1
++ WIREGUARD_TOOLS_LINK=https://git.zx2c4.com/wireguard-tools/snapshot/wireguard-tools-1.0.20200102.tar.xz
++ WIREGUARD_MODULES_VERSION=0.0.20200105
++ WIREGUARD_MODULES_CHECKSUM=c2f2d5803623a71a086eab41c29d9b15
++ WIREGUARD_MODULES_LINK=https://git.zx2c4.com/wireguard-linux-compat/snapshot/wireguard-linux-compat-0.0.20200105.tar.xz
++ registrar_wireguard
++ DOWNLOADERS+=(download_wireguard)
++ EXTRACTORS+=(extract_wireguard)
+ modules=41
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/xfsprogs.sh
++ XFSPROGS_VERSION=5.4.0
++ XFSPROGS_CHECKSUM=61232b1cc453780517d9b0c12ff1699b
++ XFSPROGS_LINK=https://mirrors.edge.kernel.org/pub/linux/utils/fs/xfs/xfsprogs/xfsprogs-5.4.0.tar.xz
++ registrar_xfsprogs
++ DOWNLOADERS+=(download_xfsprogs)
++ EXTRACTORS+=(extract_xfsprogs)
+ modules=42
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/zflist.sh
++ ZFLIST_VERSION=2.0.0-rc1
++ ZFLIST_CHECKSUM=c4dc7b9165c2024eaa0bc24cd26cb26a
++ ZFLIST_LINK=https://github.com/threefoldtech/0-flist/archive/v2.0.0-rc1.tar.gz
++ CAPNPC_VERSION=0.3
++ CAPNPC_CHECKSUM=c1836601d210c14a4a88ed55e0b7c6de
++ CAPNPC_LINK=https://github.com/opensourcerouting/c-capnproto/releases/download/c-capnproto-0.3/c-capnproto-0.3.tar.xz
++ registrar_zflist
++ DOWNLOADERS+=(download_zflist)
++ EXTRACTORS+=(extract_zflist)
+ modules=43
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/zinit.sh
++ ZINIT_VERSION=0.2.11
++ ZINIT_HASH=e1d6a6f0ff604e58735cd7bd127e5c78
++ ZINIT_BINARY=https://github.com/threefoldtech/zinit/releases/download/v0.2.11/zinit
++ registrar_zinit
++ DOWNLOADERS+=(download_zinit)
+ modules=44
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/zlib-musl.sh
++ ZLIB_MUSL_PKGNAME=zlib
++ ZLIB_MUSL_VERSION=1.2.11
++ ZLIB_MUSL_CHECKSUM=85adef240c5f370b308da8c938951a68
++ ZLIB_MUSL_LINK=https://www.zlib.net/zlib-1.2.11.tar.xz
++ registrar_zlib_musl
++ DOWNLOADERS+=(download_zlib_musl)
++ EXTRACTORS+=(extract_zlib_musl)
+ modules=45
+ for module in "${PKGDIR}"/*.sh
+ . /home/toto/0-initramfs/packages/zlib.sh
++ ZLIB_PKGNAME=zlib
++ ZLIB_VERSION=1.2.11
++ ZLIB_CHECKSUM=85adef240c5f370b308da8c938951a68
++ ZLIB_LINK=https://www.zlib.net/zlib-1.2.11.tar.xz
++ registrar_zlib
++ DOWNLOADERS+=(download_zlib)
++ EXTRACTORS+=(extract_zlib)
+ modules=46
+ green='\033[32;1m'
+ orange='\033[33;1m'
+ blue='\033[34;1m'
+ cyan='\033[36;1m'
+ white='\033[37;1m'
+ clean='\033[0m'
+ main
+ info ===============================
+ echo -e '\033[34;1m===============================\033[0m'
�[34;1m===============================�[0m
+ info '=  Zero-OS Initramfs Builder  ='
+ echo -e '\033[34;1m=  Zero-OS Initramfs Builder  =\033[0m'
�[34;1m=  Zero-OS Initramfs Builder  =�[0m
+ info ===============================
+ echo -e '\033[34;1m===============================\033[0m'
�[34;1m===============================�[0m
+ echo ''

+ prepare
+ '[' '!' -d /home/toto/0-initramfs/config ']'
+ '[' -z ']'
+ command -v go
+ echo '[-] variable GOPATH not defined'
[-] variable GOPATH not defined
+ exit 1

But the path is defined:

toto@toto:~/0-initramfs$ echo $GOPATH
/home/toto/go

Tested on Ubuntu 22.04.1 LTS x86 & in a docker running Ubuntu 18.04 on an m1 mac (see my last comment in #67).

bump iproute2

the latest & greatest incorporates ip link add test link enp6s0f1 type macvlan mode passthru
right now , mode passthru barfs.

we need passthru if we want to connect a macvlan as a bidirectional slave of a bridge (i.e. properly arps)

Make zerotier build with controller

make ZT_ENABLE_NETWORK_CONTROLLER=1

Add smem to 0-OS

smem is a tool to properly see real memory usage per process, most importantly the relation between RSS PSS and USS

https://www.selenic.com/smem/

Build using docker container does not work: no output in staging

Following: https://github.com/threefoldtech/0-initramfs#build-using-a-docker-container

Exact steps taken:

docker run --platform=linux/amd64 -ti -v "/Users/toto/dev/naiein/threefold/0-initramfs":/0-initramfs ubuntu:18.04 /bin/bash
cd 0-initramfs/
autobuild/tf-build.sh

These are the final output lines:

Processing triggers for ca-certificates (20211016ubuntu0.18.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  117M  100  117M    0     0  14.5M      0  0:00:08  0:00:08 --:--:-- 16.0M
/usr/bin/head: error reading '/proc/self/exe': Bad file descriptor
/usr/bin/head: failed to close '/proc/self/exe': Bad file descriptor
rustup: unknown platform bitness
/bin/sh: 375: [: Illegal number:
info: downloading installer
info: profile set to 'default'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2023-03-28, rust version 1.68.2 (9eb3afe9e 2023-03-27)
info: downloading component 'cargo'
  6.7 MiB /   6.7 MiB (100 %)   3.6 MiB/s in  2s ETA:  0s
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 19.5 MiB /  19.5 MiB (100 %)   8.7 MiB/s in  2s ETA:  0s
info: downloading component 'rust-std'
 29.9 MiB /  29.9 MiB (100 %)  13.1 MiB/s in  2s ETA:  0s
info: downloading component 'rustc'
 68.1 MiB /  68.1 MiB (100 %)  19.4 MiB/s in  3s ETA:  0s
info: downloading component 'rustfmt'
info: installing component 'cargo'
  6.7 MiB /   6.7 MiB (100 %)   6.0 MiB/s in  1s ETA:  0s
info: installing component 'clippy'
info: installing component 'rust-docs'
 19.5 MiB /  19.5 MiB (100 %)   1.9 MiB/s in 14s ETA:  0s
info: installing component 'rust-std'
 29.9 MiB /  29.9 MiB (100 %)   4.9 MiB/s in  6s ETA:  0s
info: installing component 'rustc'
 68.1 MiB /  68.1 MiB (100 %)   6.2 MiB/s in 11s ETA:  0s
info: installing component 'rustfmt'
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'

  stable-x86_64-unknown-linux-gnu installed - rustc 1.68.2 (9eb3afe9e 2023-03-27)


Rust is installed now. Great!

To get started you may need to restart your current shell.
This would reload your PATH environment variable to include
Cargo's bin directory ($HOME/.cargo/bin).

To configure your current shell, run:
source "$HOME/.cargo/env"
info: downloading component 'rust-std' for 'x86_64-unknown-linux-musl'
info: installing component 'rust-std' for 'x86_64-unknown-linux-musl'
 43.5 MiB /  43.5 MiB (100 %)   4.8 MiB/s in  8s ETA:  0s

Nothing in staging (expected: staging/vmlinuz.efi)

Any idea what's going wrong?

zos2: cgroups fails on mounting hugetlb in qemu

Can I please have bmon in the nodes ?

for a quick glance at network usage and pps, bmon would be a great tool... it's tiny but has immediate value if things look fishy on the node's network

0-flist package used is an rc version

the version used is "2.0.0-rc1" while "2.0.0" is available and there already is a "2.0.1" version

kernel: append 0-initramfs commit id into version

In order to easily keep track of the running version of initramfs, appending the version to the kernel would be a great help

iPXE IPv6 setting

for creating an iPXE bootloader for zOSv2, we'll need the iPXE lkrn with

#define NET_PROTO_IPV4          /* IPv4 protocol */
#define NET_PROTO_IPV6          /* IPv6 protocol */

in config/general.h

so when a machine gets an IP addr via SLAAC and has a proper RouterAdvertisement Daemon (dnsmasq or radvd) on the SubnetZeroPrefix, we can boot a machine with IPv6.

then it can register itself.

I suspect we'll need to update bootstrap.grid.tf for that too

Oops... no firmwares?

A lot of hardware drivers load firmware into cards prior to initializing them
Much needed would be FW for NICs and probably some for (i)scsi controllers.

/lib/firmware tends to be quite large from the onset, so we'll need to be selective

To wit :
bnx* brcm* cxgb* myri* mellanox* netronome* qat* qed* ql2* qlogic* rt* RTL* ti* (but without rtlwifi)

I suspect some FW for scsi controllers will still be missing, we'll see

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.