lxc / distrobuilder Goto Github PK
View Code? Open in Web Editor NEWSystem container image builder for LXC and Incus
Home Page: https://linuxcontainers.org
License: Apache License 2.0
System container image builder for LXC and Incus
Home Page: https://linuxcontainers.org
License: Apache License 2.0
For now, when the keyserver is empty in the yaml, definition.go set a default value "hkps.pool.sks-keyservers.net" for keyserver, but the keys argument is not sanitized anymore since 6c4c7ca, allowing this field to be empty if the value is not specified by the yaml used as argument of distrobuilder.
In my system (Fedora 27), gpg fails if keys isn't informed, and so the distrobuild returns an error.
My question is: Should keyserver also be permitted to be empty, and VerifyFile should return true if both keyserver and keys are empty? Or somehow these two parameters are mandatory, and it should be enforced by definition.go?
One way or another, I'm available to send a patch if this behavior needs to change.
Hi folks
When building the package, the following shows up:
import cycle not allowed
root@srv1:~/go/src/distrobuilder# make
gofmt -s -w .
go get -t -v -d ./...
import cycle not allowed
package distrobuilder/distrobuilder
imports bufio
imports bytes
imports errors
imports runtime
imports internal/bytealg
imports internal/cpu
imports runtime
Makefile:4: recipe for target 'default' failed
make: *** [default] Error 1
I have just tried to build a container for Debian stretch armel:
lxc-create -n testlxcusr -f testlxcusr.conf -t download -- --dist debian --arch armel --release stretch
And I have been given armhf binaries, which is puzzling:
You just created a Debian stretch armhf (20190617_05:24) container.
And it would seem to be so:
readelf -h testlxcusr/rootfs/bin/tar|grep Flags
Flags: 0x5000400, Version5 EABI, hard-float ABI
Here's a trace from the executing template which suggests that it was looking in the right place for armel binaries:
wget_wrapper -T 30 -q https://images.linuxcontainers.org//images/debian/stretch/armel/default/20190617_05:24//meta.tar.xz.asc -O /tmp/tmp.uKsle76i5a/meta.tar.xz.asc
So - is there something amiss with the build process ? Or should I have done something differently to get hold of the armel build ?
LXC version is 2.0.7.
Top section of the generated config:
# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d debian -a armel -r stretch
# Template script checksum (SHA-1): 740c51206e35463362b735e68b867876048a8baf
# For additional config options, please look at lxc.container.conf(5)
# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)
# Distribution configuration
lxc.include = /usr/share/lxc/config/debian.common.conf
lxc.include = /usr/share/lxc/config/debian.userns.conf
lxc.arch = linux32
The config file I gave it:
lxc.network.type = veth
#lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = fe:06:ae:7f:c6:38
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
In production we pretty often see build hangs, so far that mostly affects Alpine getting stuck somewhere in apk.
Could we get a --timeout
argument where we can feed a value (probably in seconds) after which distrobuilder kills whatever it's doing and exits?
For us, I'd probably then pass 30min for the build-dir stage and no timeout for the rest, that would take care of such issues and allow Jenkins to automatically retry things.
# go get -d -v github.com/lxc/distrobuilder
github.com/lxc/distrobuilder (download)
created GOPATH=/root/go; see 'go help gopath'
package github.com/lxc/distrobuilder: no Go files in /root/go/src/github.com/lxc/distrobuilder
We need a mechanism to seamlessly create and publish lxc containers . COPY/ADD generator to copy files/folders from local repository inside container rootfs. This will allow publishing lxc apps from git as part of CI/CD run.
Hello:
Is it possible to provide cloud-init packages with images at images.linuxcontainers.org?
Maybe this is too difficult to achieve or people use other methods to provision systems other than Ubuntu from cloud-images.ubuntu.com.
Thanks.
For distrobuilder to be practical with the way we build images, I'd like to be able to have a single yaml that I can use for all images of a given distro, covering all releases and all architectures.
I think the easiest way to do so is to avoid for key=value type override of yaml keys on the command line.
I'd therefore be able to do:
distrobuilder build-dir ubuntu.yaml new -o image.release=bionic -o image.description="Ubuntu 18.04 LTS" -o image.architecture=armhf -o source.url=http://us.ports.ubuntu.com/ubuntu-ports
Which would then give me an Ubuntu 18.04 version of the armhf image, using the proper archive mirror for the architecture.
I wrote code that does the right type guessing to go from a key=value string to a struct member in the past, you can find it here: https://github.com/nsec/askgod/blob/master/cmd/askgod/utils.go#L12
Building distrobuilder from the master branch on Archlinux gives me:
[mrakotomandimby@arch-00 distrobuilder]$ make
gofmt -s -w .
go get -t -v -d ./...
go install -v ./...
go install: no install location for directory /home/mrakotomandimby/distrobuilder/distrobuilder outside GOPATH
For more details see: 'go help gopath'
go install: no install location for directory /home/mrakotomandimby/distrobuilder/generators outside GOPATH
For more details see: 'go help gopath'
go install: no install location for directory /home/mrakotomandimby/distrobuilder/image outside GOPATH
For more details see: 'go help gopath'
go install: no install location for directory /home/mrakotomandimby/distrobuilder/managers outside GOPATH
For more details see: 'go help gopath'
go install: no install location for directory /home/mrakotomandimby/distrobuilder/shared outside GOPATH
For more details see: 'go help gopath'
go install: no install location for directory /home/mrakotomandimby/distrobuilder/sources outside GOPATH
For more details see: 'go help gopath'
make: *** [Makefile:6: default] Error 1
Go version
[mrakotomandimby@arch-00 distrobuilder]$ go version
go version go1.10.1 linux/amd64
[mrakotomandimby@arch-00 distrobuilder]$
We don't want to have to update our .yaml every month or so, it'd be good if distrobuilder could select the most recent ISO by default when source.release is empty.
Title says it all, we need tests for lxc/lxd image generation and for the file templating code.
For the Ubuntu images at least, the expectation is that the "description" and "architecture" properties in the LXD image include the architecture name as called by the distribution (so amd64 instead of x86_64), but right now I see now way to access this from pongo2.
I'd like two new file "generators":
This should let us easily take care of a good number of simple modifications to the rootfs (mostly /etc files) while avoiding the use of actions for those things.
We seem to be missing support for building Plamo Linux containers and it's one of the few remaining distros to switch over.
We need to extend the yaml to include a new template key:
source:
downloader: debootstrap
url: http://us.archive.ubuntu.com/ubuntu
keys:
- 0x790BC7277767219C42C86F933B4FE6ACC0B21F32
apt_sources: |-
deb http://{{ source.url }} {{ image.release }} main restricted universe multiverse
deb http://{{ source.url }} {{ image.release }}-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu {{ image.release }}-security main restricted universe multiverse
This should then be templated onto "/etc/apt/sources.list" inside the post-debootstrap chroot.
We also need to make sure that "apt-get update" is executed inside the chroot prior to any other package management action happening after templating.
Hi,
Since #192 , our GitLab CI is failing to build images. strace
-ing distrobuilder
, the keys are correctly fetched, but they are not being picked up. This step fails for Ubuntu-based images as well as Alpine-based ones.
Create a repository on GitLab (gitlab.com suffices), put an image template of your choice and the following .gitlab-ci.yml
file in it. Commit, push and start the CI.
# .gitlab-ci.yml
image: ubuntu:18.04
before_script:
- apt-get update
- apt-get -y install git-core golang-go debootstrap rsync gpg squashfs-tools
- 'go get -d -v github.com/lxc/distrobuilder || :'
- cd ~/go/src/github.com/lxc/distrobuilder
- make
- cd "${CI_PROJECT_DIR}"
image:
stage: build
script:
- ~/go/bin/distrobuilder build-lxd image-of-your-choice.yml
Here are relevant trace output lines for an Alpine-based image:
[pid 6739] <... read resumed> "gpg: key 293ACD0907D9495A: 2 duplicate signatures removed\ngpg: key 293ACD0907D9495A: 11 signatures not checked due to missing keys\ngpg: /tmp/distrobuilder.071160561/trustdb.gpg: trustdb created\ngpg: key 293ACD0907D9495A: public key \"Natanael Copa <[email protected]>\" imported\n", 1473) = 281
[pid 6743] read(9, "gpg: key 293ACD0907D9495A: 2 duplicate signatures removed\ngpg: key 293ACD0907D9495A: 11 signatures not checked due to missing keys\ngpg: key 293ACD0907D9495A: \"Natanael Copa <[email protected]>\" not changed\ngpg: Total number processed: 1\ngpg: unchanged: 1\n", 512) = 272
[pid 6750] write(2, "Error: Error while downloading source: Failed to import keys: 0482D84022F52DF1C4E7CD43293ACD0907D9495A\n", 103Error: Error while downloading source: Failed to import keys: 0482D84022F52DF1C4E7CD43293ACD0907D9495A ) = 103
This matches an execution where the key is correctly fetched and ignored two (2) times.
Arch Linux fails to build, using following YAML:
image:
distribution: archlinux
release: 2019.07.01
description: Arch Linux
expiry: 30d
architecture: x86_64
source:
downloader: archlinux-http
url: https://archive.archlinux.org/iso
keys:
- 4AA4767BBC9C4B1D18AE28B77F2D434B9741E8AC
targets:
lxc:
create-message: |
You just created an Arch Linux container (release={{ image.release }}, arch={{ image.architecture }})
config:
- type: all
before: 5
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/archlinux.common.conf
- type: user
before: 5
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/archlinux.userns.conf
- type: all
after: 4
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/common.conf
- type: user
after: 4
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/userns.conf
- type: all
content: |-
lxc.arch = {{ image.architecture_kernel }}
files:
- name: hostname
path: /etc/hostname
generator: hostname
- name: hosts
path: /etc/hosts
generator: hosts
packages:
manager: pacman
update: true
cleanup: true
sets:
- packages:
- systemd-sysvcompat
- neovim
action: install
$ sudo distrobuilder build-lxd archlinux.yaml
/tmp/archlinux-2019.07.01-x86_64/archlinux-bootstrap-2019.07.01-x86_64.tar.gz: 100% (6.21MB/s)
gpg: /etc/pacman.d/gnupg/trustdb.gpg: trustdb created
gpg: no ultimately trusted keys found
gpg: starting migration from earlier GnuPG versions
gpg: porting secret keys from '/etc/pacman.d/gnupg/secring.gpg' to gpg-agent
gpg: migration succeeded
gpg: Generating pacman keyring master key...
gpg: key D4298857ED8932B9 marked as ultimately trusted
gpg: directory '/etc/pacman.d/gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/etc/pacman.d/gnupg/openpgp-revocs.d/492044C0453E653D69D37B71D4298857ED8932B9.rev'
gpg: Done
==> Updating trust database...
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
==> Appending keys from archlinux.gpg...
==> Locally signing trusted keys in keyring...
-> Locally signing key D8AFDDA07A5B6EDFA7D8CCDAD6D055F927843F1C...
-> Locally signing key DDB867B92AA789C165EEFA799B729B06A680C281...
-> Locally signing key 684148BB25B49E986A4944C55184252D824B18E8...
-> Locally signing key 91FFE0700E80619CEB73235CA88E23E377514E00...
-> Locally signing key 0E8B644079F599DFC1DDC3973348882F6AC6A4C2...
-> Locally signing key AB19265E5D7D20687D303246BA1DFB64FFF979E7...
==> Importing owner trust values...
gpg: inserting ownertrust of 4
gpg: setting ownertrust to 4
gpg: setting ownertrust to 4
gpg: setting ownertrust to 4
gpg: setting ownertrust to 4
gpg: setting ownertrust to 4
==> Disabling revoked keys in keyring...
-> Disabling key 8F76BEEA0289F9E1D3E229C05F946DED983D4366...
-> Disabling key 63F395DE2D6398BBE458F281F2DBB4931985A992...
-> Disabling key 50F33E2E5B0C3D900424ABE89BDCF497A4BBCC7F...
-> Disabling key 27FFC4769E19F096D41D9265A04F9397CDFD6BB0...
-> Disabling key 39F880E50E49A4D11341E8F939E4F17F295AFBF4...
-> Disabling key 8840BD07FC24CB7CE394A07CCF7037A4F27FB7DA...
-> Disabling key 5559BC1A32B8F76B3FCCD9555FA5E5544F010D48...
-> Disabling key 0B20CA1931F5DA3A70D0F8D2EA6836E1AB441196...
-> Disabling key 07DFD3A0BC213FA12EDC217559B3122E2FA915EC...
-> Disabling key 4FCF887689C41B09506BE8D5F3E1D5C5D30DB0AD...
-> Disabling key 5A2257D19FF7E1E0E415968CE62F853100F0D0F0...
-> Disabling key 7FA647CD89891DEDC060287BB9113D1ED21E1A55...
-> Disabling key BC1FBE4D2826A0B51E47ED62E2539214C6C11350...
-> Disabling key 4A8B17E20B88ACA61860009B5CED81B7C2E5C0D2...
-> Disabling key 5696C003B0854206450C8E5BE613C09CB4440678...
-> Disabling key 8CF934E339CAD8ABF342E822E711306E3C4F88BC...
-> Disabling key F5A361A3A13554B85E57DDDAAF7EF7873CFD4BB6...
-> Disabling key 5E7585ADFF106BFFBBA319DC654B877A0864983E...
-> Disabling key 65EEFE022108E2B708CBFCF7F9E712E59AF5F22A...
-> Disabling key 40440DC037C05620984379A6761FAD69BA06C6A9...
-> Disabling key 34C5D94FE7E7913E86DC427E7FB1A3800C84C0A5...
-> Disabling key 81D7F8241DB38BC759C80FCE3A726C6170E80477...
-> Disabling key E7210A59715F6940CF9A4E36A001876699AD6E84...
-> Disabling key 5357F3B111688D88C1D88119FCF2CB179205AC90...
-> Disabling key FB871F0131FEA4FB5A9192B4C8880A6406361833...
-> Disabling key 66BD74A036D522F51DD70A3C7F2A16726521E06D...
-> Disabling key B1F2C889CB2CCB2ADA36D963097D629E437520BD...
-> Disabling key 9515D8A8EAB88E49BB65EDBCE6B456CAF15447D5...
-> Disabling key 40776A5221EF5AD468A4906D42A1DB15EC133BAD...
-> Disabling key D4DE5ABDE2A7287644EAC7E36D1A9E70E19DAA50...
-> Disabling key 44D4A033AC140143927397D47EFD567D4C7EA887...
==> Updating trust database...
gpg: key 1EB2638FF56C0C53: no user ID for key signature packet of class 10
gpg: key 1EB2638FF56C0C53: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: key 786C63F330D7CB92: no user ID for key signature packet of class 10
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 1 signed: 6 trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1 valid: 6 signed: 75 trust: 0-, 0q, 0n, 6m, 0f, 0u
gpg: depth: 2 valid: 75 signed: 7 trust: 75-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2019-10-03
:: Synchronizing package databases...
error: failed retrieving file 'core.db' from mirrors.kernel.org : Could not resolve host: mirrors.kernel.org
error: failed to update core (invalid url for server)
error: failed retrieving file 'extra.db' from mirrors.kernel.org : Could not resolve host: mirrors.kernel.org
error: failed to update extra (invalid url for server)
error: failed retrieving file 'community.db' from mirrors.kernel.org : Could not resolve host: mirrors.kernel.org
error: failed to update community (invalid url for server)
error: failed to synchronize all databases
Error: Failed to manage packages: exit status 1
There is not much documentation available, i.e. how to pass files to distrobuilder, etc etc.
The README mentions how to create. But there is no indication on how to use the created result: How to use it as LXC container?
I'm currently trying and if I succed, I will obviusly report here.
I just tried running multiple instances of distrobuilder at the same time and ran into issues because of /var/cache/distrobuilder. We should make it so that this doesn't happen. It should be possible to build multiple images at the same time.
It's unclear to me why we must change the environment of distrobuilder during init.
I can see how having a fixed value for PATH, SHELL, TERM, ... may be useful when calling the package managers, but then that should probably be done through an exec wrapper rather than modify distrobuilder's own environment.
We seem to be missing support for building Ubuntu Core containers and it's one of the few remaining distros to switch over.
This one will be slightly tricky as we need to inject our fake bootloader and some other data and I still need to figure out what's needed for core 18, so I'll be taking this one myself.
This came up on the lxc-users mailing list. We should add Fedora support or encourage the community to work on Fedora support. :)
Hi, I tried to build the centos LXC container and it turns out I can not manage to verify the sources using GPG and proceed to the build of the container without using skip_verification: true
.
I built distrobuilder on a clean CentOS 7 vm and it seems gpg
can hang sometimes and fail:
$ sudo /usr/local/bin/distrobuilder build-lxc centos
Error: Error while downloading source: Failed to verify: gpg: no valid OpenPGP data found.
gpg: the signature could not be verified.
Please remember that the signature file (.sig or .asc)
should be the first file given on the command line.
If I interrupt distrobuilder while it waits for gpg
(before I get any error message), the temporary folder is not destroyed and I have these files:
$ sudo ls /tmp/distrobuilder.157743850/
pubring.gpg secring.gpg
Is not distrobuilder.gpg
and trustdb.gpg
supposed to be in this directory?
I manage to make it work by typing gpg
commands myself, but I suspect an issue.
The following produces the expected result:
mkdir -p test && sudo chown root test && sudo chmod 0700 test && sudo gpg --homedir ./test/ --keyserver hkps://keyserver.ubuntu.com --recv-keys '0946FCA2C105B9DE'
sudo gpg --homedir ./test/ --export --output test/distrobuilder.gpg
ArchLinux, Gentoo and Sabayon at least don't have a release.
CLIs usually have a way to present the user with their version.
We should add a --version
and -v
options to report the installed version of distrobuilder.
I think those 3 keys need to have templating enabled. I'd recommend using pongo2 (flosch/pongo2) as we do in LXD, exporting the "image" struct to it so that I can do:
lxc:
create-message: |-
You just created an {% image.Description %} container (release={% image.Release %}, arch={% image.Arch %}, variant={% image.Variant %})
...
config: |
lxc.include = LXC_TEMPLATE_CONFIG/ubuntu.common.conf
lxc.arch = {% image.Arch %}
config-user: |
lxc.include = LXC_TEMPLATE_CONFIG/ubuntu.common.conf
lxc.include = LXC_TEMPLATE_CONFIG/ubuntu.userns.conf
lxc.arch = {% image.Arch %}
We seem to be missing support for building OpenSUSE containers and it's one of the few remaining distros to switch over.
Hi,
two days ago, I have followed the howto to build distrobuilder and make my own centos 7.5 image.
I needed 7.5, because the application I want to install is not yet compatible to 7.6...
Building the image was successfully, but it was not able to start. I got this error:
lxc info --show-log local:fschmid-centos-lxd-v1
Name: fschmid-centos-lxd-v1
Remote: unix://
Architecture: x86_64
Created: 2019/01/16 15:46 UTC
Status: Stopped
Type: persistent
Profiles: default
Log:
lxc fschmid-centos-lxd-v1 20190116154702.966 WARN conf - conf.c:lxc_setup_devpts:1616 - Invalid argument - Failed to unmount old devpts instance
lxc fschmid-centos-lxd-v1 20190116154702.967 ERROR start - start.c:start:2028 - No such file or directory - Failed to exec "/sbin/init"
lxc fschmid-centos-lxd-v1 20190116154702.967 ERROR sync - sync.c:__sync_wait:62 - An error occurred in another process (expected sequence number 7)
lxc fschmid-centos-lxd-v1 20190116154702.967 WARN network - network.c:lxc_delete_network_priv:2589 - Operation not permitted - Failed to remove interface "eth0" with index 20
lxc fschmid-centos-lxd-v1 20190116154702.967 ERROR lxccontainer - lxccontainer.c:wait_on_daemonized_start:842 - Received container state "ABORTING" instead of "RUNNING"
lxc fschmid-centos-lxd-v1 20190116154702.967 ERROR start - start.c:__lxc_start:1939 - Failed to spawn container "fschmid-centos-lxd-v1"
lxc 20190116154702.972 WARN commands - commands.c:lxc_cmd_rsp_recv:132 - Connection reset by peer - Failed to receive response for command "get_state"
My second problem:
After fixing first issue, I saw, that the image is missing two important templates:
Here is my centos.yaml file:
image:
distribution: centos
release: 7
variant: Minimal
description: CentOS {{ image.release }}
expiry: 3000d
architecture: x86_64
source:
downloader: centos-http
url: http://ownrepo/prod/centos/
keys:
- 24C6A8A7F4A80EB5
- 0946FCA2C105B9DE
variant: Minimal
# variant: default
skip_verification: true
targets:
lxc:
create-message: |
You just created a CentOS container (release={{ image.release }}, arch={{ image.architecture }})
config:
- type: all
before: 5
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/centos.common.conf
- type: user
before: 5
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/centos.userns.conf
- type: all
after: 4
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/common.conf
- type: user
after: 4
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/userns.conf
- type: all
content: |-
lxc.arch = {{ image.architecture_kernel }}
files:
- name: hostname
path: /etc/hostname
generator: hostname
- name: hosts
path: /etc/hosts
generator: hosts
packages:
manager: yum
update: false
cleanup: true
My fix for the first problem (image is not bootable):
197 yum --installroot=/rootfs --disablerepo=* --enablerepo=cdrom -y --releasever=%s install basesystem centos-release yum systemd binutils crontabs dhclient glibc-common iputils json logrotate openssh-clients openssh-server passwd rpm-python rsyslog tzdata vim-minimal policycoreutils json-c audit authconfig file postfix gettext net-tools rootfiles
Maybe not all of them are necessary, but most of them are. At least systemd was missing
My fix for the second problem (no networing inside contianer):
With this, I also have networking inside the container.
What I have seen, is, that there have been quite a lot changes in the last two weeks to these files:
BR Florian Schmid
Users currently need to run distrobuilder
with sudo
for building images, I guess for having the right owner/permissions in the root fs. This makes it very unconvenient for integrating distrobuilder
in automated environments or with other tools like Terraform.
I was wondering if it would be posssible to drop this requirement, maybe by unshare
ing some namespaces before starting the build...
Void is a general purpose operating system, based on the monolithic Linux® kernel. Its package system allows you to quickly install, update and remove software; software is provided in binary packages or can be built directly from sources with the help of the XBPS source packages collection.
It is available for the Intel x86®, ARM® and MIPS® processor architectures; Software packages can be built natively or cross compiling through the XBPS source packages collection.
See https://github.com/lxc/lxc-templates/blob/master/templates/lxc-voidlinux.in
Small hint: That template seems to incorrectly link to voidlinux.eu
instead of voidlinux.org
(domain shift).
hey
would like to start using distrobuilder, how do i begin?
On Fedora Core 30 the installation (make) failes at the line "go install -v ./..." with:
go install: no install location for directory XXX outside GOPATH
Solution is to execute this before:
mkdir $GOPATH/bin && export GOBIN=$GOPATH/bin
On Fedora Core 30 the lxd-build fails at mounting filesystems.
# mkdir -p $HOME/ContainerImages/fedora/
# cd $HOME/ContainerImages/fedora/
# cp $HOME/go/src/github.com/lxc/distrobuilder/doc/examples/fedora fedora.yaml
# sed -i 's/release: 28/release: 30/' fedora.yaml
# $HOME/go/bin/distrobuilder build-lxd fedora.yaml
Fedora Modular 30 - x86_64 1.2 MB/s | 2.3 MB 00:01
Fedora Modular 30 - x86_64 - Updates 131 B/s | 257 B 00:01
Fedora Modular 30 - x86_64 - Test Updates 703 kB/s | 3.1 MB 00:04
Fedora 30 - x86_64 - Test Updates 2.3 MB/s | 14 MB 00:05
Fedora 30 - x86_64 - Updates 386 B/s | 257 B 00:00
Fedora 30 - x86_64 4.3 MB/s | 54 MB 00:12
Metadata cache created.
Failed to set locale, defaulting to C
Last metadata expiration check: 0:00:58 ago on Sun Apr 14 20:49:20 2019.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Upgrading:
dbus-broker x86_64 20-2.fc30 updates-testing 160 k
langpacks-en noarch 1.0-17.fc30 updates-testing 8.2 k
systemd x86_64 241-6.gita2eaa1c.fc30 updates-testing 3.6 M
systemd-libs x86_64 241-6.gita2eaa1c.fc30 updates-testing 473 k
systemd-pam x86_64 241-6.gita2eaa1c.fc30 updates-testing 150 k
systemd-rpm-macros noarch 241-6.gita2eaa1c.fc30 updates-testing 22 k
Installing dependencies:
xkeyboard-config noarch 2.24-5.fc30 fedora 822 k
Installing weak dependencies:
libxkbcommon x86_64 0.8.3-1.fc30 fedora 108 k
Transaction Summary
================================================================================
Install 2 Packages
Upgrade 6 Packages
Total download size: 5.3 M
Downloading Packages:
(1/8): systemd-libs-241-5.git3d835d0.fc30_241-6 88 kB/s | 29 kB 00:00
(2/8): systemd-pam-241-5.git3d835d0.fc30_241-6. 48 kB/s | 19 kB 00:00
(3/8): libxkbcommon-0.8.3-1.fc30.x86_64.rpm 724 kB/s | 108 kB 00:00
(4/8): xkeyboard-config-2.24-5.fc30.noarch.rpm 6.3 MB/s | 822 kB 00:00
(5/8): langpacks-en-1.0-17.fc30.noarch.rpm 75 kB/s | 8.2 kB 00:00
(6/8): systemd-rpm-macros-241-6.gita2eaa1c.fc30 113 kB/s | 22 kB 00:00
(7/8): dbus-broker-20-2.fc30.x86_64.rpm 342 kB/s | 160 kB 00:00
(8/8): systemd-241-5.git3d835d0.fc30_241-6.gita 693 kB/s | 838 kB 00:01
[DRPM 1/3] systemd-pam-241-5.git3d835d0.fc30_241-6.gita2eaa1c.fc30.x86_64.drpm: done
/usr/share/man/man8/libnss_myhostname.so.2.8.gz: No such file or directory
cannot reconstruct rpm from disk files
/usr/share/doc/systemd/NEWS: No such file or directory
cannot reconstruct rpm from disk files
Some packages were not downloaded. Retrying.
(1/2): systemd-libs-241-6.gita2eaa1c.fc30.x86_6 415 kB/s | 473 kB 00:01
(2/2): systemd-241-6.gita2eaa1c.fc30.x86_64.rpm 1.6 MB/s | 3.6 MB 00:02
--------------------------------------------------------------------------------
Total 1.2 MB/s | 6.0 MB 00:04
Failed Delta RPMs increased 5.3 MB of updates to 6.0 MB (-13.1% wasted)
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: systemd-libs-241-6.gita2eaa1c.fc30.x86_64 1/1
Upgrading : systemd-libs-241-6.gita2eaa1c.fc30.x86_64 1/14
Running scriptlet: systemd-libs-241-6.gita2eaa1c.fc30.x86_64 1/14
Upgrading : systemd-rpm-macros-241-6.gita2eaa1c.fc30.noarch 2/14
Installing : xkeyboard-config-2.24-5.fc30.noarch 3/14
Installing : libxkbcommon-0.8.3-1.fc30.x86_64 4/14
Upgrading : systemd-pam-241-6.gita2eaa1c.fc30.x86_64 5/14
Running scriptlet: systemd-241-6.gita2eaa1c.fc30.x86_64 6/14
Upgrading : systemd-241-6.gita2eaa1c.fc30.x86_64 6/14
Running scriptlet: systemd-241-6.gita2eaa1c.fc30.x86_64 6/14
Running scriptlet: dbus-broker-20-2.fc30.x86_64 7/14
Upgrading : dbus-broker-20-2.fc30.x86_64 7/14
Running scriptlet: dbus-broker-20-2.fc30.x86_64 7/14
Upgrading : langpacks-en-1.0-17.fc30.noarch 8/14
Running scriptlet: dbus-broker-19-2.fc30.x86_64 9/14
Cleanup : dbus-broker-19-2.fc30.x86_64 9/14
Running scriptlet: dbus-broker-19-2.fc30.x86_64 9/14
Cleanup : langpacks-en-1.0-16.fc30.noarch 10/14
Running scriptlet: systemd-241-5.git3d835d0.fc30.x86_64 11/14
Cleanup : systemd-241-5.git3d835d0.fc30.x86_64 11/14
Cleanup : systemd-rpm-macros-241-5.git3d835d0.fc30.noarch 12/14
Cleanup : systemd-libs-241-5.git3d835d0.fc30.x86_64 13/14
Cleanup : systemd-pam-241-5.git3d835d0.fc30.x86_64 14/14
Running scriptlet: systemd-pam-241-5.git3d835d0.fc30.x86_64 14/14
Running scriptlet: systemd-241-6.gita2eaa1c.fc30.x86_64 14/14
Verifying : libxkbcommon-0.8.3-1.fc30.x86_64 1/14
Verifying : xkeyboard-config-2.24-5.fc30.noarch 2/14
Verifying : dbus-broker-20-2.fc30.x86_64 3/14
Verifying : dbus-broker-19-2.fc30.x86_64 4/14
Verifying : langpacks-en-1.0-17.fc30.noarch 5/14
Verifying : langpacks-en-1.0-16.fc30.noarch 6/14
Verifying : systemd-241-6.gita2eaa1c.fc30.x86_64 7/14
Verifying : systemd-241-5.git3d835d0.fc30.x86_64 8/14
Verifying : systemd-libs-241-6.gita2eaa1c.fc30.x86_64 9/14
Verifying : systemd-libs-241-5.git3d835d0.fc30.x86_64 10/14
Verifying : systemd-pam-241-6.gita2eaa1c.fc30.x86_64 11/14
Verifying : systemd-pam-241-5.git3d835d0.fc30.x86_64 12/14
Verifying : systemd-rpm-macros-241-6.gita2eaa1c.fc30.noarch 13/14
Verifying : systemd-rpm-macros-241-5.git3d835d0.fc30.noarch 14/14
Upgraded:
dbus-broker-20-2.fc30.x86_64
langpacks-en-1.0-17.fc30.noarch
systemd-241-6.gita2eaa1c.fc30.x86_64
systemd-libs-241-6.gita2eaa1c.fc30.x86_64
systemd-pam-241-6.gita2eaa1c.fc30.x86_64
systemd-rpm-macros-241-6.gita2eaa1c.fc30.noarch
Installed:
libxkbcommon-0.8.3-1.fc30.x86_64 xkeyboard-config-2.24-5.fc30.noarch
Complete!
Failed to set locale, defaulting to C
Last metadata expiration check: 0:01:24 ago on Sun Apr 14 20:49:20 2019.
Package systemd-241-6.gita2eaa1c.fc30.x86_64 is already installed.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
neovim x86_64 0.3.4-1.fc30 updates-testing 5.0 M
Installing dependencies:
libuv x86_64 1:1.27.0-1.fc30 updates-testing 127 k
libX11 x86_64 1.6.7-1.fc30 fedora 593 k
libX11-common noarch 1.6.7-1.fc30 fedora 155 k
libXau x86_64 1.0.9-1.fc30 fedora 30 k
libtermkey x86_64 0.20-6.fc30 fedora 25 k
libvterm x86_64 0-0.8.bzr681.fc30 fedora 35 k
libxcb x86_64 1.13.1-2.fc30 fedora 202 k
luajit x86_64 2.1.0-0.9beta3.fc30 fedora 327 k
msgpack x86_64 3.1.0-2.fc30 fedora 29 k
unibilium x86_64 2.0.0-4.fc30 fedora 32 k
Installing weak dependencies:
xsel x86_64 1.2.0-24.fc30 fedora 26 k
Transaction Summary
================================================================================
Install 12 Packages
Total download size: 6.6 M
Installed size: 25 M
Downloading Packages:
(1/12): libuv-1.27.0-1.fc30.x86_64.rpm 787 kB/s | 127 kB 00:00
(2/12): libX11-common-1.6.7-1.fc30.noarch.rpm 233 kB/s | 155 kB 00:00
(3/12): libX11-1.6.7-1.fc30.x86_64.rpm 683 kB/s | 593 kB 00:00
(4/12): libXau-1.0.9-1.fc30.x86_64.rpm 314 kB/s | 30 kB 00:00
(5/12): neovim-0.3.4-1.fc30.x86_64.rpm 5.0 MB/s | 5.0 MB 00:01
(6/12): libtermkey-0.20-6.fc30.x86_64.rpm 187 kB/s | 25 kB 00:00
(7/12): libvterm-0-0.8.bzr681.fc30.x86_64.rpm 374 kB/s | 35 kB 00:00
(8/12): libxcb-1.13.1-2.fc30.x86_64.rpm 1.7 MB/s | 202 kB 00:00
(9/12): luajit-2.1.0-0.9beta3.fc30.x86_64.rpm 2.1 MB/s | 327 kB 00:00
(10/12): msgpack-3.1.0-2.fc30.x86_64.rpm 203 kB/s | 29 kB 00:00
(11/12): unibilium-2.0.0-4.fc30.x86_64.rpm 347 kB/s | 32 kB 00:00
(12/12): xsel-1.2.0-24.fc30.x86_64.rpm 288 kB/s | 26 kB 00:00
--------------------------------------------------------------------------------
Total 3.5 MB/s | 6.6 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : unibilium-2.0.0-4.fc30.x86_64 1/12
Installing : libtermkey-0.20-6.fc30.x86_64 2/12
Installing : msgpack-3.1.0-2.fc30.x86_64 3/12
Installing : luajit-2.1.0-0.9beta3.fc30.x86_64 4/12
Installing : libvterm-0-0.8.bzr681.fc30.x86_64 5/12
Installing : libXau-1.0.9-1.fc30.x86_64 6/12
Installing : libxcb-1.13.1-2.fc30.x86_64 7/12
Installing : libX11-common-1.6.7-1.fc30.noarch 8/12
Installing : libX11-1.6.7-1.fc30.x86_64 9/12
Installing : xsel-1.2.0-24.fc30.x86_64 10/12
Installing : libuv-1:1.27.0-1.fc30.x86_64 11/12
Installing : neovim-0.3.4-1.fc30.x86_64 12/12
Running scriptlet: neovim-0.3.4-1.fc30.x86_64 12/12
Verifying : libuv-1:1.27.0-1.fc30.x86_64 1/12
Verifying : neovim-0.3.4-1.fc30.x86_64 2/12
Verifying : libX11-1.6.7-1.fc30.x86_64 3/12
Verifying : libX11-common-1.6.7-1.fc30.noarch 4/12
Verifying : libXau-1.0.9-1.fc30.x86_64 5/12
Verifying : libtermkey-0.20-6.fc30.x86_64 6/12
Verifying : libvterm-0-0.8.bzr681.fc30.x86_64 7/12
Verifying : libxcb-1.13.1-2.fc30.x86_64 8/12
Verifying : luajit-2.1.0-0.9beta3.fc30.x86_64 9/12
Verifying : msgpack-3.1.0-2.fc30.x86_64 10/12
Verifying : unibilium-2.0.0-4.fc30.x86_64 11/12
Verifying : xsel-1.2.0-24.fc30.x86_64 12/12
Installed:
neovim-0.3.4-1.fc30.x86_64 xsel-1.2.0-24.fc30.x86_64
libuv-1:1.27.0-1.fc30.x86_64 libX11-1.6.7-1.fc30.x86_64
libX11-common-1.6.7-1.fc30.noarch libXau-1.0.9-1.fc30.x86_64
libtermkey-0.20-6.fc30.x86_64 libvterm-0-0.8.bzr681.fc30.x86_64
libxcb-1.13.1-2.fc30.x86_64 luajit-2.1.0-0.9beta3.fc30.x86_64
msgpack-3.1.0-2.fc30.x86_64 unibilium-2.0.0-4.fc30.x86_64
Complete!
Failed to set locale, defaulting to C
47 files removed
Failed to set locale, defaulting to C
Error: Failed to mount filesystems: mkdir /var/cache/distrobuilder.175713338/rootfs/.distrobuilder/0: file exists
We seem to be missing support for building Oracle Linux containers, it should be very similar to the CentOS case and is next on my list of distros to switch over to distrobuilder.
We should be able to use http://il.us.mirror.archlinuxarm.org/os/ as the url for archlinux-http with 68B3537F39A313B3E574D06777193F152BDBE6A6 as GPG key and have it pick one of those two based on architectures:
We'll likely also need an architecture map for Arch which supports:
Looks like Leap 15 URLs have changed as they're all failing now.
Example of a now broken one:
Error: Error while downloading source: Error downloading openSUSE image: Unable to fetch https://download.opensuse.org/repositories/Virtualization:/containers:/images:/openSUSE-Leap-15.0/containers//opensuse-leap-15.0-image.x86_64-lxc.tar.xz: 404 Not Found
Tumbleweed appears to be fine
LXC has something called compat levels for images, it's effectively an ever increasing number which is used by LXC to look up version-specific overrides inside the images.
This is supported for the following files:
For example if I want to override the user config for compat level 5, I'd use config.user.5
.
Unfortunately if I want something to apply to everything before 5, I'd have to set it for level 1, 2, 3 and 4 which makes things rather repetitive, instead we probably can simply things using something like this:
In the yaml, this would likely look like:
targets:
lxc:
create-message: |
You just created an Ubuntu 16.04 LTS container (release=bionic, arch=amd64, variant=default)
To enable sshd, run: apt-get install openssh-server
For security reason, container images ship without user accounts
and without a root password.
Use lxc-attach or chroot directly into the rootfs to set a root password
or create user accounts.
config:
- type: all
before: 5
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/ubuntu.common.conf
- type: user
before: 5
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/ubuntu.userns.conf
- type: all
after: 4
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/common.conf
- type: user
after: 4
content: |-
lxc.include = LXC_TEMPLATE_CONFIG/userns.conf
- type: all
content: |-
lxc.arch = x86_64
distrobuilder needs to know what the current compat level is in LXC (currently 5) and will generate a version of "config" and "config.user" for each of those (from 1 to 5 right now). The un-suffixed file should be that of the maximum compat level. We may at some point try to be clever and de-dup the configs using symlinks, but for now we can be lazy and just generate them all.
The individual snippets should be appended in the order they're provided, so, "config.user" would then be:
lxc.include = LXC_TEMPLATE_CONFIG/common.conf
lxc.include = LXC_TEMPLATE_CONFIG/userns.conf
lxc.arch = x86_64
While "config.4" would be:
lxc.include = LXC_TEMPLATE_CONFIG/ubuntu.common.conf
lxc.arch = x86_64
This will generate all those files:
It took me a while to figure out what was going on when first trying to run distrobuilder as "--help" doesn't show you anything and running it without arguments just hangs as it tries to read from stdout.
I think it'd be best to always require some kind of input, using "-" (as already supported) to denote reading from stdin.
I'd expect the following to work at least:
Looking at the current cli structure, I suspect the easiest would be to use sub-commands for:
The following would be LXD specific options:
So it'd look something like:
distrobuilder [--help|--version]
build-lxc <filename|-> [target dir] [KEY=VALUE...]
build-lxd <filename|-> [target dir] [--type=TYPE] [--compression=COMPRESSION] [KEY=VALUE...]
build-dir <filename|-> <target dir> [KEY=VALUE...]
pack-lxc <filename|-> <source dir> [target dir]
pack-lxd <filename|-> <source dir> [target dir] [--type=TYPE] [--compression=COMPRESSION]
The expectation there as far directory handling is that for lxc and lxd, we use a temporary directory to create stuff (in /tmp) and don't leave anything behind.
In the case where we want to publish both a LXC and LXD image, I'd do:
distrobuilder build-dir ubuntu.yaml my-image/
distrobuilder pack-lxc ubuntu.yaml my-image/
distrobuilder pack-lxd ubuntu.yaml my-image/
As a reminder, the KEY=VALUE stuff is so that I can override some individual fields like:
distrobuilder build-lxd ubuntu.yaml image.release=bionic image.arch=armv7l
Right now we grab the latest ArchLinux release by parsing https://www.archlinux.org/download/ but this isn't guaranteed to match what's on the mirror we're using, leading to download failures.
We should instead pick whatever's the highest timestamp on the download mirror itself.
I think we should switch actions to be closer to what we do for files:
actions:
- trigger: post-update
action: |-
#!/bin/sh
rm -rf /run/*
- trigger: post-unpack
action: |-
#!/bin/sh
sed -i "s/foo/bar/g" /etc/hosts
releases:
- precise
- trusty
This is really just a question whether replacing:
Available Commands:
build-dir Build plain rootfs
build-lxc Build LXC image from scratch
build-lxd Build LXD image from scratch
help Help about any command
pack-lxc Create LXC image from existing rootfs
pack-lxd Create LXD image from existing rootfs
should be
Available Commands:
build dir Build plain rootfs
build lxc Build LXC image from scratch
build lxd Build LXD image from scratch
help Help about any command
pack lxc Create LXC image from existing rootfs
pack lxd Create LXD image from existing rootfs
It seems cleaner to me but feel free to close this without discussing it if you think that it doesn't make sense. :)
Actions aren't implemented yet and are likely to be needed for some of our supported images.
I just did a quick test of distrobuilder with CentOS, a few things that should get fixed:
And maybe more critically, running distrobuilder on Ubuntu 18.04, I'm getting this when running the example:
mount: /tmp/distrobuilder/iso: WARNING: device write-protected, mounted read-only.
error: open of rpm-4.11.3-25.el7.x86_64.rpm failed: No such file or directory
error: open of yum-3.4.3-154.el7.centos.noarch.rpm failed: No such file or directory
Traceback (most recent call last):
File "/bin/yum", line 28, in <module>
import yummain
ImportError: No module named yummain
Traceback (most recent call last):
File "/bin/yum", line 28, in <module>
import yummain
ImportError: No module named yummain
Traceback (most recent call last):
File "/bin/yum", line 28, in <module>
import yummain
ImportError: No module named yummain
Error: Failed to manage packages: exit status 1
with d29bfc5 the cloud init example for Debian is replaced with an example without the cloud-init support in the container.
is this intentional that the documentation is lost, or just an oversight in updating the documentation ?
Following documentation to build distrobuilder but failed with error messages below:
github.com/lxc/distrobuilder (download)
created GOPATH=/root/go; see 'go help gopath'
go: missing Git command. See https://golang.org/s/gogetcmd
package github.com/lxc/distrobuilder: exec: "git": executable file not found in $PATH
Some notes as I'm playing with it:
schedule to be discussed
Distrobuilder build is currently broken due to function signature change in lxd dependency.
Modified function signature here
Error log :
gofmt -s -w .
go get -t -v -d ./...
go install -v ./...
github.com/lxc/distrobuilder/sources
# github.com/lxc/distrobuilder/sources
sources/alpine-http.go:91:18: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
sources/archlinux-http.go:78:18: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
sources/fedora-http.go:51:18: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
sources/fedora-http.go:93:21: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
sources/gentoo.go:91:18: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
sources/sabayon.go:44:18: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
sources/ubuntu-http.go:116:19: too many arguments in call to "github.com/lxc/lxd/shared".Unpack
have (string, string, bool, bool, nil)
want (string, string, bool, bool)
make: *** [Makefile:6: default] Error 2
Because our production image building runs inside a container (privileged though), we can't use kpartx
as it attempts to mess with device-mapper which is very very unpleasant to use inside containers.
We do however have working loop devices, so using losetup
would be fine.
So we need to change distrobuilder to scan the partition table of the .img file, figure out the offset and size of the partition we want and then map that using losetup
, then mount it.
I am looking at the centos example config, and I don't see anything to tell me how to install package groups. If it is possible, can we have an instructive comment in the example config? If it is not possible, can we have a comment stating that in the example config, README.md, or some other discoverable place?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.