binary-gentoo is a collection of simple CLI tools to help build Gentoo packages on a non-Gentoo Linux host, primarily. A typical scenario is operation of an active Gentoo binary package host — an active "binhost".
Secondarily, binary-gentoo can also build Gentoo packages on a Gentoo host
with Docker isolation and a full emerge perspective
(while not affecting your host system).
There currently four CLI tools that follow the Unix philosophy and are meant to be combined using a glue language like Bash:
gentoo-build– Builds a Gentoo package with Docker isolationgentoo-clean— Clean Gentoo pkgdir/distdir files usingecleanofapp-portage/gentoolkitwith Docker isolationgentoo-local-queue– Manages simple file-based push/pop build task queuesgentoo-packages— Do operations on pkgdir (other thanemaint --fix binhost)gentoo-tree-diff– Lists packages/versions/revisions that one portdir has over anothergentoo-tree-sync– Brings a given portdir directory (and its backup) up to date
binary-gentoo
is software libre licensed under the GNU Affero GPL version 3 or later license.
Install Docker, Python >=3.8, pip and rsync and then:
# pip install binary-gentooTo set-up an active binhost using binary-gentoo you will likely need a virtual machine with…
- Enough resources, i.e. probably
- Some distribution of Linux
- Auto-updates enabled to stay reasonably secure over time
- Runtime dependencies installed:
- Docker
- Python >=3.8
- rsync
- An SSH or an HTTP daemon (to serve the produced binaries to your consuming Gentoo boxes, unless that is done from another machine)
- An up-to-date copy of the
/etc/portagefolder to use - An up-to-date text file with the names of all packages to build,
e.g. generated by
EIX_LIMIT=0 eix -I --only-names - A set of compile flags (
CFLAGS,CXXFLAGS,LDFLAGS,CPU_FLAGS_X86) that works for both the producing machine and the consuming machine(s) and ideally resolve-march-native installed — more on finding the ideal flags below - Two portdir copies — one synced with
gentoo-tree-sync(or some form ofgit pull), the other brought back in sync usingrsyncafter a call togentoo-tree-diff - A glue script that…
- respects your inclusion list
(e.g.
grep -q -f installed.txt -F <<<"${atom}" || continueif a few false positives are okay) - respects your exclusion list
(e.g. something like
*sys-kernel/*|*-bin-*|*-9999*|*acct-*/*) - pushes
gentoo-tree-diffnews onto the queue - pops off the queue,
extracts the atom using e.g.
jq -r .atom, and then let'sgentoo-builddo a build - loops forever
- sleeps every now and then
- runs
gentoo-cleanevery now and then
- respects your inclusion list
(e.g.
Let me give a concrete example.
On the consumer machine, we have this hardware situation:
# lscpu | fgrep 'Model name' | sed 's,^.\{33\},,'
Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
# resolve-march-native
-march=sandybridge -maes --param l1-cache-line-size=64 --param l1-cache-size=32 --param l2-cache-size=3072 -O2 -pipe
# cpuid2cpuflags
CPU_FLAGS_X86: aes avx mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3On the producer machine, we have this hardware situation:
# lscpu | fgrep 'Model name' | sed 's,^.\{33\},,'
QEMU Virtual CPU version 2.5+
# resolve-march-native
-march=k8-sse3 -maes -mcx16 -mno-3dnow -mno-3dnowa -mpclmul -mpopcnt -mrdrnd -msahf -msse4 -msse4.1 -msse4.2 -mssse3 -mtune=k8 --param=l1-cache-line-size=64 --param=l1-cache-size=64 --param=l2-cache-size=512 -O2 -pipe
# cpuid2cpuflags
CPU_FLAGS_X86: aes mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3Simplified, we are looking for flags that are as close to the consumer machines as possible while still resulting in binaries, that both the consumer machines and the producer machine can execute (so that the producer machine is be able to install and run the binaries of dependencies of the package of interest).
For the example above, I ended up with these values for the producer machine:
CFLAGS:-march=x86-64 -mtune=sandybridge -maes --param l1-cache-line-size=64 --param l1-cache-size=32 --param l2-cache-size=3072 -mpclmul -mpopcnt -msse4.1 -msse4.2 -mssse3 -O2 -pipe(note-marchand-mtunein particular)CXXFLAGS: same asCFLAGSLDFLAGS: whateverportageq envvar LDFLAGSon the consumer machine returnsCPU_FLAGS_X86:aes mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3(i.e. withoutavxand withoutrdrand)
-
All code in binary-gentoo must work on a non-Gentoo Linux machine, provided that it has Docker installed and working internet access.
-
Big packages like Chromium need a pile of RAM and CPU time. Therefore, the build defaults to
MAKETOPTS=-j1the package of interest is emerged separate from it dependencies. That allows to build e.g. a package of Chromium in a VM with only 8 GB of RAM. -
With regard to dependency constraints, some packages can be built without conflicts but not be installed without conflicts. Hence the default is to only install dependencies, but not the package of interest.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent