mipsn32el

mipsn32el is the "Debian style" architectural name for Mips with N32 ABI, the architecture come with a triplet of mips64el-linux-gnuabin32 or mips64el-unknown-linux-gnu.

Works ahead

• Make sure Core builds on such machine;
• Calibrate autobuild3 for mipsn32el;

Difficulties

The only machine available for Core building is my Yeeloong 8089D netbook, there is currently no plan for an automated cross build of Core.

Expected outcome

• Core can be built;
• Core can be built automatically;
• Core can be built when new release come out (e.g. future release after 3.0.5), this probably would require support for cross building;

Time to merge the build files into a single defines. Given that:

• It is way too much trouble changing stuff for each version change;
• It did not help readability anyways;

http://www.openwall.com/lists/oss-security/2017/10/27/4
http://www.openwall.com/lists/oss-security/2017/10/27/3

There are several things to do still.

• Come up with a set of compiler flags specific to mips64el.
• Architectural build config for mips64el.
• Tests.

We need some serious use of PKGPROV.

Cloog is no longer needed by GCC for Graphite loop optimizations. Therefore it holds no need for GCC dependencies, nor for binary compatibility reference, and should be removed with the GCC update.

The Graphite framework for loop optimizations no longer requires the CLooG library, only ISL version 0.14 (recommended) or 0.12.2. The installation manual contains more information about requirements to build GCC.

https://gcc.gnu.org/gcc-5/changes.html

Not doing a write up for now, but here are the things we know about at this point:

• Programs won't break immediately.
• A crap ton of rebuilds will be happenning this year.
• Further announcement and plan-of-attack on AOSCC (possibly workshop sessions).

Original mailing list announcement: http://lists.gnu.org/archive/html/info-gnu/2016-05/msg00013.html

Seems like this release have finally settled on a stable parallel interface...

This effectively excludes all non-UTF-8 locale from the default generation list.

• Need to be described in ChangeLog.
• Postinst should probably inform users of this (in case of a user who uses GBK family of encoding daily...)

GCC 6.1.0 is released, as gcc-compatible libstdcxx ABI is still available, we are looking at a bump for Core 3.6. Runtime compatiblity should be 100%, while FTBFS would be expected to the future months to come.

Upstream changes info: https://gcc.gnu.org/gcc-6/changes.html

"It is possible to use profile feedback to optimize the compiler itself. This should result in a faster compiler binary. Experiments done on x86 using gcc 3.3 showed approximately 7 percent speedup on compiling C programs. To bootstrap the compiler with profile feedback, use make profiledbootstrap.

When ‘make profiledbootstrap’ is run, it will first build a stage1 compiler. This compiler is used to build a stageprofile compiler instrumented to collect execution counts of instruction and branch probabilities. Then runtime libraries are compiled with profile collected. Finally a stagefeedback compiler is built using the information collected.

Unlike standard bootstrap, several additional restrictions apply. The compiler used to build stage1 needs to support a 64-bit integral type. It is recommended to only use GCC for this. "

"Installing GCC" documentation from GNU

It is currently known that Fedora provides such a build with exceptions on the SPARC architectures, which we are not going to concern about in the near future XD. Reference

"‘bootstrap-lto’
Enables Link-Time Optimization for host tools during bootstrapping. ‘BUILD_CONFIG=bootstrap-lto’ is equivalent to adding -flto to ‘BOOT_CFLAGS’. This option assumes that the host supports the linker plugin (e.g. GNU ld version 2.21 or later or GNU gold version 2.21 or later). "

Also an interesting configure option that we should look into. MIPS might run into issues with this, we will see about this. No other distributions are known to be using this option.

Reported as Bug 13063 at sourceware.org, https://sourceware.org/bugzilla/show_bug.cgi?id=13063. As suggested in comments, a patch was included for Bug 17563 (https://sourceware.org/bugzilla/show_bug.cgi?id=17563) - which was described to be tested with Fedora - and working.

Attempt to apply this patch and check back.

Test case from @Icenowy :

printf "%s\n" ，。 | uniq

(Moved from AOSC-Dev/aosc-os-abbs#204).

Some scripts in examples are actually fun, e.g. http://git.savannah.gnu.org/cgit/bash.git/tree/examples/functions/autoload.v3.

(Well, this one can be modified slightly with the dict-as-a-set trick.)

https://support.google.com/faqs/answer/7625886

This will save us a lot of HOSTTOOLPREFIX bullshit.

Here are some definitions for the busybox package:

# part of defines
PKGNAME=busybox
PKGPROV=sh

# overrides/etc/profile.d/80-busybox
export PATH="$PATH:/busybin"

Busybox here is chosen to be placed in /busybin so we can use it even if /usr is on a seperate dir, maintaining the possiblity of using no initramfs at all (for init scripts, we can mount --bind /busybin /bin first and in the end recover the symlink.)

Perl was not updated since the start of AOSC OS3 project (August of 2014), due to the fact that Perl (site-)modules are installed in a versioned directory /usr/lib/perl5/5.20.0/ and it is extremely time consuming to rebuild all of them modules.

A bump to Perl is advised in the near future, to the 5.22 series. However, it is not clear that if AOSC OS should rebuild the packages, or to make hacks (or even mask the directory as 5.20.0) in order for the Perl-based modules and packages to continue working.

Test suite:

LC_ALL=C.UTF-8 sed -n '/^.$/p' <<< 有 # boom
LC_ALL=C.utf8 sed -n '/^.$/p' <<< 有 # boom, optional
LC_ALL=zh_CN.UTF-8 sed -n '/^.$/p' <<< 有
LC_ALL=C sed -n '/^.$/p' <<< 没
LC_ALL=POSIX sed -n '/^.$/p' <<< 没

Currently tzdata is the only package in the Core that has the noarch specification, which can be a redundancy when building Core across multiple architectures.

An idea: split tzdata out of the build-core group, forming a full build-core-full and build-core-arch group.

gcc-runtime as reported by RPM, is missing libgcc_s.so.1 as one of its dependencies, this has resulted in a dependency issue and needs to be addressed soon.

Environment:
Fedora 25 x86_64,
libvirt-2.2.0-2.fc25.x86_64
https://mirror.anthonos.org/aosc-os/os-amd64/base/aosc-os_base_20161202.tar.xz

Steps to reproduce:

1. Make a new disk img and partition with ext4
2. sudo losetup --offset 1048576 --sizelimit $((512*41940992)) --show --find aosc.img /dev/loop0
3. sudo mount /dev/loop0 /tmp/mnt
4. sudo tar Jxvf aosc-os_base_20161202.tar.xz -C /tmp/mnt/
5. sudo umount /tmp/mnt ; sudo losetup -d /dev/loop0
6. Boot the disk in a VM (in virt-manager) with a Fedora Live DVD, chroot and create the grub and grub config
7. Reboot to the disk img.

Actual result:
After grub, it shows

Error, no symbol table.
Loading Linux 4.8.6...

Then a kernel panic happened.

[zsun@tmb113 ~]$ sudo virsh console aosc
Connected to domain aosc
Escape character is ^]
[    1.403553] gpio_it87: no device
[    1.471017] Failed to find cpu0 device node
[    1.646574] esas2r: driver will not be loaded because no ATTO esas2r devices were found
[    1.827971] i2c-parport-light: adapter type unspecified
[    1.901421] genirq: Flags mismatch irq 4. 00000000 (serial) vs. 00000080 (goldfish_pdev_bus)
[    1.907194] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    1.909411] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.8.6-aosc-main #1
[    1.911602] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-1.fc25 04/01/2014
[    1.915315]  0000000000000086 00000000a5618c00 ffff9b7dfd397d78 ffffffff835b7ed0
[    1.917367]  ffff9b7df99ec000 ffffffff8429a8a0 ffff9b7dfd397e00 ffffffff831cc3cb
[    1.919120]  ffff9b7d00000010 ffff9b7dfd397e10 ffff9b7dfd397da8 00000000a5618c00
[    1.920926] Call Trace:
[    1.921491]  [<ffffffff835b7ed0>] dump_stack+0x63/0x83
[    1.922539]  [<ffffffff831cc3cb>] panic+0xdf/0x221
[    1.924404]  [<ffffffff854a562e>] mount_block_root+0x200/0x2bf
[    1.926331]  [<ffffffff854a4884>] ? set_debug_rodata+0x12/0x12
[    1.927587]  [<ffffffff854a5752>] mount_root+0x65/0x68
[    1.929052]  [<ffffffff854a5889>] prepare_namespace+0x134/0x16c
[    1.930885]  [<ffffffff854a52e4>] kernel_init_freeable+0x32e/0x347
[    1.932463]  [<ffffffff83fdc2b9>] kernel_init+0x9/0x100
[    1.933905]  [<ffffffff83fe959f>] ret_from_fork+0x1f/0x40
[    1.935222]  [<ffffffff83fdc2b0>] ? rest_init+0x90/0x90
[    1.936575] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[    1.941356] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

• On ARM EABI (32-bit), generating a backtrace for execution contexts
  which have been created with makecontext could fail to terminate due
  to a missing .cantunwind annotation. This has been observed to lead
  to a hang (denial of service) in some Go applications compiled with
  gccgo. Reported by Andreas Schwab. (CVE-2016-6323)
• The DNS stub resolver functions would crash due to a NULL pointer
  dereference when processing a query with a valid DNS question type
  which was used internally in the implementation. The stub resolver
  now uses a question type which is outside the range of valid question
  type values. (CVE-2015-5180)

Fixed with b7e7782. And use AOSA-2017-0018.

With AOSC-Dev/autobuild3#89, Autobuild3 is able to handle architectural-neutral packages, tzdata obviously fits the criteria of being a "data" package, we should mark this package with ABHOST=NOARCH.

@Arthur2e5 As a side note... it is quite sad but aosc-aaa wouldn't be able to use this.

Enabling Golang could make it infinitely easier to bootstrap Go toolchain, and to build cgo programs.

It was disabled in the first place, with no reasonable reason what so ever. ObjC and ObjC++ support should be re-enabled with GCC 5.3 update.

The issue started with a careless mistake of excluding isl and cloog from the Core package list. Our gcc package should depend on isl and cloog as we specified in the build configuration.

What does isl and cloog do for gcc?

CLooG is the code generation library that is used in the Graphite loop transforms pass of GCC. GCC 4.4 uses the CLooG-PPL branch of CLooG that implements a backend of CLooG for the PPL.

As per described by GCC Wiki. http://gcc.gnu.org/wiki/CLooG

And isl is a dependency of cloog.

A way to fix

Introduce cloog and isl as two new packages to the "Core". As cloog and isl are only dependent on gmp, which is already in the "Core".

An exploit can be realized by creating a file or directory with a specially crafted name. A user utilizing GNU Bash’s built-in path completion by hitting the Tab button (f.e. to remove it with rm) triggers the exploit without executing a command itself. The vulnerability has been introduced on the devel -branch in May 2015.

CVE-2017-5932 was just assigned for this particular vulnerability, which is fixed with patch level 7.

Bash for AOSC OS Core has been updated to patch level 12, which includes a fix for this particular security vulnerability - with commit f6105ba. Use AOSA-2017-0019 for this issue.

CVE-2017-15804, CVE-2017-15670, CVE-2017-15671, CVE-2017-15804

... Because

• cp950 is not Big5 (EUDC mappings)
• cp936 is not GBK (Euro sign)

https://www.qualys.com/2017/12/11/buffer-overflow-cve-2017-1000408-cve-2017-1000409/cve-2017-1000408-cve-2017-1000409.txt

There are quite some favorable reports on performance improvements ("better than finalspeed") from BBR (paper) on Chinese [read: shitty] network conditions, and BBR appears optimized for wireless connections as well. Perhaps most users will find it useful.

Note: Defaulting to BBR would make fq qdisc an integral part of AOSC OS.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=636086

The issue

autobuild3, from tag v0.1.4.1 has added flags for further hardening, and with a recent commit AOSC-Dev/autobuild3@5de9897 switches for enabling/disabling those flags has been added.

Build failures are expected with PIE (position independent executable) flags passed to compiler and linker. Tests are needed to ensure that they will build under the new condition.

A sample of a fully hardened binary

root [ autobuild@dev/new-packages ] # /checksec.sh --file /usr/bin/lightdm
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH      FILE
Full RELRO      Canary found      NX enabled    PIE enabled     No RPATH   No RUNPATH   /usr/bin/lightdm

Recommended practice

First of all, enable all hardening features when possible, to achieve the expected result above.

Use the $AB_FLAGS_* variables to control the hardening-related flags when possible, as listed in /etc/autobuild/defaults/hardening:

# Work in progress: factor hardening-related flags into options
# Parameters that are likely to cause trouble.

AB_FLAGS_PIC=1
AB_FLAGS_PIE=1
AB_FLAGS_SSP=1
AB_FLAGS_RRO=1
AB_FLAGS_NOW=1

Hacks are allowed, because in cases like core-libs/glibc, it is possible to control flags in different stages.

GNU C Library gets a round-up patch every release, and it should be added to 2.22, as it's been released lately...

CVE-2017-10684, CVE-2017-10685, CVE-2017-11112, CVE-2017-11113, CVE-2017-13728, CVE-2017-13729, CVE-2017-13730, CVE-2017-13731, CVE-2017-13732, CVE-2017-13733, CVE-2017-16879

Please triage.

It was taken down as of Core 3.1.2 as a measure against unknown lock-ups and network performance issues.

bash 4.3 patch 40, 41, 42

40: https://lists.gnu.org/archive/html/bug-bash/2015-08/msg00101.html
41: https://lists.gnu.org/archive/html/bug-bash/2015-08/msg00097.html
42: https://lists.gnu.org/archive/html/bug-bash/2015-08/msg00098.html

If a package is to be provided with its respective headers, so should glibc. glibc when not dependent on linux+api is practically useless in a development scenario.

December 4, 2015

The GNU project and the GCC developers are pleased to announce the release of GCC 5.3.

This release is a bug-fix release, containing fixes for regressions in GCC 5.2 relative to previous releases of GCC.

The new Core release should bump to this version, as GCC 5.x is yet to be more mature, and some packages in aosc-os-abbs tree is having issue with the new compiler (wine is a good example), and is known to be fixed by this release.

Perl, with the Core 4 update, has a new behavior that vendor modules are installed at /usr/lib/perl5/vendor_perl and /usr/bin/vendor_perl/, which has rendered some programs unusable as our $PATH does not cover this directory.

We should include a profile script for it, something like this:

https://git.archlinux.org/svntogit/packages.git/tree/trunk/perlbin.sh?h=packages/perl

But much prettier.

(This is quite a distant update...)

GDBM should be updated to 1.12 for the next Core release, with no significant runtime incompatibility specified per the upstream changelog.

Original mailing list announcement: http://lists.gnu.org/archive/html/info-gnu/2016-05/msg00009.html

Noteworthy change in this release:

* New configuration variable COMPATINCLUDEDIR

When used with --enable-libgdbm-compat, this variable points to the
directory where the headers file dbm.h and ndbm.h will be installed.
Use this variable to avoid conflicts with already installed headers.
E.g.:

 ./configure --enable-libgdbm-compat COMPATINCLUDEDIR=/usr/include/gdbm

Seems like autobuild/beyond for this package can be dumped.

DUPLICATE FIXED AOSC-Dev/aosc-os-abbs#128 WRONG PLACE

aosc [ ~ ] ! sudo apt update && sudo apt dist-upgrade
Ign:1 http://mirrors.ustc.edu.cn/anthon/os3-next/os3-dpkg  InRelease
Get:2 http://mirrors.ustc.edu.cn/anthon/os3-next/os3-dpkg  Release [830 B]
Get:3 http://mirrors.ustc.edu.cn/anthon/os3-next/os3-dpkg  Release.gpg [473 B]
Ign:3 http://mirrors.ustc.edu.cn/anthon/os3-next/os3-dpkg  Release.gpg
Ign:4 https://repo.aosc.io/os3-next/os3-dpkg  InRelease
Get:5 https://repo.aosc.io/os3-next/os3-dpkg  Release [830 B]
Get:6 https://repo.aosc.io/os3-next/os3-dpkg  Release.gpg [473 B]
Ign:6 https://repo.aosc.io/os3-next/os3-dpkg  Release.gpg
Ign:7 http://mirrors.anthonos.org/anthon/os3-next/os3-dpkg  InRelease          
Get:8 http://mirrors.anthonos.org/anthon/os3-next/os3-dpkg  Release [830 B]
Get:9 http://mirrors.anthonos.org/anthon/os3-next/os3-dpkg  Release.gpg [473 B]
Err:9 http://mirrors.anthonos.org/anthon/os3-next/os3-dpkg  Release.gpg
  Unknown error executing apt-key
Reading package lists... Done 
W: GPG error: http://mirrors.ustc.edu.cn/anthon/os3-next/os3-dpkg  Release: Unknown error executing apt-key
E: The repository 'http://mirrors.ustc.edu.cn/anthon/os3-next/os3-dpkg  Release' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: https://repo.aosc.io/os3-next/os3-dpkg  Release: Unknown error executing apt-key
E: The repository 'https://repo.aosc.io/os3-next/os3-dpkg  Release' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://mirrors.anthonos.org/anthon/os3-next/os3-dpkg  Release: Unknown error executing apt-key

Berkeley DB 6.x is licensed under AGPL, as you can read here.

http://www.gnu.org/licenses/agpl-3.0.en.html

However, the license as it stands, may lead to potential licensing issues. Also, given that no package in the Core really needs db, and packages in AOSC OS (~20 packages) are due to be cut from db in the near future, db should be deprecated as a component of Core, and to be moved to base-databases.

Original announcement: http://mm.icann.org/pipermail/tz-announce/2016-June/000039.html.

Announcement: http://mm.icann.org/pipermail/tz-announce/2016-July/000040.html.

No comment needed, a TODO/Reminder only.

@Icenowy

A change should be applied to core-libs/glibc (that I personally believe is beneficial enough to attempt...), Unicode 8.0 extension to the package.

The change is first originated from Fedora, and the change is explained here.

https://fedoraproject.org/wiki/Changes/Unicode_8.0

Patches for this change is already available.

https://sourceware.org/ml/libc-alpha/2015-06/msg00934.html