CI: migrate build_fdroid to fdroidserver:builserver-bullseye image about navit HOT 12 CLOSED

Looking at the logs, @jkoan ’s workflow yesterday was also affected, while 24 days ago things still seemed OK.

from navit.

Just checked, our git repo does not seem to have had any CI-related changes since the last successful run. The underlying CI image has not changed since April 2022.

However, apt output indicates that 24 days ago we were still on Debian stretch, and now we’re on bullseye. This was apparently merged a few days ago: https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1012

Before the merge, all repos seemed to be HTTP only, now at least one uses HTTPS. Alas, the merge request has dropped the apt-transport-https package. If that package no longer gets installed, that would explain why apt fails. Currently testing.

from navit.

So... fdroidserver migrated from stretch to bullseye, but the base CI image (ci-images-client, also from F-Droid) is still based on stretch. fdroidserver, however, has an install script which makes a few apt-related changes, which now assume to be running on bullseye.

As long as these images are based on different Linux versions, this CI setup will no longer work.

from navit.

I will take a look tomorrow. Sounds like an easy fix

from navit.

The PR is a quick fix. On the long run, this is likely something to be solved on the F-Droid end.

I will investigate if there is an easy way to run CI jobs directly on the fdroidserver image rather than on ci-images-client. Otherwise I’d suggest that F-Droid migrate ci-images-client to bullseye.

from navit.

@jkoan if you have other ideas, feel free. For the moment, we got CI running again, but we will not get any more updates for fdroidserver.

from navit.

I’ve been trying to move build-fdroid to the fdroidserver image, see here: https://github.com/navit-gps/navit/compare/ci-fdroid-buildserver?expand=1 (or browse the ci-fdroid-buildserver branch).

Steps are mostly copied over from fdroidserver’s own CI script (they have a fdroid build step where they test the F-Droid build server against two build recipes). Some changes were needed and are marked as such in comments.

I had to swap out the sdkmanager binary in the Android tools dir, as the image has two different copies of it: the one in the Android SDK dir itself is old and does not support Java 11; the one supplied by the OS is up to date.

With all of these steps, at least fdroid starts up and successfully completes the prebuild phase (which includes installing some native components using sdkmanager). However, build fails as the cmake binary is not found – it should (and actually does) reside in /opt/android-sdk/cmake/3.6.4111459/bin/cmake but gradle does not find it – this has worked with the old CI setup.

I don’t want to mess around too much with the environment, as I want it to be as close as possible to what I would get when running on the “real” F-Droid build server. If it works on one, I want it to work on the other; if it fails one one, I want it to fail on the other – and vice versa.

@jkoan do you happen to have any ideas?

from navit.

Got in touch with the F-Droid team, and they recommend we use the fdroidserver:buildserver image for CI. In fact, the recommendation is to target a specific release, e.g. fdroidserver:buildserver-bullseye, so we can control when we migrate to a newer one. #1205 implements that, still for the fdroidserver:buildserver-stretch image, which works but is no longer current as of last week.

The new version, based on bullseye, still has some rough edges. For one, bullseye includes Java 11, which breaks the sdkmanager tool that comes with the Android SDK. There is an updeted version, but for some reason Google isn’t including that in the SDK at least for now. F-Droid has countered that by building their own sdkmanager, functionally equivalent but written in Python rather than Java.

There still seem to be some issues as our build recipe installs cmake from the Android SDK. With the F-Droid replacement, there seems to be something wrong with the environment, as the cmake executable is not found (and if I modify the path manually, it complains about CMAKE_C_COMPILER and CMAKE_MAKE_PROGRAM not being set). I suspect this is a genuine bug in the F-Droid sdkmanager version – I have filed https://gitlab.com/fdroid/sdkmanager/-/issues/12 for that.

Also, it is not clear how a build recipe should reference sdkmanager: currently we have $$SDK$$/tools/bin/sdkmanager, but that points to the now-broken old Google version. Without the path, sdkmanager runs (the F-Droid version) but I am not sure this is also true for the live build server. See https://gitlab.com/fdroid/fdroidserver/-/issues/1052

Stay tuned for updates.

from navit.

So... we got some updates:

The buildserver image should behave like the live build server. If not, this is a bug and an issue should be opened.

With the bullseye image, sdkmanager is in the default path and can be referenced by build recipes in this way.

Finally, the CI image got stripped down significantly, and quite a number of packages which were found in the stretch image are no longer present in the bullseye image. This means our build recipe needs to specify all build dependencies – this is also true on the live server.

With the great support from the guys over at F-Droid I finally got a working build on the bullseye image, still working on details.

One question: do we need libgtk2.0-dev to build for Android? The build completes without it, but I haven’t tried out the resulting APK.

from navit.

I finally got everything to build on the new image, see PR #1206. Only the question about libghtk2.0-dev is still open.

from navit.

I finally got everything to build on the new image, see PR #1206. Only the question about libgtk2.0-dev is still open.

I think we should remove it even if it's failing (what it should not) because we don't wanna build with gtk. If our detection code is flawed we should fix this in that case. So feel free to remove it

from navit.

Done.

from navit.