Test suite about bubblewrap HOT 7 OPEN

So technically we have a test case that runs just to satisfy Homu. In reality we need to have coverage of most of the options in both userns and non-userns scenarios, and cover that with PRs.

I think simplest is doing Fedora and CentOS builds, then have an "installed tests" script that assumes bubblewrap is installed. We can't (AFAIK) use travis for this - I think we should assume we run on VMs/metal, and make it easy to run the test suite after the system has been provisioned however (OpenStack/EC2/whatever).

from bubblewrap.

We can't (AFAIK) use travis for this

You can run with (at least some) privileges on Travis if you combine dist: trusty with sudo: required, as I did for dbus.

from bubblewrap.

I'm working on this with CentOS CI, which gives us total control. However a test suite should likely be able to run in both environments.

from bubblewrap.

For C code, GTest seems the obvious choice. Do you have a preferred setup for installed-tests done in other languages, such as shell scripts? (Or would you prefer "shell" scripts in Python with the subprocess module?)

When I added a Debian-specific test for #71 I used https://github.com/sstephenson/bats/ for TAP output, and added some local functions resembling Perl's Test::More for better diagnostics: see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=824968.

from bubblewrap.

bats seems interesting. Refresh my memory, if we have upstream InstalledTests style tests those can be easily bridged to Debian AutoTest right?

from bubblewrap.

Yes, GNOME-style installed-tests are easy to hook up to autopkgtest. For packages that have them, we typically run gnome-desktop-testing-runner as the main autopkgtest, then add others for "integration" stuff (testing the ability to compile and link a "hello world" program to a library using pkg-config is a common one) or for any of the installed-tests that have special dependencies (for instance flatpak-builder).

https://anonscm.debian.org/cgit/collab-maint/flatpak.git/tree/debian/tests is what I have so far for flatpak, which is a fairly typical example.

from bubblewrap.

Proposed bubblewrap tests:
https://anonscm.debian.org/cgit/users/smcv/bubblewrap.git/tree/debian/tests?h=debian/master
(obviously you'd want to change the location of the tests and helper script; I didn't want to start upstreaming until I know what structure you want).

Unfortunately I'm not actually a bubblewrap co-maintainer in Debian at the moment, so that git tree contains fixes that I've proposed, rather than what Debian actually ships.

from bubblewrap.