This requires some new mechanisms: now we're skipping individual tests,
not just whole test scripts.
There are two main reasons why autobuilder environments might not be
able to run these tests successfully, both of which apply in Debian.
Tests that rely on bwrap typically can't pass in builds that take place
in a chroot, because bwrap's use of pivot_root() assumes that the root
directory is a mount point, but a chroot will typically have an unpacked
directory somewhere below the mount point as its root.
Some autobuilder environments are also sufficiently restricted that they
can't create new user namespaces at all, as a way to harden the
autobuilder host.
As a result, Debian autobuilders can't run the majority of the Flatpak
tests. We would like to be able to continue to run the subset that don't
need bwrap, to have the best test coverage we can. For the rest we have
to rely on installed-tests (which I've wired up to Debian's autopkgtest)
rather than using build-time tests.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #2339
Approved by: matthiasclasen
Rather than regenerating it over and over we generate the test
platform as check_DATA and refer to it from the individual tests.
This cuts down "time make check -j8" from 1:25 to 1:06 in user time on
my machine, which is nice in itself, but it is also a stepping stone
for splitting out our large test-scripts into smaller ones where it
will make even more difference.
For the installed-tests case we fall back to creating the platform
once per test.
Closes: #2188
Approved by: alexlarsson
Rather than specify this in multiple places we now list the most
core binaries needed for the tests inside the script.
Closes: #2188
Approved by: alexlarsson
This is only used in the flatpak-builder tests now, not the main
flatpak tests.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #1990
Approved by: alexlarsson
In the cleanup() function used after each unit test exits, we kill the
dbus daemon process and remove the temporary test directory. However, on
an Endless Mini the daemon doesn't always exit properly before we try to
remove the test directory, leading to an error like:
rm: cannot remove '/var/tmp/test-flatpak-AlmZul/runtime/.dbus-proxy': Device or resource busy
which causes some of the tests to fail with e.g.:
ERROR: tests/test-bundle.sh - exited with status 1
I don't think this is specific to the Mini or to ARM computers; it could
probably happen on any slow computer. So this commit uses "kill -9"
instead of just "kill" to make sure the daemon exits.
Closes: #1642
Approved by: alexlarsson
This is in order to allow tests to make several apps if needed, and will
be useful when e.g. testing repos that have multiple apps with different
collection IDs.
This patch modifies the mentioned script as mentioned and updates the
places that call it.
Closes: #1587
Approved by: mwleeds
When uninstalling, if no specific installation was specified with e.g.
--user or --system, automatically chose any unique match, or error
out if there are multiple alternatives.
Fixes#1321
The kernel used on some autobuilders for the Debian PA-RISC port
can do "bwrap --ro-bind / / /bin/true", but not the bwrap invocations
made during the actual testing, which fail with "Creating new namespace
failed: Invalid argument". Make the trial bwrap invocation more like
what Flatpak actually does, so that these tests will hopefully be
skipped on such kernels.
Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876743
Signed-off-by: Simon McVittie <smcv@debian.org>
We'll need this to be able to skip system-mode tests without xattrs.
This partially reverts commit a53a752a35.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This makes it easier to reason about. In the absence of
FLATPAK_SYSTEM_CACHE_DIR, it would use /var/tmp even during tests that
prefer /tmp.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This adds variable support for collection IDs: they can either be
enabled on the server, on the server and client, or not at all. If
enabled on the server, apps and runtimes are built with collection IDs
and the repository has one set. If enabled on the client, the remote
config is added to the local repository with a collection ID and GPG
verification enabled. They are controlled with
USE_COLLECTIONS_IN_{SERVER,CLIENT}={yes,no}.
These variables are used in the new wrapper tests,
test-repo-collections.sh and test-repo-collections-server-only.sh.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The library test previously used the real ~/.cache, while the
library test and the shell-script tests would use the real ~/.config
to look up the XDG user-dirs.dirs. Other home-directory-related code
might have used the real $HOME.
As a general rule, build-time tests should not affect the real home
directory. Debian autobuilders run as a user whose home directory
does not exist, in order to catch packages whose build process could
affect or be affected by the contents of the home directory. This
caused testlibrary to fail when it tried to create that nonexistent
directory, which I think happened while trying to create ~/.cache.
Signed-off-by: Simon McVittie <smcv@debian.org>
These will be useful in upcoming tests, as they are in the right format
to be substituted into a .flatpakref file.
Generated using `gpg2 --homedir test-keyring --armor --export
${FL_GPG_ID}`, then stripping the packet header and removing line
breaks.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This mirrors the same environment variable in OSTree’s unit tests, which
keeps the temporary directory around after tests have completed (or
failed) so the developer can examine it.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This makes the ostree trivial-httpd --autoexit feature work better,
because it seems to exit whenever the root directory changes (i.e. not
only when its deleted).
This means the root dir can't be the repo (because then we can't
update the repo), or the base testdir (because we create files there
too), so instead we make the repo $testdir/repos/test and
$testdir/repos as the httpd root.
Use FLATPAK_TESTS_VALGRIND_LEAKS to check for leaks.
We're not currently passing the leak checks, so this helps letting
us make valgrind checks useful.
This catches regressions in the fix in the previous commit, where old
stale .py[oc] files can wrongly become "unstale" when we change the
.py file mtime to 1.
dbus-launch is X11-specific and contains a lot of legacy code to
support X11 autolaunching. It should not be part of the Wayland
(and/or Mir) future.
Start a dbus-daemon directly instead.
Otherwise, service activation will start a new xdg-document-portal
(because we're using a private DBUS_SESSION_BUS_ADDRESS), but because
XDG_RUNTIME_DIR is shared with the real system, it won't be able to
mount its filesystem.
We need to unmount the document portal before removing the private
XDG_RUNTIME_DIR.
Signed-off-by: Simon McVittie <smcv@debian.org>
Distribution autobuilders are often more locked-down than the
environment in which the distribution binaries actually run.
Signed-off-by: Simon McVittie <smcv@debian.org>
