Commit Graph

92 Commits

Author SHA1 Message Date
Alexander Larsson
b8db46d0e2 tests: Add some tests for subsummaries 2020-10-29 15:30:35 +01:00
Alexander Larsson
12fd8332da tests: Test stuff with the old (non-indexed) summary format too 2020-10-29 15:30:35 +01:00
Alexander Larsson
a32fb1e333 tests: Silence assert_remote_has_no_config
This is meant to fail, so redirect its stderr to get less confusing test
logs.
2020-10-29 15:30:35 +01:00
Alexander Larsson
67d3443999 tests: Fix unnecessary rebuilds of the test platform
The code to use the pre-generated test platform was buggy so it wasn't
used. Also, generate a "stable" branch of it too as that is used
by test-run.sh.
2020-10-29 15:30:35 +01:00
Alexander Larsson
95ba6c77b4 tests: Better logging of which tests are succeeding
This makes it easier to spot where things go wrong in the logs.
2020-10-29 15:30:35 +01:00
Alexander Larsson
356b7fb346 tests: Add basic test for flatpak_dir_list_unused_refs_with_options()
This is currently failing, but I believe that is caused by actual
bugs.
2020-10-08 09:14:51 +02:00
Phaedrus Leeds
617f614d50 Remove EOL runtimes upon app uninstall/upgrade
We normally don't remove a runtime when the last app using it is
uninstalled, since runtimes are large and re-downloading it in the
future may be difficult. But if the runtime is end-of-life, there's a
reasonable chance it won't be used again, so uninstall it in that case.

Similarly, if the last app using a runtime is upgraded to a different
runtime, and the runtime is EOL, uninstall it.

A unit test is included, and the subsequent unit test also had to be
modified. Otherwise we get a "Update is older than current version"
error, since the runtime is installed from test-repo but
setup_repo_no_add() calls make_runtime() which uses the one in
runtime-repo instead, which is older than the one in test-repo.
2020-10-08 09:14:51 +02:00
Alexander Larsson
620157ba12 tests: Allow specifying app id and app branch for updated apps 2020-06-22 10:51:36 +02:00
Matthew Leeds
e847bb31b6 installation: Re-implement list_installed_refs_for_update()
Re-implement flatpak_installation_list_installed_refs_for_update() using
a FlatpakTransaction, so we can guarantee it always gives the same set
of things to update as the update command. This API is used by GNOME
Software and many times in the past g-s has not shown the same list of
apps to be updated as the flatpak CLI. See:
- https://gitlab.gnome.org/GNOME/gnome-software/issues/539
- https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/430

This commit also expands the unit tests for this API, which were already
quite good. Now we test that missing subpaths of locale extensions show
up as updates, and updates that have been pulled but not deployed show
up as well. The latter is a break from how this function used to behave,
but it seems unlikely to break any application.
2020-04-16 09:22:11 +02:00
Alexander Larsson
15c761d482 Use symlinks instead of xa.sideload-repos config option
Instead of having a global config option we scan a directory for
symlinks into the sideload repos. These come from
/var/lib/flatpak/sideload-repos and /run/flatpak/sideload-repos (for
default system installation).

This is much easier to update atomically, and the two different
options are useful for persistant (the first) or dynamic (the second)
usescase.

Fixes https://github.com/flatpak/flatpak/issues/3494
2020-03-26 11:50:54 +01:00
Alexander Larsson
e4df0fa6a6 sideload: Support pulling individual objects from sideload repos
We use the localcache-repos option to ostree_repo_pull to make ostree
directly import any files that are locally available in the sideload
repo even when pulling the main commit from upstream.

This also adds a test that verifies that such files are not
pulled via http.
2020-03-24 11:46:33 +01:00
Alexander Larsson
224b60016f tests: Print calling file+lineno when assert fails 2020-03-23 17:58:04 +01:00
Alexander Larsson
241c5c7f40 tests: Hide unnecessary trace output from the test logs
We use some BASH_XTRACEFD hackery to hide the trace output from
the internals of the assert (and ok) functions, so that the
log output can focus on what is important, i.e. what asserts are
checked and what errors do they print.
2020-03-20 15:37:10 +01:00
Alexander Larsson
b1fdf6c4ef tests: Add and use "ok" helper
This allows us to print some separators for the logs also.

tests: Convert more tests to ok helper
2020-03-20 15:37:10 +01:00
Alexander Larsson
ab5f2dd7e8 tests: Fix gpg signature failure checks
It seems recent ostree reports a different error string for signature
check failures.
2020-03-16 15:01:54 +01:00
Alexander Larsson
cc4ea023f0 tests: Only fail FLATPAK_TESTS_VALGRIND_LEAKS for *definite* leaks 2019-12-19 16:52:36 +01:00
Alexander Larsson
ca5ba625a5 libtest.sh: Add gdb_bt helper
This is unused, but can be inserted in the tests if something is
segfaulting so that you can see the backtrace.
2019-11-26 16:37:01 +01:00
Alexander Larsson
227cde12ba tests: Redo httpd server setup
Rather than this weird shell loop with a timeout and polling we
move the httpd spawning entirely into the python code, and use
a pipe to synchronize the spawning. This way we can also use
the shell job control to properly clean up any running processes
from the test suite.

Additionally, this adds some (lame) support for token handling in the
test webserver, where you for any file foo can create a foo.need_token
containing a token that is needed for that file.
2019-10-09 14:25:44 +02:00
Alexander Larsson
91d9fe9c60 tests: Add simple testing portal backend and wire it up to the tests
This only supports the AccessDialog call, and always just allows
everything. Still, it tests the entire codepath for authentication.
2019-10-02 14:57:11 +02:00
Alexander Larsson
c15c1946ff test: Add test for update-portal monitoring
We add socat to the test runtime, and then we use that to run a
test app outside the sandbox as if it was inside.

The testcase connects creates a monitor and ensure we properly get signals
for updates.
2019-10-02 14:57:11 +02:00
Alexander Larsson
8832e7c7d1 Make test output a bit less noicy
This gets rid of some unnecessary spew that happens in every
test run and just makes the logs harder to read.
2019-10-02 14:57:11 +02:00
Matthew Leeds
9928ce757f libtest.sh: Delete unreachable exit command 2019-07-02 16:12:42 -07:00
Alexander Larsson
b53f937d7e Fix installable tests when running on a tty
This sets FLATPAK_FANCY_OUTPUT=0 in most tests and then special cases
the test_fancy_output() test in case the installable tests are running
on an actual tty. This was causing failures because the added bold escapes
caused some test cases to not generate the expected output.

Closes: #2907
Approved by: alexlarsson
2019-05-16 15:24:47 +00:00
Matthew Leeds
6cfad442c5 tests: Add test-p2p-security.sh
This test is meant to test the issue described in the message for the
commit "dir: Use the right keyring to verify P2P pulls".

Closes: #2705
Approved by: alexlarsson
2019-05-09 09:12:26 +00:00
Alexander Larsson
39946c3959 Support .flatpakrepo files in /etc/flatpak/remotes.d
A *.flatpakrepo file in this directory will be automatically
added as a system remote with the basename (sans extension) as the
name unless that name already exist. Also, once this is done we
record the name in the repo config so that it is not applied again if
the remote is removed.

Closes: #2884
Approved by: alexlarsson
2019-05-08 08:13:52 +00:00
Alexander Larsson
b519b69e38 Add tests for adding remotes from flatpakrefs
Closes: #2884
Approved by: alexlarsson
2019-05-08 08:13:52 +00:00
Simon McVittie
b07b48e206 Skip some tests if we can't use FUSE
On some systems we can't make use of FUSE, especially on the same
locked-down systems where distribution packages are typically built.
For example, official Debian autobuilders (buildds) are configured to
disallow module loading after boot has finished as a form of security
hardening, some build chroots don't have a valid /etc/mtab, and Docker
containers give us uid 0 but not CAP_SYS_ADMIN.

These checks are taken from libostree.

Signed-off-by: Simon McVittie <smcv@collabora.com>

Closes: #2824
Approved by: matthiasclasen
2019-04-15 03:09:00 +00:00
Matthew Leeds
0b20f3d48f run: Try to determine the branch if unspecified
Currently when the user doesn't specify a branch to the run command, for
apps it will use the current branch and for runtimes it just tries to
use "master" and fails if that doesn't work. Since runtimes don't have
current branches, it would be nice if we could determine the right one
to use rather than just erroring out. So this commit changes the
implementation so that it looks at each installed runtime and if only
one matches it is used. If there's more than one match the user is
prompted to choose. This is the first interactivity added to the run
command but I don't think that's an issue; any time it's run in a
non-interactive shell it will error out upon encountering ambiguity.

Also, add a couple unit tests for the success path and error paths.

Fixes https://github.com/flatpak/flatpak/issues/2780

Closes: #2788
Approved by: matthiasclasen
2019-03-30 13:02:10 +00:00
Matthew Leeds
3d74d4b37f tests: Add ability to test a branch other than master
This commit makes it so that a unit test can create the test app and
runtime using a branch other than master, and changes test-run.sh to use
the branch "stable". This will allow the run command to be tested better
in the following commit.

Closes: #2788
Approved by: matthiasclasen
2019-03-30 13:02:10 +00:00
Simon McVittie
752eac6859 test-override: Skip tests that need bwrap if necessary
This requires some new mechanisms: now we're skipping individual tests,
not just whole test scripts.

There are two main reasons why autobuilder environments might not be
able to run these tests successfully, both of which apply in Debian.

Tests that rely on bwrap typically can't pass in builds that take place
in a chroot, because bwrap's use of pivot_root() assumes that the root
directory is a mount point, but a chroot will typically have an unpacked
directory somewhere below the mount point as its root.

Some autobuilder environments are also sufficiently restricted that they
can't create new user namespaces at all, as a way to harden the
autobuilder host.

As a result, Debian autobuilders can't run the majority of the Flatpak
tests. We would like to be able to continue to run the subset that don't
need bwrap, to have the best test coverage we can. For the rest we have
to rely on installed-tests (which I've wired up to Debian's autopkgtest)
rather than using build-time tests.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #2339
Approved by: matthiasclasen
2018-11-29 23:26:14 +00:00
Simon McVittie
8a2a32363f Fix a typo that broke installed-tests
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #2338
Approved by: matthiasclasen
2018-11-23 13:48:10 +00:00
Alexander Larsson
b5d86fe90b tests: Only generate the test platform once
Rather than regenerating it over and over we generate the test
platform as check_DATA and refer to it from the individual tests.
This cuts down "time make check -j8" from 1:25 to 1:06 in user time on
my machine, which is nice in itself, but it is also a stepping stone
for splitting out our large test-scripts into smaller ones where it
will make even more difference.

For the installed-tests case we fall back to creating the platform
once per test.

Closes: #2188
Approved by: alexlarsson
2018-10-05 11:43:12 +00:00
Alexander Larsson
6af4d7d3fe tests: Move basic binary list into make-test-runtime.sh
Rather than specify this in multiple places we now list the most
core binaries needed for the tests inside the script.

Closes: #2188
Approved by: alexlarsson
2018-10-05 11:43:12 +00:00
Alexander Larsson
df93c9b563 tests: Make make-test-{app,runtime}.sh take repo, not repo name
Later we want to use these outside the test suite, and then we
might not use the same repo naming.

Closes: #2188
Approved by: alexlarsson
2018-10-05 11:43:12 +00:00
Simon McVittie
bf31305618 tests: Remove vestigial support for putting Python 2 in a runtime
This is only used in the flatpak-builder tests now, not the main
flatpak tests.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1990
Approved by: alexlarsson
2018-08-27 10:36:28 +00:00
Alexander Larsson
45945c5d45 tests: Add --gen-suppressions=all to valgrind commandline
Closes: #1964
Approved by: mwleeds
2018-08-13 21:31:09 +00:00
Alexander Larsson
fad723dbc4 tests: Run tests on /tmp (not /var/tmp)
Since we now are ok with no-xattrs this means we can run on tmpfs.

Closes: #1839
Approved by: alexlarsson
2018-06-29 09:59:50 +00:00
Alexander Larsson
aa55665ae1 tests: Don't skip if no user xattrs
We don't require these anymore

Closes: #1839
Approved by: alexlarsson
2018-06-29 09:59:50 +00:00
Matthew Leeds
85726180a1 tests: Run P2P tests unconditionally
Now that Flatpak always has P2P support enabled, the tests that are
dependent on it can always run.

Closes: #1804
Approved by: alexlarsson
2018-06-20 07:40:53 +00:00
Alexander Larsson
ffaf2d5386 Cli transaction: Print the full uninstall/install/update lists in ready
This lets you see what operations will happen and you can confirm
it before starting the heavy operations.

Closes: #1797
Approved by: alexlarsson
2018-06-19 08:30:21 +00:00
Matthew Leeds
cdf7c459b8 tests: Forcefully kill the dbus daemon on exit
In the cleanup() function used after each unit test exits, we kill the
dbus daemon process and remove the temporary test directory. However, on
an Endless Mini the daemon doesn't always exit properly before we try to
remove the test directory, leading to an error like:

rm: cannot remove '/var/tmp/test-flatpak-AlmZul/runtime/.dbus-proxy': Device or resource busy

which causes some of the tests to fail with e.g.:

ERROR: tests/test-bundle.sh - exited with status 1

I don't think this is specific to the Mini or to ARM computers; it could
probably happen on any slow computer. So this commit uses "kill -9"
instead of just "kill" to make sure the daemon exits.

Closes: #1642
Approved by: alexlarsson
2018-05-14 07:27:47 +00:00
Joaquim Rocha
1c8dd13f55 tests: Make the app ID in make-test-app.sh a parameter
This is in order to allow tests to make several apps if needed, and will
be useful when e.g. testing repos that have multiple apps with different
collection IDs.

This patch modifies the mentioned script as mentioned and updates the
places that call it.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Alexander Larsson
394130793c uninstall: Pick installation automatically
When uninstalling, if no specific installation was specified with e.g.
--user or --system, automatically chose any unique match, or error
out if there are multiple alternatives.

Fixes #1321
2018-04-04 17:25:20 +02:00
Gunnar Andersson
98c21473fc Fix whitespace issues (remove TAB usage)
Signed-off-by: Gunnar Andersson <gandersson@genivi.org>

Closes: #1107
Approved by: mwleeds
2017-10-17 06:21:07 +00:00
Simon McVittie
f8e5c200bc tests: Use a more realistic bwrap invocation to check for support
The kernel used on some autobuilders for the Debian PA-RISC port
can do "bwrap --ro-bind / / /bin/true", but not the bwrap invocations
made during the actual testing, which fail with "Creating new namespace
failed: Invalid argument". Make the trial bwrap invocation more like
what Flatpak actually does, so that these tests will hopefully be
skipped on such kernels.

Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876743
Signed-off-by: Simon McVittie <smcv@debian.org>
2017-09-26 10:41:49 +02:00
Simon McVittie
e65bcc9fcc Bring back skip_without_user_xattrs helper function
We'll need this to be able to skip system-mode tests without xattrs.
This partially reverts commit a53a752a35.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2017-08-31 16:22:18 +02:00
Simon McVittie
6e46dc8076 Revert "tests: Don't force use of /var/tmp"
This reverts commit 0648452fca.
Unfortunately, it appears we still need xattrs on the system cache.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2017-08-31 16:22:18 +02:00
Simon McVittie
2cfeef6bcc Put FLATPAK_SYSTEM_CACHE_DIR alongside other test data
This makes it easier to reason about. In the absence of
FLATPAK_SYSTEM_CACHE_DIR, it would use /var/tmp even during tests that
prefer /tmp.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2017-08-31 16:22:18 +02:00
Simon McVittie
0648452fca tests: Don't force use of /var/tmp
Now that we don't need user xattrs, a tmpfs is good enough.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2017-08-31 13:35:54 +02:00
Simon McVittie
a53a752a35 tests: Don't require user xattrs
Since Flatpak 0.9.6, we use libostree in bare-user-only mode, which
does not rely on xattrs.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2017-08-31 13:35:54 +02:00