Commit Graph

302 Commits

Author SHA1 Message Date
Matthias Clasen
fd60bfc96c Add a test for transaction with local install
Here we pass a file url instead of a remote name.

Closes: #2245
Approved by: alexlarsson
2018-10-15 13:39:36 +00:00
Matthias Clasen
dc5d78fa33 Add more tests
The tests added here include checks for expected types in
the library (including enum types that are needed for
introspection), and for error handling around names.

Closes: #2245
Approved by: alexlarsson
2018-10-15 13:39:36 +00:00
Matthias Clasen
acd8cfd6bb Add a test for bad remote names
These were not handled gracefully so far.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
6d696b74f5 Test more overrides
Test some of the socket settings.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
5454707464 Add more tests
Test some more instlalation apis, and some error conditions.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
34851f99d6 Add some more tests
Test more options of build-update-repo.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
263a2e5663 Add some more basic tests
Some tests for commandline error handling.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
9f7ddaf678 Add a test for --columns=help
Check that this produces the expected output.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
2ce626b069 Add various new tests
Just driving up coverage incrementally, and finding
bugs along the way.

Closes: #2244
Approved by: alexlarsson
2018-10-15 06:51:58 +00:00
Matthias Clasen
4975840826 Add various new tests
Just driving up coverage incrementally, and finding
bugs along the way.

Closes: #2239
Approved by: alexlarsson
2018-10-12 07:23:45 +00:00
Matthias Clasen
bde3a86333 Adapt tests to the previous commit
One of the transaction tests contains the situation
that was modified in the previous commit. Update it
to expect the new behavior. At the same time, add
some comments to the tests to guide the reader.
2018-10-10 19:26:58 +02:00
Matthias Clasen
03f6b1fcd2 Some test setup fixes
Actually set XDG_RUNTIME_DIR. And assert that the
paths are as we want them to be, from glib's view.

Closes: #2235
Approved by: alexlarsson
2018-10-10 14:42:32 +00:00
Matthias Clasen
10d6330eca tests: Check that updating subpaths works
This test updates an runtime to the same commit, just
changing the subpaths, and makes sure that works as
expected.
2018-10-10 12:21:15 +02:00
Matthias Clasen
ca4d4a9c3b Add a second language to the test app
This will let us test subpaths better.
2018-10-10 12:21:15 +02:00
Matthias Clasen
c8db26cc35 Make the test app not hang
We want an app that keeps running for some of the
library tests, but other tests expect it to return
right away. Make this opt-in.
2018-10-10 12:21:15 +02:00
Matthias Clasen
d4df921940 Modify the instance test
Avoid the initial sleep statement, and instead poll
for the child pid.

We also need the test app to stick around and not
exit right away, so change it to exec a shell.
2018-10-10 12:21:15 +02:00
Matthias Clasen
60948db6ac Add a test for FlatpakInstance
This shows some shortcomings of the current instance
life-cycle management, with sleeps being necessary
to work around races.
2018-10-10 12:21:15 +02:00
Matthias Clasen
ee9cfde582 fixup! Add a test for related refs
Configure the installation to install de, so we have
a known setup that we can test against.
2018-10-10 12:21:15 +02:00
Matthias Clasen
ccbd5b52e9 fixup! tests: Generate a .Locale extension 2018-10-10 12:21:15 +02:00
Matthias Clasen
91f7266f1a Add a test for related refs
Since we have a related ref now, might as well test it.
2018-10-10 12:21:15 +02:00
Matthias Clasen
c7090c5c0b tests: Generate a .Locale extension
Give the test app a Locale extension, so we can test
things around related refs and subpaths. Fix up the
library tests to work with a related ref in the repo.
2018-10-10 12:21:15 +02:00
Matthias Clasen
1709f3bc1b Add a test for --gl-drivers
It is something we can easily test, so why not.
2018-10-10 12:21:15 +02:00
Matthias Clasen
89545de1f5 Add more library tests
Various new tests, mainly about transaction and remote.
2018-10-10 12:21:15 +02:00
Matthias Clasen
a0744f1f80 Exercise all commands in test-basic.sh
We can test that every command has help output.

Closes: #2187
Approved by: alexlarsson
2018-10-05 13:34:49 +00:00
Matthias Clasen
6736c967aa Add a bigger transaction test
This test installs and uninstalls an app using
a transaction, and checks the signals that get
emitted are as expected.

Closes: #2187
Approved by: alexlarsson
2018-10-05 13:34:49 +00:00
Matthias Clasen
e72e0669ce Add some more simple library tests
Simple tests for simple things.

Closes: #2187
Approved by: alexlarsson
2018-10-05 13:34:49 +00:00
Alexander Larsson
b5d86fe90b tests: Only generate the test platform once
Rather than regenerating it over and over we generate the test
platform as check_DATA and refer to it from the individual tests.
This cuts down "time make check -j8" from 1:25 to 1:06 in user time on
my machine, which is nice in itself, but it is also a stepping stone
for splitting out our large test-scripts into smaller ones where it
will make even more difference.

For the installed-tests case we fall back to creating the platform
once per test.

Closes: #2188
Approved by: alexlarsson
2018-10-05 11:43:12 +00:00
Alexander Larsson
6af4d7d3fe tests: Move basic binary list into make-test-runtime.sh
Rather than specify this in multiple places we now list the most
core binaries needed for the tests inside the script.

Closes: #2188
Approved by: alexlarsson
2018-10-05 11:43:12 +00:00
Alexander Larsson
df93c9b563 tests: Make make-test-{app,runtime}.sh take repo, not repo name
Later we want to use these outside the test suite, and then we
might not use the same repo naming.

Closes: #2188
Approved by: alexlarsson
2018-10-05 11:43:12 +00:00
Alexander Larsson
651785c974 tests: Don't dist the .wrap files (as they don't exist) 2018-10-04 11:22:50 +02:00
Alexander Larsson
43c7e7eeec tests: Don't generate the .wrap tests, launch them via test-wrapper.sh
There is no need for these files to be generated on disk

Closes: #2170
Approved by: alexlarsson
2018-10-03 13:19:35 +00:00
Alexander Larsson
5ab1b99d62 tests: Run in /tmp
These days we don't really need xattrs for the tests to run,
so move the test runs to /tmp, which brings down make check time
from 5 to 3.5 minutes on my machine.

Closes: #2170
Approved by: alexlarsson
2018-10-03 13:19:35 +00:00
Alexander Larsson
5fa3fbf19e tests: Autogenerate test matrix
Several tests have alternative wrappers that just set some feature
flags. For instance to switch between user and system and with or
without deltas. This change makes those wrappers automatically
generated from a simple description of the features.

For example the test-run.sh test goes from:
-       tests/test-run.sh \
-       tests/test-run-system.sh \
-       tests/test-run-deltas.sh \
-       tests/test-run-system-deltas.sh \
to
+       tests/test-run.sh{{user+system},{nodeltas+deltas}} \

There is some complexity here because the TESTS variable in automake
can't be runtime generated, so we rewrite it during "make
update-test-matrix" and commit the result to git.

The way it works is that make update-test-matrix takes the
TEST_MATRIX_SOURCE list, passes it to tests/expand-test-matrix.sh
which generates all the combinations in tests/Makefile-test-matrix.am.inc
where a test like tests/test-foo.sh{user+system} will get
converted to tests/test-foo.sh@user.wrap and tests/test-foo.sh@system.wrap.

We then have a pattern rule matchin tests/*.wrap which will generate
a wrapper that sets the right environment flags based on $0 and then
spawns the wrapper.

Closes: #2170
Approved by: alexlarsson
2018-10-03 13:19:35 +00:00
Matthias Clasen
823511b87a Update tests for changed errors
Great that we're actually testing these.

Closes: #2150
Approved by: alexlarsson
2018-10-02 06:21:49 +00:00
Matthias Clasen
3eed1f42a5 Add more tests
Test some more of the FlatpakRef api, including properties.

Closes: #2108
Approved by: alexlarsson
2018-09-24 08:18:10 +00:00
Simon McVittie
36e3644d18 test-webserver: Fix race condition
If we rely on the background subshell to create the httpd-output file,
and we are unlucky, then the "cp" invocation in the loop might execute
before the file has been created, and fail. This appears to have
happened on Debian's arm64 autobuilder, which failed with:

    Running web server: PYTHONUNBUFFERED=1 setsid python3 /<<PKGBUILDDIR>>/tests/http-utils-test-server.py 0
    Web server pid: 13319
    Waiting for web server (1/300)...
    cp: cannot stat '/var/tmp/tap-test.p1cxRN/httpd-output': No such file or directory

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #2098
Approved by: alexlarsson
2018-09-24 08:06:51 +00:00
Matthew Leeds
48ceed70be Replace xa.collection-id with ostree.deploy-collection-id
Currently Flatpak reads a repo metadata key called "xa.collection-id" to
decide whether to configure a collection ID on a remote. This commit
drops support for xa.collection-id and replaces it with
ostree.deploy-collection-id, which is a key defined by OSTree but so far
only implemented here.

The reason for the change is so that collection IDs can only be deployed
to users running recent versions of Flatpak and OSTree. The current
situation is that Endless wants to deploy collection IDs (and therefore
support for doing USB updates) to users, but servers such as Flathub
can't safely set xa.collection-id in their metadata, because many users
are still using old versions of Flatpak and OSTree[1] which would hit
various bugs[2][3][4] on the P2P code paths that are enabled by
collection IDs. Defining a new key means that only users running
recent (as-yet-unreleased) versions of Flatpak and OSTree will pay
attention to it and deploy the collection ID, leaving the users on old
versions unaffected.

The reason this metadata key is being defined at the level of OSTree
instead of Flatpak, is that OSTree may want to implement it in the
future.

The functionality of deploying a collection ID by setting the
"deploy-collection-id" key in the config on the server side (which in
turn causes ostree.deploy-collection-id to be set) is already covered by
the unit tests in test-update-remote-configuration.sh; this commit just
tweaks them to use the new key.

Another solution I proposed to this problem was to have a key
"eos.collection-id" which would only be understood by the Endless fork
of Flatpak, and use that temporarily until enough people are running
recent versions of Flatpak, at which point "xa.collection-id" can be
used. But this solution (abandoning xa.collection-id upstream) allows us
to avoid that migration step and leave users on old versions of Flatpak
completely unaffected.

[1] https://ahayzen.com/direct/flathub.html#downloadsbyflatpakstacked
[2] https://github.com/ostreedev/ostree/commit/e4e6d85ea
[3] https://github.com/flatpak/flatpak/commit/5813639f
[4] https://github.com/flatpak/flatpak/commit/5b21a5b7
2018-09-19 17:23:08 -07:00
Simon McVittie
155d9116de test-webserver: Be more verbose about what we're doing
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1990
Approved by: alexlarsson
2018-08-27 10:36:28 +00:00
Simon McVittie
3224e26b0a Make test scripts bilingual Python 2/Python 3
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://github.com/flatpak/flatpak/pull/1990

Closes: #1990
Approved by: alexlarsson
2018-08-27 10:36:28 +00:00
Simon McVittie
bf31305618 tests: Remove vestigial support for putting Python 2 in a runtime
This is only used in the flatpak-builder tests now, not the main
flatpak tests.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1990
Approved by: alexlarsson
2018-08-27 10:36:28 +00:00
Simon McVittie
13fee9a9ee tests: Look for httpcache in ${test_builddir}, not PATH
For installed-tests, the installed test directory is not on the PATH.
To make this easier, put the uninstalled binary in tests/, so that
in both build-time and installed tests, it is in ${test_builddir}.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1989
Approved by: alexlarsson
2018-08-27 10:26:08 +00:00
Simon McVittie
ae0d7943a5 build: Install httpcache if installed-tests are enabled
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1989
Approved by: alexlarsson
2018-08-27 10:26:08 +00:00
Alexander Larsson
3f3625819b Dist tests/oci-registry-[server|client].py
These are needed for make distcheck to pass.
2018-08-20 10:50:57 +02:00
Alexander Larsson
6bb492cb6b test: Suppress weird thread-related stream leak in ostree
Closes: #1966
Approved by: alexlarsson
2018-08-17 08:13:15 +00:00
Alexander Larsson
45945c5d45 tests: Add --gen-suppressions=all to valgrind commandline
Closes: #1964
Approved by: mwleeds
2018-08-13 21:31:09 +00:00
Alexander Larsson
6a551c2ba7 Add suppression for ostree pthread_create issue
It seems like pthread_create allocates some tls that is not
immediately cleaned up. Suppress this for now.

Closes: #1964
Approved by: mwleeds
2018-08-13 21:31:09 +00:00
Owen W. Taylor
90be1f7515 test-oci-registry.sh: extend
Add:
 * Testing of flatpakrefs and bundles pointing to OCI remotes
 * Changing a remote from OCI to non-OCI, including during bundle
   installation.
 * Pruning origin remotes.
2018-08-13 11:23:28 +02:00
Owen W. Taylor
d7d05a8619 Use oci+http[s]:// as an URL to identify OCI registries
The old pattern of using a separate 'OCI' flag was very ugly
internally in the code once it was extended to flatpak bundles and
flatpakrefs - using a different URI scheme means that the nature
of the remote can't be accidentally lost in some part of the code.

Probing would be possible as well, but would make it difficult to
add a remote when offline, and also doesn't deal well with the
fact that our data layout is different for the two types of remotes -
the type of remote could change at any point!

As a side effect this change enables flatpakrefs and flatpak bundles for OCI
registries.
2018-08-13 11:23:28 +02:00
Alexander Larsson
3fa34ebd8a Work around libsoup recursive type hang in tests
Closes: #1952
Approved by: alexlarsson
2018-08-09 15:45:08 +00:00
Owen W. Taylor
8a32f0e718 Remove FLATPAK_ENABLE_EXPERIMENTAL_OCI envvar check
Now that appstream data and icons are retrievded from the index, the OCI
code is expected to be fully usable, and to work with registries as they
will actually be deployed.

Closes: #1910
Approved by: alexlarsson
2018-08-09 12:49:36 +00:00