The output might be written to the pipe by `flatpak --help` and/or read
from the pipe by `head -2` in more than one batch. If `head -2` reads
the first two lines before `flatpak --help` has written everything, it
will exit, causing the pipe to have no process at the read end. This
results in `flatpak --help` being killed by `SIGPIPE` next time it tries
to write to the pipe, because it has not opted out of this behaviour
(as shell tools usually shouldn't).
We're running under `set -o pipefail`, so this causes a nonzero exit
status that makes the test fail. Worse, this failure is intermittent,
because `head -2` *usually* doesn't exit until `flatpak --help` has
already written out everything it is going to write - it depends on
the precise behaviour of read(), write() and kernel scheduling.
We know that `flatpak --help` output is not *that* long, so it's OK
for `flatpak --help` not to be terminated early: we can send it all
into an intermediate file, and then run `head` on the file.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This has no practical effect (assuming environment variables are unique),
but it makes it easier to find an environment variable of interest
in a very long bwrap command-line.
Signed-off-by: Simon McVittie <smcv@collabora.com>
On a decently fast system, some files may be fetched, modified and then
fetched again within the same second. In that case, the web server
replies with a code 304 ("Not modified") to the 2nd query, causing some
tests to fail.
This commit forces the web server to ignore `If-Modified-Since` HTTP
headers, effectively disabling caching in order to mitigate the problem.
Distributions run these tests on other architectures, but hard-coding
x86_64 to look for in output dooms that to failure.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Currently if a user of libflatpak wants to list the related refs (such
as extensions and plugins) of something, they have three options:
1. They can parse the metadata manually with e.g.
flatpak_remote_ref_get_metadata() and then key-file operations, but
this means re-implementing parts of libflatpak and using key file
strings that are not actually public (FLATPAK_METADATA_KEY_...).
2. They can use flatpak_installation_list_installed_related_refs_sync()
but this only works for installed related refs not remote ones.
3. They can use flatpak_installation_list_remote_related_refs_sync() but
this lists all remotely available related refs, including ones that
may not be compatible with the installed version of the main ref
(because they don't match any of the values in the "versions"
metadata key).
So since none of these provide a way to get the remote related refs
corresponding to an installed application, add new API for that. For the
motivation of this see
https://gitlab.gnome.org/GNOME/gnome-software/-/issues/1132
There are various reasons why distro autobuilder environments might not
be able to use FUSE in build-time tests: they might not have
fusermount(1), they might be locked-down to be unable to load the FUSE
kernel module as a result of security hardening, they might not have the
/etc/mtab symlink, they might not have CAP_SYS_ADMIN in their capability
bounding set, or they might not have write access to /dev/fuse. Try to
check for all of these conditions.
Heavily based on code that I previously contributed to
xdg-desktop-portal. Disabling revokefs, instead of skipping the test
that needs it, is based on the patches applied in @alexlarsson's PPA.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This follows up from GHSA-4ppf-fxf6-vxg2 to fix missing functionality
that I noticed while resolving that vulnerability, but is not required
for fixing the vulnerability.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Some consumers of environment variables distinguish between present
with an empty value and absent. For example, if an environment variable
represents a search path like VK_ICD_FILENAMES, unsetting it often
results in use of a default, but setting it to the empty string results
in not searching any locations, which is sometimes what is desired.
The shell syntax "${BAR-unset}" expands to the value of ${BAR} if it
is set to anything (even an empty string), or to "unset" if not.
We can use that in the unit test to check that BAR is set to the
empty string in this case.
This follows up from GHSA-4ppf-fxf6-vxg2 to fix an issue that I noticed
while resolving that vulnerability, but is not required for fixing the
vulnerability.
Signed-off-by: Simon McVittie <smcv@collabora.com>
For the portal's use of --env-fd= to be safe, we want the environment
variables that it sets to end up in the environment for the program
that is run by `bwrap` as process 2, but they must not go into the
environment that gets used to run `flatpak run` or `bwrap`. Assert
that this is the case.
For completeness, we're testing both --env= and --env-fd= here,
even though the earlier commit
"portal: Do not use caller-supplied variables in environment"
always uses --env-fd=.
Part-of: https://github.com/flatpak/flatpak/security/advisories/GHSA-4ppf-fxf6-vxg2
Signed-off-by: Simon McVittie <smcv@collabora.com>
fusermount requires /etc/mtab, but not all Debian buildd chroots
have that file, either as a regular file or as a symlink to
/proc/self/mounts.
(This is similar to commit b07b48e2, but for C code.)
Signed-off-by: Simon McVittie <smcv@debian.org>
This is the environment needed to use flatpaks; a following commit will
hook this up to the systemd environment generator.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
After the --redirect-url= test is finished it leaves the remote with a
different url, which is confusing the other tests.
In particular, at some point we remove the remote and add it back,
which gets us back to the old url, but at that point the summary (from
the newer url) has a later mtime than the old one so the old one keeps
getting used.
So, we unset the redirect and set back the old url. Also, the following test
had to be tweaked for this change.
We enforce --no-update-summary when we create test apps and
runtimes, and then we ensure we always manually call update_repo
after all modifications are done.
This means we save work avoiding summary updates, but it also means we
can do special handling in update_summary and guarantee that this is
the only place this happens. For example, we want this to work around
the mtime handling of summary updates.
We remember what action we took for EOLs, and for sub-refs (ie .Locale)
we reuse that.
Also, we show if eol:ed refs are pinned (as that makes them not be
auto-uninstalled), and we list the apps that use the eol:ed runtime
ref.
Example run:
```
Looking for updates…
Info: (pinned) org.gnome.Sdk.Compat.i386 is end-of-life, with reason:
The GNOME 3.34 runtime is no longer supported as of 14th August 2020. Please ask your application developer to migrate to a supported platform.
Info: org.gnome.Platform is end-of-life, with reason:
The GNOME 3.32 runtime is no longer supported as of 11th March 2020. Please ask your application developer to migrate to a supported platform.
Applications using this runtime:
org.gnome.HexGL
```
Currently if an app has been renamed and you run "flatpak update" you
will be prompted to decide whether to install the new name of the app
(and uninstall the existing one). However, in such a situation
flatpak_installation_list_installed_refs_for_update() does not return
the app, and therefore gnome-software doesn't see any updates available
(except in the case where the installed commit is older than the commit
with the eol-rebase metadata).
So include end-of-life-rebased apps in the returned list. Patches to
make gnome-software update such apps properly are forthcoming.
I checked that the included unit test only passes with this patch to
libflatpak.
This means we do a bit less string operations, but it also allows us
to pick up the collection-id when listing refs from a file: uri, fixing
the issues described in https://github.com/flatpak/flatpak/pull/3909
This slightly changes the behaviour of collection ids and remotes.
When listing regular remotes we only ever return the refs with the
collection id specified by the configuration of that remote. However,
when specifying a file: uri for the remote name we return all the refs
in the repo, with their corresponding collection id.
This means the test suite has to change a bit as before we returned
all the collrefs from the remote, not just the one matching the configured
collection id.
getters now have three forms:
const char *foo_peek() returns pointer into data and length
const char* foo_get() returns zero terminated pointer into data
char *foo_dup() returns allocated zero terminated pointer
Not everything has all of these (as that is not always possible), but
it is clear what each one does.
We also now support storing a collection id for a
FlatpakDecomposed. This is not typically done (as we use the source
remote to define the collecion id and don't expose this concept to the
user). However, for the case when directly listing a local repo (such as
a side-load repo) we need this.
We now validate the arch string. The valid chars are a-z, A-Z, 0-9 and
_. This was't previously verified, but no arches in existance have other
chars.
There is a new flatpak_decomposed_new_from_decomposed() which lets you
modify part of another ref. Say replace the arch or the branch and
keep the rest the same.
These now take (optional) string length args:
flatpak_id_has_subref_suffix()
flatpak_levenshtein_distance()
flatpak_is_valid_name()
flatpak_is_valid_branch()
The code to use the pre-generated test platform was buggy so it wasn't
used. Also, generate a "stable" branch of it too as that is used
by test-run.sh.
This way we can get the proper eol status for the new to-be-installed
refs, rather than whatever was previously installed. This allows us to
detect when a runtime is updated and the new one is eol, and nothing
uses it, so it can be auto-uninstalled.
Rather than trying to figure out which runtimes are affected byt
the current setup of ops we run flatpak_dir_list_unused_refs() twice,
once with and once without the changes the transaction will cause.
Any unused refs after the transaction that were not unused before are
caused by the transaction and we start uninstall ops for those.
Also rename flatpak_dir_list_unused_refs_with_options() to
flatpak_dir_list_unused_refs() as it need not be so long.
In case a runtime becomes unused and then later becomes End-Of-Life, it
is currently not removed. So this commit removes such runtimes in the
update command, as discussed in #2639. A unit test is included.
I am planning to propose to use the FlatpakTransaction API added here in
gnome-software, so that users don't have to use the CLI at all for
runtimes to be cleaned up. KDE Discover already removes unused runtimes
periodically.
We normally don't remove a runtime when the last app using it is
uninstalled, since runtimes are large and re-downloading it in the
future may be difficult. But if the runtime is end-of-life, there's a
reasonable chance it won't be used again, so uninstall it in that case.
Similarly, if the last app using a runtime is upgraded to a different
runtime, and the runtime is EOL, uninstall it.
A unit test is included, and the subsequent unit test also had to be
modified. Otherwise we get a "Update is older than current version"
error, since the runtime is installed from test-repo but
setup_repo_no_add() calls make_runtime() which uses the one in
runtime-repo instead, which is older than the one in test-repo.
