This will be useful in gnome-software's flatpak plugin, which currently
iterates over the operations a few times, and it will be useful
internally as well.
This avoids polluting stdout with the output of the launched app,
which would corrupt the test's TAP output. With Autotools, we get away
with it, because Automake's TAP parser is permissive, but Meson's TAP
parser is more strict.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Recent Meson versions have warnings if you add the subprojects
directory as an include path, because the way Meson wants to consume
subprojects is by the subproject's build system producing a Meson
dependency object that encapsulates its include directory. Flatpak
doesn't have a Meson build system yet, but I'm working on that.
libglnx seems to be set up to have the libglnx directory be its include
path instead: for example, ostree (by the author of libglnx) already
uses "libglnx.h" or <libglnx.h> everywhere. Do the same here.
Signed-off-by: Simon McVittie <smcv@collabora.com>
On two different code paths we were using the "Title" field in
flatpakref files as the title of a remote, which is incorrect. In most
cases, the remote added via the RuntimeRepo key will be the same as the
remote the app is from, so when the remote is added for the runtime, its
title will be correctly set using the Title value from the flatpakrepo
file and the app will therefore have an origin remote with a title set.
This is not currently true for flatpakref files that use
SuggestRemoteName=, see https://github.com/flatpak/flatpak/pull/4513
For flatpakref files that use a different remote than the RuntimeRepo,
we don't currently have a way for the title to be set automatically;
perhaps we should (https://github.com/flatpak/flatpak/issues/4512).
Fixes https://github.com/flatpak/flatpak/issues/4499
It can happen that a related ref is installed from a different remote
than the thing it's related to. We always want to update things from
their origin remote. However as of now FlatpakTransaction resolves the
commit of a related ref to the one available from the main ref origin,
and later sets the remote for the operation to the installed origin (see
commit 6793d90b8). In case there is a newer commit in the main ref
origin than the installed origin, this leads to an update operation
being erroneously created, only to then error out with an HTTP 404
error, because the commit from the main ref origin is being pulled from
the installed ref origin. For specific steps to reproduce see
https://github.com/flatpak/flatpak/issues/3128#issuecomment-948948040
So, ensure that when a FLATPAK_TRANSACTION_OPERATION_INSTALL_OR_UPDATE
operation is created for something that's installed, whether it's a
related ref or something else, the remote used is always the origin. And
ensure that the remote is set correctly before the stage where the op is
resolved to a commit, to avoid the situation described above. This is
essentially a re-implementation of the fix in commit 6793d90b8.
Also, add a unit test for this behavior.
This commit also makes a few changes to documentation to make it clear
that this related-ref-different-origin situation is possible.
Fixes#3128
This gives new support for the new XDG_STATE_HOME addition to XDG_BASE_DIRS
which allows applications to use this without breaking because they would
assume $HOME/.local/state which may be unavailable to the flatpak
This adds it as .local/state as to make --persist=.local/state the same behaviour
as in new flatpak. This in turn means that the transition should be seamless between
old and new flatpak.
This also has the benefit of working if the application doesn't follow XDG spec thanks
to --persist=.local/state.
This fixes https://github.com/flatpak/flatpak/issues/4477
[smcv: Don't call nonexistent g_get_user_state_dir(); fix a reference
to XDG_STATE_DIR]
In a recent commit I changed flatpak_dir_list_remote_refs() to make main
refs (xa.main-ref) visible for noenumerate remotes (xa.noenumerate),
meaning that an origin remote created for a flatpakref file can be used
to get information about the ref from the file even before it is
installed. This commit extends that work to include extensions of the
main ref, and updates the unit test to check for this. The reasoning is
that if GNOME Software wants to show the user information about, say,
VLC from a flatpakref file before installing it, Software would want to
also show the available plugins so the user can decide if they want to
install those as well. I haven't checked if Software actually does that;
I'm only focusing on making libflatpak do the right thing.
Currently the xa.noenumerate option on a remote is documented as causing
the remote not to be used when presenting available apps/runtimes or
when searching for dependencies. The idea is that the remote is only
used for providing updates for things installed from it, and this
functionality is used when creating an origin remote for something
installed via a flatpakref file.
However, the implementation of this in flatpak_dir_list_remote_refs() is
buggy. It returns an empty set of refs even if something is both locally
installed and available from the remote. This is because it is using
hash table comparisons of FlatpkDecomposed objects (via
flatpak_decomposed_hash()) which take into account both the ref (or
refspec) and the collection ID, and the local refs' FlatpakDecomposed
objects are created from a refspec whereas the remote refs'
FlatpakDecomposed objects are created from a ref alone. We could fix
this by having them both use refspecs, but it is better to use a
collection-ref tuple for the following reasons:
(1) Changing flatpak_dir_list_all_remote_refs() to use a refspec to
create the FlatpakDecomposed objects would be a breaking change for the
other users of that function.
(2) Both the local and remote refs are from the same remote so we don't
need to use the remote name to disambiguate them, even if no collection
ID is configured.
(3) The whole point of collection IDs is to make refs uniquely
identifiable, so we're using them for the intended purpose.
In addition to fixing this bug, this commit adds a unit test in
testlibrary.c so it stays fixed.
When we create origin remotes for apps installed via .flatpakref files,
we set xa.noenumerate=true and
xa.main-ref=app/com.example.App/arch/branch so that the remote is only
used for the app it was intended for. This is implemented in
flatpak_dir_list_remote_refs() by only listing refs in the remote which
are already installed. This works fine after the ref is installed but in
the short timespan between when the origin remote is created and the app
is installed from it, it means that the remote appears as empty.
The use case where I ran into this is in attempting to use
flatpak_installation_fetch_remote_ref_sync() in the gnome-software
flatpak plugin, in the handling of flatpakref files, which is intended
to just display information to the user and let them decide whether to
install the app. But I was not able to create a FlatpakRemoteRef due to
the behavior described above; see #4453 for more details.
So, change the behavior so that the main ref for an origin remote is
visible even before it is installed. This technically could be a
breaking change for some consumer of libflatpak but that seems very
unlikely, and the new behavior makes more sense.
Also add a unit test for this behavior.
Currently the output produced when assert_remote_in_installation() or
assert_remote_not_in_installation() fails is not helpful: it doesn't
mention the remote name or where the assertion was called from. Fix
those problems by rewriting it as a #define.
g_key_file_get_boolean() only accepts lower case "true" and "false", so
correct some instances of "False" in key files. This makes no functional
difference as long as the error pointer isn't checked since
g_key_file_get_boolean() also returns false when it can't parse the
value as a boolean. But it still seems good to be correct, and someone
could interpret the existence of "IsRuntime=False" as implying that
"IsRuntime=True" will work, which it doesn't.
scan-build detected that res was written but never read. Presumably
the use of ref here (carried over from the previous test) is a
copy/paste error.
Signed-off-by: Simon McVittie <smcv@collabora.com>
scan-build has a lot of false positives for this codebase because it
doesn't understand __attribute__((__cleanup__)) or GLib's GError
convention, but it seems to have been right about these.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Currently if a user of libflatpak wants to list the related refs (such
as extensions and plugins) of something, they have three options:
1. They can parse the metadata manually with e.g.
flatpak_remote_ref_get_metadata() and then key-file operations, but
this means re-implementing parts of libflatpak and using key file
strings that are not actually public (FLATPAK_METADATA_KEY_...).
2. They can use flatpak_installation_list_installed_related_refs_sync()
but this only works for installed related refs not remote ones.
3. They can use flatpak_installation_list_remote_related_refs_sync() but
this lists all remotely available related refs, including ones that
may not be compatible with the installed version of the main ref
(because they don't match any of the values in the "versions"
metadata key).
So since none of these provide a way to get the remote related refs
corresponding to an installed application, add new API for that. For the
motivation of this see
https://gitlab.gnome.org/GNOME/gnome-software/-/issues/1132
There are various reasons why distro autobuilder environments might not
be able to use FUSE in build-time tests: they might not have
fusermount(1), they might be locked-down to be unable to load the FUSE
kernel module as a result of security hardening, they might not have the
/etc/mtab symlink, they might not have CAP_SYS_ADMIN in their capability
bounding set, or they might not have write access to /dev/fuse. Try to
check for all of these conditions.
Heavily based on code that I previously contributed to
xdg-desktop-portal. Disabling revokefs, instead of skipping the test
that needs it, is based on the patches applied in @alexlarsson's PPA.
Signed-off-by: Simon McVittie <smcv@collabora.com>
fusermount requires /etc/mtab, but not all Debian buildd chroots
have that file, either as a regular file or as a symlink to
/proc/self/mounts.
(This is similar to commit b07b48e2, but for C code.)
Signed-off-by: Simon McVittie <smcv@debian.org>
Currently if an app has been renamed and you run "flatpak update" you
will be prompted to decide whether to install the new name of the app
(and uninstall the existing one). However, in such a situation
flatpak_installation_list_installed_refs_for_update() does not return
the app, and therefore gnome-software doesn't see any updates available
(except in the case where the installed commit is older than the commit
with the eol-rebase metadata).
So include end-of-life-rebased apps in the returned list. Patches to
make gnome-software update such apps properly are forthcoming.
I checked that the included unit test only passes with this patch to
libflatpak.
This means we do a bit less string operations, but it also allows us
to pick up the collection-id when listing refs from a file: uri, fixing
the issues described in https://github.com/flatpak/flatpak/pull/3909
This slightly changes the behaviour of collection ids and remotes.
When listing regular remotes we only ever return the refs with the
collection id specified by the configuration of that remote. However,
when specifying a file: uri for the remote name we return all the refs
in the repo, with their corresponding collection id.
This means the test suite has to change a bit as before we returned
all the collrefs from the remote, not just the one matching the configured
collection id.
We normally don't remove a runtime when the last app using it is
uninstalled, since runtimes are large and re-downloading it in the
future may be difficult. But if the runtime is end-of-life, there's a
reasonable chance it won't be used again, so uninstall it in that case.
Similarly, if the last app using a runtime is upgraded to a different
runtime, and the runtime is EOL, uninstall it.
A unit test is included, and the subsequent unit test also had to be
modified. Otherwise we get a "Update is older than current version"
error, since the runtime is installed from test-repo but
setup_repo_no_add() calls make_runtime() which uses the one in
runtime-repo instead, which is older than the one in test-repo.
Avoid shadowing variables that are already declared in a previous scope,
and make such occurrences compile-time errors. These are not functional
changes.
In a few places do related code cleanup.
A similar ostree PR is here:
https://github.com/ostreedev/ostree/pull/2195
If a runtime is installed explicitly rather than as a dependency, pin it
so it doesn't get automatically removed when unused runtimes are being
removed. We do this because the runtime might be installed for
development or other uses.
This commit also rearranges some code in the mask and pin commands, and
adds a unit test.
Currently when searching for a remote to provide the runtime for an app,
we search remotes in priority order. This commit makes it so we search
the remote providing the app before others with the same priority, and
otherwise still search in priority order. This means for the common
case where every remote has the default priority of 1, the app's origin
will have the first chance to provide the runtime. This behavior seems
logical, but the impetus for this change was also to keep a unit test
passing in eos-updater[1] after a port to FlatpakTransaction.
Originally the eos-updater unit test was written to prioritize the
origin remote regardless of the priorities on any other remote, but
during code review it was decided to let higher priority remotes stay
above the app's origin.
In practice it's usually true that only one remote provides a runtime
and priorities aren't set at all, so this is an edge case that probably
doesn't come up much.
A unit test and documentation updates are included.
[1] eede0a8b9c/tests/test-update-install-flatpaks.c (L1919)
Don't leave options set on the remote in one of the unit tests. If
something should be set for every test it should be done in
global_setup(). This commit also changes the FlatpakRemote
implementation to allow unsetting title and default branch.
Re-implement flatpak_installation_list_installed_refs_for_update() using
a FlatpakTransaction, so we can guarantee it always gives the same set
of things to update as the update command. This API is used by GNOME
Software and many times in the past g-s has not shown the same list of
apps to be updated as the flatpak CLI. See:
- https://gitlab.gnome.org/GNOME/gnome-software/issues/539
- https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/430
This commit also expands the unit tests for this API, which were already
quite good. Now we test that missing subpaths of locale extensions show
up as updates, and updates that have been pulled but not deployed show
up as well. The latter is a break from how this function used to behave,
but it seems unlikely to break any application.
Most code that looks for a regular collection id set on the remote is
removed, as these should never happen in flatpak repo setups now.
Some is replaces with looking at xa.sideload-collection-id:
* The libflatpak FlatpakRef::collection-id property now comes comes from the sideload id
* Various CLI commands showing or changing the collection-id for a remote now uses the sideload id
* Collection id deploy in update now sets the sideload-collection-id instead
* Setting the collection id for a remote in libflatpak now sets the sideload id
Additionally we now delete the code that allows unsigned summaries
when there is a collection id (because there is none).
create-usb now uses the sideload id as as collection id source when exporting.
The direct repo operations (export, bundle, commit-from) still support
collection ids, because on the server we do want to set it so that we
can sideload.
This adds a xa.sideload-collection-id option to the remote
configuration and a global xa.sideload-repos option (which is a list
of paths to local repos).
When resolving or listing refs, if we fail to download the real remote
summary (i.e. we're offline) then we instead look into the configured
sideloaded repos for refs that match ref and the sideloaded collection
id for the remote.
For the transaction to resolve the ref we need more metadata. In the
regular summary case we use the metadata from the summary, but that
is not available in the (partial) summary in the sideload repo, so
there we load the actual commit object and use the data from there.
(The ostree-metadata branch is not used/needed.)
This actually also fixes a longstanding issue when you "flatpak update
--checksum=XYZ" because we now handle this correctly by downloading
the commit object from the remote. Before we used the metadata in the
summary which is not right for non-HEAD commits.
To handle the sideloading we record the path to the sideload repo
when sideloading and pass the url to the repo as the remote name
when pulling, which will do a direct local pull.
We avoid using sideloaded refs when offline if the timestamp in the
commits is older than what is already installed locally.
In test_transaction_install_local(), we test that the origin remote
created when installing from a local repo doesn't exist before
flatpak_transaction_run() is executed and does exist afterward. However,
the origin remote is created before the transaction is run; see the
flatpak_dir_create_origin_remote() call in
flatpak_transaction_add_ref(). The only reason this discrepancy has not
caused a test failure is that the FlatpakDir object held by the
FlatpakInstallation object is not reloaded when the origin remote is
added (so it's reading an old copy of the repo config). This issue will
be fixed in the commit following this one.
Mark installed apps to be updatable if an installed in missing
its runtime for some reasons. In pre-FlatpakTransaction world,
an app migration from runtime X to runtime Y would have rendered
an app unusable because runtime Y would not be installed
automatically by clients like gnome-software.
The goal here is that clients like gnome-software can show
those installed apps as updatable again, if such a situation
arises. The FlatpakTransaction API can automatically resolve one of
its ops to install the new runtime, provided we can mark the app
as updatable again at the first place.
While updating, if the related extension is missing on
the installation of an installed ref (could be an app or
runtime), FlatpakTransaction tends to "repair" the ref by
automatically downloading the related extension again and
restoring the overall functionality of the ref.
The related extension concerned that are the ones associated with
`should-download` to TRUE only.
Hence, teach the libflatpak API to do that same, so that clients
like gnome-software can mark those refs as updatable, if their
related extensions is missing.
FlatpakTransaction will automatically download the related ref
(.Locale ref in this case). In case of mentioning previous-ids
in the deploy file, the related ref(.Locale) should be also
be requiring an update.
Hence, flatpak_installation_list_installed_refs_for_update now
will return 2 refs (app and related .Locale extension) to be
updated, instead of 1.
Also update the tests to use the same format, and make sure to include
coverage of all forms of locale (language-only, with locale, with
codeset, and with modifier).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to configure gnome-software to show specific apps in one region
without showing to all language speakers, we allow the storage of full
locales on the extra-languages key. However, these locales are ignored when
calling flatpak_installation_get_default_languages, so locales will be reduced
to their language identifier (eg. en_IN locale will be returned as 'en', and
az_Latn_AZ will be returned as 'az'). In order to get the full locales, we can
call flatpak_installation_get_default_locales instead, which can return languages
and locales.
This ensures that when the operation to install the app completes, the
app is ready to run — rather than having to wait for subsequent
operations to install extensions which the app depends on (such as
content, or locales).
This fits in with the current ordering for *un*installing apps, where
the main app is uninstalled first, followed by its extensions.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/flatpak/flatpak/issues/3017
