Re-implement flatpak_installation_list_installed_refs_for_update() using
a FlatpakTransaction, so we can guarantee it always gives the same set
of things to update as the update command. This API is used by GNOME
Software and many times in the past g-s has not shown the same list of
apps to be updated as the flatpak CLI. See:
- https://gitlab.gnome.org/GNOME/gnome-software/issues/539
- https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/430
This commit also expands the unit tests for this API, which were already
quite good. Now we test that missing subpaths of locale extensions show
up as updates, and updates that have been pulled but not deployed show
up as well. The latter is a break from how this function used to behave,
but it seems unlikely to break any application.
Most code that looks for a regular collection id set on the remote is
removed, as these should never happen in flatpak repo setups now.
Some is replaces with looking at xa.sideload-collection-id:
* The libflatpak FlatpakRef::collection-id property now comes comes from the sideload id
* Various CLI commands showing or changing the collection-id for a remote now uses the sideload id
* Collection id deploy in update now sets the sideload-collection-id instead
* Setting the collection id for a remote in libflatpak now sets the sideload id
Additionally we now delete the code that allows unsigned summaries
when there is a collection id (because there is none).
create-usb now uses the sideload id as as collection id source when exporting.
The direct repo operations (export, bundle, commit-from) still support
collection ids, because on the server we do want to set it so that we
can sideload.
This adds a xa.sideload-collection-id option to the remote
configuration and a global xa.sideload-repos option (which is a list
of paths to local repos).
When resolving or listing refs, if we fail to download the real remote
summary (i.e. we're offline) then we instead look into the configured
sideloaded repos for refs that match ref and the sideloaded collection
id for the remote.
For the transaction to resolve the ref we need more metadata. In the
regular summary case we use the metadata from the summary, but that
is not available in the (partial) summary in the sideload repo, so
there we load the actual commit object and use the data from there.
(The ostree-metadata branch is not used/needed.)
This actually also fixes a longstanding issue when you "flatpak update
--checksum=XYZ" because we now handle this correctly by downloading
the commit object from the remote. Before we used the metadata in the
summary which is not right for non-HEAD commits.
To handle the sideloading we record the path to the sideload repo
when sideloading and pass the url to the repo as the remote name
when pulling, which will do a direct local pull.
We avoid using sideloaded refs when offline if the timestamp in the
commits is older than what is already installed locally.
In test_transaction_install_local(), we test that the origin remote
created when installing from a local repo doesn't exist before
flatpak_transaction_run() is executed and does exist afterward. However,
the origin remote is created before the transaction is run; see the
flatpak_dir_create_origin_remote() call in
flatpak_transaction_add_ref(). The only reason this discrepancy has not
caused a test failure is that the FlatpakDir object held by the
FlatpakInstallation object is not reloaded when the origin remote is
added (so it's reading an old copy of the repo config). This issue will
be fixed in the commit following this one.
Mark installed apps to be updatable if an installed in missing
its runtime for some reasons. In pre-FlatpakTransaction world,
an app migration from runtime X to runtime Y would have rendered
an app unusable because runtime Y would not be installed
automatically by clients like gnome-software.
The goal here is that clients like gnome-software can show
those installed apps as updatable again, if such a situation
arises. The FlatpakTransaction API can automatically resolve one of
its ops to install the new runtime, provided we can mark the app
as updatable again at the first place.
While updating, if the related extension is missing on
the installation of an installed ref (could be an app or
runtime), FlatpakTransaction tends to "repair" the ref by
automatically downloading the related extension again and
restoring the overall functionality of the ref.
The related extension concerned that are the ones associated with
`should-download` to TRUE only.
Hence, teach the libflatpak API to do that same, so that clients
like gnome-software can mark those refs as updatable, if their
related extensions is missing.
FlatpakTransaction will automatically download the related ref
(.Locale ref in this case). In case of mentioning previous-ids
in the deploy file, the related ref(.Locale) should be also
be requiring an update.
Hence, flatpak_installation_list_installed_refs_for_update now
will return 2 refs (app and related .Locale extension) to be
updated, instead of 1.
Also update the tests to use the same format, and make sure to include
coverage of all forms of locale (language-only, with locale, with
codeset, and with modifier).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to configure gnome-software to show specific apps in one region
without showing to all language speakers, we allow the storage of full
locales on the extra-languages key. However, these locales are ignored when
calling flatpak_installation_get_default_languages, so locales will be reduced
to their language identifier (eg. en_IN locale will be returned as 'en', and
az_Latn_AZ will be returned as 'az'). In order to get the full locales, we can
call flatpak_installation_get_default_locales instead, which can return languages
and locales.
This ensures that when the operation to install the app completes, the
app is ready to run — rather than having to wait for subsequent
operations to install extensions which the app depends on (such as
content, or locales).
This fits in with the current ordering for *un*installing apps, where
the main app is uninstalled first, followed by its extensions.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/flatpak/flatpak/issues/3017
FlatpakTransaction inherently doesn't differentiate between a
autoupdate and a regular update; both are
FLATPAK_TRANSACTION_OPERATION_UPDATE.
This getter can help differentiate between a regular update
and autoupdate in clients like gnome-software. Autoupdates
work in two separate transaction passes; first that runs with
"no-deploy" and the second pass that deploys all the downloaded
update (i.e. with "no-pull").
Assert it was successful instead. This is probably pointless from the
point of view of testing things (why would symlink() ever realistically
fail but everything else succeed?), but it does shut up a compiler
warning, which gives us a better chance to spot legitimate warnings in
future.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #3116
Approved by: alexlarsson
Currently FlatpakInstalledRef objects are constructed without the
collection-id property set. This is a problem because in the USB app
update support in the Endless fork of GNOME Software, when we find a
FlatpakRemoteRef on a USB drive which matches the ref of a
FlatpakInstalledRef object, the collection IDs of the two objects must
also match, and currently the installed one has a NULL collection ID.
So get the collection ID on the relevant configured remote when
constructing a FlatpakInstalledRef. This should be good enough in most
cases but isn't perfect; see
https://github.com/flatpak/flatpak/issues/3103Closes: #3114
Approved by: alexlarsson
In commit b8a3075d8 I changed a few places to check if the array
returned by ostree_repo_find_remotes_finish() is empty, which would mean
either none of the configured remotes (or P2P sources) provide the
requested refs, or the one(s) that do are offline. That was mostly
correct but in the case of
flatpak_installation_list_installed_refs_for_update() we don't want to
treat empty results as an error, because otherwise GNOME Software prints
an error message when offline: "No remotes found which provide these
refs: ...".
So this commit makes list_installed_refs_for_update() return an empty
array in case it can't find any updates (we don't distinguish "remotes
checked and provide no updates" from "remotes couldn't be reached").
This matches the behavior of the non-P2P code: the for loop above which
handles remotes without collection IDs prints a debug message if an
error is encountered.
Also, add a unit test that fails without this patch.
Closes: #3066
Approved by: alexlarsson
If xa.languages is set, use these, and no others. Otherwise, take the union
of xa.extra-languages, and the system default locales for system repos;
xa.extra-languages for user repo and the langs based on the user's locale
Fixes https://github.com/flatpak/flatpak/issues/3043
Currently the "test_remote()" test calls
flatpak_remote_set_gpg_verify (remote, FALSE) and disables GPG
verification on a remote while a collection ID is set on it, which
should not be possible. The remote-add command enforces that GPG
verification is used if a collection ID is set, but the library API does
not. This commit changes libflatpak to return an error when such an
invalidly configured remote is being committed to disk. Also, update the
unit test to check for the newly added error, and to unset the
collection ID before disabling GPG verification.
Later in the unit test, GPG verification is re-enabled on the remote,
but libflatpak erroneously sets gpg-verify-summary=true in addition to
gpg-verify=true (summary verification is supposed to be disabled when
collections are used, but the library doesn't notice the mistake since a
collection ID isn't set in the same transaction and was already set).
This fix addresses both issues.
Closes: #3095
Approved by: alexlarsson
This is basically modify_remote, but it fails if the remote is
already configured instead of modifying it. Although it also
has a if_needed option, and if that is set it will silently complete
as a no-op (exept resetting filters).
Closes: #2888
Approved by: alexlarsson
This copy-pastes a lot of internal code from flatpak-dir.c to mess with
the deploy file. It could be made a lot shorter...
https://phabricator.endlessm.com/T23845Closes: #2775
Approved by: alexlarsson
This commit makes it so that a unit test can create the test app and
runtime using a branch other than master, and changes test-run.sh to use
the branch "stable". This will allow the run command to be tested better
in the following commit.
Closes: #2788
Approved by: matthiasclasen
Currently a FlatpakTransaction object only operates on one installation,
but uses others (by default the system installations) for dependencies
such as runtimes. In addition to checking those dependency installations
for runtimes we check their configured remotes when deciding whether to
add a new remote as an origin for a flatpakref or as a runtime remote
for a flatpakref. This commit changes the behavior so that we only check
the installation being operated on to find out if a remote already
exists. This is the correct behavior in both cases: the origin remote
and the runtime remote. Otherwise the installation can error out when it
fails to find the runtime, or it can fail to respect the
SuggestRemoteName key which is supposed to dictate the name for the
origin remote.
One side effect of this is that a remote might be duplicated in the user
installation which already exists in the system installation, even if
the runtime it provides is already installed. But if you don't want
remotes in multiple installations you can just stick to using one
installation consistently.
Also, add a unit test for this in testlibrary.c (which required a bit of
refactoring).
Fixes https://github.com/flatpak/flatpak/issues/2758Closes: #2761
Approved by: alexlarsson
Uncrustify has an option "nl_func_var_def_blk" which is supposed to
ensure there's a newline character between the block of variable
definitions and the rest of the function body, but it gets confused and
thinks that the first instance of "g_autoptr" or "g_auto" being used on
a variable is the start of the function body. So this commit removes
those extra newline characters and removes that option in uncrustify.cfg
so they don't get re-added the next time uncrustify is run.
Here's the command I used:
perl -0777 -i -pe 's/\n(\n\s*g_auto\()/\1/g' `git ls-tree --name-only
-r HEAD | grep \\\.[ch]$ | grep -v common/valgrind-private.h |
grep -v app/flatpak-polkit-agent-text-listener\\\.[ch]`
I ran it again with "g_autoptr" in place of "g_auto", and made a few
manual edits to add back the newline when the g_auto* was in the middle
of a function body rather than at the top.
Closes: #2715
Approved by: matthiasclasen
Use only the last section in the id for origin remotes, to avoid
the origin column being really wide when listing stuff.
Closes: #2448
Approved by: alexlarsson
This avoids a crash or deadlock if the timeout is reached during test
teardown, after the GMainLoop has already been freed. That appears to
be more likely to happen when run as an installed-test.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Resolves: https://github.com/flatpak/flatpak/issues/2422Closes: #2425
Approved by: matthiasclasen
This means we pick up the built services. Also we set
FLATPAK_SYSTEM_HELPER_ON_SESSION so that the system-helper
runs on the session-bus just like in libtest.sh.
Closes: #2397
Approved by: matthiasclasen
We were relying on the order of string lists obtained
from a keyfile. But the way the keyfile is constructed
involved iterating of hash tables, which doesn't guarantee
order.
This was causing test failures with GLib master.
Closes: #2375
Approved by: matthiasclasen
This makes it clearer that we are not assuming that the test is running
on an x86_64.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #2353
Approved by: matthiasclasen
If you have a pre-existing remote configured its exact definition
might differ from the one specified in a flatpakrepo file and yet
be the same.
For example, i have:
$ flatpak --user remotes -d
Name Title URL Collection ID Priority Options
flathub Flathub https://dl.flathub.org/repo/ org.flathub.Stable 1
Yet when i install a flatpakref:
$ flatpak --user install http://flathub.org/repo/appstream/org.gnome.gedit.flatpakref
The application org.gnome.gedit depends on runtimes from:
https://dl.flathub.org/repo/
Configure this as new remote 'flathub-1' [y/n]:
Because the flathub flatpakrepo does not yet have the collection id specified.
So, we need to be more lenient when matching the pre-configured remotes.
Closes: #2324
Approved by: alexlarsson
We now avoid sleeps in the test, instead use the proper
ways to deal with races: poll for the child pid after
launching the app, and use a child watch after killing
it.
This is using the new flatpak_installation_launch_full API.
Closes: #2221
Approved by: matthiasclasen
This adds the logic from test_install_launch_uninstall() to
test_instance() and test_overrides() so that all three are skipped in CI
environments where bwrap may not be working. This is necessary because
they all use flatpak_installation_launch().
The tests added here include checks for expected types in
the library (including enum types that are needed for
introspection), and for error handling around names.
Closes: #2245
Approved by: alexlarsson
