This means we do a bit less string operations, but it also allows us
to pick up the collection-id when listing refs from a file: uri, fixing
the issues described in https://github.com/flatpak/flatpak/pull/3909
This slightly changes the behaviour of collection ids and remotes.
When listing regular remotes we only ever return the refs with the
collection id specified by the configuration of that remote. However,
when specifying a file: uri for the remote name we return all the refs
in the repo, with their corresponding collection id.
This means the test suite has to change a bit as before we returned
all the collrefs from the remote, not just the one matching the configured
collection id.
We normally don't remove a runtime when the last app using it is
uninstalled, since runtimes are large and re-downloading it in the
future may be difficult. But if the runtime is end-of-life, there's a
reasonable chance it won't be used again, so uninstall it in that case.
Similarly, if the last app using a runtime is upgraded to a different
runtime, and the runtime is EOL, uninstall it.
A unit test is included, and the subsequent unit test also had to be
modified. Otherwise we get a "Update is older than current version"
error, since the runtime is installed from test-repo but
setup_repo_no_add() calls make_runtime() which uses the one in
runtime-repo instead, which is older than the one in test-repo.
Avoid shadowing variables that are already declared in a previous scope,
and make such occurrences compile-time errors. These are not functional
changes.
In a few places do related code cleanup.
A similar ostree PR is here:
https://github.com/ostreedev/ostree/pull/2195
If a runtime is installed explicitly rather than as a dependency, pin it
so it doesn't get automatically removed when unused runtimes are being
removed. We do this because the runtime might be installed for
development or other uses.
This commit also rearranges some code in the mask and pin commands, and
adds a unit test.
Currently when searching for a remote to provide the runtime for an app,
we search remotes in priority order. This commit makes it so we search
the remote providing the app before others with the same priority, and
otherwise still search in priority order. This means for the common
case where every remote has the default priority of 1, the app's origin
will have the first chance to provide the runtime. This behavior seems
logical, but the impetus for this change was also to keep a unit test
passing in eos-updater[1] after a port to FlatpakTransaction.
Originally the eos-updater unit test was written to prioritize the
origin remote regardless of the priorities on any other remote, but
during code review it was decided to let higher priority remotes stay
above the app's origin.
In practice it's usually true that only one remote provides a runtime
and priorities aren't set at all, so this is an edge case that probably
doesn't come up much.
A unit test and documentation updates are included.
[1] eede0a8b9c/tests/test-update-install-flatpaks.c (L1919)
Don't leave options set on the remote in one of the unit tests. If
something should be set for every test it should be done in
global_setup(). This commit also changes the FlatpakRemote
implementation to allow unsetting title and default branch.
Re-implement flatpak_installation_list_installed_refs_for_update() using
a FlatpakTransaction, so we can guarantee it always gives the same set
of things to update as the update command. This API is used by GNOME
Software and many times in the past g-s has not shown the same list of
apps to be updated as the flatpak CLI. See:
- https://gitlab.gnome.org/GNOME/gnome-software/issues/539
- https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/430
This commit also expands the unit tests for this API, which were already
quite good. Now we test that missing subpaths of locale extensions show
up as updates, and updates that have been pulled but not deployed show
up as well. The latter is a break from how this function used to behave,
but it seems unlikely to break any application.
Most code that looks for a regular collection id set on the remote is
removed, as these should never happen in flatpak repo setups now.
Some is replaces with looking at xa.sideload-collection-id:
* The libflatpak FlatpakRef::collection-id property now comes comes from the sideload id
* Various CLI commands showing or changing the collection-id for a remote now uses the sideload id
* Collection id deploy in update now sets the sideload-collection-id instead
* Setting the collection id for a remote in libflatpak now sets the sideload id
Additionally we now delete the code that allows unsigned summaries
when there is a collection id (because there is none).
create-usb now uses the sideload id as as collection id source when exporting.
The direct repo operations (export, bundle, commit-from) still support
collection ids, because on the server we do want to set it so that we
can sideload.
This adds a xa.sideload-collection-id option to the remote
configuration and a global xa.sideload-repos option (which is a list
of paths to local repos).
When resolving or listing refs, if we fail to download the real remote
summary (i.e. we're offline) then we instead look into the configured
sideloaded repos for refs that match ref and the sideloaded collection
id for the remote.
For the transaction to resolve the ref we need more metadata. In the
regular summary case we use the metadata from the summary, but that
is not available in the (partial) summary in the sideload repo, so
there we load the actual commit object and use the data from there.
(The ostree-metadata branch is not used/needed.)
This actually also fixes a longstanding issue when you "flatpak update
--checksum=XYZ" because we now handle this correctly by downloading
the commit object from the remote. Before we used the metadata in the
summary which is not right for non-HEAD commits.
To handle the sideloading we record the path to the sideload repo
when sideloading and pass the url to the repo as the remote name
when pulling, which will do a direct local pull.
We avoid using sideloaded refs when offline if the timestamp in the
commits is older than what is already installed locally.
In test_transaction_install_local(), we test that the origin remote
created when installing from a local repo doesn't exist before
flatpak_transaction_run() is executed and does exist afterward. However,
the origin remote is created before the transaction is run; see the
flatpak_dir_create_origin_remote() call in
flatpak_transaction_add_ref(). The only reason this discrepancy has not
caused a test failure is that the FlatpakDir object held by the
FlatpakInstallation object is not reloaded when the origin remote is
added (so it's reading an old copy of the repo config). This issue will
be fixed in the commit following this one.
Mark installed apps to be updatable if an installed in missing
its runtime for some reasons. In pre-FlatpakTransaction world,
an app migration from runtime X to runtime Y would have rendered
an app unusable because runtime Y would not be installed
automatically by clients like gnome-software.
The goal here is that clients like gnome-software can show
those installed apps as updatable again, if such a situation
arises. The FlatpakTransaction API can automatically resolve one of
its ops to install the new runtime, provided we can mark the app
as updatable again at the first place.
While updating, if the related extension is missing on
the installation of an installed ref (could be an app or
runtime), FlatpakTransaction tends to "repair" the ref by
automatically downloading the related extension again and
restoring the overall functionality of the ref.
The related extension concerned that are the ones associated with
`should-download` to TRUE only.
Hence, teach the libflatpak API to do that same, so that clients
like gnome-software can mark those refs as updatable, if their
related extensions is missing.
FlatpakTransaction will automatically download the related ref
(.Locale ref in this case). In case of mentioning previous-ids
in the deploy file, the related ref(.Locale) should be also
be requiring an update.
Hence, flatpak_installation_list_installed_refs_for_update now
will return 2 refs (app and related .Locale extension) to be
updated, instead of 1.
Also update the tests to use the same format, and make sure to include
coverage of all forms of locale (language-only, with locale, with
codeset, and with modifier).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to configure gnome-software to show specific apps in one region
without showing to all language speakers, we allow the storage of full
locales on the extra-languages key. However, these locales are ignored when
calling flatpak_installation_get_default_languages, so locales will be reduced
to their language identifier (eg. en_IN locale will be returned as 'en', and
az_Latn_AZ will be returned as 'az'). In order to get the full locales, we can
call flatpak_installation_get_default_locales instead, which can return languages
and locales.
This ensures that when the operation to install the app completes, the
app is ready to run — rather than having to wait for subsequent
operations to install extensions which the app depends on (such as
content, or locales).
This fits in with the current ordering for *un*installing apps, where
the main app is uninstalled first, followed by its extensions.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/flatpak/flatpak/issues/3017
FlatpakTransaction inherently doesn't differentiate between a
autoupdate and a regular update; both are
FLATPAK_TRANSACTION_OPERATION_UPDATE.
This getter can help differentiate between a regular update
and autoupdate in clients like gnome-software. Autoupdates
work in two separate transaction passes; first that runs with
"no-deploy" and the second pass that deploys all the downloaded
update (i.e. with "no-pull").
Assert it was successful instead. This is probably pointless from the
point of view of testing things (why would symlink() ever realistically
fail but everything else succeed?), but it does shut up a compiler
warning, which gives us a better chance to spot legitimate warnings in
future.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #3116
Approved by: alexlarsson
Currently FlatpakInstalledRef objects are constructed without the
collection-id property set. This is a problem because in the USB app
update support in the Endless fork of GNOME Software, when we find a
FlatpakRemoteRef on a USB drive which matches the ref of a
FlatpakInstalledRef object, the collection IDs of the two objects must
also match, and currently the installed one has a NULL collection ID.
So get the collection ID on the relevant configured remote when
constructing a FlatpakInstalledRef. This should be good enough in most
cases but isn't perfect; see
https://github.com/flatpak/flatpak/issues/3103Closes: #3114
Approved by: alexlarsson
In commit b8a3075d8 I changed a few places to check if the array
returned by ostree_repo_find_remotes_finish() is empty, which would mean
either none of the configured remotes (or P2P sources) provide the
requested refs, or the one(s) that do are offline. That was mostly
correct but in the case of
flatpak_installation_list_installed_refs_for_update() we don't want to
treat empty results as an error, because otherwise GNOME Software prints
an error message when offline: "No remotes found which provide these
refs: ...".
So this commit makes list_installed_refs_for_update() return an empty
array in case it can't find any updates (we don't distinguish "remotes
checked and provide no updates" from "remotes couldn't be reached").
This matches the behavior of the non-P2P code: the for loop above which
handles remotes without collection IDs prints a debug message if an
error is encountered.
Also, add a unit test that fails without this patch.
Closes: #3066
Approved by: alexlarsson
If xa.languages is set, use these, and no others. Otherwise, take the union
of xa.extra-languages, and the system default locales for system repos;
xa.extra-languages for user repo and the langs based on the user's locale
Fixes https://github.com/flatpak/flatpak/issues/3043
Currently the "test_remote()" test calls
flatpak_remote_set_gpg_verify (remote, FALSE) and disables GPG
verification on a remote while a collection ID is set on it, which
should not be possible. The remote-add command enforces that GPG
verification is used if a collection ID is set, but the library API does
not. This commit changes libflatpak to return an error when such an
invalidly configured remote is being committed to disk. Also, update the
unit test to check for the newly added error, and to unset the
collection ID before disabling GPG verification.
Later in the unit test, GPG verification is re-enabled on the remote,
but libflatpak erroneously sets gpg-verify-summary=true in addition to
gpg-verify=true (summary verification is supposed to be disabled when
collections are used, but the library doesn't notice the mistake since a
collection ID isn't set in the same transaction and was already set).
This fix addresses both issues.
Closes: #3095
Approved by: alexlarsson
This is basically modify_remote, but it fails if the remote is
already configured instead of modifying it. Although it also
has a if_needed option, and if that is set it will silently complete
as a no-op (exept resetting filters).
Closes: #2888
Approved by: alexlarsson
This copy-pastes a lot of internal code from flatpak-dir.c to mess with
the deploy file. It could be made a lot shorter...
https://phabricator.endlessm.com/T23845Closes: #2775
Approved by: alexlarsson
This commit makes it so that a unit test can create the test app and
runtime using a branch other than master, and changes test-run.sh to use
the branch "stable". This will allow the run command to be tested better
in the following commit.
Closes: #2788
Approved by: matthiasclasen
Currently a FlatpakTransaction object only operates on one installation,
but uses others (by default the system installations) for dependencies
such as runtimes. In addition to checking those dependency installations
for runtimes we check their configured remotes when deciding whether to
add a new remote as an origin for a flatpakref or as a runtime remote
for a flatpakref. This commit changes the behavior so that we only check
the installation being operated on to find out if a remote already
exists. This is the correct behavior in both cases: the origin remote
and the runtime remote. Otherwise the installation can error out when it
fails to find the runtime, or it can fail to respect the
SuggestRemoteName key which is supposed to dictate the name for the
origin remote.
One side effect of this is that a remote might be duplicated in the user
installation which already exists in the system installation, even if
the runtime it provides is already installed. But if you don't want
remotes in multiple installations you can just stick to using one
installation consistently.
Also, add a unit test for this in testlibrary.c (which required a bit of
refactoring).
Fixes https://github.com/flatpak/flatpak/issues/2758Closes: #2761
Approved by: alexlarsson
Uncrustify has an option "nl_func_var_def_blk" which is supposed to
ensure there's a newline character between the block of variable
definitions and the rest of the function body, but it gets confused and
thinks that the first instance of "g_autoptr" or "g_auto" being used on
a variable is the start of the function body. So this commit removes
those extra newline characters and removes that option in uncrustify.cfg
so they don't get re-added the next time uncrustify is run.
Here's the command I used:
perl -0777 -i -pe 's/\n(\n\s*g_auto\()/\1/g' `git ls-tree --name-only
-r HEAD | grep \\\.[ch]$ | grep -v common/valgrind-private.h |
grep -v app/flatpak-polkit-agent-text-listener\\\.[ch]`
I ran it again with "g_autoptr" in place of "g_auto", and made a few
manual edits to add back the newline when the g_auto* was in the middle
of a function body rather than at the top.
Closes: #2715
Approved by: matthiasclasen
Use only the last section in the id for origin remotes, to avoid
the origin column being really wide when listing stuff.
Closes: #2448
Approved by: alexlarsson
