This commit fixes the handling of errors from installing/updating
related refs during a transaction, so that they're treated as non-fatal,
and so that the operation is skipped if the primary operation fails. The
current behavior is that a failure to install/update a related ref
causes the whole transaction to fail, and even after a failure to
install/update the primary ref the related ref install/update is
attempted.
I hit this error when doing an offline USB app install, when the USB
repo has an older version of the runtime and the runtime's locale
extension than what's in the local repo. Without this commit, the
failure to update the runtime (due to it being a downgrade) is treated
as a warning, but the failure to update the runtime locale is treated as
an error. With this commit, the runtime update failure is still treated
as a warning, and the locale update is not attempted. This is better
behavior because the locale extension update (or even install) is not
critical to the app install.
Closes: #1979
Approved by: alexlarsson
In https://github.com/flatpak/flatpak/pull/1689 we were meant to
have limited the receiving of broadcasts on portals, but die to a
bug in the proxy we accidentally allowed all broadcasts anyway.
The change which ignores all applied filters < POLICY_TALK fixes that.
However, it also turns out that the desktop portal actually *does*
rely on signals. For example the network portal uses property change
notification.
So, to make sure this works we allow all signal from the portal
names, but only if they are on a object path starting under
/org/freedesktop/portal (which incidentally all portal object are).
This means there is no real change in anything that is currently
deployed, but it does allow portals to opt out of this global signal
visiblity if they want by using a different object path, which we
want to use in dconf.
Closes: #1976
Approved by: alexlarsson
This commit fixes a regression that causes installing from a bundle to
fail if the bundled app's runtime was itself installed from a bundle, or
otherwise has a non-working remote (such as when the user is offline).
The fix is to treat a failure of flatpak_dir_find_latest_rev() as
non-fatal in resolve_ops() if the ref in question is already installed.
In other words, if we don't need to fetch a ref for the transaction to
succeed, errors in fetching remote info about the ref shouldn't be
fatal.
Closes: #1973
Approved by: alexlarsson
When I run `flatpak update` I get messages saying "Warning: No
xa.metadata in commit" which isn't very helpful without knowing what
commit is being referred to. So this commit adds the checksum and ref to
such error messages.
Closes: #1978
Approved by: alexlarsson
It's a good idea to NULL initialize g_autoptr/g_autofree variables, so
we can be sure uninitialized memory isn't passed to g_free or similar.
Closes: #1968
Approved by: alexlarsson
Sometimes (for example in some test-repo-collections.sh test that broke) we
update from a remote with an older ostree-metadata branch, and the
check for downgrades broke in this case.
Its unclear exactly what it the best solution here, maybe to silently
disallow the update. However, this change instead just re-allows the
downgrade for this particular case so we get the old behaviour.
When we switch the remote type, we need to clean up cached files
(appstream, OCI index/summary) because they are stored differently
for the two types of remote.
The old pattern of using a separate 'OCI' flag was very ugly
internally in the code once it was extended to flatpak bundles and
flatpakrefs - using a different URI scheme means that the nature
of the remote can't be accidentally lost in some part of the code.
Probing would be possible as well, but would make it difficult to
add a remote when offline, and also doesn't deal well with the
fact that our data layout is different for the two types of remotes -
the type of remote could change at any point!
As a side effect this change enables flatpakrefs and flatpak bundles for OCI
registries.
* Restrict the queried images to the desired architecture
* Sort query parameters as the spec requests
* Allow a fragment on the remote URI to mean "tag to query for
in the registry"
* Tweak flatpak_oci_index_ensure_cached() not to return the
index URL in the normal error case.
The normal behavior where we only list already installed refs for
a noenumerate remote doesn't work for the case where flatpak-system-helper
verifies a ref on an OCI server during installation - in that case, the
ref being installed to does not *yet* exist locally.
In general Flatpak tries to prevent downgrades of anything: apps,
runtimes, repo metadata, etc. with some exceptions such as when the user
specifies a commit they want. However at the moment the detection of a
downgrade is broken if both of the following are true: (1) a collection
ID is enabled on the relevant remote, and (2) a per-user installation
is being used instead of the system-wide one (or the system-helper is
otherwise being circumvented, such as by running flatpak as root).
This bug is a security vulnerability, but it's one with limited impact
because very few people have collection IDs enabled yet, and the
downgrade attack would require either a MITM on the network connection
(which HTTPS should prevent) or a malicious USB drive or local network
peer.
We previously made a separate request to the registry index to see if
the manifest hash of an image was the hash of the image in the registry.
Since the summary is now downloaded by the system helper and trusted, just
check if the hash matches the hash in the summary data. This is as good,
and in is a lot more efficient if the index is statically generated,
and we can't get the index data for just one image.
Closes: #1910
Approved by: alexlarsson
The OCI index information should be highly compressable (especially if
icons are remote URI's rather than data URI's) so downloading it and
storing it compressed will provide sigificant efficiency gains.
Closes: #1910
Approved by: alexlarsson
Add a new flag for flatpak_cache_http_uri() that adds Accept-Encoding: gzip
to the request, and if the result is returned compressed, stores the data
compressed. If the data result is return uncompressed, it's compressed.
Closes: #1910
Approved by: alexlarsson
Checking the registry against a previous etag is now handled inside
flatpak_cache_http_uri(), so remove the etag parameters that were
previously passed around in various places for simplicity.
Closes: #1910
Approved by: alexlarsson
Previously the code assumed that appstream data was stored in a separate
OCI image in the registry. Replace that with storing the appstream data
and icons as image annotations. When we download a new version of the
image index, the appstream data is combined, and icons are downloaded
as necessary.
Since there is no longer a content hash for the appstream data, it's
not practical for the user to download the appstream data and pass it
to the system helper, instead the system helper just downloads the
appstream data directly.
Closes: #1910
Approved by: alexlarsson
Redo the handling of generating summary information from an OCI registry
to be a two step process. First download the index, using the newly
added HTTP caching functionality. Then regenerate the summary from the
index, using mtimes to avoid duplicate work.
Closes: #1910
Approved by: alexlarsson
Add a new function, flatpak_cache_http_uri() that when passed an URL and
a local destination location, either a) downloads the content and stores
it at the destination location, storing HTTP cache header information
like Last-Modified, Etag into user xattrs (if available) or a separate
file or b) if the downloaded content is already present, checks the
header information to decide whether the downloaded content can be used
or needs to be revalidated witha conditional request.
Tests are added that use a special case test server that adds HTTP caching
headers and reacts to them based on query parameters. A small test binary
'httpcache' is added for the tests to use.
Closes: #1910
Approved by: alexlarsson
In preparation for extending the HTTP downloading function to include
caching, split HTTP related utilities into a separate file with a
separate header.
Closes: #1910
Approved by: alexlarsson
The code checked whether an OCI registry URI was an OCI image layout accessed
via HTTP by looking for /oci-layout, but distributing OCI images in this way
is not really a thing anybody does. It would be inefficient way to store
large numbers of images, since all versions need to be listed in index.json.
The code still uses OCI image layouts to represent "local registries" in
analogy to local ostree repositories.
Closes: #1910
Approved by: alexlarsson
In a couple different places, double slashes were inserted into the
generated OCI index URIs - e..g, index//static instead of index/static.
While most HTTP servers/applications will normalize double slashes,
this is not required, and such URIs are, in any case, ugly.
Closes: #1910
Approved by: alexlarsson
When Flatpak is fetching a locale extension it has to decide which
subpaths to fetch based on what language is being used on the computer.
This happens in flatpak_dir_get_locale_subpaths() which indirectly uses
the org.freedesktop.Accounts D-Bus object to check what language is
configured for each user. The problem is that if any user doesn't have a
language set, Flatpak falls back to pulling all languages, rather than
checking the system default using localed. The effect is that on Endless
OS systems, Flatpak is pulling entire locale extensions rather than just
the subset for the configured language, which is a significant waste of
bandwidth. In my testing, the "Language" property on the primary user
account is not set on Endless, but it is set on Fedora.
A side effect of this bug is to cause offline USB app installs to
sometimes fail, because if the USB only has a partial locale and you try
to pull the whole thing, the pull fails.
This commit fixes the issue by doing another D-Bus call to localed to get
the system default(s), then checking AccountsService as before, treating
an unset language for a user account as meaning "use the system
default". Then only if no languages are set for the users or the system,
fall back to pulling all languages. The code to communicate with localed
is based on the code in gnome-control-center in
panels/region/cc-region-panel.c
This extra synchronous D-Bus call adds some overhead which might be able
to be avoided; see https://github.com/flatpak/flatpak/issues/1938
Using this patch I can see that Flatpak is pulling partial locales now,
based on the output of `flatpak list -a | grep partial` after installing
Bijiben from Flathub.
Closes: #1937
Approved by: alexlarsson
This system-helper method will allow flatpak to update the ostree repo
summary while running as a non-root user that's in the "sudo" group.
This will allow the `flatpak create-usb` command to work without
requiring the user to first run `sudo ostree summary -u`, and without
requiring the system to have `core.auto-update-summary` set to true in
the ostree repo config. This is also much more efficient than
`core.auto-update-summary` because it allows us to only update the
summary when we need it rather than after every transaction.
Generating the summary basically just involves traversing the repo to
enumerate all the refs and putting this information into a file, so I
don't think there are security concerns with allowing it to happen
without authentication.
Closes: #1945
Approved by: alexlarsson
Use min-free-space-size as a replacement for min-free-space-percent.
Previously, flatpak used to disable min-free-space-percent by re-writing
the config with min-free-space-percent=0. As the new version of ostree (2018.7)
now supports min-free-space-size, we should use that and migrate from -percent
option in existing repos to -size=500MB.
Config is rewritten with min-free-space-size in case of:
1) It has min-free-space-percent=0 only. (That is probably from the
previous re-writes).
2) If there are no min-free-space-* options.
Other than that, the config remains unchanged and the co-existence(if any)
of these options is governed by ostree.
https://github.com/flatpak/flatpak/issues/1826Closes: #1912
Approved by: alexlarsson
When the system bus is not available, we set
system_helper_bus to (gpointer)1. And then we
segfault in finalize, trying to unref it.
Thats not nice, so avoid it.
Closes: #1940
Approved by: alexlarsson
When we fail to parse an installation, emit some
warnings that give the user a chance to figure out
what is going wrong. I ran into the missing quotes
issue myself when I first tried to create a custom
installation.
Closes: #1939
Approved by: alexlarsson
