To avoid the complexities of passing (and chaining) OstreeAsyncProgress
objects around, we only create one just before calling to ostree.
The rest of flatpak only ever uses the new FlatpakProgress object.
Co-authored by: Philip Chimento <philip@endlessm.com>
This used to not be set for collection-id remotes as we used the
ostree-metadata branch for resolving. However, we now use the summary
always when doing a remote install (and not ostree-metadata for local
sideloads), so we still want to verify summary.
The signature on the summary is a nice security feature, but it is also
a very efficient small file to download to verify that no new summary
needs to be downloaded in the no-op update case.
Most code that looks for a regular collection id set on the remote is
removed, as these should never happen in flatpak repo setups now.
Some is replaces with looking at xa.sideload-collection-id:
* The libflatpak FlatpakRef::collection-id property now comes comes from the sideload id
* Various CLI commands showing or changing the collection-id for a remote now uses the sideload id
* Collection id deploy in update now sets the sideload-collection-id instead
* Setting the collection id for a remote in libflatpak now sets the sideload id
Additionally we now delete the code that allows unsigned summaries
when there is a collection id (because there is none).
create-usb now uses the sideload id as as collection id source when exporting.
The direct repo operations (export, bundle, commit-from) still support
collection ids, because on the server we do want to set it so that we
can sideload.
This adds a xa.sideload-collection-id option to the remote
configuration and a global xa.sideload-repos option (which is a list
of paths to local repos).
When resolving or listing refs, if we fail to download the real remote
summary (i.e. we're offline) then we instead look into the configured
sideloaded repos for refs that match ref and the sideloaded collection
id for the remote.
For the transaction to resolve the ref we need more metadata. In the
regular summary case we use the metadata from the summary, but that
is not available in the (partial) summary in the sideload repo, so
there we load the actual commit object and use the data from there.
(The ostree-metadata branch is not used/needed.)
This actually also fixes a longstanding issue when you "flatpak update
--checksum=XYZ" because we now handle this correctly by downloading
the commit object from the remote. Before we used the metadata in the
summary which is not right for non-HEAD commits.
To handle the sideloading we record the path to the sideload repo
when sideloading and pass the url to the repo as the remote name
when pulling, which will do a direct local pull.
We avoid using sideloaded refs when offline if the timestamp in the
commits is older than what is already installed locally.
We're using the metadata from the summary, ostree-metadata or available
commit when making security sensitive decisions, so lets verify this
matches what we get in the actual commit we pulled.
We already did check that this then actually also matches what gets deployed,
so the new check shares code with that.
Note, we don't do this for OCI installs, because it seems the current
fedora flatpaks don't have this set, and we don't want to break
existing remotes.
This is a g_autoptr version of OstreeAsyncProgress that also
calls ostree_async_progress_finish() before being freed.
This should be used in all "leaf" functions that creates an asyncprogress
to avoid leaking any idle change idle sources. Using a auto* means
some code can be cleaned up to avoid goto out style handling for this.
Also, this adds a missing finish() in
_flatpak_dir_fetch_remote_state_metadata_branch().
Due to bug #3215 some systems have refs in refs/mirrors/ in addition to
the usual refs/remotes/ location. The remote refs are always at least as
new as the mirror ones since the repo_pull() invocation in
flatpak_dir_pull() which does not use OSTREE_PULL_FLAGS_MIRROR happened
after the one that did. Cleaning up these mirror refs is important since
otherwise when the remote ref is either updated or removed (by an
uninstall) disk space will be leaked since the mirror ref will point to
a no longer needed commit.
So, remove (almost) all mirror refs during flatpak repair, uninstall,
or update operations. And for the uninstall and update operations do it
in FlatpakDir so that it happens regardless of if the CLI of libflatpak
are used.
Also, add a unit test for this.
Fixes https://github.com/flatpak/flatpak/issues/3222
After an unprivileged client calls GetRevokefsFd(), the `revokefs-fuse
--backend` process busyloops as follows:
poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}], 2, -1) = 1 ([{fd=4, revents=POLLIN}])
Here is the command line for this process:
revokefs-fuse --backend --socket=3 --exit-with-fd=4 /var/lib/flatpak/repo/tmp/flatpak-cache-JBUHB0
The intention here is to poll() until fd 3 is readable (at which
point the writer process serves a client request and writes back a
response, synchronously) or fd 4 encounters an error. fd 4 is meant to
be one side of a pipe that the system helper holds the other end of;
when the pipe is broken, the system helper must have gone away, and the
`revokefs-fuse --backend` process treats this as a signal to exit.
However, fd 4 is not a pipe. In fact, it is the dirfd for the target directory:
root@camille:/var/roothome# ls -l /proc/31717/fd
total 0
lr-x------ 1 wjt wjt 64 Nov 19 21:21 0 -> /dev/null
lrwx------ 1 wjt wjt 64 Nov 19 21:21 1 -> /dev/pts/1
lrwx------ 1 wjt wjt 64 Nov 19 21:21 2 -> /dev/pts/1
lrwx------ 1 wjt wjt 64 Nov 19 21:21 3 -> 'socket:[2558007]'
lr-x------ 1 wjt wjt 64 Nov 19 21:21 4 -> /var/lib/flatpak/repo/tmp/flatpak-cache-JBUHB0
This is because revokefs_fuse_backend_child_setup() erroneously closes
fd 4 before the `revokefs-fuse --backend` process is exec()d. This
regressed in d91660fe2a.
Fix this by only closing fds 5 and above. With this change, we see the
expected set of open file descriptors:
root@camille:/var/roothome# ls -l /proc/32493/fd
total 0
lr-x------ 1 wjt wjt 64 Nov 19 21:24 0 -> /dev/null
lrwx------ 1 wjt wjt 64 Nov 19 21:24 1 -> /dev/pts/1
lrwx------ 1 wjt wjt 64 Nov 19 21:24 2 -> /dev/pts/1
lrwx------ 1 wjt wjt 64 Nov 19 21:24 3 -> 'socket:[2552594]'
lr-x------ 1 wjt wjt 64 Nov 19 21:24 4 -> 'pipe:[2552596]'
lr-x------ 1 wjt wjt 64 Nov 19 21:24 5 -> /var/lib/flatpak/repo/tmp/flatpak-cache-JBUHB0
Fixes#2882.
Use the user’s OARS filter to prevent installation or upgrade of
apps which have more extreme content than the user is allowed to see.
This uses libmalcontent to load the user’s enforced OARS filter, which
describes the extremeness of each type of content the user is allowed to
see. If an app they are trying to install exceeds the filter value in
any OARS section, installation is disallowed and an error is returned.
libmalcontent stores the parental controls policy per-user in
accountsservice, which enforces access control on the policies.
The app filter is also allowed to prevent app installation entirely,
which overrides the OARS values. This is independent from the app-install
polkit action, which determines whether an unprivileged user may install
an app system-wide. Being stored in accountsservice, the new boolean is
also easier to set per-user without having to programmatically write a
polkit JS policy file which handles multiple users (and parse it back
again).
The parental controls checks are done at deploy time, either in the
`flatpak` process (for user repositories) or in the
`flatpak-system-helper` (for system repositories). The checks use
content rating data extracted from the app’s AppData XML and stored in
the `FlatpakDeploy` cache. The checks are passed through polkit (even
for user repositories) so that users can get an admin override to
install apps which would otherwise be too extreme. This uses the new
`org.freedesktop.Flatpak.parental-controls` polkit rule.
The checks have to be done at deploy time, as that’s when the AppData
XML for the app is parsed. The downside of this arrangement is that an
app must be entirely downloaded before the parental checks can be done.
This won’t be much of an issue on normal desktops, however, since we can
assume that gnome-software will check an app’s appropriateness before
showing it to the user in the first place.
Parental controls are not enforced for non-apps/runtimes, which includes
the ostree-metadata and appstream/* refs.
One thorny issue is that flatpak unit tests may be run in an environment
with no system D-Bus available to connect to (a Jenkins instance, for
example), which means the call to `mct_manager_get_app_filter()` in
`flatpak_dir_check_parental_controls()` fails.
So this commit skips the parental controls check if the system bus is
unavailable and the environment variable
`FLATPAK_SYSTEM_HELPER_ON_SESSION` is set, since the testlibrary already
sets that variable so that the system-helper will be started on the
session bus.
The feature can be tested using something like:
```
$ malcontent-client set philip \
violence-realistic=none app/org.freedesktop.Bustle/x86_64/stable
App filter for user 1000 set
$ flatpak run org.freedesktop.Bustle
error: Running app/org.freedesktop.Bustle/x86_64/stable is not allowed by the policy set by your administrator
$ flatpak --user install flathub io.github.FreeDM
error: Failed to install io.github.FreeDM: Installing app/io.github.FreeDM/x86_64/stable is not allowed by the policy set by your administrator
```
Includes work by André Magalhães and Umang Jain.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
As per https://gitlab.gnome.org/GNOME/glib/merge_requests/490
there is a bug in glib < 2.60 where g_spawn_* can sometimes deadlock
due to using malloc in the child func to close fds.
We work around this in places where the code is (potentially) threaded
by passing glib flags to leave fds alone and then do a very naive
(but safe) fd cloexec loop ourselves.
Currently flatpak_installation_fetch_remote_ref_sync() does not work
offline. It returns an error when it fails to fetch the remote's summary
in flatpak_dir_get_remote_state(). This is a problem since GNOME
Software (or at least the Endless fork) uses this library function to
display apps it finds on a USB drive (see gs_plugin_refine_item_origin()
in gs-flatpak.c) and that's something that should work even offline.
So this commit changes flatpak_dir_get_remote_state_optional() so that
it accepts the only_cached option, and updates the call sites. Also have
fetch_remote_ref_sync() use flatpak_dir_get_remote_state_optional(),
which means that when we're offline we will use the xa.cache data in the
ostree-metadata ref as a list of refs list instead of using a summary.
However since the commit checksums are not in xa.cache, we don't have
enough information to form a FlatpakRemoteRef. So also call
ostree_repo_find_remotes_async() to get the commit from any LAN or USB
sources that may be available. This may not be very performant but at
least it only happens if the ref wasn't found in a remote summary; see
https://github.com/flatpak/flatpak/issues/1862
It's sad this code is so long-winded but it's difficult to break out a
helper function that could be shared with
list_remotes_for_configured_remote() above. Longer term we could improve
the ostree_repo_find_remotes_async() API and add options to remove the
need to manually handle OstreeRepoFinder objects.
Closes: #3114
Approved by: alexlarsson
If xa.languages is set, use these, and no others. Otherwise, take the union
of xa.extra-languages, and the system default locales for system repos;
xa.extra-languages for user repo and the langs based on the user's locale
Fixes https://github.com/flatpak/flatpak/issues/3043
We now pull the image config as well as the manifest and fall
back on the labels field if the keys we're looking for are not
in the annotations field.
This lets us support docker manifests too, which don't have
annotations (but do have labels).
Closes: #2978
Approved by: alexlarsson
The org.freedesktop.Flatpak user D-Bus service isn't just used by
flatpak(1) or applications running as Flatpaks. It's also used by
toolbox(1) for similar reasons:
* To keep various configuration files inside the container
synchronized with the host
* To let the container request certain commands to be run on the host
The org.freedesktop.Flatpak D-Bus service itself doesn't need much in
the way of dependencies, but inherits a lot of unused shared library
linkages through the libflatpak-common.la convenience library. Removing
these unused shared libraries reduces the footprint of toolbox(1) for
those who care about such things. eg., Fedora CoreOS.
This commit brings down the number of shared libraries to 19 from 62.
Closes: #3052
Approved by: alexlarsson
We enforce an umask of 022 (no world/group writable) in the cli and
the system helper. This is necessary, because we need to create
ostree repositories shared between the helper and the client, and
a more strict umask breaks this.
It would be nice if we could just set this in a thread-local way when
needed, but unfortunately umask() is not threadsafe or overridable in
any local way.
This unfortunately means this it will not automatically work for
libflatpak users...
Closes: #2856
Approved by: alexlarsson
Old versions of the create-usb command created a summary file in the
local repo being pulled from (e.g. /var/lib/flatpak/repo) but this
summary generation turned out not to be necessary and was removed. So
any computer which used the create-usb command before commit 7c5751a4f
will have a leftover /var/lib/flatpak/repo/summary file which becomes
outdated as apps are updated and installed. This causes problems for the
next invocation of (a recent version of) the create-usb command which
will use the outdated summary during the pull and fail with an error
message like:
error: Importing 3b1293596e9aa67f6fd0daeae477cb94603a4e8ca9e825f446d3dd04a2b5d5ec.commit:
fstatat(3b/1293596e9aa67f6fd0daeae477cb94603a4e8ca9e825f446d3dd04a2b5d5ec.commit): No such file or directory
So this commit makes the create-usb command delete the summary if it
exists before pulling onto the repo on the USB drive. This means USB
copies will work again for any users that used the USB app copy feature
in Endless OS 3.4.7.
Closes: #2854
Approved by: alexlarsson
As seen in https://github.com/flatpak/flatpak/issues/2829,
the system-helper crashes while trying to access an as
parameter as s. Looking at the likely culprits for this,
I find that the permission check for Deploy has an off-by-one
error where it tries to get the installation, but
passes the offset for the previous_ids parameter.
Closes: #2831
Approved by: matthiasclasen
end-of-lifed-with-rebase runs earlier in the transaction system than end-of-lifed, meaning it can modify the transaction for rebasing.
The new flatpak_transaction_add_rebase() function can then be used to apply the rebase if wanted.
Closes: #2775
Approved by: alexlarsson
In the normal case, pass the full LIBEXEC path, and in the tests,
pass the path via the env var FLATPAK_REVOKEFS_FUSE.
Closes: #2657
Approved by: alexlarsson
This is necessary so as to not leave the revokefs backend around
when the system-helper exits abruptly (e.g. OOM killer). It would
be a vulnerability if revokefs backend continues to live even after
the system-helper is killed as it might lead to write access to the
underlying directory.
Closes: #2657
Approved by: alexlarsson
If there is a pull failure in a child repo created on revokefs-fuse
mount, there is no way to go back to the system helper and notify
it to cleanup. Therefore, CancelPull is required on the pull failure
error path, so that the ongoing pull can be cleaned up nicely and
prevent any dangling mounts and subprocesses.
Closes: #2657
Approved by: alexlarsson
This adds a new helper method "GetRevokefsFd" which is responsible
for spawning the backend part of the revokefs filesystem. It takes care
of creating a cache location for the backing directory in repo/tmp.
This cache location is transferred over D-Bus to the client with the
other end socket fd.
The client on receiving the socket fd creates a mountpoint directory and
spawns the revokefs-fuse filesystem. It then creates a child repo for the
pull. In any case of failure, it fallbacks on the current code path
(which causes temporary duplication of files on disk).
The backing dir itself and all files written to it by the revokefs-fuse
backend process are owned by the "flatpak" user. After the pull in the
child repo is completed, it's ownership is then canoncalized with owner=root
and permissions as per bare-user-only in Deploy().
Now we have fulfilled all the criteria to hardlink the child repo into
the system one and avoid duplication. See [1].
If there is existing cache directory available in repo/tmp, it will be
mounted using revokefs-fuse for the current pull. Hence, it is possible
to recover the previous partial pull which might have failed due to some
error.
[1] https://github.com/ostreedev/ostree/pull/1776Closes: #2657
Approved by: alexlarsson
Uncrustify has an option "nl_func_var_def_blk" which is supposed to
ensure there's a newline character between the block of variable
definitions and the rest of the function body, but it gets confused and
thinks that the first instance of "g_autoptr" or "g_auto" being used on
a variable is the start of the function body. So this commit removes
those extra newline characters and removes that option in uncrustify.cfg
so they don't get re-added the next time uncrustify is run.
Here's the command I used:
perl -0777 -i -pe 's/\n(\n\s*g_auto\()/\1/g' `git ls-tree --name-only
-r HEAD | grep \\\.[ch]$ | grep -v common/valgrind-private.h |
grep -v app/flatpak-polkit-agent-text-listener\\\.[ch]`
I ran it again with "g_autoptr" in place of "g_auto", and made a few
manual edits to add back the newline when the g_auto* was in the middle
of a function body rather than at the top.
Closes: #2715
Approved by: matthiasclasen
Be more systematic about returning FLATPAK_ERROR unmodified
and wrap everything else in a G_DBUS_ERROR_FAILED.
Closes: #2391Closes: #2532
Approved by: alexlarsson
These services were not generating coverage data
because they always get ended by GDBus raising
SIGTERM when the bus goes away. Prevent this by
telling GDBus not to do that, and let the services
exit regularly, on their own terms.
This makes the system helper code show up in the
coverage statistics.
Closes: #2530
Approved by: matthiasclasen
This adds a version to the deply data format, assuming that if
there is no version then it is 0. Also extends all loaders so that
it can specify a required version, with the goal that most users
are fine with old versions, but if you need more recent we have
a (costly) conversion process to upgrade (will be added later).
The library and the flatpak list/info commands require the most
current version for full info, all other users can use any version.
Also, as part of this we now pass the ref to flatpak_load_deploy_data()
as this will be needed later for the backwards compat support.
Closes: #2409
Approved by: alexlarsson
The first element put in the variant created by
flatpak_dir_system_helper_call_deploy() is the repo path, but this is
being treated as the installation ID in
flatpak_authorize_method_handler(), which results in a seg fault when
dir_get_system() returns NULL and this NULL is passed to
dir_ref_is_installed(). Fix the seg fault by getting the correct element
from the variant.
Closes: #2411
Approved by: matthiasclasen
We shouldn't put the unsightly dbus error wrapping in the
UI. The cient already strips the wrapping from the system
helper call, but it can't strip the second level itself.
Closes: #2399
Approved by: mwleeds
