When we create a system child registry we also set the current token on
it. This is not used directly in the client, however its saved in a
file called .token and re-read in the system-helper, allowing it to
also do the remote registry operations it needs to verify the child
registry.
(cherry picked from commit 5d8fd2d1be)
In the implementation of flatpak_deploy_data_get_string() and a few
other getters, we borrow a value from a GVariant that is a child of the
main GVariant, and free the child via g_autoptr(). Since we then no
longer hold a reference to the child variant, if it is freed internally
in GLib, the borrowed value is no longer safe to access. And GLib frees
all children of a GVariant when serializing it; see the implementation
of g_variant_get_data().
Therefore, in the implementation of the list command the value returned
by flatpak_deploy_data_get_runtime() is not valid after
flatpak_deploy_data_get_origin() is called, and the end result is that
the values in the runtime column of the output are sometimes gibberish.
For the flatpak-1.6.x branch, fix it by ensuring that the GVariant for
deploy data is serialized directly after it is created. On the master
branch (Flatpak 1.7.x) all this GVariant code has been replaced with use
of variant-schema-compiler, and this issue is not present in that case
according to valgrind.
See also https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1462
This is a cherry-pick from https://github.com/endlessm/flatpak/pull/217
This is a bug introduced in b03916f5bd
where we check the extra_data refs against app/ or runtime/ prefix with
arguments in the wrong order.
(cherry picked from commit ed3ba39a06)
This breaks Deploy which can't find the ref. It used to work due to
the extra non-mirroring pull in flatpak_dir_setup_extra_data, but
this is not needed here.
As noticed in https://github.com/flatpak/flatpak/issues/3412 we
regressed at some point and are no longer using from-scratch deltas.
This is caused by an optimization in ostree where it decides to not
use a from-scratch deltas if theres is *some* version of the ref
locally available.
This conflicts with some code in flatpak that pulls *only* the commit
object in order to look for extra data size information so that we can
get the progress reporting right. Unfortunately the existance of
just the object triggers the above causing us to *never* use from-scratch
deltas.
We fix this by throwing away the partial pull in an aborted ostree
transaction.
In flatpak_dir_create_origin_remote() we reload the repo config after
adding an origin remote to it, but this only applies to the FlatpakDir
object used. In the case of flatpak_transaction_add_ref(), there is
another FlatpakDir object in the installation (priv->installation) which
needs to also be reloaded using flatpak_installation_drop_caches(). So
add a boolean out variable to flatpak_dir_create_origin_remote() and use
it to determine if it's necessary to call
flatpak_installation_drop_caches() (because if the origin remote already
exists we don't create another).
This commit also makes related changes at the other call sites of
create_origin_remote() (some indirectly via
flatpak_dir_ensure_bundle_remote()):
- in flatpak_dir_ensure_bundle_remote(), only set the out variable
created_remote to TRUE if a new remote was actually created
- in flatpak_installation_install_bundle(), only drop the installation
caches if a new remote was actually created
- in flatpak_transaction_resolve_bundles(), drop a redundant
flatpak_dir_recreate_repo() call and only drop installation caches
when necessary
Without these changes, this unit test failure occurs:
ERROR: testlibrary - Bail out!
flatpak:ERROR:tests/testlibrary.c:3311:test_transaction_install_local:
assertion failed (error == NULL): Remote "hello-origin" not found
(flatpak-error-quark, 7)
Initialize the related-refs array with empty GPtrArray so that if
the remote has 'url= ' (for e.g., in case of flatpak bundle's remotes),
a empty array is returned instead of NULL.
(NULL mostly implies a operation has failed and error is set)
Also, this syncs the implementation of `if (*url == 0)` with
that of flatak_dir_find_remote_related_for_metadata function.
If the local config for the remote specifies an authenticator name
and that is should in installed, automatically add it to updates
in a transaction.
The local config can either be manually configured, or automatically
from a flatpakrepo file or the summary metadata.
When we call flatpak_dir_update_remote_configuration we pass it
the pre-existing FlatpakRemoteState (if known) and also take into
account if it actually changed anything before blowing away the
cached remote state.
We also ensure we have metadata in
flatpak_dir_update_remote_configuration_for_state to ensure the passed
in optional state has metadata in it.
These should only be called at the leaf level, because the first
time its called no more change events will ever be sent on that
progress, which is not right possibly in the middle of an operation.
This is a g_autoptr version of OstreeAsyncProgress that also
calls ostree_async_progress_finish() before being freed.
This should be used in all "leaf" functions that creates an asyncprogress
to avoid leaking any idle change idle sources. Using a auto* means
some code can be cleaned up to avoid goto out style handling for this.
Also, this adds a missing finish() in
_flatpak_dir_fetch_remote_state_metadata_branch().
It turns out ostree_repo_open() overwrites custom cachedir_fd we've
set for the system using installation in case the object dir is
writable. Normally this is not a problem, because it is not writable,
but in the testsuite is *is*, which means the initial cachedir fd is
leaked, as well as using the wrong dir for summary caches during the
tests.
We work around this by setting the cache_dir after a successful
ostree_repo_open().
This fixes#3303
An event happens more often as its frequency gets higher, so these
values were confusing me.
Rename the constants to include their unit (ms) as well, to avoid
confusion.
Anything that affects public API (such as
flatpak_transaction_progress_set_update_frequency()) or libostree's
options passed to ostree_repo_pull_with_options(), is left as is.
Also, since the lower level APIs don't allow you to pass different tokens
for different parts change this function to support passing a subset
of the resolves, so that we can pass all that need a specific token in
one go, and then call this multiple times. The way we handle this is
by saving all the original ref_to_checksum hashtables for all results
and then re-create them with the subset of refs needed when pulling.
This tries to resolve the p2p resolve operation from the info in
a ostree-metadata commit. This only works if the resolve ended up
on the same commit id as what was available in the ostree-metadata
which may not be correct if the two are not synchronized.
Historically the p2p resolve code always did a parallel call to find
all the available commits for the refs, and then it took the results
and pulled only the commits for all the refs so that it could resolve
against the exact commits that were available (which might not match
with whatever metadata we have in the local ostree-metadata copy.
This splits this into two phases, the first that uses the summary only,
and a second one that pulls the commit.
The reason for this is that we want to be able to do some stuff inbetween
these, such as resolving some refs via the ostree-metadata and maybe
requesting bearer tokens that we need for pulling the commit objects.
While updating, if the related extension is missing on
the installation of an installed ref (could be an app or
runtime), FlatpakTransaction tends to "repair" the ref by
automatically downloading the related extension again and
restoring the overall functionality of the ref.
The related extension concerned that are the ones associated with
`should-download` to TRUE only.
Hence, teach the libflatpak API to do that same, so that clients
like gnome-software can mark those refs as updatable, if their
related extensions is missing.
Previously, in flatpak_dir_setup_extra_data(), n_extra_data (a gsize
which is 8 bytes wide on x86_64) was passed in a varargs list where an
unsigned int (4 bytes wide) was expected due to the "u" variant type
specifier.
This doesn't seem to have directly caused any crashes for me, but it's
undefined behaviour.
Therefore, this changes the affected keys "outstanding-extra-data" and
"total-extra-data" to be guint64 types instead of unsigned ints. The
gsize returned from g_variant_n_children() is cast to guint64 by virtue
of being assigned to a guint64-typed variable, but should not lose any
bits on supported platforms.
It's a common idiom in this codebase to push a temporary GMainContext as
the thread default context in order to run an async operation as if it
were sync. If we are not expecting progress callbacks this isn't a
problem, but it becomes a problem if we pass in an OstreeAsyncProgress
object that was created under a different GMainContext. The reason for
this is that OstreeAsyncProgress creates an idle source and attaches it
to the thread default context, so if we are iterating a temporary
context then the OstreeAsyncProgress's context never gets iterated, and
so no progress signals are fired.
To fix this, we introduce flatpak_progress_chain() and a RAII helper
FlatpakAsyncProgressChained which creates a new OstreeAsyncProgress
under the temporary GMainContext, but forwards all its state and updates
to the previous OstreeAsyncProgress's callbacks.
This is documented in a comment in the code as well.
All known instances of this problem in the existing code are fixed in
this commit.
This uses new API in libostree which is proposed in
ostreedev/ostree#1968. In anticipation of it being included in libostree
version 2019.6, the bug fix is predicated on that version being present.
If compiling against an older version, the old buggy behaviour will be
the fallback.
This problem was solved conceptually by Philip Withnall, I only wrote
the code.
Due to bug #3215 some systems have refs in refs/mirrors/ in addition to
the usual refs/remotes/ location. The remote refs are always at least as
new as the mirror ones since the repo_pull() invocation in
flatpak_dir_pull() which does not use OSTREE_PULL_FLAGS_MIRROR happened
after the one that did. Cleaning up these mirror refs is important since
otherwise when the remote ref is either updated or removed (by an
uninstall) disk space will be leaked since the mirror ref will point to
a no longer needed commit.
So, remove (almost) all mirror refs during flatpak repair, uninstall,
or update operations. And for the uninstall and update operations do it
in FlatpakDir so that it happens regardless of if the CLI of libflatpak
are used.
Also, add a unit test for this.
Fixes https://github.com/flatpak/flatpak/issues/3222
This reverts commit 915ad583a7.
This commit turned out to have unintended side effects. Specifically,
with it we do a pull with OSTREE_REPO_PULL_FLAGS_MIRROR, and then
flatpak_dir_setup_extra_data() does a non-mirror pull in the same
transaction, so the ref being pulled ends up being written to disk under
both refs/remotes/ and refs/mirrors/ in
ostree_repo_commit_transaction(). This is a problem because only the
remote ref is deleted during an uninstall, so the disk space is leaked,
and we don't have the infrastructure in place to keep both refs up to
date as they're updated.
It would be nice to consistently use OSTREE_REPO_PULL_FLAGS_MIRROR for
all pulls but that turns out to be a deep rabbit hole to go down; see
the discussion in https://github.com/flatpak/flatpak/pull/3220
So revert the commit instead (with a few exceptions: keep a
still-relevant FIXME comment, keep an assertion in the "out:"
section, and keep a debug statement printing out the resolved rev).
Note that this means that since we're no longer checking commit
signatures during ref resolution, in theory remote B could try to set
the same collection ID as remote A and serve a malicious update for
something from remote A, but the signature would be found to be invalid
during the pull phase due to our use of "ref-keyring-map" so the
transaction would fail.
All the other uses of OSTREE_REPO_PULL_FLAGS_MIRROR across the codebase
should be kept I think:
- flatpak create-usb uses it when pulling into the repo on the USB which
works perfectly well with refs/mirrors/ (and the USB is mirroring the
collection-refs!)
- it's used when pulling into a temporary "child" repo in a few places
and there it makes sense since the child repo is mirroring the refs so
they can be pulled into the main repo. In fact, in the case of
flatpak_dir_do_resolve_p2p_refs(), we need MIRROR since otherwise
ostree_repo_resolve_collection_ref() gives us the commit on-disk
rather than the just-pulled one that's in memory.
If building with --disable-system-helper, common/flatpak-dir.c might
still try to use polkit APIs. A check for libmalcontent was already in
place but not enough.
Accept the locale format as documented by `setlocale(3)`, rather than
another arbitrary format.
This reworks the validation code, and was tested to accept all the
locales on my F30 system using:
```
flatpak config --user --set extra-languages $(locale -a | tr -s '\n' ';' | head -c -1)
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to configure gnome-software to show specific apps in one region
without showing to all language speakers, we allow the storage of full
locales on the extra-languages key. However, these locales are ignored when
calling flatpak_installation_get_default_languages, so locales will be reduced
to their language identifier (eg. en_IN locale will be returned as 'en', and
az_Latn_AZ will be returned as 'az'). In order to get the full locales, we can
call flatpak_installation_get_default_locales instead, which can return languages
and locales.
Use the user’s OARS filter to prevent installation or upgrade of
apps which have more extreme content than the user is allowed to see.
This uses libmalcontent to load the user’s enforced OARS filter, which
describes the extremeness of each type of content the user is allowed to
see. If an app they are trying to install exceeds the filter value in
any OARS section, installation is disallowed and an error is returned.
libmalcontent stores the parental controls policy per-user in
accountsservice, which enforces access control on the policies.
The app filter is also allowed to prevent app installation entirely,
which overrides the OARS values. This is independent from the app-install
polkit action, which determines whether an unprivileged user may install
an app system-wide. Being stored in accountsservice, the new boolean is
also easier to set per-user without having to programmatically write a
polkit JS policy file which handles multiple users (and parse it back
again).
The parental controls checks are done at deploy time, either in the
`flatpak` process (for user repositories) or in the
`flatpak-system-helper` (for system repositories). The checks use
content rating data extracted from the app’s AppData XML and stored in
the `FlatpakDeploy` cache. The checks are passed through polkit (even
for user repositories) so that users can get an admin override to
install apps which would otherwise be too extreme. This uses the new
`org.freedesktop.Flatpak.parental-controls` polkit rule.
The checks have to be done at deploy time, as that’s when the AppData
XML for the app is parsed. The downside of this arrangement is that an
app must be entirely downloaded before the parental checks can be done.
This won’t be much of an issue on normal desktops, however, since we can
assume that gnome-software will check an app’s appropriateness before
showing it to the user in the first place.
Parental controls are not enforced for non-apps/runtimes, which includes
the ostree-metadata and appstream/* refs.
One thorny issue is that flatpak unit tests may be run in an environment
with no system D-Bus available to connect to (a Jenkins instance, for
example), which means the call to `mct_manager_get_app_filter()` in
`flatpak_dir_check_parental_controls()` fails.
So this commit skips the parental controls check if the system bus is
unavailable and the environment variable
`FLATPAK_SYSTEM_HELPER_ON_SESSION` is set, since the testlibrary already
sets that variable so that the system-helper will be started on the
session bus.
The feature can be tested using something like:
```
$ malcontent-client set philip \
violence-realistic=none app/org.freedesktop.Bustle/x86_64/stable
App filter for user 1000 set
$ flatpak run org.freedesktop.Bustle
error: Running app/org.freedesktop.Bustle/x86_64/stable is not allowed by the policy set by your administrator
$ flatpak --user install flathub io.github.FreeDM
error: Failed to install io.github.FreeDM: Installing app/io.github.FreeDM/x86_64/stable is not allowed by the policy set by your administrator
```
Includes work by André Magalhães and Umang Jain.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This will be used in upcoming commits to enforce parental controls on
app installations.
We extend version 2 of the deploy data format because it has not
appeared in a release yet.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/flatpak/flatpak/pull/2797
