If the magical io.github.containers.DeltaUrl label is set in the
index, then try to download this to use as the delta manifest for the
image. This allows servers to store deltas outside the registry
itself. The label is propagated to the xa.delta-url metadata in the
generated "fake summary" for the remote, and read back on pull.
Note that the delta manifest layers descriptor will need to have a
"urls" key where it references the blobs if the blobs are also not
stored on the registry.
If the specified manifest doesn't exist or doesn't apply to the target
image we fall back to resolving via the _deltaindex tag.
When mirroring to a local OCI dir we apply deltas and generate
uncompressed layer blobs instead of regular blobs. Then we pick this
up on the system-helper side.
This adds the nr of extra datas, as well as the total size in the sparse
cache for all refs that has them. This is what we need for in the download,
and having it in the summary means we don't have to separately download
the commit.
Additionally we add a cache version field to the summary so that we
can know if we can rely on the existance of the new data.
To avoid the complexities of passing (and chaining) OstreeAsyncProgress
objects around, we only create one just before calling to ostree.
The rest of flatpak only ever uses the new FlatpakProgress object.
Co-authored by: Philip Chimento <philip@endlessm.com>
This is a new version of --deploy-collection-id that only applies
the collection id update for new (1.7.x+) version of flatpak clients.
This allows you to enable collection ids for sideload use but not
affect older clients where the p2p codepaths are not as tested.
With the new sideload approach to collection ids it is fine to require
gpg signed summaries. (Not for the child repos or the sideload repos
though, but thoser are either trusted (sideload) or safe for other
reasons like ref-bindings and signed commits).
Allow the app-id or the DConf path to finish with a digit and still be
considered similar enough for DConf migration purposes.
This allows the org.gnome.Rhythmbox3 app-id to migrate its
/org/gnome/rhythmbox DConf path.
See https://github.com/flathub/org.gnome.Rhythmbox3/pull/26
When storing the token-type in the commit and the summary cache we
hardcode it to little-endian.
In theory this breaks the "ABI", but in practice this change is a
no-op on little-endian systems which is what most are. Additionally as most
servers are little-endian this also fixes using big-endian clients with
such servers.
This fixes:
https://github.com/flatpak/flatpak/issues/3434
We can't use the built-in bsearch from the codegen because its an array
instead of a dict, so we have to keep that but its now not using
variant at least.
In flatpak-builtins-build-commit-from.c we call flatpak_repo_collect_sizes()
without initializing the passed in download size to zero, which mean
we sum with sizes with some random value as the start.
This is fixed by having flatpak_repo_collect_sizes() always initialize
the counters to 0 at the start.
Fixes https://github.com/flatpak/flatpak/issues/3362
flatpak build-update-repo now lets you modify the
autenticator-name/install/options keys, and these are migrated to
the summary/metadata during update.
With the latest ostree that enables the chaining of progress the
testsuite broke because we were not getting changed events. Looking
into this the reason seems to be that when we run the
ostree_async_progress_finish() on the chained progress it is marked
as dead, which causes ostree_async_progress_copy_state() to not copy
any data when called from handle_chained_progress().
The fix is to copy the content manually before calling the finish().
Also, the entire callback chaining system seems wildly
overcomplicated, so I simplified it by relying on the existing change
notification of OstreeAsyncProgress.
If we're using a chained progress, it will be unchained
in the destroy notifier. However, it was newly constructed so we
need to also unref it or we'll leak it.
This also makes some minor cleanups:
1) Centralize version checks to one place and replace users
with #ifdef FLATPAK_DO_CHAIN_PROGRESS which makes it
easier to read and to test the fallback.
2) Make flatpak_progress_chain return a FlatpakAsyncProgressChained
to make it clear the two needs to be paired.
This is a slightly incompatible change, as we now only support
oci images generated with (what was before) build-export --oci-use-labels.
However, there are not a lot of OCI implementations in the wild, and
we can modify the ones in use to ensure there are labels (and
annotations if needed for older flatpak clients).
This also removes the --oci-use-label option from build-bundle --oci as
this is now the default.
This is in the same order as the xa.cache array and contains the id of
the commit that the cached data is about. This is not necessary in the
non-p2p summary metadata, because in that we always have a matching
ref -> commit array.
However, in the p2p case this information can be useful.
These are explicitly made short to save space, so lets have defines
for them to make sure we don't mistype them, especially as we
will be adding new keys.
The previous code checked whether the progress object had
"outstanding-fetches" set in order to decide whether it had been
initialized enough to show progress information. However, if the
callback saw a progress object on which flatpak_dir_setup_extra_data()
had not yet been called, then it would crash.
Therefore it seems that we should additionally be checking for the
presence of "outstanding-extra-data" which is set in
flatpak_dir_setup_extra_data().
It's likely that this wasn't previously a problem because the callback
would never get called due to the progress object's associated main
context not being iterated. It crashes now because that problem was
fixed in a previous commit.
Previously, in flatpak_dir_setup_extra_data(), n_extra_data (a gsize
which is 8 bytes wide on x86_64) was passed in a varargs list where an
unsigned int (4 bytes wide) was expected due to the "u" variant type
specifier.
This doesn't seem to have directly caused any crashes for me, but it's
undefined behaviour.
Therefore, this changes the affected keys "outstanding-extra-data" and
"total-extra-data" to be guint64 types instead of unsigned ints. The
gsize returned from g_variant_n_children() is cast to guint64 by virtue
of being assigned to a guint64-typed variable, but should not lose any
bits on supported platforms.
It's a common idiom in this codebase to push a temporary GMainContext as
the thread default context in order to run an async operation as if it
were sync. If we are not expecting progress callbacks this isn't a
problem, but it becomes a problem if we pass in an OstreeAsyncProgress
object that was created under a different GMainContext. The reason for
this is that OstreeAsyncProgress creates an idle source and attaches it
to the thread default context, so if we are iterating a temporary
context then the OstreeAsyncProgress's context never gets iterated, and
so no progress signals are fired.
To fix this, we introduce flatpak_progress_chain() and a RAII helper
FlatpakAsyncProgressChained which creates a new OstreeAsyncProgress
under the temporary GMainContext, but forwards all its state and updates
to the previous OstreeAsyncProgress's callbacks.
This is documented in a comment in the code as well.
All known instances of this problem in the existing code are fixed in
this commit.
This uses new API in libostree which is proposed in
ostreedev/ostree#1968. In anticipation of it being included in libostree
version 2019.6, the bug fix is predicated on that version being present.
If compiling against an older version, the old buggy behaviour will be
the fallback.
This problem was solved conceptually by Philip Withnall, I only wrote
the code.
This will be used for parental controls, and potentially other reasons
for denying installation of an app.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This means you can use "org.foo.bar//stable" instead of "org.foo.bar/*/stable"
which is similar to what other APIs do.
We want to use this for masking extensions too, thus the export.
Support a list of versions that are supported. This will be useful
for e.g. the extra_data for extensions once that is backported to
1.2, because that will require it to say that it is supported for
> 1.2.5 in the 1.2 series and > 1.4.2 otherwise.
Closes: #3112
Approved by: alexlarsson
flatpak_dir_collect_deployed_refs() has the order of its "branch" and "arch"
arguments wrong, as does its only caller flatpak_list_deployed_refs().
When flatpak_list_deployed_refs() is called by add_extension() the arch
is put in the branch argument and vice versa. But then in the
implementation the arch is used as if it's the branch and vice versa, so
there's no functional bug here. Fix the order for readability.
Similarly, flatpak_list_unmaintained_refs() has the order wrong, but the
confusion is only within that function, since the order is correct in
its caller add_extension() and in the function it uses,
flatpak_dir_collect_unmaintained_refs(). So there's no functional bug
there either, but fix the order.
Closes: #3067
Approved by: alexlarsson
When we export a manifest to the index, always pass the ref we're
targeting instead of relying on the org.opencontainers.image.ref.name
annotation, because that may not be set if we're using labels instead.
This is no big deal, because we know what ref we're handling anyway.
Closes: #2978
Approved by: alexlarsson
We now pull the image config as well as the manifest and fall
back on the labels field if the keys we're looking for are not
in the annotations field.
This lets us support docker manifests too, which don't have
annotations (but do have labels).
Closes: #2978
Approved by: alexlarsson
The org.freedesktop.Flatpak user D-Bus service isn't just used by
flatpak(1) or applications running as Flatpaks. It's also used by
toolbox(1) for similar reasons:
* To keep various configuration files inside the container
synchronized with the host
* To let the container request certain commands to be run on the host
The org.freedesktop.Flatpak D-Bus service itself doesn't need much in
the way of dependencies, but inherits a lot of unused shared library
linkages through the libflatpak-common.la convenience library. Removing
these unused shared libraries reduces the footprint of toolbox(1) for
those who care about such things. eg., Fedora CoreOS.
This commit brings down the number of shared libraries to 19 from 62.
Closes: #3052
Approved by: alexlarsson
