On some code paths, flatpak_dir_get_remote_state_optional() is used so
summary fetch errors and metadata fetch errors are treated as non-fatal,
and later flatpak_remote_state_ensure_summary() or
flatpak_remote_state_ensure_metadata() is used, which gives a pretty
unhelpful error message. So this commit saves the error from a failed
summary or metadata fetch into the FlatpakRemoteState object and prints
the message when the flatpak_remote_state_ensure_* function is called to
make that error message more specific and useful.
This also means that the "fail with missing gpg key" unit test in
test-repo.sh is working again. It was broken by the last commit ("dir:
make optional summary really optional) which caused the install
operation in that test to fail later than before with a different error
message.
Closes: #1637
Approved by: alexlarsson
This makes flatpak_remote_state_lookup_cache() return the metadata
from the actual FlatpakRemoteState rather than duplicating it.
All callers are updated to not free anymore and to strdup if needed.
Closes: #1594
Approved by: alexlarsson
This early bailout is not really needed, because noop updates is
pretty fast. Also, doing that breaks the timestamp updates.
Closes: #1585
Approved by: alexlarsson
When listing refs installed or from a remote, only the refs matching the
main collection-id were being returned. However, it is very important to
have access to all refs, independently from their collection-id,
especially when trying to list remotes coming from a USB repository.
These changes add the mentioned refs and update the places that use
this list, both in the lib and in the CLI.
For the implementation to become easier, we introduce also a
FlatpakCollectionRef, that should be replaced by OstreeCollectionRef
once the latter becomes part of the general API (currently it is in the
experimental one).
Closes: #1587
Approved by: mwleeds
Reusing the summary and metadata here helps us a lot as typically we
often want to look up the cache data again for every ref in the list.
Closes: #1575
Approved by: alexlarsson
This is an object that contains the state of a particular remote
at a point in time, such as the summary and the metadata. Storing
this in an object means we can reuse these data instead of constantly
looking them up.
This is more efficient, but also avoids possible inconsistencies if
the state changes during an operation.
There is also an "optional" mode for the state where we don't fail in
the p2p/collection-id case if it fails to load the summary or metadata.
This is useful in order to support certain operations "offline" (i.e.
with no network connection to the main repo).
Closes: #1575
Approved by: alexlarsson
Apparently when I rebased https://github.com/flatpak/flatpak/pull/1314
to master, git cleanly applied the patch in a different place than it
was originally. Commit c5ab9e22b moved the find_remotes_async() call to
find_latest_rev(), so all that needs to be done is pass along the
checksum in check_for_update(). This way when you're trying to downgrade
something the specified commit will be searched for instead of the
latest one.
Fixes https://github.com/flatpak/flatpak/issues/1431Closes: #1435
Approved by: barthalion
This commit breaks out the code in flatpak_dir_check_for_update() that
finds the latest revision of a ref into its own function and uses it in
flatpak_dir_check_for_appstream_update(). This allows appstream updates
to work even for offline machines.
Fixes https://github.com/flatpak/flatpak/issues/1404Closes: #1397
Approved by: alexlarsson
When building with --disable-p2p, we create a stub type for
OstreeRepoFinderResult and OstreeRepoFinderResultv to avoid having to
add #ifdefs around all uses of them throughout the code base. We also
need to create autoptr functions for them so that the code can continue
to use g_autoptr(OstreeRepoFinderResult).
Previously, we were using `void` as the GDestroyNotify function for the
stub types. This wasn’t valid (it’s not a function), but it worked.
Since g_autolist() support has landed in GLib, this has broken. Fix it
by using a static inline no-op function as the GDestroyNotify function
instead. This should never be called, so exists purely to get things to
compile.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/flatpak/flatpak/issues/1279Closes: #1293
Approved by: mwleeds
If you're installing something and its already installed, we undeploy
the old install first before deploying the new. This makes it very
easy to switch an application from one remote to another, without
having to uninstall first, which is both painful and could cause
the download to be unnecessary large.
Closes: #1241
Approved by: alexlarsson
This tries to ensure that FlatpakDir->repo is set, but if the
repo doesn't exists and we're unable to create it that is fine
and we continue running with repo == NULL.
This is useful because there are only a few operations that really
are useful/make sense with no repo set up, and we can ensure these
handle this well.
In particular, we want to allow flatpak remote-add using the
system-helper to work even if /var/lib/flatpak doesn't exist, so that
we can bootstrap a working flathub from a fresh /var.
Closes: #1195
Approved by: alexlarsson
This is mostly a cleanup of some code and extracting common code to
helper functions. But additionally, we switch the xa.languages config
value to be a list of languages, rather than dirs, although for
backwars compat we still support if it specifies dirs.
Closes: #1098
Approved by: alexlarsson
This is taken in exclusive mode whenever we prune the repo. Anything
that is not protected against object disappearing from the repo can
take the lock in a shared mode to avoid running at the same
time as a prune operation.
The two operations the are problematic in this respect is
Pull:
* During the pull we see that some object we require is already available
and doesn't need downloading. We can't have it be removed before we
commit the transaction.
* During the transaction commit we're moving the object to the
repo, and they risk being pruned as unreachable until we have updated
the ref.
Deploy:
* Once we start checking out a particular ref we assume all the object
from it is reachable. If the ref is updated in parallel some object
can become unreachable and removed.
So, we take shared locks in these operations.
In the prune operation we take the block non-blocking, and skip
the prune entirely if some other operation is outstanding, because
we don't want to block a long time, and its likely that due to the other
operation we will run prune shortly anyway.
Note: Nothing protects the system-helper case when we download to a separate
repo. However, if there is a race we will get an error when importing this
to the system repo, so we'll never end up in an inconsistent state.
We're scoping the deploy directory by the subpaths these days, so
we need to take that into account when finding the metadata file.
Fixes https://github.com/flatpak/flatpak/issues/1014
If a flatpakref has this set, for instance:
SuggestRemoteName=gnome-apps
Then flatpak install will ask if you want to configure
this as a "real" remote, rather than an origin remote (which
will only install that app).
This is useful when creating flatpakref files for remotes
that have multiple applications in them, such as e.g. flathub
or the gnome nightly builds. However, it should not be
set of one-application repositories.
This isn’t really used internally, but will be used by gnome-software
for when it configures new flatpak remotes.
This is new public API, but is only declared if compiling with
--enable-p2p.
Includes some basic smoketests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add support for collection IDs to the code which finds and pulls
related refs and other extensions.
Currently, related refs must have the same collection ID as the parent
ref — this is the most likely scenario anyway. In future, it should be
possible to extend the code to support pulling related refs from other
collections.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to eliminate some race conditions around updating the
summary{,.sig} file on the server, and to decouple signing the summary
from signing commits, and to support peer to peer mirrors of content
from multiple upstream collections: add support for unsigned summary
files.
This relaxes the requirement for gpg-verify-summary=true iff
collection-id is set in a remote’s local configuration. It depends on
some pending libostree changes to verify the ref for each commit using
the commit’s signed metadata. See
https://github.com/ostreedev/ostree/issues/983.
Metadata storage has moved from the summary file to a new
ostree-metadata well-known branch on each repository, since this can be
signed for each update and for each collection separately. If the
collection-id is set in a remote’s local configuration, flatpak will
retrieve all repository metadata from this branch rather than from the
summary file. If collection-id is unset, it will ignore this branch and
continue to use the summary file, which will continue to be updated (and
externally signed as summary.sig) for backwards compatibility.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Search for updates on peer to peer sources as well as the internet in
check_for_updates(), and pass the resulting OstreeRepoFinderResult array
to the pull() calls, so a consistent set of checksums are pulled.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Read an xa.languages key from the [core] section of
the repo config to determine which subpaths to install
for Locales. This lets us maintain a list of system
languages without inventing a new file in /etc, and
will also work for alternative install locations.
There are several places which query keys in the metadata in the summary
file. Factor out a common function to do that, and write it in such a
way that it could be extended to read the metadata from somewhere else
in future (the plan being that metadata will move to an ostree-metadata
branch rather than the summary file).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Writing things on the console every 100msec doesn't really
make any sense, the progress bar is not finegrained enough
and you can't read the text that fast anyway.
