Commit Graph

3582 Commits

Author SHA1 Message Date
Alexander Larsson
7fd085acd7 appstream: Generate appstream2/$arch branches with uncompressed xml
These delta better, and the files will be downloaded as compressed
.filez objects anyway.

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
5102a19d22 appdata: Sort by ref-order to make appdata stable
This is nice for e.g. doing deltas of them.

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
33c8e29d3f appdata: Only pull appstream branch for main arch
Now that it also includes compat refs as needed we don't need to
pull other branches.

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
484c743e4a appdata: Include compatible arches in main arch when unique
For example, if a i386 build is in the repo but no x86-64 version then
also add the i386 build to the x86-64 appstream data.  However, don't
add it if that would cause a duplicate (i.e. both the x86-64 and i386
version).

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
4fbf50d7b7 appstream: Check appstream timestamp when updating single remote too
Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
096f7d4fb5 appstream: Drop flatpak_dir_check_for_appstream_update
This early bailout is not really needed, because noop updates is
pretty fast. Also, doing that breaks the timestamp updates.

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Matthew Leeds
9afca51507 remote-ls: Allow a URI instead of remote name
This fixes the ability of the remote-ls command to take a file:// URI
instead of a remote name, which is especially useful for repos on USB
drives (created via `ostree create-usb`) which are temporary and don't
warrant being added to the repo config. This commit also updates
relevant documentation, adds a unit test, and updates a few variable
names to improve readability.

I can't find a commit in the history where this was working, but it's
working on the Endless fork of flatpak so I think there was agreement at
some point that it's desired behavior.

Fixes https://github.com/flatpak/flatpak/issues/1588

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Matthew Leeds
79907f4236 dir: Fetch ostree-metadata independently of summary
Currently _flatpak_dir_get_remote_state() only fetches the
ostree-metadata ref if it was able to fetch the remote summary, but this
is unnecessary because we don't need to know the checksum just to fetch
it, and this is especially problematic in the offline use case when the
remote summary can't be fetched. So this commit makes flatpak fetch
ostree-metadata even if the summary fetch failed, which is consistent
with how things worked before commit fedb0e5bd.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
0bb024869a lib/installation: Don't try to find dynamic remotes when not needed
When getting the dynamic remotes for the installed refs, if no refs
with a collection ID are found, then the ostree_repo_find_remotes_async
is given an array with just NULL, which makes it early return and thus
it would result in an infinite loop in
flatpak_installation_list_installed_refs_for_update.

This patch only tries to find the dynamic remotes when there are such
refs, thus preventing the mentioned error and any extra unnecessary
processing.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Matthew Leeds
5813639f15 lib: Properly handle missing Avahi daemon
In list_remotes_for_configured_remote(), we call
ostree_repo_finder_avahi_start() without checking the error status. This
means the OstreeRepoFinderAvahi instance passed to
ostree_repo_find_remotes_async() could be in an error state, which leads
to flatpak getting infinitely stuck waiting for the result of that
operation. So this commit removes the OstreeRepoFinderAvahi instance
from the array if it failed to start, and returns early if only LAN
remotes were requested. This is also consistent with how ostree itself
handles an error starting the Avahi finder instance in case NULL is
passed to ostree_repo_find_remotes_async() instead of a finders array.

This error condition happens on the Endless OBS server because there's
no Avahi daemon in the chroot used to run "make check".

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
7e0bf40758 lib: Make fetching a remote ref more robust
Fetching a remote ref may fail because of the locally configured
collection ID not matching the remote's one (because it's never been set
in the configuration or it's just different/incorrect),

However, using `flatpak_installation_fetch_remote_ref_sync` there's no
mention of the collection ID to the caller, so we should make sure we
try to do our best to match the given parameters. Thus, if the lookup
fails (because the collection IDs don't match), we resort to iterating
over every ref and comparing their names with the expected ones.

This should be however reworked, through a new method likely, in order
to accept the collection ID as an input argument, so the caller has more
power to choose which one to match.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
aa4074374a lib/installation: Don't stop getting removable remotes if one attempt fails
When listing the removable remotes, it will stop the operation after the
first configurable remote that has no collection ID set.
This is of course a problem as such an occurrence is not a reason for
not trying to list any remaining remotes.

This patch fixes that by simply reporting the failures as a debug
message instead of aborting.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
ecde260f65 lib/installation: List only static remotes when using flatpak_installation_list_remotes
When p2p is enabled, listing the remotes using
flatpak_installation_list_remotes will include all types of remotes.
However, this can lead to problems for the users since dynamic remotes
(USB/LAN) may not be used always the same way that static ones are. For
instance, when listing the refs in a remote, dyanmic ones cannot be
listed by name; so any existent code that iterates through the list of
remotes and lists the refs in them will not work properly.

To avoid this situation, this patch makes the API method in question to
return only static type remotes. Getting all remotes can still easily be
accomplished by calling flatpak_installation_list_remotes_by_type with a
0-sized array (or by specifying all types).

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
c5e0e97395 lib/installation: Add a method to list remotes by type
Since we have several types of remotes, it is important to have the
option of choosing which types one wants when listing them because
depending on the type it may have an impact in the performance (e.g.
when listing LAN remotes).

For that reason this patch adds a new method
"flatpak_installation_list_remotes_by_type" that allows to specify
which types of remotes should be returned. When giving an empty array
of types, it means that all types should be returned instead, which can
be more useful than actually asserting or not doing anything.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
508ceb74e3 doc: Mention the use of the URI for listing refs from remotes
When listing refs from remotes, a remote can be given by name or by URI.
This is an important distinction because dynamic remotes (USB/LAN) are
not internally mapped to a name, and thus their generated name cannot be
used for listing them. Thus, the solution is to use their URI in order
to list the refs directly from it.

Even though this feature has been around forever, the documentation
didn't really reflect it, so this patch mentions the described
alternative possibility.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
33f9029bb1 tests: Cover listing refs from a remote by URL
Listing refs from a remote can be done using a URL so we need to cover
this in the unit tests.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
2465cf49d4 tests: Cover listing refs with different collection IDs in remotes
This patch tests if refs of different collection IDs are returned
together when listing refs from remote.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
1c8dd13f55 tests: Make the app ID in make-test-app.sh a parameter
This is in order to allow tests to make several apps if needed, and will
be useful when e.g. testing repos that have multiple apps with different
collection IDs.

This patch modifies the mentioned script as mentioned and updates the
places that call it.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
516e249e50 lib/installation: Use "collection refs" when listing refs
When listing refs installed or from a remote, only the refs matching the
main collection-id were being returned. However, it is very important to
have access to all refs, independently from their collection-id,
especially when trying to list remotes coming from a USB repository.

These changes add the mentioned refs and update the places that use
this list, both in the lib and in the CLI.

For the implementation to become easier, we introduce also a
FlatpakCollectionRef, that should be replaced by OstreeCollectionRef
once the latter becomes part of the general API (currently it is in the
experimental one).

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
dc4d16ecbe lib/remote: Add the URL to the remote when creating it from ostree
Since we can now get the URL of the remote directly from the
OstreeRemote object, we should use it when creating a FlatpakRemote.

This will allow temporary FlatpakRemote objects (corresponding to
removable mounts) to have a URL set to them.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Alexander Larsson
abb1e6f7fd lib: Make all *Ref properties construct-only and static-strings
The Ref objects are always created by the library itself, and are
immutable.

Closes: #1591
Approved by: alexlarsson
2018-04-19 08:17:46 +00:00
Alexander Larsson
c33f842b70 lib: Add metadata info in FlatpakRemoteRef
This adds things like the size and the metadata, as well as eol strings
to FlatpakRemoteRef. We typically have this accessible anyway, in the
FlatpakRemoteState.

This makes flatpak_installation_fetch_remote_size/metadata_sync deprecated.

Closes: #1591
Approved by: alexlarsson
2018-04-19 08:17:46 +00:00
Sam Spilsbury
c7d4eeab71 build-finish: Allow exporting .service files that we are allowed to own
To implement this, we have a concept of a custom export filter
function which can be specified for each path to determine the
files that can be exported under that path.

Closes: #1589
Approved by: alexlarsson
2018-04-19 07:31:25 +00:00
Sam Spilsbury
b8cfe2e7a3 dir: Export d-bus .services files that we are allowed to own
Closes: #1589
Approved by: alexlarsson
2018-04-19 07:31:25 +00:00
Sam Spilsbury
a8620a5592 utils: Export flatpak_name_matches_one_*_prefix
We will use this later to determine what files to export
in both the build-export and build-finish stages.

Closes: #1589
Approved by: alexlarsson
2018-04-19 07:31:25 +00:00
Sam Spilsbury
b0b8f55ecb run: Export flatpak_context_get_session_bus_policy_allowed_own_names
We will use this later to work out which .service files we can export
based on names that we are allowed to own.

Closes: #1589
Approved by: alexlarsson
2018-04-19 07:31:25 +00:00
Matthew Leeds
ac690c4a1e Merge pull request #1590 from alexlarsson/batch-prunes
Batch prunes of the local repo
2018-04-18 23:48:13 -07:00
Matthew Leeds
2ec5f25599 dir: Fix grammar in a debug statement 2018-04-18 23:41:49 -07:00
Matthew Leeds
63052a51d6 dir: Update outdated comment 2018-04-18 16:16:18 -07:00
Sam Spilsbury
ceeda9b16d run: Add --no-documents-portal option
This prevents flatpak from trying to talk to the documents portal
on startup.

Closes: #1584
Approved by: alexlarsson
2018-04-18 12:05:16 +00:00
Alexander Larsson
1355713db7 Batch prunes of the local repo
This moves the prune call out of flatpak_dir_update() and
flatpak_dir_uninstall() and instead does this manually at all call
sites. The advantage is that we now only call it *once* even if you
uninstall or update multiple apps.

This means update everything is much faster as we don't have to scan
over the entire local repo for each updated app.
2018-04-18 12:00:12 +02:00
Sam Spilsbury
fcb92b3ee3 builtins-build-init: Support --extension-tag to init build dir
Closes: #1531
Approved by: alexlarsson
2018-04-18 07:43:56 +00:00
Sam Spilsbury
0cfcb90ed8 builtins-build: Search for best matching tagged extension when building extension
Closes: #1531
Approved by: alexlarsson
2018-04-18 07:43:56 +00:00
Sam Spilsbury
efa183e3d6 dir: Also handle tagged extension names in find_remote_related and find_local_related
Closes: #1531
Approved by: alexlarsson
2018-04-18 07:43:56 +00:00
Sam Spilsbury
bc5cfafb26 builtins: Support '@' tag in extension names
Closes: #1531
Approved by: alexlarsson
2018-04-18 07:43:56 +00:00
Sam Spilsbury
b6d1a20223 flatpak-utils: Remove tag from extension name before adding it
Closes: #1531
Approved by: alexlarsson
2018-04-18 07:43:56 +00:00
Matthew Leeds
14dde8a37d dir: Use child_repo_ensure_summary() more
This introduces no functional change; it just makes use of a helper
function and updates a comment.
2018-04-17 11:46:02 -07:00
Matthew Leeds
1e1ed6911e dir: Tweak an outdated comment 2018-04-17 11:31:41 -07:00
Sam Spilsbury
66e343fcb8 dir: Add support for X-Flatpak-RunOptions in exported desktop files
A desktop file can specify this key to add additional arguments
to flatpak run. Right now, only 'no-a11y-bus' is supported.

Closes: #1579
Approved by: alexlarsson
2018-04-17 08:50:56 +00:00
Alexander Larsson
4bf892a06f common: Rename FlatpakRemoteState->remote to remote_name
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
534c7a2fc8 tests: Fix unused variable warning in non-p2p case
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
482b494d9f Fix unused function warning in non-p2p case
_flatpak_dir_fetch_remote_state_metadata_branch is only used when building
with p2p, so #ifdef the entire function.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
9646f42ba8 dir: Remove unused function flatpak_dir_lookup_ref_from_summary
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
88edfda861 dir: Pass FlatpakRemoteState to flatpak_dir_pull/install/update
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
0e5baf3837 dir: Remove unused functions
flatpak_dir_get_remote_metadata and flatpak_dir_lookup_repo_metadata are now
not used anymore.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
48796fb29d dir: Pass FlatpakRemoteState to flatpak_dir_list_remote_refs
Reusing the summary and metadata here helps us a lot as typically we
often want to look up the cache data again for every ref in the list.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
2db82391c0 dir: Use FlatpakRemoteState for flatpak_dir_check_for_update
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
1936d2daae dir: Pass FlatpakRemoteState to flatpak_dir_find_remote_related
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
d791625d77 transaction: Use FlatpakRemoteState to look up metadata
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
37cdfa1d1d Transaction: Track RemoteState for all used remotes.
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00