Commit Graph

422 Commits

Author SHA1 Message Date
Philip Withnall
d87d5ed759 common: Check core.add-remotes-config-dir before modifying remote config
See the comment added in the code for an explanation: if
core.add-remotes-config-dir=true, we must avoid adding configuration in
both /etc/flatpak/remotes.d/ and /ostree/repo/config. In lieu of
actually supporting adding the configuration in the right place, bail
early to avoid leaving partial configuration in the wrong place.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

https://github.com/flatpak/flatpak/issues/1665

Closes: #1666
Approved by: alexlarsson
2018-05-14 07:18:57 +00:00
Matthew Leeds
6e61be6547 dir: Make find_remote_ref(s) work offline
This commit makes two more functions P2P friendly:
flatpak_dir_find_remote_ref() and flatpak_dir_find_remote_refs(). This
is accomplished by getting the list of remote refs from the Flatpak
cache stored on the ostree-metadata ref, rather than from the remote
summary. I put the remote refs into the same data structure that was
used before so the rest of the code can stay the same. It would be more
efficient to use a separate data structure for the cached refs so we can
do a binary search rather than a linear one, but I don't think the
performance difference will matter.

One effect of this is that offline P2P app installs (such as from a USB
drive) are working now. This is also a partial fix for
https://github.com/flatpak/flatpak/issues/1583

Fixes https://github.com/flatpak/flatpak/issues/1522

Closes: #1670
Approved by: alexlarsson
2018-05-14 07:15:46 +00:00
Matthew Leeds
9ed95fb77a dir: Save error messages from summary/metadata fetches
On some code paths, flatpak_dir_get_remote_state_optional() is used so
summary fetch errors and metadata fetch errors are treated as non-fatal,
and later flatpak_remote_state_ensure_summary() or
flatpak_remote_state_ensure_metadata() is used, which gives a pretty
unhelpful error message. So this commit saves the error from a failed
summary or metadata fetch into the FlatpakRemoteState object and prints
the message when the flatpak_remote_state_ensure_* function is called to
make that error message more specific and useful.

This also means that the "fail with missing gpg key" unit test in
test-repo.sh is working again. It was broken by the last commit ("dir:
make optional summary really optional) which caused the install
operation in that test to fail later than before with a different error
message.

Closes: #1637
Approved by: alexlarsson
2018-05-03 06:21:46 +00:00
Cosimo Cecchi
fefd42e83b dir: make optional summary really optional
We should not require a collection ID to be set for this to actually be
optional.
Doing so will make us require a network connection for flatpak installs
and updates even in case we already have the requested ref available.

A use-case that breaks is pulling a ref beforehand (e.g. using ostree
pull, or flatpak with FLATPAK_INSTALL_FLAGS_NO_DEPLOY), and later
deploying it explicitly at a time when network is not available anymore
(e.g. during boot time), using FLATPAK_INSTALL_FLAGS_NO_PULL.

This also means that creating a FlatpakRemoteState object for a disabled
remote no longer causes an error as long as the summary is optional, so
installing .flatpak bundles works again.

Fixes https://github.com/flatpak/flatpak/issues/1636

Closes: #1637
Approved by: alexlarsson
2018-05-03 06:21:46 +00:00
Matthew Leeds
013ff8d900 dir: Don't try to fetch the summary of disabled remotes
In flatpak there are two ways a remote can be disabled, an empty URL or
xa.disable set to TRUE. flatpak_dir_remote_fetch_summary() already
doesn't try to fetch the summary in the former case; this checks the
latter as well.

Closes: #1641
Approved by: alexlarsson
2018-05-02 16:23:20 +00:00
Alexander Larsson
8f428fd768 Re-fix export filename checks, this time also for build-finish
The fix in 5122766284 was not right.
First of all it didn't changethe regressing behaviour for
build-finish, just install.

Secondly, it changed the prefix matches to always use
flatpak_name_matches_one_wildcard_prefix() which had slight
differences in behaviour that the previously used
flatpak_name_matches_one_prefix. For example, it always removed
the extension, no matter what it was.

This has all been replaced with a shared (between install and
build-finish) version that is more strict with extensions, and with
what names can be exported as service files (these have to match
exactly, with the wildcard and cannot have suffixes).

To test this i tried to install these apps from flathub that has
some more complex exports:

 org.sparkleshare.SparkleShare:
  org.sparkleshare.SparkleShare.Invites.desktop
  org.sparkleshare.SparkleShare-symbolic.svg

 org.libreoffice.LibreOffice:
  org.libreoffice.LibreOffice.desktop
  org.libreoffice.LibreOffice-impress.desktop
  org.libreoffice.LibreOffice-writer.png
  org.libreoffice.LibreOffice-calc.png

 com.github.bajoja.indicator-kdeconnect:
   com.github.bajoja.indicator-kdeconnect.desktop
   com.github.bajoja.indicator-kdeconnect.settings.desktop
   com.github.bajoja.indicator-kdeconnect.tablettrusted.svg

 com.github.philip_scott.spice-up:
   com.github.philip_scott.spice-up.svg
   com.github.philip_scott.spice-up-mime.svg
   com.github.philip_scott.spice-up.mime.xml

 org.gnome.Recipes:
   org.gnome.Recipes.desktop
   org.gnome.Recipes.service
   org.gnome.Recipes-search-provider.ini
   org.gnome.Recipes.png
   org.gnome.Recipes-symbolic.symbolic.png
   org.gnome.Recipes-mime.xml

 org.gnome.Characters:
   org.gnome.Characters.desktop
   org.gnome.Characters.BackgroundService.service
   org.gnome.Characters.service
   org.gnome.Characters.search-provider.ini
   org.gnome.Characters.png
   org.gnome.Characters-symbolic.svg

 org.gnome.Weather
   org.gnome.Weather.Application.desktop
   org.gnome.Weather.Application.service
   org.gnome.Weather.BackgroundService.service
   org.gnome.Weather.Application.search-provider.ini

 org.gpodder.gpodder:
   org.gpodder.gpodder.desktop
   org.gpodder.gpodder.gpodder-url-handler.desktop
2018-05-02 14:55:20 +02:00
Alexander Larsson
5122766284 dir: Fix prefix matching to allow all you to own all org.App.Id.* names
Its always been allowed to own all subnames, but https://github.com/flatpak/flatpak/pull/1589 regressed
that as it meant apps were not allowed to export matching service files.

This fixes https://github.com/flatpak/flatpak/issues/1627
2018-04-30 14:33:13 +02:00
Matthew Leeds
2a9523b56a dir: Sometimes avoid summary use during local pulls
This commit changes flatpak_dir_pull_untrusted_local() to avoid using
the summary file in the source repo if a collection ID is set and use
ostree_repo_resolve_rev() instead in that case. Since summary signatures
are not part of the security model when collection IDs are used, this
doesn't change any security properties. But it does make pulls succeed
when there's nothing to do (i.e. the child repo has no refs because the
parent repo already has the desired commit). In that case, the summary
file will be empty if collection IDs are being used because of
child_repo_ensure_summary().

The end result of this is that "flatpak update --appstream" works when
there's a collection ID set on the remote and the appstream is already
up to date locally. This problem only recently became visible because
flatpak_dir_check_for_appstream_update() was removed in a recent commit.

This is a partial fix for https://github.com/flatpak/flatpak/issues/1615
2018-04-30 14:33:09 +02:00
Matthew Leeds
74aa0130b0 dir: Fix a memory leak of checksum data 2018-04-30 14:33:09 +02:00
Matthew Leeds
102af70c0e dir: Properly get appstream checksum from child repo
In flatpak_dir_update_appstream(), we check for the appstream ref after
pulling it and before deploying it. When the system helper is not being
used we properly include the remote name in the refspec passed to
ostree_repo_resolve_rev(), so this commit does the same for the system
helper case. Otherwise we might end up with the checksum for an
appstream ref from another remote because of the relationship between
the child and parent repos. This doesn't have any user visible effect
except preventing a "false positive" of thinking an appstream pull was
successful when it wasn't.
2018-04-30 14:33:09 +02:00
Matthew Leeds
6e4d686f8e dir: Don't hide appstream update error messages
Currently we try to update the "appstream2" branch and if that fails try
to update the "appstream" branch. If that fails too we return the error
message from the appstream2 update, which can be misleading. So this
commit combines both error messages into one so you get something like
this:

$ flatpak update --appstream tamaulipas-apps
error: Error updating appstream2: No such ref 'appstream2/x86_64' in
remote tamaulipas-apps; Error updating appstream: Update is older than
current version

That should make debugging easier for
https://github.com/flatpak/flatpak/issues/1615.
2018-04-30 14:33:09 +02:00
Matthew Leeds
460fb8e267 dir: Fix typo in error message 2018-04-26 14:02:52 -07:00
Sam Spilsbury
70bb192204 dir: Don't clobber out_matches_prefix_func
Closes: #1608
Approved by: mwleeds
2018-04-24 18:22:24 +00:00
Matthew Leeds
9dc880f70e dir: Fix return value in get_permissible_prefixes() 2018-04-24 11:18:09 -07:00
Sam Spilsbury
099278a88a dir: Start rewrite exports source_path from "", not "export"
We start exploring the directory tree relative to export/, so
we don't want to include that component.

Closes: #1602
Approved by: alexlarsson
2018-04-23 12:10:14 +00:00
Alexander Larsson
4da56c8c62 RemoteState: Drop cancallable from flatpak_remote_state_lookup_cache()
This is a no-i/o operation on immutable data, there is no need for
a cancellable.

Closes: #1594
Approved by: alexlarsson
2018-04-20 11:16:57 +00:00
Alexander Larsson
7a61658012 RemoteState: Don't unnecessary strdup the metadata
This makes flatpak_remote_state_lookup_cache() return the metadata
from the actual FlatpakRemoteState rather than duplicating it.
All callers are updated to not free anymore and to strdup if needed.

Closes: #1594
Approved by: alexlarsson
2018-04-20 11:16:57 +00:00
Alexander Larsson
30bfc901d5 appstream: Allow system-helper update for local non-gpg remotes
This is safe as we do the pull locally, and root configured the local
path. We already do the same for regular installs, so its weird that
its not done for appstream.

Should fix https://github.com/flatpak/flatpak/issues/1580

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
ef80dd9947 appstream: Update appstream2 branch by default
By default we use the new appstream2 branch if it exists in the remote,
also in this case we compress the xml when deploying to be backwards
compat with the old deploys.

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Alexander Larsson
096f7d4fb5 appstream: Drop flatpak_dir_check_for_appstream_update
This early bailout is not really needed, because noop updates is
pretty fast. Also, doing that breaks the timestamp updates.

Closes: #1585
Approved by: alexlarsson
2018-04-20 10:10:33 +00:00
Matthew Leeds
9afca51507 remote-ls: Allow a URI instead of remote name
This fixes the ability of the remote-ls command to take a file:// URI
instead of a remote name, which is especially useful for repos on USB
drives (created via `ostree create-usb`) which are temporary and don't
warrant being added to the repo config. This commit also updates
relevant documentation, adds a unit test, and updates a few variable
names to improve readability.

I can't find a commit in the history where this was working, but it's
working on the Endless fork of flatpak so I think there was agreement at
some point that it's desired behavior.

Fixes https://github.com/flatpak/flatpak/issues/1588

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Matthew Leeds
79907f4236 dir: Fetch ostree-metadata independently of summary
Currently _flatpak_dir_get_remote_state() only fetches the
ostree-metadata ref if it was able to fetch the remote summary, but this
is unnecessary because we don't need to know the checksum just to fetch
it, and this is especially problematic in the offline use case when the
remote summary can't be fetched. So this commit makes flatpak fetch
ostree-metadata even if the summary fetch failed, which is consistent
with how things worked before commit fedb0e5bd.

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Joaquim Rocha
516e249e50 lib/installation: Use "collection refs" when listing refs
When listing refs installed or from a remote, only the refs matching the
main collection-id were being returned. However, it is very important to
have access to all refs, independently from their collection-id,
especially when trying to list remotes coming from a USB repository.

These changes add the mentioned refs and update the places that use
this list, both in the lib and in the CLI.

For the implementation to become easier, we introduce also a
FlatpakCollectionRef, that should be replaced by OstreeCollectionRef
once the latter becomes part of the general API (currently it is in the
experimental one).

Closes: #1587
Approved by: mwleeds
2018-04-19 20:59:34 +00:00
Sam Spilsbury
b8cfe2e7a3 dir: Export d-bus .services files that we are allowed to own
Closes: #1589
Approved by: alexlarsson
2018-04-19 07:31:25 +00:00
Matthew Leeds
ac690c4a1e Merge pull request #1590 from alexlarsson/batch-prunes
Batch prunes of the local repo
2018-04-18 23:48:13 -07:00
Matthew Leeds
2ec5f25599 dir: Fix grammar in a debug statement 2018-04-18 23:41:49 -07:00
Matthew Leeds
63052a51d6 dir: Update outdated comment 2018-04-18 16:16:18 -07:00
Sam Spilsbury
ceeda9b16d run: Add --no-documents-portal option
This prevents flatpak from trying to talk to the documents portal
on startup.

Closes: #1584
Approved by: alexlarsson
2018-04-18 12:05:16 +00:00
Alexander Larsson
1355713db7 Batch prunes of the local repo
This moves the prune call out of flatpak_dir_update() and
flatpak_dir_uninstall() and instead does this manually at all call
sites. The advantage is that we now only call it *once* even if you
uninstall or update multiple apps.

This means update everything is much faster as we don't have to scan
over the entire local repo for each updated app.
2018-04-18 12:00:12 +02:00
Sam Spilsbury
efa183e3d6 dir: Also handle tagged extension names in find_remote_related and find_local_related
Closes: #1531
Approved by: alexlarsson
2018-04-18 07:43:56 +00:00
Matthew Leeds
14dde8a37d dir: Use child_repo_ensure_summary() more
This introduces no functional change; it just makes use of a helper
function and updates a comment.
2018-04-17 11:46:02 -07:00
Matthew Leeds
1e1ed6911e dir: Tweak an outdated comment 2018-04-17 11:31:41 -07:00
Sam Spilsbury
66e343fcb8 dir: Add support for X-Flatpak-RunOptions in exported desktop files
A desktop file can specify this key to add additional arguments
to flatpak run. Right now, only 'no-a11y-bus' is supported.

Closes: #1579
Approved by: alexlarsson
2018-04-17 08:50:56 +00:00
Alexander Larsson
4bf892a06f common: Rename FlatpakRemoteState->remote to remote_name
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
482b494d9f Fix unused function warning in non-p2p case
_flatpak_dir_fetch_remote_state_metadata_branch is only used when building
with p2p, so #ifdef the entire function.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
9646f42ba8 dir: Remove unused function flatpak_dir_lookup_ref_from_summary
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
88edfda861 dir: Pass FlatpakRemoteState to flatpak_dir_pull/install/update
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
0e5baf3837 dir: Remove unused functions
flatpak_dir_get_remote_metadata and flatpak_dir_lookup_repo_metadata are now
not used anymore.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
48796fb29d dir: Pass FlatpakRemoteState to flatpak_dir_list_remote_refs
Reusing the summary and metadata here helps us a lot as typically we
often want to look up the cache data again for every ref in the list.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
2db82391c0 dir: Use FlatpakRemoteState for flatpak_dir_check_for_update
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
1936d2daae dir: Pass FlatpakRemoteState to flatpak_dir_find_remote_related
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
d791625d77 transaction: Use FlatpakRemoteState to look up metadata
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
346dcc04c1 dir: Use FlatpakRemoteState in flatpak_dir_remote_list_refs
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
78de3ba384 dir: Use FlatpakRemoteState in flatpak_dir_fetch_remote_commit
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
76bf02cb74 dir: Use FlatpakRemoteState in flatpak_dir_remote_has_ref
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
38df7f823b dir: Use FlatpakRemoteState in flatpak_dir_find_latest_rev()
This also makes this function static, as its not needed anywhere else.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
4b73e7fc2e dir: Use FlatpakRemoteState in flatpak_dir_find_remote_related()
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
f16f8b1d8b dir: Change update_remote_configuration to use RemoteState
This means we only look up the summary once for the entire operation.

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
245c605828 dir: Add FlatpakRemoteState object
This is an object that contains the state of a particular remote
at a point in time, such as the summary and the metadata. Storing
this in an object means we can reuse these data instead of constantly
looking them up.

This is more efficient, but also avoids possible inconsistencies if
the state changes during an operation.

There is also an "optional" mode for the state where we don't fail in
the p2p/collection-id case if it fails to load the summary or metadata.
This is useful in order to support certain operations "offline" (i.e.
with no network connection to the main repo).

Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00
Alexander Larsson
60270000c8 dir: Remove unused methods fetch_remote_default_branch and fetch_remote_title
Closes: #1575
Approved by: alexlarsson
2018-04-16 15:44:06 +00:00