Commit Graph

6027 Commits

Author SHA1 Message Date
Phaedrus Leeds
097faa8411 uninstall: Note pinned runtimes in --unused output
When "flatpak uninstall --unused" is run, we don't remove unused
runtimes that are pinned. List them in the output so the user knows they
are being left installed.

This commit also adds new library API,
flatpak_installation_list_pinned_refs().
2020-08-31 16:29:03 +02:00
Phaedrus Leeds
7cd1990196 Automatically pin explicitly installed runtimes
If a runtime is installed explicitly rather than as a dependency, pin it
so it doesn't get automatically removed when unused runtimes are being
removed. We do this because the runtime might be installed for
development or other uses.

This commit also rearranges some code in the mask and pin commands, and
adds a unit test.
2020-08-31 16:29:03 +02:00
Phaedrus Leeds
2325b81401 Minor fixes for mask command 2020-08-31 16:29:03 +02:00
Matthew Leeds
d2d5397cc1 Add pin command to keep unused runtimes
As discussed here [1], we want a way to mark runtimes to be kept even
when they are unused by any apps and we are removing such runtimes.
Currently this is a command that can be run manually; a subsequent
commit will pin runtimes automatically if they are installed
independently of any app.

A unit test is included.

[1] https://github.com/flatpak/flatpak/issues/2639#issuecomment-662311756
2020-08-31 16:29:03 +02:00
Phaedrus Leeds
f24b1fdc1a testlibrary: Remove unused variable 2020-08-31 16:26:12 +02:00
Phaedrus Leeds
84a8c30e55 transaction: Remove an unused error parameter 2020-08-31 16:26:12 +02:00
Phaedrus Leeds
c6647c2b14 transaction: Fix ordering of runtime uninstallation
In case a transaction is uninstalling both an app and its runtime,
properly ensure the app is uninstalled first.
2020-08-31 16:26:12 +02:00
Phaedrus Leeds
e16574c8ee testlibrary: Add missing return value checks 2020-08-31 16:26:12 +02:00
Phaedrus Leeds
ebcad3a402 dir: Fix a typo of .changed 2020-08-31 16:26:12 +02:00
Phaedrus Leeds
3f19950936 doc/update: Fix formatting 2020-08-31 16:26:12 +02:00
Simon McVittie
27870f681d tests: Expand exports test coverage
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-28 11:07:24 +02:00
Simon McVittie
5a83c73ed8 context: Implement MODE_NONE in unparse_filesystem_flags
flatpak doesn't yet use -Wswitch-enum, but perhaps it should at some
point. Now that FLATPAK_FILESYSTEM_MODE_NONE is a member of the enum,
it should be handled; and if we're doing that, we might as well make
the same function fully responsible for it.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-28 11:00:49 +02:00
Simon McVittie
e55dcf5e2b tests: Exercise flatpak_context_save_metadata
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-28 11:00:49 +02:00
Simon McVittie
354b9a2257 tests: Use a temporary HOME directory to test contexts and exports
This gives us control over the paths that get shared (or not) and
whether they are symlinks, so that we can expand coverage later.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
09424423b9 context: Normalize home/path to ~/path, and ~ to home
Historically we didn't accept them, but there's no real reason why not.
They're normalized to the form in which earlier Flatpak releases would
want to see them.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
02094b4f39 context: Forbid --filesystem=/
Making it an equivalent of --filesystem=host would be misleading,
because it wouldn't do what you'd think it does: host mounts some host
system directories in their usual places, but others below /run/host.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
aafe1d36e0 context: Do some syntactic normalization on filesystems
Paths containing ".." are rejected: they're almost certainly a
terrible idea.

Paths containing "." or multiple slashes are syntactically normalized.

This assumes that nobody is going to use "--filesystem=/foo/bar/" to
mean "make /foo/bar available, unless it's a non-directory, in which
case fail".

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
c0faab35fa tests: Add basic unit tests for FlatpakExports, FlatpakContext
There's a limit to how many assertions we can make here right now,
because what we do here is very dependent on the "shape" of the host
filesystem. This could be extended in future by using a mock home
directory whose contents we control.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
55b27b1393 context: Expose flatpak_context_parse_filesystem for testing
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
115d82e6ff exports: Add assertions to distinguish between mode representations
When we're talking about a "mode", sometimes we mean a
FlatpakFilesystemMode, sometimes we mean a FlatpakFilesystemMode that
must be strictly greater than NONE, and sometimes we're willing to
accept the FAKE_MODE constants too.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
517ad25b5f context: Only parse filesystem/mode strings in one place
This gives us the ability for the parse function (the former verify
function) to carry out a normalization step as well.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 17:48:50 +02:00
Simon McVittie
3379c277be tests: Test container-manager interface
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 11:29:52 +02:00
Simon McVittie
414671a666 run: Tell processes in container that the container manager is Flatpak
https://systemd.io/CONTAINER_INTERFACE/ describes a generic way to tell
programs and libraries that they are running in a container: set
pid 1's ${container} to the name of the container manager in lower case,
and populate /run/host/container-manager with the same string followed
by a newline. Let's be nice to application code by doing that, instead
of requiring it to look at /.flatpak-info.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-27 11:29:52 +02:00
Phaedrus Leeds
082f50eefa Merge pull request #3809 from alexlarsson/valgrind-ci
CI: Add a valgrind check
2020-08-24 17:06:56 -07:00
Philip Withnall
78a2e45610 flatpak-dir: Pass cached summary{,.sig} to OSTree
This allows OSTree to avoid re-checking the `summary.sig` for freshness
once per pull in a transaction, since flatpak has already done that.
This avoids latency in a transaction (especially on high latency or
lossy network connections) and avoids the potential race condition of
the `summary` file changing on the server part-way through a
transaction.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Philip Withnall
95bef8e4a5 flatpak-dir: Cache the summary GBytes in FlatpakRemoteState
`FlatpakRemoteState` already caches the parsed `GVariant` form of the
summary, but it will be useful (in the following commits) to also have a
pointer to the `GBytes` which backs that variant. The `GBytes` will be
passed into OSTree in the following commits, to allow it to avoid
unnecessarily re-downloading the `summary` file.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Philip Withnall
d57db7074b flatpak-transaction: Don’t update metadata when uninstalling
When uninstalling a single app (an example of a transaction which is
entirely local-only), there’s no need to update the repository metadata
beforehand — but the code was doing that.

This avoids an unnecessary download of the `summary.sig` file (or other
repository metadata, depending on what’s out of date) when uninstalling.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Philip Withnall
254a885879 dir: Clarify a comment about caching
`flatpak-dir.c` supports two caches for `summary{,.sig}` files: one in
memory (per `FlatpakDir` instance) and one on disk (shared between all
users of the repository). Clarify in this comment which one is being
referred to.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Philip Withnall
98e372e85e dir: Tidy up definition of cache timeout
Adding brackets makes it a little more obvious that it’s 5 minutes, and
less likely that operator precedence where the macro is used will result
in the wrong value being calculated.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Philip Withnall
63b4f3a5c1 dir: Fix caching of summary files
Cache hits didn’t result in the cache entry’s timeout being extended,
and timeouts didn’t result in the entry being removed from the cache to
free up resources (even though it would never be returned as a cache hit
again).

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Philip Withnall
9bec865995 dir: Fix potential overflow when comparing cache timeouts
Also use a constant to clarify the code a little, although it’s
functionally equivalent.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-24 08:58:08 +02:00
Alexander Larsson
946cf126b7 CI: Add a valgrind check
This takes a long time, but doesn't have to be mandatory, also we
only run it if the regular make check passes.

Removes some old no-longer needed ci config workaround.

The new test uses ubuntu-20.04 so we get wider distro testing.
2020-08-24 08:55:14 +02:00
Alexander Larsson
07b917ca4e tests: Add ostree ioctl valgrind suprression
I was getting this in the CI:

--30631-- WARNING: unhandled amd64-linux syscall: 315
--30631-- You may be able to write your own handler.
--30631-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--30631-- Nevertheless we consider this a bug.  Please report
--30631-- it at http://valgrind.org/support/bug_reports.html.
runtime/org.test.Platform/x86_64/stable: 13b73140218edd02a9d18bc178af1a3ad0203049f9f1ad8c51c62b3ee5f1acd9
==30631== Syscall param ioctl(generic) points to unaddressable byte(s)
==30631==    at 0x53A437B: ioctl (syscall-template.S:78)
==30631==    by 0x4B4456E: ??? (in /usr/lib/x86_64-linux-gnu/libostree-1.so.1.0.0)
==30631==    by 0x4B466FB: ??? (in /usr/lib/x86_64-linux-gnu/libostree-1.so.1.0.0)
==30631==    by 0x4B48F29: ostree_repo_write_content (in /usr/lib/x86_64-linux-gnu/libostree-1.so.1.0.0)
==30631==    by 0x1D9161: flatpak_mtree_create_symlink (in /home/runner/work/flatpak/flatpak/_build/flatpak)
==30631==    by 0x1DF95B: flatpak_repo_generate_appstream (in /home/runner/work/flatpak/flatpak/_build/flatpak)
==30631==    by 0x157870: flatpak_builtin_build_update_repo (in /home/runner/work/flatpak/flatpak/_build/flatpak)
==30631==    by 0x135793: main (in /home/runner/work/flatpak/flatpak/_build/flatpak)
==30631==  Address 0xe is not stack'd, malloc'd or (recently) free'd
==30631==
{
   <insert_a_suppression_name_here>
   Memcheck:Param
   ioctl(generic)
   fun:ioctl
   obj:/usr/lib/x86_64-linux-gnu/libostree-1.so.1.0.0
   obj:/usr/lib/x86_64-linux-gnu/libostree-1.so.1.0.0
   fun:ostree_repo_write_content
   fun:flatpak_mtree_create_symlink
   fun:flatpak_repo_generate_appstream
   fun:flatpak_builtin_build_update_repo
   fun:main
}
2020-08-24 08:53:31 +02:00
Alexander Larsson
333c746f06 Revert "CI: Run make check in parallel (-j NCPU)"
This reverts commit 4401862ac5.
2020-08-24 08:53:12 +02:00
Alexander Larsson
4401862ac5 CI: Run make check in parallel (-j NCPU) 2020-08-21 17:15:38 +02:00
Phaedrus Leeds
332f75494b Properly validate configured collection IDs
This commit is a follow-up for the commit "dir: Validate locally
configured collection IDs". Whereas in that commit we validate the
collection ID in flatpak_remote_state_fetch_commit_object(), here we do
it in _flatpak_dir_get_remote_state(), since fetch_commit_object() is
not called on the code path normally taken for a transaction (only if
try_resolve_op_from_metadata() fails and in other niche scenarios).

This should ensure that if someone erroneously sets a collection ID on a
remote, the error will be caught quickly. It also helps the eos-updater
unit tests pass.
2020-08-21 09:35:49 +02:00
Alexander Larsson
09d57249f4 oci-auth: Don't ask for authentication if anon auth fails with weird error
If the initial anonymous fails for any other reason than "not authorized"
we immediately fail the operation instead of asking for user/password.
The later is creating a very bad UX in case of e.g. networking or
infrastructure issues, as described in #3753.
2020-08-20 16:44:56 +02:00
Alexander Larsson
83f11317e6 Allow multiarch in the apply_extra for extra-data
Some things could need to run e.g. i386 code in apply_extra (for
example #3742).  In this very limited context (almost everything is
read-only) this seems pretty secure.

We could require the app to specify a multiarch pemission to allow
this, but such permissions only really make sense for an app, and
extra data is often used for other things like runtimes and
extensions, that seems a bit weird.  Lets just enable it always.
2020-08-20 16:44:39 +02:00
Alexander Larsson
8c23c90c1e run: Fix double free in the cups config file parser
We were returning a g_autofree:d string and it was then also freed
by the parent.
2020-08-18 11:55:43 +02:00
Ryan Gonzalez
c95cb1973d run: Avoid hanging if xdg-dbus-proxy startup fails 2020-08-17 15:52:30 +02:00
Alexander Larsson
87a44588d6 dir: Cache responses from localed and account service
These are system dbus calls that could potentially be somewhat slow
and its unlikely that the local config changes during runtime of the
session (because the desktop really needs a restart to pick up a new
locale). Also, if there are any issues with these (such as #3755) each
call will be the 25sec dbus timeout, which is not great.

Fixes https://github.com/flatpak/flatpak/issues/3792
2020-08-17 14:27:02 +02:00
Phaedrus Leeds
86e150da85 tests: Use g_assert_true() not g_assert()
This way our test assertions can't be accidentally compiled out with
G_DISABLE_ASSERT.
2020-08-17 09:21:30 +02:00
Matthew Leeds
1aeceb0caf doc: Clarify remote no-enumerate docs 2020-08-17 09:21:30 +02:00
Matthew Leeds
845a877ce1 Prioritize an app's origin for its runtime
Currently when searching for a remote to provide the runtime for an app,
we search remotes in priority order. This commit makes it so we search
the remote providing the app before others with the same priority, and
otherwise still search in priority order.  This means for the common
case where every remote has the default priority of 1, the app's origin
will have the first chance to provide the runtime. This behavior seems
logical, but the impetus for this change was also to keep a unit test
passing in eos-updater[1] after a port to FlatpakTransaction.

Originally the eos-updater unit test was written to prioritize the
origin remote regardless of the priorities on any other remote, but
during code review it was decided to let higher priority remotes stay
above the app's origin.

In practice it's usually true that only one remote provides a runtime
and priorities aren't set at all, so this is an edge case that probably
doesn't come up much.

A unit test and documentation updates are included.

[1] eede0a8b9c/tests/test-update-install-flatpaks.c (L1919)
2020-08-17 09:21:30 +02:00
Alexander Larsson
676d1e0899 Update variant-schema-compiler module
There was an off-by-one bounds check error
2020-08-13 13:29:39 +02:00
Phaedrus Leeds
f1383c10be sideload-repos-systemd: Dist service unconditionally
Add flatpak-sideload-usb-repo.service.in to EXTRA_DIST regardless of if
the --enable-auto-sideloading configure option was passed. This allows
building a tarball without that option and then building from the
tarball with the option.

This matches what is done in system-helper/Makefile.am.inc with
flatpak-system-helper.service.in.

Picked from https://github.com/endlessm/flatpak/pull/228
2020-08-12 10:07:58 +02:00
Simon McVittie
12e3dc051f exports: Try harder to export usrmerge symlinks as symlinks
In applications with --filesystem=host or --filesystem=host-os, the
library directories in /run/host/usr and /run/host/lib* can be used
like a sysroot to inspect the host's library stack, regardless of
whether the host system has undergone the "/usr merge" or not. This is
particularly relevant for Steam's pressure-vessel container tool,
which imports graphics drivers from the host system, or for potential
future work on using host graphics drivers via libcapsule in Flatpak
apps.

The original implementation of this feature assumed that the /usr merge
always creates symbolic links /foo -> /usr/foo or /foo -> usr/foo, for
some value of foo. However, Arch Linux uses a variation of multilib
in which /usr/lib contains 64-bit libraries, and has a non-matching
symbolic link /lib64 -> usr/lib (instead of a chain of symlinks
/lib64 -> usr/lib64 -> lib). Similarly, Arch uses /sbin -> usr/bin
and /usr/sbin -> bin, instead of a chain of symlinks
/sbin -> usr/sbin -> bin.

This led to Flatpak ignoring the symlink and treating it like a directory,
so the host /usr/lib would be bind-mounted on both /run/host/usr/lib and
/run/host/lib64. This is *mostly* equivalent, but whether a directory
is a symlink or a real directory sometimes matters, for example when
evaluating relative symlinks that contain ".."  components; so try harder
to preserve symlinks as symlinks.

Fixes: 08d65c54 "exports: If --filesystem=host, provide /run/host/lib etc."
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-11 15:48:05 +02:00
Phaedrus Leeds
2612e9df18 doc: Update create-usb for sideloading
Update the create-usb man page to reflect the re-worked implementation
that landed in 1.7.1.
2020-08-11 15:22:57 +02:00
Matthew Leeds
73644b00c9 system-helper: Support setting 'masked' config key
This fixes the following error when the "flatpak mask" command is used
with the system-helper: "error: Unsupported key: masked"
2020-08-10 15:25:55 +02:00
Matthew Leeds
5192e94be2 tests: Test update portal in system installation too
The "flatpak mask" command needs to work for system installations, and
currently doesn't.
2020-08-10 15:25:55 +02:00