In https://github.com/flatpak/flatpak/issues/4535 it was brough to light
that the search command strips the ".desktop" suffix from app IDs even
if they genuinely end in .desktop. This commit fixes the reverse issue
now that we've ported to libappstream: don't include the .desktop suffix
even if it's present in the id of the appstream component, which it
sometimes is.
Fortunately we already ignore any components that don't have flatpak
<bundle> elements, so we can unambiguously get the ID there.
Fixes#4535
Older versions of libappstream don't provide AS_CHECK_VERSION() so we
can't use that, and we need the check because as_component_get_branch()
is only in libappstream >= 0.14.0, and the return value of
as_metadata_parse_file() changed in 0.14.0
Since the port from libappstream-glib to libappstream, these wrappers
are trivially thin and can be dropped.
This commit introduces no functional changes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
libappstream-glib is mostly unmaintained, and libappstream is more
actively developed (and up to date with the AppStream specification).
Port from libappstream-glib to libappstream. Handily, a lot of the APIs
are exactly the same. The main changes are:
* `AsApp` → `AsComponent`
* `AsStore` → `AsMetadata`
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
(Mostly done by Philip, then Phaedrus finished this)
It is common for the build directory to be within a Git checkout.
Developers never want to check this directory into Git.
Make 'flatpak build-init', and hence 'flatpak-builder', create a blanket
.gitignore file inside this directory, so the containing checkout
doesn't need to predict the name of the build directory and ignore it.
(Inspired by Meson's treatment of its build directory.)
It is already the case that when we are using ALL_DIRS, we always
combine it with OPTIONAL_REPO, meaning no need to populate empty
installations. ALL_DIRS is used for commands that iterate through all
known installations to enumerate apps/runtimes, such as `flatpak run`
and `flatpak list`; for these commands, it's reasonable to say that
if the installation does not have a libostree repository, then that's
equivalent to it having a libostree repository with no apps and no
runtimes. Make this happen automatically if forgotten.
For STANDARD_DIRS, we were inconsistent about this: `flatpak remote-list`
had OPTIONAL_REPO, but the other commands did not.
STANDARD_DIRS is used for `flatpak create-usb`, and for all the commands
that manipulate remotes.
For the commands that manipulate remotes, it seems reasonable to say
that if an installation has no libostree repository and we are unable
to create one, then that's equivalent to an installation with a
libostree repository but no remotes.
Similarly, for create-usb, an installation where we are unable to create
a libostree repository seems like it should be equivalent to an
installation whose libostree repository does not contain any of the
refs we are interested in.
Resolves: https://github.com/flatpak/flatpak/issues/4111
Signed-off-by: Simon McVittie <smcv@collabora.com>
Commits that are found to have missing or invalid objects need to be
marked partial so that when the thing referencing them is reinstalled,
the missing objects will be pulled. libostree treats non-partial commits
as complete even if they're not, since verifying their completeness is
an expensive operation.
This exactly mirrors what the "ostree fsck" command does when it finds
corruption in a commit.
This fix is especially important because corrupt repos have been an
issue lately so we at least need repair to work properly.
Relatedly, delete invalid commit objects to ensure they are
re-downloaded, though it's not clear that code path is almost ever
reachable.
Fixes https://github.com/flatpak/flatpak/issues/4618
There's no need to use polkit in the history command, so don't start the
agent in flatpak-main.c. This means we can avoid a test failure in
test-history.sh which was caused by old versions of valgrind being
unaware of syscall sched_getattr, which is used in g_bus_get_sync(),
itself called by install_polkit_agent().
Currently we include entries in the output of the history command for
pulls of appstream refs, e.g. "appstream2/x86_64". However since they
don't have an application ID the Application column shows up blank and
it seems like a pull of nothing which is confusing. These are basically
an implementation detail like the temp repo pulls we already exclude, so
I think it makes sense to exclude them from the output.
It would also make sense to exclude pulls of ostree-metadata refs, but
for some reason I don't see those in practice, even with a collection ID
set on the remote.
The history command seems to have been broken since it was changed to
use FlatpakDecomposed, since that type only works for app or runtime
refs, resulting in errors such as:
$ flatpak history
error: appstream2/x86_64 is not application or runtime
Fix this by making the logic a bit smarter, and don't let any one
invalid ref entry prevent the whole command from working.
Fixes#4332
Have heard of people running Flatpak commands with both sudo and --user,
and not expecting it work on the root user's installation. Let's just
not allow it since it's not something people ever do intentionally.
The man page states that the --all option for the list command includes
runtime extensions (e.g. that end in .Locale or .Debug) and this is the
case. When --all is omitted, we only show such extensions when the thing
they extend is not already in the list. However when you run a command
like "flatpak list --runtime --columns=application,branch", you get a
list that excludes runtime locale extensions but includes app locale
extensions, since the corresponding apps are not in the list despite
presumably being installed. This doesn't seem like the right behavior,
so tweak the logic so that app locale extensions are excluded when --all
is omitted and --runtime is used.
Strictly speaking the "extension-of" information is not guaranteed to be
present in the deploy data (which acquires it from the metadata), but in
practice it seems to be present for all but Sources extensions, and I
think it should be considered required since we are already using it to
implement functionality elsewhere in Flatpak:
https://github.com/flatpak/flatpak/issues/4585
This fixes a few issues with the unit test logs that make them ugly:
1. Currently some lines of output from a command will appear after the
line from xtrace which has the next command, since the command was
printing to stdout and xtrace uses stderr. E.g. "Installation complete."
will appear after "+ flatpak --user install -y ..." but it is from the
previous install command.
2. Lines of output have many spaces after them to pad them to the table
width but this is not needed for non-fancy output.
3. Lines of output are mixed with output from httpd since they don't end
with a newline character, e.g. "Installing… ▊
4%127.0.0.1 - - [16/Nov/2021 00:18:24] "GET /..."
This is similar to what is done for desktop files and allows
applications to use a flatpak's metainfo to retrieve e.g. required
input controllers or the supported screen sizes. Similar to what can be
done for non-flatpak'ed apps by looking at
/usr/share/{metainfo,appdata}.
Since flatpak moves the metadata from metainfo/ to appdata/ during build
we only need to export files from that single directory.
Signed-off-by: Guido Günther <agx@sigxcpu.org>
Currently if the user specifies a ref to install that has been renamed
via the end-of-life-rebased mechanism, for example "flatpak install
com.visualstudio.code.oss", Flatpak erroneously tries to install both
the old and new versions of the app. This happens because the code
handling end-of-life-rebase conditions is written assuming the rebased
app is being updated rather than installed for the first time.
Specifically, in FlatpakCliTransaction and FlatpakQuietTransaction, in
end_of_lifed_with_rebase(), we treat a failure of
flatpak_transaction_add_uninstall() as fatal and return FALSE from the
signal handler, which means that the install operation that triggered
the signal will not be skipped (see the docs for
FlatpakTransaction::end-of-lifed-with-rebase).
So, instead check for the FLATPAK_ERROR_NOT_INSTALLED error code and
ignore it, so that the installation of the old version of the renamed
app will be properly cancelled.
Fixes https://github.com/flatpak/flatpak/issues/3754
flatpak_fancy_output() determines whether the output we write is
intended for e.g. a tty or e.g. a log file, the latter being "not
fancy". Currently we write carriage return characters in the not fancy
case, but change that to new lines, which are interpreted correctly by
editors such as vim (whereas a carriage return shows up as ^M since it
is not the correct newline character on Unixy operating systems).
The <bundle> element in the appstream data unambiguously provides the
full four-part flatpak ref, so use it to determine the app ID. But fall
back to using the <id> element, since that is required to be present.
This commit changes the search command to properly output the app ID for
IDs that end in .desktop, e.g. to print org.telegram.desktop rather than
org.telegram.
Fixes https://github.com/flatpak/flatpak/issues/4535
The install command can search available remotes for a specified flatpak
when a remote wasn't specified. In case only one remote is configured,
or in case only one of the configured remotes matches the ref specified,
we currently prompt the user to confirm use of the remote anyway (unless
-y/--assumeyes was used). Skip this prompt even when -y/--assumeyes was
not used, since the remote to use will still effectively be confirmed
when the list of refs to be installed is presented for confirmation.
Fixes https://github.com/flatpak/flatpak/issues/4364
This allows callers to be checked for mismatches between format string
and arguments, and also means gcc can assume that the format string and
the arguments match up correctly when forwarding them to functions
like g_strdup_vprintf, removing the need to suppress -Wformat-nonliteral
warnings.
Signed-off-by: Simon McVittie <smcv@collabora.com>
In principle this could have been subject to a format string attack
via an argument containing %n, although in practice the code that uses
this format string is #if 0.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Currenly we only list arch in the list of things to install from a
transaction if not all the refs are the same arch. It makes more sense
to show the arch even if its unique if they are not the primary arch.
It turns out that we can't currently uninstall a ref from a
non-standard arch without specifying the arch even if there is no similar
ref installed for the main arch. (#4264)
The fundamental reason for this that `flatpak_dir_find_installed_ref(s)`
currently only returns refs with standard arches unless you explicitly
specify an arch.
This changes flatpak_dir_find_installed_refs() to always return
all the refs for all installed arches. This is generally what
we want anyway, except in the case of "flatpak run org.some.Platform" where
we don't want to prompt if there are multiple arches installed, so that
is manually changed.
This changes find_matching_ref() to look for refs in all arches, but
always prefer (without prompting) the default arch if that is installed.
This also matches what all current callers want.
Fixes#4264
By default we only download the main arch subsummary, so if you added
a ref for some other arch it failed to find the ref. This works with the
CLI, because it explicilty loads the subsummary when its trying to expand
the parial ref to the full ref. However apps using libflatpak don't do that
so they failed.
This is an optimized version of ostree_repo_prune() specialized for
archive mode repos. It is faster and uses less memory so that we can
prune larger repos (like flathub) in a realistic timeframe.
The primary reason it is faster is that it creates and uses a
`.commitmeta2` file for each commit, containing information about what
objects are reachable from that commit. This means incremental prunes
need only traverse over newly created commits.
Secondly, it uses the variant parser compiled accessors for the
various GVariants that are involved in the prune which is quite a bit
faster, especially if the repo is very large.
It also merges the scan-for-all-objects and prune-unreachable objects
phases, which means that we don't have to allocate a hashtable for
all the objects in the entire repo saving a lot of memory.
To save memory the hashtable of reachable objects, which can be quite
big on a big repo, points to a custom, very compact format for object
names.
Additionally it does the scanning for reachable objects twice, first
with a shared lock and then again (if anything changed) it with an
exclusive lock. This allows us to avoid using an exclusive lock during
the slowest part of the prune.
Unfortunately there are currently no public APIs for the ostree repo
locks. We really need to take an exclusive lock during the whole prune
or we parallel modifications (say a commit) might get their newly
written objects deleted. To work around this we have a minimal custom
implementation of an exclusive lock. Once the public API is available
we can start using that.
I created a repo with a lot of small commits to test this. It has 9M,
and pruning with depth=10 deletes 2M of them.
The original performance looks like:
Finding reachable objects: 287 seconds
Pruning unreachable: 69 seconds
Just using the pregenerated reachable data:
Finding reachable objects: 15 seconds
Pruning unreachable: 69 seconds
The final optimized prune (using pregenerated data):
Finding reachable objects: 12 seconds
Pruning unreachable: 51 seconds
The above are with the page caches cleaned, on a second run the performance
increase is even more noticeable.
As a comparison to the above, finding the reachable objects in the
actual flathub repo took 22 hours, but with the pregenerated reachable data
only 39 minutes.
scan-build complained that rest_argv_start could be used uninitialized,
because it can't see that rest_argc >= 2 implies that rest_argv_start
got initialized at the same time rest_argc was set. Make this easier
to understand.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This hasn't done anything useful since 0978826c: it just takes a
new ref to the installation, and then releases that ref without doing
anything with it. Detected by scan-build.
Signed-off-by: Simon McVittie <smcv@collabora.com>
