Commit Graph

428 Commits

Author SHA1 Message Date
Phaedrus Leeds
4f6c13e751 Merge pull request #4327 from ericcurtin/warn_unused_result
Fix some warn_unused_result warnings
2021-10-12 14:31:43 -07:00
Eric Curtin
c6845ad70b Fix some warn_unused_result warnings
While building on gcc 9.3.0
2021-10-12 13:50:14 -07:00
Simon McVittie
ae59fb7aca common: Backport g_get_language_names_with_category() more thoroughly
Even though we are going to check all the `LC_*` environment variables,
if one of them is not set, we still need to consider LANGUAGE, LC_ALL
and LANG; there is no guarantee that *every* `LC_*` environment variable
is set (and in particular, during our build-time tests, they will usually
not be).

Resolves: https://github.com/flatpak/flatpak/issues/4471
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-10-11 09:31:58 +02:00
Ryan Gonzalez
f15f926284 Retrieve user languages for all locale categories
g_get_language_names() only returns the language names for the
LC_MESSAGES category, so mixed locale scenarios would result in missing
languages. Now, the languages are listed for each individual category.

Note that this issue was only present with the user installation. For
the system installation, the locales were queried from localed, and all
categories were checked.

In order to work on GLib versions < 2.58, the code to get language
names for a category has been backported.

Fixes #3712.
2021-09-17 09:23:52 +02:00
Simon McVittie
d13f489e63 Don't disable -Wformat-nonliteral unnecessarily
These functions were already annotated with G_GNUC_PRINTF, so gcc
can assume that the format string and arguments match up.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-05-25 10:33:18 +02:00
Phaedrus Leeds
404d7c6941 Fix several memory leaks 2021-05-04 10:23:13 +02:00
Phaedrus Leeds
4562627b04 utils: Add missing check for empty string collection id 2021-05-04 10:23:13 +02:00
Phaedrus Leeds
756b9eae14 common: Fix several memory leaks 2021-05-04 10:23:13 +02:00
Phaedrus Leeds
879c9b9008 utils: Fix use-after-free and logic error
Without this change, validate_component() might free the "component"
variable, but then go on to keep using it. This change also makes the
code work as intended, to only have a flatpak-specific "bundle" in the
appstream xml (where here bundle does not mean "flatpak single-file
bundle").
2021-05-04 10:23:13 +02:00
Simon McVittie
3787db00e0 utils: Add flatpak_str_is_integer()
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-04-15 18:00:47 +02:00
Simon McVittie
db7d9cfeb5 utils: Quote empty strings when debug-logging argv
If an argument takes a value, and the value is empty, then it's
misleading to quote `{"--foo", "--empty", "", "--bar"}` as
`--foo --empty  --bar`. It's better to get `--foo --empty '' --bar`.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-04-13 14:14:10 +02:00
Simon McVittie
494feacb77 run: Sort environment before serializing it into bwrap arguments
This has no practical effect (assuming environment variables are unique),
but it makes it easier to find an environment variable of interest
in a very long bwrap command-line.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-04-06 16:19:09 +02:00
Simon McVittie
b13c8a76d2 utils: Display bundled bwrap arguments one per line
This makes them easier to deal with when debugging. Otherwise, it's easy
for the bundled arguments to wrap across 50 or more lines, and with
linebreaks in arbitrary positions that becomes very hard to read.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-04-06 16:19:09 +02:00
Alexander Larsson
f479c0f1a4 update-repo: Create an digested version of the summary index signature
In addition to the old summary.idx.sig we now create an identical file
in `summaries/$DIGEST.idx.sig`, where the digest is of the summary
index itself. This means it will have a unique name and there will
never be issues where downloading the index and signature will race
during an update.

We keep the new and the previous digested sigs every update to avoid
removing the previous one while its in use.

As suggested by simon in https://github.com/flatpak/flatpak/issues/3983
and https://github.com/ostreedev/ostree/issues/2250
2020-12-22 13:49:59 +01:00
Phaedrus Leeds
9ab02b38bc Ensure deploy files have complete metadata
Currently if a deploy file was created (because a flatpak is installed)
using an old Flatpak version (such as 0.10.1 that's in Endless 3.3.16),
it will not include every metadata key, such as "eol" and "eolr". Those
keys were added before deploy version 1. Then when the deploy file is
upgraded to version 3, the metadata is only copied so eol/eolr are still
missing even though they exist in the commit metadata.

So this commit adds deploy version 4 which ensures all metadata keys in
the deploy file are present (when they are present in the source data).
All existing deploy files will be upgraded with this code.

This change is important because we use the end-of-life information to
decide whether an unused runtime should be uninstalled, so without this
change we are not freeing up disk space that we should be.
2020-12-21 10:30:22 +01:00
Alexander Larsson
b3321e9179 extra-data: Resolve runtime from the target installation
When installing to an installation we need to find the runtime to use
for the apply-extra-data script from the installation we're targeting,
because that is the one that FlatpakTransaction guaranteed has the
required dependencies (although its possible they came from the
default dependency source too, i.e. the system repos).

In particular, we run into this issue if nothing is installed
anywhere, and then we install an extra-data app into a custom
directory. The transaction will download the runtime, and it
will not be anywhere else. Without this change flatpak only
looked for the dependency in the systam an regular user dirs
where it isn't.
2020-12-09 16:12:25 +01:00
Alexander Larsson
a03c2739df Add back the ostree.commit.timestamp per-ref metadata for summary
The old summary had a ostree.commit.timestamp in the per-ref metadata
dict. However, since this was not used anymore by flatpak I removed it.
However, it turns out that flathub has infra that depends on this,
so I'm adding it back.

We reuse the data in the old summary for unchanged refs when
rebuilding the summary, to avoid having to read all the commits. In
the new world the new format summaries are used for this, which means
we have to keep the timestamp in that too. However, to not be
unnecessary large its now using a shorter key name, as this is
duplicated for each ref in the summary.
2020-11-20 15:20:15 +01:00
Alexander Larsson
f650c303e3 Don't auto-install sources for extensions
There was a bug in the extension point matcher which made it
install `org.gnome.Totem.Videosite.YouTubeDl.Sources` (in addition to
`org.gnome.Totem.Videosite.YouTubeDl`) for the `org.gnome.Totem.Videosite`
extension.

We just need to make sure we only match the extension prefix if there
is a single element in the extension name following the extension
name (i.e. '.YouTubeDl', not '.YouTubeDl.Sources').

This fixes https://github.com/flatpak/flatpak/issues/3973
2020-11-18 17:36:10 +01:00
Alexander Larsson
754e1fb5d4 Allow configuring the subset
This adds support to remote-add, remote-modify and the flatpakrepo file
parser.
2020-11-13 10:32:19 +01:00
Alexander Larsson
3c60facb03 summary: Handle subset appstream branches
In the main summaries we drop all the subset appstream branches.
In the subset summaries we use the relevant appstream branch and drop
the normal ones.

We also leave out the old compat branch completely from the subset
summaries because it will just be used for newer flatpaks that support
the new format.
2020-11-13 10:32:19 +01:00
Alexander Larsson
4739fc147c Generate appstream branches for subsets
For all the subsets in the repo, create an appstream2/$subset-$arch branch
with the appstreams of the apps in that subset.
2020-11-13 10:32:19 +01:00
Alexander Larsson
9a7b33e4ed summary: Correctly extract the subset
We want to split $subset-$arch on the last dash, not the first,
because arch can't have dashes, while subset could.
2020-11-13 10:32:19 +01:00
Alexander Larsson
3d29c6ee5c decomposed: Convert flatpak_load_deploy_data() to FlatpakDecomposed 2020-11-10 14:32:13 +01:00
Alexander Larsson
f7593cd7b6 appstream: Use FlatpakDecomposed when generating 2020-11-10 14:32:13 +01:00
Alexander Larsson
c5b2c60609 decompose: Convert more FlatpakDir functions to use FlatpakDecomposed 2020-11-10 14:32:13 +01:00
Alexander Larsson
b6a1eb8d48 Delete some unused variables 2020-11-10 14:32:13 +01:00
Alexander Larsson
a988ee19d8 decompose: Use FlatpakDecomposed for bundles 2020-11-10 14:32:13 +01:00
Alexander Larsson
c14486f935 Use decomposed in flatpak_summary_match_subrefs() 2020-11-10 14:32:13 +01:00
Alexander Larsson
109cac60f6 decomposed: Store a FlatpakDecomposed in FlatpakRelated 2020-11-10 14:32:13 +01:00
Alexander Larsson
6897b8e2ff decomposed: Take decomposed arg in flatpak_dir_load_deployed()
Then convert all callers
2020-11-10 14:32:13 +01:00
Alexander Larsson
fa1a494566 decomposed: Convert various "current ref" functions to use FlatpakDecomposed 2020-11-10 14:32:13 +01:00
Alexander Larsson
5dae1fc6bc Break out ref helper functions to separate file
flatpak-utils.c is getting large, and also I got some issues with
include order, so split out things that are purely handling ref stings
in various ways to its own file.
2020-11-05 10:43:10 +01:00
Alexander Larsson
ed192e1044 remote-ls: Add a smarter fallback for names if appstream is missing
Rather than just taking the last element of the name we also
look at known extensions like .Platform, .Sdk, .Debug, .Locale, etc.
2020-11-05 10:43:10 +01:00
Alexander Larsson
7465ba0f18 Various FlatpakDecomposed API updates
getters now have three forms:
  const char *foo_peek() returns pointer into data and length
  const char* foo_get() returns zero terminated pointer into data
  char *foo_dup() returns allocated zero terminated pointer

Not everything has all of these (as that is not always possible), but
it is clear what each one does.

We also now support storing a collection id for a
FlatpakDecomposed. This is not typically done (as we use the source
remote to define the collecion id and don't expose this concept to the
user). However, for the case when directly listing a local repo (such as
a side-load repo) we need this.

We now validate the arch string. The valid chars are a-z, A-Z, 0-9 and
_. This was't previously verified, but no arches in existance have other
chars.

There is a new flatpak_decomposed_new_from_decomposed() which lets you
modify part of another ref. Say replace the arch or the branch and
keep the rest the same.
2020-11-05 10:43:10 +01:00
Alexander Larsson
3522d183bc summaries: When regenerating summaries, ensure they get newer mtimes
The st_mtime field is all we can guarantee to keep, so whenever we
write a new summary file for a repo we ensure it is newer than the old
by checking if st_mtime is equal (or less) and if so, use the old
st_mtime plus one.

This means sometimes the generated summary files will be in the future but
that only happens in the corner case of multiple changes inside one
second, and in that particular case we need this change.

This fixes some issues I had with the mtime chacking in the tests due to
the python httpd handling If-Modified-Since based on st_mtime.
2020-11-03 12:57:37 +01:00
Alexander Larsson
472939f538 utils: Add FlatpakDecomposed helper
This is a ref-counted version of a ref string that is guaranteed
to be in the proper form, and is pre-decomposed for efficient partial
matches.
2020-10-29 15:30:35 +01:00
Alexander Larsson
7483a98769 utils: Allow passing sizes to more utils
These now take (optional) string length args:
  flatpak_id_has_subref_suffix()
  flatpak_levenshtein_distance()
  flatpak_is_valid_name()
  flatpak_is_valid_branch()
2020-10-29 15:30:35 +01:00
Alexander Larsson
5b52b64cd1 Export flatpak_repo_load_summary_index() and flatpak_repo_load_digested_summary() 2020-10-29 15:30:35 +01:00
Alexander Larsson
98e20c18f9 summaries: Support using summary deltas on client
Also prune old unreferenced subsummaries on each transaction.
2020-10-29 15:30:35 +01:00
Alexander Larsson
2c6fec556f summaries: Generate summary deltas when updating repo 2020-10-29 15:30:35 +01:00
Alexander Larsson
0d2cf085af Add option to disable generating summary index
This is mainly useful for the tests
2020-10-29 15:30:35 +01:00
Alexander Larsson
0597f246c8 build-update-repo: Generate new summary format
In addition to the old summary file we create a summary.idx and
a set of per-arch subsummaries (and extra copies of these if any commit
specify a subset). These are much smaller, and eventually we will also
get deltas for them.

We are not yet using these new formats, although the code is there
to use them once we start downloading them.
2020-10-29 15:30:35 +01:00
Alexander Larsson
096daf91f2 summary: Implement a new, more efficient, summary format
This drops the deltas from the summary and uses the per-commit
metadata field to add the cache data to avoid the need for the separate
xa.cache and xa.sparse-cache indexes. This way we avoid repeating the
refs in multiple places.

Nothing uses this format yet, but we still pass make check if we enable
it.
2020-10-29 15:30:35 +01:00
Alexander Larsson
b094585545 utils: Expose get_compat_arch_reverse() and get_arch_for_ref()
We need to use these from some other files too.
2020-10-29 15:30:35 +01:00
Alexander Larsson
c5d76e5a85 summary: Extract the summary generation code to helper 2020-10-29 15:30:35 +01:00
Alexander Larsson
b7e90d3809 summaries: Add support for limiting which arches end up in summary
This is in preparation for adding other formats for summary and we
might not want all the arches in the fallback summary format.
2020-10-29 15:30:35 +01:00
Alexander Larsson
f84b48ac58 utils: Reimplement summary generation
This moves the generation of the summary files completely into flatpak
allowing us to (later) customise what goes into it in more detail and
generate other forms of summaries.
2020-10-29 15:30:35 +01:00
Kalev Lember
34bb5592c9 common: Allow skewered to camelcase DConf path conversion both ways
commit 6b46d9a0ed that added DConf path
skewering to camelcase conversion only allowed it in one direction
(skewered path1 and camelcase path2).

That turned out to be not enough to allow /org/gnome/sound-juicer/ to
/org/gnome/SoundJuicer/ conversion as the caller had the
flatpak_dconf_path_is_similar() arguments the other way around.

This commit implements it both ways to avoid confusion which way it
should be called.

F: Ignoring D-Conf migrate-path setting /org/gnome/sound-juicer/
2020-10-01 11:15:54 +02:00
Bastien Nocera
6b46d9a0ed common: Allow skewering when converting in app-id for DConf migration
Allow a snake-case in the app-id to convert to a '-' or '_' in the
DConf path to be considered similar enough for DConf migration purposes.

This allows the org.gnome.SoundJuicer app-id to migrate its
/org/gnome/sound-juicer DConf path.

F: Ignoring D-Conf migrate-path setting /org/gnome/sound-juicer/
2020-09-15 08:59:53 +02:00
Bastien Nocera
c6802d2c0e common: Prepare DConf path parsing for changes
Separate the variables used to keep track of which character we were
checking in each of the paths.

No functional changes.
2020-09-15 08:59:53 +02:00