Commit Graph
6156 Commits
Author SHA1 Message Date
Yuri Chornoivan df0362f012 Update Ukrainian translation 2020-11-02 08:54:45 +01:00
Phaedrus Leeds 08f3f1efb7 Merge pull request #3911 from flatpak/txn-auth-docs
transaction: Fix docs around op authentication
2020-10-30 10:25:07 -07:00
Phaedrus Leeds 08e55a36aa transaction: Fix docs around op authentication 2020-10-29 20:26:08 -07:00
Alexander Larsson a0dd41d8c4 docs: Add initial document describing the flatpak summary files. 2020-10-29 17:19:47 +01:00
Alexander Larsson 7b5a0a5c86 Fix extra-data progress reporting
We were never calling the progress callback for extra-data downloaded
bytes.
2020-10-29 17:19:38 +01:00
Mejans 2c22604f9b Adds Occitan 2020-10-29 16:12:08 +01:00
Mejans fa8f3735eb First commit for Occitan language 2020-10-29 16:12:08 +01:00
Phaedrus Leeds 835d2c78e6 testlibrary: Tweak one of the tests to avoid CLI
It seems cleaner to use library API than the CLI here.
2020-10-29 16:08:39 +01:00
Alexander Larsson 308e30c8f0 transaction: Remove transfer annotation for boolean
This doesn't do anything (and was causing warnings).
2020-10-29 16:08:12 +01:00
Alexander Larsson c04df80ee1 Drop some spew from flatpak_dir_list_unused_refs()
This was showing up in every flatpak update, making it hard
to see the important stuff.
2020-10-29 16:08:12 +01:00
Phaedrus Leeds ea7b262049 CI: Ignore some paths that don't affect build
No need to waste resources running every test after the README
changes.
2020-10-29 16:07:56 +01:00
Alexander Larsson b8db46d0e2 tests: Add some tests for subsummaries 2020-10-29 15:30:35 +01:00
Alexander Larsson e6d7a1148e summaries: Cache subsummaries per-arch
We store the cached subsummaries as `$remote-$arch-digest.sub`, this
way we can prune everything but the most recent (i.e. highest mtime)
rather than looking at the index. This way we don't prune old
subsummaries for other arches when we update one arch, which would not
let us use deltas for those.

Some details:
 * If mtimes are identical (mtime precision issue) for several
   subsummaries we keep all.
 * If we just saved a subsummary in the cache we don't prune that
   in case there is something wonky with mtimes.

The subsets for a remote are pruned every time we write a new cached
subsummary.
2020-10-29 15:30:35 +01:00
Alexander Larsson 97099bde37 CI: Build ostree from git (needed for new summary format) 2020-10-29 15:30:35 +01:00
Alexander Larsson a60e2cbe2d Remove some unused variables 2020-10-29 15:30:35 +01:00
Alexander Larsson 2c8fe5b94c Use the new FlatpakDecomposed when mathing refs
This avoids lots of re-splitting and re-validation as well as strdups.
2020-10-29 15:30:35 +01:00
Alexander Larsson 472939f538 utils: Add FlatpakDecomposed helper
This is a ref-counted version of a ref string that is guaranteed
to be in the proper form, and is pre-decomposed for efficient partial
matches.
2020-10-29 15:30:35 +01:00
Alexander Larsson 7483a98769 utils: Allow passing sizes to more utils
These now take (optional) string length args:
  flatpak_id_has_subref_suffix()
  flatpak_levenshtein_distance()
  flatpak_is_valid_name()
  flatpak_is_valid_branch()
2020-10-29 15:30:35 +01:00
Alexander Larsson f147df908a repo: Support indexed summaries in existing code and add new support
If indexes are available we now use those. Also we now allow listing
subsummaries and filtering by subset prefix.
2020-10-29 15:30:35 +01:00
Alexander Larsson 5b52b64cd1 Export flatpak_repo_load_summary_index() and flatpak_repo_load_digested_summary() 2020-10-29 15:30:35 +01:00
Alexander Larsson caa1c8c839 table printer: Add some helpers
Allow looking up pre-existing rows (by key) and appending to existing
cells.
2020-10-29 15:30:35 +01:00
Alexander Larsson 98e20c18f9 summaries: Support using summary deltas on client
Also prune old unreferenced subsummaries on each transaction.
2020-10-29 15:30:35 +01:00
Alexander Larsson 2c6fec556f summaries: Generate summary deltas when updating repo 2020-10-29 15:30:35 +01:00
Alexander Larsson 0221f5a1fa summaries: Ensure we can support non-default arches W/ indexed summaries
We need to load the ensure the right arch whenever we need it.

Also this restructures the RemoteState handling a bit in general so that
we avoid loading the same remote state multiple times when converting
partial refs to full refs.
2020-10-29 15:30:35 +01:00
Alexander Larsson dce6f748bb summaries: Rework the debug spew for summary/cache loading
This way its easier to tell exactly what happens.
2020-10-29 15:30:35 +01:00
Alexander Larsson 5821f79d88 dir: Make flatpak_transaction_ensure_remote_state non-static
This moves it to fatpak-transaction-private.h so that we can use it
internally to be able to re-use the remote-states from the transaction
outside it.
2020-10-29 15:30:35 +01:00
Alexander Larsson 12fd8332da tests: Test stuff with the old (non-indexed) summary format too 2020-10-29 15:30:35 +01:00
Alexander Larsson 0d2cf085af Add option to disable generating summary index
This is mainly useful for the tests
2020-10-29 15:30:35 +01:00
Alexander Larsson 57954e2efc summary: Enable use of indexed summaries in the client
We first try to download the index, then fall back to the old format.
2020-10-29 15:30:35 +01:00
Alexander Larsson 0597f246c8 build-update-repo: Generate new summary format
In addition to the old summary file we create a summary.idx and
a set of per-arch subsummaries (and extra copies of these if any commit
specify a subset). These are much smaller, and eventually we will also
get deltas for them.

We are not yet using these new formats, although the code is there
to use them once we start downloading them.
2020-10-29 15:30:35 +01:00
Alexander Larsson 096daf91f2 summary: Implement a new, more efficient, summary format
This drops the deltas from the summary and uses the per-commit
metadata field to add the cache data to avoid the need for the separate
xa.cache and xa.sparse-cache indexes. This way we avoid repeating the
refs in multiple places.

Nothing uses this format yet, but we still pass make check if we enable
it.
2020-10-29 15:30:35 +01:00
Alexander Larsson b094585545 utils: Expose get_compat_arch_reverse() and get_arch_for_ref()
We need to use these from some other files too.
2020-10-29 15:30:35 +01:00
Alexander Larsson c5d76e5a85 summary: Extract the summary generation code to helper 2020-10-29 15:30:35 +01:00
Alexander Larsson b7e90d3809 summaries: Add support for limiting which arches end up in summary
This is in preparation for adding other formats for summary and we
might not want all the arches in the fallback summary format.
2020-10-29 15:30:35 +01:00
Alexander Larsson f84b48ac58 utils: Reimplement summary generation
This moves the generation of the summary files completely into flatpak
allowing us to (later) customise what goes into it in more detail and
generate other forms of summaries.
2020-10-29 15:30:35 +01:00
Alexander Larsson 6fd2ea749c Bump ostree requirement to 2020.8 for the new summary features 2020-10-29 15:30:35 +01:00
Alexander Larsson 387bb8c64e Add flatpak_dir_get_remote_subset()
Reads the subset config for the remote, if any.
2020-10-29 15:30:35 +01:00
Alexander Larsson 97fa217072 build-commit-from: Add --subset option
This sets the xa.subsets property on the commit, which we will later
use to create subsets of the repo.
2020-10-29 15:30:35 +01:00
Alexander Larsson a32fb1e333 tests: Silence assert_remote_has_no_config
This is meant to fail, so redirect its stderr to get less confusing test
logs.
2020-10-29 15:30:35 +01:00
Alexander Larsson 67d3443999 tests: Fix unnecessary rebuilds of the test platform
The code to use the pre-generated test platform was buggy so it wasn't
used. Also, generate a "stable" branch of it too as that is used
by test-run.sh.
2020-10-29 15:30:35 +01:00
Alexander Larsson 95ba6c77b4 tests: Better logging of which tests are succeeding
This makes it easier to spot where things go wrong in the logs.
2020-10-29 15:30:35 +01:00
Alexander Larsson d98943b400 tests: Use "flatpak build-update-repo" instead of ostree summary -u
As we tweak the summary generation code we want to use our code, not
the ostree one.
2020-10-29 15:30:35 +01:00
Alexander Larsson 3169fa58ed tests: Convert some flatpak calls to $FLATPAK
This allows these to run under valgrind when testing in valgrind.
2020-10-29 15:30:35 +01:00
Alexander Larsson 3d65276f79 Add --enable-internal-checks option and use in CI
This enables the internal checks in the generated variant parser.
2020-10-29 15:30:35 +01:00
Alexander Larsson fb0691e895 Update to latest variant-schema-compiler with some fixes 2020-10-29 15:30:35 +01:00
Philip Withnall 7d30de344c Merge pull request #3906 from smcv/wip/tolerate-no-accountsservice
parental controls: Fail open if accountsservice is missing
2020-10-15 14:57:36 +01:00
Simon McVittie 1a58a86676 Skip parental controls checks on ServiceUnknown or NameHasNoOwner
If accountsservice isn't available on the system bus, then we can't
ask it for the user's parental controls settings, and we also can't
ask it whether it even has the malcontent extension. Since this is
not a real security boundary, fail open.

This can be dropped if we depend on a version of libmalcontent that maps
these errors to MCT_APP_FILTER_ERROR_DISABLED.

Resolves: https://github.com/flatpak/flatpak/issues/3902
Bug-Debian: https://bugs.debian.org/972138
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-10-15 09:24:14 +01:00
Philip Withnall 6c79a57114 http: Return HOST_UNREACHABLE on error 500
Sometimes a server might return a HTTP error 500 (this seems to happen
sometimes with Microsoft’s VSCode server, for example). Map this to
`G_IO_ERROR_HOST_UNREACHABLE` for now, which is a bit more specific than
returning `G_IO_ERROR_FAILED`, but without the hassle of introducing a
new public error domain which could give more detail.

In particular, this should allow gnome-software to show an error message
to the user for such failed downloads, rather than hiding the error and
logging the following:
```
not handling error failed for action download: While downloading http://packages.microsoft.com/repos/vscode/pool/main/c/code/code_1.45.1-1589445302_amd64.deb: Server returned status 500: Internal Server Error
```

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-12 11:04:40 +02:00
Yuri Chornoivan 0200348cf7 Update Ukrainian translation 2020-10-12 10:39:57 +02:00
Alexander Larsson 180d807d2a oci-authenticator: Fix crash if anon auth fails and no_interaction is set
We were clearing the error from the anon test, and then not doing any
non-anon auth, so error was NULL, causing a crash when returning an
error message.
2020-10-09 16:57:57 +02:00