20 Commits

Author SHA1 Message Date
Alexander Larsson
3ebcd200ca Remove should-retry debug spew
Every http operation (even when successfull) was spewing info about
whether to retry it which made it hard to read the logs.
2020-11-03 12:57:37 +01:00
Philip Withnall
6c79a57114 http: Return HOST_UNREACHABLE on error 500
Sometimes a server might return a HTTP error 500 (this seems to happen
sometimes with Microsoft’s VSCode server, for example). Map this to
`G_IO_ERROR_HOST_UNREACHABLE` for now, which is a bit more specific than
returning `G_IO_ERROR_FAILED`, but without the hassle of introducing a
new public error domain which could give more detail.

In particular, this should allow gnome-software to show an error message
to the user for such failed downloads, rather than hiding the error and
logging the following:
```
not handling error failed for action download: While downloading http://packages.microsoft.com/repos/vscode/pool/main/c/code/code_1.45.1-1589445302_amd64.deb: Server returned status 500: Internal Server Error
```

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-12 11:04:40 +02:00
Alexander Larsson
e0a25e7650 http: Also accept image indexes when FLATPAK_HTTP_FLAGS_ACCEPT_OCI 2020-06-05 09:35:30 +02:00
Alexander Larsson
7820057336 Add content_type output to flatpak_load_uri()
We want this to get the oci mimetypes
2020-06-05 09:35:30 +02:00
Alexander Larsson
f6a0fa41a1 Add FLATPAK_HTTP_ERROR_UNAUTHORIZED (i.e. http status 401) 2020-05-05 13:08:57 +02:00
Alexander Larsson
e3c31309a4 Rename internal error FLATPAK_OCI_ERROR to FLATPAK_HTTP_ERROR
This is what it is really, and I want to extend it with more http errors
that are not necessarily oci related.
2020-05-05 13:08:57 +02:00
Alexander Larsson
0ef69c96db http-utils: Rename flatpak_load_http_uri to flatpak_load_uri and support file:
Various places (like the new load-commit-for-extra-data-setup) needs to
support file: uris, lets make it available generically.
2020-03-30 15:00:51 +02:00
Matthew Leeds
5560132ba6 flatpak-utils-http.c: Add retry logic for transient failures
Currently if flatpak is installing an extra data app such as Spotify and
the server with the .deb file fails to complete the request, the
installation fails with a message like "Connection terminated
unexpectedly". This commit makes flatpak instead try 5 times to download
a given URI if the error returned seems like a transient one (so not,
for example, 404 not found). This is analogous to what was done in
libostree in commit 938055392fd455027a69398c441b992ae521aa87, and we use
some code from there.
2020-03-20 15:47:18 -07:00
Matthew Leeds
0b25455af1 flatpak-utils-http.c: Use more specific GIOError codes
Instead of defaulting to G_IO_ERROR_FAILED, use more specific codes when
we can. These were copied from libostree.
2020-03-20 13:46:36 -07:00
Alexander Larsson
976c88cf56 oci: Pass down token into OCI http operations
This is needed for pull operations to actually use the token if one
is given by an authenticator.
2019-12-12 12:16:53 +01:00
Matthias Clasen
e04263ca5c Fix a case of signed/unsigned confusion
G_input_stream_read_finish returns a gssize,
which we must not store in a gsize, since it
may be -1.

Closes: https://github.com/flatpak/flatpak/issues/2885
2019-12-01 23:01:10 -05:00
Alexander Larsson
45e31735f1 OCI: Accept docker v2 manifests as well as OCI manifests
Closes: #2978
Approved by: alexlarsson
2019-09-06 12:49:40 +00:00
Matthew Leeds
807f2baa53 Move interpretation of OSTREE_DEBUG_HTTP
It makes more sense to have this in flatpak_create_soup_session() so
every caller of that interprets the env var.

Closes: #2740
Approved by: matthiasclasen
2019-03-06 23:44:50 +00:00
Matthew Leeds
e8816b7663 Remove extra newlines in variable definiton blocks
Uncrustify has an option "nl_func_var_def_blk" which is supposed to
ensure there's a newline character between the block of variable
definitions and the rest of the function body, but it gets confused and
thinks that the first instance of "g_autoptr" or "g_auto" being used on
a variable is the start of the function body. So this commit removes
those extra newline characters and removes that option in uncrustify.cfg
so they don't get re-added the next time uncrustify is run.

Here's the command I used:

perl -0777 -i -pe 's/\n(\n\s*g_auto\()/\1/g' `git ls-tree --name-only
-r HEAD | grep \\\.[ch]$ | grep -v common/valgrind-private.h |
grep -v app/flatpak-polkit-agent-text-listener\\\.[ch]`

I ran it again with "g_autoptr" in place of "g_auto", and made a few
manual edits to add back the newline when the g_auto* was in the middle
of a function body rather than at the top.

Closes: #2715
Approved by: matthiasclasen
2019-02-25 18:12:30 +00:00
Owen W. Taylor
05cc3c0f66 flatpak_cache_http_uri: save downloaded files with permission 0644
Previously, downloaded files were being saved with 0600 permissions,
which prevented OCI icons downloaded by the system helper at appstream
creation time from being read by users.

Closes: #2362
Approved by: matthiasclasen
2018-11-30 23:53:10 +00:00
Alexander Larsson
9cc0f0d404 Fix leak in flatpak_cache_http_uri
Need to free the return value of soup_header_parse_param_list

Closes: #1966
Approved by: alexlarsson
2018-08-17 08:13:15 +00:00
Owen W. Taylor
c4c06b7be4 Allow using a compressed cache for HTTP downloads
Add a new flag for flatpak_cache_http_uri() that adds Accept-Encoding: gzip
to the request, and if the result is returned compressed, stores the data
compressed. If the data result is return uncompressed, it's compressed.

Closes: #1910
Approved by: alexlarsson
2018-08-09 12:49:35 +00:00
Owen W. Taylor
4dfa7721bb Remove unused etag functionality
Checking the registry against a previous etag is now handled inside
flatpak_cache_http_uri(), so remove the etag parameters that were
previously passed around in various places for simplicity.

Closes: #1910
Approved by: alexlarsson
2018-08-09 12:49:35 +00:00
Owen W. Taylor
951aed561a Add flatpak_cache_http_uri: cache downloads based on HTTP headers
Add a new function, flatpak_cache_http_uri() that when passed an URL and
a local destination location, either a) downloads the content and stores
it at the destination location, storing HTTP cache header information
like Last-Modified, Etag into user xattrs (if available) or a separate
file or b) if the downloaded content is already present, checks the
header information to decide whether the downloaded content can be used
or needs to be revalidated witha conditional request.

Tests are added that use a special case test server that adds HTTP caching
headers and reacts to them based on query parameters. A small test binary
'httpcache' is added for the tests to use.

Closes: #1910
Approved by: alexlarsson
2018-08-09 12:49:35 +00:00
Owen W. Taylor
cd6a10cb66 flatpak-utils-http.c: Split HTTP functionality into a separate file
In preparation for extending the HTTP downloading function to include
caching, split HTTP related utilities into a separate file with a
separate header.

Closes: #1910
Approved by: alexlarsson
2018-08-09 12:49:35 +00:00