The site for cookies were computed from the immediate parent `Frame`, which would allow sending a cookie that's `SameSite=Strict` from 2 levels deep under. Directly from RFC6265bis, this PR essentially implements (except for step 4, we skip host-less ancestors):
Given a Document (document), the following algorithm returns its
"site for cookies":
1. Let top-document be the active document in document's navigable's
top-level traversable.
2. Let top-origin be the origin of top-document's URI if top-
document's sandboxed origin browsing context flag is set, and
top-document's origin otherwise.
3. Let documents be a list consisting of the active documents of
document's inclusive ancestor navigables.
4. For each item in documents:
1. Let origin be the origin of item's URI if item's sandboxed
origin browsing context flag is set, and item's origin
otherwise.
2. If origin is not same-site with top-origin, return an origin
set to an opaque origin.
5. Return top-origin.
A client that disconnects might get treated as a harsher terminate failure (e.g.
watchdog). This doesn't have a huge impact, but it makes the CI flaky and it
produces more logs than is necessary.
In a terminate state, the driver will now check its inbox to see if this is a
client disconnection.
1. Abstract "Driver". Non-CDP things that referenced *CDP now reference a Driver
2. Move the NodeRegistry out of CDP. This created an artificial link between
agent / mcp and CDP
3. Add BiDi driver with enough to navigate to a page
https://github.com/lightpanda-io/browser/issues/3348
When set to 1.1, libcurl is configured to only offer HTTP 1.1. By default, or
when set to "auto", it's up to libcurl to decide how to connect. This maps to
libcurl's CURL_HTTP_VERSION_1_1 and CURL_HTTP_VERSION_NONE.
LP.configureCDP now takes an `httpVersion` field which can be "1.1" or "auto"
to control that specific browser session. Ideally this is called prior to any
navigation.
Our `Arena` from the `ArenaPool` now tracks if it's already been released. On
a subsequent release, it panics then and there. Without this, the code will
almost certainly panic anyways, but it will panic in a seemingly unrelated
place. Hopefully this makes identifying future cases of this easier (since we'll
get the stack trace of the re-releaser).
Fix 3 separate memory issues, all edge cases.
1 - an XHR error handler that re-open/sends would incorrectly null the new
transfer (maybe this isn't that odd, maybe it's a common retry-on-error).
2 - On a curl_easy_pause error (from WebSocket.zig) we now unqueue the just-
queued message, because the error will errdefer the message arena to be
cleaned up.
3 - ScriptManager now cleans up after itself on a failure prior to submit()
being called.
HttpClient guarantees that only one of done/error/shtudown callback will be
called. But that guarantee had a hole when a done/error callback would trigger
JS that caused an HttpClient.abort (e.g. by scheduling a navigation).
So far, this isn't a problem. fetch/XHR guard against this themselves, and
ScriptManager doesn't care of it happens.
But https://github.com/lightpanda-io/browser/pull/3230 doesn't guard against it
and would crash if it happens: its doneCallback frees `self` but can re-enter
shutdown which needs `self`. While 3230 could fix this issue, I'd prefer to
bake this contract into the HttpClient. Transfers now have an
`_outcome_delivered: bool` which guarantees that only one of done/error/shutdown
is called.
Network has accumulated a bit of bagged. It knows a lot about certificates, it
knows a lot of the cache. I have plans to expand Network, and wanted to clean it
up.
1 - certificate logic moved to dedicated Certificates.zig
2 - Removed CurlDebugAllocator
- this makes setup easier, to Updater can just init libcurl directly
3 - Change Updater to be a single function
4 - Cache initialization is don in the Cache
5 - ?Cache => Cache{.kind = .noop}
Add an optional per-host rate limit. This currently only applies to the top-
level navigation. This makes it simpler to implement and simpler to reason
about. The full load of a page is only ever delayed at its head, not
sporadically through the page loading.
The use-case where a RateLimiter is most useful is when the browser is crawling
multiple pages of the same site, and in that case, the top-level rate limit
still applies some degree of limit to any linked resources (e.g. a JS on a
different host).
`--http-nav-delay` is the delay, in milliseconds, to apply to top level
navigates per host. Currently defaults to 0 (disabled).
`--http-nav-burst` is the burst allowed per host. Defaults to 1 (has no impact
when `--http-nav-delay` is disabled).
1-
Per spec, dictionary objects should be loaded in lexicographical order (a). So
the following always pushes `endings` before `type`
```js
new Blob([], {
get type() { order.push('type'); return ''; },
get endings() { order.push('endings'); return 'transparent'; },
});
```
This requirement is now enforced by a comptime check. So, a lot of files were
updated to satisfy this requirement.
2-
Blob parts now work with the @@iterator protocol. This is done more generically
(via `value.iterator())` but is currently only used by Blob (and File). Other
types that need this (e.g. URLSearchParams) can be done in a follow up.
3-
Add support for Blob.textStream
4-
Reject XHR/Fetch requests on blob URLs if the method isn't GET
5-
Strip #hash from blob URL when doing lookup
All of this fixes ~175 FileAPI WPT cases, though it's possible #1 (the
dictionary ordering) helps with a few others.
(a) https://webidl.spec.whatwg.org/#js-dictionary
Builds ontop of https://github.com/lightpanda-io/browser/pull/3200 to centralize
header enforcement and standardize merge vs overwrite header logic.
The API is still a `setHeader` and `appendHeader`, with a source, but set/append
both are thin wrappers around private `putHeader`. putHeader blocks overwriting
restricted headers (user-agent). The `source` acts as a priority (ordered enum)
which further restricts the header AND depending on whether set or append were
called, controls if the value is overwritten or appended to.
3200 had an always-append which can cause problems, e.g. a script setting
Accept-Language: fr would have results in the value being appended to the
default, e.g.: 'en-US,en;q=0.9, fr'.
On a redirect, recalculate the Referer header based on the new target.
Also, allow SVG anchors to be clicked (doesn't seem related, but it came up
in referrer-policy WPT tests).
Extracted from https://github.com/lightpanda-io/browser/pull/3122. Sends RI for
redirect. Also, on a continueRequest which does redirect, restores the original
headers (continueRequest's headers are only valid for a single request).
To make this work in all drivers, CDP now decouples the transfer_id from the
intercept_id. Each unique request gets a distinct intercept_id which is managed
in CDP (with a intercept_id -> transfer_id mapping).
When we know the precise final length, prefer ensureTotalCapacityPrecise over
ensureTotalCapacity. The latter goes through `growCapacity` which will allocate
~1.5x padding.
We already had special handling for BoringSSL's RecvError on improperly closed
TLS connection. This moves the handling up, so that redirect handling is
covered by it too.
Adding headers to an HTTP request was a bit awkward due to my desire to avoid
having an intermediate representation (e.g. an ArrayList(Header)). Going
straight to a curl slist avoids double-copying the headers (first to Zig, then
to curl).
But the CORS work (https://github.com/lightpanda-io/browser/pull/3002) showcases
that this micro-optimization simply isn't worth it, since it needs that
intermediate representation anyways.
And, this change isn't just for CORS. Headers have been a silly pain in the past
like unclear ownership, and messy APIs used in _a lot_ of places (WebBotAuth,
WebSocket, Fetch, ...)
This new approach stores headers on the transfer in an ArrayList. The API is:
```
const transfer = try client.newRequest(.{...}, owner);
{
errdefer transfer.deinit();
try transfer.addHeader("Over", "9000", .{});
}
try transfer.submit();
```
This:
1 - Eliminates ambiguity about errdefer cleanup responsibility
2 - Eliminates a bunch of stringZ concat that Frame, Config, CDP were doing
3 - Transfer.arena is now available for headers
Halil Durak