This specifically fixes a crash on WPT:
/html/syntax/parsing/html5lib_scripted_webkit01.html?run_type=write
Claude wrote a simple reproducing unit test, and you can see it's a
document.write that calls document.write.
There can be cases where the data referenced by Request does not live for the
duration of a Transfer. The most obvious case is a QueuedNavigation..this is
generally a problem, but it's particularly problematic under load when the
request gets queued in the HttpClient.
Most of the data is small, so just _always_ duping it is worth it. The body
can be large, so we provide a flag for callers to set to turn off the body
duping (essentially saying "I promise to keep the body alive for the duration
of the transfer). Currently, only XHR is able to make this guarantee.
When processFrameNavigation fails, we catch the error and log the `qn.url`. But
`qn` has been been freed by this point. This refactors the code to make
`procesFrameNavigation` clearly take ownership of `qn` and of logging any
errors while `qn` is still active.
Global objects (Window, WorkerGlobalState) should have accessors and functions
defined directly on the global object. This is in addition to having it defined
on the prototype.
This prevents some weird overwriting that scripts can otherwise achieve.
Something I saw on espn where window.navigator was being overwritten by a global
`navigator` variable.
When `[]const u8` is a web api parameter, we'll coerce the value to a string,
e.g. true -> "true". This is correct for almost every API. APIs can also opt
to use a string.String (or string.String.Global). This is purely meant as an
optimization, but its behavior is currently different than `[]const u8` as the
bridge will only map actual JS strings to it.
This commit makes String and []const u8 behave the same, so that the only
decision as to which to use is about performance.
(APIs that strictly want a string should use js.String or js.Value and do the
type check)
This is driven by import-maps WPT tests. It generally does 3 high level
additions, plus various small compliance tweaks.
1 - Entries with a trailing match are used for prefix matching
2 - Supports scopes (which are entries group into a specific route-like prefix)
3 - Because of the above, resolves in correct order with fallback
Resolution is based on longest-match wins, so it doesn't require a fancy
data structures. We use ordered (by length) slices, and just iterate until we
find a match. Neither our parsing nor our matching is super efficient. While
a page might have hundreds of scripts, it likely has only 0-1 import maps and
relatively few values.
ImportMap.resolve always returns the final URL. So even if a match isn't found
based on the parsed JSON, it'll return the URL.resolve(base, url). Just to make
WPT tests pass, we do have to track invalid entries in the ImportMap, e.g.
"key": "not-a-url". In the previous version, we'd fallback to URL.resolve(base,
url). Now we return null and leave it to the caller to decide.
Firefox currently doesn't implement this API. Sites already have to check if
it exists, e.g `navigator.getBattery == undefined`. Implementing the method
but making it fail, it's the worst of both wolds.
CookieStore get/getAll return a play JS object, not an WebAPI interface.
`typeof CookieListItem === undefined` in browsers. This removes the interface
and returns the CookieListItem as a plain object.
Also adds various name/value validation based on WPT tests
Resolve relative request URLs against only the path component of the
base URL. Previously, URL.resolve searched for the last slash in the
whole base URL after the authority, so slashes inside a query string or
hash route could be mistaken for path directory separators.
The reported failure was a hash-routed page loaded at
http://127.0.0.1:8123/#/login. When that page ran
fetch("api/users/login", { method: "POST" }), browser-compatible URL
resolution should have requested
http://127.0.0.1:8123/api/users/login. Instead, Lightpanda 0.3.1
reported the response URL as
http://127.0.0.1:8123/#/api/users/login, and the HTTP server received
POST /.
That meant the server returned the single-page app shell instead of the
JSON login response. The failure broke a RealWorld/Conduit-style SPA
benchmark after login because the app used relative API URLs such as
api/users/login. Changing the benchmark fixture to root-absolute API
URLs, such as /api/users/login, worked around the benchmark failure, but
left Lightpanda incompatible with normal browser behavior for relative
request URLs on hash-routed pages.
The same issue was not limited to fragments. A base URL such as
https://example/app/page?next=/foo/bar also contains slashes after the
path component. Those slashes must not affect how path-relative inputs
such as api/users/login or ../api/users/login are merged with the base
path.
This change bounds the directory calculation at the first ? or # in the
base URL and searches for the last path slash only inside that path
component. Root-absolute inputs still keep only the scheme and
authority, query-only inputs still replace the query on the current
path, and fragment-only inputs still replace the fragment on the
current path.
This matches the behavior of the WHATWG URL algorithm, Chromium's
relative URL canonicalization, Node's WHATWG URL implementation, Go's
net/url ResolveReference, and Python's urllib.parse.urljoin for the
cases covered here. All of those implementations split the base URL into
components before merging a path-relative reference, so slashes in the
query or fragment do not change the base directory.
The URL resolver tests now cover the original hash-route repro, slashes
inside query strings and fragments, host-only bases with query or
fragment components, dot-segment paths, query-only references, and
fragment-only references. The fetch Web API tests also move a page to
/#/login and POST fetch("xhr"), expecting the request URL to resolve to
http://127.0.0.1:9582/xhr rather than the hash route.
Verified with:
mise x zig@0.15.2 -- zig fmt --check ./*.zig ./**/*.zig
mise x zig@0.15.2 rust@stable -- make ZIG=zig test
The WebSocket upgrade handshake is an HTTP/1.1 request (RFC 6455 §4.1)
and follows ordinary cookie semantics — RFC 6265 §5.4 attaches matching
cookies to "any HTTP request" by domain/path. Without this, cookie-
authenticated WebSocket endpoints (anything session-gated, e.g. Phoenix
LiveView) reject the upgrade because their auth cookie never arrives.
Read matching cookies from the session jar with the same opts shape
HTTPDocument uses (`is_http: true, is_navigation: false`), and add a
`Cookie:` request header on the upgrade if any apply.
The TestWSServer captures the upgrade's Cookie header and exposes it
to fixtures via a new `get-cookie` command. A `cookies_on_upgrade`
fixture in websocket.html sets `document.cookie` then asserts the
server received it on the upgrade.
Fixes crash in WPT /custom-elements/CustomElementRegistry.html
define has to get `observedAttributes` which itself could call define,
invalidating any GetOrPutEntry pointers. Need to do it as two distinct lookup.
With the new CookieStore, the session must be freed before the notification is.
This is how it works in CDP, but in fetch, we were pretty lazy about it. This
caused the notification to be freed first, and then the cookiestore to try to
unregister: UAF.
This is a bit all over the place.
1 - Replace libidn2 with rust-idna. It looks like there are different idna
profiles, and rust-idna (from the servo project) implements the whatwg
one. libidn2 would be too strict in some cases and not strict enough in
others. (Gemini says I could use libidn2 for this, but what it suggested
didn't work, and I couldn't figure it out myself, and claude insisted it
_did not_ have the correct implementation for what we want).
2 - We previously only ran a URL through idna if it wasn't ascii. Turns out
we also need to run it if there's a "xn--" (aka, an IDNA ACE prefix) in
there. This helps us pass hundreds of WPT cases, and it's pretty cheap.
3 - Implement more of the Area WebAPI. Mostly copied from Anchor.
4 - Add username/password accessor to Anchor/Area
5 - window.open validates the URL (i.e. tries to resolve it and handles the
error)
6 - Invalid idna conversion maps to a TypeError
7 - Cleanup closed popups on the next tick (like destroyed pages), rather than
at an interval or on shutdown. This one seems unrelated, but some of these
tests are opening hundreds (thousands?) of popups and then closing them.
If a synthetic url (blob URL) causes a navigation event, the frame abort will
deinit the transfer, causing the `defer transfer.deinit()` atop Synthetic.run
from firing. Flag the transfer as .completing to prevent this from happening.
This mimics what non-synthetic urls do.
Adds a pretty simplistic Notification WebAPI. Also adds a dummy drawImage to
CanvasRenderingContext2D.
Trying to improve how we're seen by https://bot.sannysoft.com/
Previously, Cookie.Jar.add would only conditionally take over the cookie. The
caller had no way to know whether or not to deinit it. This could result in
a double-free on certain error paths.
Cookie.Jar.add now unconditionally takes ownership of the cookie.
