Zig 0.17 removes @cImport in favor of translate-c steps in the build
system. Each library's link function now creates an addTranslateC step
exposed as a module import: curl and isocline translate their headers
straight from the dependency tree, while sqlite3 uses the artifact's
emitted include tree since the amalgamation lives in a sub-dependency.
Terminal.zig and prompt_assist.zig previously instantiated two
independent @cImport namespaces for isocline; they now share one
translated module, so its types are identical across both files.
Give accurate connectionId, connectionReused, initialPriority and securityState
values.
Always set `referrerPolicy` to `unsafe-url` as the most honest answer (we should
implement proper referrer policy!).
For workers, track the underlying frame_id so that it can be used for the
`documentURL` field.
Built against https://github.com/lightpanda-io/zig-v8-fork/tree/zig-0.16 but
it doesn't require a new v8 build.
Built against https://github.com/lightpanda-io/boringssl-zig/tree/zig-0.16
since the current fork we point to isn't updated.
A global std.Io instance, lp.io. Way easier this way and requires 0 changes to
our libcurl integration / event loop.
Network code uses a new layer that does what Zig 0.15's posix package used to
do. Again, quicker migration that way. But, as long as we have the global IO,
and given the half-baked nature of networking in std.Io 0.16, this just makes
sense. Things can be migrated as needed.
The std.time.* -> std.Io.Timestamp/Clock/Duration resulted in _a lot_ of
changes. ArrayList = .{} -> ArrayList -> .empty also resulted in a lot of
changes, but that's obviously superficial. As is the trimLeft/trimRight ->
trimStart/trimEnd rename.
Locking adopt the `Uncancelable` variants, e.g. mutex.lockUncancelable() to
preserve the error-free signature (and, because cancellation would be something
we'd have to put more thought into).
std.json.ObjectMap is now unmanaged, so the allocator had to be passed along.
However, there's still a deprecated managed variant of MemoryPool, so I switched
to it (we can do a small follow up PR to move to the unmanaged after).
I tried use_llvm = false, but it locks my computer, consuming RAM until MacOS
gives me a popup I've never seen before, begging me to start killing processes.
Agent and the networking stuff saw the most significant changes.
Removes lying dead code here and simplifies `curl_easy_setopt` type checking. Also marks passed callbacks as with C calling convention in order to get away from additional function wrapping.
Implements `url_resolve_with_encoding` and `url_resolve_without_encoding` in Rust; that way, we don't pay the cost of extra `Box`es we allocate during resolving.
I threw Claude at WPT's /WebCryptoAPI/ which represents a significant number of
failing cases.
There were a few very low-hanging fruit, like enabling CryptoKey on Worker and
supporting `{name: string}` in addition to just `string` for some functions.
Some of these "fixes" just handle the error case / validation, which tends to
represent a greater number of WPT cases, e.g. some of the importKey algorithms
aren't fully supported, but they still validate the input and return the correct
errors.
To that end, while there should be considerably more passing cases (~42K), some
of these changes merely unlocked more failing cases, so our total case count
should go up.
Claude's summary:
1. digest {name} object fix (16→80)
2. Symmetric importKey/exportKey/generateKey (AES, HMAC; raw + jwk) + CryptoKey
accessors + DataError
3. EC/OKP importKey usage validation (failure-path)
4. PBKDF2 + HKDF deriveBits, then deriveKey
5. ECDH generateKey + import (spki/pkcs8) + deriveBits/deriveKey
6. AES encrypt/decrypt (CBC/CTR/GCM)
This is a bit all over the place.
1 - Replace libidn2 with rust-idna. It looks like there are different idna
profiles, and rust-idna (from the servo project) implements the whatwg
one. libidn2 would be too strict in some cases and not strict enough in
others. (Gemini says I could use libidn2 for this, but what it suggested
didn't work, and I couldn't figure it out myself, and claude insisted it
_did not_ have the correct implementation for what we want).
2 - We previously only ran a URL through idna if it wasn't ascii. Turns out
we also need to run it if there's a "xn--" (aka, an IDNA ACE prefix) in
there. This helps us pass hundreds of WPT cases, and it's pretty cheap.
3 - Implement more of the Area WebAPI. Mostly copied from Anchor.
4 - Add username/password accessor to Anchor/Area
5 - window.open validates the URL (i.e. tries to resolve it and handles the
error)
6 - Invalid idna conversion maps to a TypeError
7 - Cleanup closed popups on the next tick (like destroyed pages), rather than
at an interval or on shutdown. This one seems unrelated, but some of these
tests are opening hundreds (thousands?) of popups and then closing them.
Previously, the CDP socket was added to the worker's multi and fully owned
by the worker. While this is simple, it introduced some issues:
1 - Cannot detect a disconnected client during JS processing ( for(;;) )
2 - A blocked worker can cause back-pressure that blocks the client. This can
cause a deadlock if the worker is blocked waiting for a CDP message
In addition to these 2 problems, there was 1 other serious CDP-related issue:
arbitrary CDP messages could be processed during JavaScript callback. For
example, a Worker calls importScripts while request interception is enabled,
this requires us to tick the HttpClient waiting for the interception response.
But, a client could sent Target.closeTarget, which we'd process and delete the
frame..all while importScripts is still blocked. Assuming importScripts unblocks
everything is a big UAF since the frame (and its workers) were cleared from
closeTarget.
The CDP socket is now read from the network (main) thread and an OTP-style
mailbox is used. The network thread posts message to the Worker's inbox and
signals it to wakeup. This solves #1 and #2. It doesn't directly solve the
reentrancy issue, but it provides the foundation. Specifically, in introduces
a queue for of CDP message and more control over when/how that queue is
processed. At "safe points" (Runner.tick, HttpClient.tick), any message can
be processed. But, when inside a JavaScript callback, we can process only non-
destructive/mutating message. Specifically, we can process only messages related
to request interception.
Links to libidn2 and builds libcurl with it. This makes libcurl work, and by
extension browser, work on international domain names, e.g.
zig build run -- fetch "https://räksmörgås.se/"
With it available, we can use it in our WebAPIs which should also support these
domains, e.g:
testing.expectEqual('xn--rksmrgs-5wao1o.se', new URL('https://räksmörgås.se').hostname);
There is more integration to be done here, but this is a first step.
claude wrote all of the build.zig code.
I don't have a strong opinion about this feature, I just dislike that our WPT
/url/* tests are at 1704 / 9095 and, this is the biggest chunk (although, this
specific commit just does the basic integration and probably won't fix too many
WPT cases directly).
Block outbound HTTP requests to specified IP ranges before TCP handshake
using libcurl CURLOPT_OPENSOCKETFUNCTION callback. Fires after DNS
resolution, reads resolved IP directly from sockaddr, does bitwise CIDR
comparison. Fail-closed: unknown address families are blocked.
--block_private_networks blocks RFC1918, localhost, link-local, ULA.
--block_cidrs blocks additional comma-separated CIDRs.
IPv4-mapped IPv6 (::ffff:x.x.x.x) is unwrapped to prevent bypass.
Uses libcurl's websocket capabilities to add support for WebSocket.
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/167
Issue: https://github.com/lightpanda-io/browser/issues/1952
This is a WIP because it currently uses the same connection pool used for all
HTTP requests. It would be pretty easy for a page to starve the pool and block
any progress.
We previously stored the *Transfer inside of the easy's private data. We now
store the *Connection, and a Connection now has a `transport` field which is
a union for `http: *Transfer` or `websocket: *Websocket`.
I've been thinking the implementation here is messy (ever since we added support for it) and thought it would be better to separate each algorithm to their respective files in order to maintain in a long run. `digest` is also refactored to prefer libcrypto instead of std.
Karl Seguin