The v8 documentation says this "Has to be called", and claude flagged it as a
potential leak.
Required a new zig-v8-fork build (v0.5.5) which I released.
Filter lists carry a few hundred rules written as JavaScript regex
literals (24 in EasyList, 165 in uBO's badware list); they parsed but
were dropped as unsupported. PCRE2 reads that syntax as-is, `\/` and
friends included, its compiled patterns are immutable so the one
blocker shared by every HTTP client thread can run them, and 10.48
ships a build.zig for 0.16, so it is wired like sqlite3.
`Regex.Context` routes every PCRE2 allocation through the blocker's
allocator, which puts the compiled patterns under the test runner's
leak detection, and caps match and depth so a broken pattern costs a
false negative rather than a stalled request. As in uBO, a regex
tests the raw URL with the case-insensitive flag unless `$match-case`.
Regex filters are still never tokenized: they ride the fallback bucket.
curl 8.22 moved the resolver sources into lib/vdns/, renamed the QUIC
backends to cf-ngtcp2*/cf-quiche, folded noproxy.c into proxy.c and
added a handful of new files. Regenerate the source list from the
tarball and drop the four config defines the 8.22 template no longer
declares (Zig errors on unused config header values).
zenai tool results now carry image parts (lightpanda-io/zenai#12, #13),
so the model-driven tool path opts into inline images: a screenshot
without `path` reaches the model as text plus the PNG, on every backend.
The slash-command path still needs `path`, its result goes to the
terminal.
https://github.com/lightpanda-io/browser/pull/3271 introduce an orderfile so
that hot sections of code were grouped together in the binary, resulting in more
efficient loading. But, it excluded v8 functions because v8 is compiled with
`-fno-unique-section-names` so each section gets the same name. Because of this
the orderfile can't target specific "hot" or "cold" functions.
As a follow up to 3271, I thought we'd be able to re-compile v8 without that
flag, but:
1 - The flag isn't directly exposed by v8, so we'd need to change v8's own build
BUILD.gn in our build process
2 - It would make the .a file ~ 40MB larger (though the final lightpanda binary
would stay the same size
3 - If we wanted to created two .a files (one with -fno-unique-section-names and
one without), it would require a full rebuild (for both x86-64 and arm).
I didn't love those compromises, so I asked Claude if the sections names could
be generated in a separate file and then that could be used when generating out
lightpanda.ld. What claude came up with was a small Zig script
(mark_hot_sections.zig) which is now run as part of the build. It takes the
v8.a file (which still has `-fno-unique-section-names`), it takes a text list
of hot functions, and it re-generates v8.a with a special .text.hot and
.rodata.hot sections. lightpanda.ld can include these.
This PR depends on https://github.com/lightpanda-io/zig-v8-fork/pull/202 but
202 doesn't require a v8 rebuild. It merely allows prebuilt_v8_path to be
a lazypath, which we need because the prebuilt_v8_path that we pass is now
generated from this build script.
The DuckDuckGo HTML endpoint the search tool fell back to is disallowed by
its robots.txt, so it is removed: the `duckduckgo` engine, the scrape via
`performGoto`, and the tool's browser-side dependency (`execSearch` no
longer needs a session). In its place `.auto` is a single walk over the
API engines in table order — each when its key is set — with Keenable
last, keyed if `KEENABLE_API_KEY` is set and its public endpoint
otherwise. Search therefore always has a rung with zero configuration,
and a set key is never shadowed by a keyless retry.
Every search now goes through zenai's `std.http.Client`, which had no
timeout at all; a stalled response would have blocked the MCP server's
shared browser thread for every session. zenai now bounds each attempt
(lightpanda-io/zenai#10, pinned here), and the tool's `timeout` argument
drives it, defaulting to 10 s. Retries are a property of the call path:
off in the cascade, where the next rung is the retry, and the engine
default for an explicit engine.
Cleanups from the #3252 review: `keyless` is derived from the client's
`api_key` type instead of a hand-set table flag; `engineIndex` no longer
depends on enum declaration order; one `searchKeyStatus` query replaces
three pub helpers for the REPL; the engine order and env-var list in the
tool description and `/searchEngine` help are generated from the table;
an empty result title renders as its URL instead of `****`.
- build.zig.zon points at lightpanda-io/zenai's merge commit of zenai#9
(same tree, so the hash is unchanged)
- the keyless rung is no longer skipped when KEENABLE_API_KEY is set:
the public endpoint has its own rate-limit regime, so it is worth
trying after a failed keyed attempt — which the search tool
description already promised
- keenable retries are disabled: the fallback cascade is the retry
mechanism, and honoring the public endpoint's 60s Retry-After would
stall the search tool short of the DDG rung
- stale SearchEngine/searchExplicit doc comments updated for keyless
zenai's review round made keyless explicit in the client's type
(init takes ?[]const u8; null selects the public endpoint) and removed
the always-empty description field from results:
- the .auto keyless rung and searchExplicit pass null instead of "";
the null-key call is comptime-gated to keyless-capable engines and
apiSearch's key parameter is anytype to admit both shapes
- a set-but-empty KEENABLE_API_KEY is now keyed and fails loudly
instead of silently switching to the keyless rate-limit regime
- formatKeenableMarkdown reads snippet only (description no longer
exists to fall back to)
- build.zig.zon repinned to the reviewed zenai commit
Adds keenable to the search tool's engines (KEENABLE_API_KEY, tried
after brave/tavily/exa in .auto). Unlike the other engines it also
answers without any key: its client routes an empty key to the public
endpoint (rate-limited per client IP), so .auto now tries that as the
last rung before the DuckDuckGo scrape, and an explicit
/searchEngine keenable works keyless instead of erroring. Engine table
entries gain init_options so the Keenable client can carry the
lightpanda attribution header.
Needs the zenai Keenable search client (lightpanda-io/zenai PR #9); the
zon pin points at that branch until it lands.
Supports prebuild-v8 shared library and adds a `download-v8-shared` make target.
Depends on: https://github.com/lightpanda-io/zig-v8-fork/pull/198
The auto-discovery of the .so is done in the build.zig, which is different than
the auto-discovery of the .a which is done in the Makefile. This is intentional
and if we inverse -Ddev_fast so that it's on by default, the .a's auto discovery
will be moved to build.zig for consistency and explicitness. As-is, we have no
way to tell whether -Dprebuilt_v8_path=v8.a is being injected by Makefile or
is explicit (which isn't a problem so long as we're not doing dev_fast by
default).
Pulls in the latest three zenai commits:
- f721fd4 feat(retry): honor Retry-After hints on retryable responses
- 103f5e1 feat(retry): treat 408 request timeout as retryable
- 75597b3 feat(types): sync with upstream Go SDK releases
Replace the Anthropic Claude-Code-import flow with OpenAI Codex (ChatGPT
subscription) OAuth: device-code login, a real token store (auth.json,
0600) with refresh-grant, and JWT ChatGPT-Account-Id extraction. Codex is
now a distinct zenai provider, so the bearer-mode credential collapses to
provider==.codex; account_id threads to the client via InitOptions.
Unit-tested (build green): JWT account-id extraction, token store
round-trip, OAuth request-body encoding, token-response parsing. The
device-login / refresh network path is unverified pending a subscription.
Pins zenai to the codex branch (6967324).
Add Brave as a second API-backed web search engine: the search tool's
auto mode tries Brave then Tavily (keyed on BRAVE_API_KEY/TAVILY_API_KEY)
before the DuckDuckGo scrape. A new /searchEngine REPL metacommand pins
the engine explicitly (auto|tavily|brave|duckduckgo) — an explicit
engine surfaces its failures instead of silently degrading — and is
persisted to .lp-agent.zon like the other REPL settings.
Bumps zenai to pick up the zenai.search.brave client.
Built against https://github.com/lightpanda-io/zig-v8-fork/tree/zig-0.16 but
it doesn't require a new v8 build.
Built against https://github.com/lightpanda-io/boringssl-zig/tree/zig-0.16
since the current fork we point to isn't updated.
A global std.Io instance, lp.io. Way easier this way and requires 0 changes to
our libcurl integration / event loop.
Network code uses a new layer that does what Zig 0.15's posix package used to
do. Again, quicker migration that way. But, as long as we have the global IO,
and given the half-baked nature of networking in std.Io 0.16, this just makes
sense. Things can be migrated as needed.
The std.time.* -> std.Io.Timestamp/Clock/Duration resulted in _a lot_ of
changes. ArrayList = .{} -> ArrayList -> .empty also resulted in a lot of
changes, but that's obviously superficial. As is the trimLeft/trimRight ->
trimStart/trimEnd rename.
Locking adopt the `Uncancelable` variants, e.g. mutex.lockUncancelable() to
preserve the error-free signature (and, because cancellation would be something
we'd have to put more thought into).
std.json.ObjectMap is now unmanaged, so the allocator had to be passed along.
However, there's still a deprecated managed variant of MemoryPool, so I switched
to it (we can do a small follow up PR to move to the unmanaged after).
I tried use_llvm = false, but it locks my computer, consuming RAM until MacOS
gives me a popup I've never seen before, begging me to start killing processes.
Agent and the networking stuff saw the most significant changes.
Updates the zenai dependency and resolves the agent's effort based on
the configured provider's default. Also dynamically updates the active
effort when switching providers.
Essentially makes it so that:
```
var w = iframe.contentWindow
iframe.src = 'spice.html';
w === iframe.contentWindow
```
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/182 and leverages
v8's own ability to re-use globals.
This uses v8
Karl Seguin