Commit Graph
7774 Commits
Author SHA1 Message Date
Adrià Arrufat 3dcbb46834 webapi: implement HTMLScriptElement.supports and DOMTokenList.supports
Module loaders (Vite legacy plugin, Shopify themes) feature-detect
HTMLScriptElement.supports("module") and
link.relList.supports("modulepreload"). Both were missing, so themes
fell back to fetch()-based legacy chunk loading: on allbirds.com that
meant 676 requests instead of 305 for two page loads, the same module
fetched 9x per page by page JS, and a ~4% CDP driver flake from the
legacy path replacing the document mid-query (0/50 repro with this
change, vs failing within 3 iterations without).

HTMLScriptElement.supports returns true for the types this engine
loads (classic, module, importmap). DOMTokenList.supports implements
the rel supported-tokens set and throws TypeError for attributes that
define none, per spec.
2026-07-04 20:27:26 +02:00
Adrià Arrufat 1c9acfa6bc Merge pull request #2878 from lightpanda-io/agent-vertex
agent: add Google Vertex AI support
2026-07-04 12:10:32 +02:00
Karl Seguin 37033003dd Merge pull request #2867 from lightpanda-io/nikneym/curl-easy-setopt-refactor
`libcurl`: refactor `curl_easy_setopt`
2026-07-04 15:43:26 +08:00
Halil Durak ca1b625d51 libcurl: reintroduce Curl* prefix for function pointer types 2026-07-04 10:01:44 +03:00
Halil Durak f17569f613 revert changes in HttpClient and WebSocket 2026-07-04 10:01:44 +03:00
Halil Durak 0011f87d82 libcurl: fix VERIFYHOST comment 2026-07-04 10:01:44 +03:00
Halil Durak f2fce9310c update all call-sites that are using curl_easy_setopt 2026-07-04 10:01:43 +03:00
Halil Durak a8a10b9ba4 libcurl: refactor curl_easy_setopt
Removes lying dead code here and simplifies `curl_easy_setopt` type checking. Also marks passed callbacks as with C calling convention in order to get away from additional function wrapping.
2026-07-04 10:01:43 +03:00
Karl Seguin 5056140bff Merge pull request #2857 from lightpanda-io/detach_global
v8: When re-using window, detach from original context first
2026-07-04 13:39:17 +08:00
Karl Seguin 1c3c6e05c4 Merge pull request #2872 from lightpanda-io/add_worker_interfaces
webapi: Add 6 more interfaces to Worker
2026-07-04 13:38:33 +08:00
Karl Seguin 62ecfb48d8 Merge pull request #2880 from lightpanda-io/script_replay_nav_fixes
script: fix two replay crashes on tool-triggered navigation and $LP_* goto URLs
2026-07-04 10:27:24 +08:00
Karl Seguin a082dc2b07 update v8 2026-07-04 07:29:40 +08:00
Adrià Arrufat cfd61ca69e tools: resolve $LP_* placeholders in startGoto
The script runtime's async goto path calls startGoto directly,
bypassing the substituteStringArgs pass that call() applies to every
other tool, so $LP_* placeholders in goto URLs (a base URL, a username
in the path) stayed literal and the navigation failed on replay.
2026-07-03 23:52:40 +02:00
Adrià Arrufat ae2359c1af script: fix segfault when a tool call triggers navigation
A press/click that submits a form or follows a link commits a
replacement Page inside the tool call itself (finalizeAction ->
awaitQueuedNavigation), freeing the Frame that
Session._tool_frame_override still pointed at; finalizeAction then
dereferenced it via requireFrame -> currentFrame. Any script whose
press/click navigated crashed on replay.

Store a frame id instead of a pointer and resolve it at every
currentFrame call. The replacement page keeps the frame id (see
commitPendingPage), so the same handle stays routable across the swap.
2026-07-03 23:52:29 +02:00
Adrià Arrufat ff7545a8c3 agent: add Google Vertex AI support
Supports both express mode (via VERTEX_API_KEY) and project mode
(via GOOGLE_CLOUD_PROJECT and gcloud auth). Handles automatic
OAuth token retrieval and refresh.
2026-07-03 20:23:51 +02:00
Karl Seguin f6fdec3300 Merge pull request #2876 from lightpanda-io/content-type-makrdown
handle text/makrdown as plain text
2026-07-03 22:46:48 +08:00
Karl Seguin 213126e08b Merge pull request #2875 from lightpanda-io/rm-100-continue
Disable Expect: 100-continue with curl
2026-07-03 22:24:30 +08:00
Karl Seguin 933dc61bf6 Merge pull request #2869 from lightpanda-io/closest_cached_selector
perf: used cachedParse for closest and waitForSelector
2026-07-03 21:18:23 +08:00
Pierre Tachoire c74f18acc2 handle text/makrdown as plain text 2026-07-03 13:25:31 +02:00
Karl Seguin 3f16812797 perf: Remove Seletor cache from Runner's waitForSelector
From feedback, using cachedParse in waitForSelector outside the loop risks
having the cached selector freed. For this reason, I'm reverting this part of
the previous commit. waitForSelector already pre-parsed the selector and I think
this is the better choice.

Pre-parsed means that we avoid the cache lookup in the [potentially] tight wait
loop. The advantage of hitting the cache inside the loop is only if
waitForSelector is called multiple times, which doesn't typically happen.
2026-07-03 19:14:41 +08:00
Pierre Tachoire e37e13b364 Disable Expect: 100-continue with curl
Curl sends by default an `Expect: 100-continue` header with some `POST`
requests. But it can slow down the whole request if the server doesn't
handle it correclty.
Forcing an `Expect:` header disable the curl's default behavior.

see https://everything.curl.dev/http/post/expect100.html or
https://gms.tf/when-curl-sends-100-continue.html
2026-07-03 11:42:35 +02:00
Karl Seguin aa3b30754d Merge pull request #2873 from lightpanda-io/tweak_parser_node_insert
refactor: Cleanup parser/non-parser Frame._insertNodeRelative
2026-07-03 17:00:22 +08:00
Karl Seguin 917a6b5176 Merge pull request #2871 from lightpanda-io/reclaim_large_ws_buffer
mem: reclaim large WS buffer
2026-07-03 17:00:00 +08:00
Karl Seguin 8c4219c2b5 Merge pull request #2870 from lightpanda-io/default_max_connections
perf: Increase default max connections
2026-07-03 16:59:42 +08:00
Karl Seguin bf4d4f7ede refactor: Cleanup parser/non-parser Frame._insertNodeRelative
_insertNodeRelative weaved the from_parser comptime through a different
conditions. It made reading the code difficult, but also optimizing it. This
commit extracts the common code (notifyChildInserted) and handles from_parser
in one cohesive chunk.

One benefit of this is that isConnected() is no longer called when from_parser
is true.
2026-07-03 13:24:30 +08:00
Karl Seguin 8a20823e78 Merge pull request #2868 from SyntaxSawdust/fix/charset-unicode-mojibake
Fix charset=unicode HTML prescan fallback
2026-07-03 11:41:32 +08:00
Karl Seguin 5a182d96f4 webapi: Add 6 more interfaces to Worker
CustomEvent, FileList, MessagePort, MessageChannel, Notification, ProgressEvent
2026-07-03 11:12:02 +08:00
Karl Seguin 23ab2df58e mem: reclaim large WS buffer
We'll grow the WS read buffer up to cdp-max-message-size (1MB default), but
never reclaim that space. A lot of drivers send large message upfront and then
settle into sending smaller ones.

This commit shrinks the buffer to 256KB (or cdp-max-message-size, whichever is
smaller) after 8 consecutive small message (tracking consecutive messages to
prevent a spat of allocation -> shink -> allocation -> shrink ->...)
2026-07-03 10:46:39 +08:00
Karl Seguin f1e49aae6e perf: Increase default max connections
max http connections 10 -> 40
max http connetions per host 4 -> 6

These are just the defaults and can still be adjusted by the command line
arguments. 6 appears to be both Chrome and FireFox's default per host (which
is probably the more important of the two settings).

The limits are really use-case specific. A use case that it multi-threading
different domains can benefit from a conservative max-host with a very large
max conn. A use case that is multi-threading the same host will need to decide
if it's safe to raise max-host.
2026-07-03 09:43:13 +08:00
Karl Seguin adf474774e perf: used cachedParse for closest and waitForSelector
closest was already hitting the cache, but it was doing it inside the loop.
Moving it outside the loop avoids the cache lookup.

Inversely, Runner.waitForSelector was parsing outside the loop, but it wasn't
caching. Repeated calls to waitForSelector with the same selector would not
leverage the cache.
2026-07-03 09:36:21 +08:00
Karl Seguin c5a5991b1c Merge pull request #2864 from lightpanda-io/cache-computed-style
cache the computed-style object per element
2026-07-03 08:28:22 +08:00
Karl Seguin c4d6c84cdb Merge pull request #2865 from lightpanda-io/network-idle-per-frame
frame: emit networkIdle lifecycle events for child frames
2026-07-03 08:19:53 +08:00
Karl Seguin 6708d5b961 Merge pull request #2862 from lightpanda-io/global_enumerable
webapi: Make all interfaces non-enumerable
2026-07-03 07:06:54 +08:00
Dustin Persek f8934f575b Fix charset unicode HTML prescan fallback 2026-07-02 17:43:04 -04:00
Pierre Tachoire ca0579413e Merge pull request #2837 from lightpanda-io/cache-simple-heuristic
Cache Simple Heuristic
2026-07-02 17:42:19 +02:00
Pierre Tachoire 656e2ee495 frame: emit networkIdle lifecycle events for child frames
Puppeteer's networkidle0 requires the networkIdle/networkAlmostIdle
lifecycle events on every frame that started loading, like Chrome
emits them — not just the root frame. Run the idle-notification
checks recursively over the frame tree in Runner._tick.

Fixes goto({waitUntil: 'networkidle0'}) timing out on pages with
iframes (e.g. reddit.com post pages).
2026-07-02 17:06:12 +02:00
Karl Seguin 53767d2dad Merge pull request #2852 from lightpanda-io/local_arena
mem: add local_arena
2026-07-02 22:46:11 +08:00
Pierre Tachoire 7e353927b9 cache the computed-style object per element
window.getComputedStyle allocated a CSSStyleProperties +
CSSStyleDeclaration pair per call; pages polling it grew the page
arena without bound. The computed variant is a stateless lazy view,
so hand out one per element from a frame-level map. This also matches
Chrome, where repeated calls return the identical object.
Pseudo-element requests keep the fresh-object path.
2026-07-02 16:33:58 +02:00
Karl Seguin 25d8b7d5c7 mem: add local_arena
The `call_arena` is currently our shortest-lived arena. Its promise is that it
will be valid for at least 1 v8 -> zig -> v8 function call, which makes it ideal
for getting values from v8 into zig, temporary work, and getting values from
zig to v8.

VBut `call_arena`'s lifetime is actually much longer. It's only reset when the
call_depth reaches 0. And the reason for that are Zig functions that invoke
v8 callbacks. If you just do it when a function ends, then if you allocate in
ZigA and then call CallbackA which calls ZigB, then when ZigB ends, your ZigA
allocation is cleared. This is something we could solve by reaching into
Zig's ArenaAllocator to capture a position to rollback from.

So, `call_arena` can end up living relatively long and accumulating quite a bit
of memory. But most calls _don't_ invoke callbacks. Hence, `local_arena` IS
reset at the end of every function.

Using `local_arena` _is_ dangerous, mostly for the case where some of our APIs
receive a *Frame (or *Execution) and thus might use a `local_arena` because they
know they aren't invoking a JS callback. BUT, those APIs might not know that
they're also invoked by some other Zig code that _could_ be invoking JS.

Dangerous? Sure. But, github has code that looks like:

```js
function onDelegatedClick(e) {
  for (const el of document.querySelectorAll('[data-action]')) {
    if (el.matches('.menu > .item:not(.disabled) a[href]')) {
      handle(el);
    }
    el.closest('.panel');
  }
}
```

Anything allocated in the `call_arena` will only be freed when the caller of
`onDelegatedClick` ends. The `querySelectorAll` returns hundreds of elements
and thus builds hundreds of parsed CSS and other scrap (x2 for `matches` and
closest`). `call_arena` peaks at 15MB. With the local_arena? 1MB. If 10x more
elements were returned, the peak would be 10x higher. With local_arena, it
stays 1MB.
2026-07-02 21:06:51 +08:00
Karl Seguin 3fbe460b04 Merge pull request #2861 from lightpanda-io/MessageEvent_getSource
crash, worker: Fix crash when MessageEvent.source is called from worker
2026-07-02 21:04:45 +08:00
Karl Seguin 0db0b89def Merge pull request #2858 from lightpanda-io/structuredClone
webapi: structuredClone for host (aka Zig) objects
2026-07-02 21:00:57 +08:00
Karl Seguin 69547db8ba webapi: Make all interfaces non-enumerable
When I added this, I was going through the list in /dom/interface-objects.html
thinking that was exhaustive. But no, no interfaces should be enumerable and
various other WPT tests (usually the idlharness ones) assert that for their
respective types.

Make it _always_ non enumerable means we no longer need Meta.enumerable to
be declared true/false (it's always false).
2026-07-02 20:54:33 +08:00
Karl Seguin 8f8df00b0f crash, worker: Fix crash when MessageEvent.source is called from worker
The source for a worker is always null. The getter cannot receive a *Frame since
it can be called from a Worker's context.
2026-07-02 19:28:09 +08:00
Pierre Tachoire 5fcfb4e344 Merge pull request #2859 from lightpanda-io/crypto-random-leak
use getrandom syscall for std.crypto.random
2026-07-02 12:03:30 +02:00
Pierre Tachoire 37a7f034c8 use writerStreaming in log 2026-07-02 11:23:45 +02:00
Pierre Tachoire e56a4f6259 use getrandom syscall for std.crypto.random
std.crypto.random's default backend mmaps a thread-local 528-byte state
page on first use and never unmaps it — there is no thread-exit hook.
With one detached thread per CDP connection (Server.handleConnection),
that leaks one resident page per connection (uuidv4 in
Page.getOrCreateOrigin touches it), ~4KB/conn of unbounded RSS growth.
Route every std.crypto.random call to the getrandom syscall instead.
.crypto_always_getrandom = true,
2026-07-02 11:19:53 +02:00
Halil Durak 60aacfa2b0 Merge pull request #2851 from lightpanda-io/nikneym/network-x509-store
networking: prefer `X509_STORE` instead of `ca_blob`
2026-07-02 09:10:15 +03:00
Halil Durak c86dad5430 libcrypto: remove unused utilities 2026-07-02 08:46:22 +03:00
Halil Durak 5e567adf1b ci: run serve command with TLS host verification disabled 2026-07-02 08:46:21 +03:00
Halil Durak fd0bff3af2 http: remove dead code 2026-07-02 08:46:21 +03:00