Each virtual time jump also moves the offset on V8's wall clock, through
the fork's v8__Platform__SetClockOffsetMillis, so Date.now() keeps pace
with the timers it drives. Pins zig-v8-fork at lightpanda-io/zig-v8-fork#203;
the prebuilt archive needs a tag bump once that lands.
Opt-in --virtual-time-budget-ms: when the page is idle except for pending
timers, jump the page clock to the next scheduler task instead of sleeping,
up to a per-navigation budget, then fall back to real time. Timers fire in
order and in-flight transfers are never skipped. Timers, performance.now,
event timestamps, resource timing, the network-idle hold and the
intersection burst guard read the page clock; the watchdog, HTTP client,
rate limiter and server keep the boot clock. Date.now stays real until the
V8 platform clock is offset too. Serve is unaffected.
main added custom property (`--*`) tracking to the same rule-building code
this branch made appendable. Both paths now share addSelectorRules, which
registers the custom declarations of a rule and its visibility rules, and
reports whether it added anything tracked. An insertRule append therefore
reaches custom_rules too, and clears the lazily parsed selectors of a
property it joins, instead of being dropped when the rule sets no
visibility property.
The raw-text path folds a block once for both kinds of declaration, so a
rebuild does not tokenize every block twice. rule_layers lives in the rule
arena so an appended rule can register its layer; the layered ranks are
still stamped by finalizeLayerRanks.
ScriptManager, XMLHttpRequest.zig, Fetch, Workers, etc. all take ownership (aka
dupe) the HTTP response from HTTPClient. They all have a headerCallback that
does something like:
```zig
if (transfer.getContentLength()) |cl| {
try self.body.ensureTotalCapacity(self.arena, cl);
}
```
But in all non-streaming cases (which is most cases), the HttpClient buffers
the response and only calls the headerCallback _after_ the body has been
received. Rather than relying on "Content-Length" header, the body buffer can
be sized to the exact body length. Why does this matter? Because the
Content-Length is the length of the body on the wire, and if the body is
compressed (like almost all .js files are), it will under-report the final
body length AND, because most callers are using an arena, the buffer growth
will retain more memory than it should.
This adds a `transfer.bodyLen()` method. Callers which dupe the body now use
this rather than the Content-Length (Content-Length is still used, e.g. for
XHR progress report).
Visibility probes no longer decide whether an element's style attribute
gets parsed into a CSSStyleProperties object. The three layout readers
(getElementAxis, positionStyle, horizontalPosition) create it on demand
through Element.inlineStyle, and StyleManager only ever folds the
attribute text. That removes the scan/materialize mode threaded through
every probe, and a JS layout read now only materializes the elements
whose inline style it actually reads.
The v8 documentation says this "Has to be called", and claude flagged it as a
potential leak.
Required a new zig-v8-fork build (v0.5.5) which I released.
Adds a `clutter` option to --strip-mode. This is based on readability.js. It
isn't a direct port (e.g. it doesn't strop bylines). It fallsback to `shell` if
it strips too much (and shell itself can fallback to not stripping anything).
But clutter rarely fallback to shell, only when a page is very small or when
it strips out _a lot_.
Also expanded shell to look at class names and ids.
Both MO and IO assumed an orderly transfer of its pending records to v8. But,
failure to transfer these objects to v8 (e.g. if the terminate flag was set)
means the records and their .tiny arenas were lost forever.
Some additional API changes:
- Add strip-mode support to pdf/png generation.
- Add LP.dump which provides greater content gathering capability to CDP,
exposing most `fetch` dump-related parameters (e.g. format, strip, selector,
...)
The new "--strip-mode shell" is designed to try to remove non-content elements
such as the header and footer. The end goal is to use readibility.js test cases
as a baseline, but this isn't a port of readibility.js.
This is just the basic implementation of this, e.g removing a few key tags, e.g.
<header>, <footer> and considering some specific roles.
Even if --strip-mode shell is used, we might decide to stick with a whole dump:
it's better to strip not enough than to strip too much. This currently works by
measuring the ratio of non-link text of the stripped vs unstripped page.
Debug and release builds each compiled their own copy of every C
dependency and of the Rust staticlib, because build.zig threaded the
top-level optimize mode into all of them. Under dev_fast the deps also
picked up the bundled-CRT target query, so even the same mode could not
share objects with a plain build.
Dependencies now build in ReleaseFast for the requested target, the way
the prebuilt V8 archive already works. Debug and release builds share
one set of cached dependency objects, and debug binaries run TLS, HTML
parsing, regex and sqlite optimized. -Ddebug_deps restores the old
behaviour for stepping into a dependency.
The Rust staticlib can only be shared by dropping the Debug-only memstats
feature: its single export, html5ever_get_memory_usage, was declared on
the Zig side but never called, and it pulled a jemalloc build into every
cold debug build. The Makefile override that existed for jemalloc's
nested make goes with it.
tighten socket ownership (on error paths)
allow reaper to be disabled
Handle window where link is being destroyed, worker is still alive, and client
attempts to re-link.
This is a small step towards WebDriver supports (non-bidi). It allows creating
and deleting a BiDi "Session" (e.g. a worker). It also allows attaching a BiDi
driver to an HTTP-created BiDi session (the typical selenium startup flow).
This change unblocks the most basic setup/teardown of Selenium, so it still
isn't enough to actually use a Selenium script as-is. But it's significant
because it models a worker (thread) that isn't tied to a WebSocket, something we
haven't had before.
A consequence of a pure HTTP Session is that we don't have a clear cleanup
signal. There is no "the socket is disconnected". There's a new HTTP reaper
which kills HTTP Sessions after --http-session-timeout. It's expected that
drivers properly DELETE /session/:id. I imagine we're going to run into
--cdp-max-connections limits and need to tweak this code. BUT, this entire flow
is only enabled with --protocol webdriver, so it won't impact exiting CDP users.
Three parameters every Playwright and Stagehand session sends were parsed
and then only logged as not implemented:
- Page.addScriptToEvaluateOnNewDocument runImmediately now also evaluates
the script in the current document, in the requested world.
- Emulation.setDeviceMetricsOverride screenWidth/screenHeight now back
window.screen, kept on the viewport override next to width/height; 0
keeps the current value, as for the other dimensions.
- Browser.setDownloadBehavior browserContextId is checked against the
loaded context, as the Storage commands already do.
Drivers focus a node before typing (chromedp's SendKeys calls DOM.focus,
then Input.dispatchKeyEvent). The method was unknown, so the keystrokes
went to the previously active element and every chromedp form fill was a
no-op. Resolve the node like the other DOM commands and call Element.focus,
which already handles focusability and the blur/focus event sequence.
An alternation or a repeat keeps only whether its text may start and
end with a token character, and read that off one marker. An optional
non-token stretch there (`\/?x`) was taken as a definite non-token,
so `\/ads(\/?x|\/y)` was filed under "ads" while `/adsx` carries no
such token. What follows the stretch answers now.
Attempt to improve https://github.com/lightpanda-io/browser/issues/3199
Meant an leaner alternative to https://github.com/lightpanda-io/browser/pull/3440
3440 is close, but it adds 16 bytes to every VisibilityRule which isn't ideal
especially since the majority of these rules have no custom properties.
Let's make a few reasonable assumptions:
1 - Most custom properties don't have many distinct selectors
(--background-color might be defined on, :root, and .card and a few others)
2 - Most custom properties aren't queried from JS (Tailwind can define thousands
of custom properties, but they're used by the rendering engine, not from JS)
3 - Most selectors don't have custom properties
The design is to inverse what we do for visibility: lookup per property. We end
up with a property-name -> [(selector, value), (selector, value)] lookup. If
there's a query for --foreground, we O(1) to get the list of (selector, value)
and then iterate through this (hopefully) short list to get the value.
PLUS, we only parse the selector on the first query. So for those sites with
thousands of custom-properties, we're not wastefully storing / parsing the
selector on every build.
With the StyleManager memo, a VisibilityCache/PointerEventsCache hit
costs the same hash probe as a memo hit, so the caches only added a
second probe per ancestor, a call_arena allocation per element, and
plumbing through SemanticTree, AXNode, CDP, links, ResizeObserver and
elementFromPoint. checkVisibilityCached becomes isVisible.
Adrià Arrufat