Review feedback: markdown caps its own output through Opts.max_bytes while
the html dump left it to each caller, so the html tool and fetch each
carried a copy of the LimitedWriter wrapping. root and deep now do it,
with a no-cap fast path so innerHTML/outerHTML are unaffected.
A wait failure on one URL aborted the whole multi-URL fetch with no
output, the wait_ms budget was computed once before the loop so N pages
could take N times the budget, navigation failures exited 0 with an empty
dump, and HTTP 4xx/5xx were indistinguishable from success by exit code.
Each page now has its own error slot: wait, navigation and dump failures
are recorded there instead of returned, every page is still written (the
JSON envelope carries the name under "error"), then the first failure is
returned so the process exits 1. The remaining budget is recomputed per
page. --fail-on-http-error turns a status >= 400 into exit 22, curl's
code for the same condition; the dump is written first either way.
--strip-mode was silently ignored for --dump markdown, and fetch had no
way to scope a dump to one element or cap its size although the markdown
and html tools have both.
--selector dumps the first matching element in any dump mode, --max-bytes
caps html and markdown with the tools' truncation marker, and markdown
now honors strip ui (images; scripts, styles and hidden elements are
never rendered, so the other groups don't apply).
The links tool returned every a[href] verbatim: hidden nav entries,
one row per duplicate href, and null text for image and icon links even
though markdown already knows to use alt. Nav-heavy pages produced
thousands of rows for an agent looking for one link.
One entry per resolved href (the first, upgraded with text from a later
duplicate), hidden anchors skipped with the same check tree uses, text
falling back to aria-label, title, then a descendant img alt. The tool
gains an optional limit.
Our `Arena` from the `ArenaPool` now tracks if it's already been released. On
a subsequent release, it panics then and there. Without this, the code will
almost certainly panic anyways, but it will panic in a seemingly unrelated
place. Hopefully this makes identifying future cases of this easier (since we'll
get the stack trace of the re-releaser).
Fix 3 separate memory issues, all edge cases.
1 - an XHR error handler that re-open/sends would incorrectly null the new
transfer (maybe this isn't that odd, maybe it's a common retry-on-error).
2 - On a curl_easy_pause error (from WebSocket.zig) we now unqueue the just-
queued message, because the error will errdefer the message arena to be
cleaned up.
3 - ScriptManager now cleans up after itself on a failure prior to submit()
being called.
A worker gets a distinct Navigator type (WorkerNavigator) which only exposes
a subset of what Navigator does.
Previously, we achieved this by exposing Navigator but with a bunch of accessors
/ functions with .{.exposed = .window}. That worked fine, except the type name
was still "Navigator". This introduces an actual WorkerNavigator.
The Page now tracks the element currently being hovered. We need to track this
so that subsequent mouse events can fire the correct events. This is largely a
WebDriver (WPT) change, but it's also triggered from CDP's
`input.dispatchMouseEvent`.
So, for example, if CDP `dispatchMouseEvent` with a mouseMoved to element1
we'll fire a mouseenter. If the same command is issue for the same element, we
should NOT trigger a mouseout and mouseenter.
This has been on my TODO for a long time. https://github.com/lightpanda-io/browser/pull/3238
added support for the @@iterator protocol and now URLSearchParams' init can
use it.
We used to only support a v8::Array, but now any type that implements the
@@iterator protocol can be passed into URLSearchParams.
This also fixes and simplifies the js.Object iterator. It now yields name +
value (so callers don't need to get the name then lookup the value). But it
isn't just about making it easier to use. The name can be a lossy UTF-8
conversion, so the value lookup can fail. By internalizing the value lookup we
can use the v8::String directly for the lookup.
Fixes a handful of WPT /url/ cases
Review feedback: navigation writes cookies and storage and a wait lets page
scripts mutate state, so neither can be auto-approved as read-only.
consoleLogs drains its buffer, so repeated calls differ.
Since all children of an invisible element are skipped, the visibility cache
serves no purpose AND we can assume that any element we do visit has a visible
parent and thus just need to check the element's own visibility.
interactiveElements and findElement returned elements under display:none
or [hidden], so agents were offered controls no user could reach and the
list disagreed with tree, which already filters on visibility. Reuse the
same cached check and skip the hidden subtree.
StyleManager ultimately ends up calling el.getOrCreateStyle() which either
returns the element's CSSStyleProperties OR (creates it AND stores it in the
Frame._element_styles for future lookups).
The goal behind this caching is twofold:
1 - Performance of not having to reparse the "style" attribute
2 - Identity: two calls from JS to get the properties should return the same
value
(2) is non-negotiable, so the 'getOrCreate' _has_ to exist for JS-facing APIs.
But (1) is CPU vs memory optimization that we've decided should always favor the
CPU. But, in any case where we dump an entire tree, that memory cost can be
significant (# of elements with a style attribute) and the CPU gains are
questionable (it isn't like a JS loop re-checking an element's properties, it's
a one-time dump). So, the StyleManager now takes a comptime `InlineAccess` which
is either `.scan` or `.materialize`. When it's `.materialize` it behaves as
before. When it's `.scan` is will use an existing `_element_styles` if available
else it will re-parse but not store the value.
The renderer's only visibility test was the tag (metadata/svg), so
display:none, [hidden], aria-hidden="true", closed <details> content and
closed <dialog>s all ended up in the output.
Check each element's own computed display:none through StyleManager plus
aria-hidden="true"; the dump root is exempt so a scoped dump of a hidden
subtree still renders it. analyzeContent/isStandaloneAnchor use the same
predicate so an anchor whose only content is hidden falls back to its label.
Also adds the missing dialog:not([open]) { display: none } UA rule.
Karl Seguin