Significant rework of the CDP/BiDi server. There are two main changes:
1 - poll replaced with EPoll/Kqueue (1)
2 - make http serving a first class citizen
The change from poll -> epoll/kqueue isn't performance driven, it's just about
tighter code. Both epoll and kqueue let you associate arbitrary data with a
socket, so we don't need to keep arrays in sync in order to associate a socket
with a CDP by index. They both provide some event/notification mechanism, which
is cleaner than the pipe required by poll.
The poll -> epoll/kqueue change could almost have been mechanical. Making HTTP
a first class citizen is the more significant of the two changes
In `main`, a new connection always spawns a thread and, until does its own
little read loop until the connection is upgraded. This is not efficient, it
uses up a connection slot, and it's inconsistent with the final WebSocket
connection which _is_ polled off the main loop. Using up a slot means that
keepalive isn't possible, else HTTP connections would quickly use up all
available slots/threads.
This commit parses and serves HTTP requests on the main thread (safe
because none of the processing is blocking). The approach is better streamlined
for HTTP requests which never upgrade (/metrics, WebDriver) without causing
any performance overhead for those that do. It simplifies some things (e.g. an
"http" socket or a "websocket" socket is monitored and read in a similar manner
(on the main loop)). It makes other things more complicated; the flow is no
longer accept -> spawn -> upgrade -> websocket loop. It's loop -> accept -> loop
-> process -> (http | ws).
This is built ontop of the BiDi branch because (a) WebDriver is what needs
better HTTP support and (b) some of the more mechanical changes already exist
in that branch (e.g. src/cdp/, src/server.zig -> src/server/*)
(1) kqueue landing in 2 commits from now on this branch.
Both globals were missing, so MDN-canonical stream setup code threw
ReferenceError. size() is a method rather than an accessor returning a
function; the streams don't consume it yet.
CURLMOPT_MAXCONNECTS was never set, so libcurl used its default of 4x the
number of easy handles currently attached to the multi. Handles are added
and removed per transfer, so between page loads that default collapses to
roughly zero and every cached connection is evicted: revisiting a host
after browsing elsewhere re-paid connect + TLS every time.
Measured over 25 navigations across 5 sites (5 rounds, ReleaseFast), median
warm navigation drops from 0.73s to 0.42s on news.ycombinator.com and from
0.71s to 0.42s on github.com; total navigation time 11.1s -> 7.3s. Loading
one host repeatedly was already fast and is unchanged; the win is on
cross-site browsing, which is what agents and crawlers actually do.
If you look at https://github.com/lightpanda-io/browser/pull/3293, you'll see
a relatively contained change that has to touch over 20 files. The issue is that
every HttpClient.newRequest needs to provide a lot of data. But `newRequest`
takes a 2nd parameter: the HttpClient.Owner. If we make that Owner a little
smarter, we can start to remove some of the individual fields needed in
newRequest. For example, we can still allow a callsite to pass frame_id but,
by default, we can use the owner's frame_id (which is what we want in most
cases).
Also updates matchCookies/onCookieChanged from the hard-coded "same-site + navigation" to areSameSite(exec.siteForCookies(), host) and is_navigation=false.
The site for cookies were computed from the immediate parent `Frame`, which would allow sending a cookie that's `SameSite=Strict` from 2 levels deep under. Directly from RFC6265bis, this PR essentially implements (except for step 4, we skip host-less ancestors):
Given a Document (document), the following algorithm returns its
"site for cookies":
1. Let top-document be the active document in document's navigable's
top-level traversable.
2. Let top-origin be the origin of top-document's URI if top-
document's sandboxed origin browsing context flag is set, and
top-document's origin otherwise.
3. Let documents be a list consisting of the active documents of
document's inclusive ancestor navigables.
4. For each item in documents:
1. Let origin be the origin of item's URI if item's sandboxed
origin browsing context flag is set, and item's origin
otherwise.
2. If origin is not same-site with top-origin, return an origin
set to an opaque origin.
5. Return top-origin.
Checked the three test files in Firefox and Chrome:
- stepUp/stepDown use the HTML step base (min, else the value attribute)
and snap off-ladder values to the next rung, counting the snap as the
first step as browsers do; clamping lands on the last rung inside
min/max.
- time strings keep a three-digit fraction.
- an empty pattern attribute is a pattern (matches only "").
- tooLong/tooShort only for values last changed by a user edit, so the
text-entry path marks the value and script/attribute values never trip
them; same for textarea.
- showPicker dropped: browsers throw NotAllowedError without a gesture,
a no-op would be a lie.
- month/week assertions skipped where the browser has no such input.
measureText was missing and roundRect, ellipse, setLineDash, gradients,
patterns, isPointInPath and the shadow/filter properties threw, so
Chart.js, D3 and ApexCharts died at init. Widths come from the
deterministic text model in text_measure.zig; nothing is painted.
https://github.com/lightpanda-io/browser/pull/3333 improved markdown rendering
for flexbox items. But screenshot generation didn't benefit from that change.
This commit introduces a RenderTree that extracts and shares functionality
between markdown and screenshots.
react-hook-form and date pickers read valueAsNumber/valueAsDate and got
undefined. The date types convert through the existing validators; the
same conversion now also drives min/max range validation for
date/time/month/week/datetime-local, which only number/range had.
click(), willValidate and the WebDriver click read only the control's own
attribute. They now use Element.isDisabled, which already walks fieldset
ancestors with the first-legend exception for :disabled. The disabled IDL
attribute keeps reflecting the attribute alone, per spec.
Three one-accessor gaps: getSetCookie returns Set-Cookie values
unjoined, getName maps a constructor back to its tag, part is a
DOMTokenList over the part attribute like relList.
Karl Seguin