This feature is significant because it adds support for processing an HTTP
request via the worker. It requires parking the connection and then having the
worker notify the loop when the response is ready. A lot of this was already
in-place (e.g. worker -> loop notification) but not quite do this extent.
This is a small step towards WebDriver supports (non-bidi). It allows creating
and deleting a BiDi "Session" (e.g. a worker). It also allows attaching a BiDi
driver to an HTTP-created BiDi session (the typical selenium startup flow).
This change unblocks the most basic setup/teardown of Selenium, so it still
isn't enough to actually use a Selenium script as-is. But it's significant
because it models a worker (thread) that isn't tied to a WebSocket, something we
haven't had before.
A consequence of a pure HTTP Session is that we don't have a clear cleanup
signal. There is no "the socket is disconnected". There's a new HTTP reaper
which kills HTTP Sessions after --http-session-timeout. It's expected that
drivers properly DELETE /session/:id. I imagine we're going to run into
--cdp-max-connections limits and need to tweak this code. BUT, this entire flow
is only enabled with --protocol webdriver, so it won't impact exiting CDP users.
Currently, the HttpClient owns the inbox and its borrowed by the Link. This is
a bit backwards, but it also means that we can't eagerly create a Link: the
Link needs the inbox, so it needs the HttpClient, which is created by the
Browser (which creates an Isolate).
Remember, the Inbox is one of the few things shared between the main thread
and the worker, so either end can own it and the other can borrow it.
This switches the ownership so that the HttpClient now borrows the Inbox from
the Server's side of the Link (the WebSocket).
The main goal of this change is to prepare for more advanced HTTP WebDriver
flows. The more we can create _without_ a Browser, the fewer edge cases we have
to deal with (Browser because it's expensive and has to be created on the
Worker thread due to how V8::Isolate works).
In order to support Selenium the way people are used to, it looks like we need
to support both WebDriver classic (WebDriver) and WebDriver BiDi (BiDi). Typical
scripts look like a mix of the two, e.g. using WebDriver to control the browser
and using BiDi to receive notifications. This commit:
1 - adds a --protocol (cdp|webdriver) CLI argument to the `serve` command to
enable one or the other protocol (defaulting to CDP)
2 - adds basic WebDriver endpoint to let a Selenium client connect. This
implementation is hackish and sits on top of our simple Handshake handler.
The handshake handler is well past its original design. Serving /json/version
and /metrics from it was one thing. But Driving the entire browser session? This
will get a follow up PR.
1. Abstract "Driver". Non-CDP things that referenced *CDP now reference a Driver
2. Move the NodeRegistry out of CDP. This created an artificial link between
agent / mcp and CDP
3. Add BiDi driver with enough to navigate to a page
Karl Seguin