`lightpanda agent <script>` replays a saved script token-free (no LLM),
but running it under `agent` reads wrong. Add `run` as a dedicated,
script-only command: `lightpanda run <script>`.
`run` exposes only a required [SCRIPT] positional plus the common
options; its LLM-session flags (--task/--save/--list-models/REPL) are
absent, so they're rejected at parse time. `parseArgs` normalizes a
`run` invocation to an `.agent` command, reusing the existing token-free
replay path unchanged, so nothing downstream needs a `.run` case.
`agent <script>` keeps working. Telemetry records `run` as its own mode
code (R) via a new Config.command field, so its adoption is measurable
separately from agent script replay (AR).
Idea here is to skip re-parsing that happen for each connection; we already use BoringSSL, so we can take more advantage of it by directly mutating cert store of `SSL_CTX`.
Remove sid. Include iid in every message. Booleans true/false => 1/0. Constant
string values => single letter. Example:
["8800df58-a5d5-4ca5-9a06-d7691a2a3780","H","fetch",0,"macos","aarch64","1.0.0-dev.7609+88b1bc671"]
["8800df58-a5d5-4ca5-9a06-d7691a2a3780","R"]
["8800df58-a5d5-4ca5-9a06-d7691a2a3780","N",1,"P"]
H => Header
R => Run
N => Navigate
B => Buffer Overflow / dropped
L => LLM
Navigate context are
P => Page
O => Open (popup)
I => Iframe
First, this adds 1 small piece of data to the navigate event: whether the
navigate was a page, frame or popup.
It also adds a session id, but as far as I'm concerned, this isn't "new"
information, or any new tracking/insight into users. Between the iid and the
"run" event, a "session" was always trackable. By giving it an explicit value,
we can shrink the size of all other messes.
This change reduces the telemetry payload by ~70% (despite the extra nav field).
I'm hoping this might remove a reason some people would consider turning it off.
It hits a /v2/ endpoint. The changes:
1 - A header is the first message in a session and contains all of the static
data, as well as a session id
2 - Every event is encoded as an array, [$SID, "event-type, params...]
```
{"sid":"92e98210a141f497","iid":"$UUID","mode":"fetch","os":"macos","arch":"aarch64","version":"$VERSION","proxy":false}
["92e98210a141f497","run"]
["92e98210a141f497","nav",false,"page"]
```
(the driver=cdp field was removed from nav, because it was always cdp).
Some notes for the server:
1 - The server can tell a header from an event based on the first character.
2 - The SID is 8 bytes, enough to be unique, but not globally unique. The
iid + sid + time window is how a events for the same SID can be grouped.
3 - A valid event is always an array of 2+ items, index 0 = SID, index 1 = type
Although positional data isn't expressive, it's still extendable.
The 4 events:
run, no parameters (mostly just used to flush the header now)
["sid", "run"]
// nav, tls, page/popup/iframe
["sid","nav",true,"page"]
// bof, # of lost telemetry events
["sid","bof",42]
// llm, provider, model (nullable)
["sid","llm","anthropic","claude"]
This reverts recent(ish) changes to telemetry which moved it from its own thread
onto the main thread.
The downside is: we have an extra thread.
The upside is largely that Network.zig becomes drastically simpler and more
efficient. There's a bunch of machinery in Network.zig to support arbitrary
workers, of which Telemetry is the only one. There's also a lot of code to
support an optional multi and requests made to is. This is all removed.
Also, fetch, agent and mcp without a cdp server no longer even need to start
the network loop. And, it IS started (e.g. serve/cdp), there's no longer an
arbitrary 250ms wakeup on poll to progress workers. Nor can telemtry block CDP.
Telemetry's implementation itself was changed. The ring buffer was removed in
favor of a double-buffer arraylist. When telemetry is disabled, this saves
64Kb of memory. When it's enabled, it creates more allocator churn, but should
still use less memory in most cases (and never more). Finally, Telemetry is
given its own easy connection rather than using one out of the pool (which
workers would maybe like to use).
@import("lightpanda") where needed.
Would also like to do this for String, Page, Session and js which all stand out
as types that are use across the codebase.
I know that a few devs are doing this in new work and I haven't heard anyone
voice an objection.
Removes manual git flags from CI and build scripts.
Versioning is now automatically derived from git and build.zig.zon.
With this PR, we follow https://semver.org/
Logic:
1. Read the version from build.zig.zon
2. If it doesn't have a `.pre` field (i.e. dev/alpha/beta) it will use that
3. Otherwise it will get the info from git: hash and number of commits since last `.0` version
4. Then build the version: `0.3.0-dev.1493+0896edc3`
Note that, since the latest stable version is `0.2.6`.
The convention is to use `0.3.0-dev`, as:
- `0.2.6` < `0.3.0.dev` < `0.3.0`
This adds a crash handler which reports a crash (if telemetry is enabled). On a
crash, this looks for `curl` (using the PATH env), and forks the process to then
call execve. This relies on a new endpoint to be setup to accept the "report".
Also, we include very little data..I figured just knowing about crashes would
be a good place to start.
A panic handler is provided, which override's Zig default handler and hooks
into the crash handler.
An `assert` function is added and hooks into the crash handler. This is
currently only used in one place (Session.zig) to demonstrate its use. In
addition to reporting a failed assert, the assert aborts execution in
ReleaseFast (as opposed to an undefined behavior with std.debug.assert).
I want to hook this into the v8 global error handler, but only after direct_v8
is merged.
Much of this is inspired by bun's code. They have their own assert (1) and
a [more sophisticated] crashHandler (2).
:
(1) https://github.com/oven-sh/bun/blob/beccd01647ec4fcb5895be5c5fb88f5254dc1e8d/src/bun.zig#L2987
(2) https://github.com/oven-sh/bun/blob/beccd01647ec4fcb5895be5c5fb88f5254dc1e8d/src/crash_handler.zig#L198
Removes optional platform, which only existed for tests.
There is now a global `@import("testing.zig").test_app` available. This is setup
when the test runner starts, and cleaned up at the end of tests. Individual
tests don't have to worry about creating app, which I assume was the reason I
Platform optional, since that woul dhave been something else that needed to be
setup.
Outputs in logfmt in release and a "pretty" print in debug mode. The format
along with the log level will become arguments to the binary at some point in
the future.
1 - Add a log_level build option to control the default log level from
the build (e.g. -Dlog_level=debug). Defaults to info
2 - Add a new boolean log_unknown_properties build option to enable
logging unknown properties. Defautls to false.
3 - Remove the log debug for script eval - this can be a huge value
(i.e. hundreds of KB), which makes the debug log unusable IMO.
Replaces the existing, very specialized Notification with something more
general.
Currently, the existing page_navigate and page_navigated have been migrated.
Telemetry's page navigation event now also hooks into these events to generate
the telemetry record.
Adrià Arrufat