mirror/browser
1
0
Fork 0
mirror of https://github.com/lightpanda-io/browser.git synced 2026-06-11 01:25:53 -04:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2628 from lightpanda-io/agent-replace-interactive-load

Browse Source 
agent: replace -i flag with /save and /load commands
This commit is contained in:
Adrià Arrufat
2026-06-03 16:43:54 +02:00
committed by GitHub
parent 6bb912716d ac82551e66
commit efb0041c83
10 changed files with 316 additions and 574 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/agent-tutorial.md
View File

@@ -15,7 +15,7 @@ One session against Hacker News:
1. Log in with your account.
2. Confirm the login by reading the username out of the header.
3. Record the whole flow to a `.js` file.
3. Save the whole flow to a `.js` file.
4. Run it offline, with no LLM.
5. Add local JavaScript logic around `extract(...)` results.
6. Save the same flow as a script from an external agent over MCP.
@@ -280,18 +280,20 @@ The schema is parsed in Zig before the page-side walker runs, so a
typo like a stray comma surfaces here as `Error: invalid /extract
schema JSON` instead of a confusing V8 stack trace.
## 4. Recording the session
## 4. Saving the session
The same flow, but recorded to a file. Quit the REPL, then:
The same flow, but exported to a file. In the same REPL, retype the
sequence — login (`/goto`, two `/fill`s, `/click`, `/waitForSelector`),
then the front-page hop and structured pull (`/goto`, multi-line
`/extract`) — then save it:
```console
./lightpanda agent -i hn_login.js
```
> /save hn_login.js
```
`-i <path>` opens an interactive REPL that appends state-mutating
commands to `<path>`. Retype the same sequence — login (`/goto`, two
`/fill`s, `/click`, `/waitForSelector`), then the front-page hop and
structured pull (`/goto`, multi-line `/extract`) — then `/quit`.
In the basic REPL (`--no-llm`) `/save` transcribes the session
deterministically; with an LLM it synthesizes an equivalent idiomatic
script. Either way `/quit` when you're done.
Inspect the result:
@@ -300,7 +302,7 @@ cat hn_login.js
```
You should see the seven mutating commands and nothing else — no
`/tree`, no `/markdown`, no read-only lookups. The recorder filters on a
`/tree`, no `/markdown`, no read-only lookups. `/save` filters on a
per-tool flag (`ToolDef.recorded`) so read-only inspection never
pollutes the script; `/extract` *is* recorded (it changes what the
script can read on replay even though it doesn't mutate the page).
@@ -326,9 +328,10 @@ prompt is kept as a `//` comment above those actions.
./lightpanda agent hn_login.js
```
No `--provider`, no LLM, no token spend. The recorded script runs top
No `--provider`, no LLM, no token spend. The saved script runs top
to bottom against a fresh browser. This is the form you want for
regression tests and CI.
regression tests and CI. From inside the REPL, `/load hn_login.js`
runs the same script against the current session.
A JavaScript script does not print its final expression automatically.
The recorded `extract(...)` call returns a local JavaScript value, so
@@ -355,11 +358,10 @@ Run it again and stdout is clean JSON:
./lightpanda agent hn_login.js > stories.json
```
`/login` and `/acceptCookies` are REPL-only LLM triggers. A pure
recording from `-i` never contains them; the recorder captures the
resulting browser tool calls instead. Lines that are neither slash
commands nor comments are also REPL-only conveniences, not script
syntax.
`/login` and `/acceptCookies` are REPL-only LLM triggers. A script
saved with `/save` never contains them; `/save` captures the resulting
browser tool calls instead. Lines that are neither slash commands nor
comments are also REPL-only conveniences, not script syntax.
## 6. Local JavaScript logic
@@ -441,7 +443,7 @@ The `save` tool's description carries the same guidance the REPL's
`/save` gives its LLM (prefer builtins, drop dead-ends, keep `$LP_*`
placeholders), and any literal `LP_*` value is scrubbed back to its
placeholder before the file is written. The output uses the same
JavaScript format as `-i hn_login.js` from section 4 and runs
JavaScript format as `/save hn_login.js` from section 4 and runs
unmodified:
```console

28
docs/agent.md
View File

@@ -35,12 +35,9 @@ etc.) without giving Lightpanda its own API key.
# Basic REPL (no LLM, slash commands only)
./lightpanda agent --no-llm
# Run a recorded script
# Run a saved script, then exit
./lightpanda agent session.js
# Replay then continue interactively, appending new commands to the file
./lightpanda agent -i session.js
# One-shot: ask a question, capture the answer on stdout
./lightpanda agent --task "what is on the front page of hn?"
```
@@ -251,20 +248,19 @@ the script, and goes away when that Session does. There is no
cross-session persistence; if you need that, use `localStorage` (which
is now origin-scoped and persists across navigations within a session).
### Recording
### Saving and loading
Interactive sessions can write back to a `.js` file:
```console
./lightpanda agent -i session.js
```
From the REPL, `/save [file.js]` writes the session back to a `.js` file
and `/load <path>` runs a script from disk against the current session.
State-mutating commands (`/goto`, `/click`, `/fill`, `/scroll`, `/hover`,
`/selectOption`, `/setChecked`, `/waitForSelector`, `/press`, `/eval`,
`/extract`) are appended; read-only commands (`/tree`, `/markdown`,
`/extract`) are saved; read-only commands (`/tree`, `/markdown`,
`/links`, `/findElement`, …) and the natural-language turns that produced
them are not. Natural-language turns are recorded as `// <prompt>` comments
above the resulting JavaScript calls so the script stays readable.
them are not. Natural-language turns are saved as `// <prompt>` comments
above the resulting JavaScript calls so the script stays readable. In the
basic REPL (`--no-llm`) `/save` transcribes the session deterministically;
with an LLM it synthesizes an equivalent idiomatic script.
### JavaScript Script Running
@@ -298,8 +294,10 @@ See [agent-script.md](agent-script.md) for the full script format reference.
JSON schema), `/provider [name]` and `/model [name]` change the active
provider/model — Tab after the space completes from detected providers and
the provider's fetched model list, and bare `/provider`/`/model` print the
current selection — `/quit` exits the REPL, `/verbosity <low|medium|high>`
tunes the log level. These are REPL-only and never recorded.
current selection — `/save [file.js]` writes the session to a script and
`/load <path>` runs one from disk (Tab completes file paths), `/quit` exits
the REPL, `/verbosity <low|medium|high>` tunes the log level. These are
REPL-only and never recorded.
```
> /goto https://example.com
> /findElement role=button name=Submit

1
src/Config.zig
View File

@@ -228,7 +228,6 @@ const Commands = cli.Builder(.{
.{ .name = "model", .type = ?[:0]const u8 },
.{ .name = "base_url", .type = ?[:0]const u8 },
.{ .name = "system_prompt", .type = ?[:0]const u8 },
.{ .name = "interactive", .short = 'i', .type = bool },
.{ .name = "task", .type = ?[]const u8 },
.{ .name = "attach", .short = 'a', .type = []const u8, .multiple = true },
.{ .name = "verbosity", .type = ?AgentVerbosity },

106
src/agent/Agent.zig
View File

@@ -28,13 +28,13 @@ const Config = lp.Config;
const Command = lp.Command;
const Schema = lp.Schema;
const Recorder = lp.Recorder;
const ScriptRuntime = lp.Runtime;
const Credentials = zenai.provider.Credentials;
const App = @import("../App.zig");
const CDPNode = @import("../cdp/Node.zig");
const Terminal = @import("Terminal.zig");
const SlashCommand = @import("SlashCommand.zig");
const ScriptRuntime = @import("ScriptRuntime.zig");
const settings = @import("settings.zig");
const truncateUtf8 = @import("../string.zig").truncateUtf8;
@@ -101,8 +101,7 @@ browser: lp.Browser,
session: *lp.Session,
node_registry: CDPNode.Registry,
terminal: Terminal,
recorder: ?Recorder,
save_buffer: Recorder.Memory,
save_buffer: Recorder,
save_path: ?[]u8,
script_runtime_mutex: std.Thread.Mutex = .{},
active_script_runtime: ?*ScriptRuntime = null,
@@ -111,7 +110,6 @@ message_arena: std.heap.ArenaAllocator,
model: []u8,
system_prompt: []const u8,
script_file: ?[]const u8,
interactive: bool,
one_shot_task: ?[]const u8,
one_shot_attachments: ?[]const []const u8,
cancel_requested: std.atomic.Value(bool) = .init(false),
@@ -154,12 +152,6 @@ pub fn init(allocator: std.mem.Allocator, app: *App, opts: Config.Agent) !*Agent
});
return error.ConflictingFlags;
}
if (opts.task != null and opts.interactive) {
log.fatal(.app, "conflicting flags", .{
.hint = "--task is one-shot and exits; drop --interactive or drop --task",
});
return error.ConflictingFlags;
}
if (opts.no_llm and opts.provider != null) {
log.warn(.app, "ignoring --provider", .{ .reason = "--no-llm takes precedence" });
}
@@ -168,7 +160,7 @@ pub fn init(allocator: std.mem.Allocator, app: *App, opts: Config.Agent) !*Agent
}
const is_one_shot = opts.task != null;
const will_repl = !is_one_shot and (opts.interactive or opts.script_file == null);
const will_repl = !is_one_shot and opts.script_file == null;
// Basic-mode REPL (no LLM) must be opted into via --no-llm. Without it,
// the REPL accepts natural language and an absent API key would only
@@ -220,10 +212,6 @@ pub fn init(allocator: std.mem.Allocator, app: *App, opts: Config.Agent) !*Agent
const history_path: ?[:0]const u8 = if (will_repl) ".lp-history" else null;
// `-i <file>` means "run then grow this file"; a script path alone is
// a pure script run and must not be mutated.
const recorder_path: ?[]const u8 = if (opts.interactive) opts.script_file else null;
self.* = .{
.allocator = allocator,
.ai_client = null,
@@ -236,7 +224,6 @@ pub fn init(allocator: std.mem.Allocator, app: *App, opts: Config.Agent) !*Agent
.session = undefined,
.node_registry = .init(allocator),
.terminal = .init(allocator, history_path, Config.agentVerbosity(opts), will_repl),
.recorder = null,
.save_buffer = .init(allocator),
.save_path = null,
.messages = .empty,
@@ -244,7 +231,6 @@ pub fn init(allocator: std.mem.Allocator, app: *App, opts: Config.Agent) !*Agent
.model = model,
.system_prompt = opts.system_prompt orelse default_system_prompt,
.script_file = opts.script_file,
.interactive = opts.interactive,
.one_shot_task = opts.task,
.one_shot_attachments = if (opts.attach.items.len == 0) null else opts.attach.items,
.available_providers = available_providers,
@@ -276,21 +262,11 @@ pub fn init(allocator: std.mem.Allocator, app: *App, opts: Config.Agent) !*Agent
if (self.model_credentials != null) _ = completionModels(self, allocator);
}
if (recorder_path) |p| {
if (Recorder.init(allocator, std.fs.cwd(), p)) |r| {
self.recorder = r;
self.terminal.printInfo("recording to {s}", .{r.path});
} else |err| {
self.terminal.printError("recording disabled: {s}", .{@errorName(err)});
}
}
return self;
}
pub fn deinit(self: *Agent) void {
self.terminal.uninstallLogSink();
if (self.recorder) |*r| r.deinit();
self.save_buffer.deinit();
if (self.save_path) |p| self.allocator.free(p);
self.terminal.deinit();
@@ -404,8 +380,7 @@ pub fn run(self: *Agent) bool {
return ok;
}
if (self.script_file) |path| {
const script_ok = self.runScript(path);
if (!self.interactive) return script_ok;
return self.runScript(path);
}
self.runRepl();
return true;
@@ -509,7 +484,6 @@ fn runRepl(self: *Agent) void {
self.terminal.printError("{s}", .{result.text});
} else {
self.printData(result.text);
if (self.recorder) |*r| r.recordRaw(line);
self.recordSaveRaw(line);
}
continue :repl;
@@ -559,7 +533,6 @@ fn runRepl(self: *Agent) void {
self.terminal.endTool();
self.printCommandResult(cmd, result);
if (!result.is_error) {
if (self.recorder) |*r| r.record(cmd);
self.recordSaveCommand(cmd);
}
self.recordSlashToolCall(trimmed, tc.name(), tc.args, result) catch |err| {
@@ -581,6 +554,7 @@ fn handleMeta(self: *Agent, arena: std.mem.Allocator, meta: *const SlashCommand.
.help => self.printSlashHelp(arena, rest),
.verbosity => self.handleVerbosity(rest),
.save => self.handleSave(arena, rest),
.load => self.handleLoad(rest),
.model => self.handleModel(arena, rest),
.provider => self.handleProvider(arena, rest),
}
@@ -600,6 +574,15 @@ fn handleVerbosity(self: *Agent, rest: []const u8) void {
self.terminal.printInfo("verbosity: {s}", .{@tagName(level)});
}
fn handleLoad(self: *Agent, rest: []const u8) void {
const path = std.mem.trim(u8, rest, &std.ascii.whitespace);
if (path.len == 0) {
self.terminal.printError("usage: /load <path>", .{});
return;
}
_ = self.runScript(path);
}
const api_keys_hint = settings.api_keys_hint;
const llm_setup_hint = "drop --no-llm and set an API key (" ++ api_keys_hint ++ ")";
@@ -1050,6 +1033,10 @@ fn printSlashHelp(self: *Agent, arena: std.mem.Allocator, target: []const u8) vo
"/save [filename.js] [prompt] — save the session to [filename.js] (a random session-*.js if omitted). With an LLM, synthesizes an idiomatic script from the session and the optional prompt; with --no-llm, dumps the recorded actions verbatim.",
.{},
),
.load => self.terminal.printInfo(
"/load <path> — read a script from disk and run it against the current session; Tab completes file paths",
.{},
),
.model => self.terminal.printInfo(
"/model [name] — change the model; Tab completes the provider's models, bare /model shows the current one",
.{},
@@ -1117,9 +1104,25 @@ fn printData(self: *Agent, text: []const u8) void {
self.terminal.printAssistant(Terminal.reindentJson(arena.allocator(), text) orelse text);
}
fn runScript(self: *Agent, path: []const u8) bool {
self.terminal.printInfo("Running script: {s}", .{path});
/// Tracks whether a `/load`-run script has emitted any `console.*` output,
/// which decides how `runScript` ends: a script that printed nothing freezes
/// the spinner into a `/goto`-style bullet; one that printed leaves its
/// output as the result.
const ScriptOutput = struct {
terminal: *Terminal,
emitted: bool = false,
/// `Runtime.ConsoleObserver` callback: on the first line, clear the live
/// spinner so output starts clean instead of colliding with the indicator.
fn observe(context: *anyopaque) void {
const self: *ScriptOutput = @ptrCast(@alignCast(context));
if (self.emitted) return;
self.emitted = true;
self.terminal.endTool();
}
};
fn runScript(self: *Agent, path: []const u8) bool {
var script_arena: std.heap.ArenaAllocator = .init(self.allocator);
defer script_arena.deinit();
@@ -1145,7 +1148,13 @@ fn runScript(self: *Agent, path: []const u8) bool {
self.cancel_requested.store(false, .release);
}
if (runtime.runSource(content, path) catch |err| {
var output: ScriptOutput = .{ .terminal = &self.terminal };
runtime.console_observer = .{ .context = @ptrCast(&output), .notify = ScriptOutput.observe };
self.terminal.beginTool("script", path);
const result = runtime.runSource(content, path);
self.terminal.endTool();
if (result catch |err| {
self.terminal.printError("Script failed: {s}", .{@errorName(err)});
return false;
}) |message| {
@@ -1153,7 +1162,10 @@ fn runScript(self: *Agent, path: []const u8) bool {
return false;
}
self.terminal.printInfo("Script completed.", .{});
// A script that printed nothing leaves no trace, so freeze the spinner
// into a green bullet (like /goto); one that printed already showed its
// result.
if (!output.emitted) self.terminal.printScriptDone("script", path);
return true;
}
@@ -1345,14 +1357,7 @@ fn processUserMessage(self: *Agent, input: TurnInput) !?[]const u8 {
if (result.cancelled) return self.drainCancellation(msg_baseline);
const file_recorder: ?*Recorder = blk: {
if (self.recorder) |*r| {
if (r.isActive()) break :blk r;
}
break :blk null;
};
const record_to_memory = input.capture_for_save;
if (file_recorder != null or record_to_memory) {
if (input.capture_for_save) {
// When the LLM tries multiple `extract` schemas in one turn, only the
// last successful one is the answer — earlier probes are noise.
var last_extract_idx: ?usize = null;
@@ -1372,19 +1377,10 @@ fn processUserMessage(self: *Agent, input: TurnInput) !?[]const u8 {
const cmd = Command.fromToolCall(tool, args);
if (!cmd.isRecorded()) continue;
if (!recorded_any) {
if (input.record_comment) |c| {
if (file_recorder) |r| r.recordComment(c);
if (record_to_memory) self.recordSaveComment(c);
}
if (input.record_comment) |c| self.recordSaveComment(c);
recorded_any = true;
}
if (file_recorder) |r| r.record(cmd);
if (record_to_memory) self.recordSaveCommand(cmd);
}
if (file_recorder) |r| {
if (!r.isActive()) {
self.terminal.printError("recording disabled (write failed); see logs", .{});
}
self.recordSaveCommand(cmd);
}
}
@@ -1545,7 +1541,7 @@ pub fn listModels(allocator: std.mem.Allocator, opts: Config.Agent) !void {
});
return error.ConflictingFlags;
}
if (opts.task != null or opts.interactive or opts.script_file != null) {
if (opts.task != null or opts.script_file != null) {
log.fatal(.app, "list-models is exclusive", .{
.hint = "--list-models only takes --provider/--model/--base-url",
});

3
src/agent/SlashCommand.zig
View File

@@ -47,7 +47,7 @@ pub const MetaCommand = struct {
/// Dispatched by `Agent.handleMeta` via an exhaustive switch so adding
/// a new meta command is a compile error until it's wired up there too.
const Tag = enum { help, quit, verbosity, save, model, provider };
const Tag = enum { help, quit, verbosity, save, load, model, provider };
};
pub const meta_commands = [_]MetaCommand{
@@ -55,6 +55,7 @@ pub const meta_commands = [_]MetaCommand{
.{ .tag = .quit, .name = "quit", .hint = "", .values = &.{}, .description = "Exit the REPL" },
.{ .tag = .verbosity, .name = "verbosity", .hint = "<low|medium|high>", .values = &.{ "low", "medium", "high" }, .description = "Set agent verbosity" },
.{ .tag = .save, .name = "save", .hint = "[filename.js] [prompt]", .values = &.{}, .description = "Save this session to a file" },
.{ .tag = .load, .name = "load", .hint = "<path>", .values = &.{}, .description = "Load and run a script from disk" },
.{ .tag = .model, .name = "model", .hint = "[name]", .values = &.{}, .description = "Change the model" },
.{ .tag = .provider, .name = "provider", .hint = "[name]", .values = &.{}, .description = "Change the provider" },
};

68
src/agent/Terminal.zig
View File

@@ -344,6 +344,11 @@ fn addMetaValueCompletions(
if (std.mem.indexOfAny(u8, body, &std.ascii.whitespace) != null) return;
const prefix = input[0 .. input.len - body.len];
if (meta.tag == .load) {
addPathCompletions(cenv, input, body, prefix, buf);
return;
}
// `/provider` / `/model` candidates are resolved at runtime, not in `meta.values`.
if (self.completion_source) |src| switch (meta.tag) {
.provider, .model => {
@@ -362,6 +367,32 @@ fn addMetaValueCompletions(
for (meta.values) |v| addPrefixedCompletion(cenv, buf, input, prefix, v, "", body);
}
/// Completes `/load`'s argument against the filesystem. The directory part of
/// the partial path is kept verbatim in each candidate; the trailing basename
/// is matched against directory entries, and directories get a `/` suffix.
fn addPathCompletions(
cenv: ?*c.ic_completion_env_t,
input: []const u8,
body: []const u8,
prefix: []const u8,
buf: *[completion_buf_len:0]u8,
) void {
const slash = std.mem.lastIndexOfScalar(u8, body, '/');
const dir_part = if (slash) |i| body[0 .. i + 1] else "";
const open_path = if (dir_part.len == 0) "." else dir_part;
var dir = std.fs.cwd().openDir(open_path, .{ .iterate = true }) catch return;
defer dir.close();
var name_buf: [completion_buf_len]u8 = undefined;
var it = dir.iterate();
while (it.next() catch return) |entry| {
const suffix: []const u8 = if (entry.kind == .directory) "/" else "";
const full = std.fmt.bufPrint(&name_buf, "{s}{s}", .{ dir_part, entry.name }) catch continue;
addPrefixedCompletion(cenv, buf, input, prefix, full, suffix, body);
}
}
/// Completes `$LP_*` against the live process environment.
fn addEnvVarCompletions(
cenv: ?*c.ic_completion_env_t,
@@ -513,9 +544,31 @@ fn renderMetaHint(self: *Terminal, meta: *const SlashCommand.MetaCommand, body:
return writeHints(if (ends_ws) "" else " ", &frags);
}
if (ends_ws) return null;
if (meta.tag == .load) return ghostPathFirstMatch(body);
return ghostFirstMatch(meta.values, body, "");
}
/// Ghosts the first filesystem entry that completes the partial path `body`,
/// appending `/` when the match is a directory.
fn ghostPathFirstMatch(body: []const u8) [*c]const u8 {
const slash = std.mem.lastIndexOfScalar(u8, body, '/');
const dir_part = if (slash) |i| body[0 .. i + 1] else "";
const base = body[dir_part.len..];
const open_path = if (dir_part.len == 0) "." else dir_part;
var dir = std.fs.cwd().openDir(open_path, .{ .iterate = true }) catch return null;
defer dir.close();
var it = dir.iterate();
while (it.next() catch return null) |entry| {
if (!std.ascii.startsWithIgnoreCase(entry.name, base)) continue;
const suffix: []const u8 = if (entry.kind == .directory) "/" else "";
const text = std.fmt.bufPrintZ(&hint_buf, "{s}{s}", .{ entry.name[base.len..], suffix }) catch return null;
return text.ptr;
}
return null;
}
/// Ghosts `lead` + the suffix of the first `names` entry that prefix-matches
/// `body`.
fn ghostFirstMatch(names: []const []const u8, body: []const u8, lead: []const u8) [*c]const u8 {
@@ -1090,6 +1143,21 @@ pub fn printToolOutcome(self: *Terminal, name: []const u8, text: []const u8, is_
std.debug.print("{s}{s}[result: {s}]{s} {s}{s}\n", .{ ansi.dim, color, name, ansi.reset, truncated, ellipsis });
}
/// Freeze the script spinner into a green bullet for a `/load` run that
/// produced no output — mirrors a `/goto` outcome line, swapping the braille
/// glyph for a `●`. Only fires when the spinner was shown (REPL + TTY);
/// otherwise the run leaves just its own output.
pub fn printScriptDone(self: *Terminal, name: []const u8, args: []const u8) void {
if (!self.spinner.isEnabled()) return;
var buf: [256]u8 = undefined;
const line = std.fmt.bufPrint(
&buf,
ansi.green ++ "●" ++ ansi.reset ++ " " ++ ansi.dim ++ "[{s} {s}]" ++ ansi.reset ++ "\n",
.{ name, args },
) catch return;
_ = std.posix.write(std.posix.STDERR_FILENO, line) catch {};
}
/// Re-indents `text` as two-space JSON, or null when it isn't a JSON object/array.
/// The `{`/`[` sniff skips the parse for the common plain-text case — `text` may
/// be up to 1 MiB.

22
src/help.zon
View File

@@ -133,16 +133,13 @@
\\ {0s} agent --provider anthropic --model claude-sonnet-4-6
\\ {0s} agent --provider ollama --model qwen3.5:9b
\\ {0s} agent --no-llm (basic slash-command-only REPL)
\\ {0s} agent script.js (run a recorded script)
\\ {0s} agent -i script.js (run then drop into REPL,
\\ appending new commands to the file)
\\ {0s} agent script.js (run a saved script, then exit)
\\
\\Arguments:
\\[SCRIPT] Optional path to a .js script.
\\ Without -i: runs the script (no LLM calls).
\\ With -i: runs if present, then enters the REPL
\\ and appends new commands to the file (creating
\\ it if it does not yet exist).
\\[SCRIPT] Optional path to a .js script. Runs the script
\\ (no LLM calls) and exits. With no script and no
\\ --task, the REPL starts; from there /load runs a
\\ script and /save exports the session to a file.
\\ Caution: .js files can contain eval(...) calls
\\ that run arbitrary JavaScript in the page. Only run
\\ scripts you trust, the same way you would a shell
@@ -183,16 +180,9 @@
\\
\\--system-prompt <STRING> Override the default system prompt.
\\
\\-i, --interactive After running the positional script (if any),
\\ drop into the REPL with the browser state
\\ preserved. When a positional script is present,
\\ any new commands entered in the REPL are appended
\\ to that file.
\\ Conflicts with --task.
\\
\\--task <STRING> One-shot mode: run a single user turn, print the
\\ final answer to stdout, and exit. Conflicts with
\\ the positional script and with --interactive.
\\ the positional script.
\\
\\-a, --attach <PATH> Feed a local file to the model alongside --task.
\\ Repeatable, one file per flag. Text files are

2
src/lightpanda.zig
View File

@@ -49,11 +49,11 @@ pub const mcp = @import("mcp.zig");
pub const Agent = @import("agent/Agent.zig");
pub const Command = @import("script/command.zig").Command;
pub const Recorder = @import("script/Recorder.zig");
pub const Runtime = @import("script/Runtime.zig");
pub const Schema = @import("script/Schema.zig");
pub const cookies = @import("cookies.zig");
pub const build_config = @import("build_config");
pub const crash_handler = @import("crash_handler.zig");
pub const AgentScriptRuntime = @import("agent/ScriptRuntime.zig");
const IS_DEBUG = @import("builtin").mode == .Debug;

520
src/script/Recorder.zig
View File

@@ -16,101 +16,83 @@
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! In-memory recorder backing the REPL `/save` command: it filters, formats,
//! and scrubs the session's commands into a JavaScript buffer, leaving
//! persistence timing to the caller.
const std = @import("std");
const lp = @import("lightpanda");
const log = lp.log;
const testing = @import("../testing.zig");
const Command = @import("command.zig").Command;
const Recorder = @This();
allocator: std.mem.Allocator,
/// Open append-mode handle while recording is active. Becomes null when a
/// write fails mid-session and the recorder self-disables; `isActive()`
/// reflects this.
file: ?std.fs.File,
/// Path of the active recording, owned by the Recorder.
path: []const u8,
/// Number of lines successfully appended since init. Bumped only on success
/// so callers see the actual file line count, not the attempt count.
/// Number of lines appended since the last reset. Bumped only on success.
lines: u32,
/// Accumulated JavaScript, returned verbatim by `bytes()`.
content: std.Io.Writer.Allocating,
/// Reused between writes so each line doesn't alloc/free.
buf: std.Io.Writer.Allocating,
/// Reset per write — backs short-lived scrub allocations so the first
/// recorded command pays the page setup and the rest reuse the bump.
/// Reset per write — backs short-lived scrub allocations.
arena: std.heap.ArenaAllocator,
/// Append-open `sub_path` under `dir`, inserting a leading newline if the
/// file is non-empty.
pub fn init(allocator: std.mem.Allocator, dir: std.fs.Dir, sub_path: []const u8) !Recorder {
const owned_path = try allocator.dupe(u8, sub_path);
errdefer allocator.free(owned_path);
const file = try openForAppend(dir, sub_path);
pub fn init(allocator: std.mem.Allocator) Recorder {
return .{
.allocator = allocator,
.file = file,
.path = owned_path,
.lines = 0,
.content = .init(allocator),
.buf = .init(allocator),
.arena = .init(allocator),
};
}
fn openForAppend(dir: std.fs.Dir, sub_path: []const u8) !std.fs.File {
const f = try dir.createFile(sub_path, .{ .truncate = false });
errdefer f.close();
try f.seekFromEnd(0);
const pos = try f.getPos();
if (pos > 0) try f.writeAll("\n");
return f;
}
pub fn deinit(self: *Recorder) void {
self.content.deinit();
self.buf.deinit();
self.arena.deinit();
if (self.file) |f| f.close();
self.allocator.free(self.path);
}
pub fn isActive(self: *const Recorder) bool {
return self.file != null;
pub fn bytes(self: *Recorder) []const u8 {
return self.content.written();
}
pub fn record(self: *Recorder, cmd: Command) void {
if (self.file == null) return;
pub fn reset(self: *Recorder) void {
self.lines = 0;
self.content.clearRetainingCapacity();
self.buf.clearRetainingCapacity();
_ = self.arena.reset(.retain_capacity);
}
pub fn record(self: *Recorder, cmd: Command) !void {
if (!cmd.isRecorded()) return;
self.tryRecord(cmd) catch |err| self.disable(err);
}
fn tryRecord(self: *Recorder, cmd: Command) !void {
self.buf.clearRetainingCapacity();
_ = self.arena.reset(.retain_capacity);
try cmd.formatJs(self.arena.allocator(), &self.buf.writer);
try self.buf.writer.writeByte('\n');
try self.writeScrubbed();
try self.appendScrubbed();
}
pub fn recordComment(self: *Recorder, comment: []const u8) void {
if (self.file == null) return;
self.tryRecordComment(comment) catch |err| self.disable(err);
pub fn recordComment(self: *Recorder, comment: []const u8) !void {
self.buf.clearRetainingCapacity();
try writeCommentLines(&self.buf.writer, comment);
try self.appendScrubbed();
}
pub fn recordRaw(self: *Recorder, line: []const u8) void {
if (self.file == null) return;
self.tryRecordRaw(line) catch |err| self.disable(err);
}
fn tryRecordRaw(self: *Recorder, line: []const u8) !void {
pub fn recordRaw(self: *Recorder, line: []const u8) !void {
self.buf.clearRetainingCapacity();
try self.buf.writer.writeAll(line);
try self.buf.writer.writeByte('\n');
try self.writeScrubbed();
try self.appendScrubbed();
}
fn tryRecordComment(self: *Recorder, comment: []const u8) !void {
self.buf.clearRetainingCapacity();
try writeCommentLines(&self.buf.writer, comment);
try self.writeScrubbed();
fn appendScrubbed(self: *Recorder) !void {
// Reverse-substitute any LP_* env-var values that snuck in as literals
// (e.g. an agent that retyped a username it saw via getUrl) so the saved
// script stays portable instead of leaking the resolved secret.
_ = self.arena.reset(.retain_capacity);
const scrubbed = try lp.tools.reverseSubstituteEnvVars(self.arena.allocator(), self.buf.written());
try self.content.writer.writeAll(scrubbed);
self.lines += @intCast(std.mem.count(u8, scrubbed, "\n"));
}
/// Emit each line of `comment` as its own `// ` line, stripping lone CRs.
@@ -127,349 +109,49 @@ fn writeCommentLines(w: *std.Io.Writer, comment: []const u8) !void {
}
}
fn writeScrubbed(self: *Recorder) !void {
// Reverse-substitute any LP_* env-var values that snuck in as literals
// (e.g. an agent that retyped a username it saw via getUrl) so the
// recording stays portable instead of leaking the resolved secret.
// Propagate scrub OOM so the recorder disables itself rather than
// silently writing the unscrubbed buffer.
_ = self.arena.reset(.retain_capacity);
const scrubbed = try lp.tools.reverseSubstituteEnvVars(self.arena.allocator(), self.buf.written());
try self.file.?.writeAll(scrubbed);
self.lines += @intCast(std.mem.count(u8, scrubbed, "\n"));
}
/// Any failure along the record path — buffer-write OOM, scrub OOM, or file
/// write — flips the recorder to inactive so subsequent calls become silent
/// no-ops and `isActive()` reflects the stopped state.
fn disable(self: *Recorder, err: anyerror) void {
log.warn(.app, "recording disabled", .{ .err = @errorName(err) });
if (self.file) |f| {
f.close();
self.file = null;
}
}
/// In-memory recorder used by the REPL `/save` command. It intentionally
/// shares the same command filter/formatter/scrubber as file recording, but
/// leaves persistence timing to the caller.
pub const Memory = struct {
allocator: std.mem.Allocator,
lines: u32,
content: std.Io.Writer.Allocating,
buf: std.Io.Writer.Allocating,
arena: std.heap.ArenaAllocator,
pub fn init(allocator: std.mem.Allocator) Memory {
return .{
.allocator = allocator,
.lines = 0,
.content = .init(allocator),
.buf = .init(allocator),
.arena = .init(allocator),
};
}
pub fn deinit(self: *Memory) void {
self.content.deinit();
self.buf.deinit();
self.arena.deinit();
}
pub fn bytes(self: *Memory) []const u8 {
return self.content.written();
}
pub fn reset(self: *Memory) void {
self.lines = 0;
self.content.clearRetainingCapacity();
self.buf.clearRetainingCapacity();
_ = self.arena.reset(.retain_capacity);
}
pub fn record(self: *Memory, cmd: Command) !void {
if (!cmd.isRecorded()) return;
self.buf.clearRetainingCapacity();
_ = self.arena.reset(.retain_capacity);
try cmd.formatJs(self.arena.allocator(), &self.buf.writer);
try self.buf.writer.writeByte('\n');
try self.appendScrubbed();
}
pub fn recordComment(self: *Memory, comment: []const u8) !void {
self.buf.clearRetainingCapacity();
try writeCommentLines(&self.buf.writer, comment);
try self.appendScrubbed();
}
pub fn recordRaw(self: *Memory, line: []const u8) !void {
self.buf.clearRetainingCapacity();
try self.buf.writer.writeAll(line);
try self.buf.writer.writeByte('\n');
try self.appendScrubbed();
}
fn appendScrubbed(self: *Memory) !void {
_ = self.arena.reset(.retain_capacity);
const scrubbed = try lp.tools.reverseSubstituteEnvVars(self.arena.allocator(), self.buf.written());
try self.content.writer.writeAll(scrubbed);
self.lines += @intCast(std.mem.count(u8, scrubbed, "\n"));
}
};
fn parseLine(arena: std.mem.Allocator, line: []const u8) Command {
return Command.parse(arena, line) catch unreachable;
}
test "record writes state-mutating commands" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "test.js");
defer recorder.deinit();
recorder.record(parseLine(aa, "/goto https://example.com"));
recorder.record(parseLine(aa, "/click selector='Login'"));
recorder.record(parseLine(aa, "/tree"));
recorder.record(parseLine(aa, "/waitForSelector '.dashboard'"));
recorder.record(parseLine(aa, "/markdown"));
recorder.record(parseLine(aa, "/scroll y=200"));
recorder.record(parseLine(aa, "/hover selector='#menu'"));
recorder.record(parseLine(aa, "/selectOption selector='#country' value='France'"));
recorder.record(parseLine(aa, "/setChecked selector='#agree'"));
recorder.record(parseLine(aa, "/setChecked selector='#newsletter' checked=false"));
recorder.record(parseLine(aa, "/extract '{\"title\":\".title\"}'"));
recorder.recordComment("LOGIN");
const file = tmp.dir.openFile("test.js", .{}) catch unreachable;
defer file.close();
var buf: [512]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
const content = buf[0..n];
try std.testing.expect(std.mem.indexOf(u8, content, "goto(\"https://example.com\");\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "click({ selector: \"Login\" });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "waitForSelector(\".dashboard\");\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "scroll({ y: 200 });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "hover({ selector: \"#menu\" });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "selectOption({ selector: \"#country\", value: \"France\" });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "setChecked({ selector: \"#agree\" });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "setChecked({ selector: \"#newsletter\", checked: false });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "extract({ title: \".title\" });\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "\n// LOGIN\n") != null);
// Read-only tools (tree, markdown) are gated out by isRecorded().
try std.testing.expect(std.mem.indexOf(u8, content, "tree(") == null);
try std.testing.expect(std.mem.indexOf(u8, content, "markdown(") == null);
}
test "recordRaw writes the JS line verbatim" {
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "raw.js");
defer recorder.deinit();
recorder.recordRaw("document.title");
recorder.recordRaw("window.scrollTo(0, 100)");
const file = tmp.dir.openFile("raw.js", .{}) catch unreachable;
defer file.close();
var buf: [256]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
try std.testing.expectEqualStrings("document.title\nwindow.scrollTo(0, 100)\n", buf[0..n]);
}
test "record skips empty and comment lines" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "test2.js");
defer recorder.deinit();
recorder.record(parseLine(aa, ""));
recorder.record(parseLine(aa, " "));
recorder.record(parseLine(aa, "# this is a comment"));
recorder.record(parseLine(aa, "/goto https://example.com"));
const file = tmp.dir.openFile("test2.js", .{}) catch unreachable;
defer file.close();
var buf: [256]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
const content = buf[0..n];
try std.testing.expectEqualStrings("goto(\"https://example.com\");\n", content);
}
test "lines counter tracks successful appends" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "count.js");
defer recorder.deinit();
recorder.record(parseLine(aa, "/goto https://example.com")); // +1
recorder.record(parseLine(aa, "/tree")); // skipped — not isRecorded()
recorder.record(parseLine(aa, "/click selector='Login'")); // +1
recorder.recordComment("a note"); // +1
try std.testing.expectEqual(@as(u32, 3), recorder.lines);
}
test "init appends to an existing file without truncating" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
// Seed a file with a prior line.
{
const seed = tmp.dir.createFile("script.js", .{}) catch unreachable;
defer seed.close();
_ = seed.writeAll("goto(\"https://example.com\");\n") catch unreachable;
}
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "script.js");
defer recorder.deinit();
recorder.record(parseLine(aa, "/click selector='Login'"));
try std.testing.expect(recorder.isActive());
try std.testing.expectEqualStrings("script.js", recorder.path);
const file = tmp.dir.openFile("script.js", .{}) catch unreachable;
defer file.close();
var buf: [256]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
const content = buf[0..n];
try std.testing.expect(std.mem.indexOf(u8, content, "goto(\"https://example.com\");\n") != null);
try std.testing.expect(std.mem.indexOf(u8, content, "click({ selector: \"Login\" });\n") != null);
// The prior line must precede the appended line.
const prior = std.mem.indexOf(u8, content, "goto").?;
const appended = std.mem.indexOf(u8, content, "click").?;
try std.testing.expect(prior < appended);
}
extern fn setenv(name: [*:0]u8, value: [*:0]u8, override: c_int) c_int;
extern fn unsetenv(name: [*:0]u8) c_int;
test "recordComment scrubs literal LP_* values back to placeholders" {
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
const var_name = "LP_RECORDER_COMMENT_TEST";
const var_value = "topsecret";
_ = setenv(@constCast(var_name), @constCast(var_value), 1);
defer _ = unsetenv(@constCast(var_name));
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "scrub.js");
defer recorder.deinit();
recorder.recordComment("a user noted that their password is topsecret");
const file = tmp.dir.openFile("scrub.js", .{}) catch unreachable;
defer file.close();
var buf: [256]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
try std.testing.expectEqualStrings(
"// a user noted that their password is $LP_RECORDER_COMMENT_TEST\n",
buf[0..n],
);
}
test "recordComment splits embedded newlines into separate comment lines" {
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "multi.js");
defer recorder.deinit();
// An attacker-controlled comment trying to smuggle a command must not
// produce an executable line on replay.
recorder.recordComment("note\n/goto https://attacker\r\nmore");
const file = tmp.dir.openFile("multi.js", .{}) catch unreachable;
defer file.close();
var buf: [256]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
try std.testing.expectEqualStrings(
"// note\n// /goto https://attacker\n// more\n",
buf[0..n],
);
}
test "record disables recorder on write failure" {
const filter: testing.LogFilter = .init(&.{.app});
defer filter.deinit();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
// Open the file read-only so writeAll fails with `error.NotOpenForWriting`.
// Struct literal (not `init`) because only this test needs to inject a
// read-only handle to exercise the failure path.
const file = blk: {
_ = tmp.dir.createFile("ro.js", .{}) catch unreachable;
break :blk tmp.dir.openFile("ro.js", .{ .mode = .read_only }) catch unreachable;
};
var recorder: Recorder = .{
.allocator = std.testing.allocator,
.file = file,
.path = try std.testing.allocator.dupe(u8, "test.js"),
.lines = 0,
.buf = .init(std.testing.allocator),
.arena = .init(std.testing.allocator),
};
defer recorder.deinit();
var test_arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer test_arena.deinit();
const aa = test_arena.allocator();
try std.testing.expect(recorder.isActive());
recorder.record(parseLine(aa, "/goto https://example.com"));
try std.testing.expect(!recorder.isActive());
try std.testing.expectEqual(@as(u32, 0), recorder.lines);
// Subsequent calls are silent no-ops, not silent successes.
recorder.record(parseLine(aa, "/click selector='Login'"));
recorder.recordComment("note");
try std.testing.expectEqual(@as(u32, 0), recorder.lines);
}
test "init creates the file if missing" {
test "record filters state-mutating commands and comments" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder: Recorder = try .init(std.testing.allocator, tmp.dir, "fresh.js");
var recorder: Recorder = .init(std.testing.allocator);
defer recorder.deinit();
recorder.record(parseLine(aa, "/goto https://example.com"));
const file = tmp.dir.openFile("fresh.js", .{}) catch unreachable;
defer file.close();
var buf: [128]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
try std.testing.expectEqualStrings("goto(\"https://example.com\");\n", buf[0..n]);
try recorder.record(parseLine(aa, "/goto https://example.com"));
try recorder.record(parseLine(aa, "/tree"));
try recorder.record(parseLine(aa, "/click selector='Login'"));
try recorder.recordComment("search for login");
try std.testing.expectEqualStrings(
"goto(\"https://example.com\");\nclick({ selector: \"Login\" });\n// search for login\n",
recorder.bytes(),
);
try std.testing.expectEqual(@as(u32, 3), recorder.lines);
recorder.reset();
try std.testing.expectEqualStrings("", recorder.bytes());
try std.testing.expectEqual(@as(u32, 0), recorder.lines);
try recorder.record(parseLine(aa, "/scroll y=200"));
try std.testing.expectEqualStrings("scroll({ y: 200 });\n", recorder.bytes());
try std.testing.expectEqual(@as(u32, 1), recorder.lines);
}
test "recordRaw writes the JS line verbatim" {
var recorder: Recorder = .init(std.testing.allocator);
defer recorder.deinit();
try recorder.recordRaw("document.title");
try recorder.recordRaw("window.scrollTo(0, 100)");
try std.testing.expectEqualStrings("document.title\nwindow.scrollTo(0, 100)\n", recorder.bytes());
}
test "record emits multi-line extract as JavaScript" {
@@ -477,54 +159,50 @@ test "record emits multi-line extract as JavaScript" {
defer arena.deinit();
const aa = arena.allocator();
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
var recorder = try Recorder.init(std.testing.allocator, tmp.dir, "triple.js");
var recorder: Recorder = .init(std.testing.allocator);
defer recorder.deinit();
const cmd_str = "/extract '{\n \"title\": \"span.title\",\n \"desc\": \"p.description\"\n}'";
recorder.record(parseLine(aa, cmd_str));
try recorder.record(parseLine(aa, cmd_str));
const file = tmp.dir.openFile("triple.js", .{}) catch unreachable;
defer file.close();
var buf: [512]u8 = undefined;
const n = file.readAll(&buf) catch unreachable;
try std.testing.expectEqualStrings(
"extract({ title: \"span.title\", desc: \"p.description\" });\n",
buf[0..n],
recorder.bytes(),
);
}
test "memory recorder mirrors file recorder filtering" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
test "recordComment splits embedded newlines into separate comment lines" {
var recorder: Recorder = .init(std.testing.allocator);
defer recorder.deinit();
var memory: Memory = .init(std.testing.allocator);
defer memory.deinit();
try memory.record(parseLine(aa, "/goto https://example.com"));
try memory.record(parseLine(aa, "/tree"));
try memory.record(parseLine(aa, "/click selector='Login'"));
try memory.recordComment("search for login");
// An attacker-controlled comment trying to smuggle a command must not
// produce an executable line on replay.
try recorder.recordComment("note\n/goto https://attacker\r\nmore");
try std.testing.expectEqualStrings(
"goto(\"https://example.com\");\nclick({ selector: \"Login\" });\n// search for login\n",
memory.bytes(),
"// note\n// /goto https://attacker\n// more\n",
recorder.bytes(),
);
try std.testing.expectEqual(@as(u32, 3), memory.lines);
memory.reset();
try std.testing.expectEqualStrings("", memory.bytes());
try std.testing.expectEqual(@as(u32, 0), memory.lines);
try memory.record(parseLine(aa, "/scroll y=200"));
try std.testing.expectEqualStrings("scroll({ y: 200 });\n", memory.bytes());
try std.testing.expectEqual(@as(u32, 1), memory.lines);
}
test "memory recorder scrubs literal LP_* values in JavaScript calls" {
test "recordComment scrubs literal LP_* values back to placeholders" {
const var_name = "LP_RECORDER_COMMENT_TEST";
const var_value = "topsecret";
_ = setenv(@constCast(var_name), @constCast(var_value), 1);
defer _ = unsetenv(@constCast(var_name));
var recorder: Recorder = .init(std.testing.allocator);
defer recorder.deinit();
try recorder.recordComment("a user noted that their password is topsecret");
try std.testing.expectEqualStrings(
"// a user noted that their password is $LP_RECORDER_COMMENT_TEST\n",
recorder.bytes(),
);
}
test "record scrubs literal LP_* values in JavaScript calls" {
var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
defer arena.deinit();
const aa = arena.allocator();
@@ -534,12 +212,12 @@ test "memory recorder scrubs literal LP_* values in JavaScript calls" {
_ = setenv(@constCast(var_name), @constCast(var_value), 1);
defer _ = unsetenv(@constCast(var_name));
var memory: Memory = .init(std.testing.allocator);
defer memory.deinit();
var recorder: Recorder = .init(std.testing.allocator);
defer recorder.deinit();
try memory.record(parseLine(aa, "/fill selector='#user' value='secret-user'"));
try recorder.record(parseLine(aa, "/fill selector='#user' value='secret-user'"));
try std.testing.expectEqualStrings(
"fill({ selector: \"#user\", value: \"$LP_RECORDER_COMMAND_TEST\" });\n",
memory.bytes(),
recorder.bytes(),
);
}

102
src/agent/ScriptRuntime.zig → src/script/Runtime.zig
View File

@@ -25,7 +25,7 @@ const CDPNode = @import("../cdp/Node.zig");
const v8 = lp.js.v8;
const ScriptRuntime = @This();
const Runtime = @This();
allocator: std.mem.Allocator,
app: *lp.App,
@@ -37,6 +37,10 @@ has_context: bool,
call_arena: std.heap.ArenaAllocator,
primitive_data: [recorded_tool_count]PrimitiveData,
console_data: [std.enums.values(ConsoleMethod).len]ConsoleData,
/// Notified before each `console.*` line is written. The REPL uses it to
/// clear the live spinner so script output starts on a clean line instead
/// of colliding with the indicator; the line still goes to stdout/stderr.
console_observer: ?ConsoleObserver = null,
/// The runtime installs exactly the recorded browser tools as script
/// primitives — the same set the recorder writes — so every recorded call
@@ -51,7 +55,7 @@ const recorded_tool_count = blk: {
};
const PrimitiveData = struct {
runtime: *ScriptRuntime,
runtime: *Runtime,
tool: BrowserTool,
};
@@ -71,10 +75,15 @@ const ConsoleMethod = enum {
};
const ConsoleData = struct {
runtime: *ScriptRuntime,
runtime: *Runtime,
method: ConsoleMethod,
};
pub const ConsoleObserver = struct {
context: *anyopaque,
notify: *const fn (context: *anyopaque) void,
};
pub const InitError = error{
OutOfMemory,
RuntimeInitFailed,
@@ -90,8 +99,8 @@ pub fn init(
app: *lp.App,
session: *lp.Session,
registry: *CDPNode.Registry,
) InitError!*ScriptRuntime {
const self = try allocator.create(ScriptRuntime);
) InitError!*Runtime {
const self = try allocator.create(Runtime);
errdefer allocator.destroy(self);
self.* = .{
@@ -119,7 +128,7 @@ pub fn init(
return self;
}
pub fn deinit(self: *ScriptRuntime) void {
pub fn deinit(self: *Runtime) void {
self.resetContext();
self.env.deinit();
self.call_arena.deinit();
@@ -127,15 +136,15 @@ pub fn deinit(self: *ScriptRuntime) void {
allocator.destroy(self);
}
pub fn terminate(self: *ScriptRuntime) void {
pub fn terminate(self: *Runtime) void {
self.env.terminate();
}
pub fn cancelTerminate(self: *ScriptRuntime) void {
pub fn cancelTerminate(self: *Runtime) void {
self.env.cancelTerminate();
}
fn createContext(self: *ScriptRuntime) InitError!void {
fn createContext(self: *Runtime) InitError!void {
var hs: lp.js.HandleScope = undefined;
hs.init(self.env.isolate);
defer hs.deinit();
@@ -159,7 +168,7 @@ fn createContext(self: *ScriptRuntime) InitError!void {
try self.installConsole(context, global);
}
fn resetContext(self: *ScriptRuntime) void {
fn resetContext(self: *Runtime) void {
if (!self.has_context) return;
v8.v8__Global__Reset(&self.context);
self.env.isolate.notifyContextDisposed();
@@ -167,7 +176,7 @@ fn resetContext(self: *ScriptRuntime) void {
}
fn installPrimitive(
self: *ScriptRuntime,
self: *Runtime,
context: *const v8.Context,
global: *const v8.Object,
name: []const u8,
@@ -188,7 +197,7 @@ fn installPrimitive(
}
fn installConsole(
self: *ScriptRuntime,
self: *Runtime,
context: *const v8.Context,
global: *const v8.Object,
) InitError!void {
@@ -207,7 +216,7 @@ fn installConsole(
}
fn setObjectProperty(
self: *ScriptRuntime,
self: *Runtime,
context: *const v8.Context,
object: *const v8.Object,
name: []const u8,
@@ -227,7 +236,7 @@ fn setObjectProperty(
/// Run script source in the agent context. Returns null on success; on a JS
/// compile/runtime exception returns a formatted error allocated in this
/// runtime's call arena and valid until deinit or the next run.
pub fn runSource(self: *ScriptRuntime, source: []const u8, name: []const u8) RunError!?[]const u8 {
pub fn runSource(self: *Runtime, source: []const u8, name: []const u8) RunError!?[]const u8 {
_ = self.call_arena.reset(.retain_capacity);
var hs: lp.js.HandleScope = undefined;
@@ -286,7 +295,7 @@ fn consoleCallback(info_handle: ?*const v8.FunctionCallbackInfo) callconv(.c) vo
data.runtime.invokeConsole(data.method, info);
}
fn invoke(self: *ScriptRuntime, tool: BrowserTool, info: *const v8.FunctionCallbackInfo) void {
fn invoke(self: *Runtime, tool: BrowserTool, info: *const v8.FunctionCallbackInfo) void {
// Owned, not shared: marshalling runs JS (`toJSON`) that can re-enter a
// primitive; a shared arena would let the nested call reset ours mid-flight.
var arena_state: std.heap.ArenaAllocator = .init(self.allocator);
@@ -322,7 +331,7 @@ fn invoke(self: *ScriptRuntime, tool: BrowserTool, info: *const v8.FunctionCallb
}
}
fn invokeConsole(self: *ScriptRuntime, method: ConsoleMethod, info: *const v8.FunctionCallbackInfo) void {
fn invokeConsole(self: *Runtime, method: ConsoleMethod, info: *const v8.FunctionCallbackInfo) void {
// Owned arena (see `invoke`): an argument's `toString` can re-enter a
// primitive mid-loop and must not reset the buffer we're accumulating.
var arena_state: std.heap.ArenaAllocator = .init(self.allocator);
@@ -343,7 +352,8 @@ fn invokeConsole(self: *ScriptRuntime, method: ConsoleMethod, info: *const v8.Fu
self.writeConsoleLine(method, aw.written());
}
fn writeConsoleLine(_: *ScriptRuntime, method: ConsoleMethod, line: []const u8) void {
fn writeConsoleLine(self: *Runtime, method: ConsoleMethod, line: []const u8) void {
if (self.console_observer) |obs| obs.notify(obs.context);
var buf: [4096]u8 = undefined;
var file = if (method.writesStderr()) std.fs.File.stderr() else std.fs.File.stdout();
var writer = file.writer(&buf);
@@ -357,7 +367,7 @@ const PrimitiveResult = union(enum) {
};
fn callTool(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
tool: BrowserTool,
args: ?std.json.Value,
@@ -382,7 +392,7 @@ const BuildArgsError = error{
};
fn buildArgs(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
tool: BrowserTool,
@@ -409,7 +419,7 @@ fn buildArgs(
}
fn singleStringOrObject(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
info: *const v8.FunctionCallbackInfo,
@@ -426,7 +436,7 @@ fn singleStringOrObject(
}
fn singleObject(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
info: *const v8.FunctionCallbackInfo,
@@ -439,7 +449,7 @@ fn singleObject(
}
fn extractArgs(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
info: *const v8.FunctionCallbackInfo,
@@ -476,7 +486,7 @@ fn normalizeExtractSchemaString(arena: std.mem.Allocator, schema: []const u8) er
}
fn argJson(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
info: *const v8.FunctionCallbackInfo,
@@ -487,7 +497,7 @@ fn argJson(
}
fn valueToJson(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
value: *const v8.Value,
@@ -504,7 +514,7 @@ fn objectWith(arena: std.mem.Allocator, key: []const u8, value: std.json.Value)
return .{ .object = obj };
}
fn normalizeExtractReturnJson(_: *ScriptRuntime, arena: std.mem.Allocator, value: []const u8) error{OutOfMemory}![]const u8 {
fn normalizeExtractReturnJson(_: *Runtime, arena: std.mem.Allocator, value: []const u8) error{OutOfMemory}![]const u8 {
if (value.len == 0) return value;
const parsed = std.json.parseFromSliceLeaky(std.json.Value, arena, value, .{}) catch |err| switch (err) {
@@ -519,11 +529,11 @@ fn normalizeExtractReturnJson(_: *ScriptRuntime, arena: std.mem.Allocator, value
return try std.json.Stringify.valueAlloc(arena, entry.value_ptr.*, .{});
}
fn setReturnString(self: *ScriptRuntime, info: *const v8.FunctionCallbackInfo, value: []const u8) void {
fn setReturnString(self: *Runtime, info: *const v8.FunctionCallbackInfo, value: []const u8) void {
self.setReturnValue(info, @ptrCast(self.env.isolate.initStringHandle(value)));
}
fn setReturnJson(self: *ScriptRuntime, context: *const v8.Context, info: *const v8.FunctionCallbackInfo, value: []const u8) void {
fn setReturnJson(self: *Runtime, context: *const v8.Context, info: *const v8.FunctionCallbackInfo, value: []const u8) void {
if (value.len == 0) {
self.setReturnValue(info, self.env.isolate.initUndefined());
return;
@@ -536,22 +546,22 @@ fn setReturnJson(self: *ScriptRuntime, context: *const v8.Context, info: *const
self.setReturnValue(info, parsed);
}
fn setReturnValue(_: *ScriptRuntime, info: *const v8.FunctionCallbackInfo, value: *const v8.Value) void {
fn setReturnValue(_: *Runtime, info: *const v8.FunctionCallbackInfo, value: *const v8.Value) void {
var rv: v8.ReturnValue = undefined;
v8.v8__FunctionCallbackInfo__GetReturnValue(info, &rv);
v8.v8__ReturnValue__Set(rv, value);
}
fn throwError(self: *ScriptRuntime, message: []const u8) void {
fn throwError(self: *Runtime, message: []const u8) void {
_ = v8.v8__Isolate__ThrowException(self.env.isolate.handle, self.env.isolate.createError(message));
}
fn throwTypeError(self: *ScriptRuntime, message: []const u8) void {
fn throwTypeError(self: *Runtime, message: []const u8) void {
_ = v8.v8__Isolate__ThrowException(self.env.isolate.handle, self.env.isolate.createTypeError(message));
}
fn formatCaught(
self: *ScriptRuntime,
self: *Runtime,
context: *const v8.Context,
try_catch: *const v8.TryCatch,
fallback: []const u8,
@@ -579,7 +589,7 @@ fn formatCaught(
}
fn valueToString(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
context: *const v8.Context,
value: *const v8.Value,
@@ -589,7 +599,7 @@ fn valueToString(
}
fn stringToOwned(
self: *ScriptRuntime,
self: *Runtime,
arena: std.mem.Allocator,
string: *const v8.String,
) error{OutOfMemory}![]const u8 {
@@ -605,20 +615,20 @@ fn stringToOwned(
return buf[0..written];
}
fn dupeError(self: *ScriptRuntime, message: []const u8) RunError![]const u8 {
fn dupeError(self: *Runtime, message: []const u8) RunError![]const u8 {
return self.call_arena.allocator().dupe(u8, message) catch error.OutOfMemory;
}
const testing = @import("../testing.zig");
fn runTestScript(runtime: *ScriptRuntime, source: []const u8) !void {
fn runTestScript(runtime: *Runtime, source: []const u8) !void {
if (try runtime.runSource(source, "agent-runtime-test.js")) |message| {
std.debug.print("agent script failed:\n{s}\n", .{message});
return error.AgentScriptFailed;
}
}
fn terminateRuntimeSoon(runtime: *ScriptRuntime) void {
fn terminateRuntimeSoon(runtime: *Runtime) void {
std.Thread.sleep(10 * std.time.ns_per_ms);
runtime.terminate();
}
@@ -630,7 +640,7 @@ test "agent script runtime: goto and eval dispatch through browser tools" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -651,7 +661,7 @@ test "agent script runtime: extract returns a JavaScript object" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -705,7 +715,7 @@ test "agent script runtime: strict-mode scripts can call primitives" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -723,7 +733,7 @@ test "agent script runtime: promise microtasks run to completion" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -744,7 +754,7 @@ test "agent script runtime: primitives re-entered from argument callbacks stay i
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -766,7 +776,7 @@ test "agent script runtime: terminate interrupts local JavaScript" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
const thread = try std.Thread.spawn(.{}, terminateRuntimeSoon, .{runtime});
@@ -784,7 +794,7 @@ test "agent script runtime: agent variables persist and page globals are isolate
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -809,7 +819,7 @@ test "agent script runtime: page eval cannot see agent primitives or bindings" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -827,7 +837,7 @@ test "agent script runtime: console is available in agent context" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
try runTestScript(runtime,
@@ -844,7 +854,7 @@ test "agent script runtime: tool errors throw and stop execution" {
var registry = CDPNode.Registry.init(testing.allocator);
defer registry.deinit();
const runtime = try ScriptRuntime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
const runtime = try Runtime.init(testing.allocator, testing.test_app, testing.test_session, &registry);
defer runtime.deinit();
const message = (try runtime.runSource(