browser/docs/agent-script.md

Agent JavaScript scripts

lightpanda agent <script.js> runs a JavaScript file that drives a Lightpanda browser session through a small object-oriented API. The format is intentionally plain JavaScript: use normal variables, functions, loops, objects, arrays, JSON.parse, JSON.stringify, and other standard ECMAScript built-ins.

./lightpanda agent session.js

The interactive REPL still uses slash commands. /save writes the recorded browser actions as JavaScript calls so the saved file can be replayed without an LLM.

Runtime Environment

Agent scripts run in their own V8 context. That context is separate from the web page's JavaScript context.

• Page is the only global. new Page() makes a page object and await page.goto(url) navigates it; every other primitive is a method on that page: const page = new Page(); await page.goto(url); page.extract({...}); page.click(sel);.
• It is not the browser page environment. There is no window, document, DOM, localStorage, navigator, or page global state in the agent script. Read page data with page.extract(...), or explicitly run page JavaScript with page.evaluate(...) when that is the right tool.
• It is not Node.js. There is no require, process, fs, path, npm package loading, command-line argument API, or Node network/filesystem API.
• Page scripts cannot see agent variables or Lightpanda primitives. Agent scripts cannot directly see page variables.
• The page.evaluate(...) method runs JavaScript in the page context, distinct from the agent context's own native eval.
• Agent variables persist for the lifetime of one script run, across navigations and primitive calls. A later lightpanda agent script.js run starts with a fresh agent context.
• page.goto(...) is async — always await it. Every other page method is synchronous: write const data = page.extract({...}), never await page.extract(...). The script body runs as an async function, so top-level await is allowed.
• Re-navigating reuses the same page. await page.goto(url2) keeps page valid and points it at the new URL. Work through one page at a time and read it before navigating away.
• Tool failures throw JavaScript Error exceptions and stop execution unless you catch them.
• return <value> is the script's output, printed automatically (objects and arrays as JSON; other values coerced). End a script with the value you want as output, e.g. return page.extract({ ... }); or return results;. A bare trailing expression is NOT printed, and neither is console.log(JSON.stringify(...)).

The agent context includes a small console object:

console.log("printed to stdout");
console.info("printed to stdout");
console.debug("printed to stdout");
console.warn("printed to stderr");
console.error("printed to stderr");

Values And Return Types

Most page methods return the browser tool's result text as a JavaScript string. page.extract(...) is the exception: it returns extracted data as a normal JavaScript value, so local script logic can use it directly. The result mirrors your schema — an object schema returns an object keyed by your fields (even with a single field), and a bare array schema returns an array:

const page = new Page();
await page.goto("https://news.ycombinator.com/");

const data = page.extract({
  title: "title",
  stories: [{
    selector: "tr.athing",
    limit: 5,
    fields: {
      id: { attr: "id" },
      title: ".titleline > a"
    }
  }]
});

return data; // printed automatically as JSON

Destructure when a single field is all you need:

const { stories } = page.extract({
  stories: [{
    selector: "tr.athing",
    limit: 5,
    fields: {
      id: { attr: "id" },
      title: ".titleline > a"
    }
  }]
});

for (const story of stories) {
  console.log(story.title);
}

page.evaluate(...) returns the page evaluate tool result text. When page evaluate(...) returns an object or array, that text is JSON.

Primitive arguments must be JSON-serializable. Strings, numbers, booleans, arrays, plain objects, and null work. undefined, functions, symbols, and cyclic objects do not.

The Page object

Page is the only global. new Page() makes a page object; everything else is a method on it. Call Page(...) without new and you get Page must be called with new.

Method	Arguments	Runs in
new Page()	—	Makes a page object. No navigation yet — call page.goto(url) before any other method.
page.goto	goto(url[, { timeout }])	Async — must be awaited. Navigates the page (re-navigating reuses the same object).
page.close	close()	Marks the page done; later method calls on it error. The page is otherwise reclaimed at script end.
page.extract	extract(schema) or extract({ schema })	Browser page via extractor; returns a JS object or array
page.evaluate	evaluate(script[, { url, timeout, save }])	Browser page JS context
page.click	click(selector) or click({ selector })	Browser page
page.fill	fill(selector, value) or fill({ selector, value })	Browser page
page.scroll	scroll() or scroll({ x, y })	Browser page
page.waitForSelector	waitForSelector(selector[, { timeout }])	Browser page
page.waitForScript	waitForScript(script[, { timeout }])	Browser page JS context
page.waitForState	waitForState(state[, { timeout }])	Browser page
page.hover	hover(selector) or hover({ selector })	Browser page
page.press	press(selector, key) or press({ key[, selector] })	Browser page
page.selectOption	selectOption(selector, value) or selectOption({ selector, value })	Browser page
page.setChecked	setChecked(selector[, checked]) or setChecked({ selector, checked })	Browser page

page.goto returns at the load event (a fast snapshot). When a page's content is still loading (rendered by post-load JS), call page.waitForState("networkidle") before reading. waitForState's state accepts "load", "domcontentloaded", "networkalmostidle", "networkidle", "done". goto's timeout defaults to 10000 ms; the waitFor* timeouts default to 5000 ms.

The [, { … }] is an optional trailing options object: leading arguments are positional (page.waitForSelector("#row", { timeout: 2000 })), and the options ride in a final object. Passing a single object with everything (page.waitForSelector({ selector: "#row", timeout: 2000 })) is equivalent — that's the shape /save records into saved scripts. An option can't be a bare positional, though: page.waitForSelector("#row", 2000) is an error. A null positional omits that field (page.press(null, "Enter") presses on the focused element), and setting the same field positionally and in the options object (page.goto(url, { url: ... })) is an invalid arguments error.

Page methods address elements by CSS selector only. The tools that hand out backendNodeIds (tree, findElement, nodeDetails) aren't installed in the script context, and a raw node ID wouldn't survive replay anyway. When you're exploring in the REPL and have a backendNodeId — e.g. the leading number on a /tree line, or a /findElement hit — run /nodeDetails backendNodeId=<id> to get a durable CSS selector, then paste that into your script.

Navigation

Use page.goto(...) to open a page. It is the only async method, so always await it:

const page = new Page();
await page.goto("https://example.com");

await page.goto({
  url: "https://example.com/app",
  timeout: 15000
});

Re-navigating reuses the same page object: a later await page.goto(url2) keeps page valid and points it at the new URL. Read a page before navigating away from it.

The call returns a status string and throws if navigation fails. A timeout does not throw: the call returns "Navigation started but the page did not finish loading before the timeout." and the page stays in whatever state it reached. Check the return value — or follow with page.waitForState(...) / page.waitForSelector(...) — when completeness matters.

Structured Extraction

Use page.extract(...) to read data from the current page without writing page-side JavaScript. This is the preferred bridge from page content into local agent logic.

const result = page.extract({
  heading: "h1",
  links: [{
    selector: "a",
    limit: 10,
    fields: {
      text: "",
      href: { attr: "href" }
    }
  }]
});

The schema forms are:

Schema value	Meaning
"<selector>"	Text of the first matching element, or null
""	Text of the current matched element inside a fields block
["<selector>"]	Text of all matching elements
{ selector: "<selector>", attr: "<name>" }	Attribute from the first match
[{ selector: "<selector>", attr: "<name>" }]	Attribute from all matches
[{ selector: "<selector>", fields: { ... } }]	Array of records, with fields resolved relative to each matched element
limit: N	Cap array extraction to N matches

Return shape follows the top-level schema:

• page.extract({ title: "h1" }) returns { title: "..." }.
• page.extract({ title: "h1", links: [{ selector: "a" }] }) returns an object with both fields.
• page.extract({ links: [{ selector: "a" }] }) returns { links: [...] } — an object schema always returns an object, even with a single field.
• page.extract([{ selector: "a" }]) is shorthand for a single anonymous array extraction and returns the array directly.

Every value is a string (trimmed text or a raw attribute) or null — parse numbers in script logic. An array field that matches nothing yields [] without complaint (a page with zero comments is a valid result), but if every field in the schema misses, page.extract(...) throws no schema selector matched any element — treat that as "my selectors are wrong", not "the page is empty".

page.extract(...) reads only the current page. For list-to-detail scraping — capture a list, then visit each row for more — capture the list, then loop in the script: await page.goto each row's URL and page.extract the detail, reading each detail page before navigating to the next. The local agent context keeps the data across navigations, so the assembly happens in plain JavaScript. See the complete example below.

When passing an object directly to page.extract(...), the runtime serializes it as the extractor schema. These forms are equivalent:

page.extract({ title: "h1" });
page.extract({ schema: { title: "h1" } });
page.extract('{ "title": "h1" }');

The wrapped form accepts only schema: the REPL's save= option does not exist in scripts (page.extract({ schema: ..., save: ... }) is rejected). Keep results in local variables instead.

Use local variables to keep extracted data available to later script logic:

const head = page.extract({ title: "title" });

Page JavaScript

page.evaluate(...) is the explicit escape hatch into the current page's JavaScript context. Its script string runs where window and document exist.

const page = new Page();
await page.goto("https://example.com");

const title = page.evaluate("document.title");
console.log(title);

Keep the boundary clear:

const selector = "h1";

// Good: local agent logic builds an extract schema.
const data = page.extract({ heading: selector });

// Bad: page evaluate cannot see local agent variables.
page.evaluate("document.querySelector(selector).textContent");

page.evaluate(...) cannot call goto, extract, or other agent primitives. Agent scripts cannot access document directly. If you need page DOM data, prefer page.extract(...); use page.evaluate(...) only for page behavior that extraction cannot express, and remember its state dies on every navigation/reload, while script variables persist.

page.waitForScript(...) also evaluates in the page context, repeatedly, until the expression is truthy or the timeout expires:

page.waitForScript("document.querySelectorAll('.row').length >= 5");

Interaction Primitives

The action methods operate on the current page. Most take one object whose fields match the browser tool schema:

page.click({ selector: "a.login" });
page.fill({ selector: "input[name='acct']", value: "$LP_HN_USERNAME" });
page.fill({ selector: "input[name='pw']", value: "$LP_HN_PASSWORD" });
page.press({ key: "Enter" });

page.waitForSelector("#logout");

page.hover({ selector: "#menu" });
page.selectOption({ selector: "select[name='country']", value: "FR" });
page.setChecked({ selector: "input[name='terms']", checked: true });
page.setChecked({ selector: "input[name='newsletter']", checked: false });

page.scroll({ y: 600 });
page.scroll();

page.setChecked defaults checked to true when the field is omitted (page.setChecked("#chk") checks the box). page.press's leading positional is the optional selector, not key: a bare page.press("Enter") binds "Enter" to selector and fails. Press on the focused element with page.press({ key: "Enter" }) or page.press(null, "Enter"); target an element with page.press("#search", "Enter") or page.press({ key: "Enter", selector: "#search" }).

$LP_* placeholders in string arguments are resolved inside the Lightpanda process. This keeps credentials out of recorded scripts and LLM prompts. In recordings, resolved LP_* values are scrubbed back to placeholders.

Recording Format

The REPL remains slash-command based:

> /goto https://example.com
> /click selector='a.login'
> /save

/save writes JavaScript by default:

const page = new Page();
await page.goto("https://example.com");
page.click({ selector: "a.login" });

Only replayable browser actions are recorded:

• Recorded: goto, click, fill, scroll, hover, press, selectOption, setChecked, waitForSelector, waitForScript, waitForState, evaluate, and extract — the same set installed as page methods.
• Not recorded: read-only exploration tools such as tree, markdown, html, links, findElement, consoleLogs, getUrl, getCookies, and getEnv.
• Natural-language prompts are written as // comments above the actions they produced.

This is the deterministic --no-llm transcription. When /save runs with an LLM it instead synthesizes an idiomatic script from the session and is asked to emit JavaScript only, so those // comments generally won't appear.

Error Handling

Primitive failures throw JavaScript exceptions:

try {
  page.waitForSelector({ selector: "#dashboard", timeout: 1000 });
} catch (err) {
  console.error("dashboard did not appear:", err.message);
  throw err;
}

Common failures:

Error	Meaning
extract is not defined (or click/fill/…)	These are methods on the page object, not globals. Use const page = new Page(); await page.goto(url); page.extract(...).
Page must be called with new	Page(...) was called without new. Use const page = new Page();.
page is not navigated or has been closed; call page.goto(url) first	A method ran on a fresh new Page() (or a closed page). await page.goto(url) first.
page handle is no longer valid	A page was used after a later goto replaced it. Read a page before navigating away.
ReferenceError: document is not defined	You tried to use browser DOM APIs in the agent context. Use page.extract(...) or page.evaluate(...).
ReferenceError: require is not defined	Agent scripts are not Node.js scripts.
no page loaded - run page.goto(url) first	A page method ran before navigation.
invalid arguments	A method received the wrong number or shape of arguments, or a non-JSON-serializable value.
extract: no schema selector matched any element	Every field in the schema missed. Fix the selectors; an empty page section yields null/[] per field, not this error.

Complete Example

This script opens Hacker News, extracts five stories, visits each comments page, and prints one JSON object. A single reusable page walks the list sequentially — read each detail page before navigating to the next. The looping and object assembly happen in the local agent script, not in the page.

const HN = "https://news.ycombinator.com";

const page = new Page();
await page.goto(HN);

const { stories } = page.extract({
  stories: [{
    selector: "tr.athing",
    limit: 5,
    fields: {
      id: { attr: "id" },
      title: ".titleline > a",
      url: { selector: ".titleline > a", attr: "href" }
    }
  }]
});

for (const story of stories) {
  story.comments = [];
  if (!story.id) continue;

  await page.goto(`${HN}/item?id=${story.id}`);
  const { comments } = page.extract({
    comments: [{
      selector: "tr.athing.comtr:has(.commtext)",
      limit: 3,
      fields: {
        author: ".hnuser",
        text: ".commtext"
      }
    }]
  });
  story.comments = comments;
}

return stories; // printed automatically as JSON