Commit Graph

5564 Commits

Author SHA1 Message Date
Adrià Arrufat
204e3aa31b Merge branch 'main' into agent 2026-06-01 08:04:44 +02:00
Karl Seguin
295bcf5cda Merge pull request #2583 from jschaf/codex/fix-relative-url-fragment
browser/URL: ignore query and fragment slashes in URL resolve
2026-06-01 13:07:06 +08:00
Karl Seguin
b17ff37f8d Merge pull request #2587 from lightpanda-io/websocket_cookies
Websocket cookies
2026-06-01 11:06:19 +08:00
Karl Seguin
14f1ef35f1 URL.isHTTPs -> URL.isSecure and consider wss://
Use conn.setCookie in websocket
2026-06-01 08:35:19 +08:00
Adrià Arrufat
668f8a07dc test: add extract follow test fixtures 2026-05-31 18:49:29 +02:00
Adrià Arrufat
cf65a00a8f extract: add declarative follow option
Allows fetching sub-pages per row and resolving nested fields against
them. Supports string templates with sibling placeholders (e.g., `{id}`)
and element-specs. Updates the JS walker to be async.
2026-05-31 17:07:42 +02:00
Adrià Arrufat
85facd2fc7 eval: auto-serialize non-JSON values in save 2026-05-31 16:31:06 +02:00
Adrià Arrufat
fc15027100 eval: clarify automatic JSON serialization
Update agent docs and tool descriptions to note that objects and
arrays are automatically serialized to JSON, making manual
`JSON.stringify` calls unnecessary.
2026-05-31 16:27:28 +02:00
Adrià Arrufat
ab3deec523 eval: serialize returned objects as JSON
Objects and arrays returned from eval now serialize to JSON instead of
"[object Object]". Native errors and functions retain their string form.
2026-05-31 16:19:34 +02:00
Adrià Arrufat
27c2fe00c7 eval: support top-level await and return
Falls back to compiling the script inside an async IIFE if the initial
block-scoped compilation fails. This enables top-level await and return
statements directly in the eval tool.
2026-05-31 16:10:42 +02:00
Adrià Arrufat
63f2706202 terminal: remove bare command validation
Stop highlighting bare tokens matching tool names as errors. Also
updates the zenai dependency.
2026-05-31 15:29:53 +02:00
Adrià Arrufat
7204de5a1b agent: prompt for provider selection when multiple found
Allows interactive selection of an LLM provider when multiple keys are
detected in the environment. Saves the choice to `.lp-agent.zon`.
2026-05-31 14:33:57 +02:00
Adrià Arrufat
de0eff05f6 terminal: simplify interactive choice selection
Remove number typing input and only support arrow keys and Enter.
2026-05-31 13:21:44 +02:00
Francis Bouvier
99ef54a557 agent: add arrow-key navigation to provider picker
Make the numbered agent choice prompt interactive on TTYs: Up/Down
moves the selected row, Enter confirms it, and numeric input still works as before.

Keep the line-based numbered prompt as the non-interactive fallback, restore
terminal settings after the raw-mode picker exits, and render raw-mode output
with CRLF so menu rows stay aligned. Dim the picker hint text to match existing
terminal command hints.
2026-05-31 13:21:44 +02:00
Joe Schafer
bf1b5e5506 browser/URL: ignore query and fragment slashes in URL resolve
Resolve relative request URLs against only the path component of the
base URL. Previously, URL.resolve searched for the last slash in the
whole base URL after the authority, so slashes inside a query string or
hash route could be mistaken for path directory separators.

The reported failure was a hash-routed page loaded at
http://127.0.0.1:8123/#/login. When that page ran
fetch("api/users/login", { method: "POST" }), browser-compatible URL
resolution should have requested
http://127.0.0.1:8123/api/users/login. Instead, Lightpanda 0.3.1
reported the response URL as
http://127.0.0.1:8123/#/api/users/login, and the HTTP server received
POST /.

That meant the server returned the single-page app shell instead of the
JSON login response. The failure broke a RealWorld/Conduit-style SPA
benchmark after login because the app used relative API URLs such as
api/users/login. Changing the benchmark fixture to root-absolute API
URLs, such as /api/users/login, worked around the benchmark failure, but
left Lightpanda incompatible with normal browser behavior for relative
request URLs on hash-routed pages.

The same issue was not limited to fragments. A base URL such as
https://example/app/page?next=/foo/bar also contains slashes after the
path component. Those slashes must not affect how path-relative inputs
such as api/users/login or ../api/users/login are merged with the base
path.

This change bounds the directory calculation at the first ? or # in the
base URL and searches for the last path slash only inside that path
component. Root-absolute inputs still keep only the scheme and
authority, query-only inputs still replace the query on the current
path, and fragment-only inputs still replace the fragment on the
current path.

This matches the behavior of the WHATWG URL algorithm, Chromium's
relative URL canonicalization, Node's WHATWG URL implementation, Go's
net/url ResolveReference, and Python's urllib.parse.urljoin for the
cases covered here. All of those implementations split the base URL into
components before merging a path-relative reference, so slashes in the
query or fragment do not change the base directory.

The URL resolver tests now cover the original hash-route repro, slashes
inside query strings and fragments, host-only bases with query or
fragment components, dot-segment paths, query-only references, and
fragment-only references. The fetch Web API tests also move a page to
/#/login and POST fetch("xhr"), expecting the request URL to resolve to
http://127.0.0.1:9582/xhr rather than the hash route.

Verified with:

mise x zig@0.15.2 -- zig fmt --check ./*.zig ./**/*.zig
mise x zig@0.15.2 rust@stable -- make ZIG=zig test
2026-05-31 00:44:33 -07:00
Adrià Arrufat
88fdeeade8 refactor: extract JSON formatting and timeout helpers 2026-05-30 23:48:50 +02:00
Adrià Arrufat
53ba47cbec agent: suggest closest slash command on typo
Implements Levenshtein distance-based suggestions for unknown slash
commands. If a typo is within two edits of a valid command, the
terminal suggests it with "Did you mean ...?".
2026-05-30 23:40:34 +02:00
Adrià Arrufat
e42862e544 terminal: add ghost hints for slash commands 2026-05-30 23:26:40 +02:00
Adrià Arrufat
58caf9faf7 agent: pretty-print JSON command results
Re-indents JSON output with 2-space indentation for better readability
in the terminal, while keeping non-JSON output unchanged.
2026-05-30 23:12:54 +02:00
Adrià Arrufat
2eb995e0ee links: return structured link objects with text and node ID
Updates `collectLinks` to return a `Link` struct containing the href,
visible text, and backend node ID. The links tool now outputs JSON.
2026-05-30 22:55:53 +02:00
Adrià Arrufat
ee96d8e813 browser: report timeout status in goto tool
Adds `WaitResult` to track whether a wait completed or timed out.
Updates the goto tool to report when a timeout occurs.
2026-05-30 22:41:24 +02:00
Tom Clarke
2ecf9ced5d Send cookies on WebSocket upgrade requests
The WebSocket upgrade handshake is an HTTP/1.1 request (RFC 6455 §4.1)
and follows ordinary cookie semantics — RFC 6265 §5.4 attaches matching
cookies to "any HTTP request" by domain/path. Without this, cookie-
authenticated WebSocket endpoints (anything session-gated, e.g. Phoenix
LiveView) reject the upgrade because their auth cookie never arrives.

Read matching cookies from the session jar with the same opts shape
HTTPDocument uses (`is_http: true, is_navigation: false`), and add a
`Cookie:` request header on the upgrade if any apply.

The TestWSServer captures the upgrade's Cookie header and exposes it
to fixtures via a new `get-cookie` command. A `cookies_on_upgrade`
fixture in websocket.html sets `document.cookie` then asserts the
server received it on the upgrade.
2026-05-30 16:37:05 -04:00
Adrià Arrufat
13e3de4b26 browser: floor remaining wait timeout to 1ms
Ensures `waitForSelector` and `waitForScript` perform at least one
check even if the timeout is 0 or already elapsed.
2026-05-30 22:34:03 +02:00
Adrià Arrufat
1ec65a00fb agent: improve command error messages
Suggest `/help` on unknown slash commands and handle `FrameNotLoaded`
by prompting the user to run `/goto <url>` first.
2026-05-30 22:31:18 +02:00
Adrià Arrufat
26cf182b38 agent: load external stylesheets by default with LLM
Enables `load_external_stylesheets` when an LLM client is active,
as LLM drivers reason about visibility and computed styles.
Updates the help documentation to reflect this change.
2026-05-30 22:16:18 +02:00
Adrià Arrufat
c9c962ec74 Merge branch 'main' into agent 2026-05-30 22:05:01 +02:00
Adrià Arrufat
c92dad165f command: add /logout and refactor LLM commands 2026-05-30 22:03:46 +02:00
Adrià Arrufat
b98a79e14e agent: derive llm commands from Command tags 2026-05-30 20:24:49 +02:00
Adrià Arrufat
74600833bc agent: add '[command]' hint to help command 2026-05-30 20:15:04 +02:00
Adrià Arrufat
17aeef886c terminal: show ghost text hints for /help arguments 2026-05-30 20:11:03 +02:00
Adrià Arrufat
47072a82e6 repl: use explicit summaries for command help
Replaces the fragile `firstSentence` parser with an explicit `summary`
field on tool definitions. Also standardizes user-facing REPL
terminology to "command" instead of "slash command" or "tool".
2026-05-30 20:06:30 +02:00
Adrià Arrufat
fb7dfc1410 refactor: clean up comments and remove redundant ping test 2026-05-30 19:53:43 +02:00
Adrià Arrufat
f51bda4d5a refactor: deduplicate comment writing and remove unused code 2026-05-30 19:32:59 +02:00
Adrià Arrufat
1f85de3d3d agent: use ZON format for remembered config
Migrates the `.lp-agent` file to `.lp-agent.zon` and uses `std.zon`
for serialization and parsing.
2026-05-30 19:10:40 +02:00
Francis Bouvier
8052f0ad81 Agent model provider picker (#2581)
* agent: add a /model command to chnage current model

And remove the pick-model CLI option

* agent: add /provider to change the current provider

* agent: extract requireLlmNoArg helper

* agent: simplify provider detection

* repl: add tab completion for /model and /provider

Changes `/model` and `/provider` to accept an optional name argument
instead of prompting with a numbered list. Bare commands now print the
current selection, while Tab dynamically completes candidates. Model
lists are fetched and cached to prevent redundant network requests.

* agent: remember last selected provider and model

Persists the last selected AI provider and model in a local
`.lp-agent` file and resumes it on startup. Removes the
interactive provider picker in favor of deterministic auto-detection.

* agent: simplify requireLlm and model resolution

Changes `requireLlm` to return a boolean instead of credentials, and
cleans up the model initialization logic to use `resolved` directly.
Also removes unused user errors.

---------

Co-authored-by: Adrià Arrufat <adria.arrufat@gmail.com>
2026-05-30 19:02:44 +02:00
Karl Seguin
490b48ecd0 zig fmt 2026-05-30 20:11:07 +08:00
Karl Seguin
eb5d46bb11 Fix potential segfault in CustomElement definition
Fixes crash in WPT /custom-elements/CustomElementRegistry.html

define has to get `observedAttributes` which itself could call define,
invalidating any GetOrPutEntry pointers. Need to do it as two distinct lookup.
2026-05-30 20:05:44 +08:00
Karl Seguin
b91b3ecd16 Merge pull request #2578 from lightpanda-io/cookie_store_crash_fix
Close session before freeing notification
2026-05-30 10:12:49 +08:00
Karl Seguin
732234c453 Merge pull request #2573 from lightpanda-io/notifiation_webapi
Add Notification WebAPI
2026-05-30 08:47:32 +08:00
Karl Seguin
a40c35ab5f Merge pull request #2574 from lightpanda-io/synthentic_transfer_double_free
Prevent double-free on Synthetic URL
2026-05-30 08:46:51 +08:00
Karl Seguin
a7d3a5968c Merge pull request #2572 from lightpanda-io/cookie_jar_ownership
Cleaner cookie ownership
2026-05-30 08:46:29 +08:00
Karl Seguin
e6332ac121 Close session before freeing notification
With the new CookieStore, the session must be freed before the notification is.
This is how it works in CDP, but in fetch, we were pretty lazy about it. This
caused the notification to be freed first, and then the cookiestore to try to
unregister: UAF.
2026-05-30 08:44:35 +08:00
Adrià Arrufat
63bcba5eab agent: add verification guidelines to system prompt
Instructs the agent to cross-check ambiguous sources and commit to a
choice for multi-candidate questions instead of abstaining.
2026-05-29 17:21:39 +02:00
Adrià Arrufat
9689aa0412 Improve extraction (#2577)
* tools: add session-scoped bridge store

Exposes `globalThis.lp` to `/eval` calls, allowing state to persist
across evaluations and page navigations. Adds a `save` parameter to
both `/eval` and `/extract` to store results in the bridge.

* browser: await promises in eval and support inline args

- Await JS Promises in `eval` tool with a 30s timeout
- Support inline arguments in multi-line slash commands
- Silence output on successful `save=`
- Add `limit` option to extract schema walker

* eval: return empty text for undefined async IIFE

* extract: support limit on simple string arrays

Treats `["<sel>"]` as sugar for `[{"selector": "<sel>"}]` in the schema
walker. This enables the `"limit"` option on simple string arrays.
Also updates agent documentation to cover cross-call state with `lp.*`.

* refactor: optimize bridge store and schema lookup

- Introduce `bridgeStorePut` to skip redundant JSON validation for
  trusted stringified values in `bridgeSync`.
- Store the schema pointer in `BlockOpener` to avoid re-parsing and
  looking up the schema in `Iterator.next`.
- Clean up error handling and optional unwrapping in `execEval`.
2026-05-29 17:15:21 +02:00
Adrià Arrufat
0a107e07a2 Merge pull request #2576 from lightpanda-io/agent_save_cmd
agent: add REPL /save command for recorded sessions
2026-05-29 15:55:49 +02:00
Adrià Arrufat
33b8af4eed agent: simplify default system prompt 2026-05-29 15:51:49 +02:00
Adrià Arrufat
135e7a0f9f Agent: simplify save handling and recorder logic
- Remove unused parameters from save helper functions.
- Inline and simplify save path duplication and file writing.
- Clean up optional unwrapping for the recorder.
2026-05-29 15:41:15 +02:00
Francis Bouvier
142c940b21 agent: add REPL /save command for recorded sessions
Add a REPL-only `/save [filename.lp]` command that persists the current
  interactive session as PandaScript without requiring the user to start the
  agent with `-i <script>`. The command records the same replayable actions as
  the existing script recorder, but keeps them in memory until the user chooses
  to save.

  Functional behavior:

  - During a REPL session, record replayable browser actions into an in-memory
    script buffer.
  - Manual slash commands are recorded through the same PandaScript formatting
    and filtering rules used by file recording.
  - Natural-language turns record their prompt as a `# ...` comment only when
    the LLM turn produces at least one successful replayable tool call.
  - Failed LLM tool calls are skipped, and repeated successful `/extract` calls
    keep only the last successful extract, matching the existing recorder logic.
  - `/save filename.lp` writes the current in-memory recording to that file.
  - Bare `/save` creates a random `session-<hex>.lp` file on first save.
  - If the first save targets an existing file, prompt with the existing numbered
    TTY picker and ask whether to replace or append.
  - After the first successful save, the REPL session is locked to that filename:
    later `/save` or `/save same-file.lp` appends to the same file without
    prompting, while `/save other-file.lp` is rejected.
  - After each successful save, reset the in-memory recorder so future saves only
    append actions entered since the previous save.
  - On any `/save` error or cancellation, keep the in-memory recorder intact so
    the user can retry without losing captured actions.
  - Restrict `/save` filenames to local file names, not paths, to keep behavior
    scoped to the current directory.

  Code changes:

  - Add `Recorder.Memory`, an in-memory recorder that shares the existing
    `Command.isRecorded`, `Command.format`, comment formatting, and `LP_*`
    reverse-substitution behavior with file recording.
  - Add `Recorder.Memory.reset()` so `/save` can clear only successfully saved
    deltas.
  - Add `/save` to the REPL meta command table and help/completion surface.
  - Add `save_buffer` and `save_path` state to `Agent`.
  - Feed manual REPL tool calls into `save_buffer` alongside the existing optional
    file recorder.
  - Extend LLM turn recording with a `capture_for_save` flag so natural-language
    REPL turns can be captured without affecting non-REPL script/self-heal paths.
  - Implement `/save` handling in `Agent`:
    - parse and validate the optional filename,
    - choose replace/append for existing first-save targets,
    - remember the first successful save path,
    - enforce the single-destination rule for the rest of the session,
    - append later deltas by default,
    - commit remembered path state only after a successful file write.
  - Add focused coverage for the memory recorder’s filtering and reset behavior.
2026-05-29 13:32:53 +02:00
Adrià Arrufat
ff95f83f74 agent: track and print token usage in one-shot mode
Updates zenai and aggregates token usage across all model calls.
Prints a `$usage` summary to stderr at the end of a task.
2026-05-29 12:44:20 +02:00
Adrià Arrufat
291364eb8c Merge branch 'main' into agent 2026-05-29 11:45:52 +02:00