Commit Graph

8347 Commits

Author SHA1 Message Date
Karl Seguin
e6332ac121 Close session before freeing notification
With the new CookieStore, the session must be freed before the notification is.
This is how it works in CDP, but in fetch, we were pretty lazy about it. This
caused the notification to be freed first, and then the cookiestore to try to
unregister: UAF.
2026-05-30 08:44:35 +08:00
Adrià Arrufat
63bcba5eab agent: add verification guidelines to system prompt
Instructs the agent to cross-check ambiguous sources and commit to a
choice for multi-candidate questions instead of abstaining.
2026-05-29 17:21:39 +02:00
Adrià Arrufat
9689aa0412 Improve extraction (#2577)
* tools: add session-scoped bridge store

Exposes `globalThis.lp` to `/eval` calls, allowing state to persist
across evaluations and page navigations. Adds a `save` parameter to
both `/eval` and `/extract` to store results in the bridge.

* browser: await promises in eval and support inline args

- Await JS Promises in `eval` tool with a 30s timeout
- Support inline arguments in multi-line slash commands
- Silence output on successful `save=`
- Add `limit` option to extract schema walker

* eval: return empty text for undefined async IIFE

* extract: support limit on simple string arrays

Treats `["<sel>"]` as sugar for `[{"selector": "<sel>"}]` in the schema
walker. This enables the `"limit"` option on simple string arrays.
Also updates agent documentation to cover cross-call state with `lp.*`.

* refactor: optimize bridge store and schema lookup

- Introduce `bridgeStorePut` to skip redundant JSON validation for
  trusted stringified values in `bridgeSync`.
- Store the schema pointer in `BlockOpener` to avoid re-parsing and
  looking up the schema in `Iterator.next`.
- Clean up error handling and optional unwrapping in `execEval`.
2026-05-29 17:15:21 +02:00
Adrià Arrufat
0a107e07a2 Merge pull request #2576 from lightpanda-io/agent_save_cmd
agent: add REPL /save command for recorded sessions
2026-05-29 15:55:49 +02:00
Adrià Arrufat
33b8af4eed agent: simplify default system prompt 2026-05-29 15:51:49 +02:00
Adrià Arrufat
135e7a0f9f Agent: simplify save handling and recorder logic
- Remove unused parameters from save helper functions.
- Inline and simplify save path duplication and file writing.
- Clean up optional unwrapping for the recorder.
2026-05-29 15:41:15 +02:00
Francis Bouvier
142c940b21 agent: add REPL /save command for recorded sessions
Add a REPL-only `/save [filename.lp]` command that persists the current
  interactive session as PandaScript without requiring the user to start the
  agent with `-i <script>`. The command records the same replayable actions as
  the existing script recorder, but keeps them in memory until the user chooses
  to save.

  Functional behavior:

  - During a REPL session, record replayable browser actions into an in-memory
    script buffer.
  - Manual slash commands are recorded through the same PandaScript formatting
    and filtering rules used by file recording.
  - Natural-language turns record their prompt as a `# ...` comment only when
    the LLM turn produces at least one successful replayable tool call.
  - Failed LLM tool calls are skipped, and repeated successful `/extract` calls
    keep only the last successful extract, matching the existing recorder logic.
  - `/save filename.lp` writes the current in-memory recording to that file.
  - Bare `/save` creates a random `session-<hex>.lp` file on first save.
  - If the first save targets an existing file, prompt with the existing numbered
    TTY picker and ask whether to replace or append.
  - After the first successful save, the REPL session is locked to that filename:
    later `/save` or `/save same-file.lp` appends to the same file without
    prompting, while `/save other-file.lp` is rejected.
  - After each successful save, reset the in-memory recorder so future saves only
    append actions entered since the previous save.
  - On any `/save` error or cancellation, keep the in-memory recorder intact so
    the user can retry without losing captured actions.
  - Restrict `/save` filenames to local file names, not paths, to keep behavior
    scoped to the current directory.

  Code changes:

  - Add `Recorder.Memory`, an in-memory recorder that shares the existing
    `Command.isRecorded`, `Command.format`, comment formatting, and `LP_*`
    reverse-substitution behavior with file recording.
  - Add `Recorder.Memory.reset()` so `/save` can clear only successfully saved
    deltas.
  - Add `/save` to the REPL meta command table and help/completion surface.
  - Add `save_buffer` and `save_path` state to `Agent`.
  - Feed manual REPL tool calls into `save_buffer` alongside the existing optional
    file recorder.
  - Extend LLM turn recording with a `capture_for_save` flag so natural-language
    REPL turns can be captured without affecting non-REPL script/self-heal paths.
  - Implement `/save` handling in `Agent`:
    - parse and validate the optional filename,
    - choose replace/append for existing first-save targets,
    - remember the first successful save path,
    - enforce the single-destination rule for the rest of the session,
    - append later deltas by default,
    - commit remembered path state only after a successful file write.
  - Add focused coverage for the memory recorder’s filtering and reset behavior.
2026-05-29 13:32:53 +02:00
Adrià Arrufat
ff95f83f74 agent: track and print token usage in one-shot mode
Updates zenai and aggregates token usage across all model calls.
Prints a `$usage` summary to stderr at the end of a task.
2026-05-29 12:44:20 +02:00
Karl Seguin
320ffa2819 Improve WPT /url/ tests
This is a bit all over the place.

1 - Replace libidn2 with rust-idna. It looks like there are different idna
    profiles, and rust-idna (from the servo project) implements the whatwg
    one. libidn2 would be too strict in some cases and not strict enough in
    others. (Gemini says I could use libidn2 for this, but what it suggested
    didn't work, and I couldn't figure it out myself, and claude insisted it
    _did not_ have the correct implementation for what we want).

2 - We previously only ran a URL through idna if it wasn't ascii. Turns out
    we also need to run it if there's a "xn--" (aka, an IDNA ACE prefix) in
    there. This helps us pass hundreds of WPT cases, and it's pretty cheap.

3 - Implement more of the Area WebAPI. Mostly copied from Anchor.

4 - Add username/password accessor to Anchor/Area

5 - window.open validates the URL (i.e. tries to resolve it and handles the
    error)

6 - Invalid idna conversion maps to a TypeError

7 - Cleanup closed popups on the next tick (like destroyed pages), rather than
    at an interval or on shutdown. This one seems unrelated, but some of these
    tests are opening hundreds (thousands?) of popups and then closing them.
2026-05-29 18:06:44 +08:00
Adrià Arrufat
291364eb8c Merge branch 'main' into agent 2026-05-29 11:45:52 +02:00
Karl Seguin
7370eb2c25 Merge pull request #2571 from lightpanda-io/storage-origin-isolation
storage: persist localStorage/sessionStorage across navigations, fix quota
2026-05-29 17:30:34 +08:00
Adrià Arrufat
f3c8595b39 storage: rename allocator to _allocator in Lookup 2026-05-29 11:04:19 +02:00
Adrià Arrufat
fc882c8238 Merge branch 'main' into storage-origin-isolation 2026-05-29 08:02:37 +02:00
Adrià Arrufat
f62f9b42ce storage: store allocator in Lookup 2026-05-29 08:00:15 +02:00
Adrià Arrufat
78312768ce storage: use getOrPut in Shed.getOrPut 2026-05-29 07:55:54 +02:00
Adrià Arrufat
ca9911c641 storage: key bucket by JS Origin key so opaque origins don't collide 2026-05-29 07:51:51 +02:00
Karl Seguin
a0c86df767 Prevent double-free on Synthetic URL
If a synthetic url (blob URL) causes a navigation event, the frame abort will
deinit the transfer, causing the `defer transfer.deinit()` atop Synthetic.run
from firing. Flag the transfer as .completing to prevent this from happening.
This mimics what non-synthetic urls do.
2026-05-29 12:21:52 +08:00
Karl Seguin
23e58b005e Add Notification WebAPI
Adds a pretty simplistic Notification WebAPI. Also adds a dummy drawImage to
CanvasRenderingContext2D.

Trying to improve how we're seen by https://bot.sannysoft.com/
2026-05-29 11:35:32 +08:00
Karl Seguin
87c6d52abc Cleaner cookie ownership
Previously, Cookie.Jar.add would only conditionally take over the cookie. The
caller had no way to know whether or not to deinit it. This could result in
a double-free on certain error paths.

Cookie.Jar.add now unconditionally takes ownership of the cookie.
2026-05-29 09:32:03 +08:00
Karl Seguin
f07eb3e264 Merge pull request #2562 from lightpanda-io/cookie-storage
Implement CookieStore web API
2026-05-29 08:20:50 +08:00
Karl Seguin
75993123b3 Merge pull request #2568 from lightpanda-io/navigate_schemaless_url
Improve navigate to schema-less URL
2026-05-29 08:03:47 +08:00
Karl Seguin
91ef1ff925 Merge pull request #2567 from lightpanda-io/rust_drop_warning
Heed Rust warning and replace copy with ignore
2026-05-29 08:02:49 +08:00
Adrià Arrufat
dab82fa29a storage: persist localStorage/sessionStorage across navigations, fix quota
Move the per-Window storage bucket to an origin-keyed Shed on the Session
so localStorage and sessionStorage survive navigation within an origin,
matching the Web Storage spec.

Also fixes two pre-existing bugs surfaced by this work:
  - setItem's quota counter was incremented on every call, never
    decremented on overwrite — five same-key overwrites tripped the cap
    spuriously. Now subtracts the old value's length first.
  - Shed.getOrPut used allocator.free on a single-pointer allocation
    where allocator.destroy was required, and inserted into _origins
    before its dependent allocations could fail. Reordered so the entry
    is only put once both key dupe and bucket creation have succeeded.

Adds an MCP test that round-trips localStorage between two origins via
the eval tool to lock in the persistence + isolation contract.
2026-05-28 20:32:39 +02:00
Pierre Tachoire
ef3faf8329 use String to return CookieListItem 2026-05-28 18:22:20 +02:00
Pierre Tachoire
b57798e4a3 Merge pull request #2569 from lightpanda-io/e2e-enable-cache
ci: always enable HTTP cache with e2e test
2026-05-28 17:58:55 +02:00
Pierre Tachoire
2c2561da50 avoid string build + parse on CookieStore.setCookie 2026-05-28 17:50:21 +02:00
Pierre Tachoire
4f2a7ba2c8 implement url option for getting cookies from CookieStore 2026-05-28 15:13:31 +02:00
Pierre Tachoire
2a47432c20 fix comments 2026-05-28 14:49:27 +02:00
Pierre Tachoire
8c9d693fd7 set max-age to 0 when cookie is expired in CookieStore 2026-05-28 14:33:30 +02:00
Pierre Tachoire
c7b9f02765 reject invalid cookie into into CookieStore 2026-05-28 14:30:36 +02:00
Pierre Tachoire
d35eaea1aa testing: deinit browser before notifications 2026-05-28 14:17:52 +02:00
Pierre Tachoire
efde428cd4 Notify the deleted cookie on change 2026-05-28 12:37:17 +02:00
Adrià Arrufat
d0a48131da Merge branch 'main' into agent 2026-05-28 11:53:47 +02:00
Pierre Tachoire
d68f1a48ad ci: always enable cache with e2e test 2026-05-28 11:25:47 +02:00
Karl Seguin
06f279c6be Improve navigate to schema-less URL
I noticed that `fetch www.openmymind.net` worked but, `fetch www.example.com`
didn't. www.openmymind.net redirects to `https://www.openmymind.net/` so
in `frameHeaderDoneCallback` we get the updated response.url(). www.example.com
doesn't redirect, so self.url remains `www.example.com` which just doesn't work
at various parts of the code (Location.init, RobotsLayer...).

Added a quick check in Navigate, if the URL isn't a "complete" URL, stick
"http://" infront. There are probably cases where this is wrong, e.g.
'javascript:...' but these don't work anyways.

(Curl works with www.example.com of course).
2026-05-28 17:16:20 +08:00
Pierre Tachoire
6179c7dde5 replace Frame with Execution into CookieStore 2026-05-28 11:14:52 +02:00
Karl Seguin
0b4ba17236 Heed Rust warning and replace copy with ignore 2026-05-28 16:48:27 +08:00
Karl Seguin
a2b3626495 Merge pull request #2563 from lightpanda-io/message_port_and_doNotTrack
postMessage / MessageEvent now allow / track MessagePost
2026-05-28 16:39:13 +08:00
Karl Seguin
b37e5288d5 Merge pull request #2565 from lightpanda-io/null_function_guard
Guard against null function
2026-05-28 16:38:42 +08:00
Karl Seguin
79afadeb42 Merge pull request #2561 from lightpanda-io/replaceable
Make various Window/Worker accessors settable
2026-05-28 16:37:43 +08:00
Karl Seguin
e40912f4b5 Merge pull request #2559 from lightpanda-io/module_evaluation_stack
In debug, try to capture a module evaluation stack
2026-05-28 16:37:14 +08:00
Karl Seguin
26a701bb32 Merge pull request #2557 from lightpanda-io/node_insert_notification_frame
Use Node's own frame when executing on nodeIsReady
2026-05-28 16:36:43 +08:00
Karl Seguin
3348eb83b2 Merge pull request #2556 from lightpanda-io/declarative_shadow_dom
Add declarative shadow dom (DSD)
2026-05-28 16:36:11 +08:00
Pierre Tachoire
27fe1d46d8 add missing .{ .dom_exception = true } into CookieStore bridge 2026-05-28 10:08:47 +02:00
Pierre Tachoire
ad0e0445a8 avoid extra allocation for 1 value slice in CookieChangeEvent 2026-05-28 10:08:46 +02:00
Pierre Tachoire
3029f8eae2 adjust CookieStore.attachToFrame comment 2026-05-28 10:08:46 +02:00
Pierre Tachoire
9c74fed309 Implement CookieChangeEvent with CookieStore 2026-05-28 10:08:45 +02:00
Pierre Tachoire
06cc808728 first draft for CookieStore API implementation 2026-05-28 10:08:41 +02:00
Pierre Tachoire
4280e28975 Merge pull request #2564 from lightpanda-io/execution_ux_improvement
This is a very small / mechanical change.
2026-05-28 10:08:24 +02:00
Karl Seguin
1ed6cd1120 Guard against null function
I'm not sure how this is happening, but we occasionally see an attempt to
execute a null v8::Function (from crash reports). It seems pretty clear that
this is a Global that is being reset, but I can't figure out a path where
a global is reset while a callback is still active. So, in != .Debug mode, we
null check functions before executing and return an error.

This potentially will work without causing any new/different issues. If this is
a global being reset, than we must be in some teardown state, and it probably
doesn't matter if a JS callback is or isn't executed.

In case anyone ends up here and wonders about the guard, here's a stack from
a crash dump:

```
Fatal V8 Error
---
location: v8::Function::Call
message: Function to be called is a null pointer
???:?:?: 0x2ac0b59 in ??? (???)
???:?:?: 0x2ace19b in ??? (???)
/src/browser/js/Function.zig:160:41: 0x268db23 in callWithThis__anon_236231 (lightpanda)
/src/browser/EventManagerBase.zig:259:30: 0x2614b53 in dispatchDirect__anon_179346 (lightpanda)
/src/browser/EventManager.zig:137:33: 0x261737e in stateChanged (lightpanda)
/src/browser/webapi/net/XMLHttpRequest.zig:580:30: 0x2621b21 in handleError (lightpanda)
/src/browser/webapi/net/XMLHttpRequest.zig:542:21: 0x2628bbd in httpErrorCallback (lightpanda)
/src/network/layer/Forward.zig:70:13: 0x29193de in errorCallback (lightpanda)
/src/browser/HttpClient.zig:1447:36: 0x2784aa6 in processMessage (lightpanda)
/src/cdp/CDP.zig:314:24: 0x26a3244 in dispatchParsed (lightpanda)
/src/cdp/CDP.zig:187:31: 0x252df34 in drainInbox (lightpanda)
/src/browser/HttpClient.zig:416:24: 0x27f01ab in handleConnection (lightpanda)
/home/runner/work/_temp/91356985-8cc1-40de-b1b8-395a29b4bd91/zig-x86_64-linux-0.15.2/lib/std/Thread.zig:509:13: 0x26877f3 in entryFn (lightpanda)
???:?:?: 0x7df4279d11f4 in ??? (libc.so.6)
Unwind information for `libc.so.6:0x7df4279d11f4` was not available, trace may be incomplete
```
2026-05-28 11:04:55 +08:00