Commit Graph
9542 Commits
Author SHA1 Message Date
Adrià Arrufat 135e7a0f9f Agent: simplify save handling and recorder logic
- Remove unused parameters from save helper functions.
- Inline and simplify save path duplication and file writing.
- Clean up optional unwrapping for the recorder.
2026-05-29 15:41:15 +02:00
Francis Bouvier 142c940b21 agent: add REPL /save command for recorded sessions
Add a REPL-only `/save [filename.lp]` command that persists the current
  interactive session as PandaScript without requiring the user to start the
  agent with `-i <script>`. The command records the same replayable actions as
  the existing script recorder, but keeps them in memory until the user chooses
  to save.

  Functional behavior:

  - During a REPL session, record replayable browser actions into an in-memory
    script buffer.
  - Manual slash commands are recorded through the same PandaScript formatting
    and filtering rules used by file recording.
  - Natural-language turns record their prompt as a `# ...` comment only when
    the LLM turn produces at least one successful replayable tool call.
  - Failed LLM tool calls are skipped, and repeated successful `/extract` calls
    keep only the last successful extract, matching the existing recorder logic.
  - `/save filename.lp` writes the current in-memory recording to that file.
  - Bare `/save` creates a random `session-<hex>.lp` file on first save.
  - If the first save targets an existing file, prompt with the existing numbered
    TTY picker and ask whether to replace or append.
  - After the first successful save, the REPL session is locked to that filename:
    later `/save` or `/save same-file.lp` appends to the same file without
    prompting, while `/save other-file.lp` is rejected.
  - After each successful save, reset the in-memory recorder so future saves only
    append actions entered since the previous save.
  - On any `/save` error or cancellation, keep the in-memory recorder intact so
    the user can retry without losing captured actions.
  - Restrict `/save` filenames to local file names, not paths, to keep behavior
    scoped to the current directory.

  Code changes:

  - Add `Recorder.Memory`, an in-memory recorder that shares the existing
    `Command.isRecorded`, `Command.format`, comment formatting, and `LP_*`
    reverse-substitution behavior with file recording.
  - Add `Recorder.Memory.reset()` so `/save` can clear only successfully saved
    deltas.
  - Add `/save` to the REPL meta command table and help/completion surface.
  - Add `save_buffer` and `save_path` state to `Agent`.
  - Feed manual REPL tool calls into `save_buffer` alongside the existing optional
    file recorder.
  - Extend LLM turn recording with a `capture_for_save` flag so natural-language
    REPL turns can be captured without affecting non-REPL script/self-heal paths.
  - Implement `/save` handling in `Agent`:
    - parse and validate the optional filename,
    - choose replace/append for existing first-save targets,
    - remember the first successful save path,
    - enforce the single-destination rule for the rest of the session,
    - append later deltas by default,
    - commit remembered path state only after a successful file write.
  - Add focused coverage for the memory recorder’s filtering and reset behavior.
2026-05-29 13:32:53 +02:00
Adrià Arrufat ff95f83f74 agent: track and print token usage in one-shot mode
Updates zenai and aggregates token usage across all model calls.
Prints a `$usage` summary to stderr at the end of a task.
2026-05-29 12:44:20 +02:00
Karl Seguin 320ffa2819 Improve WPT /url/ tests
This is a bit all over the place.

1 - Replace libidn2 with rust-idna. It looks like there are different idna
    profiles, and rust-idna (from the servo project) implements the whatwg
    one. libidn2 would be too strict in some cases and not strict enough in
    others. (Gemini says I could use libidn2 for this, but what it suggested
    didn't work, and I couldn't figure it out myself, and claude insisted it
    _did not_ have the correct implementation for what we want).

2 - We previously only ran a URL through idna if it wasn't ascii. Turns out
    we also need to run it if there's a "xn--" (aka, an IDNA ACE prefix) in
    there. This helps us pass hundreds of WPT cases, and it's pretty cheap.

3 - Implement more of the Area WebAPI. Mostly copied from Anchor.

4 - Add username/password accessor to Anchor/Area

5 - window.open validates the URL (i.e. tries to resolve it and handles the
    error)

6 - Invalid idna conversion maps to a TypeError

7 - Cleanup closed popups on the next tick (like destroyed pages), rather than
    at an interval or on shutdown. This one seems unrelated, but some of these
    tests are opening hundreds (thousands?) of popups and then closing them.
2026-05-29 18:06:44 +08:00
Adrià Arrufat 291364eb8c Merge branch 'main' into agent 2026-05-29 11:45:52 +02:00
Karl Seguin 7370eb2c25 Merge pull request #2571 from lightpanda-io/storage-origin-isolation
storage: persist localStorage/sessionStorage across navigations, fix quota
2026-05-29 17:30:34 +08:00
Adrià Arrufat f3c8595b39 storage: rename allocator to _allocator in Lookup 2026-05-29 11:04:19 +02:00
Adrià Arrufat fc882c8238 Merge branch 'main' into storage-origin-isolation 2026-05-29 08:02:37 +02:00
Adrià Arrufat f62f9b42ce storage: store allocator in Lookup 2026-05-29 08:00:15 +02:00
Adrià Arrufat 78312768ce storage: use getOrPut in Shed.getOrPut 2026-05-29 07:55:54 +02:00
Adrià Arrufat ca9911c641 storage: key bucket by JS Origin key so opaque origins don't collide 2026-05-29 07:51:51 +02:00
Karl Seguin a0c86df767 Prevent double-free on Synthetic URL
If a synthetic url (blob URL) causes a navigation event, the frame abort will
deinit the transfer, causing the `defer transfer.deinit()` atop Synthetic.run
from firing. Flag the transfer as .completing to prevent this from happening.
This mimics what non-synthetic urls do.
2026-05-29 12:21:52 +08:00
Karl Seguin 23e58b005e Add Notification WebAPI
Adds a pretty simplistic Notification WebAPI. Also adds a dummy drawImage to
CanvasRenderingContext2D.

Trying to improve how we're seen by https://bot.sannysoft.com/
2026-05-29 11:35:32 +08:00
Karl Seguin 87c6d52abc Cleaner cookie ownership
Previously, Cookie.Jar.add would only conditionally take over the cookie. The
caller had no way to know whether or not to deinit it. This could result in
a double-free on certain error paths.

Cookie.Jar.add now unconditionally takes ownership of the cookie.
2026-05-29 09:32:03 +08:00
Karl Seguin f07eb3e264 Merge pull request #2562 from lightpanda-io/cookie-storage
Implement CookieStore web API
2026-05-29 08:20:50 +08:00
Karl Seguin 75993123b3 Merge pull request #2568 from lightpanda-io/navigate_schemaless_url
Improve navigate to schema-less URL
2026-05-29 08:03:47 +08:00
Karl Seguin 91ef1ff925 Merge pull request #2567 from lightpanda-io/rust_drop_warning
Heed Rust warning and replace copy with ignore
2026-05-29 08:02:49 +08:00
Adrià Arrufat dab82fa29a storage: persist localStorage/sessionStorage across navigations, fix quota
Move the per-Window storage bucket to an origin-keyed Shed on the Session
so localStorage and sessionStorage survive navigation within an origin,
matching the Web Storage spec.

Also fixes two pre-existing bugs surfaced by this work:
  - setItem's quota counter was incremented on every call, never
    decremented on overwrite — five same-key overwrites tripped the cap
    spuriously. Now subtracts the old value's length first.
  - Shed.getOrPut used allocator.free on a single-pointer allocation
    where allocator.destroy was required, and inserted into _origins
    before its dependent allocations could fail. Reordered so the entry
    is only put once both key dupe and bucket creation have succeeded.

Adds an MCP test that round-trips localStorage between two origins via
the eval tool to lock in the persistence + isolation contract.
2026-05-28 20:32:39 +02:00
Pierre Tachoire ef3faf8329 use String to return CookieListItem 2026-05-28 18:22:20 +02:00
Pierre Tachoire b57798e4a3 Merge pull request #2569 from lightpanda-io/e2e-enable-cache
ci: always enable HTTP cache with e2e test
2026-05-28 17:58:55 +02:00
Pierre Tachoire 2c2561da50 avoid string build + parse on CookieStore.setCookie 2026-05-28 17:50:21 +02:00
Pierre Tachoire 4f2a7ba2c8 implement url option for getting cookies from CookieStore 2026-05-28 15:13:31 +02:00
Pierre Tachoire 2a47432c20 fix comments 2026-05-28 14:49:27 +02:00
Pierre Tachoire 8c9d693fd7 set max-age to 0 when cookie is expired in CookieStore 2026-05-28 14:33:30 +02:00
Pierre Tachoire c7b9f02765 reject invalid cookie into into CookieStore 2026-05-28 14:30:36 +02:00
Pierre Tachoire d35eaea1aa testing: deinit browser before notifications 2026-05-28 14:17:52 +02:00
Pierre Tachoire efde428cd4 Notify the deleted cookie on change 2026-05-28 12:37:17 +02:00
Adrià Arrufat d0a48131da Merge branch 'main' into agent 2026-05-28 11:53:47 +02:00
Pierre Tachoire d68f1a48ad ci: always enable cache with e2e test 2026-05-28 11:25:47 +02:00
Karl Seguin 06f279c6be Improve navigate to schema-less URL
I noticed that `fetch www.openmymind.net` worked but, `fetch www.example.com`
didn't. www.openmymind.net redirects to `https://www.openmymind.net/` so
in `frameHeaderDoneCallback` we get the updated response.url(). www.example.com
doesn't redirect, so self.url remains `www.example.com` which just doesn't work
at various parts of the code (Location.init, RobotsLayer...).

Added a quick check in Navigate, if the URL isn't a "complete" URL, stick
"http://" infront. There are probably cases where this is wrong, e.g.
'javascript:...' but these don't work anyways.

(Curl works with www.example.com of course).
2026-05-28 17:16:20 +08:00
Pierre Tachoire 6179c7dde5 replace Frame with Execution into CookieStore 2026-05-28 11:14:52 +02:00
Karl Seguin 0b4ba17236 Heed Rust warning and replace copy with ignore 2026-05-28 16:48:27 +08:00
Karl Seguin a2b3626495 Merge pull request #2563 from lightpanda-io/message_port_and_doNotTrack
postMessage / MessageEvent now allow / track MessagePost
2026-05-28 16:39:13 +08:00
Karl Seguin b37e5288d5 Merge pull request #2565 from lightpanda-io/null_function_guard
Guard against null function
2026-05-28 16:38:42 +08:00
Karl Seguin 79afadeb42 Merge pull request #2561 from lightpanda-io/replaceable
Make various Window/Worker accessors settable
2026-05-28 16:37:43 +08:00
Karl Seguin e40912f4b5 Merge pull request #2559 from lightpanda-io/module_evaluation_stack
In debug, try to capture a module evaluation stack
2026-05-28 16:37:14 +08:00
Karl Seguin 26a701bb32 Merge pull request #2557 from lightpanda-io/node_insert_notification_frame
Use Node's own frame when executing on nodeIsReady
2026-05-28 16:36:43 +08:00
Karl Seguin 3348eb83b2 Merge pull request #2556 from lightpanda-io/declarative_shadow_dom
Add declarative shadow dom (DSD)
2026-05-28 16:36:11 +08:00
Pierre Tachoire 27fe1d46d8 add missing .{ .dom_exception = true } into CookieStore bridge 2026-05-28 10:08:47 +02:00
Pierre Tachoire ad0e0445a8 avoid extra allocation for 1 value slice in CookieChangeEvent 2026-05-28 10:08:46 +02:00
Pierre Tachoire 3029f8eae2 adjust CookieStore.attachToFrame comment 2026-05-28 10:08:46 +02:00
Pierre Tachoire 9c74fed309 Implement CookieChangeEvent with CookieStore 2026-05-28 10:08:45 +02:00
Pierre Tachoire 06cc808728 first draft for CookieStore API implementation 2026-05-28 10:08:41 +02:00
Pierre Tachoire 4280e28975 Merge pull request #2564 from lightpanda-io/execution_ux_improvement
This is a very small / mechanical change.
2026-05-28 10:08:24 +02:00
Karl Seguin 1ed6cd1120 Guard against null function
I'm not sure how this is happening, but we occasionally see an attempt to
execute a null v8::Function (from crash reports). It seems pretty clear that
this is a Global that is being reset, but I can't figure out a path where
a global is reset while a callback is still active. So, in != .Debug mode, we
null check functions before executing and return an error.

This potentially will work without causing any new/different issues. If this is
a global being reset, than we must be in some teardown state, and it probably
doesn't matter if a JS callback is or isn't executed.

In case anyone ends up here and wonders about the guard, here's a stack from
a crash dump:

```
Fatal V8 Error
---
location: v8::Function::Call
message: Function to be called is a null pointer
???:?:?: 0x2ac0b59 in ??? (???)
???:?:?: 0x2ace19b in ??? (???)
/src/browser/js/Function.zig:160:41: 0x268db23 in callWithThis__anon_236231 (lightpanda)
/src/browser/EventManagerBase.zig:259:30: 0x2614b53 in dispatchDirect__anon_179346 (lightpanda)
/src/browser/EventManager.zig:137:33: 0x261737e in stateChanged (lightpanda)
/src/browser/webapi/net/XMLHttpRequest.zig:580:30: 0x2621b21 in handleError (lightpanda)
/src/browser/webapi/net/XMLHttpRequest.zig:542:21: 0x2628bbd in httpErrorCallback (lightpanda)
/src/network/layer/Forward.zig:70:13: 0x29193de in errorCallback (lightpanda)
/src/browser/HttpClient.zig:1447:36: 0x2784aa6 in processMessage (lightpanda)
/src/cdp/CDP.zig:314:24: 0x26a3244 in dispatchParsed (lightpanda)
/src/cdp/CDP.zig:187:31: 0x252df34 in drainInbox (lightpanda)
/src/browser/HttpClient.zig:416:24: 0x27f01ab in handleConnection (lightpanda)
/home/runner/work/_temp/91356985-8cc1-40de-b1b8-395a29b4bd91/zig-x86_64-linux-0.15.2/lib/std/Thread.zig:509:13: 0x26877f3 in entryFn (lightpanda)
???:?:?: 0x7df4279d11f4 in ??? (libc.so.6)
Unwind information for `libc.so.6:0x7df4279d11f4` was not available, trace may be incomplete
```
2026-05-28 11:04:55 +08:00
Karl Seguin 05daa4a6dc This is a very small / mechanical change.
Three changes:
- js.Execution now has a page (instead of calling exec.context.page)
- js.Execution now has a session (instead of calling exec.context.page.session)
- js.Execution.context renamed to .js to be consistent with Frame and WGS
2026-05-28 08:23:35 +08:00
Karl Seguin 3430bbddd8 postMessage / MessageEvent now allow / track MessagePost
Navigator properties moved from data properties to real accessors. This impact
how they're seen (and potentially treated) by JS code.

These change result in https://www.browserscan.net/bot-detection correctly
rendering. When paired with https://github.com/lightpanda-io/browser/pull/2561
we now render that page correctly (still detected as a bot, but we no longer
fail to render)
2026-05-28 08:13:21 +08:00
Karl Seguin 4e2f21990c Merge pull request #2560 from lightpanda-io/capture_401_407
Capture 401 and 407 bodies
2026-05-28 06:57:21 +08:00
Karl Seguin 15a7a66003 Make various Window/Worker accessors settable
window.console (and many others) should be settable. This is the [Replaceable]
WebIDL attribute.

This improves loading of https://www.browserscan.net/bot-detection which would
immediately break/fail because we would throw on window.console setting.

This appears to come from obfuscator.io's disableConsoleOutput flag, so the
issue might be relatively common.
2026-05-27 21:41:40 +08:00
Karl Seguin f614d898aa Capture 401 and 407 bodies
We currently skip capturing 401 and 407 bodies. This appears to be an
optimization with the intent that it won't be needed. While that might be true
in some cases (though, not sure when), it isn't always true. A page.navigate
to a 401 should display the content.

(Also, tried to silence meaningless BrokenPipe noise in tests).
2026-05-27 19:34:39 +08:00