Initial WIP on IndexedDB WebAPI. Uses sqlite and a new `--indexdb_dir` config.
Defaults to :memory:.
The main things missing are the IDBCursor, IDBIndex and IDBKeyRange, along with
a bunch of smaller apis.
Also, every object is currently tied to the Page arena / factory. It's possible
that's how it will have to be, but, once more of the API lands, I will check
if we can scope these better.
`--log-filter-scopes` previously took a comma-separated list of scopes to
suppress. Extend it with explicit include/exclude directives applied
left-to-right:
-X filter out scope X
X alias for -X (backward compatible)
+X filter in scope X
all target every scope
This makes "filter everything except one scope" expressible, e.g.
`-all,+cdp` suppresses all logs except `cdp`. Bare scope names keep their
old meaning, so existing invocations like `http,unknown_prop` are
unaffected.
Internally, the suppressed-scope list is replaced by a resolved per-scope
`scope_enabled` boolean array built via `log.resolveFilterScopes()`, so the
hot-path `enabled()` check is a single array index. Filtering remains
Debug-only, as before. `testing.LogFilter` is adapted to the new
representation; all existing call sites are unchanged.
Adds a unit test for resolveFilterScopes (default, backward-compat,
-all,+cdp, ordering) and updates the help text.
Move away from hard-coded 512KB (WS) and 4K (http) limits. Introduces two new
serve-specific command line arguments:
--cdp-max-message-size <INT>
Maximum allowed incoming websocket message size.
Defaults to 1048576 (1MB)
--cdp-max-http-message-size <INT>
Maximum allowed HTTP request size
Defaults to 4096 (maximum allowed: 16383)
--cdp-max-message-size has been bumped from 512KB to 1MB default.
Meant to provide a more robust solution than https://github.com/lightpanda-io/browser/pull/2717
Moves `tagNames` and `tagHint` from `SlashCommand` to `Config` for reuse.
Adds `tagJsonArray` to dynamically generate JSON enum schemas, and
uses it in `tools.zig` for the `WaitUntil` enum.
Removes the `-i`/`--interactive` CLI flag and live file-based
recording. Instead, the REPL now supports a `/load <path>` command
to run scripts from disk, and `/save` to export the in-memory
session recording.
The `Recorder` is simplified to be purely in-memory, and the script
runtime is moved to `src/script/Runtime.zig`.
BREAKING CHANGE: The `-i`/`--interactive` flag has been removed. Use
the `/save` and `/load` commands within the REPL instead.
Removes the `--self-heal` CLI option, the `scriptStep` and `scriptHeal`
MCP tools, and associated verification/iterator machinery. Replaces
"PandaScript" terminology with "slash commands" and moves shared
helpers to `tools.zig`.
BREAKING CHANGE: The `--self-heal` CLI flag and the `scriptStep` and
`scriptHeal` MCP tools have been removed.
* agent: add a /model command to chnage current model
And remove the pick-model CLI option
* agent: add /provider to change the current provider
* agent: extract requireLlmNoArg helper
* agent: simplify provider detection
* repl: add tab completion for /model and /provider
Changes `/model` and `/provider` to accept an optional name argument
instead of prompting with a numbered list. Bare commands now print the
current selection, while Tab dynamically completes candidates. Model
lists are fetched and cached to prevent redundant network requests.
* agent: remember last selected provider and model
Persists the last selected AI provider and model in a local
`.lp-agent` file and resumes it on startup. Removes the
interactive provider picker in favor of deterministic auto-detection.
* agent: simplify requireLlm and model resolution
Changes `requireLlm` to return a boolean instead of credentials, and
cleans up the model initialization logic to use `resolved` directly.
Also removes unused user errors.
---------
Co-authored-by: Adrià Arrufat <adria.arrufat@gmail.com>
fix: address various stability and reliability issues
- Cache stderr TTY check in Config to optimize log hot path.
- Cap signal handler attempts during no-hard-exit.
- Always duplicate URL segments to prevent invalid frees.
- Write live file before backup in atomic writes.
- Settle microtasks before verifying DOM elements.
Reserves the CLI flag and LP.configureLoading externalStylesheets field
so drivers can adopt the API before the fetch implementation lands in a
follow-up that depends on #2303.
The bool is intentionally unread in this PR. Mirrors the existing
--disable-subframes / --disable-workers plumbing; the CDP field extends
LP.configureLoading alongside subFrame and worker without breaking
existing callers.
Refs #2343
The `fetch` command is very practical to render pages without needing to
have a long running browser instance.
It is however masking all details on the fetch, most importantly the HTTP status code.
This is a big caveat when leveraging `lightpanda fetch` in a pipeline.
This introduces a `--json` option to provide a structured output that
contains:
* url
* HTTP status code
* response headers
* rendered content as controlled by the `--dump` option
The proposal is to always output the same JSON format even when not
using `--dump` with an option.
Adds the `-a` short flag, improves CLI validation for one-shot mode,
and ensures `--model` takes precedence over `--pick-model`.
BREAKING CHANGE: `--task-attachment` has been renamed to `--attach`.
`cli.zig` is now aware of `help` command at all situations and creates it by itself. Instead of using errors, it initializes `Command` union where `help` branch is active.
Adds validation logic to catch conflicting or missing CLI flags, such as
preventing `--task` with positional scripts and requiring a script for
`--self-heal`. Also adds a security warning to the help text regarding
arbitrary JavaScript execution in `.lp` files.
Moves the console message buffer from Frame to Session and populates it
via the notification system. This centralizes log collection for the
MCP tool and simplifies the Console Web API implementation.
Adds two ways to opt out of dedicated Web Worker loading entirely. The
Worker constructor still returns a Worker object so calling pages don't
throw, but no script fetch is initiated and the worker scope's eval
never runs (postMessage from the page is queued indefinitely with no
handler to drain it).
* CDP method LP.configureLoading { worker: bool } -- per-session
toggleable at runtime, alongside the existing { subFrame: bool }.
Both fields are now optional so callers can flip one without
resetting the other to its default. Backwards-compatible.
* CLI flag --disable-workers -- process-wide default applying to every
session and to the fetch subcommand. Operators can flip it on without
any driver changes. Mirrors --disable-subframes (#2401) exactly.
## Motivation
Reliably-reproducible SIGABRT in Worker.loadInitialScript whenever a
page constructs a Web Worker AND lightpanda is launched with
--http_proxy. Crash signature:
$msg="V8 fatal callback" location=v8::Context::Exit()
message="Cannot exit non-entered context"
Stack:
_browser.webapi.Worker.loadInitialScript
_browser.webapi.Worker.httpDoneCallback
_network.layer.InterceptionLayer.InterceptContext.doneCallback
_browser.HttpClient.processMessages
_browser.HttpClient.perform
_browser.HttpClient.tick
The Zig-side Enter/Exit pair around the worker's eval doesn't match
v8's entered_contexts stack invariant under that timing -- something
upstream of the loadInitialScript Exit leaves an extra Enter on the
stack, so v8's Utils::ApiCheck (`isolate->context() == *env`) fires
and the process aborts.
Reproducible against any Shopify storefront PDP (e.g.
https://weareallbirds.myshopify.com/products/mens-wool-runners) when
served through any HTTP proxy -- the proxy just adds enough latency
to surface the race; the same code path runs without --http_proxy
but the timing window is too tight to reliably hit. The Allbirds
trigger script is the Shopify web-pixel-extension worker, but ANY
Worker the page constructs hits the same code path.
The proper fix needs the v8 entered-contexts invariant to be
restored end-to-end through the worker eval. That's a deeper dig
into how Worker.loadInitialScript / WorkerGlobalScope.importScript /
ls.local.runMacrotasks compose with v8's microtask queues across
multiple contexts; I tried three intermediate fixes (deferring
loadInitialScript via the frame scheduler when other scripts are
mid-eval, replacing the post-eval cross-context runMacrotasks with
worker-only PerformCheckpoint, and removing runMacrotasks entirely)
and none stopped the crash. The bug is fired from inside the
synchronous tick path before the post-eval microtask handling
runs, which means the leak happens during Script::Run itself and
needs more targeted investigation.
This PR is the workaround so users hitting the SIGABRT on
storefront / analytics-heavy pages have a clean opt-in escape today.
For our use case (product catalog extraction) Workers carry no
extraction signal -- web-pixel sandboxes, analytics SDKs, marketing
tag pixels, etc. -- so disabling them removes a fragile code path
without any downside.
## Implementation
`Session.worker_loading_enabled: bool = true` -- default matches
existing behavior.
`Worker.init` short-circuits AFTER constructing the Worker /
WorkerGlobalScope / arena bookkeeping (so the JS `new Worker(url)`
expression doesn't throw):
if (!session.worker_loading_enabled) {
log.debug(.browser, "worker disabled", .{ .url = resolved_url });
return self;
}
Two ways to flip the flag, mirroring the --disable-subframes pattern:
1. LP.configureLoading { worker: bool } -- both subFrame and worker
are now optional fields in the params struct, so existing callers
passing only { subFrame } continue to work unchanged.
2. --disable-workers CLI flag -- added to CommonOptions (so it
applies to serve, fetch, mcp). New Config.disableWorkers() getter;
Session.init reads it as the initial value.
Total diff: +88 / -3 across 4 files (src/Config.zig,
src/browser/Session.zig, src/browser/webapi/Worker.zig,
src/cdp/domains/lp.zig).
## Verification
Reproducer pattern (puppeteer-core 24.42.0 + tiny CONNECT-tunnel
proxy on 127.0.0.1:9999, scripts in cdp-repros/):
serve --host 127.0.0.1 --port 9222 --http_proxy http://127.0.0.1:9999
serve --host 127.0.0.1 --port 9222 --http_proxy http://127.0.0.1:9999 --disable-workers
Driving https://weareallbirds.myshopify.com/products/mens-wool-runners:
baseline (no --disable-workers): 5/5 SIGABRT in
Worker.loadInitialScript with the v8 fatal callback above.
with --disable-workers: 10/10 successful, returns full
HTML (~1MB), no crash.
Test suite:
make test -> 637 of 637 tests passed (was 636/636 + new
cdp.lp: configureLoading toggles subFrame and worker
independently regression test).
zig fmt --check ./*.zig ./**/*.zig -> clean.
## Notes
* The CDP method is the same domain (LP.configureLoading) and same
shape as --disable-subframes' driver-side opt-in, so existing
Playwright / puppeteer integrations that already toggle
subframes don't need a separate code path -- one CDP call can
flip both.
* worker_loading_enabled = false does NOT remove Worker from the
global namespace (so feature-detection like
`if (typeof Worker !== 'undefined')` still reports true). It just
makes constructed workers no-op. Pages that postMessage to a worker
and wait for a response will hang on that promise forever (or
until the page is torn down). For our extraction use case that's
fine -- we control the worklist timeout anyway -- but it's worth
noting if upstream wants to surface the disabled state more
strongly (e.g. throw from postMessage, or remove the global
entirely behind an even-stricter flag).
* Once the underlying v8 entered-contexts invariant is restored in
Worker.loadInitialScript, this flag becomes a perf / sandboxing
tool rather than a correctness workaround. Worth keeping anyway:
blocking analytics / pixel workers is a reasonable thing to want.
## Related
* #2400 -- the iframe analog to this issue (subframe nav invalidates
executionContextId); same workaround pattern.
* #2401 -- introduced --disable-subframes / LP.configureLoading
{ subFrame } that this PR mirrors exactly for workers.
`lightpanda <subcommand> help`, `lightpanda <subcommand> --help`
now print only the relevant subcommand options plus common options,
instead of the full text.
`lightpanda help <subcommand>` is also supported
(and that's what use internally).
Karl Seguin