Commit Graph

178 Commits

Author SHA1 Message Date
Francis Bouvier
8052f0ad81 Agent model provider picker (#2581)
* agent: add a /model command to chnage current model

And remove the pick-model CLI option

* agent: add /provider to change the current provider

* agent: extract requireLlmNoArg helper

* agent: simplify provider detection

* repl: add tab completion for /model and /provider

Changes `/model` and `/provider` to accept an optional name argument
instead of prompting with a numbered list. Bare commands now print the
current selection, while Tab dynamically completes candidates. Model
lists are fetched and cached to prevent redundant network requests.

* agent: remember last selected provider and model

Persists the last selected AI provider and model in a local
`.lp-agent` file and resumes it on startup. Removes the
interactive provider picker in favor of deterministic auto-detection.

* agent: simplify requireLlm and model resolution

Changes `requireLlm` to return a boolean instead of credentials, and
cleans up the model initialization logic to use `resolved` directly.
Also removes unused user errors.

---------

Co-authored-by: Adrià Arrufat <adria.arrufat@gmail.com>
2026-05-30 19:02:44 +02:00
Adrià Arrufat
fbec560292 Ref: #123
fix: address various stability and reliability issues

- Cache stderr TTY check in Config to optimize log hot path.
- Cap signal handler attempts during no-hard-exit.
- Always duplicate URL segments to prevent invalid frees.
- Write live file before backup in atomic writes.
- Settle microtasks before verifying DOM elements.
2026-05-25 18:05:11 +02:00
Adrià Arrufat
7df1e80ce5 config: fix agent mode 2026-05-21 13:49:08 +02:00
Adrià Arrufat
54482831ed Merge branch 'main' into agent 2026-05-20 17:33:07 +02:00
Pierre Tachoire
af6175cc56 rewrite help in a more compact way
Inspired by go help output
2026-05-20 16:05:17 +02:00
Francis Bouvier
dffa961f45 Remove options from main help 2026-05-20 16:05:13 +02:00
Adrià Arrufat
989932b40a Merge branch 'main' into agent 2026-05-20 14:57:32 +02:00
Pierre Tachoire
2ad2c9d878 Merge pull request #2487 from navidemad/feat/external-stylesheets-flag
Add --enable-external-stylesheets flag with fetch + parse
2026-05-20 13:41:59 +02:00
Karl Seguin
b6fd09c5ab Merge pull request #2502 from lightpanda-io/max-cdp-conn
by using httpClient, fetch generates a call to Config.maxConnections
2026-05-20 16:28:56 +08:00
Pierre Tachoire
6eb25d5c44 by using httpClient, fetch generates a call to Config.maxConnections 2026-05-20 10:14:17 +02:00
Navid EMAD
6ed41ea346 Add --enable-external-stylesheets flag (no-op surface)
Reserves the CLI flag and LP.configureLoading externalStylesheets field
so drivers can adopt the API before the fetch implementation lands in a
follow-up that depends on #2303.

The bool is intentionally unread in this PR. Mirrors the existing
--disable-subframes / --disable-workers plumbing; the CDP field extends
LP.configureLoading alongside subFrame and worker without breaking
existing callers.

Refs #2343
2026-05-19 15:50:11 +02:00
Marc Helbling
a89a28a4a2 feat: add --json to fetch command
The `fetch` command is very practical to render pages without needing to
have a long running browser instance.
It is however masking all details on the fetch, most importantly the HTTP status code.
This is a big caveat when leveraging `lightpanda fetch` in a pipeline.

This introduces a `--json` option to provide a structured output that
contains:
* url
* HTTP status code
* response headers
* rendered content as controlled by the `--dump` option

The proposal is to always output the same JSON format even when not
using `--dump` with an option.
2026-05-19 12:08:23 +02:00
Adrià Arrufat
8d102b8511 Merge branch 'main' into agent 2026-05-19 11:33:46 +02:00
Adrià Arrufat
8d250ac7b0 agent: rename --task-attachment to --attach
Adds the `-a` short flag, improves CLI validation for one-shot mode,
and ensures `--model` takes precedence over `--pick-model`.

BREAKING CHANGE: `--task-attachment` has been renamed to `--attach`.
2026-05-19 11:02:46 +02:00
Pierre Tachoire
23a3d5476b Merge pull request #2458 from lightpanda-io/nikneym/cli-help-rework
`help`: rework `help` command
2026-05-18 11:54:29 +02:00
Adrià Arrufat
de00deb899 Merge branch 'main' into agent 2026-05-17 19:13:54 +02:00
Halil Durak
b2d8c2b834 help.zon: introduce help.zon
Separates `help` explanation from configuration.
2026-05-15 18:31:09 +03:00
Halil Durak
f361f12316 cli.zig: change the way help command and sub-command detected
`cli.zig` is now aware of `help` command at all situations and creates it by itself. Instead of using errors, it initializes `Command` union where `help` branch is active.
2026-05-15 18:31:09 +03:00
Halil Durak
9c40cd9fb2 send Accept header when navigating 2026-05-15 18:30:18 +03:00
Adrià Arrufat
882ed74e89 agent: update zenai and default models 2026-05-14 19:35:38 +02:00
Adrià Arrufat
e8ac1bada7 agent: make verbosity default context-aware 2026-05-14 17:49:05 +02:00
Adrià Arrufat
dbd0197576 cli: validate conflicting flags and add security warning
Adds validation logic to catch conflicting or missing CLI flags, such as
preventing `--task` with positional scripts and requiring a script for
`--self-heal`. Also adds a security warning to the help text regarding
arbitrary JavaScript execution in `.lp` files.
2026-05-13 09:58:26 +02:00
Adrià Arrufat
15f06474cf refactor(browser): move console log buffering to Session
Moves the console message buffer from Frame to Session and populates it
via the notification system. This centralizes log collection for the
MCP tool and simplifies the Console Web API implementation.
2026-05-13 08:57:44 +02:00
Adrià Arrufat
71e39e9df3 Merge branch 'main' into agent 2026-05-13 08:48:41 +02:00
Scott Taylor
b2998470c2 Add --disable-workers + LP.configureLoading worker opt-out
Adds two ways to opt out of dedicated Web Worker loading entirely. The
Worker constructor still returns a Worker object so calling pages don't
throw, but no script fetch is initiated and the worker scope's eval
never runs (postMessage from the page is queued indefinitely with no
handler to drain it).

* CDP method LP.configureLoading { worker: bool } -- per-session
  toggleable at runtime, alongside the existing { subFrame: bool }.
  Both fields are now optional so callers can flip one without
  resetting the other to its default. Backwards-compatible.
* CLI flag --disable-workers -- process-wide default applying to every
  session and to the fetch subcommand. Operators can flip it on without
  any driver changes. Mirrors --disable-subframes (#2401) exactly.

## Motivation

Reliably-reproducible SIGABRT in Worker.loadInitialScript whenever a
page constructs a Web Worker AND lightpanda is launched with
--http_proxy. Crash signature:

    $msg="V8 fatal callback" location=v8::Context::Exit()
    message="Cannot exit non-entered context"
    Stack:
      _browser.webapi.Worker.loadInitialScript
      _browser.webapi.Worker.httpDoneCallback
      _network.layer.InterceptionLayer.InterceptContext.doneCallback
      _browser.HttpClient.processMessages
      _browser.HttpClient.perform
      _browser.HttpClient.tick

The Zig-side Enter/Exit pair around the worker's eval doesn't match
v8's entered_contexts stack invariant under that timing -- something
upstream of the loadInitialScript Exit leaves an extra Enter on the
stack, so v8's Utils::ApiCheck (`isolate->context() == *env`) fires
and the process aborts.

Reproducible against any Shopify storefront PDP (e.g.
https://weareallbirds.myshopify.com/products/mens-wool-runners) when
served through any HTTP proxy -- the proxy just adds enough latency
to surface the race; the same code path runs without --http_proxy
but the timing window is too tight to reliably hit. The Allbirds
trigger script is the Shopify web-pixel-extension worker, but ANY
Worker the page constructs hits the same code path.

The proper fix needs the v8 entered-contexts invariant to be
restored end-to-end through the worker eval. That's a deeper dig
into how Worker.loadInitialScript / WorkerGlobalScope.importScript /
ls.local.runMacrotasks compose with v8's microtask queues across
multiple contexts; I tried three intermediate fixes (deferring
loadInitialScript via the frame scheduler when other scripts are
mid-eval, replacing the post-eval cross-context runMacrotasks with
worker-only PerformCheckpoint, and removing runMacrotasks entirely)
and none stopped the crash. The bug is fired from inside the
synchronous tick path before the post-eval microtask handling
runs, which means the leak happens during Script::Run itself and
needs more targeted investigation.

This PR is the workaround so users hitting the SIGABRT on
storefront / analytics-heavy pages have a clean opt-in escape today.
For our use case (product catalog extraction) Workers carry no
extraction signal -- web-pixel sandboxes, analytics SDKs, marketing
tag pixels, etc. -- so disabling them removes a fragile code path
without any downside.

## Implementation

`Session.worker_loading_enabled: bool = true` -- default matches
existing behavior.

`Worker.init` short-circuits AFTER constructing the Worker /
WorkerGlobalScope / arena bookkeeping (so the JS `new Worker(url)`
expression doesn't throw):

    if (!session.worker_loading_enabled) {
        log.debug(.browser, "worker disabled", .{ .url = resolved_url });
        return self;
    }

Two ways to flip the flag, mirroring the --disable-subframes pattern:

1. LP.configureLoading { worker: bool } -- both subFrame and worker
   are now optional fields in the params struct, so existing callers
   passing only { subFrame } continue to work unchanged.
2. --disable-workers CLI flag -- added to CommonOptions (so it
   applies to serve, fetch, mcp). New Config.disableWorkers() getter;
   Session.init reads it as the initial value.

Total diff: +88 / -3 across 4 files (src/Config.zig,
src/browser/Session.zig, src/browser/webapi/Worker.zig,
src/cdp/domains/lp.zig).

## Verification

Reproducer pattern (puppeteer-core 24.42.0 + tiny CONNECT-tunnel
proxy on 127.0.0.1:9999, scripts in cdp-repros/):

  serve --host 127.0.0.1 --port 9222 --http_proxy http://127.0.0.1:9999
  serve --host 127.0.0.1 --port 9222 --http_proxy http://127.0.0.1:9999 --disable-workers

Driving https://weareallbirds.myshopify.com/products/mens-wool-runners:

  baseline (no --disable-workers): 5/5 SIGABRT in
    Worker.loadInitialScript with the v8 fatal callback above.

  with --disable-workers:           10/10 successful, returns full
    HTML (~1MB), no crash.

Test suite:

  make test  -> 637 of 637 tests passed (was 636/636 + new
    cdp.lp: configureLoading toggles subFrame and worker
    independently regression test).

  zig fmt --check ./*.zig ./**/*.zig  -> clean.

## Notes

* The CDP method is the same domain (LP.configureLoading) and same
  shape as --disable-subframes' driver-side opt-in, so existing
  Playwright / puppeteer integrations that already toggle
  subframes don't need a separate code path -- one CDP call can
  flip both.

* worker_loading_enabled = false does NOT remove Worker from the
  global namespace (so feature-detection like
  `if (typeof Worker !== 'undefined')` still reports true). It just
  makes constructed workers no-op. Pages that postMessage to a worker
  and wait for a response will hang on that promise forever (or
  until the page is torn down). For our extraction use case that's
  fine -- we control the worklist timeout anyway -- but it's worth
  noting if upstream wants to surface the disabled state more
  strongly (e.g. throw from postMessage, or remove the global
  entirely behind an even-stricter flag).

* Once the underlying v8 entered-contexts invariant is restored in
  Worker.loadInitialScript, this flag becomes a perf / sandboxing
  tool rather than a correctness workaround. Worth keeping anyway:
  blocking analytics / pixel workers is a reasonable thing to want.

## Related

* #2400 -- the iframe analog to this issue (subframe nav invalidates
  executionContextId); same workaround pattern.
* #2401 -- introduced --disable-subframes / LP.configureLoading
  { subFrame } that this PR mirrors exactly for workers.
2026-05-12 23:46:45 -04:00
Adrià Arrufat
19f8a1a2e5 config: handle agent mode in disableSubframes 2026-05-12 09:00:07 +02:00
Adrià Arrufat
7d312ce5e9 Merge branch 'main' into agent 2026-05-12 08:56:29 +02:00
Karl Seguin
e9b2aa4946 Merge pull request #2426 from lightpanda-io/feat/cdp-disable-iframes
Feat/cdp disable iframes
2026-05-12 13:16:16 +08:00
Karl Seguin
cdfabf7953 Minor tweaks
Remove repeating description/comment of flag.

Change CDP lp command to be a general configuration endpoint.
2026-05-12 12:56:33 +08:00
Francis Bouvier
31ef5246bc Add per-subcommand help via help or --help argument
`lightpanda <subcommand> help`, `lightpanda <subcommand> --help`
now print only the relevant subcommand options plus common options,
instead of the full text.

`lightpanda help <subcommand>` is also supported
(and that's what use internally).
2026-05-11 22:16:52 +02:00
Adrià Arrufat
75b7a8ec6d Merge branch 'main' into agent 2026-05-11 17:52:35 +02:00
Halil Durak
19401dc950 Config: update --inject-script documentation 2026-05-11 15:15:36 +03:00
Halil Durak
60d721caa2 Config: increase read file size for --inject-script-file 2026-05-11 15:15:36 +03:00
Halil Durak
246f91d1f8 --inject-script: prefer splice bytes into <head> directly 2026-05-11 15:15:35 +03:00
Halil Durak
c566d0c41c introduce --inject-script and --inject-script-file
* Prefer `--inject-*` prefix.
* Support injecting multiple scripts (also allows using both variants together).
* Instead of executing scripts in JS context, actually insert them to `<head>` for correct dump output.
2026-05-11 15:15:35 +03:00
Halil Durak
39f12a5669 fetch: add support for --script option
Allows passing a JS file as an arg to be executed alongside other scripts.
2026-05-11 15:15:35 +03:00
Adrià Arrufat
938795ec8d agent: add --pick-model for interactive selection 2026-05-09 20:11:52 +02:00
Adrià Arrufat
16b83f5093 agent: add provider auto-detection and --no-llm flag 2026-05-09 19:52:42 +02:00
Adrià Arrufat
357033eb0c agent: add --list-models flag 2026-05-09 19:16:48 +02:00
Scott Taylor
b272b0e33c Add --disable-subframes CLI flag
Complementary to LP.setSubframeLoading (preceding commit): exposes
the same iframe-skip behavior as a CLI option that applies to all
sessions in the process. Useful for:

  * the 'fetch' subcommand (no CDP driver to call LP.setSubframeLoading)
  * 'serve' deployments where the operator wants iframes off by
    default for every connecting client (the LP method can still
    re-enable per-session if needed)
  * Playwright's chromium.connectOverCDP, which can't reliably issue
    custom CDP methods on Lightpanda today: BrowserContext.newCDPSession
    and Browser.newBrowserCDPSession both attach a new CRSession that
    collides with the STARTUP-session reuse from #2399, triggering a
    Playwright internal assertion. With --disable-subframes set on the
    server, Playwright doesn't need to issue any custom CDP \u2014 every
    session inherits subframes-off and the executionContextId churn
    from #2400 never trips.

Verified:

  serve --disable-subframes + plain puppeteer-core goto
    [ok] goto status=200 elapsed=6354ms frameAttached=0

  fetch --disable-subframes --dump html https://www.allbirds.com/...
    exit=0
    html bytes: 1021562
    title: <title>Allbirds Wool Runners, Men's | ...</title>
    iframe count in dumped html: 2  (still in DOM, just not loaded)

521/521 unit tests pass.
2026-05-08 17:12:54 -04:00
Adrià Arrufat
372c0a12a2 agent: simplify logic and cleanup comments 2026-05-08 08:23:19 +02:00
Adrià Arrufat
7bf69a9a34 agent: remove integrated mcp server
Removes the `--mcp` flag and the internal `task` tool from the agent.
Users should use `lightpanda mcp` for external agent integrations.
2026-05-07 17:12:46 +02:00
Adrià Arrufat
a254e4041a agent: refine verbosity levels and tool output UI 2026-05-07 16:11:28 +02:00
Adrià Arrufat
8e9e82b478 cli: support short flags and hide tool results in REPL 2026-05-07 14:22:26 +02:00
Adrià Arrufat
25f8dfcab3 refactor: consolidate common logic and structs
- Extract `modeNeedsHttp` helper in `Config.zig`.
- Move `CallParams` to `protocol.zig` to deduplicate MCP code.
- Extract `lookupLpEnv` in `browser/tools.zig` for environment lookups.
2026-05-07 10:11:25 +02:00
Adrià Arrufat
93d08a486b agent: add --verbosity flag to control stderr output 2026-05-07 08:15:51 +02:00
Adrià Arrufat
eb14783af7 agent: rename Pandascript to PandaScript and .panda to .lp 2026-04-30 17:29:36 +02:00
Adrià Arrufat
300fdfb34c agent: add MCP server mode with task tool 2026-04-30 17:11:48 +02:00
Adrià Arrufat
d4d8cb6450 Merge branch 'main' into agent 2026-04-28 17:58:32 +02:00
Halil Durak
c5b16cb18e Config: add a custom validator for --log-level 2026-04-28 12:37:16 +03:00