Commit Graph

72 Commits

Author SHA1 Message Date
Halil Durak
f5b9bdb51b rebase and backport new feature from main 2026-04-22 16:07:04 +03:00
Halil Durak
85e624356b Commands: more aliases 2026-04-22 16:07:03 +03:00
Halil Durak
886448aaa3 Commands: add shortcuts for host and port of serve 2026-04-22 16:07:02 +03:00
Halil Durak
10914d6288 cli: fix --log-filter-scopes regression 2026-04-22 16:07:02 +03:00
Halil Durak
56b6fbe011 cli: many improvements
* Options with optional types are introduced to null by default
* Options with boolean types cannot be optional (nullable)
* Options with boolean types are introduced with false by default
* introduce shortcuts for options that can be provided via single dash
* introduce validators; custom parsing logic can be inserted for niche cases
2026-04-22 16:07:01 +03:00
Halil Durak
81b89e67b7 Config: adapt to new CLI builder 2026-04-22 16:07:01 +03:00
Karl Seguin
6d0003ad2b Sqlite
This adds an app.storage which is a union around configurable storage engine
(currently, only sqlite).

It is _not_ being used anywhere right now. The goal is to get feedback on
the implementation and then move cache to it.

This doesn't expose a generic query API. The goal is that the storage will
expose high level methods, e.g. `cacheGet(req: CacheGetRequest)` and every
storage engine will translate the `storage.CacheGetRequest` as needed.

A thin wrapper around the Sqlite C api is included, e.g. exec(SQL, .{args}) a
`rows` and `row` fetcher. A connection pool is included. By default, an
in-memory DB is currently created. And a `migrations` table with an id of `1`
is created/inserted. I don't imagine needing fancy migratations.
2026-04-20 17:13:06 +08:00
Karl Seguin
2d20e57f80 Change all @import("...../log.zig") to const log = lp.log;
@import("lightpanda") where needed.

Would also like to do this for String, Page, Session and js which all stand out
as types that are use across the codebase.

I know that a few devs are doing this in new work and I haven't heard anyone
voice an objection.
2026-04-20 12:40:04 +08:00
Pierre Tachoire
e922bf7369 update help for --cookies and --cookie-jar option 2026-04-15 10:48:23 -04:00
Matt Van Horn
35991a1b32 refactor: split --cookies-file into --cookie/--cookie-jar per curl convention
Split the single --cookies-file flag into two flags following curl's
convention as requested by @krichprollsch:

- --cookie (read-only): loads cookies at startup for fetch, mcp, and
  serve/CDP commands
- --cookie-jar (write-only): saves cookies on exit for fetch and mcp
  only (CDP cookie-jar deferred per maintainer guidance)

Add cookie integration to MCP server (load in init, save in deinit)
and CDP session creation (load only). The serve command now rejects
--cookie-jar with a clear error message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:09:13 -04:00
Matt Van Horn
494ef345a1 fix: resolve Zig 0.15 compilation errors in cookies and config
- Escape braces in help text format string to avoid std.debug.print
  interpreting them as format specifiers
- Use writer.interface for std.Io.Writer methods (writeAll, stringify)
  instead of calling them directly on fs.File.Writer
- Replace writer.flush() with writer.end() per codebase convention

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:09:13 -04:00
Matt Van Horn
4d384dfe01 feat: add --cookies-file flag for session persistence
Add a --cookies-file CLI option that loads cookies from a JSON file
at startup and saves them back on exit. This enables AI agents to
maintain login sessions across multiple Lightpanda invocations.

The cookie format matches CDP Network.Cookie (compatible with
Puppeteer's page.cookies() export):
  [{"name":"sid","value":"abc","domain":".example.com","path":"/",
    "expires":1234567890,"secure":true,"httpOnly":true}]

Closes #335

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 10:09:12 -04:00
Karl Seguin
5c33c8fc3c Merge pull request #2158 from lightpanda-io/websocket_tweaks
WS.close returns DOMException
2026-04-15 07:08:37 +08:00
Karl Seguin
eb0af793c2 WS.close returns DOMException
Cleanup WS after sending disconnetion events

Mapping support for []f32 and []f64

Default max WS connections 8 -> 64
2026-04-14 21:58:43 +08:00
Karl Seguin
5226df9968 Tweak tests
De-duplicate user agent validation
2026-04-14 17:11:54 +08:00
Pierre Tachoire
1589445ec0 zig fmt 2026-04-13 10:36:35 +02:00
zed
9fe3a48c3a test: add tests for setting CDP user agent 2026-04-13 10:23:54 +02:00
Pierre Tachoire
a4617390de Merge pull request #2104 from lightpanda-io/feat/add-ip-filter
Feat/add ip filter
2026-04-10 08:46:06 +02:00
Karl Seguin
8eaeafe16c Fix a lot of typos.
I used https://github.com/crate-ci/typos, it worked well.

Also, make sure cdp-initiated KeyboardEvent is freed when no element is in focus
2026-04-10 06:51:10 +08:00
Pierre Tachoire
f884b562ba user-agent ovveride must not contain mozilla 2026-04-08 12:11:09 +02:00
Pierre Tachoire
efb2fa9c22 Send Sec-Ch-Ua http header 2026-04-08 12:11:09 +02:00
Pierre Tachoire
ae9b4d3fc6 stricter user-agent rule 2026-04-08 12:11:09 +02:00
Trevin Chow
f0aacad52e feat: add --user-agent flag for full User-Agent override
When --user-agent is set, the provided string replaces the entire
User-Agent header instead of appending to "Lightpanda/1.0".
The existing --user-agent-suffix behavior is unchanged.

Fixes #2029
2026-04-08 12:11:08 +02:00
Lucien Coffe
7f5abfc9cf fix: use dashes in CLI flag names for consistency
Rename --block_private_networks to --block-private-networks and
--block_cidrs to --block-cidrs to match the existing flag naming
convention (e.g. --http-proxy, --proxy-bearer-token).
2026-04-08 12:10:46 +02:00
Lucien Coffe
fb6c4e4978 feat: add allow-list exclusions to --block_cidrs
CIDRs prefixed with '-' are treated as allow rules that exempt matching
IPs from blocking. Allow rules take precedence over both
--block_private_networks and custom block CIDRs.

Example: --block_private_networks --block_cidrs -10.0.0.42/32
blocks all private ranges except 10.0.0.42.

Adds 3 new tests for allow-list behavior.
2026-04-08 12:10:46 +02:00
Lucien Coffe
f5cfc4d315 feat: add --block_private_networks and --block_cidrs CLI flags
Block outbound HTTP requests to specified IP ranges before TCP handshake
using libcurl CURLOPT_OPENSOCKETFUNCTION callback. Fires after DNS
resolution, reads resolved IP directly from sockaddr, does bitwise CIDR
comparison. Fail-closed: unknown address families are blocked.

--block_private_networks blocks RFC1918, localhost, link-local, ULA.
--block_cidrs blocks additional comma-separated CIDRs.
IPv4-mapped IPv6 (::ffff:x.x.x.x) is unwrapped to prevent bypass.
2026-04-08 12:10:42 +02:00
Adrià Arrufat
afd6c11b34 Config: remove mcp version flag and simplify usage 2026-04-07 12:24:19 +02:00
Karl Seguin
14dcb7895a Give websockets their own connection pool, improve websocket message logging 2026-04-04 07:00:24 +08:00
Muki Kiboigo
ca5fa2b866 change --cache-dir -> --http-cache-dir 2026-04-03 07:23:32 -07:00
Muki Kiboigo
619a2653d1 update cacheDir config option 2026-04-03 07:23:31 -07:00
Muki Kiboigo
6633b6effc add cache dir to configuration opts 2026-04-03 07:22:20 -07:00
Karl Seguin
b6020e4770 Merge pull request #2066 from lightpanda-io/fix/agent-integration-and-mcp-cdp
mcp: improve navigation reliability and add CDP support
2026-04-03 08:27:55 +08:00
Karl Seguin
0604056f76 Improve network naming consistency
1.
Runtime.zig -> Network.zig (especially since most places imported it as
`const Network = @import("Runtime.zig")`

2.
const net_http = @import(...) -> const http = @import(...)
2026-04-01 18:46:03 +08:00
Adrià Arrufat
58fc60d669 mcp: improve navigation reliability and add CDP support
- Configurable navigation timeouts and wait strategies in MCP tools.
- Default navigation timeout increased from 2s to 10s.
- Added navigate, eval, and screenshot MCP tools.
- Supported running a CDP server alongside MCP using --cdp-port.
- Fixed various startup crashes when running CDP in MCP mode.
- Hardened MCP server error handling.
2026-04-01 12:41:56 +02:00
Karl Seguin
ab6c63b24b Add --wait-selector, --wait-script and --wait-script-file options to fetch
These new optional parameter run AFTER --wait-until, allowing the (imo) useful
combination of `--wait-until load --wait-script "report.complete === true"`.
However, if `--wait-until` IS NOT specified but `--wait-selector/script` IS,
then there is no default wait and it'll just check the selector/script. If
neither `--wait-selector` or `--wait-script/--wait-script-file` are specified
 then  `--wait-until` continues to default to `done`.

These waiters were added to the Runner, and the existing Action.waitForSelector
now uses the runner's version. Selector querying has been split into distinct
parse and query functions, so that we can parse once, and query on every tick.

We could potentially optimize --wait-script to compile the script once and call
it on each tick, but we'd have to detect page navigation to recompile the script
in the new context. Something I'd rather optimize separately.
2026-03-31 12:30:46 +08:00
Adrià Arrufat
d99599fa21 zig fmt 2026-03-30 07:24:08 +02:00
Adrià Arrufat
20e62a5551 mcp: inline mcpVersion helper from Config 2026-03-30 07:13:45 +02:00
Adrià Arrufat
e083d4a3d1 Config: remove LIGHTPANDA_MCP_VERSION env var 2026-03-30 07:07:23 +02:00
Adrià Arrufat
81d4bdb157 mcp: change default protocol version to 2024-11-05 2026-03-29 08:34:24 +02:00
Adrià Arrufat
cf5e4d7d1e mcp: allow configuring protocol version
Closes #2023
2026-03-29 08:29:04 +02:00
Karl Seguin
679f2104f4 Fix --wait-until default value.
This was `load`, but it should have been (and was documented as `done`). This
is my fault. Sorry.

Should help with: https://github.com/lightpanda-io/browser/issues/1947#issuecomment-4120597764
2026-03-26 18:06:14 +08:00
Karl Seguin
61dabdedec Support (and prefer) dash-separated command line arguments
--log_level -> --log-level

Underscored arguments are still supported for backwards compatibility.
2026-03-24 12:55:08 +08:00
Karl Seguin
5453630955 Merge pull request #1958 from lightpanda-io/runner
Extract Session.wait into a Runner
2026-03-24 07:28:18 +08:00
Karl Seguin
c9bc370d6a Extract Session.wait into a Runner
This is done for a couple reasons. The first is just to have things a little
more self-contained for eventually supporting more advanced "wait" logic, e.g.
waiting for a selector.

The other is to provide callers with more fine-grained controlled. Specifically
the ability to manually "tick", so that they can [presumably] do something
after every tick. This is needed by the test runner to support more advanced
cases (cases that need to test beyond 'load') and it also improves (and fixes
potential use-after-free, the lp.waitForSelector)
2026-03-23 12:30:41 +08:00
Karl Seguin
4e2e895cd9 Add --advertise_host option to serve command
Allows overwriting the --host for the json/version payload. When --host is set
to 0.0.0.0, we want to provide a mechanism to specify the specific address to
connect to in /json/version (or anywhere else that we "advertise" the address).

Inspired by https://github.com/lightpanda-io/browser/pull/1923 but rather than
defaulting to 127.0.0.1 (which seems just as unsafe), adds the explicit config
option.
2026-03-22 12:40:17 +08:00
shaewe180
e2be8525c4 Config: remove js_enum_from_string constant 2026-03-19 09:40:40 +08:00
shaewe180
c15afa23ca Session: fix page pointer handling in wait loop
- Refactor `wait` and `_wait` to handle `page` as `*Page` instead of `**Page`, preventing stale references during navigations.
- Update `networkidle` wait condition to use `_notified_network_idle == .done`.
- Document `--wait_ms` and `--wait_until` options in `Config.zig` help text.
2026-03-19 09:36:42 +08:00
shaewe180
09327c3897 feat: fetch add wait_until parameter for page loads options
Add `--wait_until` and `--wait_ms` CLI arguments to configure session wait behavior. Updates `Session.wait` to evaluate specific page load states (`load`, `domcontentloaded`, `networkidle`, `fixed`) before completing the wait loop.
2026-03-18 15:08:51 +08:00
Muki Kiboigo
d365240f91 fix cli argument for WebBotAuth domain 2026-03-12 09:04:12 -07:00
Muki Kiboigo
02198de455 add support for WebBotAuth in Client 2026-03-12 09:04:10 -07:00