Assuming the conditions are met for -dev_fast to work (linux, debug, x86-64
without sanitizers), the -dev_fast now defaults to true.
download-v8 make target always downloads the .a (as before) and on linux x86-64
it also downloads the .so.
Supports prebuild-v8 shared library and adds a `download-v8-shared` make target.
Depends on: https://github.com/lightpanda-io/zig-v8-fork/pull/198
The auto-discovery of the .so is done in the build.zig, which is different than
the auto-discovery of the .a which is done in the Makefile. This is intentional
and if we inverse -Ddev_fast so that it's on by default, the .a's auto discovery
will be moved to build.zig for consistency and explicitness. As-is, we have no
way to tell whether -Dprebuilt_v8_path=v8.a is being injected by Makefile or
is explicit (which isn't a problem so long as we're not doing dev_fast by
default).
Zig 0.17 removes @cImport in favor of translate-c steps in the build
system. Each library's link function now creates an addTranslateC step
exposed as a module import: curl and isocline translate their headers
straight from the dependency tree, while sqlite3 uses the artifact's
emitted include tree since the amalgamation lives in a sub-dependency.
Terminal.zig and prompt_assist.zig previously instantiated two
independent @cImport namespaces for isocline; they now share one
translated module, so its types are identical across both files.
Make sure jemallocator is disabled in Release builds (we only use it in Debug
builds to collect metrics. Supposedly something changed in our 0.16 build that
causes it to be initialized even though it isn't used. Not sure, for this, I'm
just doing what Claude tell me, but disabling something in Release that I know
we aren't using sounds ok).
Built against https://github.com/lightpanda-io/zig-v8-fork/tree/zig-0.16 but
it doesn't require a new v8 build.
Built against https://github.com/lightpanda-io/boringssl-zig/tree/zig-0.16
since the current fork we point to isn't updated.
A global std.Io instance, lp.io. Way easier this way and requires 0 changes to
our libcurl integration / event loop.
Network code uses a new layer that does what Zig 0.15's posix package used to
do. Again, quicker migration that way. But, as long as we have the global IO,
and given the half-baked nature of networking in std.Io 0.16, this just makes
sense. Things can be migrated as needed.
The std.time.* -> std.Io.Timestamp/Clock/Duration resulted in _a lot_ of
changes. ArrayList = .{} -> ArrayList -> .empty also resulted in a lot of
changes, but that's obviously superficial. As is the trimLeft/trimRight ->
trimStart/trimEnd rename.
Locking adopt the `Uncancelable` variants, e.g. mutex.lockUncancelable() to
preserve the error-free signature (and, because cancellation would be something
we'd have to put more thought into).
std.json.ObjectMap is now unmanaged, so the allocator had to be passed along.
However, there's still a deprecated managed variant of MemoryPool, so I switched
to it (we can do a small follow up PR to move to the unmanaged after).
I tried use_llvm = false, but it locks my computer, consuming RAM until MacOS
gives me a popup I've never seen before, begging me to start killing processes.
Agent and the networking stuff saw the most significant changes.
The /save script documentation lived as a hand-maintained string in
Agent.zig whose primitives table drifted whenever a tool changed. Move
it to src/script/skill.zig and render the reference (signatures, option
lists, enums, defaults, per-parameter descriptions) from Schema.all()
at first use, keeping the curated per-tool notes behind an exhaustive
switch on Tool so a new or renamed tool is a compile error until its
doc entry exists.
The rendered skill is shared by three consumers:
- the /save and revision system prompts (built lazily in Agent.zig)
- a new mcp://skill/pandascript resource
- `zig build skills`, which writes zig-out/skills/<name>/SKILL.md with
Claude Code frontmatter via a registry-based generator exe, so
future Lightpanda skills are one registry entry each
Schema.FieldEntry now retains per-parameter schema descriptions, which
previously existed only in the raw JSON.
This is a bit all over the place.
1 - Replace libidn2 with rust-idna. It looks like there are different idna
profiles, and rust-idna (from the servo project) implements the whatwg
one. libidn2 would be too strict in some cases and not strict enough in
others. (Gemini says I could use libidn2 for this, but what it suggested
didn't work, and I couldn't figure it out myself, and claude insisted it
_did not_ have the correct implementation for what we want).
2 - We previously only ran a URL through idna if it wasn't ascii. Turns out
we also need to run it if there's a "xn--" (aka, an IDNA ACE prefix) in
there. This helps us pass hundreds of WPT cases, and it's pretty cheap.
3 - Implement more of the Area WebAPI. Mostly copied from Anchor.
4 - Add username/password accessor to Anchor/Area
5 - window.open validates the URL (i.e. tries to resolve it and handles the
error)
6 - Invalid idna conversion maps to a TypeError
7 - Cleanup closed popups on the next tick (like destroyed pages), rather than
at an interval or on shutdown. This one seems unrelated, but some of these
tests are opening hundreds (thousands?) of popups and then closing them.
AFAIK, these aren't being used and I've personally not have reason to reference
/ look at them for months.
I have no issues though if we want to keep them in.
The build script wrote the version line to stdout, polluting any pipeline
that captures program output via 'zig build run' or similar. Banners
belong on stderr.
Replaces the linenoise library with isocline for improved REPL
functionality. Updates build scripts and moves completion state from
global variables into the Terminal struct.
config.h already has a hand-added _LIBIDN2_LP_DECLS guarded block at
the end declaring extern strverscmp for the same gnulib-overlay reason.
Move the strchrnul prototype into that block (still #ifdef __APPLE__
since glibc Linux already declares it) so all project-added externs
live in one place, the autoconf-generated section stays untouched, and
the precedent for "how this codebase handles missing glibc extensions"
is one consistent pattern instead of two.
Refresh the build.zig and strchrnul.c comments to point at the
relocated declaration.
No functional change — verified across -target x86_64-macos.{13.0,
14.0,15.4} (lookup.c pattern + the three TUs that include both
<config.h> and <string.h>) plus x86_64-linux-gnu, all under
-Wall -Wextra -Werror.
Move the macOS strchrnul declaration from a side-channel header injected
via -include into vendor/libidn2/config.h itself, gated on __APPLE__.
config.h is the canonical place where libidn2 documents its
HAVE_STRCHRNUL detection — co-locating the platform fallback prototype
there mirrors gnulib's own approach (which declares missing symbols in
its substituted <string.h>) and removes the per-platform build-flag
asymmetry plus the dedicated include path.
The Darwin-only strchrnul.c implementation is unchanged; build.zig now
just adds it on Darwin without touching the lib/ flags array.
Verified that on both `-target x86_64-macos.14.0` (no libc strchrnul)
and `-target x86_64-macos.15.4` (libc has it) a TU mimicking lookup.c
(`#include "config.h"` then call strchrnul) compiles cleanly with no
implicit-declaration error and no redeclaration warning.
libidn2's lib/lookup.c calls strchrnul() unconditionally — a glibc
extension that macOS libc neither defines nor declares. Upstream's
portable build relies on gnulib substituting <string.h> with a
declaration and linking gl/strchrnul.c, but that overlay is not wired
up here. The result was an "implicit function declaration" hard error
on every macOS nightly build.
Add a Darwin-only shim under vendor/libidn2/darwin/ and inject its
prototype into the libidn2 lib/ sources via -include. The shim mirrors
strchrnul's semantics without falling through to rawmemchr (also
glibc-only) since libidn2 only ever searches for '.'.
Links to libidn2 and builds libcurl with it. This makes libcurl work, and by
extension browser, work on international domain names, e.g.
zig build run -- fetch "https://räksmörgås.se/"
With it available, we can use it in our WebAPIs which should also support these
domains, e.g:
testing.expectEqual('xn--rksmrgs-5wao1o.se', new URL('https://räksmörgås.se').hostname);
There is more integration to be done here, but this is a first step.
claude wrote all of the build.zig code.
I don't have a strong opinion about this feature, I just dislike that our WPT
/url/* tests are at 1704 / 9095 and, this is the biggest chunk (although, this
specific commit just does the basic integration and probably won't fix too many
WPT cases directly).
Profiling `zig build --release=fast` after a one-file edit showed the
cost was three parallel exe compiles of the same module — each re-runs
LLVM codegen when anything in `lightpanda_module` changes. All C/C++
deps (v8, sqlite, curl, brotli, nghttp2, boringssl) already cache across
builds and are not the bottleneck.
Default install now builds only the main `lightpanda` exe.
`lightpanda-snapshot-creator` and `legacy_test` move to a new named step
and can be built explicitly via `zig build extras`. CI is unaffected:
nightly.yml already invokes `snapshot_creator` as an explicit step, and
no job consumes the legacy_test artifact.
Measured (release=fast, one source file modified, 36/36 steps):
before: real 3m32s, compile exe lightpanda ~3m, others ~2m each
after: real 2m43s, only compile exe lightpanda
Also tried and rejected:
- self-hosted Zig backend for Debug: fatal linker error, unhandled
relocation type R_X86_64_PC64 in V8 objects.
- mold via `mold -run`: ~4s delta, within noise — Zig invokes LLD
as a library, so no external `ld` process exists to intercept.
This adds an app.storage which is a union around configurable storage engine
(currently, only sqlite).
It is _not_ being used anywhere right now. The goal is to get feedback on
the implementation and then move cache to it.
This doesn't expose a generic query API. The goal is that the storage will
expose high level methods, e.g. `cacheGet(req: CacheGetRequest)` and every
storage engine will translate the `storage.CacheGetRequest` as needed.
A thin wrapper around the Sqlite C api is included, e.g. exec(SQL, .{args}) a
`rows` and `row` fetcher. A connection pool is included. By default, an
in-memory DB is currently created. And a `migrations` table with an id of `1`
is created/inserted. I don't imagine needing fancy migratations.
Some WPT tests need to interact with the browser in a way that isn't possible
with web apis. Browsers need to expose a way for tests to do this and then use
the testdriver-vendor.js to hook into these special WPT actions.
This commit sets up the infrastructure for supporting this and includes
the delete_all_cookies functionality needed by various cookie tests (e.g.
/cookies/attributes/attributes-ctl.sub.html).
A new compilation flag, `-Dwpt_extensions`, can be specified. When specified
a `window.webdriver` accessor is defined and a `WebDriver` type is exposed.
Note that, while I only implemented delete_all_cookies for now, I've seen other
tests fail because of missing vendor-specific implementation.
Introduces `lightpanda agent` command that provides a REPL where users
can chat with an AI that uses the browser's tools (goto, markdown, click,
fill, etc.) to browse the web. Uses zenai for multi-provider LLM support
(Anthropic, OpenAI, Gemini) and linenoise v2 for terminal line editing.
Uses libcurl's websocket capabilities to add support for WebSocket.
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/167
Issue: https://github.com/lightpanda-io/browser/issues/1952
This is a WIP because it currently uses the same connection pool used for all
HTTP requests. It would be pretty easy for a page to starve the pool and block
any progress.
We previously stored the *Transfer inside of the easy's private data. We now
store the *Connection, and a Connection now has a `transport` field which is
a union for `http: *Transfer` or `websocket: *Websocket`.
Removes manual git flags from CI and build scripts.
Versioning is now automatically derived from git and build.zig.zon.
With this PR, we follow https://semver.org/
Logic:
1. Read the version from build.zig.zon
2. If it doesn't have a `.pre` field (i.e. dev/alpha/beta) it will use that
3. Otherwise it will get the info from git: hash and number of commits since last `.0` version
4. Then build the version: `0.3.0-dev.1493+0896edc3`
Note that, since the latest stable version is `0.2.6`.
The convention is to use `0.3.0-dev`, as:
- `0.2.6` < `0.3.0.dev` < `0.3.0`
- Add git_version option to build.zig (similar to git_commit)
- Update version command to output git_version when available
- Falls back to git_commit when not on a tagged release
- CI can pass -Dgit_version=$(git describe --tags --exact-match) for releases
Fixes#1867
Karl Seguin