Profiling `zig build --release=fast` after a one-file edit showed the
cost was three parallel exe compiles of the same module — each re-runs
LLVM codegen when anything in `lightpanda_module` changes. All C/C++
deps (v8, sqlite, curl, brotli, nghttp2, boringssl) already cache across
builds and are not the bottleneck.
Default install now builds only the main `lightpanda` exe.
`lightpanda-snapshot-creator` and `legacy_test` move to a new named step
and can be built explicitly via `zig build extras`. CI is unaffected:
nightly.yml already invokes `snapshot_creator` as an explicit step, and
no job consumes the legacy_test artifact.
Measured (release=fast, one source file modified, 36/36 steps):
before: real 3m32s, compile exe lightpanda ~3m, others ~2m each
after: real 2m43s, only compile exe lightpanda
Also tried and rejected:
- self-hosted Zig backend for Debug: fatal linker error, unhandled
relocation type R_X86_64_PC64 in V8 objects.
- mold via `mold -run`: ~4s delta, within noise — Zig invokes LLD
as a library, so no external `ld` process exists to intercept.
This adds an app.storage which is a union around configurable storage engine
(currently, only sqlite).
It is _not_ being used anywhere right now. The goal is to get feedback on
the implementation and then move cache to it.
This doesn't expose a generic query API. The goal is that the storage will
expose high level methods, e.g. `cacheGet(req: CacheGetRequest)` and every
storage engine will translate the `storage.CacheGetRequest` as needed.
A thin wrapper around the Sqlite C api is included, e.g. exec(SQL, .{args}) a
`rows` and `row` fetcher. A connection pool is included. By default, an
in-memory DB is currently created. And a `migrations` table with an id of `1`
is created/inserted. I don't imagine needing fancy migratations.
Some WPT tests need to interact with the browser in a way that isn't possible
with web apis. Browsers need to expose a way for tests to do this and then use
the testdriver-vendor.js to hook into these special WPT actions.
This commit sets up the infrastructure for supporting this and includes
the delete_all_cookies functionality needed by various cookie tests (e.g.
/cookies/attributes/attributes-ctl.sub.html).
A new compilation flag, `-Dwpt_extensions`, can be specified. When specified
a `window.webdriver` accessor is defined and a `WebDriver` type is exposed.
Note that, while I only implemented delete_all_cookies for now, I've seen other
tests fail because of missing vendor-specific implementation.
Introduces `lightpanda agent` command that provides a REPL where users
can chat with an AI that uses the browser's tools (goto, markdown, click,
fill, etc.) to browse the web. Uses zenai for multi-provider LLM support
(Anthropic, OpenAI, Gemini) and linenoise v2 for terminal line editing.
Uses libcurl's websocket capabilities to add support for WebSocket.
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/167
Issue: https://github.com/lightpanda-io/browser/issues/1952
This is a WIP because it currently uses the same connection pool used for all
HTTP requests. It would be pretty easy for a page to starve the pool and block
any progress.
We previously stored the *Transfer inside of the easy's private data. We now
store the *Connection, and a Connection now has a `transport` field which is
a union for `http: *Transfer` or `websocket: *Websocket`.
Removes manual git flags from CI and build scripts.
Versioning is now automatically derived from git and build.zig.zon.
With this PR, we follow https://semver.org/
Logic:
1. Read the version from build.zig.zon
2. If it doesn't have a `.pre` field (i.e. dev/alpha/beta) it will use that
3. Otherwise it will get the info from git: hash and number of commits since last `.0` version
4. Then build the version: `0.3.0-dev.1493+0896edc3`
Note that, since the latest stable version is `0.2.6`.
The convention is to use `0.3.0-dev`, as:
- `0.2.6` < `0.3.0.dev` < `0.3.0`
- Add git_version option to build.zig (similar to git_commit)
- Update version command to output git_version when available
- Falls back to git_commit when not on a tagged release
- CI can pass -Dgit_version=$(git describe --tags --exact-match) for releases
Fixes#1867
This adds a crash handler which reports a crash (if telemetry is enabled). On a
crash, this looks for `curl` (using the PATH env), and forks the process to then
call execve. This relies on a new endpoint to be setup to accept the "report".
Also, we include very little data..I figured just knowing about crashes would
be a good place to start.
A panic handler is provided, which override's Zig default handler and hooks
into the crash handler.
An `assert` function is added and hooks into the crash handler. This is
currently only used in one place (Session.zig) to demonstrate its use. In
addition to reporting a failed assert, the assert aborts execution in
ReleaseFast (as opposed to an undefined behavior with std.debug.assert).
I want to hook this into the v8 global error handler, but only after direct_v8
is merged.
Much of this is inspired by bun's code. They have their own assert (1) and
a [more sophisticated] crashHandler (2).
:
(1) beccd01647/src/bun.zig (L2987)
(2) beccd01647/src/crash_handler.zig (L198)
There are two layers here. The first is that, on startup, a v8 SnapshotCreator
is created, and a snapshot-specific isolate/context is setup with our browser
environment. This contains most of what was in Env.init and a good chunk of
what was in ExecutionWorld.createContext. From this, we create a v8.StartupData
which is used for the creation of all subsequent contexts. The snapshot sits
at the application level, above the Env - it's re-used for all envs/isolates, so
this gives a nice performance boost for both 1 connection opening multiple pages
or multiple connections opening 1 page.
The second layer is that the Snapshot data can be embedded into the binary, so
that it doesn't have to be created on startup, but rather created at build-time.
This improves the startup time (though, I'm not really sure how to measure that
accurately...).
The first layer is the big win (and just works as-is without any build / usage
changes).
with snapshot
total runs 1000
total duration (ms) 7527
avg run duration (ms) 7
min run duration (ms) 5
max run duration (ms) 41
without snapshot
total runs 1000
total duration (ms) 9350
avg run duration (ms) 9
min run duration (ms) 8
max run duration (ms) 42
To embed a snapshot into the binary, we first need to create the snapshot file:
zig build -Doptimize=ReleaseFast snapshot_creator -- src/snapshot.bin
And then build using the new snapshot_path argument:
zig build -Dsnapshot_path=../../snapshot.bin -Doptimize=ReleaseFast
The paths are weird, I know...since it's embedded, it needs to be inside the
project path, hence we put it in src/snapshot.bin. And since it's embedded
relative to the embedder (src/browser/js/Snapshot.zig) the path has to be
relative to that, hence ../../snapshot.bin. I'm open to suggestions on
improving this.
