Make sure jemallocator is disabled in Release builds (we only use it in Debug
builds to collect metrics. Supposedly something changed in our 0.16 build that
causes it to be initialized even though it isn't used. Not sure, for this, I'm
just doing what Claude tell me, but disabling something in Release that I know
we aren't using sounds ok).
We've been climbing the last couple weeks, and Zig 0.16 appears to have added
~400KB. I'll try to go hunt it, but I don't want this to delay the merge.
(Pierre agrees)
Runs on pushes to main and on pull requests. The config starts from
the generated defaults and is scoped to this repo: the few controls
that are off carry a note explaining why and what would let them
come back on.
Reports are uploaded to code scanning, except on PRs from forks
where the token cannot write security events. Each run on main also
publishes the score through OIDC, which feeds the live badge added
to the README next to the existing ones.
Mutable tags like @v6 can be re-pointed by whoever controls the action,
so the next run executes code nobody here reviewed, with the job's
token and secrets. Pinning to the full commit sha freezes what runs.
Version tags are kept as comments so renovate or dependabot can still
track updates.
dtolnay/rust-toolchain is pinned to a master commit with the toolchain
moved to an explicit input, as its readme recommends for sha pinning.
Mirror the wpt/e2e-integration pattern: tee the suite output to a log,
strip the ANSI colors, and upload it to the #ci-agent channel
(AGENT_SLACK_CHANNEL_ID) via the CI slack bot, with the final
pass/fail summary as the comment. The post also happens when the
suite fails, without masking its exit code (explicit bash pipefail).
Add a nightly cron trigger (03:37, offset from wpt's 02:21) so the
suite actually runs unattended; workflow_dispatch stays for manual
runs with a model override.
The suite moved to the demo repo (machinery in agent/, targets the
existing public/ sites — lightpanda-io/demo#206). Delete the in-repo
copy and point the agent-deterministic job, agent-regression.yml, and
make test-agent at the demo checkout, passing the binary via LPD_PATH
(the same variable wptrunner uses) — same consumption model as
runner/integration/wpt.
Includes a temporary push trigger on agent-regression.yml to validate
the cross-repo wiring pre-merge; dropped before merging.
Two layers driven by test/agent/run.sh:
- deterministic: replay a golden PandaScript against frozen local HN
fixtures and diff the returned JSON exactly against a golden file.
No API key, no network; gates every PR via e2e-test.yml.
- live (GOOGLE_API_KEY): closed-form Q&A over a local fixture page
(substring match), and a live Hacker News scrape saved with --save,
replayed token-free and validated against a jq shape invariant.
Runs nightly and on demand via agent-regression.yml.
The live output contract lives in cases/hn-live.task (prompt) +
schemas/hn-live.jq (invariant). Model pinned to gemini-3.5-flash in
run.sh; per-task timeouts and a $usage token ceiling guard runaway
loops. make test-agent for local runs.
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/179
An improvement to https://github.com/lightpanda-io/browser/pull/2515 to prevent
a v8 assertion if we terminate as an inspector dispatch is happening.
The problem is that if we just immediately terminate, we aren't sure what the
worker thread is doing, and, apparently, if we terminate then dispatch a message
to the inspector, we fail an assertion.
With the way the code was, the only safe solution would be to hold a mutex
over the session dispatch, but that could block the network thread.
So instead of terminating from the network thread, we now ask v8 to execute
a callback. This gets executed on the worker thread, which can then terminate
the execution.
The initial version of 2515 delayed the termination from the network thread.
It's possible that solution would "solve" the issue, simply because it's very
unlikely that a worker would be "stuck" for 5 seconds and then get unstuck.
More likely that it exits immediately, or is stuck in an endless loop. But
that would still leave a window where we could terminate in network and then
dispatch in the worker. Less likely, but still possible. Hopefully this new
mechanism eliminates this from being a problem in all circumstances.
Replace `grep '"id":N' | jq -e ...` with `jq -ec 'select(.id == N) | ...'`.
The grep form also matched `"id":10`, `"id":11`, ... and any tool description
containing that substring; numeric `select` is type-correct. `jq -e` still
fails the job when `select` produces no output (exit 4), so the smoke
semantics are preserved.
Also add `jq --version` up front so the job fails fast and loud if the
`ubuntu-latest` image ever stops shipping jq.
Sends initialize + notifications/initialized + tools/list over stdin
and asserts the JSON-RPC responses with jq. Catches regressions in
the agentic surface (./lightpanda mcp) without needing a node client.
Reuses the existing lightpanda-build-release artifact, so the new
job costs about a minute on top of zig-build-release.
cache=true is problematic for a few reasons
1 - The current cache implementation is known to cause timing issues; i.e. it
executes callbacks synchronously.
2 - Unlike something like robots.txt or proxy, cache tests need to be explicitly
tested. The response has to include cache headers and the resource loaded
again.
