Scott Taylor f013990ff3 Fix Fetch.enable + http-proxy CDP send/recv deadlock (#2462)
WsConnection.send used to switch the socket to blocking mode on
EWOULDBLOCK, with a comment claiming this 'should virtually never
happen'. Under CDP Fetch.enable + --http-proxy it is routine: the
CONNECT+TLS proxy round-trip means many subresources are in flight
simultaneously, lightpanda emits a flood of Fetch.requestPaused events,
the kernel send buffer fills, lightpanda blocks on write, and
puppeteer's matching Fetch.continueRequest replies pile up in
lightpanda's TCP recv buffer (which lightpanda can't drain because it
is blocked on the write). Both peers wedge until the client times out.

Other contributing problems all collapse paused-intercept sites where
an outer loop polls without ever draining the CDP socket, OR where
disconnect-time cleanup re-enters JS through paths the runtime can no
longer satisfy:

* HttpClient.perform skipped the CDP socket poll entirely whenever
  processMessages() returned non-empty, so a steady stream of HTTP
  completions could starve CDP reads.
* ScriptManagerBase.waitForImport spun on `client.tick(200)` and
  discarded the .cdp_socket return, so a script `import()` whose
  request was paused at the InterceptionLayer hung forever.
* BrowserContext.deinit aborted pending intercepts via `transfer.abort`,
  which fired XHR/script error_callback chains into a half-torn-down
  V8 context (the inspector had already been stopped two lines above).
* Headers.deinit was non-idempotent, so a value-copied Headers (the
  hazard RobotsLayer documents) double-freed its curl_slist on the
  second deinit; the symptom was an "incorrect alignment" panic
  inside ZigToCurlAllocator.free.
* Transfer.deinit was non-idempotent, so a cascade out of
  error_callback (e.g. Script.errorCallback -> manager.evaluate() ->
  JS execution -> Frame.deinit -> abortOwner -> Transfer.kill ->
  Transfer.deinit) reached `arena_pool.release` twice on the same
  arena.

Coordinated changes:

* src/network/WsConnection.zig: On EWOULDBLOCK, instead of switching
  to a blocking write, poll for both POLLOUT and POLLIN. While waiting
  for write space, drain any incoming bytes into the reader buffer
  (without dispatching - that would re-enter send and recurse). Adds
  tryRead/bufferedBytes accessors.

* src/browser/HttpClient.zig:
  - Add has_buffered_input to CDPClient. In perform(), return
    .cdp_socket when buffered input exists, and always do at least a
    non-blocking poll on the CDP socket so HTTP completions can no
    longer starve CDP reads.
  - Make Transfer.deinit idempotent by claiming ownership through
    `client.transfers.remove(self.id)`. Second deinits (cascades out
    of error_callback) early-return.
  - Make `Transfer.kill` public (was `fn`) so BrowserContext.deinit
    can use it.
  - Tighten RequestParams.deinit / Request.deinit to take `*` instead
    of `*const` so they can call into `Headers.deinit` (now mutating).

* src/network/http.zig: Headers.deinit now nulls out `self.headers`
  after `curl_slist_free_all`, so a second deinit is a no-op. Without
  this guard a value-copied Headers double-frees the curl_slist (the
  hazard RobotsLayer's call site already documents).

* src/browser/ScriptManagerBase.zig: waitForImport now drains pending
  CDP messages on every iteration (matching the syncRequest pattern)
  and re-fetches the imported_modules entry per iteration. The cached
  entry was a use-after-free risk because the CDP-drain step above
  re-enters JS, and a transitively-imported module's preloadImport()
  -> getOrPut() can rehash the map and invalidate the prior entry
  pointer.

* src/cdp/CDP.zig:
  - Wire hasBufferedInput.
  - Replace read() with tryRead() in readSocket and tolerate the
    no_new_data case so we still process messages drained during a
    backpressured send.
  - BrowserContext.deinit aborts pending intercepts via
    `transfer.kill` instead of `transfer.abort`. `kill` fires
    shutdown_callback (or no-op for transfers without one), avoiding
    error_callback's re-entry into JS through XHR/script error
    handlers - those crash because the V8 context and inspector this
    BC owns have either been torn down already or are about to be.

Verified end-to-end against a puppeteer + http-proxy reproducer:

| URL                        | before               | after            |
|----------------------------|----------------------|------------------|
| example.com                | OK 124ms             | OK 117ms         |
| github.com                 | HANG 20s (12/82)     | OK 1410ms (82/82)|
| shopify.com                | HANG 20s (1/4)       | OK 1973ms (66/66)|
| allbirds.com               | HANG 20s (12/53)     | OK 3944ms (372)  |
| allbirds wool-runners PDP  | HANG 20s (12/53)     | OK 6286ms (459)  |

(nike.com still doesn't reach `load` event but all 68 continueRequests
process cleanly - the remaining stall is third-party widgets keeping
the page in `loading` state, not the CDP/HTTP deadlock this PR fixes.)

Lightpanda survives 18 back-to-back navigation runs across the matrix
above (3 per URL) without crashing.

Fixes #2462.
2026-05-18 11:30:18 +02:00
2026-04-25 13:22:41 +08:00
2026-05-12 14:06:17 +08:00
2024-11-26 09:35:14 +01:00
2026-05-08 09:50:09 +02:00
2026-05-11 06:50:03 -07:00
2024-05-13 12:10:15 +02:00
2026-03-25 10:42:52 +01:00
2026-05-16 20:07:32 +08:00
2026-03-30 11:28:46 +02:00

Logo

Lightpanda Browser

The headless browser built from scratch for AI agents and automation.
Not a Chromium fork. Not a WebKit patch. A new browser, written in Zig.

License Twitter Follow GitHub stars Discord

Benchmarks

Requesting 933 real web pages over the network on a AWS EC2 m5.large instance. See benchmark details.

Metric Lightpanda Headless Chrome Difference
Memory (peak, 100 pages) 123MB 2GB ~16 less
Execution time (100 pages) 5s 46s ~9x faster

Quick start

Install

Package Managers

Latest nightly from Homebrew:

brew install lightpanda-io/browser/lightpanda

Latest nightly from Arch Linux User Repository:

yay -S lightpanda-nightly-bi

Download from the nightly builds

You can download the last binary from the nightly builds for Linux and MacOS for both x86_64 and aarch64.

For Linux

curl -L -o lightpanda https://github.com/lightpanda-io/browser/releases/download/nightly/lightpanda-x86_64-linux && \
chmod a+x ./lightpanda

Verify the binary before running anything:

./lightpanda version

Linux aarch64 is also available

Note: The Linux release binaries are linked against glibc. On musl-based distros (Alpine, etc.) the binary fails with cannot execute: required file not found because the glibc dynamic linker is missing. Use a glibc-based base image (e.g., FROM debian:bookworm-slim or FROM ubuntu:24.04) or build from sources.

For MacOS

curl -L -o lightpanda https://github.com/lightpanda-io/browser/releases/download/nightly/lightpanda-aarch64-macos && \
chmod a+x ./lightpanda

MacOS x86_64 is also available

For Windows + WSL2

Lightpanda has no native Windows binary. Install it inside WSL following the Linux steps above.

WSL not installed? Run wsl --install from an administrator shell, restart, then open wsl. See Microsoft's WSL install guide for details.

Your automation client (Puppeteer, Playwright, etc.) can run either inside WSL or on the Windows host. WSL forwards localhost:9222 automatically.

Install from Docker

Lightpanda provides official Docker images for both Linux amd64 and arm64 architectures. The following command fetches the Docker image and starts a new container exposing Lightpanda's CDP server on port 9222.

docker run -d --name lightpanda -p 127.0.0.1:9222:9222 lightpanda/browser:nightly

Dump a URL

./lightpanda fetch --obey-robots --dump html --log-format pretty  --log-level info https://demo-browser.lightpanda.io/campfire-commerce/

You can use --dump markdown to convert directly into markdown. --wait-until, --wait-ms, --wait-selector and --wait-script are available to adjust waiting time before dump.

Start a CDP server

./lightpanda serve --obey-robots --log-format pretty  --log-level info --host 127.0.0.1 --port 9222

Once the CDP server started, you can run a Puppeteer script by configuring the browserWSEndpoint.

Example Puppeteer script
import puppeteer from 'puppeteer-core';

// use browserWSEndpoint to pass the Lightpanda's CDP server address.
const browser = await puppeteer.connect({
  browserWSEndpoint: "ws://127.0.0.1:9222",
});

// The rest of your script remains the same.
const context = await browser.createBrowserContext();
const frame = await context.newPage();

// Dump all the links from the frame.
await frame.goto('https://demo-browser.lightpanda.io/amiibo/', {waitUntil: "networkidle0"});

const links = await frame.evaluate(() => {
  return Array.from(document.querySelectorAll('a')).map(row => {
    return row.getAttribute('href');
  });
});

console.log(links);

await frame.close();
await context.close();
await browser.disconnect();

Native MCP and skill

The MCP server communicates via MCP JSON-RPC 2.0 over stdio.

Add to your MCP configuration:

{
  "mcpServers": {
    "lightpanda": {
      "command": "/path/to/lightpanda",
      "args": ["mcp"]
    }
  }
}

Read full documentation

A skill is available in lightpanda-io/agent-skill.

Telemetry

By default, Lightpanda collects and sends usage telemetry. This can be disabled by setting an environment variable LIGHTPANDA_DISABLE_TELEMETRY=true. You can read Lightpanda's privacy policy at: https://lightpanda.io/privacy-policy.

Status

Lightpanda is in Beta and currently a work in progress. Stability and coverage are improving and many websites now work. You may still encounter errors or crashes. Please open an issue with specifics if so.

Here are the key features we have implemented:

  • CORS #2015
  • HTTP loader (Libcurl)
  • HTML parser (html5ever)
  • DOM tree
  • Javascript support (v8)
  • DOM APIs
  • Ajax
    • XHR API
    • Fetch API
  • DOM dump
  • CDP/websockets server
  • Click
  • Input form
  • Cookies
  • Custom HTTP headers
  • Proxy support
  • Network interception
  • Respect robots.txt with option --obey-robots

NOTE: There are hundreds of Web APIs. Developing a browser (even just for headless mode) is a huge task. Coverage will increase over time.

Build from sources

Prerequisites

Lightpanda is written with Zig 0.15.2. You have to install it with the right version in order to build the project.

Lightpanda also depends on v8, Libcurl and html5ever.

To be able to build the v8 engine, you have to install some libs:

For Debian/Ubuntu based Linux:

sudo apt install xz-utils ca-certificates \
    pkg-config libglib2.0-dev \
    clang make curl git

You also need to install Rust.

For systems with Nix, you can use the devShell:

nix develop

For MacOS, you need cmake and Rust.

brew install cmake

Build and run

You can build the entire browser with make build or make build-dev for debug env.

But you can directly use the zig command: zig build run.

Embed v8 snapshot

Lighpanda uses v8 snapshot. By default, it is created on startup but you can embed it by using the following commands:

Generate the snapshot.

zig build snapshot_creator -- src/snapshot.bin

Build using the snapshot binary.

zig build -Dsnapshot_path=../../snapshot.bin

See #1279 for more details.

Test

Unit Tests

You can test Lightpanda by running make test.

make test                                       # Run all tests
make test F="server"                            # Filter by substring
TEST_FILTER="WebApi: #selector_all" make test   # Filter main + subtest (separator: #)
TEST_VERBOSE=true make test
TEST_FAIL_FIRST=true make test
METRICS=true make test                          # Capture allocation/duration metrics as JSON

End to end tests

To run end to end tests, you need to clone the demo repository into ../demo dir.

You have to install the demo's node requirements

You also need to install Go > v1.24.

make end2end

Web Platform Tests

Lightpanda is tested against the standardized Web Platform Tests.

We use a fork including a custom testharnessreport.js.

For reference, you can easily execute a WPT test case with your browser via wpt.live.

Configure WPT HTTP server

To run the test, you must clone the repository, configure the custom hosts and generate the MANIFEST.json file.

Clone the repository with the fork branch.

git clone -b fork --depth=1 git@github.com:lightpanda-io/wpt.git

Enter into the wpt/ dir.

Install custom domains in your /etc/hosts

./wpt make-hosts-file | sudo tee -a /etc/hosts

Generate MANIFEST.json

./wpt manifest

Use the WPT's setup guide for details.

Run WPT test suite

An external Go runner is provided by github.com/lightpanda-io/demo/ repository, located into wptrunner/ dir. You need to clone the project first.

First start the WPT's HTTP server from your wpt/ clone dir.

./wpt serve

Run a Lightpanda browser

zig build run -- --insecure-disable-tls-host-verification

Then you can start the wptrunner from the demo's clone dir:

cd wptrunner && go run .

Or one specific test:

cd wptrunner && go run . Node-childNodes.html

wptrunner command accepts --summary and --json options modifying output. Also --concurrency define the concurrency limit.

⚠️ Running the whole test suite will take a long time. In this case, it's useful to build in releaseFast mode to make tests faster.

zig build -Doptimize=ReleaseFast run

Contributing

See CONTRIBUTING.md for guidelines. You must sign our CLA during the pull request process.

Why Lightpanda?

Javascript execution is mandatory for the modern web

Simple HTTP requests used to be enough for web automation. That's no longer the case. Javascript now drives most of the web:

  • Ajax, Single Page Apps, infinite loading, instant search
  • JS frameworks: React, Vue, Angular, and others

Chrome is not the right tool

Running a full desktop browser on a server works, but it does not scale well. Chrome at hundreds or thousands of instances is expensive:

  • Heavy on RAM and CPU
  • Hard to package, deploy, and maintain at scale
  • Many features are not necessary in headless made

Lightpanda is built for performance

Supporting Javascript with real performance meant building from scratch rather than forking Chromium:

  • Not based on Chromium, Blink, or WebKit
  • Written in Zig, a low-level language with explicit memory control
  • No graphical rendering engine
Description
No description provided
Readme AGPL-3.0 44 MiB
Languages
Zig 76.7%
HTML 21.9%
Rust 0.7%
JavaScript 0.5%