Commit Graph
38 Commits
Author SHA1 Message Date
Karl Seguin 47cffb8819 telemetry: Optimize the payload size, add navigation type
First, this adds 1 small piece of data to the navigate event: whether the
navigate was a page, frame or popup.

It also adds a session id, but as far as I'm concerned, this isn't "new"
information, or any new tracking/insight into users. Between the iid and the
"run" event, a "session" was always trackable. By giving it an explicit value,
we can shrink the size of all other messes.

This change reduces the telemetry payload by ~70% (despite the extra nav field).
I'm hoping this might remove a reason some people would consider turning it off.

It hits a /v2/ endpoint. The changes:

1 - A header is the first message in a session and contains all of the static
    data, as well as a session id
2 - Every event is encoded as an array, [$SID, "event-type, params...]

```
{"sid":"92e98210a141f497","iid":"$UUID","mode":"fetch","os":"macos","arch":"aarch64","version":"$VERSION","proxy":false}
["92e98210a141f497","run"]
["92e98210a141f497","nav",false,"page"]
```

(the driver=cdp field was removed from nav, because it was always cdp).

Some notes for the server:

1 - The server can tell a header from an event based on the first character.
2 - The SID is 8 bytes, enough to be unique, but not globally unique. The
    iid + sid + time window is how a events for the same SID can be grouped.
3 - A valid event is always an array of 2+ items, index 0 = SID, index 1 = type

Although positional data isn't expressive, it's still extendable.

The 4 events:

run, no parameters (mostly just used to flush the header now)
["sid", "run"]

// nav, tls, page/popup/iframe
["sid","nav",true,"page"]

// bof, # of lost telemetry events
["sid","bof",42]

// llm, provider, model (nullable)
["sid","llm","anthropic","claude"]
2026-07-01 06:33:30 +08:00
Pierre Tachoire d44bdaa0c9 telemetry: remove allocation for llm event
The model length is limited to 23 char in DB.
2026-06-05 11:09:08 +02:00
Pierre TachoireandKarl Seguin 3f012cee4c remove useless import
Co-authored-by: Karl Seguin <karlseguin@users.noreply.github.com>
2026-06-05 08:16:50 +02:00
Pierre Tachoire 3f5d19d7dc telemetry: send a new llm event 2026-06-04 17:35:26 +02:00
Pierre Tachoire 495d207aad telemetry: send agent_replay mode on non-interactive usage 2026-06-04 16:18:02 +02:00
Karl Seguin 2d20e57f80 Change all @import("...../log.zig") to const log = lp.log;
@import("lightpanda") where needed.

Would also like to do this for String, Page, Session and js which all stand out
as types that are use across the codebase.

I know that a few devs are doing this in new work and I haven't heard anyone
voice an objection.
2026-04-20 12:40:04 +08:00
Karl Seguin cec9628e37 zig fmt 2026-04-10 06:51:50 +08:00
Karl Seguin 255fa247c3 Reduces the size of Telemetry.Lightpanda
32920 -> 8344

We buffer [1024]Telemetry.Event, and the event is unnecessarily large.
2026-04-10 06:51:50 +08:00
Nikolay Govorov 5fb561dc9c Used ring buffer for telemetry events buffer 2026-03-17 23:23:08 +00:00
Nikolay Govorov a6d699ad5d Use common network runtime for telemetry messages 2026-03-17 23:21:57 +00:00
Karl Seguin 2eab4b84c9 Rename all ArrayListUnmanaged -> ArrayList
ArrayListAlignedUnmanaged has been deprecated for a while, and I occasionally
replace them, but doing one complete pass gets it done once and for all.
2026-02-05 11:49:15 +08:00
Nikolay Govorov f71aa1cad2 Centralizes configuration, eliminates unnecessary copying of config 2026-02-04 07:57:59 +00:00
Nikolay Govorov dbf18b90a7 Removes telemetry dependence on notifications 2026-02-04 07:30:38 +00:00
Karl Seguin 0f9c9e2089 Improve crash handling
This adds a crash handler which reports a crash (if telemetry is enabled). On a
crash, this looks for `curl` (using the PATH env), and forks the process to then
call execve. This relies on a new endpoint to be setup to accept the "report".
Also, we include very little data..I figured just knowing about crashes would
be a good place to start.

A panic handler is provided, which override's Zig default handler and hooks
into the crash handler.

An `assert` function is added and hooks into the crash handler. This is
currently only used in one place (Session.zig) to demonstrate its use. In
addition to reporting a failed assert, the assert aborts execution in
ReleaseFast (as opposed to an undefined behavior with std.debug.assert).

I want to hook this into the v8 global error handler, but only after direct_v8
is merged.

Much of this is inspired by bun's code. They have their own assert (1) and
a [more sophisticated] crashHandler (2).
:

(1) https://github.com/oven-sh/bun/blob/beccd01647ec4fcb5895be5c5fb88f5254dc1e8d/src/bun.zig#L2987
(2) https://github.com/oven-sh/bun/blob/beccd01647ec4fcb5895be5c5fb88f5254dc1e8d/src/crash_handler.zig#L198
2026-01-19 07:36:46 +08:00
Karl Seguin b047cb6dc1 remove libdom 2025-10-27 22:14:59 +08:00
Karl Seguin 1443f38e5f Zig 0.15.1
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/89
2025-08-29 10:42:06 +08:00
Karl Seguin 687f09d952 Make the App own the Platform
Removes optional platform, which only existed for tests.

There is now a global `@import("testing.zig").test_app` available. This is setup
when the test runner starts, and cleaned up at the end of tests. Individual
tests don't have to worry about creating app, which I assume was the reason I
Platform optional, since that woul dhave been something else that needed to be
setup.
2025-08-29 10:14:06 +08:00
Karl Seguin f65a39a3e3 Re-enable telemetry
Start work on supporting navigation events (clicks, form submission).
2025-08-11 21:37:00 +08:00
Karl Seguin 54ab1326e5 Switch XHR to new http client
get puppeteer/cdp.js working again

make test are all passing
2025-08-11 21:37:00 +08:00
Karl Seguin 7d9951aa3c Replace SessionState directly with the Page. 2025-05-27 20:31:34 +08:00
Karl Seguin 2feba3182a Replace std.log with a structured logger
Outputs in logfmt in release and a "pretty" print in debug mode. The format
along with the log level will become arguments to the binary at some point in
the future.
2025-05-27 19:57:58 +08:00
Karl Seguin b8e5e130b9 remove unused code, mostly imports 2025-05-05 13:29:41 +08:00
Karl Seguin 6c592669da Introduce more general notification capabilities
Replaces the existing, very specialized Notification with something more
general.

Currently, the existing page_navigate and page_navigated have been migrated.

Telemetry's page navigation event now also hooks into these events to generate
the telemetry record.
2025-04-30 17:33:51 +08:00
Karl Seguin 3d6dd06b99 Generate non-persisted iid if app_path is null 2025-03-22 23:58:57 +08:00
Karl Seguin ba8a0179d5 Share the HTTP client globally 2025-03-19 11:09:58 +08:00
Karl Seguin 2e7342a59c add driver field to navigate telemetry 2025-03-18 10:40:04 +08:00
Karl Seguin c9bc5be42b add additition navigate fields 2025-03-18 09:56:57 +08:00
Karl Seguin 1e6a1bd3af store iid in application data directory 2025-03-18 08:27:47 +08:00
Karl Seguin b0a2087015 fix unit test 2025-03-18 08:27:47 +08:00
Karl Seguin a5ee34a2db send telemetry synchronously in a background thread 2025-03-18 08:27:47 +08:00
Karl Seguin 3b4de6a405 remove [incorrect] data version 2025-03-18 08:27:32 +08:00
Karl Seguin 75512602c3 Add log to display telemetry state 2025-03-18 08:27:02 +08:00
Karl Seguin cd33a089d1 flatten events, include aarch + os, remove eid 2025-03-18 08:26:58 +08:00
Karl Seguin 6b83281539 Add navigate telemetry 2025-03-18 08:25:44 +08:00
Karl Seguin 2609671982 don't try (and fail) to get userData after clearing context 2025-03-18 08:02:09 +08:00
Karl Seguin accf2c0e5e use async-client for telemetry 2025-03-18 08:02:09 +08:00
Karl Seguin 53f6e66c23 Remove plausible, leave a dummy provider for now
Add batching, add install optional id (persisted) and execution id (per run)
2025-03-18 08:02:09 +08:00
Karl Seguin 56ddcc8e29 Initial usage telemetry 2025-03-18 08:02:09 +08:00