Files
web/e2e
Ben Meadors ad692eef74 test(e2e): real-device Playwright messaging suite (#1121)
* fix(connections): connect to the just-added connection via the live store

addConnectionAndConnect() adds a connection and then connects to it in the
same tick, but connect() resolved the id against the memoized `connections`
closure, which is stale until the hook re-renders. The just-added id was
therefore reported as an unknown connection id and Save silently never
connected any HTTP/Serial/Bluetooth device. Read savedConnections from
useDeviceStore.getState() so the lookup always sees the live store.

* test(e2e): real-device Playwright messaging suite

Drives the actual web app in Chromium against real meshtasticd firmware over
the HTTP phone API and verifies text messaging in both directions across a
two-node mesh. Nodes mesh over the firmware's built-in UDP multicast
(224.0.0.69) with no MQTT/relay; distinct node numbers, real encryption.

- Default backend: two Docker meshtasticd sim nodes (daily-debian). The same
  specs run against physical hardware via E2E_DEVICE_MODE=hardware.
- An off-browser Python meshtastic peer (e2e/peer/peer.py) drives/asserts the
  non-browser node over the TCP phone API, mirroring firmware mcp-server tests.
- Coverage: connect over HTTPS, mesh->web receive, web->mesh send. Direct
  messages are fixme'd (see below). CI workflow runs it on Linux.

Bugs surfaced by the suite:
- Fixed (prior commit): connect-on-save never connected (stale-closure id
  lookup in useConnections).
- Not fixed: apps/web/src/core/subscriptions.ts throws 'ReferenceError:
  nodeDB is not defined' on every device-metrics telemetry packet (the #1050
  migration removed that store); caught per-packet, so messaging still works.
- Not fixed: direct messages are blocked by a PKI 'Keys Mismatch' (the SDK's
  stored peer public key != the key presented during NodeInfo exchange), seen
  even with fresh sim nodes.

* test(e2e): address Copilot review feedback

- waitForTcp(): destroy the probe socket on the error path so repeated
  connection failures don't accumulate sockets/FDs across the retry loop.
- Don't remove the mesh containers in Playwright globalTeardown in CI — it
  raced the workflow's failure log capture. Teardown is now gated on
  E2E_DOCKER_DOWN only; CI dumps device logs on failure and tears the mesh
  down in a final always() workflow step.

* fix(sdk): fold device-metrics telemetry into nodes

apps/web/src/core/subscriptions.ts called nodeDB.addDeviceMetrics() on every
device-metrics telemetry packet, but the #1050 migration removed that store —
so it threw 'ReferenceError: nodeDB is not defined' on each telemetry packet
(caught per-packet by the SDK's HandleFromRadio, so messaging still worked but
the error spammed the console).

Route device metrics into the SDK NodesClient via onTelemetryPacket instead —
mirroring the existing position handler; the Node domain already carries a
deviceMetrics field — and drop the dead app-side handler. Adds a NodesClient
test covering the fold.

* docs(e2e): accurate DM root cause + bug status

The direct-message fixme is a simulator limitation, not a web-app bug: the
keyless meshtasticd sim nodes NAK a DM with NO_CHANNEL (routing error 6) — no
Curve25519 keypair is provisioned/shared, and current firmware can't deliver a
direct message without a per-node key / decryptable channel. The app surfaces
this correctly (key-refresh dialog). Re-enable against hardware or once the sim
provisions keys.

Also: mark the nodeDB telemetry bug fixed and note the CI teardown change.

* docs(e2e): precise DM root cause (firmware/sim PKI)

Followed up on the suggestion to provision keys in config.security: the keys
ARE settable and persist (verified via admin), but on the native meshtasticd
sim they don't sync to the node's owner / NodeInfo key — owner.public_key stays
empty and the node keeps its MAC-derived num — so the two nodes never exchange
keys. Combined with the firmware refusing non-PKI DMs ('Unknown public key for
destination ... refusing to send legacy DM'), the DM is NAK'd with NO_CHANNEL.
A firmware/sim limitation; DMs work on real hardware. Spec stays fixme.

* docs(e2e): definitive DM root cause (SimRadio PKC payload limit)

Per the steer to research the firmware: PKI keygen is gated on a set LoRa
region (NodeDB.cpp:3051) and the sim boots region-UNSET — setting lora.region
via admin DOES make the nodes generate and exchange keys (verified both ways).
But a PKI-encrypted DM still can't traverse the SimRadio: the PKC overhead
exceeds its payload limit ('Payload size larger than compressed message allows!
Send empty payload'), so the packet is truncated and the receiver NAKs
NO_CHANNEL ('No suitable channel found for decoding, hash 0x0'). The firmware
skips PKC under --sim (Router.cpp:730) for exactly this reason, but --sim also
disables the config-file loading the web app needs, so they're mutually
exclusive. DMs work on real hardware; spec stays fixme with this detail.

---------

Co-authored-by: Dan Ditomaso <dan.ditomaso@gmail.com>
2026-06-20 11:19:50 -04:00
..

Real-device E2E messaging suite

Playwright tests that drive the real web app in Chromium against a real Meshtastic device over the HTTP(S) phone API and verify text messaging in both directions across a two-node mesh.

By default the "devices" are two simulated meshtasticd firmware nodes running in Docker, meshed over the firmware's built-in UDP multicast LAN transport (224.0.0.69:4403) — real firmware, real encryption, distinct node numbers, and no MQTT/relay. The same tests can run against physical hardware.

   Playwright (headless Chromium)                 Python peer (meshtastic lib)
   ── HTTPS phone API :9443 ────┐                 ┌──── TCP phone API :4403 ────
                                ▼                 ▼
                    ┌─────────────────┐  UDP multicast  ┌─────────────────┐
                    │  Node A (DUT)   │  224.0.0.69     │   Node B (peer) │
                    │ meshtasticd sim │◀─── mesh ──────▶│ meshtasticd sim │
                    └─────────────────┘                 └─────────────────┘
  • Node A is the device-under-test the browser connects to (HTTPS).
  • Node B is driven/observed by the Python peer (e2e/peer/peer.py) over TCP.

Layout

Path What
playwright.config.ts Config (root): chromium, serial, dev server on :3100, global setup/teardown
e2e/global-setup.ts / global-teardown.ts Bring the mesh up / wait for readiness / tear down
e2e/device/docker-compose.yml + nodeA.yaml / nodeB.yaml The two meshtasticd sim nodes
e2e/peer/peer.py + requirements.txt The off-browser mesh peer (TCP meshtastic lib)
e2e/fixtures/ peer.ts (peer wrapper) + test.ts (page-object + device fixtures)
e2e/pages/ ConnectionPage.ts, MessagesPage.ts
e2e/tests/ connect, messaging.broadcast, messaging.direct

Running locally (Docker sim — default)

Prerequisites: Docker, Node + pnpm (11.5.2), Python 3.11+.

pnpm install
pnpm exec playwright install chromium
python -m venv e2e/peer/.venv && e2e/peer/.venv/bin/pip install -r e2e/peer/requirements.txt

pnpm test:e2e            # brings up the mesh, runs the suite
pnpm test:e2e:report     # open the HTML report

Global setup runs docker compose up -d (idempotent) and waits for the device. The mesh is left running between runs for speed; set E2E_DOCKER_DOWN=1 to tear it down on exit. CI leaves the containers up through the run (so the workflow can dump device logs on failure) and tears them down in a final step.

Environment variables

Var Default Purpose
E2E_DEVICE_MODE docker docker (sim) or hardware (skip compose)
E2E_NODE_A_URL https://127.0.0.1:9443 Device-under-test the browser connects to
E2E_PEER_HOST / E2E_PEER_PORT 127.0.0.1 / 14404 TCP phone API the Python peer drives
E2E_WEB_PORT 3100 Dev-server port for the app under test
E2E_PEER_PYTHON e2e/peer/.venv/bin/python Python used to run the peer
E2E_DOCKER_DOWN unset 1 to compose down on teardown (CI tears down in a final workflow step)

Running against real hardware

Point the suite at two physical devices on the same channel/region. Node A must expose the HTTP(S) phone API (Wi-Fi); the peer reaches Node B over TCP (Wi-Fi) — both on the same LoRa mesh, so the radio is the bridge (no MQTT):

E2E_DEVICE_MODE=hardware \
E2E_NODE_A_URL=https://<deviceA-ip> \
E2E_PEER_HOST=<deviceB-ip> E2E_PEER_PORT=4403 \
pnpm test:e2e

What the tests cover

  • connect — add an HTTP(S) connection in the UI, complete the config handshake, land on the messages view.
  • messaging.broadcastmesh → web (peer broadcasts, the browser renders it) and web → mesh (the browser sends, the peer node confirms receipt over the real mesh).
  • messaging.directfixme (see Known limitations).

Gotchas baked in (firmware/sim specifics)

  • Image tag: use meshtastic/meshtasticd:daily-debian. :latest is 2.7.15 and predates the EnableUDP multicast feature (no mesh between sim nodes).
  • Do not pass --sim: force_simradio takes an early branch in portduinoSetup() that skips config-file loading — Webserver / EnableUDP / MACAddress would all be ignored. Select the sim radio via Lora: Module: sim in the config instead.
  • Distinct MACAddress per node → distinct node numbers (else the UDP handler drops the peer's packets as "spoofed local origin").
  • Webserver is HTTPS-only (self-signed cert on 9443) — Playwright uses ignoreHTTPSErrors + --ignore-certificate-errors; the dialog's HTTPS toggle is on. The app is served over plain HTTP to avoid mixed-content.
  • Send readiness: the composer renders before the SDK chat client is ready (the SQLite/OPFS sqlocal store times out in headless Chromium and falls back to in-memory). MessagesPage.waitReady() gates on the "Connected" status so an immediate send isn't silently dropped.

Known limitations

  • Direct messages (messaging.direct) are fixme — a SimRadio limitation, not a web-app issue. DMs go out PKI-encrypted. PKI keygen is gated on a set LoRa region (NodeDB.cpp:3051; the sim boots region-UNSET) — setting lora.region via admin does make the nodes generate and exchange keys (verified: both learn each other's public key). But a PKI DM still can't traverse the SimRadio: the PKC overhead exceeds its payload limit (Payload size larger than compressed message allows! Send empty payload), so the packet is truncated and the receiver NAKs NO_CHANNEL (No suitable channel found for decoding, hash 0x0). The firmware skips PKC under --sim (Router.cpp:730) for exactly this reason, but --sim also disables the config-file loading (Webserver/EnableUDP/MAC) the web app needs, so the two are mutually exclusive. The app behaves correctly (key-refresh dialog). Broadcast covers bidirectional messaging; re-enable against real hardware, where real LoRa carries PKC fine.

App bugs surfaced by this suite (fixed on this branch)

  1. Connect-on-save race (apps/web/src/pages/Connections/useConnections.ts): connect() read the just-added connection from a stale memoized closure, so "Save" never actually connected ("unknown connection id"). Fixed to read from the live store.
  2. ReferenceError: nodeDB is not defined (apps/web/src/core/subscriptions.ts): the device-metrics telemetry handler called a node store the #1050 migration removed, throwing on every telemetry packet. Fixed by folding device metrics into the node inside the SDK NodesClient (onTelemetryPacket) and dropping the dead app-side handler.