firmware/mcp-server/tests/README.md

Meshtastic MCP Server — Test Harness

Automated test suite for the MCP server, organized around real operator concerns rather than generic "unit vs hardware".

Tiers

Dir	Hardware	Question this tier answers
unit/	none	Do the parsing / filtering / profile-generation primitives work?
provisioning/	1 device, per-test bake	Did my pre-bake recipe stick? Does it survive a factory reset?
admin/	1 device, shared bake	Do my daily admin ops (owner, channel URL, config writes) round-trip?
mesh/	2 devices, shared bake	Do my devices actually form a mesh? Send + receive? ACKs?
telemetry/	2 devices, shared bake	Is telemetry reporting? Is position broadcast correct?
monitor/	1 device, shared bake	Is the boot log clean (no panics)?
fleet/	varies	Are my CI runs isolated from each other? Are reflashes idempotent?

Quick start

cd mcp-server
pip install -e ".[test]"

# No hardware — 33 unit tests, ~3 seconds
pytest tests/unit -v

# Hub attached (nRF52840 + ESP32-S3) — first run bakes, then exercises everything
pytest tests/ --html=report.html

# Hub already baked with session profile (dev loop) — skip bake
pytest tests/ --assume-baked --html=report.html

# Force a rebake (new firmware, new seed, etc.)
pytest tests/ --force-bake --html=report.html

CLI flags

• --force-bake — always reflash both roles at session start, even if the current state matches the session profile.
• --assume-baked — skip test_00_bake.py entirely. Use when you know the devices are already baked and want a fast dev loop.
• --hub-profile=<yaml> — point at a YAML file for non-default hub hardware. Default targets VID 0x239a (nRF52) and 0x303a/0x10c4 (ESP32-S3).
• --no-teardown-rebake — skip the session-end rebake that provisioning/ and fleet/ tests perform. Useful in rapid iteration.

Environment variables

• MESHTASTIC_FIRMWARE_ROOT — firmware repo path (defaults to ../ from tests/)
• MESHTASTIC_MCP_ENV_NRF52 — PlatformIO env for the nRF52 role (default rak4631)
• MESHTASTIC_MCP_ENV_ESP32S3 — PlatformIO env for the ESP32-S3 role (default heltec-v3)
• MESHTASTIC_MCP_SEED — override the session PSK seed (default: pytest-<unix-ts>). Set this to reproduce a specific failing run.

Fixtures you'll use when adding tests

All defined in conftest.py:

• hub_devices → {"nrf52": "/dev/cu.X", "esp32s3": "/dev/cu.Y"}. Auto- skips the test if a required role isn't present.
• test_profile → USERPREFS dict for the session (build_testing_profile).
• no_region_profile → variant without USERPREFS_CONFIG_LORA_REGION.
• baked_mesh → verifies both devices are baked with the session profile (does NOT reflash — that's test_00_bake.py's job).
• baked_single → single verified baked device; parametrize request.param to pick role.
• serial_capture → factory; cap = serial_capture("esp32s3") starts a pio device monitor session, drains into a per-test buffer, attaches the buffer to the pytest-html report on failure.
• wait_until → exponential-backoff polling helper; wait_until(lambda: predicate(), timeout=60) replaces flaky time.sleep() patterns.

Reports

pytest --html=report.html produces a self-contained HTML with:

• Per-test pass/fail/skip with timings
• On failure: serial log capture from any serial_capture fixture used
• On failure: device_info + lora config JSON for every role on the hub
• Session seed and session start time (for reproducibility)

pytest --junitxml=junit.xml produces CI-integration XML.

tool_coverage.json is emitted at session end in the tests directory — shows which of the 38 MCP tools the run exercised. Useful for closing test gaps.

Adding a new test

1. Pick the category that matches the operator concern (not the technical surface). "Does my fleet's owner name persist" is admin/, not unit/.
2. If you need both devices, depend on baked_mesh. If you need one, depend on baked_single. If you need to mutate hardware state, put it in provisioning/ or fleet/ and add a try/finally teardown that re-bakes the session profile.
3. Use wait_until for anything involving LoRa timing — fixed sleep() produces flakes.
4. Use serial_capture when you need to observe firmware log output (e.g. "did the packet get decoded?").
5. Add a @pytest.mark.timeout(N) — mesh tests routinely hit LoRa-airtime waits; default pytest timeout is infinite.

Troubleshooting

• All hardware tests SKIP → hub not detected. Plug in the USB hub, verify with pytest tests/ --collect-only or python -c "from meshtastic_mcp import devices; print(devices.list_devices())".
• baked_mesh fails with "devices not baked" → run pytest tests/test_00_bake.py first, or pass --force-bake on the full run.
• Mesh formation tests time out → check that both devices are on the same session profile (--force-bake forces both to the current seed).
• Provisioning tests leave device in bad state → teardowns re-bake, but if a test crashes between "bake broken state" and "bake good state", run pytest tests/test_00_bake.py --force-bake to recover.