mirror of
https://github.com/meshtastic/firmware.git
synced 2026-11-01 11:59:23 -05:00
renovate/python-3.x
863
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 renovate[bot]
|
1d0fe91aa9 | Update python | ||
|
Austin
|
1dff54cabf |
Actions: pin python to 3.14 where platformio is used (#12210)
Pin python 3.14 (not `3.x`) now that 3.15 exists. Fixes currently broken wasm / test jobs. |
||
|
Jonathan BennettandClaude Opus 5
|
2cf97c1b64 |
Pin membrowse-action to a SHA to unbreak CI (#12209)
A push to membrowse/membrowse-action's v1 tag today shipped an action.yml that
uses ${{ github.workspace }} in the `description:` of its new source_root input.
Expressions are evaluated in the `inputs:` section too, and the github context
does not exist there, so the runner refuses to load the manifest:
membrowse/membrowse-action/v1/action.yml (Line: 77, Col: 18):
Unrecognized named-value: 'github'
Every membrowse job now fails in "Set up job", before any of our steps run, and
build_firmware.yml's membrowse job feeds ci-gate - so the required check is red
on every PR in the repo.
Pin both references to 34719bdb0472, the SHA v1 resolved to on the last green
run (membrowse 1.2.15, 03:34 UTC today), which also stops a third-party tag
move from reaching us unannounced again. comment-action/action.yml is identical
at both SHAs; pinning it is for consistency, not because it broke.
The repo's own size enforcement is unaffected either way: size-budget-gate
reads the manifest-* artifacts from pio-build, not anything MemBrowse produces.
Claude-Session: https://claude.ai/code/session_01JjAvRt6xuztC3hKqWkW9Dw
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Jonathan BennettandClaude Opus 5
|
329b7ea918 |
Fail the native simulator job on a sanitizer fault, and fix the one it found (#12205)
* Fail the native simulator job on a sanitizer fault, and fix the one it found The integration step backgrounded meshtasticd and ended in `wait "$PID" 2>/dev/null || true`, which discards the simulator's exit status. The coverage env is built with -fsanitize=address, so a fault there prints a full report and exits non-zero on the way out, after the Python harness has already printed its own success - and the job concluded green with an AddressSanitizer heap-use-after-free sitting in the log. Gate it: capture the simulator's output, read the status `wait` reports, and scan the output for a sanitizer report. The pattern moves to bin/lib/sanitizer.sh so bin/run-tests.sh and the new bin/check-sanitizer-log.sh share one matcher rather than drifting apart; the output is uploaded as an artifact, since a fault on the exit path prints after the last test has passed. Leak detection is off, as in bin/test-config-check.sh - exit_simulator exit()s with the node's globals still live. The fault it had been hiding: OSThread::run() points currentThread at the running thread for the whole of runOnce(), and exit_simulator calls exit(0) from AdminModule while ServerAPI is that thread. ServerAPI is declared `public StreamAPI, private OSThread`, so its Thread base frees ThreadName first and PhoneAPI::~PhoneAPI's own LOG_DEBUG then reads it back through currentThread. Clear the pointer in ~OSThread when it names the thread being destroyed, which covers any thread deleted from inside its own runOnce, not just this one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JjAvRt6xuztC3hKqWkW9Dw * Detach observers when an Observable is destroyed The new sanitizer gate turned the simulator job red on a second heap-use-after-free on the same exit path, this one in Observable::removeObserver. ~Observer removes itself from every observable it watches, but Observable has no destructor and dies with its watchers still pointing at it. exit_simulator exit()s from AdminModule, and static destruction takes the global xModem - whose packetReady is an Observable that PhoneAPI::handleStartConfig attached to - before the heap-allocated ServerAPI that is watching it. PhoneAPI::close() then calls unobserve(&xModem.packetReady) on the dead observable, and ~Observer would walk it again afterwards. Both halves of the contract, so neither side has to outlive the other: ~Observable drops itself from each watcher's list, and unobserve() checks membership before reading through the pointer it is given. Also take CodeRabbit's grep -m 4 in bin/check-sanitizer-log.sh: with more than four matching lines, head closing the pipe is a SIGPIPE that pipefail reports and set -e exits on, so the script gave 141 where it documents 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JjAvRt6xuztC3hKqWkW9Dw --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Jonathan BennettandClaude Opus 5.5
|
fad52bba0d |
CI: let a running push run finish instead of cancelling it (#12182)
* CI: let a running push run finish instead of cancelling it #12162 put pushes in one concurrency group per branch with cancel-in-progress, so a merge landing mid-run threw away a part-built full matrix. Set cancel-in-progress only for non-push events. A push now lets the running run finish and waits as the group's single pending run; a newer push replaces that pending run before it starts, so pushes that land mid-run collapse into one follow-up run. PRs still cancel their superseded runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: correct the concurrency group comment A PR groups on head_ref and a push on the full ref, so the two never share a group; the comment said they did. The expression is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Jonathan BennettandClaude Opus 5.5
|
c4b08e82a8 |
CI: build no firmware in the merge queue; trim repeated CI work (#12162)
* CI: run pio check on PRs only, not in the merge queue The merge queue re-ran cppcheck on the same --level pr boards the PR run had already checked. Restrict the check job to pull_request; ci-gate already treats a skipped check as passing, so merge_group runs gate on the builds alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: cancel superseded push runs on the same branch Push runs fell back to github.run_id for their concurrency group, so every merge to develop ran the full 156-board matrix to completion even when a newer merge had already landed. Group pushes by github.ref so the newer push cancels the older run. PRs keep grouping by head_ref, and merge_group, schedule and workflow_dispatch runs keep a group of their own, so the queue, the nightly and releases are never cancelled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: move axiometa-genesis-mini out of the PR board set Set its board_level to release. PRs and the merge queue now build 17 boards instead of 18; the board is still built in the full matrix on push, nightly and release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: reuse the merge queue's PR-level builds in the post-merge run A post-merge push to master/develop lands the exact commit its merge_group run already built, so the PR-level boards were compiled twice. When the push's merge_group run is found and its ci-gate passed, setup marks those boards 'reuse' and build_firmware.yml copies their firmware and manifest artifacts from that run instead of building. The artifacts keep their names, so the size report baseline, the RAM/flash budget gate and MemBrowse are unchanged. If no such run is found (event/* and feature/* pushes, a push that bypassed the queue, an API failure), every board builds as before. ci-gate is read through the check-runs API: the run's jobs endpoint returns 502 for a run this size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: skip macOS, Windows and debian-src on reused post-merge pushes When setup finds the merge_group run that already built this commit (reuse_run_id), these jobs ran on that exact commit there and nothing on push consumes their output: the debian-src artifact is only read by release-artifacts on workflow_dispatch. test-native, WASM and Docker still run, because only pushes to develop save their caches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: only skip OS builds on push when the merge queue run passed them ci-gate needs only build and check, so a merge_group run whose macOS, Windows or debian-src job failed still merges, and skipping those jobs on the push hid the failure. setup now sets skip_os_builds only when the latest attempt of every MacOS, Windows and build-debian-src check run in that merge_group run concluded success and all three are present; the jobs skip on that flag instead of reuse_run_id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: harden the post-merge reuse lookup (review fixes) - Find the merge_group run at any status. The queue merges once ci-gate passes, often while jobs outside it still run, so requiring a completed run usually found nothing and rebuilt every board. - Mark a board 'reuse' only if its firmware and manifest artifacts are still present and unexpired in that run; any other board compiles. setup now needs version to build the exact artifact names. - Never skip MacOS, Windows or build-debian-src on the default branch: those jobs save their caches only on its pushes. The skip stays for master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: drop the macOS/Windows/debian-src skip on post-merge pushes Those jobs save their caches only on default-branch pushes, so the skip was already limited to master, where it saved too little to justify the check-run lookup. They now run on every push exactly as before this PR. Reuse of the PR-level firmware builds is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: look up the merge_group run by workflow file, not display name Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L * CI: build no firmware in the merge queue; drop post-merge artifact reuse Per review: the merge queue now only batches PRs into develop/master. setup is skipped on merge_group, which skips the build and check matrices that depend on it, and ci-gate accepts a skipped build on merge_group only; a PR must still build. Post-merge pushes build the full matrix as before. With no firmware built in the queue there is nothing to reuse, so the post-merge artifact copy (reuse-build, the merge_group run lookup and setup's dependency on version) is removed and build_firmware.yml is back to develop's version. The merge_group branches of setup's matrix generation are removed with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DdE8uBENCwUMApqYdut2L --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Austin
|
38ae9aab86 |
Renovate: Fix lookup errors / tag formatting (#12176)
Also unpins actions/checkout... no need for this on official Actions (needless renovate PR churn). |
||
|
|
1e3a38fc8a |
Add native Portduino mDNS discovery with Avahi and Bonjour (#12032)
* Add native Portduino mDNS discovery via Avahi and Bonjour * Guard native discovery in Portduino WASM builds * Add linux dependencies in packaging, support "buildroot" Adds libavahi dependencies for Debian, RPM, Docker and CI. Moves the libavahi definition to the shared `build_flags` (which is only consumed on Linux targets) * Fix platformio formatting * fix(portduino): disable Avahi when daemon unavailable --------- Co-authored-by: Jake-B <jake-b@users.noreply.github.com> Co-authored-by: vidplace7 <vidplace7@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Austin
|
8e4ccb029a |
Increase timeout for dput publish steps to 45 minutes (#12140)
`dput` PPA upload operations began showing their status properly after #11941, revealing that many uploads are reaching ~85% and then timing out. This change increases the timeout for dput publish steps to prevent these failures. |
||
|
Austin
|
373bff393f |
Actions: Update/pin to Ubuntu 26.04 (#11941)
ubuntu-latest means we don't own the break (if it happens), always pin! |
||
|
Jonathan BennettandClaude Opus 5
|
a8120ef176 |
Gate verbose radio logging behind LOG_DEBUG_RADIO (RADIO_DEBUG) (#12071)
Console I/O is expensive on some platforms - on a T-Beam over HWCDC a log line can block behind someone else's USB drain - so chatty radio logging sits on the path it is meant to observe. Add a gate in the shape of GPS_DEBUG/LOG_DEBUG_GPS: RADIO_DEBUG (default 0) and LOG_DEBUG_RADIO(), which costs no flash when off. It lives in DebugConfiguration.h beside LOG_TRACE, since every radio driver already includes it. Moved behind it: chip bring-up detail (constructor pin lines, TCXO probing, RF-switch and irqDioNum selection, current limit, FW version, DCDC and PA table) and the recurring per-event chatter (standby failures, sleep entry, AGC reset, frequency-offset reads on every RX, noise-floor reset, invalid RSSI samples, sleep veto, cancelSending, no-LoRa and pending-TX drops, the weighted Tx delay printed per relayable packet). Unchanged: packet lines (printPacket "Lora RX"/"Started Tx"), init results, region/frequency/bandwidth/power summaries, and every LOG_WARN/LOG_ERROR anomaly. Two constructor lines that were LOG_WARN for no reason (LR11x0, LR20x0) are now gated like their SX126x/SX128x/RF95 counterparts, and the noise-floor reset line drops from LOG_INFO, since it fires with every 60 s AGC upkeep. Claude-Session: https://claude.ai/code/session_019nsLa5cFMMcYhbDSRp6hyW Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Jonathan BennettandClaude Opus 5
|
c7826791d4 |
feat(portduino): admin keys in config.yaml, beyond the protobuf's three (#12056)
* feat(portduino): admin keys in config.yaml, beyond the protobuf's three meshtasticd can now list the public keys authorized to administer the node in its own config, as Security.AdminKeys, and list more than the three config.security.admin_key holds. The host's file is the authority: it is root-owned, so unlike the persisted protobuf an authorized remote cannot rewrite it, and applyHostKeys() re-asserts it at boot and after every remote write of the security config. A key removed from the file loses access on the next boot instead of lingering in flash. The protobuf array stays at three. It is not an arbitrary cap: a security set_config already encodes to ~207 bytes against the 221 a PKC admin packet can carry (233 payload bytes less 12 of PKC overhead), so a fourth key would not fit in one admin message on any platform. Four sites decided who may administer a node, each unrolling admin_key[0..2] by hand - the PKC gate in AdminModule, the same gate in DMShell, the is_managed guard, and the admin-key trial decrypt in Router::perhapsDecode. They now share AdminKeys::isAuthorized(), so a key source reaches all four at once, and enumeration skips empty slots rather than trusting admin_key_count. Each configured key costs one trial decryption per PKI packet from an unknown sender, so the host list is capped at 16 and duplicates are dropped. Files in config.d append to the list, one per operator. A key that is not exactly 32 canonical base64 bytes stops meshtasticd rather than silently authorizing something else, and --check reports it with its line instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SP5hpFwdTBC2K9ggGFLpcs * test(portduino): config-check fixtures for Security.AdminKeys bin/test-config-check.sh drives a built meshtasticd against the fixtures, which is the only place the two process-level behaviours here are reachable: what --check reports, and that a normal boot refuses the file. A clean two-key list, a key three characters short of 32 bytes, and one key written as a scalar instead of a list - the last two as errors under --check and as a non-zero exit on a normal boot, since a truncated admin key would authorize something other than what was written and a file whose only key is unreadable would otherwise boot a node nobody can administer. adminkeys-append/ pins the append-across-config.d rule by value through --output-yaml, with the first key in config.yaml so the effective list does not depend on the filesystem's order. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SP5hpFwdTBC2K9ggGFLpcs --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Jonathan BennettandClaude Opus 5
|
8c0abbd522 |
feat(portduino): BLE peripheral support via BlueZ for meshtasticd (Raspberry Pi) (#11396)
* feat(portduino): BLE peripheral support via BlueZ for meshtasticd on Linux Adds the standard Meshtastic BLE service (toRadio/fromRadio/fromNum/logRadio) to the Linux native target, so a Raspberry Pi running meshtasticd can be paired and used over BLE like any other Meshtastic device. Implementation: a new LinuxBluetooth backend registers a GATT application, LE advertisement and pairing agent with bluetoothd over the org.bluez D-Bus APIs, using sdbus-c++ (both the 1.x and 2.x major versions, via a small compat shim - Debian bookworm/Ubuntu 24.04 ship 1.x, trixie/Fedora ship 2.x). When the sdbus-c++ dev package is absent the whole backend compiles out via __has_include, the same optional-dependency idiom as the ulfius webserver. Threading follows the NimbleBluetooth model, simplified: the sdbus event loop runs its own thread, and all PhoneAPI calls happen on the main thread. Writes queue to the main loop; reads park the D-Bus reply and are completed from the main thread after queued writes, so write-then-read clients see their answer without any busy-waiting. Enablement is a double opt-in: a new `Bluetooth:` config.yaml section (Enabled, default false; AdapterId, default hci0) must turn BLE on for the host, and the regular device config bluetooth.enabled must be on. The config-check schema and fixtures cover the new section. Pairing honors config.bluetooth.mode: NO_PIN maps to a NoInputNoOutput just-works agent; RANDOM_PIN to DisplayOnly with the kernel-generated passkey shown on screen/log via the existing BluetoothStatus plumbing. FIXED_PIN falls back to random-passkey semantics with a warning - BlueZ does not support forcing a passkey. PIN modes enforce encrypt-authenticated-read/write on all mesh characteristics. Packaging: install a D-Bus system policy so the meshtasticd user may talk to org.bluez, add it to the bluetooth group, order the unit after bluetooth.service, and add libsdbus-c++-dev to debian/rpm/docker/CI deps. Verified in-container against a mock bluetoothd: registration flow, GATT tree enumeration, advertisement properties, and a full config download (ToRadio wantConfig -> 47 FromRadio packets) through the D-Bus bridge. Real-hardware pairing/notify testing on a Pi still pending. Known limitations (v1): meshtasticd must be restarted if bluetoothd restarts; FIXED_PIN degrades to a random passkey; getRssi() returns 0 (same as nRF52). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): BLE fixes from first real-hardware pass (Pi CM5 + RAK6421) Findings from testing PR #11396 on a Raspberry Pi CM5 (Pi OS trixie, BlueZ/sdbus-c++ 2.1 - the v2 compat path) with a RAK6421 HAT and an Android phone: - getMacAddr() leaked its HCI socket on every call and never closed it, and on failure returned without touching the caller's buffer - which getDeviceName() passed in uninitialized. Close the socket on all paths and cache the MAC after the first successful read; it cannot change at runtime and this now runs on every bluetoothd property read. - getDeviceName() zero-initializes its MAC buffer, and LinuxBluetooth snapshots the name once at setup() on the main thread: the advertisement's LocalName getter runs on the D-Bus event-loop thread and getDeviceName()'s static buffer is not thread-safe. - Restore NimBLE-style config-phase packet prefetch (depth 3). The initial port answered every FromRadio read with a D-Bus -> main-loop round trip, which made the config download noticeably slow; ReadValue now answers straight from the prefetch queue on the event-loop thread, with NimBLE's safety rules (never in STATE_SEND_PACKETS, writes always observed before reads, queue cleared on disconnect). - Set advertising MinInterval/MaxInterval to 20-100ms (BlueZ >= 5.71; older versions ignore the properties). btmon showed the kernel default of 1.28s otherwise, and Android's background-connect scan windows are sparse enough that tap-to-connect took 8-14s; 20ms is the same floor NimBLE uses on ESP32. Verified on hardware: scan, passkey pairing, connect, config download, reconnect after bond wipe. Also diagnosed (no code change): the node identity MAC comes from the RAK HAT EEPROM by design, so the BLE name suffix follows the HAT rather than the BT adapter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): address CodeRabbit review on BLE support - setBluetoothEnable: handle disable before the config gate, so a running BLE stack is always stoppable even after the device config turns Bluetooth off underneath it - getMacAddr: read the adapter configured as Bluetooth.AdapterId instead of hardcoding hci0, falling back to hci0 for unparseable names - systemd unit: Wants=bluetooth.service so bluetoothd is pulled up when present (After= only orders, it does not start it) - debian/rpm: Recommends: bluez as the runtime contract for BLE - dbus policy: document why the org.bluez rule is destination-wide rather than a per-interface allowlist Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): show the BLE pairing code on BaseUI screens onDisplayPasskey published the passkey to bluetoothStatus and triggered PowerFSM, but never called screen->startAlert(), so on BaseUI the code only ever reached the log. BluetoothStatus has no BaseUI consumer -- only InkHUD's PairingApplet and StatusLEDModule read it -- so a Pi driving a HUB75/OLED panel showed nothing while BlueZ sat waiting for the user to type a code they could not see. NimBLE and nRF52 draw it via startAlert(); this adds the missing half for Linux. The agent callbacks run on the sdbus event-loop thread while the screen is owned by the main thread, so the passkey is handed over as a pending flag and drawn from runOnce(), matching the existing disconnectCleanupPending pattern rather than reaching into the screen from the event loop. Dismissed on all four exits, so a stale code cannot stick on an always-on panel: Paired -> true (newly watched in PropertiesChanged, which previously only looked at Connected), agent Cancel, peer disconnect (moved out of the lastGone branch so a peer leaving mid-pairing clears the code even when another device is still connected), and doDeinit() -- applied inline there because runOnce() may never be scheduled again after teardown. Verified on a Pi 5 + BlueZ 5.66 in RANDOM_PIN mode: the code renders on a HUB75 panel and clears once the phone completes pairing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: install libsdbus-c++-dev for the native test build setup-native-test landed on develop while this branch was adding libsdbus-c++-dev to setup-native, so the new action's "full setup-native list" of C libraries is missing it. Without the package the test job builds with HAS_BLUETOOTH 0 and never compiles LinuxBluetooth.cpp. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): warn when a factory reset cannot clear BLE bonds factoryReset(eraseBleBonds) silently did nothing on Linux when the BLE backend was not running, so the reset reported success while the host's pairings stayed. Removing them needs a live connection to bluetoothd that a disabled backend never opened, so say so rather than imply they went. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): gate the factory-reset bond clear on an enabled backend setup() leaves linuxBluetooth allocated with its bus torn down when it throws, so a pointer check alone let factoryReset log "Clear bluetooth bonds" for a clear that clearBonds() then declined to perform. isEnabled() is only true after setup() completes, which routes that case to the warning that says the bonds were left alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * style: reformat under clang-format 20 #11909 moved trunk from clang-format 16 to 20, which spaces C-style casts differently and reindents the comment above the HAS_WIFI block. Both files are ones this branch already touches, and trunk's fmt linter grades whole files, so its check fails until they are reformatted. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): three BLE config and lifecycle fixes from review Bluetooth config keys are now assigned individually rather than per section. loadConfig() runs once for every file in config.d, so reading an absent key as its default let a later file that named only one of them silently reset the other: `AdapterId: hci1` alone turned Bluetooth off, and `Enabled: true` alone dragged the adapter back to hci0. Only what a file actually states should override what an earlier one set. A backend that failed to come up is now retried. setup() can leave linuxBluetooth non-null but disabled - bluetoothd not ready, adapter missing, policy refusing - and every later enable then called resumeAdvertising(), which returns immediately while disabled. A transient failure at boot kept BLE off until the process restarted. doSetup() already opens with `if (enabled) return` and tears the bus down on every failure path, so calling it again is safe. Bluetooth.AdapterId is now checked for the hci<digits> form. LinuxBluetooth uses the value verbatim as the BlueZ object path while the MAC fallback reads only the leading hciN, so "hci1junk" looks plausible, yields a MAC, then finds no adapter and BLE never comes up. Covered by a new fixture and suite case, which is the kind of silent no-op that directory exists to catalogue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
e0c76fd41e |
Improve performance of encrypted packets with a shared secret cache (#11979)
* Improve performance of encrypted packets with a shared secret cache Every PKI encrypt, decrypt and ack-proof ran Curve25519::dh2 plus a SHA256 to derive the pairwise key, so a node in a conversation paid a full X25519 per packet, on the main loop, under cryptLock. On a RAK4631 that is ~96 ms of the ~210 ms it takes to handle a DM. setCryptoSharedSecret() derives the key only when it is not already held for that peer, keeping the last MAX_CACHED_SHARED_SECRETS derivations (8 on nRF52, 2 on STM32WL, 10 elsewhere, under 400 bytes) and evicting the least recently used. encryptCurve25519, decryptCurve25519 and ackProofCompute all go through it, so the ack proof gets the same cache without a second DH path. The cache is emptied whenever our own private key changes, since every secret in it is then stale. The lookup key is the first 4 bytes of the peer's public key. A collision makes us derive against the wrong cached secret, which costs a failed decrypt for that pair; it cannot disclose either secret. Co-Authored-By: Jonathan Bennett <jbennett@incomsystems.biz> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLHxcWJuvSoSSMpz3LWdt3 * Do not let an empty cache slot answer for a zero-prefixed peer key The cache used lookup_key == 0 to mean "slot unused", so a peer key whose first 4 bytes are zero matched every unused slot and was handed that slot's zeroed secret as a hit. The all-zero key is exactly such a key, which is how test_proof_rejects_weak_peer_key caught it: dh2's weak-point check never ran. Entries carry an explicit valid flag instead, which the struct's existing padding absorbs. A weak peer key is also now rejected before the cache is consulted, since with 4 bytes of lookup key it could otherwise collide with a cached peer and be served that peer's secret rather than being refused. Co-Authored-By: Jonathan Bennett <jbennett@incomsystems.biz> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLHxcWJuvSoSSMpz3LWdt3 * Key the shared secret cache by the whole peer public key A four-byte lookup key is grindable: anyone can generate a keypair whose public key shares those bytes with a peer they want to shadow, get their own entry cached, and then be handed the secret this node uses to talk to that peer - readable by them, since they hold the matching private key. That is disclosure of traffic meant for the peer, and a forgeable ack proof in its name, not the failed exchange a chance collision would cause. Entries hold the peer key itself and are matched on all 32 bytes, so the weak-key check dh2 does on a miss can no longer be skipped either and the explicit isWeakPoint guard goes away with it. The cache costs 66 bytes per entry: 528 on nRF52, 660 where the default 10 entries apply. Co-Authored-By: Jonathan Bennett <jbennett@incomsystems.biz> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLHxcWJuvSoSSMpz3LWdt3 * Allowlist the cache's uptime shift for the millis deadline guard The guard's regex reads `millis() >> 22` as a comparison against the uptime clock. It is a right shift, coarsening uptime into the ~1.165 hour units the cache stamps entries with, and the eviction arithmetic handles that stamp's 8-bit wrap itself. Co-Authored-By: Jonathan Bennett <jbennett@incomsystems.biz> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JLHxcWJuvSoSSMpz3LWdt3 --------- Co-authored-by: Jason B. Cox <contact@jasonbcox.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
f5940ab5dd |
Meshtasticd notifications (#10216)
* First attempt at sending message notifications on Linux
* Add files via upload
* Notifications: Use libnotify
Use libnotify for meshtasticd desktop notifications.
Add "HAS_LIBNOTIFY" macro to guard for builds where libnotify is expected.
This is not included in buildroot/openwrt builds. (no libnotify except when extra repos are added).
Install desktop icon in the correct location on Debian and Fedora packages.
Update dependencies in packaging and dockerfiles.
* Add libnotify to setup-native (GitHub Actions)
* Address review feedback on the meshtasticd notification path
- platformio.ini: only define HAS_LIBNOTIFY when pkg-config actually finds
libnotify. The probe previously ran unguarded and the macro was defined
unconditionally, so a native build without libnotify-dev both hard-failed at
config time and claimed the feature was available.
- Fix the sender lookup for develop's flattened NodeInfoLite: has_user/user.*
are gone, replaced by nodeInfoLiteHasUser() and direct long_name/short_name.
This is a silent semantic conflict - it merges cleanly but does not compile.
- Move the desktop notification off the packet path. notify_notification_show()
is a synchronous DBus round trip and meshtasticd's packet handling is
single-threaded, so a slow or wedged notification daemon could stall the
radio. handleReceived() now resolves the strings and queues them (bounded at
16); a dedicated worker owns every libnotify call.
- Stop retrying forever: a failed notify_init(), or three consecutive failed
shows, latches desktop notifications off instead of re-logging per message.
- NodeDB: the ARCH_PORTDUINO default-enable block was nested inside
#ifdef HAS_I2S, which Portduino never defines, so it never ran. Hoist it out.
* fix(portduino): close input-broker guard before the libnotify block
The libnotify implementation was inserted ahead of the #endif that closed
#if !MESHTASTIC_EXCLUDE_INPUTBROKER, so the file's trailing #endif closed
#if HAS_LIBNOTIFY instead and the input-broker conditional was never closed.
Every build target failed to preprocess:
src/modules/ExternalNotificationModule.cpp:630: error: unterminated #if
Close the guard immediately after handleInputEvent(), as develop does, so the
two conditionals stay independent and HAS_LIBNOTIFY still resolves when the
input broker is excluded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): back off desktop notifications instead of latching them off
notifyDisabled was doing two jobs: "libnotify looks unusable" and "the
destructor wants the worker to exit". Because the worker's only exit path was
also its only failure path, three failed shows turned notifications off for the
process lifetime with no way back. The packaged daemon is exactly that case:
bin/meshtasticd.service runs as User=meshtasticd and the rpm spec creates that
account with /sbin/nologin, so there is no session bus and every show() fails.
A stock install logged three warnings and then went silent forever, while
NodeDB now enables the module by default on Portduino.
Split the flag. notifyShutdown is destructor-only and remains the worker's one
exit; a retry window replaces the latch. After maxNotifyFailures the worker
arms a backoff (30s, doubling to a 15min cap), drops the queue rather than
holding stale popups, and keeps looping. The producer refuses to queue while
the window is open, so the retry is driven by the next message after it expires
rather than by a timer - no idle wakeups, no probe notifications. Any success
resets the backoff. notify_init() moved inside the loop so a retry can pick up
a session bus that was absent at startup.
The worker also no longer calls the LOG_ macros. RedirectablePrint formats into
a shared static buffer that nothing guards and every other writer to it is on
the main thread, so the worker now records the state change under the mutex it
already holds and reportNotifyStatus() emits it from portduinoNotify() and
runOnce(). Per-message failure spam becomes one line per transition: the reason
and retry interval when it goes down, and a line when it comes back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): sanitize mesh text before it reaches libnotify
The notification body was the raw decoded payload and the summary was a node
name, both attacker-controlled, and neither was checked before being handed to
libnotify. g_variant_new_string() rejects invalid UTF-8: a GLib CRITICAL and an
"[Invalid UTF-8]" body by default, and a hard abort under G_DEBUG=fatal-criticals,
so an unauthenticated mesh packet could terminate meshtasticd on any install
running with that flag. An embedded NUL separately truncated the body at the
first one, hiding the rest of the message.
Route both strings through sanitizedMeshText(), which replaces embedded NULs and
then applies the existing sanitizeUtf8() helper. TypeConversions already
sanitizes names on the way into NodeDB, but an abort is too sharp an edge to
leave resting on an invariant owned by another file.
Escape the body for Pango markup as well. Servers advertising "body-markup"
parse a markup subset there, so a message can inject formatting to dress itself
up as trusted UI, and where the server also advertises body-images or
body-hyperlinks it can inject tags that make the notification daemon fetch a
remote URL. The escaping is unconditional rather than gated on
notify_get_server_caps(): a caps query that fails, or goes stale across a daemon
restart, fails open, while on a server without body-markup the only cost is
entities rendering literally, and mesh text carries no markup worth preserving.
The summary is not escaped - the spec gives it no markup, so escaping it would
only ever show entities.
Also call notify_uninit() in the destructor, after the join: the worker owns
every libnotify call, so tearing down while it is still running would be a
use-after-uninit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): gate the notification default-on behind HAS_LIBNOTIFY
The default-enable block was gated on ARCH_PORTDUINO, so a native build where
pkg-config could not find libnotify still shipped external_notification enabled.
portduinoNotify() is not compiled into that build, and native defines
EXT_NOTIFICATION_MODULE_OUTPUT as 0 with setup() guarding the pin writes behind
output > 0, so nothing was driven - it only exposed a config surface that can
do nothing. Gate it on the same macro that governs the code it exists to feed.
HAS_LIBNOTIFY is a build flag for the whole env and is simply absent when the
probe fails, so this reads as 0 on every other target, matching how
ExternalNotificationModule.cpp already tests it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): fail closed when markup escaping returns NULL
escapedNotificationBody() returned its input unescaped if g_markup_escape_text()
gave back NULL, which would hand libnotify the exact attacker-controlled string
the function exists to neutralize. The branch is unreachable for the input we
pass - already sanitized to valid UTF-8, and GLib documents no NULL return for
it - but an error path whose fallback is the unsafe action is the wrong shape
for a helper on this boundary. Drop the body instead.
Also note in the doc comment why running this on the caller's thread does not
weaken the rule that the worker owns every libnotify call: it is a pure GLib
string function that touches no libnotify or DBus state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* style: format ExternalNotificationModule.h per the pinned clang-format
Trunk Check Runner failed on
|
||
|
Austin
|
9d8ce1c780 | Actions: Support MemBrowse memory/flash tracking (#12006) | ||
|
Tom
|
f90b48ea6c |
chore: trunk fmt --all (#11938)
Whitespace and comment-alignment only: the output of `trunk fmt --all` on
develop @
|
||
|
Tom
|
08cd97ea2d |
test(harness): make run-tests.sh drivable by a caller that cannot see the terminal (#11862)
* test(harness): make run-tests.sh drivable by a caller that cannot see the terminal A tool call or a fresh session reads a captured output file, gets interrupted mid-run, and starts cold. Two things in the wrapper tripped that caller: an interrupt left pio's build tree running with no record of it (the next invocation started a second build into the same .pio/build/, or pgrep'd and matched itself); and pio's own "[PASSED]" / "N succeeded" lines made a half-finished output file read as green. run-tests.sh: - Run record in .pio/runtests/current.tsv for the life of a run. A second invocation prints RESULT: BUSY and exits 4 without touching the build directory. Valid while the holder pid OR the recorded process group is alive, so a SIGKILLed wrapper with live scons children reads ORPHANED rather than clear. - --status / --wait / --abort. --abort kills the whole tree by pgid. - pio runs under setsid with its pgid recorded; INT/TERM/HUP kill the tree and record RESULT: ABORTED (exit 5), log kept. - One result() for every verdict: prints to the stdout the script started with (a signal can arrive inside a redirected pio call) and writes .pio/runtests/last-result.tsv with head, args, env, finish time, kept log and a tree fingerprint (HEAD + working-tree diff + untracked files). --status marks the last verdict STALE when the tree has changed since. - Banner naming the final RESULT: line as the only verdict. - Non-Linux host: RESULT: UNSUPPORTED, exit 6, instead of the AMBER code. - A failed build removes .pio/build/<env>/meshtasticd, which run bare would reprint the last good run. - FILTERED lists the not-run count, not 77 suite names. bin/run-tests.cmd forwards into WSL with the exit code passed through, so the same command line works from cmd.exe and PowerShell; no logic is duplicated. test/README.md, copilot-instructions.md and the mirrors document the new codes and the rule. * test(harness): one-suite warm-up, and name the build phase instead of freezing the counter Measured on a full native run: the warm-up, `pio test --without-testing` with no filter, builds AND links every suite - 78 links, 2320 s, 29.7 s each, 39 minutes before the first test ran - and prints a "[PASSED]" line for each program it merely linked. CI never did this; its warm-up is one `platformio run`. The shared src objects are the same whichever suite links them, so the warm-up now links one: the filtered suite when there is one, else test_utf8. The run itself still builds every suite, as it must. The heartbeat counted objects newer than its marker, which sits still through PlatformIO's single-threaded scons dependency scan and through each link - twelve minutes at "430/754 objs, ETA 17m" on that run, which reads as a hung build to a caller who cannot run ps. It now names the phase from the processes in the recorded group: [scons] / [compile] (with the ETA) / [link] / [test], and --status prints the same phase word. * test(harness): define phase_of_run before --status can call it * test(harness): bind --wait to the run it observed; serialize the run-record publish Review findings on #11862, all three valid: - --wait stored a state and then waited for any record to clear, so a run that finished and a second that started between polls would be followed to the second's verdict, and a run that turned ORPHANED mid-wait could print a stale last-result. Each run now has an id (pid-start) in current.tsv and last-result.tsv; --wait captures it and reports only a matching verdict, else ABORTED-without-verdict. - run_state() then the current.tsv write was a check-then-act pair: two invocations in the same instant could both see IDLE. The pair is now one critical section under a short-lived flock, and both records are published by rename so no reader can see a partial file. The record stays the ownership token; the lock only serializes the handoff (a SIGKILLed holder releases flock but not the record, which is why flock alone was rejected). - The FILTERED and AMBER examples in copilot-instructions.md carried a literal suite count, which the same document says never to do. Verified on a live run: --status RUNNING, a concurrent start refused BUSY, a --wait started before --abort reported the aborted run's own verdict with the matching id, exit 5; no build process survived. * fix(waypoints): do not create an empty store file on clear clearAllWaypoints() wrote a two-byte empty store unconditionally, so test_waypoint_expiry left Waypoints_default.wpts behind in every run and the suite has read AMBER (undeclared shared state) since it landed. An existing file - stale or unreadable included - is still rewritten as empty, so a reset after a failed load clears flash as before; a file that is not there is left not there. |
||
|
Austin
|
43479aa4e1 |
Actions: exclude mutable action tag rule from Semgrep scans (#11944)
We are not pedantic enough to pin Actions versions to a SHA. Re-enable semgrep scans on previously-ignored .github/workflows and disable yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag |
||
|
Austin
|
bf2ef8f2d6 |
Actions: Fix setup-python v7 version pinning (#11942)
We pin official actions to their major-version floating tag, we don't care about minor-revs here. |
||
|
Tom
|
f49cc46476 |
docs(agents): list nRF54 as a platform, drop the phantom nRF52833/nRF52832 (#11865)
* docs(agents): list nRF54L15 as a platform, drop the phantom nRF52833/nRF52832 The supported-platform list named nRF52833 and the warm-tier section excluded "bare nRF52832"; neither part exists anywhere in the tree. The nRF54L15 (ARCH_NRF54L on the nRF52 platform layer, out-of-tree meshtastic/platform-nordicnrf54 with the s145 SoftDevice core) does, with xiao_nrf54l15_lr2021 as the per-PR canary, and was missing. The warm tier's real exclusion is MEM_CLASS_TINY (STM32WL only); the nRF54L15 falls into MEM_CLASS_SMALL and keeps its warm records in /prefs/warm.dat, not the nRF52840 raw-flash ring. * docs(copilot-instructions): nRF54 has its own platform layer; warm.dat only where WARM_NODE_COUNT > 0 Matches #11867: src/platform/nrf54 (architecture.h, main-nrf54.cpp), env base nrf54_base in variants/nrf54l15/nrf54.ini, sharing only NRF52Bluetooth.cpp, Nrf52SaadcLock.cpp, alloc.cpp and hardfault.cpp with src/platform/nrf52. The persistence line named the condition instead of "everywhere else", which read as including STM32WL, which has no warm tier. * docs(copilot-instructions): the platform is nRF54; no chip qualifier * docs(copilot-instructions): state the warm-tier gate before the exception The section opened with the arch condition folded into a parenthetical beside STM32WL - "On every arch except STM32WL (`WARM_NODE_COUNT > 0`; the only `MEM_CLASS_TINY` part)" - which reads as though STM32WL is the part with the tier, contradicting the bullets below it. Lead with the rule instead. The gate is the memory class, not the part: mesh-pb-constants.h zeroes WARM_NODE_COUNT under MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY, and STM32WL is merely the only part in that class today, so naming the part as the rule would go stale the moment another tiny board lands. Saying it once up front also lets the Persistence bullet drop its own copy of the caveat. * ci: run the PR matrix on docs-only PRs so ci-gate reports ci-gate is the required status check for develop, and it lives in this workflow, whose pull_request trigger ignored "**.md". A PR that touches only Markdown never runs the workflow, never reports ci-gate, and stays BLOCKED with every other check green - which is what this PR has been for a week. Drop the paths-ignore on pull_request only. push keeps its filter: nothing gates on the post-merge run. A docs-only PR now costs one --level pr matrix, which is rare enough to be cheaper than an unmergeable class of PRs. |
||
|
Austin
|
d49cf21c3a | Actions: Add a basic retry for PPA uploads (#11923) | ||
|
James Rich
|
2a398fb2fd |
Cancel superseded runs of Check PR Labels and Semgrep Differential Scan (#11906)
* Cancel superseded runs of Check PR Labels and Semgrep Differential Scan Both workflows trigger on pull_request without a concurrency group, so every push, label or edit on a PR queues a fresh run while the earlier ones keep their place in the org's shared runner queue. Check PR Labels listens to six event types, so opening, labelling and editing one PR queued three identical runs within a minute today. Give each the same head_ref-keyed group with cancel-in-progress that CI, Tests and the trunk check already use. Develop pushes are unaffected: they fall through to run_id, as before. * Key the new concurrency groups by PR number, not head_ref Fork PRs often come from a branch named develop, so two of them share github.head_ref and one would cancel the other's run. The PR number is unique per PR. |
||
|
Austin
|
683b3533fd |
setup-base: Move python dependencies into a requirements.txt, pin versions for caching (#11876)
Moves python dependencies declared in `setup-base` to a separate requirements.txt file (which renovate can keep updated) and aligns it with gh-action-firmware. Also removes `pio upgrade`, this is pointless (we just installed the latest platformio in the previous step) |
||
|
Austin
|
a70fe9ea10 |
Actions: Build MacOS 27, Drop 15 (#11851)
MacOS 15 will still be built in homebrew, but not tested as part of CI here. |
||
|
Austin
|
c3df4914b4 |
Make Nightly index beautiful, commit nightly release_notes (#11816)
Link back to the Action, mention the commit, build date, make it pretty with CSS. Also moves release notes into .github/nightly so they can be more easily edited. (Contents unchanged). |
||
|
Austin
|
6554194e66 |
Generate (very basic) HTML index for Meshtastic Nightlies (#11807)
So https://nightly.meshtastic.org doesn't just 404. We ❤️ our nightly testers, if they don't want to use the Flasher that is fine! |
||
|
Austin
|
7aaf647064 |
Actions: Fix uploading zips to GitHub Releases (#11793)
The last two GitHub releases were *empty*. master -> develop bit us here, update the workflows to track the default branch instead. |
||
|
Tom
|
e09b961f39 |
docs(agents): exempt test headers from the two-line comment limit (#11742)
* docs(agents): exempt test headers from the two-line comment limit The one-or-two-line comment rule rests on "the diff and commit message carry the rationale". For a test that premise is false. A test outlives the PR that added it, and the next person to read it is reading it because it failed - months later, in someone else's change, with the original discussion out of reach. That reader has one decision to make: real regression, or an expectation that has gone stale? The assertions alone cannot answer it, so the justification has to live in the file. The new "Test comments" section requires three things of a test header - what is under test by symbol and file, why that behavior is required, and the specific regression that returns if the assertions are deleted or relaxed - and grants whatever length they need. Authoritative statement lives in that section of .github/copilot-instructions.md, anchored at #test-comments. AGENTS.md and CLAUDE.md link to it rather than restate it. The test/** entry in .coderabbit.yaml carries the one permitted copy, since a YAML instruction cannot follow a link. This documents existing practice rather than changing it: 57 of the 72 suites with a test_main.cpp already open with three or more comment lines, test/test_gps_fix_hold/test_main.cpp with eleven. The rule as written forbade all of them, and automated reviewers acted on it. Scoped to the single comment bullet, not to the whole "General Style" section. Unlike "Naming Conventions", that section also holds the logging tiers and the Throttle rule, which bind test code as hard as src/; a blanket preamble would have quietly exempted tests from millis() discipline as well. The exception is bounded. The CodeRabbit entry still flags narrative that carries no contract - debugging journey, changelog prose, restating what the assertions plainly do - and per-case comments that merely repeat the test name. The documentation-does-not-live-here rule is untouched. * docs(agents): cut the test-comment rule to its essential statements A section about comment length had no business running to 373 words. Canonical section down to 194: dropped the elaboration of why the commit message is unavailable, the aside about which bullet reviewers most often cut, the "not licence for narrative" preamble, and the closing flourish. What remains is what an agent has to act on - the premise that fails for tests, the three things a header states, the instruction to reject a shorten-this review comment, and the worked example. Pointer files carry scope, not argument. The AGENTS.md bullet is one sentence and a link, matching the Test naming pointer directly above it. The .coderabbit.yaml copy keeps its imperatives, since a YAML instruction cannot follow the link to find them, but loses the causal explanation it did not need in order to act. |
||
|
AustinandBen Meadors
|
0becda3017 |
Actions: Only publish nightly releases to R2 (#11719)
Depends on: https://github.com/meshtastic/web-flasher/pull/427 Do not publish high-churn nightly releases to meshtastic.github.io. Now R2-only. Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Austin
|
7b8a02bb4c |
Revert "fix(ci): build with pioarduino core instead of upstream platformio (#…" (#11760)
This reverts commit
|
||
|
Thomas Göttgens
|
f631428309 |
fix(ci): build with pioarduino core instead of upstream platformio (#11759)
* fix(ci): build with pioarduino core instead of upstream platformio
The espressif32 platform is the pioarduino fork, but setup-base installed
upstream platformio and then ran pio upgrade, so every firmware build ran
a core the platform is not built against.
Upstream 6.2.0, released 2026-09-05, moved its tool-scons core dependency
to ~4.41101.0 (SCons 4.11.1). The lazy "import SCons.Tool.FortranCommon"
that smart_link() uses to choose a linker fails there, so every ESP target
died at link-action resolution before compiling a file. Core resolves
tool-scons as a core dependency, so a platform_packages pin cannot help:
core installs its own version and removes the pinned one.
pioarduino core pins SCons 4.8.1 by URL rather than by range, so upstream
releases cannot reach it. Dropping pio upgrade as well, since it re-pulled
the latest upstream core regardless of what pip installed.
Only setup-base changes. The matrix-generation jobs install platformio to
parse the ini files and never build firmware, and the native test suites
pass on upstream core because that platform does not reach smart_link.
* Revert "fix(ci): pin tool-scons to 4.8.1 for ESP targets (#11756)"
This reverts commit
|
||
|
renovate[bot]
|
5920d05f5b |
chore(deps): update actions/setup-python action to v7 (#11727)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Thomas Göttgens
|
5b987d4897 |
docs(agents): document the test naming rules and why camelCase does not apply (#11734)
* docs(agents): document that test names are snake_case, not camelCase
Automated reviewers repeatedly ask for test suite directories and test_*
functions to be renamed to camelCase to match the src/ convention. That
rename breaks the harness and the rule was never written down.
Authoritative statement lives in the "Test naming" section of
.github/copilot-instructions.md, anchored at #test-naming. AGENTS.md and
CLAUDE.md link to it rather than restate it.
bin/run-tests.sh enumerates suites with -name 'test_*' but matches
PlatformIO verdict lines against test_[a-z0-9_]+, lowercase only, so an
uppercase suite directory is counted missing and the run reports AMBER
instead of GREEN. RUN_TEST in test/TestUtil.h passes #func to Unity and to
the state-checkpoint hooks, making the function name the only attribution a
CI failure carries.
.coderabbit.yaml gains a test/** path_instruction stating the rule inline,
since YAML cannot follow the link.
* docs(agents): separate the suite-directory and test-function naming rules
Review feedback on the previous commit was correct on both points.
The section called the test-function form snake_case while every example used
camelCase segments. The tree holds 743 test functions with an uppercase segment
and 675 without, so snake_case was wrong for more than half of them. Split the
two rules that were conflated: suite directories are strictly test_[a-z0-9_]+,
while test functions require only the test_ prefix and underscore separators,
with segment case free. States what is actually forbidden - dropping the prefix,
or collapsing the segments into one camelCase identifier.
The canonical-copy policy forbade restating the rule anywhere, then restated it
in AGENTS.md and .coderabbit.yaml. Name the YAML entry as the single permitted
copy, since a YAML instruction cannot follow a link, and reduce the AGENTS.md
bullet to a pointer.
* docs(agents): make the AGENTS.md and CLAUDE.md pointers neutral
Both still carried the "snake_case, not camelCase" label that
|
||
|
renovate[bot]andThomas Göttgens
|
76e45c5b44 |
Update actions/setup-python action to v7 (#11081)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> |
||
|
Austin
|
d5873ead0a |
R2: Cache harder! (#11710)
Increase the cache-control age limits for R2 uploads. Releases: Cache for 1 day in browser, 1 month on CDN Nightly: Cache for 1 hour in browser, 1 day on CDN Add metadata to R2 uploads so we can track which commit and which GitHub Actions run produced the upload. |
||
|
Thomas Göttgens
|
9850b76351 |
ci(test): shard the native test suite across a matrix (#11706)
* ci(test): shard the native test suite across a matrix Replace the single sequential runner with a matrix populated by bin/test-shards.py from the test/ tree: areas over --max-suites are split, smaller ones packed, and a --max-shards budget bounds the fan-out. A collector job merges the per-shard JUnit reports, checks the union against the canonical suite set, and states the verdict. Native PlatformIO Tests remains as the single required check over the matrix. Drop the --without-testing warm build. PlatformIO links every native test program to the same $BUILD_DIR/$PROGNAME, so the area run relinked each suite regardless. ccache carries the shared src objects between shards instead; one shard is flagged cache_writer so a single entry is saved. The coverage-event-policy and coverage-channel-table envs and the attribution canary move into their own matrix rows and job. Harden the new paths: bound the matrix row count so a branch cannot size the fan-out, reject multi-line or empty $GITHUB_OUTPUT values, fail the whole-run attribution gate on an empty expected set, upload exact report and tracefile names instead of globs, and pass the repo path to bin/lib/shuffle.sh as an argument rather than into bash -c source text. 12 shards, largest 9 suites. * ci(test): minimal test toolchain, cap shard runtime, fix pack overflow Add .github/actions/setup-native-test, used by the shard and canary jobs in place of setup-native. It drops the redundant second checkout, both submodules (src/mesh/generated is tracked, meshtestic is the hardware harness), cppcheck, and the adafruit-nrfutil, poetry and meshtastic pip installs, and folds in ccache and lcov. setup-base and setup-native are unchanged, so the firmware matrix and every other consumer keep theirs. Cap the shard job at 30 minutes. A lost runner held one for 48 of the 360 GitHub allows by default, and there are twelve of them. pack() could exceed --max-suites: ceil(total / cap) is a lower bound and whole areas do not divide, so three areas of 6 at cap 10 put 12 in one of two bins. Grow the bin count until every bin fits. Validate the fixed-env test_filter tokens against SUITE_RE. PlatformIO accepts globs there, and those tokens reach the same word-split and the same attribution gate as discovered names. Split with read -ra so a token cannot glob against the workspace either. Report the suite count rather than the length of the -f argument array, which counted every name twice. Trim comments to the one or two lines AGENTS.md asks for. * ci(test): quote the $GITHUB_OUTPUT redirects Applied to all five, including the three that predate this branch, so the file is consistent rather than half-converted. |
||
|
Austin
|
fca4fa88c5 |
Bump version (2.8.1) (#11695)
Bump version to 2.8.1 Fix broken meshtasticd version bumps while we're in here |
||
|
Austin
|
39383b9fcf | Actions: Also upload release / nightly builds to R2 (#11689) | ||
|
Thomas Göttgens
|
9fbc176e91 |
Extend userPrefs coverage to the whole channel table and the missing config fields (#11624)
* Extend userPrefs coverage to the whole channel table and the missing config fields initDefaultChannel() handled only indices 0-2, so USERPREFS_CHANNELS_TO_WRITE above 3 produced live secondary channels carrying the public default PSK; it now covers all eight slots, with bin/platformio-custom.py completing every field of a configured index so indices 0-2 stay byte-identical. Adds USERPREFS_CHANNEL_<n>_IS_MUTED, USERPREFS_CONFIG_DEVICE_REBROADCAST_MODE, USERPREFS_CONFIG_DEVICE_NODE_INFO_BROADCAST_SECS, USERPREFS_CONFIG_LORA_CONFIG_OK_TO_MQTT, USERPREFS_CONFIG_SECURITY_IS_MANAGED and USERPREFS_CANNED_MESSAGES, applied after installRoleDefaults() and validated the way AdminModule validates a set-config. Adds test_userprefs_channels, covering the configured table under coverage-channel-table and the stock defaults under every other env. * Address review: hex channel count, PSK width assert, canned-message termination USERPREFS_CHANNELS_TO_WRITE now parses 0x-prefixed hex, matching the format userPrefs.jsonc documents, without int(x, 0)'s rejection of a leading-zero decimal such as "03". A static_assert rejects a USERPREFS_CHANNEL_<n>_PSK literal wider than psk.bytes, which memcpy would otherwise write over the fields after it. The USERPREFS_CANNED_MESSAGES copy keeps strncpy's zero-padding and terminates explicitly, rather than shortening the length, which would have left the last byte unwritten. |
||
|
Thomas Göttgens
|
eb6df1c649 |
ci: resolve the PR diff base without a shallow refetch (#11623)
The setup job checks out with fetch-depth: 0, then refetched the base branch with --depth=1 before calling merge-base. A depth-limited fetch into a complete clone writes .git/shallow and grafts the fetched tip as parentless, so merge-base finds no common commit once the base branch has moved past the pull request merge commit. The step then failed under set -e with a bare exit 1 and no message. Use the origin/<base> ref the checkout already provides. |
||
|
Austin
|
4de20187f5 |
Actions: Update to trunk-io/trunk-action v2 -- remove annotations (#11563)
trunk-action v2 removed support for PR annotations (they have been broken for a while anyways) |
||
|
Thomas Göttgens
|
68bfe015e6 |
ci: build newly added variants in the PR matrix (#11549)
* ci: build newly added variants in the PR matrix A new board declares board_level = release, so it gets no CI build until after merge. Build the first env of each platformio.ini added by a PR, regardless of board_level. Only added files qualify; adding an env to an existing config does not. * ci: also detect added variants in merge_group runs merge_group uses the same --level pr subset as pull_request, so a newly added variant was skipped there. Derive the diff base from github.event.merge_group.base_sha for those runs. * ci: fail the matrix step when the variant diff errors Process substitution hides the exit status, so a failed diff silently yielded an empty list and dropped the new board from the matrix. Capture into a variable so 'set -e' aborts the step instead. |
||
|
Tom
|
bca7c0b480 |
Tom fiddles with the test suite - again (#11517)
* test: make every suite run its own binary, and fail the run when it does not PlatformIO links every native test program to the one $BUILD_DIR/$PROGNAME path and attributes Unity output by text alone, never checking that the source file a case came from belongs to the suite it thinks it ran. Both harnesses had been split into a build pass (--without-testing) and a run pass (--without-building), and for a non-embedded platform the run pass never relinks - so all 57 suites executed whichever suite was linked last, each reporting PASSED under its own name. Introduced for CI in |
||
|
Thomas Göttgens
|
4d524320b5 |
Add agent guideline: documentation belongs in the docs repo (#11492)
Mirrored in AGENTS.md, .github/copilot-instructions.md and CLAUDE.md, with a matching CodeRabbit path instruction for **/*.md. |
||
|
Austin
|
119cd261be | Cache docker image layers in Registry (#11495) | ||
|
Austin
|
3e71c679c1 |
Actions: Add PIO caching to build-debian-src workflow (#11465)
Prevent a few more transient fails |
||
|
|
fdb644e0b7 |
Fix millis() rollover in deadline, interval, and timestamp handling (#11291)
* Add native test coverage for the UptimeClock monotonic seam
src/UptimeClock.{h,cpp} shipped without a dedicated test suite. Port the six
tests from the monotonic-time branch (test/test_time), retargeted to the
renamed header.
The wrap test crosses 0xFFFFFFFF via advanceTestMillis() rather than a second
setTestMillis(): setTestMillis() sets clockSourceChanged, which makes
getMillis64() rebase its accumulator and swallow the wrap.
* NextHopRouter: fix 49.7-day millis() rollover in retransmission timing
Resolves the "FIXME, handle 51 day rolloever here!!!" in
NextHopRouter::doRetransmissions() by switching the retransmission-due
comparison from plain unsigned <= to a signed-difference cast.
The previous p.nextTxMsec <= now comparison silently breaks across the
~49.7 day millis() wraparound: pending retransmissions either stall
for the remainder of the wrap window, or all fire simultaneously at
the rollover boundary. Long-running router/infrastructure nodes do hit
this in practice.
The replacement (int32_t)(p.nextTxMsec - now) <= 0 is the standard
Arduino/embedded idiom for rollover-safe deadline checks and behaves
identically to the original for any non-wrap timing.
* Address Copilot review: use unsigned half-range for rollover-safe retransmit check
Review feedback from @Copilot on PR #10227: casting a uint32_t
subtraction to int32_t is implementation-defined in C++ when the
unsigned value exceeds INT32_MAX (even though it works on typical
two's-complement targets).
Switch to the fully well-defined unsigned half-range form:
nextTxMsec is in the past-or-equal iff (now - nextTxMsec) has not
wrapped past 2^31 ms. Future offsets < 2^31 ms wrap into the top
half and read as 'not yet'.
Same semantics as the signed-cast version on every two's-complement
platform we care about, but portable to any conforming C++ impl.
* Use monotonic time for airtime windows
* Document monotonic airtime windows
* Fix test_packet_signing sentinel that #10227's rollover fix inverts
test_C3_invalid_repeated_packet_cannot_ack_or_change_retry_state parked a
pending packet at nextTxMsec = UINT32_MAX to mean "never retransmit", then
asserted that a rejected repeated packet leaves the retry state untouched.
NextHopRouter::doRetransmissions() now tests whether a retransmit is due with
an unsigned half-range compare, (uint32_t)(now - nextTxMsec) < 0x80000000u,
so that retransmission timing survives the ~49.7 day millis() wrap. Under it
now - 0xFFFFFFFF == now + 1, a small positive delta, so UINT32_MAX reads as
~1ms in the past: the retransmit fires and rewrites nextTxMsec, and the test
failed with "Expected 4294967295 Was 6247".
Use a representable future time instead. Production is unaffected either way -
nextTxMsec is only ever written as millis() + d, and UINT32_MAX came from the
test harness alone - so the sentinel is what needs to go, not the comparison.
Special-casing UINT32_MAX in the retransmit path would keep a value that reads
as "expired" under any wrap-correct compare.
The value is held in a local because millis() advances across
runPipelineIngress(), so recomputing it at the assertion would compare against
a different number.
Reported upstream on meshtastic/firmware#10227, whose branch predates this test.
* Make Throttle time-injectable and add hasElapsed()
Throttle backs ~94 call sites, which makes it the highest-leverage place in
the tree to put the clock seam: reading Time::getMillis() instead of millis()
in its three call sites turns all of them into time-injectable code at once,
without touching any of them. The 32-bit millis() wrap is not otherwise
reachable from a native test.
The read is behaviour-preserving - Time::getMillis() returns millis() unless a
test injects a clock - and the full native suite passes with it live.
Also add hasElapsed(), the complement of isWithinTimespanMs(), because 51 of
the 94 call sites are spelled !isWithinTimespanMs and read poorly. Its
boundary is inclusive (>=) since isWithinTimespanMs uses <; both are
documented. It deliberately does not treat lastExecutionMs == 0 as "never
run": call sites pair that test with the interval check themselves, and
absorbing a sentinel into the one helper every module depends on is exactly
the value-overloading hazard being removed elsewhere.
Migrating the existing !isWithinTimespanMs sites is cosmetic and deliberately
left out of this commit.
test/test_throttle/ covers window semantics, both boundaries, the complement
identity, execute()'s first-run and throttled paths, and - the point of the
exercise - a window opened before the wrap closing correctly after it,
including at the 24h interval that is the longest in the tree.
* Stop disarmed deadline sentinels reaching the comparison
Two deadline variables encoded "inactive" as a magic value that only reads as
"never" because the comparison against it is a naive millis() compare. Under
any rollover-correct comparison both invert to "expired ~49 days ago", so they
have to be untangled before those comparisons can be fixed.
Power::reboot() set rebootAtMsec = -1 on platforms with no reboot
implementation, intending "never fire". Every reader already treats 0 as the
disarm value - powerCommandsCheck() tests `if (rebootAtMsec && ...)`, and
AdminModule writes 0 to cancel - so -1 was both wrong and unnecessary. Use 0.
Left as UINT32_MAX it would reboot-loop the moment the comparison is corrected.
ExternalNotificationModule's nag window compared against nagCycleCutoff, which
holds UINT32_MAX once stopped and 1 at boot. isNagging is the real armed flag,
so test it first and short-circuit: a disarmed cutoff can no longer reach the
arithmetic, while an idle module still takes the same sleep path that the
boot-time value of 1 was relying on.
Note this fixes the sentinel only. The comparison itself is still a naive
`nagCycleCutoff < millis()` and remains on the list to convert.
* Fix millis() rollover in every deadline and interval comparison
Roughly 20 sites compared against millis() directly - `millis() > deadline`,
`deadline < millis()`, `last + interval < millis()`. All of them break for
about 24 days after the 32-bit millis() wrap: depending on which side of the
wrap each value sits, the action either stalls for weeks or fires immediately
and repeatedly. The longest affected interval is the 12 hour NTP renewal, a
~50x margin against the wrap, so none of these needed the range - only the
correct comparison.
Add Throttle::deadlinePassed(deadlineMs) for sites that store an absolute
deadline they cannot re-express as "interval since an event". It uses the same
unsigned half-range test as NextHopRouter::doRetransmissions() rather than
introducing a competing signed-cast idiom, and unlike the signed cast it is
defined for every input. Sites that do store an event use the existing
isWithinTimespanMs / hasElapsed. Nothing gained new state.
Because both helpers read Time::getMillis(), every converted site is now
reachable from a native test that drives the clock across the wrap; the
comparison itself is covered directly in test/test_throttle/.
Sentinel handling is the reason this could not be a mechanical rewrite. The
disarm convention is not uniform: 0 means "inactive" for rebootAtMsec,
shutdownAtMsec, alertBannerUntil, fixHoldEnds, suppressUntilMs and
touchResumeBlockUntilMs; 0 means "due now" for ntp_renew, which is forced to 0
at link-up; UINT32_MAX means "inactive" for nagCycleCutoff; and
alertBannerUntil == 0 in isOverlayBannerShowing() means "show indefinitely".
Every inactive marker is arithmetically far in the past, so a correct
comparison fires on it - each site tests its sentinel before the arithmetic,
and keeps the meaning it had.
Two sites carried a second bug found on the way:
BME680Sensor tested (stateUpdateCounter * STATE_SAVE_PERIOD) < millis(). With
a 6 hour period and a uint16_t counter that product overflows uint32_t after
about 198 saves, independently of the millis() wrap. It now measures the
interval since the last save.
EInkDynamicDisplay had `if (previousRunMs > millis()) return;` as a millis()
overflow guard, which skipped rate limiting entirely for the whole post-wrap
period - the bug it meant to prevent. Every check below it already goes
through Throttle, so the guard is removed rather than fixed.
MotionSensor's calibration countdown is converted to a signed delta rather
than deadlinePassed, because it needs the remaining magnitude and not a
boolean; that matches the already-correct check in the same file.
* Remove getMillis64() and use Throttle for the NodeInfo reply window
getMillis64() had exactly one caller and no callers in tests. It also carried
obligations that made it the wrong shape for this firmware: a wrap accumulator
in mutable statics, which is not ISR-safe, and which must be polled at least
once every ~49.7 days or it silently misses a wrap and returns a time ~49 days
short.
Its one caller only wanted to know whether a 12 hour suppression window had
elapsed - which Throttle answers correctly across the wrap without any
accumulator. NodeInfoModule now stores Time::getMillis() in lastNodeInfoSeen
and tests the window with Throttle::isWithinTimespanMs, so the map holds
milliseconds rather than seconds derived from a 64-bit read.
USERPREFS_NODEINFO_REPLY_SUPPRESS_SECS is user-overridable and now feeds a
multiply by 1000, so a static_assert rejects any value too large to express in
milliseconds instead of letting it wrap.
clockSourceChanged goes too. It existed solely to rebase getMillis64()'s
accumulator when a test swapped clock sources, and it made the wrap untestable
through the injection API: setTestMillis() set the flag, so a wrap crossed by
two setTestMillis() calls was swallowed. With the accumulator gone the flag has
nothing to rebase, and the injection API is a plain settable clock.
The three getMillis64 tests are dropped as they no longer describe anything.
One test replaces them, pinning that advanceTestMillis() wraps past
0xFFFFFFFF rather than saturating, since the Throttle wrap tests rely on it.
Also fix eviction in pruneLastNodeInfoCache(): it picked the entry with the
smallest stored stamp, which is the wrong victim once some stamps sit on the
far side of the wrap. It now evicts the largest elapsed time.
* Add CI guard and docs rule against naive millis() comparisons
Fixing the existing sites does not stop the next one being added. The
millis-deadline-check job rejects millis() placed directly next to a comparison
operator, in either order, anywhere in src/. It lives in test_native.yml
alongside suite-count-check, which sets the precedent for a repo-hygiene guard
that CI enforces and bin/run-tests.sh does not.
The correct idioms all subtract before comparing, so none of them match the
pattern. Line comments are stripped first, so documentation is free to name the
broken form - as the guard's own comment and the coding conventions both do.
Writing the check before finishing the sweep turned out to be worth it: it
found roughly 14 sites that a by-hand audit of deadline variables had missed,
including two extra nagCycleCutoff compares, both boot-screen timeouts, and a
6 hour sensor save interval that was also overflowing a uint32_t multiply.
.github/millis-deadline-allowlist.txt covers the cases that are genuinely not
deadline tests. Both current entries are uptime thresholds - "has the device
been up N ms" - with no stored deadline and no event to measure from: a 30s
button holdoff against phantom shutdown from floating pins, and a 10s window
for the OEM boot logo. Each re-crosses its threshold once per wrap, which is
harmless for boot-holdoff logic and not worth new state to avoid. Entries are
keyed on file plus exact source text, without line numbers, so an edit above an
entry does not silently invalidate it.
Locally the guard reports 19 matches before the sweep and 2 after, both
allowlisted.
The Throttle bullet in the coding conventions is rewritten from "prefer
Throttle for rate limiting" to "never compare against millis() directly", lists
all four helpers with when to use which, names the CI guard, and documents the
sentinel hazard with the rebootAtMsec = -1 case that would have become a reboot
loop. Mirrored into AGENTS.md; CLAUDE.md gets a pointer row.
* Trim rollover comments to what the code needs
The comments added with the millis() rollover fixes carried too much of the
investigation that produced them: how many sites were found, which document
recorded them, what the old code used to do. That belongs in the commit history,
not in the source, and some of it was already stale - Power::reboot() still
described the check it disarms as "a naive millis() > deadline" when that
comparison had been fixed in the same series.
What stays is the non-obvious part at each site: which sentinel value the
variable overloads and what it means there, since that differs between call
sites and is what a correct comparison gets wrong. 0 means "not scheduled" for
rebootAtMsec, "renew now" for ntp_renew, and "show indefinitely" in
isOverlayBannerShowing().
Exposition is kept where it earns its place: the Throttle helpers, the uptime
clock's note on why there is no 64-bit variant, and the tests. The Throttle
docs lose only the site count and the "longest interval in the firmware"
statistic, both of which would age badly; the range trade-off between the two
forms is what a caller actually needs.
Comments only - no code changed, verified by diff.
* possible fixes
* Address review feedback on the rollover fixes
- BME680Sensor: checkpoint lastStateSaveMs after a successful write instead of
at the interval test. The first save (IAQ accuracy >= 2) left it at 0, timing
the next save from boot, and stamping before the write deferred the retry a
full period when the write failed. Reads Time::getMillis(), the same clock
Throttle compares against.
- Throttle: add deadlinePassedAt(now, deadline) for loops that snapshot the
clock once and test many deadlines; deadlinePassed() now delegates to it.
NextHopRouter::doRetransmissions() uses it, replacing the inline half-range
compare adopted from #10227 (nightjoker7) - same arithmetic, credited at the
call site - and takes its snapshot from Time::getMillis() so setNextTx()
deadlines and the due test cannot diverge under an injected test clock.
- test_native.yml: set -euo pipefail in the millis-deadline guard, matching the
sibling suite-count job. Without -e a partially failed scan could report "no
violations" from truncated output.
- test_packet_signing: build the not-due deadline from Time::getMillis() rather
than millis(), so the test and the router read one clock.
- test_throttle: cover deadlinePassedAt(), and correct a wrapped-value comment
(0xFFFFFF00 + 400 is 0x00000090, not 0x00000094).
Two review comments were declined: the AirTime mutex (every airTime-> caller
runs in the single cooperative loop, WebServerThread included) and the
MotionSensor 0-sentinel countdown (the calibration frame is only installed
while a window is open).
clod helped out here
* Correct the described failure window of a naive millis() compare
The comments and agent docs said a bare `millis() > deadline` "breaks for ~24
days after the wrap". That figure belongs to the fix, not the bug: it is the
half-range limit of deadlinePassed(), which reads deadlines more than 2^31 ms
ahead as already passed, and the range over which a UINT32_MAX sentinel reads
as passed.
The naive compare's actual failure is an inversion lasting only while the
deadline sits on the far side of the wrap, so it is bounded by the interval:
the action fires immediately and loses its wait, or blocks for about the wait
it should have performed - days for the nRF52 flash-corruption backoff,
one skipped cycle for a seconds-long retransmit timer.
Comments and docs only; the ~24.8 day statements that correctly describe
deadlinePassed()'s own range are left as they were.
clod helped out here
* Restore a monotonic uptime clock and consolidate the wrap counters
Time::getMillisMonotonic() is the getMillis64() shape - a 32-bit wrap
counter carried across reads - promoted to the shared timebase, with
Time::getUptimeSecs() as the derived whole-seconds view. This deliberately
reverses the earlier removal of getMillis64(), and the distinction matters:
removal was right for a lazily-read accumulator with one rare caller, where
a 49.7-day gap between reads silently swallowed a wrap. Here every read is
the poll and AirTime::runOnce() guarantees one per second; the missed-wrap
contract is pinned by a test rather than left as a footnote.
Three private wrap counters collapse into it:
- AirTime::syncNow() takes its seconds from Time::getUptimeSecs() and drops
its lastSyncMsec checkpoint; window rotation is unchanged.
- DeviceTelemetryModule loses refreshUptime()/uptimeWrapCount/uptimeLastMs;
uptime_seconds comes from Time::getUptimeSecs(), which also removes the
0.296s-per-wrap truncation of (0xFFFFFFFF / 1000) * wraps. Its two
interval checks move to Throttle::hasElapsed().
- HostMetricsModule's copies of those members were never read (its uptime
comes from /proc/uptime) - deleted.
Not ISR-safe (unguarded mutable carry): ISRs keep using getMillis(), which
stays a pure read. Audited: no interrupt-context file reads getTime(),
getValidTime(), or the new accessors.
test/native-suite-count 44 -> 45: the bump for test_uptime_clock was lost
in a branch history rewrite, leaving every later value off by one -
run-tests.sh reports AMBER and CI's suite-count-check fails on the current
push until this correction.
* Anchor the wall clock in monotonic milliseconds
getTime() computed elapsed-since-time-set as a 32-bit millis() delta, so a
node that took time once and stayed up past 49.7 days reported a wall clock
one full cycle in the past - and last_heard, rx_time, message and position
stamps all inherited it. The anchor is now the 64-bit monotonic count
(timeStartMsec -> timeStartMs64) and the elapsed term is computed in 64-bit,
so the wall clock is exact at any uptime.
All six anchor writers follow: the five hardware-RTC read branches and
perhapsSetRTC(), which keeps a truncated 32-bit copy of the same instant for
its Throttle-checked rate-limit stamps. The test seams anchor the same way.
Two native regression tests drive getTime() across the wrap through the
Time seam - one anchored before the wrap and read after it, one anchored
after a counted wrap - with the test epoch derived from BUILD_EPOCH so the
plausibility window cannot rot as the build date advances.
* Stamp the rx_time placeholder in monotonic uptime seconds
computeRxTimeStamp() stamped Time::getMillis() when the clock was untrusted,
and reconcilePendingRxTimes() back-calculated with a 32-bit millis() delta -
correct within one wrap, but a placeholder older than 49.7 days aliased to a
small elapsed value and reconciled to a plausible-but-wrong recent epoch:
the exact failure has_rx_time exists to prevent, reachable by an ordinary
unattended router whose phone connects two months in.
The placeholder is now Time::getUptimeSecs(). Both stamps come off the
monotonic counter, so the elapsed term is exact at any age and the aliasing
window is gone outright rather than widened. If elapsed somehow exceeds the
epoch itself, the packet stays un-dated (absent, never wrong) instead of
clamping to a pre-1970 value. Defence in depth: a placeholder that leaks
needs ~50 years of uptime to cross MIN_PLAUSIBLE_EPOCH, where milliseconds
took 18.3 days.
The stream-API reconciliation tests keep their scenarios with the placeholder
unit switched, and ScopedTimeFixture resets the monotonic carry so uptime
seconds are deterministic per case.
* Date nodes heard before the clock arrives, without polluting last_heard
A node first heard while the wall clock was untrusted got no last_heard at
all, and nothing backfilled it once time arrived - the phone showed "Last
heard: unknown" for a node it had just announced. The arrival instant now
waits in a RAM-only sidecar (NodeNum -> uptime seconds, 32 slots,
reuse-oldest - the RouteHealth shape) and is converted to a real epoch on
the clock-becoming-trusted transition, beside the existing rx_time
reconciliation. last_heard itself never holds anything but a real epoch or
0: it persists to flash and the warm tier, where an uptime-relative value
would be meaningless after reboot.
The sidecar's write sites are updateFrom()'s no-trusted-clock path (the
rx_time placeholder already carries the arrival instant, so this is a store,
not a second clock read) and addFromContact's anti-eviction stamps, which
previously wrote a bare getTime() - boot-relative seconds on a clockless
node, the exact value lastHeardIsWallClock() exists to catch. Eviction
ranking honours the stamps: heard-this-boot outranks every stored epoch,
ordered among themselves, so a stamped contact is not the first victim.
PhoneAPI re-reads last_heard at nodeinfo send time: a record prefetched
before the clock became trusted can carry 0 while the store has since been
backfilled, and re-reading at the pop makes handshake ordering (time-set vs
node-list download) irrelevant. Backfill never moves last_heard backwards
and skips the pathological elapsed-exceeds-epoch case. A node evicted to
the warm tier before time arrives is still absorbed with last_heard 0 -
same as before, bounded to the untrusted window.
* Update the agent docs for the monotonic timebase
The conventions bullet asserted there is deliberately no 64-bit millis; the
monotonic uptime clock restored for timestamps changes that contract. State
the split explicitly: Throttle for deadlines and intervals (no carry state),
Time::getMillisMonotonic()/getUptimeSecs() for timestamps, polled by
construction and not ISR-safe.
* Publish the monotonic wrap carry from a single writer
getMillisMonotonic() was a read-modify-write on two unguarded statics, and it
is reached off the main loop: the nRF52 Bluefruit task via
onFromRadioAuthorize() -> PhoneAPI::getFromRadio -> getValidTime(), and the
portduino civetweb workers via the same path. Two readers interleaving inside
the wrap window could each increment the carry, putting every uptime and
wall-clock reading 2^32 ms ahead for the rest of the boot - a permanent ~49.7
day jump in rx_time, last_heard and ClientNotification.time.
Readers no longer write. serviceMonotonic() publishes a snapshot behind a
seqlock and is the only writer; a reader adds its own unsigned elapsed time to
that snapshot, which is exact across the wrap, so it never inspects the
boundary and cannot miscount it. The main loop publishes every iteration, so
the once-per-49.7-days obligation now has the whole window of margin instead of
resting on an instruction-wide race.
AirTime was the guaranteed poller and is now a pure reader, so the two airtime
wrap tests step the clock the way loop() does. The test clock itself is atomic
so a suite can drive it from one thread while others read.
* Re-arm the GPS ephemeris hold when none is in force
The rollover sweep guarded the hold re-arm with `fixHoldEnds != 0 &&`, which
reads like the sentinel rule but inverts this site. The comparison it replaced,
`(fixHoldEnds + GPS_THREAD_INTERVAL) < millis()`, was always true when nothing
was armed - that was the point, since 0 means "not holding" and so is a reason
to arm. With the guard, a publish that cleared the hold without sleeping (the
`shouldPublish && !tooLong && !holdExpired` path, which does not call down())
left hasValidLocation set and prev_fixQual non-zero, so no disjunct held:
nothing re-armed, nothing published, and the receiver stayed powered at the
200ms poll until searchedTooLong() fired.
State the question positively instead. fixHoldInForce() is the only place the
sentinel is interpreted, and both of runOnce()'s decisions derive from it - the
asymmetry is now visible rather than implied, since arming does not require a
prior hold but expiring does. Its `!= 0` test is not redundant with the
arithmetic: deadlinePassed() is an unsigned half-range test, so past 2^31 ms of
uptime the sentinel reads as a deadline ~24.9 days in the future.
Kept beside its caller rather than in a header; the native test build compiles
GPS.cpp, so the suite declares the prototypes.
Also converts the getACK() wait to isWithinTimespanMs(start, interval): it has
both the start instant and the interval in hand, which gives the full 49.7-day
range instead of 24.8 days ahead, and takes its anchor from Time::getMillis()
so the wait is injectable.
* Date the NodeInfo reply window in uptime seconds
The 12h reply-suppression stamp regressed from wrap-immune 64-bit seconds to
raw 32-bit milliseconds, and pruneLastNodeInfoCache() evicts only by node count
and DB membership - never by age. A stable mesh under the node cap therefore
keeps every stamp indefinitely, and once uptime passes 49.7 days an old one
aliases back into the window: `now - stamp` computes as ~0 and a legitimate
NodeInfo request goes unanswered for up to 12h. It self-heals and repeats once
per wrap cycle.
Store Time::getUptimeSecs() instead, which does not wrap for 136 years, and
drop the millisecond conversion the previous shape needed. Entries past the
window are now evicted too: they can only ever decide "don't suppress".
N8-N11 cover the window from both sides, and N10 pins the regression - it needs
a full 2^32 ms of uptime to elapse, not merely a crossing of the boundary,
because that is when a millisecond stamp reads as "answered this instant".
tearDown() now restores the injected clock and C14's region and TX bucket. A
failing assertion aborts the test body, so restoring at the end of it leaked
that state into every later case.
* Update the agent docs for the single-writer clock and sentinel direction
Two rules the preceding three commits changed.
The monotonic clock is no longer maintained by whoever happens to read it:
serviceMonotonic() is the only writer, readers are pure, and calling it from
anywhere but the main loop reintroduces the double-count.
The sentinel guidance gained the half it was missing. It named UINT32_MAX as a
sentinel while prescribing an idiom that only covers 0, and it assumed the
sentinel always means "suppress" - at the GPS fix-hold site it meant "fire",
which is how that regression passed review looking like the rule.
* Name the fix-hold expiry predicate and arm it from the injected clock
holdJustExpired() gives the second reading of the fixHoldEnds sentinel a
name beside the first, so both are pinned by test/test_gps_fix_hold/ and
neither can be respelled at the call site. The old inline form could not
be tested: written as a literal, its guard folds at compile time and the
assertion asserts nothing.
The arm site used bare millis() while the evaluation reads the Throttle
clock; same value in production, but it kept that write out of reach of
Time::setTestMillis(). Remap a deadline that lands on 0, which would
otherwise read as no hold at all.
* Share the extend formula between the clock's reader and writer
getMillisMonotonic() and serviceMonotonic() carried byte-identical wrap
arithmetic. A one-sided edit to either would drift the published carry
from what readers report, so keep one copy.
* Trim the NodeInfo dedup comment to the house limit
* todo note for potential future imrpovments
* fix some simple deadlines
* Trim the hold-expiry test comment to the house limit
* Fix non-blocking uptime publication and pre-clock recency edges (#29)
* fix(time): avoid blocking monotonic readers
* test(time): make paused-publisher check deterministic
* fix(time): address review portability gaps
* Init the eviction sentinel to the newest possible recency
EvictionRecency{} is {0, false}, which evictionRecencyOlder() ranks as older than
every candidate: without the oldestIndex/oldestBoringIndex guards nothing would
ever be selected and a full node DB would stop evicting entirely.
Init to the genuine maximum instead, so the sentinel is correct on its own. The
index guards stay: two independent reasons the scan is right beats one.
* Keep the deadline-guard check name branch protection matches
The guard was widened to cover Time::getMillis() and unqualified getMillis(),
and renamed to suit. Upstream branch protection matches required checks by name,
so a rename means the old name never reports and merges block on a check that
will never arrive.
Widen the guard, keep the name; the descriptive text carries the broader scope.
* Correct native-suite-count to 47 after the develop merge
Upstream #11293 added test_nmea_wpl and took develop's count to 43; this branch
had independently reached 46. Merging develop resolved the counter textually,
keeping 46, while the directory set became the union of both sides at 47.
The suite-count CI gate fails on the mismatch, and it gates the native test jobs,
so the tests themselves were being skipped.
* test(uptime): make the wrap fall where the comment says it does
The concurrent-reader case started at 0xFFFFF000, leaving 0x1000 to the wrap, so
the 0x800 advance annotated "cross the wrap" fell short and the wrap actually
happened during the following 60s advance.
Start at 0xFFFFF800 instead, so the first advance lands exactly on the wrap while
the readers are running and the second is the ordinary time after it - the shape
both comments already described. Total elapsed is unchanged, so the closing
assertion still holds.
* Respond to human comments
* Did I ever tell you about the time I went to Shelbyville? I wore an onion on my belt, which was the style at the time.
* Convert the I2S nag deadline develop dragged in
The HAS_I2S_SPEAKER_NRF52 RTTTL block arrived from develop with a raw
nagCycleCutoff >= millis(), which the deadline guard rejects. Use the same
Throttle::deadlinePassed() form as the two sibling paths in this function.
* Arm the LittleFS format guard with a flag, not a zero timestamp
preFSBegin() runs in the first millisecond of boot, so millis() can legitimately
return 0 there. Both readers of last_format_ms treated 0 as "nothing formatted
this boot", which would skip the repeat-corruption escalation and let a dead
flash reformat-loop instead of reporting FLASH_CORRUPTION_UNRECOVERABLE.
* Note the single-thread contract on AirTime
* Note the AirTime locking TODO, and tighten the thread note
The two constant getters are not constrained, and getSilentMinutes() reads the
buckets without rotating them, so "the accessors mutate" was not accurate.
* trunk: ignore trufflehog false positives on millis-wrap test constants
test_throttle and test_uptime_clock pin dense clusters of hex boundary
constants (0xFFFFFF00u and neighbors) to exercise 32-bit millis()
rollover. trufflehog's Lob detector stitches nearby hex literals into
one candidate string, and the result happens to match a Lob API key
shape - not a secret, just test fixtures.
Same pattern already used for the gitleaks/nodedb-fixture false
positive in this file.
---------
Co-authored-by: nightjoker7 <mattdeering7@gmail.com>
Co-authored-by: Clive Blackledge <clive@ansible.org>
Co-authored-by: Benjamin Faershtein <119711889+RCGV1@users.noreply.github.com>
Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
|
||
|
Jonathan BennettandClaude Fable 5
|
5baad2e2a8 |
logging: compile out LOG_TRACE by default, demote chatty DEBUG lines, drop redundant logs (#11391)
* logging: gate LOG_TRACE behind MESHTASTIC_TRACE_LOGGING, drop redundant reclock logs LOG_TRACE now compiles out by default so trace-level diagnostics cost no flash; enable with -DMESHTASTIC_TRACE_LOGGING. Portduino keeps it on for the traceFilename packet-trace feature. Remove the 66 caller-side I2C reclock/restore log lines in the telemetry sensors: ReClockI2C::setClock/restoreClock already log both frequencies internally (now at trace level, since they fire every sensor read). Also unify near-duplicate literals (colon/case/punctuation variants) so linker string dedup applies, and drop an information-free bare 'done'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 * logging: demote chatty per-packet/per-poll DEBUG lines to trace level With LOG_TRACE compiled out by default, per-iteration chatter (packet bookkeeping, sensor poll values, e-ink refresh reasons, GPS pin states, UI runState traces) now costs no flash on device builds while remaining one -DMESHTASTIC_TRACE_LOGGING away. 108 lines demoted, 4 information- free lines removed; failure paths, drop reasons, and one-time init logs all stay at debug level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 * logging: address CodeRabbit review on trace-gate PR - GPS: pass serial-derived buffers as %s args, never as format strings (untrusted bytes could contain % directives) - 0x%08x for packet id / NodeNum per convention (Router, CannedMessage, NeighborInfo); unsigned casts for size_t args; %u for uint32_t delta - EInk: async full-refresh begin/complete back to DEBUG (rare state transitions); per-frame SKIPPED lines stay trace Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 * logging: gate trace on the flag's value, not its presence -DMESHTASTIC_TRACE_LOGGING=0 previously *enabled* trace logging because the gate tested definedness. The flag now defaults per-platform (portduino 1, else 0) and both backends test the value, so =0 disables, =1 or a bare -D enables. Also cast tx_after-millis() to uint32_t for %u (millis() is unsigned long on native). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 * logging: clang-format rewrap after specifier widening Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 * Even fewer bytes! * logging: keep compile-gated debug lines at debug level; fix native-suite-count Lines already inside default-off #ifdef blocks (GPS_DEBUG, DEBUG_LOOP_TIMING) cost no flash and should stay visible at debug level when their gate is enabled, rather than also requiring MESHTASTIC_TRACE_LOGGING. test/native-suite-count lags the two test_event_channel_* suites added by #11045 (develop's Native Suite Count check has the same mismatch); bump 46 -> 47. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 * gps: route GPS_DEBUG diagnostics through a LOG_DEBUG_GPS() macro (#11414) Replaces 27 log-only #ifdef GPS_DEBUG blocks across GPS.cpp, PositionModule, MeshService, and GPSStatus.h with a single-line LOG_DEBUG_GPS() call (src/gps/GPSLog.h, modeled on LOG_MIGRATION: value-gated, ((void)0) when off). Blocks containing declarations, control flow, hexDump, or nested conditionals keep an explicit '#if GPS_DEBUG' guard. RTC.cpp's per-reading raw time dumps and per-candidate rejection chatter fold under the same gate; quality transitions and boot-time seeding stay at debug. Also fixes the '// define GPS_DEBUG' missing-# typo in two variant headers and updates all seven commented examples to the value form ('#define GPS_DEBUG 1') required by the value-based gate. Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 Co-authored-by: Claude <noreply@anthropic.com> * gps: declare RTC gmtime result as pointer to const (cppcheck) With the setTime debug dump gated behind GPS_DEBUG, all remaining uses of t are reads; cppcheck (constVariablePointer) now flags it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBiZc9sfPrH1MZ2L3Fxgt1 --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Jonathan BennettandClaude Fable 5
|
af56a11f00 |
Replace native-suite-count file with dynamic test discovery (#11413)
* Derive the native suite count on the fly instead of registering it in a file test/native-suite-count was a manually-maintained register of the test_* directory count, reconciled against the actual directories by bin/run-tests.sh (as an AMBER verdict) and by a dedicated suite-count-check CI job. The reconciliation only ever guarded the file itself: the check that matters - suites that actually ran vs. the test_* directories on disk - already derives its expected count from a directory walk, so the file added a bookkeeping step to every suite addition/removal without adding signal. Remove the file and everything that existed to keep it honest: - bin/run-tests.sh: drop the canonical-count file read, the count-mismatch AMBER verdict, and the [canonical: x/y] suffix; the verdict lines already carry ran/expected from the directory walk. The shuffle seed suffix stays. - test_native.yml: delete the suite-count-check job and its needs: edges. - Docs (copilot-instructions.md, AGENTS.md, test/README.md) and the test-script comments now describe the count as derived from test/test_* at run time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APCEfNjd1X7ErDHEzT6Dqd * Add suite-shrinkage-check: fail a PR that silently loses a test_* suite With test/native-suite-count gone, nothing in CI noticed the suite set shrinking: platformio test discovers and runs whatever test_* directories exist, and bin/run-tests.sh derives its expected count from the same walk, so a suite directory lost in a bad rebase or an overzealous cleanup just means fewer suites run - every remaining check stays green. Restore that tripwire git-aware instead of file-based: on pull_request runs, compare the test_* directory list at the PR's merge base against the PR result. A vanished suite fails the job unless its name appears in the PR title, PR body, or a commit message in the PR's range - a deliberate removal satisfies that by stating what it removes; an accidental loss cannot. Other events skip: they have no natural base, and PRs are where accidents arrive. No job depends on this one (a skipped job would skip its dependents). Incidentally: test/ currently holds 47 test_* directories while the deleted count file said 46 - the manual register had already drifted, which is exactly the bookkeeping failure mode this replaces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APCEfNjd1X7ErDHEzT6Dqd * Re-pad the verdict table after shortening the AMBER row Shrinking the AMBER cell left the table's column padding inconsistent, which trunk (prettier + markdownlint MD060) rejects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APCEfNjd1X7ErDHEzT6Dqd --------- Co-authored-by: Claude <noreply@anthropic.com> |