mirror of
https://github.com/meshtastic/firmware.git
synced 2026-10-31 04:19:58 -04:00
develop
101
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
1e3a38fc8a |
Add native Portduino mDNS discovery with Avahi and Bonjour (#12032)
* Add native Portduino mDNS discovery via Avahi and Bonjour * Guard native discovery in Portduino WASM builds * Add linux dependencies in packaging, support "buildroot" Adds libavahi dependencies for Debian, RPM, Docker and CI. Moves the libavahi definition to the shared `build_flags` (which is only consumed on Linux targets) * Fix platformio formatting * fix(portduino): disable Avahi when daemon unavailable --------- Co-authored-by: Jake-B <jake-b@users.noreply.github.com> Co-authored-by: vidplace7 <vidplace7@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Austin
|
9015b2213f |
Update portduino to gnu++20 (#12069)
Upstream dependencies in the newest distros are beginning to require c++20. This PR bumps all portduino-based targets to c++20, removes the shared c++17 build_flag, and adds it to each variant family where it is needed. |
||
|
|
dee945cd6a |
CH341: fewer waits on the USB bus per radio command (#12013)
* CH341: fewer waits on the USB bus per radio command Every RadioLib command to a radio behind a CH341 waits on the USB bus about five times: the BUSY read before it, CS low, the command, CS high, and the BUSY read after, with a 1 us settle between the last two that usleep() turns into ~50 us of timer slack. - A BUSY read that follows the read that found BUSY low after a command is answered from it, once, within 1 ms and only if nothing else has been sent to the adapter in between: RadioLib's wait before a command then costs nothing after the wait that ended the last one. - Delays under 100 us spin on the steady clock instead of sleeping. - With a libch341 that has pinedio_transceive_select(), and where Lora.CS is D0, the CS levels travel with the SPI transfer and RadioLib's separate CS writes are dropped. Against an older libch341 this part compiles out. The HAL learns the radio's CS and BUSY pins from initLoRa(), since USBHal.h is included by PortduinoGlue.h ahead of portduino_config. Measured on an SX1262 over a CH341 (SHORT_TURBO, 6 v 6 runs of 300 s): the three-command CAD setup fell 1.01 -> 0.62 ms, CAD arm to air 6.60 -> 5.23 ms, and total packet loss 1.51 -> 1.13% (p=0.048), with collisions 4.83 -> 0.67 per run (p=0.002). Co-Authored-By: Jonathan Bennett <jbennett@incomsystems.biz> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01REkPVFh6kvG4AZJ5A8AtM9 * Bump libch341-spi-userspace to 027cde5 for pinedio_transceive_select() Picks up libch341#6 (1 ms interrupt poll, poll thread woken on detach) and #7 (pinedio_transceive_select()), so the packed-CS path in USBHal.h builds in rather than compiling out. Co-Authored-By: Jonathan Bennett <jbennett@incomsystems.biz> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01REkPVFh6kvG4AZJ5A8AtM9 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Jonathan BennettandClaude Opus 5
|
8c0abbd522 |
feat(portduino): BLE peripheral support via BlueZ for meshtasticd (Raspberry Pi) (#11396)
* feat(portduino): BLE peripheral support via BlueZ for meshtasticd on Linux Adds the standard Meshtastic BLE service (toRadio/fromRadio/fromNum/logRadio) to the Linux native target, so a Raspberry Pi running meshtasticd can be paired and used over BLE like any other Meshtastic device. Implementation: a new LinuxBluetooth backend registers a GATT application, LE advertisement and pairing agent with bluetoothd over the org.bluez D-Bus APIs, using sdbus-c++ (both the 1.x and 2.x major versions, via a small compat shim - Debian bookworm/Ubuntu 24.04 ship 1.x, trixie/Fedora ship 2.x). When the sdbus-c++ dev package is absent the whole backend compiles out via __has_include, the same optional-dependency idiom as the ulfius webserver. Threading follows the NimbleBluetooth model, simplified: the sdbus event loop runs its own thread, and all PhoneAPI calls happen on the main thread. Writes queue to the main loop; reads park the D-Bus reply and are completed from the main thread after queued writes, so write-then-read clients see their answer without any busy-waiting. Enablement is a double opt-in: a new `Bluetooth:` config.yaml section (Enabled, default false; AdapterId, default hci0) must turn BLE on for the host, and the regular device config bluetooth.enabled must be on. The config-check schema and fixtures cover the new section. Pairing honors config.bluetooth.mode: NO_PIN maps to a NoInputNoOutput just-works agent; RANDOM_PIN to DisplayOnly with the kernel-generated passkey shown on screen/log via the existing BluetoothStatus plumbing. FIXED_PIN falls back to random-passkey semantics with a warning - BlueZ does not support forcing a passkey. PIN modes enforce encrypt-authenticated-read/write on all mesh characteristics. Packaging: install a D-Bus system policy so the meshtasticd user may talk to org.bluez, add it to the bluetooth group, order the unit after bluetooth.service, and add libsdbus-c++-dev to debian/rpm/docker/CI deps. Verified in-container against a mock bluetoothd: registration flow, GATT tree enumeration, advertisement properties, and a full config download (ToRadio wantConfig -> 47 FromRadio packets) through the D-Bus bridge. Real-hardware pairing/notify testing on a Pi still pending. Known limitations (v1): meshtasticd must be restarted if bluetoothd restarts; FIXED_PIN degrades to a random passkey; getRssi() returns 0 (same as nRF52). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): BLE fixes from first real-hardware pass (Pi CM5 + RAK6421) Findings from testing PR #11396 on a Raspberry Pi CM5 (Pi OS trixie, BlueZ/sdbus-c++ 2.1 - the v2 compat path) with a RAK6421 HAT and an Android phone: - getMacAddr() leaked its HCI socket on every call and never closed it, and on failure returned without touching the caller's buffer - which getDeviceName() passed in uninitialized. Close the socket on all paths and cache the MAC after the first successful read; it cannot change at runtime and this now runs on every bluetoothd property read. - getDeviceName() zero-initializes its MAC buffer, and LinuxBluetooth snapshots the name once at setup() on the main thread: the advertisement's LocalName getter runs on the D-Bus event-loop thread and getDeviceName()'s static buffer is not thread-safe. - Restore NimBLE-style config-phase packet prefetch (depth 3). The initial port answered every FromRadio read with a D-Bus -> main-loop round trip, which made the config download noticeably slow; ReadValue now answers straight from the prefetch queue on the event-loop thread, with NimBLE's safety rules (never in STATE_SEND_PACKETS, writes always observed before reads, queue cleared on disconnect). - Set advertising MinInterval/MaxInterval to 20-100ms (BlueZ >= 5.71; older versions ignore the properties). btmon showed the kernel default of 1.28s otherwise, and Android's background-connect scan windows are sparse enough that tap-to-connect took 8-14s; 20ms is the same floor NimBLE uses on ESP32. Verified on hardware: scan, passkey pairing, connect, config download, reconnect after bond wipe. Also diagnosed (no code change): the node identity MAC comes from the RAK HAT EEPROM by design, so the BLE name suffix follows the HAT rather than the BT adapter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): address CodeRabbit review on BLE support - setBluetoothEnable: handle disable before the config gate, so a running BLE stack is always stoppable even after the device config turns Bluetooth off underneath it - getMacAddr: read the adapter configured as Bluetooth.AdapterId instead of hardcoding hci0, falling back to hci0 for unparseable names - systemd unit: Wants=bluetooth.service so bluetoothd is pulled up when present (After= only orders, it does not start it) - debian/rpm: Recommends: bluez as the runtime contract for BLE - dbus policy: document why the org.bluez rule is destination-wide rather than a per-interface allowlist Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): show the BLE pairing code on BaseUI screens onDisplayPasskey published the passkey to bluetoothStatus and triggered PowerFSM, but never called screen->startAlert(), so on BaseUI the code only ever reached the log. BluetoothStatus has no BaseUI consumer -- only InkHUD's PairingApplet and StatusLEDModule read it -- so a Pi driving a HUB75/OLED panel showed nothing while BlueZ sat waiting for the user to type a code they could not see. NimBLE and nRF52 draw it via startAlert(); this adds the missing half for Linux. The agent callbacks run on the sdbus event-loop thread while the screen is owned by the main thread, so the passkey is handed over as a pending flag and drawn from runOnce(), matching the existing disconnectCleanupPending pattern rather than reaching into the screen from the event loop. Dismissed on all four exits, so a stale code cannot stick on an always-on panel: Paired -> true (newly watched in PropertiesChanged, which previously only looked at Connected), agent Cancel, peer disconnect (moved out of the lastGone branch so a peer leaving mid-pairing clears the code even when another device is still connected), and doDeinit() -- applied inline there because runOnce() may never be scheduled again after teardown. Verified on a Pi 5 + BlueZ 5.66 in RANDOM_PIN mode: the code renders on a HUB75 panel and clears once the phone completes pairing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: install libsdbus-c++-dev for the native test build setup-native-test landed on develop while this branch was adding libsdbus-c++-dev to setup-native, so the new action's "full setup-native list" of C libraries is missing it. Without the package the test job builds with HAS_BLUETOOTH 0 and never compiles LinuxBluetooth.cpp. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): warn when a factory reset cannot clear BLE bonds factoryReset(eraseBleBonds) silently did nothing on Linux when the BLE backend was not running, so the reset reported success while the host's pairings stayed. Removing them needs a live connection to bluetoothd that a disabled backend never opened, so say so rather than imply they went. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): gate the factory-reset bond clear on an enabled backend setup() leaves linuxBluetooth allocated with its bus torn down when it throws, so a pointer check alone let factoryReset log "Clear bluetooth bonds" for a clear that clearBonds() then declined to perform. isEnabled() is only true after setup() completes, which routes that case to the warning that says the bonds were left alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * style: reformat under clang-format 20 #11909 moved trunk from clang-format 16 to 20, which spaces C-style casts differently and reindents the comment above the HAS_WIFI block. Both files are ones this branch already touches, and trunk's fmt linter grades whole files, so its check fails until they are reformatted. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0184v7MyCLuJHW2ebZ9r8NmQ * fix(portduino): three BLE config and lifecycle fixes from review Bluetooth config keys are now assigned individually rather than per section. loadConfig() runs once for every file in config.d, so reading an absent key as its default let a later file that named only one of them silently reset the other: `AdapterId: hci1` alone turned Bluetooth off, and `Enabled: true` alone dragged the adapter back to hci0. Only what a file actually states should override what an earlier one set. A backend that failed to come up is now retried. setup() can leave linuxBluetooth non-null but disabled - bluetoothd not ready, adapter missing, policy refusing - and every later enable then called resumeAdvertising(), which returns immediately while disabled. A transient failure at boot kept BLE off until the process restarted. doSetup() already opens with `if (enabled) return` and tears the bus down on every failure path, so calling it again is safe. Bluetooth.AdapterId is now checked for the hci<digits> form. LinuxBluetooth uses the value verbatim as the BlueZ object path while the MAC fallback reads only the leading hciN, so "hci1junk" looks plausible, yields a MAC, then finds no adapter and BLE never comes up. Covered by a new fixture and suite case, which is the kind of silent no-op that directory exists to catalogue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
f5940ab5dd |
Meshtasticd notifications (#10216)
* First attempt at sending message notifications on Linux
* Add files via upload
* Notifications: Use libnotify
Use libnotify for meshtasticd desktop notifications.
Add "HAS_LIBNOTIFY" macro to guard for builds where libnotify is expected.
This is not included in buildroot/openwrt builds. (no libnotify except when extra repos are added).
Install desktop icon in the correct location on Debian and Fedora packages.
Update dependencies in packaging and dockerfiles.
* Add libnotify to setup-native (GitHub Actions)
* Address review feedback on the meshtasticd notification path
- platformio.ini: only define HAS_LIBNOTIFY when pkg-config actually finds
libnotify. The probe previously ran unguarded and the macro was defined
unconditionally, so a native build without libnotify-dev both hard-failed at
config time and claimed the feature was available.
- Fix the sender lookup for develop's flattened NodeInfoLite: has_user/user.*
are gone, replaced by nodeInfoLiteHasUser() and direct long_name/short_name.
This is a silent semantic conflict - it merges cleanly but does not compile.
- Move the desktop notification off the packet path. notify_notification_show()
is a synchronous DBus round trip and meshtasticd's packet handling is
single-threaded, so a slow or wedged notification daemon could stall the
radio. handleReceived() now resolves the strings and queues them (bounded at
16); a dedicated worker owns every libnotify call.
- Stop retrying forever: a failed notify_init(), or three consecutive failed
shows, latches desktop notifications off instead of re-logging per message.
- NodeDB: the ARCH_PORTDUINO default-enable block was nested inside
#ifdef HAS_I2S, which Portduino never defines, so it never ran. Hoist it out.
* fix(portduino): close input-broker guard before the libnotify block
The libnotify implementation was inserted ahead of the #endif that closed
#if !MESHTASTIC_EXCLUDE_INPUTBROKER, so the file's trailing #endif closed
#if HAS_LIBNOTIFY instead and the input-broker conditional was never closed.
Every build target failed to preprocess:
src/modules/ExternalNotificationModule.cpp:630: error: unterminated #if
Close the guard immediately after handleInputEvent(), as develop does, so the
two conditionals stay independent and HAS_LIBNOTIFY still resolves when the
input broker is excluded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): back off desktop notifications instead of latching them off
notifyDisabled was doing two jobs: "libnotify looks unusable" and "the
destructor wants the worker to exit". Because the worker's only exit path was
also its only failure path, three failed shows turned notifications off for the
process lifetime with no way back. The packaged daemon is exactly that case:
bin/meshtasticd.service runs as User=meshtasticd and the rpm spec creates that
account with /sbin/nologin, so there is no session bus and every show() fails.
A stock install logged three warnings and then went silent forever, while
NodeDB now enables the module by default on Portduino.
Split the flag. notifyShutdown is destructor-only and remains the worker's one
exit; a retry window replaces the latch. After maxNotifyFailures the worker
arms a backoff (30s, doubling to a 15min cap), drops the queue rather than
holding stale popups, and keeps looping. The producer refuses to queue while
the window is open, so the retry is driven by the next message after it expires
rather than by a timer - no idle wakeups, no probe notifications. Any success
resets the backoff. notify_init() moved inside the loop so a retry can pick up
a session bus that was absent at startup.
The worker also no longer calls the LOG_ macros. RedirectablePrint formats into
a shared static buffer that nothing guards and every other writer to it is on
the main thread, so the worker now records the state change under the mutex it
already holds and reportNotifyStatus() emits it from portduinoNotify() and
runOnce(). Per-message failure spam becomes one line per transition: the reason
and retry interval when it goes down, and a line when it comes back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): sanitize mesh text before it reaches libnotify
The notification body was the raw decoded payload and the summary was a node
name, both attacker-controlled, and neither was checked before being handed to
libnotify. g_variant_new_string() rejects invalid UTF-8: a GLib CRITICAL and an
"[Invalid UTF-8]" body by default, and a hard abort under G_DEBUG=fatal-criticals,
so an unauthenticated mesh packet could terminate meshtasticd on any install
running with that flag. An embedded NUL separately truncated the body at the
first one, hiding the rest of the message.
Route both strings through sanitizedMeshText(), which replaces embedded NULs and
then applies the existing sanitizeUtf8() helper. TypeConversions already
sanitizes names on the way into NodeDB, but an abort is too sharp an edge to
leave resting on an invariant owned by another file.
Escape the body for Pango markup as well. Servers advertising "body-markup"
parse a markup subset there, so a message can inject formatting to dress itself
up as trusted UI, and where the server also advertises body-images or
body-hyperlinks it can inject tags that make the notification daemon fetch a
remote URL. The escaping is unconditional rather than gated on
notify_get_server_caps(): a caps query that fails, or goes stale across a daemon
restart, fails open, while on a server without body-markup the only cost is
entities rendering literally, and mesh text carries no markup worth preserving.
The summary is not escaped - the spec gives it no markup, so escaping it would
only ever show entities.
Also call notify_uninit() in the destructor, after the join: the worker owns
every libnotify call, so tearing down while it is still running would be a
use-after-uninit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): gate the notification default-on behind HAS_LIBNOTIFY
The default-enable block was gated on ARCH_PORTDUINO, so a native build where
pkg-config could not find libnotify still shipped external_notification enabled.
portduinoNotify() is not compiled into that build, and native defines
EXT_NOTIFICATION_MODULE_OUTPUT as 0 with setup() guarding the pin writes behind
output > 0, so nothing was driven - it only exposed a config surface that can
do nothing. Gate it on the same macro that governs the code it exists to feed.
HAS_LIBNOTIFY is a build flag for the whole env and is simply absent when the
probe fails, so this reads as 0 on every other target, matching how
ExternalNotificationModule.cpp already tests it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* fix(portduino): fail closed when markup escaping returns NULL
escapedNotificationBody() returned its input unescaped if g_markup_escape_text()
gave back NULL, which would hand libnotify the exact attacker-controlled string
the function exists to neutralize. The branch is unreachable for the input we
pass - already sanitized to valid UTF-8, and GLib documents no NULL return for
it - but an error path whose fallback is the unsafe action is the wrong shape
for a helper on this boundary. Drop the body instead.
Also note in the doc comment why running this on the caller's thread does not
weaken the rule that the worker owns every libnotify call: it is a pure GLib
string function that touches no libnotify or DBus state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EZstXBJtzRyBmUD1h2FLs
* style: format ExternalNotificationModule.h per the pinned clang-format
Trunk Check Runner failed on
|
||
|
renovate[bot]
|
fa87e7d29c |
chore(deps): update libch341-spi-userspace digest to d85aceb (#11907)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
renovate[bot]
|
d4166d0426 |
chore(deps): update libch341-spi-userspace digest to eaaef01 (#11895)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
renovate[bot]
|
dbbd42687b |
chore(deps): update meshtastic/crypto digest to 1c817c2 (#11879)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Benjamin FaershteinandManuel
|
2c595e935d |
feat(native): add macOS MUI simulator target (#11739)
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com> |
||
|
renovate[bot]
|
fa65b0797e |
chore(deps): update lovyangfx to v1.2.28 (#11615)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Andrew YongandBen Meadors
|
db84bdf3b4 |
Reduce ExternalNotificationModule flash usage (RTTTL + InputBroker) (#10989)
* Generalize RTTTL exclusion into MESHTASTIC_EXCLUDE_RTTTL ExternalNotificationModule already stubbed out RTTTL playback for STM32WL/portduino/ESP32C6 via a raw ARCH/CONFIG_IDF check, but the ringtone config plumbing around it (protobuf message, encode/decode tables, /prefs/ringtone.proto persistence, admin get/set-ringtone handlers) still compiled in even though it can never do anything on those platforms. Introduce MESHTASTIC_EXCLUDE_RTTTL and gate the dead ringtone plumbing behind it too. The flag is set in each architecture's *_base build_flags (stm32_base, esp32c6_base, portduino_base) rather than in the module itself - this matches how every other MESHTASTIC_EXCLUDE_* flag in the tree is set (e.g. stm32_base already sets ten of them directly, and esp32c6_base already excludes PAXCOUNTER for an analogous platform-can't-support-this reason), rather than introducing a new per-architecture C header pattern. Behavior is unchanged on all three platforms; overridable via -D like every other MESHTASTIC_EXCLUDE_* flag. Also guard the two HAS_I2S ringtone-playback call sites with !MESHTASTIC_EXCLUDE_RTTTL alongside HAS_I2S, since rtttlConfig itself is now only declared when RTTTL is not excluded. No current platform defines both HAS_I2S and MESHTASTIC_EXCLUDE_RTTTL simultaneously, so this has no effect today, but prevents a future HAS_I2S platform that also excludes RTTTL from failing to compile. Saves 368 bytes flash / 236 bytes RAM on wio-e5 with no loss to the GPIO on/off notification toggle itself, which does not depend on RTTTL. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * Skip unused InputBroker observer in ExternalNotificationModule The inputObserver CallbackObserver member was declared unconditionally, even though its only use site was already gated behind MESHTASTIC_EXCLUDE_INPUTBROKER (set for all of stm32 in stm32.ini). Because it's a non-trivial member, the compiler still generated its constructor/destructor as part of ExternalNotificationModule's own lifecycle even when InputBroker is compiled out entirely. Gate the member and its only consumer, handleInputEvent(), behind the same flag as their use site, and match the codebase's dominant !MESHTASTIC_EXCLUDE_X style (used ~330 times) rather than !defined(MESHTASTIC_EXCLUDE_X) (used ~20 times) while touching this flag's other call site. Saves an additional 288 bytes flash on wio-e5, no RAM change, no functional impact since InputBroker was already unused on this platform. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * fix(native-wasm): exclude RTTTL to unbreak build The browser node builds its own build_flags from arduino_base rather than inheriting portduino_base, so it did not pick up the MESHTASTIC_EXCLUDE_RTTTL flag added to portduino_base. With the inline ARCH_PORTDUINO stub in ExternalNotificationModule.h now replaced by that flag, native-wasm tried to include the unavailable NonBlockingRtttl.h. Set MESHTASTIC_EXCLUDE_RTTTL=1 directly in the native-wasm env alongside its other exclusion flags. Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Andrew Yong <me@ndoo.sg> --------- Signed-off-by: Andrew Yong <me@ndoo.sg> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Thomas Göttgens
|
9fbc176e91 |
Extend userPrefs coverage to the whole channel table and the missing config fields (#11624)
* Extend userPrefs coverage to the whole channel table and the missing config fields initDefaultChannel() handled only indices 0-2, so USERPREFS_CHANNELS_TO_WRITE above 3 produced live secondary channels carrying the public default PSK; it now covers all eight slots, with bin/platformio-custom.py completing every field of a configured index so indices 0-2 stay byte-identical. Adds USERPREFS_CHANNEL_<n>_IS_MUTED, USERPREFS_CONFIG_DEVICE_REBROADCAST_MODE, USERPREFS_CONFIG_DEVICE_NODE_INFO_BROADCAST_SECS, USERPREFS_CONFIG_LORA_CONFIG_OK_TO_MQTT, USERPREFS_CONFIG_SECURITY_IS_MANAGED and USERPREFS_CANNED_MESSAGES, applied after installRoleDefaults() and validated the way AdminModule validates a set-config. Adds test_userprefs_channels, covering the configured table under coverage-channel-table and the stock defaults under every other env. * Address review: hex channel count, PSK width assert, canned-message termination USERPREFS_CHANNELS_TO_WRITE now parses 0x-prefixed hex, matching the format userPrefs.jsonc documents, without int(x, 0)'s rejection of a leading-zero decimal such as "03". A static_assert rejects a USERPREFS_CHANNEL_<n>_PSK literal wider than psk.bytes, which memcpy would otherwise write over the fields after it. The USERPREFS_CANNED_MESSAGES copy keeps strncpy's zero-padding and terminates explicitly, rather than shortening the length, which would have left the last byte unwritten. |
||
|
renovate[bot]
|
4c640270f2 |
chore(deps): update lovyangfx to v1.2.27 (#11533)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Tom
|
bca7c0b480 |
Tom fiddles with the test suite - again (#11517)
* test: make every suite run its own binary, and fail the run when it does not PlatformIO links every native test program to the one $BUILD_DIR/$PROGNAME path and attributes Unity output by text alone, never checking that the source file a case came from belongs to the suite it thinks it ran. Both harnesses had been split into a build pass (--without-testing) and a run pass (--without-building), and for a non-embedded platform the run pass never relinks - so all 57 suites executed whichever suite was linked last, each reporting PASSED under its own name. Introduced for CI in |
||
|
Thomas Göttgens
|
34680833b8 |
fix(test): make the native-windows test suite build and run (#11482)
* fix(test): make the native-windows test suite build and run pio test -e native-windows failed every suite at the build stage. Five independent causes, all Windows-only: - TestUtil.cpp called lstat(), which MinGW-w64 does not provide. The state-checkpoint walk added in #11322 is fenced with ARCH_PORTDUINO, which native-windows also satisfies, so all 53 suites failed to compile. Route it through a stat() shim on _WIN32. - test_default, test_http_content_handler, test_meshpacket_serializer and test_serial define no setUp/tearDown and relied on the weak defaults PlatformIO emits in unity_config.c. GCC lowers a weak definition on PE-COFF to a weak external, leaving the symbol undefined, so it does not satisfy unity.c's reference and the link fails. Define them explicitly, as the other 49 suites already do. - test_mqtt included <arpa/inet.h>, absent on MinGW, for htonl(). Use winsock2.h there. - test_gps_update_scheduling uses TEST_ASSERT_DOUBLE_WITHIN. Unity omits double support unless UNITY_INCLUDE_DOUBLE is defined, so the assertion compiled to an unconditional failure. Define it for the env. - test_getfiles_rejects_overlong_path is excluded on _WIN32. Overrunning the 228-byte file_name needs at least 229 bytes below the portduino root, and that root is already ~34 bytes, so every qualifying path passes the 260-byte MAX_PATH: the nested mkdir() fails, the file is never created, and getFiles() has nothing to drop. No component layout satisfies both limits. Each of the seven suites that failed on Windows was verified individually after the change. test_fscommon_getfiles still fails in a full run, for a cause outside this change: rmDir() does not remove directories on Windows, so empty dirs left by an earlier run survive setUp() and make getFiles() report a depth truncation. That is a pre-existing FSCommon bug, reported separately. No Linux or macOS behaviour changes: every guard is _WIN32-only except UNITY_INCLUDE_DOUBLE, which is scoped to env:native-windows. * fix(test): define UNITY_INCLUDE_DOUBLE for every native env The flag was scoped to env:native-windows, but the gap is not Windows-specific. Verified on Debian with gcc against the Linux env's own Unity 2.6.1 and PlatformIO's generated native unity_config: UNITY_INCLUDE_DOUBLE : NOT defined UNITY_EXCLUDE_DOUBLE : defined test_double_within:FAIL: Unity Double Precision Disabled UNITY_INCLUDE_DOUBLE appears nowhere in the repo, the ini files, the workflow, or PlatformIO's unity runner, which adds only UNITY_INCLUDE_CONFIG_H. So TEST_ASSERT_DOUBLE_* is an always-failing stub on Linux and macOS too, not only on Windows. Moved to portduino_base.build_flags_common, which every native env resolves: native, native-tft, native-fb, native-tft-debug, coverage, coverage-event-policy, native-macos, native-windows and native-wasm. This does change Linux and macOS: TEST_ASSERT_DOUBLE_* becomes a real comparison instead of a stub. test_gps_update_scheduling is the only suite using those macros and its arithmetic is integer-based and bit-identical across platforms, so it should pass wherever it runs. Note it currently reports PASSED on CI in 0.03s while emitting no Unity output at all, so those assertions appear never to execute there; that is tracked separately and is not addressed here. |
||
|
|
546b9d9e40 |
Block coordinate traffic on configured event channels (#11045)
* Block coordinate traffic on configured event channels Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Suppress event coordinates in reliable relay paths Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Reject blocked phone coordinates before rate limiting Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Prevent event coordinates from reaching MQTT Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Add event coordinate policy preference Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Test event coordinate policy in native CI Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Make event policy test tolerate a full NodeDB Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Test Router event coordinate enforcement Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Test PhoneAPI event coordinate retry handling Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Test reliable event coordinate suppression Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Test MQTT event coordinate suppression Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * Run event policy behavioral suites in native CI Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> * tests: address CodeRabbit review feedback - test_event_channel_phone_api: complete the setUp/tearDown save-restore pair. GlobalState now carries cryptLock and myNodeInfo; setUp() nulls cryptLock before constructing MockRouter (Router's ctor asserts it is unset), and tearDown() restores both so the suite leaves no global mutated. Not reachable today - the globals start null in this binary - but the pair was asymmetric. - Replace the strcpy calls this branch added on Channel.settings.name (char[12]) with the bounded form the rest of the test tree already uses, strncpy(dst, src, sizeof(dst) - 1). Covers the flagged site in test_nexthop_routing plus the six equivalents in test_event_channel_phone_api, test_mqtt and test_position_precision, which trip the same ast-grep dangerous-buffer-functions-cpp rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tom
|
de6b23190a |
Test suite rebuild (#11322)
* docs(nodedb): make the native node cap unambiguous The native node cap was stated in four places that disagreed, and the disagreement already caused a wrong diagnosis: a saturated 200-node database looked arithmetically impossible because the cap had been read as 248, computed from a header that does not apply on this platform. The real value is 198. On portduino MAX_NUM_NODES is not a compile-time constant at all - the variant defines it as `portduino_config.MaxNodes`, resolved at runtime, default 200 and settable per host with `General: MaxNodes`. variant.h is reached before mesh-pb-constants.h, so that header's ARCH_PORTDUINO branch never fires and its plausible-looking 250 is dead code. - #error-guard the dead branch rather than leave a wrong number where people grep. The guard found a real defect: seven translation units reach mesh-pb-constants.h without configuration.h (SerialConsole.cpp, StreamAPI.cpp, PacketAPI.cpp, ServerAPI.cpp, PiWebServer.cpp, ServiceEnvelope.cpp, MeshtasticOTA.cpp, and test/TestUtil.cpp), so each was compiling with a different MAX_NUM_NODES - and therefore a different PACKETHISTORY_MAX - than the rest of the build. Each now includes configuration.h first. It cannot be included from mesh-pb-constants.h itself: that reaches SerialConsole.h through DebugConfiguration.h and closes a cycle. - Name the bare 250 in getMaxNodesAllocatedSize() NODEDB_MIGRATION_LOAD_CEILING. It is a decode allowance for files written by larger-cap firmware, not a cap, and it read like one. - Fix docs/node_info_stores.md, which named the wrong source and a "10-250" range that is wrong for native, and the copilot-instructions tunables line that said "portduino 250". * test(harness): give each suite its own scratch HOME and report leftovers Native suites shared one directory. Every suite that constructs a NodeDB loads and saves ~/.portduino/default/prefs/ - nodes.proto, config.proto, channels.proto, module.proto, device.proto, warm.dat, transmit_history.dat - and nothing cleared it, so state leaked suite -> suite within a run and run -> every run after it. A test run could also rewrite a real meshtasticd node database on the same machine. Per-run isolation does not fix this: the leak is generated inside a single run, so the boundary has to be per suite. bin/pio-test-isolate.sh runs each suite in its own scratch $HOME, registered as test_testing_command for env:native and env:coverage so a bare `pio test` and CI get the same boundary, not just bin/run-tests.sh. It runs the binary unchanged and exits with its exit code, so PlatformIO's pass/fail is untouched. Overriding HOME here rather than around `pio` also sidesteps the blocker that a bare HOME= breaks pio's own ~/.platformio/penv/bin/pio lookup. Leftovers are reported as a second axis, PASS/FAIL x CLEAN/DIRTY, because an unintended write has no matching assertion by definition - nobody writes TEST_ASSERT for a save they do not know is happening. The harness asserts it from outside, so it applies to every suite without the author opting in. - Only the *set of changed paths* is asserted, never contents. Hashes answer the boolean "did this change?" and nothing more; content baselines over protobuf bytes would churn on every NodeInfoLite field added, which is how snapshot suites become noise. - Deliberate writes are declared in test/state-manifest.tsv - one central file, suite / flags / mandatory reason. run-tests.sh prints the opt-out count on every run. - Granularity follows the state flag, so the two ship together: per-test by default (TestUtil redefines RUN_TEST to checkpoint after each test, naming the exact test that dirtied things), suite boundary for state=per-suite, where carrying state across test cases is the declared behaviour. - A declared write that does NOT happen is reported as MISSING, not folded into DIRTY. It catches silently broken persistence; a warning for now, since some are conditional. - Graded AMBER, not RED. With isolation in place DIRTY means "undeclared", not "dangerous", and a check that lands red on day one gets switched off. Guard the guard, both halves: state_assert_empty() refuses to run a suite against a sandbox that is not empty (otherwise the after-diff measures against the wrong baseline and reports CLEAN while meaning nothing), and bin/test-state-check.sh drives the real wrapper with fixtures asserting CLEAN / CLEAN / DIRTY / MISSING plus both directions of the empty assertion. A checker that silently matches everything would otherwise pass forever. --write-manifest proposes entries for a human to paste and justify; it never applies them, and neither does CI. * test(harness): stop reporting Unity's exit code as a signal A native suite ends in exit(UNITY_END()), and UNITY_END() returns the failure count. PlatformIO's native runner reads that non-zero exit code as a POSIX signal number, so four failures print "Program received signal SIGILL", five print "SIGTRAP", and the suite is classified [ERRORED] rather than [FAILED]. There is no crash. The signal name tracks the failure count and nothing else - it moved SIGILL -> SIGTRAP when a diagnostic probe added a fifth failure - and it cost hours of hunting a memory bug that did not exist, on an env (native) that carries no sanitizer at all. It also explains the phantom extra test case in the totals: the runner adds a synthetic entry for the signal it thinks it saw. run-tests.sh now says so inline whenever a signal line appears, and the three agent-facing docs say it too. * test(admin): isolate NodeDB and globals per test setUp() did `if (!nodeDB) nodeDB = new NodeDB();` and never deleted it, so 83 of the 85 tests shared one never-reset database and never restored config, owner, devicestate or channelFile. The fixture that does restore them was opt-in and armed by exactly two tests. The setUp comment claiming the rest "set their own config/region state and are unaffected" was not true - the admin handlers under test write all four globals. Route every test through the fixture instead: setUp saves the globals and installs a fresh NodeDB, tearDown restores and deletes it. The two tests that armed it themselves no longer need to. All 85 pass, so nothing was silently relying on the shared state. It costs about 7% of the suite's runtime (a NodeDB construction is a loadFromDisk plus, with a region set, key generation) - worth paying to write the phase 3 tests against a clean fixture rather than 83 tests' residue. Also cap the per-test attribution in the run summary at five entries; the full list stays in the suite's sandbox. * test(fs): cover the bounded file-manifest walk getFiles() runs on every phone sync via STATE_SEND_FILEMANIFEST, and nothing asserted any of its bounding behaviour. It does execute unasserted from test_stream_api's handshakes, but the cap, the depth limit, the wasLimited paths, overlong-path rejection and capacity release were all unguarded. Eight tests, all describing what the code does today: today's code is already correct here, since #10778 landed the by-reference collectFiles(), the 64-entry cap, the strlcpy bounds and the swap-idiom release. They pass on arrival, which is the point - this is the baseline a later change has to leave alone. Two things they do not cover, and cannot: - Moving reserve() outside the __cpp_exceptions guard. Exceptions are on natively, so the #else branch is not compiled. The suite's job there is to prove that change alters nothing observable. - The file.name() null guard. No in-tree backend returns null; the guard is defensive. The manifest-release test pins the swap idiom rather than calling PhoneAPI's releaseFilesManifest(), which is file-local. It asserts capacity() == 0, not just size() == 0 - a size-only check passes on clear(), which is the bug #7924 shipped. Suite count 43 -> 44, recounted against the directories rather than copied. * test(admin): assert node-DB metadata saves skip the radio reload set_favorite_node, set_ignored_node and toggle_muted_node each persist a NodeInfoLite bit and nothing else. MeshService::reloadConfig() gates its region re-derivation and configChanged notification on saveWhat & (SEGMENT_CONFIG | SEGMENT_CHANNELS), so a SEGMENT_NODEDATABASE-only save already skips the live radio reconfigure. Pure characterization - all three pass on develop. Worth pinning because that reconfigure is the path implicated in the WisMesh Tag favourite-node crash, and develop asserts nothing about it: widening the saveWhat mask or reordering the check would currently go unnoticed. Ported from the config-save series along with ConfigChangedCounter (an Observer<void *> counting configChanged notifications, the only externally visible signal that the reload branch was taken) and TEST_NODE_NUM. They join the existing suite, so no suite-count change. * refactor(menu): extract the mute toggle into a named function The node menu's mute action was inline in a banner-callback lambda, and that lambda only ever runs via screen->showOverlayBanner() - which is why nothing in MenuHandler.cpp was reachable from a test. Lift the `selected == Mute` branch into menuHandler::toggleNodeMuted(uint32_t) and call it from the lambda. Behaviour-neutral by construction: same statements, same order, same bare saveToDisk(). The null check moves into the function, so the call site no longer needs its own lookup. Verified by the native build and suite; the byte-identical-image check on a headroom-constrained nRF52 board was not run locally - CI's firmware-size comment covers it. Three tests come with it, all describing today's behaviour: - the bit flips both ways and no configChanged fires (develop never calls reloadConfig on this path); - an unknown node is a no-op rather than a write; - and the segment mask. Flipping one NodeInfoLite bit currently rewrites all five segments via bare saveToDisk(). That is asserted deliberately, with the comment naming it as characterization of a known defect: a pending fix narrows it to SEGMENT_NODEDATABASE, and when it lands this assertion is expected to change, which makes the improvement visible in the diff instead of silent. saveToDisk() is not virtual, so the mask is observed through its effect - remove the five prefs files, toggle, and see which reappear. * docs(test): make every suite count a pointer to the canonical one test/native-suite-count is the registered total and is machine-checked against test/test_* on every full run and by the suite-count-check CI job. Every other statement of the count is a copy that drifts: copilot-instructions said 12, AGENTS.md said 19, and the real number is 44. Replace both literals with a pointer to the file, say explicitly that no document should state the count as a literal, and reframe the two suite listings as descriptions rather than inventories - they carry per-suite information the count does not, so they stay, but nothing should infer completeness from their length. Register the new FS suite in both. * test(harness): randomise suite order, reproducibly Landed last, deliberately. Randomising an order-dependent suite set does not find bugs so much as convert a silent pass into intermittent red, and the first instinct is to revert the randomisation rather than fix the coupling. Phases 1-2 removed the coupling; this keeps it removed. Both runners previously hid order dependence behind a fixed order that happened to differ between them, and neither order was chosen: CI's area rules put admin first, PlatformIO's local discovery is reverse alphabetical and put it last. CI was green by accident. - bin/run-tests.sh --shuffle / --seed <n>. The seed defaults to HEAD's short SHA: one order per commit, so a red is replayable and attributable to the diff instead of flaky, while the project keeps exploring orders. Printed at the start and carried into the RESULT line, so a verdict is replayable from that line alone; the full order is printed on failure, because for an order-dependent failure the order is the diagnostic. - The shuffle is a Fisher-Yates over a MINSTD generator rather than awk's rand(), whose sequence differs between gawk and mawk. A seed that does not reproduce the same order on another machine is not a seed. - Shuffling needs one `pio test -f <suite>` invocation per suite - PlatformIO orders by its own os.walk() over test/ and filters only select - which measures at about 4.7s per suite of extra startup. - CI shuffles its area order, seeded from GITHUB_SHA and printed with the command to replay it locally. Intra-area order stays PlatformIO's; controlling it there would mean per-suite invocations, which is a cost worth deciding separately. Also records the 16 measured entries in test/state-manifest.tsv, each with its reason, taken from a full run's --write-manifest output rather than guessed. * test(default): cover the region-throttle interval overload getConfiguredOrDefaultMsScaled(configured, default, nodes, TrafficType) is the overload every telemetry and position module actually calls, and nothing referenced TrafficType anywhere under test/. All four of its behaviours were unguarded: the no-region guard, the throttle <= 1 short-circuit, the multiply, and the 64-bit overflow clamp. The throttles are real, not hypothetical - EU_866 carries PROFILE_LITE, which sets both positionThrottle and telemetryThrottle to 10, so a change here moves broadcast spacing in that region by an order of magnitude. Each test pins numOnlineNodes at the congestion threshold and uses ROUTER, which never congestion-scales, so the coefficient is 1 and the throttle is the only variable. The overflow case needs a base above INT32_MAX/10, hence three days rather than one. * ci(test): keep pull-request suite order fixed, seed the rest Shuffling the area order on every run - including pull_request - would turn a contributor's PR red for an ordering they did not choose, which is how a randomisation gets reverted instead of the coupling being fixed. That is the exact dynamic the ordering work was sequenced last to avoid, and the previous commit walked straight into it. - pull_request keeps the fixed declared area order. - push and schedule shuffle, seeded from the commit SHA: deterministic per commit, printed, attributable, and never blocking someone else's PR. - A suite_order_seed input on workflow_call and workflow_dispatch overrides both, so a specific failing order can be replayed anywhere, including on a PR. The run log prints which mode it took, the resulting order, and the local command to replay it. * ci(test): satisfy CKV_GHA_7 and yamllint on the seed input The seed is reachable through workflow_call, which callers can pass programmatically. The workflow_dispatch copy tripped checkov's "workflow_dispatch inputs MUST be empty" rule, and suppressing it was not worth it: replaying a specific order is a local operation, and the run log already prints the exact bin/run-tests.sh command to do it. * style(menu): apply the node-ID format convention RadioInterface.cpp documents the rule: 0x%08x in logs, !%08x in user-facing display. MenuHandler held every remaining exception - seven logs printing bare %08X, and two display labels doing the same. Repo-wide there are now no bare %08X node IDs left in log calls. * ci(test): pass workflow inputs through env, not shell interpolation suite_order_seed and github.event_name were spliced into the run: script as ${{ }} text, so a value carrying shell metacharacters would execute as code on the runner rather than being read as data. semgrep (run-shell-injection) and zizmor (template-injection) both flag it. Both now arrive as environment variables and are read as "$VAR". * refactor(test): share the seeded shuffle between the harness and CI bin/run-tests.sh and test_native.yml each carried a byte-identical copy of the MINSTD Fisher-Yates awk. The workflow prints "replay locally: ./bin/run-tests.sh --shuffle --seed $seed" after a shuffled CI run, and that instruction is only true while the two agree - drift would be announced by a replay quietly reproducing a different order than the one that failed. Extract shuffle_suites() to bin/lib/shuffle.sh and source it from both. Permutations verified identical across seeds before and after the move. * fix(test): correct the shared-state MISSING check and summary join Three defects in the new harness: state_classify() matched declarations two different ways - state_path_declared() for "undeclared", a hand-rolled regex for "missing". Interpolating an entry into an ERE also let a metacharacter in a manifest name match a file that is not the declared one. Both directions now go through the one helper. `paste -sd'; '` does not join with "; ": with -s, paste cycles through a multi-character delimiter one character per join, so paths rendered as "a;b c;d e". Replaced with an awk join. test-state-check.sh ran on after a failed cd instead of stopping (SC2164). ./bin/test-state-check.sh: 6/6 fixtures pass, MISSING included. * fix(portduino): bound General.MaxNodes MaxNodes was validated only for <= 0. Any positive value, including a typo'd or pasted-in one, propagates to MAX_NUM_NODES and scales both the node DB and the nodes.proto decode ceiling - failing at boot with no obvious cause. The ceiling is a sanity bound, not a capability limit; raise it if a host genuinely needs more. * docs(nodedb): reconcile the capacity tables The property matrix omitted the ESP32-S3 100-node flash tier that the platform table above it lists, and neither mentioned that the WASM build overrides MaxNodes to 80 in wasm_config_apply(). * fix(nodedb): make mesh-pb-constants.h self-sufficient on portduino The ARCH_PORTDUINO #error assumed it was unreachable in a normal build. It is not: the vendored device-ui sources include this header without configuration.h, which broke both native-tft docker builds. Include configuration.h here instead, ahead of every compile-time default - variant.h overrides MAX_RX_TOPHONE as well as MAX_NUM_NODES, so placing it lower in the file just moves the divergence to a redefinition. The #error stays as a backstop for the case where that include genuinely stops providing the cap. Verified with the native env's own flags: a TU including only this header now compiles, normal-order use of both macros compiles, and NodeDB.cpp compiles. * fix(portduino): raise the MaxNodes ceiling to 16000 Marked artificial: nothing in the node DB fails at 16001. 16000 sits just under the 16384 (128 x 128) population where HopScalingModule saturates its sampling denominator and starts dropping nodes, so a host inside the bound still gets meaningful hop recommendations. * lint(trunk): advise on node IDs logged as bare %08x RadioInterface.cpp documents the convention - 0x%08x in logs, !%08x in display - but nothing enforced it, which is how the MenuHandler cluster drifted. 22 call sites in PacketHistory, NodeInfoModule and PositionModule are still off it. A trunk linter rather than a CI grep job, because trunk checks changed files: new violations get flagged without a 22-site cleanup landing in an unrelated PR. Modelled on the existing too-many-defined definition. Scoped to values it can tell are IDs - an ID-shaped argument (->num, .from, getNodeNum) or message text naming one. A 32-bit hex that is not an ID is out of scope, so the CRC32 logs in ethOTA.cpp are correctly ignored. Emits "note", trunk's only non-blocking level: "warning" and "info" both exit non-zero and would gate CI, which is not what a log-format nit deserves. The pre-existing sites are line-scoped in the allowlist, so a new bad call in those same files is still caught. * lint(trunk): stop exempting the known node-id-format sites The seeded allowlist made the rule green by declaring the backlog acceptable. Empty it instead, so the 22 pre-existing sites are reported and get cleaned up by whoever next edits those files. Costs nothing to do: the rule emits "note", so these are non-blocking either way. The allowlist stays for its real purpose - a value the linter misreads as an ID. * style: log node and packet IDs as 0x%08x Clears the 22 sites the node-id-format linter reports, so the rule starts from zero rather than from a backlog nobody can see - trunk suppresses pre-existing findings by default, so left alone these would not have surfaced on edit the way an empty allowlist implies. Format strings only; no argument or control flow changes. The !%08x user-facing display forms are deliberately untouched - that is the other half of the same convention. * test(harness): build once up front, so suite timings mean something run-tests.sh fused build and run in a single pio invocation, so whichever suite PlatformIO's directory walk reached first absorbed the entire src compile and reported it as its own duration. On a real run that made a 0.03s suite report 13m21s, and hid the build cost from every other number in the summary. Do what .github/workflows/test_native.yml already does: one --without-testing build pass, then run with --without-building. Measured on a full 44-suite run - the build is now a single reported figure and 968 test cases execute in 1.9s, with no suite above 0.084s. Build output goes to its own log rather than $LOG: the outcome regexes match "error:" and "[ERRORED]", so a compiler diagnostic sharing that file would read as a test failure. Both red paths now keep the log they quote from. $LOG and the build log are mktemps the EXIT trap removes, so the three grepped lines were previously all anyone ever saw - and the cause is usually further up than the first [FAILED]. * test(harness): keep the run log on every red path bin/pio-test-isolate.sh already keeps a failing or DIRTY suite's sandbox and log under .pio/test-state/<suite>/. What was missing is the cross-suite view: $LOG is a mktemp the EXIT trap deletes, so run-tests.sh quoted three grepped lines from a file that no longer existed by the time anyone looked. Preserve it as .pio/build/<env>/test-failure.log from both red paths - including "no success summary found", which said "see log" while preserving nothing, and which is exactly the case where the build died before any suite ran and so left no per-suite sandbox either. Cleared at the start of every run, so a green run cannot leave a red one's log lying around looking current. * fix(test): report the real failure count on a shuffled red A shuffled run is one `pio test` invocation per suite, all appending to the same log, so the log carries one PlatformIO "N test cases:" summary per suite. verdict_red() took `tail -1`, which reports whatever the LAST suite did: a failure in suite 3 printed a "0 failed" summary from suite 44 directly under "RED - failures detected:". Sum the summaries instead. A single summary line - every unshuffled run - is passed through verbatim, so the familiar output is byte-identical. The patterns are passed to the awk helper as strings rather than /regex/ literals: awk evaluates a regex literal in argument position as `$0 ~ /re/`, so the callee would receive 0 or 1 and silently sum garbage. * fix(test): do not emit an empty suite name for an empty shuffle `printf '%s\n' "$@"` with no arguments still writes one empty line, and both callers read shuffle_suites through mapfile, so an empty suite list arrived as a single suite named "". Return before the printf when there is nothing to shuffle. * test(harness): state and enforce the Linux host requirement The native harness is a Linux tool: bash 4+ (mapfile), GNU coreutils and GNU find (-printf, md5sum, -executable). Most of that predates this branch - mapfile and both find predicates are already on develop - but none of it was written down, so the requirement was there to be discovered rather than read. Refuse to start on a non-Linux uname instead of degrading. On a BSD userland this would not fail cleanly: it would mis-hash the sandbox and mis-read the suite list, and still print a verdict. A state check that silently measures the wrong thing is worse than one that declines to run. Carrying a per-host fallback was the alternative, and it buys a second code path that nothing in CI exercises. bin/test-native-docker.sh already exists for macOS and non-Linux hosts, and the native-macos PlatformIO env is a build target for meshtasticd, not a test host - the isolation wrapper is registered for env:native and env:coverage only. Documented in the script header, test/README.md, and both agent docs. * fix(test): terminate every suite with exit(UNITY_END()) Two sites across two suites ended on a bare UNITY_END(). That ends the reporting, not the suite: setup() returns, the runtime goes on calling loop(), and the process runs forever. PlatformIO does not notice - it reports a suite from its Unity output, not from process exit - so the suite passes, the run goes green, and the binary stays resident. Thirteen of them had accumulated on one dev box, the oldest 19 hours old. The costs are quiet by construction: - the per-suite sandbox is deleted underneath a live process, so its CLEAN/DIRTY verdict describes what the suite had written when the harness stopped looking, not what it left behind; - .gcda coverage and LeakSanitizer's report both flush from atexit handlers, so a suite that never exits contributes no coverage and gets no leak check; - each survivor pins its own deleted 94 MB binary, which du cannot see. One of the two is the #else of an architecture guard, which is the easiest one to get wrong - it looks like there is nothing to clean up. test_mqtt has a correct exit(UNITY_END()) in its live branch, so a "does this file call exit() anywhere" check passes the file whole. test_serial had two more. develop's serial-config validation rework restructured that suite - the architecture guard is gone and both remaining branches now exit correctly - so this commit no longer has anything to change there; bin/lint-unity-exit.sh, added later on this branch, is what keeps it that way. test/README.md gets a section on it, since the skeleton showing the right shape had not stopped this happening. * test(harness): detect and reap suites that outlive their run A suite that never exits was invisible: PlatformIO reports a suite from its Unity output, so the run stayed green while the binary kept running. Two checks, because they fail differently. Runtime, in bin/pio-test-isolate.sh: the sandbox $HOME is mktemp-unique per suite, so any process still holding it is a survivor of that suite. Matching on the environment rather than a remembered PID identifies one whatever its parentage - a fork, a grandchild, a process already reparented to init - none of which a $! comparison catches. Reaped before the after-fingerprint is taken, so that fingerprint measures a tree nobody is still writing to, and so a run cannot leave processes accumulating on the host. Recorded as a sixth summary column and graded AMBER: the tests did pass, but the CLEAN verdict and the coverage were measured under a false assumption. Author-time, as bin/lint-unity-exit.sh, wired into trunk at "note" like node-id-format: every UNITY_END() must be wrapped in exit(). The rule is per occurrence, and that is the point - a file-level "calls exit() somewhere" check passes test_serial and test_mqtt, which have a correct one in their live branch and a bare one in the #else. Running it over the tree turned up test_mqtt, which the file-level pass had missed. It allows `int rc = UNITY_END(); ...; exit(rc)`, used by test_packet_signing to restore globals between the summary and the exit. That is where the rule gives ground: capturing and never exiting would leak and is not flagged. Flagging a correct idiom would push someone to "fix" working code. bin/test-state-check.sh gains a survivor fixture, asserting the wrapper both reports and reaps - a detector that only reports leaves the host accumulating processes, which is half the harm. 8/8. * fix(lint): make the unity-exit scanner statement-aware The rule judged one physical line at a time, which reports two kinds of correct code as bare: /* a comment that happens to mention UNITY_END() */ <- interior lines were never stripped exit( UNITY_END()); <- exit( and the macro never met On a probe of both, two of three findings were wrong. This is a note-level rule whose whole job is advice, and bin/lint-node-id-format.sh already says why that matters: a false positive costs more than a miss. One that cries wolf gets ignored, and the real finding goes with it. Carry /* ... */ state across lines and accumulate logical statements before testing, with a 12-line cap so one unclosed call cannot swallow the rest of the file - the same structure lint-node-id-format.sh uses, so the two custom linters in bin/ work alike rather than each having its own idea. Verified both directions: the develop-era sources still produce the same four findings, the fixed tree produces none, and a probe covering block-comment interiors, wrapped exit(), line comments, return UNITY_END() and capture-then- exit reports only the genuinely bare calls - including a complete block comment followed by real bare code on the same line, which the state machine has to keep live. Reported by CodeRabbit on #11322. * fix(lint): tokenise instead of pattern-matching, and self-test it Second round of review findings on the same scanner, all confirmed by direct test before changing anything. Six defects, one root cause: layered regexes cannot tokenise C++. False positives (correct code reported): - UNITY_END() inside a string literal read as code False negatives (real leaks missed): - a string containing "/*" opened comment state and swallowed later lines - greedy .* removed everything between two block comments on one line, taking a bare call with it - myexit(UNITY_END()) matched the exit() exemption as a substring - x == UNITY_END() and total += UNITY_END() matched the assignment exemption Replaced with a character-level scan carrying comment state, and token-bounded exemptions: exit must be a whole identifier, and the capture form must be a plain `=`. Raw string literals are still not modelled - there are none under test/, and delimiter tracking for a case that does not occur would be untested code guarding untested code, so it is documented rather than guessed at. Also drops the `return UNITY_END()` exemption. It only terminates from main(), there is no main() under test/, and from a helper it just returns a count. bin/test-lint-unity-exit.sh pins all fifteen cases, every false positive and false negative found in review among them. The rule has been wrong twice in a way that looked fine by inspection; it needed a self-test more than it needed another careful reading. Two further findings in the same review: - bin/run-tests.sh dropped PASSTHRU in shuffled mode, so `--shuffle -vvv` built verbosely and then ran quietly. The shuffled loop now forwards EXTRA_ARGS, which is PASSTHRU minus the -f pair it supplies per suite. - bin/run-tests.sh did not guard `cd "$ROOT_DIR"`. And one that did not reproduce: the survivor fixture's glob does find the pid file (verified with the lookup instrumented - the earlier failure was an artifact of running the script from /tmp, where SCRIPT_DIR cannot resolve). The assertion was still weak, because an empty pid took the "not running" branch and passed vacuously. It now fails if the pid was never recorded, and finds the file by search rather than assuming a directory depth. Reported by CodeRabbit on #11322. * fix(lint): report each UNITY_END occurrence at its own location The self-test only asked "did the linter say anything", so it could not have caught a wrong line, a wrong column, or a missing second finding. Fixtures now assert the exact diagnostics as line:col, and the first run of that assertion found two real problems. The caret pointed at the wrong occurrence. For `exit(UNITY_END()); UNITY_END();` the verdict was right but the column was 17 - the wrapped call - because the scanner stripped terminating forms out of the whole statement and then reported the first occurrence it had seen. Two bare calls on one line reported once. Judged per occurrence now, by looking back through whitespace at what wraps it, so both the count and the caret are right. That also needed a position map from strip_noncode(): removing a comment or collapsing a literal shifts every later column, and counting occurrences in the raw line does not recover it either - TEST_MESSAGE("... UNITY_END() ..."); UNITY_END(); has two occurrences in the raw text and one in the code. Four of the expected columns I wrote by hand were also wrong, off by one. The linter was right in every case; the assertions were not. They are computed from the fixture text now rather than pasted from output, because a baseline accepted from the tool it is testing asserts nothing. 17 fixtures, including the two-on-one-line case from review and its mirror. Reported by CodeRabbit on #11322. |
||
|
Thomas GöttgensandAustin
|
ecd59e3120 |
Package meshtasticd for Windows as an MSI (#11289)
* Package meshtasticd for Windows as an MSI Adds a --service flag connecting meshtasticd to the Service Control Manager, a WiX MSI installing it as an auto-start LocalSystem service with config in %ProgramData%\Meshtastic, and a CI step attaching the MSI to releases. * Address review comments Bind workflow expressions to env vars in run: bodies, and build the service status per call with an atomic checkpoint. * Fix service stop state and CI lint Latch the stop under a mutex so a startup report cannot walk the state back. Ignore the new workflows in semgrep and checkov, as main_matrix already is. * Drop the checkov ignore for the winget workflow Resolve the newest release inside the job instead of taking workflow_dispatch inputs, so CKV_GHA_7 no longer fires and checkov stays active on the file. * Carry the MSI architecture into the winget manifest Parse it from the asset name instead of defaulting to x64, and fail on a multi-arch release rather than validating one at random. * Restore release/.gitignore * Leave the main matrix alone Release attachment moves to the matrix rework in #11151. The MSI is still built and uploaded as a CI artifact. --------- Co-authored-by: Austin <vidplace7@gmail.com> |
||
|
Thomas Göttgens
|
047c4e9feb |
add LR 2021 to portduino, and allow Framebuffer devices to rotate the screen from config (#11252)
* add LR 2021 to portduino, and allow Framebuffer devices to rotate the screen from config. Requires https://github.com/meshtastic/device-ui/pull/355 and supersedes https://github.com/meshtastic/firmware/pull/10567 and https://github.com/meshtastic/firmware/pull/11138 Many thanks to the original authors https://github.com/a-li3n and https://github.com/jessm33 * Build LR2021Interface.cpp in the wasm env initLoRa() constructs LR2021Interface for Lora.Module: lr2021, so excluding the file from the native-wasm source filter left the constructor undefined at link time. LR20x0Interface.cpp stays excluded; it is template-only and comes in via the InterfacesTemplates.cpp amalgamation. Also replace the non-UTF-8 degree signs in the framebuffer rotation comment and correct the rfswitch alias cleanup comment. * Trim comments * Report setenv failure for the framebuffer rotation |
||
|
Tom
|
2024bb8384 |
Arrival time fix perhaps (#11274)
* Add explicit presence for MeshPacket.rx_time (arrival time) rx_time is now proto3 optional with a has_rx_time presence bit, matching the rx_rssi treatment. A node with no GPS and no phone connected yet has no time source at all, so a bare 0 was indistinguishable from a genuine 1970-01-01 reading; downstream consumers (replay packets, JSON serialization) now check has_rx_time instead of the value. * Dedupe rx_time stamping into a shared helper; trim a debug log string Extract the repeated haveTime/rx_time/has_rx_time stamp logic (5 call sites across Router.cpp, MeshBeaconModule.cpp, MeshService.cpp) into Router::computeRxTimeStamp()/stampRxTime(). Also shorten the new RTC.cpp LOG_DEBUG string. Saves 48 bytes of flash on rak4631 (measured), no behavior change. * Fix has_rx_rssi presence carried unconditionally through StoreForward replay preparePayload() set has_rx_rssi = true unconditionally on replay, regardless of whether the packet's rx_rssi at store time was a genuine measurement (e.g. MQTT-relayed packets carry no real RSSI). Store the presence bit alongside rx_rssi in PacketHistoryStruct and restore it on replay instead. Flagged by Copilot on #11271 (same root cause the has_rx_time explicit presence work fixes) but never addressed before that PR merged. * Trim comment blocks to the repo's 1-2 line guideline .github/copilot-instructions.md:338 caps code comments at 1-2 lines; several blocks added across the rx_time explicit-presence work ran well past that. Also consolidates Time.cpp's file-level doc comment into Time.h, where the rest of the Time:: API contract already lives. No behavior change. * Add rx_time explicit-presence test coverage - test_meshpacket_serializer: has_rx_time=false fixture plus tests asserting JsonSerialize/JsonSerializeEncrypted emit 0 rather than leaking the millis() placeholder, alongside the has_rx_time=true baseline. - test_stream_api: two tests driving a real PhoneAPI handshake (want_config_id through STATE_SEND_PACKETS) that simulate a phone time-giving transaction arriving before vs. after a queued packet is drained - covering both the reconciled and the ships-with-placeholder-absent paths of MeshService::reconcilePendingRxTimes(). * Fix three correctness issues flagged in review - Time.h: drop the reserved-identifier include guard (_MT_TIME_H); pragma once already covers it, matching convention elsewhere (e.g. RTC.h). - Time.cpp: rebase getMillis64()'s wrap accumulator when the test seam swaps clock sources, so a real<->injected clock jump isn't miscounted as a genuine 32-bit wrap. - NodeInfoModule: the 12h reply-suppression window is a local dedup duration, not a wall-clock reading - switch it to Time::getMillis64() so RTC-quality jumps and replayed packets' stale rx_time can't perturb it. - StoreForwardModule: has_rx_time was derived from *current* RTC quality at replay time rather than stored at capture time, so a history entry saved while time-blind could be misreported as a valid epoch once the clock later improved. Persist the presence bit in PacketHistoryStruct instead. * tryfix CI * post review fixes * more test fixes |
||
|
Manuel
|
5f51bb1c1e |
MUI: Portduino curl maps (#11214)
* add libcurl, fix missing SDL2 * device-ui commit reference PR353 * add curl dependencies to CI build * switch to gnutls curl version * Update device-ui library dependency URL * Update device-ui dependency to new version * Fix formatting in Dockerfile package installation * Update device-ui library dependency URL |
||
|
renovate[bot]
|
36fdbddd57 |
Update LovyanGFX to v1.2.26 (#11157)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Jonathan Bennett
|
bd71ad283c |
Remove BASEUI_HAS_GAMES flag from portduino.ini (#11113)
Removed BASEUI_HAS_GAMES flag from build configuration. |
||
|
Thomas Göttgens
|
a808e992a1 | Add native Windows build of meshtasticd (#11031) | ||
|
Austin
|
a4ea55f919 |
Fix renovate comments (#11065)
Fix renovate comments for: - Crypto - Adafruit_nRF52_Arduino - adafruit/Adafruit-MLX90614-Library - meshtastic/Fusion |
||
|
renovate[bot]
|
b56b35fcbb |
chore(deps): update platform-native digest to 86c62ed (#11054)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
|
8d1fbbf55f |
Snake! (#10936)
* Snake! * Add spiLock to snake score saving * Check fixes * More careful locking * WIP: Big Display Node * Update src/graphics/HUB75Display.cpp Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Add HUB75 Native * Add Tetris game module with core logic and UI integration - Implement TetrisGame class for game logic, including piece movement, rotation, and line clearing. - Create TetrisModule class to manage game state, input handling, and rendering on OLED display. - Introduce high score tracking with persistence and optional mesh broadcasting. - Define UI states for title, playing, paused, game over, and high scores. - Implement input handling for game controls and state transitions. - Add rendering functions for the game board, high scores, and title screen. * feat(snake): add snake graphics and update display logic in SnakeModule * Prompt for Initials for Tetris, too * refactor games module * Games refactor * hub75 native double buffer * Games tuning * Make joystick repeat events on held button * Add clouds and colors * Fix breakout and more color * difficulty tuning * trunk * refactor game announcements, etc * Scale chirpy gravity as game speed increases * Portduino, check for hub75 display before reading in hub75 options * Final(?) games tuning * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Properly ignore input when games screen not shown * Fail gracefully when HUB75 is selected but not supported. --------- Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Ixitxachitl <kramerfm@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
renovate[bot]
|
5b1c1a9e04 |
Update LovyanGFX to v1.2.25 (#10968)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Ben Meadors
|
6c7ee8afc7 |
Add MemAudit: per-subsystem heap accounting in the boot log (#10900)
* Add MemAudit: per-subsystem heap accounting in the boot log The 2.8.0 nRF52840 heap-exhaustion field reports had to be diagnosed by hand, reconstructing each subsystem's heap footprint from source and build flags one report at a time. This makes every future report self-diagnosing from the serial log: a tiny fixed-size registry (src/memory/MemAudit.*) that big long-lived allocations report into, printed as one line at the end of setup() and alongside the periodic "Heap free:" log: MemAudit[boot]: tmm=2500 warm=4000 pkthist=5824 nodedb=13440 msgstore=2200 pktpool(live)=3270 total=31234 Instrumented: NodeDB hot vector (nodedb) + satellite maps (satmaps, rb-tree overhead estimated), WarmNodeStore (warm), PacketHistory records and hash index (pkthist), TrafficManagement caches (tmm/tmm_ni), MessageStore text pool (msgstore), TFT line/repaint buffers (display), and live in-flight packets (pktpool(live)) via an optional audit tag on the packet pool allocator - the one hot path, counted with a relaxed 32-bit atomic add (single instructions on Cortex-M, no locks). Cost: 128 B RAM for the 16-tag table, well under 1 KB flash on rak4631. MESHTASTIC_MEM_AUDIT=0 compiles it out to inline no-op stubs (call sites need no ifdefs); STM32WL, the tightest flash target, defaults off. New native suite test_mem_audit covers add/set/snapshot arithmetic, tag reuse (pointer and cross-TU strcmp fallback), null/unknown tags, and table-full behavior; test/native-suite-count bumped to 28. * native-wasm: add src/memory/ to the curated source filter The wasm env denies all sources and adds an explicit file list; MemAudit callers (main, MeshService, NodeDB, PacketHistory) are in that list but src/memory/MemAudit.cpp was not, so wasm-ld failed on undefined memaudit:: symbols. |
||
|
Ben Meadors
|
0e84c1a827 |
Harden XEdDSA unsigned-packet policy and add coverage (#10858)
Audit of the XEdDSA packet-signing implementation (#10478) surfaced several issues in when unsigned packets are accepted on receive or emitted on send. This fixes them and adds regression coverage. - Unicast NodeInfo exchange no longer breaks against signer nodes: the NodeInfoModule downgrade drop is gated to broadcasts, since senders never sign unicast (want_response replies, directed exchanges). - Replace the payload-size sign heuristic with an exact encoded-size gate (signedDataFits) and mirror it on the receive side, removing a dead band where 167-168 B broadcasts were signed then failed TOO_LARGE. - Extract the receive policy into checkXeddsaReceivePolicy() and apply it to plaintext-MQTT decoded downlink, which previously skipped signature verification and downgrade protection entirely. - Reject signatures whose length is neither 0 nor 64 as malformed, so a crafted partial signature can't inflate the size estimate and dodge the unsigned-downgrade drop. - Hold cryptLock on the MQTT verify path (shared Ed25519 key cache). - Clear any client-preset signature on packets we originate, on all builds. - Randomized (hedged) signing per the Signal XEdDSA spec: bump the meshtastic/Crypto pin to the build where XEdDSA::sign mixes 32 bytes of caller randomness into the nonce as Z (meshtastic/Crypto#3), and seed those bytes in xeddsa_sign from HardwareRNG (checked, with a seeded-CSPRNG fallback). test_crypto pins that repeated signs differ and both verify. Adds test coverage: test_packet_signing groups A-E (receive matrix, send policy, NodeInfo backstop, encoding invariants, decoded-ingress policy), test_mqtt end-to-end downlink cases, and a test_crypto randomization check. |
||
|
Tom
|
3becaf2d95 | emdashes begone (#10847) | ||
|
Ben Meadors
|
23bb55d965 |
chore(deps): forward-port LovyanGFX + audio-driver renovate bumps from master (#10826)
Two more renovate updates landed on master (2.7) after #10824; reconcile develop to their values (value reconcile, consistent with #10824): - LovyanGFX 1.2.21 -> 1.2.24 (#10783) -- 18 lovyan03 custom.pio pins - pschatzmann arduino-audio-driver v0.2.1 -> v0.3.0 (#10770) -- tlora-pager, tracksenger Left untouched (deliberate develop-only pins; master's renovate didn't bump them): - lovyan03/LovyanGFX@1.2.19 (single variant pin) - mverch67/LovyanGFX fork (seeed-sensecap-indicator variant) |
||
|
Ben MeadorsandAustin
|
0baf8b1db6 |
Portduino WASM hello world (still experimental) (#10793)
* Add ARCH_PORTDUINO_WASM build: meshtasticd in WebAssembly over WebUSB
Compile the full portduino firmware to WebAssembly (Emscripten) so a real node runs in a browser tab or headless Node, driving a LoRa radio over WebUSB through a CH341 — the desktop Ch341Hal path with its libusb backend swapped for a WebUSB one. The native/desktop portduino build is unchanged.
New build env under src/platform/portduino/wasm/ (excluded from the native build_src_filter): WebUSB libpinedio backend, config/FS/region/MAC/PhoneAPI glue, wasm setup/loop, JS WebUSB runtime, and build stubs. bin/build-portduino-wasm.sh runs a standalone cached emcc build to build/wasm/meshnode.{mjs,wasm}.
Six firmware sources gain #ifdef ARCH_PORTDUINO_WASM guards (single-threaded cooperative emscripten_sleep loop, continuous RX, US region default, std RNG, no-popen exec); none affect non-wasm builds.
* PortDuino WASM: CI build job, cross-platform libdeps, configurable adapter
bin/build-portduino-wasm.sh: auto-detect native-macos (macOS) vs native (Linux/CI) libdeps with a NATIVE_ENV override, and use the SAME env's Crypto with an XEdDSA-present guard. Drops the heltec-v3/Crypto borrow — the meshtastic/Crypto pin already ships XEdDSA; the native libdeps cache was just stale. No env pin change.
Add .github/workflows/build_portduino_wasm.yml: build the ARCH_PORTDUINO_WASM target in CI (ubuntu + emsdk, native libdeps) so it can't silently bit-rot; asserts build/wasm/meshnode.{mjs,wasm}.
src/platform/portduino/wasm: add wasm_set_lora_* setters so the JS host can configure any CH341 LoRa adapter (module, USB ids, DIO/TCXO, SPI speed, pins); wasm_config_apply falls back to the MeshToad defaults when unset.
* WASM: build as a first-class [env:wasm] via platform-wasm (retire emcc script)
Replace the standalone emcc build (bin/build-portduino-wasm.sh) with a normal
PlatformIO env, `pio run -e wasm`, using the new meshtastic/platform-wasm
platform (emcc/em++ + Asyncify, the WASM sibling of platform-native). The
portduino WebAssembly node is now built the same way as every other target.
- variants/native/portduino/platformio.ini: add [env:wasm] (platform pinned to
platform-wasm, board wasm). Translates the script's source set into a curated
build_src_filter, the ~30 EXCLUDE_* defines, and the lib set; defines
ARCH_PORTDUINO_WASM in-repo so correctness doesn't hinge on the platform's
board.json.
- extra_scripts/wasm_link_flags.py: the firmware-specific emcc *link* settings
(EXPORT_NAME, EXPORTED_RUNTIME_METHODS, EXPORTED_FUNCTIONS, ASYNCIFY_IMPORTS).
PlatformIO feeds build_flags to compile only, so these must ride LINKFLAGS;
without it the WebUSB Asyncify seam and the JS host's runtime methods are
dropped (the _wasm_* exports survive only via EMSCRIPTEN_KEEPALIVE).
- PortduinoGlue.{h,cpp}: guard the yaml-cpp dependency out of the WASM build
(#ifndef ARCH_PORTDUINO_WASM around the include, emit_yaml/loadConfig/
readGPIOFromYaml). The browser node configures via the wasm_set_lora_* setters
and dead-strips the YAML path; this drops the host yaml-cpp build dependency
entirely. Native is unchanged (guards are inert there).
- portduino_glue_wasm.cpp / portduino_main_wasm.cpp: repair EM_ASM JS that a
formatter had mangled (!== -> != =, regex split) in the prior landing; the
emcc link succeeds regardless, so CI now runs `node --check meshnode.mjs`.
- .github/workflows/build_portduino_wasm.yml: build via `pio run -e wasm`
(artifacts under .pio/build/wasm/), trigger on the shared inputs the env
inherits (root platformio.ini, bin/platformio-*.py).
- NodeDB.cpp: drop the dead ARCH_PORTDUINO_WASM region-default branch (region
now defaults the same as native).
- Crypto renovate pins: add the missing gitBranch so they track upstream.
Output: .pio/build/wasm/meshnode.{mjs,wasm} (ES module, factory createMeshNode).
Verified: pio run -e wasm (against the published platform archive), node --check,
module instantiates in Node with all exports; native-macos + Docker native unit
tests (450/450) still pass.
* Fix name on the Piggystick
* wasm: pin platform-wasm at the GPL-3.0-relicensed commit
platform-wasm's LICENSE was always GPLv3 (matching this firmware), but its
platform.json/README still declared Apache-2.0 (mis-copied from platform-native).
That's fixed upstream in b83fa5b; bump the [env:wasm] pin to it. Build output is
unchanged (license metadata only). Verified: pio run -e wasm against b83fa5b.
* wasm: make reboot() actually restart the node (was a no-op)
In wasm the reboot path is live (main.cpp -> Power::powerCommandsCheck ->
Power::reboot), but Power::reboot's ARCH_PORTDUINO arm tore down SPI/Wire/Serial
and then called the no-op ::reboot() stub — leaving the node running with a dead
radio until the tab was manually reloaded. Triggers include an admin/phone
reboot, factory reset, the "reconfigure failed" path, and the 60 s stuck-TX
hardware watchdog (RadioLibInterface).
- Power::reboot(): add an ARCH_PORTDUINO_WASM arm (before ARCH_PORTDUINO, since
the wasm build defines both) that skips the host teardown and just calls
::reboot(). notifyReboot already let modules persist.
- ::reboot() (glue): hand off to the JS host — browser reloads the tab (NodeDB
state survives via IDBFS, same identity returns); headless calls Module.onReboot
if provided, else logs. Loose !=/== so clang-format doesn't mangle the EM_ASM JS.
- README: document the reboot handoff + the Module.onReboot hook.
Verified: pio run -e wasm + node --check (EM_ASM intact); native-macos unaffected.
* wasm: rename env to native-wasm and run it in the main CI matrix
Rename [env:wasm] -> [env:native-wasm] for consistency with the portduino
native family (native, native-macos, native-tft). The build dir follows to
.pio/build/native-wasm/ (artifact is still meshnode.{mjs,wasm}); the PIOENV
guard in extra_scripts/wasm_link_flags.py, the README, and the companion wrapper
move with it. The board stays `wasm`.
Also wire the build into normal CI: build_portduino_wasm.yml becomes a reusable
workflow (workflow_call) invoked as the `build-wasm` job of main_matrix.yml, so
the WebAssembly node is built like every other platform instead of on a separate
path trigger.
* native-wasm: auto-locate the Emscripten SDK (pre-build script)
`pio run -e native-wasm` failed with "emcc not found" whenever it was invoked
from a shell that hadn't sourced emsdk_env.sh — a VS Code task, an IDE build
button, a bare terminal. Add a pre: extra script that probes the usual emsdk
locations ($EMSDK_ENV, $EMSDK, ~/emsdk, ./.emsdk, the sibling companion
checkout), sources emsdk_env.sh, and imports the resulting environment so the
platform builder and emcc see PATH/EMSDK/EM_CONFIG. No-op when emcc is already
reachable (CI), silent when no SDK is found (the platform emits its own error).
* wasm: address PR review feedback
- js/bridge.js: import CH341 from "./ch341.js" (sibling in this layout), not
"../src/ch341.js" which doesn't resolve here.
- js/ch341.js: a zero-length transferIn while MISO bytes are still outstanding
now throws instead of breaking out with a partially-filled buffer — silent SPI
corruption becomes a loud error, matching the comment above it.
- libpinedio_webusb.c: webusb_set_auto_cs honors the AUTO_CS option (? 1 : 0)
instead of the always-on ? 1 : 1. Runtime behavior is unchanged — Ch341Hal sets
AUTO_CS=0 right after pinedio_init (RadioLib drives the active-low NSS); the
option just isn't set yet at init, so this now correctly defaults off.
- SX126xInterface.cpp: the RX-start error log now names the method actually
called (startReceive vs startReceiveDutyCycleAuto) instead of hardcoding the
duty-cycle name in the WASM branch.
* native-wasm: drop the emsdk bootstrap shim (now in platform-wasm)
The Emscripten SDK auto-location moved into the platform-wasm builder, so the
firmware no longer needs its own pre: extra script. Remove
extra_scripts/wasm_emsdk_env.py and bump the platform pin to the build that
carries the bootstrap. The wasm_link_flags.py post script stays — those exported
fns / runtime methods / Asyncify import seam are firmware-app-specific.
* wasm: use the canonical companion name (meshtasticd-wasm-node)
The companion repo was renamed meshtastic-web-node -> meshtasticd-wasm-node; fix
the stale name in the wasm README and bump the platform pin to the build that
promotes the canonical name in its emsdk auto-location.
* wasm: re-entrancy guard for the API/region entry points + flaky-open retry
Two robustness fixes for the browser node:
- Re-entrancy guard. The node is single-threaded + Asyncify: while setup()/loop()
is suspended inside a WebUSB transfer, the JS event loop is free, so a stray
DOM/timer callback that re-enters a wasm_* entry point starts a second Asyncify
unwind ("async operation already in flight" abort) or clobbers shared PhoneAPI
state (observed as a "PhoneAPI::available unexpected state" flood). Add a
g_wasm_in_firmware flag set around setup()/loop() (portduino_main_wasm.cpp); the
wasm_set_region / wasm_api_to_radio / wasm_api_from_radio / wasm_api_available
entry points now reject a mid-tick call (return busy) instead of corrupting or
aborting. The host must still call them between ticks — this is the safety net
the design lacked, not a substitute for the JS queue.
- CH341 open retry (js/bridge.js). First-connect WebUSB is flaky — the interface
is briefly unclaimable right after the grant, or held by a prior session,
giving a transient "Could not open SPI: -1". Retry the open with a short
backoff, closing the device between attempts so claimInterface starts clean.
* wasm: exclude emscripten-only sources from cppcheck
The `check` board matrix runs `pio check` (cppcheck) over all of src/,
including src/platform/portduino/wasm/. cppcheck can't parse the EM_ASYNC_JS/
EM_JS macros (Syntax Error: AST broken at libpinedio_webusb.c:39,
internalAstError) and these sources are not part of any checked board build
([env:native-wasm] is board_level=extra, compiled by the build-wasm CI job).
Suppress the wasm dir in suppressions.txt, the same way generated/ and .pio/
are already excluded.
* wasm: coalesce FS.syncfs so two never run at once
IDBFS syncfs is async; the explicit wasm_fs_sync (5s timer + post-save +
beforeunload) could overlap a prior in-flight sync, warning "2 FS.syncfs
operations in flight at once". Serialize: if a sync is running, mark a pending
re-sync and let the in-flight one chain it on completion — at most one in flight,
trailing writes still flushed. (Companion drops IDBFS autoPersist so this is the
single persistence path.)
* wasm: silence false-positive SAST on the emscripten glue
- extra_scripts/wasm_link_flags.py: restore the trunk-ignore-all(ruff/F821,
flake8/F821) header every other SCons extra_script carries; Import/env are
SConscript-injected globals, so ruff/flake8 flag them as undefined.
- .semgrepignore: exclude src/platform/portduino/wasm/js/ (browser WebUSB glue,
not part of the firmware binary). The unsafe-formatstring rule false-positives
on its benign retry/diagnostic console logs.
* Update .github/workflows/build_portduino_wasm.yml
Co-authored-by: Austin <vidplace7@gmail.com>
---------
Co-authored-by: Austin <vidplace7@gmail.com>
|
||
|
|
00f5fa80be |
Add native Portduino malloc shim (#10677)
* Add native Portduino malloc shim * Taking copilots advice Honestly I don't think it matters that much, but if it makes copilot happy, return ret if something weird happens. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
TomandClaude Sonnet 4.6
|
c51c01607d |
Traffic Management module: dedup, rate limiting, role-aware policing (#10706)
Adds the Traffic Management module (TMM) plus the NodeDB/warm-store and next-hop foundations it builds on: - Unified per-node cache (flat array, 8-bit relative ticks) shared by all features; role-aware throttles for tracker / lost-and-found. - Position deduplication: drop unchanged position rebroadcasts within a configurable interval; precision driven off the channel ceiling (clamped to the public-key max on well-known channels). Enabled by default at 11h. - Per-node rate limiting and unknown-packet filtering (config-driven; a non-zero companion field enables each feature -- no bool toggles). - NodeInfo direct response from cache with role-based hop clamps. - Persistent next-hop overflow store: confirmed hops have no TTL, are seeded from NodeInfoLite at boot, and survive hot-store eviction. - Three-tier sender-role resolution (hot NodeInfoLite -> warm store -> TMM cache). Role is cached write-time (seeded on first track, refreshed from NodeInfo), pins its cache entry like a next-hop hint, and is evicted last. - Warm store caches device role + protected category across reboot/eviction. - PositionModule stationary floor for tracker / lost-and-found. - PSRAM gating for warm/satellite/TMM cache sizes; STM32WL excluded. Protobufs: TrafficManagementConfig trimmed to the five uint32 fields actually used; submodule repointed to protobufs develop. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Jonathan Bennett
|
ca7d82629d |
Native sensors (#10748)
* Enable Sensirion libraries on native * Bump platform-native to get bug fix |
||
|
Jonathan BennettandCopilot Autofix powered by AI
|
1cf6d7648f |
Leave src/platform out of the build filter by default (#10726)
* Leave src/platform out of the build filter by default, and only add back what each build needs. * typo fix Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
|
8267bb22bd |
Packet Signing via XEdDSA (#10478)
* Test commit for XEdDSA support * Update to Crypto lib in Meshtatic org * Generate a new node identity on key generation (#7628) * Generate a new node identity on key generation * Fixes * Fixes * Fixes * Messed up * Fixes * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Figured it out! * Cleanup * Update src/mesh/NodeDB.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update crypto commit hash * Some fixes for xeddsa pr (#9610) * fix: add null check for getMeshNode() in NodeInfoModule getMeshNode() can return nullptr for unknown nodes. Dereferencing without a check crashes the firmware when receiving NodeInfo from a node not yet in the database. * fix: enforce XEdDSA signature verification and prevent stripping Previously, failed signature verification still allowed the packet through, making signatures purely cosmetic. Now: - Failed verification drops the packet (DECODE_FAILURE) - Successfully verified nodes get HAS_XEDDSA_SIGNED bitfield set - Unsigned packets from previously-signing nodes are rejected - Log levels reduced from WARN/ERROR to DEBUG/WARN as appropriate * fix: include packet metadata in XEdDSA signature The signature now covers [fromNode | packetId | portnum | payload] instead of just the payload bytes. This prevents: - Replay attacks (different packetId fails verification) - Reattribution (different fromNode fails verification) - Portnum redirection (different portnum fails verification) Also adds a key initialization check to xeddsa_sign (returns false if XEdDSA keys are all zeros) and checks the return value in the encode path. * fix: handle existing key pair in AdminModule security config When a user provides both a valid private key and public key via admin config, the crypto engine's DH private key and owner public key were never loaded. DMs and XEdDSA signing would silently break. Add an else branch to load both keys into the crypto engine. * perf: cache Ed25519 public key conversion in xeddsa_verify curve_to_ed_pub() performs field element parsing, inversion, and multiplication on every call. Since packets from the same node tend to arrive in bursts, a single-entry cache avoids repeating this expensive conversion for consecutive packets from one sender. * fix: skip identity cleanup when node number is unchanged createNewIdentity() was called on every generateCryptoKeyPair(), including normal boots where the same key is regenerated. This caused unnecessary NodeDB writes and old-node cleanup logic to run when the node number hadn't actually changed. Also fixes only zeroing byte[0] of the old node's public key instead of clearing the entire array. * fix: replace hardcoded 120 with derived XEDDSA_SIGNATURE_SIZE constant The payload size check for XEdDSA signing used a magic number (120). Replace with a derivation from DATA_PAYLOAD_LEN and XEDDSA_SIGNATURE_SIZE so the limit adjusts automatically if constants change. This also increases the max signable payload from 120 to 169 bytes, which is still safe since the actual encoded size is checked after pb_encode. * fix: add const qualifiers to XEdDSA verify and curve_to_ed_pub inputs pubKey, payload, and signature parameters in xeddsa_verify are input-only and should not be modified. Same for curve_pubkey in curve_to_ed_pub. * chore: remove commented-out old Crypto dependency in portduino.ini * Leave out the admin module change for now --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> * trunk * protobuf re-update * Protobufs * Merge resolution fix * Put XEDDSA on the right bit * NodeDB update to new nodeInfoLite accessors, etc * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Refine unsigned packet rejection logic in Router (#10534) * use hardware random to fill the first 32 signature bytes with entropy prior to signing. * Add XEdDSA packet-signing policy tests and update dependencies for macos * Minor fixes * integrate XEdDSA support and update dependencies across multiple modules --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Wessel <github@weebl.me> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> |
||
|
Ben Meadors
|
93f87c57b9 | MacOS fixes | ||
|
TomandCopilot
|
de345939af |
Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Thomas Göttgens
|
f86cb7781e | Remove fragile JSON libraries from the firmware while retaining Meshtasticd JSON support (#10152) | ||
|
Thomas Göttgens
|
1747e2d8e5 | T-Echo-Card support (#10267) | ||
|
|
4a1ff18f57 |
feat: add Nordic nRF54L15-DK variant (Zephyr + BLE + LoRa) (#10193)
* feat: add Nordic nRF54L15-DK variant (Zephyr + BLE + LoRa)
Adds a community hardware variant for the Nordic nRF54L15-DK (PCA10156)
with an external EBYTE E22-900M30S (SX1262) LoRa module. First Meshtastic
port running on the Zephyr RTOS; all other Nordic targets use the nRF5
SoftDevice stack.
Scope
-----
- New Zephyr-based platform layer under src/platform/nrf54l15/ providing
Arduino-compatible shims (Arduino.h, SPI, Wire, Print, Stream) over the
Zephyr APIs plus a LittleFS-backed InternalFileSystem on SPIM20.
- Bluetooth LE peripheral (NRF54L15Bluetooth.*) built on the Zephyr BT
host stack, exposing the Meshtastic GATT service with legacy
connectable advertising, just-works pairing, dynamic MTU exchange
(up to 247 bytes), and iOS connection-parameter tweaks.
- Variant directory variants/nrf54l15/nrf54l15dk/ with pin map for the
E22 module on connector J1, PlatformIO env (nrf54l15dk), Zephyr
DT overlay and a wiring README.
- Zephyr project config (zephyr/prj.conf + board overlay) tuned for
BT + LoRa: 16 KB main stack, 4 KB BT RX thread, RTT logging in
immediate mode, newlib-nano heap sized to leave room for the GATT
pools while still allowing ATT MTU=247.
- extra_scripts/nrf54l15_linker.py works around a PlatformIO + old Ninja
issue where Zephyr's two-pass linker script generation does not run
automatically; the post-script parses build.ninja and invokes the
gcc -E step directly before the final link.
- boards/nrf54l15dk.json board definition (PlatformIO needs it for the
DK; the Seeed platform only ships the XIAO variants).
- variants/rp2350/rp2350.ini excludes platform/nrf54l15/ from RP2350
build_src_filter so the shared platform tree does not leak between
targets.
- .gitignore: add nRF J-Link / RTT debug artifacts (flash.jlink,
rtt_*.txt).
Shared source changes
---------------------
- src/main.{cpp,h}, src/RedirectablePrint.cpp, src/FSCommon.{cpp,h},
src/mesh/{Channels,NodeDB,RadioLibInterface,MeshService,PhoneAPI}.cpp,
src/mesh/RadioLibInterface.h, src/modules/AdminModule.cpp: add small
guards / helpers so the Zephyr build compiles alongside the Arduino
targets. Behavior on existing boards is unchanged.
Hardware model
--------------
HW_VENDOR maps to meshtastic_HardwareModel_PRIVATE_HW until a dedicated
protobuf enum value is assigned upstream. The variant declares
custom_meshtastic_hw_model = 132 so the maintainers can wire the new
enum value through the protobufs repo after merge.
Hardware note
-------------
The E22-900M30S does not connect its DIO2 pin to TXEN internally — a
wire/solder bridge between DIO2 and TXEN on the module is required for
TX to work. Details and full pin map are in the variant README.
Validation
----------
Built clean against develop. On real hardware (April 2026) the port
passes end-to-end: iOS companion app pairs and connects, configuration
round-trip works, LoRa TX/RX reaches a canonical tbeam on the same mesh
channel, NodeDB updates propagate both ways, and traceroute completes.
* fix(nrf54l15): use atomic fs_rename instead of copy fallback
Zephyr LittleFS on nrf54l15 supports fs_rename natively, so route it
through the same atomic path as ESP32. The previous copyFile+remove
fallback truncated the destination before copying, leaving 0-byte files
if interrupted mid-write.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(nrf54l15): expand storage_partition from 36KB to 700KB
LittleFS on the default 9-block (36KB) storage_partition ran out of
space during copy-on-write of config.proto, causing fs_write to return
ENOSPC and pb_encode to surface "io error" when saving configuration
via the mobile app.
Reclaim slot1_partition (the MCUboot secondary slot — unused since we
flash directly via J-Link) and grow storage_partition to span
0xb6000..0x165000 (~175 blocks).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(nrf54l15): drop USERPREFS_LORACONFIG_* so LoRa config stays mutable
NodeDB rewrites LoRa config from USERPREFS_LORACONFIG_* on every boot,
which prevented reconfiguration via the BLE/serial app. Drop the
variant-level defaults; users configure region and modem preset through
the app like every other variant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(nrf54l15): enforce MITM passkey pairing on GATT service
- Add MESH_PERM_READ/MESH_PERM_WRITE macros (READ_AUTHEN/WRITE_AUTHEN)
on all mesh service characteristics so clients must complete passkey
exchange before accessing fromNum/fromRadio/toRadio/logRadio.
- Wire FIXED_PIN mode to bt_passkey_set() so the device advertises a
known PIN (config.bluetooth.fixed_pin); RANDOM_PIN keeps default
per-pairing random passkey.
- Reduce BleDeferredThread HARD_WATCHDOG_MS from 3min to 1min.
- prj.conf: CONFIG_BT_SMP_ENFORCE_MITM=y, CONFIG_BT_FIXED_PASSKEY=y,
CONFIG_BT_SMP_SC_PAIR_ONLY=n (legacy fallback for clients that abort
SC pairing with reason 0x01 within 150ms).
* fix(nrf54l15): resolve develop-merge conflict + cppcheck warnings
The `Merge branch 'develop'` left two ~RadioLibInterface() declarations
in src/mesh/RadioLibInterface.h: the inline version added upstream by
PR #10254 (which independently applied the same UAF guard this PR was
carrying) and the out-of-line version this PR introduced. GCC rejects
the duplicate, breaking every platform build. Drop the out-of-line
declaration + definition; keep upstream's inline form.
Also silence the 13 cppcheck low warnings introduced by the new
nrf54l15 Arduino shim — Arduino's `String`/`SPISettings` API contract
relies on implicit single-arg constructors used pervasively by
existing Meshtastic code, so suppress `noExplicitConstructor` inline
with a comment instead of breaking the API. The few mechanical wins
(`const tmp[2]`, `const uint32_t *sp`) are applied directly.
* fmt: fix Trunk Check lint issues on nrf54l15-port
- extra_scripts/nrf54l15_linker.py: move regular imports above
Import("env") to silence E402, add trunk-ignore-all(F821) for the
PIO/SCons SConstruct injection (matches esp32_pre.py / nrf52_extra.py
convention)
- src/platform/nrf54l15/NRF54L15Bluetooth.cpp: clang-format 16.0.3
- boards/nrf54l15dk.json + variants/nrf54l15/nrf54l15dk/README.md:
prettier 3.8.3 (also resolves markdownlint MD060 on README tables)
No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(nrf54l15): address Copilot review comments + correct clang-format style
Six review threads from the 2026-04-30 Copilot review:
- src/platform/nrf54l15/nrf54l15_main.cpp: validate PSP against the nRF54L15
SRAM range (0x20000000..0x20040000) and 4-byte alignment before walking the
faulting thread's stack, and clamp the walk so it never reads past the end
of RAM. Prevents a second fault inside the fatal handler when PSP is
corrupted (common in real faults).
- src/platform/nrf54l15/nrf54l15_arduino.cpp: gate the bring-up printk traces
in digitalWrite/digitalRead (CS/NRESET toggle log, BUSY-before-NRESET
snapshot, BUSY periodic timeline) behind a new -DNRF54L15_GPIO_DEBUG flag
that is off by default. The "dev NOT READY" message stays unconditional —
it indicates a genuine hardware/DTS misconfig.
- src/modules/AdminModule.cpp: don't mutate config.device.output_gpio_enabled
from handleGetConfig(). Reflect the live pin state in the response payload
only — a getter must not write back to disk-persisted state.
- src/platform/nrf54l15/InternalFileSystem.h: derive totalBytes() from
FIXED_PARTITION_SIZE(storage_partition) at compile time so it tracks the
DK overlay's ~700 KB partition instead of the stale 36 KB hard-coded value.
Updated the file header comment accordingly.
- extra_scripts/nrf54l15_linker.py: make _extract_gcc_command() handle the
POSIX Ninja COMMAND format (no `cmd.exe /C "..."` wrapper) in addition to
the Windows form, so the script doesn't hard-fail on Linux/macOS hosts.
- src/platform/nrf54l15/NRF54L15Bluetooth.cpp: clamp NO_PIN to RANDOM_PIN
with a one-shot LOG_WARN. The mesh GATT permissions are declared with
BT_GATT_PERM_*_AUTHEN and prj.conf sets CONFIG_BT_SMP_ENFORCE_MITM=y, so
NO_PIN with no auth callbacks would leave every characteristic returning
BT_ATT_ERR_AUTHENTICATION. Falling back to RANDOM_PIN keeps the link
usable instead of silently broken. Also re-formatted this file with the
project's .trunk/configs/.clang-format (Linux braces, 4-space indent,
130-col) — the previous lint-fix commit
|
||
|
Ben Meadors
|
8110887be2 | Turns out it's already excluded | ||
|
Ben Meadors
|
593909c26b | Radiolib excludes | ||
|
Ben Meadors
|
784e3748e2 | Merge remote-tracking branch 'origin/master' into develop | ||
|
renovate[bot]
|
d559af8477 |
Update LovyanGFX to v1.2.21 (#10373)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Jonathan Bennett
|
bb86cf4e81 | Merge remote-tracking branch 'origin/master' into develop | ||
|
renovate[bot]
|
7cb071c780 |
Update platform-native digest to cab4b21 (#10372)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> |
||
|
Austin Lane
|
0240a00d09 | MacOS: Re-Add Orcania/Yder |