* asdf
* Implement SphereOfInfluenceModule for traffic management and eviction tracking
* Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor
* Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy
* Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule
* Enable variable hop limits and role-based hop floors in Sphere of Influence module
* Respond to copilot review
* Disable variable hop limits and role-based hop floors in Sphere of Influence module
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Implement adaptive sampling for unique node ID tracking in Sphere of Influence module
* Add state persistence for Sphere of Influence module
* Enhance Sphere of Influence module with state management and adaptive sampling adjustments
* Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule
- Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule.
- Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation.
- Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity.
- Introduced state persistence for hop scaling parameters to maintain continuity across reboots.
- Enhanced thread safety and modularity by utilizing concurrency features.
* Guard out STM32. Sowwy.
* Refactor HopScalingModule: enhance sampling logic and improve state management
* Add unit tests for HopScalingModule: implement mock database and various test scenarios
* Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity
* Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability
* Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity
* Remove unnecessary delay in setup function for improved test performance
* Add missing include for MeshTypes in test_main.cpp
* Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity
* Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases
* Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc.
* Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management
* Add sample traffic injection for HopScaling tests to enhance sampledEst visibility
* Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting
* Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior
* Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests
* Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output
* Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests
* Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation
* Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points
* Implement CompactHistogram for parallel hop scaling sampling
- Added CompactHistogram class to track node hop distances with bitwise sampling.
- Integrated CompactHistogram into HopScalingModule for independent packet sampling.
- Updated NodeDB to feed both the hop scaling module and the new histogram sampler.
- Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics.
- Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling.
- Updated existing tests to validate the integration of the new histogram sampling mechanism.
* Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior
* CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution
* Refactor HopScalingModule and CompactHistogram integration
- Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic.
- Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling.
- Enhanced logging in HopScalingModule to provide detailed histogram statistics.
- Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic.
- Improved node ID distribution in tests to better exercise sampling mechanisms.
- Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling.
* Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes
- Updated the bitfield structure to accommodate 13-hour seen tracking.
- Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation.
- Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios.
- Adjusted filtering and sampling logic to reflect the new 13-hour tracking period.
- Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes.
* Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making
- Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling.
- Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management.
- Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection.
- Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops.
- Enhanced unit tests to validate new sampling logic and memory layout changes.
* Expose hashNodeId for testing in CompactHistogram
* Add mesh trend statistics to CompactHistogram for enhanced activity tracking
* Implement histogram state persistence in CompactHistogram with save and load functions
* Refactor CompactHistogram to improve entry management and enhance rollHour logging
* feat: add HopScalingModule for adaptive hop limit recommendations
Introduces HopScalingModule, a sampled hop-distance histogram that
recommends the minimum hop limit needed to reach ~40 nodes, and
automatically reducing the hops as the mesh grows.
Key design:
- 512-byte packed histogram (128 × 4-byte Record entries) embedded
in a new HopScalingModule.
- Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap
- Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept;
denominator doubles on overflow and halves when utilisation is low
- Hourly rollHour(): tallies per-hop counts, walks scaled buckets to
find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a
politeness extension based on recent/older activity ratio, shifts all
seen bitmaps, and persists state to /prefs/hopScalingState.bin
- Hop recommendation gated by bootstrap (requires >=1 rollHour before
overriding HOP_MAX)
- NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet
- Module also estimates total mesh size and logs useful information about
mesh characteristics.
Changes:
- src/modules/HopScalingModule.h/.cpp: new module
- src/mesh/NodeDB.cpp: wire up samplePacketForHistogram
- src/mesh/Router.cpp: consume getLastRequiredHop()
- test/test_hop_scaling/: 12-test suite covering all mesh topologies and
anticipated operational requirements
* test: increase run iterations in sparse to dense transition test
* feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging
* feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions
* refactor: remove CompactHistogram module and related files
* address copilot review comments
* Tweak: packet sampling only lora
* ove role-based hop floor logic and related definitions into the module - keep it in one place.
* Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting
* Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact.
* Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency
* refactor: improve test output organization and clarity in hop scaling tests
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* macOS: enable CH341 LoRa-hardware path — fix serial truncation, document setup
Verified on Apple Silicon with a CH341A USB-SPI bridge (VID 0x1A86,
PID 0x5512) wired to an SX1262 (Meshstick variant) that the existing
`pine64/libch341-spi-userspace` lib_dep works on macOS as-is — Apple's
bundled CH34x driver only matches the CH340 *UART* variant
(PID 0x7523), so the CH341A's interface 0 is left unclaimed and
libusb opens / configures / claims it directly via IOUSBHostInterface.
End-to-end test: meshtasticd boots, libusb claim succeeds, SX1262 init
returns 0, TCP API serves the meshtastic CLI's --info / --sendtext flow.
Two changes:
1. **`PortduinoGlue.cpp:497`**: pass `sizeof(serial)` (= 9) instead of
the literal `8` to `Ch341Hal::getSerialString()`. The function in
`USBHal.h:61-68` treats `len` as buffer size and reserves one slot
for the null terminator (`bytesCopied = (len - 1) < 8 ? (len - 1) : 8`),
so passing 8 produced a 7-char serial — which then broke the
`strlen(serial) == 8` check at line 502, skipping the auto-MAC
derivation from serial + product string. On Linux this was masked
by the BlueZ HCI MAC fallback in `getMacAddr()` at lines 139-157,
but on macOS that fallback is `__linux__`-guarded so the serial path
is mandatory and the truncation left `mac_address` empty, causing
the daemon to exit with `*** Blank MAC Address not allowed!`.
2. **`variants/native/portduino/platformio.ini`**: expand the
`[env:native-macos]` comment block with a "Real LoRa hardware on
macOS" section. Documents:
- Why no upstream library change is needed (Apple kext targets
CH340/UART, not CH341A/SPI; libusb's `#ifdef __linux__` skip is
correct for macOS in this case).
- How to point `meshtasticd` at an existing platform-agnostic
`bin/config.d/lora-*.yaml` for CH341 hardware.
- The auto-MAC-derivation contract (now working with this fix).
- `ioreg` and `LIBUSB_DEBUG=4` diagnostic recipes for the failure
mode where a third-party WCH `CH34xVCPDriver` *would* claim
interface 0 (`kmutil unload -b <bundleID>` workaround).
No upstream library forks, no PR chain, no additional lib_deps —
the existing `pine64/libch341-spi-userspace` + libusb-1.0 stack does
the right thing on macOS already.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* macOS: enable CH341 LoRa-hardware path — fix serial truncation, document setup
Verified on Apple Silicon with a CH341A USB-SPI bridge (VID 0x1A86,
PID 0x5512) wired to an SX1262 (Meshstick variant) that the existing
`pine64/libch341-spi-userspace` lib_dep works on macOS as-is — Apple's
bundled CH34x driver only matches the CH340 *UART* variant
(PID 0x7523), so the CH341A's interface 0 is left unclaimed and
libusb opens / configures / claims it directly via IOUSBHostInterface.
End-to-end test: meshtasticd boots, libusb claim succeeds, SX1262 init
returns 0, TCP API serves the meshtastic CLI's --info / --sendtext flow.
Two changes:
1. **`PortduinoGlue.cpp:497`**: pass `sizeof(serial)` (= 9) instead of
the literal `8` to `Ch341Hal::getSerialString()`. The function in
`USBHal.h:61-68` treats `len` as buffer size and reserves one slot
for the null terminator (`bytesCopied = (len - 1) < 8 ? (len - 1) : 8`),
so passing 8 produced a 7-char serial — which then broke the
`strlen(serial) == 8` check at line 502, skipping the auto-MAC
derivation from serial + product string. On Linux this was masked
by the BlueZ HCI MAC fallback in `getMacAddr()` at lines 139-157,
but on macOS that fallback is `__linux__`-guarded so the serial path
is mandatory and the truncation left `mac_address` empty, causing
the daemon to exit with `*** Blank MAC Address not allowed!`.
2. **`variants/native/portduino/platformio.ini`**: expand the
`[env:native-macos]` comment block with a "Real LoRa hardware on
macOS" section. Documents:
- Why no upstream library change is needed (Apple kext targets
CH340/UART, not CH341A/SPI; libusb's `#ifdef __linux__` skip is
correct for macOS in this case).
- How to point `meshtasticd` at an existing platform-agnostic
`bin/config.d/lora-*.yaml` for CH341 hardware.
- The auto-MAC-derivation contract (now working with this fix).
- `ioreg` and `LIBUSB_DEBUG=4` diagnostic recipes for the failure
mode where a third-party WCH `CH34xVCPDriver` *would* claim
interface 0 (`kmutil unload -b <bundleID>` workaround).
No upstream library forks, no PR chain, no additional lib_deps —
the existing `pine64/libch341-spi-userspace` + libusb-1.0 stack does
the right thing on macOS already.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add ESP32 Power Management lessons learned document
Documents our experimentation with ESP-IDF DFS and why it doesn't
work well for Meshtastic (RTOS locks, BLE locks, USB issues).
Proposes simpler alternative: manual setCpuFrequencyMhz() control
with explicit triggers for when to go fast vs slow.
* Addition of traffic management module
* Fixing compile issues, but may still need to update protobufs.
* Fixing log2Floor in cuckoo hash function
* Adding support for traffic management in PhoneAPI.
* Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE.
* Adding station-g2 and portduino varients to be able to use this module.
* Spoofing from address for nodeinfo cache
* Changing name and behavior for zero_hop_telemetry / zero_hop_position
* Name change for exhausting telemetry packets and setting hop_limit to 1 so it will be 0 when sent.
* Updated hop logic, including exhaustRequested flag to bypass some checks later in the code.
* Reducing memory on nrf52 nodes further to 12 bytes per entry, 12KB total using 8 bit hashes with 0.4% collision. Probably ok. Adding portduino to the platforms that don't need to worry about memory as much.
* Fixing hopsAway for nodeinfo responses.
* traffic_management.nodeinfo_direct_response_min_hops -> traffic_management.nodeinfo_direct_response_max_hops
* Removing dry run mode
* Updates to UnifiedCacheEntry to use a common cache, created defaults for some values, reduced a couple bytes per entry by using a resolution-scale time selection based on configuration value.
* Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants.
* Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails.
* Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit.
* Creating consistent log messages
* Remove docs/ESP32_Power_Management.md from traffic_module
* Add unit tests for Traffic Management Module functionality
* Fixing compile issues, but may still need to update protobufs.
* Adding support for traffic management in PhoneAPI.
* Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE.
* Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants.
* Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails.
* Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit.
* Add mock classes and unit tests for Traffic Management Module functionality.
* Refactor setup and loop functions in test_main.cpp to include extern "C" linkage
* Update comment to include reduced memory requirements
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Re-arranging comments for programmers with the attention span of less than 5 lines of code.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update comments in TrafficManagementModule to reflect changes in timestamp epoch handling and memory optimization details.
* bug: Use node-wide config_ok_to_mqtt setting for cached NodeInfo replies.
* Better way to handle clearing the ok_to_mqtt bit
* Add bucketing to cuckoo hashing, allowing for 95% occupied rate before major eviction problems.
* Extend nodeinfo cache for psram devices.
* Refactor traffic management to make hop exhaustion packet-scoped. Nice catch.
* Implement better position precision sanitization in TrafficManagementModule.
* Added logic in TrafficManagementModule to invalidate stale traffic state. Also, added some tests to avoid future me from creating a regression here.
* Fixing tests for native
* Enhance TrafficManagementModule to improve NodeInfo response handling and position deduplication logic. Added tests to ensure local packets bypass transit filters and that NodeInfo requests correctly update the requester information in the cache. Updated deduplication checks to prevent dropping valid position packets under certain conditions.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Remove noop CANNED_MESSAGE_MODULE_ENABLE define
* Remove over-eager warning removal
* Remove unused LED_CONN
* Dead defines removal
* Rename oddball LED pin name
* Rename second oddball LED pin name
* Remove another dead define
* Plain RAK4631 should not compile EInk and TFT display code
* Add USE_TFTDISPLAY to variant files.
* Derp
* Undo the platformio.ini changes to heltec_v4
* Drop unneeded src_filter lines
---------
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
Co-authored-by: Jason P <applewiz@mac.com>
* Force coverage tests to run in simulation mode
* Revert "Force coverage tests to run in simulation mode"
This reverts commit e4ec719e6f.
* Force coverage tests to run in simulation mode
---------
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
* Start portduino_config refactor
* refactor GPIOs to new portduino_config
* More portduino_config work
* More conversion to portduino_config
* Finish portduino_config transition
* trunk
* yaml output work
* Simplify the GPIO config
* Trunk
