49 Commits

Include renames

Author	SHA1	Message	Date
4 people	80cfa52665	Add zero guards on time calculations where they were missing (#11692) ... * time: add skipZero/safeMillis/timerEndsAtMillis helpers skipZero() steps a millis value past 0, since stored stamps and deadlines conventionally use 0 for "unset" and the one tick per ~49.7-day wrap that lands on 0 would otherwise read as never-set. safeMillis() covers a bare stamp; timerEndsAtMillis(delayMs) covers a deadline, where the sum is what has to dodge 0 - a non-zero read plus a delay lands there once per wrap - so it is not safeMillis() + delayMs. * time: replace hand-rolled zero-dodging with the UptimeClock helpers PacketHistory rxTimeMsec, EncryptedStorage s_lastFailMillis (stamps), and SGM41562 lastRefreshMs_ / NextHopRouter learnedAtMsec (ternary stamps) each hand-rolled skipZero() in place; swap in safeMillis()/skipZero() directly. HapticFeedback pulseOffAt/delayedPulseAt and GPS fixHoldEnds hand-rolled the deadline form - millis() + delay, then remap a 0 result to 1 - swap in timerEndsAtMillis(delay). No behavior change; each site keeps the value it already computed. * time: guard the remaining 0-means-unset deadline/stamp writes rebootAtMsec, shutdownAtMsec, and NotificationRenderer::alertBannerUntil are all read back with a bare == 0 / != 0 check for 'not scheduled', but every write site computed millis() + delay (or a bare millis() stamp) with no guard against landing exactly on 0 - the same wrap hazard skipZero() exists for, just never applied here. Route every rebootAtMsec/shutdownAtMsec/alertBannerUntil write through timerEndsAtMillis()/safeMillis(); RadioLibInterface's reboot-on-stuck-tx sums an already-captured stamp rather than "now", so it goes through skipZero() directly instead. No behavior change outside the ~1-in-2^32 wrap window each site was already exposed to. * time: guard three more 0-means-unset deadline writes ntp_renew (ethClient.cpp), suppressTouchTapUntilMs (Events.cpp), and tx_after (RadioLibInterface.cpp) all read back 0 as a real state - forced NTP renewal, no suppress window active, no TX delay armed, respectively - but each arm site wrote a bare millis()/getMillis() + delay with no guard against the sum landing exactly on 0. Route each through Time::timerEndsAtMillis(). No behavior change outside the wrap window each site was already exposed to. Refresh the Throttle.h TODO list to note ntp_renew is converted too. * motion: guard the calibration deadline and use Throttle::deadlinePassed endCalibrationAt's arm site wrote millis() + calibrateFor with no guard against landing on 0, the same value finishCalibrationIfExpired()/ drawFrameCalibration() treat as "not calibrating". Route it through Time::timerEndsAtMillis(). Also swap finishCalibrationIfExpired()'s hand-rolled (int32_t)(now - deadline) < 0 for Throttle::deadlinePassed(): same wrap-safe comparison the codebase already provides, without the signed-cast pattern Throttle.h documents as implementation-defined past INT32_MAX, and it drops the file's last direct millis() call in favor of the Time:: wrapper the rest of it already uses. * time: fix Throttle::execute()'s own zero-dodging Both places execute() writes *lastExecutionMs - the first-ever-run branch and the regular update - used bare Time::getMillis() with no guard against landing on 0, which is the exact sentinel this function reads back as "never run" one line above. A hit there makes the next call re-fire immediately instead of respecting minumumIntervalMs. Capture now via Time::safeMillis() once; every use downstream (the elapsed comparison, the stored value) is then safe by construction instead of needing the guard reapplied at each write. * revert some safeMillis cases where overflow is a bad thing * test(uptime): pin skipZero/safeMillis/timerEndsAtMillis at the wrap boundary Covers the zero case, an ordinary nonzero value, and a sum that lands exactly on 0 from a nonzero start - the case timerEndsAtMillis() exists for, and the one the prior suite had no direct coverage of. * time: restore the route-health write normalization and put it on one clock noteRouteLearned()/noteRouteSuccess() lost their `now ? now : 1` normalization, leaving learnedAtMsec able to store 0 - which getOrAllocRouteHealth() reads as an ever-growing age, making the slot the first eviction candidate and permanently stale. Normalize at the write, where the block comment already says it happens, so every caller is covered rather than just today's two. Both callers, the two isRouteStale() sites and doRetransmissions() now read Time::getMillis(), so the stamp and every comparison against it share a clock. doRetransmissions() goes back to getMillis(): its `now` feeds only comparisons, never a 0-sentinel field, so skipping zero there only cost accuracy. * time: read the haptic, InkHUD and calibration deadlines on the write's clock These three deadlines were converted to Time::timerEndsAtMillis() on the write side while their reads stayed on millis(), so each spanned two clocks and would fire immediately or never under an injected test clock. Convert the reads to match: HapticFeedback::scheduleNext()/runOnce(), the InkHUD tap-suppression window, and the calibration countdown's read-back of screen->getEndCalibration(). MotionSensor's sampledAtMs is left alone - its write and read are both millis() and consistent already. * time: correct the sentinel notes to match what the code actually does The Throttle.h enumeration claimed the remaining timerEndsAtMillis() callers "already dodge the sentinel", which reads as a completeness claim the same branch contradicts: RadioLibInterface's tx_after and activeReceiveStart are both 0=unarmed and both still arm from bare millis(). Name them instead, so the deadline-type conversion has the real list. The ntp_renew entry now separates a deliberate 0 ("due now", forced at link-up) from a computed one, which is what changed there. The three TODO(elapsed-stamp) blocks ran four and five lines against the repo's one-or-two rule, and two of them argued their case wrongly. Throttle.cpp implied safeMillis() simply doesn't help; in fact neither store is safe on the wrap tick - the 1 underflows a same-instant read, the 0 re-takes the never-run branch - which is the symmetry worth recording. PacketHistory.cpp called its dodge "reflecting the previous pattern" when it is load-bearing: rxTimeMsec 0 means "empty slot" (PacketHistory.h:21) and insert() drops a record stamped 0 outright, so without it a packet arriving on the wrap tick is never stored and loses its dedup. Also picks up trunk fmt's trailing-whitespace fix in Throttle.cpp and the comment realignment in SGM41562.cpp that this branch's added comment knocked out. * test(nexthop): pin the route-health stamp against the 0 sentinel The uptime suite covers skipZero/safeMillis/timerEndsAtMillis themselves, but nothing covered a call site, so the branch deleted noteRouteLearned()'s normalization and stayed green. None of the existing route-health tests pass 0 as `now` - they use 1000, learnAt, or millis() - (TTL + 5000) - which is exactly the gap the regression went through. Both new tests fail with "Expected 0 to be not equal to 0" when the skipZero() is backed out of NextHopRouter, and pass with it. noteRouteSuccess() only refreshes an existing record, so its twin learns a route first to reach the write. Also drops a self-referential assertion in the uptime suite: comparing getMillis() against safeMillis() passes even if safeMillis() does no dodge at all, so it now asserts the literal. * discard safemillis for skipzero (better semantics and therefore maintainability) and make consistent use of getmillis where it is called (to permit testing) * more wrapzero safety * STM gets some too * time: stop the next 0-means-unset deadline being armed from raw millis() The fields this branch armed through Time::timerEndsAtMillis() / Time::skipZero() are the kind that get added by copy-paste: `rebootAtMsec = millis() + N` appears at twenty-odd sites across six files, and the next module to defer a reboot will be written from one of them. Nothing catches the mistake afterwards - the sum lands on 0 for one tick per ~49.7-day wrap, so a test run, a soak and a bench session all pass while a pending reboot, shutdown, DFU jump or banner expiry is silently dropped. Two guards, at the two places it can go wrong. The helpers themselves: skipZero() is constexpr, so its contract is now pinned by static_assert in the header rather than only by test_uptime_clock. The asserts are chosen against the two plausible rewrites - `ms | 1` perturbs every even value and `ms + 1` turns the last tick of the wrap into the 0 the function exists to avoid. Both compile, and both pass a test that only checks skipZero(0); each trips a distinct assert here, naming the failure mode. The call sites: bin/lint-unset-sentinel-millis.sh flags a sentinel field in src/ assigned from a raw millis()/getMillis() read, and names the helper to use. It is name-driven because the 0 contract is declared in src/main.h and enforced in six other files, so no single-file scan can infer it; every one of the thirteen fields was checked to actually test against 0 before being listed. nagCycleCutoff and LinuxJoystick's nextRepeatX/nextRepeatY are deliberately absent - their unset state is a separate bool - and the nine remaining `millis() + x` sites in src/ are locals that never store 0 for anything to misread. Blocking, unlike its note-level neighbours: there is no run-time enforcer to pair with, and the tree has zero violations today, so gating costs nothing. Scoped to src/ so test_uptime_clock can keep building raw wrap values on purpose. bin/test-lint-unset-sentinel-millis.sh pins the scanner against 23 fixtures - reads, disarms, shadowing locals, comments, string literals and the already-fixed forms all have to stay quiet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * time: guard the four 0-means-unset stamps this branch had missed Sweeping src/ for the `if (stamp && <deadline check>)` idiom - the shape that makes 0 mean "unset" - turned up four stamps still armed from a raw clock read, so the new lint rule would have had to either ignore them or go red on checkout. Each is the same one-tick-per-wrap hole the rest of the branch closes: * TrackballInterruptBase lastInterruptTime, armed in all four ISR handlers and explicitly disarmed to 0 at the threshold reset. getMillis() is the ISR-safe read by construction - it compiles to millis() outside PIO_UNIT_TESTING - and skipZero() is pure, so neither adds anything to interrupt context. * NeighborInfoModule lastSentReply, read as `if (lastSentReply && ...)` before the 3-minute reply throttle. Needed the UptimeClock.h include. * PositionModule lastSentReply, same throttle; already on the injectable clock but still missing the guard. * NodeDB lastSort, whose own read spells the sentinel out as `lastSort == 0 ||`. On the wrap tick each would read as never-stamped: a trackball debounce window lost, a neighbour or position reply sent inside the throttle it was meant to respect, one extra NodeDB sort. Cheap individually, which is why they were missed. All four are now listed in bin/lint-unset-sentinel-millis.sh, so the rule covers every field in the tree that actually tests against 0 rather than a subset, and the header records the eight stamps left off for the opposite reason - their unset state is a separate flag (isNagging, busyTx, heldX/heldY, formatted_this_boot, heartbeat, gotwind, haveSample, lastIaqValid), so 0 is a value they may legally hold. The rule is silent across src/ on this tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * lint: let a site opt out of the sentinel rule, with its reason on the record The rule is blocking, so it needs an escape hatch for the site where 0 genuinely is a legal timestamp - and the hatch should cost something, or it becomes the first thing anyone reaches for. `unset-sentinel-ok: <reason>` in a comment on the write, or on a comment line above it, suppresses that one statement: // unset-sentinel-ok: busyTx carries the armed state, so 0 is a legal stamp here lastTxStart = Time::getMillis(); The reason is mandatory. A bare `unset-sentinel-ok`, or a colon with nothing after it, is reported instead of honoured - with a message saying so - so the only way to silence a site is to write down why it is safe. trunk-ignore still works, but this states the justification at the write and also applies when the script runs outside trunk. The marker is read from comment text collected during the same character-level pass that strips comments and literals, not by re-scanning the raw line. That is what keeps it out of reach of data: LOG_DEBUG("unset-sentinel-ok: ...") mutes nothing, because a string literal is not a comment. It is also consumed by the statement it was written for, so it cannot leak onto the next write - while still carrying across any number of intervening comment lines to the statement below, which is where a real justification wants to be written. Twelve fixtures added for the new behaviour: both comment styles, block and multi-line block comments, the bare form, the marker-in-a-string cases, and three leak cases. 35 total, all green, under bash 3.2 as well. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * lint: watch the separate-flag stamps too, with their exemption stated at the write The nine stamps whose armed state lives in a companion boolean were previously just absent from the rule's list, which meant the reasoning for leaving them out existed only as prose in a shell script. They are now listed and individually opted out at the write, naming the flag that actually carries the armed state: // unset-sentinel-ok: haveSample carries the armed state, so 0 is a legal stamp lastSampleMs = Time::getMillis(); The point is what happens later. If someone rewrites `if (haveSample && ...)` as `if (lastSampleMs && ...)`, the field has silently acquired the 0 contract; with the opt-out sitting at the write, the claim to re-examine is in front of whoever makes that edit instead of buried in bin/. Every exemption was checked against its real read sites before being written, and three candidates did not survive that check. They stay off the list, because listing one would mean stamping an opt-out over a claim that does not hold: * nagCycleCutoff. handleInputEvent reads `if (nagCycleCutoff != UINT32_MAX)` without consulting isNagging, so at that read the field is its own armed flag with UINT32_MAX as the sentinel - and the arm at ExternalNotificationModule .cpp:521 can land exactly there. skipZero() cannot help: it lifts 0 to 1 and leaves UINT32_MAX alone, which UptimeClock.h's own static_assert pins. There is also a live boot-state bug behind this - the in-class initializer is 1 while isNagging starts false - and fixing the read is a behaviour change that belongs in its own PR. * TouchScreenBase::_start. Overloaded as an event stamp AND a `+ 30000` suppression deadline compared by signed subtraction, so a near-zero value reads as "long ago" rather than "armed 30s out" and LONG_PRESS re-fires. skipZero() does not fix this one either: 1 reads as long-ago exactly as 0 does. It needs the stamp and the deadline held separately. * StoreForwardModule::retry_delay. No reads at all today, so nothing misbehaves yet; exempting it now would pre-approve the raw arm for whoever implements the retry its own comment promises. The rule is silent across src/ on this tree, and the header records all three rejections so the next person does not have to re-derive them. The self-test's negative fixture no longer uses nagCycleCutoff as its example of a safely unlisted field - that would have encoded the opposite of what the header says. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(time,lint): guard the recomputed tx_after, and judge one write at a time Two review findings, both real. setTransmitDelay() recomputes p->tx_after from a clamp of three candidates, and that recomputation was still raw. Two lines above it, `if (p->tx_after)` is the read that takes 0 as "no delay wanted", so a clamp landing on 0 drops the CSMA backoff and the packet goes out immediately instead of after its computed delay. The first arm site in this function was already guarded; this one was missed because the value is not a plain `now + delay` and so does not fit timerEndsAtMillis() - it takes skipZero() instead. The narrowing order matters here and is spelled out at the site: add_delay is unsigned long, 64-bit on the portduino host, so the clamp can exceed UINT32_MAX there. skipZero() on the wide value would pass 0x100000000 through as non-zero and the store to this uint32_t field would then truncate it back to the 0 being avoided, so the cast comes first. The lint rule judged each write by the wrong text. rhs was taken from the write to the end of the accumulated statement, so a neighbour on the same line decided the verdict - and it was wrong in both directions: rebootAtMsec = millis() + 5; shutdownAtMsec = Time::timerEndsAtMillis(10); the later helper call suppressed a genuine raw arm rebootAtMsec = otherDeadline; shutdownAtMsec = millis(); the later millis() reported a safe copy rhs is now cut at its own semicolon. Six fixtures cover it, including both cases above, two raw writes on one line, two helper writes on one line, and a statement split across lines, which must still see its whole right-hand side. 41 fixtures total, green under bash 3.2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Tom <116762865+Nestpebble@users.noreply.github.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-12 10:54:50 +00:00
6 people	fdb644e0b7	Fix millis() rollover in deadline, interval, and timestamp handling (#11291) ... * Add native test coverage for the UptimeClock monotonic seam src/UptimeClock.{h,cpp} shipped without a dedicated test suite. Port the six tests from the monotonic-time branch (test/test_time), retargeted to the renamed header. The wrap test crosses 0xFFFFFFFF via advanceTestMillis() rather than a second setTestMillis(): setTestMillis() sets clockSourceChanged, which makes getMillis64() rebase its accumulator and swallow the wrap. * NextHopRouter: fix 49.7-day millis() rollover in retransmission timing Resolves the "FIXME, handle 51 day rolloever here!!!" in NextHopRouter::doRetransmissions() by switching the retransmission-due comparison from plain unsigned <= to a signed-difference cast. The previous p.nextTxMsec <= now comparison silently breaks across the ~49.7 day millis() wraparound: pending retransmissions either stall for the remainder of the wrap window, or all fire simultaneously at the rollover boundary. Long-running router/infrastructure nodes do hit this in practice. The replacement (int32_t)(p.nextTxMsec - now) <= 0 is the standard Arduino/embedded idiom for rollover-safe deadline checks and behaves identically to the original for any non-wrap timing. * Address Copilot review: use unsigned half-range for rollover-safe retransmit check Review feedback from @Copilot on PR #10227: casting a uint32_t subtraction to int32_t is implementation-defined in C++ when the unsigned value exceeds INT32_MAX (even though it works on typical two's-complement targets). Switch to the fully well-defined unsigned half-range form: nextTxMsec is in the past-or-equal iff (now - nextTxMsec) has not wrapped past 2^31 ms. Future offsets < 2^31 ms wrap into the top half and read as 'not yet'. Same semantics as the signed-cast version on every two's-complement platform we care about, but portable to any conforming C++ impl. * Use monotonic time for airtime windows * Document monotonic airtime windows * Fix test_packet_signing sentinel that #10227's rollover fix inverts test_C3_invalid_repeated_packet_cannot_ack_or_change_retry_state parked a pending packet at nextTxMsec = UINT32_MAX to mean "never retransmit", then asserted that a rejected repeated packet leaves the retry state untouched. NextHopRouter::doRetransmissions() now tests whether a retransmit is due with an unsigned half-range compare, (uint32_t)(now - nextTxMsec) < 0x80000000u, so that retransmission timing survives the ~49.7 day millis() wrap. Under it now - 0xFFFFFFFF == now + 1, a small positive delta, so UINT32_MAX reads as ~1ms in the past: the retransmit fires and rewrites nextTxMsec, and the test failed with "Expected 4294967295 Was 6247". Use a representable future time instead. Production is unaffected either way - nextTxMsec is only ever written as millis() + d, and UINT32_MAX came from the test harness alone - so the sentinel is what needs to go, not the comparison. Special-casing UINT32_MAX in the retransmit path would keep a value that reads as "expired" under any wrap-correct compare. The value is held in a local because millis() advances across runPipelineIngress(), so recomputing it at the assertion would compare against a different number. Reported upstream on meshtastic/firmware#10227, whose branch predates this test. * Make Throttle time-injectable and add hasElapsed() Throttle backs ~94 call sites, which makes it the highest-leverage place in the tree to put the clock seam: reading Time::getMillis() instead of millis() in its three call sites turns all of them into time-injectable code at once, without touching any of them. The 32-bit millis() wrap is not otherwise reachable from a native test. The read is behaviour-preserving - Time::getMillis() returns millis() unless a test injects a clock - and the full native suite passes with it live. Also add hasElapsed(), the complement of isWithinTimespanMs(), because 51 of the 94 call sites are spelled !isWithinTimespanMs and read poorly. Its boundary is inclusive (>=) since isWithinTimespanMs uses <; both are documented. It deliberately does not treat lastExecutionMs == 0 as "never run": call sites pair that test with the interval check themselves, and absorbing a sentinel into the one helper every module depends on is exactly the value-overloading hazard being removed elsewhere. Migrating the existing !isWithinTimespanMs sites is cosmetic and deliberately left out of this commit. test/test_throttle/ covers window semantics, both boundaries, the complement identity, execute()'s first-run and throttled paths, and - the point of the exercise - a window opened before the wrap closing correctly after it, including at the 24h interval that is the longest in the tree. * Stop disarmed deadline sentinels reaching the comparison Two deadline variables encoded "inactive" as a magic value that only reads as "never" because the comparison against it is a naive millis() compare. Under any rollover-correct comparison both invert to "expired ~49 days ago", so they have to be untangled before those comparisons can be fixed. Power::reboot() set rebootAtMsec = -1 on platforms with no reboot implementation, intending "never fire". Every reader already treats 0 as the disarm value - powerCommandsCheck() tests `if (rebootAtMsec && ...)`, and AdminModule writes 0 to cancel - so -1 was both wrong and unnecessary. Use 0. Left as UINT32_MAX it would reboot-loop the moment the comparison is corrected. ExternalNotificationModule's nag window compared against nagCycleCutoff, which holds UINT32_MAX once stopped and 1 at boot. isNagging is the real armed flag, so test it first and short-circuit: a disarmed cutoff can no longer reach the arithmetic, while an idle module still takes the same sleep path that the boot-time value of 1 was relying on. Note this fixes the sentinel only. The comparison itself is still a naive `nagCycleCutoff < millis()` and remains on the list to convert. * Fix millis() rollover in every deadline and interval comparison Roughly 20 sites compared against millis() directly - `millis() > deadline`, `deadline < millis()`, `last + interval < millis()`. All of them break for about 24 days after the 32-bit millis() wrap: depending on which side of the wrap each value sits, the action either stalls for weeks or fires immediately and repeatedly. The longest affected interval is the 12 hour NTP renewal, a ~50x margin against the wrap, so none of these needed the range - only the correct comparison. Add Throttle::deadlinePassed(deadlineMs) for sites that store an absolute deadline they cannot re-express as "interval since an event". It uses the same unsigned half-range test as NextHopRouter::doRetransmissions() rather than introducing a competing signed-cast idiom, and unlike the signed cast it is defined for every input. Sites that do store an event use the existing isWithinTimespanMs / hasElapsed. Nothing gained new state. Because both helpers read Time::getMillis(), every converted site is now reachable from a native test that drives the clock across the wrap; the comparison itself is covered directly in test/test_throttle/. Sentinel handling is the reason this could not be a mechanical rewrite. The disarm convention is not uniform: 0 means "inactive" for rebootAtMsec, shutdownAtMsec, alertBannerUntil, fixHoldEnds, suppressUntilMs and touchResumeBlockUntilMs; 0 means "due now" for ntp_renew, which is forced to 0 at link-up; UINT32_MAX means "inactive" for nagCycleCutoff; and alertBannerUntil == 0 in isOverlayBannerShowing() means "show indefinitely". Every inactive marker is arithmetically far in the past, so a correct comparison fires on it - each site tests its sentinel before the arithmetic, and keeps the meaning it had. Two sites carried a second bug found on the way: BME680Sensor tested (stateUpdateCounter * STATE_SAVE_PERIOD) < millis(). With a 6 hour period and a uint16_t counter that product overflows uint32_t after about 198 saves, independently of the millis() wrap. It now measures the interval since the last save. EInkDynamicDisplay had `if (previousRunMs > millis()) return;` as a millis() overflow guard, which skipped rate limiting entirely for the whole post-wrap period - the bug it meant to prevent. Every check below it already goes through Throttle, so the guard is removed rather than fixed. MotionSensor's calibration countdown is converted to a signed delta rather than deadlinePassed, because it needs the remaining magnitude and not a boolean; that matches the already-correct check in the same file. * Remove getMillis64() and use Throttle for the NodeInfo reply window getMillis64() had exactly one caller and no callers in tests. It also carried obligations that made it the wrong shape for this firmware: a wrap accumulator in mutable statics, which is not ISR-safe, and which must be polled at least once every ~49.7 days or it silently misses a wrap and returns a time ~49 days short. Its one caller only wanted to know whether a 12 hour suppression window had elapsed - which Throttle answers correctly across the wrap without any accumulator. NodeInfoModule now stores Time::getMillis() in lastNodeInfoSeen and tests the window with Throttle::isWithinTimespanMs, so the map holds milliseconds rather than seconds derived from a 64-bit read. USERPREFS_NODEINFO_REPLY_SUPPRESS_SECS is user-overridable and now feeds a multiply by 1000, so a static_assert rejects any value too large to express in milliseconds instead of letting it wrap. clockSourceChanged goes too. It existed solely to rebase getMillis64()'s accumulator when a test swapped clock sources, and it made the wrap untestable through the injection API: setTestMillis() set the flag, so a wrap crossed by two setTestMillis() calls was swallowed. With the accumulator gone the flag has nothing to rebase, and the injection API is a plain settable clock. The three getMillis64 tests are dropped as they no longer describe anything. One test replaces them, pinning that advanceTestMillis() wraps past 0xFFFFFFFF rather than saturating, since the Throttle wrap tests rely on it. Also fix eviction in pruneLastNodeInfoCache(): it picked the entry with the smallest stored stamp, which is the wrong victim once some stamps sit on the far side of the wrap. It now evicts the largest elapsed time. * Add CI guard and docs rule against naive millis() comparisons Fixing the existing sites does not stop the next one being added. The millis-deadline-check job rejects millis() placed directly next to a comparison operator, in either order, anywhere in src/. It lives in test_native.yml alongside suite-count-check, which sets the precedent for a repo-hygiene guard that CI enforces and bin/run-tests.sh does not. The correct idioms all subtract before comparing, so none of them match the pattern. Line comments are stripped first, so documentation is free to name the broken form - as the guard's own comment and the coding conventions both do. Writing the check before finishing the sweep turned out to be worth it: it found roughly 14 sites that a by-hand audit of deadline variables had missed, including two extra nagCycleCutoff compares, both boot-screen timeouts, and a 6 hour sensor save interval that was also overflowing a uint32_t multiply. .github/millis-deadline-allowlist.txt covers the cases that are genuinely not deadline tests. Both current entries are uptime thresholds - "has the device been up N ms" - with no stored deadline and no event to measure from: a 30s button holdoff against phantom shutdown from floating pins, and a 10s window for the OEM boot logo. Each re-crosses its threshold once per wrap, which is harmless for boot-holdoff logic and not worth new state to avoid. Entries are keyed on file plus exact source text, without line numbers, so an edit above an entry does not silently invalidate it. Locally the guard reports 19 matches before the sweep and 2 after, both allowlisted. The Throttle bullet in the coding conventions is rewritten from "prefer Throttle for rate limiting" to "never compare against millis() directly", lists all four helpers with when to use which, names the CI guard, and documents the sentinel hazard with the rebootAtMsec = -1 case that would have become a reboot loop. Mirrored into AGENTS.md; CLAUDE.md gets a pointer row. * Trim rollover comments to what the code needs The comments added with the millis() rollover fixes carried too much of the investigation that produced them: how many sites were found, which document recorded them, what the old code used to do. That belongs in the commit history, not in the source, and some of it was already stale - Power::reboot() still described the check it disarms as "a naive millis() > deadline" when that comparison had been fixed in the same series. What stays is the non-obvious part at each site: which sentinel value the variable overloads and what it means there, since that differs between call sites and is what a correct comparison gets wrong. 0 means "not scheduled" for rebootAtMsec, "renew now" for ntp_renew, and "show indefinitely" in isOverlayBannerShowing(). Exposition is kept where it earns its place: the Throttle helpers, the uptime clock's note on why there is no 64-bit variant, and the tests. The Throttle docs lose only the site count and the "longest interval in the firmware" statistic, both of which would age badly; the range trade-off between the two forms is what a caller actually needs. Comments only - no code changed, verified by diff. * possible fixes * Address review feedback on the rollover fixes - BME680Sensor: checkpoint lastStateSaveMs after a successful write instead of at the interval test. The first save (IAQ accuracy >= 2) left it at 0, timing the next save from boot, and stamping before the write deferred the retry a full period when the write failed. Reads Time::getMillis(), the same clock Throttle compares against. - Throttle: add deadlinePassedAt(now, deadline) for loops that snapshot the clock once and test many deadlines; deadlinePassed() now delegates to it. NextHopRouter::doRetransmissions() uses it, replacing the inline half-range compare adopted from #10227 (nightjoker7) - same arithmetic, credited at the call site - and takes its snapshot from Time::getMillis() so setNextTx() deadlines and the due test cannot diverge under an injected test clock. - test_native.yml: set -euo pipefail in the millis-deadline guard, matching the sibling suite-count job. Without -e a partially failed scan could report "no violations" from truncated output. - test_packet_signing: build the not-due deadline from Time::getMillis() rather than millis(), so the test and the router read one clock. - test_throttle: cover deadlinePassedAt(), and correct a wrapped-value comment (0xFFFFFF00 + 400 is 0x00000090, not 0x00000094). Two review comments were declined: the AirTime mutex (every airTime-> caller runs in the single cooperative loop, WebServerThread included) and the MotionSensor 0-sentinel countdown (the calibration frame is only installed while a window is open). clod helped out here * Correct the described failure window of a naive millis() compare The comments and agent docs said a bare `millis() > deadline` "breaks for ~24 days after the wrap". That figure belongs to the fix, not the bug: it is the half-range limit of deadlinePassed(), which reads deadlines more than 2^31 ms ahead as already passed, and the range over which a UINT32_MAX sentinel reads as passed. The naive compare's actual failure is an inversion lasting only while the deadline sits on the far side of the wrap, so it is bounded by the interval: the action fires immediately and loses its wait, or blocks for about the wait it should have performed - days for the nRF52 flash-corruption backoff, one skipped cycle for a seconds-long retransmit timer. Comments and docs only; the ~24.8 day statements that correctly describe deadlinePassed()'s own range are left as they were. clod helped out here * Restore a monotonic uptime clock and consolidate the wrap counters Time::getMillisMonotonic() is the getMillis64() shape - a 32-bit wrap counter carried across reads - promoted to the shared timebase, with Time::getUptimeSecs() as the derived whole-seconds view. This deliberately reverses the earlier removal of getMillis64(), and the distinction matters: removal was right for a lazily-read accumulator with one rare caller, where a 49.7-day gap between reads silently swallowed a wrap. Here every read is the poll and AirTime::runOnce() guarantees one per second; the missed-wrap contract is pinned by a test rather than left as a footnote. Three private wrap counters collapse into it: - AirTime::syncNow() takes its seconds from Time::getUptimeSecs() and drops its lastSyncMsec checkpoint; window rotation is unchanged. - DeviceTelemetryModule loses refreshUptime()/uptimeWrapCount/uptimeLastMs; uptime_seconds comes from Time::getUptimeSecs(), which also removes the 0.296s-per-wrap truncation of (0xFFFFFFFF / 1000) * wraps. Its two interval checks move to Throttle::hasElapsed(). - HostMetricsModule's copies of those members were never read (its uptime comes from /proc/uptime) - deleted. Not ISR-safe (unguarded mutable carry): ISRs keep using getMillis(), which stays a pure read. Audited: no interrupt-context file reads getTime(), getValidTime(), or the new accessors. test/native-suite-count 44 -> 45: the bump for test_uptime_clock was lost in a branch history rewrite, leaving every later value off by one - run-tests.sh reports AMBER and CI's suite-count-check fails on the current push until this correction. * Anchor the wall clock in monotonic milliseconds getTime() computed elapsed-since-time-set as a 32-bit millis() delta, so a node that took time once and stayed up past 49.7 days reported a wall clock one full cycle in the past - and last_heard, rx_time, message and position stamps all inherited it. The anchor is now the 64-bit monotonic count (timeStartMsec -> timeStartMs64) and the elapsed term is computed in 64-bit, so the wall clock is exact at any uptime. All six anchor writers follow: the five hardware-RTC read branches and perhapsSetRTC(), which keeps a truncated 32-bit copy of the same instant for its Throttle-checked rate-limit stamps. The test seams anchor the same way. Two native regression tests drive getTime() across the wrap through the Time seam - one anchored before the wrap and read after it, one anchored after a counted wrap - with the test epoch derived from BUILD_EPOCH so the plausibility window cannot rot as the build date advances. * Stamp the rx_time placeholder in monotonic uptime seconds computeRxTimeStamp() stamped Time::getMillis() when the clock was untrusted, and reconcilePendingRxTimes() back-calculated with a 32-bit millis() delta - correct within one wrap, but a placeholder older than 49.7 days aliased to a small elapsed value and reconciled to a plausible-but-wrong recent epoch: the exact failure has_rx_time exists to prevent, reachable by an ordinary unattended router whose phone connects two months in. The placeholder is now Time::getUptimeSecs(). Both stamps come off the monotonic counter, so the elapsed term is exact at any age and the aliasing window is gone outright rather than widened. If elapsed somehow exceeds the epoch itself, the packet stays un-dated (absent, never wrong) instead of clamping to a pre-1970 value. Defence in depth: a placeholder that leaks needs ~50 years of uptime to cross MIN_PLAUSIBLE_EPOCH, where milliseconds took 18.3 days. The stream-API reconciliation tests keep their scenarios with the placeholder unit switched, and ScopedTimeFixture resets the monotonic carry so uptime seconds are deterministic per case. * Date nodes heard before the clock arrives, without polluting last_heard A node first heard while the wall clock was untrusted got no last_heard at all, and nothing backfilled it once time arrived - the phone showed "Last heard: unknown" for a node it had just announced. The arrival instant now waits in a RAM-only sidecar (NodeNum -> uptime seconds, 32 slots, reuse-oldest - the RouteHealth shape) and is converted to a real epoch on the clock-becoming-trusted transition, beside the existing rx_time reconciliation. last_heard itself never holds anything but a real epoch or 0: it persists to flash and the warm tier, where an uptime-relative value would be meaningless after reboot. The sidecar's write sites are updateFrom()'s no-trusted-clock path (the rx_time placeholder already carries the arrival instant, so this is a store, not a second clock read) and addFromContact's anti-eviction stamps, which previously wrote a bare getTime() - boot-relative seconds on a clockless node, the exact value lastHeardIsWallClock() exists to catch. Eviction ranking honours the stamps: heard-this-boot outranks every stored epoch, ordered among themselves, so a stamped contact is not the first victim. PhoneAPI re-reads last_heard at nodeinfo send time: a record prefetched before the clock became trusted can carry 0 while the store has since been backfilled, and re-reading at the pop makes handshake ordering (time-set vs node-list download) irrelevant. Backfill never moves last_heard backwards and skips the pathological elapsed-exceeds-epoch case. A node evicted to the warm tier before time arrives is still absorbed with last_heard 0 - same as before, bounded to the untrusted window. * Update the agent docs for the monotonic timebase The conventions bullet asserted there is deliberately no 64-bit millis; the monotonic uptime clock restored for timestamps changes that contract. State the split explicitly: Throttle for deadlines and intervals (no carry state), Time::getMillisMonotonic()/getUptimeSecs() for timestamps, polled by construction and not ISR-safe. * Publish the monotonic wrap carry from a single writer getMillisMonotonic() was a read-modify-write on two unguarded statics, and it is reached off the main loop: the nRF52 Bluefruit task via onFromRadioAuthorize() -> PhoneAPI::getFromRadio -> getValidTime(), and the portduino civetweb workers via the same path. Two readers interleaving inside the wrap window could each increment the carry, putting every uptime and wall-clock reading 2^32 ms ahead for the rest of the boot - a permanent ~49.7 day jump in rx_time, last_heard and ClientNotification.time. Readers no longer write. serviceMonotonic() publishes a snapshot behind a seqlock and is the only writer; a reader adds its own unsigned elapsed time to that snapshot, which is exact across the wrap, so it never inspects the boundary and cannot miscount it. The main loop publishes every iteration, so the once-per-49.7-days obligation now has the whole window of margin instead of resting on an instruction-wide race. AirTime was the guaranteed poller and is now a pure reader, so the two airtime wrap tests step the clock the way loop() does. The test clock itself is atomic so a suite can drive it from one thread while others read. * Re-arm the GPS ephemeris hold when none is in force The rollover sweep guarded the hold re-arm with `fixHoldEnds != 0 &&`, which reads like the sentinel rule but inverts this site. The comparison it replaced, `(fixHoldEnds + GPS_THREAD_INTERVAL) < millis()`, was always true when nothing was armed - that was the point, since 0 means "not holding" and so is a reason to arm. With the guard, a publish that cleared the hold without sleeping (the `shouldPublish && !tooLong && !holdExpired` path, which does not call down()) left hasValidLocation set and prev_fixQual non-zero, so no disjunct held: nothing re-armed, nothing published, and the receiver stayed powered at the 200ms poll until searchedTooLong() fired. State the question positively instead. fixHoldInForce() is the only place the sentinel is interpreted, and both of runOnce()'s decisions derive from it - the asymmetry is now visible rather than implied, since arming does not require a prior hold but expiring does. Its `!= 0` test is not redundant with the arithmetic: deadlinePassed() is an unsigned half-range test, so past 2^31 ms of uptime the sentinel reads as a deadline ~24.9 days in the future. Kept beside its caller rather than in a header; the native test build compiles GPS.cpp, so the suite declares the prototypes. Also converts the getACK() wait to isWithinTimespanMs(start, interval): it has both the start instant and the interval in hand, which gives the full 49.7-day range instead of 24.8 days ahead, and takes its anchor from Time::getMillis() so the wait is injectable. * Date the NodeInfo reply window in uptime seconds The 12h reply-suppression stamp regressed from wrap-immune 64-bit seconds to raw 32-bit milliseconds, and pruneLastNodeInfoCache() evicts only by node count and DB membership - never by age. A stable mesh under the node cap therefore keeps every stamp indefinitely, and once uptime passes 49.7 days an old one aliases back into the window: `now - stamp` computes as ~0 and a legitimate NodeInfo request goes unanswered for up to 12h. It self-heals and repeats once per wrap cycle. Store Time::getUptimeSecs() instead, which does not wrap for 136 years, and drop the millisecond conversion the previous shape needed. Entries past the window are now evicted too: they can only ever decide "don't suppress". N8-N11 cover the window from both sides, and N10 pins the regression - it needs a full 2^32 ms of uptime to elapse, not merely a crossing of the boundary, because that is when a millisecond stamp reads as "answered this instant". tearDown() now restores the injected clock and C14's region and TX bucket. A failing assertion aborts the test body, so restoring at the end of it leaked that state into every later case. * Update the agent docs for the single-writer clock and sentinel direction Two rules the preceding three commits changed. The monotonic clock is no longer maintained by whoever happens to read it: serviceMonotonic() is the only writer, readers are pure, and calling it from anywhere but the main loop reintroduces the double-count. The sentinel guidance gained the half it was missing. It named UINT32_MAX as a sentinel while prescribing an idiom that only covers 0, and it assumed the sentinel always means "suppress" - at the GPS fix-hold site it meant "fire", which is how that regression passed review looking like the rule. * Name the fix-hold expiry predicate and arm it from the injected clock holdJustExpired() gives the second reading of the fixHoldEnds sentinel a name beside the first, so both are pinned by test/test_gps_fix_hold/ and neither can be respelled at the call site. The old inline form could not be tested: written as a literal, its guard folds at compile time and the assertion asserts nothing. The arm site used bare millis() while the evaluation reads the Throttle clock; same value in production, but it kept that write out of reach of Time::setTestMillis(). Remap a deadline that lands on 0, which would otherwise read as no hold at all. * Share the extend formula between the clock's reader and writer getMillisMonotonic() and serviceMonotonic() carried byte-identical wrap arithmetic. A one-sided edit to either would drift the published carry from what readers report, so keep one copy. * Trim the NodeInfo dedup comment to the house limit * todo note for potential future imrpovments * fix some simple deadlines * Trim the hold-expiry test comment to the house limit * Fix non-blocking uptime publication and pre-clock recency edges (#29) * fix(time): avoid blocking monotonic readers * test(time): make paused-publisher check deterministic * fix(time): address review portability gaps * Init the eviction sentinel to the newest possible recency EvictionRecency{} is {0, false}, which evictionRecencyOlder() ranks as older than every candidate: without the oldestIndex/oldestBoringIndex guards nothing would ever be selected and a full node DB would stop evicting entirely. Init to the genuine maximum instead, so the sentinel is correct on its own. The index guards stay: two independent reasons the scan is right beats one. * Keep the deadline-guard check name branch protection matches The guard was widened to cover Time::getMillis() and unqualified getMillis(), and renamed to suit. Upstream branch protection matches required checks by name, so a rename means the old name never reports and merges block on a check that will never arrive. Widen the guard, keep the name; the descriptive text carries the broader scope. * Correct native-suite-count to 47 after the develop merge Upstream #11293 added test_nmea_wpl and took develop's count to 43; this branch had independently reached 46. Merging develop resolved the counter textually, keeping 46, while the directory set became the union of both sides at 47. The suite-count CI gate fails on the mismatch, and it gates the native test jobs, so the tests themselves were being skipped. * test(uptime): make the wrap fall where the comment says it does The concurrent-reader case started at 0xFFFFF000, leaving 0x1000 to the wrap, so the 0x800 advance annotated "cross the wrap" fell short and the wrap actually happened during the following 60s advance. Start at 0xFFFFF800 instead, so the first advance lands exactly on the wrap while the readers are running and the second is the ordinary time after it - the shape both comments already described. Total elapsed is unchanged, so the closing assertion still holds. * Respond to human comments * Did I ever tell you about the time I went to Shelbyville? I wore an onion on my belt, which was the style at the time. * Convert the I2S nag deadline develop dragged in The HAS_I2S_SPEAKER_NRF52 RTTTL block arrived from develop with a raw nagCycleCutoff >= millis(), which the deadline guard rejects. Use the same Throttle::deadlinePassed() form as the two sibling paths in this function. * Arm the LittleFS format guard with a flag, not a zero timestamp preFSBegin() runs in the first millisecond of boot, so millis() can legitimately return 0 there. Both readers of last_format_ms treated 0 as "nothing formatted this boot", which would skip the repeat-corruption escalation and let a dead flash reformat-loop instead of reporting FLASH_CORRUPTION_UNRECOVERABLE. * Note the single-thread contract on AirTime * Note the AirTime locking TODO, and tighten the thread note The two constant getters are not constrained, and getSilentMinutes() reads the buckets without rotating them, so "the accessors mutate" was not accurate. * trunk: ignore trufflehog false positives on millis-wrap test constants test_throttle and test_uptime_clock pin dense clusters of hex boundary constants (0xFFFFFF00u and neighbors) to exercise 32-bit millis() rollover. trufflehog's Lob detector stitches nearby hex literals into one candidate string, and the result happens to match a Lob API key shape - not a secret, just test fixtures. Same pattern already used for the gitleaks/nodedb-fixture false positive in this file. --------- Co-authored-by: nightjoker7 <mattdeering7@gmail.com> Co-authored-by: Clive Blackledge <clive@ansible.org> Co-authored-by: Benjamin Faershtein <119711889+RCGV1@users.noreply.github.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2026-08-12 16:49:17 -05:00
Andrew YongandBen Meadors	b71c1adb26	stm32wl: add hardware RTC support (rak3172) (#10961) ... * stm32wl: add hardware RTC support infrastructure Wires the STM32WL chip's internal RTC (running off the LSE 32.768kHz crystal) into meshtastic's existing time-of-day framework (perhapsSetRTC()/readFromRTC()), following the same pattern already used for I2C RTC chips (RV3028, PCF8563/85063, RX8130CE). LSE is started and polled manually before ever calling into the STM32RTC library, with our own bounded timeout - the library's own internal LSE startup path has no bounded fallback and hangs forever via Error_Handler() if the crystal never locks, so this is required for a board with a missing/faulty crystal to boot normally rather than hang. Gated behind a new HAS_LSE variant flag (currently unset everywhere, so this is inert until a variant opts in - see follow-up commit). Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * gps: qualify RTC.h includes to avoid case-insensitive filesystem collision with STM32RTC The stm32duino STM32RTC library (added to lib_deps in a follow-up commit) ships its own src/rtc.h. On case-insensitive filesystems (the macOS default), an unqualified #include "RTC.h"/<RTC.h> from any file outside src/gps/ resolves to the library's rtc.h instead of src/gps/RTC.h, since PlatformIO's LDF puts lib_deps include paths ahead of the project's own -Isrc/gps. Qualify every include as gps/RTC.h so it can't collide with any same-named header a future dependency might ship, regardless of filesystem case sensitivity. Purely mechanical, no behavior change. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl(rak3172): enable hardware RTC support Opts rak3172 into the HAS_LSE infrastructure added previously: sets STM32WL_LSE_DRIVE to a conservative default and pulls in the STM32RTC library. rak3172 has ~63KB flash headroom going in; build-verified at 76.7% flash usage after this change (up from a 73.8% baseline), well within budget. wio-e5 is not opted in here despite sharing the same STM32WLE5 chip - it's already at 96.8% flash usage today (GPS + I2C sensor support compiled in, unlike rak3172), leaving too little headroom to safely add STM32RTC without first trimming something else. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl: add docstrings for LSE/RTC setup functions Addresses CodeRabbit's docstring coverage check on PR #10961. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl: address CodeRabbit nitpicks on PR #10961 - Brace the single-statement HAS_LSE branch in perhapsSetRTC() to match the sibling readFromRTC() branch's style. - Quote the RTC.h include in PhoneAPI.cpp for consistency with every other qualified include site. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> --------- Signed-off-by: Andrew Yong <me@ndoo.sg> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2026-07-16 19:00:45 -05:00
Tom	3becaf2d95	emdashes begone (#10847)	2026-07-01 19:01:27 -05:00
3 people	e64d20548c	feat(eth): port mesh/http to RP2350 + W5500 (HTTP/80 + HTTPS/443) (#10573) ... * add watchdog to rp2xx0 * feat(eth-api): phase 0 skeleton — HTTP API server on TCP/80 (503 only) First step of porting mesh/http/ to RP2350 + W5500 (today ESP32-only). Phase 0 stands up the listener over the existing arduino-libraries/Ethernet stack (no Mongoose — its built-in TCP/IP would conflict with EthernetServer and break OTA/MQTT/NTP). Skeleton parses request line, logs it, replies 503. Real handlers (/api/v1/{info,fromradio,toradio}) come in later phases. - New mesh/eth/ethApiServer.{h,cpp} gated by HAS_ETHERNET_API - Wired into ethClient init+loop in parallel with existing ethHttpOTA (port 4244) - Enabled in both wiznet_5500_evb_pico2_e22p and pico2_w5500_e22 variants - Build impact on wiznet: +~9 KB flash (63.0% used), <1 KB RAM * feat(eth-api): phase 1 — implement /api/v1/{fromradio,toradio} with PhoneAPI Replace phase 0 skeleton with the real handlers that bridge the HTTP transport to PhoneAPI, mirroring mesh/http/ContentHandler (ESP32) semantics. - EthHttpAPI : public PhoneAPI (api_type=TYPE_HTTP, checkIsConnected=true), outside the MESHTASTIC_EXCLUDE_WEBSERVER gate so it builds on RP2350. - Minimal HTTP parser: method/path/query/Content-Length, no allocations beyond Arduino String, capped at 32 header lines x 256 bytes (anti-DoS). - OPTIONS preflight -> 204 with CORS + X-Protobuf-Schema. - GET /api/v1/fromradio[?all=true]: stream-write up to 64 protobufs from webAPI.getFromRadio(), Connection: close framing (HTTP/1.0 style). - PUT /api/v1/toradio: read Content-Length bytes (<=512), call webAPI.handleToRadio(), echo body back. - 404 default for unknown paths, 405 for wrong method, 400 for bad body. Validated e2e against wiznet @ 192.168.1.143: - OPTIONS /api/v1/fromradio -> 204 + correct CORS headers - PUT ToRadio{want_config_id=1} (2 bytes) -> 200 + echo - GET /api/v1/fromradio?all=true -> 200 + 3476 bytes (config+channels+nodeinfo) - GET /api/v1/nonexistent -> 404 unknown endpoint - OTA HTTP on port 4244 untouched and still responsive Build impact on wiznet_5500_evb_pico2_e22p: +2 KB flash (63.1%), +1.8 KB RAM (17.6%) vs phase 0. * fix(eth-api): move accept loop to dedicated OSThread (20ms tick) The API server was being polled from the Ethernet client Periodic which runs every 5s. Measured impact before this fix on wiznet @ 192.168.1.143: req 1: ttfb=6.458s total=6.509s (matches the 5s tick + handler overhead) req 2: connection refused (W5500 sockets exhausted) req 3: connection refused After: same hardware, same network, back-to-back requests: req 2: ttfb=0.287s req 3: ttfb=0.015s req 4: ttfb=0.022s req 5: ttfb=0.020s The web client (meshtastic/web served from localhost) was visibly stalling mid-handshake — it had pulled 109 nodes + 19 messages but channels and device info never arrived. With the periodic-only polling, every request takes 5s and the W5500's 4 hardware sockets fill up under the burst. EthApiServerThread mirrors the WebServerThread pattern from ESP32 mesh/http/WebServer.cpp: adaptive interval — 20ms when there's recent traffic, 100ms after 5s of idle, 500ms after 30s. Auto-registers with the OSThread scheduler on construction. ethHttpOTA still ticks from the periodic; left unchanged because OTA is one large transfer that tolerates the latency, and minimizing scope here to one behaviour change at a time. * refactor(eth-api): extract handlers to shared module via IStreamReadWrite Phase 2.0 of the TLS port — separate transport from request handling so the upcoming HTTPS server can reuse the exact same parser + routing logic. - New ethApiHandlers.{h,cpp}: Request, parser, CORS helpers, fromradio/toradio handlers, EthHttpAPI : PhoneAPI subclass. All driven by a single IStreamReadWrite interface that inherits Print (so client.print(...) keeps working transparently). - ethApiServer.cpp slimmed to ~90 LOC: now just the EthernetServer(80) + OSThread + an EthernetClientStream adapter that forwards reads/writes to the underlying EthernetClient. No behaviour change. Validated on wiznet @ 192.168.1.143: 5 curl requests TTFB 18-110ms (same as pre-refactor), protobuf round-trip PUT 200 + GET ?all=true 3499 bytes. Flash impact: +200 bytes (63.2%); RAM unchanged (17.6%). * feat(eth-tls): ECDSA P-256 self-signed cert generation for HTTPS API mbedTLS-based cert generation module that produces a SAN=IP self-signed ECDSA P-256 server certificate, persisted under LittleFS so subsequent boots reuse the same key. Generation runs once on a dedicated OSThread so the ECDSA keygen path (~430 ms) never blocks the Periodic stack or the Ethernet reconnect loop. - mbedTLS 3.6.2 sources compiled in via scripts/add_mbedtls_sources.py (BuildSources of pico-sdk/lib/mbedtls/library/*.c, all 108 files). MBEDTLS_USER_CONFIG_FILE injected as CPPDEFINES tuple in the script — build_flags shell escape mangles the embedded quotes on Windows. - src/mbedtls_user_config.h undefs MBEDTLS_HAVE_TIME, HAVE_TIME_DATE, TIMING_C, NET_C, FS_IO, PSA_ITS_FILE_C, PSA_CRYPTO_STORAGE_C, and defines MBEDTLS_NO_PLATFORM_ENTROPY (entropy_poll.c uses a raw platform check, not gated by a flag). - src/mesh/eth/ethCert.{h,cpp}: ECDSA P-256 keypair + cert with SAN(IP=current) using pico-sdk get_rand_64() directly as f_rng, bypassing mbedtls_entropy. DER buffers heap-allocated to keep the OSThread stack within budget. Cert + key + ip persisted under /prefs/eth_*.der so subsequent boots reuse the same identity. - Gated by HAS_ETHERNET_TLS_API. Standalone phase: generation only; HTTPS server (TCP/443) wired up in the follow-up commit. * feat(eth-tls-api): phase 2.2 — HTTPS server on TCP/443 reusing handlers Brings up an mbedTLS server on port 443 that defers to the ECDSA P-256 self-signed cert produced by [[ethCert]]. Reuses the HTTP request / response handlers from [[ethApiHandlers]] via the IStreamReadWrite interface — there is exactly one code path for routing / CORS / PhoneAPI integration regardless of whether the transport is plain TCP or TLS. EthTlsApiServerThread (OSThread): - Phase A: poll isEthCertReady() every 500 ms while the cert worker runs. Once true, parse cert chain + key, build ssl_config (TLS server, stream transport, default preset, VERIFY_NONE since we are the cert issuer), install own cert, run ssl_setup, bind tlsServer on 443. - Phase B: standard adaptive accept loop (20 / 100 / 500 ms tick), identical to the plain-HTTP server. Per-connection flow: 1. session_reset on the static ssl context (1 in-flight session — multi- session pool is Phase 3 if needed) 2. set_bio routes mbedtls I/O to two C callbacks (netSend / netRecv) that bridge to the live EthernetClient via the void* ctx 3. handshake loop (sync, blocking, with 10 s recv timeout) — mbedTLS errors are logged with mbedtls_strerror 4. wrap (ssl, client) in MbedTlsStream → handleApiClient(stream) 5. close_notify + client.stop Stack budget continues the Phase 2.1-bis discipline: every large buffer (ssl context, cert chain, pk_key, ssl_config) lives in BSS as a static global. The OSThread stack only holds the per-connection EthernetClient adapter and small mbedtls return codes. Validated on wiznet 192.168.1.143: - cert load-from-FS path: 13 ms (regen skipped on second boot) - cert gen path: 290 ms (first boot only) - ssl_setup chain: parse cert, parse key, ssl_config_defaults, conf_own_cert, ssl_setup all return 0 - end-to-end: curl -k https://192.168.1.143/api/v1/fromradio → 200 OK with application/x-protobuf, CORS, X-Protobuf-Schema headers, server initiates close_notify cleanly Footprint vs 2.0: - flash 70.5% → 87.3% (+220 KB for mbedtls_ssl + x509 server-side code) - RAM static 17.6% → 19.8% (+11 KB BSS for ssl_context + cert chain) - heap at runtime adds ~32 KB per active session (mbedtls in/out record buffers, default MBEDTLS_SSL_*_CONTENT_LEN=16384) Next: validate from Firefox direct (https://192.168.1.143 → warning accept → JSON visible), then from client.meshtastic.org hosted to confirm the mixed-content block is gone. * fix(eth-tls): add KeyUsage + EKU serverAuth and cap TLS 1.2 for browser compat Two browser-compat fixes that surfaced in Firefox validation: 1. Cert v1 only had Basic Constraints + SAN. NSS / Firefox refuse to treat a cert as a TLS server cert without an Extended Key Usage extension naming id-kp-serverAuth since 2023 — the error surfaces as a non-overridable 'Secure Connection Failed' with no 'Accept the Risk' path. Add KeyUsage(digitalSignature + keyEncipherment, critical) and ExtendedKeyUsage(serverAuth, critical). Bump cert/key/ip file paths to '_v2' so live boards regenerate on next start instead of loading a v1 cert that the browser silently refuses. 2. pico-sdk mbedtls defines MBEDTLS_SSL_PROTO_TLS1_3 in its default config, but the server-side 1.3 plumbing in this vendored build is incomplete: Firefox and openssl-3.5's s_client default to 1.3 and the handshake dies 4 ms in with MBEDTLS_ERR_ERROR_GENERIC_ERROR (-0x0001). curl/SChannel happened to default to 1.2 so it masked the issue earlier. Cap min/max_tls_version to TLS 1.2 — clients downgrade transparently and we keep the ECDHE-ECDSA + AES-GCM / CHACHA20-POLY1305 suites that already work end-to-end. Validated on wiznet 192.168.1.143: - cert v2 dumps with the 4 extensions visible (openssl x509 -text): BasicConstraints CA:FALSE, KeyUsage(critical) digitalSignature + keyEncipherment, ExtendedKeyUsage(critical) serverAuth, SAN IP. - openssl s_client (no version flag): downgrades to TLS 1.2, handshake completes, verify_return=18 (self-signed, expected). - Firefox: warning self-signed -> Advanced -> Accept Risk -> handshake OK in 632 ms with CHACHA20-POLY1305, request reaches handleApiClient and returns the protobuf body. * perf(eth-api): HTTP/1.1 keep-alive — one handshake per session, not per request Before this change /fromradio used 'Connection: close' framing (HTTP/1.0 style with no Content-Length), forcing each poll to redo the full TLS handshake. client.meshtastic.org needs ~80 sequential /fromradio polls during initial sync (one per packet from the config-replay state machine: MyInfo, channels, every Config_*, every ModuleConfig_*, every NodeInfo), so the user-visible load time was ~50 s of pure handshake overhead (80 requests * ~625 ms ECDSA P-256 each). Three coordinated changes: 1. handleApiClient() now loops on the same connection until the peer closes or parseRequest hits its 3 s idle timeout. requestsServed counter keeps the 'bad/timeout request' debug log from firing on the natural idle close after a keep-alive sequence. 2. handleFromRadio() buffers all packets into a std::vector before writing, so it can emit a real Content-Length and 'Connection: keep-alive'. Buffer is dynamic — common 1-packet response only allocates ~256 B; ?all=true keeps the 64-packet cap which tops out around 16 KB. handleToRadio + sendPreflight switched to keep-alive too (they already had real Content-Length). sendError keeps close — errors are terminal. 3. ethTlsApiServer netRecv RECV_TIMEOUT_MS dropped from 10 s to 3 s so mbedtls_ssl_read can't outlast the handler's idle deadline (a quiet browser leaving the socket open would otherwise wedge the OSThread for 10 s past the natural close). Measured on wiznet 192.168.1.143 against client.meshtastic.org: - one TLS handshake (641 ms, CHACHA20-POLY1305) - ~80 requests pipelined over the same session - full config + 40 NodeInfos in ~6-8 s (vs ~50 s before) - per-request latency post-handshake: ~10-25 ms - curl -kv with two URLs: 'Reusing existing https: connection', server returns 'Connection: keep-alive' explicitly. * fix(eth-tls): watchdog + Chrome compat — handshake stability across browsers Phase 3 keep-alive shipped a working Firefox flow but client.meshtastic.org loops + Chrome 'Test Connection' both rebooted the board. Four distinct issues; collectively they kept the OSThread inside serveClient() too long or spin-looping without yielding, and pico-sdk mbedtls' TLS 1.3 code path choked on Chrome's modern ClientHello. 1. Watchdog reset during keep-alive idle. Once a client drained the replay queue and entered 3 s poll mode, the OSThread sat inside netRecv()'s busy-wait waiting for the next request. Two consecutive 3 s waits plus prior handler time crossed the 8 s RP2350 hardware watchdog. Pet the watchdog inside netRecv()'s poll loop (every 2 ms) so a quiet client can never starve the watchdog. Same fix in the ethApiHandlers per-request yield path. 2. Cap session at 64 requests + yield() between. Defense-in-depth: a pathological client can't monopolize serveClient indefinitely; after the cap it just re-handshakes (~625 ms), still vastly cheaper than the per-request handshake we had before keep-alive. 3. TLS 1.3 code compiled out of mbedtls entirely (#undef MBEDTLS_SSL_PROTO_TLS1_3 in mbedtls_user_config). Capping max_tls_version=TLS1_2 at runtime is enough for Firefox / openssl (they downgrade cleanly), but Chrome's ClientHello carries TLS 1.3 extensions — post-quantum key shares, Encrypted ClientHello, etc. — that the vendored mbedtls 1.3 parser crashes on before the downgrade decision happens. Removing the 1.3 sources sidesteps the parser; ServerHello just announces TLS 1.2 and Chrome accepts. 4. netSend infinite WANT_WRITE spin. When W5500's TX buffer momentarily filled mid-handshake (Chrome draining slower than Firefox during ServerKeyExchange), EthernetClient::write() returned 0, our netSend returned MBEDTLS_ERR_SSL_WANT_WRITE without delay, mbedtls retried immediately, repeat at ~180k iter/sec until ... well, until the board's other threads got nothing done. Log signature: ret=-0x6880 tight-looping in the handshake iter trace. Rewrite netSend to block with delay(2) + watchdog_update() and a 3 s timeout — same shape as netRecv. Return MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY on disconnect (was incorrectly returning WANT_WRITE). Also added granular per-iter handshake logging gated on first 20 iters + every 50th after that, so any future regression localizes itself in COM9 without RTT JLink. Validated on wiznet 192.168.1.143: - Firefox: client.meshtastic.org full sync + idle poll stable (no reset during the previously-crashing 'replay drain complete' phase) - Chrome: 'Test Connection' accepts the cert prompt and connects - Edge: same as Chrome - openssl s_client default + tls1_2 forced: both negotiate TLS 1.2 with ECDHE-ECDSA + AES-GCM / CHACHA20-POLY1305, verify=18 (self- signed, expected) * chore(eth-tls): drop verbose debug logs from cert + TLS init paths The cert pipeline + TLS context init had step-by-step LOG_INFOs and ubiquitous Serial.flush() that were essential while diagnosing the Phase 2.1-bis stack overflow, the Chrome handshake crash, and the keep-alive watchdog reset. Once those bugs were fixed the logs just clutter COM9 on every boot. Kept on hand: - cert: 'loaded from FS', 'generating ...', 'generated N B in T ms', 'persisted to LittleFS', plus all LOG_ERROR / LOG_WARN paths - tls: 'server listening on TCP port 443', 'client connected from', 'handshake OK in N ms ciphersuite=', 'handshake failed -0xXXXX (...)', plus all init LOG_ERRORs Dropped: - cert: 'step 1/8 pk_setup' through 'step 8/8 copy key DER', 'thread woke', 'pipeline OK' (now silent on success) - tls: 'cert is ready, initializing', 'parsing cert chain', 'parsing key', 'ssl_config_defaults', 'conf_own_cert', 'ssl_setup', 'server worker scheduled', and the per-iter handshake trace - obsolete 'Optional mbedtls debug bridge' commented-out stub - all Serial.flush() that were added defensively for the debug-the-crash phase Bin shrinks ~40 KB (logs + format strings). Validated on wiznet 192.168.1.143: HTTP 200 round-trip works post-flash, no regression. * fix(eth): harden API/TLS build guards (review #10573) - ethApiHandlers.{h,cpp}: gate on (HAS_ETHERNET_API || HAS_ETHERNET_TLS_API) so the shared handleApiClient/IStreamReadWrite still compile for an HTTPS-only variant (the TLS server depends on them). - ethTlsApiServer.{cpp,h}, ethCert.{cpp,h}: require ARCH_RP2040 (they use Pico SDK get_rand_64()/watchdog_update()); the ethClient.cpp TLS include + call site are tightened to match, so enabling the flag on a non-RP2040 Ethernet target is a clean no-op instead of a cryptic build break. - clang-format (16) the touched eth files to satisfy Trunk Check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(eth): address review — listener/cert recovery + keep-alive framing - Rebind HTTP/80 and HTTPS/443 after a W5500 chip reset. reconnectETH() now tears the API/TLS servers down (deInitEthApiServer/deInitEthTlsApiServer) so the restart path recreates them; the singleton guards previously left both bound to dead sockets until reboot. The workers are kept (only the listener is dropped) so nothing is deleted from another thread's runOnce. - Stop the keep-alive loop after a handler error. /fromradio and /toradio return whether the connection may stay open; a 400/405/408 response (Connection: close) now ends the loop so unread body bytes can't be parsed as the next request. - Validate the cached cert/key before trusting it (parse both DERs), and clear the IP commit-marker before rewriting the pair and write it last, so a reset mid-persist regenerates instead of handing the TLS server a truncated or mismatched pair (which would hard-disable HTTPS). - Track a cert generation counter so the TLS server reloads and rebinds when a DHCP lease change regenerates the cert for a new IP (the SAN must follow, or browsers reject the new address). The cert worker is no longer one-shot. - Silence the SCons F821/E402 lint on add_mbedtls_sources.py to match the other extra_scripts. Validated on a W5500 board: a forced chip reset rebinds 80 and 443 without an MCU reboot; a truncated cached cert regenerates and HTTPS recovers; a simulated lease change reissues the cert with the new IP in the SAN. * fix(eth): clear cert readiness on regen failure + verify cached pair Follow-up to the review: - Reset ready_/certIp_ when ensureCertForIp() fails, so isReady() no longer reports true with empty material — otherwise a later TLS reload fails initTlsContext() and stays disabled instead of retrying on the next poll. - certKeyParse() now also checks the cached cert and key form a matching pair (mbedtls_pk_check_pair), rejecting an independently-parseable but mismatched cert/key (e.g. if the IP commit-marker survived a failed clear). Validated on a W5500 board: a valid cached pair still loads from FS, and a regenerated cert reloads cleanly. --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 05:51:14 -05:00
Carlos ValdesandClaude Opus 4.8	4f0e2dde98	feat: add Ethernet OTA support for RP2350/W5500 boards (#10136) ... * feat: add Raspberry Pi Pico 2 + W5500 + E22-900M30S variant Adds community variant for Raspberry Pi Pico 2 (RP2350, 4 MB flash) with external WIZnet W5500 Ethernet module and EBYTE E22-900M30S LoRa module (SX1262, 30 dBm PA, 868/915 MHz). Key details: - LoRa on SPI1: GP10/11/12/13 (SCK/MOSI/MISO/CS), RST=GP15, DIO1=GP14, BUSY=GP2, RXEN=GP3 (held HIGH via SX126X_ANT_SW) - W5500 on SPI0: GP16/17/18/19/20 (MISO/CS/SCK/MOSI/RST) - SX126X_DIO2_AS_RF_SWITCH: DIO2→TXEN bridge on module handles PA - SX126X_DIO3_TCXO_VOLTAGE 1.8: TCXO support via EBYTE_E22 flags - DHCP timeout reduced to 10 s to avoid blocking LoRa startup - GPS on UART1/Serial2: GP8 TX, GP9 RX - Reuses WIZNET_5500_EVB_PICO2 code paths for Ethernet init Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * pico2_w5500_e22: rename define and address review feedback Rename WIZNET_5500_EVB_PICO2 to PICO2_W5500_E22 so the variant-specific define matches the variant directory name and isn't confused with an on-board EVB SKU. Review fixes from PR #10135: - Gate the 10 s Ethernet DHCP timeout behind PICO2_W5500_E22 so other Ethernet builds keep the default 60 s behavior; apply the same timeout to reconnectETH() for consistency. - Drop the unused -D EBYTE_E22 flag; EBYTE_E22_900M30S already selects TX_GAIN_LORA / SX126X_MAX_POWER in src/configuration.h. - Rewrite "on-board W5500" comments to describe the external module. - Correct README TX_GAIN_LORA value (7, not 10) and drop the EBYTE_E22 row. * fix(pico2_w5500_e22): drop DEBUG_RP2040_PORT=Serial The arduino-pico framework hooks _write() when DEBUG_RP2040_PORT=Serial is set and dumps raw debug bytes onto USB CDC, corrupting any binary protobuf stream sent through StreamAPI (e.g. `meshtastic --port COMx`). The variant excludes BT and WiFi, so the primary client transport is Ethernet TCP via ethServerAPI — unaffected — but users who configure the node over USB serial would see protobuf decode failures from debug-byte interleaving. Removing the flag restores clean USB CDC. Debug output can still be enabled per-build by adding -D DEBUG_RP2040_PORT=Serial1 to redirect to UART0 instead of USB CDC. * feat: add Ethernet OTA support for RP2350/W5500 boards Adds over-the-air firmware update capability for RP2350-based boards with a WIZnet W5500 Ethernet module (e.g. pico2_w5500_e22). Protocol (MOTA): - SHA256 challenge-response authentication with a configurable PSK (override via USERPREFS_OTA_PSK; default key ships in source) - 12-byte header: magic "MOTA" + firmware size + CRC32 - Firmware received in 1 KB chunks, verified with CRC32, written via Updater (picoOTA), then device reboots to apply - Constant-time hash comparison prevents timing attacks on auth - 30s inactivity timeout + 5s cooldown after failed auth - Response codes 0x00-0x08 map 1:1 to OTAResponse enum Firmware side: - ethOTA.cpp / ethOTA.h: OTA TCP server on port 4243 - ethClient.cpp: wire initEthOTA/ethOTALoop into reconnect loop - main-rp2xx0.cpp: hardware watchdog (8s, paused during debug) - pico2_w5500_e22/platformio.ini: HAS_ETHERNET_OTA flag, filesystem_size bumped to 0.75m for OTA staging Host side: - bin/eth-ota-upload.py: Python uploader with progress and full result-code mapping (matches OTAResponse 0x00-0x08) * style(eth): clang-format ethOTA.cpp per repo .clang-format Reformat to the repo trunk clang-format config (IndentWidth 4, ColumnLimit 130). Resolves the Trunk Check 'Incorrect formatting' failure on PR #10136. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-06-18 16:28:10 +02:00
3 people	5b7a5b2c22	feat: add Raspberry Pi Pico 2 + W5500 + E22-900M30S variant (#10135) ... * feat: add Raspberry Pi Pico 2 + W5500 + E22-900M30S variant Adds community variant for Raspberry Pi Pico 2 (RP2350, 4 MB flash) with external WIZnet W5500 Ethernet module and EBYTE E22-900M30S LoRa module (SX1262, 30 dBm PA, 868/915 MHz). Key details: - LoRa on SPI1: GP10/11/12/13 (SCK/MOSI/MISO/CS), RST=GP15, DIO1=GP14, BUSY=GP2, RXEN=GP3 (held HIGH via SX126X_ANT_SW) - W5500 on SPI0: GP16/17/18/19/20 (MISO/CS/SCK/MOSI/RST) - SX126X_DIO2_AS_RF_SWITCH: DIO2→TXEN bridge on module handles PA - SX126X_DIO3_TCXO_VOLTAGE 1.8: TCXO support via EBYTE_E22 flags - DHCP timeout reduced to 10 s to avoid blocking LoRa startup - GPS on UART1/Serial2: GP8 TX, GP9 RX - Reuses WIZNET_5500_EVB_PICO2 code paths for Ethernet init Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * pico2_w5500_e22: rename define and address review feedback Rename WIZNET_5500_EVB_PICO2 to PICO2_W5500_E22 so the variant-specific define matches the variant directory name and isn't confused with an on-board EVB SKU. Review fixes from PR #10135: - Gate the 10 s Ethernet DHCP timeout behind PICO2_W5500_E22 so other Ethernet builds keep the default 60 s behavior; apply the same timeout to reconnectETH() for consistency. - Drop the unused -D EBYTE_E22 flag; EBYTE_E22_900M30S already selects TX_GAIN_LORA / SX126X_MAX_POWER in src/configuration.h. - Rewrite "on-board W5500" comments to describe the external module. - Correct README TX_GAIN_LORA value (7, not 10) and drop the EBYTE_E22 row. * fix(pico2_w5500_e22): drop DEBUG_RP2040_PORT=Serial The arduino-pico framework hooks _write() when DEBUG_RP2040_PORT=Serial is set and dumps raw debug bytes onto USB CDC, corrupting any binary protobuf stream sent through StreamAPI (e.g. `meshtastic --port COMx`). The variant excludes BT and WiFi, so the primary client transport is Ethernet TCP via ethServerAPI — unaffected — but users who configure the node over USB serial would see protobuf decode failures from debug-byte interleaving. Removing the flag restores clean USB CDC. Debug output can still be enabled per-build by adding -D DEBUG_RP2040_PORT=Serial1 to redirect to UART0 instead of USB CDC. * style(pico2_w5500_e22): apply trunk fmt — fixes Trunk Check - variant.h: clang-format 16.0.3 (drop manual #define alignment) - README.md: prettier + add `text` language to fenced code blocks (markdownlint MD040) - wiring.svg: svgo optimization Resolves the Trunk Check Runner failure on this PR (3 unformatted files + 8 markdownlint issues). No functional changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(pico2_w5500_e22): address review — move to rp2350/diy, generic guards Maintainer feedback from NomDeTom on PR #10135: - Move the variant from variants/rp2350/ to variants/rp2350/diy/ to distinguish DIY from prebuilt boards (matches the variants/*/diy/ pattern; still discovered via the existing variants/*/diy/*/platformio.ini glob in the root platformio.ini). - Replace the board-name macro PICO2_W5500_E22 in shared code with a generic capability macro USE_ARDUINO_ETHERNET, defined from variant.h. DebugConfiguration.h / ethServerAPI.h / ethClient.cpp no longer reference a board name. - Drop the architecture.h hook entirely: variant.h now defines PRIVATE_HW, which the existing `#elif defined(PRIVATE_HW)` branch already handles. No functional change. Build verified: pico2_w5500_e22 SUCCESS (RAM 19.2%, Flash 28.1%). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(pico2_w5500_e22): drop unoptimized wiring.svg to fix trunk fmt check --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2026-05-25 14:05:46 -05:00
5 people	8e99ffbe7e	ThinkNode M7 (#8077) ... * ThinkNode G3, ETH support WIP * ThinkNode G3, ETH support WIP * ThinkNode G3, ETH support WIP * ThinkNode G3, ETH support WIP * ThinkNode G3, ETH support WIP * ThinkNode G3, ETH support WIP * ThinkNode G3, ETH support WIP * rename variant and add guard macros * older G3 operational. M7 next. * Split out G3 and M7 to different variants. Completely new PCB design. The G3 stays on 'PRIVATE_HW' * Define button behaviour and use all of the device flash --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com> Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>	2026-05-11 11:33:13 -05:00
Philip LykovandBen Meadors	5319bc7c2c	Fix W5100S socket exhaustion blocking MQTT and additional TCP clients (#9770) ... The W5100S Ethernet chip has only 4 hardware sockets. On RAK4631 Ethernet gateways with syslog and NTP enabled, all 4 sockets were permanently consumed (NTP UDP + Syslog UDP + TCP API listener + TCP API client), leaving none for MQTT, DHCP lease renewal, or additional TCP connections. - NTP: Remove permanent timeClient.begin() at startup; NTPClient::update() auto-initializes when needed. Add timeClient.end() after each query to release the UDP socket immediately. - Syslog: Remove socket allocation from Syslog::enable(). Open and close the UDP socket on-demand in _sendLog() around each message send. - MQTT: Fix socket leak in isValidConfig() where a successful test connection was never closed (PubSubClient destructor does not call disconnect). Add explicit pubSub->disconnect() before returning. Made-with: Cursor Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2026-03-30 07:12:23 -05:00
3 people	f04746a928	Fix RAK4631 Ethernet gateway API connection loss after W5100S brownout (#9754) ... * Fix RAK4631 Ethernet gateway API connection loss after W5100S brownout PoE power instability can brownout the W5100S while the nRF52 MCU keeps running, causing all chip registers (MAC, IP, sockets) to revert to defaults. The firmware had no mechanism to detect or recover from this. Changes: - Detect W5100S chip reset by periodically verifying MAC address register in reconnectETH(); on mismatch, perform full hardware reset and re-initialize Ethernet interface and services - Add deInitApiServer() for clean API server teardown during recovery - Add ~APIServerPort destructor to prevent memory leaks - Switch nRF52 from EthernetServer::available() to accept() to prevent the same connected client from being repeatedly re-reported - Add proactive dead-connection cleanup in APIServerPort::runOnce() - Add 15-minute TCP idle timeout to close half-open connections that consume limited W5100S hardware sockets Fixes meshtastic/firmware#6970 Made-with: Cursor * Log actual elapsed idle time instead of constant timeout value Address Copilot review comment: log millis() - lastContactMsec to show the real time since last client activity, rather than always logging the TCP_IDLE_TIMEOUT_MS constant. Made-with: Cursor * Update src/mesh/api/ServerAPI.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Stop UDP multicast handler during W5100S brownout recovery After a W5100S chip brownout, the udpHandler isRunning flag stays true while the underlying socket is dead. Without calling stop(), the subsequent start() no-ops and multicast is silently broken after recovery. Made-with: Cursor * Address Copilot review: recovery flags and timeout constant Move ethStartupComplete and ntp_renew reset to immediately after service teardown, before Ethernet.begin(). Previously, if DHCP failed the early return left ethStartupComplete=true, preventing service re-initialization on subsequent retries. Replace #define TCP_IDLE_TIMEOUT_MS with static constexpr uint32_t for type safety and better C++ practice. Made-with: Cursor --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2026-03-19 08:37:39 -05:00
Martin Emrich	986d70db6a	Pioarduino preparation (#9223) ... * Resolve naming conflict of Syslog class with namespace * do not include libpax headers if pax counter is excluded * clean only top-level sdkconfigs, keep them in the variants directories * Fix code formatting	2026-01-12 09:52:39 -06:00
Jorropo	beb268ff25	Revert "add a .clang-format file (#9154)" (#9172) ... I thought git would be smart enough to understand all the whitespace changes but even with all the flags I know to make it ignore theses it still blows up if there are identical changes on both sides. I have a solution but it require creating a new commit at the merge base for each conflicting PR and merging it into develop. I don't think blowing up all PRs is worth for now, maybe if we can coordinate this for V3 let's say. This reverts commit 0d11331d18.	2026-01-04 05:15:53 -06:00
Jorropo	0d11331d18	add a .clang-format file (#9154)	2026-01-03 14:19:24 -06:00
Thomas Göttgens	b851b15a73	fix UDP builds on nRF (#7394) ... * fix UDP builds on nRF * fix rp2040 too	2025-07-20 23:13:50 +02:00
3 people	a9c9b96eb6	UDP for RAK4631 Eth Gw and the t-eth-elite. Solves #7149 (#7385) ... * UDP for RAK4631 Eth Gw and the t-eth-elite. Also enable IP output on Portduino. Solves #7149 * Copilot suggestion Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix portduino build * initialize local port --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2025-07-20 06:22:00 -05:00
7 people	4feaec651f	Unify the native display config between legacy display and MUI (#6838) ... * Add missed include * Another Warning fix * Add another HAS_SCREEN * Namespace fixes * Removed depricated destination types and re-factored destination screen * Get rid of Arduino Strings * Clean up after Copilot * SixthLine Def, Screen Rename Added Sixth Line Definition Screen Rename, and Automatic Line Adjustment * Consistency is hard - fixed "Sixth" * System Frame Updates Adjusted line construction to ensure we fit maximum content per screen. * Fix up notifications * Add a couple more ifdef HAS_SCREEN lines * Add screen->isOverlayBannerShowing() * Don't forget the invert! * Adjust Nodelist Center Divider Adjust Nodelist Center Divider * Fix variable casting * Fix entryText variable as empty before update to fix validation * Altitude is int32_t * Update PowerTelemetry to have correct data type * Fix cppcheck warnings (#6945) * Fix cppcheck warnings * Adjust logic in Power.cpp for power sensor --------- Co-authored-by: Jason P <applewiz@mac.com> * More pixel wrangling so things line up NodeList edition * Adjust NodeList alignments and plumb some background padding for a possible title fix * Better alignment for banner notifications * Move title into drawCommonHeader; initial screen tested * Fonts make spacing items difficult * Improved beeping booping and other buzzer based feedback (#6947) * Improved beeping booping and other buzzer based feedback * audible button feedback (#6949) * Refactor --------- Co-authored-by: todd-herbert <herbert.todd@gmail.com> * Sandpapered the corners of the notification popup * Finalize drawCommonHeader migration * Update Title of Favorite Node Screens * Update node metric alignment on LoRa screen * Update the border for popups to separate it from background * Update PaxcounterModule.cpp with CommonHeader * Update WiFi screen with CommonHeader and related data reflow * It was not, in fact, pointing up * Fix build on wismeshtap * T-deck trackball debounce * Fix uptime on Device Focused page to actually detail * Update Sys screen for new uptime, add label to Freq/Chan on LoRa * Don't display DOP any longer, make Uptime consistent * Revert Uptime change on Favorites, Apply to Device Focused * Label the satelite number to avoid confusion * Boop boop boop boop * Correct GPS positioning and string consistency across strings for GPS * Fix GPS text alignment * Enable canned messages by default * Don't wake screen on new nodes * Cannedmessage list emote support added * Fn+e emote picker for freetext screen * Actually block CannedInput actions while display is shown * Add selection menu to bannerOverlay * Off by one * Move to unified text layouts and spacing * Still my Fav without an "e" * Fully remove EVENT_NODEDB_UPDATED * Simply LoRa screen * Make some char pointers const to fix compilation on native targets * Update drawCompassNorth to include radius * Fix warning * button thread cleanup * Pull OneButton handling from PowerFSM and add MUI switch (#6973) * Trunk * Onebutton Menu Support * Add temporary clock icon * Add gps location to fsi * Banner message state reset * Cast to char to satisfy compiler * Better fast handling of input during banner * Fix warning * Derp * oops * Update ref * Wire buzzer_mode * remove legacy string->print() * Only init screen if one found * Unsigned Char * More buttonThread cleaning * screen.cpp button handling cleanup * The Great Event Rename of 2025 * Fix the Radiomaster * Missed trackball type change * Remove unused function * Make ButtonThread an InputBroker * Coffee hadn't kicked in yet * Add clock icon for Navigation Bar * Restore clock screen definition code - whoops * ExternalNotifications now observe inputBroker * Clock rework (#6992) * Move Clock bits into ClockRenderer space * Rework clock into all device navigation * T-Watch Actually Builds Different * Compile fix --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> * Add AM/PM to Digital Clock * Flip Seconds and AM/PM on Clock Display * Tik-tok pixels are hard * Fix builds on Thinknode M1 * Check for GPS and don't crash * Don't endif til the end * Rework the OneButton thread to be much less of a mess. (#6997) * Rework the OneButton thread to be much less of a mess. And break lots of targets temporarily * Update src/input/ButtonThread.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix GPS toggle * Send the shutdown event, not just the kbchar * Honor the back button in a notificaiton popup * Draw the right size box for popup with options * Try to un-break all the things --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * 24-hour Clock Should have leading zero, but not 12-hour * Fixup some compile errors * Add intRoutine to ButtonThread init, to get more responsive user button back * Add Timezone picker * Fix Warning * Optionally set the initial selection for the chooser popup * Make back buttons work in canned messages * Drop the wrapper classes * LonPressTime now configurable * Clock Frame can not longer be blank; just add valid time * Back buttons everywhere! * Key Verification confirm banner * Make Elecrow M* top button a back button * Add settings saves * EInk responsiveness fixes * Linux Input Fixes * Add Native Trackball/Joystick support, and move UserButton to Input * No Flight Stick Mode * Send input event * Add Channel Utilization to Device Focused frame * Don't shift screens when we draw new ones * Add showOverlayBanner arguments to no-op * trunk * Default Native trackball to NC * Fix crash in simulator mode * Add longLong button press * Get the args right * Adjust Bluetooth Pairing Screen to account for bottom navigation. * Trackball everywhere, and unPhone buttons * Remap visionmaster secondary button to TB_UP * Kill ScanAndSelect * trunk * No longer need the canned messages input filter * All Canned All the time * Fix stm32 compile error regarding inputBroker * Unify tft lineheights (#7033) * Create variable line heights based upon SCREEN_HEIGHT * Refactor textPositions into method -> getTextPositions * Update SharedUIDisplay.h --------- Co-authored-by: Jason P <applewiz@mac.com> * Adjust top distance for larger displays * Adjust icon sizes for larger displays * Fix Paxcounter compile errors after code updates * Pixel wrangling to make larger screens fit better * Alert frame has precedence over banner -- for now * Unify on ALT_BUTTON * Align AM/PM to the digit, not the segment on larger displays * Move some global pin defines into configuration.h * Scaffolding for BMM150 9-axis gyro * Alt button behavior * Don't add the blank GPS frames without HAS_GPS * EVENT_NODEDB_UPDATED has been retired * Clean out LOG_WARN messages from debugging * Add dismiss message function * Minor buttonThread cleanup * Add BMM150 support * Clean up last warning from dev * Simplify bmm150 init return logic * Add option to reply to messages * Add minimal menu upon selecting home screen * Move Messages to slot 2, rename GPS to Position, move variables nearer functional usage in Screen.cpp * Properly dismiss message * T-Deck Trackball press is not user button * Add select on favorite frame to launch cannedMessage DM * Minor wording change * Less capital letters * Fix empty message check, time isn't reliable * drop dead code * Make UIRenderer a static class instead of namespace * Fix the select on favorite * Check if message is empty early and then 'return' * Add kb_found, and show the option to launch freetype if appropriate * Ignore impossible touchscreen touches * Auto scroll fix * Move linebreak after "from" for banners to maximize screen usage. * Center "No messages to show" on Message frame * Start consolidating buzzer behavior * Fixed signed / unsigned warning * Cast second parameter of max() to make some targets happy * Cast kbchar to (char) to make arduino string happy * Shorten the notice of "No messages" * Add buzzer mode chooser * Add regionPicker to Lora icon * Reduce line spacing and reorder Position screen to resolve overlapping issues * Update message titles, fix GPS icons, add Back options * Leftover boops * Remove chirp * Make the region selection dismissable when a region is already set * Add read-aloud functionality on messages w/ esp8266sam * "Last Heard" is a better label * tweak the beep * 5 options * properly tear down freetext upon cancel * de-convelute canned messages just a bit * Correct height of Mail icon in navigation bar * Remove unused warning * Consolidate time methods into TimeFormatters * Oops * Change LoRa Picker Cancel to Back * Tweak selection characters on Banner * Message render not scrolling on 5th line * More fixes for message scrolling * Remove the safety next on text overflow - we found that root cause * Add pin definitions to fix compilation for obscure target * Don't let the touchscreen send unitialized kbchar values * Make virtual KB just a bit quicker * No more double tap, swipe! * Left is left, and Right is right * Update horizontal lightning bolt design * Move from solid to dashed separator for Message Frame * Single emote feature fix * Manually sort overlapping elements for now * Freetext and clearer choices * Fix ESP32 InkHUD builds on the unify-tft branch (#7087) * Remove BaseUI branding * Capitalization is fun * Revert Meshtastic Boot Frame Changes * Add ANZ_433 LoRa region to picker * Update settings.json --------- Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Jason P <applewiz@mac.com> Co-authored-by: todd-herbert <herbert.todd@gmail.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-06-21 06:36:04 -05:00
Eric SeveranceandBen Meadors	50b7d6a0f7	Establish MQTT connection only from MQTT::runOnce (#6057) ... Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2025-02-14 20:32:41 -06:00
3 people	9d710041c4	Add MESHTASTIC_EXCLUDE_SOCKETAPI (#5729) ... MESHTASTIC_EXCLUDE_SOCKETAPI disables the API Server when set. Co-authored-by: mverch67 <manuel.verch@gmx.de> Co-authored-by: GUVWAF <thijs@havinga.eu>	2025-01-03 09:01:10 +08:00
Bernd Giesecke	9af8c58c40	Add Ethernet RAK13800 support to RAK11310 (#5707)	2024-12-31 13:36:49 +01:00
Ben Meadors	982190936d	More log reductions. I'll probably stop now ;-) (#5263)	2024-11-06 21:03:25 +08:00
Ben Meadors	f769c50fa5	More reduction (#5256) ... * Now with even fewer ings * Ye * Mo * QMA6100PSensor	2024-11-04 19:15:59 -06:00
Ben Meadors	50dac38a1b	Pass#2: Lots more savings in logs and string reduction surgery (#5251) ... * Pass#2: Lots more savings in logs and string reduction surgery * Don't need Thread suffix either * Warn	2024-11-04 12:16:25 -06:00
Thomas GöttgensandBen Meadors	05e4a639a1	remove newline from logging statements. (#5022) ... remove newline from logging statements in code. The LOG_* functions will now magically add it at the end. --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2024-10-14 15:11:43 +11:00
zerolint	d72a836e07	RAK13800 Ethernet improvements ... Fixes (#3618) by allowing more time for slower requests. Resolve Syslog not maintaining client causing issues on RAK13800. Resolve Ethernet static IP setting subnet as gateway IP. Reduce comment and log message ambiguity around API. Remove duplicate #if !MESHTASTIC_EXCLUDE_WEBSERVER block.	2024-09-06 11:45:43 +02:00
geeksville	f59d98482f	Fix build when HAS_NETWORKING is false on nrf52 (#4237) ... (tested on a rak4631 by setting HAS_ETHERNET false when shrinking image)	2024-07-07 07:08:49 -05:00
Jim WhitelawandBen Meadors	acc32916c3	Add multiple configuration options for a minimized build (GPS,WiFi,BT,MQTT,Screen). (#3469) ... Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2024-03-25 06:33:57 -05:00
Andre KandBen Meadors	4056d34bed	fix: ipv4_config byte order already little endian (#3073) ... Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2024-01-17 18:14:44 -06:00
Ben Meadors	6e96216ba3	MQTT client proxying (#2587) ... * WIP on MQTT proxy message queue * Fix copy paste goof * Progress on uplink * Has packets * Avoid trying to connect if we're proxying * Pointer correctly * Remove wifi guards * Client proxy subscribe * Fixed method that got bababababorked somehow... personally I blame CoPilot * Short circuit logic * Remove canned settings * Missed some stuff in the move * Guard pubsub client for non-networked variants * Has networking guard * else * Return statement for fall-thru * More gaurd removals * Removed source filters. No wonder I was confused * Bounding * Scope guard around else and fix return * Portduino * Defs instead * Move macro up to actually fix portduino * Size_t * Unsigned int * Thread interval * Protos * Protobufs ref	2023-07-08 20:37:04 -05:00
Michel Jung	194833d77f	Fix static ethernet config (#2544) ... With static ethernet config, `status` stayed `0` which let the function return without setting `ethEvent`. Therefore, `reconnectETH` was never called and network services were never started. Also, the RAK4631 uses little endian, which is why the IP addresses need to be converted before setting them. Fixes #2543	2023-06-06 19:26:13 -05:00
Thomas Göttgens	829318046a	rsyslog is working	2023-02-05 00:11:00 +01:00
Thomas Göttgens	661894f9f9	fix nRF52 and linter errors.	2023-01-31 14:20:04 +01:00
Thomas Göttgens	104ffe36b2	Merge branch 'master' into rsyslog-client	2023-01-30 19:06:42 +01:00
Thomas Göttgens	6fdb93cd16	re-add namespacing in protobufs. Let's see what i missed. Portduino likely ... ... Checking in generated on purpose.	2023-01-21 21:23:24 +01:00
Thomas Göttgens	51b2c431d9	trunk roundhouse kick	2023-01-21 21:23:24 +01:00
Ben Meadors	ff029ad752	Formatting	2023-01-18 15:37:23 -06:00
Thomas Göttgens	f86eef66c8	Simple UDP calls, if wired up a fair bit of this can go again. ... this is preliminary work	2023-01-18 21:35:51 +01:00
Thomas Göttgens	5fd00b2538	- make a template class for API Server ... - Skip a lot of duplicate code - add a hexDump output - might come in handy - refactor directory names - remove unused debugOut that was generating template errors	2023-01-09 17:03:52 +01:00
Tim Wilkinson	0b9c8e62ea	Make sure we dont use a multicast MAC for ethernet	2023-01-05 22:31:57 -08:00
Ben Meadors	021fc8235c	Renamed eth_mode to address_mode	2023-01-03 06:24:47 -06:00
Ben Meadors	f1cdfd163d	Replaced all of the logging with proper log levels	2022-12-30 10:27:07 -06:00
Ben Meadors	0b5cae5393	Log levels refactoring	2022-12-29 20:41:37 -06:00
Ben Meadors	82bcd391cd	Fix those refs	2022-10-26 16:50:58 -05:00
Thomas Göttgens	d5a9e3114a	Update ethClient.cpp	2022-10-26 18:37:51 +02:00
Thomas Göttgens	7417729482	debug removed	2022-10-26 18:37:20 +02:00
Thomas Göttgens	d5c407c098	remove accidental commit	2022-10-26 18:36:41 +02:00
Thomas Göttgens	497c0b7a47	don't rush failed time updates	2022-10-26 17:38:53 +02:00
Thomas Göttgens	b7ebe03ca8	API Server and DHCP Lease Management	2022-10-26 11:09:59 +02:00
Thomas Göttgens	a66538fe55	MQTT is working over ethernet	2022-10-26 00:07:02 +02:00
Thomas Göttgens	f3fee5f4fb	first murmurs of ethernet support	2022-10-22 16:29:50 +02:00