Commit Graph
63 Commits
Author SHA1 Message Date
Ben Meadors 905482ccce fix(serial): don't sleep forever with pending PhoneAPI output on UART consoles (#11500)
* fix(serial): don't sleep forever with pending PhoneAPI output on UART consoles

Since #11164 bounded the stream drain, a config dump can end a dispatch
with output still queued. On UART-console ESP32 boards runOnce() then
returns INT32_MAX with no RX pending, and neither rxInt() nor
onNowHasData() fires for the remaining output, so the download wedges
mid nodeinfo stream until the client happens to send a byte.

Add StreamAPI::hasPendingOutput() (transport-retained frame or queued
PhoneAPI data) and have SerialConsole::runOnce() short-poll (<=25ms)
while it holds instead of sleeping INT32_MAX. The #11164 write budget
is unchanged; idle sleep behavior with a drained queue is unchanged.
The retained-frame probe also covers the ESP32-S2 USB-CDC branch,
which takes the same INT32_MAX path.

* test(serial): restore scratch NodeDB via tearDown, trim comments to house style

A failed TEST_ASSERT longjmps out of a Unity test without running
destructors, so RAII cannot restore the swapped nodeDB pointer; install
the scratch NodeDB explicitly and restore/delete it in tearDown(),
which runs after every test outcome. Also shorten the new comments to
the two-line house limit.
2026-08-14 10:41:44 -05:00
Tom de6b23190a Test suite rebuild (#11322)
* docs(nodedb): make the native node cap unambiguous

The native node cap was stated in four places that disagreed, and the disagreement
already caused a wrong diagnosis: a saturated 200-node database looked arithmetically
impossible because the cap had been read as 248, computed from a header that does not
apply on this platform. The real value is 198.

On portduino MAX_NUM_NODES is not a compile-time constant at all - the variant defines
it as `portduino_config.MaxNodes`, resolved at runtime, default 200 and settable per
host with `General: MaxNodes`. variant.h is reached before mesh-pb-constants.h, so that
header's ARCH_PORTDUINO branch never fires and its plausible-looking 250 is dead code.

- #error-guard the dead branch rather than leave a wrong number where people grep. The
  guard found a real defect: seven translation units reach mesh-pb-constants.h without
  configuration.h (SerialConsole.cpp, StreamAPI.cpp, PacketAPI.cpp, ServerAPI.cpp,
  PiWebServer.cpp, ServiceEnvelope.cpp, MeshtasticOTA.cpp, and test/TestUtil.cpp), so
  each was compiling with a different MAX_NUM_NODES - and therefore a different
  PACKETHISTORY_MAX - than the rest of the build. Each now includes configuration.h
  first. It cannot be included from mesh-pb-constants.h itself: that reaches
  SerialConsole.h through DebugConfiguration.h and closes a cycle.
- Name the bare 250 in getMaxNodesAllocatedSize() NODEDB_MIGRATION_LOAD_CEILING. It is a
  decode allowance for files written by larger-cap firmware, not a cap, and it read like
  one.
- Fix docs/node_info_stores.md, which named the wrong source and a "10-250" range that
  is wrong for native, and the copilot-instructions tunables line that said "portduino
  250".

* test(harness): give each suite its own scratch HOME and report leftovers

Native suites shared one directory. Every suite that constructs a NodeDB loads and
saves ~/.portduino/default/prefs/ - nodes.proto, config.proto, channels.proto,
module.proto, device.proto, warm.dat, transmit_history.dat - and nothing cleared it,
so state leaked suite -> suite within a run and run -> every run after it. A test run
could also rewrite a real meshtasticd node database on the same machine.

Per-run isolation does not fix this: the leak is generated inside a single run, so the
boundary has to be per suite.

bin/pio-test-isolate.sh runs each suite in its own scratch $HOME, registered as
test_testing_command for env:native and env:coverage so a bare `pio test` and CI get the
same boundary, not just bin/run-tests.sh. It runs the binary unchanged and exits with its
exit code, so PlatformIO's pass/fail is untouched. Overriding HOME here rather than
around `pio` also sidesteps the blocker that a bare HOME= breaks pio's own
~/.platformio/penv/bin/pio lookup.

Leftovers are reported as a second axis, PASS/FAIL x CLEAN/DIRTY, because an unintended
write has no matching assertion by definition - nobody writes TEST_ASSERT for a save they
do not know is happening. The harness asserts it from outside, so it applies to every
suite without the author opting in.

- Only the *set of changed paths* is asserted, never contents. Hashes answer the boolean
  "did this change?" and nothing more; content baselines over protobuf bytes would churn
  on every NodeInfoLite field added, which is how snapshot suites become noise.
- Deliberate writes are declared in test/state-manifest.tsv - one central file, suite /
  flags / mandatory reason. run-tests.sh prints the opt-out count on every run.
- Granularity follows the state flag, so the two ship together: per-test by default
  (TestUtil redefines RUN_TEST to checkpoint after each test, naming the exact test that
  dirtied things), suite boundary for state=per-suite, where carrying state across test
  cases is the declared behaviour.
- A declared write that does NOT happen is reported as MISSING, not folded into DIRTY. It
  catches silently broken persistence; a warning for now, since some are conditional.
- Graded AMBER, not RED. With isolation in place DIRTY means "undeclared", not
  "dangerous", and a check that lands red on day one gets switched off.

Guard the guard, both halves: state_assert_empty() refuses to run a suite against a
sandbox that is not empty (otherwise the after-diff measures against the wrong baseline
and reports CLEAN while meaning nothing), and bin/test-state-check.sh drives the real
wrapper with fixtures asserting CLEAN / CLEAN / DIRTY / MISSING plus both directions of
the empty assertion. A checker that silently matches everything would otherwise pass
forever.

--write-manifest proposes entries for a human to paste and justify; it never applies
them, and neither does CI.

* test(harness): stop reporting Unity's exit code as a signal

A native suite ends in exit(UNITY_END()), and UNITY_END() returns the failure count.
PlatformIO's native runner reads that non-zero exit code as a POSIX signal number, so
four failures print "Program received signal SIGILL", five print "SIGTRAP", and the suite
is classified [ERRORED] rather than [FAILED].

There is no crash. The signal name tracks the failure count and nothing else - it moved
SIGILL -> SIGTRAP when a diagnostic probe added a fifth failure - and it cost hours of
hunting a memory bug that did not exist, on an env (native) that carries no sanitizer at
all. It also explains the phantom extra test case in the totals: the runner adds a
synthetic entry for the signal it thinks it saw.

run-tests.sh now says so inline whenever a signal line appears, and the three
agent-facing docs say it too.

* test(admin): isolate NodeDB and globals per test

setUp() did `if (!nodeDB) nodeDB = new NodeDB();` and never deleted it, so 83 of the 85
tests shared one never-reset database and never restored config, owner, devicestate or
channelFile. The fixture that does restore them was opt-in and armed by exactly two
tests. The setUp comment claiming the rest "set their own config/region state and are
unaffected" was not true - the admin handlers under test write all four globals.

Route every test through the fixture instead: setUp saves the globals and installs a
fresh NodeDB, tearDown restores and deletes it. The two tests that armed it themselves no
longer need to.

All 85 pass, so nothing was silently relying on the shared state. It costs about 7% of
the suite's runtime (a NodeDB construction is a loadFromDisk plus, with a region set, key
generation) - worth paying to write the phase 3 tests against a clean fixture rather than
83 tests' residue.

Also cap the per-test attribution in the run summary at five entries; the full list stays
in the suite's sandbox.

* test(fs): cover the bounded file-manifest walk

getFiles() runs on every phone sync via STATE_SEND_FILEMANIFEST, and nothing asserted any
of its bounding behaviour. It does execute unasserted from test_stream_api's handshakes,
but the cap, the depth limit, the wasLimited paths, overlong-path rejection and capacity
release were all unguarded.

Eight tests, all describing what the code does today: today's code is already correct
here, since #10778 landed the by-reference collectFiles(), the 64-entry cap, the strlcpy
bounds and the swap-idiom release. They pass on arrival, which is the point - this is the
baseline a later change has to leave alone.

Two things they do not cover, and cannot:

- Moving reserve() outside the __cpp_exceptions guard. Exceptions are on natively, so the
  #else branch is not compiled. The suite's job there is to prove that change alters
  nothing observable.
- The file.name() null guard. No in-tree backend returns null; the guard is defensive.

The manifest-release test pins the swap idiom rather than calling
PhoneAPI's releaseFilesManifest(), which is file-local. It asserts capacity() == 0, not
just size() == 0 - a size-only check passes on clear(), which is the bug #7924 shipped.

Suite count 43 -> 44, recounted against the directories rather than copied.

* test(admin): assert node-DB metadata saves skip the radio reload

set_favorite_node, set_ignored_node and toggle_muted_node each persist a NodeInfoLite bit
and nothing else. MeshService::reloadConfig() gates its region re-derivation and
configChanged notification on saveWhat & (SEGMENT_CONFIG | SEGMENT_CHANNELS), so a
SEGMENT_NODEDATABASE-only save already skips the live radio reconfigure.

Pure characterization - all three pass on develop. Worth pinning because that reconfigure
is the path implicated in the WisMesh Tag favourite-node crash, and develop asserts
nothing about it: widening the saveWhat mask or reordering the check would currently go
unnoticed.

Ported from the config-save series along with ConfigChangedCounter (an Observer<void *>
counting configChanged notifications, the only externally visible signal that the reload
branch was taken) and TEST_NODE_NUM. They join the existing suite, so no suite-count
change.

* refactor(menu): extract the mute toggle into a named function

The node menu's mute action was inline in a banner-callback lambda, and that lambda only
ever runs via screen->showOverlayBanner() - which is why nothing in MenuHandler.cpp was
reachable from a test. Lift the `selected == Mute` branch into
menuHandler::toggleNodeMuted(uint32_t) and call it from the lambda.

Behaviour-neutral by construction: same statements, same order, same bare saveToDisk().
The null check moves into the function, so the call site no longer needs its own lookup.
Verified by the native build and suite; the byte-identical-image check on a
headroom-constrained nRF52 board was not run locally - CI's firmware-size comment covers
it.

Three tests come with it, all describing today's behaviour:

- the bit flips both ways and no configChanged fires (develop never calls reloadConfig on
  this path);
- an unknown node is a no-op rather than a write;
- and the segment mask. Flipping one NodeInfoLite bit currently rewrites all five
  segments via bare saveToDisk(). That is asserted deliberately, with the comment naming
  it as characterization of a known defect: a pending fix narrows it to
  SEGMENT_NODEDATABASE, and when it lands this assertion is expected to change, which
  makes the improvement visible in the diff instead of silent.

saveToDisk() is not virtual, so the mask is observed through its effect - remove the five
prefs files, toggle, and see which reappear.

* docs(test): make every suite count a pointer to the canonical one

test/native-suite-count is the registered total and is machine-checked against test/test_*
on every full run and by the suite-count-check CI job. Every other statement of the count
is a copy that drifts: copilot-instructions said 12, AGENTS.md said 19, and the real
number is 44.

Replace both literals with a pointer to the file, say explicitly that no document should
state the count as a literal, and reframe the two suite listings as descriptions rather
than inventories - they carry per-suite information the count does not, so they stay, but
nothing should infer completeness from their length. Register the new FS suite in both.

* test(harness): randomise suite order, reproducibly

Landed last, deliberately. Randomising an order-dependent suite set does not find bugs so
much as convert a silent pass into intermittent red, and the first instinct is to revert
the randomisation rather than fix the coupling. Phases 1-2 removed the coupling; this
keeps it removed.

Both runners previously hid order dependence behind a fixed order that happened to differ
between them, and neither order was chosen: CI's area rules put admin first, PlatformIO's
local discovery is reverse alphabetical and put it last. CI was green by accident.

- bin/run-tests.sh --shuffle / --seed <n>. The seed defaults to HEAD's short SHA: one
  order per commit, so a red is replayable and attributable to the diff instead of flaky,
  while the project keeps exploring orders. Printed at the start and carried into the
  RESULT line, so a verdict is replayable from that line alone; the full order is printed
  on failure, because for an order-dependent failure the order is the diagnostic.
- The shuffle is a Fisher-Yates over a MINSTD generator rather than awk's rand(), whose
  sequence differs between gawk and mawk. A seed that does not reproduce the same order on
  another machine is not a seed.
- Shuffling needs one `pio test -f <suite>` invocation per suite - PlatformIO orders by
  its own os.walk() over test/ and filters only select - which measures at about 4.7s per
  suite of extra startup.
- CI shuffles its area order, seeded from GITHUB_SHA and printed with the command to
  replay it locally. Intra-area order stays PlatformIO's; controlling it there would mean
  per-suite invocations, which is a cost worth deciding separately.

Also records the 16 measured entries in test/state-manifest.tsv, each with its reason,
taken from a full run's --write-manifest output rather than guessed.

* test(default): cover the region-throttle interval overload

getConfiguredOrDefaultMsScaled(configured, default, nodes, TrafficType) is the overload
every telemetry and position module actually calls, and nothing referenced TrafficType
anywhere under test/. All four of its behaviours were unguarded: the no-region guard, the
throttle <= 1 short-circuit, the multiply, and the 64-bit overflow clamp.

The throttles are real, not hypothetical - EU_866 carries PROFILE_LITE, which sets both
positionThrottle and telemetryThrottle to 10, so a change here moves broadcast spacing in
that region by an order of magnitude.

Each test pins numOnlineNodes at the congestion threshold and uses ROUTER, which never
congestion-scales, so the coefficient is 1 and the throttle is the only variable. The
overflow case needs a base above INT32_MAX/10, hence three days rather than one.

* ci(test): keep pull-request suite order fixed, seed the rest

Shuffling the area order on every run - including pull_request - would turn a
contributor's PR red for an ordering they did not choose, which is how a randomisation
gets reverted instead of the coupling being fixed. That is the exact dynamic the ordering
work was sequenced last to avoid, and the previous commit walked straight into it.

- pull_request keeps the fixed declared area order.
- push and schedule shuffle, seeded from the commit SHA: deterministic per commit,
  printed, attributable, and never blocking someone else's PR.
- A suite_order_seed input on workflow_call and workflow_dispatch overrides both, so a
  specific failing order can be replayed anywhere, including on a PR.

The run log prints which mode it took, the resulting order, and the local command to
replay it.

* ci(test): satisfy CKV_GHA_7 and yamllint on the seed input

The seed is reachable through workflow_call, which callers can pass programmatically. The
workflow_dispatch copy tripped checkov's "workflow_dispatch inputs MUST be empty" rule,
and suppressing it was not worth it: replaying a specific order is a local operation, and
the run log already prints the exact bin/run-tests.sh command to do it.

* style(menu): apply the node-ID format convention

RadioInterface.cpp documents the rule: 0x%08x in logs, !%08x in user-facing
display. MenuHandler held every remaining exception - seven logs printing bare
%08X, and two display labels doing the same.

Repo-wide there are now no bare %08X node IDs left in log calls.

* ci(test): pass workflow inputs through env, not shell interpolation

suite_order_seed and github.event_name were spliced into the run: script as
${{ }} text, so a value carrying shell metacharacters would execute as code on
the runner rather than being read as data. semgrep (run-shell-injection) and
zizmor (template-injection) both flag it.

Both now arrive as environment variables and are read as "$VAR".

* refactor(test): share the seeded shuffle between the harness and CI

bin/run-tests.sh and test_native.yml each carried a byte-identical copy of the
MINSTD Fisher-Yates awk. The workflow prints "replay locally: ./bin/run-tests.sh
--shuffle --seed $seed" after a shuffled CI run, and that instruction is only
true while the two agree - drift would be announced by a replay quietly
reproducing a different order than the one that failed.

Extract shuffle_suites() to bin/lib/shuffle.sh and source it from both.
Permutations verified identical across seeds before and after the move.

* fix(test): correct the shared-state MISSING check and summary join

Three defects in the new harness:

state_classify() matched declarations two different ways - state_path_declared()
for "undeclared", a hand-rolled regex for "missing". Interpolating an entry into
an ERE also let a metacharacter in a manifest name match a file that is not the
declared one. Both directions now go through the one helper.

`paste -sd'; '` does not join with "; ": with -s, paste cycles through a
multi-character delimiter one character per join, so paths rendered as
"a;b c;d e". Replaced with an awk join.

test-state-check.sh ran on after a failed cd instead of stopping (SC2164).

./bin/test-state-check.sh: 6/6 fixtures pass, MISSING included.

* fix(portduino): bound General.MaxNodes

MaxNodes was validated only for <= 0. Any positive value, including a typo'd or
pasted-in one, propagates to MAX_NUM_NODES and scales both the node DB and the
nodes.proto decode ceiling - failing at boot with no obvious cause.

The ceiling is a sanity bound, not a capability limit; raise it if a host
genuinely needs more.

* docs(nodedb): reconcile the capacity tables

The property matrix omitted the ESP32-S3 100-node flash tier that the platform
table above it lists, and neither mentioned that the WASM build overrides
MaxNodes to 80 in wasm_config_apply().

* fix(nodedb): make mesh-pb-constants.h self-sufficient on portduino

The ARCH_PORTDUINO #error assumed it was unreachable in a normal build. It is
not: the vendored device-ui sources include this header without configuration.h,
which broke both native-tft docker builds.

Include configuration.h here instead, ahead of every compile-time default -
variant.h overrides MAX_RX_TOPHONE as well as MAX_NUM_NODES, so placing it lower
in the file just moves the divergence to a redefinition. The #error stays as a
backstop for the case where that include genuinely stops providing the cap.

Verified with the native env's own flags: a TU including only this header now
compiles, normal-order use of both macros compiles, and NodeDB.cpp compiles.

* fix(portduino): raise the MaxNodes ceiling to 16000

Marked artificial: nothing in the node DB fails at 16001. 16000 sits just under
the 16384 (128 x 128) population where HopScalingModule saturates its sampling
denominator and starts dropping nodes, so a host inside the bound still gets
meaningful hop recommendations.

* lint(trunk): advise on node IDs logged as bare %08x

RadioInterface.cpp documents the convention - 0x%08x in logs, !%08x in display -
but nothing enforced it, which is how the MenuHandler cluster drifted. 22 call
sites in PacketHistory, NodeInfoModule and PositionModule are still off it.

A trunk linter rather than a CI grep job, because trunk checks changed files:
new violations get flagged without a 22-site cleanup landing in an unrelated PR.
Modelled on the existing too-many-defined definition.

Scoped to values it can tell are IDs - an ID-shaped argument (->num, .from,
getNodeNum) or message text naming one. A 32-bit hex that is not an ID is out of
scope, so the CRC32 logs in ethOTA.cpp are correctly ignored.

Emits "note", trunk's only non-blocking level: "warning" and "info" both exit
non-zero and would gate CI, which is not what a log-format nit deserves. The
pre-existing sites are line-scoped in the allowlist, so a new bad call in those
same files is still caught.

* lint(trunk): stop exempting the known node-id-format sites

The seeded allowlist made the rule green by declaring the backlog acceptable.
Empty it instead, so the 22 pre-existing sites are reported and get cleaned up
by whoever next edits those files.

Costs nothing to do: the rule emits "note", so these are non-blocking either
way. The allowlist stays for its real purpose - a value the linter misreads as
an ID.

* style: log node and packet IDs as 0x%08x

Clears the 22 sites the node-id-format linter reports, so the rule starts from
zero rather than from a backlog nobody can see - trunk suppresses pre-existing
findings by default, so left alone these would not have surfaced on edit the way
an empty allowlist implies.

Format strings only; no argument or control flow changes. The !%08x
user-facing display forms are deliberately untouched - that is the other half of
the same convention.

* test(harness): build once up front, so suite timings mean something

run-tests.sh fused build and run in a single pio invocation, so whichever suite
PlatformIO's directory walk reached first absorbed the entire src compile and
reported it as its own duration. On a real run that made a 0.03s suite report
13m21s, and hid the build cost from every other number in the summary.

Do what .github/workflows/test_native.yml already does: one --without-testing
build pass, then run with --without-building. Measured on a full 44-suite run -
the build is now a single reported figure and 968 test cases execute in 1.9s,
with no suite above 0.084s.

Build output goes to its own log rather than $LOG: the outcome regexes match
"error:" and "[ERRORED]", so a compiler diagnostic sharing that file would read
as a test failure.

Both red paths now keep the log they quote from. $LOG and the build log are
mktemps the EXIT trap removes, so the three grepped lines were previously all
anyone ever saw - and the cause is usually further up than the first [FAILED].

* test(harness): keep the run log on every red path

bin/pio-test-isolate.sh already keeps a failing or DIRTY suite's sandbox and log
under .pio/test-state/<suite>/. What was missing is the cross-suite view: $LOG is
a mktemp the EXIT trap deletes, so run-tests.sh quoted three grepped lines from a
file that no longer existed by the time anyone looked.

Preserve it as .pio/build/<env>/test-failure.log from both red paths - including
"no success summary found", which said "see log" while preserving nothing, and
which is exactly the case where the build died before any suite ran and so left
no per-suite sandbox either.

Cleared at the start of every run, so a green run cannot leave a red one's log
lying around looking current.

* fix(test): report the real failure count on a shuffled red

A shuffled run is one `pio test` invocation per suite, all appending to the
same log, so the log carries one PlatformIO "N test cases:" summary per suite.
verdict_red() took `tail -1`, which reports whatever the LAST suite did: a
failure in suite 3 printed a "0 failed" summary from suite 44 directly under
"RED - failures detected:".

Sum the summaries instead. A single summary line - every unshuffled run - is
passed through verbatim, so the familiar output is byte-identical.

The patterns are passed to the awk helper as strings rather than /regex/
literals: awk evaluates a regex literal in argument position as `$0 ~ /re/`,
so the callee would receive 0 or 1 and silently sum garbage.

* fix(test): do not emit an empty suite name for an empty shuffle

`printf '%s\n' "$@"` with no arguments still writes one empty line, and both
callers read shuffle_suites through mapfile, so an empty suite list arrived as
a single suite named "". Return before the printf when there is nothing to
shuffle.

* test(harness): state and enforce the Linux host requirement

The native harness is a Linux tool: bash 4+ (mapfile), GNU coreutils and GNU
find (-printf, md5sum, -executable). Most of that predates this branch -
mapfile and both find predicates are already on develop - but none of it was
written down, so the requirement was there to be discovered rather than read.

Refuse to start on a non-Linux uname instead of degrading. On a BSD userland
this would not fail cleanly: it would mis-hash the sandbox and mis-read the
suite list, and still print a verdict. A state check that silently measures
the wrong thing is worse than one that declines to run.

Carrying a per-host fallback was the alternative, and it buys a second code
path that nothing in CI exercises. bin/test-native-docker.sh already exists
for macOS and non-Linux hosts, and the native-macos PlatformIO env is a build
target for meshtasticd, not a test host - the isolation wrapper is registered
for env:native and env:coverage only.

Documented in the script header, test/README.md, and both agent docs.

* fix(test): terminate every suite with exit(UNITY_END())

Two sites across two suites ended on a bare UNITY_END(). That ends the
reporting, not the suite: setup() returns, the runtime goes on calling loop(),
and the process runs forever. PlatformIO does not notice - it reports a suite
from its Unity output, not from process exit - so the suite passes, the run
goes green, and the binary stays resident. Thirteen of them had accumulated on
one dev box, the oldest 19 hours old.

The costs are quiet by construction:

- the per-suite sandbox is deleted underneath a live process, so its
  CLEAN/DIRTY verdict describes what the suite had written when the harness
  stopped looking, not what it left behind;
- .gcda coverage and LeakSanitizer's report both flush from atexit handlers,
  so a suite that never exits contributes no coverage and gets no leak check;
- each survivor pins its own deleted 94 MB binary, which du cannot see.

One of the two is the #else of an architecture guard, which is the easiest one
to get wrong - it looks like there is nothing to clean up. test_mqtt has a
correct exit(UNITY_END()) in its live branch, so a "does this file call exit()
anywhere" check passes the file whole.

test_serial had two more. develop's serial-config validation rework
restructured that suite - the architecture guard is gone and both remaining
branches now exit correctly - so this commit no longer has anything to change
there; bin/lint-unity-exit.sh, added later on this branch, is what keeps it
that way.

test/README.md gets a section on it, since the skeleton showing the right
shape had not stopped this happening.

* test(harness): detect and reap suites that outlive their run

A suite that never exits was invisible: PlatformIO reports a suite from its
Unity output, so the run stayed green while the binary kept running. Two
checks, because they fail differently.

Runtime, in bin/pio-test-isolate.sh: the sandbox $HOME is mktemp-unique per
suite, so any process still holding it is a survivor of that suite. Matching
on the environment rather than a remembered PID identifies one whatever its
parentage - a fork, a grandchild, a process already reparented to init - none
of which a $! comparison catches. Reaped before the after-fingerprint is
taken, so that fingerprint measures a tree nobody is still writing to, and so
a run cannot leave processes accumulating on the host. Recorded as a sixth
summary column and graded AMBER: the tests did pass, but the CLEAN verdict and
the coverage were measured under a false assumption.

Author-time, as bin/lint-unity-exit.sh, wired into trunk at "note" like
node-id-format: every UNITY_END() must be wrapped in exit(). The rule is per
occurrence, and that is the point - a file-level "calls exit() somewhere"
check passes test_serial and test_mqtt, which have a correct one in their live
branch and a bare one in the #else. Running it over the tree turned up
test_mqtt, which the file-level pass had missed.

It allows `int rc = UNITY_END(); ...; exit(rc)`, used by test_packet_signing
to restore globals between the summary and the exit. That is where the rule
gives ground: capturing and never exiting would leak and is not flagged.
Flagging a correct idiom would push someone to "fix" working code.

bin/test-state-check.sh gains a survivor fixture, asserting the wrapper both
reports and reaps - a detector that only reports leaves the host accumulating
processes, which is half the harm. 8/8.

* fix(lint): make the unity-exit scanner statement-aware

The rule judged one physical line at a time, which reports two kinds of correct
code as bare:

    /* a comment that happens to
       mention UNITY_END() */          <- interior lines were never stripped

    exit(
        UNITY_END());                  <- exit( and the macro never met

On a probe of both, two of three findings were wrong. This is a note-level rule
whose whole job is advice, and bin/lint-node-id-format.sh already says why that
matters: a false positive costs more than a miss. One that cries wolf gets
ignored, and the real finding goes with it.

Carry /* ... */ state across lines and accumulate logical statements before
testing, with a 12-line cap so one unclosed call cannot swallow the rest of the
file - the same structure lint-node-id-format.sh uses, so the two custom linters
in bin/ work alike rather than each having its own idea.

Verified both directions: the develop-era sources still produce the same four
findings, the fixed tree produces none, and a probe covering block-comment
interiors, wrapped exit(), line comments, return UNITY_END() and capture-then-
exit reports only the genuinely bare calls - including a complete block comment
followed by real bare code on the same line, which the state machine has to
keep live.

Reported by CodeRabbit on #11322.

* fix(lint): tokenise instead of pattern-matching, and self-test it

Second round of review findings on the same scanner, all confirmed by direct
test before changing anything. Six defects, one root cause: layered regexes
cannot tokenise C++.

False positives (correct code reported):
  - UNITY_END() inside a string literal read as code

False negatives (real leaks missed):
  - a string containing "/*" opened comment state and swallowed later lines
  - greedy .* removed everything between two block comments on one line,
    taking a bare call with it
  - myexit(UNITY_END()) matched the exit() exemption as a substring
  - x == UNITY_END() and total += UNITY_END() matched the assignment exemption

Replaced with a character-level scan carrying comment state, and token-bounded
exemptions: exit must be a whole identifier, and the capture form must be a
plain `=`. Raw string literals are still not modelled - there are none under
test/, and delimiter tracking for a case that does not occur would be untested
code guarding untested code, so it is documented rather than guessed at.

Also drops the `return UNITY_END()` exemption. It only terminates from main(),
there is no main() under test/, and from a helper it just returns a count.

bin/test-lint-unity-exit.sh pins all fifteen cases, every false positive and
false negative found in review among them. The rule has been wrong twice in a
way that looked fine by inspection; it needed a self-test more than it needed
another careful reading.

Two further findings in the same review:

  - bin/run-tests.sh dropped PASSTHRU in shuffled mode, so `--shuffle -vvv`
    built verbosely and then ran quietly. The shuffled loop now forwards
    EXTRA_ARGS, which is PASSTHRU minus the -f pair it supplies per suite.
  - bin/run-tests.sh did not guard `cd "$ROOT_DIR"`.

And one that did not reproduce: the survivor fixture's glob does find the pid
file (verified with the lookup instrumented - the earlier failure was an
artifact of running the script from /tmp, where SCRIPT_DIR cannot resolve).
The assertion was still weak, because an empty pid took the "not running"
branch and passed vacuously. It now fails if the pid was never recorded, and
finds the file by search rather than assuming a directory depth.

Reported by CodeRabbit on #11322.

* fix(lint): report each UNITY_END occurrence at its own location

The self-test only asked "did the linter say anything", so it could not have
caught a wrong line, a wrong column, or a missing second finding. Fixtures now
assert the exact diagnostics as line:col, and the first run of that assertion
found two real problems.

The caret pointed at the wrong occurrence. For `exit(UNITY_END()); UNITY_END();`
the verdict was right but the column was 17 - the wrapped call - because the
scanner stripped terminating forms out of the whole statement and then reported
the first occurrence it had seen. Two bare calls on one line reported once.

Judged per occurrence now, by looking back through whitespace at what wraps it,
so both the count and the caret are right. That also needed a position map from
strip_noncode(): removing a comment or collapsing a literal shifts every later
column, and counting occurrences in the raw line does not recover it either -
TEST_MESSAGE("... UNITY_END() ..."); UNITY_END(); has two occurrences in the raw
text and one in the code.

Four of the expected columns I wrote by hand were also wrong, off by one. The
linter was right in every case; the assertions were not. They are computed from
the fixture text now rather than pasted from output, because a baseline accepted
from the tool it is testing asserts nothing.

17 fixtures, including the two-on-one-line case from review and its mirror.

Reported by CodeRabbit on #11322.
2026-08-06 14:05:07 +00:00
Thomas Göttgens 023351a979 Remove dead code (#11082)
Removed:
- MessageStore::addFromPacket and addFromString, superseded by
  tryAddFromPacket
- GeoCoord rangeRadiansToMeters, distanceTo, bearingTo
- Router::rawSend, declared virtual with no override and no caller
- ContentHandler handleHotspot, handleFs, handleAdminSettings,
  handleAdminSettingsApply, handleDeleteFsContent and their commented
  route registrations, plus the now unreachable htmlDeleteDir and the
  handleUpdateFs declaration that had no definition
- ContentHelper replaceAll
- OnScreenKeyboardModule popup chain: showPopup, clearPopup, drawPopup,
  drawPopupOverlay and their state, unreachable since the frame based UI
  was replaced by baseUI
- DebugRenderer drawDebugInfoTrampoline, drawDebugInfoSettingsTrampoline
  and the orphaned drawFrameSettings
- NodeListRenderer calculateMaxScroll, drawColumns and a stale extern
  haveGlyphs declaration with no definition
- UIRenderer::haveGlyphs, Screen::blink,
  NotificationRenderer::showKeyboardMessagePopupWithTitle,
  VirtualKeyboard::getInputText
- InkHUD touchNavLeft, touchNavRight, Applet::getActiveNodeCount,
  ThreadedMessageApplet::saveMessagesToFlash
- TwoButton::setHandlerUp, TwoButtonExtended setHandlerUp,
  setJoystickDownHandlers, setJoystickUpHandlers
- CannedMessageModule LaunchRepeatDestination, isCharInputAllowed,
  hasMessages
- TrafficManagementModule resetStats, recordRouterHopPreserved,
  saturatingIncrement
- UnitConversions::MetersPerSecondToMilesPerHour
- EncryptedStorage getSessionRemainingSeconds
- BMI270Sensor::writeRegisters, GPS::hasFlow, FSCommon copyFile,
  SerialConsole consolePrintf, buzz playLongPressLeadUp,
  memGet displayPercentHeapFree
2026-07-20 11:55:17 +02:00
Ben Meadors f6f3284cfc fix(lockdown): re-lock serial console admin auth on USB link drop (#11028)
On lockdown builds (MESHTASTIC_PHONEAPI_ACCESS_CONTROL, nRF52) the SerialConsole
is a process-lifetime singleton, so the per-connection admin-auth slot keyed by
its inherited PhoneAPI* is reused for every USB/serial client for the whole boot.
An operator's admin unlock stayed latched across serial client swaps: an attacker
plugging into the USB/serial port before the prior session's 15-minute inactivity
timeout inherited admin authorization -- the serial analog of the BLE stale-session
reuse bug closed by resetting state in onConnect()/onDisconnect().

Sample the USB-CDC host link (DTR/mount) state each runOnce(); on the link-drop
edge call close(), which frees the auth slot and resets PhoneAPI state so whoever
connects next re-locks via handleStartConfig()'s !isConnected() branch on their
first want_config -- the same physical-link boundary BLE enforces in onConnect().
On the nRF52 TinyUSB (Adafruit) core, (bool)Port == tud_cdc_n_connected(), which
goes false on cable unplug or host port-close. Console transports without a real
DTR line fall back to the existing inactivity timeout, no worse than before.

Entirely nRF52-lockdown-gated; non-lockdown builds are byte-identical.
2026-07-17 07:05:11 -05:00
p0ns ca833d944c Fix serial protobuf corruption on short USB CDC writes (#10976)
* Fix serial protobuf corruption on short writes

SerialConsole shared raw debug output and framed protobuf traffic on one
HWCDC stream. Raw text could interleave inside an active frame when API
logging was disabled. Separately, HWCDC deliberately returns a short write
after bounded backpressure; StreamAPI abandoned that frame after PhoneAPI
had already advanced, so the next 0x94c3 header landed inside the previous
declared payload.

Suppress all unframed output after protobuf mode starts and honor the
existing config-replay log pause. For HWCDC, retain short frame tails in the
persistent tx buffers and finish them on later loop passes before dequeuing
another FromRadio packet. Logs remain best-effort and only start when their
complete frame fits; main packets are deferred rather than dropped if a
synchronous log is pending. Do not call HWCDC flush for framed serial output,
since its no-progress path can discard queued bytes. TCP and non-HWCDC
transports keep their existing behavior.

Validated on Cardputer ADV (200-node DB) and Heltec Tracker V2: 400 initial
full-DB sessions plus 140 final sessions across both API-log settings, zero
malformed frames/timeouts/incomplete DBs; 2-20s forced reader stalls resume
with complete DBs; abrupt stalled-client replacement 5/5 per board; 150s
post-close reboot counts stable. Builds pass for Cardputer, Heltec, tbeam,
and rak4631.

* Add serial frame continuation regression tests

Extract the HWCDC pending/deferred frame state machine into a small
transport-independent helper so native tests exercise the same production
logic used by SerialConsole. Keep framing, persistent buffer ownership and
locking in SerialConsole.

Cover short-tail continuation, deferred main-frame ordering, best-effort log
admission, bounded zero-progress calls, generic StreamAPI failure semantics,
PhoneAPI advancement gating, framed-log gating and raw output suppression.
The coverage suite passes 31/31 suites and 582/582 tests.

* Address review: guard flush, assert deferred invariant, dedup framing

- Make SerialConsole::flush() a no-op in protobuf mode: HWCDC::flush()'s
  no-progress path discards queued TX bytes, which would tear a framed
  stream when the sleep path flushes with a stalled host.
- Assert the single required-frame producer invariant in
  StreamFrameWriter::writeFrame() instead of silently dropping a second
  required frame.
- Hoist 0x94C3 header construction into StreamAPI::buildFrameHeader() so
  SerialConsole no longer re-hardcodes the framing constants.

* Test retained serial tail across client replacement

Model a replacement client arriving while an older required frame has an
unwritten tail. Require the old frame to complete before the new frame starts,
so a new 0x94C3 header can never land inside the old declared payload.

* Document serial frame APIs and regression tests

Add concise Doxygen comments for the frame continuation hooks, production
helper, native test doubles, and regression scenarios. Document why retained
frame tails intentionally survive client disconnects: HWCDC may still hold the
accepted prefix, so dropping metadata could insert a new frame header inside
the old declared payload.
2026-07-11 06:27:50 -05:00
p0nsandBen Meadors 144b07986b Fix ESP32-S3 USB CDC: post-disconnect task-WDT reboot from blocking console log writes (#10956)
* Fix ESP32-S3 USB CDC: post-disconnect task-WDT reboot from blocking console log writes

When a serial API client disconnects (USB cable still attached), the host
stops draining the USB-Serial/JTAG CDC buffer. The next raw-text debug log
write can then block the main loop task indefinitely (measured: a single
write blocked 52.4 s). The loop task stops feeding the app task watchdog
(APP_WATCHDOG_SECS = 90 s, trigger_panic), so the device reboots with
esp_reset_reason = ESP_RST_TASK_WDT ~97 s after every serial disconnect.

On 2.7.x (arduino-esp32 2.x / IDF 4.4) the reboot also re-enumerated USB;
since the Arduino 3.x migration the reboot is silent (USB stays enumerated),
making it look like a random reboot ~90 s after using the CLI.

Fix: on USB CDC targets, keep console TX in non-blocking mode (txTimeout 0,
drop-oldest) whenever no API client is provably alive, and restore the
normal bounded timeout while a client is connected so protobuf API frames
are never truncated. Toggle points: boot, handleToRadio (host sent bytes),
and onConnectionChanged (set non-blocking before disconnect handling emits
more log lines to a dead port).

Repro/validation on HELTEC_WIRELESS_TRACKER_V2 (macOS + Linux hosts):
open+close any meshtastic-python session, wait 150 s: reboot_count +1 every
time on unpatched builds; flat with this fix. Max observed log write stall
drops from 52405 ms to 30 ms.

* Condense comments per review feedback

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-07-09 11:42:15 -05:00
Jonathan Bennett 3778fb021c Refactor SerialConsole to simplify timeout handling (#10802)
Removed conditional compilation for timeout check on supported architectures.
2026-07-02 20:01:54 -05:00
Jonathan Bennett 125c1b7f13 Make consoleInit() Reentrant, and initialize it earlier on native (#10156) 2026-04-14 06:41:04 -05:00
Thomas Göttgens 54781cf51a hopefully fix remaining cppcheck issues (#9745) 2026-02-25 10:27:21 +01:00
Jorropo beb268ff25 Revert "add a .clang-format file (#9154)" (#9172)
I thought git would be smart enough to understand all the whitespace changes but even with all the flags I know to make it ignore theses it still blows up if there are identical changes on both sides.

I have a solution but it require creating a new commit at the merge base for each conflicting PR and merging it into develop.

I don't think blowing up all PRs is worth for now, maybe if we can coordinate this for V3 let's say.

This reverts commit 0d11331d18.
2026-01-04 05:15:53 -06:00
Jorropo 0d11331d18 add a .clang-format file (#9154) 2026-01-03 14:19:24 -06:00
531cad5e88 Add API types, state, and log message in Debug screen. Added persistent "Connected" icon (#8576)
* Add API types, state, and log message in Debug screen

* un-goober the API state tracking

* Set the SerialConsole api_type

* Add api_type for Ethernet

* Remove API state debugging code

* Update wording for client connection states

* Improve string width for smaller screen devices

* Reserve space on navigation bar to fit link indicator

* Add persistent Connected icon to screen

* Connect System frame to ensure text doesn't overflow

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Jason P <applewiz@mac.com>
Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com>
2025-11-07 15:03:56 -06:00
Austin b7f6a2acb6 ESP32s2 doesn't implement HWCDC (#8199) 2025-10-03 18:52:51 -05:00
Links2004 17ecd69416 onReceive does only exist for HardwareSerial not for USB CDC serial but we can at least check for USB connection in a longer interval 2025-09-24 22:09:32 +02:00
Links2004 14e64d6b9e move SerialConsole to event based trigger 2025-09-24 17:14:14 +02:00
0903ed8232 Mesh solar integrate (#7764)
* Added HELTEC MeshSolar board. (#7499)

* Added HELTEC MeshSolar board.

* Set emergency shutdown pin as high impedance

* Set emergency shutdown pin as high impedance

Set emergency shutdown pin as high impedance

* Update variants/nrf52840/heltec_mesh_solar/variant.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update variants/nrf52840/heltec_mesh_solar/variant.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update variants/nrf52840/heltec_mesh_solar/variant.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update I2C SCL pin definition in variant.h

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Updates

---------

Co-authored-by: Quency-D <55523105+Quency-D@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-27 06:02:54 -05:00
Tom Fifield e394bc6f8f De-duplicate log-level determination (#5148)
RedirectablePrint had a method, getLogLevel, which did exactly
what code in SerialConsole did. Let's use that method rather than
duplicating the code.
2024-10-26 12:06:50 +02:00
Thomas Göttgens c650e7d273 finish TLora C6 Support, without bluetooth for now 2024-09-28 13:42:32 +02:00
Ben Meadors c442cd7267 Remove some straggler now 2024-09-23 15:53:42 -05:00
Ben MeadorsandTom Fifield e8829b8f52 Refactor and consolidate time window logic (#4826)
* Refactor and consolidate windowing logic

* Trunk

* Fixes

* More

* Fix braces and remove unused now variables.

There was a brace in src/mesh/RadioLibInterface.cpp that was breaking
compile on some architectures.

Additionally, there were some brace errors in
src/modules/Telemetry/AirQualityTelemetry.cpp
src/modules/Telemetry/EnvironmentTelemetry.cpp
src/mesh/wifi/WiFiAPClient.cpp

Move throttle include in WifiAPClient.cpp to top.

Add Default.h to sleep.cpp

rest of files just remove unused now variables.

* Remove a couple more meows

---------

Co-authored-by: Tom Fifield <tom@tomfifield.net>
2024-09-23 08:58:14 -05:00
Jonathan BennettandBen Meadors 74afd13171 Re-implement PKI from #1509 (#4379)
* Re-implement PKI from #1509
co-authored-by: edinnen <ethanjdinnen@protonmail.com>

* Set the key lengnth to actually make PKI work.

* Remove unused variable and initialize keys to null

* move printBytes() to meshUtils

* Don't reset PKI key son reboot unless needed.

* Remove double encryption for PKI messages

* Cleanup encrypt logic

* Add the MESHTASTIC_EXCLUDE_PKI option, and set it for minimal builds. Required for STM32 targets for now.

* Use SHA-256 for PKI key hashing, and add MESHTASTIC_EXCLUDE_PKI_KEYGEN for STM32

* Fix a crash when node is null

* Don't send PKI encrypted packets while licensed

* use chIndex 8 for PKI

* Don't be so clever, that you corrupt incoming packets

* Pass on channel 8 for now

* Typo

* Lock keys once non-zero

* We in fact need 2 scratch buffers, to store the encrypted bytes, unencrypted bytes, and decoded protobuf.

* Lighter approach to retaining known key

* Attach the public key to PKI decrypted packets in device memory

* Turn PKI back off for STM32 :(

* Don't just memcp over a protobuf

* Don't PKI encrypt nodeinfo packets

* Add a bit more memory logging around nodeDB

* Use the proper macro to refer to NODENUM_BROADCAST

* Typo fix

* Don't PKI encrypt ROUTING (naks and acks)

* Adds SecurityConfig protobuf

* Add admin messages over PKI

* Disable PKI for the WIO-e5

* Add MINIMUM_SAFE_FREE_HEAP macro and set to safe 1.5k

* Add missed "has_security"

* Add the admin_channel_enabled option

* STM32 again

* add missed configuration.h at the top of files

* Add EXCLUDE_TZ and RTC

* Enable PKI build on STM32 once again

* Attempt 1 at moving PKI to aes-ccm

* Fix buffers for encrypt/decrypt

* Eliminate unused aes variable

* Add debugging lines

* Set hash to 0 for PKI

* Fix debug lines so they don't print pointers.

* logic fix and more debug

* Rather important typo

* Check for short packets before attempting decrypt

* Don't forget to give cryptoEngine the keys!

* Use the right scratch buffer

* Cleanup

* moar cleanups

* Minor hardening

* Remove some in-progress stuff

* Turn PKI back off on STM32

* Return false

* 2.5 protos

* Sync up protos

* Add initial cryptography test vector tests

* re-add MINIMUM_SAFE_FREE_HEAP

* Housekeeping and comment fixes

* Add explanatory comment about weak dh25519 keys

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2024-08-10 13:45:41 -05:00
Ben Meadors 93ba19d1e1 Make LogRecord protobuf serial logging over Phone API opt-in instead (#4358)
* Make LogRecord protobuf serial logging over Phone API opt-in instead of enabled by default

* debug_log_enabled
2024-07-30 15:05:33 -05:00
Thomas Göttgens 94a10e011c Merge branch 'master' into wio-lr1110-refresh 2024-07-22 15:37:34 +02:00
Thomas Göttgens bdd1c53072 Revert "Sync Wio lr1110 refresh with master (#4288)"
This reverts commit 5cc8ca59a3.

Revert "Sync Wio lr1110 refresh with master (#4251)"

This reverts commit d97e6b86b8.

Revert "update SD_FLASH_SIZE to 0x27000 (#4232)"

This reverts commit 2df8093fef.
2024-07-22 15:30:36 +02:00
+7 d97e6b86b8 Sync Wio lr1110 refresh with master (#4251)
* Fix protobuf structs handling (#4140)

* Fix protobuf structs handling

* Log instead of assert

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* BLE based logging (#4146)

* WIP log characteristic

* Bluetooth logging plumbing

* Characteristic

* Callback

* Check for nullptr

* Esp32 bluetooth impl

* Formatting

* Add thread name and log level

* Add settings guard

* Remove comments

* Field name

* Fixes esp32

* Open it up

* Whoops

* Move va_end past our logic

* Use `upload_protocol = esptool` as with the other heltec devices instead of `esp-builtin` (#4151)

* Standardize lat/lon position logs (#4156)

* Standardize lat/lon position logs

* Missed sone and condensed logs

* [create-pull-request] automated change (#4157)

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Pause BLE logging during want_config flow (#4162)

* Update NimBLE to 1.4.2 (#4163)

* Implement replies for all telemetry types based on variant tag (#4164)

* Implement replies for all telemetry types based on variant tag

* Remove check for `ignoreRequest`: modules can set this, don't need to check

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* Esptool is better

* Explicitly set characteristic

* fix INA3221 sensor (#4168)

- pass wire to begin()
- remove redundant setAddr() (already set in header)

* Show compass on waypoint frame; clear when waypoint deleted (#4116)

* Clear expired or deleted waypoint frame

* Return 0 to CallbackObserver

* Add a missing comment

* Draw compass for waypoint frame

* Display our own waypoints

* [create-pull-request] automated change (#4171)

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Add semihosting support for nrf52 devices (#4137)

* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes https://github.com/meshtastic/firmware/issues/4135

* fix my botched merge - keep board_level = extra flag for rak3631_dbg

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* [create-pull-request] automated change (#4174)

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Display alerts (#4170)

* Move static functions into Screen.h, show compass during calibration

* Move to _fontHeight macro to avoid collision

* Move some alert functions to new alert handler

* Catch missed reboot code

* ESP32 fixes

* Bump esp8266-oled-ssd1306

* Fixes for when a device has no screen

* Use new startAlert(char*) helper class

* Add EINK bits back to alert handling

* Add noop class for no-display devices

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* Send file system manifest up on want_config (#4176)

* Send file system manifest up on want_config

* Platform specific methods

* Helps to actually make the change

* Clear

* tell vscode, if formatting, use whatever our trunk formatter wants (#4186)

without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* fix the build - would loop forever if there were no files to send (#4188)

* Show owner.short_name on boot (and E-Ink sleep screen) (#4134)

* Show owner.short_name on boot and sleep screen (on e-ink)

* Update Screen.cpp - new line for short_name

Boot screen short_name now below the region setting.
Looks better on small screens.

* Draw short_name on right

---------

Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: todd-herbert <herbert.todd@gmail.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* nrf52 soft device will watchdog if you use ICE while BT on... (#4189)

so have debugger disable bluetooth.

* correct xiao_ble build preventing sx1262 init (#4191)

* Force a compile time failur if FromRadio or ToRadio get larger than (#4190)

a BLE packet size. We are actually very close to this threshold so
important to make sure we don't accidentally pass it.

* Clear vector after complete config state (#4194)

* Clear after complete config

* Don't collect . entries

* Log file name and size

* [create-pull-request] automated change (#4200)

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Make the logs Colorful! (#4199)

* Squash needlessly static functions (#4183)

* Trim extra vprintf and filter for unprintable characters

* Deprecate Router Client role (and make it Client) (#4201)

* [create-pull-request] automated change (#4205)

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Move waypoint (#4202)

* Move waypoint screen draw into the waypoint module

* Get the observer set up for the waypoint screen draw

* Static squashing: screen dimensions
Macros moved back to Screen.cpp, as a band-aid until we eventually move all those static functions into the Screen class.

* Move getCompassDiam into Screen class
(supress compiler warnings)
At this stage, the method is still static, because it's used by drawNodeInfo, which has no tidy reference to our screen instance.
This is probably just another band-aid until these static functions all move.

* Use new getCompassDiam function in AccelerometerThread

* Properly gate display code in WaypointModule

---------

Co-authored-by: Todd Herbert <herbert.todd@gmail.com>

* Fix flakey phone api transition from file manifest to complete (#4209)

* Try fix flakey phone api transition from file manifest to complete

* Skip

* enable colors in platformio serial monitor (#4217)

* When talking via serial, encapsulate log messages in protobufs if necessary (#4187)

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* [create-pull-request] automated change (#4218)

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Fix SHT41 support (#4222)

* Add SHT41 Serial to I2c Detection Code

On the Seeed Wio-WM1110 Dev Kit board, the SHT41 chip was being
incorrectly detected as SHT31.

This patch adds the necessary serial number for the SHT41 chip to
be correctly detected.

fixes meshtastic/firmware#4221

* Add missing sensor read for SHT41

* Typo fix in logs - mhz - MHz (#4225)

As reported by karamo, a few different places in our logs had
incorrect capitalization of MHz.

fixes meshtastic/firmware#4126

* New new BLE logging characteristic with LogRecord protos  (#4220)

* New UUID

* New log radio characteristic with LogRecord protobuf

* LogRecord

* Merge derp

* How did you get there

* Trunk

* Fix length

* Remove assert

* minor cleanup proposal (#4169)

* MESHTASTIC_EXCLUDE_WIFI and HAS_WIFI cleanup...
Our code was checking HAS_WIFI and the new MESHTASTIC_EXCLUDE_WIFI
flags in various places (even though EXCLUDE_WIFI forces HAS_WIFI
to 0).  Instead just check HAS_WIFI, only use EXCLUDE_WIFI inside
configuration.h

* cleanup: use HAS_NETWORKING instead of HAS_WIFI || HAS_ETHERNET
We already had HAS_NETWORKING as flag in MQTT to mean 'we have
tcpip'.  Generallize that and move it into configuration.h so that
we can use it elsewhere.

* Use #pragma once, because supported by gcc and all modern compilers
instead of #ifdef DOTHFILE_H etc...

---------

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>

* Add PowerMon support (#4155)

* Turn off vscode cmake prompt - we don't use cmake on meshtastic

* Add rak4631_dap variant for debugging with NanoDAP debug probe device.

* The rak device can also run freertos (which is underneath nrf52 arduino)

* Add semihosting support for nrf52840 devices
Initial platformio.ini file only supports rak4630
Default to non TCP for the semihosting log output for now...
Fixes https://github.com/meshtastic/firmware/issues/4135

* powermon WIP (for https://github.com/meshtastic/firmware/issues/4136 )

* oops - mean't to mark the _dbg variant as an 'extra' board.

* powermon wip

* Make serial port on wio-sdk-wm1110 board work
By disabling the (inaccessible) adafruit USB

* Instrument (radiolib only for now) lora for powermon
per https://github.com/meshtastic/firmware/issues/4136

* powermon gps support
https://github.com/meshtastic/firmware/issues/4136

* Add CPU deep and light sleep powermon states
https://github.com/meshtastic/firmware/issues/4136

* Change the board/swversion bootstring so it is a new "structured" log msg.

* powermon wip

* add example script for getting esp S3 debugging working
Not yet used but I didn't want these nasty tricks to get lost yet.

* Add PowerMon reporting for screen and bluetooth pwr.

* make power.powermon_enables config setting work.

* update to latest protobufs

* fix bogus shellcheck warning

* make powermon optional (but default enabled because tiny and no runtime impact)

* tell vscode, if formatting, use whatever our trunk formatter wants
without this flag if the user has set some other formatter (clang)
in their user level settings, it will be looking in the wrong directory
for the clang options (we want the options in .trunk/clang)

Note: formatOnSave is true in master, which means a bunch of our older
files are non compliant and if you edit them it will generate lots of
formatting related diffs.  I guess I'll start letting that happen with
my future commits ;-).

* add PowerStress module

* nrf52 arduino is built upon freertos, so let platformio debug it

* don't accidentally try to Segger ICE if we are using another ICE

* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

* update to latest protobufs (needed for powermon goo)

* PowerStress WIP

* fix linter warning

* Cleanup buffer

* Merge hex for wm1110 target(s)

* Only sdk

* Sudo

* Fix exclude macros (#4233)

* fix MESHTASTIC_EXCLUDE_BLUETOOTH

* fix HAS_SCREEN=0

* fix MESHTASTIC_EXCLUDE_GPS

* fix typo in build-nrf52.sh (#4231)

chmod is the command, '+x' is the argument.

* Tidy Wireless Paper variant files (#4238)

* Quick tidy of pins_arduino.h
Matches requests made at https://github.com/meshtastic/firmware/pull/4226#discussion_r1664183480)

* Tidy variant.h

* Change deprecated ADC attenuation parameter
From 11dB to 12dB. Resolves compiler warning. Allegly, no impact on function: `This is deprecated, it behaves the same as `ADC_ATTEN_DB_12`

* Updated raspbian CI to update apt repository ahead of libbluetooth. (#4243)

* Fix BLE logging on nrf52 (#4244)

* allow ble logrecords to be fetched either by NOTIFY or INDICATE ble types

This allows 'lossless' log reading.  If client has requested INDICATE
(rather than NOTIFY) each log record emitted via log() will have to fetched
by the client device before the meshtastic node can continue.

* Fix serious problem with nrf52 BLE logging.
When doing notifies of LogRecords it is important to use the
binary write routines - writing using the 'string' write won't work.
Because protobufs can contain \0 nuls inside of them which if being
parsed as a string will cause only a portion of the protobuf to be sent.
I noticed this because some log messages were not getting through.

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

* Fix build when HAS_NETWORKING is false on nrf52 (#4237)

(tested on a rak4631 by setting HAS_ETHERNET false when shrinking
image)

* If `toPhoneQueue` is full, still increment `fromNum` to avoid client never getting packets (#4246)

* Update to SoftDevice 7.3.0 for wio-sdk-wm1110 and wio-tracker-wm1110 (#4248)

* Update variant.h

* Update wio-tracker-wm1110.json

* Update wio-sdk-wm1110.json

* Update platformio.ini

* Update platformio.ini

* Add files via upload

* Add files via upload

* Update variant.h

---------

Co-authored-by: Mike <mikhael.skvortsov@gmail.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Mike G <mkgin@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
Co-authored-by: GUVWAF <78759985+GUVWAF@users.noreply.github.com>
Co-authored-by: Warren Guy <5602790+warrenguy@users.noreply.github.com>
Co-authored-by: todd-herbert <herbert.todd@gmail.com>
Co-authored-by: geeksville <kevinh@geeksville.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
Co-authored-by: Alexander <156134901+Dorn8010@users.noreply.github.com>
Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: quimnut <github@dopegoat.com>
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com>
Co-authored-by: Agent Blu, 006 <blu006@ucr.edu>
Co-authored-by: Mark Trevor Birss <markbirss@gmail.com>
2024-07-08 06:03:23 -05:00
geeksvilleandBen Meadors 3219d65387 When talking via serial, encapsulate log messages in protobufs if necessary (#4187)
* clean up RedirectablePrint::log so it doesn't have three very different implementations inline.

* remove NoopPrint - it is no longer needed

* when talking to API clients via serial, don't turn off log msgs instead encapsuate them

* fix the build - would loop forever if there were no files to send

* don't use Segger code if not talking to a Segger debugger

* when encapsulating logs, make sure the strings always has nul terminators

* nrf52 soft device will watchdog if you use ICE while BT on...
so have debugger disable bluetooth.

* Important to not print debug messages while writing to the toPhone scratch buffer

* don't include newlines if encapsulating log records as protobufs

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2024-06-30 18:41:27 -05:00
Jonathan Bennett 6f3d7ca4d2 Trim extra vprintf and filter for unprintable characters 2024-06-28 23:30:39 -05:00
Thomas Göttgens ecf5519b56 Moar LR1110 Targets 2024-06-20 16:26:04 +02:00
Jonathan Bennett c009c0db1e Elimate non-text output for Portduino 2024-05-06 22:27:12 -05:00
tuxphone 70712d859c Enable compiling with gccnoneeabi 12.3.1 for nRF52 targets, additional small fixes (#3778)
* Fix type of nodeNum

Type of nodeNum is NodeNum, not uint

* typo

fixed typo "resumeAdverising()"

* fix missing #include "time.h"

Missing include breaks compilation with gccnoneeabi 12.3.1 for nrf52 targets on windows hosts.

* change type uint to unsigned int

uint is not a standard type. Using uint breaks compilation with gccnoneeabi 12.3.1 for nRF52 targets on windows hosts.

* fix type of channel_num

Type of channel_num should be uint32_t (as this is the type of hash() and numChannels).

Using uint non-standard type uint breaks compilation with gccnoneeabi 12.3.1 for nRF52 targets on windows hosts.

* Update nrf52.ini

Default build type should be "release" as this is the default of platformio.

* Update GPS.cpp

uint to unsigned int
2024-05-03 15:49:22 -05:00
Ben Meadors 9586606229 Handle for heartbeat toradio packets (#3420) 2024-03-15 18:40:48 -05:00
Thomas Göttgens 34bc22f94d (2/3) Add Slow Clock Support for RP2040 platform. This will disable USB Softserial. 2024-03-15 20:16:36 +01:00
Jonathan Bennett 79cfc4b725 Avoid Serial output mangling with RTOS. 2023-08-31 20:40:01 -05:00
1dfa8f2d9e RAK11310 (#2299)
* POC. Board definition JSON upcoming. Generic for now

* side-effect: RP2040 is building again.

* WIP Pico Targets

* current state of affairs

* ahem

* POC. Board definition JSON upcoming. Generic for now

* side-effect: RP2040 is building again.

* WIP Pico Targets

* current state of affairs

* ahem

* fmt

* update toolkit and fmt

* Add built in LED pin

* Use arduino pins

* init SPI bus on right pins.

* Use SPI1 and control chip select manually

* Use macro define for SPI selection. This needs to be defined in the ini file since portduino needs it inside the framework source

* Remove manual CS; works when not using setCS()

* Remove whoopsie debug line

* we are not ARDUINO_AVR_NANO_EVERY any more

* fix rp2040 compilation

* fix RadioLibHAL

* Use new arduino-pico core

* Use cortex-m0plus for BSEC2 library

* Forgot RAK11310 target for BSEC2 library

* That branch was merged

* RAK11310 is working too

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: GUVWAF <thijs@havinga.eu>
2023-05-23 16:19:36 -05:00
Thomas Göttgens 51b2c431d9 trunk roundhouse kick 2023-01-21 21:23:24 +01:00
Thomas Göttgens d077be1496 add a possibility to flush the serial TX buffer on the console. 2023-01-04 10:37:27 +01:00
Thomas Göttgens f73b8661de Wait for 5 seconds for serial port on ESP32-S2/3 too... 2022-12-29 21:34:18 +01:00
Thomas Göttgens a289406863 refactor classes to accomodate SerialModule 2022-12-22 18:24:42 +01:00
Thomas Göttgens f7331a2e41 fix #1713
stop output to serial debug after config has been loaded (if serial is disabled), and only accept protobuf packets from that point on.
2022-10-05 10:33:39 +02:00
Thomas Göttgens 7c5a36ce38 Update SerialConsole.cpp 2022-09-21 15:46:56 +02:00
Thomas Göttgens 0b81a25fda Output serial console data without buffering 2022-09-21 15:34:48 +02:00
ade32b1827 lay groundwork for a possible future architecture (#1571)
* lay groundwork for a possible future architecture

* switch from feature opt-out to feature opt-in

* lay groundwork for a possible future architecture

* switch from feature opt-out to feature opt-in

* fix USE_RTC in variant.h for rak4631_epaper and t-echo

* ensure Screen.h is not included without configuration.h

Co-authored-by: Peter Lawrence <12226419+majbthrd@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2022-07-31 07:11:47 -05:00
Ben Meadors 5106433572 Removed phone_timeout_secs pref references (#1503) 2022-06-12 09:44:23 -05:00
Ben Meadors 4ab831c103 Fix nagging bool linter warnings (#1491) 2022-06-04 07:28:58 -05:00
Thomas Göttgens 53e9f4df46 Segemented config works for me (TM)
Small GPS Fix that cropped up while testing included.
2022-05-21 22:38:33 +02:00
Sacha Weatherstone 6b0ce6b729 Finish config transition 2022-05-07 20:31:21 +10:00
Thomas Göttgens c8254f9ec8 SerialDebug NRF
On slow systems, it takes up to 5 seconds to init serial
2022-03-26 11:15:57 +01:00
Thomas Göttgens 54d802f15d Wait up to 2 seconds for port being connected.
This will send serial debug log from the beginning and not somewhere in between.
2022-03-25 23:06:31 +01:00
Kevin Hester 17a1262382 pull in new epaper lib (which required importing configuration.h everywhere
for dumb reasons)
2021-06-27 10:56:28 -07:00
Kevin Hester d179bda728 serious bug: connection to phones not being properly tracked 2021-05-03 14:46:30 +08:00