firmware

12,651 Commits 189 Branches 329 Tags

2 Commits

Include renames

Author	SHA1	Message	Date
Ben Meadors	b565a07a83	Remove proprietary Bosch BSEC blob; open in-tree IAQ estimator for BME680 (#11381 ) * Remove proprietary Bosch BSEC blob; open in-tree IAQ estimator for BME680 BSEC2 cost ~37-39 KB flash and ~4-5 KB static RAM on ~190 of ~240 build targets, linked whether or not a BME680 was attached, and was a no-source proprietary archive inside GPLv3 release binaries. The firmware consumed exactly one BSEC-exclusive output: the IAQ value. - New BME680IaqEstimator: clean-room log-domain baseline tracker (humidity-compensated gas resistance vs a rise-fast/decay-slow ceiling, 0-500 scale matching the existing UI bands), pure math, unit-tested on native (test_bme680_iaq, 15 tests incl. a deep-sleep reboot simulation). Warm-up/burn-in progress persists to /prefs/bme680.dat via SafeFile so one-sample-per-wake SENSOR nodes converge across reboots; stale /prefs/bsec.dat is removed once. - BME680Sensor: single-path rewrite on Adafruit_BME680 with async once-per-minute sampling (~20x lower heater duty than BSEC LP mode), a hard 2-minute publish-freshness bound (a dead sensor stops reporting instead of freezing its last reading on the wire), and suppression of bogus gas_resistance=0 points from heater-unstable cycles. - platformio.ini: environmental_extra_common/_extra/_no_bsec collapsed into one section; Bosch BSEC2 + BME68x deps deleted; per-variant BSEC link-path hacks and the TEMPORARY promicro lib_ignore removed. nrf52_promicro_diy_tcxo regains BME680 support at 36 KB clear of the warm-store cap; rak4631 lands at 75 KB clear. - EnvironmentTelemetry: iaq rendering gates on has_iaq (a genuine IAQ of 0 now displays); stale BSEC comments rewritten. - rak4631 size budgets tightened (113000->108000 RAM, 786000->746000 flash) to lock in the reclaimed headroom. - bin/bme680_iaq_replay.cpp: host-side replay harness for tuning the estimator against captured BSEC traces (mean abs error + band agreement), no reflashing needed. Measured (develop -> this branch): rak4631 -38.8 KB flash / -4.9 KB RAM; heltec-v3 -36.4 KB / -4.0 KB; tlora-v2-1-1_6 +1.3 KB (its IAQ approximation had been dead code since #9663 due to an inverted isfinite check and now actually runs). Note: gas_resistance stays kOhm on the wire for fleet compatibility; the proto comment claiming MOhm gets a separate meshtastic/protobufs docs PR. * Address CodeRabbit review feedback - Use Throttle::isWithinTimespanMs for all elapsed-time predicates in BME680Sensor per coding guidelines (deadline math for the async reading completion stays raw, as it targets an absolute timestamp) - Make the state file name members static constexpr - Replay tool: cast uint16_t before %u (default argument promotion), report malformed input lines instead of silently skipping, and fail non-zero on stream read errors * Address CodeRabbit nitpicks - Replace the local clampf helper with std::clamp (meshUtils.h's clamp drags in Arduino.h, which would break the estimator's standalone host build that the replay harness depends on) - Trim the replay tool's file header to a two-line summary; the full build, capture, and tuning workflow moves to docs/bme680_iaq_replay.md	2026-08-13 13:21:16 -04:00
Ben Meadors	24c1dccf00	CI: track RAM (.data+.bss) in size reports and gate on per-env budgets (#10899 ) * CI: track RAM (.data+.bss) in size reports and gate on per-env budgets The 2.8.0 nRF52840 heap regression (99% heap in field reports) shipped invisibly because CI only tracked flash. On nRF52840 the heap arena is the linker gap after .bss, so every byte of static .data+.bss growth shrinks the usable heap 1:1 - RAM needs the same guardrails flash already has. What's added: - bin/platformio-custom.py emits ram_bytes (.data + .bss from the ELF, via the toolchain size tool) into each .mt.json manifest. Heap/stack placeholder sections are deliberately excluded. - bin/collect_sizes.py records {flash_bytes, ram_bytes} per env; bin/size_report.py grows RAM and RAM-delta columns in the PR size report. Older artifacts without ram_bytes (and legacy int-schema baselines) degrade to "n/a" instead of crashing. - bin/ram_budgets.json: per-env RAM/flash budgets, enforced only for envs listed there. Seeded with rak4631: ram 113,000 (current 110,948 + ~2 KB slack), flash 786,000 (current 765,192 + ~20 KB; the app region is 0x27000..0xEA000 = 798,720 and the image must stay clear of the warm-store ring guard in extra_scripts/nrf52_warm_region.py). - New size-budget-gate CI job runs size_report.py --enforce-budgets and fails the build on violation; the informational firmware-size-report job now also renders budget usage into the PR comment. - src/main.cpp: opt-in boot heap watermark (-DMESHTASTIC_HEAP_WATERMARK_CHECK) logs LOG_ERROR when less than 20% of the heap is free at the end of setup(). Off by default; skipped on platforms without heap accounting. How budgets are raised: deliberately, never automatically. If a change needs more headroom, bump the env's limit in bin/ram_budgets.json in the same PR and justify the increase in the PR description. Verified: python3 bin/test_size_scripts.py (23/23 pass, including ram_bytes parsing, n/a fallback, and over/under/missing-env budget-gate cases). * Address review: fail the budget gate closed, fix RAM section matching - size-budget-gate workflow: drop continue-on-error on the manifest download and the empty-dir fallback, so the job fails when the data it gates on cannot be fetched. - size_report.py --enforce-budgets now fails closed on every missing-data path instead of trivially passing: empty collected sizes, a budgeted env that was not built, or a manifest without the budgeted metric. Report-only mode keeps rendering those as n/a. - load_budgets() rejects zero/negative/non-integer budgets with a clear error (a typo'd budget could previously crash budget_markdown with ZeroDivisionError or silently skip the check); the percentage render keeps a defensive guard for direct callers. - compute_ram_bytes(): count RISC-V small-data sections (.sdata/.sbss) and exclude ESP-IDF .rtc.* sections, which live outside the heap-competing SRAM. - Trim the heap-watermark comment in main.cpp to two lines. bin/test_size_scripts.py: 27/27 - the two fail-open assertions are flipped to fail-closed, with new report-only counterparts plus cases for empty sizes under enforcement and malformed budgets.	2026-07-06 14:52:22 -05:00

Author

SHA1

Message

Date

Ben Meadors

b565a07a83

Remove proprietary Bosch BSEC blob; open in-tree IAQ estimator for BME680 (#11381 )

* Remove proprietary Bosch BSEC blob; open in-tree IAQ estimator for BME680

BSEC2 cost ~37-39 KB flash and ~4-5 KB static RAM on ~190 of ~240 build
targets, linked whether or not a BME680 was attached, and was a no-source
proprietary archive inside GPLv3 release binaries. The firmware consumed
exactly one BSEC-exclusive output: the IAQ value.

- New BME680IaqEstimator: clean-room log-domain baseline tracker
  (humidity-compensated gas resistance vs a rise-fast/decay-slow ceiling,
  0-500 scale matching the existing UI bands), pure math, unit-tested on
  native (test_bme680_iaq, 15 tests incl. a deep-sleep reboot simulation).
  Warm-up/burn-in progress persists to /prefs/bme680.dat via SafeFile so
  one-sample-per-wake SENSOR nodes converge across reboots; stale
  /prefs/bsec.dat is removed once.
- BME680Sensor: single-path rewrite on Adafruit_BME680 with async
  once-per-minute sampling (~20x lower heater duty than BSEC LP mode),
  a hard 2-minute publish-freshness bound (a dead sensor stops reporting
  instead of freezing its last reading on the wire), and suppression of
  bogus gas_resistance=0 points from heater-unstable cycles.
- platformio.ini: environmental_extra_common/_extra/_no_bsec collapsed
  into one section; Bosch BSEC2 + BME68x deps deleted; per-variant BSEC
  link-path hacks and the TEMPORARY promicro lib_ignore removed.
  nrf52_promicro_diy_tcxo regains BME680 support at 36 KB clear of the
  warm-store cap; rak4631 lands at 75 KB clear.
- EnvironmentTelemetry: iaq rendering gates on has_iaq (a genuine IAQ of
  0 now displays); stale BSEC comments rewritten.
- rak4631 size budgets tightened (113000->108000 RAM, 786000->746000
  flash) to lock in the reclaimed headroom.
- bin/bme680_iaq_replay.cpp: host-side replay harness for tuning the
  estimator against captured BSEC traces (mean abs error + band
  agreement), no reflashing needed.

Measured (develop -> this branch): rak4631 -38.8 KB flash / -4.9 KB RAM;
heltec-v3 -36.4 KB / -4.0 KB; tlora-v2-1-1_6 +1.3 KB (its IAQ
approximation had been dead code since #9663 due to an inverted isfinite
check and now actually runs).

Note: gas_resistance stays kOhm on the wire for fleet compatibility; the
proto comment claiming MOhm gets a separate meshtastic/protobufs docs PR.

* Address CodeRabbit review feedback

- Use Throttle::isWithinTimespanMs for all elapsed-time predicates in
  BME680Sensor per coding guidelines (deadline math for the async reading
  completion stays raw, as it targets an absolute timestamp)
- Make the state file name members static constexpr
- Replay tool: cast uint16_t before %u (default argument promotion), report
  malformed input lines instead of silently skipping, and fail non-zero on
  stream read errors

* Address CodeRabbit nitpicks

- Replace the local clampf helper with std::clamp (meshUtils.h's clamp drags
  in Arduino.h, which would break the estimator's standalone host build that
  the replay harness depends on)
- Trim the replay tool's file header to a two-line summary; the full build,
  capture, and tuning workflow moves to docs/bme680_iaq_replay.md

2026-08-13 13:21:16 -04:00

Ben Meadors

24c1dccf00

CI: track RAM (.data+.bss) in size reports and gate on per-env budgets (#10899 )

* CI: track RAM (.data+.bss) in size reports and gate on per-env budgets

The 2.8.0 nRF52840 heap regression (99% heap in field reports) shipped
invisibly because CI only tracked flash. On nRF52840 the heap arena is the
linker gap after .bss, so every byte of static .data+.bss growth shrinks the
usable heap 1:1 - RAM needs the same guardrails flash already has.

What's added:

- bin/platformio-custom.py emits ram_bytes (.data + .bss from the ELF, via
  the toolchain size tool) into each .mt.json manifest. Heap/stack
  placeholder sections are deliberately excluded.
- bin/collect_sizes.py records {flash_bytes, ram_bytes} per env;
  bin/size_report.py grows RAM and RAM-delta columns in the PR size report.
  Older artifacts without ram_bytes (and legacy int-schema baselines)
  degrade to "n/a" instead of crashing.
- bin/ram_budgets.json: per-env RAM/flash budgets, enforced only for envs
  listed there. Seeded with rak4631: ram 113,000 (current 110,948 + ~2 KB
  slack), flash 786,000 (current 765,192 + ~20 KB; the app region is
  0x27000..0xEA000 = 798,720 and the image must stay clear of the
  warm-store ring guard in extra_scripts/nrf52_warm_region.py).
- New size-budget-gate CI job runs size_report.py --enforce-budgets and
  fails the build on violation; the informational firmware-size-report job
  now also renders budget usage into the PR comment.
- src/main.cpp: opt-in boot heap watermark (-DMESHTASTIC_HEAP_WATERMARK_CHECK)
  logs LOG_ERROR when less than 20% of the heap is free at the end of
  setup(). Off by default; skipped on platforms without heap accounting.

How budgets are raised: deliberately, never automatically. If a change needs
more headroom, bump the env's limit in bin/ram_budgets.json in the same PR
and justify the increase in the PR description.

Verified: python3 bin/test_size_scripts.py (23/23 pass, including ram_bytes
parsing, n/a fallback, and over/under/missing-env budget-gate cases).

* Address review: fail the budget gate closed, fix RAM section matching

- size-budget-gate workflow: drop continue-on-error on the manifest
  download and the empty-dir fallback, so the job fails when the data it
  gates on cannot be fetched.
- size_report.py --enforce-budgets now fails closed on every
  missing-data path instead of trivially passing: empty collected sizes,
  a budgeted env that was not built, or a manifest without the budgeted
  metric. Report-only mode keeps rendering those as n/a.
- load_budgets() rejects zero/negative/non-integer budgets with a clear
  error (a typo'd budget could previously crash budget_markdown with
  ZeroDivisionError or silently skip the check); the percentage render
  keeps a defensive guard for direct callers.
- compute_ram_bytes(): count RISC-V small-data sections (.sdata/.sbss)
  and exclude ESP-IDF .rtc.* sections, which live outside the
  heap-competing SRAM.
- Trim the heap-watermark comment in main.cpp to two lines.

bin/test_size_scripts.py: 27/27 - the two fail-open assertions are
flipped to fail-closed, with new report-only counterparts plus cases for
empty sizes under enforcement and malformed budgets.

2026-07-06 14:52:22 -05:00