Commit Graph

11 Commits

Author SHA1 Message Date
Andrew Yong
1eb860a3fc fix(stm32wl,nrf52,fs): flash hardening, FS platform unification, write-behind LFS cache (FORMAT BREAK) (#10171)
* stm32wl: check HAL_FLASH_Unlock() return in _internal_flash_erase

_internal_flash_prog already checks HAL_FLASH_Unlock() and returns
LFS_ERR_IO on failure. _internal_flash_erase discarded the return
value, proceeding to erase even if the flash was not unlocked.

Apply the same check for consistency and safety.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl: fix _internal_flash_prog to abort on first write error

Previously the programming loop continued to the next doubleword after
HAL_FLASH_Program() failed, potentially writing to invalid addresses
and returning a misleading error code only at the end (last iteration).
HAL_FLASH_Lock() was also skipped on the mid-loop early return path.

- Move bounds check before the loop (validate full range at once)
- Break on first HAL error so subsequent doublewords are not written
- Move HAL_FLASH_Lock() after the loop so it always runs

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl: clear stale flash SR error flags before erase and program

Stale error flags in FLASH->SR from a previous failed operation can
cause HAL_FLASH_Program() or HAL_FLASHEx_Erase() to return HAL_ERROR
immediately without attempting the operation.

Add __HAL_FLASH_CLEAR_FLAG(FLASH_FLAG_ALL_ERRORS) after each
HAL_FLASH_Unlock() in both _internal_flash_prog and
_internal_flash_erase to ensure a clean state before each operation.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl: reject flash prog writes not aligned to 8-byte doubleword

The STM32WL HAL minimum write unit is one 64-bit doubleword (8 bytes).
_internal_flash_prog silently truncated any trailing bytes when size % 8
!= 0 because dw_count = size / 8 drops the remainder. Return LFS_ERR_INVAL
early so LittleFS sees the error rather than a silent short write.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(nrf52,fs): use atomic SafeFile rename instead of direct write

NRF52 was bypassing the .tmp/readback/rename path entirely — openFile()
deleted the target file and wrote directly to it, and close() returned
true without verifying the write or renaming anything.

Adafruit_LittleFS::rename() calls lfs_rename() directly (confirmed at
Adafruit_LittleFS.cpp:205). Remove both ARCH_NRF52 guards so NRF52
follows the same write-to-.tmp → readback-hash → rename path used by
all other platforms.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(admin): skip uiconfig.proto save on devices without a screen

handleStoreDeviceUIConfig() was writing /prefs/uiconfig.proto
unconditionally. MenuHandler.cpp is already gated behind #if HAS_SCREEN,
so there is no path that populates UI config on screen-less platforms.
Guard the save with #if HAS_SCREEN to avoid wasting a flash block on
devices that will never use it.

The read path (handleGetDeviceUIConfig) does not touch the filesystem
and needs no change.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fs: enable format-on-retry for all platforms in saveToDisk

The FSCom.format() call on save failure was guarded to ARCH_NRF52 with
a comment that other platforms were not ready (bug #4184). STM32WL was
added to the guard in a prior commit. All platforms now expose format
semantics and the retry logic is identical — remove the guard.

To keep NodeDB.cpp platform-agnostic and fix a CI failure on native-tft
(portduino's fs::FS has no format() method), introduce fsFormat() in
FSCommon as the single call-site for all callers:

  - Embedded (ESP32, NRF52, STM32WL, RP2040): delegates to FSCom.format()
  - Portduino: rmDir("/prefs") + FSBegin() (a no-op on portduino).
    rmDir("/prefs") is already called unconditionally by factoryReset()
    (NodeDB.cpp:504), so both primitives are proven on portduino.

Replace both direct FSCom.format() calls in NodeDB.cpp with fsFormat().

Note: we do not run portduino locally — portduino/native build testers
please verify the format-on-retry path.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* DO NOT MERGE: nrf52(fs): add File() default constructor bound to InternalFS

Adds File() to the Adafruit LittleFS File class (in the Meshtastic
Adafruit_nRF52_Arduino fork), delegating to File(InternalFS). This
matches the default-constructible File API on all other platforms.

The constructor is implemented in Adafruit_LittleFS_File.cpp rather
than inline in the header to avoid a circular include between
Adafruit_LittleFS_File.h and InternalFileSystem.h.

FOLLOW-UP REQUIRED: nrf52.ini points to a commit SHA on the
mesh-malaysia/Adafruit_nRF52_Arduino fork instead of the upstream
meshtastic framework. Once meshtastic/Adafruit_nRF52_Arduino#5 is
merged, revert nrf52.ini to point back to the upstream meshtastic
framework URL.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl(fs): add File() default constructor and document LFS tunables

Adds File() to STM32_LittleFS_Namespace::File, delegating to
File(InternalFS). Implemented in the .cpp to avoid a circular include
between STM32_LittleFS_File.h (which cannot include LittleFS.h) and
the InternalFS extern declaration.

This matches the File API on ESP32/RP2040/Portduino and is a
prerequisite for removing the ARCH_STM32WL guard in xmodem.h.

No behavior change — the constructor leaves the file in the same
closed/unattached state as File(InternalFS) would.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fs: remove arch-specific ifdefs from FSCommon, SafeFile, xmodem

Now that NRF52 and STM32WL have File() default constructors and NRF52
has working atomic SafeFile rename, the capability gaps are closed.
Remove all per-arch guards across the shared FS layer:

FSCommon.cpp — renameFile():
  Use FSCom.rename() on all platforms. Adafruit_LittleFS::rename()
  calls lfs_rename() directly (Adafruit_LittleFS.cpp:205). The
  copy+delete fallback on NRF52/RP2040 was never necessary.

FSCommon.cpp — getFiles():
  Replace four ARCH_ESP32 guards with a single filepath pointer at
  the top of the loop (file.path() on ESP32, file.name() elsewhere).
  Fix strcpy(fileInfo.file_name, filepath): bounded to
  sizeof(fileInfo.file_name)-1 with explicit NUL termination to prevent
  overflow of the 228-byte meshtastic_FileInfo::file_name array.

FSCommon.cpp — listDir():
  Same filepath pointer approach. NRF52/STM32WL were in an else-branch
  that only logged but never deleted — now all platforms follow the
  unified del path. 12 guards → 2.
  Fix three strncpy(buffer, ..., sizeof(buffer)) calls that did not
  NUL-terminate when source length >= sizeof(buffer) (255 bytes).
  Add explicit buffer[sizeof(buffer)-1] = '\0' after each.

FSCommon.cpp — rmDir():
  Use listDir(del=true) everywhere. The ARCH_NRF52 rmdir_r() path and
  the ARCH_ESP32|RP2040|PORTDUINO listDir() path collapse to one line.

SafeFile.cpp:
  ARCH_NRF52 bypass removed (handled in preceding commit).

xmodem.h:
  File file; now works on all platforms via default constructors
  added in the two preceding commits.

Remaining #ifdef ARCH_ESP32 in FSCommon.cpp: exactly 4, all for the
file.path() vs file.name() API difference (ESP32 Arduino LittleFS
returns the full path; all others return only the name). That
difference lives in the framework and cannot be closed here.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl(fs): add write-behind page cache, reduce virtual block size and FS reservation (FORMAT BREAK)

Adds a write-behind (RMW) page cache to the STM32WL LittleFS driver,
modelled after the NRF52 Adafruit approach (flash_cache.c). This allows
LFS to use 256-byte virtual blocks backed by 2048-byte physical pages:
the erase/prog callbacks accumulate changes in a 2 KB RAM buffer; the
sync callback (and page eviction on page-change) flushes with a single
HAL physical-erase + doubleword-program pass.

LFS tunables changed (FORMAT BREAK — superblock parameters):
  block_size:  2048 B → 256 B  (8 virtual blocks per physical page)
  read_size:   2048 B → 256 B  (= block_size)
  prog_size:   2048 B → 256 B  (= block_size; hardware min is 8 B)
  block_count: 112   → 80     (14 phys pages → 10 phys pages = 20 KiB)

Benefits:
  - Internal fragmentation: max 2047 B/file → max 255 B/file
  - Heap per open LFS file: ~4 KB → 512 B (prog + read buffers)
  - Code flash headroom: 6.7 KB → ~14.1 KB (+7.4 KB)
  - Block budget: 80 virtual blocks, worst-case peak ~20, ~60 free

Updates board_upload.maximum_size in wio-e5/platformio.ini from 233472
(256 KB − 28 KB) to 241664 (256 KB − 20 KB) to match the reduced FS
reservation.

Justification for the format break: the prior STM32WL firmware had
several flash write bugs fixed earlier in this series (missing error
flag clearing, no abort on first write failure, unaligned write
acceptance). These bugs very likely caused silent config corruption on
deployed devices. The format break should be treated as an enhancement:
it provides a clean, reliably-written starting point. Users will need
to reconfigure their device once after this update.

Correctness fixes applied to the cache implementation:
  - alignas(8) on _page_cache: the buffer was uint8_t[] (alignment 1)
    but _flash_cache_flush casts it to const uint64_t* — undefined
    behaviour per C++ standard, potential Cortex-M hardfault. alignas(8)
    guarantees the required alignment for the doubleword cast.
  - HAL_FLASH_Lock() return value: was discarded. Now assigned to
    lock_rc and propagated into rc if prior writes succeeded, so LFS
    sees the error rather than a false success.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl(fs): reduce FS reservation from 10 pages to 7 pages (FORMAT BREAK)

Reduces LFS_FLASH_TOTAL_SIZE from 10 × 2 KiB pages (20 KiB) to
7 × 2 KiB pages (14 KiB), freeing 6 KiB for firmware.

board_upload.maximum_size updated accordingly across all STM32WL variants:
  241664 (256 KiB - 20 KiB) → 247808 (256 KiB - 14 KiB)

This is a FORMAT BREAK: existing filesystems must be erased before use.

Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Andrew Yong <me@ndoo.sg>

* fix(fs): return false in renameFile() when FSCom is not defined

Avoids undefined behavior and -Wreturn-type warnings in configurations
that compile FSCommon.cpp without a filesystem backend.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Signed-off-by: Andrew Yong <me@ndoo.sg>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-01 08:25:19 -05:00
Mike Robbins
e99853f660 SafeFile: use atomic rename-with-overwrite, rather than non-atomic delete-then-rename (#9296) 2026-01-13 05:57:04 -06:00
Ben Meadors
dc100e4d3e Cleanup 2025-03-16 08:19:46 -05:00
Ben Meadors
0d860882a8 NRF52 - Remove file totally before opening write (#5916)
* Remove prefs first

* Remove file first

* Remove truncate

* No longer needed

* Missed a param

* That wasn't supposed to be there

* Remove vestigal lfs assert

* Durr
2025-01-23 19:12:20 -06:00
Eric Severance
c4fcbad372 Reboot before formatting LittleFS (#5900)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2025-01-20 11:43:35 -06:00
Ben Meadors
85de193845 Fix NRF52 default append write mode of files (#5858)
* Fix NRF52 default append write mode of files

* Inside the lock
2025-01-15 06:46:12 -06:00
Ben Meadors
1c0f43c8e2 NRF52 SafeFile should not remove / rename files (#5840) 2025-01-13 06:28:18 -06:00
Tom Fifield
e1aaafb77a Cherrypick "add more locking for shared SPI devices (#5595) " (#5728)
* add more locking for shared SPI devices (#5595)

* add more locking for shared SPI devices
* call initSPI before the lock is used
* remove old one
* don't double lock
* Add missing unlock
* More missing unlocks
* Add locks to SafeFile, remove from `readcb`, introduce some LockGuards
* fix lock in setupSDCard()
* pull radiolib trunk with SPI-CS fixes
* change ContentHandler to Constructor type locks, where applicable

---------

Co-authored-by: mverch67 <manuel.verch@gmx.de>
Co-authored-by: GUVWAF <thijs@havinga.eu>
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com>

* mesh-tab: lower I2C touch frequency

---------

Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: mverch67 <manuel.verch@gmx.de>
Co-authored-by: GUVWAF <thijs@havinga.eu>
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com>
2025-01-03 10:05:26 +08:00
Thomas Göttgens
05e4a639a1 remove newline from logging statements. (#5022)
remove newline from logging statements in code. The LOG_* functions will now magically add it at the end.

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2024-10-14 15:11:43 +11:00
geeksville
62a0321c7d Fixes for #4395: nrf52 flash filesystem reliability (#4406)
* bug #4184: fix config file loss due to filesystem write errors
* Use SafeFile for atomic file writing (with xor checksum readback)
* Write db.proto last because it could be the largest file on the FS (and less critical)
* Don't keep a tmp file around while writing db.proto (because too big to fit two files in the filesystem)
* generate a new critial fault if we encounter errors writing to flash
either CriticalErrorCode_FLASH_CORRUPTION_RECOVERABLE or CriticalErrorCode_FLASH_CORRUPTION_UNRECOVERABLE
(depending on if the second write attempt worked)
* reformat the filesystem if we detect it is corrupted (then rewrite our config files) (only on nrf52 - not sure
yet if we should bother on ESP32)
* If we have to format the FS, make sure to preserve the oem.proto if it exists

* add logLegacy() so old C code in libs can log via our logging

* move filesList() to a better location (used only in developer builds)

* Reformat with "trunk fmt" to match our coding conventions

* for #4395: don't use .exists() to before attempting file open
If a LFS filesystem is corrupted, .exists() can fail when a mere .open()
attempt would have succeeded.  Therefore better to do the .open() in hopes that
we can read the file (in case we need to reformat to fix the FS).
(Seen and confirmed in stress testing)

* for #4395 more fixes, see below for details:
* check for LFS assertion failures during file operations (needs customized lfs_util.h to provide suitable hooks)
* Remove fsCheck() because checking filesystem by writing to it is very high risk, it makes likelyhood that we will
be able to read the config protobufs quite low.
* Update the LFS inside of adafruitnrf52 to 1.7.2 (from their old 1.6.1) to get the following fix:
97d8d5e96a

* use disable_adafruit_usb.py now that we are (temporarily?) using a forked adafruit lib
We need to reach inside the adafruit project and turn off USE_TINYUSB, just doing that
from platformio.ini is no longer sufficient.

Tested on a wio-sdk-wm1110 board (which is the only board that had this problem)

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2024-08-13 06:45:39 -05:00
geeksville
66c41e683d bug #4184: fix config file loss due to filesystem write errors (#4397)
* Use SafeFile for atomic file writing (with xor checksum readback)
* Write db.proto last because it could be the largest file on the FS (and less critical)
* Don't keep a tmp file around while writing db.proto (because too big to fit two files in the filesystem)
* generate a new critial fault if we encounter errors writing to flash
either CriticalErrorCode_FLASH_CORRUPTION_RECOVERABLE or CriticalErrorCode_FLASH_CORRUPTION_UNRECOVERABLE
(depending on if the second write attempt worked)
* reformat the filesystem if we detect it is corrupted (then rewrite our config files) (only on nrf52 - not sure
yet if we should bother on ESP32)
* If we have to format the FS, make sure to preserve the oem.proto if it exists

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2024-08-06 13:59:06 -05:00