mirror of
https://github.com/meshtastic/firmware.git
synced 2026-09-16 08:30:04 -04:00
Jonathan BennettandClaude Fable 5
af56a11f00
* Derive the native suite count on the fly instead of registering it in a file test/native-suite-count was a manually-maintained register of the test_* directory count, reconciled against the actual directories by bin/run-tests.sh (as an AMBER verdict) and by a dedicated suite-count-check CI job. The reconciliation only ever guarded the file itself: the check that matters - suites that actually ran vs. the test_* directories on disk - already derives its expected count from a directory walk, so the file added a bookkeeping step to every suite addition/removal without adding signal. Remove the file and everything that existed to keep it honest: - bin/run-tests.sh: drop the canonical-count file read, the count-mismatch AMBER verdict, and the [canonical: x/y] suffix; the verdict lines already carry ran/expected from the directory walk. The shuffle seed suffix stays. - test_native.yml: delete the suite-count-check job and its needs: edges. - Docs (copilot-instructions.md, AGENTS.md, test/README.md) and the test-script comments now describe the count as derived from test/test_* at run time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APCEfNjd1X7ErDHEzT6Dqd * Add suite-shrinkage-check: fail a PR that silently loses a test_* suite With test/native-suite-count gone, nothing in CI noticed the suite set shrinking: platformio test discovers and runs whatever test_* directories exist, and bin/run-tests.sh derives its expected count from the same walk, so a suite directory lost in a bad rebase or an overzealous cleanup just means fewer suites run - every remaining check stays green. Restore that tripwire git-aware instead of file-based: on pull_request runs, compare the test_* directory list at the PR's merge base against the PR result. A vanished suite fails the job unless its name appears in the PR title, PR body, or a commit message in the PR's range - a deliberate removal satisfies that by stating what it removes; an accidental loss cannot. Other events skip: they have no natural base, and PRs are where accidents arrive. No job depends on this one (a skipped job would skip its dependents). Incidentally: test/ currently holds 47 test_* directories while the deleted count file said 46 - the manual register had already drifted, which is exactly the bookkeeping failure mode this replaces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APCEfNjd1X7ErDHEzT6Dqd * Re-pad the verdict table after shortening the AMBER row Shrinking the AMBER cell left the table's column padding inconsistent, which trunk (prettier + markdownlint MD060) rejects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APCEfNjd1X7ErDHEzT6Dqd --------- Co-authored-by: Claude <noreply@anthropic.com>
103 lines
4.7 KiB
Bash
Executable File
103 lines
4.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Self-test for bin/lint-unity-exit.sh.
|
|
#
|
|
# This exists because the scanner has been wrong twice in review, both times in a way that looked
|
|
# fine by inspection: layered regexes cannot tokenise C++, so a `/*` inside a string literal flipped
|
|
# comment state, a greedy `.*` swallowed code between two comments, `myexit(...)` matched the `exit`
|
|
# exemption as a substring, and `==` matched the assignment exemption. Every one of those is pinned
|
|
# below as a fixture, so the next rewrite has to keep them all passing.
|
|
#
|
|
# Each fixture is a snippet of C++ plus the exact diagnostics it must produce, as a comma-separated
|
|
# list of <line>:<col> - empty for none. Asserting the locations rather than just "did it say
|
|
# anything" is what catches a rule that reports the right number of findings in the wrong places, or
|
|
# that collapses two findings on one line into one.
|
|
#
|
|
# Not a Unity suite and not a test_* directory, so outside the suite count run-tests.sh derives
|
|
# from test/ - same arrangement as bin/test-state-check.sh, and for the same reason: it asserts
|
|
# the behaviour of a process.
|
|
#
|
|
# Usage: ./bin/test-lint-unity-exit.sh (exit 0 = all fixtures behaved)
|
|
|
|
set -uo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
|
cd "$ROOT_DIR" || exit 1
|
|
|
|
WORK="$(mktemp -d -t meshlintunity.XXXXXX)"
|
|
trap 'rm -rf "$WORK"' EXIT
|
|
mkdir -p "$WORK/test/probe"
|
|
|
|
PASSES=0
|
|
FAILURES=0
|
|
|
|
# expect "<line>:<col>[,<line>:<col>...]" <label> <body> ("" = no diagnostics expected)
|
|
expect() {
|
|
local want="$1" label="$2" body="$3"
|
|
local f="$WORK/test/probe/case.cpp"
|
|
printf '%s\n' "$body" >"$f"
|
|
|
|
# Reduce each diagnostic to line:col. The message text is asserted once, separately, so a
|
|
# reworded message does not churn every fixture here.
|
|
local got
|
|
got="$(cd "$WORK" && "$SCRIPT_DIR/lint-unity-exit.sh" test/probe/case.cpp |
|
|
awk -F: '{printf "%s%s:%s", (NR > 1 ? "," : ""), $2, $3}')"
|
|
|
|
if [[ $got == "$want" ]]; then
|
|
echo " PASS $label - [${want:-none}]"
|
|
PASSES=$((PASSES + 1))
|
|
else
|
|
echo " FAIL $label - expected [${want:-none}], got [${got:-none}]"
|
|
FAILURES=$((FAILURES + 1))
|
|
fi
|
|
}
|
|
|
|
echo "Terminating forms (must NOT be reported):"
|
|
expect "" "exit(UNITY_END()) on one line" 'void s() { UNITY_BEGIN(); exit(UNITY_END()); }'
|
|
expect "" "exit( and the macro on separate lines" 'void s() {
|
|
exit(
|
|
UNITY_END());
|
|
}'
|
|
expect "" "capture-then-exit" 'void s() { const int rc = UNITY_END(); restore(); exit(rc); }'
|
|
|
|
echo
|
|
echo "Bare calls (MUST be reported):"
|
|
expect "1:27" "plain bare call" 'void s() { UNITY_BEGIN(); UNITY_END(); }'
|
|
expect "2:27" "bare call in an #else branch" '#else
|
|
void s() { UNITY_BEGIN(); UNITY_END(); }
|
|
#endif'
|
|
# `return` finalises the report and returns a count; it does not terminate the runner, and there is
|
|
# no main() under test/ from which it would.
|
|
expect "1:19" "return UNITY_END() does not terminate" 'void s() { return UNITY_END(); }'
|
|
# `exit` must be a whole identifier, not a suffix of some other function.
|
|
expect "1:19" "myexit(UNITY_END()) is not exit()" 'void s() { myexit(UNITY_END()); }'
|
|
# The assignment exemption is for capture; comparison and compound assignment are not capture.
|
|
expect "1:21" "== is not an assignment" 'void s() { if (x == UNITY_END()) return; }'
|
|
expect "1:21" "+= is not an assignment" 'void s() { total += UNITY_END(); }'
|
|
|
|
echo
|
|
echo "Comments and literals (the two classes that broke it before):"
|
|
expect "" "line comment mentioning the macro" 'void s() { exit(UNITY_END()); } // call UNITY_END() at the end'
|
|
expect "" "block comment interior mentioning the macro" '/* a comment
|
|
that mentions UNITY_END()
|
|
across lines */
|
|
void s() { exit(UNITY_END()); }'
|
|
expect "" "the macro inside a string literal" 'void s() { TEST_MESSAGE("call UNITY_END() when done"); exit(UNITY_END()); }'
|
|
expect "1:34" "a string containing /* must not open a comment" 'void s() { const char *p = "/*"; UNITY_END(); }'
|
|
expect "1:24" "code between two block comments on one line" 'void s() { /* first */ UNITY_END(); /* second */ }'
|
|
expect "1:36" "escaped quote inside a string does not end it" 'void s() { const char *p = "a\"b"; UNITY_END(); }'
|
|
|
|
echo
|
|
echo "Multiple occurrences on one line (count and caret must both be right):"
|
|
# The caret must land on the BARE call at column 31, not the wrapped one at 17.
|
|
expect "1:31" "one wrapped and one bare on the same line" 'void s() { exit(UNITY_END()); UNITY_END(); }'
|
|
expect "1:12,1:25" "two bare calls on one line report twice" 'void s() { UNITY_END(); UNITY_END(); }'
|
|
|
|
echo
|
|
if ((FAILURES > 0)); then
|
|
echo "RESULT: RED lint-unity-exit self-test - $FAILURES of $((PASSES + FAILURES)) fixtures behaved unexpectedly"
|
|
exit 1
|
|
fi
|
|
echo "RESULT: GREEN lint-unity-exit self-test - $PASSES/$PASSES fixtures behaved as specified"
|
|
exit 0
|