Files
Andrew Tridgell 93e0d01798 testsuite: fail closed on every malformed skip-list spec
Review of the previous commit found four ways the parser or its test fell
short of what that commit claimed:

  - An empty or comment-only list, and an empty entry within a spec (`a,,b`,
    which is what an unset shell variable expands to), both expanded quietly
    to a smaller expected set.  A shrunken expectation is a weaker oracle, so
    these are hard errors now; a wholly empty spec remains the legitimate
    "expect no skips".
  - Name validation accepted a path, so `../testsuite/acls` passed as a test
    name.  Names must be plain and resolve to a regular file.
  - skiplist-spec_test.py built @FILE paths from a relative srcdir and handed
    them back to a srcdir-relative API, which doubled the prefix -- it would
    have failed under `make installcheck` (--srcdir=../src).  It also proved
    the sort check with a name that does not exist, so deleting that check was
    masked by the stale-name check; it now uses two real tests out of order,
    and covers the cases above.  Each guard verified by mutation.
  - fleettest returned an extras-only expected set for a pass whose workflow
    has no matching step (a non-Linux target with protocols=[29]), a
    guaranteed mismatch.  An unpinned lane is now simply unpinned.

Also restores the fleettest CI path filter that the previous commit dropped:
that job must run when runtests.py or a skip list changes.
2026-07-31 21:02:55 +10:00

74 lines
3.0 KiB
YAML

name: Test fleettest harness
# Bitrot check for testsuite/fleettest.py (the developer fleet CI harness).
# fleettest is meant to be run by developers on a modern Ubuntu box, so this
# job runs only on ubuntu-latest: it stands up a one-host "fleet" of two
# targets that both ssh to localhost and runs a real fleettest pass against it.
# It does not run on the BSD/Solaris/macOS/Cygwin matrix.
on:
push:
branches: [ master ]
paths:
- 'testsuite/fleettest.py'
- '.github/workflows/fleettest.yml'
- 'runtests.py'
- 'testsuite/skiplist/**'
- 'testsuite/skiplist-spec_test.py'
pull_request:
types: [opened, synchronize, reopened, labeled]
paths:
- 'testsuite/fleettest.py'
- '.github/workflows/fleettest.yml'
- 'runtests.py'
- 'testsuite/skiplist/**'
- 'testsuite/skiplist-spec_test.py'
workflow_dispatch:
schedule:
- cron: '17 7 * * 1'
jobs:
fleettest:
# temporary gate: PR CI runs only for PRs labeled 'run-ci', to save CI
# minutes; labels need triage access, so fork PRs can't self-enable.
if: github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'run-ci')
runs-on: ubuntu-latest
name: fleettest against localhost
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y gcc g++ gawk autoconf automake \
acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev \
python3-cmarkgfm openssl rsync openssh-server
- name: set up ssh to localhost
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
ssh-keygen -t ed25519 -N '' -f ~/.ssh/id_ed25519
cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
sudo systemctl start ssh || sudo service ssh start
# fleettest connects with `ssh -o BatchMode=yes localhost`, which won't
# answer a host-key prompt -- so pre-trust localhost in known_hosts.
ssh-keyscan -H localhost 127.0.0.1 >> ~/.ssh/known_hosts 2>/dev/null
ssh -o BatchMode=yes -o ConnectTimeout=15 localhost 'echo ssh-to-localhost-ok'
- name: write localhost fleet config
run: |
cat > fleettest-ci.json <<'EOF'
{ "targets": [
{ "name": "local-a", "ssh_host": "localhost", "workflow": "none.yml",
"configure_flags": [], "builddir": "rsync-citest-a", "privilege": "sudo" },
{ "name": "local-b", "ssh_host": "localhost", "workflow": "none.yml",
"configure_flags": [], "builddir": "rsync-citest-b", "privilege": "sudo" }
] }
EOF
- name: fleettest --list (config sanity)
run: python3 testsuite/fleettest.py --fleet fleettest-ci.json --list
- name: run fleettest against localhost
# Two targets both on localhost exercise the parallel multi-target path
# and the per-run dir / port isolation; exit 0 iff every cell is OK.
run: python3 testsuite/fleettest.py --fleet fleettest-ci.json --timing