Adds .github/workflows/ubuntu-version-mix.yml (ubuntu-latest) and a
per-release manifest testsuite/expect/rsync_<ver>.expect for each of the
nine peers. The workflow builds the current rsync, then runs the two-
sided suite against every old binary over both the pipe and --use-tcp
daemon transports. All peers run in a SINGLE looped job (not a matrix)
so the PR shows one check line; each peer/transport is a foldable log
group and a failure annotates which one broke.
A new phony `check-progs` target builds rsync plus the test helper
programs and check symlinks without running the suite -- the build half
of `make check` -- so the workflow's direct runtests.py invocation has
the helpers it needs.
Notable expected results encoded in the manifests:
- The four May-2026 security tests xfail against every released peer:
the suite demonstrates each release is vulnerable to those findings
while current master is fixed.
- symlink-dirlink-basis xfails on 3.4.0/3.4.1 (issue #715: their
secure_relative_open O_NOFOLLOW-confines the basedir, breaking a -K
dir-symlink update; current master fixes it with secure_basis_open).
- Older peers carry more xfails for options/negotiation they lack;
2.6.0 (protocol 27) fails most daemon tests. reverse-daemon-delta
passes against all peers, confirming backward compat down to 2004.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nine statically-linked, stripped binaries for the version-mixing test
suite (and ad-hoc cross-version behaviour checks): every x.y.0 release
from 2.6.0 (2004, protocol 27) through 3.4.0, plus the 3.1.3/3.2.7/3.4.1
point releases. 2.6.0 is the practical floor; older tags need more
porting to build on a current toolchain.
build_static.sh rebuilds any release from its git tag, applying the
minimal patches needed to compile old sources on a modern toolchain:
K&R lseek64 redecl, gettimeofday, -std=gnu11, --disable-openssl, and
_FORTIFY_SOURCE disabled (modern FORTIFY=3 turns latent benign over-reads
in old rsync into aborts when it runs as a server). Pre-3.0 trees ship
configure.in, so it regenerates configure (autoheader/autoconf) after
neutralizing the dead AC_LIBOBJ replacement fallbacks, generates proto.h,
and stubs the dropped vendored lib/addrinfo.h -- all guarded to no-op on
newer versions.
.gitattributes marks the binaries binary (so the text=auto rule can't
corrupt them) and export-ignore (kept out of the release tarball).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every other two-sided test drives with the current binary, covering
new-client -> old-server. This adds the backward-compat direction that
matters most for a project shipping new servers to a world of old
clients: a current daemon must keep serving the installed base of old
rsync clients.
reverse-daemon-delta_test.py starts the daemon with the current build
(via start_test_daemon's rsync_cmd override) and drives it with the old
binary. It does a push and a pull, each with and without -z, with the
receiving side pre-seeded with an older version of the file so the delta
algorithm actually runs -- exercising delta encoding both ways (old->new
on push, new->old on pull) and compression negotiation both ways. It
asserts the bytes crossing the wire are far smaller than the file, so a
silent fallback to a whole-file copy is caught, and accepts both the
modern "sent/received" and the old "wrote/read" summary wording so an
old client's output parses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Let the suite run with two rsync binaries so the current build can be
tested against the actual old code of a previous release, rather than
only forcing the current binary to speak an old protocol (check29/30).
--rsync-bin2 PATH exports RSYNC_PEER, the binary used for the SERVER
side of two-sided transfers (the daemon process and
the remote-shell --rsync-path target). Defaults to
RSYNC, so single-binary runs are byte-for-byte
unchanged.
--expect-result F the manifest's listed tests ARE the run set; each
test's actual outcome (pass/skip/fail/xfail) is
compared to its expected one and any mismatch --
including an unexpected pass (xpass) -- fails the
run. --expect-skipped and the default exit logic
are untouched.
rsyncfns gains the RSYNC_PEER global and launches the daemon with it
(start_rsyncd / start_test_daemon, the latter with an optional rsync_cmd
override used by the reverse-direction test); the remote-shell tests
pass --rsync-path={RSYNC_PEER}. All no-ops when no peer is selected.
Direction is fixed: the current binary always drives (only it
understands the new test scripts); the old binary is only ever the
server/daemon side.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some tests cannot run in certain build/CI environments. In particular the
protected-regular test self-re-execs under "unshare --map-users" to exercise
fs.protected_regular handling, and that user-namespace path hangs in a
restricted buildd chroot (e.g. Launchpad/sbuild), tripping the per-test
timeout and failing the whole "make check".
Add an --exclude option (comma-separated test names/globs), with an
RSYNC_EXCLUDE environment fallback so it can be set without touching the
make/check command line. Excluded tests are dropped before running -- they
are neither executed nor reported as skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the new ppa:rsyncproject/rsync-latest (development snapshots rebuilt
from git master) alongside the existing stable PPA in INSTALL.md and the
download page. Notes that snapshot versions (3.5.0~git...) sort below the
matching stable release, so the two PPAs can coexist without a stable
release being silently replaced by a snapshot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
when a symlink is to the same directory as the source then it can be
considered unsafe if it goes via a path outside the directory.
This came up on the mailing list, added a test to make the case clear
GitHub Actions artifact storage is approaching our quota. Each `make`/build
job uploads its rsync binary + manpages, the coverage job uploads its full
HTML tree, and Android uploads its dist/ -- 11 jobs producing artifacts per
PR/push, all kept for the repo default of 90 days.
Set retention-days: 45 explicitly on every upload-artifact step so they
expire at half the previous lifetime; older artifacts can still be re-built
from the commit if needed. No other workflow behaviour changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tests are launched with subprocess.run(..., cwd=TOOLDIR) so the
subprocess's argv[0] resolves against TOOLDIR, not the runner's
invocation cwd. A user-supplied --rsync-bin=../foo/rsync therefore
worked when invoked from inside TOOLDIR but silently failed (or
ENOENT'd inside individual tests) when invoked from a sibling
directory.
Fix: absolutize rsync_bin via os.path.abspath() at parse time, before
it propagates into build_rsync_cmd()/RSYNC. abspath() captures
os.getcwd() now, which is the operator's invocation cwd -- exactly
what the --rsync-bin=../path form expresses.
Regression check:
cd /tmp/somewhere-else
ln -s /path/to/rsync ./alt/rsync
python3 /path/to/rsync-git/runtests.py \
--rsync-bin=./alt/rsync \
--srcdir=/path/to/rsync-git --tooldir=/path/to/rsync-git \
00-hello
Before this commit the test failed at subprocess time with the relative
path being looked up under TOOLDIR; after, it passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds .github/workflows/actionlint.yml which runs rhysd/actionlint over
.github/workflows/*.yml on push and PR to master. Triggers only when
something in .github/workflows/ (or the actionlint config) changes, so
the rest of the platform matrix isn't billed when nothing here moves.
The job downloads a pinned actionlint binary (1.7.12) via the upstream
download script (which verifies a SHA256) -- no third-party Action
dependency, matching the inline-install style of the existing
ubuntu/macos/cygwin workflows. Bump the pinned version deliberately.
actionlint catches a) GitHub Actions expression / type errors, b)
unsupported runner images, c) missing secrets / inputs, and d) the
embedded shellcheck class of issues in 'run:' scripts that the previous
commit cleaned up. Keeping it in CI prevents regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
actionlint (rhysd/actionlint) reported a handful of shellcheck-class issues
across the GitHub Actions workflows. All are 1-line mechanical fixes:
* Replace legacy backticks in --rsync-bin=`pwd`/rsync with
--rsync-bin="$PWD/rsync" (SC2006 + SC2046; almalinux-8-build,
macos-build, ubuntu-22.04-build, ubuntu-build).
* Quote >>$GITHUB_PATH redirects as >>"$GITHUB_PATH"
(SC2086; coverage, macos-build, ubuntu-22.04-build, ubuntu-build).
After this commit `actionlint .github/workflows/*.yml` exits 0.
(Also cleaned up 6 editor backup *.yml~ files from the local working
tree; those weren't tracked -- *~ is gitignored -- so the cleanup is
local-only and not part of this commit.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
symlink-dirlink-basis assert the --backup file holds the pre-update content,
not merely that the backup file exists.
acls-default check that clearing the inherited default ACL actually
succeeded, so the no-default-ACL cases can't silently
test against the scratch dir's seeded default ACL.
alt-dest assert --copy-dest produces a distinct inode from the
alt-dir candidate (a copy, not a hard link) -- the
property that distinguishes it from --link-dest, which
checkit's tree comparison alone doesn't capture.
(crtimes' "independently pin the historical create time" gap is left as-is: the
touch-trick pinning is APFS-specific and not locally verifiable, and a mistuned
probe would make the test skip on macOS and break its expected-skip set.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace loose/partial oracles with exact ones:
omit-times under -O, require EVERY directory mtime to be omitted, not
just one (the old "at least one differs" missed partial bugs).
dir-sgid assert the created dirs' actual gid: a setgid parent makes
them inherit its group (set to a secondary group to be
discriminating), while the non-setgid case gets the process's.
relative-implied pin a deterministic umask and assert the exact default mode
(0o755) for --no-implied-dirs, not merely "not the source's".
safe-links / compare the preserved symlink TARGET strings via readlink,
unsafe-links not just that a symlink exists.
preallocate verify do_punch_hole via st_blocks on the --inplace --sparse
case (guarded by a sparse-capability probe).
Note: --preallocate --sparse leaves the file fully allocated on a fresh write
(the zero run is not punched), so that case stays content-only rather than
asserting hole-punching -- see the test comment; rsync.1's claim that the
combination yields sparse blocks does not hold for the fresh-write path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several tests proved only that rsync exited cleanly (or that a file merely
exists), so a no-op/short transfer would pass:
protected-regular compare the dst bytes to the source after --inplace.
00-hello re-assert one/two were copied on the RSYNC_OLD_ARGS=1
env-var path (the explicit --old-args case already did).
missing check the dry-run's exit status in test 1.
mkpath compare transferred bytes (not just existence) and add a
negative control: a transfer WITHOUT --mkpath must fail
and create no intermediate path.
size-filter compare each kept file's content to its source.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These daemon tests confirmed refusals/exclusions but accepted the allowed
transfers on exit status alone, so a transfer that exited cleanly while moving
nothing would pass:
daemon-refuse allowed() imported verify_dirs but never called it; now it
confirms the allowed push/pull actually populated the dest.
daemon-filter pull()/the incoming push ignored their exit status, and the
outgoing-chmod loop iterated only files that exist -- a
zero-file pull passed vacuously. Check the codes and require
at least one file to have been mode-checked.
daemon run_and_check's unused `expected` param is dropped; the
hidden-module and glob listings now compare the exact set of
listed paths (catching a leaked extra path), replacing the
per-path containment check and the dead normalise() helper
whose regex never matched the -r listing format anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The symlink-race tests only asserted that an outside sentinel was unchanged or
unlisted while ignoring rsync's exit status, so an attack transfer/listing that
failed before reaching the vulnerable receiver/sender path would pass without
the security property ever being exercised. Add a positive control to each --
an ordinary in-module write (bare-do-open, chdir) or an in-module listing
(sender-flist-leak) that must succeed -- so a globally broken/refusing daemon
can no longer make the sentinel checks vacuous, and assert the attack run did
not die from a signal.
clean-fname-underflow now also enforces a non-zero exit: clean_fname()
collapses "a/../test" to "test", whose merge file is absent, so rsync must
reject it; accepting it (rc 0) would mean the crafted name was mis-collapsed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several subcases ran rsync without checking the exit status, so a silent
failure could pass as the expected (often empty) output -- most notably -q,
which only asserted empty stdout. Route every expected-success run through a
helper that asserts the exit status, and verify -q actually transferred the
tree. Replace the "-h/-8 didn't break the transfer" check with positive format
assertions: -h must render byte counts with a K/M/G suffix (and the default
must not), and -8 must leave a high-bit filename byte unescaped (\#371 absent)
where the default escapes it -- best-effort, self-skipping where the platform
can't store the raw byte.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compress-options only checked that each requested algorithm yielded
byte-identical output, which proves parsing/non-corruption but not that the
advertised algorithm was actually used -- the test would pass if the choice
were silently ignored. Capture --debug=NSTR (compat.c / checksum.c) and assert
the selected compressor, compress level, and checksum match the request
(anchored so zlib != zlibx). --skip-compress / --checksum-seed stay content
checks: they have no comparable negotiation-string signal.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both fuzzy tests asserted only that the final file content matched, which a
full transfer that ignored --fuzzy would also satisfy -- so a broken fuzzy
basis selection would pass undetected. Drive rsync directly with --debug=FUZZY
and assert the generator reports the expected basis ("fuzzy basis selected
for <f>: <basis>", generator.c find_fuzzy): rsync2.c for fuzzy, and the
closest-named candidate archive-v1.tar for fuzzy-basis. fuzzy switches from
checkit() to a manual run plus verify_dirs() so the output can be captured.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rsync.1 says combining --preallocate with --sparse yields sparse blocks
wherever the filesystem can punch holes, but since 2019 (commit c2da3809,
"keep file-size 0 when possible") it has silently left the file fully
allocated. Two problems, both rooted in that commit switching --preallocate /
--inplace to fallocate(FALLOC_FL_KEEP_SIZE):
* do_fallocate() then returned 0 instead of the reserved length, so the
receiver's preallocated_len was 0 and write_sparse() always lseek'd over
null runs instead of punching them (and the over-preallocation trim in
receiver.c never fired either).
* more fundamentally, KEEP_SIZE leaves the file size at 0 while data is
written incrementally, so the FALLOC_FL_PUNCH_HOLE call lands on blocks
beyond EOF and is a silent no-op -- the reserved blocks are never freed.
Fix both: don't request KEEP_SIZE when --sparse is also active, so the file is
preallocated at full size and the punch lands within it; and return the
reserved length from do_fallocate() so preallocated_len drives the punch
decision and the over-allocation trim. --preallocate without --sparse keeps
the KEEP_SIZE (file-size-0) behaviour. t_stub.c gains a sparse_files stub since
do_fallocate now references it and the test helpers link syscall.o.
preallocate_test.py now asserts via st_blocks (where the filesystem can punch
holes) that --preallocate --sparse ends up sparse, guarding the regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The OpenBSD job runs inside a nested VM. At -j8 the --use-tcp run starts
many concurrent loopback daemons, and under that resource pressure the
daemon connection handshake occasionally loses a timing race and one test
hangs to the 300s runner timeout. It is an environment artifact, not an
rsync defect: the daemon handshake writes-then-reads with unbuffered early
I/O (no flush/mutual-wait deadlock), the indefinite wait is the documented
no-timeout daemon behaviour, and it does not reproduce off OpenBSD even with
the full suite pinned to a single CPU at -j8.
Drop just this job's --use-tcp parallelism to -j2 so the nested VM stops
over-subscribing; the pipe `make check` and every other platform are
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Target previously-uncovered functions in the path/file-operation files the
resolver restructure touches, confirmed hit under coverage:
preallocate --preallocate (syscall.c do_fallocate) and sparse hole-punching
via --preallocate --sparse and --inplace --sparse (do_punch_hole),
on a file several levels deep.
fuzzy-basis --fuzzy basis selection with similar-named candidates and no
exact match, so the generator scores them (util1.c fuzzy_distance).
delete-deep add a --backup --delete case so removing an extraneous
backup-suffixed file consults delete.c is_backup_file.
preallocate probes --preallocate support up front and skips where it is
unavailable: macOS, the *BSDs and Solaris build without fallocate/posix_fallocate
(and FALLOC_FL_PUNCH_HOLE is Linux-only), and reject the option outright. It runs
on Linux and Cygwin. fuzzy-basis and delete-deep are plain local transfers with
no skips. All green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The --expect-skipped check compared the skip list as an ordered string, so the
per-platform RSYNC_EXPECT_SKIPPED lists had to match runtests' collection order
(sorted filenames) exactly -- a subtle, easy-to-break ordering dependency.
Compare the skipped SET instead; which tests skipped is what matters.
Register the new require_tcp test daemon-access-ip in the per-platform
expected-skipped lists (it skips in the pipe-transport make check, like
daemon-chroot-acl and proxy-response-line-too-long).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The coverage report counted bundled third-party code (zlib/, popt/, and the
PostgreSQL/ISC lib/ imports getaddrinfo/getpass/inet_ntop/inet_pton) that rsync
ships but does not own, muddying the percentages. Add a COVERAGE_EXCLUDE gcovr
filter (shared by all coverage targets) so the report reflects rsync's own code:
on the same data, lines 63.9%->65.5%, functions 81.4%->85.0%, branches
55.0%->56.5% (rsync's own md5/mdfour/wildmatch/etc. stay in the report).
Add 'make coverage-all': run the suite under pipe + --protocol=30 + --protocol=29
+ --use-tcp, accumulating into the shared .gcda (not cleared between runs), then
one merged scoped report -- covers the daemon/TCP and protocol-compat paths a
single pipe run misses (lines 67.6%, functions 87.6%, branches 58.6%). Also add
'make coverage-fallback' for a separate --disable-openat2 build (different .gcno,
so it can't merge with the openat2 report). CI is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
acls-depth skips where ACLs/setfacl are unavailable (macOS, Cygwin) like the
existing acls tests, and sparse skips on APFS (macOS), where a seek-written
hole isn't allocated sparsely. Add them to the per-platform RSYNC_EXPECT_SKIPPED
lists so the skip-set assertion stays accurate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Builds with --enable-coverage and runs the suite under both transports
(make coverage, then make coverage-tcp). gcovr's line/branch/decision totals
are printed to the step log and also written to the GitHub step summary, so the
coverage numbers are visible directly in the CI output; the HTML reports are
uploaded as an artifact. make coverage exits with the suite's status, so a test
regression fails the job.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
coverage-tcp reuses the coverage recipe with --use-tcp (daemon tests over a real
loopback rsyncd, which also runs the require_tcp-only tests) and a separate
report directory, via COVERAGE_RUNFLAGS / COVERAGE_DIR. Verified end to end:
pipe run reports 63.9% lines, the TCP run 64.5% (it exercises more code).
Also drop gcovr's --branches flag: it is deprecated in gcovr 8 and branch +
decision coverage still appear in --print-summary and the HTML without it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
partial_test.py sub-test 5 deterministically asserts a delta (--no-whole-file)
resume from an absolute, outside-tree --partial-dir reproduces the source and
consumes the basis -- the regression guard for the receiver fix. Sub-test 4
keeps asserting the cross-directory partial WRITE on interrupt. Drop the
--whole-file workaround and the 'broken on master' notes in the docstring and
COVERAGE.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A delta (--no-whole-file) resume whose basis is an absolute --partial-dir
looped forever on exit code 23 ("failed verification -- update put into
partial-dir"), stranding the correct data in the partial-dir and never
populating the destination.
Cause: an absolute --partial-dir makes the basis path absolute, but the
receiver opened it with secure_relative_open(NULL, fnamecmp, ...), which by
design rejects an absolute relpath (EINVAL). The basis fd was then -1, so
receive_data() mapped no basis and (because the matched-block sum_update() is
guarded by "if (mapbuf)") computed the whole-file verification checksum over
the literal data only -> a spurious mismatch every run. (The data itself was
correct, since the in-place update leaves the matched basis bytes in place.)
Under a non-chroot daemon the in-place write went through the same call and
failed outright.
Fix: add secure_basis_open(), which treats an operator-trusted absolute basis
path as (trusted directory + confined leaf) -- the same way secure_relative_open
already trusts an absolute basedir while keeping O_NOFOLLOW on the leaf -- and
use it for both the basis read and the inplace-partial write. The strict
"reject absolute relpath" contract of secure_relative_open is left intact.
Defense-in-depth: receive_data() now treats a block-match token with no mapped
basis as a protocol inconsistency (it can only arise from a basis that the
generator opened but the receiver could not), failing cleanly instead of
silently dropping those bytes from the verify checksum or the output.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
COVERAGE.md is the living checklist mapping every CLI option (~142) and daemon
parameter (~54) to its test(s), with depth / cross-dir status and remaining
gaps, so the path-resolution restructure can see exactly what is guarded.
update_test.py closes two of the documented gaps: -u/--update (keep a newer
destination, update an older one) and --force (replace a non-empty destination
directory with a file), both at depth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two test-coverage build knobs (both behaviour-neutral by default):
--enable-coverage appends '--coverage -fprofile-update=atomic -O0' and adds
a 'make coverage' target (whole suite, run serially, then
gcovr HTML with branch + decision coverage). rsync forks
and its children exit without running the gcov atexit
flush -- the generator via its SIGUSR1 handler
(_exit_cleanup) and the receiver via the SIGUSR2 handler
-- so under GCOV_COVERAGE we call __gcov_dump() at both, or
receiver.c/generator.c record no coverage at all.
--disable-openat2 gates the Linux openat2(RESOLVE_BENEATH) sites in syscall.c
on HAVE_OPENAT2 (defined by default), so disabling it forces
the portable per-component O_NOFOLLOW resolver to run as the
primary on ordinary Linux -- exercising and
coverage-counting that fallback tier without a pre-5.6
kernel. NOTE: coordinate with the parallel syscall.c
path-resolution restructure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add resolve_beneath_supported() to rsyncfns: it functionally probes whether the
rsync binary can follow an in-tree directory symlink under its secure resolver
(an initial transfer plus a delta update through a dir-symlink, the operation
issue #715 is about). This tracks the actual binary instead of a platform name.
Use it in symlink-dirlink-basis_test.py in place of the SunOS/OpenBSD/NetBSD/
Cygwin name check: it skips on those platforms too, and additionally on
Linux < 5.6, a seccomp-blocked openat2, and the new --disable-openat2 build,
where the portable O_NOFOLLOW fallback rejects the in-tree symlink.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Breadth pass for options not yet exercised:
output-options output shape of --version/--help/-i/-n/--stats/
--out-format/--list-only/-q/--progress/-h/-8 (these control
output, not path handling, so they're checked for shape).
compare -c and -I catch a stealth change (same size+mtime, new
content) deep in the tree; --size-only skips a same-size
change; --modify-window absorbs a 1s mtime difference.
compress-options --compress-choice for every advertised compressor,
--compress-level, --skip-compress, --checksum-choice for
every advertised checksum, and --checksum-seed -- each a
clean byte-identical transfer at depth.
Green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drive a loopback daemon (secure stdio-pipe transport by default, also green
under --use-tcp) via the new write_daemon_conf helper and assert the behaviour
of the security-relevant rsyncd.conf parameters, transferring >=3-deep trees:
daemon-access path / read only / write only / list, incl. a deep sub-path
pull and that a list=no module is hidden yet usable by name.
daemon-filter daemon exclude hides matching files everywhere; incoming /
outgoing chmod rewrite modes of every transferred file.
daemon-auth auth users + secrets file accept the right password, reject a
wrong one and an unauthenticated request; strict modes rejects
a world-readable secrets file.
daemon-exec pre-/post-xfer exec run with RSYNC_MODULE_NAME /
RSYNC_EXIT_STATUS; a failing pre-xfer exec aborts the transfer
(marker files polled for, since post-xfer exec runs after the
client disconnects under TCP).
daemon-munge munge symlinks stores incoming links with the /rsyncd-munged/
prefix and strips it on the way out.
daemon-refuse refuse options: a named option, a wildcard, and the '* !a !v'
allow-list idiom.
Green on master under pipe and --use-tcp transports and under --protocol=29.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Assert exactly which entries are/aren't transferred, deep in the tree:
filter-depth --exclude/--include precedence on files at every level, and
a -F per-directory .rsync-filter loaded from a deep dir that
applies to that subtree only (not above it).
cvs-exclude -C built-in cruft patterns (*.o, *~) at every level plus a
deep per-directory .cvsignore scoped to its subtree.
size-filter --max-size / --min-size select the right files all the way
down.
files-from-depth --files-from selects only the listed deep paths (implied
parents created); --from0 NUL-delimited; --exclude-from /
--include-from filter at depth.
(--existing / --ignore-existing are covered in delete-deep_test.py.)
Green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Set each attribute distinctively on a file AND a directory at every level of a
>=3-deep tree and verify it per entry after transfer (metadata is applied as a
single-component op on an entry whose parent chain the resolver restructure
rewrites):
metadata-depth -p preserves exact file/dir modes; -t preserves file
mtimes; --chmod=D710,F600 rewrites them.
omit-times -O omits directory times (files still preserved); -J omits
symlink times.
sparse -S preserves a deep file's hole (allocated << size);
--no-sparse fills it.
xattrs-depth -X reproduces a user xattr on every entry (gated on xattr
support).
acls-depth -A reproduces a POSIX ACL on every entry (gated on ACL
support + setfacl/getfacl).
ownership-depth --groupmap and --chown=:GROUP remap the group of every
entry (non-root, to a secondary group); -o/--usermap gated
on root.
All green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cover the structure and link options at >=3 levels and across directories,
asserting each option's specific effect:
links -l keeps a symlink, -L dereferences it, -k follows a
directory symlink -- all on a symlink several levels deep.
dirs -d copies the top layer (file + empty dir) without recursing.
prune-empty-dirs -m drops empty chains and chains emptied by an exclude,
keeps populated ones.
hardlinks-deep -H preserves a hard link whose names live in different
directories at depth; without -H they become separate inodes.
delete-deep --delete removes a deep extraneous file/subtree; the four
delete-timing variants agree; --max-delete caps deletions;
--existing / --ignore-existing select/skip correctly.
relative-implied -R mirrors an implied directory's mode at depth;
--no-implied-dirs does not (proto 30+).
Green on master and under --protocol=29/30 (the --no-implied-dirs sub-case is
gated to protocol >= 30, where multi-component sender paths are accepted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fill the highest-restructure-risk gap: options that do two-directory / rename /
outside-tree work, asserted at >=3 levels deep with the aux tree kept outside
the main tree, and asserting the option's specific property rather than just
tree equality (which the ported tests already cover).
alt-dest-deep --link-dest hardlinks unchanged files (same inode), --copy-dest
copies (never links), --compare-dest omits unchanged files;
ref tree outside both src and dest.
temp-dir cross-dir temp->final rename at depth; temp dir left clean; a
missing --temp-dir fails (so the option is proven consulted).
partial --partial keeps the partial in the dest file; relative
--partial-dir stages per-directory at depth (pre-seed +
interrupt/resume); absolute --partial-dir writes the partial
outside the tree.
inplace --inplace keeps the destination inode across a delta update;
the default temp+rename path replaces it.
append --append completes truncated files tail-only; --append-verify
repairs a corrupted prefix (protocol >= 30).
backup-deep --suffix saves <name>S beside the new file; --backup-dir
relocates old files to a parallel deep tree outside the dest
and captures deletions under --delete.
All green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add helpers for the option-coverage expansion (the path-handling restructure
changes parent-component resolution, so options must be exercised at depth and
across directory boundaries):
* make_tree() builds a tree with a regular file at every level so a property
can be checked at the tree root and >=3 levels deep;
* walk_files()/walk_dirs() iterate entries for per-level assertions;
* assert_same/assert_mode/assert_mtime_close/assert_is_symlink/
assert_hardlinked/assert_not_hardlinked/assert_exists/assert_not_exists
assert the concrete property an option controls (not just dest == src);
* write_daemon_conf() writes an arbitrary rsyncd.conf (globals + modules)
for daemon-parameter tests, beyond build_rsyncd_conf's fixed four modules;
* forced_protocol() lets protocol-sensitive tests gate sub-cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
xattr_set() sets attributes with the native os.setxattr(), but
xattr_dump() read them back by running "getfattr -d". That asymmetry
breaks "make check" on any system where rsync is built with xattr
support (libattr headers present) but the attr package's CLI tools are
not installed -- common on Android/Termux and minimal CI images: setting
succeeds via os.setxattr, then xattr_dump's getfattr raises
FileNotFoundError, which crashes the test (reported FAIL) instead of
running or skipping it. That's why "make check" was failing here on
xattrs / xattrs-hlink.
Read the xattrs natively with os.listxattr()/os.getxattr() on Linux,
symmetric with xattr_set(), so the suite needs no external getfattr; the
output still mimics "getfattr -d" and only has to be self-consistent
between the source and destination dumps. Cygwin keeps the CLI path
(Python there lacks os.*xattr). make check now passes with no attr
package installed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Python rewrite of the suite carried over the shell habit of
populating the test tree by capturing "ls -l /etc" / "ls -l /bin"
(falling back to "ls /"): hands_setup() built etc-ltr-list / bin-lt-list
that way, and longdir_test.py did the same for its leaf files. That ties
the fixtures to the host filesystem layout -- those directories are
absent or unreadable on Android/Termux and other minimal environments,
where "ls /" fails outright -- and the captured content was never
reproducible from run to run.
Add a deterministic make_text_file() helper to rsyncfns.py and use it for
hands_setup()'s two fixture files and longdir's leaf files. The names
etc-ltr-list / bin-lt-list are unchanged (chmod, chmod-temp-dir and
alt-dest reference them by name); only the content source changes, so the
fixtures are now self-contained and identical on every platform. This
also drops longdir_test.py's date(1) and ls(1) subprocess calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a link to the rsync Discord server (https://discord.gg/Avfvy9zhdp)
below the mailing lists section in README.md and on the lists.html web
page.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Python rewrite had gated the xattr / fake-super tests (xattrs,
xattrs-hlink, chown-fake, devices-fake) to Linux because it used the
Linux-only os.*xattr. Restore them on macOS, FreeBSD, Cygwin and Solaris
via a per-OS xattr surface in rsyncfns.py (xattrs_supported / xattr_set /
xattr_dump):
* Linux -- os.*xattr
* macOS -- xattr
* FreeBSD -- setextattr / lsextattr / getextattr
* Cygwin -- getfattr / setfattr (from the `attr` package; CPython on
Cygwin has no os.*xattr)
* Solaris -- runat(1), with the script on stdin and the attr name/value
passed via the environment (the runat -c form mangles args)
Test attribute names are logical; the "user." namespace prefix is added
only on the Linux-style platforms (Linux, Cygwin). RSYNC_PREFIX/RUSR vary
per OS (macOS and Solaris use rsync.nonuser to avoid rsync's reserved
rsync.* space). The macOS and Cygwin workflows no longer skip these tests;
the FreeBSD/Solaris jobs use IGNORE skip-checking so need no change.
Verified on real Linux, macOS, FreeBSD, Cygwin and Solaris hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chmod-option: pin umask to the suite-wide 022 baseline (mirroring the
old rsync.fns) so rsync's --chmod `D+w` is computed and applied under
the same umask -- fixes failures under a different ambient umask (077).
* daemon module-list test: assert the `list = no` module does NOT leak
into the listing (the substring check alone missed regressions).
* claim_ports() lock file: open with O_NOFOLLOW and only fchmod a file we
O_EXCL-created, rejecting a symlink OR hard link planted at the
well-known /tmp path -- which, with the TCP tests running under sudo in
CI, could otherwise chmod an arbitrary root-owned target. Require a
pristine (regular, nlink==1) file.
* CI: extend the Linux/Cygwin expected-skip lists for the gated tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
socketpair_tcp() fakes a connected socket pair via a loopback TCP
self-connect (socket -> bind 127.0.0.1:0 -> listen -> connect ->
accept), used by sock_exec() for RSYNC_CONNECT_PROG. Its comment has
long promised that "nobody else can attach to the socket, or if they
do that this function fails", but nothing actually verified it: the
code accept()ed whatever connection arrived first without checking it
was the one our own connect() made.
Between listen() and accept() the ephemeral loopback port is
connectable by any local user. With backlog 1 a same-host attacker who
races a connection in before our connect() lands could have their
socket returned by accept(), handing them one end of the rsync
protocol stream. The exposure is small (loopback only, random
ephemeral port, sub-millisecond window, local users only), but the
promised guarantee was simply not enforced.
Enforce it: after the connection is established, require that the peer
address of the accepted end (fd[0]) equals the local address of our
connecting end (fd[1]), and that both are 127.0.0.1. A hijacked
connection has a different source port and is rejected (errno EPERM,
fail closed). The legitimate self-connect always matches, so there is
no behaviour change for the normal path.
Verified: rebuilds clean with -Wall -W; the full testsuite still
passes in both transports (pipe `make check` 57/3, `runtests.py
--use-tcp` 59/1) -- the pipe transport exercises this code path on
every daemon test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Daemon-mode tests default to the stdio-pipe transport (RSYNC_CONNECT_PROG),
which opens no listening socket -- so `make check` never exposes a network
service. Real TCP is opt-in via `runtests.py --use-tcp`, with the daemon
bound to loopback (127.0.0.1) on a claim_ports()-reserved port; CI runs the
suite both ways.
start_test_daemon() is the single seam every daemon test uses: the secure
pipe by default, a real rsyncd on a claimed loopback port under --use-tcp.
Tests with no pipe equivalent (the fake-proxy listener and the reverse-DNS
hostname-ACL daemon test) are gated behind require_tcp().
`make check` also now runs the suite in parallel by default (CHECK_J=8);
the claim_ports() byte-range locks make that safe across concurrent runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rsyncfns.claim_ports(*ports) takes exclusive POSIX byte-range locks on
/tmp/rsync_test.lck (offset = port number) so any number of test
processes can run concurrently without colliding on a TCP port: a test
asking for a port already held blocks until the holder exits. The
kernel drops the locks automatically when the holding process dies, so
a crashed test releases its ports with no manual cleanup.
Ports are claimed in sorted order so two callers requesting the same
set in different orders can't deadlock. The lock file is forced to
mode 0o666 after creation (the umask would otherwise trim it and lock
out a second user on a shared CI runner; EPERM when we're not the
owner is fine).
proxy-response-line-too-long is the first user: it switches from an
ephemeral port to a claimed fixed port (12873).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
