operator-path-backup-chown probes for a cross-device directory, because
make_backup() renames into a same-filesystem backup dir and never reaches
the set_file_attrs() path the test is about. On macOS there is normally
no such directory, so the test is a macOS-wide expected skip -- but this
target puts the scratch trees on a separate HFS+ volume, which supplies
exactly the condition it was looking for.
Same reason backup-crossdev-copy and chmod-temp-dir are already omitted
here. The test PASSES on the target (10.9s); only the expected-skip
bookkeeping was wrong.
The base gained 13 commits. Three needed hand-porting because the change
lands on code the VFS split moved or renamed:
- 31130112 (--confine-root) is almost entirely in syscall.c, which does
not exist here. abspath_excluded_by_module() becomes
abspath_outside_confinement() in vfs/dirstack.c, taking its root from
vfs.module_dir when we are a daemon and from confine_root otherwise,
plus the fd-pin helpers; ona_open() in vfs/owner_walk.c gains the
getcwd() seed and the pin_transit exemption. The VFS passes is_operator
as an argument where the base reads operator_path_resolve, so the
refusal takes it from there rather than the deleted global.
- d09edb85 (--link-dest hard-link fallback) keeps the VFS call
vfs_link_at(cmpbuf, fname, !am_daemon ? VFS_OPERATOR_PATH : 0, 0) and
adopts the base's cannot_hardlink/match_level fallback around it.
- t_stub.c gains the confine_root/confine_rootlen stubs beside the VFS
curr_dir note.
Tree is byte-identical to the merge oracle (tag merge-reference-11).
The previous commit gave set_file_attrs()' path-based chmod/chown the
operator ownership walk, but nothing exercised the chown side: the
instrumented branch was reached 92 times by the existing suite, all of
them chmod. That is the same gap that let the upstream fix ship inert.
The uncovered path is a SYMLINK backup. op_pin cannot pin a symlink --
there is no O_NOFOLLOW open of one -- so make_backup()'s set_file_attrs()
falls through to the full-path lchown. Without the walk, a backup parent
flipped to an attacker-owned symlink redirects that lchown onto a victim
outside the backup tree and retags it as the attacker's.
Two things make this awkward to test, and both are why it was missed:
- On one filesystem make_backup() hard-links or renames the item into
the backup dir and never calls set_file_attrs() at all. The fixture
therefore puts the backup dir on tmpfs so link and rename fail EXDEV
and the recreate path runs. backup-crossdev-copy probes the same way.
- A statically planted symlink proves nothing: rsync's own backup-dir
validation deletes a non-directory component before using it. The
plant has to be a live flip, as in operator-path-backup-symlink.
The positive control checks both that the symlink reached the backup tree
and that the backup copy carries the attacker's uid -- i.e. that an lchown
actually ran. Without that second assertion the race would pass
vacuously on any build where the chown never happens.
RED on the parent of the previous commit (victim outside/f94 retagged
away from root); GREEN here. Runs on Linux CI as root; registered as an
expected skip on Cygwin (root-only) and macOS (root-only, and no
cross-device tmpfs -- backup-crossdev-copy is listed there for the same
reason). Full expected-skip oracle passes exactly.
Base commit 0bfcd3b0 taught do_chmod_at()/do_lchown_at() to resolve an
operator-supplied path through the ownership walk. The previous commit
ported that into vfs_chmod()/vfs_lchown(), but on this branch the policy
is a per-call argument rather than the ambient operator_path_resolve
global, and set_file_attrs() still passed 0 -- so the ported branch was
unreachable and the fix was inert.
op_pin already covers a reg/dir/fifo leaf with a pinned fd, which is
stronger than the walk. What it does not cover reaches the path-based
fallbacks: a symlink or device leaf never enters op_pin, and a non-root
operator can fail the pin open with an ordinary EACCES and fall through
with op_refuse clear. Both then resolved the full operator path with a
bare lchown()/chmod().
Derive the flag from ATTRS_OPERATOR_PATH and pass it to the two
VFS_AT_FDCWD fallbacks only; the held-dirfd arms stay at 0, since a
pinned parent already confines them. vfs_chmod()'s operator branch
skips S_ISLNK on its own, so a symlink-as-object keeps the
lchmod()/setattrlist() path, matching the base.
Also drop ten .gitignore entries the rebase duplicated: this branch's
own test-helper block and the base's new one list the same binaries.
Suite 259/0/83; the operator-path, backup and temp-dir families pass as
root over --use-tcp. Instrumenting the branch shows it is now reached
92 times across those tests (the chmod side); the lchown side has no
test exercising it, which is how the gap survived upstream.
The base gained 99 commits since the last rebase. Nine files needed a
hand-port because the change lands on code the VFS split moved or
renamed, and syscall.c no longer exists here:
- backup.c: make_path() now runs on a private copy of backup_dir_buf
(c933f622), so vfs_make_path() takes dirbuf and drops the restore.
- clientserver.c: keep both the module-root snapshot and the new
daemon_config_filter_file window.
- exclude.c: the peer-driven merge-file confinement is the
vfs_open_owner_walk() is_operator argument, not a global.
- fileio.c: the coalesced --sparse writer's new helpers use
vfs_lseek()/vfs_punch_hole().
- generator.c/vfs/mknod.c: gen_entry_mknod() falls back through
vfs_mknod(); the atfd path keys its mknodat() off HAVE_MKNODAT.
- receiver.c: secure_recv_open() passes VFS_OPERATOR_PATH instead of
toggling operator_path_resolve; open_readonly_inplace() uses the VFS
stat/chmod/open wrappers.
- sender.c: absolute --relative cleanup anchors at "/" via
vfs_resolve_open(), the copy-links walk uses
vfs_resolve_open_at_beneath(), and the source removal goes through
vfs_unlink().
- vfs/chmod.c, vfs/chown.c: VFS_OPERATOR_PATH now takes the ownership
walk, and the no-follow chmod grows the non-Linux fd path.
- vfs/secure_open.c: the fd-anchored resolver splits into a shared
internal with an allow-dotdot entry point, and secure_walk_at() routes
a literal "."/".." through ds_descend() before the leaf fast paths.
Tree is byte-identical to the merge oracle (tag merge-reference-10).
Linear-rebase counterpart of the conflict resolutions made when the
sec-fixes base was re-integrated (see the merge for reference). Three of
the base's new commits touch code this branch relocated or reworked:
- options.c (3fe1ed51 "rsync: confine the daemon files-from open to the
module root"): the base wraps the files-from open in the
operator_path_resolve global, which no longer exists here. Pass the
operator context explicitly instead: vfs_open_owner_walk(..., 1).
- receiver.c (cfd40f55 "receiver: confine peer-selected partial basis
paths"): the new relative-partial-basis branch uses the VFS resolver
name, vfs_resolve_open().
- vfs/chmod.c (7b16872e "syscall: silence scan-build dead-store in
do_fchmodat_nofollow fallback"): syscall.c is deleted here, so the
mode masking move and the unused-arg casts land in the relocated
do_fchmodat_nofollow.
Tree is byte-identical to the validated merge result.
Two fixes from codex review of the no-AT_FDCWD port:
- vfs/stat.c: the held-dirfd vfs_lstat branch fell back to
fstatat(dirfd, path, st, 0) when AT_SYMLINK_NOFOLLOW is unavailable,
which FOLLOWS the leaf and breaks lstat's no-follow contract. On a
system with SUPPORT_LINKS but no AT_SYMLINK_NOFOLLOW, return ENOSYS
instead (mirroring the held-fd vfs_lchown arm) so a symlink-sensitive
caller fails loud rather than silently following; the !SUPPORT_LINKS
arm keeps fstatat(...,0) since there is nothing to follow. The CI
compile-check config also undefines AT_FDCWD so no held fd is produced
there; this hardens the standalone "no AT_SYMLINK_NOFOLLOW" shape.
- Makefile.in: the vfs-no-at-fdcwd.o compile loop wrote every object to
$@, so a mid-loop failure left a fresh-timestamped $@ and a retry could
skip the check. Compile to $@.tmp and mv to $@ only after the whole
loop succeeds (rm the stale target up front); clean the .tmp too.
Linear-rebase counterpart of the conflict resolutions made when the
sec-fixes base was re-integrated (see the merge for reference). The base
gained 5cb4b829 ("syscall: build without AT_SYMLINK_NOFOLLOW") and its CI
compile-check, which touch code this branch relocated into vfs/:
- Move syscall.c's RSYNC_TEST_NO_AT_FDCWD undef block into vfs/vfs.h,
before the VFS_AT_FDCWD sentinel binds (so the sentinel takes its
no-AT_FDCWD value rather than dangling on the undefined AT_FDCWD).
- Port the AT_SYMLINK_NOFOLLOW-absent fallbacks into the relocated code:
vfs/chown.c (vfs__lchown_secure + held-fd vfs_lchown gate on
AT_SYMLINK_NOFOLLOW), vfs/stat.c (do_xstat_at's unused-arg casts, the
vfs_lstat AT_SYMLINK_NOFOLLOW-absent arm, held-fd gate), vfs/mkdir.c
(guard rand_bytes on AT_FDCWD, its only caller).
- Retarget the CHECK_COMPILE_OBJS compile-check from syscall.c to a
portable shell loop over the vfs sources built with
-DRSYNC_TEST_NO_AT_FDCWD (Makefile.in).
Tree is byte-identical to the validated merge result.
t_hashtable_overflow, t_iwildmatch, t_clean_fname, and t_safe_arg were
built by the suite but missing from .gitignore, so a stray git add -A
sweeps them into a commit (as happened during this rebase round). List
them alongside the other t_* harnesses.
daemon-exclude-namebased bound its daemon on 13010, and the
setup_chroot_inner helper hashed into 12940-13139 -- both reach into
13000+, where ASUS Armoury Crate on the Cygwin CI host parks localhost
listeners (13010, 13030-13032), making the port probe fail the test.
The helper also used str hash(), which is per-process randomized
(PYTHONHASHSEED), so its port wandered run to run.
Move the fixed port to 12931 and the helper to a deterministic
crc32-based slot in the otherwise-unused 12800-12859 band.
The #else arm for platforms without AT_FDCWD called the four-argument
vfs_symlink() with two arguments -- a compile error on such systems.
Call vfs__symlink_plain(), matching the base's do_symlink() fallback.
Found by codex review; predates this rebase round.
Linear-rebase counterpart of the conflict resolutions made when the
sec-fixes base was integrated (see the merge for reference):
- Port ddda7ba5's operator-path confinement of do_symlink_at and
do_rmdir_at into the relocated VFS code. vfs__symlink_secure gains
the VFS_OPERATOR_PATH ownership-walk branch (parent confined via
vfs_owner_walk_parent, shared leaf-creation preserved so fake-super
emulation still applies); vfs__unlink_secure extends its existing
operator branch to the rmdir/AT_REMOVEDIR case. Callers pass the
policy explicitly where the base set operator_path_resolve: the
keep_backup symlink create (backup.c), the backup-tree rmdir in
delete_item (delete.c, DEL_FOR_BACKUP), and handle_partial_dir's
rmdir (util1.c).
- Port 1f8f89c2's robust_rename EXDEV-fallback confinement into
vfs/robust.c: an absolute --temp-dir/--partial-dir operand routes the
copy_file dest-write and the source-unlink through the ownership walk
(VFS_OPERATOR_PATH), so a raced parent symlink can't redirect either
out of the module.
- Drop the stale "no ownership-walk branch" notes in vfs/vfs.h and
vfs/symlink.c now that symlink and rmdir carry the branch.
Tree is byte-identical to the validated merge result.
The HAVE_SOLARIS_ACLS facl(2) paths called secure_relpath_active() -- the
base-branch name -- which does not exist on the VFS branch (here the gate is
vfs_relpath_active()). Linux/BSD never compile that branch, so it stayed latent;
a real Solaris build fails with an implicit-declaration error. Rename both call
sites (set_rsync_acl default-ACL delete and the access/default set fallback).
Guard the two receiver decision points that drive the xattr/ACL setters down a
raw path-based branch -- set_file_attrs() and gen_entry_copy_xattrs(). In a
strict build, a confined pinnable non-operator leaf that reached the setters
without a confined fd (held_fd/xfd < 0 and not refused) aborts: that is exactly
the copy-xattrs fallback class. The guards mirror the pin/re-pin conditions, so
they cannot false-abort a legitimate transfer.
Verified: under --enable-strict-confinement the suite is 210/0 with zero aborts;
neutering the re-pin makes copy-xattrs-symlink-race abort at the guard.
Add a CI/dev hardening mode (--enable-strict-confinement) that turns any
confined-regime raw path-based metadata op into a hard abort, so a reintroduced
copy-xattrs-class fallback fails the test suite instead of silently escaping
through a flipped parent symlink.
vfs_must_be_confined() is the predicate: the modern *at/O_NOFOLLOW primitives are
present, the path is non-operator, relative and multi-component (a parent the
attacker could flip), and vfs_relpath_active(). vfs_strict_confine_fail() logs
and aborts. Both are absent/no-op without STRICT_CONFINEMENT, so there is no
production behaviour change.
set_file_attrs() pins the entry via the cached held dir fd and drives the xattr
/ACL ops off that fd (fsetxattr). But when the pin missed -- vfs_cached_dirfd()
returns -1 (its dir isn't the held one, or the path is deeper than the dirfd
cache), or the leaf openat() loses a race -- held_fd stayed -1 and
set_stat_xattr()/set_xattr()/get_acl_fdat()/set_acl_fdat() fell through to the
path-based branch (sys_lsetxattr(fname,...)). Unlike the chmod/chown/times path
wrappers (which secure-resolve), that raw lsetxattr re-resolves the parent, so a
concurrent flip of a dest parent component to a symlink->outside lands the xattr
OUTSIDE the destination tree (the intermittent copy-xattrs-symlink-race escape
that surfaces under -j load, which widens the open->setxattr window).
Re-pin through the secure resolver when the cached pin misses on a confined,
non-operator receiver path, so the xattr/ACL ops always use a confined fd -- NOT
a raw path lsetxattr; if the re-pin also fails (a genuinely raced parent/leaf
symlink) skip the path-based ops (xattr_refuse) rather than redirecting them.
The re-pin passes O_DIRECTORY for a directory leaf (the secure resolver refuses a
bare dir open with EISDIR), so dir xattrs/ACLs are still preserved on a cache
miss. Apply the same re-pin/refuse to gen_entry_copy_xattrs() (the dir xattr
copy), whose dfd<0 path likewise fell to copy_xattrs() with dest_fd==-1.
chmod/chown/times are unchanged (already confined via their *at wrappers);
operator paths keep op_pin/op_refuse.
Confirmed: under heavy parallel load the dangerous lsetxattr fallback was reached
52+ times before and 0 times after; dir xattrs preserved; suite 210/0/66,
root metadata PoC tests pass, builds +/-xattr.
Extend the t_rename_secure harness with PS-refuse/PS-follow: rename to the SAME
operator-owned escaping symlink path (oplink -> ../trap) with the SAME operator
old-side flag, differing ONLY in the new-side flag. The ownership walk (operator)
follows the operator's own uid0/euid symlink; the secure receiver resolve
(transfer, flag 0) refuses it because it leaves the cwd anchor. PS-refuse must be
rejected and PS-follow must succeed -- proving the new side's confinement is
independent of the old side's policy.
Deterministic, non-root, no race. Verified RED on the old whole-call behaviour
(emulated by OR-ing both operands' flags, the new side then follows oplink and
escapes to ../trap) and GREEN with the per-operand split. The driver adds the
oplink fixture and the per-side source files.
Suggested-by: Zen Dodd <mail@steadytao.com>
vfs_rename_at() and vfs_link_at() took a single vfs_flags applied to BOTH
operands, so VFS_OPERATOR_PATH (ownership walk: follow uid0/euid symlinks, refuse
foreign) was applied to a transfer operand even when only one side was operator-
supplied -- relaxing the secure receiver resolve that a transfer/file-list path
should get. The default branch confined each side by absolute-vs-relative, but
that missed a *relative* operator path (--backup-dir=bdir, --partial-dir=.rsync,
relative --link-dest). Reported in the rsync-private PR #30 review.
Take old_flags and new_flags separately and resolve each operand under its own
policy via a shared vfs_twopath_side() helper (operator walk if the side is
VFS_OPERATOR_PATH or absolute; secure receiver resolve if relative-with-slash;
AT_FDCWD for a bare name). An operator basis/backup path on one side can no
longer relax the other side's confinement.
Callers now pass the correct per-side policy:
- backup link/rename: source (live dest file) = transfer (0), backup target = operator.
- receiver partial-dir rename: partialptr = operator, final dest = transfer.
- hard_link_one / generator link-dest: basis source = operator (non-daemon), dest = transfer.
- finish_transfer / gen_entry_rename / robust fallback: both transfer (0,0; default per-side).
- robust_unlink ETXTBSY sibling rename: both share the caller's policy.
Also drops the operator flag on backup's "just in case" robust_unlink of the
transfer-side source.
Behaviour-preserving for the operator side (the backup/partial-dir parents still
owner-walk); it only tightens the transfer side back to the secure resolve.
Builds +/-xattr; suite 210/0/66; root operator-path PoC tests all pass.
Suggested-by: Zen Dodd <mail@steadytao.com>
A backup-tree delete (delete_item with DEL_FOR_BACKUP -- removing an existing
leaf under an absolute --backup-dir before the new backup is placed) fell through
del_unlink() to robust_unlink(fbuf, 0), which resolves the leaf's parent by path
with no ownership walk. A local attacker who flips a backup-path parent to a
symlink in that window could redirect the unlink outside the backup tree.
The base confines this: make_backup() wraps make_backup_inner() in
operator_path_resolve, so the same unlink reaches do_unlink_at()'s owner-walk
branch. The VFS refactor replaced the global with explicit vfs_flags but left
del_unlink() passing 0, dropping the confinement. Thread VFS_OPERATOR_PATH into
del_unlink()'s path-based fallback when DEL_FOR_BACKUP is set (the exact cases
that ran under the base's operator wrap); a held-dirfd delete is already confined
and ignores it. The rmdir path keeps no owner-walk branch, matching base.
The -X theirs rebase replayed the 47 VFS commits onto the new base, letting each
VFS commit win its refactor on conflict; that drops the round-3 base changes that
overlap the operator-path code (they were authored against the deleted syscall.c
and the pre-vfs_flags model). This commit snaps those files byte-for-byte from
the validated merge oracle (merge-reference-3 = 6afdd389), so the branch tip is
tree-identical to the merge that built clean +/-xattr and passed the suite +
root operator-path PoC tests.
Reconciled: the syscall.c->vfs/ ports (secure_open/owner_walk/dirstack/rename/
link/chmod/chown/times/copy_file + vfs.h/vfs_internal.h), the set_file_attrs
op_pin via ATTRS_OPERATOR_PATH (rsync.c/rsync.h + backup.c callers), the
secure_basis_open/basis_link_stat operator branches (receiver.c/generator.c), the
copy_file/gen_entry_copy_xattrs held-fd source reads, the change_dir/sender
opt-out sites (util1.c/sender.c/clientserver.c), and Makefile.in.
Linear-rebase counterpart of the round-2 merge resolution: 37dbb263 fixed
do_mknod_at()'s operator branch in syscall.c (deleted on this branch) to
fall back to mkfifoat() for a FIFO and EOPNOTSUPP for a nested socket when
mknodat() can't make a special file on the BSDs/macOS/Solaris. Re-express
it in the VFS_OPERATOR_PATH branch of vfs__mknod_secure() in vfs/mknod.c,
mirroring the secure-relpath branch already in that function.
Tree is byte-identical to the validated round-2 merge result.
Linear-rebase counterpart of the two conflict resolutions made when the
sec-fixes base was integrated (see the merge for reference):
- Port 8a7e9a16's fixed-array dirstack into the relocated VFS code:
vfs/vfs_internal.h (DS_MAXDEPTH + inline int fds[]) and vfs/dirstack.c
(drop malloc/realloc/free; ENOMEM past the cap). 8a7e9a16 changed the
dirstack in syscall.c, which this branch deleted, so the fix is
re-expressed in vfs/.
- Map the base's new F_XATTR ndx<0 guard (1d36a565) from the old
do_chmod_at() to the unified vfs_chmod(VFS_AT_FDCWD, fname, ..., 0).
Tree is byte-identical to the validated merge result.
Three issues from the post-unification review:
- The !SUPPORT_XATTRS x_stat/x_lstat macros hardcoded VFS_ALLOW_SYMLINK,
silently downgrading an operator-path stat (backup-dir) from the ownership
walk + module confinement to plain follow-stat in no-xattr builds (a
pre-existing gap the unified vfs_stat now lets us close). Forward the
caller's vfs_flags instead, so no-xattr builds get the same policy as
xattr builds.
- Held-fd ".." was accepted by the stat/chmod/lchown dispatchers, which
would operate on the PARENT of the pinned dirfd -- the wrong default for a
security boundary even though no current call site passes it. Reject ".."
while still allowing "." (a stat/chmod/chown of the dir itself, which
link_stat_at and set_file_attrs legitimately do for directory entries).
- Two stale comment/test strings still said vfs_chmod_at (now vfs_chmod).
Verified with a local --disable-xattr build; full suite 190/50.
vfs_stat/vfs_lstat are now vfs_<op>(int dirfd, path, st, int flags); the
no-xattr fallback macros still expanded to the old 2-arg vfs_stat(fn,fst),
breaking the build without xattr support. Map them to the unified plain
form: vfs_stat(VFS_AT_FDCWD, fn, fst, VFS_ALLOW_SYMLINK) (and lstat), which
preserves the prior plain-stat behaviour. Caught by the fleettest on
openbsd; verified with a local --disable-xattr build.
Rewrite the vfs.h header contract to describe the current model: the two-layer
vfs/ structure (primitives + compounds), the single vfs_<op>(int dirfd, ...,
int flags) form, the meaning of VFS_AT_FDCWD / VFS_ALLOW_SYMLINK /
VFS_OPERATOR_PATH / VFS_REMOVEDIR, that the operator policy is an explicit
per-call flag (not ambient state), and which ops keep explicit forms
(rename/link two-path, open variants, fd-based fstat/fileio).
Update the stale threat-model cross-references that pointed at the old
vfs_chmod_at() (now the static vfs__chmod_secure()) and the two
vfs_mknod_at()/vfs_symlink_at() mentions left by the unification.
Collapse the remaining three-form ops into one call each, matching
mkdir/mknod/symlink/unlink:
vfs_stat(int dirfd, path, st, flags)
vfs_lstat(int dirfd, path, st, flags)
vfs_chmod(int dirfd, path, mode, flags)
vfs_lchown(int dirfd, path, owner, group, flags)
dirfd == VFS_AT_FDCWD resolves the path (VFS_ALLOW_SYMLINK = plain libc op,
default 0 = secure receiver resolve, VFS_OPERATOR_PATH = ownership walk for
stat); a real held dirfd operates on a single component under it. The old
plain/_at/_atfd bodies become static helpers behind the dispatchers;
vfs_fstat stays (fd-based). chmod/lchown keep their no-owner-walk behaviour
(VFS_OPERATOR_PATH resolves the same as the default secure walk).
Pure API-narrowing -- the operator policy was already explicit; no global is
involved. This is behaviour-preserving (plain sites -> VFS_ALLOW_SYMLINK,
_at -> the flag they already carried, _atfd -> held-fd 0).
Held-fd validation differs by op: mkdir/mknod/symlink/unlink reject "."/".."
(creating/removing them is nonsensical), but stat/chmod/lchown ALLOW "."
(a read or metadata op on the dir itself is legitimate -- link_stat_at and
set_file_attrs do it for directory entries); only empty and multi-component
("/") names are rejected there. Caught by the suite (chmod/metadata/
ownership-depth + the implied-"." dir mkdir path).
Full suite 190/50.
The stat/lstat threading added a vfs_flags argument to x_stat()/x_lstat(),
but on a build without xattr support those are 3-arg macros in rsync.h
(mapping to plain vfs_stat/vfs_lstat), so the 4-arg call sites failed to
compile ("too many arguments to macro"). Give the macros the extra
(ignored) parameter. Caught by the fleettest on openbsd; verified with a
local --disable-xattr build.
With every operator-context call site now passing VFS_OPERATOR_PATH
explicitly, nothing reads vfs.operator_path_resolve any more. Remove the
four now-dead set/clear blocks (make_backup, handle_partial_dir, and the two
generator in-place-backup paths) and delete the field from struct vfs.
The operator-supplied path resolution policy -- which selects the ownership
walk and the daemon module-root confinement for --backup-dir/--temp-dir/
--partial-dir/--link-dest operations -- is no longer ambient dynamic-scope
state poked into the VFS from mainline. It travels as an explicit per-call
VFS_OPERATOR_PATH flag through the primitives and compounds, resolving the
layering violation that motivated this work.
Full suite 190/50.
vfs_stat_at / vfs_lstat_at (and their shared do_xstat_at helper) gain a
vfs_flags argument so the ownership-walk branch reads VFS_OPERATOR_PATH
instead of vfs.operator_path_resolve -- this was the last primitive still
reading the global. The metadata wrappers x_stat()/x_lstat() (which stay in
xattrs.c, above the VFS) gain a vfs_flags argument forwarded to the
primitives.
Call sites classified: every backup-dir stat (validate_backup_dir,
copy_valid_path's x_stat, make_backup_inner's x_lstat + bak lstat) and the
partial-dir lstat (handle_partial_dir) pass VFS_OPERATOR_PATH; the transfer
enumeration (flist link_stat + friends), the in-place dir stat (generator),
and the set_stat_xattr lstat pass 0. generator's non-daemon --link-dest
basis lookup passes 0 too (is_operator only gates the daemon module
confinement, a no-op there).
With this, NOTHING reads vfs.operator_path_resolve any more -- it is now
write-only (set by four now-dead blocks, removed next). Full suite 190/50;
backup/partial/link-dest/xattr/daemon subset green.
Give vfs_link_at an explicit vfs_flags argument (kept two-parent form) so its
ownership-walk branch reads VFS_OPERATOR_PATH instead of the global.
hard_link_one() gains a vfs_flags argument forwarded to vfs_link_at, so the
generator's alt-dest hard-link no longer sets vfs.operator_path_resolve
around the call: it passes (!am_daemon ? VFS_OPERATOR_PATH : 0), matching the
prior op = !am_daemon gating (a non-daemon --link-dest uses the ownership
walk; a daemon keeps the stronger module-anchored vfs_relpath_active path).
That operator_path_resolve block is removed.
Other call sites: the backup link (make_backup link_or_rename) passes
VFS_OPERATOR_PATH; the try_dests alt-dest link and the hlink finish-up
hard_link_one pass 0 (transfer context, as the global was there).
Set-sites: 5 -> 4 (backup make_backup, generator 2085/2130, util1
handle_partial_dir -- all wrapping the not-yet-flagged metadata layer
set_file_attrs/x_stat/x_lstat). Full suite 190/50; hardlink/link-dest/
backup/daemon subset green.
Give vfs_open_owner_walk an explicit is_operator argument so its module-root
confinement (abspath_excluded_by_module) reads the policy from the caller
instead of the vfs.operator_path_resolve global -- the last owner-walk
function still reading it. secure_basis_open gains an is_operator parameter
threaded to it; its gate (owner-walk vs strict resolve) now branches on that
param too.
Faithful conversion (no behaviour change): every direct caller passes the
value the global held at that site -- config/log/motd/early-input/files-from/
batch/connection/exclude/authenticate/params, change_dir's daemon dest-chdir,
and vfs_secure_mkstemp's --temp-dir all pass 0; secure_basis_open passes
VFS_OPERATOR_PATH only for the operator cases (a --partial-dir basis,
fnamecmp_type == FNAMECMP_PARTIAL_DIR; and one_inplace partial-dir staging).
This preserves the existing --temp-dir behaviour (unconfined) by deliberate
choice; hardening that is a separate decision.
The receiver partial-dir-basis and one_inplace operator_path_resolve blocks
are removed (receiver.c is now free of the global). Set-sites: 7 -> 5
(backup make_backup, generator 1071/2085/2130, util1 handle_partial_dir).
The operator-path-partial-dir-daemon test caught a real regression mid-change
(secure_basis_open's gate still read the global after its set/clear block was
removed, dropping the confinement); fixed by gating on is_operator.
Full suite 190/50.
Give vfs_rename_at an explicit vfs_flags argument (it keeps its two-parent
form, per the kept-explicit scope for rename) so its ownership-walk branch
reads VFS_OPERATOR_PATH instead of the vfs.operator_path_resolve global.
Call sites: the backup link/rename (make_backup) and the in-place/partial
rename (receiver) pass VFS_OPERATOR_PATH; the transfer renames (generator
entry rename, finish_transfer) and robust_rename pass 0; robust_unlink's
ETXTBSY rename-retry now forwards robust_unlink's own vfs_flags -- so
robust_unlink is fully explicit (both its unlink and its rename retry).
With unlink and rename both flagged, three operator_path_resolve set/clear
blocks that wrapped only a single now-flagged op are removed: the receiver
partial-dir rename and the two partial-dir unlinks (generator + receiver).
The remaining seven blocks still wrap not-yet-flagged ops (link, stat/lstat,
chmod/lchown via set_file_attrs, secure_basis_open) and stay for now.
Full suite 190/50; rename/partial/backup/inplace/daemon subset green.
Collapse vfs_unlink / vfs_unlink_at / vfs_unlink_atfd / vfs_rmdir /
vfs_rmdir_at into one vfs_unlink(int dirfd, const char *path, int flags).
VFS_REMOVEDIR selects rmdir/AT_REMOVEDIR; dirfd == VFS_AT_FDCWD resolves the
path, a real held dirfd removes a single validated component. The secure
body reads the operator context from VFS_OPERATOR_PATH instead of the
global; the rmdir path keeps its pre-existing no-owner-walk behaviour (it
never read the global -- matched the old vfs_rmdir_at).
robust_unlink() gains a vfs_flags argument threaded to vfs_unlink, so the
operator policy reaches it explicitly: the backup-tree callers (make_backup
link_or_rename, generator in-place backup) and copy_file's dest unlink (when
copying a backup) pass VFS_OPERATOR_PATH; delete and robust_rename's
transfer-context calls pass 0. Its ETXTBSY rename-retry still uses
vfs_rename_at (the rename primitive isn't flagged yet), which reads the
global -- still set by the surrounding backup blocks -- so behaviour is
unchanged.
Call sites classified: --remove-source-files fallback (sender) follows
symlinks (VFS_ALLOW_SYMLINK); partial-dir unlink/rmdir (handle_partial_dir)
and in-place-backup/partial unlinks operator; the rest secure or held-fd.
Full suite 190/50; delete/backup/partial/remove-source/daemon subset green.
Wiring phase: give vfs_open_at an explicit vfs_flags argument so the
operator-path resolution policy reaches it from the caller instead of the
vfs.operator_path_resolve global, and thread that argument through
copy_file()/unlink_and_reopen() to the dest create.
Call sites classified: the in-place backup open (generator) and the backup
copy (backup.c, generator in-place) pass VFS_OPERATOR_PATH; the copy-dest
basis copy (copy_altdest_file) passes 0 -- it must NOT use the ownership
walk (re-opening the copy_xattrs parent-symlink race, see generator.c
comment), which the explicit argument now expresses directly; robust_rename
runs in transfer context and passes 0.
copy_file's dest UNLINK (robust_unlink) still reads the operator global,
still set by the surrounding backup blocks, so behaviour is unchanged at
every site (flag and global agree). Full suite 190/50; backup/copy-dest/
link-dest/partial/daemon subset green.
These compiled test harnesses (t_*_secure, t_acl, simdtest) were
accidentally committed by a git add -A; ignore them like the other
built test programs (t_unsafe, wildtest, ...) so they stay out of the tree.
Relocate the file-copy / robust-unlink / robust-rename cluster from util1.c
into vfs/copy_file.c and vfs/robust.c. These are filesystem mechanics built
on the vfs_* open/read/write/unlink/rename primitives, so they belong in the
VFS compound layer; this is a verbatim move (no behaviour change).
The cluster is mutually coupled -- robust_rename falls back to copy_file for
the cross-filesystem (EXDEV) case, and copy_file's unlink_and_reopen uses
robust_unlink -- so all four move together. They call out to the metadata/
protocol layer for what isn't pure filesystem (copy_xattrs for the held-fd
xattr copy, handle_partial_dir for the EXDEV partial-dir path); those stay in
their current modules, keeping the dependency one-way (vfs -> nothing
above it pulled in). safe_read (a static helper only copy_file used) moves
with it.
Operator context still flows via the vfs.operator_path_resolve field for now
(legitimately VFS-internal once these callers live in vfs/); a later commit
threads it as an explicit vfs_flags argument and removes the field. The
t_*_secure harnesses don't reference the cluster, so no xattr/protocol
symbols are pulled into them. Full suite 190/50; copy-dest/backup/partial/
link-dest subset green.
First step of the VFS three-layer architecture: filesystem-mechanic helpers
that compose the vfs_* primitives belong inside vfs/, so the operator-path
resolution policy travels as an explicit argument instead of leaking across
the vfs<->mainline boundary as ambient state.
make_path() (recursive mkdir over vfs_mkdir/vfs_stat) moves verbatim from
util1.c to vfs/make_path.c and becomes vfs_make_path(fname, mkp_flags,
vfs_flags): mkp_flags keeps the path-handling bits (MKP_DROP_NAME/
MKP_SKIP_SLASH), vfs_flags carries the resolution policy (VFS_OPERATOR_PATH
for the operator-supplied --backup-dir tree, else 0). The transitional
MKP_OPERATOR flag added in the mkdir step is dropped in favour of the
explicit vfs_flags argument.
Callers: get_backup_name (backup dir) passes VFS_OPERATOR_PATH; the transfer
callers (main/receiver/generator) pass 0. Full suite 190/50.
Collapse vfs_symlink / vfs_symlink_at / vfs_symlink_atfd into one call.
dirfd == VFS_AT_FDCWD resolves `path` (secure receiver resolve by default,
plain follow under VFS_ALLOW_SYMLINK); a real held dirfd makes `path` a
single validated component. The old bodies become static helpers behind
the public dispatcher.
Unlike mkdir/mknod, the symlink secure path has NO ownership-walk branch --
it never read vfs.operator_path_resolve. That pre-existing asymmetry is
preserved: VFS_OPERATOR_PATH is accepted but resolves the same as the
default secure walk (only the parent dir is confined; the link target is
stored verbatim and never resolved at creation). The backup-symlink site
passes flags=0, which exactly reproduces the old vfs_symlink_at behavior.
Full suite 190/50.
Collapse vfs_mknod / vfs_mknod_at / vfs_mknod_atfd into one call, mirroring
vfs_mkdir. dirfd == VFS_AT_FDCWD resolves the path (secure receiver resolve
by default, ownership walk under VFS_OPERATOR_PATH, plain follow under
VFS_ALLOW_SYMLINK); a real held dirfd makes path a single validated
component. The old plain/secure/atfd bodies become static helpers
(vfs__mknod_plain/_secure/_atfd) behind the public dispatcher; the secure
body reads the operator context from VFS_OPERATOR_PATH instead of the
vfs.operator_path_resolve global.
Call sites: backup file node (make_backup_inner, operator) ->
VFS_OPERATOR_PATH; transfer device/fifo (generator) -> secure / held-fd.
The t_symlink_secure harness's plain PoC call becomes VFS_ALLOW_SYMLINK
(the vulnerable follow it demonstrates) and its secure calls flags=0.
Full suite 190/50.
Collapse vfs_mkdir / vfs_mkdir_at / vfs_mkdir_atfd into one call. dirfd ==
VFS_AT_FDCWD resolves the path argument; a real held dirfd makes path a
single component created directly under it (the held-fd form validates
that path is a lone harmless component -- rejects empty, any '/', "." and
".." -- so it cannot reintroduce path resolution under the pinned dir).
The plain-vs-secure choice is now an explicit per-call flag instead of a
function-name choice plus the vfs.operator_path_resolve global:
VFS_ALLOW_SYMLINK the call site asserts it is safe to follow symlinks
(the old plain vfs_mkdir); checked first, wins over
VFS_OPERATOR_PATH (no caller passes both)
VFS_OPERATOR_PATH operator-supplied path: ownership walk + module
confinement (the old global)
default 0 secure receiver resolve (the old vfs_mkdir_at)
VFS_AT_FDCWD is a VFS-owned sentinel (maps to AT_FDCWD where available,
else a value that routes the held-fd form to ENOSYS) so mainline code
never has to mention AT_FDCWD directly -- avoids a compile break on
platforms lacking it.
mkdir stops reading vfs.operator_path_resolve. vfs_owner_walk_parent now
takes the operator context as an is_operator parameter; its non-mkdir
callers (the other _at wrappers, generator hard-link) still pass the
global verbatim and convert in later steps. make_path() gains MKP_OPERATOR
(translated to VFS_OPERATOR_PATH) so the backup-dir creation stays an
operator path by path-semantics, not by relying on the global being set.
Call sites classified: dest dir (main.c) follows symlinks (user's local
dest); transfer dirs (generator, make_path transfer callers) secure;
backup-dir (backup.c, get_backup_name->make_path) and partial-dir
(handle_partial_dir) operator. The operator_path_resolve=1 blocks stay
for now since they still wrap non-mkdir ops; the global value and the
explicit flag agree at every converted mkdir site.
Full suite 190/50; backup/partial/daemon/exclude/filter/symlink-race
subset green.
First step of the API redesign. Define the per-call VFS_* flags
(VFS_ALLOW_SYMLINK / VFS_OPERATOR_PATH / VFS_REMOVEDIR) and stop
abspath_excluded_by_module() from reading the vfs.operator_path_resolve
global directly -- it now takes an explicit is_operator argument.
The strict resolver stays confined beneath its anchor (always within the
module), so its two call sites pass is_operator=0 (the check never fires
there anyway). The ownership walk is reached in BOTH operator context
(--backup-dir/--temp-dir/--partial-dir) and non-operator context (daemon
log-file/motd/config opens, which may legitimately live outside the
module), so vfs_open_owner_walk()/vfs_owner_walk_parent() capture the
global and pass it down through ona_open(). No behavior change -- the
value still comes from the global; later commits replace that source
with the explicit flag and delete the global.
Fold the served module root (module_dir / module_dirlen / module_dirfd)
into the vfs.module_* snapshot so the confinement checks read the VFS's
own state rather than implicit clientserver.c externs.
clientserver.c calls vfs_set_module_root() in two stages, matching when
the values become final: once right after the module path is settled
(before the daemon opens any operator path -- filter/include files, the
log file), with the root dirfd still -1, and again once that dirfd is
pinned by identity. The dirfd is borrowed (open_anchor_dirfd dup()s it);
the VFS never closes it. vfs_init() now clears the whole snapshot so a
forked connection can never inherit a stale module root.
The vfs/ readers (dirstack, secure_open, owner_walk) switch from the
externs to vfs.module_*; clientserver/flist/main keep their own globals.
Behavior is identical: between the two calls vfs.module_dirfd is -1, so
open_anchor_dirfd re-resolves the path exactly as before the pin.
Refresh the header comment now that the wrappers are vfs_* and syscall.c
is gone, and document the three operation forms callers choose between:
the plain path wrapper, the parent-resolved _at form (race-safe receiver
path), and the _atfd form (single component under a pinned dirfd).
Comment-only; no code change. Validated at protocol 30 and 29.
Every filesystem wrapper has moved into vfs/, so syscall.c held nothing
but its includes and an orphaned comment. Delete it and drop syscall.o
from the rsync link and from each test-harness object list (they reach
the vfs_* symbols through libvfs.a now). The S_BLKSIZE fallback define,
used by vfs/fileio.c, moves there. No behavior change.
The chmod family carries macOS setattrlist() and the Linux SYS_fchmodat2
raw-syscall fast path, but the move into vfs/chmod.c dropped the two
platform headers that syscall.c had included at file scope: <sys/attr.h>
(macOS) and <sys/syscall.h> (Linux). A Linux build hid this -- with
SYS_fchmodat2 undefined the code silently fell back to fchmodat() -- but
macOS failed to compile setattrlist(). Re-add both, matching the
original syscall.c includes.
Found by fleettest (mac2 BUILD-FAIL).
Relocate do_ftruncate/do_lseek/do_fallocate/do_punch_hole out of
syscall.c into vfs/fileio.c as the vfs_* names, declared in vfs/vfs.h.
The SUPPORT_PREALLOCATION / HAVE_FALLOCATE / HAVE_SYS_FALLOCATE /
FALLOC_FL_PUNCH_HOLE guards travel verbatim. This was the last
operation family: syscall.c no longer defines any filesystem wrapper.
No behavior change. (Portability-sensitive; wants a fleettest.)
Relocate the timestamp wrappers (do_utimensat/_at/_atfd, do_lutimes,
do_utimes, do_utime) and the crtime paths (do_setattrlist_times/_crtime,
get_create_time, do_SetFileTime) out of syscall.c into vfs/times.c as the
vfs_* names. The struct create_time / #pragma pack / Cygwin windows.h
and sys/attr.h includes travel with them, as do the SUPPORT_CRTIMES /
HAVE_SETATTRLIST / HAVE_GETATTRLIST / HAVE_UTIMENSAT / HAVE_LUTIMES /
HAVE_UTIMES / HAVE_UTIME guards. No behavior change. (Portability-
sensitive; wants a fleettest.)
Relocate do_mknod/do_mknod_at/do_mknod_atfd out of syscall.c into
vfs/mknod.c as the vfs_* names, declared in vfs/vfs.h. The
HAVE_MKNOD/HAVE_MKNODAT/HAVE_MKFIFO guards and the AF_UNIX socket-bind
fallback (with its <sys/un.h> include) travel verbatim. No behavior
change. (Portability-sensitive family; wants a fleettest.)
Relocate do_lchown/do_lchown_at/do_lchown_atfd out of syscall.c into
vfs/chown.c as the vfs_* names, declared in vfs/vfs.h. The HAVE_LCHOWN
fallback guard travels verbatim. No behavior change.
Relocate do_mkdir/do_mkdir_at/do_mkdir_atfd and do_mkstemp/
do_mkstemp_atfd/secure_mkstemp out of syscall.c into vfs/mkdir.c as the
vfs_* names (secure_mkstemp -> vfs_secure_mkstemp). The
trim_trailing_slashes path helper (used by trimslash and the mkdir
wrappers) moves with them, keeping its name and now declared in vfs.h.
The static rand_bytes helper, used only by the mkstemp create loop,
moves along too. No behavior change.
Relocate do_link/do_link_at/do_link_atfd out of syscall.c into
vfs/link.c as the vfs_* names, declared in vfs/vfs.h. The
HAVE_LINK/HAVE_LINKAT guards travel verbatim. No behavior change.
Relocate do_symlink/do_symlink_at/do_symlink_atfd and do_readlink/
do_readlink_atfd out of syscall.c into vfs/symlink.c as the vfs_* names.
The fake-super (NO_SYMLINK_*XATTRS) placeholder handling travels
verbatim. vfs_readlink stays a function only in fake-super builds and a
macro -> readlink() otherwise (the rsync.h macro is renamed to match);
its vfs.h declaration is guarded accordingly. No behavior change.
Relocate do_chmod/do_chmod_at/do_chmod_atfd (and the leaf-safe
do_fchmodat_nofollow helper) out of syscall.c into vfs/chmod.c as the
vfs_* names, declared in vfs/vfs.h. The HAVE_CHMOD / HAVE_LCHMOD /
HAVE_SETATTRLIST / SYS_fchmodat2 platform guards travel verbatim.
Function bodies unchanged; no behavior change.
(Portability-sensitive family; wants a fleettest with the rest of the
#ifdef-heavy moves.)
Relocate do_open/do_open_at/do_open_atfd/do_open_nofollow/
do_open_checklinks out of syscall.c into vfs/open.c as the vfs_* names,
declared in vfs/vfs.h. Function bodies unchanged; no behavior change.
Relocate do_unlink/do_unlink_at/do_unlink_atfd and do_rmdir/do_rmdir_at
out of syscall.c into vfs/unlink.c as the vfs_* names, declared in
vfs/vfs.h. Function bodies unchanged; no behavior change.
Relocate do_rename / do_rename_at / do_rename_atfd out of syscall.c into
vfs/rename.c as vfs_rename / vfs_rename_at / vfs_rename_atfd, declared in
vfs/vfs.h. Function bodies unchanged.
Also centralize the option-global externs (dry_run, am_root, am_sender,
inplace, preserve_*, open_noatime, copy_*, insecure_links, module_id, …)
in vfs/vfs_internal.h, replacing syscall.c's local extern block, so each
relocated family picks them up from one place. No behavior change.
Relocate the stat/lstat/fstat wrappers out of syscall.c into vfs/stat.c
with the vfs_* names:
do_stat -> vfs_stat do_stat_at -> vfs_stat_at
do_lstat -> vfs_lstat do_lstat_at -> vfs_lstat_at
do_fstat -> vfs_fstat do_stat_atfd -> vfs_stat_atfd
do_lstat_atfd-> vfs_lstat_atfd
do_xstat_at stays a file-local helper. The x_stat/x_lstat/x_fstat
fallback macros in rsync.h now expand to the vfs_* names. First Phase-3
family move, so the shared RETURN_ERROR_IF* dry-run/read-only guard
macros (and the read_only/list_only externs they expand to) move from
syscall.c into vfs/vfs_internal.h where every vfs/ source can use them.
Function bodies unchanged; no behavior change.
Move the operator-path resolver-mode flag into vfs.operator_path_resolve.
Its definition leaves vfs/owner_walk.c, the extern declarations (in
backup/generator/receiver/util1 and vfs_internal.h) are dropped, and the
uses across the do_*_at wrappers and the vfs/ internals are updated to
the struct field directly. This completes moving the scattered VFS state
(dirfd cache, curr_dir, operator_path_resolve) into struct vfs.
No behavior change.
Move the logical-cwd globals into vfs.curr_dir / vfs.curr_dir_len. The
definition leaves syscall.c, the per-file `extern char curr_dir[]` /
`extern unsigned int curr_dir_len` declarations are dropped (the struct
is reached via the vfs.h `extern struct vfs vfs`), and the uses across
exclude/flist/log/main/util1 and vfs/secure_open are updated directly --
no compatibility alias macros. change_dir() writes vfs.curr_dir. The
separate curr_dir_depth global is untouched. No behavior change.
Replace the file-local dpc_* statics in vfs/dircache.c (anchor, base,
fd[], name[][], depth) with the vfs.dpc fields already declared in
struct vfs and initialized by the designated initializer in vfs/vfs.c.
Pure encapsulation: the cache logic is unchanged, DPC_MAXDEPTH becomes
the shared VFS_DPC_MAXDEPTH, and vfs_dircache_reset() now resets vfs.dpc
(consistent with vfs_init()). No behavior change.
Post-relocation cleanup (no behavior change):
- fix a doubled-prefix typo "vfs_vfs_owner_walk_parent" in the
vfs/owner_walk.c header comment
- vfs/vfs.c comment now points at vfs/dircache.c (not syscall.c) for the
live dpc_* cache
- drop unused extern decls left from assembling the moved files
(am_sender/insecure_links in owner_walk.c, curr_dir/curr_dir_len in
dircache.c)
Relocate the persistent ancestor-dirfd cache out of syscall.c into
vfs/dircache.c with the vfs_* public names:
open_dir_secure -> vfs_opendir
get_dir_fd -> vfs_get_dirfd
held_dir_path_fd -> vfs_path_dirfd
held_dfd_for -> vfs_cached_dirfd
reset_dir_fd_cache -> vfs_dircache_reset
The dpc_* cache statics and dpc_dir_fd stay file-local (they fold into
the vfs struct in a later commit). Function bodies are unchanged; the
callers in generator/receiver/sender/delete/util1/rsync are updated and
the five entry points declared in vfs/vfs.h. This completes moving the
security core (resolver, owner-walk, dirfd cache) out of syscall.c.
No behavior change.
Relocate the operator-supplied-path resolver out of syscall.c into
vfs/owner_walk.c with the vfs_* public names:
open_no_attacker_symlinks -> vfs_open_owner_walk
owner_walk_parent -> vfs_owner_walk_parent
The static helpers ona_open and abspath_step move with them, as does the
operator_path_resolve flag definition (commit-8 will fold it into the vfs
struct). Function bodies are unchanged; the call sites across the daemon
and option-parsing files are updated and the two entry points declared in
vfs/vfs.h.
With the owner-walk gone, syscall.c no longer references am_daemon, so
the now-dead `extern int am_daemon` declarations (the do_*_at wrappers
delegate to vfs_relpath_active) are dropped. No behavior change.
Relocate the race-safe resolver and its policy gates out of syscall.c
into vfs/secure_open.c, and give them the vfs_* public names that the
mainline code will use going forward:
secure_relative_open -> vfs_resolve_open
secure_relative_open_at -> vfs_resolve_open_at
secure_relpath_active -> vfs_relpath_active
symlink_optout_allowed -> vfs_symlink_optout_allowed
secure_walk_at stays file-local. The function bodies are unchanged; the
call sites across receiver/sender/generator/flist/util1/clientserver/
main/options and the test harnesses are updated to the new names, and
the four entry points are declared in vfs/vfs.h.
The resolver's only consumer of am_chrooted in syscall.c left with it,
so the now-dead `am_chrooted` is dropped from the do_*_at wrappers'
local externs (and the file-scope extern), which is otherwise unused.
No behavior change.
First step of relocating the security core. The component-walk dirfd
stack (struct dirstack + the ds_* helpers) and the module-confinement
helpers (path_has_dotdot_component, abspath_excluded_by_module,
open_anchor_dirfd) move verbatim out of syscall.c into vfs/dirstack.c.
The secure resolver (still in syscall.c) and the held-dirfd cache reach
them through a new private header vfs/vfs_internal.h.
The functions are byte-identical to before; only their linkage changes
(the ones syscall.c still calls become non-static; ds_path_push/
ds_path_pop/ds_push stay file-local). struct dirstack and
SECURE_OPEN_MAXSYMLINKS now live in vfs_internal.h so both sides see one
definition. vfs_internal.h also centralizes the option/daemon externs
the VFS internals read. No behavior change.
Introduce a vfs/ subtree that will house rsync's filesystem-handling code
(the do_* syscall wrappers, the race-safe path resolver, the held-dirfd
cache, the operator-path ownership walk and daemon module confinement),
separating those security-critical details from the protocol/transfer
logic. This first commit only stands up the layer; no code is moved yet,
so behavior is identical.
- vfs/vfs.h: public interface, included by rsync.h just after proto.h.
Declares "struct vfs" -- the single global that will hold the state
currently scattered across syscall.c statics (the dirfd cache, curr_dir,
operator_path_resolve) and the clientserver.c module_* globals -- plus
vfs_init().
- vfs/vfs.c: defines the global instance with a designated initializer so
the cache and module snapshot are safe by construction (a plain
definition would zero base/module_dirfd, making fd 0 look valid); the
t_*_secure harnesses never run main(), so this must not rely on
vfs_init(). vfs_init() resets between transfers (inert for now).
- Build: bundle the VFS into a static libvfs.a linked last on rsync and
every test harness, so later commits can move code out of syscall.o
without breaking a harness link (the linker pulls only what it needs).
AC_CHECK_TOOL(AR)/AC_PROG_RANLIB added for portable archiving; a
vfs/dummy config-file output creates vfs/ in VPATH builds.
- main.c calls vfs_init() early (establishes the call site).
Date the release and bring the security section up to the full set: it
described 20 CVEs, and the release fixes 33.
The thirteen later items are added in three groups -- the peer-triggerable
memory-corruption findings from the daemon-protocol fuzzing pass, the daemon
availability and access-control issues, and the two client-side ones.
Several changes were previously described here as carrying no CVE and now do,
so those claims are removed rather than left to contradict the advisories:
rsync-ssl's unverified TLS is CVE-2026-70454, the non-positive MSG_IO_TIMEOUT
is part of CVE-2026-70462, and the early-protocol argument-count bound is part
of CVE-2026-70464. What is left under "no CVE assigned" is only the proxy
header bounds and the xattr expansion cap.
A stable-backport branch runs a newer suite than its own code. fleettest
already reads testsuite/skiplist/backport.txt from the tree being built and
excludes those tests; runtests.py did not, so running the suite directly --
which is what the backport branches' CI job does -- tried to run tests that
base cannot support.
Read the same file from tooldir and drop its names from both the run and the
expected-skip set: an excluded test never runs, so leaving it in the expected
set would make the oracle demand a skip that cannot happen. A stale name is
an error rather than a silent no-op.
Also note that backport CI, once it exists, has to consume these lists the
same way fleettest does -- read backport.txt from the branch being built and
pass it as RSYNC_EXCLUDE -- or it will fail on every entry and get switched
off.
A test named in a backport's backport.txt never runs, so it cannot skip either.
If the suite's expected-skip list also names it -- the two --compress-threads
tests are declared as expected skips because they need --use-tcp -- the oracle
waits for a skip that can no longer happen and every pipe cell reports a skip
mismatch.
Emit a '-name' removal for those, as the per-target expect_skip_omit already
does. Only for names the spec actually contains: runtests rejects a '-name'
that removes a name nothing added, and most of a backport's exclusions (a test
for a feature it lacks) are not expected skips at all. Deciding that needs the
@FILE references expanded locally, which is what _expand_spec_names does.
v3.4.1 with this: 5/5 cells OK on ubuntu-2404, from 3 OK / 36 not OK across the
fleet before the mechanism existed.
Running the 3.5.0 suite against an older branch (--repo BACKPORT
--testsuite-repo .) reports a wall of failures that are not regressions: tests
for fixes the branch does not carry, and tests whose unit-test helpers its
Makefile cannot build. Both backport branches came back 3 OK / 36 not OK with
every distinct failure explained that way, which makes the run useless as an
oracle -- a real regression would not stand out.
A backport now declares those in its own testsuite/skiplist/backport.txt, read
from the tree being BUILT rather than the one providing the suite: only the
built tree knows what it lacks. The names go to runtests.py as RSYNC_EXCLUDE
rather than as an expected-skip declaration, because some of them fail rather
than skip and an expected-skip list cannot describe a failure.
The overlay that puts a newer testsuite/ onto an older tree is a merge with no
delete, so a file that exists only on the backport survives it. skiplist-spec
exempts the name from its every-list-must-be-referenced rule, since nothing
references this one by design.
Under inc_recurse the first flist (ndx_start == 1) has no parent entry of its
own, so recv_file_list() trusts the peer's "." entry to be the transfer root and
leaves parent_ndx at the flist_new() default of 0 -- dir_flist->files[0]. Only
S_ISDIR entries are appended to dir_flist, so a peer that sends "." with a
NON-directory mode keeps dir_flist->used at 0 while the basename strcmp still
passes: parent_ndx stays 0 and the consumers index a never-written slot.
Drives a real daemon with the pure-Python sender: an inc_recurse push whose only
flist is a regular file "." plus a regular file "a" (no directory anywhere, and
"." sorts lowest; "a" keeps file_total != 1 so the receiver doesn't divert into
recv_additional_file_list). The file list alone is what does it -- the
generator crashes in generate_files() before any transfer phase -- reproduced on
released 3.2.7, 3.4.0 and 3.4.1.
The oracle needs both halves: a positive control that the daemon logged
"receiving file list", and the condition-specific refusal. Accepting any
"rsync error:" line is not enough -- with "." sent as a valid directory and a
bogus file index, that form passes on "File-list index 1000000 not in 0 - 2"
without the crafted transfer root ever reaching the parser.
A fixed daemon has already refused the list and exited by the time the ndx-0
token is sent, so that send and the drain can hit a closed socket; Linux and
FreeBSD swallow it, Solaris, the other BSDs, macOS x86 and Cygwin raise
EPIPE/ECONNRESET. Treat it as an expected outcome, not a result.
The header records what this does not prove: it gates the attack shape rather
than the parent_ndx clause (only the parse-time transfer-root check fires on a
current build), it does not exercise the receiver-side consumer, and the
dereferenced slot is not guaranteed NULL since dir_flist->files[] comes from
realloc(), not calloc().
The fix shipped in the test10 snapshot but was never written up: safe_arg()'s
filename-mode buffer sizing disagreed with the writer, leaving an uninitialized
heap byte in the argument handed to the remote shell when --protect-args is off.
The daemon sets its deadline with time(NULL) (set_daemon_handshake_timeout,
io.c), and this test measured the elapsed time with CLOCK_MONOTONIC. Those
agree on a quiet machine and diverge on a stalled one: a virtualised guest
resyncs its wall clock after the host deschedules it, while monotonic keeps
its own count. The daemon then closes exactly when it meant to and the test
reports it closed early.
That is what a NetBSD CI run showed -- "closed after 39.55s, before the
expected timeout window (58.75s)" -- and it is the same shape as the macOS
failure that turned out to be the machine sleeping mid-test.
Measure the bound on the clock the daemon decides with. Monotonic still
drives the poll budget, where the job is only "do not hang forever".
On its own that would trade a false failure for a false pass, which is worse:
a refusal or a crash arriving just as the guest's wall clock caught up would
read as a clean timeout, and no diagnostic would fire because the test would
be green. So when the two clocks disagree -- wall says on time, monotonic
says early -- neither settles it, and the daemon has to have recorded its own
deadline firing. The offset of its log is taken before each observation, so a
timeout it logged earlier cannot vouch for this one.
Failures now carry both clocks and that window of the daemon's log, bounded
and with control bytes escaped. The clock note states the discrepancy without
concluding from it: a stalled host produces it, but so does an NTP step, and
either can accompany a real failure.
The same kernel-side missed wakeup on the other side of the connection: a
blocking connect() can sleep forever on a connection that is already
established, with the 4-tuple ESTABLISHED at both ends and the listener's
greeting queued unread. Without --contimeout nothing breaks it.
Wait for the connect with poll() in slices rather than blocking in the
kernel, re-checking the socket on each pass, and take the result from
SO_ERROR. A finished slice is not a failure -- looping is what re-examines
the socket and recovers a missed wakeup.
--contimeout is unchanged: the alarm still fires and the caller still
reports RERR_CONTIMEOUT. The per-address errno is now stashed before
close()/alarm() can overwrite it.
Measured the same way, against a real loopback daemon: 20 hangs in 48,000
connects before, 0 in 48,000 after, with equal wall clock. This is the half
of the OpenBSD flakiness that the socketpair_tcp() fix does not cover: the
--use-tcp pass talks to a real rsyncd over a port, so it hangs here rather
than in accept().
On OpenBSD a blocking accept() can sleep forever on a connection the kernel
has already completed: the 4-tuple is ESTABLISHED at both ends, the
connection is queued on the listener, and the accept()ing process is still
asleep in netacc. Nothing bounds that wait, so rsync hangs for good.
Poll the listener instead, with a non-blocking accept(), so a missed wakeup
costs another pass rather than the process. The accepted fd is put back
into blocking mode explicitly because BSD accept() gives it the listener's
non-blocking flag. A time(NULL) deadline bounds the whole wait the way
io.c bounds its own, rather than counting passes -- a signal on every pass
must not extend it and a poll() that returns at once must not consume it.
A listener that reports ready without yielding a connection (the peer can
reset first) pauses rather than spinning.
Measured on an OpenBSD 7.8 VM, driving the real binary through
RSYNC_CONNECT_PROG with 8 concurrent workers, alternating stock/patched
rounds: 111 hangs in 120,000 invocations before, 0 in 120,000 after, with
no change in throughput.
Every daemon test reaches socketpair_tcp() through RSYNC_CONNECT_PROG in
the default transport, so the hang landed on whichever daemon test happened
to be connecting. See dev-notes/openbsd-socketpair-accept-wedge.txt.
Nothing in CI or the fleet has ever set --enable-roll-simd, --enable-roll-asm
or --enable-md5-asm, which is why the over-read above sat behind a "fixed"
label for two months, and why the fix applied for it went to the wrong
assembly file.
mac-x86-asm is the same host and OS as mac-x86 with all three on. Mach-O is
the interesting part -- both problems reported against these flags were
macOS-x86-64 -- and it is the only machine in the fleet that can build the
x86-64 assembly at all.
It needs MacPorts clang 19 through CC/CXX, because Apple clang 10 (the ceiling
on macOS 10.13) rejects configure's target("default") multiversioning probe.
mac-x86 keeps the stock Apple compiler, which is what caught #161, so the two
cover different ground rather than one replacing the other.
simd-checksum is a macOS-wide expected skip, since simdtest is only built when
SIMD is enabled; this target subtracts it, because running it is the point.
Also corrects mac-x86's comment, which claimed the probe "cannot compile here
with any clang". It is a compiler-version limit: clang 19 on that same box
compiles, links and runs it.
simdtest allocated 64 spare bytes so it could test an unaligned buffer, which
is exactly the slack that hid a 64-byte over-read in the AVX2 assembly for as
long as it existed. Add a pass that places the buffer flush against a
PROT_NONE page, so a read past the end faults in the test rather than in
somebody's transfer.
Every length from 128 to 4096, so each remainder mod 64 and both alignments
are covered, and all four implementations are checked -- the assembly was the
one at fault here, but the intrinsic paths preload too.
It fails closed. A guard this test cannot set up means it is not testing what
the caller thinks, so a failed sysconf/mmap/mprotect is a failure rather than a
pass that looks identical to a real one. And because the dispatcher falls back
on a CPU without AVX2 -- where the guard loop proves nothing about the code
under test -- it says which of the two happened rather than letting a fallback
run read as coverage.
Without the fix this segfaults; the suite's simd-checksum test reports the
non-zero exit.
The loop is software-pipelined: each iteration folds in the 64 bytes it
preloaded last time and preloads the next 64. Nothing stopped the final
iteration doing that preload, so it always read the 64 bytes after the region
it was asked to checksum.
Not an edge case. The assembly processes len&~63 and leaves the remainder to
the caller, so the remainder is by construction under 64 bytes and the preload
passed buf+len on every call, by 64 minus the remainder.
It normally landed in slack inside the map_ptr() window and nothing noticed.
Where the buffer ended near an unmapped page it was a SIGSEGV in the middle of
a transfer -- reported on macOS x86-64 by Roland Kletzing, whose `partial` run
died with "connection unexpectedly closed" because the generator had crashed.
A guard page reproduces it on Linux too, so it was latent there, not absent.
Run the pipelined loop one block short and finish the last block in .last,
which does the same arithmetic without the preload. No per-iteration cost, and
checksums are bit-identical -- simdtest compares every implementation against
the C reference.
The earlier fix for that report, "lib: use .balign in md5 x86-64 asm", was to
the md5 assembly. It addressed the linker alignment warning that appeared
alongside, not this.
An LD_PRELOAD hook refuses linkat() for a symlink source only, so the arm is
reachable on a filesystem that hard-links symlinks perfectly well. Three
controls keep it from proving less than it looks:
- the regular file in the same transfer must still be hard-linked, or "it fell
back" would also be satisfied by --link-dest having been abandoned;
- the itemised run must emit exactly one "cL... sym -> some-target" line. A
plain -a run cannot see a duplicated itemisation, which is how that defect
reached an HFS+ target before this was added;
- EPERM and ENOSYS must fall back too, since errno does not separate "cannot"
from "may not".
itemize picked its expected change-type letter from the build capability, which
is the wrong question -- the link happens on whichever filesystem holds the test
data. Ask that one too, and drop the XFAIL the old mismatch needed.
The hook is Linux-only, so the test joins the macOS and Cygwin skip lists,
which are required to be sorted.
CAN_HARDLINK_SYMLINK and CAN_HARDLINK_SPECIAL are decided by configure running
linkat() on whatever filesystem the build tree happened to sit on. The
destination is free to disagree, and one host can hold both answers: macOS
builds on APFS, which can hard-link a symlink, and backs up to HFS+, which
returns ENOTSUP.
A build that said yes had no fallback left. try_dests_non() reported the
refusal as a transfer error and returned a matched basis, so the caller created
the entry anyway -- correctly -- and the run still exited 23. Every
neighbouring case copes: a regular file whose link() fails goes to try_a_copy,
and a build compiled without either macro resorts to --copy-dest behaviour.
This was the same situation, discovered a little later, and the only one
treated as fatal.
Take the existing fallback on any refusal. Singling out the "cannot" errnos is
not possible: link(2) documents EPERM both for a filesystem with no hard-link
support and for an ordinary permission refusal, and FUSE reports ENOSYS for the
same thing. It is also what the regular-file path next door has always done
(try_dests_reg -> hard_link_one -> try_a_copy), and consistency between the two
was the point. Where the errno does matter the surrounding transfer says so
anyway: ENOSPC, EDQUOT and EROFS fail the creation independently, EMLINK and
EXDEV mean the link was never possible. EIO alone goes unremarked; reporting
it would put a line into --link-dest's itemised output, so it is left out on
purpose.
Returning -3 rather than -2 keeps the caller out of the "already up to date,
skip it" arm, which under --link-dest would drop the entry entirely. Both
callers give -3 the treatment the compile-time fallback already gets -- clearing
itemizing and code -- because try_dests_non() has itemised the match itself and
would otherwise report the entry twice.
The fallback is silent, matching a build that cannot link these at compile
time; documented under --link-dest instead.
Uses the exclude-only merge form, which leaves no diagnostic to assert on: the
escape shows up as a file silently missing from the transfer, so the test reads
the oracle the same way an attacker would. Pull mode, so no --delete is
involved. Each case requires the transfer to have succeeded as well, since
refusing outright would hand the peer a denial of service.
A second escape reaches the source through a symlink, which is what makes
rsync's tracked cwd and the real one disagree -- the shape that catches a
lexical seed. That one drives --confine-root directly: rrsync rejects the
argument spellings that would carry it, so routing it through the wrapper would
pass either way and prove nothing.
Both controls repeat their escape with an in-tree merge target and require it
to be read AND obeyed, since "the transfer failed" and "every merge file is
refused" would otherwise satisfy the escape assertions on their own.
Filter rules arrive over the protocol, long after the wrapper has exec'd rsync,
so no argv-level check can see them. A client can name a merge file outside
the restricted dir in a dir-merge rule and have the server read it in as filter
rules; on a pull that needs neither --delete nor any verbosity. Pass
--confine-root so the server bounds the open itself, which is the only end that
can.
Both directions: a dir-merge is read by whichever side its rule applies to, so
unlike --drop-D this is not receiver-only. Skipped for a "/" restricted dir,
where there is nothing to confine.
The ownership walk that resolves operator-supplied paths asks who planted a
symlink, not where the path came out, so a symlink owned by uid 0 or the euid
is followed wherever it points. A daemon already narrows that with the served
module root; nothing else has a root to narrow it with.
That leaves a wrapper serving a restricted directory over a remote shell with
no way to bound the resolution. rrsync can vet the argv it is handed, but
filter rules travel over the protocol instead: a dir-merge rule can name a
merge file outside the restricted dir and the server reads it in as rules.
Redacting the resulting diagnostics does not close it, because an exclude-only
merge produces none -- every line becomes a pattern, so nothing fails to parse,
and the client reads the file's contents off which of its own names went
missing from the file list.
--confine-root gives that wrapper the root the daemon has. The existing module
check becomes a root check that takes its root from module_dir when we are a
daemon and from the option otherwise, so daemon behaviour is unchanged; a
daemon ignores the option outright, since it arrives in a peer-supplied argv
and could only widen the module.
The tracker is seeded from getcwd() rather than curr_dir, which is only the
lexical name change_dir() was given: descend into a source argument through a
trusted symlink and the two sit at different depths, so a ".." that really
escapes looks like it landed inside. When the cwd cannot be read there is
nothing to measure against and the open is refused -- an empty tracker does not
deny by itself, because a leading ".." pops nothing from it and an empty path
reads as an ancestor of the root.
An fd pin (/proc/self/fd/N, which rrsync uses so no later symlink can redirect
a validated option path) is spelled outside the root by construction, so the
walk transits the pin namespace and the pin is judged by what it points at.
Only a bare ".../fd/<digits>" is resolved that way, and one that will not
readlink to an absolute path is refused; rrsync's ".../fd/N/<leaf>" spelling
resolves through the magic link and has its remaining components checked
normally.
--insecure-links is refused alongside it: that opt-out returns the legacy open
before the walk that enforces the root runs, so the pair would have quietly
meant no confinement at all.
The exclude-self rule that a ":e" merge synthesizes is built by hand with
new0(), so it inherited no flags. While the merge file was still being
parsed the global parse state masked that, but once parsing finished the
stored rule looked argument-origin, and report_filter_result() printed its
pattern -- a merge file's own text -- verbatim:
[sender] hiding file PAT-x9 because of pattern PAT-[x]9 [per-dir ...]
Plain -vv reaches this on a stock client; no --debug is involved. That is
the fifth site of this shape, and the first to get there by constructing a
rule rather than by printing one, so the redaction helper could not catch it.
Also fix the location a per-directory merge reports. Its fname points into
dirbuf, which is cut back to the directory before the name was saved, so the
error said "<rule from .../src/ line 1>" instead of naming .rsync-filter --
no leak, but it breaks the "redact what, keep where" bargain the rest of this
work depends on. Save the name before the truncation.
The rrsync test's claim to close "the rest of the FILTER trace family" was
too strong and is corrected: options.c maps verbosity onto the debug flags,
so -vvv still raises a restricted server to FILTER2 and its trace metadata
comes back. Rule text stays redacted at every verbosity, which is the
property that matters; -vvv is added to the unaffected-transfer cases.
The --debug=FILTER traces print rule text and merge-file names that came
out of a file's contents -- and a word-split per-dir merge (":w- FILE")
turns every word of a file into a merge-file name, so the trace echoes
what the syntax errors no longer do, with nothing failing to parse.
Redacting every trace would mean carrying provenance on each rule, which
a deferred ":" merge does not currently keep. For a restricted account
the cheaper answer is to deny the peer the switch: server_options() only
ever forwards --info, so no stock client sends --debug to a server and
the only way it arrives is a deliberate -M--debug=. An operator
debugging their own server is unaffected.
Disabled rather than deleted from the table, because that table is
generated by the cull-options script and a regeneration would put the
line back; the test would then catch it.
A filter rule that fails to parse was printed back verbatim. When the
rule came from a file rather than an argument, that text is file
CONTENT, and the peer picks which file gets merged: a per-directory
merge rule travels over the protocol, so no argument of ours ever names
it and nothing a wrapper can see mentions it either. Any line that is
not valid filter syntax therefore came straight back to the peer -- a
read-any-line oracle over an rrsync restricted account or a daemon
module, neither of which confines the merge open.
The syntax errors turned out to be the smaller half. The MATCH trace
names the pattern that acted, and report_filter_result() logs at level 1
for a sender or generator, so plain -vv -- no --debug, nothing a stock
client cannot send -- returns a server-side merge file's rules:
[generator] protecting file X because of pattern <the file's text>
So provenance is carried on the rule itself (FILTRULE_FROM_FILE), not
just in the parser: a deferred ":" merge is processed long after the
file that named it was read, and its own name is file content too.
TEXT_FROM_FILE() consults the parse-time context and the rule, so both
the immediate and the deferred paths redact.
Rather than test the provenance at each message -- which is how the last
few of these were found, one at a time, after the ones before them were
fixed -- every string that is or is built from a rule's own text goes
through rule_text(). It returns the text for an argument-supplied rule
and a description of where it came from otherwise, so a message added
later cannot reintroduce the leak by forgetting to check, and there is
one place to audit. rule_detail() does the same for the extra detail a
message adds ABOUT the text: a character of it, an offset into it, the
[not found] bit.
Thirteen sites now route through them: the syntax errors; the modifier
character (one byte of the file, a slower oracle but still one); the
failed-open and merge-depth messages, whose pathname is file content
whenever a rule named it -- and errno with them, since it answers "does
this path exist"; both over-long messages, the deferred one of which
needed no verbosity at all; both merge-name overflows; the long-named
directory error; the [not found] openability bit; the match trace; the
add_rule, parse_filter_file and daemon-hidden traces; and the per-dir
mergelist label, which had the name baked in.
rule_detail() covers more than it first looks: the trailing-whitespace
CAUTION is computed from the rule's last byte, and "hidden by daemon
filter" distinguishes a daemon-filter rejection from an ordinary open,
so both would answer questions about text the peer cannot see.
The regression proves the chokepoint rather than the sites: making
rule_text() return its input unconditionally fails the test. It also
pins what must NOT change for the user's own rules -- the whitespace
warning still fires, and an over-long argument rule is still reported at
full length (the helper buffers at BIGPATHBUFLEN, as rprintf does, so
redaction does not quietly truncate what the user typed).
Bounded and left alone: the numeric rflags in the FILTER2 trace and the
in/exclude wording still describe a file-derived rule without quoting
it, and the daemon's own FLOG line records the name it filtered -- that
one goes to the operator's log, not the peer.
Rules given AS arguments are still echoed in full -- that text is the
user's own, and hiding it would only make ordinary typos harder to fix.
Where a rule did come from a file, the diagnostic names the file and
line instead, which is more useful anyway.
Two things the location itself needed: fname can point into
parse_merge_name()'s static buffer, which a merge rule inside the same
file overwrites while we are still reading it, so a rule after a nested
merge was blamed on the nested file -- keep our own copy. And a CRLF
pair was counted as two line endings while word-split mode counted
tokens rather than lines, so the number pointed at nothing; consume the
LF of a CRLF (preserving the byte for the next rule if pushback ever
fails), and report word-split sources without a line number.
Not covered, deliberately: a rule's provenance is not serialized by
send_filter_list(), so it does not survive to the far side. That is
right -- only the client sends that list, and the server already knows
the patterns the peer gave it.
A daemon test could leave an orphaned rsyncd squatting its port even when
every test PASSED, so nothing in the results pointed at it. On Cygwin the
orphan then wedged the whole fleet: it kept the ssh session from closing,
so fleettest's run_on() blocked until its 2400s timeout and unrelated
tests failed with 300s timeouts as collateral. One such wedge cost a fleet
run 21 minutes.
Cause: rsyncd forks a child per connection, but _stop_rsyncd only killed
the parent -- the one pid the Popen handle knows. A child still winding up
or down when the test ended survived, inherited the listening socket, and
was reparented to init. Cygwin turned that from untidy into unrecoverable:
its signals are cooperative, delivered by a helper thread inside the
target, so a process sitting in a Windows call ignores even SIGKILL. kill,
killpg and pkill all failed against it, which also defeated the orphan
reapers and fleettest --cleanup.
Snapshot the daemon's children before killing it (once the parent is gone
they are reparented and no longer identifiable as ours) and kill them too,
re-checking with _pid_is_rsync before each signal so a pid recycled in the
meantime is never signalled. Where signals cannot win, fall back to
terminating the winpid via taskkill; fleettest --cleanup gets the same
fallback, so it can no longer report SURVIVED and leave the port squatted.
_reap_group() reports success only once the daemon is confirmed gone
rather than when a signal was merely accepted -- on Cygwin a signal is
routinely accepted by a process that then ignores it -- and confirms with
a bounded poll, because SIGKILL is asynchronous and calling a
still-terminating process "alive" would make _probe_bindable() skip its
retry and fail a test for a port that was about to free itself.
_cleanup_rsyncd() keeps the port's pid record only while it still names a
live rsync. Keying that on the port being busy instead looks safer but is
worse: a port sits in TIME_WAIT after a passing test, so a record naming
an already-dead pid would be retained forever, and nothing clears such a
record -- yet no reaper can use it either, since they all reject it at the
_pid_is_rsync guard, leaving only the hazard that its pid is recycled onto
an unrelated rsync.
The daemon stays in the TEST's process group on purpose: runtests.py
killpg's that group on a per-test timeout, and that is what keeps a
timed-out test from stranding its daemon. An earlier version of this fix
gave the daemon its own group so one killpg would catch the children --
which silently broke that sweep, and a full Cygwin pass then stranded two
parent daemons when variety hit its timeout.
Two residual limitations are documented in the code rather than left to be
rediscovered: _kill_pid's check-then-signal is inherently a TOCTOU
(narrowed to microseconds, not closed; closing it needs pidfd or retained
Windows handles across seven platforms), and _stop_rsyncd cannot collect
children when the parent has already exited on its own, because the
parent-child link it relies on is gone by then.
Measured on a Cygwin VM, 4 proxy/daemon tests x 8 runs at -j4: before 5/8
runs left an orphan (one left two), after 0/10. All tests passed in every
run, before and after -- which is the point: the leak was invisible to the
suite. A test killed by the runner's timeout still leaves no daemon
behind.
The tcp pass re-ran the whole suite over the same build the pipe pass had
just swept, but --use-tcp is observable through exactly one code path:
RSYNC_TEST_USE_TCP is read once (rsyncfns USE_TCP) and acted on once (in
start_test_daemon). A test that never reaches there cannot tell the two
passes apart, so 186 of the 340 tests were producing the same result
twice.
runtests.py --daemon-tests-only keeps the tests that can reach the daemon
transport, matched against the closure of every rsyncfns helper leading to
USE_TCP/start_rsyncd/claim_ports plus the modules that open a daemon
connection themselves. The token list is deliberately over-broad and an
unreadable test is kept, so the filter can only ever run too much; audited
against the tests it drops, none of which reach the transport (their
"daemon" hits are the unix username, a macOS ACL principal, mount --bind,
and docstrings declaring the test local-only). The dropped count is always
printed rather than left implicit.
The narrowing is only sound as the second half of a pipe+tcp pair, so it
is gated on the pipe pass having run: under --transport tcp that pass is
the only one there is, and narrowing it would drop the other 186 tests
from the run altogether. --full-tcp forces the full sweep either way.
Measured on the full suite: serial work 558s -> 367s.
The race tests are the suite's slowest by a wide margin -- a race test is
a negative oracle, so it passes by spending its entire budget. Most of
them wrote `max(RACE_TIMEOUT, 10.0)`, which ignored --race-timeout below
10s: the documented knob did nothing for 10 of the 16 tests.
Replace the floor idiom with race_budget(default), where the per-test
default applies only when the operator did not pass --race-timeout, and
runtests.py exports race_timeout only when the flag was actually given.
Defaults are unchanged (measured identical at 15.3s/10.3s/5.2s).
Validate the value rather than take it on trust. A race test loops
`while monotonic() < deadline`, so a zero, negative or NaN budget runs the
body zero times and the test reports PASS without ever racing, and an
infinite one runs until the unrelated per-test timeout; the old
max(..., 10.0) floor had made all of that unreachable, so removing the
floor had to come with rejecting the input. An unparsable value in the
environment counts as unset for the same reason -- falling back to the 5s
baseline while still counting as "set" would silently halve a 10s or 15s
oracle that nobody asked to shorten.
NB the *_test.py glob spans four committed symlinks (chown-fake,
devices-fake, exclude-lsh, xattrs-hlink); sed -i would replace each with a
copy of its target, so they are rewritten with --follow-symlinks semantics
and left as symlinks.
A fleet run costs a full configure+build on every machine, and the report
only names the tests that failed -- so seeing WHY one failed meant paying
for a second whole run, against a race test that may not fail the same
way twice.
--keep-on-fail saves the full build/test output of every target that came
back with anything unexpected, and keeps that target's remote run dir
(with the scratch trees the failing tests left). Clean targets are swept
as before.
--timing now also asks each target's runtests.py for its own per-test
table, so a slow cell can be attributed to actual tests rather than just
named as the hold-up.
The suite reported which tests ran, never how long any of them took, so
"the fleet is slow" could not be attributed to anything. Time each test
and, with --timing, print the slowest first.
The footer gives the two bounds that decide what to do about a slow run:
the serial sum (what one worker would take) and the floor set by the
longest single test, which no amount of -j can beat.
The test needs a real listening socket to stall, so it require_tcp()s and skips
on the default pipe transport -- like daemon-chroot-acl and the proxy tests
alongside it. runtests.py compares the skip set against RSYNC_EXPECT_SKIPPED on
a FULL run, so without an entry every pipe-mode CI job reports an unexpected
skip and fails, while the tcp jobs pass.
rsyncd.conf(5) says of "timeout": "Using this parameter you can ensure that
rsync won't wait on a dead client forever." That did not hold before a module
was known. set_io_timeout() ran at the very end of rsync_module(), so the
greeting, authentication and the whole argument list were read with no I/O
timeout at all -- a peer could stall at any of them and the child waited
indefinitely. Measured: 20 connections sending "@RSYNCD: 31.0" with no newline
were all still alive well past timeout=5, and only went away when the client
hung up.
The consequence is worse than an idle process. claim_connection() runs BEFORE
auth_server(), so naming a module is enough to take a slot: an attacker with no
credentials could occupy every "max connections" slot of an authenticated
module and hold them for as long as it kept the sockets open, with the
documented control unable to recover them. It costs the attacker nothing --
five stalled children measured 0 CPU ticks over 5s -- so this is descriptor and
slot exhaustion, not load.
Bound the handshake at min(configured, 60s). "timeout" is a Locals parameter,
so lp_timeout(-1) reads the global section -- the same -1 idiom start_daemon()
already uses for lp_reverse_lookup().
Both halves of that minimum matter. "timeout" DEFAULTS TO 0, so honouring only
the configured value would leave the daemon most exposed to this -- one whose
administrator never set a timeout -- exactly as pinnable as before. And capping
matters because an operator who sets "timeout = 86400" for slow links is asking
for patience during a TRANSFER, not for a stranger to hold a pre-auth slot for a
day. The pre-module phase has no legitimate reason to take even a minute.
The bound is retired the moment the module is known, which is what lets the
configured value still govern the transfer. That retirement is load-bearing:
the per-module test only ever LOWERS the timeout (`lp_timeout(module_id) <
io_timeout`), so leaving the handshake bound in place would silently clamp a
module that asked for more -- "timeout = 300" would get 60. It is cleared
before that test runs, and only when io_timeout is still the value we armed,
since the client's own --timeout is parsed in between and must win on its own
terms. Verified: with no global timeout and "timeout = 120" in the module, a
connection idles past 75s rather than being dropped at 60.
Applied only for a real socket daemon (am_daemon > 0): an rsh-run daemon has no
listener to exhaust.
Verified end to end with max connections = 2 and timeout = 5: with two stalled
unauthenticated connections holding both slots, a legitimate client is refused
during the timeout window and served once it elapses. Before this change it was
refused both times.
Reported by Chamal De Silva. Not a regression -- 3.2.7 behaves the same way.
An idle timeout alone is not enough, which the review of the first version of
this change made concrete: safe_read() consults it only when poll() TIMES OUT,
so a peer sending a byte more often than allowed_lull (timeout/2) is never
checked at all. Measured: one byte every 20s held the handshake open for 182s
against a 60s bound, keeping its max-connections slot the whole time -- the
reported attack, merely with the attacker typing.
Non-positive configured values are treated as "use the built-in bound":
"timeout" is parsed with atoi(), so "timeout = -1" would otherwise reach
set_io_timeout() (which reads it as no timeout) and alarm() (which would take it
as a huge unsigned count), disabling the very bound it looks like it configures.
The client's own --timeout is no longer inferred by comparing values, which could
not distinguish it from an identical armed value: io_timeout is zeroed before
parse_arguments(), so anything non-zero afterwards came from the client.
So the bound is absolute and lives in the READ PATH, next to the idle timeout
it complements: safe_read() caps each poll() at whatever is left of it and
gives up when it expires, so it is re-checked on every iteration and a peer
that keeps typing cannot outrun it.
It is deliberately NOT alarm()/SIGALRM. Three earlier attempts used one and
each regressed something: fork() clears pending alarms, so the "post-xfer exec"
parent -- which waits for the ENTIRE transfer -- kept the deadline and _exit()ed
mid-transfer, skipping the hook and releasing the max-connections fcntl lock
while the transfer child ran on; "pre-xfer exec" and the name converter are
operator scripts that may legitimately outlast any handshake bound; and the
cancellation sat inside an exec-environment compile guard, so a build without
setenv/putenv kept it armed through the transfer. A deadline consulted only
where the daemon is already blocked reading a peer has none of those hazards.
It is also kept entirely separate from io_timeout, which is an idle timeout the
module or client may set. Mixing them clamped a module asking for more than the
bound ("timeout = 300" became 60) and leaked the handshake value into the
transfer. Verified: module 300 stays 300, and a client --timeout=7 still wins.
Armed for each peer-driven phase and cleared between them: at the start of the
handshake, tightened by the module's own timeout once the module is known and
its slot claimed, cleared across the hook/fork setup, re-armed before
"@RSYNCD: OK" so it spans BOTH read_args() calls including secluded args, and
cleared before the transfer.
That argument-read coverage is the part that matters most. auth_server()
returns immediately when a module sets no "auth users", so on an ANONYMOUS
module nothing is authenticated: without a bound there, a peer could claim the
slot, take the OK, and trickle an unterminated argument line forever. Measured:
still open after 150s before, closed at 60s after.
.gitignore lists the older helpers (tls, getgroups, wildtest, trimslash,
t_unsafe, getfsdev) but not the ones the security work added, so a `git add -A`
in a built tree stages ~4 MB of ELF -- which is exactly how nine of them ended
up committed on this branch before being removed again.
The three operator-path-traversal daemon tests failed with "escaped: a '..'
traversal reached the excluded subtree" when the build path contained a space.
That reads like a confinement failure and is not one.
rsyncd.conf's "exclude" is a SPACE-SEPARATED list of patterns, so
"exclude = /ws test/.../secret/" is two patterns, neither of which is the
directory meant to be protected. Nothing was excluded, so the traversal
reached a subtree that was never actually off limits.
Confirmed by running the same case with a "filter" rule, which the parser
deliberately does not split at an internal space: it passes, so the traversal
protection itself holds.
Left on "exclude" rather than switched to "filter" -- these tests exist to
cover the exclude path -- and skipped with the reason when the scratch path
makes that config inexpressible.
Worth knowing outside the testsuite: an operator whose module paths contain a
space gets no warning that "exclude" silently matched nothing.
Third layer of the space-in-build-path work, and the first part that is not
test-only.
rsync-ssl expanded the helper program paths unquoted -- "exec
$RSYNC_SSL_OPENSSL s_client ...", likewise for gnutls and stunnel -- so an
openssl installed under a path containing a space is split and never runs.
That affects anyone with such a path, not just the testsuite. Quoted; the
neighbouring $caopt/$certopt/... stay unquoted because they are option lists
that rely on word splitting. Its own re-exec passes --rsh="$0 --HELPER",
which rsync then tokenises, so $0 is single-quoted for rsync's parser.
On the test side, the same shape in generated shell scripts: redirect targets
("printf ... > {capture}") and daemon hook commands, which rsync runs through a
shell, both interpolated a path with no quoting.
In a directory with a space: 235 pass, 18 fail, from 0 able to run.
Unchanged in a normal path: 257 passed, 0 failed.
Second layer of the space-in-build-path work. Quoting the Makefile got the
runner started; these are the places that then hand the binary's path to
something that splits on whitespace.
- RSYNC_CONNECT_PROG is run by a shell. This was the big one: an unquoted
daemon command turned every daemon-mode test into
"sh: 1: /path/to/ws: Permission denied".
- RSYNC_RSH / --rsh is tokenised by rsync itself (do_cmd() in main.c, which
honours ' and "), so support/lsh.sh needs quoting when srcdir has a space.
- --rsync-path is a command line run by the REMOTE shell, so rsync passes it
through unsplit and lsh.sh's eval re-parses it.
- The generated rsync-shim scripts interpolate RSYNC into "#!/bin/sh\nexec
...", where it is shell syntax rather than an argv entry.
rsync_path_arg() and rsh_cmd() build those strings by splitting the command and
re-joining with shlex, so a plain path with a space comes back quoted while a
wrapper command ("valgrind ... /build/rsync") stays several words.
split_rsync_cmd() also has to cope with RSYNC once a test has appended options
to it -- chown-fake and friends do -- where the string is no longer a filename.
It now takes the longest leading run that names an existing file as the program
and splits only what follows.
In a directory with a space: 231 pass, 22 fail, from 0 able to run before the
first commit. Unchanged in a normal path: 257 passed, 0 failed.
`make check` died immediately when the build directory had a space in it:
./runtests.py --rsync-bin=`pwd`/rsync -j 8
rsync_bin /Volumes/Untitled is not a file
Reported by Roland Kletzing building in "/Volumes/Untitled 2"; it reproduces
anywhere, and is not macOS-specific.
Makefile.in interpolated an unquoted `pwd` into --rsync-bin at five sites, so
the shell word-split it. Quote those, and --tooldir at the installcheck site,
which had the same bug and was not in the report. Quote "$(srcdir)/runtests.py"
too: the script's own path word-splits just as readily.
That alone only gets as far as starting the runner. rsync_argv() then did
shlex.split(RSYNC), which turns "/ws test/rsync" into two nonexistent programs.
RSYNC may legitimately be a wrapper command line ("valgrind ... /build/rsync"),
so it cannot simply stop splitting; split_rsync_cmd() checks whether the string
names an existing file first -- a path that exists is one word by definition --
and only falls back to shlex for a real command line. Nine tests that called
shlex.split(RSYNC)/(RSYNC_PEER) directly go through it as well.
Deliberately a function called at use time rather than a pre-split constant:
chown-fake, devices-fake, chown, devices and partial_nowrite append
' --fake-super' or ' --super' to rsyncfns.RSYNC part-way through, and a cached
split hands back the pre-mutation command. Caching it is what broke those two
tests while I was writing this.
The suite is still not space-clean -- in a directory with a space 157 pass and
97 fail, against 0 able to run before. The rest is a separate problem: mostly
transfers whose --rsync-path is re-parsed by a remote shell, which needs
quoting at a different layer. No change in a normal path: 257 passed, 0 failed.
io.c and socket.c include <poll.h> unconditionally, but configure only
required the function. A system that exposes poll() through some other
header would pass configure and then fail to compile -- the AC_CHECK_HEADERS
result for poll.h was collected and never used.
Require the header too, with its own message. Verified both ways: a normal
configure still succeeds, and forcing ac_cv_header_poll_h=no now stops with
"rsync requires <poll.h>" rather than failing later in the build.
highfd-hang probes FD_SETSIZE by compiling a snippet, and passed $CC to
subprocess as a single argv[0]. CC='ccache gcc' then looks for a program
literally named "ccache gcc" and the test dies with FileNotFoundError
instead of probing -- and ccache is wired into PATH on the CI fleet, so
this was reachable rather than theoretical.
Split it with shlex, and treat an unusable CC as "cannot probe" (skip)
rather than an error: the fallback to cc/gcc already handles a missing CC,
and a broken one should behave the same way.
The MSG_IO_TIMEOUT cap and set_io_timeout()'s negative/overflow guards
were written when these loops used select(), and their comments explain
the danger as a tight select()-EINVAL spin on a negative tv_sec.
Under poll() the failure mode inverts: the timeout is a millisecond count
where a negative value means "wait forever", so a wrapped allowed_lull
hangs the process instead of spinning it. The guards are still needed and
unchanged -- only their stated reason was wrong, and a rationale that no
longer matches the code is what gets a guard removed later.
poll_timeout_ms() clamps the value too, so the guards are now belt and
braces; noted so neither looks redundant on its own.
Follow-up to the FD_SETSIZE fix, covering the points raised in review.
Negative/overflowing I/O timeouts. set_io_timeout() could produce a negative
select_timeout (a peer-supplied MSG_IO_TIMEOUT value was applied unchecked),
and every wait now passes select_timeout * 1000 to poll(), where a negative
millisecond count means "wait forever" -- so a hostile or buggy peer could
stall the other side and bypass keepalives entirely. select() used to reject
that with EINVAL, which kept the loop and check_timeout() running. Clamp a
negative argument to 0, compute allowed_lull without overflowing near INT_MAX
(secs / 2 + secs % 2), ignore a non-positive MSG_IO_TIMEOUT value, and funnel
all three waits through poll_timeout_ms(), which keeps the count positive and
bounded.
The daemon accept loop had the same fd_set overflow. start_accept_loop() still
stored listening sockets in an fd_set, so a daemon started with enough
descriptors already open got listener fds >= FD_SETSIZE and hit the same
undefined behaviour at startup -- verified: with the old code a transfer
through such a daemon yields nothing, with this change it succeeds. Converted
it to poll() as well.
Readiness testing. Treating any non-zero revents as ordinary readiness was
wrong: poll() reports POLLERR/POLLHUP/POLLNVAL unrequested, and an invalid fd
shows up as POLLNVAL on a successful poll() rather than -1/EBADF, which left
the EBADF branches dead and let an invalid ff_forward_fd reach
forward_filesfrom_data() (where EBADF reads as EOF). Use role-specific masks
(POLL_RD_BITS / POLL_WR_BITS), handle POLLNVAL explicitly in all three loops,
and request POLLPRI so select()'s old exception set is not silently dropped.
A bidirectional fd is no longer entered twice. A direct daemon connection uses
one fd for both directions; it now occupies a single pollfd row with OR-ed
events instead of two rows carrying different masks, which also avoids the
Cygwin < 3.3.6 duplicate-entry readiness bug.
poll() is now a declared requirement: configure.ac checks for poll.h and
poll(), failing with a clear message rather than leaving it implicit.
The test no longer hardcodes FD_SETSIZE (1024 on glibc but 65536 on 64-bit
Solaris, where it would have opened too few fds and passed vacuously); it asks
the C library for the real value via a small compiled probe and skips if that
is unavailable. Its description now also covers the fortified-libc case, where
the pre-fix result is an abort rather than a hang.
(cherry picked from commit 7ef165dd45)
rsync's I/O loops (safe_read, safe_write, and the main perform_io
multiplexer) waited for readiness with select() and fd_set bitmaps. An
fd_set can only represent descriptors below FD_SETSIZE (1024 with glibc).
When rsync is started with many descriptors already open -- e.g. inherited
from a parent process that leaked fds, a high "ulimit -n", or a busy daemon
-- its own socket and pipe fds get allocated at or above 1024. FD_SET() and
FD_ISSET() then index past the end of the fixed-size fd_set, which is
undefined behavior: select() reports the fd as ready, but FD_ISSET() reads
the out-of-bounds bit as 0, so the read or write never happens and rsync
spins at 100% CPU forever with no progress. This is the long-standing
"rsync hangs at 100% CPU on large systems" report, and it matches the
MemorySanitizer use-of-uninitialized-value seen in perform_io.
Convert the three loops to poll(), which identifies descriptors by value in
a small array and has no FD_SETSIZE ceiling, so a high-numbered fd works
fine. rsync only ever waits on a handful of fds (at most three in
perform_io: in_fd, out_fd, and the files-from forward fd), so poll() is as
fast as -- or faster than -- select() here; the select()-vs-poll() cost gap
only appears when watching thousands of descriptors, which rsync never
does. The remaining select(0, ...) call is a pure timed sleep with no fds
and is unaffected.
The conversion is behavior-preserving: the same max_fd bookkeeping decides
when there is nothing to wait on, the per-fd readiness checks map to the
matching pollfd revents, and the timeout is the same (now expressed in
milliseconds).
testsuite/highfd-hang_test.py reproduces the hang deterministically by
opening enough inheritable dummy fds to push rsync's descriptors past
FD_SETSIZE before an ordinary transfer; it hangs (caught by a timeout) on
the select() code and passes instantly with poll().
(cherry picked from commit 4a751a2ceb)
The test needed strace, so it skipped on every platform without it (macOS, the
BSDs, Solaris, and the AlmaLinux container). runtests.py compares the skip set
against RSYNC_EXPECT_SKIPPED and treats any unexpected skip as a failure, so it
turned the macOS and AlmaLinux 8 jobs red and would have needed an entry in
each platform's expected-skip list -- an entry that would itself go stale the
moment strace became available.
It also earned its keep poorly: it guarded a syscall-count property rather than
correctness, and it was not what caught the --inplace --sparse hole regression
in this series (review and differential fuzzing did). Correctness of the sparse
paths is already covered by the sparse and preallocate tests; the write-count
improvement is recorded, with measurements, in the commit that made it.
(cherry picked from commit b7639c8c71)
Review caught a release-blocking regression in the previous commit: with
--inplace --sparse, interior zero runs inside *matching* blocks were left
allocated.
The scan I added applied only to the write path. The use_seek branch --
reached via skip_matched() when an in-place update finds identical data --
still trimmed just the leading and trailing zeros and lseek()'d over the whole
middle. Before the change, write_file() fed that data through
SPARSE_WRITE_SIZE slices, so an all-zero slice in the middle of a large
matching block became a deferred hole like any other; afterwards those blocks
stayed fully allocated. Reproduced with the reported case: an 8 MiB file whose
every 32 KiB block is 4 KiB data / 24 KiB zeros / 4 KiB data, copied onto an
identical destination with --inplace --sparse --no-whole-file
--block-size=32768, occupied 2048 KiB before this series, 8192 KiB after the
previous commit, and 2048 KiB again with this one. Content was byte-identical
throughout; only the on-disk sparseness regressed.
Rather than duplicate the scan in the use_seek branch, drop that branch and run
both cases through the one loop, with the sole difference factored into
emit_sparse_span(): a span that is not becoming a hole is written normally, or
merely seeked past when the bytes on disk already match. The hole itself is
flushed by the existing flush_sparse_hole(), which already picks do_punch_hole()
over do_lseek() while inside the preallocated extent -- and for an in-place
transfer the receiver sets preallocated_len to the basis size, so a matched
interior hole is genuinely deallocated rather than skipped over.
Verified by differential fuzzing against the pre-series binary: 37 file shapes
(including the reported one, runs either side of the SPARSE_WRITE_SIZE
threshold, all-zero and hole-free files, and randomised mixes) across both the
plain and --inplace modes, comparing contents and allocated blocks. Contents
match and allocation is never worse than before the series.
(cherry picked from commit 231de22f7c)
write_file()'s sparse path sliced each span into SPARSE_WRITE_SIZE (1024-byte)
pieces and write_sparse() issued one write() syscall per slice. Copying a
large *non-sparse* file with --sparse therefore cost roughly one write() per
kilobyte -- about a million write() calls for a 1 GiB file -- which on real
storage ran far slower than the same copy without --sparse (the bug report
measured 1.36 MB/s vs 391 MB/s, ~280x). The 1024-byte chunk is also smaller
than a filesystem block, so it cannot even create finer holes than a plain
copy could.
Rewrite write_sparse() to scan the whole span itself: it looks for interior
runs of zeros that are at least SPARSE_WRITE_SIZE long -- the same hole
granularity rsync has always used -- and emits each intervening non-zero
region (which may include shorter zero runs not worth a hole) with a single
write(). do_punch_hole() advances the file offset just like the lseek() path,
so flushing a deferred hole between segments keeps the position correct.
The hole granularity is unchanged, so sparseness is identical; only the
syscall pattern changes. Measured on a 100 MiB random (hole-free) file:
write() syscalls drop from 100,730 to 6,125 (~16x), now tracking the data's
natural chunking rather than its size in kilobytes. Verified byte-identical
and equally sparse output for hole-free, large-hole, small-interior-hole,
all-zero, --inplace, and --preallocate cases.
testsuite/sparse-write-count_test.py copies a 16 MiB hole-free file under
strace and asserts the write() count stays far below the old size/1024
behaviour (it skips where strace is unavailable).
(cherry picked from commit 5ecab683a8)
The ENOSYS test was the wrong discriminator. It was meant to detect "this
build compiled no fd-relative create at all", but a live mknodat() or
mkfifoat() returns ENOSYS too -- an unimplemented FUSE mknod does, and
seccomp can synthesise it -- so a runtime failure could route a create
through the unconfined path on a platform that has the secure primitive.
On Linux the fall-through lands in do_mknod_at(), which re-confines with
secure_relative_open(), so no escape was reachable there. The real gap is
a mixed-capability build (mkfifoat() but no mknodat()): a runtime ENOSYS
from a real mkfifoat() reached the unconfined fallback even though an
fd-relative FIFO primitive existed.
Whether a primitive exists is a property of the build, so decide it there:
no_atfd_mknod_primitive() is false wherever mknodat() covers the node type,
where mkfifoat() covers a FIFO, or where --fake-super creates through
openat() -- which is always present and was previously able to fall back on
its own unrelated failures.
Also correct the SECURITY.md residual, which overstated the loss. Plain
mknod()/mkfifo() do not follow a planted leaf symlink; they fail EEXIST,
verified directly against a symlink to a victim file. What a no-mknodat
platform actually loses is the pinned parent, so the residual is a
parent-component race rather than a followed basename, and ordinary
fake-super placeholder creation stays confined via openat(O_NOFOLLOW).
And narrow t_symlink_secure's skip: it skipped the whole helper without
mknodat(), including do_symlink_at() assertions that do not depend on it.
Only the do_mknod_at() checks are now skipped, and the helper still skips
outright when neither applies rather than passing vacuously.
The hardlink_symlinks==false branch set five values that differ from the
true branch. Only one of them is real.
Measured on macOS 10.13 -- the one platform that takes it, where
linkat(AT_FDCWD, sym, ..., 0) is EOPNOTSUPP so a symlink cannot be
hard-linked even though ordinary hard links work -- rsync prints the
attribute field as blanks rather than 'c.t.' + dots, says "foo/sym is
uptodate" rather than "foo/sym -> ../bar/baz/rsync", emits no trailing
--copy-dest line at all, and uses .L where the branch expected cL.
Only the change-type letter genuinely differs, and only where the symlink
itself is transferred: hL when hard-linked, cL when copied. So the other
four knobs are gone rather than corrected -- keeping them as variables
that hold the same value on both paths would preserve the suggestion that
something varies.
It went unnoticed because every platform that had run this test takes the
other branch: Linux, FreeBSD and OpenBSD all report hardlink_symlinks
true, and Cygwin skips itemize. macOS before 13 is the first target to
reach it.
Derived by collecting every mismatching block in one run rather than
fixing them one at a time, so the values are what rsync emits rather than
a guess that makes one block pass and leaves the next wrong.
That host is macOS 10.13 and has no mknodat(), so it differs from the
shared macOS list in both directions:
symlink-mknod-fakesuper-symlink-race skips there and only there --
do_mknod_at() is the unconfined fallback on such a build, so the
test skips itself rather than asserting a property the build does
not have. mac2 is macOS 26 and still runs it.
sender-remove-source-root-anchor the macOS list expects it to
skip; this host runs it.
Neither can go in testsuite/skiplist/macos.txt: both Macs share that
file and they disagree. The per-target extra/omit fields exist for
exactly this.
do_mknod_atfd() returns ENOSYS on a platform that compiled no
fd-relative create at all -- older macOS has mknod() and mkfifo() but
neither mknodat() nor mkfifoat() -- and gen_entry_mknod() returned that
straight to the caller, so a FIFO or device node could not be created:
rsync: [generator] mknod ".../afifo" failed: Function not implemented (78)
That is not the stance SECURITY.md sets out. Where an operation can be
secured on some platforms but not others, rsync takes the race-safe path
where it exists and falls back to the historical unconfined behaviour
where it does not, "rather than refusing the operation outright". Its
one stated exception is the nested-socket bind(), which gen_entry_mknod()
already routes away from this path.
So fall through to do_mknod_at(), which on such a platform is do_mknod()
by design. Only on ENOSYS: any other errno is a real failure and must
not be retried through the unconfined path.
None of this was reachable before: the platform did not link at all until
the previous commit, which is why a refusal sitting where the documented
rule says fall back went unnoticed.
The race helper skips itself where mknodat() is absent. do_mknod_at() IS
do_mknod() there -- the held-dirfd walk and the O_NOFOLLOW leaf create are
compiled out, not failing -- so its checks were asserting a property the
build deliberately does not have, and reported the accepted residual as a
module escape.
SECURITY.md gains that residual under "Known residuals": it previously
covered only the socket case, and said nothing about the whole special-
file path degrading where mknodat() is missing.
Checked by rewriting config.h the way macOS 10.13 has it (mknod and
mkfifo yes, mknodat and mkfifoat no): --specials now creates the FIFO
where it previously failed with ENOSYS, the race test skips instead of
failing, and the suite is 254/0. Unchanged on Linux at 255/0.
It used HAVE_MKNOD. Older Darwin has mknod() but not mknodat(), so the
call was compiled and then failed to link:
"_mknodat", referenced from:
_do_mknod_atfd in syscall.o
ld: symbol(s) not found for architecture x86_64
Reported on macOS 10.13.6 x86_64, where config.h carries
/* #undef HAVE_MKNODAT */ next to #define HAVE_MKNOD 1. The sibling
do_mknod_at() already keys off HAVE_MKNODAT and its comment names this
exact platform.
Both occurrences change together. The second guards
return -1; /* mknodat()'s errno (regular/device node) */
against an ENOSYS fallback, so leaving it on HAVE_MKNOD would report
"mknodat()'s errno" on a build where the call was never compiled -- a
stale errno from whatever ran last.
Checked by rewriting config.h the way that platform has it and compiling
syscall.c: the parent leaves one unresolved mknodat reference, this
leaves none. A FIFO still goes to mkfifoat() where that exists, and a
socket still returns EOPNOTSUPP; only the regular/device-node path
becomes ENOSYS, which is what a platform without mknodat() can offer.
A checked receiver-side directory option whose leaf does not exist can
be created as attacker-controlled transfer content and then consumed by
the same transfer: --backup-dir=a, with "a" arriving in-band as a
symlink pointing out of the restricted directory. The same shape works
against --copy-dest, where it reads an outside file and delivers it to
the client. Passing the leaf under the parent's /proc/self/fd pin is
not enough: the peer's symlink wins the race to the name, and rsync
creates or reads through it.
The answer is to hand rsync an inode rather than a name -- but what
inode depends on what rsync does with the option, so the policy is per
option rather than per type ("type 2" means "check when receiving", not
"is a directory"):
--backup-dir, --partial-dir rsync creates them on demand. rrsync
creates them instead, walking down from
the restricted dir one component at a
time with O_NOFOLLOW (nested names too:
make_bak_dir() builds a hierarchy), and
pins the result. The partial dir is
made 0700, as rsync makes it.
--temp-dir rsync requires it to exist, so a missing
one stays an error.
--link-dest, --compare-dest, rsync only reads through these, and a
--copy-dest missing one is the ordinary first-run
case that must keep working. rrsync
pins an empty directory it then unlinks:
the transfer behaves as with a missing
one, and there is no name left for the
peer to take over. Not quite identical:
rsync prints "--link-dest arg does not
exist" for a genuinely missing basis and
the placeholder suppresses that.
If the transfer later replaces a created name, the held inode is merely
detached -- the backup fails, it does not escape.
Without /proc/self/fd there is no way to name an inode, so on those
platforms every missing type-2 option path is refused instead -- the six
above, not the type-3 paths covered at the end.
That is the same fail-closed behaviour this change originally had
everywhere; the pinning is what buys back first use where it can.
An earlier version refused every missing type-2 leaf on every platform.
That closed the pivot but broke first-use --backup-dir and --partial-dir
-- including --partial-dir=.rsync-partial, the documented resumable-
upload idiom -- and would have broken first-run --link-dest, which is
how every rotating-snapshot script starts.
The regressions run their security assertions BEFORE their controls, so
an environment where a control fails for an unrelated reason cannot mask
the escape check by aborting first -- which is exactly what happened to
one reviewer. The alt-dest one also has to beat a race: the generator
runs ahead of the receiver, so sorting the pivot symlink first does not
guarantee it is installed before the basis lookup, and a run where the
generator won would pass vacuously. A few thousand files in between
give the receiver the head start, and the test asserts the symlink was
really installed rather than trusting the ordering. Measured 5/5 RED on
the parent, 3/3 GREEN here.
The regressions cover each branch of the policy: the backup-dir pivot,
the --copy-dest read escape (its own test), first-use --backup-dir,
nested first-use --backup-dir, first-use --partial-dir including its
mode, a refused missing --temp-dir, an accepted first-run --link-dest
with no placeholder left behind, and an existing basis still working.
Each requires the specific mechanism rather than just "the outside file
was left alone", which any unrelated failure would satisfy, and each
takes the refusal branch where the pin primitive is unavailable.
Behaviour worth knowing about, since rrsync now creates these rather
than rsync: they are created while the ARGUMENTS are parsed, so they
appear even under --dry-run, where rsync's own make_path() deliberately
does not mkdir; and one is left behind if the transfer then fails. A
pinned --partial-dir is also one directory rather than one per
destination directory, and rsync will not auto-remove it -- which is not
new, rrsync already rewrote an existing relative partial dir to its pin.
A nested --partial-dir now behaves differently from plain rsync, which
documents creating "just the last directory -- not the whole path". With
--partial-dir=a/b and "a" missing, rsync creates nothing and rrsync
creates both. Containment is unaffected -- each component is made
beneath the fd already held, O_NOFOLLOW -- but rrsync accepts a shape
rsync would not honour, and since rsync only removes the last component
of a relative partial dir, an empty parent is left behind.
Not fixed here, and NOT claimed to be: this covers the six type-2
directory options. --files-from, --log-file and receiver positional
paths are type 3; where the leaf and its parent are both missing, or on
a platform without /proc/self/fd, those still reach rsync unpinned, as
SECURITY.md describes. That is the rest of the issue, not this one.
The wrapper is handed a shim rather than RSYNC directly: RSYNC is a
multi-word command whenever the runner forces --protocol=N, and rrsync
execlp()s its RSYNC as a single executable name, so this test died
before reaching the policy under test. A fleet run caught it on the
protocol columns of three targets.
The expected-skip lists are now files referenced as @FILE and composed
(common + platform + protocol), and they are expanded here, on the
target, against the tree that shipped. That is deliberate, but it means
nobody upstream of this point can SUBTRACT: the name lives inside a file
the composer does not read.
fleettest needs exactly that. A target may run the same build against a
different filesystem, and then tests its platform list expects to skip
genuinely run -- a scratch dir on a second volume makes backup-crossdev
-copy and chmod-temp-dir work. With additions only, such a target can
never be green.
Accept a '-name' entry, applied after every addition so order does not
matter. A test name never begins with '-', so the token is unambiguous.
Every removal must remove something: a name nothing added is stale, and
a repeated removal is that same no-op written twice. Quietly shrinking
the expected set is the failure this parser exists to prevent, so both
are refused rather than left to sit in a config unnoticed.
Targets were all submitted to the pool at once, and the per-run build
directory was named for the run alone -- <builddir>-<run_id>, identical
on every target. Both assume one target per machine.
Two targets naming the same host break that, and the fleet now has such
a pair: mac2 and mac2-hfs are one Mac, differing only in where the
tests' scratch trees live. They pushed into the same directory and
built over each other, and BOTH reported BUILD-FAIL -- a failure that
looks exactly like the code under test not compiling. Either target run
by itself was fine, which is the worst way for this to present.
The build directory now carries the target name too, and a machine's
targets run one after another, with a log line saying so. Serialising
matters beyond the shared directory: two suites on one host would fight
over the fixed ports the daemon tests claim, and over every other piece
of host-global state, so separate directories alone would not be enough.
Different machines still run concurrently, which is where the
parallelism actually was.
The target name is reduced to [A-Za-z0-9._-] before it goes into a path
that cleanup later feeds to rm -rf, so a name cannot contribute a path
separator, a shell metacharacter or a leading dash. --cleanup still
globs <builddir>-*, which the longer name matches.
make_variety_tree() sets an xattr on every file, including the ones it
deliberately creates read-only, and tolerates a refusal:
try:
xattr_set('variety', os.path.basename(str(p)), p)
except OSError:
pass
That handler works only on Linux. There xattr_set() calls os.setxattr()
and a refusal is an OSError; every other platform shells out to a CLI
with check=True and raises CalledProcessError, which is not an OSError
and sails straight past. So a refusal the suite tolerates on Linux can
kill variety and variety-symlink-traversal on any CLI-backed platform.
macOS, Cygwin, FreeBSD and Solaris all carry the defect; macOS is where
Roland Kletzing hit it, on test8 through test10, on the perm7 file:
xattr: [Errno 13] Permission denied: '.../d1/d2/.../d7/perm7'
subprocess.CalledProcessError: ... returned non-zero exit status 1
What triggers it there is a non-root run meeting the mode-0400 files the
tree deliberately creates, which their own owner cannot attach an xattr
to. That is why the fleet, which runs as root, never saw it. Root is
not immune to every refusal, just to that one.
Route the four CLI branches through a helper that raises XattrError, an
OSError subclass, so one handler covers every platform. It carries an
errno only when the tool named one, and only macOS's xattr(1) does:
setfattr and setextattr just say "Permission denied", and guessing an
errno back out of localised strerror text would be worse than admitting
we do not know. The match is anchored to that tool's own prefix on the
first line -- the rest of the line is a filename, and a file can perfectly
well be called "[Errno 5]".
devices/devices-fake had already worked around this locally by catching
both types; that catch is now dead, so drop it.
Verified by forcing the CLI branch on Linux and running variety non-root:
it fails with Roland's traceback, on the same perm7, and passes with
this. That establishes the exception path every CLI branch takes, not
macOS's xattr(1) in particular -- his report supplies that half.
runtests.py already honours $scratchbase, but a target could not use it:
the sudo branch runs `sudo -n env PATH="$PATH" ...`, which drops whatever
env_prefix exported. Setting it there looked like it worked and silently
ran on the default filesystem instead -- the first HFS+ run came back
green for that reason. Give it a target field carried inside the env
string, on both the root and non-root paths, shell-quoted so a volume
name containing a space does not turn into a stray argument. The
non-root pass also clears the relocated scratch, which a prior sudo run
leaves root-owned outside builddir.
expect_skip_omit is the mirror of expect_skip_extra: entries the
workflow expects to skip which a target actually RUNS. Relocating the
scratch supplies conditions the workflow's host lacks -- a separate
volume makes backup-crossdev-copy and chmod-temp-dir reachable -- and
without a way to subtract, such a target can never be green.
mac2-hfs runs the same host and build as mac2 with the scratch on HFS+.
It verifies the mount rather than assuming it: a stale directory, or a
name collision attaching at "RsyncHFS 1", would otherwise leave the
tests on APFS reporting green, which is how the first version lied.
Ownership must be on as well, and is now checked rather than attempted:
a user-attached image mounts "noowners", under which every uid/gid and
permission check is meaningless, and that alone accounted for 28 of the
31 failures the first honest run produced.
mac-x86 is the x86-64 Mac -- the only target that can build the x86-64
md5 assembly, since mac2 is arm64 where configure refuses
--enable-md5-asm outright. It needs MacPorts for autotools, python3 and
the crypto/hash libs. --enable-roll-simd is not set and cannot be: that
probe uses GCC-style function multiversioning, which clang does not
support on Mach-O, failing identically under Apple clang 10 and clang 19.
mac-x86 currently BUILD-FAILs on the unguarded mknodat() in
do_mknod_atfd() (#161), which it reproduced on its first run.
itemize picks between two expectation sets using rsync's own
"hardlink_symlinks" build capability. That says nothing about the
filesystem underneath: on macOS the build reports true while HFS+ returns
ENOTSUP for link()ing a symlink, and the run dies with
failed to hard-link .../foo/sym with foo/sym: Operation not supported (45)
Selecting the other expectation set does not help and would assert
something untrue: the itemisation follows the BUILD capability, so rsync
still prints "foo/sym is uptodate" and ".L foo/sym -> ..." even though
the link failed. Neither set describes that combination.
It is an rsync gap rather than a test one. generator.c reports the
runtime linkat() failure as FERROR_XFER and the transfer exits 23, after
which rsync creates the symlink anyway -- while a regular file in the
same position falls back to a local copy, and so does a build compiled
WITHOUT symlink-hardlink support. Falling back on ENOTSUP would make
this pass by itself.
So XFAIL rather than skip: the failure stays visible and flips back to a
pass once rsync falls back. XFAILing the whole test is blunter than the
one --link-dest case deserves, but the symlink expectations are threaded
through every assertion here rather than confined to one.
The probe answers only the question it is asked: a link() refused for any
other reason -- EPERM, ENOSPC, EMLINK, a quota -- propagates instead of
being reported as a capability difference and quietly reshaping the
expectations.
operator-path-temp-dir and operator-path-partial-dir decided whether a
symlink had been followed by sampling the target directory's
st_mtime_ns, sleeping 10ms, and looking for a change. The temp file is
renamed away, so an mtime bump was the only trace left.
On a filesystem whose timestamps have 1-second granularity -- HFS+, and
it is not alone -- a change within the same second is invisible. The
delta is zero, the test concludes the symlink was not followed, and
reports the operator's OWN euid-owned symlink as refused when it was
followed correctly. Both fail that way on HFS+ while passing on APFS,
and operator-path-partial-dir is one of the failures Roland Kletzing
reported on macOS.
Pin the directory's mtime to a fixed past epoch instead, read back what
the filesystem actually stored, and ask afterwards whether it still
holds -- reading back because a filesystem may clamp or round the value,
and comparing against the requested epoch would then read an unfollowed
symlink as followed. temp-dir-symlink-injection already works this way.
This is not proof against every clock: a directory whose mtime lands
exactly on the stored sentinel would still read as unfollowed. That
needs the host clock set to 2001 or a deliberate restore, where the old
10ms delta failed on any coarse-granularity filesystem.
Verified in both directions by running as root, where the matrix also
exercises the cross-uid cells: a followed symlink moves the mtime off
the sentinel, a refused one leaves it.
The %H allow-list was wrong in both directions.
Several accepted characters change an argument's MEANING rather than its
text when they lead the value, which quoting cannot prevent because the
word stays intact -- that IS the problem:
'-' and '+' introduce options to plenty of programs; with
RSYNC_CONNECT_PROG="prog %H", hosts "-c" and "+x" arrive as
options, and "sh +x" is as real as "sh -x";
'~' is tilde-expanded by the nested shell, turning ~root into /root;
'%' is expanded by a nested fish, where %self becomes its pid.
An empty host has the same shape from the other end: it survives a direct
exec as an empty argument but disappears when a nested shell re-splits the
command, shifting everything after it. rsync://:873/m/ and ::m/ both
produce one. None of these can begin a real hostname, so refuse them in
first position only -- mid-word each is literal, which matters because an
IPv6 zone id carries its '%' mid-word.
The other direction: '+' and '~' were refused outright. They execute
nothing, and RSYNC_CONNECT_PROG exists for custom transports where %H is
often an alias the program resolves itself rather than a name the
resolver sees. Refusing them mid-word breaks that use case for no gain.
A non-ASCII host stays refused. That is a policy choice rather than a
free one: a custom connect program never calls getaddrinfo, so a Unicode
alias would otherwise work, and this does exclude it. A punycode A-label
is unaffected.
What this cannot do is bound what the named program makes of the value.
"host:-rf" arrives intact, and a program that splits on ':' may
reinterpret the tail; that boundary belongs to whoever writes the
command.
The test asserted only that a marker file was absent -- equally true when
rsync failed to parse its arguments, when socketpair_tcp is blocked, or
when touch was missing. Worse, the marker path was absolute, and a URL
authority ends at the first '/', so the injected `touch` never received
an operand and the check could not fail even with the guard gone. It now
runs with cwd set to the scratch directory and injects a bare name, so
the marker is genuinely reachable; requires the specific refusal message;
checks the exit status; and checks that ordinary hosts still arrive at
the connect program with their text intact, which is the part an
absence-only test can never show.
Fault-injected separately: dropping the guard, dropping just the
first-character check, and narrowing the set back each fail the test on
their own, in both the default and --use-tcp transports.
The refused set was built from the characters that obviously execute
something, and missed three that a SECOND shell acts on:
'!' negates in command position. A hook written as an access check --
`pre-xfer exec = sh -c '%RSYNC_USER_NAME% false'` -- becomes
`! false`, reports success, and serves the transfer. An
authenticated user named "!" turns a denial into an approval, which
is precisely the case the fail-closed comment above exists for.
'~' is tilde-expanded, so ~root becomes /root.
'{' and '}' brace-expand in bash and zsh.
None of them execute anything on their own, which is how a set built from
the obvious metacharacters came to miss them. That is also the standing
weakness of the approach: this is a deny-list, and the two rounds of
review it took to find '!' are the argument for eventually inverting it.
The documentation is corrected with it -- it claimed every shell-active
character was refused, which this disproves -- and now lists the set.
Each listed character is pinned by the test, which needed its module
paths to EXIST first: a missing path fails the transfer on its own, so
checking the exit status alone passed whether or not the character was
refused. Removing any single character from the set now fails the test.
It asserted that a marker was absent and that rsync exited non-zero. Both
are equally true when authentication failed, when the daemon never
started, or when the globbed command was missing -- so it passed for any
of those, with or without the guard. Changing only the password to a
wrong value left it green.
Require the daemon log to carry the specific refusal, so a transfer
stopped for some other reason no longer reads as the value having been
refused.
Add a positive control. The test now runs the same nested-shell
expansion itself first and requires it to work; without that, a pass
could equally mean the attack was inert here and rsync was never tested
against a live one.
Stop globbing onto /usr/bin/touch. The command the expansion selects is
now one the test writes into its own directory: the build never
guaranteed a system touch, and with the old oracle a missing one produced
a pass. Exactly one file there matches "touc?", so the expansion is
unambiguous.
The docstring described the value becoming "find arguments", which is not
what the hook does -- it selects /usr/bin/touc? and glob-expands it to a
command. Say what actually happens.
Checked in both directions: the wrong-password mutation that used to pass
now fails, narrowing the refused set back to the pre-existing one fails,
and the unmodified test passes under both the default transport and
--use-tcp.
Refusing a shell-active %VAR% in an exec hook is a real usability cost
and it is not confined to hostile input: the check runs on every
%RSYNC_*% value, so a module whose path holds a space cannot be
interpolated into a hook at all. `path = /srv/My Backups` with a command
mentioning %RSYNC_MODULE_PATH% refuses every transfer of that module,
with no attacker involved.
That is deliberate rather than an oversight. rsync escapes a
substitution for the quoting context it sits in, which is right for the
one shell that runs the command and wrong for a command that starts a
second one -- `sh -c '... %RSYNC_USER_NAME% ...'` hands the inner shell a
bare value, where "touc?" is glob-expanded against /usr/bin and chooses
the command rather than being data for it. Escaping for an unknown
number of passes is not possible, so the value is refused instead. Nor
can the check exempt operator-supplied values: `path` may itself be
templated from a peer one (`path = /home/%RSYNC_USER_NAME%`), and once
both are inside the same string rsync cannot tell them apart.
Say so in the manual, and give the way out: the same names are exported
to the command, so $RSYNC_MODULE_PATH inside a script is unrestricted.
The test pins both halves, since a documented behaviour with no oracle
drifts. Both are fault-injected: narrowing the refused set back makes
the interpolated half pass when the manual says it must not, and dropping
the RSYNC_MODULE_PATH export fails the workaround half.
An invalidly encoded list must exit 2 like any other unreadable one, but no
case created invalid bytes, so dropping UnicodeDecodeError from the except
would have left the test green (failing closed with a traceback instead).
Follow-up review found three guards that the test observed only indirectly:
removing the one-name-per-line check, or relaxing the regular-file check back
to exists(), or dropping a separator check still produced a hard error via the
"no such test" path, so the test could not tell them apart. Each now runs
against a stand-in suite directory containing fixtures that make the malformed
name look real ("acls sparse_test.py", a directory named adir_test.py), so the
guard under test is the only thing that can reject it. Verified by mutation:
each guard removed in turn makes the test fail on its own case.
Also: reject a comma in a name (it is the separator of the csv this returns
and of the summary line the fleet parses back); catch UnicodeDecodeError
alongside OSError so a mis-encoded list exits 2 rather than tracebacking; and
exercise a non-trivial relative srcdir, since in-tree os.path.relpath() is
just "." and would not have caught prefix doubling.
Two comments overstated things: a truncated list does not silently weaken the
oracle -- the comparison is exact, so it surfaces as a page of unexpected
skips. Rejecting it is about failing where the mistake is, not about the
oracle going blind. And extras merge into the passes the workflow pins, not
into every pass.
Review of the previous commit found four ways the parser or its test fell
short of what that commit claimed:
- An empty or comment-only list, and an empty entry within a spec (`a,,b`,
which is what an unset shell variable expands to), both expanded quietly
to a smaller expected set. A shrunken expectation is a weaker oracle, so
these are hard errors now; a wholly empty spec remains the legitimate
"expect no skips".
- Name validation accepted a path, so `../testsuite/acls` passed as a test
name. Names must be plain and resolve to a regular file.
- skiplist-spec_test.py built @FILE paths from a relative srcdir and handed
them back to a srcdir-relative API, which doubled the prefix -- it would
have failed under `make installcheck` (--srcdir=../src). It also proved
the sort check with a name that does not exist, so deleting that check was
masked by the stale-name check; it now uses two real tests out of order,
and covers the cases above. Each guard verified by mutation.
- fleettest returned an extras-only expected set for a pass whose workflow
has no matching step (a non-Linux target with protocols=[29]), a
guaranteed mismatch. An unpinned lane is now simply unpinned.
Also restores the fleettest CI path filter that the previous commit dropped:
that job must run when runtests.py or a skip list changes.
Every branch that added a test which skips somewhere had to edit
RSYNC_EXPECT_SKIPPED, a single ~3 KB YAML line duplicated across seven
workflow steps -- so two such branches always conflicted, and the conflict
was in the one format git cannot merge.
The lists move to testsuite/skiplist/*.txt, one name per line with the reason
as a comment, and RSYNC_EXPECT_SKIPPED takes @FILE entries which runtests.py
expands (relative to srcdir, so out-of-tree builds work). Several compose,
which lets the 46 names common to Linux/macOS/Cygwin live in one file: adding
a require_tcp test now edits one line of common.txt instead of three lists in
three files.
Lists must be sorted, duplicate-free, and name real tests, and an unreadable
or malformed list is a hard error -- it must never degrade to "expect no
skips", which would silently disarm the oracle on that job. fleettest passes
the spec through to the remote runtests.py, which expands it against the tree
that was staged there.
The oracle itself is unchanged. Verified on Linux by running the full suite
in all three lanes (check, check30, check29) against the new files: same
expected sets, all green.
The check29 steps reused the plain check list, so six tests that skip only
under --protocol=29 were unaccounted for and the run failed its expected-skip
comparison. They gate on the wire version rather than the platform: ACL and
xattr transfers need protocol 30+, and the stdio_daemon helper speaks 30.
Verified by running the full suite at both protocols on Linux: the 29 skip set
is the default set plus exactly these six. acl-symlink-race already carried a
comment saying its protocol gate had to be represented here.
--ignore-existing protects the live transfer destination, but three
peer-selectable options reach other existing objects inside the
restricted directory: --log-file appends to one, --partial-dir consumes
and then renames or unlinks one, and --delay-updates does the same
through its implicit .~tmp~ directory. Refuse all three under
-no-overwrite.
Refusal rather than confinement, because confining these paths does not
help: keeping the --log-file append inside the tree still appends to an
existing file, and for the partial directories the peer controls both
the directory and the transferred basename, so a collision is always
reachable. A pre-exec emptiness check would be raceable. The cost is
that a push naming --partial-dir on the remote receiver, or using
--delay-updates, is now refused for a -no-overwrite account.
Each regression drives the option through a wrapper WITHOUT
-no-overwrite as well, and requires that to be accepted. Without that
control the tests cannot tell "refused under -no-overwrite" from
"refused always", and would still pass if the options were disabled for
every rrsync deployment -- verified by making the refusal unconditional,
which the controls then catch.
The wrapper is handed a shim rather than RSYNC directly: RSYNC is a
multi-word command whenever the runner forces --protocol=N, and rrsync
execlp()s its RSYNC as a single executable name, so every one of these
tests died before reaching the policy under test. A fleet run caught it
on the protocol columns of three targets.
Backup mode belongs in the same set. Publishing a backup onto a name
that already exists deletes what is there (backup.c make_backup()), and
deleting a file backs it up first (delete.c), so a --delete of an
unrelated file can land on a protected name -- overwriting a file that
--ignore-existing was holding, with rc=0 and no diagnostic. Disable -b
and --backup-dir. --suffix is left enabled: with both of those refused
nothing can turn backups on, so it is inert.
The short option must be disabled before short_no_arg_re is built, since
a stock client sends b inside the remote short-option bundle and the
regex is snapshotted there; the whole -no-overwrite block therefore
moves up beside the other policy gates rather than sitting after the
chdir. Its regression drives the collision through a wrapper without
-no-overwrite, passing --ignore-existing by hand, which both proves the
refusal is conditional and shows the primitive defeating the very
protection -no-overwrite forces.
rrsync.1.md now states what -no-overwrite costs: no explicit
--partial-dir, no --delay-updates, no server-side --log-file, no backups.
Write-only mode rejects ordinary downloads, but a remote --files-from
makes the receiving child open a server-side file and send it back over
the upload protocol. Reject server-local --files-from paths under -wo,
keeping the exact "-" sentinel that a client-local files-from upload
sends.
The control now uploads a second, unlisted source file and requires it
NOT to arrive. With a single file present an ordinary recursive upload
looks identical, so the control passed whether or not the list selected
anything.
The wrapper is handed a shim rather than RSYNC directly: RSYNC is a
multi-word command whenever the runner forces --protocol=N, and rrsync
execlp()s its RSYNC as a single executable name, so this test died
before reaching the policy under test. A fleet run caught it on the
protocol columns of three targets.
Known gap: this closes the argv route only. A per-directory merge
filter delivered over the protocol still reads a file outside the
restricted directory and returns its content in an "Unknown filter rule"
error -- reproduced with a stock client against -wo. rsync's existing
filter-file confinement does not apply because it is gated on am_daemon
and an rrsync server is not a daemon. Tracked separately.
A daemon parses the peer-supplied server argv, so a client naming a large
--compress-threads on a pull makes the daemon-side sender materialize
that many Zstandard workers: 256 was measured as 257 threads in one
connection. No custom client is needed -- a stock rsync forwards it with
-M--compress-threads=N -- and on an anonymous module no authentication
happens first. Clamp it to 8 on a daemon; local and remote-shell
invocations keep the operator-requested value.
A push parses and clamps it too, but creates no workers there: the
option affects compression, not decompression.
The test asserts the implementation's own bound, 8 workers plus the main
thread, rather than a looser threshold that a build unable to create
workers at all would also satisfy -- so it first requires a worker pool
to be reachable and skips if it is not, then requires it to be bounded.
It needs --use-tcp and is declared in the workflows that enforce a skip
set.
Whether the platform can be counted at all is asked once, before the
answer is folded into a max(): thread_count() returns -1 where it has no
way to look, and max(0, -1) is 0, so the -1 could never reach the check
meant to catch it and an uncountable host looked instead like a sender
that died. A fleet run had Cygwin failing for exactly that reason.
Left deliberately open: the cap is silent, is not expressible in
rsyncd.conf, and does not bound the total across connections, since
max connections defaults to unlimited.
parse_one_refuse_match() marked only the first long_options row whose
long name matched the configured spelling, then broke out for a
non-wildcard rule. --compress-threads and --zt are separate popt rows
that both write &do_compression_threads, so "refuse options =
compress-threads" disabled the canonical row and left the alias
accepted: the refused capability was still reachable under its other
name. The same shape covers zc/compress-choice and zl/compress-level.
An exact rule names a capability, not one spelling of it, so mark every
row that does the same thing. Comparing the raw table fields is not
enough for that: popt's `val` means different things per argInfo. For
POPT_ARG_VAL it IS the value stored in `arg`, while elsewhere a nonzero
`val` is an action code for the parser's switch, and POPT_ARG_NONE with
a destination stores 1 whatever `val` says. --del is
POPT_ARG_NONE/&delete_during/0 and --delete-during is
POPT_ARG_VAL/&delete_during/1: the same destination and the same
resulting value, but unequal as table entries, so "refuse options =
delete-during" was still evaded by --del and the mirror held too.
Compare what a row does instead -- the destination and the constant it
assigns, falling back to table-entry equality for rows that store a
runtime value or only dispatch an action. Enumerating all 258 rows,
this couples exactly one pair the field comparison missed, del and
delete-during, and changes nothing else. Opposite switches such as
--foo and --no-foo stay distinct because they assign different values.
Two regressions. The compress-threads one drives the raw daemon
protocol -- not to preserve the spelling, which -M--zt=N would do just as
well, but because it goes on to observe the worker pool the bypass
delivers. Its oracle is the refusal itself -- the alias connection torn
down and
"configured to refuse --zt" logged -- and deliberately not the resulting
worker count: an accepted --zt is a defeated refuse rule however few
threads it produces, and the daemon worker cap being added alongside
this holds that count to 9, so a count-based assertion passes while the
alias is still accepted. Run that test against the cap without this
parser fix and it does exactly that; the two changes were covering for
each other.
The delete one needs neither zstd nor a socket: --remote-option puts the
option in the daemon's argv verbatim, which is the reach an ordinary
user already has, so it drives a stock client both ways round against
modules refusing each spelling, with an unrefused module as the control.
The compress-threads test needs --use-tcp, so it skips in every other
column and is declared in the workflows that enforce a skip set, which a
fleet run otherwise reports as an unexpected skip on fourteen cells.
malicious-server-partial-basis-symlink-overwrite waits for the client's
generator to ask for index 1 before sending its forged basis-type
response. It waited by searching the raw stream for one hard-coded
three-byte marker: the ndx delta, then exactly
ITEM_TRANSFER|ITEM_IS_NEW.
The generator is entitled to send more than that. When it has already
chosen an alternate basis it also sets ITEM_BASIS_TYPE_FOLLOWS and
appends a basis-type byte, which it does once it has noticed the
partial-dir file this test plants. Whether it does varies between
environments -- consistently not on this Linux box, consistently so on
the OpenBSD VM. I have not identified what differs; the planted file
and the victim both exist before rsync starts and the malicious sender
does not touch them, so calling it a race would be a guess.
Where the extra flag appears the marker never matched, and the test
timed out after ten seconds reporting "generator did not request file
index 1" -- which was untrue. The captured bytes decode as the
auto-added perishable filter rule "-p .rsync-partial/", the filter-list
terminator, then ndx 1 with flags 0xa800 and the byte 0x81,
FNAMECMP_PARTIAL_DIR: the very constant this test defines. It failed in
five of six fleet runs and reproduced on an isolated single-target run,
so it was not load-related flakiness.
Read the request properly instead: consume the filter list, then the
index, the item flags, and the optional basis-type and xname fields,
using the framing rather than hunting for a byte pattern that can also
occur inside file data.
Both outcomes are worth driving, and they prove different things:
* without the flag the response SUBSTITUTES a partial basis the
generator never asked for -- the unbound-basis-type bug;
* with it the generator asked for that basis itself, so the response
substitutes nothing and the test proves the receiver CONFINES a
basis it did request.
The comment claimed the first unconditionally, which was wrong wherever
the second happens. Any other basis type now fails as a fixture change.
Failing the old way was safe -- the forged response was never sent, so
no false pass was possible -- but it cost a red cell on the fleet and
hid whatever else that target had to report.
A restricted dir must not let a client have the spawned rsync create
devices or special files in the served tree, but -a bundles -D into the
client's short options, so refusing -D outright breaks every ordinary
`rsync -a`. 88cee089 forced --no-D instead. That option also clears
the rdev framing, and rrsync sets it on one end only, so the file list
desynchronised: a FIFO push hung at protocol 29 and corrupted the list
at 30, and a device push failed at every protocol including 32. Use
--drop-D, which withholds the creation without touching the wire.
Only on the receiving side. A sender creates no received device or
special entry in the served tree, so there is nothing to deny and
--drop-D is a no-op there; forcing --no-D on a pull was the same
one-sided change in the other direction, and broke pulls the same way.
3.4.4 forced nothing at all and is the behaviour a pull now gets back.
rrsync-specials-denied asserted the option on a "--server --sender"
command line, which conflated the two directions. It now checks that
the receiving side forces --drop-D and still forwards the client's own
-D -- without which the two ends frame the list differently again --
that the sending side forces neither, and, rather than only what is
forwarded, that a real push cannot create a FIFO while the rest of the
transfer succeeds. An ordinary file alongside is the control, since a
push that failed outright would "deny" the FIFO too.
The device case gets its own push, because a device desynchronises at
every protocol while a FIFO only does so below 31. It needs no mknod
privilege: rsync's fake-super "%stat" xattr is what makes a file a
device to rsync, and running the RECEIVER under --fake-super too lets it
record one without privilege -- so the case asserts that nothing of that
name appears, not merely that the transfer survived.
rrsync-pull-arg-shapes could previously assert only that pulling a FIFO
did not hang, because forcing the option on that side broke the transfer
outright. It now requires the pull to succeed and deliver a FIFO, which
is what a pristine 3.4.4 rrsync does.
-D and --no-D do two jobs at once: they decide whether devices and
special files are created, and they decide whether those entries carry
their rdev fields on the wire. send_file_entry() and recv_file_entry()
frame those fields with the same condition, but each end evaluates its
own preserve_devices/preserve_specials, so the two only agree because
both normally parse the same command line.
That makes --no-D unusable for a wrapper that controls one end of a
connection and wants to deny creation. Give it to the receiver alone
and the client's -D sender writes rdev the receiver never reads: the
file list desynchronises from that entry on. A FIFO or socket breaks
below protocol 31 -- a hang at 29, "File-list index 0 not in 0 - -1" at
30 -- and a device node breaks at EVERY protocol, current ones included,
because its arm of the condition has no protocol clause at all.
--drop-D separates the two jobs: it refuses the creation and leaves the
encoding alone. The entry is skipped through the existing non-regular
fall-through, so the visible result matches --no-D, and because it
touches no wire state it can be applied to one end by itself.
It has no effect on a sending rsync, which creates nothing.
Two shapes a pristine 3.4.4 rrsync transfers, and 88cee089 broke, both
from one cause: the pin opens the argument's CONTENT, when for a sender
rsync often only needs to name or describe it.
* an in-tree FIFO wedged rrsync before exec. O_RDONLY on a FIFO blocks
until a writer appears, so an authorised user naming one could
accumulate stuck processes indefinitely.
* an in-tree dangling symlink failed the transfer. realpath() resolved
it to a missing target and the ENOENT was reported as a detected
race, though a dangling link is an ordinary archive entry that rsync
transmits by its target string without opening anything.
So only a regular file or a directory gets its content opened; anything
else keeps the realpath()-validated name. The sender never opens these,
it only describes them.
The leaf is still spelled beneath a pinned directory. An
earlier form of this commit left the bare name for rsync to re-resolve,
on the reasoning that 3.4.4 passes it that way -- but that puts every
component back in play and reintroduces CVE-2026-53783 for the shape:
with an in-tree "dir/target" that is a dangling symlink, flipping "dir"
to a symlink pointing outside leaked the outside file's content in 3 of
83 raced pulls. With the parent pinned it is 0 in 104 -- but a race only
samples the window, and zero in 104 still leaves a few per cent of
per-attempt risk unmeasured, so rrsync-sender-parent-pin closes it
deterministically instead: a stub standing in for rsync inherits the
pinned descriptor and blocks, the parent is swapped for a symlink out of
the tree while it is blocked, and only then does the stub resolve the
argument. It reports the in-tree leaf with the pin and the attacker's
file without it, so it fails outright if the pin is removed rather than
depending on winning anything. A control first proves the swap really
does redirect the bare name, or the assertions would prove nothing.
Pinning the
parent costs nothing here -- pin_dir() opens it O_PATH, so the special
file itself is still never opened and a FIFO still cannot block, and
whatever the leaf becomes afterwards is reached only from beneath the
held one.
Which directory that is, sender_pinned_arg() already decides, and for
every shape except one it is the immediate parent. The exception is a
--relative argument with no client "/./": there the whole argument is
the transmitted name, so only the anchor it starts from can be pinned
and the components below it stay raceable. That limit predates this
commit and NEWS states it; "the parent is pinned" is not true of that
one shape.
Two boundaries this must NOT cross, each found the hard way:
* a trailing "/" or "/." argument keeps its leaf pin: rsync opens that
one and does follow a symlink there. Declining it made
rrsync-sender-leaf-flip leak the outside directory's content.
* the decision is not gated on HAVE_PROC_SELF_FD. It is about what
rsync does with the argument, not about whether we can pin it, so
gating it left the dangling-symlink failure in place on the BSDs,
macOS, Solaris and Cygwin.
The shape matrix grows fifo, dangling-symlink and symlink-to-file cases,
and now asserts what each delivered entry IS -- kind, symlink target and
content -- on every case rather than spot-checking a couple at the end.
A name-only comparison is satisfied by an empty directory called "f1",
or by the correctly-named but empty symlinks that handing the sender a
magic link produced. It still passes against a pristine 3.4.4 rrsync.
The FIFO case asserts only that the pull does not hang. What a special
file does on the wire is decided by the --no-D that a restricted dir
forces on the remote side alone: the sender then omits the old-protocol
rdev fields that the client's own -D receiver still reads, so protocol
29 and 30 fail regardless of this change. Verified by running the FIFO
case under fakeroot at protocol 29 with and without the parent pin --
it hangs identically either way, so the pinned name is not the cause.
That asymmetry is a pre-existing rrsync bug and is tracked separately.
The CVE-2026-53783 entry claimed rrsync "inode-pins each validated
component and exec's against the pinned fd". It pins the path and roots
the argument there, which is not the same thing for a sender argument,
and the primitive is Linux-only -- elsewhere rrsync keeps the
realpath()-validated name, as it always did.
The HAVE_PROC_SELF_FD probe checked that readlink of a DIRECTORY's entry
returned the right path, which is not evidence of an inode pin, and two
platforms fail that assumption in opposite directions:
* NetBSD makes the entry a symlink for directories only -- readlink of
a regular file's entry fails with EINVAL -- so the probe passed and
then every pull of a file died in the post-pin check with
"post-pin readlink failed (race?): f1 Invalid argument". This is
not new: the same failure reproduces on the branch base.
* Cygwin's readlink returns the right path, but opening the magic link
RE-RESOLVES it. Renaming a directory out from under a held fd lets
the magic link reach the replacement, so the pin protected nothing
while appearing to. rrsync-sender-leaf-flip caught this as a real
outside-content leak, not as flakiness.
Only Linux provides the inode-bound magic link this depends on, so
require that explicitly and keep the runtime probes as a guard for
Linux-like environments where /proc is absent or restricted. Elsewhere
rrsync falls through to the unpinned path, as it already did on the BSDs
and macOS. proc_self_fd_pins() mirrors the rule so the race tests skip
rather than report the intended gap, and Cygwin's workflow expects both
of them to skip -- rrsync-symlink only ran there because the old probe
wrongly reported support.
88cee089 rewrote every validated argument to /proc/self/fd/N so the
spawned rsync re-resolves it to the pinned inode. That is right for a
receiver, which open()s its destination, but a sender never opens its
source argument: send_file_list() lstat()s it first, and lstat of a
procfs magic link is always S_IFLNK. So the sender described the
argument as a symlink and sent no data -- silent data loss on
"rsync -a user@host:file dest/", the most ordinary command there is.
Of the argument shapes now covered, only a trailing-slash directory
survived.
Leonid Bugaev reported the regression, diagnosed the lstat-vs-magic-link
mechanism, and supplied the first regression test.
Which pin is usable depends on what rsync does with the argument:
* a trailing "/" or "/." directory is opened, not lstat()ed, and rsync
does follow a symlink there, so it keeps the leaf pin -- verified:
without it a flipped leaf transfers the outside directory's content;
* anything else pins one level up and passes the leaf by name. rsync
will not follow a symlink at that position (it sends the symlink
itself), -L/-k/--copy-unsafe-links are already disabled for a
restricted dir, and rsync's own leaf open is O_NOFOLLOW.
The directory pin resolves normally, including a symlink at its last
component, which is legitimate and which 3.4.4 accepts; the readlink
check afterwards is what proves the held inode is in-tree. It uses
O_PATH because reaching a known name beneath a directory needs only
search permission, and a mode 0111 parent is an ordinary way to publish
a file without letting it be listed.
Under --relative the transmitted name is the whole argument rather than
its basename, so the pin moves up to where that name starts and the rest
is spelled after a /./ marker. The client's own first marker wins if it
supplied one, including when nothing follows it; -R is parsed out of the
short-option cluster rather than sniffed for the letter, so the trailing
capability blob (-e.iLsfxC) cannot be mistaken for it.
Directory pins are keyed by (st_dev, st_ino), so a glob whose matches
share a parent inherits one descriptor rather than one per argument.
Every shape now delivers what a pristine 3.4.4 delivers, which is what
rrsync-pull-arg-shapes asserts -- it passes against 3.4.4 itself, so the
expectations are that behaviour and not this implementation's. The
"-R --no-implied-dirs" case is gated on protocol >= 30: at protocol 29
the receiver rejects it with "invalid path from sender" and transfers
nothing, which 3.4.4 does identically.
Moving the sender's pin off the leaf invalidates rrsync-symlink's oracle,
so it is reworked here rather than left failing. It patches rsync to a
stub that open()s its last argument, which is a faithful model for an
intermediate path component -- whatever rsync does with the final name,
it must not reach it through a flipped parent -- but not for the leaf: a
sender lstat()s its source and transmits a symlink there rather than
reading through it. So it now flips an intermediate directory, and the
leaf is covered against the real binary by rrsync-sender-leaf-flip, which
races both a plain file argument and a trailing-slash directory and
asserts no outside CONTENT is delivered rather than requiring a symptom
from a race that may not be won on a given run. Its trailing-slash half
is RED against a pristine 3.4.4 rrsync, which delivers outside/dir/loot.
rrsync-symlink is now sender-only. Measured over a 5s race, the stub
reached the outside marker 28 times in 98 runs as a sender and 25 in 97
as a receiver against 3.4.4; with the pin it is 0 as a sender but still
7-9 as a receiver, on the branch base as well as here. That residual is
the receiver's not-yet-existing-destination fallback, which predates this
work and is tracked in #139 rather than folded in.
Clearing FD_CLOEXEC goes through F_SETFD rather than os.set_inheritable(),
which prefers ioctl(FIONCLEX) and gets EBADF from an O_PATH descriptor on
older kernels -- every sender pull on Ubuntu 18.04 aborted with "Bad file
descriptor" the moment a directory pin was taken.
Co-authored-by: Leonid Bugaev <leonsbox@gmail.com>
A pull with a local --files-from does not send the list file to the
server: it sends the literal "--files-from=-" and streams the names down
the protocol connection. 88cee089 started inode-pinning every checked
option value, so rrsync tried to realpath() and open a file named "-" in
the restricted dir and killed the connection:
post-realpath open failed (race detected): - No such file or directory
Every --files-from pull through a restricted account was broken; 3.4.4
delivers the files. Exempt the exact string "-" only, so a list file
that really is a pathname is still validated and pinned -- which the new
test's control case checks, using the command shape rrsync actually
accepts so that it reaches the pathname check rather than dying earlier
at the syntax check.
auth_server() tokenised on commas AND whitespace, ignoring the documented
comma-only form, so an entry containing a space was torn in two: the rule
the administrator wrote never matched, and a rule they never wrote
appeared from its tail. For "@Group Name:deny" that means the deny is
skipped and a later :rw entry can match instead -- an authorization
bypass for a member of the denied group.
conf_strtok() already implements the documented behaviour and the
daemon's gid field already uses it (clientserver.c); this consumer was
missed when that one was fixed.
Reported by Andres Berbescu. Refs #137.
Four modules -- two defect cases and a control apiece -- driving the
defect in both directions.
"spaced" needs no groups at all:
auth users = ,@nosuchgroup authuser:deny, authuser:rw
* parsed as documented -- one entry naming a group that does not
exist, so no match, then "authuser:rw" grants access;
* split on whitespace -- "@nosuchgroup", then "authuser:deny", which
matches the username and refuses a transfer that should succeed.
"grpdeny" drives the direction that was actually reported, a member of a
denied group getting in:
auth users = ,@[! []*:deny, <realuser>:rw
"[! []*" is a wildmatch class holding a space, matching any ordinary
group name -- one with no "/" in it, which wildmatch treats as a path
separator -- whose first character is neither a space nor a "[". So the
entry contains a space without needing an NSS group named with one,
which a test cannot create. Parsed as documented the deny fires; split
on whitespace it becomes "@[!" and "[]*:deny", both unterminated classes
that match nothing, and the later ":rw" lets the member in.
The "[" is excluded from the class for the sake of that second half: the
simpler "[! ]*" splits to "]*", which matches any name beginning with
"]", so on a host with such a user the buggy parser would deny for the
wrong reason and look correct.
That half authenticates as the invoking user rather than the
secrets-file name, because the daemon must resolve the name to a real
uid or getallgroups() finds nothing and no group rule of any spelling
could match.
Each direction needs its own control, because "refused" is the expected
outcome of grpdeny and almost anything can produce a refusal. "plain"
proves the synthetic credential and the transfer work; "grpctl"
(auth users = @*:rw) proves the daemon resolved the real user to a uid,
enumerated its groups, accepted its secret and could store the file.
Without them, pointing grpdeny at a nonexistent module or breaking the
real user's secret both made it pass while proving nothing.
The deny itself is checked in the daemon log, not the client's output.
A client is told only "auth failed" whatever the server decided, so
"denied by rule", "no matching rule" and "password mismatch" are
indistinguishable to it -- and a parse yielding no rules at all produces
"no matching rule", which would satisfy any client-side check without
the deny having matched anything.
filter-leak plants a backup-dir symlink owned by another uid, so it
needs root and skips without it. Cygwin runs the suite as an ordinary
user, so it skips and the workflow says so.
Not AlmaLinux: that job is privileged, so the test runs there -- listing
it made the run fail with "expected-but-ran". Caught by the fleet, not
by inspection, which is the argument for running it before merging a
skip-list change.
Confining every parse_filter_file() open to the module root also caught
"filter", "include from" and "exclude from" from rsyncd.conf. Those name
operator-configured paths and pointing them outside the module -- at
/etc/rsync/excludes, say -- is the ordinary way to write them; rsyncd.conf(5)
puts no constraint on where the file lives. The result was not a refused
rule but a refused connection:
failed to open exclude file /etc/rsync/excludes:
Too many levels of symbolic links (40)
rsync error: error in file IO (code 11) at exclude.c(1582)
with no symlink involved anywhere -- just a regular file outside the module.
Mark the window in which the daemon loads its own parameters and skip the
confinement there. Everything else, in particular the peer-driven dir-merge
the leak test exercises, is still confined. Also fix the trailing whitespace
in the original hunk.
Follow-up to 6b885e51/51618b74, which I merged without updating the
per-workflow expected-skip lists, so every fleet run since has reported
a skip mismatch on eight targets.
Both skips are legitimate:
readonly-partial-abort-mode-regression exits 77 as root ("root
bypasses the read-only output-file precondition"), and the fleet runs
most targets as root -- so it only ever executes in a non-root run.
daemon-leaf-type-race-fchmod needs Darwin and --use-tcp, so outside
the macOS tcp cell it always skips.
Worth noting rather than burying: this means neither sec-regression test
runs in the fleet's default cells. The read-only one is exercised only
by a non-root local run, and the leaf-type one only by macOS over TCP.
The test only means anything when the scratch directory's group is one
the caller cannot grant, since that is what makes macOS refuse the
setgid bit. Taking that group from the build tree is fine for a
checkout under a shared parent but not for one under a home directory --
there the group is the user's own and the test skips silently. I only
got RED/GREEN out of it by chgrp'ing the scratch tree by hand.
So it falls back to /private/tmp, which is group wheel. On macOS as an
ordinary user it runs and passes.
It does NOT run in our macOS CI: that workflow drives the suite with
sudo, and root can grant every group, so the condition cannot exist --
hence the entry in the macOS expected-skip list alongside the others.
Making it run there needs a separate non-root invocation, not attempted
here. The skip message says which case it is instead of blaming the
scratch group.
The /private/tmp directory is outside SCRATCHDIR, which the harness
cleans, so it gets a mkdtemp() name and an atexit hook: a fixed name in
a sticky world-writable directory would let concurrent runs delete each
other's live fixture, and a leftover owned by another user would make
every later run skip. Verified on macOS that a run leaves nothing
behind.
macOS's fchmodat(..., AT_SYMLINK_NOFOLLOW) returns EPERM and applies
NOTHING when the requested setgid bit is ungrantable, so
"rsync -a --chmod=D2750,F0640" exits 23 and leaves the destination at
0755/0700 where 3.4.4 leaves 0750. fchmod() on an already-open
descriptor does the right thing: it succeeds, drops the setgid bit it
cannot grant, and applies the ordinary bits.
That fd path existed but was fenced behind "#if defined __linux__".
Guard it on O_NOFOLLOW so every platform that can open a leaf without
following a symlink uses it, and leave the fchmodat/fchmodat2 fallbacks
under __linux__. Measured on macOS with an ungrantable group:
fchmodat(2750, NOFOLLOW) EPERM, 0700 -> 0700 (dir and FIFO alike)
fchmod(fd, 2750) ok, 0700 -> 0750
fchmodat(0750, NOFOLLOW) ok, 0700 -> 0750
A FIFO observed by the lstat takes the pathname call instead, then one
retry without S_ISGID. Opening a FIFO -- even O_NONBLOCK -- makes this
process a reader for as long as the descriptor lives, which wakes a
writer blocked in open(O_WRONLY) and can cost it a SIGPIPE or the bytes
it writes before we close; the third line above is why that is
avoidable. This does not make the function FIFO-open-free: the type
comes from the lstat, so a leaf swapped to a FIFO after it is still
opened, and set_file_attrs() opens FIFOs elsewhere for ACL/xattr work.
Closing that needs the open constrained to the observed type, which is
tracked separately.
The retry is a pathname call and does not pin the inode, so a leaf
swapped for another object of the same name is chmod'd instead;
AT_SYMLINK_NOFOLLOW still keeps it off a symlink's target, and the held
parent fd still confines the ancestors, so the out-of-tree boundary is
unaffected. Only S_ISGID is retried -- clearing S_ISUID too could
discard a bit that was grantable when only setgid caused the failure.
Linux keeps the fd-first order it has always had.
The raced-to-a-symlink refusal after openat() also accepts EMLINK and
EFTYPE. ELOOP is not universal for O_NOFOLLOW on a symlink -- FreeBSD
documents EMLINK and NetBSD EFTYPE -- so those two silently skipped the
refusal and fell through to the (still symlink-safe) pathname call.
Verified on real macOS: the regression test FAILS on 93a67aa9 with rc=23
and modes 0755/0700, and PASSES here with 0750/0750.
The fleet run for this change reported it as an unexpected skip there.
Cygwin has no real FIFO for fake-super to represent, so the test skips
by design; every other target runs it.
From PR #90, whose code change is superseded by the preceding commit:
that helper already requires S_ISREG on the fd it chmods, and declines
the recovery entirely on the local/chrooted path #90 left untouched.
The test is kept because it is the only oracle for the directory-
substitution case.
Verified on macOS over TCP: FAIL on acfc94ef, PASS on the preceding
commit. Two limits worth knowing: it needs both Darwin and --use-tcp,
so it only runs in the macOS tcp cell, and its daemon sets
"use chroot = no", so it covers the fd-based branch only. It also
accepts a run in which the race window was never reached, so it can
report success without having exercised the check.
The EACCES recovery added by c1d7b5c6 chmods a read-only destination to
0600 so an --inplace update can proceed, and only restores the mode
after the transfer. Any abort in between -- peer EOF, checksum failure,
a signal -- leaves the file permanently owner-writable. 3.4.4 fails the
transfer and leaves 0444, so this is new exposure in 3.5.0.
Record the existing mode, add only owner-write, open the writable
descriptor, and put the old mode back before any network data is
consumed. The descriptor stays writable afterwards, so a complete
read-only --inplace update still works.
What this does NOT promise. Restoration is best effort, not a
guarantee: an unprivileged fchmod() silently drops S_ISGID when the
file's group is outside the process's groups, so 02444 can come back as
0444 with both calls reporting success, and a signal inside the
chmod/open/restore window still strands the relaxed mode. The window
goes from "the whole transfer" to a few syscalls, which is the point,
but it is not closed.
The helper also requires a regular file, which closes PR #90's finding:
O_NOFOLLOW refuses a symlink at the leaf but not a directory swapped in
after the type probe, and the recovery would otherwise fchmod that
directory from 0755 to 0600. On the fd-based branch that check is an
fstat() of the descriptor being chmod'd, so it is genuine. On the
local/chrooted branch it is only a type check on a stable path --
do_stat() follows a leaf symlink and every later call re-resolves the
name -- so that branch is confined by the chroot, not by this check.
Recovery is also skipped outright when the file is already
owner-writable, since adding S_IWUSR cannot be what such an EACCES is
about and each needless chmod risks a special bit.
Reworked from PR #102, which was written against a tree that already had
secure_recv_open() and deleted it: its helper resolved through
secure_relative_open(), dropping the one_inplace operator-path ownership
policy from the recovery window. (Its initial O_CREAT open toggled
operator_path_resolve by hand and kept the policy; the Linux
protected-regular retry and the recovery did not.) partial-protected-
regular-retry-linux catches that -- it passes on the base, fails with
the PR as submitted, and passes here. This version keeps the recovery
on secure_recv_open(..., one_inplace) and gates it on
"use_secure_symlinks || one_inplace" like every other open in the block.
Follow-ups from review of the lazy-resolution fix:
- Guard resolution against re-entry. If dlsym() ever reaches an
interposed function the nested wrapper would recurse; it now takes the
raw path instead.
- Forward a mode for O_TMPFILE as well as O_CREAT. rsync itself never
uses it, but the hook interposes every library in the process. The
test is an equality one because Linux defines O_TMPFILE as
__O_TMPFILE|O_DIRECTORY, so a plain & would also match O_DIRECTORY.
- Treat death by signal as a failure rather than a skip. The load marker
is written by the hook, so a crash before that point is
indistinguishable from the hook never loading -- which is precisely how
the AlmaLinux SIGSEGV stayed hidden.
Note the signal branch is not exercised by any current configuration:
with the raw openat(2) fallback in place the hook no longer crashes even
when resolution fails, which is why reconstructing the pre-fix behaviour
does not reproduce it.
The copy-links confinement gate null-checks module_dir but then passes
anchor to strcmp() without checking it, which the scan-build gate flags:
sender.c:291:7: warning: Null pointer passed to 1st parameter
expecting 'nonnull' [core.NonNullParamChecker]
Not reachable today -- the one caller passes module_dir -- but NULL is a
legitimate value for this parameter: secure_relative_open() reads it as
"relative to the cwd", which the else branch relies on. Only this branch
would dereference it.
RHEL-family LTS coverage in the fleet, matching the almalinux-8-build.yml
CI job that until now was the only place this family ran. Its container
and this VM do not agree on everything, so two box-specific skips are
recorded: no separate filesystem for a cross-device temp dir, and the
old static client the source-only push omits.
fs.protected_regular is enabled on the box (persisted in
/etc/sysctl.d/90-rsync-fleettest.conf) so protected-regular exercises the
real kernel behaviour here instead of skipping.
sender-remove-source-root-anchor runs and passes there -- the job is
privileged and / is writable -- so listing it as an expected skip made
the whole run fail on the mismatch. partial-protected-regular-retry-linux
is deliberately not added: with the hook fix it runs there too.
A preloaded open() interposes for the whole process the moment the loader
maps the library -- including calls made from OTHER shared objects'
constructors. The order constructors run between unrelated objects is
unspecified, so resolving real_open in our own constructor is a race we
do not always win.
On AlmaLinux 8 we lose it: OPENSSL_init_library() calls open() from its
constructor before ours runs, real_open is still NULL, and the process
dies in the loader:
#0 0x0000000000000000
#1 open () from hook.so
#2 OPENSSL_init_library () from libcrypto.so.1.1
#3 call_init ... dl-init.c
Every rsync run under the hook segfaulted, the load marker never
appeared, and the test reported "hook was not loaded" -- so a crash on a
supported platform surfaced only as a skip. Not a glibc-version thing:
ubuntu-1804 (glibc 2.27, older than AlmaLinux 8's 2.28) wins the race and
passes.
Resolve on demand at the top of each wrapper instead, with a raw
openat(2) fallback for the case where even dlsym() is unusable that
early. The test now runs, and passes, on AlmaLinux 8.
Its hand-rolled protocol client greets with version 30 and sends a
protocol-30 argument string. When the run pins the daemon lower the two
sides cannot agree and the client just sits there until it times out, so
the test failed on every check29 target rather than reporting anything
about copy-links.
The behaviour under test is not protocol-specific: the sibling
daemon-copylinks-parent-escape drives the same sender paths with the real
rsync client and passes at protocol 29, so skipping here loses no
coverage. Registered in the check29 expected-skip lists.
secure_walk_at() has two fast paths for the final component that call
openat(ds_cur(&ds), part, ...) directly instead of going through
ds_descend(). A final component of ".." therefore never met the anchor
floor, and what happened depended entirely on the caller's flags:
".." with O_DIRECTORY -> ELOOP (refused)
".." with O_DIRECTORY|O_NOFOLLOW -> fd for the directory ABOVE
the anchor
".." without O_DIRECTORY -> parent opened, then closed,
EISDIR returned
That was harmless while every literal ".." was rejected at the front
door, but secure_relative_open_at_beneath() now admits them and
documents the held-fd stack as refusing every climb above the anchor.
The sender's own call passes O_RDONLY|O_DIRECTORY and so was never
affected -- but the guarantee the new API advertises has to hold for
whatever flags the next caller picks.
Route a literal "." or ".." through ds_descend() before the leaf fast
paths. All three flag combinations now refuse a bare ".." with ELOOP,
and t_secure_relpath covers the matrix.
The shipped regression test covers the in-module target that the fix
enables. The fix is a loosening, though -- the resolver used to refuse
every literal ".." at the front door, which guarded the module boundary
by accident -- so the escape needs an end-to-end guard too, not just the
unit coverage in t_secure_relpath.
Cover a file and a directory symlink in each direction. Both types are
needed because they take different paths through the sender: the
directory one already resolved ".." via the dirstack walk while the file
one hit the front-door EINVAL, which is exactly the asymmetry reported on
this PR (a "../dir" symlink copied, a "../file" one was skipped).
The oracle is what landed on disk, not the exit status: a refused escape
legitimately makes rsync exit 23. The in-module assertions matter as
much as the leak ones -- without them "nothing leaked" would also be
satisfied by refusing everything, i.e. by the bug being fixed.
Two races made this test report a vacuous result -- neither FIFO opened,
so no traversal was attempted and there was nothing to confine -- on the
slower fleet VMs. It failed 9 runs in 12 on NetBSD.
Wait for each FIFO helper to reach its blocking open() before starting
the transfer, instead of assuming a freshly spawned process is already
there, and retry a run that comes back vacuous. A vacuous run is a setup
failure, not a security signal: an ESCAPE still fails immediately and is
never retried, so the oracle keeps its strength.
With the stale-object fix as well, NetBSD is 15 passes in 15.
build_patched_rsync() copies the configured tree, including its prebuilt
objects, then rewrites one source and runs make. copytree() preserves
mtimes, so on a target whose clock lags the host that pushed the tree the
copied sender.o is NEWER than the freshly patched sender.c: make reuses
it and the instrumentation never makes it into the binary. The test then
drives an unmodified peer and reports a vacuous result -- basis-xname-
traversal did exactly that on NetBSD, whose clock ran ~1h behind (gmake
warned "modification time in the future" during the build).
Drop the object for each patched unit, and the prebuilt binary too, so
neither the compile nor the link can be skipped. The function already
carried a comment about the same hazard on Cygwin's coarse mtimes.
claim_ports() fails loudly when a port is occupied, which is correct for
the 36 tests that bind the port themselves: they must not silently drift
away from the number they are about to bind. start_test_daemon() owns
both the bind and the URL it returns, so it can move instead -- and needs
to, because a fixed test port can be permanently held by unrelated
software on a shared CI box. An ASUS service was found sitting on 13010
on the Windows/Cygwin target, which no amount of orphan reaping frees, so
daemon-exclude-namebased failed there on every run.
Add claim_free_port(), which tries the preferred port and then a few
nearby ones, and use it at that single seam. _probe_bindable() grows a
non-fatal mode to support it; its default behaviour is unchanged.
The probe compiles the real authenticate.c with a hand-written include
list and ignores the CPPFLAGS configure recorded. Where a dependency
lives outside the default search path -- openssl from brew on macOS --
that fails at <openssl/sha.h>, for reasons unrelated to O_CLOEXEC, so
the test failed permanently on the macOS fleet target.
Take CPPFLAGS (and CC, when the environment does not override it) from
the configured Makefile so the probe matches the production build.
The absolute-source branch strips every leading slash and resolves the
parent beneath "/". On Cygwin clean_fname() deliberately preserves
exactly two leading slashes, because //server/share is a separate UNC
namespace -- so //server/share/f would be resolved as /server/share/f,
a different object, and the size/mtime guard would then be comparing the
wrong file before the unlink. That is the same wrong-target removal this
branch exists to prevent.
Decline the confined open for that shape (errno 0) so the caller falls
back to the path-based cleanup, as it did before. Exactly two slashes
matches clean_fname's own rule: three or more still collapse to one.
The sibling anchor test only covers the nested case, where re-anchoring
the cleanup at the sender's CWD merely fails with EINVAL. For a source
that is a direct child of / the parent component is empty, so the
cwd-backed cache handed back the sender's own working directory and
--remove-source-files unlinked a same-named entry there -- the real
consequence of the defect, and silent: the requested source survived and
the exit status was 0.
Needs root and a writable /, so it skips elsewhere; registered as an
expected skip on the non-root and sealed-root platforms.
The decoy's mtime is copied at nanosecond precision on purpose: the
sender's changed-file guard compares sub-second mtime too, and a
whole-second copy makes it skip the removal for an unrelated reason,
which would leave the test passing on a vulnerable build.
Context-aware quoting is only correct for one level of shell parsing. A
hook may re-parse the substituted word in a nested shell:
pre-xfer exec = sh -c 'printf %s %RSYNC_USER_NAME% >out'
The level-1 quotes are removed before the inner shell sees the value, so
an authenticated peer's username still reaches it as syntax however
carefully it was escaped. Escaping cannot fix this; refuse instead.
A %RSYNC_*% value substituted into a shell-executed hook (early exec,
name converter, pre-/post-xfer exec) is now rejected if it holds any
character that can become shell syntax in any context: quote, backtick,
dollar, backslash, semicolon, ampersand, pipe, redirection, parenthesis,
or a control character. Word-splitting and glob characters are left
alone -- they cannot execute anything and paths legitimately contain
them. The refusal is fail-closed and logged: a hook may be an access
check, so silently skipping it is not an option.
Also fix the quote tracker itself, which moved to SHELL_SINGLE_QUOTED on
an apostrophe even inside "...", where it is an ordinary character. That
made a value in `printf %s "it's %RSYNC_USER_NAME%"` escape for the wrong
context. With the refusal above this is defence in depth, and it matters
if the refused set is ever narrowed.
The two existing hook-injection tests asserted that a metacharacter value
was quoted and the transfer still succeeded; both now expect the refusal.
partial-protected-regular-retry-policy is Darwin-only and its new Linux
twin is Linux-only, so each skips on the other's platforms; the Linux one
also skips under check29, which cannot negotiate CF_INPLACE_PARTIAL_DIR.
None of that was in any RSYNC_EXPECT_SKIPPED list, which made every Linux
and Cygwin cell report a skip mismatch.
The existing partial-dir recovery test only runs under dyld interposing,
so it skips everywhere except Darwin -- and the fs.protected_regular
compatibility retry it is meant to cover sits inside "#ifdef linux",
which Darwin never compiles. That arm therefore had no coverage on any
platform.
Add a Linux twin driven by LD_PRELOAD: hook open/openat to model the
EACCES on the O_CREAT open of the existing partial leaf and swap the
partial dir for a symlink in the recovery window, and hook fstatat (with
an __fxstatat fallback for glibc < 2.33) to model the swap as foreign-
owned so the ownership walk refuses it. Between the two tests both
recovery arms are now covered.
The staging path needs one_inplace, i.e. the protocol-30
CF_INPLACE_PARTIAL_DIR capability, so skip below that rather than fail a
control the older protocol can never satisfy.
Order the assertions so the escape is reported before the ownership-walk
control: a vulnerable build runs no walk at all, and that must read as an
escape rather than an inconclusive result. Apply the same ordering to
the Darwin test, whose foreign-owner marker was built but never asserted.
Every other mutating do_*_at() wrapper (unlink, symlink, link, mknod,
rmdir, open, mkdir, rename) resolves an operator-supplied path through
owner_walk_parent() when operator_path_resolve is set. do_chmod_at()
and do_lchown_at() did not look at the flag at all, and both hand an
absolute name straight to the unconfined full-path do_chmod()/do_lchown().
set_file_attrs() is called with operator_path_resolve = 1 precisely so
that "a flipped temp-dir parent then can't redirect the chmod/chown"
(rsync.c). With no held dirfd -- an absolute --temp-dir or
--partial-dir -- both fell back to these two wrappers, so that promise
did not hold. Give them the same ownership-walk branch the others use.
S_ISLNK(mode) still takes do_chmod()'s lchmod()/setattrlist() path.
The missing branch was spotted by Omar Elsayed in review on the
partial-dir EACCES recovery PR, together with the fix approach.
Suggested-by: Omar Elsayed <omarelsayed161@gmail.com>
secure_remove_source_file() called unlinkat() directly, dropping the
dry_run no-op and the read-only/list-only refusal that do_unlink()
applies on the non-fd path. That is what let --only-write-batch (which
implies dry_run) really delete the source files once a MSG_SUCCESS
reached the sender. Use do_unlink_atfd(), which carries both guards.
When neither AT_FDCWD nor AT_SYMLINK_NOFOLLOW is available, the function body is
a no-op warning that never reads mode or dfd, so the leading 'mode &= CHMOD_BITS'
became a dead store and dfd an unused parameter -- which the pinned clang-18
scan-build gate flags (deadcode.DeadStores). Move the mask inside the
AT_SYMLINK_NOFOLLOW guard where mode is actually used, and mark dfd/mode used in
the fallback. No behavior change on any platform that has the symlink-safe
primitive.
malicious-dot-dir-delete-scope and peer-legacy-implied-delete-scope both need a
real TCP socket (require_tcp), so they skip on the pipe and protocol check
passes. Add them to RSYNC_EXPECT_SKIPPED for the check/check30/check29 steps so
the CI skip-set matches. (The squash-merge of the dot-content-scope fix dropped
this registration.)
daemon-dot-file-force-wipe and malicious-dot-file-delete-scope both need a real
TCP socket (require_tcp), so they skip on the pipe and protocol check passes.
Add them to RSYNC_EXPECT_SKIPPED for the check/check30/check29 steps so the
fleet skip-set matches.
The new test needs a real TCP socket (require_tcp), so it skips on the pipe and
protocol check passes. Add it to RSYNC_EXPECT_SKIPPED for the check/check30/
check29 steps so the fleet skip-set matches.
The daemon rejection for a peer-selected FNAMECMP_PARTIAL_DIR basis that the
confined open declined fired at every protocol. In-place partial updates are
only negotiated at protocol 30+ (CF_INPLACE_PARTIAL_DIR); at protocol 29 no
partial-basis redirect is possible, and the receiver already handled such a
transfer safely by completing it with no basis. Gate the abort on
inplace_partial so a pre-30 daemon falls back to the safe no-basis path instead
of aborting a legitimate transfer with a protocol error.
Fixes operator-path-partial-dir-daemon at protocol 29.
Differential test for the daemon files-from/backup-symlink out-of-module read.
It races a --backup-dir push against a parent-swap flipper until a root-owned
backup symlink to an out-of-module secret lands in the backup tree, then tries
--files-from=:backup/sub/<name> and fails if the secret's content is read back
as the file list. RED before the module-root confinement, GREEN after.
Requires root plus an untrusted uid to plant the cross-uid symlink; skips
otherwise. Registered in the Cygwin expected-skip list.
Based on a report and proof-of-concept test by seks99x.
A daemon serving a writable, non-chrooted module reads a client-requested
--files-from=:LIST through open_no_attacker_symlinks(), which follows a
symlink owned by uid 0 or the euid. The module-root confinement in that
resolver (abspath_excluded_by_module) only fires when operator_path_resolve
is set, and this open left it clear -- so a trusted-owned symlink whose
target escapes the module was followed.
An attacker can obtain such a symlink without owning it: a --backup-dir push
makes the daemon back up the old destination symlink with the daemon's own
(root) ownership, and a parent-swap race can leave that root-owned backup
symlink pointing outside the module. A later --files-from=:backup/... then
reads out-of-module file content as the file list, bypassing the same-uid
ownership constraint that normally protects files-from.
Set operator_path_resolve around the files-from open so the ownership walk
also refuses a trusted-owned symlink that redirects the list outside the
module root. A daemon has no rsyncd.conf "files from" of its own, so this
path is always client-requested and confining it is unconditional. No-op off
a daemon (the module-root check only fires when am_daemon).
The same ownership-walk opener backs the daemon merge/--exclude-from reads in
exclude.c, but those also load the module's own "include from"/"exclude from"
admin files, which on a non-chrooted module may legitimately live outside the
module; confining them there needs a client-vs-admin distinction and is left
to a separate change.
The basis-xname-traversal test builds an instrumented sender via
build_patched_rsync(), which skips on Cygwin (coarse NTFS mtimes leave the
patched unit unbuilt, and forcing the rebuild trips -fno-common relinks). Add
it to the Cygwin RSYNC_EXPECT_SKIPPED set so its clean skip there is expected
rather than a skip-mismatch.
read_ndx_and_attrs() sanitized the wire-supplied xname (the alternate-basis
leaf name sent with ITEM_XNAME_FOLLOWS) only when sanitize_paths was set,
which is the daemon side. A client receiver has sanitize_paths == 0, so a
malicious server could send an xname containing ".." and, joined to an
operator basedir (--link-dest / --compare-dest / --copy-dest, or the fuzzy
dir), have the client open an out-of-tree file as the delta basis -- a
client-side arbitrary-read / file-existence-oracle / FIFO-hang. The ownership
walk in secure_basis_open() does not stop this: it deliberately follows a
plain ".." to a regular file (the legitimate --link-dest=../01 sibling, #915)
and only refuses foreign-owned symlink components.
Sanitize xname unconditionally. The operator basedir may legitimately be
relative, but the leaf name that arrives over the wire never legitimately
needs ".." or a leading "/".
Reported by z3r0s.
Builds a malicious daemon-sender (env-gated xname injection patched into
sender.c) and pulls with --link-dest through the production receiver. A FIFO
one level above the link-dest dir, plus a helper blocked in open(O_WRONLY),
detects whether the receiver opened the traversed "../secret" basis. RED on
an unsanitized-xname receiver, GREEN once xname is sanitized.
Builds the tree under test with -fwrapv (build_patched_rsync gains an
append_cflags option, which drops the copied tree's prebuilt objects so the flag
is actually applied on the full rebuild) to make the signed overflow
deterministic, then drives set_io_timeout(INT_MAX) via --timeout and asserts the
copy completes instead of spinning in the tight select()-EINVAL loop. RED on the
unfixed computation, GREEN once the arithmetic is overflow-safe.
io_timeout can reach INT_MAX -- from an operator --timeout (options.c parses it
as a plain int, unbounded and even negative) or a peer's MSG_IO_TIMEOUT (now
also capped at 86400 in read_a_msg). Several signed computations then misbehave:
* allowed_lull = (io_timeout + 1) / 2 overflows to a negative allowed_lull /
select_timeout; select() then returns EINVAL on the negative tv_sec, which
isn't EBADF, so the read loop spins at 100% CPU forever (io_timeout ~= 68
years never fires check_timeout), plus a keepalive flood. Compute
ceil(io_timeout/2) in a wider type so "+ 1" cannot overflow.
* the generator and sender derive an int loop-check limit as allowed_lull * 5
(generator.c, sender.c), which overflows for a large allowed_lull. Cap
allowed_lull so that product stays in range -- invisible to real use, as
allowed_lull is the keep-alive half-interval and INT_MAX/5 seconds is over
13 years.
* a negative --timeout drove allowed_lull / select_timeout negative the same
way; treat secs < 0 as "no timeout" up front.
The overflows are undefined behaviour, so plain -O2 gcc/clang happen to keep
select_timeout at 60, but -fwrapv / -fno-strict-overflow (common hardening) wrap
to the spin and -ftrapv aborts.
Reported by z3r0s.
A malicious server can send MSG_IO_TIMEOUT with val near INT_MAX
(0x7FFFFFFF). The existing val <= 0 guard prevents timeout disabling,
but a large positive value passes through to set_io_timeout() where
(io_timeout + 1) / 2 overflows signed int, wrapping allowed_lull and
select_timeout negative. Every subsequent select() returns EINVAL
immediately, trapping the client in a tight CPU loop.
Cap the accepted timeout at 86400 seconds (24 hours), which is well
above any practical timeout and avoids the overflow in set_io_timeout().
Reported-by: z3r0s <https://github.com/z3r0s6>
Document the security and bug fixes integrated after the initial 3.5.0
security-fix set: peer io_error masking (io.c + flist trailer), log-file
control-character escaping (CWE-117), the --safe-links/--backup hard-link
bypass, the operator-path backup leaf sinks (do_symlink_at/do_rmdir_at), the
hash_search() chain bound (issue #217), and the clean_fname/robust_rename
resolver hardenings; plus the %%, .cvsignore "!", --chmod=a+s and
bracket-expression case-fold bug fixes.
NEWS.md gains the new SECURITY RELATED items and a BUG FIXES section;
SECURITY.md notes the peer error-flag masking, log-injection escaping and
hash_search bound in the malicious-peer section.
The outside-module victim-intact oracle holds at every protocol and
stays unconditional. The stronger 'daemon actively rejects the forced
--partial-dir operand' behavior is protocol-30+ only: at protocol 29 the
operand is handled differently and the transfer completes normally with
the victim still untouched (verified). So gate that check on protocol
30+, and at protocol 29 require the known-good normal outcome (rc==0,
dest replaced) rather than skipping -- keeping the branch non-vacuous.
Surfaced by the fleet's proto29 pass.
%C only renders a hex digest for a canonical checksum. At protocol < 30
the negotiated file checksum is a non-canonical MD4 variant, so %C (via
sum_as_hex) renders empty -- there is no digest to compare and no F_SUM
read to over-run. Gate the over-wide %C sub-case on protocol 30+, where
checksum_for() still fails on a missing digest so a real regression is
caught. The %% literal checks are protocol-independent and still run.
Surfaced by the fleet proto29 pass on ubuntu-2204/2404/2604.
Both new tests skip unless run as root; the Cygwin CI job runs non-root
and enforces an exact RSYNC_EXPECT_SKIPPED set, so an unregistered skip
fails the suite. Add them to the list (they run and pass on the
root/sudo Linux, BSD, Solaris and macOS targets).
do_symlink_at() and do_rmdir_at() were the only operator-path syscall
wrappers still missing the operator_path_resolve branch that
do_mknod_at()/do_open_at()/do_rename_at()/do_unlink_at() already carry:
an absolute operator path (e.g. an absolute --backup-dir) took the
'*path == "/"' arm straight to the bare do_symlink()/rmdir(), whose
libc path resolution follows a parent-component symlink. A local
attacker who owns a parent component of the backup tree could thus
redirect a backup symlink creation or a backup-dir rmdir outside the
intended tree.
Route both through owner_walk_parent() + symlinkat()/unlinkat() when
operator_path_resolve is set, mirroring the existing wrappers, so a
foreign-owned parent component is refused while the operator's own is
followed. symlink_optout_allowed() (--insecure-links / 'insecure
links =') restores the legacy following.
Tests: operator-path-backup-symlink and operator-path-backup-rmdir
drive a live parent-component swap (native C flipper) against a local
--backup-dir push and confirm nothing escapes the backup tree (RED
before, GREEN after).
Co-authored-by: Andrew Tridgell <andrew@tridgell.net>
The new source-change-size-continues test skips on non-Linux platforms
(it needs LD_PRELOAD + /proc/self/fd). macOS and Cygwin are the two
non-Linux CI jobs that enforce RSYNC_EXPECT_SKIPPED, so an unregistered
skip there fails the suite (runtests.py treats an unexpected skip as a
mismatch). Add the test to both lists. The Linux enforcing jobs
(ubuntu, ubuntu-22.04, almalinux-8) run it and are unaffected; the
BSD/Solaris jobs do not enforce the skip set. The mac2/cygwin fleet
targets read these same lists, so the fleet is covered too.
The header claimed protocol-level crafting of MSG_IO_ERROR is covered by
msg-io-* crafted-server tests; no such test exists (msg-io-timeout-zero
crafts MSG_IO_TIMEOUT). Say what the test actually locks down.
log_formatted() renders %% as a literal '%', but log_format_has() still
rescanned the literal '%' as the start of a new escape, so a format such
as --out-format='%%i' misdetected the 'i' and turned on itemizing (and
'%%b'/'%%c'/'%%C' likewise perturbed log_before_transfer and checksum
retention). Skip the literal so both parsers agree.
Extends the ki58 test: an attribute-only change must not be logged under
--out-format='%%i %n', while a transferred file still renders the
literal '%i'.
The --safe-links guard on the backup hard-link fast path only skipped the
backup when do_readlink() succeeded (llen > 0) and the target escaped. A
failed readlink (e.g. the link vanished between the lstat and the
readlink) fell through to link_or_rename(), which could hard-link the
symlink into the backup area unchecked -- the same bypass the guard was
added to close.
Fail closed: skip the backup when the target can't be read.
Extend the KI-72 test with a safe (in-tree) symlink case to confirm the
guard doesn't over-block and drop legitimate safe symlinks.
recv_file_list() OR's the wire-supplied end-of-list error value straight
into the local io_error at three sites (the varint-flags path, the
XMIT_IO_ERROR_ENDLIST byte path, and the protocol < 30 int flag). Like
the MSG_IO_ERROR path fixed in io.c, a malicious peer could set arbitrary
undefined bits, which then accumulate in io_error and can be re-forwarded
to other peers.
Mask each with IOERR_VALID_MASK so only the defined IOERR_* bits survive,
matching the io.c MSG_IO_ERROR handler.
ki62-io-error-mask needs --use-tcp (it SIGKILLs the daemon-side sender), so it
skips under the default pipe-mode make check. Register it alongside the other
tcp-only daemon test (daemon-argv-limit) in every workflow that pins
RSYNC_EXPECT_SKIPPED, so its pipe-mode skip is expected rather than flagged.
Two of the integrated MC/DC-audit tests were environment-fragile:
- ki58: asserted the exact string '100% done percentfile', but %f expands to the
transfer-relative path (with leading dirs) when the source is an absolute path,
so the basename assumption failed. Assert the literal-percent escape '100% done '
plus the file name instead (still RED on a broken %%: it emits '100%% done').
- ki62: killed daemon.kill() -- the listener -- but rsyncd forks a child per
connection, so the child sender kept streaming and the receiver hung. Parse the
transfer child's pid from the daemon log ('[pid] rsync on <mod>/') and kill that.
killpg is not usable: the test daemon shares this test's process group.
Fixes are unchanged; only the test drivers.
A peer-supplied MSG_IO_ERROR value was OR'd into the local io_error
without masking, allowing a malicious peer to set arbitrary bits that
propagate to exit codes and get re-forwarded to other peers.
Fix: mask the incoming value with IOERR_VALID_MASK (IOERR_GENERAL |
IOERR_VANISHED | IOERR_DEL_LIMIT) before OR'ing.
Test: testsuite/ki62-io-error-mask_test.py
The log_formatted() switch had no case for '%', so %% did not
produce a literal percent character. Add case '%' to output
a single '%' character, matching the printf convention.
Test: testsuite/ki58-log-format-percent_test.py
The CLEAR_LIST guard in parse_rule_tok checked rule->rflags for
FILTRULE_NO_PREFIXES, but NO_PREFIXES is a template-level flag
excluded from FILTRULES_FROM_CONTAINER inheritance. The guard
was always true for CVS rules, causing RERR_SYNTAX abort instead
of clearing the list.
Fix: check template->rflags instead of rule->rflags.
Regression test: testsuite/ki73-cvs-clear-list_test.py
When CAN_HARDLINK_SYMLINK is defined (Linux, macOS), the backup
hardlink fast path at link_or_rename() succeeded for symlinks and
'goto success' skipped the safe_symlinks check. An escaping symlink
(pointing outside the transfer tree) was silently preserved in the
backup area despite --safe-links.
Fix: check safe_symlinks BEFORE the hardlink path. If the symlink
target escapes, skip the backup (same as non-hardlink-symlink systems).
Regression test: testsuite/ki72-safe-links-backup_test.py
The vmactions OpenBSD VM has the same kernel bugs as the fleet's
OpenBSD box: a connect()-under-rename-load lost-wakeup and an FFS
rename-storm corruption that hang the symlink-race flipper tests to
the 300s timeout for non-rsync reasons (sender-remove-source-secure
just did so in the --use-tcp pass). Exclude the same three tests the
fleet config already excludes there; the protections they exercise
are verified on the Linux and other BSD targets.
Corrections to the preceding man page rewrite, found in review:
- --max-alloc: 0 is rejected as invalid since 3.5.0 (CVE-2026-53794),
it does not mean "no limit"
- -t: restore "next transfer" -- it is the subsequent run that behaves
like --ignore-times when times aren't preserved; also note that the
timestamp-range limit is protocol < 30, not just an old remote
- --max-delete: re-add the caution that a pre-3.0.0 CLIENT treats
--max-delete=0 as unlimited (a new client forwards it as -1, so the
client version is the boundary that matters)
- --stats: the deleted-files line needs negotiated protocol >= 31, not
just a new remote rsync
- --version: the server does not ignore it; only the repeated-option
JSON output is client-side only
- SECURITY: restore the sentence distinguishing the pre-transfer -c
checksum from the whole-file transfer-verification checksum (which
runs unless --checksum-choice=none); fix a singular/plural clash in
the escape-path paragraph
- remove a leftover XXX comment marker (the paragraph it questioned is
correct: user@ sets the rsync module user, ssh -l sets the login)
- typo/mechanical: "If it is sufficient", "is output in a JSON",
missing space before --read-batch, unbalanced paren after
rsyncd.conf(5), missing "the" before --whole-file, trailing
whitespace
Updates to make statements more definite and explicit and improve
English expression, including:
- Globally change dir to directory, arg to argument, parens to
parentheses.
- Remove some references to behaviour of past versions, on the grounds
that the man page should describe the current version, and only
describe past behaviour when talking about how to get a current rsync
client to deal with an older remote rsync version.
- In most cases where a version number is mentioned, add the month
and year of release of that version.
- Be more explicit about when "file" means "regular file" vs. "any
filesystem object".
- Remove some parentheses, where the parenthesized comment is important
enough to warrant making it a first-class part of the sentence.
Both callers pass non-null from/to, but the 'to &&' test taught the
clang analyzer that to may be null, and it then walked a null to
through copy_file -> unlink_and_reopen -> robust_unlink into glibc's
nonnull-annotated strlcpy, failing the scan-build gate. The args are
required non-null, so test the first byte directly.
T_SAFE_ARG_OBJ used $(filter-out main.o,$(OBJS)), a GNU-make-only
function that BSD make and Solaris make silently expand to nothing,
so t_safe_arg linked with no rsync objects at all and failed with
undefined symbols (am_daemon, io_timeout, send_msg_int, ...). The
fleet missed this because those VMs build with gmake; the FreeBSD,
OpenBSD and Solaris CI VMs use the native make.
Spell out the object list via a new OBJS1_NO_MAIN macro instead;
plain macro expansion works in every make.
Now that pull_request triggers fire for any base branch, limit runner
minutes by skipping PR jobs unless the PR carries the 'run-ci' label.
Applying a label needs triage access, so a fork PR can't enable the
matrix by itself. The 'labeled' trigger type is added so applying the
label starts a run immediately; skipped jobs cost no runner minutes.
Push, schedule and manual dispatch runs are unaffected.
The pull_request triggers were filtered to base branch master, so PRs
targeting staging branches (e.g. pr-rsync350-sec-fixes) got no CI at
all. A PR is a deliberate request for review, so run the checks on
every PR; the push trigger keeps its master filter to avoid running
the whole matrix on every WIP branch push.
Adds a reusable harness (mutatefns.py) that mutates a source resource in the
window after the file list is built but before that resource is sent -- the
same window the growing-file regression exposed -- plus tests for a source
file that shrinks or vanishes mid-transfer. Both are handled gracefully
today (no protocol abort, later files still transfer); these lock that in.
Note: metadata (perms/mtime/xattr/ACL) is captured at flist-scan time and
applied from the flist, so a data-phase change is a no-op -- content is the
only resource re-read at send time, so it is the meaningful surface here.
The sender records each file's length when it scans the file list, then
re-stats and maps the file at its current size when it later sends the
data (sender.c: do_fstat + map_file + match_sums use st.st_size). A file
appended to in that window -- a live log written during a nightly backup
is the common case -- is transmitted longer than its flist-recorded
length.
A hardening check added in d33e599c (offset + i > total_size and
offset + len > total_size in receive_data) turned that benign, common
condition into a fatal "received more data than file length"
RERR_PROTOCOL, tearing down the whole connection so every file after the
growing one is skipped. Stock rsync has no such check: the receiver
writes to a temp file, so the extra bytes just extend it, and the
whole-file checksum-verify already contains a malicious sender -- the
bound bought little while breaking a routine case.
Remove both checks to restore the stock behavior.
Regression test: testsuite/growing-file_test.py (RED before, GREEN after).
The KI-54 helper linked options.o compiled with -ffunction-sections and relied on
-Wl,--gc-sections to drop the option parser. That is GNU-ld-only: the fleet
showed macOS (ld64; also its custom rule dropped the openssl include path) and the
cygwin PE linker leave parse_arguments and its deps undefined, failing the build
of every CHECK_PROGS target on those hosts. Instead link the real rsync objects
(the same set as the rsync binary, minus main.o -- supplied renamed via
t_safe_arg_main.o) so safe_arg's deps all resolve with a plain link on every
platform. t_safe_arg_main.o depends on $(HEADERS) so the generated proto.h is
built before it under a parallel make. No behaviour change to the test itself.
Reported-by: Leonid Bugaev
dowild()'s literal path folds the pattern char, but the [class]/[a-z] path
compared unfolded pattern bytes against the (folded) text, so iwildmatch() was
still case-asymmetric for character classes and ranges -- e.g. a daemon
'hosts deny = [A-Z]*.EVIL.COM' failed to match a lower-case host (access-control
fail-open, same class as the literal case). Fold p_ch for the escaped-member,
range-endpoint and plain-member comparisons; the range start (prev_ch) picks up
the folded value too. Only active under force_lower_case (iwildmatch), so
case-sensitive wildmatch() is unchanged.
Reported-by: Leonid Bugaev
The KI-53 fix folds only the literal-match path, so a bracket-expression
pattern ([A-Z], [ABC], [\A]) still compares unfolded pattern bytes against the
folded text -- an upper-case character class / range fails to match a lower-case
host, the same access-control fail-open class as the literal case. Add bracket
cases (plain, range, mid-pattern, and escaped member; RED until the class/range
path folds too).
Reported-by: Leonid Bugaev
After the backward walk, s points at the first char of the prior component and
s[-1] is its leading '/', so the boundary test must read s[-1] (not *s) and t
must reset to s (not s+1). The old off-by-one left CFN_COLLAPSE_DOT_DOT_DIRS
dead for all multi-component and absolute paths. The peer-traversal guard
CFN_REFUSE_DOT_DOT_DIRS is checked first and is unaffected, so this is a
normalization-correctness fix, not a traversal hole.
Reported-by: Leonid Bugaev
t_clean_fname links the real util1.o and checks that clean_fname() with
CFN_COLLAPSE_DOT_DOT_DIRS collapses ".." for multi-component and absolute
paths. RED before the fix: an off-by-one left the collapse dead for all such
paths.
Reported-by: Leonid Bugaev
The cross-filesystem fallback copied to the dest and unlinked the source without
operator-path confinement, so an absolute --temp-dir/--partial-dir on another
filesystem was opened/unlinked via plain libc -- a raced parent symlink could
redirect the dest-write or source-unlink out of the module. The source READ was
already confined; flip operator_path_resolve for an absolute (operator) path
around the copy_file dest-open and the do_unlink_at, leaving relative in-module
paths on the secure_relative_open arm.
This is the EXDEV-fallback backstop for the same absolute --partial-dir /
foreign-owned-parent-symlink escape that operator-path-partial-dir_test.py
already exercises at the handle_partial_dir() layer (which confines the staging
dir before this code runs). A dedicated test for the EXDEV copy_file path itself
would need the main tmp->final rename to fail first and then hit EXDEV on a
foreign-owned raced parent -- a nested, timing-dependent trigger.
Reported-by: Leonid Bugaev
logit() wrote the message to the log file with a raw fprintf, so an
attacker-controlled filename could inject terminal escapes that an admin later
executes when cat'ing the log (CWE-117). Route it through filtered_fwrite(),
keeping the trailing newline raw. Also escape C1 controls (0x80-0x9f, incl CSI)
on filtered_fwrite's use_isprint=0 path, which previously caught only C0.
Reported-by: Leonid Bugaev
The C1 escaping is gated by an escape_c1 flag set only for the log path, so
--8-bit-output / iconv terminal output still passes 8-bit bytes (incl. UTF-8)
through unchanged.
Transfers files whose names carry C0 (0x1b) and C1 (0x9b) control bytes via
--log-file and checks the log contains no raw control bytes (only \#NNN
escapes). RED before the fix: logit() writes the raw filename to the log
(CWE-117), and C1 controls slip through filtered_fwrite. Skips if the fs
rejects control-char names.
Reported-by: Leonid Bugaev
parse_chmod()'s 'a' clause set the "where" bits but not topbits, so "a+s" fell
through to setuid only and dropped setgid (chmod(1) sets both). Add
S_ISUID|S_ISGID to topbits for 'a'.
Reported-by: Leonid Bugaev
Transfers with --chmod=a+s / u+s / g+s (needs -p to apply set-id bits) and
checks the resulting mode. RED before the fix: a+s sets setuid only and
drops setgid.
Reported-by: Leonid Bugaev
The escape counter reserved a slot for every backslash, but the writer
suppresses the escaping backslash before a wildcard and -- via the
strchr(WILD_CHARS, '\0') footgun -- on a trailing backslash, so counter and
writer disagreed and left an uninitialized heap byte in the returned string that
is handed to the legacy remote shell (protect_args=0). Make the counter mirror
the writer and guard the strchr with f[1] (which also correctly doubles a
trailing backslash).
Reported-by: Leonid Bugaev
t_safe_arg links the real options.o (via --gc-sections so only safe_arg is
pulled in), poisons the heap, and checks that filename-mode quoting is exact.
RED before the fix: the counter/writer backslash miscount leaks an
uninitialized heap byte into the returned string.
Reported-by: Leonid Bugaev
iwildmatch() folded only the text to lower case, not the pattern, so it was
asymmetric rather than case-insensitive. An upper-case "hosts deny" token
(e.g. *.BADDOMAIN.COM) then failed to match a lower-case peer name and the
blocked host was admitted -- an access-control fail-open. Fold the pattern
char too under force_lower_case.
Reported-by: Leonid Bugaev
t_iwildmatch links the real lib/wildmatch.o and checks that iwildmatch()
folds case on BOTH the pattern and the text. RED before the fix: an
upper-case pattern token (a daemon 'hosts deny = *.BADDOMAIN.COM') fails to
match a lower-case host -> access-control fail-open.
Reported-by: Leonid Bugaev
patched_rrsync() rewrote support/rrsync's hardcoded RSYNC path to a test stub
with a value-specific str.replace("RSYNC = '/usr/bin/rsync'", ...). Python's
str.replace silently returns the text unchanged when the needle is absent, so on
a tree whose shipped rrsync uses a different path -- e.g. FreeBSD's net/rsync
port patches it to /usr/local/bin/rsync -- the rewrite no-oped and
rrsync-under-test kept exec'ing the real system rsync, which blocks in server
mode on stdin and hangs the rrsync tests (a testsuite timeout on the BSDs).
Match the RSYNC assignment line itself (re.subn on ^RSYNC\s*=.*$) rather than a
specific value, using a callable replacement so the path is inserted verbatim,
and fail loudly via test_fail if it is not found exactly once. Centralize
rrsync-symlink_test.py's duplicate inline replace onto the same helper.
Reported-by: Rodrigo Osorio <rodrigo@FreeBSD.org>
Solaris has facl(2), which performs ACL operations on an already-open file
descriptor. This adds Solaris fd-based helpers for the ACL operations that
rsync needs while keeping the existing path-based fallback for callers without
a held fd.
On Solaris, setting an ACL on a directory replaces the combined access and
default ACL set. The path-based sys_acl_set_file() already handled this by
reading the other half of the directory ACL, merging access and default entries,
and then calling acl(..., SETACL, ...). The new sys_acl_set_fd_type() preserves
that behavior with fd-based operations: it uses fstat() to identify directories,
reads the other ACL half with sys_acl_get_fd_type(), combines the access and
default entries, marks default entries with ACL_DEFAULT, and finally writes the
combined ACL with facl(2). Deleting a default ACL similarly fetches the access
ACL through the fd and rewrites only that access ACL with facl(2), which removes
the default ACL without re-resolving the path.
Driving the apply off the held fd also closes a symlink-race on the Solaris ACL
apply (the path-based sys_acl_*file re-resolves the path -- the CVE-2026-53799
class, unfixed on Solaris until now). Added on integration: for a root receiver
a missing held fd on a confined receiver means the leaf was raced to a symlink
(acl_set_file follows it), so refuse the path-based set/delete rather than apply
the ACL to a redirected inode; a plain non-root receiver keeps the path-based
fallback for a legitimately un-pinnable owned leaf (e.g. a 0300 dir), matching
the operator-path op_pin rule.
Don't let close() clobber the errno from a failed write, and report a close()
failure on an otherwise-successful write.
(The short-write length, the unsigned bufpos = -1 returning success, and the
size == 0 returning failure that Vladimir also reported were already fixed
in-tree.)
Use a level-aware compressor for the compress-options --compress-level check.
The default compressor can be lz4, which has no tunable compression levels and
therefore reports level 0. Select zlibx or zlib for this subtest so it verifies
that --compress-level is passed to a compressor that supports levels.
Read the build config when setting up the symlink-placeholder test. The
t_symlink_secure helper is compiled against the build directory config.h; use
that same config for the Python test setup so it creates the symlink placeholder
PoC inputs when the compiled helper will exercise them.
The da7c4208 oracle-exec guard only caught OSError (can't-exec / ENOEXEC, on the
BSDs and macOS). Solaris execs the Linux x86-64 old_versions/rsync_3.2.7 binary
without ENOEXEC but it SIGSEGVs, so subprocess.run('--version') returned -11
without raising and ORACLE_BIN stayed set -- the oracle daemon launch then died
("rsyncd exited before listening on port 12910, status=-11") and failed the test.
Require the probe to exit cleanly (returncode == 0); a non-zero/signal exit, a
hang (TimeoutExpired), or a can't-exec (OSError) all degrade to the static
contract.
The test relied on a held-dirfd cache MISS coinciding with the parent flip, so it
only caught the escape occasionally (it read as -j-load flakiness). Bury the
flipped dir below the held-dirfd cache depth (DEEP, 70 levels > VFS_DPC_MAXDEPTH
64) so set_file_attrs() takes the fd-less metadata path for every file on every
push -- removing the cache-miss race and leaving only the parent flip landing in
the create->lsetxattr window. Widen that window (64 KB payload) and raise the
file count (N=120) for more attempts per push.
Now reliably RED on the unfixed code (8/8 under load, the CI condition; ~5/6
standalone) and reliably GREEN with the set_file_attrs re-pin/refuse fix
(no false positives, standalone and under load). Same inline 3-rename flipper
(the reset-each-push model the EXCHANGE c-flipper can't drive).
set_file_attrs() pins the entry via the cached held dir fd (held_dfd_for) and
drives the xattr/ACL ops off that fd (fsetxattr). But when the pin missed --
held_dfd_for() returns -1 (the path is deeper than the dirfd cache, or its dir
isn't the held one), or the leaf openat() loses a race -- held_fd stayed -1 and
set_stat_xattr()/set_xattr()/get_acl_fdat()/set_acl_fdat() fell through to the
path-based branch (sys_lsetxattr(fname,...)). Unlike the chmod/chown/times path
wrappers (which secure-resolve), that raw lsetxattr re-resolves the parent, so a
concurrent flip of a dest parent component to a symlink->outside lands the xattr
OUTSIDE the destination tree (the intermittent copy-xattrs-symlink-race escape
that surfaces under -j load, which widens the open->setxattr window).
Re-pin through secure_relative_open() when the cached pin misses on a confined,
non-operator receiver path, so the xattr/ACL ops always use a confined fd -- NOT
a raw path lsetxattr; if the re-pin also fails (a genuinely raced parent/leaf
symlink) skip the path-based ops (xattr_refuse) rather than redirecting them.
The re-pin passes O_DIRECTORY for a directory leaf. Apply the same re-pin/refuse
to gen_entry_copy_xattrs() (the dir xattr copy), whose dfd<0 path likewise fell
to copy_xattrs() with dest_fd==-1. chmod/chown/times are unchanged (confined via
their *at wrappers); operator paths keep op_pin/op_refuse.
The test runs and passes on AlmaLinux 8 (chroot works and CI runs as root), so
listing it in RSYNC_EXPECT_SKIPPED tripped the strict skip-set check (expected to
skip but ran). It only genuinely skips where chroot is unavailable -- cygwin --
whose entry is kept.
daemon-symlink-escape-matrix prefers the in-tree old_versions/rsync_3.2.7 as its
legacy oracle, but that is a Linux x86-64 static binary. On a full checkout (the
per-platform CI runners) it is present yet cannot exec on a BSD/macOS/Solaris host,
so start_test_daemon() crashed with OSError ENOEXEC and failed the test. (The
git-archive fleet never hit this: .gitattributes export-ignores old_versions/, so
the binary is absent there and the test already took its static-contract path.)
Probe the selected oracle binary (rsync --version) and degrade to the static
contract on any OSError, the same path used when no oracle binary is present.
NEWS.md: extend the robustness-hardening section with the second-pass source
audit (hashtable/flist size-computation integer overflows, non-positive
MSG_IO_TIMEOUT reject, async-signal-safe SIGUSR2 handler) and the source-side
xattr/ACL metadata read confinement plus the fake-super cross-tree metadata-apply
fd-pin.
SECURITY.md: note in the operator-directory residual writeup that the cross-tree
metadata apply on a --temp-dir/--backup-dir leaf is fd-pinned the same way the
reads now are, including under --fake-super (previously am_root >= 0 only).
The hardened receiver confined the destination side of an xattr/ACL copy (the
fsetxattr/acl_set_fd through a held O_NOFOLLOW fd) but still read the SOURCE side
by path: copy_xattrs() did get_xattr_names/get_xattr_data on the source path, and
make_backup() cached the backed-up file's ACL/xattr via get_acl()/get_xattr() by
path. A local module writer could race the source/basis parent to a symlink
after the confined content/stat open and before that path-based metadata read,
so out-of-module xattrs/ACLs got copied onto an in-module destination or backup.
Thread a source fd through the read side, mirroring the existing dest-fd plumbing:
- get_xattr_data(), get_xattr() and get_xattr_acl() gain an fd arg (get_xattr_names
already had one) and use sys_fgetxattr/sys_flistxattr when fd >= 0; this also
covers the --fake-super ACL-as-xattr read in get_rsync_acl().
- copy_xattrs() gains a source_fd; copy_file() passes its held source fd (ifd)
and keeps it open across the xattr copy (closing it on the fsync error path
too); gen_entry_copy_xattrs() O_NOFOLLOW-opens the basis leaf under the
confined resolver (with O_DIRECTORY for a directory basis) and passes it.
- make_backup() pins the source leaf with a confined O_NOFOLLOW fd
(backup_source_fd, like set_file_attrs's op_leaf_fd) and reads its ACL via
get_acl_fdat() and its xattrs via get_xattr(fd); the in-place delta-backup in
the generator pins fname the same way. On a hardened receiver a raced/absent
leaf skips the cache rather than reading through a flippable path.
Non-hardened receivers (fd < 0) keep the path-based behaviour unchanged. The
basis COMPARE reads (the generator deciding a match) stay path-based: they never
copy out-of-module metadata onto a file, so they are not part of this sink.
set_file_attrs() pins a cross-tree operator leaf (an absolute --temp-dir /
--backup-dir / --*-dest path) with an O_NOFOLLOW fd (op_leaf_fd) and drives
chmod/chown/xattr/ACL/times off it so a flipped parent can't redirect them, but
the pin was opened only when am_root >= 0. A daemon module with "fake super =
yes" runs with am_root < 0, so the cross-tree leaf kept op_leaf_fd/held_fd == -1
and the fake-super %stat, preserved-xattr and ACL-as-xattr writes fell back to
path-based sys_lsetxattr(): a local module writer racing the staging parent to a
symlink could redirect those metadata writes outside the module
(CVE-2026-53799 residual on the fake-super path).
Open the pin for fake-super too -- it has nothing to do with privilege: the
daemon owns the freshly-staged leaf it is about to set metadata on, so any
O_NOFOLLOW open failure is a race and is refused rather than redirected through a
re-resolvable path. Every metadata op already routes through
held_fd/op_leaf_fd/op_refuse, so they all become fd-based; strace confirms the
cross-tree fake-super write uses fsetxattr()/fchmod(), never the l-variant.
The previous matrix only drove --copy-dirlinks/--keep-dirlinks, which route
entirely through send_directory -- the one daemon descent site that already
honoured the opt-out -- so it passed even while explicit-path traversal
(change_dir) and alt-dest basis (basis_link_stat) ignored "insecure links =
yes". It also asserted a hardcoded contract rather than measuring stock 3.2.7.
Add read-plain/write-plain vectors (a plain pull/push through a symlinked
directory -> change_dir) and a compare-dest vector (--compare-dest=/symlink ->
basis_link_stat), and stand up a second daemon running old_versions/rsync_3.2.7
as a live oracle: every insecure=yes cell must follow iff 3.2.7 follows, and no
insecure=no cell may ever escape the module. Falls back to a static contract
when the 3.2.7 binary is absent. Red on the pre-fix build for both the escape
and the opt-out follow gap; green after.
The opt-out is meant to restore stock-3.2.7 "follow existing symlinks in the
module" behaviour, but several daemon symlink-resolution sites called the
confined resolver unconditionally and never consulted symlink_optout_allowed(),
so a module with "insecure links = yes" still refused to follow a symlinked
directory (change_dir ELOOP) or an alt-dest basis. Gate every such site on the
opt-out -- follow plainly like 3.2.7 when set, confine otherwise:
- change_dir() relative daemon branch (util1.c): plain chdir() under the
opt-out instead of secure_relative_open(), so a peer can read a path through
a symlinked directory again.
- basis_link_stat() (generator.c): the chrooted-relative and non-chroot
branches honour the opt-out. The non-chroot ABSOLUTE branch (a basis rooted
under the module by check_alt_basis_dirs, which can reach an in-module
symlink) is, when NOT opted out, resolved through owner_walk_parent so a
target landing outside the module root is refused -- closing a confirmed
--compare-dest=/symlink out-of-module read oracle. The leaf is taken
O_NOFOLLOW under the confined parent so --copy-links can't follow a leaf
symlink out; --fake-super folds its %stat via the held fd. A relative
sibling basis (--link-dest=../01) keeps the plain path (#915/#930).
- secure_basis_open() (receiver.c): plain do_open() under the opt-out.
- use_secure_symlinks (clientserver.c): cleared under the opt-out, so the
receiver's protected-regular EACCES write fallback is legacy too.
- secure_sender_parent_fd() (sender.c): declines (errno=0) under the opt-out
so --remove-source-files re-stats the plain path.
Default modules (insecure links = no) are unchanged and stay fully confined;
the opt-out is per-module and a client cannot enable it.
A daemon must clamp a peer-supplied --link-dest/--compare-dest/--copy-dest basis
dir to the served module: --link-dest=../sibling (or an absolute path) must not
let the daemon stat files outside the module (an existence/size oracle -- the
KI-48 / CVE-53795 surface). rsync confines this lexically already (longstanding
sanitize behavior, preserved through this branch's path-confinement rework); the
test locks that in so the rework can't regress it. The audit's "runtime-
confirmed" KI-48 was a local receiver, where --link-dest=.. is the operator's
own intended choice, not a module-boundary crossing.
abspath_excluded_by_module() is module-ROOT confinement only by design (the
daemon name/exclude filter is name-based visibility, not a physical-path
boundary -- munge symlinks is that defense), so its name_is_dir parameter was
always discarded ((void)name_is_dir). Remove it, and the now-redundant
absent-leaf second call in owner_walk_parent()'s leaf check that re-tested the
SAME leafabs with name_is_dir=1 for an identical result, plus the
fstatat()/isdir/absent that only fed the ignored logic. Behavior-preserving
(the refuse decision is unchanged at every call site); drops one incidental
lstat.
(Leonid Bugaev May-2026 re-audit, KI-49 -- dead defense-in-depth arm, no escape.)
sigusr2_handler called output_summary() (rprintf/vsnprintf/fwrite/iconv/malloc)
and close_all() (fstat/shutdown/close) directly from signal context before
_exit(). SIGUSR2 is sent by the generator/parent to tell the receiver child to
print its summary and exit; if it interrupted the receiver while it was inside
malloc/stdio, the handler re-entered those and could deadlock or corrupt.
The handler now only sets a flag (got_sigusr2, a volatile sig_atomic_t); the
actual summary + shutdown moves to receive_sigusr2(), run from a safe point in
the receiver's post-transfer wait paths:
- the perform_io() flag checks (next to got_kill_signal);
- the safe_read()/safe_write() loops (which a --read-batch / --write-batch fd
uses without going through perform_io);
- the whine_about_eof() kluge loop, where the receiver waits out the race of
the sender dying before the kill-signal arrives -- this loop polls for the
signal, so without the flag check it slept the full 10s and then errored with
RERR_STREAMIO instead of exiting cleanly;
- the trailing `while (!got_sigusr2) msleep()` loop in do_recv().
(Leonid Bugaev May-2026 re-audit, KI-14.)
Add t_hashtable_overflow (links the real hashtable.o, sets a realistic
--max-alloc, requests an absurd size) and a test asserting hashtable_create now
rejects it with RERR_MALLOC instead of under-allocating and crashing on the OOB
node access a regressed build would hit. The helper defines its own
info_levels/debug_levels (as the other t_* helpers do) for the DEBUG_GTE macro
in the linked hashtable.o.
hashtable_create() and the grow path computed the slot-array byte count as
new_array0(char, size * node_size) in 32-bit int arithmetic; for a large
peer/data-driven size the product wrapped to a tiny value, bypassing my_alloc's
--max-alloc guard (which only saw the already-wrapped count), so the table was
under-allocated while tbl->size kept the huge size -- a later node access then
ran out of bounds (heap overflow; ASan-confirmed). Pass size and node_size as
SEPARATE factors so my_alloc checks both before multiplying; guard each *2
doubling against int overflow BEFORE it happens; make HASH_LOAD_LIMIT divide
before multiplying; promote the HT_NODE index multiply to size_t; and test
size < 16 first so a negative req short-circuits the size-1 (INT_MIN UB).
flist_expand()'s int growth math (used+extra and *=4 / *=2 / += FLIST_LINEAR)
could overflow past INT_MAX on a very large file list; guard each operation
before it overflows and refuse rather than under-size the realloc.
(Leonid Bugaev May-2026 re-audit, KI-11/12/13.)
Drives a tiny crafted rsync daemon that completes the handshake and sends
MSG_IO_TIMEOUT(0) as its first multiplex frame, then holds the socket open well
past the client's --timeout. The fix ignores the non-positive value so the
client keeps its timeout and self-exits; a vulnerable client disables its
timeout and hangs until the test's watchdog kills it (test_fail). Runs in any
transport mode (it connects to the local crafted server over a rsync:// URL).
RED on stock 3.4.x, GREEN on the fix.
(Leonid Bugaev May-2026 re-audit, KI-47.)
Pin the two cross-uid operator-path races fixed in "confine the remaining
cross-tree operator-path syscalls" and "copy_file: confine an absolute operator
source ...":
temp-dir-symlink-injection absolute --temp-dir rename pulls an attacker's
out-of-tree file into the destination (do_rename_at
absolute-side confinement)
copy-dest-symlink-readleak --copy-dest basis read follows a flipped foreign
parent symlink, leaking out-of-tree content into
the destination (copy_file source confinement, KI-46)
Both root+nobody gated (the cross-uid plant needs root), RED on stock 3.4.x and
under --insecure-links, GREEN on the fix; cygwin runs make check non-root so
they skip there (added to its RSYNC_EXPECT_SKIPPED).
copy_file() routed a RELATIVE source through secure_relative_open (parents
confined) but opened an ABSOLUTE source -- an operator basis such as an absolute
--copy-dest -- with bare do_open_nofollow, which refuses only a leaf symlink and
follows every parent. basis_link_stat() refuses a foreign-owned basis at stat
time, but a parent flipped to a foreign symlink between that stat and this open
redirects the basis read out of tree (an out-of-tree content read-leak into the
destination; RED on 3.4.x, GREEN here).
Resolve an absolute source's parents through owner_walk_parent (foreign-owned
parent symlink refused, operator's own dirs/uid0/euid symlinks followed).
operator_path_resolve is set only across the walk -- so module-exclude is
enforced -- and restored, leaving the caller's value for the dest side; that is
why confining the source here does not re-open the copy_xattrs dest race that
wrapping the whole copy_altdest_file would (copy-xattrs-symlink-race stays green).
(Leonid Bugaev May-2026 re-audit, KI-46.)
A peer may use MSG_IO_TIMEOUT only to ask us to adopt a SHORTER I/O timeout
(a stricter cap). A crafted server sending val <= 0 would instead zero the
client's --timeout via set_io_timeout(0), disabling it entirely and letting the
server hang the client indefinitely. Ignore a non-positive value.
(Leonid Bugaev May-2026 re-audit, KI-47; pre-existing since 2009.)
The backup-dir fix pinned chmod/chown for an operator leaf; extend the same
confinement to every other sink that re-resolves a cross-tree operator path
(an absolute --temp-dir/--partial-dir/--*-dest), each confirmed by a cross-uid
race PoC that is RED on stock 3.2.7 and GREEN here:
- set_file_attrs (rsync.c): route times (do_futimens), and -- by aliasing
held_fd to the pinned op_leaf_fd -- the xattr/ACL ops through the same
O_NOFOLLOW leaf fd, not a re-resolvable path. A raced/refused pin skips them
(op_refuse) rather than redirecting. finish_transfer now wraps the pre-rename
set_file_attrs in operator_path_resolve so an absolute --temp-dir temp file's
metadata is pinned too (in-tree temps keep their held dirfd, so op_pin is off).
- do_rename_at / do_link_at (syscall.c): an ABSOLUTE side was left at AT_FDCWD
and followed a flipped parent symlink, letting a name-disclosed --temp-dir /
predictable --partial-dir rename pull an attacker file into the destination
(content injection). Resolve an absolute (operator) side via the ownership
walk -- with module-exclude enforced -- while a relative (transfer) side stays
on secure_relative_open. --insecure-links keeps the legacy path.
- secure_basis_open (receiver.c): an alt-dest basis read
(--copy-dest/--compare-dest/--link-dest) on a non-daemon receiver used a bare
do_open; route it through the ownership walk (refuses a foreign-owned basis
symlink, still allows the "../sibling" basis of #915). Daemons keep their
existing confinement branch.
- configure.ac: probe futimens (do_futimens is gated on HAVE_FUTIMENS).
New fd wrappers: do_fchmod, do_fchown, do_futimens. Documented residuals left
as-is: copy_altdest_file's basis copy (routing it re-opens copy-xattrs-symlink-
race; basis_link_stat already refuses the foreign symlink), crtimes
(do_setattrlist_crtime/do_SetFileTime, path-based on macOS/Cygwin), and
device/socket leaf metadata (pinning a device via open has side effects).
Pins the fix in "backup: confine cross-tree operator-path metadata via a pinned
fd". A root operator runs `rsync -a -b --backup-dir=<abs> ...` while a non-root
attacker flips a backup parent component between a real dir and a foreign-owned
symlink -> outside; pre-fix, rsync's own backup-dir attribute mirroring lchowns
the planted symlink to root, laundering it into a trusted symlink the owner-walk
then follows, so the backup escapes the tree.
Root+nobody gated (cross-uid plant needs root); RED on stock 3.2.7 and under
--insecure-links, GREEN on the fix. Uses the compiled flipper for a reliable
RED oracle. cygwin runs make check non-root so the test skips there -- add it to
that workflow's RSYNC_EXPECT_SKIPPED; the root workflows (almalinux-8 container,
sudo ubuntu/macos) run it for real.
The Python path-flipper used by the symlink-race tests wins the race window
unreliably on a journaled disk fs -- on the vulnerable binary it reproduced the
escape only ~1/3 of the time, because the interpreter loop caps the swap rate.
Add compile_c_flipper()/start_c_flipper(): a small C flipper, built on demand
against the build's config.h (CC and -I taken from TOOLDIR, then SRCDIR), that
swaps two sibling names with renameat2(RENAME_EXCHANGE) where available -- one
atomic syscall, no transient missing-name window -- and a self-healing 3-rename
fallback elsewhere. Measured ~2x (plain rename) to ~7x (EXCHANGE) the swap rate
on disk, which turns a flaky RED oracle into a reliable one. It self-terminates
on parent exit plus a deadline backstop (like start_path_flipper) so a killed
test can't leak an orphan, and falls back to the Python flipper where no
compiler is available.
A symlink race on a non-daemon --backup-dir let a local attacker redirect
rsync's backup writes and chmods outside the backup tree when rsync runs as
root. make_backup() sets operator_path_resolve, but set_file_attrs() had no
held parent dir fd for an absolute backup path (held_dfd_for() returns -1), so
its chmod/chown fell through to the path-based wrappers and, for an absolute
path, to raw chmod()/lchown(). copy_valid_path() mirrors the source dir's
attrs onto each backup subdir; when an attacker flips a backup component to a
symlink in that window the raw lchown retags the planted symlink as root-owned
-- laundering it into a "trusted" (uid 0) symlink that the owner-walk then
follows, so the backup rename/chmod escapes the tree.
Pin the leaf inode of a cross-tree operator path with an O_NOFOLLOW open via
the operator owner-walk resolver and drive fchmod/fchown off that fd; a raced
symlink leaf makes the open fail and the op is refused, never redirected. Gate
on the INTENDED type (new_mode), not the attacker-controlled on-disk type. As
root any open failure is the race (a real owned leaf never fails); a non-root
operator, which cannot launder a uid-0 symlink, falls back to the legacy path
op on a benign EACCES. --insecure-links opts back out.
Adds do_fchmod()/do_fchown() fd wrappers. Residual cross-tree metadata sinks
(times, ACLs, xattrs, and --temp-dir's finish_transfer set_file_attrs) are not
covered here and are tracked for a follow-up.
Tighten man-page prose for options whose documented behaviour was
imprecise or outright wrong, matching the behaviour now pinned by the
new oracle tests and verified against the C source:
- strict modes: the real secrets-file rule is "st_mode & 06" (other
read/write) plus root-owner-when-root, so 640 is accepted and 644
rejected -- not the old "any user ID other than the daemon's".
- munge symlinks: rewrite the default in terms of chroot and the "/./"
path split (disabled only for a plain chrooted module serving the
chroot root); drop the bogus "daemon chroot" clause. Fix the helper
reference: support/munge-symlinks is a python script, not perl.
- --no-implied-dirs: spell out that an existing in-tree dest symlink is
followed.
- --files-from: ".." handling is collapse-then-reject-survivors.
- --copy-unsafe-links: describe the lexical unsafe-symlink rule instead
of the old "verbose output" phrasing.
Add behaviour tests that nail down option semantics the man pages
describe vaguely, each verified to pass against both this branch and the
3.2.7 oracle (so they document long-standing behaviour, not regressions):
daemon-strict-modes-matrix secrets-file mode rule (st_mode & 06)
daemon-chroot-munge-default munge-symlinks default vs chroot/path /./
safe-links-unsafe-def --copy-unsafe-links lexical unsafe rule
no-implied-dirs-symlink --no-implied-dirs follows in-tree dest symlink
files-from-path-clamp --files-from collapse-then-reject ".."
relative-implied-symlink --relative sends implied dirs as real dirs
keep-dirlinks-rule --keep-dirlinks opening rule
backup-dir-relative --backup-dir resolves relative to dest
no-implied-dirs-symlink relies on the -R "/./" implied-dir marker, a
protocol-30+ feature; under protocol 29 the generator rejects the
multi-component path (same as the 3.2.7 oracle), so it passes through
without testing, matching the sibling relative-implied test.
daemon-chroot-munge-default needs root to exercise the chroot regimes
and skips otherwise; add it to RSYNC_EXPECT_SKIPPED only in the
almalinux-8 and cygwin workflows, which run make check non-root. The
macos workflow runs it as root, so the test runs there for real.
The "Robustness against malicious peers" summary enumerated the classes of
peer-triggerable faults closed by the fuzzing/static-analysis pass. Add the two
classes that the later scanner batch introduced -- reads past a file-list
allocation (mostly bounded over-reads of an entry's extra slots) and
option-argument-driven length bounds (plus the suffix-list recursion sink) --
keeping the summary general (no per-finding detail; not every over-read
disclosed memory).
daemon-symlink-escape-matrix exercises, for a writable non-chroot module, every
combination of `insecure links` {no,yes} x `munge symlinks` {no,yes} x link
origin {pre-existing, uploaded} x op {read pull, write push} x five symlink
target types (rel-within, rel-outside, rel-transits [.. above the module root
then back in], abs-outside, abs-inside).
It pins the contract: the secure default follows only an in-tree (rel-within)
link and NEVER reaches an out-of-module target (read or write); the
`insecure links = yes` opt-out restores legacy following on sender AND receiver
(so an outside target escapes, matching stock 3.2.7); and an uploaded link never
escapes regardless (munge prefixes it, munge-off sanitises it). A secure-default
out-of-module access is a hard failure. require_tcp + root gated; listed in the
per-platform RSYNC_EXPECT_SKIPPED pipe make-check sets.
secure_relpath_active() (the gate that routes receiver-side filesystem ops --
get_dir_fd/dpc, do_*_at, open_tmpfile, make_path, link_stat -- through the
symlink-race-safe resolver) checked only am_daemon/am_chrooted/am_sender, not
the symlink_optout_allowed() opt-out. So `insecure links = yes` (or a non-daemon
--insecure-links) restored the legacy follow only on the SENDER enumeration
(which checks the opt-out directly), while the receiver still confined writes,
mkdirs, renames, unlinks and stats through a pre-existing in-module symlink --
i.e. the admin opt-out did not actually reproduce the pre-3.4.3 behaviour it
documents (rsyncd.conf(5) "munge symlinks"/"insecure links").
Have secure_relpath_active() return 0 when symlink_optout_allowed(), so the
opt-out uniformly disables the secure resolver on both sides. No effect on the
default (opt-out off): confinement is unchanged.
The probe() helper sends the @RSYNCD greeting after the PROXY header, then
reads the daemon's response. For a `want='drop'` leg (untrusted peer / a
daemon with no `proxy protocol hosts`) the daemon closes the connection,
which on some CI runners surfaces as EPIPE/ECONNRESET on our sendall() before
we ever read -- an uncaught BrokenPipeError that failed the test (seen on
AlmaLinux 8 and Ubuntu 22.04, a timing race; other runners closed read-side).
Wrap the send/recv in `except OSError` and leave `out` empty: for want='drop'
the absent greeting is the expected outcome; want='ok'/'denied' still fail
correctly on an absent greeting.
The four tests added with the audit fixes skip on the standard (non-ASan,
stdio-pipe) CI/fleet runs: daemon-deny-dns-failopen needs a TCP peer
(require_tcp), and the three leak reproducers need an AddressSanitizer build
(require_asan). Add them to RSYNC_EXPECT_SKIPPED so make check / the fleet
report clean instead of flagging an expected skip as a mismatch.
(cherry picked from commit 272341682b668424e1f87fd1e8f8a5878db272c2)
Five error-path/cleanup memory leaks found by an external audit:
- flist.c send_file_name: free the ACL loaded by get_acl() when a later
get_xattr() fails (and on the get_acl error path).
- xattrs.c copy_xattrs: free the xattr datum buffer when the setxattr fails.
- generator.c recv_generator: free real_sx at the cleanup label (the
directory branch loaded its ACL via set_file_attrs but only the
regular-file path freed it); zero-init real_sx so the early gotos are safe.
- uidlist.c send_one_list: free the strdup'd id-0 name after send_one_name.
- clientserver.c start_inband_exchange: free modname on the early error
returns (it was freed only on the success path).
ASan/LSan regression tests cover the generator, uidlist and clientserver
leaks; the flist and xattrs leaks need a forced syscall failure and are
covered by the audit's standalone harnesses.
Reported-by: Leonid Bugaev <leonsbox@gmail.com>
(cherry picked from commit 078f3b99f4004510d418ee9d97d9b775dc8587bc)
match_hostname() did a forward-DNS lookup of a config-specified hostname
token and, on gethostbyname() failure, returned "no match" -- which
allow_access() cannot distinguish from a real non-match, so a daemon with
"hosts deny = <hostname>" silently admitted the host whenever the token
could not resolve (a resolver-less chroot, or a transient DNS failure).
No attacker DNS control required.
Thread a deny flag through access_match()/match_hostname() and, on a
forward-DNS failure, treat an unresolvable DENY-list token as a match so
the connection is refused (fail closed); allow-list tokens still fail as a
non-match. Sibling of CVE-2026-43617, which fixed only the reverse path.
Reported-by: Leonid Bugaev <leonsbox@gmail.com>
(cherry picked from commit 84e469ea03a98ef31326d3d861336d7e7d582ce1)
Python 3.14's strict UTF-8 text mode raised UnicodeDecodeError when tls
emitted a non-UTF-8 byte (a filename or symlink target with high bytes),
aborting every test that calls rsync_ls_lR via hands_setup() -- ssh-basic,
hands, delete, files-from, alt-dest, daemon-gzip-*. Decode the tls output
with errors='backslashreplace' so a stray high byte renders as \xNN in the
listing rather than crashing the run; the result stays a clean str that
write_text()/print() can consume without re-raising.
Adds six coverage tests from the code-scanner run, each closing a measured
gap in a daemon or metadata code path the suite never reached:
- daemon-include-maxconn: rsyncd.conf &include/&merge directives +
`max connections`/`lock file` (params.c include_config, connection.c
claim_connection, util1.c lock_range).
- fake-super-acl-xattr: --fake-super -A stores ACLs as user.rsync.%aacl/
%dacl xattrs (acls.c am_root<0 IVAL/SIVAL pack, xattrs.c get/set/
del_def_xattr_acl); Linux-only (the user.rsync.* namespace).
- backup-crossdev-copy: make_backup() EXDEV copy-fallback for non-regular
files (do_symlink_at/do_mknod_at/copy_file); skips without a cross-dev
tmpfs.
- daemon-http-proxy: RSYNC_PROXY HTTP CONNECT (socket.c
establish_proxy_connection + base64 Proxy-Authorization + 503 branch).
- daemon-module-options: motd file, socket options, incoming/outgoing
chmod, dont compress, list=no, comment, --sockopts.
- daemon-chroot: `use chroot = yes` incl. the /outer/./inner split and
`temp dir`; probes CAP_SYS_CHROOT and skips cleanly without it.
clientserver.c flushes gcov counters just before chroot() in rsync_module()
so the per-connection child's pre-chroot lines reach disk (the build-tree
.gcda paths are unreachable post-chroot); no-op without --enable-coverage.
CI: the require_tcp-gated tests (daemon-chroot/-http-proxy/-module-options)
plus the Linux-only fake-super-acl-xattr and the cross-dev backup-crossdev-copy
are listed in the per-platform RSYNC_EXPECT_SKIPPED sets where they skip on the
pipe-transport make-check jobs.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The operator_path_resolve branch of do_mknod_at() called mknodat() and
returned its result directly, without the FIFO/socket fallback that the
bare-path do_mknod() and the secure-relpath branch below it both have.
mknodat() can make a FIFO only on Linux; on the BSDs/macOS/Solaris it fails
with EINVAL, so creating a special file under an operator-supplied path --
e.g. backing up a FIFO into a --backup-dir -- failed there. Retry race-safely
with mkfifoat() on the held parent dirfd, and fail a nested socket closed
(EOPNOTSUPP) exactly as the secure-relpath path does.
io.c only treats a short read as the EOF sentinel when the fd is still
open; xattrs.c never stores a -1 from find_matching_xattr() and guards
ndx < 0 in set_xattr; rsync-ssl refuses the gnutls backend without
RSYNC_SSL_CA_CERT.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cap -v repetition so the argstr[64] global can't overflow, clamp a
negative --info/--debug level out of counts[], and cap a --skip-compress
suffix token at 32 bytes.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Move the uid/gid/acls/xattrs *_ndx assignments past check_batch_flags():
a mismatched-flag batch otherwise wrote F_XATTR(file) at offset 0 of
every file_struct, clobbering file->dirname. parse_negotiate_str() no
longer short-circuits on am_server, so each side picks its own #1 mutual
digest/checksum/compress choice rather than deferring to the peer's
order; man pages updated to match.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
A '!'-prefixed delete-delay entry computed one byte short, dropping the
final character of the name.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
hlink.c must confirm S_ISREG before quick_check_ok(FT_REG,...) reads
F_SUM, and start_server() must set sender_keeps_checksum when a daemon
sender runs -c with a %C log format so make_file() allocates
SUM_EXTRA_CNT. Without these, F_SUM() reads past the pool slot and (for
%C) hex-encodes adjacent heap into the transfer log.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
recv_file_entry (normal and XMIT_HLINKED abbrev branches) and make_file
left F_RDEV_P / the symlink-name slot uninitialized when the matching
preserve option was off, so a later read walked into adjacent pool
memory. Empty the symlink name when !preserve_links, zero F_RDEV_P when
!preserve_devices, and mirror both in the abbreviated branch. receiver.c
saves/restores the --write-devices S_IFBLK mode flip around receive_data
so dest_mode() never sees the mutated mode.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Adds the coverage/regression tests from the code-scanner run and the
gcov plumbing they rely on:
- scanner-argv-bounds, scanner-batch-flag-mismatch,
scanner-delete-delay-overread, scanner-daemon-log-checksum:
regression tests for the argv/-v/--info/--skip-compress bounds, the
batch metadata-ndx corruption, the read_delay_line off-by-one, and
the daemon -c/%C checksum-slot leak.
- daemon-proxy-protocol, daemon-early-exec-nameconv, daemon-auth-group,
daemon-standalone-detach, misc-coverage, nonroot-restrictive-perms,
backup-acl-xattr-cache: daemon and path coverage tests.
- rsyncfns.py: CAP_MKNOD probe in devices_supported().
- gcov_flush() macro (rsync.h) + calls in the daemon fork/_exit paths
(clientserver.c, socket.c); no-op without --enable-coverage. Makefile.in
COVERAGE_EXCLUDE / gcovr / setuid .gcda refinements.
- CI: list the new TCP/root/ACL tests in the per-platform
RSYNC_EXPECT_SKIPPED sets.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add a per-target `xfail` field (merged with the global --xfail) so a known
platform/version-specific failure can be tolerated persistently without a
command-line flag -- the test still runs, and if it passes the entry is a no-op.
Mark crtimes xfail on mac2: older backport binaries (3.4.x/3.2.7) drive APFS
birthtime via setattrlist differently than current rsync, so the 3.5.0
testsuite's crtimes check fails there; it passes for a current binary.
variety is the heaviest test in the suite; on slow platforms (Cygwin) the
per-component O_NOFOLLOW resolver pushes it past the default 300s per-test
timeout. Give it the same 600s budget the hardlinks test already gets.
The race-safe resolver's dirstack grew its fd array with realloc() inside the
recursive ds_descend() walk and freed it via a passed pointer. clang's
unix.Malloc analyzer cannot model that ownership and reported a false "Potential
leak of ds.fds", failing the pinned-clang-18 scan-build gate.
The walk holds one open fd per path component, so its depth is already bounded
by RLIMIT_NOFILE; use a fixed inline array (DS_MAXDEPTH, mirroring DPC_MAXDEPTH)
and drop the malloc/realloc/free entirely. ds_push() fails with ENOMEM past the
cap. No behaviour change -- 1024 levels exceeds any reachable depth.
--insecure-links and the SECURITY sections in rsync(1)/rsyncd.conf(5), the
name-based exclude/filter + munge-symlinks clarification, the rsync-ssl hostname
note, and the README (incl. the thank-you to Wayne Davison, 2004-2024).
NEWS.md: the 3.5.0 security-update section. SECURITY.md: the platform-residuals
policy, symlink-race-safe path resolution, operator-path symlink defense, the
name-based daemon exclude/filter clarification, and the known residuals.
testsuite/fleettest.py builds the branch and runs the suite across a fleet of
remote VMs (BSDs/Solaris/Ubuntu/macOS/Cygwin) over multiple transports, with a
--cleanup that reaps orphaned daemons; the expect/*.expect files are the
version-mixing manifests (current rsync vs old static peers).
Gate the build on a pinned clang-18 analyzer run (deterministic checker set,
--status-bugs fails on any new report) and run the latest clang informationally.
Run the security test suite (pipe + real-TCP daemon transports, proto30/29, and a
targeted non-root pass) across Ubuntu/macOS/Cygwin/AlmaLinux, with per-platform
expected-skip baselines for the tests that legitimately skip there.
The exclude/filter is a name filter, not a symlink boundary (3.2.7-equivalent):
symlink-exclude family, daemon-exclude-namebased, the operator-path exclude /
traversal / dir-daemon cases, filter-merge and implied-trailing-backslash.
Co-authored-by: Omar Elsayed <omarelsayed161@gmail.com>
The TOCTOU / symlink-race suite for the secure resolver and operator-supplied
paths: chdir/chmod/rename/mknod/source/dest symlink races, relative make_path and
symlinked-parent cases, the operator-path matrix (--temp/partial/backup-dir,
alt-dest basis, files-from, log-file, insecure-links), and the admin-file opens
(--password-file / daemon secrets / config / log-file / early-input symlinks),
plus the daemon module-confinement and chroot inner-module cases.
Co-authored-by: Omar Elsayed <omarelsayed161@gmail.com>
The t_rename_secure / t_symlink_secure / t_acl unit harnesses (C), the rsyncfns.py
helper library (daemon fixtures, symlink matrix, tree compare, xattr/ACL drivers),
runtests.py, and the rsync_proto/xrsync/cmptree/mkvariety helper scripts that the
security tests build on.
Pin each validated path component (and a receiver-side new destination's parent)
with O_RDONLY|O_NOFOLLOW and pass /proc/self/fd/N to the exec'd rsync so the child
cannot re-resolve the path; probe the /proc/self/fd magic-symlink at runtime (not
just isdir); fail closed on a readlink anomaly; and don't abort when flock() is
unavailable (Solaris).
batch.c: single-quote every --write-batch replay-script argument, quote a "--opt="
prefix unless it is a plain option token, and refuse a newline in a filter rule
written to the replay script. rsync-ssl: bind the server certificate to the
requested hostname in stunnel mode.
authenticate.c: seed gen_challenge() from /dev/urandom, add an "auth digest" floor
to refuse weak negotiated digests, and fstat the opened --password-file fd rather
than re-stat the pathname; checksum.c carries auth_digest_rank(). socket.c: reject
control bytes in the daemon host before a proxy CONNECT and bind the stunnel server
cert to the requested hostname. clientserver/access: warn when proxy-protocol
fail-closes. loadparm + daemon-parm: only shell-quote %RSYNC_*% for shell-executed
hooks, and add the auth-digest / proxy-protocol-hosts module parameters.
Refuse malformed/hostile wire input that could crash or corrupt the receiver:
io.c (out-of-range file index, count*blength OFF_T overflow, read_args NUL room,
deferred in_multiplexed), flist.c (sub-flist after the final flist is freed,
FLAG_HLINKED on dirs / gated on preserve_hard_links, parent_ndx bound, cleared-
slot ndx in the transfer phase), hlink.c (undeclared cross-flist gnum -> error not
assert), match.c (clamp peer flength, re-check len before want_i), log.c (drop
peer-reachable asserts and F_SUM deref), exclude.c (merge-file recursion cap,
trailing-backslash heap fix), lib/pool_alloc.c (ASan redzone for pool underflow).
Apply ACLs and xattrs through a held file descriptor instead of by path, closing
the symlink-race where an attacker swaps the leaf between the transfer and the
metadata set. lib/acl.c provides fd/at POSIX-ACL primitives (the system libacl
*_at where available, else a /proc/self/fd compat that never follows on the
fallback); acls.c routes through them and stays functional (path-based) where the
OS lacks a race-safe primitive; xattrs.c routes copy_xattrs through a held fd; -VV
(usage.c) reports the runtime race-safe-ACL capability.
The daemon exclude/filter chain is a name-based visibility/tamper filter, as in
stock rsync (verified against 3.2.7): a symlink whose own name is not excluded is
followed to an excluded target, and the documented symlink defense is `munge
symlinks`, not the filter. Collapse ".." (via sanitize_path) before the daemon
dest / temp-dir / backup-dir / partial-dir / basis filter checks so a "../excluded"
path is matched by name like 3.2.7 (clean_fname's CFN_COLLAPSE_DOT_DOT_DIRS does
not collapse "a/b/c/../../../secret"), and keep a leading "/" for a "path = /"
module so an absolute filter rule still matches. The module-ROOT confinement of
operator paths is unchanged (previous commit); only the in-module name match is
restored to its 3.2.7 behaviour.
Route every operator-supplied directory path (--temp-dir / --partial-dir /
--backup-dir / alt-dest basis) and the transfer engine's own dest/source opens
through the secure resolver + ownership walk, so a symlink owned by another uid
can no longer redirect a read, a backup, a staging open, a rename, an unlink or a
new-destination create outside the module. Covers backup.c, generator.c (alt/
link/in-place basis), receiver.c (basis open), sender.c (remove-source / source
open / copy-links leaf), rsync.c (held-fd attr stat), clientserver.c (pid-file
parent pin), and main.c (relative-basis make-absolute, mkpath dest-arg guard).
--insecure-links / "insecure links = yes" is the local opt-out (a daemon never
honors a peer-forwarded one).
The core symlink-race (TOCTOU) defense for the CVE-2026-29518 class: a portable
secure resolver that walks a path one component at a time holding an O_NOFOLLOW
dir fd per level (the dirfd-stack), plus the do_*_at() filesystem wrappers, the
held-directory fd cache, secure_relative_open[_at](), the operator-path ownership
walk (owner_walk_parent / open_no_attacker_symlinks, follow a uid0/euid symlink,
refuse a foreign one) and its module-root confinement (abspath_excluded_by_module).
util1.c routes change_dir / robust_rename / make_path / handle_partial_dir through
it; the resolver bounds its deep-path fd use against RLIMIT_NOFILE. Also drops the
obsolete android.c openat2 path and confines delete.c via the held dirfd.
configure.ac: detect fdopendir + a working dirfd() (a macro/inline on the BSDs,
so the default link probe mis-detects) and getrlimit/setrlimit for the resolver's
deep-path fd budget; add the race-safe-ACL build probes.
Makefile.in: build lib/acl.o; build the new t_rename_secure / t_symlink_secure /
t_acl unit harnesses; drop the obsolete android.o object.
mkgitver: version a git build by the dev version + commit (not the nearest tag),
and harden the version.h parse.
The description of user@host::module transfers over a remote shell only
documented the "ssh -l ssh-user" form, which led readers to conclude that
user@ never reaches the remote shell. In fact, for the simple
`--rsh=ssh user@host::module` form the user@ prefix is used both as the
ssh login user (ssh -l user) and as the rsync-user offered to the module;
the two are the same name. rsync only omits its own -l when the remote
shell command already specifies one, in which case user@ becomes the
rsync-user alone.
Spell out the default behaviour and why the explicit -l is needed to use
a different ssh login than the rsync-user.
hash_search() walks the entire hash-table chain for the current rolling
checksum at every byte offset of the source file. Disk and VM images
contain large runs of identical blocks, so a single weak checksum
(get_checksum1) can collide thousands of times and pile every one of
those blocks onto one chain. When the sender then rolls across a region
whose weak checksum keeps landing on that chain without ever producing a
strong-checksum match, it re-walks the whole chain for every byte, giving
O(file_size * chain_length) behaviour. The result is rsync sitting at
100% CPU for hours with no apparent progress -- the long-standing "rsync
hangs on large files" reports.
Cap the number of same-weak-checksum candidates examined per offset at
MAX_CHAIN_LEN. Once the cap is hit we treat the offset as a non-match and
roll forward a byte; any block skipped this way is simply sent as literal
data, so the transferred result is always correct -- only the transfer
size is marginally affected. This is purely a sender-side search limit:
it changes no checksum, emitted byte, or protocol field, so a capped
sender interoperates with an unmodified receiver and vice versa.
On a synthetic 40000-block basis sharing one weak checksum, syncing a
60KB source dropped from ~18.4s to ~0.7s; the unbounded cost grows with
the square of the file size.
testsuite/hashsearch-chain_test.py reproduces the pathology with a tiny
basis of weak-checksum-colliding decoy blocks and asserts, via the
existing false_alarms counter (--debug=deltasum1), that the per-hash-hit
chain walk stays bounded. The assertion is exact and machine-independent
rather than timing-based.
Split the scan-build workflow into two non-gating jobs, each uploading
its HTML report as an artifact:
- pinned-clang18: clang-18 / clang-tools-18 on ubuntu-24.04, so the
checker set -- and thus the report -- is deterministic.
- informational-latest: whatever clang ubuntu-latest ships, to surface
what newer analyzers see.
Both are informational (no --status-bugs): the tree still has known
clang-18 findings, so the run reports without blocking the build. Once
the tree is at zero for clang-18, re-add --status-bugs to the pinned job
to turn it back into a gate. Installs libpopt-dev so configure finds
popt under the scan-build compiler wrapper.
clientserver.c: close the --early-input-file FILE* on the
fstat/oversize/early-EOF error returns; it was only closed on the
success path.
getgroups.c: free the gid list before returning.
Remove stores that are never read before being overwritten or going
out of scope. No behavior change except batch.c write_opt, which now
accumulates the leading-space write error into the return value
(consistent with the arg branch) instead of discarding it.
simd-checksum-x86_64.cpp, options.c, util1.c, batch.c
clang's static analyzer doesn't model SIVAL/SIVAL64/SIVALu or
getpeername/getsockname as initializing their target bytes, so it
reports false "garbage value" reads. Zero-init the affected buffers;
the bytes are always overwritten at runtime, so this only quiets the
analyzer.
io.c: write_varint/write_varlong b[]
hashtable.c: hash_search buf[]
socket.c: accepted_peer/our_local
Under --valgrind some tests run rsync with reduced privileges: partial_nowrite
wraps it in "setpriv --inh-caps -all --bounding-set -all" to force EACCES, and
chdir-symlink-race's daemon drops to the module's uid. Such a child cannot
create valgrind's --log-file in a root-owned scratchbase, so valgrind aborts at
startup and the test fails (seen only in the root + --use-tcp cell).
Put the logs in a 1777 valgrind-logs/ subdir so a privilege-dropped child can
always write them. Scan and cleanup are unchanged; the logs just move one
directory down.
this fixes a valgrind error where we could read an uninitialised sx.st
field when we don't fill the stat data.
Also drop the now-obsolete testsuite/valgrind.supp stanzas for these
reads (atomic_create/delete_item, plus the rwrite strlcpy over-read that
master already fixed) -- they are no longer needed now the reads are gone.
Thanks to report from Michael Mess <michael@michaelmess.de>
main()'s line parser stepped through the fgets() buffer with `*++s` in
three places without first checking for the terminating NUL, so a test
line whose last token runs to the end of the buffer (e.g. a final line
with no trailing newline) could advance s past the NUL and read out of
bounds.
Guard the flag-separator check and rewrite the two whitespace-skip loops
so they never step past the NUL. No behaviour change for well-formed
input: the existing wildtest.txt still passes, and the crafted overflow
input is now clean under valgrind.
Fixes#776
Reported-by: vikk777 (@vikk777)
The valgrind memcheck CI flagged 'Conditional jump depends on uninitialised
value(s)' in rwrite() -> strlcpy() (log.c) and the subsequent logit() fprintf.
rwrite()'s daemon/logfile branch did strlcpy(msg, buf, MIN(sizeof msg, len+1)),
but strlcpy() scans the whole source with strlen(); buf is the data buffer from
read_a_msg() (io.c) holding exactly len bytes of a forwarded MSG_* payload with
no NUL terminator, so strlen() reads past the message into uninitialised stack.
Copy exactly len (bounded) bytes with memcpy() and NUL-terminate, matching the
(buf, len) contract the rest of rwrite() already honours. Behaviour is
unchanged for the NUL-terminated callers; the over-read is gone.
Full testsuite under valgrind (1572 logs) now reports zero unsuppressed errors.
`make` alone does not build the CHECK_PROGS test helpers (tls, trimslash,
t_chmod_secure, ...), so runtests.py exited immediately with "missing
test helper program(s)", produced no valgrind logs, and the scan step
failed every job with "the suite did not run". Use `make check-progs`,
which builds rsync plus the helpers and symlink fixtures without running
the suite.
rsync groups the "sent/received N bytes" summary numbers using the
locale's thousands separator (e.g. de_DE uses '.'), which broke the
[\d,]+ parser and failed the test for testers in non-C locales. Run the
peer client under LC_ALL=C so the output is deterministic.
Reported-by: Michael Mess <michael@michaelmess.de>
Add a .github/workflows/valgrind.yml that runs the full suite under
valgrind in a 2x2 matrix (user/root x pipe/tcp transport) and gates on
memory errors. It uses --leak-check=no: rsync intentionally leaves
file-list/socket/option memory unfreed at exit, so a leak check is
inherently noisy; the gate flags uninitialised reads, invalid
read/write, bad frees and uninit syscall params instead.
Add testsuite/valgrind.supp covering the known-benign reports (rwrite
strlcpy over-read on a non-NUL-terminated peer message, atomic_create/
delete_item st_mode read under fakeroot, libfakeroot msgsnd padding,
plus popt/xxhash leaks for manual --leak-check audits). runtests.py
--valgrind now loads it automatically.
The hardening in c44c90e9 added a check in simple_recv_token() rejecting
any uncompressed literal-run length > CHUNK_SIZE (32k). That assumption
breaks interoperability: other rsync implementations -- e.g. the acrosync
library used by the iOS "PhotoBackup" app -- use a 64k block size and
send literal runs of 65536 bytes, which 3.4.3+ now rejects with
"invalid uncompressed token length 65536".
The check was unnecessary: simple_recv_token() already reads the run
CHUNK_SIZE bytes at a time via the residue loop (n = MIN(CHUNK_SIZE,
residue)), so read_buf() never writes past the static CHUNK_SIZE buffer
regardless of the wire-supplied length. Drop the check to restore
interop; the compressed-token integer-overflow fix from c44c90e9 (the
MAX_TOKEN_INDEX / rx_token caps) is left unchanged.
Fixes#1002
Reported-by: Jack Whitham
FreeBSD and OpenBSD return EFTYPE (errno 79) when chmod-ing a sticky bit
onto a regular file as non-root, rather than EPERM/EACCES. Catch OSError
and check errno against the expected skip set so the test skips correctly
on those platforms instead of erroring out.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
testsuite/abdiff.py runs the same benign transfer with two rsync binaries
(A = build under test, B = a baseline) and compares the OUTCOME -- exit code,
stderr, --stats "Literal data", the destination tree (content + full metadata),
the --itemize list, and (with --cost) peak process-group RSS. For benign input
the two must be indistinguishable; any divergence is a regression candidate.
It is a developer tool, NOT a runtests.py test (does not end in _test.py).
Capabilities:
- Scenario sweeps over options / path shapes / file types / sizes / modes /
selection / placement / wire / transports, plus domain-knowledge pairwise +
combo sweeps and a stochastic fuzzer/role matrix.
- Transport lanes: local, ssh split (lsh.sh), stdio-pipe daemon, a REAL TCP
daemon (bound port + greeting/handshake/auth challenge-response), and the
restricted rrsync wrapper (support/rrsh.sh; each binary paired with its own
version's rrsync via --rrsync-a/--rrsync-b, since rrsync ships in the script).
- Stability gate: each binary is run N times and escalated on a candidate diff;
nondeterministic scenarios are quarantined FLAKY, never reported as regressions.
- Parallel (-j, default 20) with a per-run findings log; --loop runs until
--timelimit (or Ctrl-C), feeding the pool a half-random / half-systematic
stream of new combinations. As root an "all" run also folds in the root-only
sweeps (priv, daemonchroot).
- General coverage levers: a cost oracle (--cost, peak RSS over the whole process
group), transport lifted as an orthogonal axis, a resume/redo sweep, and
type-transition / nanosecond-mtime / scale (--scale N) fixtures.
Documented in testsuite/README.md.
A standalone dev tool (run directly, not via runtests.py) for catching
performance regressions between rsync releases. Given two rsync binaries it
builds one deterministic test tree -- heavy-tailed file sizes, a directory
spine, symlinks, hard links and a spread of permission modes, modelled on the
gentestdata generator -- then runs the two binaries ALTERNATELY for N loops,
timing each transfer, and reports the mean and standard deviation per binary.
Each loop times a full copy into an emptied destination and an incremental
no-op against an already-synced one (rsync's scan/file-list/stat overhead,
where many regressions hide); --mode selects. The first run of each binary is
dropped to reduce page-cache impact, the run order alternates to cancel drift,
and a B-vs-A slowdown is flagged only when it exceeds the run-to-run noise.
The file used ".align 16" intending 16-byte alignment (GNU/ELF semantics).
On macOS the Mach-O assembler reads ".align N" as 2^N, so it requested
64KB alignment for __TEXT,__text, producing:
ld: warning: reducing alignment of section __TEXT,__text from 0x10000
to 0x1000 because it exceeds segment maximum alignment
The linker clamps it back, so it was harmless, but .balign 16 means
16 bytes on every target and silences the warning.
When built with --enable-roll-asm, get_checksum1() called the AVX2 asm
routine get_checksum1_avx2_asm() unconditionally. Unlike the intrinsic
path (get_checksum1_avx2_64), which is function-multiversioned with a
target("default") fallback and so resolves safely on any CPU, the asm
routine is a single AVX2-only symbol with no fallback. On an x86-64 host
without AVX2 (an older CPU, or a VM that does not expose AVX2) the first
block checksum executes a VEX-encoded instruction and dies with SIGILL,
which surfaces as "connection unexpectedly closed (0 bytes received so
far)" and a code-12 protocol error.
Gate the asm call on a cached __builtin_cpu_supports("avx2") check, the
same signal the intrinsic resolver uses. When AVX2 is absent we skip it
and the SSSE3/SSE2/scalar steps (safe everywhere) do the work. Apply the
same guard in the simdtest harness so it can run on non-AVX2 hosts too.
Run the clang static analyzer over a check-progs build, publish the HTML report
as an artifact, and print the bug count to the run summary. INFORMATIONAL only:
it does not pass --status-bugs, so it surfaces new analyzer findings without
going red on the existing (overwhelmingly false-positive) reports.
Runs on push/PR to master and via workflow_dispatch. No cron: it is
informational and its output only changes with the code (push/PR) or the clang
version, so a daily run on an unchanged tree would add noise without value.
Add a clang AddressSanitizer + UndefinedBehaviorSanitizer workflow that builds
rsync with -fsanitize=address,undefined -fno-sanitize-recover=undefined -DNDEBUG
and runs the full test suite over both the stdio-pipe and TCP daemon transports.
UBSAN_OPTIONS=halt_on_error=1 together with -fno-sanitize-recover=undefined makes
any undefined behaviour fatal, so this job gates: the tree must stay UBSan-clean.
The remaining findings are fixed in code (hashtable/mdfour shifts, xattrs, and
log.c's file_struct, kept aligned via rounding.h); only byteorder.h's intentional
unaligned accessors are suppressed, with no_sanitize. -DNDEBUG builds as a release
does (assert() compiled out) so ASan covers the production code paths.
Runs on push/PR to master and via workflow_dispatch, plus a weekly cron to
catch breakage from a moving ubuntu-latest/clang toolchain (push/PR already
cover every code change, so daily would just re-run an unchanged tree).
The cherry-picked #428 wrapped no_sanitize attributes on read_varint() and
read_varlong() in `#ifndef CAREFUL_ALIGNMENT`, but byteorder.h always
#defines CAREFUL_ALIGNMENT (to 0 or 1), so that guard is never true and the
attributes were dead code.
They are also unnecessary: both functions read the assembled value through
an aligned union member (union { char b[5]; int32 x; }), not an unaligned
cast, so UBSan's alignment check never fires there (verified: the ASan+UBSan
suite is clean without them). Remove the whole block rather than fix the
guard. (The byteorder.h annotations from #428, which are real and correctly
placed inside the !CAREFUL_ALIGNMENT branch, are kept.)
rsync sets CAREFUL_ALIGNMENT for architectures which do not support
unaligned access. Disable UBSAN for functions which may use unaligned
accesses when CAREFUL_ALIGNMENT is set.
Bug: https://github.com/WayneD/rsync/issues/427
Signed-off-by: Sam James <sam@gentoo.org>
(cherry picked from commit 11c1e934e8)
log_delete() builds a struct file_struct inside a char buffer offset by the
(EXTRA_LEN-granular) extra data. The EXTRA_ROUNDING block that rounds that
offset up to the struct's alignment (exactly as flist.c does for its pool
allocations) was dead code here: log.c never included rounding.h, so
EXTRA_ROUNDING was undefined and the rounding never ran, leaving the
file_struct pointer potentially under-aligned. That trips UBSan's alignment
check and would fault on strict-alignment arches.
Include rounding.h (and add the Makefile dependency) so the existing rounding
actually applies -- fixing the alignment at the source rather than suppressing
the sanitizer.
Three pre-existing issues UBSan flags during the xattr tests:
* xattr_lookup_hash(): the summed hashlittle2() values overflow the
signed int64 accumulator (UB). Accumulate in uint64_t and convert back
at return -- the key is only used for hash-table equality, so the value
is unchanged.
* rsync_xal_get(): for an empty list (count == 0) the loop init
`rxa += count-1` forms `items - 1` on a NULL `items` (UB). Guard with
`if (count)`.
* rsync_xal_store(): `memcpy(dst, xalp->items, 0)` passes a NULL source for
an empty list (UB). Guard with `if (xalp->count)`.
UBSan flags two spots that shift a value into the top bits of a word via a
signed operand:
* lib/mdfour.c copy64(): `in[i] << 24` promotes the uchar to int, so a
byte >= 128 overflows int (UB). Cast each byte to uint32.
* hashtable.c NON_ZERO_64(): `(int64)(x) << 32` overflows int64 whenever
x's high bit is set. Shift as uint64_t (covers all four call sites).
Behavior-preserving -- only the intermediate type changes; the resulting
bit pattern is identical.
In a git worktree .git is a file (a gitdir pointer), not a directory,
so os.path.isdir('.git') wrongly aborted with "no .git dir" when the
release was run from a worktree. Use os.path.exists() so it works from
both a normal checkout and a linked worktree.
Every platform build (the BSD/Solaris/macOS/cygwin/almalinux/ubuntu jobs),
coverage, the version-mix job and the android static build ran on a daily cron
*in addition to* push and pull_request to master. Since push/PR already cover
every code change, the cron only adds drift coverage -- catching breakage from a
moving runner image or toolchain that no commit triggers. Those images do not
change daily, so a daily run mostly re-tests an unchanged tree.
Move them all to a weekly cron (Mondays, keeping each job's existing time) to
keep that drift coverage at roughly a seventh of the Actions spend and log
noise. fleettest was already weekly. Per-change CI on push/PR is unchanged, and
workflow_dispatch still allows an on-demand run.
A run killed without a parent-death backstop can strand a TOCTOU path-flipper
(a busy `python -c` rename loop that pins a CPU) and an orphaned test rsyncd
(--no-detach --address=127.0.0.1) that squats its fixed port -- the wedge the
claim_ports() bind-probe now reports and points at --cleanup. Sweep both, best
effort, before removing the run dirs.
Each sweep counts the pattern, kills it (with a `sudo -n` retry for a process a
root-running test left), then re-counts after a settle: KILLED reports what
actually died, and a process that survives (pkill blocked, no passwordless sudo,
missing/limited pkill) is reported as SURVIVED and fails the run instead of
falsely claiming success.
Run-dir removal falls back to `sudo -n rm` so a dir whose contents a root test
owns is removed instead of failing with "Permission denied" (the failure mode
seen on the ubuntu/mac targets); only a dir that survives even sudo is failed.
The kill patterns use the pgrep self-exclusion trick ('r[e]name', 'det[a]ch')
so they match a real process's "rename"/"detach" but not the literal pattern in
the cleanup shell's own argv -- run_on() passes the whole script as the remote
argv, so without it --cleanup would signal itself. The patterns are host-global
(not scoped to one run), so --cleanup is documented to run between runs, not
during one.
claim_ports() takes a POSIX byte-range lock per port, which serializes
concurrent live test runs. But the kernel drops that lock the instant the
holding process dies, even if the run left an orphaned rsync --daemon still
bound to the port -- which happens when a run is SIGKILLed on a platform with
no parent-death backstop (rsyncfns only arms PR_SET_PDEATHSIG, Linux-only, so
the BSDs/Solaris/macOS can strand a daemon). A later run then wins the freed
lock while the socket is still squatted and dies with a cryptic "bind() failed:
Address already in use" / "did not see server greeting".
After taking each lock, actually bind the port (SO_REUSEADDR, so a port merely
in TIME_WAIT is not a false positive; only a live squatter fails) and close it
immediately. On failure stop with an actionable message naming the port and the
likely orphaned daemon. Closes the gap that masked the OpenBSD daemon-auth wedge.
When --testsuite-repo provides the suite, the build tree (--repo) need not
carry runtests.py -- it may be an older release whose shell testsuite predates
the Python runtests.py (e.g. a 3.4.1 backport branch built and tested with the
current suite). Check runtests.py in TESTSUITE_REPO and only require the build
tree to be rsync source (rsync.h).
--repo couples the built source and the test suite that exercises it.
--testsuite-repo PATH overlays runtests.py + testsuite/ from a second tree onto
the staged build tree, and sources the expected-skip workflows from it, so one
can build an older release (e.g. a 3.4.x stable branch) and run the current
comprehensive suite against that binary. Defaults to --repo, so the existing
single-tree behaviour is unchanged.
The shell testsuite was removed in 1f689ec0 (rewritten in Python); only
*_test.py remain, yet collect_tests still globbed *.test and _testbase mapped
foo.test and foo_test.py to the same canonical name. Harmless on a master tree
(no .test files), but when an older tree's *.test files are present -- e.g.
fleettest --testsuite-repo building a 3.4.x release whose shell suite still
exists -- both glob to the same test name and scratch dir and race under -j,
producing spurious failures. Drop .test discovery entirely.
The regression test honestly skips when it cannot force the receiver's
output mkstemp() to fail -- as root (root bypasses DAC) and on Cygwin
(chmod 0555 does not deny the owner a write). The ubuntu, ubuntu-22.04,
almalinux and macOS jobs run `make check` as root, and Cygwin can't
enforce the unwritable directory, so the test skips on all of them.
runtests.py fails a run on any skip-set mismatch, so add the test to
those jobs' RSYNC_EXPECT_SKIPPED lists; the BSD/Solaris jobs run as root
too but enforce no expected-skip set, so they need no change.
Also tighten the pass condition. The post-chmod writability probe already
guarantees the receiver discards (mkstemp must fail), so an exit 0 would
mean the file actually transferred and the discard path was never
exercised -- a silent false-pass. Require exactly exit 23 (the forced
discard leaves the file untransferred); 12 remains the pre-fix crash.
Drives a real sender<->receiver pair (client sender -> daemon receiver,
both the binary under test in the default pipe transport) so the receiver
actually takes the recv_files discard path -- a local `rsync a b` does
not. The basis and source share a leading block so the generator emits
real sums and the receiver gets a block MATCH; the destination directory
is made unwritable so the receiver's output mkstemp() fails and it
discards the delta. Pre-fix the receiver SIGSEGVs in full_fname(NULL),
which the client sees as a protocol-data-stream error (code 12); post-fix
it drains the delta and reports a benign code 23 (or 0).
Skips (exit 77) when run as root, since root bypasses DAC and the
unwritable destination would not make mkstemp() fail -- so the discard
path, and the bug, would never be reached.
Verified red-on-buggy / green-on-fixed against the 0d0399bb receiver.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
receive_data() crashed a receiver that was merely DISCARDING a file's
delta stream. discard_receive_data() calls receive_data() with
fname == NULL and fd == -1, so size_r == 0 and mapbuf == NULL. A normal
block-MATCH token (against a block the basis and source share) then
reaches the !mapbuf branch added in 31fbb17d ("receiver: fix absolute
--partial-dir delta resume"), which calls full_fname(fname). full_fname()
dereferences its argument unconditionally (util1.c: `if (*fn == '/')`),
so fname == NULL faults there -> receiver SIGSEGV.
This is a normal-operation crash with a stock cooperating sender, not an
adversarial one. The generator hands the sender real block sums whenever
the basis is readable and we're in delta mode; the receiver only decides
to discard afterwards, when its output cannot be produced -- e.g. the
destination directory is not writable (mkstemp fails), the basis turns
out to be a directory, or a --partial-dir resume is skipped. A MATCH
token arriving during that discard hit the NULL deref.
The 31fbb17d branch is correct only for a REAL output transfer (fd != -1,
fname valid): there, a block match with no mapped basis is a genuine
protocol inconsistency (the generator promised a basis the receiver could
not open), and honoring it would silently omit those bytes from the
verification checksum or leave a hole, so hard-erroring -- and
full_fname(fname) -- is right. It conflated that with the discard path.
The discriminator is fd, not mapbuf: on the discard path fd == -1 always;
on the real-output inconsistency fd != -1. Scope the "no basis file"
protocol error to fd != -1 (where fname is non-NULL and full_fname is
safe) and, on the discard path (fd == -1), absorb the matched bytes
benignly (offset += len; continue) -- symmetric with the literal-token
handling just above, and restoring the pre-31fbb17d behavior. The
real-transfer inconsistency check is preserved unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A slow or heavily-loaded fleet box can occasionally flake a concurrency-
sensitive test (e.g. a daemon/lsh test under -j8 on a nested-VM Solaris box).
Rather than dropping the whole target to a lower -j, add a per-target
"max_retry" property: after a run, each failed test is re-run on its own up to
max_retry more times, and any that then pass are dropped from the failure list.
Recovered tests are listed in a new "RECOVERED" report section, so a flake is
surfaced, never silently hidden.
Applies to every pass for the target (pipe, tcp, protoNN, nonroot). Default 0
keeps the current no-retry behaviour.
A target can list older "protocols" (e.g. [30, 29]) in the fleet config;
each runs as an extra stdio-pipe pass with runtests --protocol=N, the fleet
analogue of a workflow's check30/check29 steps. The passes reuse the same
parsed RSYNC_EXPECT_SKIPPED list as the default pipe run and appear as protoNN
columns in the report and --timing breakdown. Targets without the key run only
the default protocol and show "-" there.
The example config's ubuntu-2604 target (mirroring ubuntu-build.yml, which has
check30/check29 steps) now sets protocols: [30, 29].
Covers both halves: a --mkpath file-to-file --dry-run must succeed and
match the real run (the #880 abort), and a plain file-to-file --dry-run
onto an existing differing destination must still itemize the real change
rather than report it as brand new. Both compare "--dry-run -i" output
against the real run.
Co-authored-by: Stiliyan Tonev (Bark) <stiliyan21@gmail.com>
A single-file --mkpath copy whose destination parent does not exist
failed under --dry-run: make_path() only *reports* the directories it
would create in a dry run, so change_dir#3 then tried to chdir into a
parent that isn't there and aborted with "change_dir#3 ... failed".
When the parent is genuinely missing in a dry run, skip the chdir and
mark the destination as not-yet-present (dry_run++), exactly as the
multi-file/dir-creation path already does, so the generator doesn't
probe the missing tree. Gating it on the missing-parent case keeps an
ordinary file-to-file dry run chdir'ing into and itemizing against an
existing destination.
Fixes: #880
Co-authored-by: Stiliyan Tonev (Bark) <stiliyan21@gmail.com>
doc/rsync.sgml is a 1996-2002 DocBook user manual (with README-SGML
describing the docbook-utils build) that was long ago superseded by the
markdown man pages. It is unmaintained and referenced by nothing in the
build. This empties doc/.
rsync3.txt and rsyncsh.txt are Martin Pool's 2001 design proposals
("notes towards a new version of rsync", an interactive rsync shell),
neither of which reflects the current implementation. doc/profile.txt is
stale profiling notes. None are referenced by the build, tests, or docs.
This Python 2 test-tree generator (print statements, string.letters,
.next()) has been broken on modern Python for years and is referenced
nowhere in the build, tests, or any script. Drop it.
send_deflated_token() adds a matched block to the compressor history with
deflate(Z_INSERT_ONLY). Our bundled zlib implements Z_INSERT_ONLY (it
produces no output and consumes the input in one call), but a build
against a system zlib lacks it and falls back to Z_SYNC_FLUSH (see the top
of the file), which emits a flush block into obuf. For a large
incompressible matched token that block exceeds AVAIL_OUT_SIZE(CHUNK_SIZE),
so deflate returned with avail_in != 0 and the transfer aborted:
"deflate on token returned 0 (N bytes left)" at token.c
The insert output is never sent -- the receiver rebuilds the matching
history itself in see_deflate_token() -- so loop, resetting the output
buffer, and discard it. Drain with the same condition as the data loop
above: until the input is consumed AND avail_out != 0. Stopping at
avail_in == 0 alone can leave pending output in the deflate stream (a
full output buffer with bytes still buffered), which would then be emitted
by the next real deflate send and corrupt the stream. A bundled-zlib
build still finishes in one iteration.
Fixes: #951
fleettest is a developer tool meant to run on a modern Ubuntu box, so a
bitrot check belongs in its own ubuntu-latest job rather than in the
testsuite (which runs on the BSD/Solaris/macOS/Cygwin matrix, whose
older Pythons may not even parse it).
The job sets up passwordless ssh to localhost, writes a two-target
fleet config that both ssh to localhost (distinct build dirs), and runs
a real fleettest pass. Two targets exercise the parallel multi-target
path and the per-run dir / port isolation; the run exits 0 only if
every cell is OK. Triggered on changes to fleettest.py or this
workflow, manually, and weekly.
Records wall-clock per phase (push, build, each test transport, nonroot)
plus a total in TargetResult, and with --timing prints a breakdown after
the report, sorted slowest-target-first. Targets run in parallel, so the
run is gated by the slowest one; the phase columns show whether that
hold-up is the push, the build, or a test pass. A target that failed
early (no total) falls back to the sum of the phases it reached.
Address review findings on the cleanup paths:
- --cleanup no longer removes a bare <builddir>, only the suffixed
<builddir>-* run dirs it created. This keeps the sweep within its
documented scope and avoids clobbering an unrelated tree.
- Add _unsafe_builddir(): reject empty/root/$HOME and any absolute path
directly under / (e.g. a misconfigured builddir of "/tmp") before
building a destructive command, in both cleanup paths.
- Use `rm -rf --` so a path with a leading dash can't be read as options.
- Soften the docs: run-dir removal on Ctrl-C/kill is best-effort (a
signal arriving mid-push can still leave a remnant for --cleanup).
Each run now builds in its own randomly-named dir on every target
(<builddir>-<run_id>), so two or three fleettest runs can share the same
fleet without colliding on the pushed tree, the build, or the testtmp
scratch. Port collisions were already handled by claim_ports() locks.
The run dir is removed when the run ends -- on success, failure, or
Ctrl-C/kill (atexit + SIGINT/SIGTERM handlers); --keep retains it. A new
--cleanup mode sweeps stray <builddir>-* dirs left by a SIGKILL.
Incremental builds are dropped (every run is a fresh dir + full build):
--no-push removed, --clean removed.
Also look for the fleet config at ~/.fleettest.json first, then
testsuite/fleettest.json (still overridable with --fleet PATH).
Maps every source group to a second group the test user belongs to via a
daemon upload (--groupmap='*:GID') and checks the wildcard took effect.
Runs both arg modes: the default path (the '*' is safe_arg-escaped and the
daemon must un-backslash it -- the regression) and --secluded-args (the '*'
is sent raw over the protected channel, a guard that the fix left that path
alone). Needs no root -- a non-root receiver can chgrp to a member group --
and was verified RED on a pre-fix binary (the escaped '\*' is ignored, gid
unchanged) and GREEN after the fix.
Without --secluded-args, the client's safe_arg() backslash-escapes shell
and wildcard chars in option values before sending them to the server, so
--chown's --usermap=*:user is transmitted as --usermap=\*:user. Over ssh a
remote shell removes the backslashes before rsync parses the args, but a
daemon has no shell and read_args() stored option args verbatim -- so the
receiver saw the literal "\*", the usermap/groupmap wildcard never matched,
and the module's configured uid/gid won instead. A regression from the
secluded-args hardening; rsync 3.2.3 (protocol 31) worked.
Un-backslash option args in read_args() on the daemon's first
(non-protected) read, mirroring what the ssh-side shell does. File args
after the dot are already handled by glob_expand(); the protected (NUL,
already-unescaped) re-read and the server's stdin read pass unescape=0 so
their raw args are left untouched.
Fixes: #829
do_mknod_at() (the symlink-race-safe variant used by a non-chrooted
daemon receiver) calls mknodat()/mkfifoat(), but the at-variant was
gated only on AT_FDCWD. Older Darwin declares AT_FDCWD without
mknodat(), so the build failed with "mknodat undeclared".
Probe mknodat()/mkfifoat() in configure and require HAVE_MKNODAT for the
at-variant; without it do_mknod_at() falls back to do_mknod(), exactly
as it already does where AT_FDCWD is missing. Linux keeps the mknodat
path since HAVE_MKNODAT is defined there.
Fixes: #896
Commit d046525d made my_alloc() calloc every fresh allocation and made
expand_item_list() memset the freshly grown tail, to hand out predictably
zeroed memory. But that forces the kernel to back pages callers never
touch: each per-directory file_list pre-allocates a FLIST_START-entry
(32768) pointer array -- 256KB -- and calloc now zeroes the whole array
even for an empty directory. With incremental recursion over many
directories the resident set explodes; 80000 empty dirs went from ~336MB
to ~10.8GB.
Restore the pre-d046525d malloc/calloc split: fresh allocations use
malloc (so untouched tails stay lazy) and only explicit do_calloc
requests (new_array0) are zeroed. Callers that need zeroed memory
already ask for it, and the full test suite passes.
Fixes: #959
Forces --checksum-choice=xxh64 (an 8-byte transfer checksum) with a
corrupted-prefix --append-verify so the full-checksum redo path runs.
Before the generator capped s2length at MIN(SUM_LENGTH, xfer_sum_len)
this died with "Invalid checksum length 16 [sender]"; the test is RED on
the prior generator and GREEN with the cap. Reproduces on any build that
has xxhash, so it guards the fix without an old-libxxhash host; skips when
xxh64 is absent (a build without xxhash).
sum_sizes_sqroot() capped the strong-sum length at SUM_LENGTH (16), the
legacy MD4/MD5 digest size. Since 0902b52f the sum2 array elements are
xfer_sum_len bytes and the sender rejects a sums header whose s2length
exceeds xfer_sum_len. When the negotiated transfer checksum is shorter
than 16 bytes -- xxh64 (8), used when the build's libxxhash lacks
xxh128/xxh3 (e.g. Ubuntu 20.04) -- the generator still emitted s2length
up to 16, so --append-verify and other full-checksum (redo) transfers
died with "Invalid checksum length 16 [sender]" (protocol incompatibility).
Cap s2length at MIN(SUM_LENGTH, xfer_sum_len): unchanged for any checksum
>= 16 bytes (md5/xxh128/sha1), corrected for short ones. Also closes a
latent over-read of the xfer_sum_len-sized digest buffer.
Android's seccomp sandbox traps openat2() with SECCOMP_RET_TRAP, which
raises SIGSYS and kills the process instead of returning ENOSYS, so the
secure resolver cannot simply try openat2() and inspect errno. Add
openat2_usable() in a new android.c: it probes openat2() once behind a
temporary SIGSYS handler and caches the result.
Gate every SYS_openat2 call on openat2_usable(): in the resolver via an
openat2_beneath() wrapper, and in t_chmod_secure's kernel probe directly,
so a blocked openat2 reports ENOSYS and the caller falls back to the
portable O_NOFOLLOW resolver. Only openat2 is gated -- a plain openat()
(e.g. opening an operator-trusted absolute basedir) is left free.
The probe body compiles only on Android -- __ANDROID__ is a Bionic target
macro, so it is set for NDK cross-builds and native Termux alike and unset
everywhere else, where openat2_usable() collapses to a constant 1. Link
android.o into the secure-resolver test helpers too so their self-tests
survive on Termux.
Adapted from PR #909.
The openat2 secure resolver in syscall.c needs struct open_how and
RESOLVE_BENEATH from <linux/openat2.h>, not only the SYS_openat2 syscall
number. Some setups expose the syscall number via glibc without the
kernel header present, so probing SYS_openat2 alone still left the build
broken (#905). Exercise the header and struct in the configure check so
HAVE_OPENAT2 is defined only when both are actually usable.
To prevent using openat2() in situations where it is not supported, use
#if defined(__linux__) && defined(HAVE_OPENAT2)
in t_chmod_secure.c, just like it was already being done in syscall.c.
Signed-off-by: Markus Mayer <mmayer@broadcom.com>
Let configure detect if the openat2() syscall is supported by the kernel
headers we are building against. Do not attempt to use openat2() if
support is not present.
Users can still disable using the openat2() syscall manually if so
desired.
Signed-off-by: Markus Mayer <mmayer@broadcom.com>
fleettest.py builds the committed HEAD of a checkout on a fleet of remote machines over ssh and runs the test suite under both the stdio-pipe and --use-tcp transports in parallel, reporting only the unexpected results. Each target mirrors a .github/workflows/*.yml job: its configure flags, and the RSYNC_EXPECT_SKIPPED list parsed from the workflow.
The fleet is described by a JSON file (testsuite/fleettest.json, git-ignored); fleettest.json.example is a worked template. Use --fleet to point at another config and --repo to build a tree other than the current directory.
A target with nonroot:true reruns, as the unprivileged ssh user, the tests that declare a module-level fleet_nonroot=True (here ownership-depth and daemon). The set lives in the test files, so new privilege-sensitive tests join the non-root pass with no fleet-config change.
Also rename testsuite/README.testsuite to README.md and rewrite it as markdown documenting the current testsuite: runtests.py, the make check/check29/check30/installcheck/coverage targets, the result/exit-code conventions, and fleettest.py.
The symlink-race hardening routed the receiver's basis open through
secure_relative_open(), which rejects any '..' -- so a sibling
--link-dest=../01 on a use-chroot=no daemon was silently ignored and every file
re-transferred (#915/#928, a regression from 3.4.1).
Narrow the confinement to the sanitizing daemon (am_daemon && !am_chrooted) and
re-anchor it at the module root, the real trust boundary: secure_relative_open()
prefixes the cwd's module-relative path (from rsync's logical curr_dir[], a
guaranteed lexical prefix of module_dir) and resolves beneath module_dir, so
RESOLVE_BENEATH permits an in-module '..' climb while still rejecting one that
escapes the module. secure_basis_open() opens with a bare do_open() in the
non-sanitizing cases. t_stub.c gains weak curr_dir[]/curr_dir_len for the
helpers (via #pragma weak on non-GNU compilers, where rsync.h erases
__attribute__).
Two tests: link-dest-relative-basis asserts the in-module '..' is honoured;
link-dest-module-escape asserts a --link-dest=../../OUTSIDE climb that leaves
the module is refused (not hard-linked to an outside file). See upstream
PR #930.
A daemon module with path=/ makes F_PATHNAME absolute, so the secure_path built
for the content open starts with '/'. secure_relative_open() rejects an
absolute relpath with EINVAL, so a use-chroot=no daemon with path=/ could not
send any file ('failed to open ...: Invalid argument (22)') -- a regression
from 3.4.2. Strip leading slashes to a module-relative path; resolution stays
confined beneath module_dir.
--delete-missing-args (missing_args==2) sends a missing --files-from arg as a
mode-0 entry (IS_MISSING_FILE), the generator's delete signal. The mode-type
validation in recv_file_entry() rejected mode 0 as an invalid file type,
aborting the transfer with 'invalid file mode 00 ... code 2' before the
generator could act (a regression from 3.4.1). Allow mode 0 through only when
missing_args==2 (the delete mode -- not --ignore-missing-args, which never
sends a mode-0 entry); all other modes are still rejected.
The regression tests use test_xfail() (exit 78) to assert a known, documented
residual on platforms where the fix can't apply -- e.g. link-dest-relative-basis
XFAILs where the receiver has no openat2/O_RESOLVE_BENEATH and the portable
resolver rejects the '..' for safety. runtests.py counted exit 78 in the
generic else->failed branch, so a bare XFAIL failed the whole suite; tally it
separately ('N xfailed (expected)') and exclude it from the failure exit code.
Also add --race-timeout plumbing (race_timeout env) for race tests.
Adds .github/workflows/ubuntu-version-mix.yml (ubuntu-latest) and a
per-release manifest testsuite/expect/rsync_<ver>.expect for each of the
nine peers. The workflow builds the current rsync, then runs the two-
sided suite against every old binary over both the pipe and --use-tcp
daemon transports. All peers run in a SINGLE looped job (not a matrix)
so the PR shows one check line; each peer/transport is a foldable log
group and a failure annotates which one broke.
A new phony `check-progs` target builds rsync plus the test helper
programs and check symlinks without running the suite -- the build half
of `make check` -- so the workflow's direct runtests.py invocation has
the helpers it needs.
Notable expected results encoded in the manifests:
- The four May-2026 security tests xfail against every released peer:
the suite demonstrates each release is vulnerable to those findings
while current master is fixed.
- symlink-dirlink-basis xfails on 3.4.0/3.4.1 (issue #715: their
secure_relative_open O_NOFOLLOW-confines the basedir, breaking a -K
dir-symlink update; current master fixes it with secure_basis_open).
- Older peers carry more xfails for options/negotiation they lack;
2.6.0 (protocol 27) fails most daemon tests. reverse-daemon-delta
passes against all peers, confirming backward compat down to 2004.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nine statically-linked, stripped binaries for the version-mixing test
suite (and ad-hoc cross-version behaviour checks): every x.y.0 release
from 2.6.0 (2004, protocol 27) through 3.4.0, plus the 3.1.3/3.2.7/3.4.1
point releases. 2.6.0 is the practical floor; older tags need more
porting to build on a current toolchain.
build_static.sh rebuilds any release from its git tag, applying the
minimal patches needed to compile old sources on a modern toolchain:
K&R lseek64 redecl, gettimeofday, -std=gnu11, --disable-openssl, and
_FORTIFY_SOURCE disabled (modern FORTIFY=3 turns latent benign over-reads
in old rsync into aborts when it runs as a server). Pre-3.0 trees ship
configure.in, so it regenerates configure (autoheader/autoconf) after
neutralizing the dead AC_LIBOBJ replacement fallbacks, generates proto.h,
and stubs the dropped vendored lib/addrinfo.h -- all guarded to no-op on
newer versions.
.gitattributes marks the binaries binary (so the text=auto rule can't
corrupt them) and export-ignore (kept out of the release tarball).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every other two-sided test drives with the current binary, covering
new-client -> old-server. This adds the backward-compat direction that
matters most for a project shipping new servers to a world of old
clients: a current daemon must keep serving the installed base of old
rsync clients.
reverse-daemon-delta_test.py starts the daemon with the current build
(via start_test_daemon's rsync_cmd override) and drives it with the old
binary. It does a push and a pull, each with and without -z, with the
receiving side pre-seeded with an older version of the file so the delta
algorithm actually runs -- exercising delta encoding both ways (old->new
on push, new->old on pull) and compression negotiation both ways. It
asserts the bytes crossing the wire are far smaller than the file, so a
silent fallback to a whole-file copy is caught, and accepts both the
modern "sent/received" and the old "wrote/read" summary wording so an
old client's output parses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Let the suite run with two rsync binaries so the current build can be
tested against the actual old code of a previous release, rather than
only forcing the current binary to speak an old protocol (check29/30).
--rsync-bin2 PATH exports RSYNC_PEER, the binary used for the SERVER
side of two-sided transfers (the daemon process and
the remote-shell --rsync-path target). Defaults to
RSYNC, so single-binary runs are byte-for-byte
unchanged.
--expect-result F the manifest's listed tests ARE the run set; each
test's actual outcome (pass/skip/fail/xfail) is
compared to its expected one and any mismatch --
including an unexpected pass (xpass) -- fails the
run. --expect-skipped and the default exit logic
are untouched.
rsyncfns gains the RSYNC_PEER global and launches the daemon with it
(start_rsyncd / start_test_daemon, the latter with an optional rsync_cmd
override used by the reverse-direction test); the remote-shell tests
pass --rsync-path={RSYNC_PEER}. All no-ops when no peer is selected.
Direction is fixed: the current binary always drives (only it
understands the new test scripts); the old binary is only ever the
server/daemon side.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some tests cannot run in certain build/CI environments. In particular the
protected-regular test self-re-execs under "unshare --map-users" to exercise
fs.protected_regular handling, and that user-namespace path hangs in a
restricted buildd chroot (e.g. Launchpad/sbuild), tripping the per-test
timeout and failing the whole "make check".
Add an --exclude option (comma-separated test names/globs), with an
RSYNC_EXCLUDE environment fallback so it can be set without touching the
make/check command line. Excluded tests are dropped before running -- they
are neither executed nor reported as skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the new ppa:rsyncproject/rsync-latest (development snapshots rebuilt
from git master) alongside the existing stable PPA in INSTALL.md and the
download page. Notes that snapshot versions (3.5.0~git...) sort below the
matching stable release, so the two PPAs can coexist without a stable
release being silently replaced by a snapshot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
when a symlink is to the same directory as the source then it can be
considered unsafe if it goes via a path outside the directory.
This came up on the mailing list, added a test to make the case clear
GitHub Actions artifact storage is approaching our quota. Each `make`/build
job uploads its rsync binary + manpages, the coverage job uploads its full
HTML tree, and Android uploads its dist/ -- 11 jobs producing artifacts per
PR/push, all kept for the repo default of 90 days.
Set retention-days: 45 explicitly on every upload-artifact step so they
expire at half the previous lifetime; older artifacts can still be re-built
from the commit if needed. No other workflow behaviour changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tests are launched with subprocess.run(..., cwd=TOOLDIR) so the
subprocess's argv[0] resolves against TOOLDIR, not the runner's
invocation cwd. A user-supplied --rsync-bin=../foo/rsync therefore
worked when invoked from inside TOOLDIR but silently failed (or
ENOENT'd inside individual tests) when invoked from a sibling
directory.
Fix: absolutize rsync_bin via os.path.abspath() at parse time, before
it propagates into build_rsync_cmd()/RSYNC. abspath() captures
os.getcwd() now, which is the operator's invocation cwd -- exactly
what the --rsync-bin=../path form expresses.
Regression check:
cd /tmp/somewhere-else
ln -s /path/to/rsync ./alt/rsync
python3 /path/to/rsync-git/runtests.py \
--rsync-bin=./alt/rsync \
--srcdir=/path/to/rsync-git --tooldir=/path/to/rsync-git \
00-hello
Before this commit the test failed at subprocess time with the relative
path being looked up under TOOLDIR; after, it passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds .github/workflows/actionlint.yml which runs rhysd/actionlint over
.github/workflows/*.yml on push and PR to master. Triggers only when
something in .github/workflows/ (or the actionlint config) changes, so
the rest of the platform matrix isn't billed when nothing here moves.
The job downloads a pinned actionlint binary (1.7.12) via the upstream
download script (which verifies a SHA256) -- no third-party Action
dependency, matching the inline-install style of the existing
ubuntu/macos/cygwin workflows. Bump the pinned version deliberately.
actionlint catches a) GitHub Actions expression / type errors, b)
unsupported runner images, c) missing secrets / inputs, and d) the
embedded shellcheck class of issues in 'run:' scripts that the previous
commit cleaned up. Keeping it in CI prevents regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
actionlint (rhysd/actionlint) reported a handful of shellcheck-class issues
across the GitHub Actions workflows. All are 1-line mechanical fixes:
* Replace legacy backticks in --rsync-bin=`pwd`/rsync with
--rsync-bin="$PWD/rsync" (SC2006 + SC2046; almalinux-8-build,
macos-build, ubuntu-22.04-build, ubuntu-build).
* Quote >>$GITHUB_PATH redirects as >>"$GITHUB_PATH"
(SC2086; coverage, macos-build, ubuntu-22.04-build, ubuntu-build).
After this commit `actionlint .github/workflows/*.yml` exits 0.
(Also cleaned up 6 editor backup *.yml~ files from the local working
tree; those weren't tracked -- *~ is gitignored -- so the cleanup is
local-only and not part of this commit.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
symlink-dirlink-basis assert the --backup file holds the pre-update content,
not merely that the backup file exists.
acls-default check that clearing the inherited default ACL actually
succeeded, so the no-default-ACL cases can't silently
test against the scratch dir's seeded default ACL.
alt-dest assert --copy-dest produces a distinct inode from the
alt-dir candidate (a copy, not a hard link) -- the
property that distinguishes it from --link-dest, which
checkit's tree comparison alone doesn't capture.
(crtimes' "independently pin the historical create time" gap is left as-is: the
touch-trick pinning is APFS-specific and not locally verifiable, and a mistuned
probe would make the test skip on macOS and break its expected-skip set.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace loose/partial oracles with exact ones:
omit-times under -O, require EVERY directory mtime to be omitted, not
just one (the old "at least one differs" missed partial bugs).
dir-sgid assert the created dirs' actual gid: a setgid parent makes
them inherit its group (set to a secondary group to be
discriminating), while the non-setgid case gets the process's.
relative-implied pin a deterministic umask and assert the exact default mode
(0o755) for --no-implied-dirs, not merely "not the source's".
safe-links / compare the preserved symlink TARGET strings via readlink,
unsafe-links not just that a symlink exists.
preallocate verify do_punch_hole via st_blocks on the --inplace --sparse
case (guarded by a sparse-capability probe).
Note: --preallocate --sparse leaves the file fully allocated on a fresh write
(the zero run is not punched), so that case stays content-only rather than
asserting hole-punching -- see the test comment; rsync.1's claim that the
combination yields sparse blocks does not hold for the fresh-write path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several tests proved only that rsync exited cleanly (or that a file merely
exists), so a no-op/short transfer would pass:
protected-regular compare the dst bytes to the source after --inplace.
00-hello re-assert one/two were copied on the RSYNC_OLD_ARGS=1
env-var path (the explicit --old-args case already did).
missing check the dry-run's exit status in test 1.
mkpath compare transferred bytes (not just existence) and add a
negative control: a transfer WITHOUT --mkpath must fail
and create no intermediate path.
size-filter compare each kept file's content to its source.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These daemon tests confirmed refusals/exclusions but accepted the allowed
transfers on exit status alone, so a transfer that exited cleanly while moving
nothing would pass:
daemon-refuse allowed() imported verify_dirs but never called it; now it
confirms the allowed push/pull actually populated the dest.
daemon-filter pull()/the incoming push ignored their exit status, and the
outgoing-chmod loop iterated only files that exist -- a
zero-file pull passed vacuously. Check the codes and require
at least one file to have been mode-checked.
daemon run_and_check's unused `expected` param is dropped; the
hidden-module and glob listings now compare the exact set of
listed paths (catching a leaked extra path), replacing the
per-path containment check and the dead normalise() helper
whose regex never matched the -r listing format anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The symlink-race tests only asserted that an outside sentinel was unchanged or
unlisted while ignoring rsync's exit status, so an attack transfer/listing that
failed before reaching the vulnerable receiver/sender path would pass without
the security property ever being exercised. Add a positive control to each --
an ordinary in-module write (bare-do-open, chdir) or an in-module listing
(sender-flist-leak) that must succeed -- so a globally broken/refusing daemon
can no longer make the sentinel checks vacuous, and assert the attack run did
not die from a signal.
clean-fname-underflow now also enforces a non-zero exit: clean_fname()
collapses "a/../test" to "test", whose merge file is absent, so rsync must
reject it; accepting it (rc 0) would mean the crafted name was mis-collapsed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several subcases ran rsync without checking the exit status, so a silent
failure could pass as the expected (often empty) output -- most notably -q,
which only asserted empty stdout. Route every expected-success run through a
helper that asserts the exit status, and verify -q actually transferred the
tree. Replace the "-h/-8 didn't break the transfer" check with positive format
assertions: -h must render byte counts with a K/M/G suffix (and the default
must not), and -8 must leave a high-bit filename byte unescaped (\#371 absent)
where the default escapes it -- best-effort, self-skipping where the platform
can't store the raw byte.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compress-options only checked that each requested algorithm yielded
byte-identical output, which proves parsing/non-corruption but not that the
advertised algorithm was actually used -- the test would pass if the choice
were silently ignored. Capture --debug=NSTR (compat.c / checksum.c) and assert
the selected compressor, compress level, and checksum match the request
(anchored so zlib != zlibx). --skip-compress / --checksum-seed stay content
checks: they have no comparable negotiation-string signal.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both fuzzy tests asserted only that the final file content matched, which a
full transfer that ignored --fuzzy would also satisfy -- so a broken fuzzy
basis selection would pass undetected. Drive rsync directly with --debug=FUZZY
and assert the generator reports the expected basis ("fuzzy basis selected
for <f>: <basis>", generator.c find_fuzzy): rsync2.c for fuzzy, and the
closest-named candidate archive-v1.tar for fuzzy-basis. fuzzy switches from
checkit() to a manual run plus verify_dirs() so the output can be captured.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rsync.1 says combining --preallocate with --sparse yields sparse blocks
wherever the filesystem can punch holes, but since 2019 (commit c2da3809,
"keep file-size 0 when possible") it has silently left the file fully
allocated. Two problems, both rooted in that commit switching --preallocate /
--inplace to fallocate(FALLOC_FL_KEEP_SIZE):
* do_fallocate() then returned 0 instead of the reserved length, so the
receiver's preallocated_len was 0 and write_sparse() always lseek'd over
null runs instead of punching them (and the over-preallocation trim in
receiver.c never fired either).
* more fundamentally, KEEP_SIZE leaves the file size at 0 while data is
written incrementally, so the FALLOC_FL_PUNCH_HOLE call lands on blocks
beyond EOF and is a silent no-op -- the reserved blocks are never freed.
Fix both: don't request KEEP_SIZE when --sparse is also active, so the file is
preallocated at full size and the punch lands within it; and return the
reserved length from do_fallocate() so preallocated_len drives the punch
decision and the over-allocation trim. --preallocate without --sparse keeps
the KEEP_SIZE (file-size-0) behaviour. t_stub.c gains a sparse_files stub since
do_fallocate now references it and the test helpers link syscall.o.
preallocate_test.py now asserts via st_blocks (where the filesystem can punch
holes) that --preallocate --sparse ends up sparse, guarding the regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The OpenBSD job runs inside a nested VM. At -j8 the --use-tcp run starts
many concurrent loopback daemons, and under that resource pressure the
daemon connection handshake occasionally loses a timing race and one test
hangs to the 300s runner timeout. It is an environment artifact, not an
rsync defect: the daemon handshake writes-then-reads with unbuffered early
I/O (no flush/mutual-wait deadlock), the indefinite wait is the documented
no-timeout daemon behaviour, and it does not reproduce off OpenBSD even with
the full suite pinned to a single CPU at -j8.
Drop just this job's --use-tcp parallelism to -j2 so the nested VM stops
over-subscribing; the pipe `make check` and every other platform are
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Target previously-uncovered functions in the path/file-operation files the
resolver restructure touches, confirmed hit under coverage:
preallocate --preallocate (syscall.c do_fallocate) and sparse hole-punching
via --preallocate --sparse and --inplace --sparse (do_punch_hole),
on a file several levels deep.
fuzzy-basis --fuzzy basis selection with similar-named candidates and no
exact match, so the generator scores them (util1.c fuzzy_distance).
delete-deep add a --backup --delete case so removing an extraneous
backup-suffixed file consults delete.c is_backup_file.
preallocate probes --preallocate support up front and skips where it is
unavailable: macOS, the *BSDs and Solaris build without fallocate/posix_fallocate
(and FALLOC_FL_PUNCH_HOLE is Linux-only), and reject the option outright. It runs
on Linux and Cygwin. fuzzy-basis and delete-deep are plain local transfers with
no skips. All green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The --expect-skipped check compared the skip list as an ordered string, so the
per-platform RSYNC_EXPECT_SKIPPED lists had to match runtests' collection order
(sorted filenames) exactly -- a subtle, easy-to-break ordering dependency.
Compare the skipped SET instead; which tests skipped is what matters.
Register the new require_tcp test daemon-access-ip in the per-platform
expected-skipped lists (it skips in the pipe-transport make check, like
daemon-chroot-acl and proxy-response-line-too-long).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The coverage report counted bundled third-party code (zlib/, popt/, and the
PostgreSQL/ISC lib/ imports getaddrinfo/getpass/inet_ntop/inet_pton) that rsync
ships but does not own, muddying the percentages. Add a COVERAGE_EXCLUDE gcovr
filter (shared by all coverage targets) so the report reflects rsync's own code:
on the same data, lines 63.9%->65.5%, functions 81.4%->85.0%, branches
55.0%->56.5% (rsync's own md5/mdfour/wildmatch/etc. stay in the report).
Add 'make coverage-all': run the suite under pipe + --protocol=30 + --protocol=29
+ --use-tcp, accumulating into the shared .gcda (not cleared between runs), then
one merged scoped report -- covers the daemon/TCP and protocol-compat paths a
single pipe run misses (lines 67.6%, functions 87.6%, branches 58.6%). Also add
'make coverage-fallback' for a separate --disable-openat2 build (different .gcno,
so it can't merge with the openat2 report). CI is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
acls-depth skips where ACLs/setfacl are unavailable (macOS, Cygwin) like the
existing acls tests, and sparse skips on APFS (macOS), where a seek-written
hole isn't allocated sparsely. Add them to the per-platform RSYNC_EXPECT_SKIPPED
lists so the skip-set assertion stays accurate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Builds with --enable-coverage and runs the suite under both transports
(make coverage, then make coverage-tcp). gcovr's line/branch/decision totals
are printed to the step log and also written to the GitHub step summary, so the
coverage numbers are visible directly in the CI output; the HTML reports are
uploaded as an artifact. make coverage exits with the suite's status, so a test
regression fails the job.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
coverage-tcp reuses the coverage recipe with --use-tcp (daemon tests over a real
loopback rsyncd, which also runs the require_tcp-only tests) and a separate
report directory, via COVERAGE_RUNFLAGS / COVERAGE_DIR. Verified end to end:
pipe run reports 63.9% lines, the TCP run 64.5% (it exercises more code).
Also drop gcovr's --branches flag: it is deprecated in gcovr 8 and branch +
decision coverage still appear in --print-summary and the HTML without it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
partial_test.py sub-test 5 deterministically asserts a delta (--no-whole-file)
resume from an absolute, outside-tree --partial-dir reproduces the source and
consumes the basis -- the regression guard for the receiver fix. Sub-test 4
keeps asserting the cross-directory partial WRITE on interrupt. Drop the
--whole-file workaround and the 'broken on master' notes in the docstring and
COVERAGE.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A delta (--no-whole-file) resume whose basis is an absolute --partial-dir
looped forever on exit code 23 ("failed verification -- update put into
partial-dir"), stranding the correct data in the partial-dir and never
populating the destination.
Cause: an absolute --partial-dir makes the basis path absolute, but the
receiver opened it with secure_relative_open(NULL, fnamecmp, ...), which by
design rejects an absolute relpath (EINVAL). The basis fd was then -1, so
receive_data() mapped no basis and (because the matched-block sum_update() is
guarded by "if (mapbuf)") computed the whole-file verification checksum over
the literal data only -> a spurious mismatch every run. (The data itself was
correct, since the in-place update leaves the matched basis bytes in place.)
Under a non-chroot daemon the in-place write went through the same call and
failed outright.
Fix: add secure_basis_open(), which treats an operator-trusted absolute basis
path as (trusted directory + confined leaf) -- the same way secure_relative_open
already trusts an absolute basedir while keeping O_NOFOLLOW on the leaf -- and
use it for both the basis read and the inplace-partial write. The strict
"reject absolute relpath" contract of secure_relative_open is left intact.
Defense-in-depth: receive_data() now treats a block-match token with no mapped
basis as a protocol inconsistency (it can only arise from a basis that the
generator opened but the receiver could not), failing cleanly instead of
silently dropping those bytes from the verify checksum or the output.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
COVERAGE.md is the living checklist mapping every CLI option (~142) and daemon
parameter (~54) to its test(s), with depth / cross-dir status and remaining
gaps, so the path-resolution restructure can see exactly what is guarded.
update_test.py closes two of the documented gaps: -u/--update (keep a newer
destination, update an older one) and --force (replace a non-empty destination
directory with a file), both at depth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two test-coverage build knobs (both behaviour-neutral by default):
--enable-coverage appends '--coverage -fprofile-update=atomic -O0' and adds
a 'make coverage' target (whole suite, run serially, then
gcovr HTML with branch + decision coverage). rsync forks
and its children exit without running the gcov atexit
flush -- the generator via its SIGUSR1 handler
(_exit_cleanup) and the receiver via the SIGUSR2 handler
-- so under GCOV_COVERAGE we call __gcov_dump() at both, or
receiver.c/generator.c record no coverage at all.
--disable-openat2 gates the Linux openat2(RESOLVE_BENEATH) sites in syscall.c
on HAVE_OPENAT2 (defined by default), so disabling it forces
the portable per-component O_NOFOLLOW resolver to run as the
primary on ordinary Linux -- exercising and
coverage-counting that fallback tier without a pre-5.6
kernel. NOTE: coordinate with the parallel syscall.c
path-resolution restructure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add resolve_beneath_supported() to rsyncfns: it functionally probes whether the
rsync binary can follow an in-tree directory symlink under its secure resolver
(an initial transfer plus a delta update through a dir-symlink, the operation
issue #715 is about). This tracks the actual binary instead of a platform name.
Use it in symlink-dirlink-basis_test.py in place of the SunOS/OpenBSD/NetBSD/
Cygwin name check: it skips on those platforms too, and additionally on
Linux < 5.6, a seccomp-blocked openat2, and the new --disable-openat2 build,
where the portable O_NOFOLLOW fallback rejects the in-tree symlink.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Breadth pass for options not yet exercised:
output-options output shape of --version/--help/-i/-n/--stats/
--out-format/--list-only/-q/--progress/-h/-8 (these control
output, not path handling, so they're checked for shape).
compare -c and -I catch a stealth change (same size+mtime, new
content) deep in the tree; --size-only skips a same-size
change; --modify-window absorbs a 1s mtime difference.
compress-options --compress-choice for every advertised compressor,
--compress-level, --skip-compress, --checksum-choice for
every advertised checksum, and --checksum-seed -- each a
clean byte-identical transfer at depth.
Green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drive a loopback daemon (secure stdio-pipe transport by default, also green
under --use-tcp) via the new write_daemon_conf helper and assert the behaviour
of the security-relevant rsyncd.conf parameters, transferring >=3-deep trees:
daemon-access path / read only / write only / list, incl. a deep sub-path
pull and that a list=no module is hidden yet usable by name.
daemon-filter daemon exclude hides matching files everywhere; incoming /
outgoing chmod rewrite modes of every transferred file.
daemon-auth auth users + secrets file accept the right password, reject a
wrong one and an unauthenticated request; strict modes rejects
a world-readable secrets file.
daemon-exec pre-/post-xfer exec run with RSYNC_MODULE_NAME /
RSYNC_EXIT_STATUS; a failing pre-xfer exec aborts the transfer
(marker files polled for, since post-xfer exec runs after the
client disconnects under TCP).
daemon-munge munge symlinks stores incoming links with the /rsyncd-munged/
prefix and strips it on the way out.
daemon-refuse refuse options: a named option, a wildcard, and the '* !a !v'
allow-list idiom.
Green on master under pipe and --use-tcp transports and under --protocol=29.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Assert exactly which entries are/aren't transferred, deep in the tree:
filter-depth --exclude/--include precedence on files at every level, and
a -F per-directory .rsync-filter loaded from a deep dir that
applies to that subtree only (not above it).
cvs-exclude -C built-in cruft patterns (*.o, *~) at every level plus a
deep per-directory .cvsignore scoped to its subtree.
size-filter --max-size / --min-size select the right files all the way
down.
files-from-depth --files-from selects only the listed deep paths (implied
parents created); --from0 NUL-delimited; --exclude-from /
--include-from filter at depth.
(--existing / --ignore-existing are covered in delete-deep_test.py.)
Green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Set each attribute distinctively on a file AND a directory at every level of a
>=3-deep tree and verify it per entry after transfer (metadata is applied as a
single-component op on an entry whose parent chain the resolver restructure
rewrites):
metadata-depth -p preserves exact file/dir modes; -t preserves file
mtimes; --chmod=D710,F600 rewrites them.
omit-times -O omits directory times (files still preserved); -J omits
symlink times.
sparse -S preserves a deep file's hole (allocated << size);
--no-sparse fills it.
xattrs-depth -X reproduces a user xattr on every entry (gated on xattr
support).
acls-depth -A reproduces a POSIX ACL on every entry (gated on ACL
support + setfacl/getfacl).
ownership-depth --groupmap and --chown=:GROUP remap the group of every
entry (non-root, to a secondary group); -o/--usermap gated
on root.
All green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cover the structure and link options at >=3 levels and across directories,
asserting each option's specific effect:
links -l keeps a symlink, -L dereferences it, -k follows a
directory symlink -- all on a symlink several levels deep.
dirs -d copies the top layer (file + empty dir) without recursing.
prune-empty-dirs -m drops empty chains and chains emptied by an exclude,
keeps populated ones.
hardlinks-deep -H preserves a hard link whose names live in different
directories at depth; without -H they become separate inodes.
delete-deep --delete removes a deep extraneous file/subtree; the four
delete-timing variants agree; --max-delete caps deletions;
--existing / --ignore-existing select/skip correctly.
relative-implied -R mirrors an implied directory's mode at depth;
--no-implied-dirs does not (proto 30+).
Green on master and under --protocol=29/30 (the --no-implied-dirs sub-case is
gated to protocol >= 30, where multi-component sender paths are accepted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fill the highest-restructure-risk gap: options that do two-directory / rename /
outside-tree work, asserted at >=3 levels deep with the aux tree kept outside
the main tree, and asserting the option's specific property rather than just
tree equality (which the ported tests already cover).
alt-dest-deep --link-dest hardlinks unchanged files (same inode), --copy-dest
copies (never links), --compare-dest omits unchanged files;
ref tree outside both src and dest.
temp-dir cross-dir temp->final rename at depth; temp dir left clean; a
missing --temp-dir fails (so the option is proven consulted).
partial --partial keeps the partial in the dest file; relative
--partial-dir stages per-directory at depth (pre-seed +
interrupt/resume); absolute --partial-dir writes the partial
outside the tree.
inplace --inplace keeps the destination inode across a delta update;
the default temp+rename path replaces it.
append --append completes truncated files tail-only; --append-verify
repairs a corrupted prefix (protocol >= 30).
backup-deep --suffix saves <name>S beside the new file; --backup-dir
relocates old files to a parallel deep tree outside the dest
and captures deletions under --delete.
All green on master and under --protocol=29/30.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add helpers for the option-coverage expansion (the path-handling restructure
changes parent-component resolution, so options must be exercised at depth and
across directory boundaries):
* make_tree() builds a tree with a regular file at every level so a property
can be checked at the tree root and >=3 levels deep;
* walk_files()/walk_dirs() iterate entries for per-level assertions;
* assert_same/assert_mode/assert_mtime_close/assert_is_symlink/
assert_hardlinked/assert_not_hardlinked/assert_exists/assert_not_exists
assert the concrete property an option controls (not just dest == src);
* write_daemon_conf() writes an arbitrary rsyncd.conf (globals + modules)
for daemon-parameter tests, beyond build_rsyncd_conf's fixed four modules;
* forced_protocol() lets protocol-sensitive tests gate sub-cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
xattr_set() sets attributes with the native os.setxattr(), but
xattr_dump() read them back by running "getfattr -d". That asymmetry
breaks "make check" on any system where rsync is built with xattr
support (libattr headers present) but the attr package's CLI tools are
not installed -- common on Android/Termux and minimal CI images: setting
succeeds via os.setxattr, then xattr_dump's getfattr raises
FileNotFoundError, which crashes the test (reported FAIL) instead of
running or skipping it. That's why "make check" was failing here on
xattrs / xattrs-hlink.
Read the xattrs natively with os.listxattr()/os.getxattr() on Linux,
symmetric with xattr_set(), so the suite needs no external getfattr; the
output still mimics "getfattr -d" and only has to be self-consistent
between the source and destination dumps. Cygwin keeps the CLI path
(Python there lacks os.*xattr). make check now passes with no attr
package installed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Python rewrite of the suite carried over the shell habit of
populating the test tree by capturing "ls -l /etc" / "ls -l /bin"
(falling back to "ls /"): hands_setup() built etc-ltr-list / bin-lt-list
that way, and longdir_test.py did the same for its leaf files. That ties
the fixtures to the host filesystem layout -- those directories are
absent or unreadable on Android/Termux and other minimal environments,
where "ls /" fails outright -- and the captured content was never
reproducible from run to run.
Add a deterministic make_text_file() helper to rsyncfns.py and use it for
hands_setup()'s two fixture files and longdir's leaf files. The names
etc-ltr-list / bin-lt-list are unchanged (chmod, chmod-temp-dir and
alt-dest reference them by name); only the content source changes, so the
fixtures are now self-contained and identical on every platform. This
also drops longdir_test.py's date(1) and ls(1) subprocess calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a link to the rsync Discord server (https://discord.gg/Avfvy9zhdp)
below the mailing lists section in README.md and on the lists.html web
page.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Python rewrite had gated the xattr / fake-super tests (xattrs,
xattrs-hlink, chown-fake, devices-fake) to Linux because it used the
Linux-only os.*xattr. Restore them on macOS, FreeBSD, Cygwin and Solaris
via a per-OS xattr surface in rsyncfns.py (xattrs_supported / xattr_set /
xattr_dump):
* Linux -- os.*xattr
* macOS -- xattr
* FreeBSD -- setextattr / lsextattr / getextattr
* Cygwin -- getfattr / setfattr (from the `attr` package; CPython on
Cygwin has no os.*xattr)
* Solaris -- runat(1), with the script on stdin and the attr name/value
passed via the environment (the runat -c form mangles args)
Test attribute names are logical; the "user." namespace prefix is added
only on the Linux-style platforms (Linux, Cygwin). RSYNC_PREFIX/RUSR vary
per OS (macOS and Solaris use rsync.nonuser to avoid rsync's reserved
rsync.* space). The macOS and Cygwin workflows no longer skip these tests;
the FreeBSD/Solaris jobs use IGNORE skip-checking so need no change.
Verified on real Linux, macOS, FreeBSD, Cygwin and Solaris hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chmod-option: pin umask to the suite-wide 022 baseline (mirroring the
old rsync.fns) so rsync's --chmod `D+w` is computed and applied under
the same umask -- fixes failures under a different ambient umask (077).
* daemon module-list test: assert the `list = no` module does NOT leak
into the listing (the substring check alone missed regressions).
* claim_ports() lock file: open with O_NOFOLLOW and only fchmod a file we
O_EXCL-created, rejecting a symlink OR hard link planted at the
well-known /tmp path -- which, with the TCP tests running under sudo in
CI, could otherwise chmod an arbitrary root-owned target. Require a
pristine (regular, nlink==1) file.
* CI: extend the Linux/Cygwin expected-skip lists for the gated tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
socketpair_tcp() fakes a connected socket pair via a loopback TCP
self-connect (socket -> bind 127.0.0.1:0 -> listen -> connect ->
accept), used by sock_exec() for RSYNC_CONNECT_PROG. Its comment has
long promised that "nobody else can attach to the socket, or if they
do that this function fails", but nothing actually verified it: the
code accept()ed whatever connection arrived first without checking it
was the one our own connect() made.
Between listen() and accept() the ephemeral loopback port is
connectable by any local user. With backlog 1 a same-host attacker who
races a connection in before our connect() lands could have their
socket returned by accept(), handing them one end of the rsync
protocol stream. The exposure is small (loopback only, random
ephemeral port, sub-millisecond window, local users only), but the
promised guarantee was simply not enforced.
Enforce it: after the connection is established, require that the peer
address of the accepted end (fd[0]) equals the local address of our
connecting end (fd[1]), and that both are 127.0.0.1. A hijacked
connection has a different source port and is rejected (errno EPERM,
fail closed). The legitimate self-connect always matches, so there is
no behaviour change for the normal path.
Verified: rebuilds clean with -Wall -W; the full testsuite still
passes in both transports (pipe `make check` 57/3, `runtests.py
--use-tcp` 59/1) -- the pipe transport exercises this code path on
every daemon test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Daemon-mode tests default to the stdio-pipe transport (RSYNC_CONNECT_PROG),
which opens no listening socket -- so `make check` never exposes a network
service. Real TCP is opt-in via `runtests.py --use-tcp`, with the daemon
bound to loopback (127.0.0.1) on a claim_ports()-reserved port; CI runs the
suite both ways.
start_test_daemon() is the single seam every daemon test uses: the secure
pipe by default, a real rsyncd on a claimed loopback port under --use-tcp.
Tests with no pipe equivalent (the fake-proxy listener and the reverse-DNS
hostname-ACL daemon test) are gated behind require_tcp().
`make check` also now runs the suite in parallel by default (CHECK_J=8);
the claim_ports() byte-range locks make that safe across concurrent runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rsyncfns.claim_ports(*ports) takes exclusive POSIX byte-range locks on
/tmp/rsync_test.lck (offset = port number) so any number of test
processes can run concurrently without colliding on a TCP port: a test
asking for a port already held blocks until the holder exits. The
kernel drops the locks automatically when the holding process dies, so
a crashed test releases its ports with no manual cleanup.
Ports are claimed in sorted order so two callers requesting the same
set in different orders can't deadlock. The lock file is forced to
mode 0o666 after creation (the umask would otherwise trim it and lock
out a second user on a shared CI runner; EPERM when we're not the
owner is fine).
proxy-response-line-too-long is the first user: it switches from an
ephemeral port to a claimed fixed port (12873).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the entire shell-based testsuite with Python. runtests.py
already drove the suite (it had replaced runtests.sh earlier); this
converts all 60 test scripts from *.test shell to *_test.py and adds
testsuite/rsyncfns.py as the shared helper module -- the Python
counterpart of the now-removed rsync.fns.
runtests.py:
* Discovers and runs both *.test and *_test.py; dispatches the
Python tests via the same python3 that runs the harness.
* Extends PYTHONPATH so tests can `import rsyncfns`.
testsuite/rsyncfns.py provides everything the ports need:
* environment wiring (scratchdir / srcdir / TOOLDIR / RSYNC /
TLS_ARGS, and HOME pointed at the per-test scratch dir);
* result reporting -- test_fail / test_skipped / test_xfail mapping
to the 0 / 1 / 77 / 78 exit-code convention;
* the transfer-and-verify helpers checkit, checkdiff, verify_dirs,
rsync_ls_lR, check_perms and the v_filt output filter;
* fixture builders hands_setup, build_symlinks, build_rsyncd_conf,
make_data_file, cp_p / cp_touch, makepath / rmtree.
All 60 tests are converted, including the four split-variant tests
that share one source via a Makefile-built symlink (chown/chown-fake,
devices/devices-fake, xattrs/xattrs-hlink, exclude/exclude-lsh);
Makefile.in's CHECK_SYMLINKS now points at the *_test.py names.
The dead rsync.fns shell library is removed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-compiles statically-linked rsync binaries with the Android NDK for
arm64-v8a (all modern phones) and armeabi-v7a (older 32-bit devices), and
uploads them as workflow artifacts for adb push / Termux use.
The build is self-contained (optional external libraries disabled; keeps
md5/md4 and the bundled zlib) and forces a few configure cache values
that can't be probed when cross-compiling: lchmod()/lutimes() off (Bionic
doesn't declare them until API 36 though the symbols link), and
socketpair / mknod-FIFO / mknod-socket on (Android runs a Linux kernel,
so these match the native result). IPv6 is enabled explicitly.
Since the binaries are cross-compiled the test suite can't run; the job
instead asserts each binary is static and the correct architecture, and
smoke-tests `--version` under qemu-user.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
symlink-dirlink-basis.test and chdir-symlink-race.test both
require a multi-kilobyte non-trivial-content source file for the
rsync delta algorithm to exercise. Both used dd / head against
/dev/urandom directly, which fails on platforms that don't ship
/dev/urandom (e.g. HPE NonStop). The dd error gets swallowed by
'2>/dev/null' and the test then fails with a misleading 'failed
to create test file' that hides the real cause.
Add make_data_file <path> <size> to testsuite/rsync.fns. Prefers
/dev/urandom when readable (kernel-provided randomness, fast),
falling back to a deterministic awk LCG seeded from PID and a
POSIX cksum of the destination path. Output is constrained to
printable ASCII (33..126) so the helper survives two awk-portability
quirks:
- printf '%c', 0 terminates the string in some awks, emitting
fewer than sz bytes;
- gawk in UTF-8 locales encodes printf '%c', N for N > 127 as
a 2-byte UTF-8 sequence, emitting more than sz bytes.
The tests don't need 8-bit binary entropy -- they just need
non-trivial bytes for rsync's block-matching algorithm.
Update both call sites to use the helper. Linux/FreeBSD/macOS
still take the /dev/urandom fast path; NonStop and any other
platform missing the device get the awk fallback transparently.
Both paths verified locally with the symlink-dirlink-basis test.
The chmod-symlink-race test was previously a no-op on Solaris,
OpenBSD, NetBSD, and Cygwin via a case 'uname -s' skip. The skip
was too broad: of the four scenarios the helper exercises, only
the 'legitimate within-tree dir-symlink' one actually needs
RESOLVE_BENEATH-equivalent kernel support. The other three
(attack rejection, plain relative path, top-level file) behave
identically on the per-component O_NOFOLLOW fallback and would
have caught the t_stub.c max_alloc=0 bug fixed in the previous
commit if the test had been allowed to run.
Make the helper probe the running kernel for either
openat2(RESOLVE_BENEATH) on Linux 5.6+ or openat(O_RESOLVE_BENEATH)
on FreeBSD 13+ / macOS 15+ by opening '.' under the requested
confinement. Honour the result:
- If RESOLVE_BENEATH-equivalent confinement is available, the
within-tree symlink scenario must succeed (status quo).
- If not, the per-component O_NOFOLLOW fallback rejects every
symlink including legitimate ones; expect the within-tree
symlink scenario to be rejected (rc != 0) and the file mode
to remain unchanged.
The attack-rejection, plain-path and top-level scenarios are
unchanged: they expect the same outcome on both code paths.
Drop the case-based skip from chmod-symlink-race.test so the test
runs everywhere and the per-component fallback gets the CI
coverage that the SunOS/OpenBSD/NetBSD/Cygwin runners can
provide. HPE NonStop -- which lacks RESOLVE_BENEATH but isn't in
the existing skip list -- is also covered by this change.
The t_stub.c shim defined max_alloc = 0 as a placeholder to satisfy
the link against util2.o. This was harmless when the test helpers
made no allocations, but the secure_relative_open() implementation
in 3.4.0+ calls my_strdup() in its per-component O_NOFOLLOW
fallback (syscall.c around line 1857), and the 3.4.3 do_*_at()
hardening series added more such calls. With max_alloc=0, every
allocation in that path trips the 'exceeded --max-alloc=0' check in
util2.c's my_alloc(), and t_chmod_secure (which exercises
do_chmod_at via secure_relative_open) fails on the very first
my_strdup.
The failure is invisible on Linux 5.6+ / FreeBSD 13+ / macOS 15+ /
recent Cygwin because those platforms take the kernel-enforced
openat2(RESOLVE_BENEATH) or openat(O_RESOLVE_BENEATH) branch and
never reach the per-component fallback. It also goes unobserved
on the SunOS/OpenBSD/NetBSD/CYGWIN* CI runners because the
chmod-symlink-race.test script case-skips on those platforms (the
legitimate dir-symlink scenario the test exercises can't pass on
the per-component fallback). HPE NonStop is the first platform
that lacks RESOLVE_BENEATH support AND isn't in the skip list AND
has someone actually running the test suite, so it surfaced the
latent bug.
Raise max_alloc to SIZE_MAX so the helpers can allocate freely.
A follow-up patch makes t_chmod_secure adapt at runtime so the
skip list can be removed and the per-component fallback gets real
CI coverage.
The .filt file in /home/ftp/pub/rsync on samba.org controls which
subtrees release.py's FTP mirror excludes (currently /binaries/
and /generated-files/). Without it, step-10-push-ftp's
'rsync --del' would propagate local deletions to the server even
for those archive subtrees.
Until now the only copy of this two-line file lived on the server.
Bundle it in source at packaging/ftp.filt so it survives a disaster
on samba.org, and have step_1_fetch seed FTP_DIR/.filt from the
bundled copy on every run (with --exclude=/.filt on the rsync pull,
so the server's copy can't silently drift the bundled one).
step-10-push-ftp then propagates any in-source updates to the
filter back to the server.
Both scripts were pre-release.py legacy helpers:
* samba-rsync rsync'd ~/samba-rsync-{ftp,html}/ to the samba.org
server. release.py step-10-push-ftp and step-11-push-html now
do exactly this, using ../release/rsync-{ftp,html}/ as the
local mirrors.
* send-news copied README/INSTALL/NEWS .md + .html files into
~/samba-rsync-ftp/ and rsync'd them to samba.org.
release.py step-8-update-ftp already does this
(./md-convert --dest=FTP_DIR README.md NEWS.md INSTALL.md and
the surrounding rsync of html files into FTP_DIR), and
step-10-push-ftp pushes the result.
Update the trailing instructions printed at the end of
step-12-push-git to drop the now-obsolete 'run packaging/send-news'
suggestion, and tighten the comment in step_1_fetch that referred
to samba-rsync as a current sibling tool.
Track the move of rsync-web from sibling git checkout to a regular
subdirectory of the rsync source tree:
* HTML_SRC: '../rsync-web' -> 'rsync-web'.
* step_1_fetch: drop the .git-presence probe and the 'make sure
it's up to date' reminder. Both made sense when rsync-web was
a separate repo the maintainer had to clone and pull, but the
directory is now part of the same checkout as this script.
* rsync invocation no longer needs --exclude=/.git: there is no
.git inside rsync-web/ (it is just a subdir of the parent
rsync-git checkout).
* Header comment block and step-1 help text rewritten to describe
the new layout.
Fold the standalone rsync-web repo into the rsync source tree as
rsync-web/, eliminating the sibling-checkout convention and the
drift it causes between the release-time HTML snapshot in
../release/rsync-html and the source of truth in ../rsync-web.
Flat-copy import (no git history merge). The standalone repo at
github.com/RsyncProject/rsync-web is retained for historical
reference and will be archived once the in-tree copy proves itself.
Add /rsync-web/ to .gitattributes with export-ignore so the
website content does not bloat the release source tarball
produced by 'git archive' in packaging/release.py step_7_tarball.
A follow-up commit repoints HTML_SRC in packaging/release.py at
the new in-tree location.
Most Ubuntu users landing on INSTALL.md want to install rsync, not
build it. Add a short section near the top that offers the
Launchpad PPA as the one-line path for the four currently supported
series (jammy 22.04 LTS, noble 24.04 LTS, questing 25.10,
resolute 26.04 LTS), and clarify that the rest of the file is about
building from source.
Drops the "dev" suffix on RSYNC_VERSION ahead of the
2026-05-20 00:00 UTC public release.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Set the date to 20 May 2026, add a SECURITY FIXES section listing
all six May 2026 CVEs (CVE-2026-29518, -43617, -43618, -43619,
-43620, -45232) with reach, root cause, fix and reporter for each,
plus a note on the defence-in-depth hardening that goes with them.
Also list the new symlink-race regression tests under DEVELOPER
RELATED.
fixes a one byte stack overflow when using RSYNC_PROXY with a
malicious proxy.
Reach: only when RSYNC_PROXY is set and a malicious or MITM'd
proxy returns the pathological response. The byte written is
always '\0' and the attacker doesn't choose the offset, so impact
is corruption of one adjacent stack byte and possible later
misbehaviour or crash -- no information disclosure beyond the
existing rprintf of buffer contents.
Reported by Aisle Research via Michal Ruprich
read_del_stats() in main.c accumulates 5 wire-supplied counts into
the int32 stats.deleted_files field:
stats.deleted_files = read_varint_bounded(..., MAX_WIRE_DEL_STAT, ...);
stats.deleted_files += stats.deleted_dirs = ...;
stats.deleted_files += stats.deleted_symlinks = ...;
stats.deleted_files += stats.deleted_devices = ...;
stats.deleted_files += stats.deleted_specials = ...;
With the previous MAX_WIRE_DEL_STAT = 2^30 (1.07 GB) the worst-case
sum is 5 * 2^30 = 5.37 GB; three maximal values already exceed
INT32_MAX = 2.15 GB on the third "+=", triggering signed integer
overflow (C99 6.5/5 -- undefined behaviour, the compiler may assume
it cannot happen and elide subsequent checks).
The bound was introduced in f0155902 ("defence-in-depth: bound
wire-supplied counts and lengths") with a commit message claiming
"per-summand cap so the total can't overflow", but 2^30 * 5 does
overflow. Lower the per-summand cap to 2^28 (= 268M) so the worst
case is 5 * 2^28 = 1.34 GB < INT32_MAX with margin. 2^28 deletions
per category is still vastly above any plausible real transfer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two assorted audit findings:
- receive_data() never bounds-checked the block index returned
by recv_token() against sum.count before computing offset2
and feeding it to map_ptr(). An out-of-bounds index from a
hostile sender produces invalid memory access. Add a
sum.count bounds check.
- read_delay_line()'s strchr() call could return NULL when no
space was found, but the code unconditionally added 1 to the
result before dereferencing. Low impact (just a disconnect on
exit of the client-specific forked process) but the NULL
deref is real. Guard the NULL.
Both reported by Joshua Rogers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cumulative-snprintf patterns in log.c (rsyserr) and main.c
(output_itemized_counts) had the shape
len = snprintf(buf, sizeof buf, ...);
len += snprintf(buf+len, sizeof buf - len, ...);
with no guard between calls. snprintf returns the would-have-been
length on truncation, so a truncated first call leaves
"sizeof buf - len" as a negative-then-promoted-to-size_t value,
underflowing into a huge size_t and writing past buf.
Realistic exposure is small in both cases (log header well under
buffer, only ~5 itemized iterations writing ~25 chars each into a
1024-byte buffer) but the defect class matches bb0a8118 and the
fix is cheap. Guard before each subsequent call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Multiple receiver-side fields read from the wire were trusted
without upper-bound checks. A hostile peer could either request
extreme allocations (DoS via --max-alloc) or, on platforms where
read_varint returned a negative value, push ~SIZE_MAX through the
size_t conversion to wrap downstream length checks.
Introduce read_int_bounded(), read_varint_bounded() and
read_varint_size() in io.c so wire-derived integer ranges are
checked at the read site rather than scattered across each
caller, with RERR_PROTOCOL on out-of-range input.
Apply the bounded primitives to:
- sum->count (checksum count -- previously could overflow
(size_t)count * xfer_sum_len on 32-bit with raised max-alloc)
- xattrs: count, name_len, datum_len, plus rel_pos overflow
detect to stop chain wrapping the num accumulator
- acls: ida-entry count
- flist: file mode S_IFMT validation, modtime_nsec range check
- delete-stat counters in main: per-summand cap so the total
can't overflow a signed 32-bit accumulator
Reporters include Joshua Rogers (checksum-count overflow finding).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On an rsync daemon configured with "daemon chroot", the reverse-DNS
lookup of the connecting client was performed *after* the chroot
had been entered. If the chroot did not contain the files glibc
needs for resolution (/etc/resolv.conf, /etc/nsswitch.conf,
/etc/hosts, NSS service modules), the lookup failed and
client_name() returned "UNKNOWN". Hostname-based deny rules
("hosts deny = *.evil.example") therefore could not match, and
an attacker controlling their PTR record could connect from a
hostname the administrator had intended to deny. IP-based ACLs
were unaffected.
Do the reverse DNS lookup before chroot/setuid; client_name()
caches its result, so the post-chroot call uses the cached value
and hostname-based ACLs work even when DNS is unavailable
post-chroot.
Adds testsuite/daemon-chroot-acl.test as end-to-end regression
coverage. The test sets up an empty chroot directory, configures
"hosts deny = <localhost-resolved-name>" with daemon chroot, and
asserts the connection is refused with @ERROR access denied.
Uses unshare --user --map-root-user for non-root CAP_SYS_CHROOT;
skips cleanly on non-Linux or when user namespaces aren't
available.
Reporter: Joshua Rogers (MegaManSec).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 797e17f ("fixed an invalid access to files array") added a
parent_ndx < 0 guard to send_files() in sender.c, but the visually-
identical block in recv_files() in receiver.c was not updated. A
malicious rsync:// server can therefore drive any connecting client
into the same out-of-bounds dir_flist->files[-1] read followed by a
file_struct dereference in f_name() one line later.
Reach: protocol-30+ default (inc_recurse) makes flist.c:2745 set
parent_ndx = -1 on the first received flist when the sender omits a
leading "." entry; rsync.c flist_for_ndx() does not reject ndx == 0
in that state because the range check evaluates 0 < 0 = false; and
read_ndx_and_attrs() only validates ndx with the ITEM_TRANSFER bit
set, so iflags=ITEM_IS_NEW (or any other non-transfer iflag word)
bypasses the check.
Apply the same guard receiver-side. Confirmed: the same PoC (a
minimal Python rsyncd that handshakes with CF_INC_RECURSE, sends a
no-leading-"." flist, and emits ndx=0 with ITEM_IS_NEW) crashes
unpatched 3.4.2 with SEGV_MAPERR si_addr=0x4101a-class in the
receiver child; with this guard it exits cleanly with code 2
(RERR_PROTOCOL).
The attack surface delta over the sender variant is large:
the original was malicious-client -> daemon, this is
malicious-server -> any rsync client doing a normal rsync://
or remote-shell pull.
Reported by Pratham Gupta (alchemy1729).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a daemon-refuse-compress test that builds a module configured with
'refuse options = compress' and asserts that:
1. an attempted -z transfer to that module fails with an error
mentioning --compress, and
2. the same transfer without -z still succeeds.
This pins down the documented way to disable all compression on a
daemon, which previously had no automated coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The receiver's three compressed-token decoders --
recv_deflated_token (zlib), recv_zstd_token, and
recv_compressed_token (lz4) -- accumulated rx_token (a 32-bit
signed counter) without overflow checking. A malicious sender
could craft a compressed-token stream that walked rx_token past
INT32_MAX, with careful manipulation leaking process memory
contents to the wire (environment variables, passwords, heap
pointers, library pointers -- significantly weakening ASLR
and facilitating further exploitation).
Cap rx_token at MAX_TOKEN_INDEX = 0x7ffffffe. Fold the
bookkeeping into recv_compressed_token_num() and
recv_compressed_token_run() shared by all three decoders. Reject
negative or out-of-range token values explicitly. Also cap the
simple_recv_token literal-block length at the source: any
wire-supplied length > CHUNK_SIZE is ill-formed (the matching
simple_send_token never writes a chunk larger than CHUNK_SIZE),
so reject before looping on attacker-controlled bytes.
Reach: an authenticated daemon connection with compression
enabled (the default for protocols >= 30 when both peers
advertise it). Disabling compression on the daemon
("refuse options = compress" in rsyncd.conf) is the available
workaround.
Reporter: Omar Elsayed (seks99x).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cygwin lacks RESOLVE_BENEATH-equivalent kernel support and the
per-component O_NOFOLLOW fallback also can't be exercised meaningfully
under the cygwin runner's filesystem semantics, so every test that
asserts the secure_relative_open / do_*_at machinery actually blocks
the attack would skip. Make those skips expected in the workflow's
RSYNC_EXPECT_SKIPPED list:
- chdir-symlink-race
- chmod-symlink-race
- bare-do-open-symlink-race
- sender-flist-symlink-leak
- daemon-chroot-acl
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
testsuite/chdir-symlink-race.test runs an actual rsync daemon
(via RSYNC_CONNECT_PROG to avoid the network) configured with
"use chroot = no", plants a symlink at module/subdir -> ../outside,
and runs four flavours of attacker-shaped transfer (single-file
poc_chmod, -r push into the symlinked subdir with --size-only and
without, -r push into the module root). All four must leave the
outside-the-module sentinel file's mode AND content unchanged.
Portability:
- file_mode() helper falls back to BSD stat -f %Lp when GNU
stat -c %a is unavailable (macOS, FreeBSD).
- Pre-saved pristine copy + cmp(1) replaces sha1sum, which
differs across platforms (sha1sum / shasum / sha1).
Tests are kept running as root in the user-namespace re-exec
wrapper used by symlink-race tests so the daemon's setuid path
doesn't drop into the test user's identity (which on Linux
would mean the chmod-escape code path can't trigger because
the test user doesn't have CAP_FOWNER over the outside file).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three related codex audit findings:
Finding 3a: copy_file()'s source open in util1.c used
do_open_nofollow(), which only rejects a final-component
symlink. A parent-component symlink (e.g. --copy-dest=cd where
cd -> /outside) follows freely and reads outside the module.
Route through secure_relative_open() with O_NOFOLLOW.
Finding 3b: generator.c's in-place backup-file create still
used a bare do_open with O_CREAT, leaving a tiny but reachable
parent-symlink window between the secure unlink (already
through do_unlink_at) and the create. Add do_open_at() that
goes through a secure parent dirfd, and route the call site
through it.
Finding 3c: copy_file()'s destination open in
unlink_and_reopen() had the same bare-do_open pattern; route
through do_open_at as well.
Adds testsuite/copy-dest-source-symlink.test and
testsuite/bare-do-open-symlink-race.test as regression coverage
for both attack shapes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the rest of the path-based syscall wrappers and migrate every
receiver-side caller:
- do_lchown_at, do_rename_at, do_mkdir_at, do_symlink_at,
do_mknod_at, do_link_at, do_unlink_at, do_rmdir_at,
do_utimensat_at, do_stat_at, do_lstat_at
Same shape as do_chmod_at: open each parent under
secure_relative_open(), call the *at() variant against the dirfd,
fall through to the bare path-based syscall in non-daemon /
chrooted / absolute-path / no-parent cases. macOS's
setattrlist-based set_times tier is also routed through the
utimensat_at path on daemon-no-chroot.
Hardenings to secure_relative_open() itself:
- confine basedir resolution under the same kernel mechanism
used for relpath (basedirs from --copy-dest / --link-dest are
sender-controllable in daemon mode)
- reject any '..' component (bare '..', 'foo/..', 'subdir/..')
so the per-component O_NOFOLLOW fallback can't escape
- return the dirfd we built up from the per-component fallback
when the caller passed O_DIRECTORY (otherwise every do_*_at
failed with EINVAL on platforms without RESOLVE_BENEATH)
Adds testsuite/alt-dest-symlink-race.test and
testsuite/secure-relpath-validation.test (with t_secure_relpath
helper) as regression coverage for the new hardenings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The receiver's chdir(2) into a destination subdirectory followed
attacker-planted symlinks at every path component. Once CWD
escaped the module, every subsequent path-relative syscall (open,
chmod, lchown, ...) inherited the escape -- defeating
secure_relative_open's RESOLVE_BENEATH anchor against AT_FDCWD,
since the anchor itself was now outside the module.
Route change_dir's relative target through secure_relative_open()
and fchdir() to the resulting dirfd in am_daemon && !am_chrooted
mode, so the chdir step itself can no longer follow a parent-
symlink. Same treatment applied to the CD_SKIP_CHDIR /
set_path_only path so it also can't follow attacker symlinks
during path tracking.
Adds testsuite/sender-flist-symlink-leak.test covering the
sender-side flist resolution variant of the same primitive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CVE-2026-29518's fix routed the receiver's open() through
secure_relative_open(), but every other path-based syscall the
receiver runs on sender-controllable paths is vulnerable to the
same TOCTOU primitive. This commit closes the chmod variant.
Add do_chmod_at() that opens the parent of fname under
secure_relative_open() and uses fchmodat() against the resulting
dirfd. Gate the secure path on am_daemon && !am_chrooted (the same
gate use_secure_symlinks already uses for the receiver basis-file
open), so non-daemon callers and chrooted daemons keep the original
do_chmod() fast path.
Migrate the receiver-side do_chmod() call sites in delete.c,
generator.c, rsync.c, and xattrs.c.
Adds testsuite/chmod-symlink-race.test (with t_chmod_secure helper)
as regression coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sender's file open was vulnerable to the same TOCTOU symlink
race as the receiver-side basis-file open. change_pathname() calls
chdir() into subdirectories, which follows symlinks; an attacker
could race to swap a directory for a symlink between the chdir and
the file open, allowing reads of privileged files through the
daemon.
Reconstruct the full relative path (F_PATHNAME + fname) and open
via secure_relative_open() from the trusted module_dir, which
walks each path component without following symlinks. This is
independent of CWD, so the chdir race is neutralised.
CVE-2026-29518.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CVE-2026-29518: an rsync daemon configured with "use chroot = no"
is exposed to a TOCTOU race on parent path components. A local
attacker with write access to a module can replace a parent
directory component with a symlink between the receiver's check
and its open(), redirecting reads (basis-file disclosure) and
writes (file overwrite) outside the module. Under elevated daemon
privilege this allows privilege escalation. Default
"use chroot = yes" is not exposed.
Add secure_relative_open() in syscall.c. It walks the parent
components under RESOLVE_BENEATH (Linux 5.6+) /
O_RESOLVE_BENEATH (FreeBSD 13+, macOS 15+) / per-component
O_NOFOLLOW elsewhere, anchored at a trusted dirfd, so a parent-
symlink swap is rejected by the kernel. Route the receiver's
basis-file open in receiver.c through it when use_secure_symlinks
is set in clientserver.c rsync_module().
Reporters: Nullx3D (Batuhan SANCAK); Damien Neil; Michael Stapelberg.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The default python3 on AlmaLinux 8 is 3.6, but runtests.py uses
subprocess.run(capture_output=...) and check_output(text=...) which
were introduced in 3.7. Install the python39 module stream and point
/usr/bin/python3 at it via alternatives so the existing shebang
resolves correctly.
Reproduced as: TypeError: __init__() got an unexpected keyword
argument 'capture_output' at runtests.py line 75.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The intent is to validate that future security fixes still build and
test cleanly on the oldest still-supported LTS releases of the two
mainstream Linux families, so backports can be developed against the
same CI surface as the trunk:
- ubuntu-22.04: oldest GitHub Actions runner image still available
(20.04 was retired in April 2025). Mirrors the existing
ubuntu-build.yml step list.
- almalinux-8: RHEL 8 rebuild, full support until 2029. Runs in an
almalinux:8 container on ubuntu-latest because GHA has no native
runner for the Fedora/RHEL family. Pulls libzstd/xxhash/lz4 dev
headers from PowerTools + EPEL; commonmark via pip for the man
page generator.
Both jobs follow the same paths-ignore convention as the other
workflows so a workflow-only change to one file won't fan out across
the whole CI matrix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Use unshare with user namespace UID mapping to run the
protected-regular test without real root privileges. Falls back
to skipping if unshare or uidmap is not available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The test correctly skips on Cygwin (which lacks RESOLVE_BENEATH), but
the workflow's RSYNC_EXPECT_SKIPPED list still treats any change in
the skipped set as a CI failure. Add the new test name so the
skipped/got comparison matches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
secure_relative_open() has a kernel-enforced "stay below dirfd" path
on Linux 5.6+ (openat2 RESOLVE_BENEATH) and FreeBSD 13+ (openat
O_RESOLVE_BENEATH). On Solaris, OpenBSD, NetBSD, and Cygwin the code
falls back to the per-component O_NOFOLLOW walk, which by design
rejects every directory symlink in the path -- the very case this
test exercises. Mark the test skipped there rather than have it
fail with a known regression that's tracked separately.
macOS is intentionally not in the skip list: although it does not
have O_RESOLVE_BENEATH either, the test passes there in practice;
investigation of the underlying reason is left as follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FreeBSD and MacOS have O_RESOLVE_BENEATH as an openat() flag with the same
"must not escape dirfd" semantics as Linux's RESOLVE_BENEATH. The
kernel rejects ".." escapes, absolute symlinks, and symlinks whose
target lies outside dirfd, while still following symlinks that
resolve within it -- the same trade-off that fixes issue #715 on
Linux.
Add a parallel BSD path in secure_relative_open(), gated on
declared. Unlike Linux, BSD doesn't have the header/runtime split
where the symbol can exist without kernel support, so no runtime
fallback is needed: if the flag compiles in, the kernel honours it.
OpenBSD and NetBSD have no equivalent kernel primitive and continue
to use the existing per-component O_NOFOLLOW walk; issue #715
remains visible on those platforms (a userland resolver or
unveil(2)-based fence would be follow-up work).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CVE fix in commit c35e283 made secure_relative_open() walk every
component of relpath with O_NOFOLLOW. That blocks every symlink in the
path, which is stricter than the threat model required: legitimate
directory symlinks within the destination tree (e.g. when using -K /
--copy-dirlinks) are also rejected, breaking delta transfers with
"failed verification -- update discarded". See issue #715.
On Linux 5.6+, openat2(RESOLVE_BENEATH | RESOLVE_NO_MAGICLINKS) gives
us exactly what we want: the kernel rejects any resolution that would
escape the starting directory (via "..", absolute paths, or symlinks
pointing outside dirfd) while still following symlinks that resolve
within it. /proc magic-links are blocked too.
Use openat2 first; fall back to the existing per-component O_NOFOLLOW
walk on ENOSYS (kernel < 5.6). The lexical "../" checks at the head
of the function are kept as defense in depth. The Linux gate is
plain #ifdef __linux__: the runtime ENOSYS fallback covers the only
case that actually matters (header present + old kernel), and any
Linux build environment without linux/openat2.h will fail with a
clear "no such file" error rather than silently disabling the
protection.
Verified manually that openat2(RESOLVE_BENEATH) blocks all four
escape patterns (absolute symlink, ../ symlink, lexical .., absolute
path) while allowing direct and within-tree symlinks. The new
testsuite/symlink-dirlink-basis.test (taken from PR #864 by Samuel
Henrique) exercises the issue #715 regression and passes; full
make check passes 47/47.
Test: testsuite/symlink-dirlink-basis.test (8 scenarios)
Fixes: https://github.com/RsyncProject/rsync/issues/715
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Solaris xls() function listed every entry in the file's xattr
directory, which on Solaris includes OS-managed SUNWattr_ro and
SUNWattr_rw pseudo-attributes. SUNWattr_rw embeds the file creation
time, so its bytes naturally differ between the source and destination
files, making the xattrs and xattrs-hlink tests fail with diffs that
have nothing to do with rsync.
Rsync's own listxattr wrapper already filters these out
(lib/sysxattrs.c), so the right fix is to filter them in the test
display too. Other platforms are unaffected because each has its own
xls() branch in the case statement.
With the test now actually passing on Solaris, drop the CI hack that
overwrote testsuite/xattrs.test with a skip stub.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror the existing FreeBSD workflow for OpenBSD and NetBSD using
vmactions/openbsd-vm and vmactions/netbsd-vm so we get cross-BSD
coverage on push, PR, and the nightly schedule.
Also extend the FreeBSD and Solaris workflows to actually exercise the
test suite by running 'make check' after the build. The Linux, macOS,
and Cygwin jobs already did this.
The Solaris xattrs and xattrs-hlink tests are removed before 'make
check' because the Solaris SUNWattr_ro / SUNWattr_rw system attributes
leak into the test diff; that's a real rsync-on-Solaris issue to follow
up on, but skip the tests for now so the suite goes green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When invoked directly (rather than via 'make check'), runtests.py
previously left the user with a wall of confusing "not found" errors
from inside individual test scripts if the CHECK_PROGS helpers had not
been built. Detect this up front and point the user at the make
target that builds them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
localtime/localtime_r need /etc/localtime for timezone info.
After chroot this file is inaccessible, causing log timestamps
to fall back to UTC. Calling tzset() before chroot ensures the
timezone data is cached by glibc for subsequent calls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sorted() call reordered skipped test names alphabetically,
causing CI expected-skipped mismatches (e.g. acls,acls-default
instead of acls-default,acls). Sort by original test order instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add parallel test execution using concurrent.futures. With -j8 the
test suite completes in ~4s vs ~29s sequential (~7x speedup).
Also fix two issues that caused failures under parallel execution:
- rsync_ls_lR now prunes testtmp/ so parallel tests don't see each
other's temp files when scanning the source tree
- clean-fname-underflow.test now uses $scratchdir instead of /tmp
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite the test runner in Python with proper command-line options
including --valgrind which directs valgrind output to per-process
log files so it doesn't interfere with test output comparisons.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Glibc 2.43 added C23 const-preserving overloads to various string functions,
which change the return type depending on the constness of the argument(s).
Currently this leads to warnings from calls to strtok() or strchr().
Fix this by properly declaring the respective variable types.
Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Change my_alloc() to use calloc instead of malloc so all fresh
allocations return zeroed memory. Also zero the expanded portion
in expand_item_list() after realloc, since it knows both old and
new sizes. This gives more predictable behaviour in case of bugs
where uninitialised or stale memory is accidentally accessed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
this fixes the count passed to the sort of the xattr list. This issue
was reported here:
https://www.openwall.com/lists/oss-security/2026/04/16/2
the bug is not exploitable due to the fork-per-connection design of
rsync, the attack is the equivalent of the user closing the socket
themselves.
The len field in the proxy v2 header was declared as signed char,
allowing a negative size to bypass the validation check and cause
a stack buffer overflow when passed to read_buf() as size_t.
This bug was reported by John Walker from ZeroPath, many thanks for
the clear report!
With the current code this bug does not represent a security issue as
it only results in the exit of the forked process that is specific to
the attached client, so it is equivalent to the client closing the
socket, so no CVE for this, but it is good to fix it to prevent a
future issue.
The bundled zlib 1.2.8 used K&R-style function definitions which are
rejected by clang 16+ as hard errors. Convert all 90 functions across
9 files to ANSI-style prototypes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The new simd-checksum test is skipped on platforms where SIMD
instructions are unavailable (macOS ARM, Cygwin). Add it to the
RSYNC_EXPECT_SKIPPED lists so CI doesn't fail on the mismatch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The AVX2 get_checksum1_avx2_64() read mul_one before initializing it,
which is undefined behavior. Replace the cmpeq/abs trick with
_mm256_set1_epi8(1) to match the SSSE3 and SSE2 versions.
Add a TEST_SIMD_CHECKSUM1 test mode that verifies all SIMD paths
(SSE2, SSSE3, AVX2, and the full dispatch chain) produce identical
results to the C reference, across multiple buffer sizes with both
aligned and unaligned buffers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Validate that token numbers read from compressed streams are
non-negative. A negative token value would cause the return value
of recv_*_token() to become positive, which callers interpret as
literal data length, but no data pointer is set on this code path.
While this only causes the receiver to crash (which is process-isolated
and only affects the attacker's own connection), it's still undefined
behavior.
Reported-by: Will Sergeant <wlsergeant@gmail.com>
The static buf1 pointer was only allocated when len > len1, but on
first call with len == 0, this condition is false (0 > 0), leaving
buf1 NULL when passed to memcpy().
Fixes#673
this was found by Calum Hutton from Rapid7. It is a real bug, but
analysis shows it can't be leverged into an exploit. Worth fixing
though.
Many thanks to Calum and Rapid7 for finding and reporting this
If poptGetContext returns NULL, perhaps due to OOM,
a NULL pointer is passed into poptReadDefaultConfig()
which in turns SEGVs when trying to dereference it.
This was found using https://github.com/sahlberg/malloc-fail-tester.git
$ ./test_malloc_failure.sh rsync -Pav crash crosh
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Int32x32To64 macro internally truncates the arguments to int32,
while time_t is 64-bit on most/all modern platforms.
Therefore, usage of this macro creates a Year 2038 bug.
This can happen when the tests are unable to `stat(2)` some files in
`/etc`, `/bin`, or `/`, due to Unix permissions or other sandboxing. We
still guard against serious errors, which use exit code 2.
In 2015, the attr/xattr.h header was fully removed from upstream attr.
In 2020, rsync started preferring the standard header, if it exists:
https://github.com/RsyncProject/rsync/pull/22
But the fix was incomplete. We still looked for the getxattr function in
-lattr, and used it if -lattr exists. This was the case even if the
system libc was sufficient to provide the needed functions. Result:
overlinking to -lattr, if it happened to be installed for any other
reason.
```
checking whether to support extended attributes... Using Linux xattrs
checking for getxattr in -lattr... yes
```
Instead, use a different autoconf macro that first checks if the
function is available for use without any libraries (e.g. it is in
libc).
Result:
```
checking whether to support extended attributes... Using Linux xattrs
checking for library containing getxattr... none required
```
Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>
The test was added in dc34990, it turns out that it's flaky. It failed
once on the Debian build infra, cf. [1].
The problem is that the command `rsync -aH '$fromdir/sym' '$todir'`
updates the mod time of `$todir`, so there might be a diff between the
output of `rsync_ls_lR $fromdir` and `rsync_ls_lR $todir`, if ever rsync
runs 1 second (or more) after the directories were created.
To clarify: it's easy to make the test fails 100% of the times with this
change:
```
makepath "$fromdir/sym" "$todir"
+sleep 5
checkit "$RSYNC -aH '$fromdir/sym' '$todir'" "$fromdir" "$todir"
```
With the fix proposed here, we don't use `checkit` anymore, instead we
just run the rsync command, then a simple `diff` to compare the two
directories. This is exactly what the other `-H` test just above does.
In case there's some doubts, `diff` fails if `sym` is missing:
```
$ mkdir -p foo/sym bar
$ diff foo bar || echo KO!
Only in foo: sym
KO!
```
I tested that, after this commit, the test still catches the `-H`
regression in rsync 3.4.0.
Fixes: https://github.com/RsyncProject/rsync/issues/735
[1]: https://buildd.debian.org/status/fetch.php?pkg=rsync&arch=ppc64el&ver=3.4.1%2Bds1-1&stamp=1741147156&raw=0
atime of source files could sometimes be overwritten
even though --open-noatime option was used.
To fix that, optional O_NOATIME flag was added
to do_open_nofollow which is also used to open regular
files since fix:
"fixed symlink race condition in sender"
Previously optional O_NOATIME flag was only in do_open.
From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1093201:
Whilst working on the Reproducible Builds effort [0], we noticed that
rsync could not be built reproducibly.
This is because the date in the manual page can vary depending on
whether there is a .git directory and the modification time of version.h
and Mafile, which might get modified when patching via quilt.
A patch is attached that makes this use SOURCE_DATE_EPOCH, which
will always be reliable.
when we open a file that we don't expect to be a symlink use
O_NOFOLLOW to prevent a race condition where an attacker could change
a file between being a normal file and a symlink
Replace unsafe generic function pointer cast with proper type cast for
qsort() comparison function. This fixes a potential type mismatch
warning without changing the behavior.
Signed-off-by: Charalampos Mitrodimas <charmitro@posteo.net>
rsbackup (https://github.com/ewxrjk/rsbackup) uses "ssh <host> true" to
check that the host in question is reachable. I like to configure my
backed-up hosts to force the backup system to go via `rrsync`, but I
always have to add a local tweak to allow `SSH_ORIGINAL_COMMAND=true` to
work. I think this would be safe enough to include in rrsync.
Clang rightfully complains about conflicting prototypes, as both lseek() variants
are redefined:
syscall.c:394:10: warning: a function declaration without a prototype is deprecated
in all versions of C and is treated as a zero-parameter prototype in C2x, conflicting
with a previous declaration [-Wdeprecated-non-prototype]
off64_t lseek64();
^
/usr/include/unistd.h:350:18: note: conflicting prototype is here
extern __off64_t lseek64 (int __fd, __off64_t __offset, int __whence)
^
1 warning generated.
The point of the #ifdef is to build for the configured OFF_T; there is
no reason to redefine lseek/lseek64, which should have been found
via configure.
Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Clang rightfully complains about invoking bomb(..) without a proper prototype:
lib/pool_alloc.c:171:16: warning: passing arguments to a function without a prototype
is deprecated in all versions of C and is not supported in C2x [-Wdeprecated-non-prototype]
(*pool->bomb)(bomb_msg, __FILE__, __LINE__);
^
1 warning generated.
Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Building with clang-16 complains with:
./simd-checksum-x86_64.cpp:204:25: warning: passing 1-byte aligned argument to
16-byte aligned parameter 1 of '_mm_store_si128' may result in an unaligned pointer
access [-Walign-mismatch]
Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Since 05278935 (- Call mkdir_defmode() instead of do_mkdir(). - Define
orig_umask in this file, not options.c. - Made orig_umask a mode_t, not an
int., 2006-02-24), the type for the global was changed, and therefore on
systems where sizeof(mode_t) != sizeof(int), writes or reads to them will
overflow to adjacent bytes.
Change the type to the one used everywhere else and avoid this problem.
While at it, silence again a warning that is being triggered by
Apple's clang 15.
- Put sum2_array into sum_struct to hold an array of sum2 checksums
that are each xfer_sum_len bytes.
- Remove sum2 buf from sum_buf.
- Add macro sum2_at() to access each sum2 array element.
- Throw an error if a sums header has an s2length larger than
xfer_sum_len.
- Change the developer flow to not require updating the git-version repo
that the builds used to download a git-version.h file. The Actions now
do a full repo fetch so that the .h file can be generated via the git
history.
- Get rid of the gensend Makefile target that was used for the above.
- Get rid of the pre-push git hook file that called "Make gensend".
- Change the FreeBSD build to save an artifact with its built binaries.
[buildall]
Fixing this warning escalated to an error, resuting in no IPv6 support:
```
configure.sh:7679: checking whether to enable ipv6
configure.sh:7718: clang -o conftest -g -O2 -DHAVE_CONFIG_H -Wall -W conftest.c >&5
conftest.c:73:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
main()
^
int
1 error generated.
configure.sh:7718: $? = 1
configure.sh: program exited with status 1
```
Fortified (-D_FORTIFY_SOURCE=2 for gcc) builds make strlcpy() crash when
its third parameter (size) is larger than the buffer:
$ rsync -FFXHav '--filter=merge global-rsync-filter' Align-37-43/ xxx
sending incremental file list
*** buffer overflow detected ***: terminated
It's in the exclude code in setup_merge_file():
strlcpy(y, save, MAXPATHLEN);
Note the 'y' pointer was incremented, so it no longer points to memory
with MAXPATHLEN "owned" bytes.
Fix it by remembering the number of copied bytes into the 'save' buffer
and use that instead of MAXPATHLEN which is clearly incorrect.
Fixes#511.
The exclude.test file continues to run local copies (which are a special
kind of "push") while the exclude-lsh.test symlink runs a a "pull" using
the lsh.sh script as the "remote" shell.
When conversing with a protocol 29 or earlier rsync, the modtime values
are arriving as 4-byte integers. This change interprets these short
values as unsigned integers, allowing the time that can be conveyed to
range from 1-Jan-1970 to 7-Feb-2106 instead of the signed range of
13-Dec-1901 to 19-Jan-2038. Given that we are fast approaching 2038,
any old-protocol transfers will be better served using the unsigned
range rather than the signed.
It is important to keep in mind that protocol 30 & 31 convey the full
8-byte mtime value (plus nanoseconds), allowing for a huge span of time
that is not affected by this change.
- The sanitize_paths variable was set too often. It only needs to be set
when the "inner" path is not "/". This change avoids sanitizing &
munging things for a path=/ module just because chroot is off.
- The default for "use chroot" is now "unset" instead of "true". When
unset it checks if chrooting works, and if not, it proceeds with a
sanitized copy instead of totally failing to work. This makes it
easier to setup a non-root rsync daemon, for instance. It will have
no effect on a typical Linux root-run daemon where the default will
continue to use chroot (because chrooting works). A config file can
explicitly set "use chroot = true | false" to force the choice.
- Try to improve the "use chroot" manpage.
When using the --fuzzy option to try and find close matches locally,
the edit distance algorithm used is O(N^2), which can get painful on
CPU constrained systems when working in folders with tens of thousands
of files in it.
The lower bound on the calculated Levenshtein distance is the difference
of the two strings being compared, so if that difference is larger than
the current best match, the calculation of the exact edit distance between
the two strings can be skipped.
Testing on the OpenSUSE package repo has shown a 50% reduction in the CPU time
required to plan the rsync transaction.
The main purpose of the SHA checksums are to allow the daemon auth code
to pick a stonger digest method when negotiating the auth digest to use.
However, the SHA digests are also available for use in file checksums,
should someon really want to use one of them.
The new digests are listed from strongest to weakest at the start of the
daemon auth list, giving them the highest priority.
The new digests are listed from weakest to strongest near the end of the
checksum list, giving them the lowest priority of use for file
checksums.
- Size flist checksum data to hold the active size, not the max.
- Add a negotiated hash method to the daemon auth code.
- Use EVP for all openssl digests. This makes it easy to add more
openssl digest methods and avoids deprecation warnings.
- Support a way to re-enable deprecated digests via openssl conf
file and allow a default file to be configured.
- Supply a simple openssl-rsync.cnf file to enable legacy digests.
A local_server copy now includes the dev+ino info from the destination
file so that the sender can make sure that it is not going to delete
the destination file. Fixes mistakes such as:
rsync -aiv --remove-source-files dir .
- Avoid implied rules on generator and (with extra certainty) on server
- Add -R implied-directory path elements as directory includes
- Log about extra file-list checking using a new --debug=FILTER3 level
Some systems apparently put strlcpy() into a separate bsd/strings.h file
without putting the function into a separate library. Thus, configure
finds that the function exists for linking but the build does not have
the declaration (which rsync only supplies if it is also supplying its
own version of the function).
When pedantic errors are enabled, SIGNED_CHAR_OK was no longer
being set correctly. This would cause the checksum code to use
"char" instead of "signed char", and if the default for a "char"
was unsigned, the checksum code would fail to compute the right
hash values. Fixes bug #317.
Solve the following problems:
* mishandling of commit message lines similar to committer lines
* UnicodeDecodeError with commit messages that cannot be interpreted
as utf-8
* add tests to exercise copy_file
* Extract new function unlink_and_reopen from copy_file
The argument `ofd` to copy_file is always set to -1 unless
`open_tmpfile()` is called at generator.c:909
This change
* removes assignment to a function argument
* renames argument `ofd` to `tmpfilefd` in line with existing uses
* extracts a new function `unlink_and_reopen` which is static to util1.c
* rewrites header comments for copy_file
- Make the SIMD ASM code off by default. Use configure --enable-simd-asm
to enable.
- Allow MD5 ASM code to be requested even when OpenSSL is handling MD4
checksums. Use configure --enable-md5-asm to enable.
- Add link targets for all option choices, not just the first one.
- Tweak cross-link arg format.
- Add more links, including some in the latest NEWS.
- Split out a few numbered lists.
- Improve NEWS heading's link targets using version info.
- Optimize regex compilation.
- Make sure every link target is unique.
- Allow link targets to start with a number.
If the double "remain" value is so large that it overflows an int, make
the estimated seconds output as :00 instead of :-8. Similar for the
estimated remaining minutes. Support larger hours values.
The new default is to protect args and options from unintended shell
interpretation using backslash escapes. See the new `--old-args` option
for a way to get the old-style splitting. This idiom was chosen over
making `--protect-args` enabled by default because it is more backward
compatible (e.g. it works with rrsync). Fixes#272.
The `--stop-at`, `--stop-after`, and `--time-limit`` options should have their
limit checked when receiving and sending data, not just when receiving.
Fixes#177.
The compression level of the first file in the transfer no longer sets
the level for all files that follow it. Document that per-file level
switching has no current effect (except for a global "dont compress = *"
rule in the daemon).
- Convert rrsync to python.
- Enhance security of arg & option checking.
- Reject `-L` (`--copy-links`) by default.
- Add `-munge` and `-no-del` options.
- Tweak the logfile line format.
- Created an rrsync man page.
- Use `configure --with-rrsync` if you want `make install` to install
rrsync and its man page.
- Give lsh more rrsync testing support.
- rsync-no-vanished now avoids joining stdout & stderr, avoids affecting
a non-client run, and gets the rsync status code correctly.
- rsync-slash-strip now avoids affecting a non-client run.
The Linux fs.protected_regular sysctl setting could cause rsync to fail to write a file in-place with the O_CREAT flag set, so the code now tries an open without O_CREAT when it might help to avoid an EACCES error. A testsuite script is included (and slightly improved by Wayne to ensure that it outputs a SKIP when fs.protected_regular is turned off).
Some hosts were not running `mkgitver` when they should, so tweak the
script to only update the timestamp when the file's data changes and
then always run the script when performing a build.
In 2004, an allocation optimization has been added to the file
list handling code, that preallocates 32k of file_struct pointers
in a file_list. This optimization predates the incremental
recursion feature, for which it is not appropriate anymore. When
copying a tree containing a large number of small directories,
using the incremental recursion, rsync allocates many short
file_lists. Suddenly, the unused file_struct pointers can easily
take 90-95% of the memory allocated by rsync.
- Make SIMD & ASM only default to enabled on linux for now (due to
FreeBSD & MacOS issues).
- Improve the enable/disable help messages so that they don't look
wrong when the opposite --enable-X/--disable-X arg is specified.
- use `grep -E` and `grep -F` (`egrep` and `fgrep` are non-standard)
- use same hashbang style for all test scripts
- use explicit comparisons in test scripts
- remove redundant ; from test scripts
- make test script not executable, just like all the other scripts
- unify codestyle across all test scripts
- make openssl license exception clearer by having it at the top
- use modern links in COPYING. The text now matches:
https://www.gnu.org/licenses/gpl-3.0.txt
- fix typo
* Eventually add write permission when setting extended attributes
When we need to set extended atributes of file which does not
allow write then temporarily add write permission and after
attributes are set, remove it again.
Resolves#208
Co-authored-by: Wayne Davison <wayne@opencoder.net>
Legacy configure behaviour was to detect IPv6 support through known IPv6
capable version of common standard libraries. Now: it runs a POSIX test
to determine if IPv6 is usable (in case it has not been disabled).
Patch originally from Pierre-Olivier Mercier <nemunaire@nemunai.re>.
Signed-off-by: Jonathan Davies <jpds@protonmail.com>
Without a DISPLAY var, ssh won't try to forward X11 when making an
ssh connection. This patch also makes use of setenv() and unsetenv()
if they are available.
Since a non-cygwin gmake trips up the github cygwin action, let's just
require that the user put a good "make" early on their path (a simple
`ln -s `which gmake` ~/bin/make` with the right $PATH works fine).
Replace runtime SIMD check with a compile-only test in case of
cross-compilation.
You can still use '--enable-simd=no' to build x86_64 code without
SIMD instructions.
- Use -pedantic-errors with gcc to make an array-init fatal.
- Fix all the extra warnings that gcc outputs due to this option.
- Also add -Wno-pedantic to gcc if we're using the internal popt
code (since it has lots of pedantic issues).
- Rename unchanged_file() to quick_check_ok().
- Enhance quick_check_ok() to work with non-regular files.
- Add a get_file_type() function to the generator.
- Use the new functions in the generator code to make the logic simpler.
- Fix a bug where the `--alt-dest` functions were not checking if a
special file fully matched the non-permission mode bits before
deciding if we have found an alt-dest match.
- Enhance the `--info=skip --ignore-existing` output to include extra
info on if the existing file differs in type or passes the standard
quick-check logic.
- Add `--info=skip2` that authorizes rsync to perform a slow checksum
"quick check" when ignoring existing files. This provides the uptodate
and differs info even if we need to checksum a file to get it.
For a non-git build or for a git build w/o adequate git history, we now
allow the git-version.h file to be provided before the build. If the
file does not exist, we either create an empty file or put a define of
RSYNC_GITVER in it based on the output of git describe. The github
builds now snag the git-version.h file that was generated for the last
commit so that they all get the same version string, even with a shallow
checkout.
Change the logic in compat.c to construct the client_info string value
for a local copy so that the various checks of the string don't need to
make an exception for local_server.
If the receiving side read a hard-linked device, it needs to set the
value of rdev_major to the value it snags from the hard-linked data
because the sender set their rdev_major value for that file entry.
- Rename daemon_over_rsh -> daemon_connection since it is also used to
indicate if a non-rsh daemon connection is active.
- Move the daemon-over-rsh exception out of server_options() to the one
caller that needs that behavior.
- Don't allow noop_io_until_death() to be short-circuited when talking
to a daemon over a socket, because it can't send errors via stderr.
This is based on the long-standing patch but with the protocol changed
to just use newlines as delimiters instead of null chars (since names
should not contain a newline AND it makes it easier to write a helper
script). Lots of other small improvements and a better default value
for "numeric ids" when using "use chroot" with "name converter".
- Use strdup(do_big_num(...)) to replace num_to_byte_string(...).
- Allow a ',' for a decimal point in a SIZE option in some locales.
- Get rid of old (now unused) strdup() compatibility function.
- Try harder to include the newline in a single error message write.
- Use C99 flexible arrays when possible, and fall back on the existing
fname[1] kluges on older compilers.
- Avoid static initialization of a flexible array, which is not really
in the C standard.
The code now derives all the struct defines, default value assignments,
parser-param defines, and lp_foo() accessor functions from a single list
of daemon parameters.
The release script & the patch management script now require the use of
an auto-build-save dir that makes it much easier to keep the generated
files from melding together, and remembers the configure setup for each
patch branch.
We now put the configure.sh, config.h.in, and aclocal.m4 files in the
alternate build dir along with the other generated files. This requires
that we create symlinks for configure.ac & m4 in the build dir, which is
handled on the first run of configure or prepare-source. I also changed
the patch-branch handling away from the .gen-stash dir to an automatic
build/$PATCH subdir idiom that will keep each branch's configuration
separated. These automatic build dirs are only used when there is a
.git dir, a build/master dir, and no top-dir Makefile. You'll also
want to have package/make early on your path for optimal ease of use.
- All the memory-allocation macros now auto-check for failure and exit
with a failure message that incudes the caller's file and lineno
info. This includes strdup().
- Added the `--max-alloc=SIZE` option to be able to override the memory
allocator's sanity-check limit. It defaults to 1G (as before).
Fixes bugzilla bug 12769.
The client does not pass "none" as a negotiation choice unless it's from
the user's environment list. The server still passes the "none" value
to the client unless its environment var excludes it.
- The env on the server side now affects the negotiated strings
that are sent to the client.
- A too-old remote rsync gets a default negotiated string value
so that an env restriction now handles old clients the same way
as new ones.
When building out of source tree, we can't find rsync-ssl in the current
directory and installation fails. Fix it by using the srcdir variable for the
path to rsync-ssl.
Signed-off-by: Hiroshi Takekawa <sian.ht@gmail.com>
- Only use the asm code if we're on x86_64.
- More changes to decouple asm from simd.
- Check if the -Wa,--noexecstack option works.
- Support --disable-asm configure option.
- Switch .s -> .S to enable the preprocessor.
- Move some defines from mdigest.h to md-defines.h.
- Tweak the asm file to use md-defines.h.
- Add a couple missing .h dependencies in the Makefile.
The Mach-O x86-64 model doesn't seem to support ".type" and
".size" directives in assembly. Add ifdefs that should allow for
the file to build without issues in Apple Clang.
xattr headers have been provided by glibc (at least on Linux/glibc)
for many years now. Reorder the inclusion of xattr headers to
attempt compatibility/legacy after the common case.
This prevents the warning without changing compatibility to
non-glibc systems.
* Add dependency on lib/sysxattrs.h header in Makefile
Co-authored-by: Wayne Davison <wayne@opencoder.net>
- In html, use css more for non-breakability.
- In nroff, mark more dashes as non-breaking in code->bold sections,
and get rid of backslashed dashes in preformatted blocks.
Using a non-breaking zero-width char after a dash makes the browser
avoiding breaking on that dash and also makes it match a dash in a
search. This is better than a non-breaking dash char, which does not
match a dash in a search.
* x86-64 SIMD build fixes
configure.ac was modified to detect g++ >=5 and clang++ >=7. Additionally
some script malfunctions on FreeBSD were corrected.
The get_checksum1() code has been modified to fix clang and g++ 10
compilation.
This version of the code and configure.ac has been tested on:
Ubuntu 16 - gcc 7.3.0, clang 6.0.0
Debian 10 - gcc 5.4.0, 6.4.0, 7.2.0, 8.4.0, 9.2.1, 10.0.1, clang 5.0.2,
6.0.1, 7.0.1, 8.0.0, 9.0.0, 10.0.0
ArchLinux 20200605 - gcc 10.1.0, clang 10.0.0
FreeBSD 12.1 - gcc 9.3.0, clang 8.0.1
It is unknown if it will work on gcc 5.0-5.3, but the script currently
allows it.
- Change default_cvsignore char[] into a define.
- Make the DEFAULT_DONT_COMPRESS and DEFAULT_CVSIGNORE defines get set
based on their info in rsync.1.md.
- Add a few more don't-compress suffixes from Simon Matter.
- Stop setting the mtime on a file we didn't transfer (or didn't verify
the checksum) when the time diff is within the modify window.
- Stop computing a time difference (-1|0|1) when all we care about is
time equality.
This removes the yodl dependency, which is sometimes hard to track down.
Instead, this uses a python3 script that leverages the cmarkgfm library
to turn the source file into html. Then, the script parses the html in
order to turn the tag stream into a nroff stream using a simple state
machine. While it's doing that it also implements one added format rule
that turns an ordinal list that starts at 0 into a description list
(since markdown doesn't have an easy description list idiom).
Add the OpenSSL license exception also to the COPYING file which
contains the license related information.
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Allow the receiver to increase their iobuf.msg xbuf if it fills up. This
ensures that the receiver will never block trying to output a message,
and thus it will always drain the data from the sender and keep the
whole thing from clogging up.
- Some manpage changes to make them more consistent and to add a section
that the release script expects in rsync-ssl.
- Fixed some issues in release-rsync pertaining to various file changes,
such as the .md file changes.
- Change the gpg handling to stop prompting for a passphrase since gpg
now makes use of gpg-agent (and the old gpg script is apparently not
passing through fd 2 that git needs to get status).
- Set am_daemon to -1 (from 1) when the daemon is run via rsh.
- Only disable --msgs2stderr for a normal (socket) daemon.
- Forward a -q to the server if --msgs2stderr was also specified.
- Added --no-msgs2stderr option to allow it to be overridden.
- Make "len" parameter of do_punch_hole an OFF_T.
- Clear sparse_past_write in sparse_end(), otherwise when write_sparse()
is called for the next file, do_punch_hole() will be called with a pos
that's not actually the current position in file, causing it to fail.
Previously files were hashed in blocks of CSUM_CHUNK (64) bytes. This
causes significant overhead. The CSUM_CHUNK define cannot be changed as
md5.c depends on it, but there is no obvious reason to use it in
file_checksum(). By using CHUNK_SIZE (32 kB) instead, in some test
cases throughput more than doubles.
- Add the zlibx (external-code compatible) compression name.
- Re-enable zlib support with the external library so it can be
tried as a fallback if zlibx isn't available.
- Add --compress-choice=STR (aka -zz=STR) option.
- Make --cc=STR an alias for --checksum-choice=STR.
- Hook up the new compression negotiation logic.
- Add/improve --debug=CSUM2 messages.
- Add an "xxh64" alias for "xxhash" name because we should be
getting a few more xxhash variations in the future.
- Tweak the matching code to handle entries that have multiple
names.
- Tweak some of the vars/defines.
Avoid a newline issue during the output of --DEBUG=CSUM info from
both the server and the client -- we need to output the full message
with its newline as much as possible.
Originally created by Marc Bevand and placed in the public domain.
Enable/disabled via the same --enable-simd configure switch as
the rolling checksum optimizations.
Additionally restructures build switches and defines from SSE2 to SIMD,
to allow potential reuse should patches become available with SIMD
instructions for other processor architectures.
(Some minor tweaks of Jorrit's patch to avoid requiring GNU make and to
avoid C++ comments in .c files.)
- Add checksum negotiation to the protocol so that we can easily add new
checksum algorithms and each will be used when both sides support it.
- Increase the size of the compat_flags value in the protocol from a
byte to an int.
Fix the code that writes the options and the default destination path
into the batch.sh file to be able to handle options being specified
after source/dest args.
Requires compilation using GCC C++ front end, build scripts have been
modified accordingly. C++ is only used when the optimization is enabled
(g++ as compiler, x86-64 build target, --enable-sse2 is passed to
configure).
(Wayne made a few tweaks, including making it disabled by default.)
If both sides support the "V" compatibility flag, we send the file-list
flags as a varint instead of a 1-or-2 byte value. This upgrades the
number of reserved flag bits from 1 to 17 with very few extra bytes in
typical file-list data.
- Make the rsync-ssl default behavior more user friendly.
- Install rsync-ssl & rsync-ssl-rsh in the regular install rules.
- Add a manpage for rsync-ssl (which is also installed).
- Get rid of the rsync-ssl-client package in our spec file.
The new rsh-ssl-rsync helper script (replacing stunnel-rsync) supports
openssl in addition to stunnel. The RSYNC_SSL_TYPE environment variable
can be set to specify which type of connection to use, and the first arg
to rsync-ssl can be --type=stunnel or --type=openssl to override the env
var or the default of "stunnel". The helper script now looks for
stunnel4 or stunnel on the PATH at runtime instead of having configure
look for it at compile time.
I replaced git-set-file-times with an improved version that I wrote
recently (in python3). A new script uses it to figure out the
last-modified year for each *.[ch] file and updates its copyright.
It also puts the latest year into the latest-year.h file for the
output of --version.
The default value of the skip-compress list actually comes from the
daemon's default lp_dont_compress() value, but a while back the vars
stopped getting default values in a non-daemon run. I added a call to
reset_daemon_vars() so that the "Vars" values get set from "Defaults".
Add a flag for calling get_dirlist() and for send_directory() that
indicates that the dirname is allowed to not be a directory. Based
on a patch by Ben Rubson. Fixes bug #13445.
On BSD-ish systems you can type Ctrl+T to see the current file and
the progress output (in --info=progress2 format). On hosts w/o
SIGINFO, use something like "killall -VTALRM rsync" or a more
targetted "kill -VTALRM PID ..." call (as needed).
This is a fleshed out version of the old one in the patches repo with
documentation & proper handling of the implied --inplace option for a
daemon's option-rufusing considerations. I ommitted the -w short option
as I would hate for someone to turn this on accidentally.
This can be used by a root-run rsync to try to make reading or writing
files safer in a situation where you can't run the whole rsync command
as a non-root user.
If the alternate-destination code was scanning multiple alt dirs and it
found the right size/mtime/checksum info but not the right xattrs, it
would keep scanning the other dirs for a better xattr match, but it
would omit the unchanged-file check that needs to happen first.
In parse_arguments when --protect-args is encountered the function exits
early. The caller is expected to check protect_args, and recall
parse_arguments setting protect_args to 2. This patch prevents the
client from resetting protect_args during the second pass of
parse_arguments. This prevents parse_arguments returning early the
second time before it's able to sanitize the arguments it received.
This allows the daemon to run chrooted as any uid+gid you like
(prior to the transfer possibly changing the chroot and/or the
uid+gid further). Based on the patch in #12817.
This patch avoids inconsistent evaluation of options in the
show_filelist_p() function by turning it into a var. We
also avoid setting "output_needs_newline" if --quiet was
specified.
The new code tries to punch holes in the destination file using newer
Linux fallocate features. It also supports a --whole-file + --sparse +
--inplace copy on any filesystem by truncating the destination file.
As a testcase I've used one directory on gpfs with 1000000 files,
each with an xattr called 'name$i' having a value of 'value$i'.
So we also have 1000000 unique xattrs. The source and dest directories
are already in sync before. So the rsync command is basically a noop,
just verifying that everything is already in sync.
The results before this patchset are:
[gpfs]# time rsync -a -P -X -q source-xattr/ dest-with-xattr/
real 8m46.191s
user 6m29.016s
sys 0m24.883s
[gpfs]# time rsync -a -P -q source-xattr/ dest-without-xattr/
real 1m58.462s
user 0m0.957s
sys 0m11.801s
With the patchset I got:
[gpfs]# time /gpfs/rsync.install/bin/rsync -a -P -X -q source-xattr/ dest-with-xattr/
real 2m4.150s
user 0m1.917s
sys 0m17.077s
[gpfs]# time /gpfs/rsync.install/bin/rsync -a -P -q source-xattr/ dest-without-xattr/
real 1m59.534s
user 0m0.924s
sys 0m11.599s
It means the time in userspace dropped from 6m29.016s down to 0m1.917s!
Without -X we get ~ 0m0.9s with or without the patch.
Part of a patchset for bug 5324.
We have the global 'item_list rsync_xal_l', this maintains an array
of rsync_xa_list structure, one per file.
Each rsync_xa_list structure maintains an array of rsync_xa structure,
while each represent a single xattr with name and value.
Part of a patchset for bug 5324.
The abbreviated-xattr code can get requests that are not in the same
order as the xattr list, so we need to support wrap-around scanning
of the available xattrs. Fixes bug 6590.
This patch adds the ability to specify --modify-window=-1 (aka -@-1) to
ask rsync to compare files with the full nanosecond timestamps. The
default is still -@0 for the moment, which ignores nanoseconds in time
comparisons. Changing the default to -1 would cause a copy from ext4 to
ext3 to constantly compare as different, or a copy there and back again
to do a full copy as it zeroed all the nanosecond times. Such a change
might be too much of a functional difference for things like backup
solutions to handle without a warning period. The current plan is to
support nanosecond comparisons for those that want them, and possibly
change the default window value from 0 to -1 at some point in the
future.
The try_dests_reg() function could sometimes tweak the stat struct's
info when it should have been left unchanged. This fixes bug 11545
(where an ACL check of a file that was mistakenly thought to be a
directory failed).
The %b and %c escapes were outputting cumulative values when logged via
--log-file only (the bug didn't affect daemon transfer logging or the
output of the client's --out-format info). Also unified the %b & %c
switch case to make it easier to maintain. Fixes bug 11496.
If the receiving side cannot hard-link symlinks and/or special files
(including devices) then we now properly handle incoming hard-linked
items (creating separate identical items).
I added a compatibility flag for protocol 31 that will let both sides
know if they should be using the xattr optimization that attempted to
avoid sending xattr info for hardlinked files. Since this optimization
was causing some issues, this compatibility flag will ensure that both
sides know if they should be trying to use the optimization or not.
Adding new-style compression that only compresses the literal data that
is sent over the wire and not also matching file data that was not sent.
This new-style compression is compatible with external zlib instances,
and will eventually become the default (once enough time has passed that
all servers support the --new-compress and --old-compress options).
NOTE: if you build rsync with an external zlib (i.e. if you specified
configure --with-included-zlib=no) you will ONLY get support for the
--new-compress option! A client will treat -z as uncompressed (with a
warning) and a server will exit with an error (unless -zz was used).
If the receiver gets a filename with a leading slash (w/o --relative)
and/or a filename with an embedded ".." dir in the path, it dies with
an error (rather than continuing). Those invalid paths should never
happen in reality, so just reject someone trying to pull a fast one.
If the receiver is running without --relative, it shouldn't be receiving
any filenames with a leading slash. To ensure that the sender doesn't
try to pull a fast one on us, we now make flist_sort_and_clean() strip a
leading slash even if --relative isn't specified.
I'm backing out the xattr optimization that was put in to try
to make xattr data sending more optimal on hard-linked files.
The code was causing hard-to-reproduce bugs, and it's better to
get things done fully & correctly over fully optimally.
The make_path() utility function was not returning the right status
when --dry-run was used, so I added some stat() checking that only
happens for -n. I also noticed that the function was not handling
the case where the whole path needed to be created, so I fixed that.
Fixes bug 10209.
If the client is the sender and it is wanting to log deletes, the
current generator code neglects to send MSG_DELETED to the client side
unless some delete verbosity is enabled. With this new version on the
generator side, the logfile will now mention deletes, even if the
sending (client) side is an older rsync. Fixes bug 10182.
When running with --*-dest & -X, some alt-dest-found files would not
use the right name when looking up old attrs in itemize(), causing a
weird error for a --dry-run copy. Fixes bug 10238.
This fix avoids the sending of keep-alive messages from the receiver
to the sender when we are still sending the file list (at which time
an older rsync would die if it received such a keep-alive message).
The messages aren't actually needed, since we haven't forked yet, and
the single flow of data keeps the procs alive.
Fix a problem where sparse_seek could get left non-zero when we
did not finish writing all the data that would take us to that
sparse gap. Issue pointed out by David Taylor.
When checking a checksum that refers to a part of an --inplace file that
has been overwritten w/o getting SUMFLG_SAME_OFFSET set, we remove the
checksum from the list. This will speed up files that have a lot of
identical checksum blocks (e.g. sequences of zeros) that we can't use
due to them not getting marked as being the same. Patch provided by
Michael Chapman.
Added the client rsync-ssl script and various client/daemon support
files needed for talking to an rsync daemon over SSL on port 874 (no
tls support). This uses an elegant stunnel setup that was detailed
by dozzie (see the resources page) now that stunnel4 has improved
command-spawning support. Also incorporates some tweaks by devzero
(e.g. the nice no-tmpfile-config client-side code) and a few by me
(including logging of the actual remote IP that came in to the
stunnel process). This probably still needs a little work.
The cleanup code will try to flush the output buffer in some
circumstances, which is not valid if we're handling an async signal
(since it might have interrupted some partial I/O in the main thread).
These signals now set a flag and try to let the main I/O handler take
care of the exit strategy. Fixes a protocol error that could happen
when trying to exit after a kill signal.
- If iconv() returns EINVAL or EILSEQ and the error is being ignored, make
sure that there is room in the output buffer to store the erroneous char.
- When accepting an erroneous char, be sure to break if there are no more
input characters (without calling iconv() with a zero input length).
When a daemon is sent multiple request args, they are now joined into a
single return value (separated by spaces) so that the RSYNC_REQUEST
environment variable is accurate for any "pre-xfer exec". The values
in RSYNC_ARG# vars are no longer truncated at the "." arg, so that all
the request values are also listed (separately) in RSYNC_ARG#.
The sender no longer allows a filelist to be sent in the middle of
parsing an incoming message, so that the directory sending doesn't block
all further input reading. The generator no longer allows recursive
reading of info/error messages when it is waiting for the message buffer
to flush. This avoids a stack overflow when lots of messages are coming
from the receiver and the sender is not reading things quickly enough.
The I/O code now avoids sending debug messages that could mess up the
I/O buffer it was in the middle of tweaking. This fixes an infinite
loop in reduce_iobuf_size() with high levels of debug enabled. Several
I/O-related messages were changed to output only when --msgs2stderr is
enabled.
If we have the attropen() function, allow OS conditional enabling of
extended attribute support. This removes the need to pass
--enable-extended-attributes to force the feature activation on Solaris.
- Drop one leading '.' from the filename (before adding our own).
- Drop one trailing '.' from a (possibly truncated) name prior to
the .XXXXXX suffix being added.
- Allow the temp-name to collapse to just the .XXXXXX suffix
if the path is long enough to require that.
Note that we don't try to remove multiple dots from a filename
that actually has multiple consecutive dots, since we might as
well learn early if the final name is going to fail or not.
The code now avoids any special internal meaning for uid/gid -1, which
allows it to be mapped to a better value (use 4294967295 instead of -1
as the ID to map). Replaced atol() with something than can return a
value > 0x7FFFFFFF and that will error-out if the value overflows. If
chown() is called with a uid or gid of -1, complain that the ID is not
settable and signal a transfer error. Fixes bug 6936.
The I/O code can receive incremental file-list chunks during deletion,
and their OPT_EXTRA fields would get corrupted when file_extra_cnt is
incremented.
Instead of temporarily enabling uid_ndx to find out whether the user
owns a file, have make_file() set a flag for that purpose.
Applied with a few minor tweaks by Wayne. Fixes bug 7936.
Based on a patch by Matt, but further tweaked to deal with -q=foo.
Ultimately this should be upstreamed, but for now lets get this
functionality into rsync.
Rsync was showing the full destination path, which was confusing because
nothing is created at that path and was especially bogus in combination
with the source name of a solo file.
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=506830
- The receiver now sends keep-alive messages to the generator
when it is actively doing work and hasn't sent anything
recently. This ensures that the generator won't timeout
if the receiver is working hard.
- The perform_io() code has improved keep-alive participation.
- Allow the sender to send some keep-alive messages, which
ensures that if it is in a lull, it can probe the socket.
The receiving side also switches timeout handling from the receiver to
the generator, which obviates the need for the sender to send any
keep-alive messages at all (for protocol 31 and beyond). Given this
setup, all keep-alive messages are now sent as empty MSG_DATA messages,
with MSG_NOOP messages only being understood and (when necessary) acted
upon to forward a keep-alive event to an older receiver. This is both
safer and more compatible with older versions.
- The receiver notifies the generator if it is exiting with an error,
and then, if it is a server, waits around for the generator to die.
This ensures that the client side has time to read the error.
- The generator or sender will notifiy the other side of the transfer of
an error-exit value if protocol 31 is in effect. This will get rid of
some "connection unexpectedly closed" errors that are really expected
events due to a fatal exit on the other side.
Files-from data is now sent as multiplexed I/O so that it can mingle
with any messages (such as debug output). Requires protocol 31.
Protocol 31 no longer disables output verbosity in a couple instances
that used to cause protocol issues.
Got rid of MSG_* messages that have implied raw data that follows after
them. We instead send a negative index value as a part of the raw data
stream, which is guaranteed to be output together with the following
data. This only affects the (in-progress) protocol 31 and the (self-
contained) communication stream from the receiver to the generator.
Added --debug=IO and improved --debug=FLIST. Some --debug=IO output
requires --msgs2stderr to be used to see it (i.e. sending a message
about sending a message would send another message, ad infinitum).
If a symlink, device, special-file, or hard-linked file is replacing
an existing non-directory, the new file is created using a temporary
filename and then renamed into place. Also changed the handling of
a cluster of hard-linked symlinks/devices/special-files to always
ensure the first item in the cluster is correct, since it doesn't
really save any significant work to try to find an existing correct
item later in the cluster to link with.
- Improve function name: parse_rule -> parse_filter_str (to make the
similarity with parse_filter_file clearer, and better indicate that
it can parse multiple rules when FILTRULE_WORD_SPLIT is specified).
- In preparation for rule prefixes containing information beyond the
rflags, change the code to pass around a full "template" filter_rule
instead of just rflags. Callers of parse_filter_{str,file} that want
to specify only rflags can use rule_template(rflags) .
- Remove the MODIFIERS_* strings and instead hand-code the condition
under which each modifier is valid. This should make it easier to
see that the conditions are correct.
- Tighten up default modifiers on merge rules:
- Disallow "!" because it isn't useful.
- If the merge rule specifies a side via "s" or "r", the rules in the
file cannot also specify a side via "s", "r", "hide", etc.
[Patch was changed by Wayne a bit prior to application.]
Since the value is not needed, protocol 31 no longer sends it, while
older protocols are optimized so the sender just sends a valid rdev
value as efficiently as possible. The receiver no longer caches an
rdev value for special files, and the generator will always pass a 0
rdev value to do_mknod() for special files. Fixes bug #6280.
- Changed get_backup_name() to verify the backup path, and make any
missing directories. This avoids accidental use of a symlink as a dir
in a backup path, and gets rid of any other non-dirs that are in the
way. It also avoids the need for various operations to retry after
calling make_bak_dir(), simplifying several pices of code.
- Changed create_directory_path() to make_path(), giving it flags that
lets the caller decide if it should skip a leading slash or drop the
trailing filename.
- Mention when we create the backup directory, so the user is not caught
unaware when rsync uses a directory they didn't expect.
- Got rid of some dir-moving backup code that is not used.
- Added a little more backup-debug output.
- Implement --ignore-missing-args.
- In the absence of --*-missing-args, a missing source arg is an
FERROR_XFER, but doesn't need to be an IOERR_GENERAL.
- Revise the man page.
recursion scan is still active. Mention the output change more
prominently in the NEWS file. Updated the --progress output in
the manpage, with mention of the new "ir-chk" string's meaning.
This avoids a problem where an extra message from the sender
could give the generator time to start sending data that will
not be understood by the sender's use of read_msg_fd().
Initialize both the Globals and Locals back to their default values
when reading the config. This fixes a bug where locals set in the
global section were not getting reset to their default value if the
config item was removed from the file.
- Backups do not interfere with an atomic update (when possible).
- Backing up a file will remove a directory that is in the way
and visa versa.
- Unify the backup-dir and non-backup-dir code in backup.c.
- Improved the backup tests a little bit.
This fixes an issue with -K noticed by eric casteleijn, avoids some
inconsistent itemizing when a file/dir is replaced by a dir/file,
and removes a now-obsolete chunk of code from make_file().
- Mention how many files were created (protocol >= 29).
- Mention how many files were deleted (new in protocol 31).
- Follow the file-count, created-count, and deleted-count
with a break-out list of each count by type.
- Free a mergelist's parent_dirscanned filters the last time it is
popped or as soon as the filters are discarded due to the "n"
modifier. Aside from not leaking memory, this is needed to clean up
any mergelists defined during the parent_dirscan to avoid crashing by
trying to restore nonexistent state for them in pop_local_filters.
- Make push_local_filters save the current mergelist_cnt, and make
pop_local_filters assert that it has the correct number of mergelists
before restoring their state.
- Assert that mergelists get deactivated in strict LIFO order to catch
any glitches as soon as they happen. Free linked lists of filters in
reverse order to make that the case.
- Add a bunch of mergelist-related debug output (--debug=filter2).
mention the protocol number have the right value, that the check-in date
for a protocol-change release is specified, and that a pre-release with
a protocol change doesn't have SUBPROTOCOL_VERSION set to 0. Prompt for
releasing a branch if -b option was not used and we're on a branch.
list that is transferred. This fixes a glitch where a failed
--iconv conversion on the receiving side could prevent deletions
from happening in the root-dir of the transfer.
- Make sure that handle_partial_dir() never returns a truncated fname.
- Make robust_rename() return that it failed to do a cross-device
copy if the partial-dir could not be created.
(making pools aligned by default). Added the missing code to make the
documented behavior of pool_free() with a NULL addr work. Updated the
pool_alloc.3 manpage.
option to the server (which is only useful for protocols 30 and above
anyway). This gives the user an easy way to talk to a restricted
server that has overly restrictive option-checking.
(instead of the start) in order to be extra sure that an error won't
overwrite it. We also ensure that the progress option can't be enabled
on the server side.
[if grep sys/sysmacros.h conftest.err >/dev/null; then
ac_cv_header_sys_types_h_makedev=no
else
ac_cv_header_sys_types_h_makedev=yes
fi],
[ac_cv_header_sys_types_h_makedev=no])
])
if test $ac_cv_header_sys_types_h_makedev = no; then
AC_CHECK_HEADER(sys/mkdev.h,
[AC_DEFINE(MAJOR_IN_MKDEV, 1,
[Define to 1 if `major', `minor', and `makedev' are
declared in <mkdev.h>.])])
if test $ac_cv_header_sys_mkdev_h = no; then
AC_CHECK_HEADER(sys/sysmacros.h,
[AC_DEFINE(MAJOR_IN_SYSMACROS, 1,
[Define to 1 if `major', `minor', and `makedev'
are declared in <sysmacros.h>.])])
fi
fi
])
Loaded 100 of 755 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Andrew Tridgell