* Add --delay-symlinks to create symlinks after all files are in place
The generator creates a symlink as soon as it reaches it in the file
list, while regular files arrive later through the sender and receiver
(and, with --delay-updates, are only renamed into place at the end).
For the rest of the transfer a new symlink can point at a file that
isn't there yet, so anything reading the destination during the sync --
e.g. a loader resolving libfoo.so -> libfoo.so.2 -- can fail.
The new --delay-symlinks option makes the generator defer new and
changed symlinks and create them once the receiver reports that every
file, including any --delay-updates renames, is in place. Each link is
itemized and logged after it has been created, before the generator
ends the delay-updates phase, and the deletion stats are sent after any
directory that a deferred link replaces.
The option implies --no-inc-recursive on the receiving side and is
passed to a remote receiver, which must also support it. It is an
error to combine it with --remove-source-files, --hard-links,
--compare-dest, --copy-dest, or --link-dest, or to use it with a
protocol older than 29. support/rrsync accepts the option and refuses
it under -no-overwrite, as it does --delay-updates.
Add delay-symlinks_test.py, which holds a push partway through the
link's target to check that new and changed links aren't created early,
and rrsync-no-overwrite-delay-symlinks_test.py.
* Document --delay-symlinks limits and test how it is passed to the server
The man page now says the option narrows the inconsistency window rather
than making the update atomic, that an interrupted transfer leaves new
symlinks missing and changed ones at their old targets, and that
--delete-before can also remove an old target early.
Add delay-symlinks-remote_test.py. Through a remote shell that records
the server command, it checks that the option is passed to a remote
receiver on a push but not to a remote sender on a pull, and that
incremental recursion is off in both directions. It also checks that
--protocol=28 fails with the protocol error, and that a push to an older
rsync (old_versions/rsync_3.4.1) is refused before anything is
transferred while a pull from it works.
* testsuite: seed delay-symlinks fixture with rsync
* NEWS: document --delay-symlinks
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: group file-list and directory counts
Adds the thousand separators as file-list is computed. Useful on large
dirtrees:
Behavior before: 2523297101 files to consider
After: 2,523,297,101 files to consider (2.523.297.101 with a comma decimal point)
Also covers the file-list counter, "created N directories" and --max-delete.
* NEWS: record human-readable output changes
Use O_PATH on Linux for directory descriptors used only for path
traversal, fchdir(), or as *at() parents. Reopen final directory
endpoints with the caller-requested access mode and retain the existing
O_RDONLY fallback elsewhere.
Add coverage for exact sources and readable descendants beneath
search-only directories, known-file creation beneath write/search-only
destinations, and retained refusal to enumerate unreadable directories.
Fixes#1064.
CAN_HARDLINK_SYMLINK and CAN_HARDLINK_SPECIAL are decided by configure running
linkat() on whatever filesystem the build tree happened to sit on. The
destination is free to disagree, and one host can hold both answers: macOS
builds on APFS, which can hard-link a symlink, and backs up to HFS+, which
returns ENOTSUP.
A build that said yes had no fallback left. try_dests_non() reported the
refusal as a transfer error and returned a matched basis, so the caller created
the entry anyway -- correctly -- and the run still exited 23. Every
neighbouring case copes: a regular file whose link() fails goes to try_a_copy,
and a build compiled without either macro resorts to --copy-dest behaviour.
This was the same situation, discovered a little later, and the only one
treated as fatal.
Take the existing fallback on any refusal. Singling out the "cannot" errnos is
not possible: link(2) documents EPERM both for a filesystem with no hard-link
support and for an ordinary permission refusal, and FUSE reports ENOSYS for the
same thing. It is also what the regular-file path next door has always done
(try_dests_reg -> hard_link_one -> try_a_copy), and consistency between the two
was the point. Where the errno does matter the surrounding transfer says so
anyway: ENOSPC, EDQUOT and EROFS fail the creation independently, EMLINK and
EXDEV mean the link was never possible. EIO alone goes unremarked; reporting
it would put a line into --link-dest's itemised output, so it is left out on
purpose.
Returning -3 rather than -2 keeps the caller out of the "already up to date,
skip it" arm, which under --link-dest would drop the entry entirely. Both
callers give -3 the treatment the compile-time fallback already gets -- clearing
itemizing and code -- because try_dests_non() has itemised the match itself and
would otherwise report the entry twice.
The fallback is silent, matching a build that cannot link these at compile
time; documented under --link-dest instead.
The ENOSYS test was the wrong discriminator. It was meant to detect "this
build compiled no fd-relative create at all", but a live mknodat() or
mkfifoat() returns ENOSYS too -- an unimplemented FUSE mknod does, and
seccomp can synthesise it -- so a runtime failure could route a create
through the unconfined path on a platform that has the secure primitive.
On Linux the fall-through lands in do_mknod_at(), which re-confines with
secure_relative_open(), so no escape was reachable there. The real gap is
a mixed-capability build (mkfifoat() but no mknodat()): a runtime ENOSYS
from a real mkfifoat() reached the unconfined fallback even though an
fd-relative FIFO primitive existed.
Whether a primitive exists is a property of the build, so decide it there:
no_atfd_mknod_primitive() is false wherever mknodat() covers the node type,
where mkfifoat() covers a FIFO, or where --fake-super creates through
openat() -- which is always present and was previously able to fall back on
its own unrelated failures.
Also correct the SECURITY.md residual, which overstated the loss. Plain
mknod()/mkfifo() do not follow a planted leaf symlink; they fail EEXIST,
verified directly against a symlink to a victim file. What a no-mknodat
platform actually loses is the pinned parent, so the residual is a
parent-component race rather than a followed basename, and ordinary
fake-super placeholder creation stays confined via openat(O_NOFOLLOW).
And narrow t_symlink_secure's skip: it skipped the whole helper without
mknodat(), including do_symlink_at() assertions that do not depend on it.
Only the do_mknod_at() checks are now skipped, and the helper still skips
outright when neither applies rather than passing vacuously.
do_mknod_atfd() returns ENOSYS on a platform that compiled no
fd-relative create at all -- older macOS has mknod() and mkfifo() but
neither mknodat() nor mkfifoat() -- and gen_entry_mknod() returned that
straight to the caller, so a FIFO or device node could not be created:
rsync: [generator] mknod ".../afifo" failed: Function not implemented (78)
That is not the stance SECURITY.md sets out. Where an operation can be
secured on some platforms but not others, rsync takes the race-safe path
where it exists and falls back to the historical unconfined behaviour
where it does not, "rather than refusing the operation outright". Its
one stated exception is the nested-socket bind(), which gen_entry_mknod()
already routes away from this path.
So fall through to do_mknod_at(), which on such a platform is do_mknod()
by design. Only on ENOSYS: any other errno is a real failure and must
not be retried through the unconfined path.
None of this was reachable before: the platform did not link at all until
the previous commit, which is why a refusal sitting where the documented
rule says fall back went unnoticed.
The race helper skips itself where mknodat() is absent. do_mknod_at() IS
do_mknod() there -- the held-dirfd walk and the O_NOFOLLOW leaf create are
compiled out, not failing -- so its checks were asserting a property the
build deliberately does not have, and reported the accepted residual as a
module escape.
SECURITY.md gains that residual under "Known residuals": it previously
covered only the socket case, and said nothing about the whole special-
file path degrading where mknodat() is missing.
Checked by rewriting config.h the way macOS 10.13 has it (mknod and
mkfifo yes, mknodat and mkfifoat no): --specials now creates the FIFO
where it previously failed with ENOSYS, the race test skips instead of
failing, and the suite is 254/0. Unchanged on Linux at 255/0.
-D and --no-D do two jobs at once: they decide whether devices and
special files are created, and they decide whether those entries carry
their rdev fields on the wire. send_file_entry() and recv_file_entry()
frame those fields with the same condition, but each end evaluates its
own preserve_devices/preserve_specials, so the two only agree because
both normally parse the same command line.
That makes --no-D unusable for a wrapper that controls one end of a
connection and wants to deny creation. Give it to the receiver alone
and the client's -D sender writes rdev the receiver never reads: the
file list desynchronises from that entry on. A FIFO or socket breaks
below protocol 31 -- a hang at 29, "File-list index 0 not in 0 - -1" at
30 -- and a device node breaks at EVERY protocol, current ones included,
because its arm of the condition has no protocol clause at all.
--drop-D separates the two jobs: it refuses the creation and leaves the
encoding alone. The entry is skipped through the existing non-regular
fall-through, so the visible result matches --no-D, and because it
touches no wire state it can be applied to one end by itself.
It has no effect on a sending rsync, which creates nothing.
set_file_attrs() pins the entry via the cached held dir fd (held_dfd_for) and
drives the xattr/ACL ops off that fd (fsetxattr). But when the pin missed --
held_dfd_for() returns -1 (the path is deeper than the dirfd cache, or its dir
isn't the held one), or the leaf openat() loses a race -- held_fd stayed -1 and
set_stat_xattr()/set_xattr()/get_acl_fdat()/set_acl_fdat() fell through to the
path-based branch (sys_lsetxattr(fname,...)). Unlike the chmod/chown/times path
wrappers (which secure-resolve), that raw lsetxattr re-resolves the parent, so a
concurrent flip of a dest parent component to a symlink->outside lands the xattr
OUTSIDE the destination tree (the intermittent copy-xattrs-symlink-race escape
that surfaces under -j load, which widens the open->setxattr window).
Re-pin through secure_relative_open() when the cached pin misses on a confined,
non-operator receiver path, so the xattr/ACL ops always use a confined fd -- NOT
a raw path lsetxattr; if the re-pin also fails (a genuinely raced parent/leaf
symlink) skip the path-based ops (xattr_refuse) rather than redirecting them.
The re-pin passes O_DIRECTORY for a directory leaf. Apply the same re-pin/refuse
to gen_entry_copy_xattrs() (the dir xattr copy), whose dfd<0 path likewise fell
to copy_xattrs() with dest_fd==-1. chmod/chown/times are unchanged (confined via
their *at wrappers); operator paths keep op_pin/op_refuse.
The hardened receiver confined the destination side of an xattr/ACL copy (the
fsetxattr/acl_set_fd through a held O_NOFOLLOW fd) but still read the SOURCE side
by path: copy_xattrs() did get_xattr_names/get_xattr_data on the source path, and
make_backup() cached the backed-up file's ACL/xattr via get_acl()/get_xattr() by
path. A local module writer could race the source/basis parent to a symlink
after the confined content/stat open and before that path-based metadata read,
so out-of-module xattrs/ACLs got copied onto an in-module destination or backup.
Thread a source fd through the read side, mirroring the existing dest-fd plumbing:
- get_xattr_data(), get_xattr() and get_xattr_acl() gain an fd arg (get_xattr_names
already had one) and use sys_fgetxattr/sys_flistxattr when fd >= 0; this also
covers the --fake-super ACL-as-xattr read in get_rsync_acl().
- copy_xattrs() gains a source_fd; copy_file() passes its held source fd (ifd)
and keeps it open across the xattr copy (closing it on the fsync error path
too); gen_entry_copy_xattrs() O_NOFOLLOW-opens the basis leaf under the
confined resolver (with O_DIRECTORY for a directory basis) and passes it.
- make_backup() pins the source leaf with a confined O_NOFOLLOW fd
(backup_source_fd, like set_file_attrs's op_leaf_fd) and reads its ACL via
get_acl_fdat() and its xattrs via get_xattr(fd); the in-place delta-backup in
the generator pins fname the same way. On a hardened receiver a raced/absent
leaf skips the cache rather than reading through a flippable path.
Non-hardened receivers (fd < 0) keep the path-based behaviour unchanged. The
basis COMPARE reads (the generator deciding a match) stay path-based: they never
copy out-of-module metadata onto a file, so they are not part of this sink.
The opt-out is meant to restore stock-3.2.7 "follow existing symlinks in the
module" behaviour, but several daemon symlink-resolution sites called the
confined resolver unconditionally and never consulted symlink_optout_allowed(),
so a module with "insecure links = yes" still refused to follow a symlinked
directory (change_dir ELOOP) or an alt-dest basis. Gate every such site on the
opt-out -- follow plainly like 3.2.7 when set, confine otherwise:
- change_dir() relative daemon branch (util1.c): plain chdir() under the
opt-out instead of secure_relative_open(), so a peer can read a path through
a symlinked directory again.
- basis_link_stat() (generator.c): the chrooted-relative and non-chroot
branches honour the opt-out. The non-chroot ABSOLUTE branch (a basis rooted
under the module by check_alt_basis_dirs, which can reach an in-module
symlink) is, when NOT opted out, resolved through owner_walk_parent so a
target landing outside the module root is refused -- closing a confirmed
--compare-dest=/symlink out-of-module read oracle. The leaf is taken
O_NOFOLLOW under the confined parent so --copy-links can't follow a leaf
symlink out; --fake-super folds its %stat via the held fd. A relative
sibling basis (--link-dest=../01) keeps the plain path (#915/#930).
- secure_basis_open() (receiver.c): plain do_open() under the opt-out.
- use_secure_symlinks (clientserver.c): cleared under the opt-out, so the
receiver's protected-regular EACCES write fallback is legacy too.
- secure_sender_parent_fd() (sender.c): declines (errno=0) under the opt-out
so --remove-source-files re-stats the plain path.
Default modules (insecure links = no) are unchanged and stay fully confined;
the opt-out is per-module and a client cannot enable it.
Five error-path/cleanup memory leaks found by an external audit:
- flist.c send_file_name: free the ACL loaded by get_acl() when a later
get_xattr() fails (and on the get_acl error path).
- xattrs.c copy_xattrs: free the xattr datum buffer when the setxattr fails.
- generator.c recv_generator: free real_sx at the cleanup label (the
directory branch loaded its ACL via set_file_attrs but only the
regular-file path freed it); zero-init real_sx so the early gotos are safe.
- uidlist.c send_one_list: free the strdup'd id-0 name after send_one_name.
- clientserver.c start_inband_exchange: free modname on the early error
returns (it was freed only on the success path).
ASan/LSan regression tests cover the generator, uidlist and clientserver
leaks; the flist and xattrs leaks need a forced syscall failure and are
covered by the audit's standalone harnesses.
Reported-by: Leonid Bugaev <leonsbox@gmail.com>
(cherry picked from commit 078f3b99f4004510d418ee9d97d9b775dc8587bc)
A '!'-prefixed delete-delay entry computed one byte short, dropping the
final character of the name.
Co-authored-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Route every operator-supplied directory path (--temp-dir / --partial-dir /
--backup-dir / alt-dest basis) and the transfer engine's own dest/source opens
through the secure resolver + ownership walk, so a symlink owned by another uid
can no longer redirect a read, a backup, a staging open, a rename, an unlink or a
new-destination create outside the module. Covers backup.c, generator.c (alt/
link/in-place basis), receiver.c (basis open), sender.c (remove-source / source
open / copy-links leaf), rsync.c (held-fd attr stat), clientserver.c (pid-file
parent pin), and main.c (relative-basis make-absolute, mkpath dest-arg guard).
--insecure-links / "insecure links = yes" is the local opt-out (a daemon never
honors a peer-forwarded one).
this fixes a valgrind error where we could read an uninitialised sx.st
field when we don't fill the stat data.
Also drop the now-obsolete testsuite/valgrind.supp stanzas for these
reads (atomic_create/delete_item, plus the rwrite strlcpy over-read that
master already fixed) -- they are no longer needed now the reads are gone.
Thanks to report from Michael Mess <michael@michaelmess.de>
sum_sizes_sqroot() capped the strong-sum length at SUM_LENGTH (16), the
legacy MD4/MD5 digest size. Since 0902b52f the sum2 array elements are
xfer_sum_len bytes and the sender rejects a sums header whose s2length
exceeds xfer_sum_len. When the negotiated transfer checksum is shorter
than 16 bytes -- xxh64 (8), used when the build's libxxhash lacks
xxh128/xxh3 (e.g. Ubuntu 20.04) -- the generator still emitted s2length
up to 16, so --append-verify and other full-checksum (redo) transfers
died with "Invalid checksum length 16 [sender]" (protocol incompatibility).
Cap s2length at MIN(SUM_LENGTH, xfer_sum_len): unchanged for any checksum
>= 16 bytes (md5/xxh128/sha1), corrected for short ones. Also closes a
latent over-read of the xfer_sum_len-sized digest buffer.
Two assorted audit findings:
- receive_data() never bounds-checked the block index returned
by recv_token() against sum.count before computing offset2
and feeding it to map_ptr(). An out-of-bounds index from a
hostile sender produces invalid memory access. Add a
sum.count bounds check.
- read_delay_line()'s strchr() call could return NULL when no
space was found, but the code unconditionally added 1 to the
result before dereferencing. Low impact (just a disconnect on
exit of the client-specific forked process) but the NULL
deref is real. Guard the NULL.
Both reported by Joshua Rogers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 797e17f ("fixed an invalid access to files array") added a
parent_ndx < 0 guard to send_files() in sender.c, but the visually-
identical block in recv_files() in receiver.c was not updated. A
malicious rsync:// server can therefore drive any connecting client
into the same out-of-bounds dir_flist->files[-1] read followed by a
file_struct dereference in f_name() one line later.
Reach: protocol-30+ default (inc_recurse) makes flist.c:2745 set
parent_ndx = -1 on the first received flist when the sender omits a
leading "." entry; rsync.c flist_for_ndx() does not reject ndx == 0
in that state because the range check evaluates 0 < 0 = false; and
read_ndx_and_attrs() only validates ndx with the ITEM_TRANSFER bit
set, so iflags=ITEM_IS_NEW (or any other non-transfer iflag word)
bypasses the check.
Apply the same guard receiver-side. Confirmed: the same PoC (a
minimal Python rsyncd that handshakes with CF_INC_RECURSE, sends a
no-leading-"." flist, and emits ndx=0 with ITEM_IS_NEW) crashes
unpatched 3.4.2 with SEGV_MAPERR si_addr=0x4101a-class in the
receiver child; with this guard it exits cleanly with code 2
(RERR_PROTOCOL).
The attack surface delta over the sender variant is large:
the original was malicious-client -> daemon, this is
malicious-server -> any rsync client doing a normal rsync://
or remote-shell pull.
Reported by Pratham Gupta (alchemy1729).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three related codex audit findings:
Finding 3a: copy_file()'s source open in util1.c used
do_open_nofollow(), which only rejects a final-component
symlink. A parent-component symlink (e.g. --copy-dest=cd where
cd -> /outside) follows freely and reads outside the module.
Route through secure_relative_open() with O_NOFOLLOW.
Finding 3b: generator.c's in-place backup-file create still
used a bare do_open with O_CREAT, leaving a tiny but reachable
parent-symlink window between the secure unlink (already
through do_unlink_at) and the create. Add do_open_at() that
goes through a secure parent dirfd, and route the call site
through it.
Finding 3c: copy_file()'s destination open in
unlink_and_reopen() had the same bare-do_open pattern; route
through do_open_at as well.
Adds testsuite/copy-dest-source-symlink.test and
testsuite/bare-do-open-symlink-race.test as regression coverage
for both attack shapes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the rest of the path-based syscall wrappers and migrate every
receiver-side caller:
- do_lchown_at, do_rename_at, do_mkdir_at, do_symlink_at,
do_mknod_at, do_link_at, do_unlink_at, do_rmdir_at,
do_utimensat_at, do_stat_at, do_lstat_at
Same shape as do_chmod_at: open each parent under
secure_relative_open(), call the *at() variant against the dirfd,
fall through to the bare path-based syscall in non-daemon /
chrooted / absolute-path / no-parent cases. macOS's
setattrlist-based set_times tier is also routed through the
utimensat_at path on daemon-no-chroot.
Hardenings to secure_relative_open() itself:
- confine basedir resolution under the same kernel mechanism
used for relpath (basedirs from --copy-dest / --link-dest are
sender-controllable in daemon mode)
- reject any '..' component (bare '..', 'foo/..', 'subdir/..')
so the per-component O_NOFOLLOW fallback can't escape
- return the dirfd we built up from the per-component fallback
when the caller passed O_DIRECTORY (otherwise every do_*_at
failed with EINVAL on platforms without RESOLVE_BENEATH)
Adds testsuite/alt-dest-symlink-race.test and
testsuite/secure-relpath-validation.test (with t_secure_relpath
helper) as regression coverage for the new hardenings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CVE-2026-29518's fix routed the receiver's open() through
secure_relative_open(), but every other path-based syscall the
receiver runs on sender-controllable paths is vulnerable to the
same TOCTOU primitive. This commit closes the chmod variant.
Add do_chmod_at() that opens the parent of fname under
secure_relative_open() and uses fchmodat() against the resulting
dirfd. Gate the secure path on am_daemon && !am_chrooted (the same
gate use_secure_symlinks already uses for the receiver basis-file
open), so non-daemon callers and chrooted daemons keep the original
do_chmod() fast path.
Migrate the receiver-side do_chmod() call sites in delete.c,
generator.c, rsync.c, and xattrs.c.
Adds testsuite/chmod-symlink-race.test (with t_chmod_secure helper)
as regression coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
when we open a file that we don't expect to be a symlink use
O_NOFOLLOW to prevent a race condition where an attacker could change
a file between being a normal file and a symlink
When using the --fuzzy option to try and find close matches locally,
the edit distance algorithm used is O(N^2), which can get painful on
CPU constrained systems when working in folders with tens of thousands
of files in it.
The lower bound on the calculated Levenshtein distance is the difference
of the two strings being compared, so if that difference is larger than
the current best match, the calculation of the exact edit distance between
the two strings can be skipped.
Testing on the OpenSUSE package repo has shown a 50% reduction in the CPU time
required to plan the rsync transaction.
A local_server copy now includes the dev+ino info from the destination
file so that the sender can make sure that it is not going to delete
the destination file. Fixes mistakes such as:
rsync -aiv --remove-source-files dir .
- Use -pedantic-errors with gcc to make an array-init fatal.
- Fix all the extra warnings that gcc outputs due to this option.
- Also add -Wno-pedantic to gcc if we're using the internal popt
code (since it has lots of pedantic issues).
- Rename unchanged_file() to quick_check_ok().
- Enhance quick_check_ok() to work with non-regular files.
- Add a get_file_type() function to the generator.
- Use the new functions in the generator code to make the logic simpler.
- Fix a bug where the `--alt-dest` functions were not checking if a
special file fully matched the non-permission mode bits before
deciding if we have found an alt-dest match.
- Enhance the `--info=skip --ignore-existing` output to include extra
info on if the existing file differs in type or passes the standard
quick-check logic.
- Add `--info=skip2` that authorizes rsync to perform a slow checksum
"quick check" when ignoring existing files. This provides the uptodate
and differs info even if we need to checksum a file to get it.
- All the memory-allocation macros now auto-check for failure and exit
with a failure message that incudes the caller's file and lineno
info. This includes strdup().
- Added the `--max-alloc=SIZE` option to be able to override the memory
allocator's sanity-check limit. It defaults to 1G (as before).
Fixes bugzilla bug 12769.
