* hlink: fix hard-link failure abort (#741) and the latent drain hang it exposes
When a hard-link group member fails its direct link() (EMLINK or
EXDEV) it is written as a regular file and finishes first, claiming
the group's prior_hlinks node; a later member's finish then hits the
"Hlink node data ... already has path" check and aborts the whole
transfer (code 13). Tolerate the duplicate finish: keep the first
leader's path, log an FINFO warning, and continue.
Removing the abort exposes a second, latent bug: finish_hard_link()
decrements flist->in_progress for every file in the completion chain,
but members that failed their direct link() in hard_link_check() were
never counted, so the counter drifts negative and the final drain
hangs forever. Add FLAG_HLINK_WAIT, set at the three increment
sites, and gate the two affected decrements on it.
Tested with a 65000-link pinned-inode EMLINK fixture (full repro and
evidence in the PR description), an -H --link-dest regression matrix,
and the upstream hardlink/link-dest test suite.
hlink: fix hard-link failure abort (#741) and the latent drain hang it exposes
When a hard-link group member fails its direct link() (EMLINK or
EXDEV) it is written as a regular file and finishes first, claiming
the group's prior_hlinks node; a later member's finish then hits the
"Hlink node data ... already has path" check and aborts the whole
transfer (code 13). Tolerate the duplicate finish: keep the first
leader's path, log an FINFO warning, and continue.
Removing the abort exposes a second, latent bug: finish_hard_link()
decrements flist->in_progress for every file in the completion chain,
but members that failed their direct link() in hard_link_check() were
never counted, so the counter drifts negative and the final drain
hangs forever. Add FLAG_HLINK_WAIT, set at the three increment
sites, and gate the two affected decrements on it.
Tested with a 65000-link pinned-inode EMLINK fixture (full repro and
evidence in the PR description), an -H --link-dest regression matrix,
and the upstream hardlink/link-dest test suite.
* hlink: re-root a hard-link group whose root is in a released file list
Address the review comment on commit 350425a8 (RsyncProject/rsync#1144):
"This will still abort when the hard-link group spans incremental
file-list chunks. Re-root the group without retaining released
file-list indices."
When a group spans chunks, a member's F_HL_PREV can point into a
file-list that is released before the member's finish_hard_link()
walks the completion chain, and the fatal flist_for_ndx() call
aborts the transfer. Two changes in finish_hard_link():
1. The completion-chain lookup now uses the graceful (NULL) variant
of flist_for_ndx() and stops the walk when an index is no longer
in a live file-list. The chain only moves to earlier ndx values,
so once one is released, everything further back is already
finished and there is nothing left to re-link. No live file
retains a released index: the finishing file's F_HL_PREV is reset
on entry, and each walked member's F_HL_PREV is pointed at the
finishing file.
2. A duplicate finish (the group already recorded as finished at
some path) now re-roots the group at the newly finished file
instead of keeping the recorded path. A duplicate finish only
happens when the recorded root could not be hard-linked (e.g.
EMLINK, EXDEV) and that member was written as a regular file, so
that member's inode is the only one the group can still link to;
later members then link to this live file instead of the stale
root.
The abort is reproducible locally with a small chunk-spanning fixture
whose first file-list is under MIN_FILECNT_LOOKAHEAD/2, both on the
series base 8718cd28 and with only commit 350425a8 applied: the
hard-link lookahead at the top of generate_files() then matches the
extra file-list before the group leader is itemized, so the members'
F_HL_PREV pointers into the leader's file-list survive the match,
and the fatal lookup aborts deterministically: "File-list index 3
not in 11 - 13 (finish_hard_link)" with "protocol incompatibility"
(code 2). With this commit the same run completes with rc 23 and
the group re-rooted on a fresh inode; a simulated WAN delay for the
last member gives the identical abort/fix behavior (fixture and full
evidence in the description of RsyncProject/rsync#1144). Also
verified: a 1200-file chunk-spanning EMLINK fixture with the
destination inode pinned at the ext4 65000-link cap completes in
0.2 s with rc 23 and the group re-rooted on a fresh inode; the
single-flist EMLINK fixture takes 0.15 s, rc 23, all 101 members on
one fresh inode; the hard-link regression tree is all rc 0 with no
spurious output; the upstream hardlink/link-dest/keep-dirlinks suite
(9 tests) passes.
Co-authored-by: LIM Coding Assistant <298007100+statefullm@users.noreply.github.com>
* hlink: cut stale F_HL_PREV indexes into released file lists
Address the review comment on ad01ac16 (RsyncProject/rsync#1144):
"Breaking here only hides the stale index."
The break added by ad01ac16 stops the finish_hard_link() walk from
dereferencing a released file-list, but it only avoided the stale
index: a live file_struct could still hold its F_HL_PREV into a
released list (for example, a FLAG_HLINK_WAIT member keeps its own
file-list alive while an earlier member's file-list is released),
and the index sat in place until that member's own completion
re-pointed or reset it. That is harmless in practice - every
F_HL_PREV reader uses the NULL-tolerant flist_for_ndx() - but the
index is now cut instead of retained, at the places where a live
file_struct reads or re-acquires one:
1. check_prior() resets F_HL_PREV to -1 when its walk ends because
the target index is no longer in a live file-list (previously the
stale value was retained when the group had already recorded a
finished path).
2. The FLAG_HLINK_WAIT re-point in hard_link_check() copies the
previous member's F_HL_PREV only while that index is still in a
live list, and cuts it to -1 otherwise.
Cutting is safe because a file-list is released only after every
file in it has completed: check_for_finished_files() gates
flist_free() on in_progress and to_redo being zero, and in_progress
counts every sent file and every FLAG_HLINK_WAIT file. So the
members of a released list are all finished and the chain has
nothing left to do there. The finish_hard_link() walk itself
re-points each walked member at the finishing file before testing
the next hop, and the finishing file's F_HL_PREV is reset on entry,
so after the walk no live file_struct points into a released list;
its break remains as the last resort where an earlier member's index
is already gone.
Verified: the small chunk-spanning fixture (first file-list under
MIN_FILECNT_LOOKAHEAD/2) completes with rc 23 and the group re-rooted
on a fresh inode, with a temporary probe confirming that at the
walk's break no live file_struct retains an index into a released
list (every live hlinked file's F_HL_PREV is either -1 or in a live
list); the 1200-file chunk-spanning EMLINK fixture and the
single-flist EMLINK fixture (101 members on one fresh inode) behave
as before with rc 23 and the group re-rooted; the hard-link
regression tree is all rc 0 with no spurious output; the upstream
hardlink/link-dest/keep-dirlinks suite (9 tests) passes.
Co-authored-by: LIM Coding Assistant <298007100+statefullm@users.noreply.github.com>
* hlink: transfer files after hard-link failure
---------
Co-authored-by: LIM Coding Assistant <298007100+statefullm@users.noreply.github.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
* match: guard sender_file_sum[-1] write when xfer_sum_len is 0
If xfer_sum_len were 0, the loop exits immediately with i=0, and the
subsequent sender_file_sum[i-1] access becomes sender_file_sum[-1] — UB.
Add an `i > 0` guard to make the read-error checksum corruption path safe
for a zero-length sum.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Shreenidhi Shedi <yesshedi@gmail.com>
* generator: remove dead NULL check after strdup in atomic_create
rsync redefines strdup to my_strdup (via rsync.h), which calls out_of_memory
and never returns NULL. The NULL check after the strdup call was therefore
unreachable and misleading — a reader might infer the system strdup is in use
or that a NULL return is possible here.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Shreenidhi Shedi <yesshedi@gmail.com>
---------
Signed-off-by: Shreenidhi Shedi <yesshedi@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* testsuite: fix timestamp race in exclude test
The same-newness files must have equal modification times, but
separate touch calls can straddle a second boundary and make the
destination newer. This causes --update to skip it unexpectedly.
Copy the source timestamps to the destination explicitly.
* testsuite: only copy update-test mtime
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* access: match hosts allow/deny hostname tokens against IPv6 peers
match_hostname() resolved hostname tokens with gethostbyname(), which
only returns IPv4 addresses, so a hostname token never matched an IPv6
peer: allow entries locked such clients out and deny entries silently
failed open. Resolve with getaddrinfo(PF_UNSPEC) instead, mirroring
match_address() in the same file.
Fixes#563.
* access: compare forward-DNS hostname matches in binary form
Address review feedback on the getaddrinfo conversion: parse the peer
address once and compare each resolved address against it byte-wise
within the same family, as match_address() does, instead of comparing
inet_ntop() strings; keep the IPv6 cases under #ifdef INET6 so a
--disable-ipv6 build compiles the IPv4 path only.
Add daemon-access-hostname regressions: hostname-token allow and deny
over IPv4 and IPv6 (with reverse lookup off, so only the forward path
can match), forward lookup disabled, and unresolvable allow/deny
tokens.
* access: tighten IPv6 hostname matching portability
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* auth: read only the first password-file line
strtok() skips leading delimiters, so an empty first line followed by a
password was interpreted as that password. Stop at the first CR/LF
instead, keeping an empty first line empty and making file and stdin
behavior consistent.
Add daemon-auth regression coverage for LF, CRLF, empty input, an empty
first line, and a final line without a newline for both input sources.
Fixes#1134.
Signed-off-by: Eduard Zateev <hackerowskiy@gmail.com>
* tests: reject password-file fallback for empty first lines
Supply the valid daemon password through RSYNC_PASSWORD in negative
cases, then require exit status 1 and the password read diagnostic.
Signed-off-by: Eduard Zateev <hackerowskiy@gmail.com>
---------
Signed-off-by: Eduard Zateev <hackerowskiy@gmail.com>
* rrsync: pin the parent of existing receiver file arguments
Passing an existing destination as the leaf's /proc/self/fd/N makes
rsync lstat the procfs link and try to replace it, failing with EPERM.
Use the held parent fd plus the leaf name, as for a missing destination.
Keep directory arguments pinned directly and verify the parent's leaf
still matches the already validated inode before releasing its fd.
Reuse pin_dir() for the parent checks and execute-only directory support.
Add end-to-end coverage for overwrite, creation, directories, pulls,
rejected paths, an in-place leaf replacement and a parent rename.
Based on the parent-pin approach described by JeanFrancoisGagne in #1107.
Fixes#1107.
Assisted-by: OpenAI Codex
Signed-off-by: Eduard Zateev <hackerowskiy@gmail.com>
* testsuite: expect rrsync file-argument skips
* testsuite: allow slow aligned overflow scans
---------
Signed-off-by: Eduard Zateev <hackerowskiy@gmail.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
* Fix hash_search() duplicate-chain regression (#1115) and collision flood
When a hash chain contains > MAX_CHAIN_LEN identical colliding blocks
(such as 0x00 sparse regions colliding with 0xFF blocks at 64K/128K
block sizes), walking every duplicate entry exhausts MAX_CHAIN_LEN before
reaching valid matching blocks further down the chain (#1115).
Track the strong checksum of the last failed chain entry (last_false_sum2)
and unlink subsequent duplicate entries sharing the same strong checksum
(unless protected by SUMFLG_SAME_OFFSET in --inplace mode) so duplicates
are neither re-walked nor counted repeatedly against MAX_CHAIN_LEN.
Additionally, if 4 consecutive 1-byte rolling offsets experience a
weak-checksum collision where the entire chain fails to match, advance
offset by a full block rather than computing a full strong checksum on
every single byte.
* Split hash_search() collision tests and add boundary/--inplace coverage
Split testsuite/collision_test.py into two dedicated regression suites:
1. testsuite/hashsearch-dup-chain_test.py:
- Verifies forward chains (> MAX_CHAIN_LEN duplicates),
multi-group interleaved duplicate chains, and --inplace
SUMFLG_SAME_OFFSET preservation when mutating the chain after a
subsequent duplicate is overwritten on disk.
2. testsuite/hashsearch-collision-skip_test.py:
- Asserts explicit O(N/B) hash_hits CPU bounds and exact literal/matched
bandwidth bounds across skip boundaries (3, BLOCK, BLOCK+3, BLOCK+4,
2*BLOCK+3, 2*BLOCK+6, and 10*BLOCK+137) in both standard and
--inplace modes, and documents the bunched-perturbation hazard case.
* Unlink redundant SUMFLG_SAME_OFFSET duplicates in hash_search()
In --inplace mode, when > MAX_CHAIN_LEN identical basis blocks match
at their aligned offsets (such as 0x00 sparse runs), every matching
entry acquires SUMFLG_SAME_OFFSET. Skipping deduplication for all
SUMFLG_SAME_OFFSET entries left all pinned duplicates in the chain,
exhausting MAX_CHAIN_LEN, inflating false_alarms, and hiding valid
basis blocks behind them.
Track the flags of the entry that set last_false_sum2 (last_false_flags).
Once a SUMFLG_SAME_OFFSET entry has been retained as last_false_sum2,
unlink subsequent duplicate entries carrying SUMFLG_SAME_OFFSET as well,
while still preventing unpinned entries from unlinking a pinned entry.
Add Case 4 to testsuite/hashsearch-dup-chain_test.py to verify that
> MAX_CHAIN_LEN pinned duplicate blocks are unlinked and trailing valid
blocks behind them are matched with 0 extra literal bytes.
When updating_basis_file is active (--inplace), hash_search() advances
aligned_offset by s->blength and increments the 32-bit signed counter
aligned_i until aligned_offset catches up to offset.
If the source file is larger than 2 GiB and the peer supplies a small
s->blength and s->count, continuing to increment aligned_i past s->count
serves no purpose (as no basis blocks exist beyond s->count - 1) and can
overflow INT32_MAX into a negative index that bypasses the one-sided
aligned_i < s->count check.
Stop the catch-up loop once aligned_i reaches s->count so aligned_i
remains strictly within [0, s->count] and cannot overflow.
Reported-by: Mominul Islam Hemal <mominulislam-2021411216@cs.du.ac.bd>
* log: avoid recursive deletion-message overflow
A quiet daemon receiver queued one MSG_DELETED per path until the generator hit --max-alloc, so skip names the client did not request. If requested output legitimately fills the queue, keep allocator error reporting from re-entering the same buffer and exhausting the stack. Preserve explicit deletion requests across the daemon max-verbosity clamp.
* NEWS: note daemon deletion-message overflow fix
* sender: resolve --files-from entries under a module subdirectory
A --files-from pull whose source is a subdirectory of a daemon module
(rsync://host/mod/subdir/) looked up every listed file at the module
root. A missing file was reported as "file has vanished" and the
transfer ended with code 24; a same-named file at the module root was
sent in its place. With --remove-source-files, the sent files were
then kept with "Skipping sender remove for changed file".
The sender opens each file from the module root with
secure_relative_open(), rebuilding the path as F_PATHNAME + "/" +
f_name, and secure_sender_parent_fd() rebuilds the parent the same way.
A --files-from entry has no F_PATHNAME: like any relative pathname it
is relative to orig_dir, the transfer's source directory, which here is
a module subdirectory. Dropping orig_dir made both look under
mod/<entry> instead of mod/subdir/<entry>.
Build the module-relative path in one place, module_relative_path(),
and prefix a missing or relative F_PATHNAME with orig_dir's
module-relative part, as change_pathname() does when it chdirs.
orig_dir is no longer static in flist.c so sender.c can read it.
Add daemon-files-from-subdir_test.py, which fails without the fix.
* sender: keep the --files-from subdirectory fix minimal and test the module filter
* sender, testsuite: shorten the orig_subdir() comment and the test
* cleanup: don't flush the write buffer from a signal handler
When SIGUSR1 aborts an --inplace/--append transfer, _exit_cleanup()
runs inside the handler and flushes the receiver's write buffer. If
the handler interrupted flush_write_file() after write() returned but
before wf_writeBufCnt was updated, the same bytes are written a second
time, and with --append the file keeps a duplicated 256 KiB region.
Skip that flush when called from a signal handler. A SIGINT, SIGTERM
or SIGHUP that reaches the receiver itself still exits from normal code
through got_kill_signal, so its flush is unaffected.
* testsuite: abort --append/--inplace in the middle of a write
Hold the receiver right after its first write() to the destination,
with every signal pending, send it SIGUSR1 and check that the
destination is still a prefix of the source. This fails without the
previous commit. The hold is an LD_PRELOAD shim, so the test is
Linux-only and is expected to skip on macOS and Cygwin.
* Add --delay-symlinks to create symlinks after all files are in place
The generator creates a symlink as soon as it reaches it in the file
list, while regular files arrive later through the sender and receiver
(and, with --delay-updates, are only renamed into place at the end).
For the rest of the transfer a new symlink can point at a file that
isn't there yet, so anything reading the destination during the sync --
e.g. a loader resolving libfoo.so -> libfoo.so.2 -- can fail.
The new --delay-symlinks option makes the generator defer new and
changed symlinks and create them once the receiver reports that every
file, including any --delay-updates renames, is in place. Each link is
itemized and logged after it has been created, before the generator
ends the delay-updates phase, and the deletion stats are sent after any
directory that a deferred link replaces.
The option implies --no-inc-recursive on the receiving side and is
passed to a remote receiver, which must also support it. It is an
error to combine it with --remove-source-files, --hard-links,
--compare-dest, --copy-dest, or --link-dest, or to use it with a
protocol older than 29. support/rrsync accepts the option and refuses
it under -no-overwrite, as it does --delay-updates.
Add delay-symlinks_test.py, which holds a push partway through the
link's target to check that new and changed links aren't created early,
and rrsync-no-overwrite-delay-symlinks_test.py.
* Document --delay-symlinks limits and test how it is passed to the server
The man page now says the option narrows the inconsistency window rather
than making the update atomic, that an interrupted transfer leaves new
symlinks missing and changed ones at their old targets, and that
--delete-before can also remove an old target early.
Add delay-symlinks-remote_test.py. Through a remote shell that records
the server command, it checks that the option is passed to a remote
receiver on a push but not to a remote sender on a pull, and that
incremental recursion is off in both directions. It also checks that
--protocol=28 fails with the protocol error, and that a push to an older
rsync (old_versions/rsync_3.4.1) is refused before anything is
transferred while a pull from it works.
* testsuite: seed delay-symlinks fixture with rsync
* NEWS: document --delay-symlinks
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: group file-list and directory counts
Adds the thousand separators as file-list is computed. Useful on large
dirtrees:
Behavior before: 2523297101 files to consider
After: 2,523,297,101 files to consider (2.523.297.101 with a comma decimal point)
Also covers the file-list counter, "created N directories" and --max-delete.
* NEWS: record human-readable output changes
* human-readable: do not add trailing space in output
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: use dynamic precision length
Let's lower precision for huge numbers.
The output used to be:
3.45M -> 46.73M -> 523.11M -> 1.24G -> ...
With this change the code always gives the three most significant digits:
3.45M -> 46.7M -> 523M -> 1.24G -> ...
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: also handle num < mult with the same code
... just make sure no precision is added.
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: add an "i" to unit to indicate binary (1024) base
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: also use it to format rate in progress
Let's also simplify the code for rate in progress, and benefit from
same functionality.
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: append suffix "B" for byte to ofs...
... and also fix the formatting in man page.
Co-authored-by: Zen Dodd <mail@steadytao.com>
* human-readable: unbreak for do_big_dnum() with fract
Co-authored-by: Christian Hesse <mail@eworm.de>
* testsuite: cover human-readable output
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* Add fchmodat() as an option to set symlink Unix access permissions
syscall.c:
Add fchmodat(), if it exists on the system, as a last-resort attempt to change
the Unix access permissions for a symlink (on those systems that support it).
This is done just prior to finally abandoning such an attempt and after any
other available options for this, such as lchmod() or setattrlist().
This option is required to preserve symlink access permissions when making
copies using rsync on OpenBSD. The patch itself is not OS-specific.
testsuite:
Add a test, symlink-unix-perms_test.py, to testsuite to check on the success
of setting symlink access changes using rsync. The test is skipped on those
systems that do not allow symlink access permission changes. The test checks
at runtime whether or not the system permits symlink access changes. As with
the patch, this test is not OS-specific.
* testsuite: Modify the symlink-unix-perms test to conform to proper standards.
1) Set the proper python shebang
2) Remove the use of random test access values in favour of fixed values.
3) Skip the test only on "not implemented" or "not supported" exceptions. All
other unexpected errors will fail the test.
4) Test using symlinks to a specifically created directory or file as opposed
to the '.' directory. The latter could potentially lead to cyclical paths.
Some code tweaking has also been done for readability.
* symlink-unix-perms_test.py: Perform some useful cleanup.
1) Remove unnecessary import of "random".
2) Consolidate some unnecessarily verbose and inefficient code.
* testsuite: expect unsupported symlink permission skips
This drops a lot of `else if` blocks and extends units by "E" (Exa),
which is 2^60 and thus the last to fit into int64.
Co-authored-by: Zen Dodd <mail@steadytao.com>
rsh_cmd() returns the lsh.sh path quoted for rsync's own --rsh tokenizer,
so a build directory containing a character shlex.quote() will not pass
bare -- the '~' in a Debian snapshot version such as
rsync-3.5.1~git20260921 -- comes back wrapped in quotes. The ssh-basic
probe passed that string straight to subprocess.run() as argv[0], where
no shell strips the quotes, and failed with FileNotFoundError. Split it
with shlex first.
Found via the rsync-latest PPA, where every snapshot build fails
`make check` on exactly this test while the release builds pass.
* Add 4 KiB logical block footprint tracking to stats
This adds a 'Number of 4 KiB logical blocks touched' metric to the --stats output to decouple network delta payload from actual local file modifications. Previously, a small amount of literal data scattered across a file (especially with --inplace) could result in a massive number of local file write operations with no visibility, and large sparse files masked their true write opreations (ignoring punch holes).
Technical details:
- Implemented a stateful block tracker in the receiver that calculates touched 4K boundaries using file offsets and lengths, including strict lseek awareness to accurately skip sparse file holes.
- Enforced strict per-file lifetime state with a reset hook inside receive_data(), successfully mitigating POSIX file descriptor (FD) recycling state leaks.
- Created MSG_BLOCK_STATS multiplex message to tunnel the block footprint safely out of the isolated receiver process and relay it over the network.
- Bumped PROTOCOL_VERSION to 33 and SUBPROTOCOL_VERSION to 8392 for safe PR testing.
- Added test suite covering contiguous, scattered, zero-byte, sparse file (hole-skipping), multi-file (FD reuse), batch mode, and maximum-I/O boundary conditions.
* io: fix block stats integration
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* testsuite: fix fleettest example json for IDN
match CI for IDN tests
* testsuite: fix fleettest example skip extras and pin Cygwin python
#1069 replaced daemon-max-alloc-zero with max-alloc-zero, which no
longer needs the old static client, but the per-box expect_skip_extra
entries still named the old test, so runtests refused the skip list
("no such test") on every Linux target. Drop those entries.
Pin python3.9 on the cygwin target to match the workflow, which installs
only python39: the box's default python3 (3.12) has a working AF_UNIX
socketpair, so daemon-stdin-local-socket ran where cygwin.txt expects a
skip. Note that python39 must be 3.9.25+ for os.getxattr, which the
xattr tests need to run as they do in CI.
* testsuite: run bash without POSIXLY_CORRECT; tolerate Cygwin symlink copystat
runtests.py exports POSIXLY_CORRECT=1 to every test, and bash before 5.1
disables process substitution in POSIX mode, so pseudo-paths,
pseudo-paths-daemon and read-batch-pipe silently skipped on bash 4.4/5.0
hosts (Ubuntu 18.04/20.04, AlmaLinux 8). Strip the variable from bash's
environment: the tests require bash precisely for that syntax.
That was also why they skipped in the AlmaLinux 8 CI container, so its
skip list (which held only those three) goes away along with the
workflow's reference to it. The macOS list keeps them: there they skip
as Linux-only.
malicious-sender-delete-scope copies the source tree with symlinks=True.
Cygwin's python 3.9.25 gained os.listxattr, and copystat() on a fresh
symlink then fails with EACCES, although the link itself was created.
Ignore copytree errors that are only about symlink pairs; anything else
still raises.
* testsuite: update max-alloc entry
* Tighten alt-dest path resolution and partial-dir state validation
This introduces architectural best-practices to harden the receiver's state machine
and harmonize symlink handling across the delta-basis engine, addressing protocol
edge-cases reported by Fyyre (James).
- receiver.c (recv_files): Added explicit validation to ensure one_inplace is
only triggered when partial_dir is configured and the FNAMECMP_PARTIAL_DIR
token is legitimate.
- receiver.c (secure_basis_open): Enforced O_NOFOLLOW on leaf components when
resolving operator-supplied paths, aligning it on operator-path behavior.
Co-authored-by: Fyyre <fyyre@fyyre.net>
* receiver: validate partial-dir basis state
* fix: reject alt-dest leaf symlinks
* fix: preserve basis open flags
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* Fix --files-from confinement for local and SSH transfers
The recent path confinement patch caused an ELOOP error when local or SSH users tried to use a --files-from list located outside the confined root.
This happened because the code treated the argument as if its always an operator-path peer. CLI arguments provided locally or over SSH are trusted, so the strict boundary check should only apply to untrusted clients connecting to a background daemon.
* tests: assert local files-from copy
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
Problem:
The value of an arithmetic expression 'value << state->bits' is a subject to overflow because its operands are not cast to a larger data type before performing arithmetic
Solution: Cast value to unsigned long before the left shift to prevent signed integer overflow during shift evaluation.
Signed-off-by: a.burlakov@fobos-nt.ru
Signed-off-by: a.burlakov@fobos-nt.ru
Co-authored-by: a.burlakov <a.burlakov@aburlakov.ru>
Using --log-file or --files-from with /dev/stdin, /dev/stdout, or /dev/stderr
attached to a pipe previously failed in two ways:
1. On standard hosts, it failed with ENOENT because the symlink target
(pipe:[N]) was treated as a relative file path rather than a secure
kernel pseudo-path.
2. Inside user namespaces (e.g., rootless podman, unshare), it aborted
with ELOOP ("refusing to follow a symlink owned by an untrusted user").
The symlink ownership reported the kernel overflow UID (65534), which
completely blocked the walker from following the pseudo-paths.
This patch resolves the issues by refining the symlink path walker and
trust mechanisms:
* Updated fd_pin_tail() to natively recognize /dev/stdin, /dev/stdout,
and /dev/stderr, parsing them directly to their corresponding /0, /1,
and /2 descriptor tails.
* Introduced the is_anchored variable to strictly enforce absolute paths,
ensuring malformed paths cannot bypass the check using relative forms
like dev/fd/ or proc/self/.
* Removed pin_transit from the namespace_pin check and replaced it by
adding the is_anchored variable to the check. Checking for root confinement
or daemon status is unnecessary when we are already validating and restricting
traversal to known safe paths (Daemon also refuses any symlinks pointing to /).
* Added new cases to the test suite to validate standard stream pseudo-path
handling and ensure namespace overflow UID bypasses work correctly
without regression.
* Fix ona_open to safely resolve bash process substitution pseudo-paths
Bash process substitution (e.g., `<(...)` or `>(...)`) exposes file
descriptors as symlinks under `/proc/self/fd/X` pointing to kernel
pseudo-paths such as `pipe:[12345]`. Previously, `ona_open()` would read
this target and attempt to resolve it as a literal file path on disk,
causing the operation to fail with `ENOENT` and breaking legitimate local
process substitution.
This patch safely intercepts and resolves these pseudo-paths while
maintaining strict confinement boundaries and averting TOCTOU risks:
- Detects kernel pseudo-paths (`pipe:[`, `socket:[`, `anon_inode:`)
only when `fd_pin_tail` confirms the path resolves precisely to a
direct child of a valid FD directory.
- Categorically rejects pseudo-path resolution if `confine_root` is
active (yielding `ENOENT`).
- Strips `O_NOFOLLOW` for legitimate leaf pseudo-paths, allowing
`openat()` to correctly delegate resolution.
- Reverts `fd_pin_tail` to its upstream signature, as manual PID
validation is no longer required due to the secure `openat()` design.
* testsuite: expect pseudo-path skip on Alma
* syscall: reject trailing pseudo-path components
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* main: allow --contimeout for daemon connections over --rsh
The --contimeout guard rejected the option whenever connect_timeout was
set and the connection was not being made by a socket, so a daemon
reached through a remote shell (daemon_connection == 1) was refused the
same way as a plain non-daemon remote-shell transfer. That is how
rsync-ssl talks to a daemon: it runs rsync with --rsh pointing at its
helper, so "rsync-ssl --contimeout=60 host::mod" died with "The
--contimeout option may only be used when connecting to an rsync
daemon" before it could even connect.
Only reject --contimeout when there is no daemon connection at all, so
the option is accepted for a daemon-via-rsh connection (and still works
unchanged for the socket daemon path, which returns earlier through
start_socket_client).
Add a regression test that drives a daemon-via-rsh invocation and
asserts the option is not rejected, while a plain remote-shell
destination is still refused.
* main, io: honor --contimeout for daemon-via-rsh connections
The previous commit made --contimeout legal for a daemon reached
through a remote shell, but rsync still did not act on it: the socket
path times its connect() via an alarm, while the daemon-via-rsh path
spawned the helper and then blocked in start_inband_exchange() waiting
for the daemon greeting with no connect timeout at all. A hung helper
-- e.g. rsync-ssl's openssl connect to an unreachable host -- would hang
the transfer forever.
Give the client a second wall-clock deadline, client_connect_deadline,
set from --contimeout around do_cmd() + start_inband_exchange() and
cleared once the greeting exchange finishes. handshake_poll_timeout_ms()
now also checks it and exits with RERR_CONTIMEOUT ("timeout waiting for
daemon connection"), matching the socket path's exit code, instead of the
daemon-handshake deadline's RERR_TIMEOUT.
This covers the helper spawn, its connect/TLS handshake, and the greeting
exchange as one connection-establishment phase, so it works uniformly for
openssl, stunnel, and gnutls rsync-ssl backends.
Extend the contimeout-rsh test to drive a helper that never connects and
assert rsync aborts with exit 35 within the timeout, and that a working
daemon-via-rsh connection still completes.
* options: accept --max-alloc=0 again, resolved to the parser's own ceiling
3.5.0 rejected --max-alloc=0 (CVE-2026-53794). The stated rationale was that a
zero cap "disabled the per-allocation size cap (the defense behind
CVE-2024-12084)". That was true up to 3.2.7, where the check short-circuited:
if (max_alloc && num >= max_alloc/size)
but 2f9b963a ("Make `--max-alloc=0` safer", 3.3.0) removed the short-circuit and
mapped 0 to SIZE_MAX at parse time, leaving the guard unconditional. Since the
guard admits an allocation only when num < max_alloc/size, num*size stays below
max_alloc at every setting, so the num*size overflow check was armed for 0 just
as for any other value. From 3.3.0 onward, 0 raised the magnitude ceiling and
nothing else -- and an explicit 8191P raises it exactly as far, is accepted, and
is forwarded to the peer, so rejecting 0 removed no capability.
What it did remove is the only portable spelling. The parser's ceiling is
SIZE_MAX/2, so it tracks the build's word size: on ILP32 the suffix multiplier
alone exceeds the bound, making every P and T value an error whatever the digits
and capping the option at 2047M. Because max_alloc_arg goes on the wire
un-normalized, a 0 was re-resolved by each side against its own SIZE_MAX; any
literal is resolved once on the client and shipped verbatim, so nothing above
2047M survives a 64-bit client talking to a 32-bit daemon. There is no number a
user can compute that does what 0 did, which is what #1056 ran into.
So accept 0 again, but resolve it to SIZE_ARG_MAX (SIZE_MAX/2) rather than
SIZE_MAX, so it lands exactly on the largest value that could also be typed and
is no longer a limit only the 0 spelling can reach. Keep forwarding it
verbatim: that per-side resolution is the property worth having. This leaves
the substantive half of the 3.5.0 hardening -- bounding parse_size_arg() against
the unbounded `size *= atof(size_arg)` -- untouched.
The residual concern, a 0 forwarded to a <= 3.2.7 daemon that honours it, is not
something a client-side check can address: the client is the attacker's own
code, as daemon-max-alloc-zero_test.py noted in its own docstring. Operators
who want peers kept off their cap have `refuse options = max-alloc`.
Also drops the now-unreachable rejection message, which left the min-value error
recommending a value the parser refused ("min: 1.00M or 0 for unlimited").
Tests: max-alloc-zero replaces max-alloc-zero-rejected and
daemon-max-alloc-zero, whose assertions are the behaviour being reverted. It
checks that 0 is accepted, that it reaches the peer as the literal "0" rather
than a resolved number (verified against a negative control that normalizes it),
and that the parser's upper bound still rejects an out-of-range value.
Fixes#1056
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* testsuite: refresh Cygwin expectations
* docs: note max-alloc safety
Zen Dodd