Relocate do_ftruncate/do_lseek/do_fallocate/do_punch_hole out of
syscall.c into vfs/fileio.c as the vfs_* names, declared in vfs/vfs.h.
The SUPPORT_PREALLOCATION / HAVE_FALLOCATE / HAVE_SYS_FALLOCATE /
FALLOC_FL_PUNCH_HOLE guards travel verbatim. This was the last
operation family: syscall.c no longer defines any filesystem wrapper.
No behavior change. (Portability-sensitive; wants a fleettest.)
Relocate the timestamp wrappers (do_utimensat/_at/_atfd, do_lutimes,
do_utimes, do_utime) and the crtime paths (do_setattrlist_times/_crtime,
get_create_time, do_SetFileTime) out of syscall.c into vfs/times.c as the
vfs_* names. The struct create_time / #pragma pack / Cygwin windows.h
and sys/attr.h includes travel with them, as do the SUPPORT_CRTIMES /
HAVE_SETATTRLIST / HAVE_GETATTRLIST / HAVE_UTIMENSAT / HAVE_LUTIMES /
HAVE_UTIMES / HAVE_UTIME guards. No behavior change. (Portability-
sensitive; wants a fleettest.)
Relocate do_mknod/do_mknod_at/do_mknod_atfd out of syscall.c into
vfs/mknod.c as the vfs_* names, declared in vfs/vfs.h. The
HAVE_MKNOD/HAVE_MKNODAT/HAVE_MKFIFO guards and the AF_UNIX socket-bind
fallback (with its <sys/un.h> include) travel verbatim. No behavior
change. (Portability-sensitive family; wants a fleettest.)
Relocate do_lchown/do_lchown_at/do_lchown_atfd out of syscall.c into
vfs/chown.c as the vfs_* names, declared in vfs/vfs.h. The HAVE_LCHOWN
fallback guard travels verbatim. No behavior change.
Relocate do_mkdir/do_mkdir_at/do_mkdir_atfd and do_mkstemp/
do_mkstemp_atfd/secure_mkstemp out of syscall.c into vfs/mkdir.c as the
vfs_* names (secure_mkstemp -> vfs_secure_mkstemp). The
trim_trailing_slashes path helper (used by trimslash and the mkdir
wrappers) moves with them, keeping its name and now declared in vfs.h.
The static rand_bytes helper, used only by the mkstemp create loop,
moves along too. No behavior change.
Relocate do_link/do_link_at/do_link_atfd out of syscall.c into
vfs/link.c as the vfs_* names, declared in vfs/vfs.h. The
HAVE_LINK/HAVE_LINKAT guards travel verbatim. No behavior change.
Relocate do_symlink/do_symlink_at/do_symlink_atfd and do_readlink/
do_readlink_atfd out of syscall.c into vfs/symlink.c as the vfs_* names.
The fake-super (NO_SYMLINK_*XATTRS) placeholder handling travels
verbatim. vfs_readlink stays a function only in fake-super builds and a
macro -> readlink() otherwise (the rsync.h macro is renamed to match);
its vfs.h declaration is guarded accordingly. No behavior change.
Relocate do_chmod/do_chmod_at/do_chmod_atfd (and the leaf-safe
do_fchmodat_nofollow helper) out of syscall.c into vfs/chmod.c as the
vfs_* names, declared in vfs/vfs.h. The HAVE_CHMOD / HAVE_LCHMOD /
HAVE_SETATTRLIST / SYS_fchmodat2 platform guards travel verbatim.
Function bodies unchanged; no behavior change.
(Portability-sensitive family; wants a fleettest with the rest of the
#ifdef-heavy moves.)
Relocate do_open/do_open_at/do_open_atfd/do_open_nofollow/
do_open_checklinks out of syscall.c into vfs/open.c as the vfs_* names,
declared in vfs/vfs.h. Function bodies unchanged; no behavior change.
Relocate do_unlink/do_unlink_at/do_unlink_atfd and do_rmdir/do_rmdir_at
out of syscall.c into vfs/unlink.c as the vfs_* names, declared in
vfs/vfs.h. Function bodies unchanged; no behavior change.
Relocate do_rename / do_rename_at / do_rename_atfd out of syscall.c into
vfs/rename.c as vfs_rename / vfs_rename_at / vfs_rename_atfd, declared in
vfs/vfs.h. Function bodies unchanged.
Also centralize the option-global externs (dry_run, am_root, am_sender,
inplace, preserve_*, open_noatime, copy_*, insecure_links, module_id, …)
in vfs/vfs_internal.h, replacing syscall.c's local extern block, so each
relocated family picks them up from one place. No behavior change.
Relocate the stat/lstat/fstat wrappers out of syscall.c into vfs/stat.c
with the vfs_* names:
do_stat -> vfs_stat do_stat_at -> vfs_stat_at
do_lstat -> vfs_lstat do_lstat_at -> vfs_lstat_at
do_fstat -> vfs_fstat do_stat_atfd -> vfs_stat_atfd
do_lstat_atfd-> vfs_lstat_atfd
do_xstat_at stays a file-local helper. The x_stat/x_lstat/x_fstat
fallback macros in rsync.h now expand to the vfs_* names. First Phase-3
family move, so the shared RETURN_ERROR_IF* dry-run/read-only guard
macros (and the read_only/list_only externs they expand to) move from
syscall.c into vfs/vfs_internal.h where every vfs/ source can use them.
Function bodies unchanged; no behavior change.
Move the operator-path resolver-mode flag into vfs.operator_path_resolve.
Its definition leaves vfs/owner_walk.c, the extern declarations (in
backup/generator/receiver/util1 and vfs_internal.h) are dropped, and the
uses across the do_*_at wrappers and the vfs/ internals are updated to
the struct field directly. This completes moving the scattered VFS state
(dirfd cache, curr_dir, operator_path_resolve) into struct vfs.
No behavior change.
Move the logical-cwd globals into vfs.curr_dir / vfs.curr_dir_len. The
definition leaves syscall.c, the per-file `extern char curr_dir[]` /
`extern unsigned int curr_dir_len` declarations are dropped (the struct
is reached via the vfs.h `extern struct vfs vfs`), and the uses across
exclude/flist/log/main/util1 and vfs/secure_open are updated directly --
no compatibility alias macros. change_dir() writes vfs.curr_dir. The
separate curr_dir_depth global is untouched. No behavior change.
Replace the file-local dpc_* statics in vfs/dircache.c (anchor, base,
fd[], name[][], depth) with the vfs.dpc fields already declared in
struct vfs and initialized by the designated initializer in vfs/vfs.c.
Pure encapsulation: the cache logic is unchanged, DPC_MAXDEPTH becomes
the shared VFS_DPC_MAXDEPTH, and vfs_dircache_reset() now resets vfs.dpc
(consistent with vfs_init()). No behavior change.
Post-relocation cleanup (no behavior change):
- fix a doubled-prefix typo "vfs_vfs_owner_walk_parent" in the
vfs/owner_walk.c header comment
- vfs/vfs.c comment now points at vfs/dircache.c (not syscall.c) for the
live dpc_* cache
- drop unused extern decls left from assembling the moved files
(am_sender/insecure_links in owner_walk.c, curr_dir/curr_dir_len in
dircache.c)
Relocate the persistent ancestor-dirfd cache out of syscall.c into
vfs/dircache.c with the vfs_* public names:
open_dir_secure -> vfs_opendir
get_dir_fd -> vfs_get_dirfd
held_dir_path_fd -> vfs_path_dirfd
held_dfd_for -> vfs_cached_dirfd
reset_dir_fd_cache -> vfs_dircache_reset
The dpc_* cache statics and dpc_dir_fd stay file-local (they fold into
the vfs struct in a later commit). Function bodies are unchanged; the
callers in generator/receiver/sender/delete/util1/rsync are updated and
the five entry points declared in vfs/vfs.h. This completes moving the
security core (resolver, owner-walk, dirfd cache) out of syscall.c.
No behavior change.
Relocate the operator-supplied-path resolver out of syscall.c into
vfs/owner_walk.c with the vfs_* public names:
open_no_attacker_symlinks -> vfs_open_owner_walk
owner_walk_parent -> vfs_owner_walk_parent
The static helpers ona_open and abspath_step move with them, as does the
operator_path_resolve flag definition (commit-8 will fold it into the vfs
struct). Function bodies are unchanged; the call sites across the daemon
and option-parsing files are updated and the two entry points declared in
vfs/vfs.h.
With the owner-walk gone, syscall.c no longer references am_daemon, so
the now-dead `extern int am_daemon` declarations (the do_*_at wrappers
delegate to vfs_relpath_active) are dropped. No behavior change.
Relocate the race-safe resolver and its policy gates out of syscall.c
into vfs/secure_open.c, and give them the vfs_* public names that the
mainline code will use going forward:
secure_relative_open -> vfs_resolve_open
secure_relative_open_at -> vfs_resolve_open_at
secure_relpath_active -> vfs_relpath_active
symlink_optout_allowed -> vfs_symlink_optout_allowed
secure_walk_at stays file-local. The function bodies are unchanged; the
call sites across receiver/sender/generator/flist/util1/clientserver/
main/options and the test harnesses are updated to the new names, and
the four entry points are declared in vfs/vfs.h.
The resolver's only consumer of am_chrooted in syscall.c left with it,
so the now-dead `am_chrooted` is dropped from the do_*_at wrappers'
local externs (and the file-scope extern), which is otherwise unused.
No behavior change.
First step of relocating the security core. The component-walk dirfd
stack (struct dirstack + the ds_* helpers) and the module-confinement
helpers (path_has_dotdot_component, abspath_excluded_by_module,
open_anchor_dirfd) move verbatim out of syscall.c into vfs/dirstack.c.
The secure resolver (still in syscall.c) and the held-dirfd cache reach
them through a new private header vfs/vfs_internal.h.
The functions are byte-identical to before; only their linkage changes
(the ones syscall.c still calls become non-static; ds_path_push/
ds_path_pop/ds_push stay file-local). struct dirstack and
SECURE_OPEN_MAXSYMLINKS now live in vfs_internal.h so both sides see one
definition. vfs_internal.h also centralizes the option/daemon externs
the VFS internals read. No behavior change.
Introduce a vfs/ subtree that will house rsync's filesystem-handling code
(the do_* syscall wrappers, the race-safe path resolver, the held-dirfd
cache, the operator-path ownership walk and daemon module confinement),
separating those security-critical details from the protocol/transfer
logic. This first commit only stands up the layer; no code is moved yet,
so behavior is identical.
- vfs/vfs.h: public interface, included by rsync.h just after proto.h.
Declares "struct vfs" -- the single global that will hold the state
currently scattered across syscall.c statics (the dirfd cache, curr_dir,
operator_path_resolve) and the clientserver.c module_* globals -- plus
vfs_init().
- vfs/vfs.c: defines the global instance with a designated initializer so
the cache and module snapshot are safe by construction (a plain
definition would zero base/module_dirfd, making fd 0 look valid); the
t_*_secure harnesses never run main(), so this must not rely on
vfs_init(). vfs_init() resets between transfers (inert for now).
- Build: bundle the VFS into a static libvfs.a linked last on rsync and
every test harness, so later commits can move code out of syscall.o
without breaking a harness link (the linker pulls only what it needs).
AC_CHECK_TOOL(AR)/AC_PROG_RANLIB added for portable archiving; a
vfs/dummy config-file output creates vfs/ in VPATH builds.
- main.c calls vfs_init() early (establishes the call site).
Date the release and bring the security section up to the full set: it
described 20 CVEs, and the release fixes 33.
The thirteen later items are added in three groups -- the peer-triggerable
memory-corruption findings from the daemon-protocol fuzzing pass, the daemon
availability and access-control issues, and the two client-side ones.
Several changes were previously described here as carrying no CVE and now do,
so those claims are removed rather than left to contradict the advisories:
rsync-ssl's unverified TLS is CVE-2026-70454, the non-positive MSG_IO_TIMEOUT
is part of CVE-2026-70462, and the early-protocol argument-count bound is part
of CVE-2026-70464. What is left under "no CVE assigned" is only the proxy
header bounds and the xattr expansion cap.
A stable-backport branch runs a newer suite than its own code. fleettest
already reads testsuite/skiplist/backport.txt from the tree being built and
excludes those tests; runtests.py did not, so running the suite directly --
which is what the backport branches' CI job does -- tried to run tests that
base cannot support.
Read the same file from tooldir and drop its names from both the run and the
expected-skip set: an excluded test never runs, so leaving it in the expected
set would make the oracle demand a skip that cannot happen. A stale name is
an error rather than a silent no-op.
Also note that backport CI, once it exists, has to consume these lists the
same way fleettest does -- read backport.txt from the branch being built and
pass it as RSYNC_EXCLUDE -- or it will fail on every entry and get switched
off.
A test named in a backport's backport.txt never runs, so it cannot skip either.
If the suite's expected-skip list also names it -- the two --compress-threads
tests are declared as expected skips because they need --use-tcp -- the oracle
waits for a skip that can no longer happen and every pipe cell reports a skip
mismatch.
Emit a '-name' removal for those, as the per-target expect_skip_omit already
does. Only for names the spec actually contains: runtests rejects a '-name'
that removes a name nothing added, and most of a backport's exclusions (a test
for a feature it lacks) are not expected skips at all. Deciding that needs the
@FILE references expanded locally, which is what _expand_spec_names does.
v3.4.1 with this: 5/5 cells OK on ubuntu-2404, from 3 OK / 36 not OK across the
fleet before the mechanism existed.
Running the 3.5.0 suite against an older branch (--repo BACKPORT
--testsuite-repo .) reports a wall of failures that are not regressions: tests
for fixes the branch does not carry, and tests whose unit-test helpers its
Makefile cannot build. Both backport branches came back 3 OK / 36 not OK with
every distinct failure explained that way, which makes the run useless as an
oracle -- a real regression would not stand out.
A backport now declares those in its own testsuite/skiplist/backport.txt, read
from the tree being BUILT rather than the one providing the suite: only the
built tree knows what it lacks. The names go to runtests.py as RSYNC_EXCLUDE
rather than as an expected-skip declaration, because some of them fail rather
than skip and an expected-skip list cannot describe a failure.
The overlay that puts a newer testsuite/ onto an older tree is a merge with no
delete, so a file that exists only on the backport survives it. skiplist-spec
exempts the name from its every-list-must-be-referenced rule, since nothing
references this one by design.
Under inc_recurse the first flist (ndx_start == 1) has no parent entry of its
own, so recv_file_list() trusts the peer's "." entry to be the transfer root and
leaves parent_ndx at the flist_new() default of 0 -- dir_flist->files[0]. Only
S_ISDIR entries are appended to dir_flist, so a peer that sends "." with a
NON-directory mode keeps dir_flist->used at 0 while the basename strcmp still
passes: parent_ndx stays 0 and the consumers index a never-written slot.
Drives a real daemon with the pure-Python sender: an inc_recurse push whose only
flist is a regular file "." plus a regular file "a" (no directory anywhere, and
"." sorts lowest; "a" keeps file_total != 1 so the receiver doesn't divert into
recv_additional_file_list). The file list alone is what does it -- the
generator crashes in generate_files() before any transfer phase -- reproduced on
released 3.2.7, 3.4.0 and 3.4.1.
The oracle needs both halves: a positive control that the daemon logged
"receiving file list", and the condition-specific refusal. Accepting any
"rsync error:" line is not enough -- with "." sent as a valid directory and a
bogus file index, that form passes on "File-list index 1000000 not in 0 - 2"
without the crafted transfer root ever reaching the parser.
A fixed daemon has already refused the list and exited by the time the ndx-0
token is sent, so that send and the drain can hit a closed socket; Linux and
FreeBSD swallow it, Solaris, the other BSDs, macOS x86 and Cygwin raise
EPIPE/ECONNRESET. Treat it as an expected outcome, not a result.
The header records what this does not prove: it gates the attack shape rather
than the parent_ndx clause (only the parse-time transfer-root check fires on a
current build), it does not exercise the receiver-side consumer, and the
dereferenced slot is not guaranteed NULL since dir_flist->files[] comes from
realloc(), not calloc().
The fix shipped in the test10 snapshot but was never written up: safe_arg()'s
filename-mode buffer sizing disagreed with the writer, leaving an uninitialized
heap byte in the argument handed to the remote shell when --protect-args is off.
The daemon sets its deadline with time(NULL) (set_daemon_handshake_timeout,
io.c), and this test measured the elapsed time with CLOCK_MONOTONIC. Those
agree on a quiet machine and diverge on a stalled one: a virtualised guest
resyncs its wall clock after the host deschedules it, while monotonic keeps
its own count. The daemon then closes exactly when it meant to and the test
reports it closed early.
That is what a NetBSD CI run showed -- "closed after 39.55s, before the
expected timeout window (58.75s)" -- and it is the same shape as the macOS
failure that turned out to be the machine sleeping mid-test.
Measure the bound on the clock the daemon decides with. Monotonic still
drives the poll budget, where the job is only "do not hang forever".
On its own that would trade a false failure for a false pass, which is worse:
a refusal or a crash arriving just as the guest's wall clock caught up would
read as a clean timeout, and no diagnostic would fire because the test would
be green. So when the two clocks disagree -- wall says on time, monotonic
says early -- neither settles it, and the daemon has to have recorded its own
deadline firing. The offset of its log is taken before each observation, so a
timeout it logged earlier cannot vouch for this one.
Failures now carry both clocks and that window of the daemon's log, bounded
and with control bytes escaped. The clock note states the discrepancy without
concluding from it: a stalled host produces it, but so does an NTP step, and
either can accompany a real failure.
The same kernel-side missed wakeup on the other side of the connection: a
blocking connect() can sleep forever on a connection that is already
established, with the 4-tuple ESTABLISHED at both ends and the listener's
greeting queued unread. Without --contimeout nothing breaks it.
Wait for the connect with poll() in slices rather than blocking in the
kernel, re-checking the socket on each pass, and take the result from
SO_ERROR. A finished slice is not a failure -- looping is what re-examines
the socket and recovers a missed wakeup.
--contimeout is unchanged: the alarm still fires and the caller still
reports RERR_CONTIMEOUT. The per-address errno is now stashed before
close()/alarm() can overwrite it.
Measured the same way, against a real loopback daemon: 20 hangs in 48,000
connects before, 0 in 48,000 after, with equal wall clock. This is the half
of the OpenBSD flakiness that the socketpair_tcp() fix does not cover: the
--use-tcp pass talks to a real rsyncd over a port, so it hangs here rather
than in accept().
On OpenBSD a blocking accept() can sleep forever on a connection the kernel
has already completed: the 4-tuple is ESTABLISHED at both ends, the
connection is queued on the listener, and the accept()ing process is still
asleep in netacc. Nothing bounds that wait, so rsync hangs for good.
Poll the listener instead, with a non-blocking accept(), so a missed wakeup
costs another pass rather than the process. The accepted fd is put back
into blocking mode explicitly because BSD accept() gives it the listener's
non-blocking flag. A time(NULL) deadline bounds the whole wait the way
io.c bounds its own, rather than counting passes -- a signal on every pass
must not extend it and a poll() that returns at once must not consume it.
A listener that reports ready without yielding a connection (the peer can
reset first) pauses rather than spinning.
Measured on an OpenBSD 7.8 VM, driving the real binary through
RSYNC_CONNECT_PROG with 8 concurrent workers, alternating stock/patched
rounds: 111 hangs in 120,000 invocations before, 0 in 120,000 after, with
no change in throughput.
Every daemon test reaches socketpair_tcp() through RSYNC_CONNECT_PROG in
the default transport, so the hang landed on whichever daemon test happened
to be connecting. See dev-notes/openbsd-socketpair-accept-wedge.txt.
Nothing in CI or the fleet has ever set --enable-roll-simd, --enable-roll-asm
or --enable-md5-asm, which is why the over-read above sat behind a "fixed"
label for two months, and why the fix applied for it went to the wrong
assembly file.
mac-x86-asm is the same host and OS as mac-x86 with all three on. Mach-O is
the interesting part -- both problems reported against these flags were
macOS-x86-64 -- and it is the only machine in the fleet that can build the
x86-64 assembly at all.
It needs MacPorts clang 19 through CC/CXX, because Apple clang 10 (the ceiling
on macOS 10.13) rejects configure's target("default") multiversioning probe.
mac-x86 keeps the stock Apple compiler, which is what caught #161, so the two
cover different ground rather than one replacing the other.
simd-checksum is a macOS-wide expected skip, since simdtest is only built when
SIMD is enabled; this target subtracts it, because running it is the point.
Also corrects mac-x86's comment, which claimed the probe "cannot compile here
with any clang". It is a compiler-version limit: clang 19 on that same box
compiles, links and runs it.
simdtest allocated 64 spare bytes so it could test an unaligned buffer, which
is exactly the slack that hid a 64-byte over-read in the AVX2 assembly for as
long as it existed. Add a pass that places the buffer flush against a
PROT_NONE page, so a read past the end faults in the test rather than in
somebody's transfer.
Every length from 128 to 4096, so each remainder mod 64 and both alignments
are covered, and all four implementations are checked -- the assembly was the
one at fault here, but the intrinsic paths preload too.
It fails closed. A guard this test cannot set up means it is not testing what
the caller thinks, so a failed sysconf/mmap/mprotect is a failure rather than a
pass that looks identical to a real one. And because the dispatcher falls back
on a CPU without AVX2 -- where the guard loop proves nothing about the code
under test -- it says which of the two happened rather than letting a fallback
run read as coverage.
Without the fix this segfaults; the suite's simd-checksum test reports the
non-zero exit.
The loop is software-pipelined: each iteration folds in the 64 bytes it
preloaded last time and preloads the next 64. Nothing stopped the final
iteration doing that preload, so it always read the 64 bytes after the region
it was asked to checksum.
Not an edge case. The assembly processes len&~63 and leaves the remainder to
the caller, so the remainder is by construction under 64 bytes and the preload
passed buf+len on every call, by 64 minus the remainder.
It normally landed in slack inside the map_ptr() window and nothing noticed.
Where the buffer ended near an unmapped page it was a SIGSEGV in the middle of
a transfer -- reported on macOS x86-64 by Roland Kletzing, whose `partial` run
died with "connection unexpectedly closed" because the generator had crashed.
A guard page reproduces it on Linux too, so it was latent there, not absent.
Run the pipelined loop one block short and finish the last block in .last,
which does the same arithmetic without the preload. No per-iteration cost, and
checksums are bit-identical -- simdtest compares every implementation against
the C reference.
The earlier fix for that report, "lib: use .balign in md5 x86-64 asm", was to
the md5 assembly. It addressed the linker alignment warning that appeared
alongside, not this.
An LD_PRELOAD hook refuses linkat() for a symlink source only, so the arm is
reachable on a filesystem that hard-links symlinks perfectly well. Three
controls keep it from proving less than it looks:
- the regular file in the same transfer must still be hard-linked, or "it fell
back" would also be satisfied by --link-dest having been abandoned;
- the itemised run must emit exactly one "cL... sym -> some-target" line. A
plain -a run cannot see a duplicated itemisation, which is how that defect
reached an HFS+ target before this was added;
- EPERM and ENOSYS must fall back too, since errno does not separate "cannot"
from "may not".
itemize picked its expected change-type letter from the build capability, which
is the wrong question -- the link happens on whichever filesystem holds the test
data. Ask that one too, and drop the XFAIL the old mismatch needed.
The hook is Linux-only, so the test joins the macOS and Cygwin skip lists,
which are required to be sorted.
CAN_HARDLINK_SYMLINK and CAN_HARDLINK_SPECIAL are decided by configure running
linkat() on whatever filesystem the build tree happened to sit on. The
destination is free to disagree, and one host can hold both answers: macOS
builds on APFS, which can hard-link a symlink, and backs up to HFS+, which
returns ENOTSUP.
A build that said yes had no fallback left. try_dests_non() reported the
refusal as a transfer error and returned a matched basis, so the caller created
the entry anyway -- correctly -- and the run still exited 23. Every
neighbouring case copes: a regular file whose link() fails goes to try_a_copy,
and a build compiled without either macro resorts to --copy-dest behaviour.
This was the same situation, discovered a little later, and the only one
treated as fatal.
Take the existing fallback on any refusal. Singling out the "cannot" errnos is
not possible: link(2) documents EPERM both for a filesystem with no hard-link
support and for an ordinary permission refusal, and FUSE reports ENOSYS for the
same thing. It is also what the regular-file path next door has always done
(try_dests_reg -> hard_link_one -> try_a_copy), and consistency between the two
was the point. Where the errno does matter the surrounding transfer says so
anyway: ENOSPC, EDQUOT and EROFS fail the creation independently, EMLINK and
EXDEV mean the link was never possible. EIO alone goes unremarked; reporting
it would put a line into --link-dest's itemised output, so it is left out on
purpose.
Returning -3 rather than -2 keeps the caller out of the "already up to date,
skip it" arm, which under --link-dest would drop the entry entirely. Both
callers give -3 the treatment the compile-time fallback already gets -- clearing
itemizing and code -- because try_dests_non() has itemised the match itself and
would otherwise report the entry twice.
The fallback is silent, matching a build that cannot link these at compile
time; documented under --link-dest instead.
Uses the exclude-only merge form, which leaves no diagnostic to assert on: the
escape shows up as a file silently missing from the transfer, so the test reads
the oracle the same way an attacker would. Pull mode, so no --delete is
involved. Each case requires the transfer to have succeeded as well, since
refusing outright would hand the peer a denial of service.
A second escape reaches the source through a symlink, which is what makes
rsync's tracked cwd and the real one disagree -- the shape that catches a
lexical seed. That one drives --confine-root directly: rrsync rejects the
argument spellings that would carry it, so routing it through the wrapper would
pass either way and prove nothing.
Both controls repeat their escape with an in-tree merge target and require it
to be read AND obeyed, since "the transfer failed" and "every merge file is
refused" would otherwise satisfy the escape assertions on their own.
Filter rules arrive over the protocol, long after the wrapper has exec'd rsync,
so no argv-level check can see them. A client can name a merge file outside
the restricted dir in a dir-merge rule and have the server read it in as filter
rules; on a pull that needs neither --delete nor any verbosity. Pass
--confine-root so the server bounds the open itself, which is the only end that
can.
Both directions: a dir-merge is read by whichever side its rule applies to, so
unlike --drop-D this is not receiver-only. Skipped for a "/" restricted dir,
where there is nothing to confine.
The ownership walk that resolves operator-supplied paths asks who planted a
symlink, not where the path came out, so a symlink owned by uid 0 or the euid
is followed wherever it points. A daemon already narrows that with the served
module root; nothing else has a root to narrow it with.
That leaves a wrapper serving a restricted directory over a remote shell with
no way to bound the resolution. rrsync can vet the argv it is handed, but
filter rules travel over the protocol instead: a dir-merge rule can name a
merge file outside the restricted dir and the server reads it in as rules.
Redacting the resulting diagnostics does not close it, because an exclude-only
merge produces none -- every line becomes a pattern, so nothing fails to parse,
and the client reads the file's contents off which of its own names went
missing from the file list.
--confine-root gives that wrapper the root the daemon has. The existing module
check becomes a root check that takes its root from module_dir when we are a
daemon and from the option otherwise, so daemon behaviour is unchanged; a
daemon ignores the option outright, since it arrives in a peer-supplied argv
and could only widen the module.
The tracker is seeded from getcwd() rather than curr_dir, which is only the
lexical name change_dir() was given: descend into a source argument through a
trusted symlink and the two sit at different depths, so a ".." that really
escapes looks like it landed inside. When the cwd cannot be read there is
nothing to measure against and the open is refused -- an empty tracker does not
deny by itself, because a leading ".." pops nothing from it and an empty path
reads as an ancestor of the root.
An fd pin (/proc/self/fd/N, which rrsync uses so no later symlink can redirect
a validated option path) is spelled outside the root by construction, so the
walk transits the pin namespace and the pin is judged by what it points at.
Only a bare ".../fd/<digits>" is resolved that way, and one that will not
readlink to an absolute path is refused; rrsync's ".../fd/N/<leaf>" spelling
resolves through the magic link and has its remaining components checked
normally.
--insecure-links is refused alongside it: that opt-out returns the legacy open
before the walk that enforces the root runs, so the pair would have quietly
meant no confinement at all.
The exclude-self rule that a ":e" merge synthesizes is built by hand with
new0(), so it inherited no flags. While the merge file was still being
parsed the global parse state masked that, but once parsing finished the
stored rule looked argument-origin, and report_filter_result() printed its
pattern -- a merge file's own text -- verbatim:
[sender] hiding file PAT-x9 because of pattern PAT-[x]9 [per-dir ...]
Plain -vv reaches this on a stock client; no --debug is involved. That is
the fifth site of this shape, and the first to get there by constructing a
rule rather than by printing one, so the redaction helper could not catch it.
Also fix the location a per-directory merge reports. Its fname points into
dirbuf, which is cut back to the directory before the name was saved, so the
error said "<rule from .../src/ line 1>" instead of naming .rsync-filter --
no leak, but it breaks the "redact what, keep where" bargain the rest of this
work depends on. Save the name before the truncation.
The rrsync test's claim to close "the rest of the FILTER trace family" was
too strong and is corrected: options.c maps verbosity onto the debug flags,
so -vvv still raises a restricted server to FILTER2 and its trace metadata
comes back. Rule text stays redacted at every verbosity, which is the
property that matters; -vvv is added to the unaffected-transfer cases.
The --debug=FILTER traces print rule text and merge-file names that came
out of a file's contents -- and a word-split per-dir merge (":w- FILE")
turns every word of a file into a merge-file name, so the trace echoes
what the syntax errors no longer do, with nothing failing to parse.
Redacting every trace would mean carrying provenance on each rule, which
a deferred ":" merge does not currently keep. For a restricted account
the cheaper answer is to deny the peer the switch: server_options() only
ever forwards --info, so no stock client sends --debug to a server and
the only way it arrives is a deliberate -M--debug=. An operator
debugging their own server is unaffected.
Disabled rather than deleted from the table, because that table is
generated by the cull-options script and a regeneration would put the
line back; the test would then catch it.
A filter rule that fails to parse was printed back verbatim. When the
rule came from a file rather than an argument, that text is file
CONTENT, and the peer picks which file gets merged: a per-directory
merge rule travels over the protocol, so no argument of ours ever names
it and nothing a wrapper can see mentions it either. Any line that is
not valid filter syntax therefore came straight back to the peer -- a
read-any-line oracle over an rrsync restricted account or a daemon
module, neither of which confines the merge open.
The syntax errors turned out to be the smaller half. The MATCH trace
names the pattern that acted, and report_filter_result() logs at level 1
for a sender or generator, so plain -vv -- no --debug, nothing a stock
client cannot send -- returns a server-side merge file's rules:
[generator] protecting file X because of pattern <the file's text>
So provenance is carried on the rule itself (FILTRULE_FROM_FILE), not
just in the parser: a deferred ":" merge is processed long after the
file that named it was read, and its own name is file content too.
TEXT_FROM_FILE() consults the parse-time context and the rule, so both
the immediate and the deferred paths redact.
Rather than test the provenance at each message -- which is how the last
few of these were found, one at a time, after the ones before them were
fixed -- every string that is or is built from a rule's own text goes
through rule_text(). It returns the text for an argument-supplied rule
and a description of where it came from otherwise, so a message added
later cannot reintroduce the leak by forgetting to check, and there is
one place to audit. rule_detail() does the same for the extra detail a
message adds ABOUT the text: a character of it, an offset into it, the
[not found] bit.
Thirteen sites now route through them: the syntax errors; the modifier
character (one byte of the file, a slower oracle but still one); the
failed-open and merge-depth messages, whose pathname is file content
whenever a rule named it -- and errno with them, since it answers "does
this path exist"; both over-long messages, the deferred one of which
needed no verbosity at all; both merge-name overflows; the long-named
directory error; the [not found] openability bit; the match trace; the
add_rule, parse_filter_file and daemon-hidden traces; and the per-dir
mergelist label, which had the name baked in.
rule_detail() covers more than it first looks: the trailing-whitespace
CAUTION is computed from the rule's last byte, and "hidden by daemon
filter" distinguishes a daemon-filter rejection from an ordinary open,
so both would answer questions about text the peer cannot see.
The regression proves the chokepoint rather than the sites: making
rule_text() return its input unconditionally fails the test. It also
pins what must NOT change for the user's own rules -- the whitespace
warning still fires, and an over-long argument rule is still reported at
full length (the helper buffers at BIGPATHBUFLEN, as rprintf does, so
redaction does not quietly truncate what the user typed).
Bounded and left alone: the numeric rflags in the FILTER2 trace and the
in/exclude wording still describe a file-derived rule without quoting
it, and the daemon's own FLOG line records the name it filtered -- that
one goes to the operator's log, not the peer.
Rules given AS arguments are still echoed in full -- that text is the
user's own, and hiding it would only make ordinary typos harder to fix.
Where a rule did come from a file, the diagnostic names the file and
line instead, which is more useful anyway.
Two things the location itself needed: fname can point into
parse_merge_name()'s static buffer, which a merge rule inside the same
file overwrites while we are still reading it, so a rule after a nested
merge was blamed on the nested file -- keep our own copy. And a CRLF
pair was counted as two line endings while word-split mode counted
tokens rather than lines, so the number pointed at nothing; consume the
LF of a CRLF (preserving the byte for the next rule if pushback ever
fails), and report word-split sources without a line number.
Not covered, deliberately: a rule's provenance is not serialized by
send_filter_list(), so it does not survive to the far side. That is
right -- only the client sends that list, and the server already knows
the patterns the peer gave it.
A daemon test could leave an orphaned rsyncd squatting its port even when
every test PASSED, so nothing in the results pointed at it. On Cygwin the
orphan then wedged the whole fleet: it kept the ssh session from closing,
so fleettest's run_on() blocked until its 2400s timeout and unrelated
tests failed with 300s timeouts as collateral. One such wedge cost a fleet
run 21 minutes.
Cause: rsyncd forks a child per connection, but _stop_rsyncd only killed
the parent -- the one pid the Popen handle knows. A child still winding up
or down when the test ended survived, inherited the listening socket, and
was reparented to init. Cygwin turned that from untidy into unrecoverable:
its signals are cooperative, delivered by a helper thread inside the
target, so a process sitting in a Windows call ignores even SIGKILL. kill,
killpg and pkill all failed against it, which also defeated the orphan
reapers and fleettest --cleanup.
Snapshot the daemon's children before killing it (once the parent is gone
they are reparented and no longer identifiable as ours) and kill them too,
re-checking with _pid_is_rsync before each signal so a pid recycled in the
meantime is never signalled. Where signals cannot win, fall back to
terminating the winpid via taskkill; fleettest --cleanup gets the same
fallback, so it can no longer report SURVIVED and leave the port squatted.
_reap_group() reports success only once the daemon is confirmed gone
rather than when a signal was merely accepted -- on Cygwin a signal is
routinely accepted by a process that then ignores it -- and confirms with
a bounded poll, because SIGKILL is asynchronous and calling a
still-terminating process "alive" would make _probe_bindable() skip its
retry and fail a test for a port that was about to free itself.
_cleanup_rsyncd() keeps the port's pid record only while it still names a
live rsync. Keying that on the port being busy instead looks safer but is
worse: a port sits in TIME_WAIT after a passing test, so a record naming
an already-dead pid would be retained forever, and nothing clears such a
record -- yet no reaper can use it either, since they all reject it at the
_pid_is_rsync guard, leaving only the hazard that its pid is recycled onto
an unrelated rsync.
The daemon stays in the TEST's process group on purpose: runtests.py
killpg's that group on a per-test timeout, and that is what keeps a
timed-out test from stranding its daemon. An earlier version of this fix
gave the daemon its own group so one killpg would catch the children --
which silently broke that sweep, and a full Cygwin pass then stranded two
parent daemons when variety hit its timeout.
Two residual limitations are documented in the code rather than left to be
rediscovered: _kill_pid's check-then-signal is inherently a TOCTOU
(narrowed to microseconds, not closed; closing it needs pidfd or retained
Windows handles across seven platforms), and _stop_rsyncd cannot collect
children when the parent has already exited on its own, because the
parent-child link it relies on is gone by then.
Measured on a Cygwin VM, 4 proxy/daemon tests x 8 runs at -j4: before 5/8
runs left an orphan (one left two), after 0/10. All tests passed in every
run, before and after -- which is the point: the leak was invisible to the
suite. A test killed by the runner's timeout still leaves no daemon
behind.
Andrew Tridgell