Compare commits

..
Author SHA1 Message Date
Zen Dodd 7fef1c827e docs: correct devices and specials behaviour (#1078) 2026-09-04 16:54:28 +10:00
Zen Dodd 59be391641 syscall: explain untrusted symlink refusal (#1074)
* syscall: explain untrusted symlink refusal
* syscall: emphasise insecure-links warning
2026-09-03 17:05:03 +10:00
Omar Elsayed a93490077d batch.c: Allow FIFO pipes in batch file processing. Regression fix (#1060) 2026-09-03 17:03:54 +10:00
Omar ElsayedandZen Dodd a68b32cd5c Fix ENOENT when resolving kernel pseudo-paths in ona_open (#1054)
* Fix ona_open to safely resolve bash process substitution pseudo-paths
Bash process substitution (e.g., `<(...)` or `>(...)`) exposes file
descriptors as symlinks under `/proc/self/fd/X` pointing to kernel
pseudo-paths such as `pipe:[12345]`. Previously, `ona_open()` would read
this target and attempt to resolve it as a literal file path on disk,
causing the operation to fail with `ENOENT` and breaking legitimate local
process substitution.

This patch safely intercepts and resolves these pseudo-paths while
maintaining strict confinement boundaries and averting TOCTOU risks:

- Detects kernel pseudo-paths (`pipe:[`, `socket:[`, `anon_inode:`)
  only when `fd_pin_tail` confirms the path resolves precisely to a
  direct child of a valid FD directory.
- Categorically rejects pseudo-path resolution if `confine_root` is
  active (yielding `ENOENT`).
- Strips `O_NOFOLLOW` for legitimate leaf pseudo-paths, allowing
  `openat()` to correctly delegate resolution.
- Reverts `fd_pin_tail` to its upstream signature, as manual PID
  validation is no longer required due to the secure `openat()` design.

* testsuite: expect pseudo-path skip on Alma
* syscall: reject trailing pseudo-path components

---------

Co-authored-by: Zen Dodd <mail@steadytao.com>
2026-09-03 08:11:16 +10:00
LIChengGang a4bfa170fa main: allow --contimeout for daemon connections over --rsh (#1071)
* main: allow --contimeout for daemon connections over --rsh

The --contimeout guard rejected the option whenever connect_timeout was
set and the connection was not being made by a socket, so a daemon
reached through a remote shell (daemon_connection == 1) was refused the
same way as a plain non-daemon remote-shell transfer.  That is how
rsync-ssl talks to a daemon: it runs rsync with --rsh pointing at its
helper, so "rsync-ssl --contimeout=60 host::mod" died with "The
--contimeout option may only be used when connecting to an rsync
daemon" before it could even connect.

Only reject --contimeout when there is no daemon connection at all, so
the option is accepted for a daemon-via-rsh connection (and still works
unchanged for the socket daemon path, which returns earlier through
start_socket_client).

Add a regression test that drives a daemon-via-rsh invocation and
asserts the option is not rejected, while a plain remote-shell
destination is still refused.

* main, io: honor --contimeout for daemon-via-rsh connections

The previous commit made --contimeout legal for a daemon reached
through a remote shell, but rsync still did not act on it: the socket
path times its connect() via an alarm, while the daemon-via-rsh path
spawned the helper and then blocked in start_inband_exchange() waiting
for the daemon greeting with no connect timeout at all.  A hung helper
-- e.g. rsync-ssl's openssl connect to an unreachable host -- would hang
the transfer forever.

Give the client a second wall-clock deadline, client_connect_deadline,
set from --contimeout around do_cmd() + start_inband_exchange() and
cleared once the greeting exchange finishes.  handshake_poll_timeout_ms()
now also checks it and exits with RERR_CONTIMEOUT ("timeout waiting for
daemon connection"), matching the socket path's exit code, instead of the
daemon-handshake deadline's RERR_TIMEOUT.

This covers the helper spawn, its connect/TLS handshake, and the greeting
exchange as one connection-establishment phase, so it works uniformly for
openssl, stunnel, and gnutls rsync-ssl backends.

Extend the contimeout-rsh test to drive a helper that never connects and
assert rsync aborts with exit 35 within the timeout, and that a working
daemon-via-rsh connection still completes.
2026-09-03 08:10:05 +10:00
Samuel Henrique c529eccff7 options: accept --max-alloc=0 again, resolved to the parser's own ceiling (#1069)
* options: accept --max-alloc=0 again, resolved to the parser's own ceiling

3.5.0 rejected --max-alloc=0 (CVE-2026-53794).  The stated rationale was that a
zero cap "disabled the per-allocation size cap (the defense behind
CVE-2024-12084)".  That was true up to 3.2.7, where the check short-circuited:

	if (max_alloc && num >= max_alloc/size)

but 2f9b963a ("Make `--max-alloc=0` safer", 3.3.0) removed the short-circuit and
mapped 0 to SIZE_MAX at parse time, leaving the guard unconditional.  Since the
guard admits an allocation only when num < max_alloc/size, num*size stays below
max_alloc at every setting, so the num*size overflow check was armed for 0 just
as for any other value.  From 3.3.0 onward, 0 raised the magnitude ceiling and
nothing else -- and an explicit 8191P raises it exactly as far, is accepted, and
is forwarded to the peer, so rejecting 0 removed no capability.

What it did remove is the only portable spelling.  The parser's ceiling is
SIZE_MAX/2, so it tracks the build's word size: on ILP32 the suffix multiplier
alone exceeds the bound, making every P and T value an error whatever the digits
and capping the option at 2047M.  Because max_alloc_arg goes on the wire
un-normalized, a 0 was re-resolved by each side against its own SIZE_MAX; any
literal is resolved once on the client and shipped verbatim, so nothing above
2047M survives a 64-bit client talking to a 32-bit daemon.  There is no number a
user can compute that does what 0 did, which is what #1056 ran into.

So accept 0 again, but resolve it to SIZE_ARG_MAX (SIZE_MAX/2) rather than
SIZE_MAX, so it lands exactly on the largest value that could also be typed and
is no longer a limit only the 0 spelling can reach.  Keep forwarding it
verbatim: that per-side resolution is the property worth having.  This leaves
the substantive half of the 3.5.0 hardening -- bounding parse_size_arg() against
the unbounded `size *= atof(size_arg)` -- untouched.

The residual concern, a 0 forwarded to a <= 3.2.7 daemon that honours it, is not
something a client-side check can address: the client is the attacker's own
code, as daemon-max-alloc-zero_test.py noted in its own docstring.  Operators
who want peers kept off their cap have `refuse options = max-alloc`.

Also drops the now-unreachable rejection message, which left the min-value error
recommending a value the parser refused ("min: 1.00M or 0 for unlimited").

Tests: max-alloc-zero replaces max-alloc-zero-rejected and
daemon-max-alloc-zero, whose assertions are the behaviour being reverted.  It
checks that 0 is accepted, that it reaches the peer as the literal "0" rather
than a resolved number (verified against a negative control that normalizes it),
and that the parser's upper bound still rejects an out-of-range value.

Fixes #1056

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* testsuite: refresh Cygwin expectations
* docs: note max-alloc safety
2026-09-03 07:42:34 +10:00
Zen Dodd 270202affb ci: adjust Cygwin common skip expectations (#1073) 2026-09-03 07:22:38 +10:00
Zen Dodd 90992088d6 ci: apt-update before install (#1072) 2026-09-03 06:43:29 +10:00
Zayd Rajab 240bd9df96 syscall: use O_PATH for held directory traversal (#1065)
Use O_PATH on Linux for directory descriptors used only for path
traversal, fchdir(), or as *at() parents. Reopen final directory
endpoints with the caller-requested access mode and retain the existing
O_RDONLY fallback elsewhere.

Add coverage for exact sources and readable descendants beneath
search-only directories, known-file creation beneath write/search-only
destinations, and retained refusal to enumerate unreadable directories.

Fixes #1064.
2026-08-28 19:37:01 +10:00
LIChengGang 7c20b077c9 rsync-ssl, testsuite: accept --type=SSL_TYPE anywhere in the args (#1047)
* rsync-ssl, testsuite: accept --type=SSL_TYPE anywhere in the args

The --type=... option was only recognized as the first argument, so a
command such as "rsync-ssl --dry-run --type=stunnel host::mod" passed
the option through to the underlying rsync, which rejected it as
unknown.  Scan the full argument list for --type=..., export
RSYNC_SSL_TYPE, and drop the option before handing the remaining args
to rsync.  The manpage no longer says the option must be first.

Adds a test that runs rsync-ssl with a fake rsync in PATH and checks
that --type= is consumed in first, middle, and last positions.

* fix(rsync-ssl): stop interpreting wrapper options at --

rsync-ssl consumed a --type=... operand that appeared after a --
argument, even though -- explicitly protects the rest of the command
line from option parsing.  Stop scanning for --type=... at a --
argument: preserve the -- and every subsequent argument verbatim,
passing them through to rsync unchanged.  Add regression coverage for
this case and document the behavior in the manpage.

* test(rsync-ssl): assert RSYNC_SSL_TYPE and exit status in the type-option test

The rsync-ssl-type-option test only checked that --type= tokens were
removed from the rsync argv, so it would also pass if the wrapper
silently discarded the requested SSL implementation instead of
exporting RSYNC_SSL_TYPE.  The fake rsync now records both the argv it
receives and the RSYNC_SSL_TYPE value it observes; each invocation
asserts the expected value (or UNSET when the wrapper must not consume
anything), and run() fails the test unless rsync-ssl exits
successfully.
2026-08-20 19:27:55 +10:00
ᴄʜʀɪsᴛᴏᴘʜᴇʀ ᴍᴇɴɢ f0177d82a8 Add IDN support (#986)
* Add IDN support

rsync can now connect to IDN (internationalized domain name) hosts, and
IDN names are recognized in a daemon's hosts allow/deny.

* idn: convert host names label by label

The IDNA mapping folds some non-ASCII characters onto ASCII ones, so
running a whole hosts allow/deny token through idn2_to_ascii_8z() could
hand back a pattern the admin never wrote: a "*" (U+FF0A FULLWIDTH
ASTERISK) entry came back as "*" and let every host in.

Convert label by label instead, keeping an ASCII label byte for byte and
using a converted label only when it comes back as a bare A-label. An
ASCII-only config now behaves as it did before there was IDN support, and
a token that cannot be converted is left alone and so matches nothing. The
client side shares the same helper, and neither side truncates a name at
its 1024-byte buffer any more. strlower() folds only ASCII now, since its
one caller is the hosts allow/deny list, which can hold UTF-8.

Adds testsuite/daemon-access-idn and extends testsuite/idn to cover
Unicode, punycode, mixed-case and invalid input on both sides.

* idn: hand libidn2 the same flags on both sides

The client path called idn2_lookup_ul() without IDN2_NFC_INPUT while the
daemon path passed it to idn2_to_ascii_8z(). Both normalize either way --
idn2_lookup_ul() ors the flag in itself, and TR46 normalizes as it maps --
but there is no reason for the two calls to read differently, so pass one set
of flags from one place. The flag asks libidn2 to normalize the label rather
than promising that it already is: it gates the u32_normalize() call, and
without it a decomposed label comes back IDN2_NOT_NFC.

Adds composed/decomposed cases to testsuite/idn, which sees the exact host
name rsync hands out, and a decomposed hosts allow token to
testsuite/daemon-access-idn.
2026-08-20 18:51:23 +10:00
Samuel Henrique 0145b9128d testsuite: interpose the large-file spellings of open/openat/fstatat (#1063)
Which symbol names this test's LD_PRELOAD hook exports is decided by the
compiler that builds it, and which names rsync imports is decided by
configure -- and the two do not have to agree.

Where off_t is not already 64 bits, configure's AC_SYS_LARGEFILE adds
-D_FILE_OFFSET_BITS=64 (i386 and alpha in Debian), so glibc redirects every
open()/openat()/fstatat() call in rsync to open64(), openat64() and
fstatat64().  The hook is compiled by a bare "cc", so on those architectures
it defines only the unsuffixed names: the receiver's opens never reach it, no
EACCES is injected, the marker the positive control looks for is never
written, and the test fails with

    positive control failed: receiver did not open the existing partial file
    with O_CREAT (rc=0, output='')

It happens to work on Debian's 64-bit time_t ports (armhf, hppa, powerpc,
...) only by luck: their gcc predefines -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64,
so glibc's __REDIRECT renames the hook's own DEFINITIONS as well and it ends
up exporting exactly the *64 names rsync imports.

Define both spellings explicitly so the hook interposes whichever set the
rsync under test was linked against, and #undef the two macros at the top of
the hook so that renaming cannot happen -- otherwise, on precisely those ports
whose compiler predefines them, open() would be emitted as open64() and
collide with the explicit wrapper ("symbol `open64' is already defined"),
leaving the hook unbuildable and the test skipped.

The new pointers are resolved through hook_resolve(), so the existing
nested-dlsym recursion guard covers them as well.

Failing build logs:
  i386  https://buildd.debian.org/status/fetch.php?pkg=rsync&arch=i386&ver=3.5.0%2Bds1-1&stamp=1786938290&raw=0
  alpha https://buildd.debian.org/status/fetch.php?pkg=rsync&arch=alpha&ver=3.5.0%2Bds1-1&stamp=1786973432&raw=0
2026-08-19 20:37:09 +10:00
Samuel Henrique 9371b33320 testsuite: probe the punch granularity each hole assertion relies on (#1062)
The last section of preallocate_test.py writes 256 blocks of
[4 KiB data][24 KiB zeros][4 KiB data] and requires --inplace --sparse to
deallocate at least half of the file.  A punch only frees whole allocation
units, and that interior run sits 4 KiB into each 32 KiB block: with a 4 KiB
unit it covers six whole units (24 KiB freed per block), but with a 16 KiB
unit it covers none at all -- [0,16K) holds live data at [0,4K) and [16K,32K)
holds live data at [28K,32K) -- so st_blocks cannot move whatever rsync does.

That fails the build on Debian loong64, whose kernel uses 16 KiB pages:

    --inplace --sparse left matching interior zero runs allocated: 8388608 of
    8388608 bytes remain allocated after a 8388608-byte matched-block update

i.e. exactly "nothing was freed".  fs_can_punch_holes() does not catch it
because it punches a whole 64 KiB file, which frees blocks at any granularity.

Generalise that helper into punch_frees(offset, length, size) so each
assertion can probe the shape it actually depends on, and gate the interior
one on the exact layout written just above it.  Where the filesystem can free
that run the assertion runs exactly as before; where it cannot, the test says
so rather than reporting a regression.  The content check is unconditional
either way.

While here, call fallocate64() instead of fallocate().  ctypes declares the
offset and length as c_longlong, but glibc's fallocate() takes 32-bit off_t
where off_t is 32 bits, so the callee reads the high half of `offset` as its
`length`, the call fails with EINVAL, and can_punch comes back False -- which
is why none of this file's hole-punching assertions have ever run on a 32-bit
port.  On i386:

    fs_can_punch_holes() as written : before=128 ret=-1 errno=22 -> False
    the same probe via fallocate64  : before=128 ret=0            -> True

fallocate64() takes off64_t on every architecture and is a plain alias of
fallocate() where off_t is already 64 bits.  The probe also writes urandom
rather than a repeated byte, so a filesystem that compresses does not answer
"nothing was freed" merely because nothing was allocated.
2026-08-19 20:26:04 +10:00
Zen Dodd 7f01ae0af1 testsuite: sort platform skip lists (#1061) 2026-08-16 22:02:30 +10:00
Zen Dodd 3d2caaa0cb testsuite: bound the unshare probe (#1049) 2026-08-16 21:55:00 +10:00
Zen Dodd 3b1eb8dd7f syscall: use O_PATH for directory traversal (#1052) 2026-08-16 21:52:11 +10:00
Zen Dodd 324a1f0716 rrsync: support fd pins in user namespaces (#1048)
* rrsync: support fd pins in user namespaces
* rrsync: support /dev/fd pins in user namespaces
2026-08-16 21:51:07 +10:00
Zen Dodd 010aa15dec testsuite: test install-strip tool selection (#1027) 2026-08-16 21:50:52 +10:00
Alessandro Di NepiandClaude Opus 4.8 a49f085a4f Honor $(STRIP) in install-strip for cross-compilation (#1024)
* Honor $(STRIP) in install-strip for cross-compilation

The install-strip target hard-coded `install -s`, which strips via the
install program using the build host's strip and ignores the STRIP
variable. When cross-compiling this runs the host strip against a
target binary and fails.

Pass --strip-program=$(or $(STRIP),strip) so the target strip is used
when STRIP is set (as cross toolchains and build systems provide),
falling back to plain `strip` for native builds. A plain `make install`
is unaffected.

* Make install-strip portable (address review)

- Detect the target strip via AC_CHECK_TOOL([STRIP],[strip],[strip]) in
  configure.ac (picks up the cross-prefixed strip when cross-compiling,
  defaults to plain strip otherwise) and substitute @STRIP@ in Makefile.in.
- Rewrite install-strip to run a normal install then $(STRIP) on the
  installed rsync binary, dropping the GNU Make $(or ...) and the GNU
  install --strip-program extension that broke with install-sh/BSD install.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-16 21:14:14 +10:00
62 changed files with 2183 additions and 365 deletions

No files matched your search

-1
View File
@@ -30,7 +30,6 @@ permissions:
jobs:
actionlint:
runs-on: ubuntu-latest
timeout-minutes: 15
name: actionlint
steps:
- uses: actions/checkout@v4
+3 -6
View File
@@ -24,13 +24,9 @@ on:
schedule:
- cron: '42 8 * * *'
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 45
container:
image: almalinux:8
name: Test rsync on AlmaLinux 8
@@ -56,7 +52,8 @@ jobs:
attr libattr-devel acl libacl-devel \
zstd libzstd-devel \
lz4 lz4-devel \
xxhash xxhash-devel
xxhash xxhash-devel \
libidn2 libidn2-devel
alternatives --set python3 /usr/bin/python3.9
pip3 install commonmark
- name: configure
@@ -70,7 +67,7 @@ jobs:
# crtimes-not-supported skip matches the other Linux jobs;
# daemon-chroot-acl and proxy-response-line-too-long skip because
# the default (secure) transport opens no listening socket.
run: RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/linux.txt make check
run: RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/linux.txt,@testsuite/skiplist/almalinux-8.txt make check
- name: check (TCP daemon transport)
# Second run exercising the real loopback-TCP daemon path.
run: ./runtests.py --rsync-bin="$PWD/rsync" --use-tcp -j 8
+1 -5
View File
@@ -28,9 +28,6 @@ on:
- cron: '42 8 * * 1'
workflow_dispatch:
permissions:
contents: read
env:
# Minimum supported API level. 24 (Android 7.0) runs on every modern
# phone while keeping broad reach; bump if you need newer Bionic APIs.
@@ -39,7 +36,6 @@ env:
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 45
name: ${{ matrix.abi }}
strategy:
fail-fast: false
@@ -84,7 +80,7 @@ jobs:
# checksums and its bundled zlib.
./configure --host=${{ matrix.triple }} --build=x86_64-pc-linux-gnu \
--enable-ipv6 \
--disable-zstd --disable-lz4 --disable-xxhash --disable-openssl \
--disable-zstd --disable-lz4 --disable-xxhash --disable-openssl --disable-idn \
--disable-iconv --disable-iconv-open \
--disable-acl-support --disable-xattr-support \
--disable-md2man --disable-roll-simd \
+1 -5
View File
@@ -23,13 +23,9 @@ on:
- cron: '42 9 * * 1'
workflow_dispatch:
permissions:
contents: read
jobs:
asan:
runs-on: ubuntu-latest
timeout-minutes: 45
name: rsync ASan+UBSan (clang)
env:
# rsync intentionally leaks small allocations at process exit, so leak
@@ -49,7 +45,7 @@ jobs:
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y clang acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev openssl
sudo apt-get install -y clang acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev openssl
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
# -DNDEBUG builds as a shipped release does (assert() compiled out), so
+11 -7
View File
@@ -4,20 +4,24 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
# Coverage duplicates the Ubuntu suite to measure it rather than protect a
# distinct PR failure mode. Keep that cost scheduled and available on demand.
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/coverage.yml'
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/coverage.yml'
schedule:
- cron: '42 9 * * 1'
workflow_dispatch:
permissions:
contents: read
jobs:
coverage:
runs-on: ubuntu-latest
timeout-minutes: 45
name: gcov coverage
steps:
- uses: actions/checkout@v4
@@ -26,7 +30,7 @@ jobs:
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev python3-cmarkgfm openssl gcovr
sudo apt-get install -y acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev python3-cmarkgfm openssl gcovr
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
run: ./configure --enable-coverage --with-rrsync
+4 -7
View File
@@ -18,13 +18,9 @@ on:
schedule:
- cron: '42 8 * * *'
permissions:
contents: read
jobs:
test:
runs-on: windows-2022
timeout-minutes: 60
name: Test rsync on Cygwin
steps:
- uses: actions/checkout@v4
@@ -34,7 +30,7 @@ jobs:
run: |
$setup = Join-Path $Env:RUNNER_TEMP 'setup-x86_64.exe'
$sums = Join-Path $Env:RUNNER_TEMP 'cygwin-sha512.sum'
$packages = 'make,autoconf,automake,gcc-core,attr,libattr-devel,python39,python39-pip,libzstd-devel,liblz4-devel,libssl-devel,libxxhash0,libxxhash-devel'
$packages = 'make,autoconf,automake,gcc-core,attr,libattr-devel,python39,python39-pip,libzstd-devel,liblz4-devel,libssl-devel,libxxhash0,libxxhash-devel,libidn2-devel'
Invoke-WebRequest https://cygwin.com/setup-x86_64.exe -OutFile $setup
Invoke-WebRequest https://cygwin.com/sha512.sum -OutFile $sums
@@ -89,8 +85,9 @@ jobs:
# (rsyncfns.py drives xattrs via getfattr/setfattr from the `attr`
# package installed above), verified on a real Cygwin host. The real
# chown/devices tests still skip (need root/mknod), as do the
# RESOLVE_BENEATH symlink-race tests.
run: bash -c 'RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/cygwin.txt make check'
# RESOLVE_BENEATH symlink-race tests. Cygwin runs non-root, so the
# namecvt empty-response regression can run despite its common root skip.
run: bash -c 'RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/cygwin.txt,-daemon-namecvt-empty-response make check'
- name: check (TCP daemon transport)
# Second run with daemon tests over a real loopback rsyncd; the default
# 'make check' above uses the secure stdio-pipe transport.
+1 -5
View File
@@ -31,13 +31,9 @@ on:
schedule:
- cron: '17 7 * * 1'
permissions:
contents: read
jobs:
fleettest:
runs-on: ubuntu-latest
timeout-minutes: 45
name: fleettest against localhost
steps:
- uses: actions/checkout@v4
@@ -47,7 +43,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y gcc g++ gawk autoconf automake \
acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev \
acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev \
python3-cmarkgfm openssl rsync openssh-server
- name: set up ssh to localhost
run: |
+1 -5
View File
@@ -18,13 +18,9 @@ on:
schedule:
- cron: '42 8 * * 1'
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 45
name: Test rsync on FreeBSD
steps:
- uses: actions/checkout@v4
@@ -39,7 +35,7 @@ jobs:
pkg install -y bash autotools m4 devel/xxhash zstd liblz4 python3 archivers/liblz4 git
run: |
freebsd-version
./configure --with-rrsync -disable-zstd --disable-md2man --disable-xxhash --disable-lz4
./configure --with-rrsync -disable-zstd --disable-md2man --disable-xxhash --disable-lz4 --disable-idn
make
./rsync --version
make check
+1 -5
View File
@@ -18,13 +18,9 @@ on:
schedule:
- cron: '42 8 * * *'
permissions:
contents: read
jobs:
test:
runs-on: macos-latest
timeout-minutes: 45
name: Test rsync on macOS
steps:
- uses: actions/checkout@v4
@@ -32,7 +28,7 @@ jobs:
fetch-depth: 0
- name: prep
run: |
brew install automake openssl xxhash zstd lz4
brew install automake openssl xxhash zstd lz4 libidn2
pip3 install --user --break-system-packages commonmark
echo "$(brew --prefix)/bin" >>"$GITHUB_PATH"
- name: configure
+1 -5
View File
@@ -18,13 +18,9 @@ on:
schedule:
- cron: '42 8 * * 1'
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 45
name: Test rsync on NetBSD
steps:
- uses: actions/checkout@v4
@@ -40,7 +36,7 @@ jobs:
ln -sf /usr/pkg/bin/python3.12 /usr/pkg/bin/python3
run: |
uname -a
./configure --with-rrsync --disable-zstd --disable-md2man --disable-xxhash --disable-lz4
./configure --with-rrsync --disable-zstd --disable-md2man --disable-xxhash --disable-lz4 --disable-idn
make
./rsync --version
make check
+1 -5
View File
@@ -18,13 +18,9 @@ on:
schedule:
- cron: '42 8 * * 1'
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 45
name: Test rsync on OpenBSD
steps:
- uses: actions/checkout@v4
@@ -41,7 +37,7 @@ jobs:
uname -a
export AUTOCONF_VERSION=2.71
export AUTOMAKE_VERSION=1.16
./configure --with-rrsync --disable-zstd --disable-md2man --disable-xxhash --disable-lz4
./configure --with-rrsync --disable-zstd --disable-md2man --disable-xxhash --disable-lz4 --disable-idn
make
./rsync --version
# The flipper (symlink-race) tests are excluded on OpenBSD, as on the
+2 -7
View File
@@ -17,9 +17,6 @@ on:
- '!.github/workflows/scan-build.yml'
workflow_dispatch:
permissions:
contents: read
jobs:
# GATING run: pinned clang-18 on a pinned runner so the checker set -- and
# thus the expected zero -- is deterministic. The tree is kept clean for
@@ -28,7 +25,6 @@ jobs:
# and the runner (ubuntu-24.04, whose apt repos carry those packages).
gate-clang18:
runs-on: ubuntu-24.04
timeout-minutes: 45
name: scan-build gate (clang-18, pinned)
steps:
- uses: actions/checkout@v4
@@ -37,7 +33,7 @@ jobs:
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y clang-18 clang-tools-18 acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libpopt-dev openssl
sudo apt-get install -y clang-18 clang-tools-18 acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev libpopt-dev openssl
- name: configure (under scan-build)
# Run configure under scan-build so its analyzer compiler-wrapper is baked
# into the Makefile's $(CC); --disable-md2man avoids the doc toolchain.
@@ -70,7 +66,6 @@ jobs:
# broken run from affecting the workflow's required status.
informational-latest:
runs-on: ubuntu-latest
timeout-minutes: 45
name: scan-build (latest clang, informational)
continue-on-error: true
steps:
@@ -80,7 +75,7 @@ jobs:
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y clang clang-tools acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libpopt-dev openssl
sudo apt-get install -y clang clang-tools acl libacl1-dev attr libattr1-dev liblz4-dev libidn2-dev libzstd-dev libxxhash-dev libpopt-dev openssl
- name: configure (under scan-build)
run: scan-build ./configure --with-rrsync --disable-md2man
- name: scan-build (informational)
+1 -5
View File
@@ -18,13 +18,9 @@ on:
schedule:
- cron: '42 8 * * 1'
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 45
name: Test rsync on Solaris
steps:
- uses: actions/checkout@v4
@@ -39,7 +35,7 @@ jobs:
pkg install bash automake gnu-m4 pkg://solaris/runtime/python-35 autoconf gcc git
run: |
uname -a
./configure --with-rrsync -disable-zstd --disable-md2man --disable-xxhash --disable-lz4
./configure --with-rrsync -disable-zstd --disable-md2man --disable-xxhash --disable-lz4 --disable-idn
make
./rsync --version
make check
+70
View File
@@ -0,0 +1,70 @@
name: Test rsync on Ubuntu 22.04
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
# Older-LTS coverage to help with backporting security fixes. ubuntu-22.04
# is currently the oldest GitHub Actions runner image (20.04 was retired
# in April 2025).
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-22.04-build.yml'
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-22.04-build.yml'
schedule:
- cron: '42 8 * * *'
jobs:
test:
runs-on: ubuntu-22.04
name: Test rsync on Ubuntu 22.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
run: |
sudo apt-get update
sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev python3-cmarkgfm openssl
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
run: ./configure --with-rrsync
- name: make
run: make
- name: install
run: sudo make install
- name: info
run: rsync --version
- name: check
run: sudo RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/linux.txt make check
- name: check30
run: sudo RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/linux.txt make check30
- name: check29
run: sudo RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/common.txt,@testsuite/skiplist/linux.txt,@testsuite/skiplist/proto29.txt make check29
- name: check (TCP daemon transport)
# Second run with daemon tests over a real loopback rsyncd; the default
# 'make check' above uses the secure stdio-pipe transport.
run: sudo ./runtests.py --rsync-bin="$PWD/rsync" --use-tcp -j 8
- name: ssl file list
run: rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4
with:
retention-days: 45
name: ubuntu-22.04-bin
path: |
rsync
rsync-ssl
rsync.1
rsync-ssl.1
rsyncd.conf.5
rrsync.1
rrsync
+5 -21
View File
@@ -18,33 +18,18 @@ on:
schedule:
- cron: '42 8 * * *'
permissions:
contents: read
jobs:
test:
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-latest
name: Test rsync on Ubuntu latest
artifact: ubuntu-bin
nonroot: true
- runner: ubuntu-22.04
name: Test rsync on Ubuntu 22.04
artifact: ubuntu-22.04-bin
nonroot: false
runs-on: ${{ matrix.runner }}
timeout-minutes: 45
name: ${{ matrix.name }}
runs-on: ubuntu-latest
name: Test rsync on Ubuntu
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
run: |
sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev python3-cmarkgfm openssl
sudo apt-get update
sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev python3-cmarkgfm openssl
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
run: ./configure --with-rrsync
@@ -67,7 +52,6 @@ jobs:
# is env-dependent here (chroot-acl), so leave RSYNC_EXPECT_SKIPPED unset.
run: sudo ./runtests.py --rsync-bin="$PWD/rsync" --use-tcp -j 8
- name: check (non-root, targeted)
if: matrix.nonroot
# Every run above is root (sudo), so privilege-sensitive tests never hit
# their non-root path. Run those here as the unprivileged 'runner' user
# (NO sudo). Explicit test names make runtests.py full_run False, so
@@ -89,7 +73,7 @@ jobs:
uses: actions/upload-artifact@v4
with:
retention-days: 45
name: ${{ matrix.artifact }}
name: ubuntu-bin
path: |
rsync
rsync-ssl
+2 -5
View File
@@ -39,13 +39,9 @@ on:
schedule:
- cron: '52 8 * * 1'
permissions:
contents: read
jobs:
version-mix:
runs-on: ubuntu-latest
timeout-minutes: 45
name: rsync version-mix
steps:
- uses: actions/checkout@v4
@@ -53,7 +49,8 @@ jobs:
fetch-depth: 0
- name: prep
run: |
sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev python3-cmarkgfm openssl
sudo apt-get update
sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev libidn2-dev python3-cmarkgfm openssl
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
run: ./configure --with-rrsync
+12 -7
View File
@@ -4,20 +4,25 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
# ASan+UBSan remains the per-PR memory-safety gate. Run the four slower
# Valgrind combinations daily and on demand instead of occupying PR runners.
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/valgrind.yml'
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/valgrind.yml'
schedule:
- cron: '17 4 * * *'
workflow_dispatch:
permissions:
contents: read
jobs:
memcheck:
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
@@ -32,7 +37,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y valgrind acl libacl1-dev attr libattr1-dev \
liblz4-dev libzstd-dev libxxhash-dev python3-cmarkgfm openssl
liblz4-dev libzstd-dev libxxhash-dev libidn2-dev python3-cmarkgfm openssl
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
run: ./configure --with-rrsync --enable-debug
+16
View File
@@ -114,6 +114,16 @@ checksums.
[4]: https://www.openssl.org/docs/man1.0.2/man3/crypto.html
## libidn2
The [libidn2 library][5] converts an internationalized domain name into the
IDNA A-label ("Punycode") form that a resolver understands. Installing this
development library lets rsync connect to a daemon whose name has non-ASCII
characters in it, and lets a daemon's "hosts allow" & "hosts deny" settings be
written the same way.
[5]: https://www.gnu.org/software/libidn/#libidn2
## Package summary
To help you get the libraries installed, here are some package install commands
@@ -131,6 +141,7 @@ like.
> sudo apt install -y libzstd-dev
> sudo apt install -y liblz4-dev
> sudo apt install -y libssl-dev
> sudo apt install -y libidn2-dev
Or run support/install_deps_ubuntu.sh
@@ -144,6 +155,7 @@ Or run support/install_deps_ubuntu.sh
> sudo yum -y install libzstd-devel
> sudo yum -y install lz4-devel
> sudo yum -y install openssl-devel
> sudo yum -y install libidn2-devel
> python3 -mpip install --user commonmark
- For Fedora 33:
@@ -154,6 +166,7 @@ Or run support/install_deps_ubuntu.sh
> sudo dnf -y install libzstd-devel
> sudo dnf -y install lz4-devel
> sudo dnf -y install openssl-devel
> sudo dnf -y install libidn2-devel
- For FreeBSD (this assumes that the python3 version is 3.7):
@@ -161,6 +174,7 @@ Or run support/install_deps_ubuntu.sh
> sudo pkg install -y xxhash
> sudo pkg install -y zstd
> sudo pkg install -y liblz4
> sudo pkg install -y libidn2
- For macOS:
@@ -169,6 +183,7 @@ Or run support/install_deps_ubuntu.sh
> brew install zstd
> brew install lz4
> brew install openssl
> brew install libidn2
- For Cygwin (with all cygwin programs stopped, run the appropriate setup program from a cmd shell):
@@ -177,6 +192,7 @@ Or run support/install_deps_ubuntu.sh
> setup-x86_64 --quiet-mode -P libzstd-devel
> setup-x86_64 --quiet-mode -P liblz4-devel
> setup-x86_64 --quiet-mode -P libssl-devel
> setup-x86_64 --quiet-mode -P libidn2-devel
Sometimes cygwin has commonmark packaged and sometimes it doesn't. Now that
its python38 has stabilized, you could install python38-commonmark. Or just
+3 -1
View File
@@ -21,6 +21,7 @@ LIBOBJDIR=lib/
INSTALLCMD=@INSTALL@
INSTALLMAN=@INSTALL@
STRIP=@STRIP@
srcdir=@srcdir@
MKDIR_P=@MKDIR_P@
@@ -116,7 +117,8 @@ install-ssl-daemon: stunnel-rsyncd.conf
install-all: install install-ssl-daemon
install-strip:
$(MAKE) INSTALL_STRIP='-s' install
$(MAKE) install
$(STRIP) $(DESTDIR)$(bindir)/rsync$(EXEEXT)
.PHONY: uninstall
uninstall:
+34 -28
View File
@@ -146,28 +146,29 @@ following symlinks by design.
### The mechanism
Resolution of attacker-influenceable paths goes through `secure_relative_open()`
and the `do_*_at()` wrappers in `syscall.c`, never a raw `open()`/`rename()`/
`chmod()` on a full path string. The principle is: **trust the operator-named
transfer root, and confine all resolution beneath it**, rejecting escapes via
`..` above the anchor, absolute symlinks, or out-of-tree symlinks.
`secure_relative_open()` resolves the parent directory by walking it one
component at a time on a stack of held directory fds, then operates on the final
component with an at-style call on the resulting directory fd.
Resolution of attacker-influenceable paths goes through `secure_relative_open()`,
`secure_relative_dirfd()`, and the `do_*_at()` wrappers in `syscall.c`, never a
raw `open()`/`rename()`/`chmod()` on a full path string. The principle is:
**trust the operator-named transfer root, and confine all resolution beneath
it**, rejecting escapes via `..` above the anchor, absolute symlinks, or
out-of-tree symlinks. `secure_relative_open()` opens the resolved endpoint with
the caller's requested access. `secure_relative_dirfd()` instead returns
traversal authority for `fchdir()` or an at-style operation on a known child;
it does not imply permission to enumerate the directory.
For per-entry work the receiver and generator go one step further and hold the
parent directory open: `open_dir_secure()` resolves an entry's directory once
(via `secure_relative_open()`), `held_dfd_for()` caches that descriptor for the
duration of the entry, and every operation on the entry — `lstat`, the temp-file
`mkstemp`, the temp->final `rename`, `chmod`/`chown`/`utimes`, `mkdir`, special-
file and symlink creation, the delta-basis open, and the recursive delete — runs
through that one held fd via an `*at()` call (`do_*_atfd()`). Because the
descriptor is pinned to the directory inode, a parent component flipped to a
symlink *after* the open cannot redirect any of those operations. The alternate-
destination lookups are confined the same way (`basis_link_stat()` in
`generator.c` and `secure_basis_open()` in `receiver.c`), so a peer-chosen
`--link-dest`/`--compare-dest`/`--copy-dest` basis index cannot reach an
out-of-module file through a symlinked parent.
as traversal authority, `held_dfd_for()` caches that descriptor for the
duration of the entry, and every operation on the entry — `lstat`, the
temp-file `mkstemp`, the temp->final `rename`, `chmod`/`chown`/`utimes`,
`mkdir`, special-file and symlink creation, the delta-basis open, and the
recursive delete — runs through that one held fd via an `*at()` call
(`do_*_atfd()`). Because the descriptor is pinned to the directory inode, a
parent component flipped to a symlink *after* the open cannot redirect any of
those operations. The alternate-destination lookups are confined the same way
(`basis_link_stat()` in `generator.c` and `secure_basis_open()` in
`receiver.c`), so a peer-chosen `--link-dest`/`--compare-dest`/`--copy-dest`
basis index cannot reach an out-of-module file through a symlinked parent.
The sender's source-directory *enumeration* is confined the same way as its
content open. `send_directory()` opens each scanned directory through
@@ -190,13 +191,17 @@ symlink would otherwise introduce.
### Path resolution
`secure_relative_open()` resolves a path with a single portable mechanism on
every platform: a per-component walk on a stack of held directory fds. Each
component is opened relative to the held parent with `openat(parent_fd,
"component", O_NOFOLLOW)`; descending into a real subdirectory pushes its fd, a
`..` pops back to the already-held parent (a pop at the anchor is refused), and an
in-tree directory symlink is followed by reading its target and walking that off
the same stack (absolute targets refused, symlink hops bounded). The final
component is opened `O_NOFOLLOW`.
every platform: a per-component walk on a stack of held directory fds. On
Linux, anchors and traversal components use
`O_PATH|O_DIRECTORY|O_NOFOLLOW`; other platforms retain the
`O_RDONLY|O_DIRECTORY` fallback. Descending into a real subdirectory pushes
its fd, a `..` pops back to the already-held parent (a pop at the anchor is
refused), and an in-tree directory symlink is followed by reading its target
and walking that off the same stack (absolute targets refused, symlink hops
bounded). A final directory endpoint is reopened with the caller's requested
flags. Thus `secure_opendir()` still receives a readable fd, while known-name
operations beneath a searchable but unreadable directory do not require
permission to list it.
Because every component is opened relative to a *pinned* fd under `O_NOFOLLOW`,
and `..` is resolved by the held-fd stack rather than by the kernel, the walk is
@@ -235,8 +240,9 @@ chmod-ing through a raced leaf symlink.
influenced by the remote peer or by another local user, use a `do_*_at()`
wrapper (or `secure_relative_open()`), not a raw full-path syscall.
* When introducing a new operation, add a matching `do_<op>_at()` wrapper that
resolves the parent with `secure_relative_open()` and acts via an at-style call
on the returned dirfd.
resolves a parent used only as at-style authority with
`secure_relative_dirfd()`. Use `secure_relative_open()` when the returned fd
itself must be readable or otherwise support the caller's requested access.
* Do not assume a non-daemon transfer is safe; the question is whether rsync has
more authority than whoever controls the path components.
* On platforms whose API lacks an at-style equivalent (e.g. `setattrlist()`),
+11
View File
@@ -33,6 +33,9 @@ static int match_hostname(const char **host_ptr, const char *addr, const char *t
struct hostent *hp;
unsigned int i;
const char *host = *host_ptr;
#ifdef SUPPORT_IDN
char idn_tok[1024];
#endif
if (!host || !*host)
return 0;
@@ -42,6 +45,14 @@ static int match_hostname(const char **host_ptr, const char *addr, const char *t
return innetgr(tok + 1, host, NULL, NULL);
#endif
#ifdef SUPPORT_IDN
/* A hostname reaches us from DNS as ASCII, so fold an IDN token to its
* A-label form before comparing. An all-ASCII token, and a token we
* can't fold, are both left as they are. */
if (idn_to_ascii(tok, 0, idn_tok, sizeof idn_tok))
tok = idn_tok;
#endif
/* First check if the reverse-DNS-determined hostname matches. */
if (iwildmatch(tok, host))
return 1;
+4 -4
View File
@@ -271,12 +271,12 @@ void open_batch_files(void)
exit_cleanup(RERR_FILEIO);
}
/* --read-batch: the file's bytes drive the protocol parser, so refuse
* non-regular files (FIFO, device, socket) at the batch path. */
/* --read-batch: the file's bytes drive the protocol parser,
* allow FIFOs used by shell process substitution while continuing to reject other non-regular inputs. */
if (!write_batch && batch_fd != STDIN_FILENO) {
STRUCT_STAT st;
if (do_fstat(batch_fd, &st) == 0 && !S_ISREG(st.st_mode)) {
rprintf(FERROR, "Batch file %s is not a regular file\n",
if (do_fstat(batch_fd, &st) == 0 && !S_ISREG(st.st_mode) && !S_ISFIFO(st.st_mode)) {
rprintf(FERROR, "Batch file %s is neither a regular file nor a FIFO\n",
full_fname(batch_name));
exit_cleanup(RERR_FILEIO);
}
+23 -1
View File
@@ -13,7 +13,7 @@ AC_CHECK_HEADERS(poll.h sys/fcntl.h sys/select.h fcntl.h sys/time.h sys/unistd.h
sys/acl.h acl/libacl.h attr/xattr.h sys/xattr.h sys/extattr.h dl.h \
popt.h popt/popt.h linux/falloc.h netinet/in_systm.h netgroup.h \
zlib.h xxhash.h openssl/md4.h openssl/md5.h zstd.h lz4.h sys/file.h \
sys/resource.h bsd/string.h)
sys/resource.h bsd/string.h idn2.h)
AC_CHECK_HEADERS([netinet/ip.h], [], [], [[#include <netinet/in.h>]])
AC_HEADER_MAJOR_FIXED
@@ -59,6 +59,7 @@ AC_PROG_CXX
AC_PROG_AWK
AC_PROG_EGREP
AC_PROG_INSTALL
AC_CHECK_TOOL([STRIP], [strip], [strip])
AC_PROG_MKDIR_P
AC_SUBST(SHELL)
AC_PATH_PROG([PERL], [perl])
@@ -626,6 +627,27 @@ else
AC_MSG_RESULT(no)
fi
AC_MSG_CHECKING([whether to enable IDN support])
AC_ARG_ENABLE([idn],
AS_HELP_STRING([--disable-idn], [disable to omit IDN (Internationalized Domain Name) support]))
AH_TEMPLATE([SUPPORT_IDN],
[Undefine if you do not want IDN support. By default this is defined.])
if test x"$enable_idn" != x"no"; then
if test x"$ac_cv_header_idn2_h" = x"yes"; then
AC_MSG_RESULT(yes)
AC_SEARCH_LIBS(idn2_lookup_ul, idn2,
[AC_DEFINE(SUPPORT_IDN)],
[err_msg="$err_msg$nl- Failed to find idn2_lookup_ul function in idn2 lib.";
no_lib="$no_lib idn"])
else
AC_MSG_RESULT(no)
err_msg="$err_msg$nl- Failed to find idn2.h for IDN support."
no_lib="$no_lib idn"
fi
else
AC_MSG_RESULT(no)
fi
if test x"$no_lib" != x; then
echo ""
echo "Configure found the following issues:"
+1 -1
View File
@@ -1052,7 +1052,7 @@ static int basis_link_stat(const char *path, STRUCT_STAT *stp)
if (dlen >= sizeof dir) { errno = ENAMETOOLONG; return -1; }
memcpy(dir, path, dlen);
dir[dlen] = '\0';
if ((dfd = secure_relative_open(NULL, dir, O_RDONLY | O_DIRECTORY, 0)) < 0)
if ((dfd = secure_relative_dirfd(NULL, dir)) < 0)
return -1;
r = link_stat_at(dfd, slash + 1, stp, 0);
e = errno;
+33 -7
View File
@@ -117,6 +117,13 @@ static time_t last_io_out;
* transfer timeout and may be supplied by the module or client. */
static time_t daemon_handshake_deadline;
/* Wall-clock bound the client puts on establishing a daemon connection made
* through a remote shell (daemon_connection == 1): spawning the helper, its
* connect()/TLS handshake, and the exchange of the daemon greeting all happen
* before any buffered I/O begins, so --contimeout can time the whole phase the
* same way the socket path times its connect(). */
static time_t client_connect_deadline;
static int write_batch_monitor_in = -1;
static int write_batch_monitor_out = -1;
@@ -146,17 +153,28 @@ static int handshake_poll_timeout_ms(void)
time_t now, left;
int timeout = poll_timeout_ms();
if (!daemon_handshake_deadline)
if (!daemon_handshake_deadline && !client_connect_deadline)
return timeout;
now = time(NULL);
left = daemon_handshake_deadline - now;
if (left <= 0) {
rprintf(FERROR, "[%s] daemon handshake timeout -- exiting\n", who_am_i());
exit_cleanup(RERR_TIMEOUT);
if (daemon_handshake_deadline) {
left = daemon_handshake_deadline - now;
if (left <= 0) {
rprintf(FERROR, "[%s] daemon handshake timeout -- exiting\n", who_am_i());
exit_cleanup(RERR_TIMEOUT);
}
if (left <= INT_MAX / 1000 && left * 1000 < timeout)
timeout = (int)left * 1000;
}
if (client_connect_deadline) {
left = client_connect_deadline - now;
if (left <= 0) {
rprintf(FERROR, "[%s] connection timed out -- exiting\n", who_am_i());
exit_cleanup(RERR_CONTIMEOUT);
}
if (left <= INT_MAX / 1000 && left * 1000 < timeout)
timeout = (int)left * 1000;
}
if (left <= INT_MAX / 1000 && left * 1000 < timeout)
timeout = (int)left * 1000;
return timeout;
}
@@ -1303,6 +1321,14 @@ void set_daemon_handshake_timeout(int secs)
daemon_handshake_deadline = 0;
}
void set_client_connect_timeout(int secs)
{
if (secs > 0)
client_connect_deadline = time(NULL) + secs;
else
client_connect_deadline = 0;
}
static void check_for_d_option_error(const char *msg)
{
static const char rsync263_opts[] = "BCDHIKLPRSTWabceghlnopqrtuvxz";
+20 -1
View File
@@ -521,6 +521,16 @@ static pid_t do_cmd(char *cmd, char *machine, char *user, char **remote_argv, in
char *args[MAX_ARGS], *need_to_free = NULL;
pid_t pid;
int dash_l_set = 0;
#ifdef SUPPORT_IDN
char idn_machine[1024];
/* A daemon-over-remote-shell host is ours to resolve, so give the helper
* the A-label form. A "host:path" transfer is left alone because that
* name belongs to the user's ssh, which may be matching it against an
* ssh_config Host pattern. */
if (machine && daemon_connection > 0 && idn_to_ascii(machine, 1, idn_machine, sizeof idn_machine))
machine = idn_machine;
#endif
if (!read_batch && !local_server) {
char *t, *f, in_quote = '\0';
@@ -1602,7 +1612,7 @@ static int start_client(int argc, char *argv[])
exit_cleanup(RERR_SYNTAX);
}
if (connect_timeout) {
if (connect_timeout && !daemon_connection) {
rprintf(FERROR, "The --contimeout option may only be "
"used when connecting to an rsync daemon.\n");
exit_cleanup(RERR_SYNTAX);
@@ -1634,6 +1644,14 @@ static int start_client(int argc, char *argv[])
(void)env_port;
#endif
/* For a daemon reached through a remote shell, the "connection" rsync
* waits on is: the helper is spawned, it establishes its own link (e.g.
* rsync-ssl's openssl connect + TLS handshake), and the daemon greeting is
* exchanged. Bound that whole phase with --contimeout so the option
* behaves for daemon-via-rsh the way it does for a socket connection. */
if (daemon_connection && connect_timeout > 0)
set_client_connect_timeout(connect_timeout);
pid = do_cmd(shell_cmd, shell_machine, shell_user, remote_argv, remote_argc, &f_in, &f_out);
/* if we're running an rsync server on the remote host over a
@@ -1641,6 +1659,7 @@ static int start_client(int argc, char *argv[])
if (daemon_connection) {
int tmpret;
tmpret = start_inband_exchange(f_in, f_out, shell_user, remote_argc, remote_argv);
set_client_connect_timeout(0);
if (tmpret < 0)
return tmpret;
}
+15 -6
View File
@@ -1157,6 +1157,12 @@ static int count_args(const char **argv)
return i;
}
/* The largest value parse_size_arg() will accept when no explicit max_value is
* given. It is SIZE_MAX/2 rather than SIZE_MAX because the parser computes and
* returns the size as a signed ssize_t (with a negative return meaning error),
* so this keeps every accepted size representable as a positive ssize_t. */
#define SIZE_ARG_MAX ((ssize_t)(SIZE_MAX / 2))
/* If the size_arg is an invalid string or the value is < min_value, an error
* is put into err_buf & the return is -1. Note that this parser does NOT
* support negative numbers, so a min_value < 0 doesn't make any sense. */
@@ -1166,7 +1172,7 @@ static ssize_t parse_size_arg(const char *size_arg, char def_suf, const char *op
int reps, mult, len;
const char *arg, *err = "invalid", *min_max = NULL;
ssize_t limit = -1, size = 1;
ssize_t size_max = max_value >= 0 ? max_value : (ssize_t)(SIZE_MAX / 2);
ssize_t size_max = max_value >= 0 ? max_value : SIZE_ARG_MAX;
double dsize;
for (arg = size_arg; isDigit(arg); arg++) {}
@@ -2067,14 +2073,17 @@ int parse_arguments(int *argc_p, const char ***argv_p)
ssize_t size = parse_size_arg(max_alloc_arg, 'B', "max-alloc", 1024*1024, -1, True);
if (size < 0)
goto cleanup;
if (size == 0) {
snprintf(err_buf, sizeof err_buf, "max-alloc must be greater than zero\n");
goto cleanup;
}
max_alloc = size;
}
/* A 0 value means "as large as this build allows". We resolve it to the
* same ceiling parse_size_arg() enforces, so that --max-alloc=0 is exactly
* the largest value a user could also have typed, and never a limit that
* only the 0 spelling can reach. Note that max_alloc_arg is forwarded to
* the peer un-normalized (see server_options()), which is what lets each
* side resolve 0 against its own SIZE_MAX -- a 64-bit client and a 32-bit
* daemon each get their own ceiling from the one portable spelling. */
if (!max_alloc)
max_alloc = SIZE_MAX;
max_alloc = SIZE_ARG_MAX;
if (old_style_args < 0) {
if (!am_server && protect_args <= 0 && (arg = getenv("RSYNC_OLD_ARGS")) != NULL && *arg) {
+15 -4
View File
@@ -226,9 +226,20 @@ if [[ "$1" == --HELPER ]]; then
rsync_ssl_helper "${@}"
fi
if [[ "$1" == --type=* ]]; then
export RSYNC_SSL_TYPE="${1/--type=/}"
shift
fi
args=()
dash_dash_seen=false
for arg in "$@"; do
if [[ $dash_dash_seen == true ]]; then
args+=("$arg")
elif [[ "$arg" == "--" ]]; then
args+=("$arg")
dash_dash_seen=true
elif [[ "$arg" == --type=* ]]; then
export RSYNC_SSL_TYPE="${arg#--type=}"
else
args+=("$arg")
fi
done
set -- "${args[@]}"
rsync_ssl_run "${@}"
+5 -2
View File
@@ -25,12 +25,15 @@ rsync version to be at least 3.2.0.
## OPTIONS
If the **first** arg is a `--type=SSL_TYPE` option, the script will only use
If an arg is a `--type=SSL_TYPE` option, the script will only use
that particular program to open an ssl connection instead of trying to find an
openssl or stunnel executable via a simple heuristic (assuming that the
`RSYNC_SSL_TYPE` environment variable is not set as well -- see below). This
option must specify one of `openssl` or `stunnel`. The equal sign is
required for this particular option.
required for this particular option. The wrapper's option scan stops at a
`--` argument: the `--` and everything after it are passed through to rsync
unchanged, so a `--type=...` token after a `--` is not consumed by the
wrapper.
All the other options are passed through to the rsync command, so consult the
**rsync**(1) manpage for more information on how it works.
+39 -15
View File
@@ -412,6 +412,20 @@ when scripting rsync.
WARNING: On some systems, environment variables are visible to all users. On
those systems using [`--password-file`](#opt) is recommended.
If rsync was built with IDN support (look for "IDN" in `rsync --version`), the
daemon host may contain non-ASCII characters: those labels are converted to
their IDNA A-label ("Punycode") form before the name is looked up. The name is
read using your locale's character encoding, so be sure your locale is set
correctly. A name typed with combining characters is normalized on the way, so
it is looked up the same as its precomposed spelling. Only the non-ASCII
labels change, so an address literal, a name you punycoded yourself, and a name
that is not a valid IDN are all looked up just as you typed them.
This applies to the host of a daemon connection only. The host of a plain
remote-shell transfer (the single-colon syntax) is passed to your remote-shell
program unchanged, since that name may well be an ssh_config "Host" alias
rather than a DNS name.
You may establish the connection via a web proxy by setting the environment
variable [`RSYNC_PROXY`](#) to a hostname:port pair pointing to your web proxy.
Note that your web proxy's configuration must support proxy connections to port
@@ -1741,20 +1755,20 @@ sign) if you want the local shell to expand it.
0. `--devices`
This option causes rsync to transfer character and block device files to
the remote system to recreate these devices. If the receiving rsync is not
being run as the super-user, rsync silently skips creating the device files
(see also the [`--super`](#opt) and [`--fake-super`](#opt) options).
the remote system to recreate these devices. This option has no effect if
the receiving rsync is not run as the super-user and neither
[`--super`](#opt) nor [`--fake-super`](#opt) is in effect.
By default, rsync generates a "non-regular file" warning for each device
file encountered when this option is not set. You can silence the warning
by specifying [`--info=nonreg0`](#opt).
When a device file is not created, rsync generates the usual "non-regular
file" warning. You can silence the warning by specifying
[`--info=nonreg0`](#opt).
0. `--specials`
This option causes rsync to transfer special files, such as named sockets
and fifos. If the receiving rsync is not being run as the super-user,
rsync silently skips creating the special files (see also the
[`--super`](#opt) and [`--fake-super`](#opt) options).
and fifos. Creating these files does not normally require super-user
privileges. Use [`--drop-D`](#opt) to make the receiving rsync refuse to
create them regardless of what the transfer requested.
By default, rsync generates a "non-regular file" warning for each special
file encountered when this option is not set. You can silence the warning
@@ -2325,12 +2339,22 @@ sign) if you want the local shell to expand it.
See the [`--max-size`](#opt) option for a description of how SIZE can be
specified. The default suffix if none is given is bytes.
Beginning in 3.2.7, a value of 0 was an easy way to specify SIZE_MAX (the
largest limit possible). However, beginning with 3.5.0, a value of 0 is
rejected as invalid for security reasons (a 0-byte cap could be used to
disable the allocation limit, which could lead to a denial-of-service via
memory exhaustion). Use an explicit very large value if you want a very
high limit.
A value of 0 is an easy way to say "the largest limit this build supports".
It resolves to the same ceiling an explicit SIZE is checked against, so it
is never a higher limit than one you could have typed out yourself.
Because the option is passed to the remote rsync as you wrote it, each side
resolves a 0 against its own maximum. That makes 0 the only spelling that
is correct for both ends of a transfer between hosts of different word
sizes: a literal value large enough to be useful on a 64-bit client is
rejected as too large by a 32-bit daemon.
A value of 0 was accepted beginning in 3.2.3 and rejected in 3.5.0; the
release after 3.5.0 accepts it again.
A daemon administrator who does not want clients to change the configured
allocation ceiling can set `refuse options = max-alloc` in the module's
`rsyncd.conf`. This refuses every client-supplied value, including 0.
You can set a default value using the environment variable
[`RSYNC_MAX_ALLOC`](#) using the same SIZE values as supported by this
+14
View File
@@ -853,6 +853,20 @@ in the values of parameters. See that section for details.
- an '@' followed by a netgroup name, which will match if the reverse DNS
of the connecting IP is in the specified netgroup.
If rsync was built with IDN support (look for "IDN" in `rsync --version`),
a hostname pattern may contain non-ASCII characters: this file is read as
UTF-8, and each non-ASCII label is converted to its IDNA A-label
("Punycode") form before matching, since the name the daemon has for a
client always reaches it from DNS as ASCII. A pattern written with
combining characters is normalized on the way, so it matches the same as
its precomposed spelling. Only those labels change, so an address, a
mask, an already-punycoded name, and the wildcard characters are all
matched exactly as written. A pattern that cannot be converted that way
is matched as it stands, and thus matches nothing. That includes a
pattern whose conversion would have introduced a character it was not
written with, such as the U+FF0A FULLWIDTH ASTERISK that the IDNA mapping
turns into a "*".
Note IPv6 link-local addresses can have a scope in the address
specification:
+6 -9
View File
@@ -127,8 +127,7 @@ static int secure_sender_parent_fd(struct file_struct *file, const char *fname,
#endif
while (*rel == '/')
rel++;
return secure_relative_open("/", rel,
O_RDONLY | O_DIRECTORY, 0);
return secure_relative_dirfd("/", rel);
}
/* held_dir_path_fd returns a cache-OWNED fd; the caller closes
* what we return, so hand back an owned dup and leave the cache's
@@ -141,7 +140,7 @@ static int secure_sender_parent_fd(struct file_struct *file, const char *fname,
return dup(dfd);
if (errno != 0)
return -1;
return secure_relative_open(NULL, dir, O_RDONLY | O_DIRECTORY, 0);
return secure_relative_dirfd(NULL, dir);
}
errno = 0; /* top-level file: no parent component to confine */
return -1;
@@ -176,9 +175,9 @@ static int secure_sender_parent_fd(struct file_struct *file, const char *fname,
}
memcpy(dir, relp, dlen);
dir[dlen] = '\0';
dfd = secure_relative_open(module_dir, dir, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(module_dir, dir);
} else
dfd = secure_relative_open(module_dir, "", O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(module_dir, "");
/* The leaf is the same last component either way; take it from the caller's
* persistent fname buffer, not the local secure_path. */
@@ -292,11 +291,9 @@ static int sender_open_copylinks_confined(const char *anchor, const char *relpat
* only this branch would hand it to strcmp(). */
if (am_daemon && module_dirfd >= 0 && module_dir && anchor
&& strcmp(anchor, module_dir) == 0)
pdfd = secure_relative_open_at_beneath(module_dirfd, dir,
O_RDONLY | O_DIRECTORY, 0);
pdfd = secure_relative_dirfd_at_beneath(module_dirfd, dir);
else
pdfd = secure_relative_open(anchor, dir,
O_RDONLY | O_DIRECTORY, 0);
pdfd = secure_relative_dirfd(anchor, dir);
if (pdfd < 0)
return -1;
n = do_readlink_atfd(pdfd, bname, tgt, sizeof tgt - 1);
+8
View File
@@ -343,6 +343,14 @@ int open_socket_out(char *host, int port, const char *bind_addr, int af_hint)
int proxied = 0;
char buffer[1024];
char *proxy_user = NULL, *proxy_pass = NULL;
#ifdef SUPPORT_IDN
char idn_host[1024];
/* The resolver only speaks ASCII, so an IDN host goes out as A-labels.
* An all-ASCII host is passed along untouched. */
if (idn_to_ascii(host, 1, idn_host, sizeof idn_host))
host = idn_host;
#endif
/* if we have a RSYNC_PROXY env variable then redirect our
* connection via a web proxy at the given address. */
+1
View File
@@ -9,3 +9,4 @@ sudo apt install -y libxxhash-dev
sudo apt install -y libzstd-dev
sudo apt install -y liblz4-dev
sudo apt install -y libssl-dev
sudo apt install -y libidn2-dev
+140 -53
View File
@@ -74,6 +74,20 @@ extern unsigned int confine_rootlen;
extern char curr_dir[MAXPATHLEN]; /* defined below; fwd-declared for the seed */
extern int operator_path_resolve; /* defined below; fwd-declared for the exclude check */
/* A directory fd used only for pathname traversal, fchdir(), or as *at()
* authority does not need read permission on Linux. Keep the portable
* O_RDONLY fallback for systems without O_PATH. */
static int directory_traverse_flags(void)
{
#if defined O_PATH && defined O_DIRECTORY
return O_PATH | O_DIRECTORY;
#elif defined O_DIRECTORY
return O_RDONLY | O_DIRECTORY;
#else
return O_RDONLY;
#endif
}
#if defined AT_FDCWD && defined O_NOFOLLOW && defined O_DIRECTORY
/* Open a trusted absolute anchor directory as an owned dirfd. When the anchor is
* the served module root and the daemon pinned it by identity (module_dirfd), dup
@@ -86,7 +100,7 @@ static int open_anchor_dirfd(const char *path)
{
if (module_dirfd >= 0 && am_daemon && module_dir && strcmp(path, module_dir) == 0)
return dup(module_dirfd);
return openat(AT_FDCWD, path, O_RDONLY | O_DIRECTORY);
return openat(AT_FDCWD, path, directory_traverse_flags());
}
#endif
@@ -143,13 +157,18 @@ static const char *confinement_root(unsigned int *lenp)
return confine_root;
}
/* Split the "/proc/<self|pid>/fd" prefix off `p`, returning the tail -- "" for
* the pin directory itself, otherwise a string starting with '/'. NULL when `p`
* is not in the fd-pin namespace at all. */
/* Split a recognised fd-pin prefix off `p`, returning the tail -- "" for the
* pin directory itself, otherwise a string starting with '/'. NULL when `p`
* is not in an fd-pin namespace. */
static const char *fd_pin_tail(const char *p)
{
const char *s;
if (strncmp(p, "/dev/fd", 7) == 0) {
s = p + 7;
return (*s == '\0' || *s == '/') ? s : NULL;
}
if (strncmp(p, "/proc/", 6) != 0)
return NULL;
s = p + 6;
@@ -168,8 +187,8 @@ static const char *fd_pin_tail(const char *p)
return (*s == '\0' || *s == '/') ? s : NULL;
}
/* An EXACT pin entry, "/proc/self/fd/7" -- the one spelling whose target is what
* confinement must judge. rrsync also writes a pinned parent as
/* An EXACT pin entry, such as "/proc/self/fd/7" or "/dev/fd/7", whose target is
* what confinement must judge. rrsync also writes a pinned parent as
* ".../fd/7/<leaf>", but the walk resolves the magic link itself and checks the
* components past it, so only the bare entry is resolved here. Requiring all
* digits keeps a planted name like ".../fd/outside-secret" out. */
@@ -285,12 +304,13 @@ static int abspath_step(char *abspath, size_t cap, const char *comp, size_t comp
* uses it to filter-check the (otherwise unchecked) leaf basename. */
static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, size_t out_cap)
{
#if defined AT_FDCWD && defined O_NOFOLLOW
#if defined AT_FDCWD && defined O_NOFOLLOW && defined O_DIRECTORY
/* O_CLOEXEC predates some still-supported targets; mirror rand_bytes()'s
* fallback in syscall.c so a build without it still compiles. */
#ifndef O_CLOEXEC
#define O_CLOEXEC 0
#endif
const int dir_traverse_flags = directory_traverse_flags() | O_CLOEXEC;
if (!path || !*path) {
errno = EINVAL;
return -1;
@@ -309,8 +329,7 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
* (abspath_outside_confinement). A relative operator path starts at the
* daemon's cwd == the module root; an absolute one (or a followed absolute
* symlink target) restarts at "/". */
char abspath[MAXPATHLEN];
abspath[0] = '\0';
char abspath[MAXPATHLEN] = {0};
if (am_daemon && module_dir && module_dir[0] == '/')
strlcpy(abspath, module_dir, sizeof abspath); /* "/" for a path=/ module */
else if (confine_root) {
@@ -334,7 +353,8 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
* reach the magic link. This only suspends the check for that prefix:
* following the link restarts the walk at its absolute target, and every
* component of THAT is checked, so a pin aimed outside is still refused. */
int pin_transit = !am_daemon && confine_root && fd_pin_tail(path) != NULL;
const char *ptail = fd_pin_tail(path);
int pin_transit = !am_daemon && confine_root && ptail != NULL;
/* Path-walk state. `remaining` is the unconsumed tail; we splice
* symlink targets back into it as we go. Sized 2x MAXPATHLEN so a
@@ -348,7 +368,7 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
/* Absolute path: pin "/" as the starting dfd. */
if (remaining[0] == '/') {
dfd = open("/", O_RDONLY | O_DIRECTORY | O_CLOEXEC);
dfd = open("/", dir_traverse_flags);
if (dfd < 0)
return -1;
dfd_owns = 1;
@@ -401,9 +421,18 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
}
if (S_ISLNK(lst.st_mode)) {
/* Symlink: untrusted owner is refused; trusted owner
* is followed via readlinkat + splice. */
if (lst.st_uid != 0 && lst.st_uid != trusted_uid) {
/* Symlink: untrusted owner is refused; trusted owner is followed
* via readlinkat + splice. In a user namespace the /proc/self and
* /dev/fd symlinks may report the overflow uid, so
* allow those exact components while traversing a recognised pin. */
int namespace_pin = pin_transit
&& ((strcmp(abspath, "/proc") == 0 && strcmp(comp, "self") == 0)
|| (strcmp(abspath, "/dev") == 0 && strcmp(comp, "fd") == 0));
if (!namespace_pin && lst.st_uid != 0 && lst.st_uid != trusted_uid) {
rprintf(FERROR,
"refusing to follow a symlink owned by an untrusted user; "
"use --insecure-links locally or \"insecure links = yes\" in a "
"daemon module ONLY if every path component is trusted\n");
saved_errno = ELOOP;
goto out;
}
@@ -419,6 +448,41 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
}
target[n] = '\0';
/* Detect Linux kernel pseudo-paths (pipes, sockets, anon_inodes).
* These are not real paths on disk and never contain slashes. */
const char *abstail = fd_pin_tail(abspath);
int is_fd_dir = (abstail != NULL && *abstail == '\0' && ptail != NULL);
if (is_fd_dir && (strncmp(target, "pipe:[", 6) == 0
|| strncmp(target, "socket:[", 8) == 0
|| strncmp(target, "anon_inode:", 11) == 0)) {
if (!is_last) {
saved_errno = ENOTDIR;
goto out;
}
if (confine_root) {
/* Anonymous objects cannot be proven to reside beneath
* the confinement root. */
saved_errno = ENOENT;
goto out;
}
/* Process substitution exposes /dev/fd/X as a symlink to a
* kernel object. Reopen the validated leaf without O_NOFOLLOW
* so the kernel applies the caller's requested open flags. */
retfd = openat(dfd, comp, (flags & ~O_NOFOLLOW) | O_CLOEXEC, mode);
/* Refuse a descriptor that changed to a filesystem object
* between validation and openat(). */
if (retfd >= 0) {
STRUCT_STAT pst;
if (fstat(retfd, &pst) < 0 || S_ISREG(pst.st_mode) || S_ISDIR(pst.st_mode)) {
close(retfd);
retfd = -1;
errno = ELOOP;
}
}
saved_errno = retfd < 0 ? errno : 0;
goto out;
}
/* Splice: new `remaining` = <target> + <tail-after-comp>.
* Absolute target restarts the walk from "/". */
char tail[MAXPATHLEN];
@@ -435,7 +499,7 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
if (target[0] == '/') {
if (dfd_owns) close(dfd);
dfd = open("/", O_RDONLY | O_DIRECTORY | O_CLOEXEC);
dfd = open("/", dir_traverse_flags);
if (dfd < 0) {
saved_errno = errno;
dfd_owns = 0;
@@ -490,7 +554,7 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
saved_errno = ELOOP;
goto out;
}
int next = openat(dfd, comp, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
int next = openat(dfd, comp, dir_traverse_flags | O_NOFOLLOW);
if (next < 0) {
saved_errno = errno;
goto out;
@@ -510,12 +574,12 @@ static int ona_open(const char *path, int flags, mode_t mode, char *out_abs, siz
}
/* Path resolved entirely to a directory (no leaf component left).
* If the caller wanted O_DIRECTORY we already hold the dirfd we
* built up; otherwise it's an EISDIR. */
* Reopen the held traversal fd with the caller's requested access mode;
* an O_PATH fd is sufficient for traversal and fchdir but not operations
* such as fchmod. */
if (flags & O_DIRECTORY) {
retfd = dfd;
dfd_owns = 0; /* caller now owns it */
saved_errno = 0;
retfd = openat(dfd, ".", flags | O_NOFOLLOW, mode);
saved_errno = retfd < 0 ? errno : 0;
if (out_abs && out_cap)
/* Root-resolved (".." popped abspath empty) tracked daemon walk:
* hand back "/" so owner_walk_parent still leaf-checks (path=/ bypass). */
@@ -540,6 +604,14 @@ int open_no_attacker_symlinks(const char *path, int flags, mode_t mode)
return ona_open(path, flags, mode, NULL, 0);
}
/* Open a directory for traversal or as *at()/fchdir() authority. Unlike an
* O_RDONLY directory endpoint, this accepts a searchable but unreadable
* directory on Linux. */
int open_no_attacker_symlinks_dirfd(const char *path)
{
return ona_open(path, directory_traverse_flags(), 0, NULL, 0);
}
/* When set, the do_*_at() wrappers resolve their path as an OPERATOR-supplied
* directory path (an absolute or relative --backup-dir/--temp-dir/--*-dest)
* using the ownership walk -- follow a symlink owned by uid 0 or our euid,
@@ -565,7 +637,7 @@ int owner_walk_parent(const char *path, const char **bname)
*bname = slash ? slash + 1 : path;
pabs[0] = '\0';
if (!slash)
dfd = ona_open(".", O_RDONLY | O_DIRECTORY, 0, pabs, sizeof pabs);
dfd = ona_open(".", directory_traverse_flags(), 0, pabs, sizeof pabs);
else {
dlen = slash == path ? 1 : (size_t)(slash - path); /* "/x" -> parent "/" */
if (dlen >= sizeof dir) {
@@ -574,7 +646,7 @@ int owner_walk_parent(const char *path, const char **bname)
}
memcpy(dir, path, dlen);
dir[dlen] = '\0';
dfd = ona_open(dir, O_RDONLY | O_DIRECTORY, 0, pabs, sizeof pabs);
dfd = ona_open(dir, directory_traverse_flags(), 0, pabs, sizeof pabs);
}
if (dfd < 0)
return -1;
@@ -703,7 +775,7 @@ int do_unlink_at(const char *path)
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -810,7 +882,7 @@ int do_symlink_at(const char *lnk, const char *path)
memcpy(dirpath, path, dlen);
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
owns = True;
@@ -1012,7 +1084,7 @@ int do_link_at(const char *old_path, const char *new_path)
memcpy(old_dirpath, old_path, old_dlen);
old_dirpath[old_dlen] = '\0';
old_bname = old_slash + 1;
old_dfd = secure_relative_open(NULL, old_dirpath, O_RDONLY | O_DIRECTORY, 0);
old_dfd = secure_relative_dirfd(NULL, old_dirpath);
if (old_dfd < 0)
return -1;
old_owns = True;
@@ -1051,7 +1123,7 @@ int do_link_at(const char *old_path, const char *new_path)
&& memcmp(old_dirpath, new_dirpath, old_dlen) == 0) {
new_dfd = old_dfd;
} else {
new_dfd = secure_relative_open(NULL, new_dirpath, O_RDONLY | O_DIRECTORY, 0);
new_dfd = secure_relative_dirfd(NULL, new_dirpath);
if (new_dfd < 0) {
e = errno;
if (old_owns) close(old_dfd);
@@ -1154,7 +1226,7 @@ int do_lchown_at(const char *fname, uid_t owner, gid_t group)
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -1325,7 +1397,7 @@ int do_mknod_at(const char *pathname, mode_t mode, dev_t dev)
memcpy(dirpath, pathname, dlen);
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
owns = True;
@@ -1447,7 +1519,7 @@ int do_rmdir_at(const char *pathname)
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -1543,7 +1615,7 @@ int do_open_at(const char *pathname, int flags, mode_t mode)
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -1825,7 +1897,7 @@ int do_chmod_at(const char *fname, mode_t mode)
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -1942,7 +2014,7 @@ int do_rename_at(const char *old_path, const char *new_path)
memcpy(old_dirpath, old_path, old_dlen);
old_dirpath[old_dlen] = '\0';
old_bname = old_slash + 1;
old_dfd = secure_relative_open(NULL, old_dirpath, O_RDONLY | O_DIRECTORY, 0);
old_dfd = secure_relative_dirfd(NULL, old_dirpath);
if (old_dfd < 0)
return -1;
old_owns = True;
@@ -1981,7 +2053,7 @@ int do_rename_at(const char *old_path, const char *new_path)
&& memcmp(old_dirpath, new_dirpath, old_dlen) == 0) {
new_dfd = old_dfd;
} else {
new_dfd = secure_relative_open(NULL, new_dirpath, O_RDONLY | O_DIRECTORY, 0);
new_dfd = secure_relative_dirfd(NULL, new_dirpath);
if (new_dfd < 0) {
e = errno;
if (old_owns) close(old_dfd);
@@ -2112,7 +2184,7 @@ int do_mkdir_at(char *path, mode_t mode)
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -2238,7 +2310,7 @@ static int do_xstat_at(const char *path, STRUCT_STAT *st, int at_flags, int (*fa
dirpath[dlen] = '\0';
bname = slash + 1;
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -2490,7 +2562,7 @@ int do_utimensat_at(const char *path, STRUCT_STAT *stp)
t[1].tv_nsec = 0;
#endif
dfd = secure_relative_open(NULL, dirpath, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirpath);
if (dfd < 0)
return -1;
@@ -2871,13 +2943,13 @@ static int ds_push(struct dirstack *ds, int fd)
return 0;
}
/* Detach the current dir as an owned fd the caller must close. At the anchor
* (top 0) the anchor is borrowed, so return a fresh dup of it instead. */
/* Detach the current traversal dirfd as an owned fd the caller must close. At
* the anchor (top 0) the anchor is borrowed, so open a fresh traversal fd. */
static int ds_take(struct dirstack *ds)
{
if (ds->top > 0)
return ds->fds[ds->top--];
return openat(ds->fds[0], ".", O_RDONLY | O_DIRECTORY);
return openat(ds->fds[0], ".", directory_traverse_flags());
}
static int ds_walk_path(struct dirstack *ds, char *path, int *hops);
@@ -2902,7 +2974,7 @@ static int ds_descend(struct dirstack *ds, const char *part, int *hops)
return 0;
}
int fd = openat(ds_cur(ds), part, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
int fd = openat(ds_cur(ds), part, directory_traverse_flags() | O_NOFOLLOW);
if (fd != -1) { /* a real subdirectory */
if (ds_push(ds, fd) < 0)
return -1;
@@ -3025,7 +3097,7 @@ static int secure_walk_at(int anchor_fd, const char *anchor_abspath,
goto cleanup;
if (is_last) {
if (flags & O_DIRECTORY)
retfd = ds_take(&ds);
retfd = openat(ds_cur(&ds), ".", flags | O_NOFOLLOW, mode);
else
errno = EISDIR;
goto cleanup;
@@ -3045,7 +3117,8 @@ static int secure_walk_at(int anchor_fd, const char *anchor_abspath,
goto cleanup;
}
}
int next_fd = openat(ds_cur(&ds), part, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
int next_fd = openat(ds_cur(&ds), part,
directory_traverse_flags() | O_NOFOLLOW);
if (next_fd == -1 && (errno == ENOTDIR || errno == ENOENT)) {
retfd = openat(ds_cur(&ds), part, flags | O_NOFOLLOW, mode);
goto cleanup;
@@ -3059,7 +3132,7 @@ static int secure_walk_at(int anchor_fd, const char *anchor_abspath,
/* O_DIRECTORY|O_NOFOLLOW leaf: the caller's O_NOFOLLOW governs the leaf. */
if (is_last && (flags & O_NOFOLLOW)) {
retfd = openat(ds_cur(&ds), part, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
retfd = openat(ds_cur(&ds), part, flags | O_NOFOLLOW, mode);
goto cleanup;
}
@@ -3071,17 +3144,17 @@ static int secure_walk_at(int anchor_fd, const char *anchor_abspath,
goto cleanup;
}
if (is_last) {
retfd = ds_take(&ds);
retfd = openat(ds_cur(&ds), ".", flags | O_NOFOLLOW, mode);
goto cleanup;
}
}
/* Empty relpath: hand back a real anchor for an O_DIRECTORY caller (ds_take
* dups the borrowed anchor), else EISDIR. An AT_FDCWD anchor is not a
* resolvable target, so it fails rather than silently returning the cwd. */
/* Empty relpath: reopen the anchor with the caller's requested directory
* access, else EISDIR. An AT_FDCWD anchor is not a resolvable target, so it
* fails rather than silently returning the cwd. */
if (!saw_component) {
if ((flags & O_DIRECTORY) && anchor_fd != AT_FDCWD)
retfd = ds_take(&ds);
retfd = openat(anchor_fd, ".", flags | O_NOFOLLOW, mode);
else
errno = EISDIR;
}
@@ -3234,6 +3307,14 @@ int secure_relative_open(const char *basedir, const char *relpath, int flags, mo
#endif // O_NOFOLLOW, O_DIRECTORY
}
/* Resolve a directory for traversal or as *at()/fchdir() authority. Callers
* that read directory entries or need a read-capable fd must continue to use
* secure_relative_open(..., O_RDONLY | O_DIRECTORY, ...). */
int secure_relative_dirfd(const char *basedir, const char *relpath)
{
return secure_relative_open(basedir, relpath, directory_traverse_flags(), 0);
}
/* Common fd-anchored resolver. A caller may explicitly allow literal ".."
* components when the fd itself is the confinement boundary: secure_walk_at()
* resolves each one by popping its held-dirfd stack and refuses a pop above the
@@ -3288,6 +3369,12 @@ int secure_relative_open_at_beneath(int anchor_fd, const char *relpath,
return secure_relative_open_at_internal(anchor_fd, relpath, flags, mode, 1);
}
int secure_relative_dirfd_at_beneath(int anchor_fd, const char *relpath)
{
return secure_relative_open_at_internal(anchor_fd, relpath,
directory_traverse_flags(), 0, 1);
}
#if defined O_NOFOLLOW && defined O_DIRECTORY && defined AT_FDCWD
/* Fill buf with len random bytes. Prefers /dev/urandom for cryptographic
* quality; falls back to rand() if /dev/urandom cannot be opened or read
@@ -3424,8 +3511,8 @@ int secure_mkstemp(char *template, mode_t perms, int operator_path)
dir = dirbuf;
}
dirfd = operator_path
? open_no_attacker_symlinks(dir, O_RDONLY | O_DIRECTORY, 0)
: secure_relative_open(dir, ".", O_RDONLY | O_DIRECTORY, 0);
? open_no_attacker_symlinks_dirfd(dir)
: secure_relative_dirfd(dir, ".");
if (dirfd < 0)
return -1;
}
@@ -3494,14 +3581,14 @@ int open_dir_secure(const char *dirname)
if (!dirname || !*dirname) {
/* The transfer root itself (file->dirname == NULL): the cwd. */
dfd = openat(AT_FDCWD, ".", O_RDONLY | O_DIRECTORY);
dfd = openat(AT_FDCWD, ".", directory_traverse_flags());
} else if (dirname[0] == '/') {
/* An absolute dirname is not expected for an in-transfer entry;
* leave it to the legacy path. */
errno = 0;
return -1;
} else {
dfd = secure_relative_open(NULL, dirname, O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(NULL, dirname);
}
if (dfd >= 0) {
+1 -1
View File
@@ -209,7 +209,7 @@ proc = subprocess.run(
# node fails first with ENXIO, which is an equally valid refusal at the open.
# A clean (returncode 0) run would be the real failure -- rsync accepting the
# device as a batch file.
refused = ('is not a regular file' in proc.stderr
refused = ('is neither a regular file nor a FIFO' in proc.stderr
or ('open error' in proc.stderr and proc.returncode != 0))
if not refused:
test_fail(
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
# --contimeout is documented as a "daemon connection timeout". The guard that
# rejects it only looked at whether connect_timeout was set, so a daemon
# connection made through a remote shell (e.g. rsync-ssl, which runs rsync with
# --rsh pointing at its helper) was rejected with the same syntax error as a
# plain non-daemon remote-shell transfer. Only reject the option when there is
# no daemon connection at all: a daemon reached via --rsh (daemon_connection ==
# 1) is still a daemon connection, and rsync now also times that connection's
# establishment phase with --contimeout the same way it times a socket connect.
import subprocess
import time
from rsyncfns import SCRATCHDIR, SRCDIR, rsync_argv, rmtree, test_fail
RERR_CONTIMEOUT = 35
base = SCRATCHDIR / 'contimeout-rsh'
rmtree(base)
base.mkdir(parents=True)
def run(*args):
return subprocess.run(rsync_argv(*args), capture_output=True, text=True)
rejected_marker = "may only be used when connecting to an rsync daemon"
# A remote-shell command that fails immediately: the option guard runs before
# rsync ever tries to exec it, so it only has to exist as a plausible --rsh
# target to put rsync into its daemon-via-rsh connection mode.
rsh_prog = str(SRCDIR / 'support' / 'lsh.sh')
# --- Daemon via --rsh must accept --contimeout (rsync-ssl's shape of call).
proc = run('--contimeout=5', '--rsh=' + rsh_prog,
'-av', 'rsync://127.0.0.1:9/mod/', str(base / 'dest'))
if rejected_marker in (proc.stderr or ''):
test_fail(f"--contimeout was rejected for a daemon-via-rsh connection:\n{proc.stderr}")
# --- A plain remote-shell (non-daemon) destination must still be rejected.
proc = run('--contimeout=5', '-av', str(base / 'src'), 'localhost:' + str(base / 'dst'))
if rejected_marker not in (proc.stderr or ''):
test_fail("--contimeout was not rejected for a non-daemon remote shell:\n" +
(proc.stderr or '') + (proc.stdout or ''))
# --- A daemon-via-rsh connection that never establishes must time out with the
# daemon-connection timeout exit code. The fake helper sleeps instead of
# connecting, so the only thing that can end the run is --contimeout firing.
fake_rsh = base / 'hang-rsh'
# The helper inherits rsync's stderr; redirect it so an orphaned "sleep" does
# not keep the harness's captured-pipe open after rsync has already exited.
fake_rsh.write_text("#!/bin/sh\nexec 2>/dev/null\nsleep 60\n")
fake_rsh.chmod(0o755)
start = time.monotonic()
proc = run('--contimeout=1', '--rsh=' + str(fake_rsh),
'-av', 'rsync://127.0.0.1:9/mod/', str(base / 'dest2'))
elapsed = time.monotonic() - start
if proc.returncode != RERR_CONTIMEOUT:
test_fail(f"--contimeout did not abort the hung connection with exit "
f"{RERR_CONTIMEOUT}; got {proc.returncode}:\n{proc.stderr}")
if elapsed >= 15:
test_fail(f"--contimeout=1 took {elapsed:.1f}s; the timeout did not bound "
"the connection establishment phase")
print("contimeout-rsh: --contimeout is accepted for a daemon-via-rsh "
"connection, rejected for a non-daemon remote shell, and times out a "
"connection that never establishes")
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env python3
"""Daemon coverage: IDN hosts allow / hosts deny matching (access.c).
A daemon gets its peer's name from DNS as ASCII, so an rsyncd.conf entry
written with non-ASCII characters is folded to its IDNA A-label form before
being matched. The IDNA mapping also folds some non-ASCII characters onto
ASCII ones (U+FF0A FULLWIDTH ASTERISK becomes '*'), so the checks below cover
both directions: a Unicode name that has to match, and the tokens that must
stay denied rather than turn into a wildcard or an address/mask. Punycode,
mixed case and an unconvertible name are covered too.
The peer name isn't assumed: a throwaway daemon comes up first to log the name
this host's resolver gives it, and the real config is written around that.
"forward lookup" is off throughout, which pins the match on the reverse-DNS
name and keeps a denied module from waiting out a resolver timeout on a name
that deliberately doesn't exist.
Like daemon-access-ip_test.py this needs a real TCP peer (--use-tcp), and the
config sets no global hosts allow so each module's own patterns decide.
"""
import re
import subprocess
from rsyncfns import (
FROMDIR, SCRATCHDIR,
claim_ports, make_tree, require_tcp, rmtree, rsync_argv, start_rsyncd,
start_test_daemon, test_fail, test_skipped,
)
PROBE_PORT = 12896
DAEMON_PORT = 12898
require_tcp("hosts allow/deny hostname matching needs a real TCP peer")
if '"IDN": true' not in subprocess.run(rsync_argv('-VV'), capture_output=True,
text=True).stdout:
test_skipped("rsync built without IDN support")
src = FROMDIR
rmtree(src)
make_tree(src, depth=2)
def write_conf(path, modules, log, pidfile):
lines = [
'# autogenerated by daemon-access-idn_test.py',
f'pid file = {pidfile}',
'use chroot = no',
'forward lookup = no',
f'log file = {log}',
'',
]
for mod, params in modules:
lines.append(f'[{mod}]')
lines.append(f'\tpath = {src}')
lines.append('\tread only = yes')
lines += [f'\t{k} = {v}' for k, v in params.items()]
lines.append('')
# rsyncd.conf is read as UTF-8 by the daemon whatever the test's locale is.
path.write_text('\n'.join(lines) + '\n', encoding='utf-8')
return path
# --- find out what this host's resolver calls the loopback peer -------------
# A throwaway daemon with one wide-open module: connect once, read the name it
# logged for us, then shut it down before the real config goes up.
probe_log = SCRATCHDIR / 'rsyncd-idn-probe.log'
probe_conf = write_conf(SCRATCHDIR / 'access-idn-probe.conf', [('probe', {})],
probe_log, SCRATCHDIR / 'rsyncd-idn-probe.pid')
claim_ports(PROBE_PORT)
probe = start_rsyncd(probe_conf, PROBE_PORT)
try:
subprocess.run(rsync_argv('-r', f'rsync://localhost:{PROBE_PORT}/probe/'),
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
finally:
probe.terminate()
probe.wait(timeout=5)
m = re.search(r'connect from (\S+) \(', probe_log.read_text(errors='replace'))
if not m:
test_fail(f"no 'connect from' line in the probe daemon log {probe_log}")
peer = m.group(1)
print(f"daemon sees its peer as {peer!r}")
def fullwidth(name):
"""The fullwidth-forms spelling of an ASCII name.
IDNA (UTS #46) maps U+FF01..U+FF5E back onto ASCII, so this gives us a
genuinely non-ASCII name that folds to a peer name we can predict without
controlling DNS. Label separators stay ASCII dots.
"""
return ''.join(chr(ord(c) - 0x21 + 0xFF01) if '!' <= c <= '~' and c != '.'
else c for c in name)
if fullwidth(peer) == peer:
test_skipped(f"peer name {peer!r} has no ASCII to respell in fullwidth")
ZWSP = '' # maps to nothing, leaving no label at all
FW_STAR = '' # FULLWIDTH ASTERISK, which IDNA maps to '*'
FW_SLASH = '' # FULLWIDTH SOLIDUS, which IDNA maps to '/'
modules = [
('ascii-name', {'hosts allow': peer}),
('ascii-upper', {'hosts allow': peer.upper()}),
('ascii-wild', {'hosts allow': peer[:1] + '*'}),
# The same name in fullwidth forms, which only matches once the token has
# been folded to A-labels.
('idn-name', {'hosts allow': fullwidth(peer)}),
('idn-mixedcase', {'hosts allow': fullwidth(peer.upper())}),
('idn-deny', {'hosts deny': fullwidth(peer)}),
# A real IDN that is not the peer, plus its punycode spelling: both must
# stay denied, and neither may be mistaken for a wildcard.
('idn-other', {'hosts allow': 'čičku.example'}),
('idn-puny', {'hosts allow': 'xn--iku-eqab.example'}),
# idn-other's token respelled with combining carons: an equivalent name
# under Unicode, so it has to be treated the same way. (A decomposed
# token that *matches* isn't constructible here -- the peer name comes
# from DNS and is ASCII -- so idn_test checks the two spellings convert
# alike, and this checks the daemon agrees they don't match.)
('idn-nfd', {'hosts allow': 'c\u030ci' 'c\u030cku.example'}),
# Tokens whose IDNA mapping yields ASCII the author never wrote. Left
# unconverted they match nothing; converted blindly, the first two would
# allow every host and the third would read as an address/mask.
('wide-star', {'hosts allow': FW_STAR}),
('wide-star-dom', {'hosts allow': FW_STAR + '.example'}),
('wide-mask', {'hosts allow': '127.0.0.0' + FW_SLASH + '8'}),
# An IDN that can't be converted at all (its label maps to nothing).
('bad-idn', {'hosts allow': ZWSP + '.example'}),
]
conf = write_conf(SCRATCHDIR / 'access-idn.conf', modules,
SCRATCHDIR / 'rsyncd.log', SCRATCHDIR / 'rsyncd.pid')
url = start_test_daemon(conf, DAEMON_PORT)
def connect(mod):
"""Return rsync's exit code for listing the module over the daemon."""
return subprocess.run(rsync_argv('-r', f'{url}{mod}/'),
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
text=True).returncode
def allowed(mod, why):
if connect(mod) != 0:
test_fail(f"connection to {mod} should be ALLOWED ({why}) but was refused")
def denied(mod, why):
if connect(mod) == 0:
test_fail(f"connection to {mod} should be DENIED ({why}) but succeeded")
allowed('ascii-name', "the peer's own name in a hosts allow")
allowed('ascii-upper', "hostname matching is case-insensitive")
allowed('ascii-wild', "an ASCII wildcard still matches")
allowed('idn-name', f"fullwidth {peer!r} folds to the peer's name")
allowed('idn-mixedcase', "IDNA case-folds the token")
denied('idn-deny', "hosts deny sees the folded token too")
denied('idn-other', "a different IDN must not match the peer")
denied('idn-puny', "an A-label for a different host must not match the peer")
denied('idn-nfd', "a decomposed spelling of that name must not match either")
denied('wide-star', "U+FF0A must not become a '*' that allows every host")
denied('wide-star-dom', "U+FF0A must not become a wildcard label")
denied('wide-mask', "U+FF0F must not become an address/mask separator")
denied('bad-idn', "an unconvertible IDN must not match anything")
print("daemon-access-idn: IDN hosts allow/deny matching + no wildcard widening")
-83
View File
@@ -1,83 +0,0 @@
#!/usr/bin/env python3
"""Daemon-mode: the server must reject a wire-supplied --max-alloc=0.
max-alloc-zero-rejected_test.py only proves the *local* client refuses
--max-alloc=0. That alone doesn't protect a daemon: a modified or older client
still forwards --max-alloc=0 on the wire, and an unpatched daemon honours it and
disables its my_alloc() allocation cap (the defence behind CVE-2024-12084 and
friends). This test drives an older rsync client -- which lacks the reject-zero
check and so forwards the option -- against the current rsync daemon, and
asserts the *daemon* refuses it.
It uses the in-tree old_versions/rsync_3.2.7 as the client (3.2.7 predates the
reject-zero fix, so it forwards --max-alloc=0 on the wire). If that binary is
missing or can't run here (e.g. a non-Linux host that can't run the static
archive) the test skips.
"""
import subprocess
from pathlib import Path
from rsyncfns import (
FROMDIR, RSYNC, SCRATCHDIR,
makepath, rmtree, start_test_daemon, test_fail, test_skipped,
write_daemon_conf,
)
DAEMON_PORT = 12932
REJECT_MSG = 'max-alloc must be greater than zero'
OLD_CLIENT = Path(__file__).resolve().parents[1] / 'old_versions' / 'rsync_3.2.7'
if not OLD_CLIENT.exists():
test_skipped(f"{OLD_CLIENT} not present")
# Confirm the static binary actually runs as rsync on this OS/arch before we
# depend on it: exec of a foreign-arch/OS binary raises OSError, while one that
# loads but can't run won't print the rsync banner. (3.2.7 predates the
# reject-zero fix, so once it runs it forwards --max-alloc=0 on the wire.)
try:
probe = subprocess.run([str(OLD_CLIENT), '--version'],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=True)
except OSError as e:
test_skipped(f"cannot run {OLD_CLIENT.name} on this OS/arch: {e}")
if probe.returncode != 0 or 'version 3.2.7' not in probe.stdout:
test_skipped(f"{OLD_CLIENT.name} does not run as rsync on this OS/arch")
# Module served by the *current* (patched) daemon.
src = FROMDIR
rmtree(src)
makepath(src)
(src / 'file.txt').write_text('hello\n')
conf = write_daemon_conf([('mod', {'path': str(src), 'read only': 'yes'})])
url = start_test_daemon(conf, DAEMON_PORT, rsync_cmd=RSYNC)
dest = SCRATCHDIR / 'out.txt'
def run_client(*extra):
argv = [str(OLD_CLIENT), *extra, f'{url}mod/file.txt', str(dest)]
return subprocess.run(argv, stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE, text=True)
# Positive control: the old client and current daemon transfer fine without the
# option, so the failure below is specifically the daemon refusing the option.
dest.unlink(missing_ok=True)
ctrl = run_client()
if ctrl.returncode != 0:
test_fail(f"old client could not talk to the current daemon:\n{ctrl.stderr}")
# The attack: a forwarded --max-alloc=0 must be refused by the daemon.
dest.unlink(missing_ok=True)
proc = run_client('--max-alloc=0')
if proc.returncode == 0:
test_fail("daemon accepted a wire-supplied --max-alloc=0")
if REJECT_MSG not in proc.stderr:
test_fail("daemon did not reject --max-alloc=0 with the expected message; "
f"stderr:\n{proc.stderr}")
print("daemon-max-alloc-zero: daemon refuses a wire-supplied --max-alloc=0 "
f"(client {OLD_CLIENT.name})")
+5 -5
View File
@@ -91,10 +91,10 @@
]
},
{
"_comment": "Ubuntu 18.04 older-LTS backport coverage on a root@ box; no 18.04 runner image exists so it mirrors the 22.04 matrix lane.",
"_comment": "Ubuntu 18.04 older-LTS backport coverage on a root@ box; no 18.04 runner image exists so it mirrors the 22.04 workflow.",
"name": "ubuntu-1804",
"ssh_host": "root@ubuntu-1804",
"workflow": "ubuntu-build.yml",
"workflow": "ubuntu-22.04-build.yml",
"_python_comment": "18.04's default python3 is 3.6, but runtests.py uses subprocess capture_output (3.7+); run the suite under the 3.10 that's also installed.",
"python": "python3.10",
"_configure_flags_comment": "18.04's libzstd 1.3.3 lacks ZSTD_minCLevel (configure aborts), so disable zstd; also disable lz4 so the default -z compressor is zlib -- with lz4 as the default, compress-options' --compress-level=9 check fails since lz4 has no levels. xxhash is fine.",
@@ -110,10 +110,10 @@
]
},
{
"_comment": "Ubuntu 20.04 older-LTS backport coverage on a root@ box; no 20.04 runner image exists so it mirrors the 22.04 matrix lane.",
"_comment": "Ubuntu 20.04 older-LTS backport coverage on a root@ box; no 20.04 runner image exists so it mirrors the 22.04 workflow.",
"name": "ubuntu-2004",
"ssh_host": "root@ubuntu-2004",
"workflow": "ubuntu-build.yml",
"workflow": "ubuntu-22.04-build.yml",
"configure_flags": [
"--with-rrsync"
],
@@ -126,7 +126,7 @@
"_comment": "Builds unprivileged (like a CI runner) and runs the suite via sudo; the nonroot pass reruns the privilege-sensitive tests as the ssh user. protocols: [30, 29] runs the check30/check29 passes on this NO-xattrat kernel: the CI's proto-29 step runs here, but the only other proto-29 fleet box (ubuntu-2604) is xattrat, so without this the (no-xattrat x proto-29) cell is uncovered.",
"name": "ubuntu-2204",
"ssh_host": "runner@ubuntu-2204",
"workflow": "ubuntu-build.yml",
"workflow": "ubuntu-22.04-build.yml",
"privilege": "sudo",
"nonroot": true,
"protocols": [
+5 -5
View File
@@ -6,10 +6,10 @@ Builds the committed HEAD of an rsync checkout on a fleet of remote machines
--use-tcp) in parallel, and prints one report of only the UNEXPECTED results --
a fast local pre-flight for the GitHub CI matrix.
Each target maps to a .github/workflows/*.yml job or matrix lane: the per-target
configure flags mirror that lane, and the pipe-run RSYNC_EXPECT_SKIPPED list is
PARSED from the workflow (not hardcoded). The --use-tcp run never sets an
expected-skip list (matching the workflows), so only test FAILs matter there.
Each target maps 1:1 to a .github/workflows/*.yml job: the per-target configure
flags mirror that workflow, and the pipe-run RSYNC_EXPECT_SKIPPED list is PARSED
from the workflow (not hardcoded). The --use-tcp run never sets an expected-skip
list (matching the workflows), so only test FAILs matter there.
The tcp pass runs only the tests that can reach the daemon transport, because it
follows a full pipe pass over the very same build: --use-tcp is observable only
@@ -167,7 +167,7 @@ PUSH_EXCLUDES = [
class Target:
name: str
ssh_host: str | None # null in JSON => run locally
workflow: str # workflow containing the matching job or matrix lane
workflow: str # filename under .github/workflows
configure_flags: list[str]
make: str = "make" # e.g. "gmake" on the BSDs/Solaris
env_prefix: str = "" # exported before configure AND make (e.g. PATH)
+186
View File
@@ -0,0 +1,186 @@
#!/usr/bin/env python3
# Verify that rsync converts an IDN (internationalized domain name) host to
# its IDNA A-label (Punycode) form, and that it leaves an ASCII host name
# alone. Only the labels that are not ASCII get rewritten, so an address
# literal, an already-punycoded name, and a name that isn't a valid IDN all
# reach the resolver as typed. A name typed with combining marks is normalized
# on the way, so it converts the same as its precomposed spelling.
#
# Two daemon connection methods carry the host name out of rsync, so both are
# checked:
# * daemon over a remote shell (what rsync-ssl does): the host is handed to
# the --rsh helper.
# * direct daemon socket: observed through a dummy HTTP proxy (RSYNC_PROXY) on
# loopback, so this part only runs under --use-tcp.
# A plain remote-shell transfer (host:path) is intentionally left alone, since
# that name belongs to the user's ssh.
#
# The daemon side of IDN -- hosts allow/deny matching -- is daemon-access-idn.
import os
import shlex
import socket
import subprocess
import sys
import threading
from rsyncfns import (
RSYNC, SCRATCHDIR, USE_TCP, claim_ports, run_rsync,
test_fail, test_skipped,
)
if '"IDN": true' not in run_rsync('-VV', check=True, capture_output=True).stdout:
test_skipped("rsync built without IDN support")
def find_utf8_locale():
try:
out = subprocess.check_output(['locale', '-a'], text=True,
stderr=subprocess.DEVNULL)
except (OSError, subprocess.CalledProcessError):
return None
avail = out.split()
for want in ('C.UTF-8', 'C.utf8', 'en_US.UTF-8', 'en_US.utf8'):
if want in avail:
return want
for loc in avail:
if loc.lower().replace('-', '').endswith('utf8'):
return loc
return None
utf8_locale = find_utf8_locale()
if not utf8_locale:
test_skipped("no UTF-8 locale available to encode the IDN host")
idn_host = "\u010ci\u010dku.example"
ascii_host = "xn--iku-eqab.example"
# The same name with each caron letter spelled as a plain "c" plus a combining
# caron (U+030C). Unicode calls the two spellings equivalent, so both have to
# come out as the same A-label; libidn2 is what normalizes them.
nfd_host = "c\u030ci" "c\u030cku.example"
env = os.environ.copy()
env['LC_ALL'] = utf8_locale
out_dir = (str(SCRATCHDIR / 'out') + '/').encode()
def run_idn(url, *extra, extra_env=None):
# A bytes argv keeps the UTF-8 host intact regardless of Python's
# filesystem encoding.
e = dict(env)
if extra_env:
e.update(extra_env)
argv = [a.encode() for a in shlex.split(RSYNC)]
argv += [a.encode() for a in extra]
argv += [url.encode('utf-8'), out_dir]
return subprocess.run(argv, capture_output=True, env=e, timeout=30)
# --- daemon over a remote shell (the rsync-ssl mechanism) ------------------
helper = SCRATCHDIR / 'idn-rsh.sh'
helper.write_text('#!/bin/sh\nprintf %s "$1" > "$IDN_RSH_OUT"\nexit 1\n')
helper.chmod(0o755)
hostfile = SCRATCHDIR / 'idn-rsh-host'
def rsh_host(url_host):
"""The host name rsync hands the --rsh helper for rsync://<url_host>/."""
if hostfile.exists():
hostfile.unlink()
run_idn(f"rsync://{url_host}/module/", f"--rsh={helper}",
extra_env={'IDN_RSH_OUT': str(hostfile)})
if not hostfile.exists():
test_fail(f"the --rsh helper never ran for {url_host!r}")
return hostfile.read_bytes().decode('utf-8', 'surrogateescape')
def check_rsh(url_host, want, what):
got = rsh_host(url_host)
if got != want:
test_fail(f"daemon-over-rsh sent host {got!r} for {what} "
f"({url_host!r}), expected {want!r}")
print(f"OK: {what} -> {got}")
# A U-label becomes its A-label, case-folded by the IDNA mapping. An ASCII
# label is handed on byte for byte, case included, since DNS doesn't care.
check_rsh(idn_host, ascii_host, "a Unicode host")
check_rsh(nfd_host, ascii_host, "a decomposed Unicode host")
check_rsh("C\u030cI" "C\u030cKU.Example", "xn--iku-eqab.Example",
"a decomposed mixed-case Unicode host")
check_rsh("ČIČKU.Example", "xn--iku-eqab.Example",
"a mixed-case Unicode host")
check_rsh(ascii_host, ascii_host, "an already-punycoded host")
check_rsh("XN--IKU-EQAB.Example", "XN--IKU-EQAB.Example",
"a mixed-case punycoded host")
# A name that isn't a valid IDN goes out as-is instead of being rewritten into
# some other name (the U+200B one would map to ".example"), so the resolver
# fails on it just as it did before.
check_rsh("xn--0.example", "xn--0.example", "an undecodable A-label")
check_rsh("ـx.example", "ـx.example", "a label with a disallowed character")
check_rsh(".example", ".example", "a label that maps to nothing")
# --- direct daemon socket, observed via a dummy proxy -----------------------
if not USE_TCP:
print("direct-socket proxy check needs --use-tcp; skipping that part")
sys.exit(0)
PROXY_PORT = 13335
claim_ports(PROXY_PORT)
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind(('127.0.0.1', PROXY_PORT))
listener.listen(1)
captured = {}
def serve_one():
conn, _ = listener.accept()
conn.settimeout(5)
data = b""
try:
while b"\r\n\r\n" not in data and len(data) < 65536:
chunk = conn.recv(8192)
if not chunk:
break
data += chunk
except socket.timeout:
pass
captured['request'] = data
try:
conn.sendall(b"HTTP/1.0 403 Forbidden\r\n\r\n")
conn.shutdown(socket.SHUT_RDWR)
except OSError:
pass
conn.close()
t = threading.Thread(target=serve_one)
t.daemon = True
t.start()
proc = run_idn(f"rsync://{idn_host}:873/whatever/",
extra_env={'RSYNC_PROXY': f'127.0.0.1:{PROXY_PORT}'})
t.join(timeout=15)
listener.close()
if proc.returncode >= 128:
sys.stderr.write(proc.stderr.decode('latin1'))
test_fail(f"rsync killed by signal (status={proc.returncode})")
request = captured.get('request', b'')
if not request:
test_fail("dummy proxy received no CONNECT request from rsync")
if ascii_host.encode() not in request:
sys.stderr.write("proxy received: %r\n" % request.split(b"\r\n", 1)[0])
test_fail(f"expected A-label {ascii_host} in the proxy CONNECT request")
print(f"OK: direct-socket CONNECT host sent as {ascii_host}")
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
import filecmp
import os
import re
import shlex
import shutil
import subprocess
from rsyncfns import SCRATCHDIR, SRCDIR, TOOLDIR, test_fail
make_vars = {}
for line in (TOOLDIR / 'Makefile').read_text().splitlines():
name, separator, value = line.partition('=')
if separator and name in ('prefix', 'exec_prefix', 'bindir',
'EXEEXT', 'STRIP'):
make_vars[name] = value.strip()
if not make_vars.get('STRIP'):
test_fail('configured Makefile does not define STRIP')
source_rsync = TOOLDIR / f"rsync{make_vars.get('EXEEXT', '')}"
if not source_rsync.is_file():
test_fail('cannot find the built rsync binary')
builddir = SCRATCHDIR / 'build'
builddir.mkdir()
makefile_text = (TOOLDIR / 'Makefile').read_text()
if 'install: all\n' not in makefile_text:
test_fail('cannot isolate the install target from the shared build')
if 'Makefile: Makefile.in config.status configure.sh config.h.in\n' not in makefile_text:
test_fail('cannot isolate Makefile regeneration from the shared build')
(builddir / 'Makefile').write_text(
makefile_text
.replace('install: all\n', 'install:\n', 1)
.replace('Makefile: Makefile.in config.status configure.sh config.h.in\n',
'Makefile:\n', 1)
)
for name in (source_rsync.name, 'install-sh', 'rsync-ssl', 'rrsync', 'rsync.1',
'rsync-ssl.1', 'rsyncd.conf.5', 'rrsync.1'):
source = TOOLDIR / name
if not source.is_file():
source = SRCDIR / name
if source.is_file():
shutil.copy2(source, builddir / name)
def expand_make_value(value):
for _ in range(10):
expanded = re.sub(
r'\$\{([^}]+)\}|\$\(([^)]+)\)',
lambda match: make_vars.get(match.group(1) or match.group(2),
match.group(0)),
value,
)
if expanded == value:
return expanded
value = expanded
test_fail(f'cannot expand Makefile value {value!r}')
bindir = expand_make_value(make_vars.get('bindir', ''))
if not bindir or '$' in bindir:
test_fail(f'cannot determine configured bindir from {bindir!r}')
make = shlex.split(os.environ.get('MAKE', 'make'))
tools = SCRATCHDIR / 'tools'
tools.mkdir()
for strip_name in ('strip', 'aarch64-linux-gnu-strip'):
destdir = SCRATCHDIR / 'roots' / strip_name
strip_log = SCRATCHDIR / f'{strip_name}.log'
strip = tools / strip_name
strip.write_text('#!/bin/sh\nprintf \'%s\\n\' "$@" >"$STRIP_LOG"\n')
strip.chmod(0o755)
env = os.environ.copy()
env['STRIP_LOG'] = str(strip_log)
proc = subprocess.run(
[*make, f'DESTDIR={destdir}', f'STRIP={strip}', 'install-strip'],
cwd=builddir, env=env, capture_output=True, text=True,
)
if proc.returncode != 0:
test_fail(f'install-strip failed with {strip_name}:\n'
f'{proc.stdout}{proc.stderr}')
installed = destdir / bindir.lstrip('/') / source_rsync.name
if not installed.is_file():
test_fail(f'install-strip did not install {installed}')
if not strip_log.is_file():
test_fail(f'install-strip did not call {strip_name}')
if strip_log.read_text().splitlines() != [str(installed)]:
test_fail(f'{strip_name} was not called with {installed}')
if not filecmp.cmp(source_rsync, installed, shallow=False):
test_fail(f'{strip_name} unexpectedly changed the installed test binary')
@@ -1,9 +0,0 @@
#!/usr/bin/env python3
from rsyncfns import SCRATCHDIR, rsync_argv
from rsyncfns import expect_fail
expect_fail(
rsync_argv('--max-alloc=0', str(SCRATCHDIR / 'missing-src'), str(SCRATCHDIR / 'missing-dst')),
'max-alloc must be greater than zero',
)
print("max-alloc-zero-rejected: --max-alloc=0 is rejected")
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""``--max-alloc=0`` means "the largest limit this build supports".
Three things are asserted, and the second is the reason 0 is worth keeping as a
spelling at all:
1. 0 is accepted, and a transfer using it works.
2. 0 reaches the peer as the literal "0", not as a resolved number. Each side
then resolves it against its own SIZE_MAX. That is what makes 0 the only
value correct for both ends of a mixed-word-size pairing: the ceiling is
SIZE_MAX/2, so any number large enough to be worth setting on a 64-bit
client (over 2047M) is refused as "too large" by a 32-bit daemon.
3. The parser's upper bound is still enforced. Accepting 0 again must not
bring back the unbounded ``size *= atof(size_arg)`` that was fixed in
3.5.0, so an out-of-range value is still rejected rather than wrapping.
The forwarding check in (2) deliberately inspects the argv the remote shell is
handed rather than a transfer outcome: a resolved number also copies files
happily on a same-word-size pair, so an outcome-based assertion would pass on
exactly the configuration this behaviour does not matter for.
"""
import os
import shlex
from rsyncfns import (
SCRATCHDIR, SRCDIR, expect_fail, rsh_cmd, rmtree, rsync_argv,
rsync_path_arg, run_rsync, test_fail,
)
base = SCRATCHDIR / 'max-alloc-zero'
rmtree(base)
src = base / 'from'
dst = base / 'to'
src.mkdir(parents=True)
dst.mkdir(parents=True)
(src / 'file.txt').write_text('hello\n')
# --- 1. 0 is accepted -------------------------------------------------------
run_rsync('-r', '--max-alloc=0', f'{src}/', f'{dst}/')
if (dst / 'file.txt').read_text() != 'hello\n':
test_fail('--max-alloc=0 did not copy the file')
# --- 2. 0 goes on the wire un-normalized ------------------------------------
argv_log = base / 'server-argv'
wrapper = base / 'log-rsh.sh'
wrapper.write_text(
'#!/bin/sh\n'
'# Log the command line built for the peer, then behave like lsh.sh.\n'
f'printf \'%s\\n\' "$*" >> {shlex.quote(str(argv_log))}\n'
f'exec {shlex.quote(str(SRCDIR / "support" / "lsh.sh"))} "$@"\n'
)
wrapper.chmod(0o755)
rmtree(dst)
dst.mkdir()
os.environ['RSYNC_RSH'] = rsh_cmd(str(wrapper))
run_rsync('-r', '--max-alloc=0', f'--rsync-path={rsync_path_arg()}',
f'localhost:{src}/', f'{dst}/')
del os.environ['RSYNC_RSH']
if (dst / 'file.txt').read_text() != 'hello\n':
test_fail('--max-alloc=0 did not copy the file over the remote shell')
logged = argv_log.read_text() if argv_log.exists() else ''
if not logged:
test_fail('the remote-shell wrapper logged no command line')
if '--max-alloc=0' not in f' {logged} '.replace('\n', ' '):
test_fail('--max-alloc=0 was not forwarded verbatim; the peer was sent:\n'
f'{logged}'
'\nA resolved number here would be rejected as "too large" by a '
'peer with a smaller SIZE_MAX.')
# --- 3. the upper bound still holds -----------------------------------------
# 8192P is one step past SIZE_ARG_MAX (SIZE_MAX/2) on a 64-bit build; on a
# 32-bit one the P multiplier alone already exceeds it. Either way: too large.
expect_fail(rsync_argv('--max-alloc=8192P', f'{src}/', f'{dst}/'), 'is too large')
# And the min-value message must keep advertising a spelling that works.
expect_fail(rsync_argv('--max-alloc=1', f'{src}/', f'{dst}/'),
'or 0 for unlimited')
print('max-alloc-zero: 0 is accepted, forwarded verbatim, and the bound holds')
@@ -37,6 +37,17 @@ if _proto is not None and _proto < 30:
hook_code = r'''
#define _GNU_SOURCE
/* Build the hook itself WITHOUT large-file redirection, whatever the compiler
* defaults to. Debian's armhf/hppa/powerpc gcc predefines
* -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 (check with "gcc -v -E -"), and under
* those macros glibc's __REDIRECT renames the DEFINITIONS below -- open()
* becomes open64(), fstatat() becomes __fstatat64_time64() -- which then
* collide with the explicit large-file wrappers further down ("symbol `open64'
* is already defined"). Undefining them here keeps each name declared exactly
* once, so the hook always exports both spellings and interposes whichever set
* the rsync under test was linked against. */
#undef _FILE_OFFSET_BITS
#undef _TIME_BITS
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
@@ -56,6 +67,9 @@ static int (*real_open)(const char *, int, ...);
static int (*real_openat)(int, const char *, int, ...);
static int (*real_fstatat)(int, const char *, struct stat *, int);
static int (*real_fxstatat)(int, int, const char *, struct stat *, int);
static int (*real_fstatat64)(int, const char *, struct stat64 *, int);
static int (*real_fxstatat64)(int, int, const char *, struct stat64 *, int);
static int (*real_fstatat64_time64)(int, const char *, struct stat64 *, int);
/* Resolve on demand rather than trusting our constructor to have run. A
* preloaded open() interposes for the whole process the moment the loader maps
@@ -77,6 +91,10 @@ static void hook_resolve(void)
if (!real_openat) real_openat = dlsym(RTLD_NEXT, "openat");
if (!real_fstatat) real_fstatat = dlsym(RTLD_NEXT, "fstatat");
if (!real_fxstatat) real_fxstatat = dlsym(RTLD_NEXT, "__fxstatat");
if (!real_fstatat64) real_fstatat64 = dlsym(RTLD_NEXT, "fstatat64");
if (!real_fxstatat64) real_fxstatat64 = dlsym(RTLD_NEXT, "__fxstatat64");
if (!real_fstatat64_time64)
real_fstatat64_time64 = dlsym(RTLD_NEXT, "__fstatat64_time64");
hook_resolving = 0;
}
@@ -151,6 +169,10 @@ static int swap_and_deny(void)
# define HOOK_TAKES_MODE(f) ((f) & O_CREAT)
#endif
#ifndef O_LARGEFILE
# define O_LARGEFILE 0
#endif
static int is_victim_write(const char *path, int flags)
{
return path && strcmp(path, "victim") == 0
@@ -231,6 +253,49 @@ int open(const char *path, int flags, ...)
return fd;
}
/* --- large-file spellings -------------------------------------------------
* Which names the RECEIVER calls is settled by its own build: where off_t is
* not already 64 bits, configure's AC_SYS_LARGEFILE adds -D_FILE_OFFSET_BITS=64
* (i386, alpha, ...), and distro CPPFLAGS add it -- along with -D_TIME_BITS=64
* -- on the 64-bit time_t ports, so glibc redirects each open()/openat()/
* fstatat() call to open64()/openat64()/fstatat64()/__fstatat64_time64().
* "objdump -T rsync | grep UND" says which set a given build imports.
*
* Which names THIS HOOK exports is a different question with a different
* answer, settled by whatever the "cc" below it defaults to -- see the #undef
* at the top. Nothing keeps the two in step, so define every spelling and let
* the loader match them up. With only the unsuffixed ones the receiver's opens
* sail straight past the hook, no EACCES is ever injected, and the test reports
* "positive control failed" having exercised nothing at all.
*
* O_LARGEFILE is the only thing open64() adds over open(), so the wrappers
* below can hand the call to the unsuffixed interposer above. */
int open64(const char *path, int flags, ...)
{
mode_t mode = 0;
if (HOOK_TAKES_MODE(flags)) {
va_list ap;
va_start(ap, flags);
mode = (mode_t)va_arg(ap, int);
va_end(ap);
}
return open(path, flags | O_LARGEFILE, mode);
}
int openat64(int dfd, const char *path, int flags, ...)
{
mode_t mode = 0;
if (HOOK_TAKES_MODE(flags)) {
va_list ap;
va_start(ap, flags);
mode = (mode_t)va_arg(ap, int);
va_end(ap);
}
return openat(dfd, path, flags | O_LARGEFILE, mode);
}
/* ona_open() decides via fstatat(..., AT_SYMLINK_NOFOLLOW) and refuses a
* component owned by neither root nor the euid. Model the attacker as a
* different uid so a retry that kept the ownership walk refuses the swap. */
@@ -276,6 +341,68 @@ int __fxstatat(int ver, int dfd, const char *path, struct stat *st, int flags)
return rc;
}
/* The stat family's large-file spellings. st_mode and st_uid sit ahead of the
* timestamps in every glibc struct stat layout, so the time32/time64 variants
* of the buffer are interchangeable for the two fields touched here. */
static void model_foreign_owner64(int rc, const char *path, struct stat64 *st)
{
if (rc == 0 && swapped && path && strcmp(path, "pdir") == 0
&& S_ISLNK(st->st_mode)) {
st->st_uid = geteuid() + 1;
mark(getenv("RSYNC_PARTIAL_RETRY_FOREIGN_MARKER"));
}
}
int fstatat64(int dfd, const char *path, struct stat64 *st, int flags)
{
int rc, saved_errno;
hook_resolve();
if (!real_fstatat64) {
errno = ENOSYS;
return -1;
}
rc = real_fstatat64(dfd, path, st, flags);
saved_errno = errno;
model_foreign_owner64(rc, path, st);
errno = saved_errno;
return rc;
}
int __fxstatat64(int ver, int dfd, const char *path, struct stat64 *st, int flags)
{
int rc, saved_errno;
hook_resolve();
if (!real_fxstatat64) {
errno = ENOSYS;
return -1;
}
rc = real_fxstatat64(ver, dfd, path, st, flags);
saved_errno = errno;
model_foreign_owner64(rc, path, st);
errno = saved_errno;
return rc;
}
/* A 32-bit port built with -D_TIME_BITS=64 (Debian's armhf/armel/hppa/powerpc,
* ...) reaches fstatat() under this third name. */
int __fstatat64_time64(int dfd, const char *path, struct stat64 *st, int flags)
{
int rc, saved_errno;
hook_resolve();
if (!real_fstatat64_time64) {
errno = ENOSYS;
return -1;
}
rc = real_fstatat64_time64(dfd, path, st, flags);
saved_errno = errno;
model_foreign_owner64(rc, path, st);
errno = saved_errno;
return rc;
}
__attribute__((constructor)) static void hook_loaded(void)
{
hook_resolve();
+40 -12
View File
@@ -38,12 +38,33 @@ if run_rsync('-a', '--preallocate', f'{src}/', f'{TODIR}/',
check=False, capture_output=True).returncode != 0:
test_skipped("--preallocate not supported on this platform")
def fs_can_punch_holes():
"""True only where the kernel can deallocate blocks via FALLOC_FL_PUNCH_HOLE
-- the mechanism do_punch_hole uses for --sparse. A filesystem may report
seek-based sparseness yet still keep every block on a punch (e.g. where
rsync's punch falls back to writing zeros), so probe the real capability and
assert the hole only where it actually frees blocks."""
def punch_frees(offset, length, size):
"""True where punching [offset, offset+length) out of a `size`-byte file
really deallocates blocks -- the mechanism do_punch_hole uses for --sparse.
Two separate things can leave st_blocks untouched, so each assertion below
probes the exact shape it relies on. A filesystem may report seek-based
sparseness yet still keep every block on a punch (e.g. where rsync's punch
falls back to writing zeros), which a whole-file probe catches. And a punch
only frees storage in whole allocation units, which are not always 4 KiB: a
tmpfs frees whole pages, 16 KiB on loongarch/loong64 (and 64 KiB on a
64k-page ppc64el or arm64 kernel), and a filesystem may be formatted with a
block size above the page size. An interior run spanning no whole unit is
zeroed rather than deallocated, so st_blocks does not move and an assertion
phrased in st_blocks would report a hole-punching regression that is really
just the filesystem's granularity.
fallocate64() rather than fallocate(): where off_t is 32 bits the latter
takes 32-bit offsets, so ctypes' 64-bit arguments do not line up with what
it reads (on i386 it takes the high half of `offset` as its `length`) and
every probe fails with EINVAL -- which is why every assertion below has
silently done nothing on all the 32-bit ports. fallocate64() takes off64_t
everywhere and is a plain alias of fallocate() where off_t is already 64
bits wide.
The probe data has to be incompressible: a filesystem that compresses
(btrfs with compress=) stores a run of one repeated byte in almost no
blocks, leaving a successful punch with nothing to free."""
import ctypes
import ctypes.util
KEEP_SIZE, PUNCH_HOLE = 0x01, 0x02
@@ -52,12 +73,12 @@ def fs_can_punch_holes():
try:
libc = ctypes.CDLL(ctypes.util.find_library('c') or 'libc.so.6',
use_errno=True)
libc.fallocate.argtypes = [ctypes.c_int, ctypes.c_int,
ctypes.c_longlong, ctypes.c_longlong]
libc.fallocate64.argtypes = [ctypes.c_int, ctypes.c_int,
ctypes.c_longlong, ctypes.c_longlong]
fd = os.open(p, os.O_CREAT | os.O_RDWR | os.O_TRUNC, 0o644)
os.write(fd, b'\xff' * 65536)
os.write(fd, os.urandom(size))
before = os.fstat(fd).st_blocks
ret = libc.fallocate(fd, PUNCH_HOLE | KEEP_SIZE, 0, 65536)
ret = libc.fallocate64(fd, PUNCH_HOLE | KEEP_SIZE, offset, length)
return ret == 0 and os.fstat(fd).st_blocks < before
except (OSError, AttributeError, ValueError):
return False
@@ -70,7 +91,7 @@ def fs_can_punch_holes():
pass
can_punch = fs_can_punch_holes()
can_punch = punch_frees(0, 65536, 65536)
def seed_plain(size=1_000_000):
@@ -139,6 +160,13 @@ with open(src / deep, 'wb') as source, open(TODIR / deep, 'wb') as dest:
source.write(block)
dest.write(block)
# Only assert the interior punch where the filesystem can free a 24 KiB run
# sitting 4 KiB into a 32 KiB block -- the exact shape written just above.
can_punch_interior = can_punch and punch_frees(4096, 24576, 32768)
if can_punch and not can_punch_interior:
print("preallocate: interior-hole assertion skipped: this filesystem's "
"allocation unit cannot free a 24 KiB run inside a 32 KiB block")
matched_size = os.path.getsize(TODIR / deep)
matched_before = allocated(TODIR / deep)
run_rsync('-a', '--ignore-times', '--inplace', '--sparse', '--no-whole-file',
@@ -146,7 +174,7 @@ run_rsync('-a', '--ignore-times', '--inplace', '--sparse', '--no-whole-file',
assert_same(TODIR / deep, src / deep,
label='--inplace --sparse matched-block content')
matched_after = allocated(TODIR / deep)
if (can_punch and matched_before >= matched_size
if (can_punch_interior and matched_before >= matched_size
and matched_after * 2 >= matched_before):
test_fail(f"--inplace --sparse left matching interior zero runs allocated: "
f"{matched_after} of {matched_before} bytes remain allocated "
+11 -5
View File
@@ -49,11 +49,17 @@ if not _chown_5001(workdir / 'dst'):
if not os.environ.get('RSYNC_UNSHARED'):
unshare = shutil.which('unshare')
if unshare is not None:
probe = subprocess.run(
[unshare, '--user', '--map-root-user',
'--map-users', '5001:100000:1', 'true'],
capture_output=True,
)
try:
probe = subprocess.run(
[unshare, '--user', '--map-root-user',
'--map-users', '5001:100000:1', 'true'],
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=5,
)
except subprocess.TimeoutExpired:
test_skipped("Can't chown (unshare probe timed out)")
if probe.returncode == 0:
print("Re-running under unshare with UID mapping...")
env = os.environ.copy()
+138
View File
@@ -0,0 +1,138 @@
"""Process substitution /dev/fd/ write pipe pseudo-paths for --log-file must not crash and must successfully write logs, but must be rejected if confined root."""
import shlex
import shutil
import subprocess
import sys
from pathlib import Path
from rsyncfns import (
SCRATCHDIR, makepath, rmtree, rsync_argv, test_fail, test_skipped,
)
if not sys.platform.startswith('linux'):
test_skipped('Kernel pseudo-path string is a Linux-specific procfs feature')
raise SystemExit(0)
# We require bash specifically because standard POSIX /bin/sh does not
# guarantee support for >(...) process substitution syntax.
bash = shutil.which('bash')
if bash is None:
test_skipped('bash is unavailable, cannot test process substitution')
# Verify the host bash actually supports process substitution
probe = subprocess.run(
[bash, '-c', 'echo "probe" > >(cat > /dev/null)'],
capture_output=True
)
if probe.returncode != 0:
test_skipped('bash process substitution is not supported on this system')
base = Path(SCRATCHDIR / 'rsync-pseudo-path').resolve()
src = base / 'src'
dest = base / 'dest'
log_out = base / 'test_log.txt'
log_out_confined = base / 'test_log_confined.txt'
makepath(src, dest)
(src / 'transfer_me.txt').write_text('sync this\n')
rsync_base_cmd = shlex.join(rsync_argv('-a'))
src_path = shlex.quote(str(src) + '/')
dest_path = shlex.quote(str(dest) + '/')
log_path = shlex.quote(str(log_out))
log_path_confined = shlex.quote(str(log_out_confined))
# -------------------------------------------------------------------------
# TEST 1: Unconfined process substitution (Should Succeed)
# -------------------------------------------------------------------------
bash_script = f"{rsync_base_cmd} -v --log-file=>(cat > {log_path}) {src_path} {dest_path}"
try:
proc = subprocess.run(
[bash, '-c', bash_script],
capture_output=True,
text=True,
timeout=10,
)
except subprocess.TimeoutExpired:
rmtree(base)
test_fail('process substitution test timed out')
ctx = f'rc={proc.returncode}, stderr={proc.stderr.strip()!r}'
if proc.returncode != 0:
rmtree(base)
test_fail(f'rsync crashed writing to a pseudo-path log pipe ({ctx})')
if not (dest / 'transfer_me.txt').is_file():
rmtree(base)
test_fail(f'rsync failed to transfer the allowed file ({ctx})')
if not log_out.exists() or log_out.stat().st_size == 0:
rmtree(base)
test_fail(f'rsync survived, but failed to write data to the log pipe ({ctx})')
log_data = log_out.read_text()
if "transfer_me.txt" not in log_data:
rmtree(base)
test_fail(f'Log pipe received data, but is missing expected output: {log_data[:100]}')
print('Test 1 Passed: rsync successfully wrote logs to a process substitution pseudo-path')
# -------------------------------------------------------------------------
# TEST 2: Confined Root (Should Reject Pseudo-path)
# -------------------------------------------------------------------------
bash_script_confined = f"{rsync_base_cmd} --confine-root={dest_path} -v --log-file=>(cat > {log_path_confined}) {src_path} {dest_path}"
try:
proc_confined = subprocess.run(
[bash, '-c', bash_script_confined],
capture_output=True,
text=True,
timeout=10,
)
except subprocess.TimeoutExpired:
rmtree(base)
test_fail('confined process substitution test timed out')
ctx_confined = f'rc={proc_confined.returncode}, stderr={proc_confined.stderr.strip()!r}'
# Rsync considers log-file failure a warning, so it still exits 0.
stderr_lower = proc_confined.stderr.lower()
if "no such file or directory" in stderr_lower and "failed to open" in stderr_lower:
if log_out_confined.exists() and log_out_confined.stat().st_size > 0:
rmtree(base)
test_fail(f'rsync printed an error but still wrote the confined log! ({ctx_confined})')
print('Test 2 Passed: rsync correctly rejected the pseudo-path when confine_root was active')
else:
rmtree(base)
test_fail(f'rsync failed to reject the pseudo-path or had an unexpected error ({ctx_confined})')
# A pseudo-path is valid only when its descriptor number is the final component.
rmtree(dest)
makepath(dest)
trailing_script = (
f'pipe_path=<(printf "transfer_me.txt\\n"); '
f'{rsync_base_cmd} --exclude-from="$pipe_path/trailing" {src_path} {dest_path}'
)
try:
proc_trailing = subprocess.run(
[bash, '-c', trailing_script],
capture_output=True,
text=True,
timeout=10,
)
except subprocess.TimeoutExpired:
rmtree(base)
test_fail('trailing-component pseudo-path test timed out')
if proc_trailing.returncode == 0:
rmtree(base)
test_fail('/dev/fd/N/trailing unexpectedly opened descriptor N')
if (dest / 'transfer_me.txt').exists():
rmtree(base)
test_fail('transfer continued after accepting a trailing pseudo-path component')
rmtree(base)
raise SystemExit(0)
+80
View File
@@ -0,0 +1,80 @@
#!/usr/bin/env python3
"""--read-batch process substitution /dev/fd/ pipe must not crash with strict file-type checks."""
import os
import shlex
import shutil
import subprocess
import tempfile
import sys
from pathlib import Path
from rsyncfns import SCRATCHDIR, makepath, rmtree, rsync_argv, test_fail, test_skipped
# We require bash specifically because standard POSIX /bin/sh does not
# guarantee support for <(...) process substitution syntax.
if not sys.platform.startswith('linux'):
test_skipped('This test requires Linux platform')
bash = shutil.which('bash')
if bash is None:
test_skipped('bash is unavailable, cannot test process substitution')
# Verify the host bash actually supports process substitution
probe = subprocess.run(
[bash, '-c', 'cat <(echo "probe")'],
capture_output=True)
if probe.returncode != 0:
test_skipped('bash process substitution is not supported on this system')
base = Path(SCRATCHDIR / 'rsync-batch-fifo')
src = base / 'src'
dest = base / 'dest'
batch_file = base / 'update.batch'
makepath(src, dest)
# 1. Create dummy data
(src / 'payload.txt').write_text('batch payload data\n')
# 2. Generate a valid batch file so `cat` actually has a real file to read.
# Note: This operation also copies the file to `dest` as a side effect.
subprocess.run([*rsync_argv('-a', f'--write-batch={batch_file}'), f'{src}/', f'{dest}/'], check=True)
# must wipe and recreate the destination directory so the test can
# properly prove that --read-batch recreates the files from scratch.
rmtree(dest)
makepath(dest)
# 3. Now we can test reading it via bash process substitution
rsync_base_cmd = shlex.join(rsync_argv('-a'))
batch_path = shlex.quote(str(batch_file))
dest_path = shlex.quote(str(dest) + '/')
# Construct the bash command: rsync -a --read-batch=<(cat /path/to/batch) /dest/
bash_script = f"{rsync_base_cmd} --read-batch=<(cat {batch_path}) {dest_path}"
try:
proc_read = subprocess.run(
[bash, '-c', bash_script],
capture_output=True,
text=True,
timeout=10,
)
except subprocess.TimeoutExpired:
rmtree(base)
test_fail('process substitution batch test timed out')
ctx = f'rc={proc_read.returncode}, stderr={proc_read.stderr.strip()!r}'
# Evaluate result against the strict S_ISREG check bug
if proc_read.returncode != 0:
rmtree(base)
test_fail(f'rsync crashed reading batch file from pipe ({ctx})')
if not (dest / 'payload.txt').is_file():
rmtree(base)
test_fail(f'rsync exited successfully but payload is missing in target ({ctx})')
rmtree(base)
print('rsync successfully parsed batch stream via process substitution pseudo-path')
raise SystemExit(0)
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env python3
"""A confined fd pin must remain usable inside a Linux user namespace."""
import os
import shlex
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from rsyncfns import makepath, rmtree, rsync_argv, test_fail, test_skipped
if not sys.platform.startswith('linux'):
test_skipped('rrsync-userns-procfs is Linux-specific')
if not os.environ.get('RSYNC_USERNS_PROCFS'):
unshare = shutil.which('unshare')
if unshare is None:
test_skipped('unshare is unavailable')
env = os.environ.copy()
env['RSYNC_USERNS_PROCFS'] = '1'
launch_dir = Path(tempfile.mkdtemp(prefix='rsync-userns-launch-'))
launch_dir.chmod(0o755)
testdir = Path(__file__).resolve().parent
child_test = launch_dir / Path(__file__).name
for source in (Path(__file__), testdir / 'rsyncfns.py',
testdir / 'exitcodes.py'):
shutil.copy2(source, launch_dir / source.name)
rsync_cmd = shlex.split(env['RSYNC'])
for i, arg in enumerate(rsync_cmd):
if Path(arg).name in ('rsync', 'rsync.exe') and Path(arg).is_file():
staged_rsync = launch_dir / Path(arg).name
shutil.copy2(arg, staged_rsync)
staged_rsync.chmod(0o755)
rsync_cmd[i] = str(staged_rsync)
break
else:
rmtree(launch_dir)
test_fail(f'cannot locate the rsync executable in {env["RSYNC"]!r}')
env['RSYNC'] = shlex.join(rsync_cmd)
launcher = []
if os.geteuid() == 0:
setpriv = shutil.which('setpriv')
if setpriv is None:
test_skipped('setpriv is unavailable for the root-run testsuite')
launcher = [setpriv, '--reuid=65534', '--regid=65534', '--clear-groups']
unshare_argv = [unshare, '--user', '--map-root-user', '--mount', '--pid',
'--fork', '--mount-proc']
probe = subprocess.run(
launcher + unshare_argv + ['true'],
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
if probe.returncode != 0:
rmtree(launch_dir)
print(f'user namespaces unavailable (rc={probe.returncode})')
raise SystemExit(0)
try:
proc = subprocess.run(
launcher + unshare_argv
+ [sys.executable, str(child_test)],
env=env,
timeout=30,
)
except subprocess.TimeoutExpired:
test_fail('user-namespace regression test timed out')
finally:
rmtree(launch_dir)
if proc.returncode != 0:
test_fail(f'user-namespace regression test failed (rc={proc.returncode})')
print('rrsync fd pin works inside a user namespace')
raise SystemExit(0)
proc_uid = os.lstat('/proc/self').st_uid
if proc_uid in (0, os.geteuid()):
test_skipped('/proc/self does not expose an overflow uid in this namespace')
base = Path(tempfile.mkdtemp(prefix='rsync-userns-procfs-'))
src = base / 'src'
dest = base / 'dest'
outside = base / 'outside'
makepath(src, dest, outside)
(src / 'file').write_text('content\n')
fd_roots = ['/proc/self/fd']
if Path('/dev/fd').exists():
fd_roots.append('/dev/fd')
dest_fd = os.open(dest, os.O_RDONLY | os.O_DIRECTORY)
try:
for index, fd_root in enumerate(fd_roots):
log_file = dest / f'rsync-{index}.log'
proc = subprocess.run(
rsync_argv('-a', f'--confine-root={dest}',
f'--log-file={fd_root}/{dest_fd}/{log_file.name}',
str(src) + '/', str(dest) + '/'),
pass_fds=(dest_fd,),
capture_output=True,
text=True,
)
ctx = (f'fd_root={fd_root!r}, rc={proc.returncode}, '
f'stderr={proc.stderr.strip()[:300]!r}')
if proc.returncode != 0:
test_fail(f'confined transfer through an fd pin failed ({ctx})')
if not log_file.is_file():
test_fail(f'confined log path through an fd pin was rejected ({ctx})')
if (dest / 'file').read_text() != 'content\n':
test_fail(f'confined transfer did not deliver the file ({ctx})')
finally:
os.close(dest_fd)
outside_list = outside / 'files-from'
outside_list.write_text('file\n')
for fd_root in fd_roots:
outside_fd = os.open(outside_list, os.O_RDONLY)
try:
proc = subprocess.run(
rsync_argv('-a', f'--confine-root={dest}',
f'--files-from={fd_root}/{outside_fd}',
str(src) + '/', str(dest) + '/'),
pass_fds=(outside_fd,),
capture_output=True,
text=True,
)
finally:
os.close(outside_fd)
if proc.returncode == 0 or 'failed to open files-from file' not in proc.stderr:
test_fail(f'outside {fd_root} pin was not observably refused: '
f'rc={proc.returncode}, stderr={proc.stderr!r}')
rmtree(base)
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python3
# rsync-ssl only recognized --type=SSL_TYPE as the FIRST argument, so
# "rsync-ssl --dry-run --type=stunnel host::mod" passed the option through to
# the underlying rsync, which rejected it with "--type=stunnel: unknown option".
# Fix: scan the whole argument list for --type=..., export RSYNC_SSL_TYPE, and
# drop the option before handing the remaining args to rsync.
# A `--` arg stops the wrapper-option scan: `--` and everything after it are
# passed through to rsync verbatim, so an operand such as `--type=stunnel`
# that was protected from option parsing is not consumed by the wrapper.
#
# A fake rsync in PATH records the args it receives and the RSYNC_SSL_TYPE it
# observes; rsync-ssl is run in its normal (non-HELPER) mode with --type= in
# various positions. The recorded args must contain every other option but
# never a --type= token, RSYNC_SSL_TYPE must match what the wrapper consumed,
# and rsync-ssl must exit successfully.
import os
import subprocess
from rsyncfns import SCRATCHDIR, SRCDIR, rmtree, test_fail
base = SCRATCHDIR / 'rsync-ssl-type-opt'
rmtree(base)
base.mkdir(parents=True)
args_capture = base / 'rsync_args'
type_capture = base / 'rsync_ssl_type'
fakebin = base / 'bin'
fakebin.mkdir(parents=True)
fake_rsync = fakebin / 'rsync'
fake_rsync.write_text(
f"#!/usr/bin/env bash\n"
f"printf '%s\\n' \"$@\" > {args_capture}\n"
f"printf '%s\\n' \"${{RSYNC_SSL_TYPE-UNSET}}\" > {type_capture}\n"
f"exit 0\n")
fake_rsync.chmod(0o755)
env = {**os.environ, 'PATH': str(fakebin) + os.pathsep + os.environ.get('PATH', '')}
for v in ('RSYNC_SSL_TYPE', 'RSYNC_SSL_OPENSSL', 'RSYNC_SSL_STUNNEL'):
env.pop(v, None)
def run(args, expect_type):
for capture in (args_capture, type_capture):
if capture.exists():
capture.unlink()
proc = subprocess.run(['bash', str(SRCDIR / 'rsync-ssl')] + args, env=env,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
if proc.returncode != 0:
test_fail(f"rsync-ssl exited {proc.returncode} for args {args!r}:\n{proc.stdout}")
got_args = args_capture.read_text().splitlines() if args_capture.exists() else []
got_type = type_capture.read_text().strip() if type_capture.exists() else 'UNSET'
if got_type != expect_type:
test_fail(f"RSYNC_SSL_TYPE is {got_type!r}, expected {expect_type!r} "
f"for args {args!r}:\n{proc.stdout}")
return got_args
# --- The reported failure: --type= in the middle of the rsync args.
got = run(['--dry-run', '--type=stunnel', 'host::mod'], 'stunnel')
if any(a.startswith('--type=') for a in got):
test_fail(f"--type= was passed through to rsync instead of being consumed:\n{got}")
for want in ('--dry-run', 'host::mod'):
if want not in got:
test_fail(f"missing rsync arg {want!r} after --type= handling:\n{got}")
if not any(a.startswith('--rsh=') for a in got):
test_fail(f"missing the --rsh= helper option:\n{got}")
# --- First, last, and no --type= keep working.
for pos_args, expect_type in ((['--type=stunnel', '--dry-run', 'host::mod'], 'stunnel'),
(['-av', 'host::mod', '--type=openssl'], 'openssl'),
(['-av', 'host::mod'], 'UNSET')):
got = run(pos_args, expect_type)
if any(a.startswith('--type=') for a in got):
test_fail(f"--type= was passed through for args {pos_args!r}:\n{got}")
for want in pos_args:
if want.startswith('--type='):
continue
if want not in got:
test_fail(f"missing rsync arg {want!r} for args {pos_args!r}:\n{got}")
# --- `--` stops the wrapper-option scan: the protected operand is preserved.
rsh_arg = "--rsh='{}' --HELPER".format(SRCDIR / 'rsync-ssl')
got = run(['--', '--type=stunnel', 'host::mod'], 'UNSET')
if got != [rsh_arg, '--', '--type=stunnel', 'host::mod']:
test_fail(f"args after -- must be preserved verbatim (no --type= consumed):\n{got}")
# --- A wrapper option before `--` is still consumed; the protected one is not.
got = run(['--type=openssl', '--', '--type=stunnel', 'host::mod'], 'openssl')
if got != [rsh_arg, '--', '--type=stunnel', 'host::mod']:
test_fail(f"--type= before -- is consumed, args after -- are preserved:\n{got}")
print("rsync-ssl-type-option: --type=SSL_TYPE is consumed in any argument "
"position (until a -- stops the wrapper-option scan), exported as "
"RSYNC_SSL_TYPE, and rsync-ssl exits successfully")
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env python3
"""The receiver must traverse a searchable but unreadable destination parent.
Android exposes /sdcard through such a path, so the race-safe destination walk
must use directory descriptors that require search permission only.
"""
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from rsyncfns import SCRATCHDIR, rmtree, rsync_argv, test_fail, test_skipped
if not sys.platform.startswith('linux'):
test_skipped('search-only-destination is Linux-specific')
launcher = []
if os.geteuid() == 0:
setpriv = shutil.which('setpriv')
if setpriv is None:
test_skipped('setpriv is unavailable for the root-run testsuite')
launcher = [setpriv, '--reuid=65534', '--regid=65534', '--clear-groups']
external_base = os.geteuid() == 0
if external_base:
base = Path(tempfile.mkdtemp(prefix='rsync-search-only-'))
base.chmod(0o755)
else:
base = SCRATCHDIR / 'search-only-destination'
src = base / 'src'
parent = base / 'search-only'
dest = parent / 'dest'
rmtree(base)
src.mkdir(parents=True)
dest.mkdir(parents=True)
(src / 'probe').write_text('search-only destination\n')
if os.geteuid() == 0:
for path in (src, src / 'probe', dest):
os.chown(path, 65534, 65534)
try:
parent.chmod(0o111)
try:
probe = subprocess.run(
launcher + ['test', '-r', str(parent)],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
if probe.returncode == 0:
test_skipped('filesystem does not enforce the search-only test mode')
if probe.returncode != 1:
test_fail(f'search-only permission probe failed with exit {probe.returncode}')
proc = subprocess.run(
launcher + rsync_argv('-a', f'{src}/', f'{dest}/'),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
finally:
parent.chmod(0o755)
copied = (dest / 'probe').read_text() if (dest / 'probe').is_file() else None
finally:
if external_base:
rmtree(base)
if proc.returncode != 0:
test_fail(
'receiver could not enter a destination below a searchable, unreadable '
f'parent (exit {proc.returncode}): {proc.stderr.strip()}'
)
if copied != 'search-only destination\n':
test_fail('receiver did not copy into the search-only destination')
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""Known-name operations must not require permission to list parent dirs.
The confined resolver holds directory descriptors to prevent symlink races.
On Linux, those traversal and *at() anchor descriptors can use O_PATH: opening
a known file beneath a searchable directory, or creating one beneath a
writable/searchable directory, does not require directory read permission.
"""
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from rsyncfns import (
SCRATCHDIR, forced_protocol, rmtree, rsync_argv, test_fail, test_skipped,
)
if not sys.platform.startswith('linux'):
test_skipped('search-only held-dirfd coverage is Linux-specific')
launcher = []
if os.geteuid() == 0:
setpriv = shutil.which('setpriv')
if setpriv is None:
test_skipped('setpriv is unavailable for the root-run testsuite')
launcher = [setpriv, '--reuid=65534', '--regid=65534', '--clear-groups']
external_base = os.geteuid() == 0
base = (
Path(tempfile.mkdtemp(prefix='rsync-search-only-held-dirfd-'))
if external_base
else SCRATCHDIR / 'search-only-held-dirfd'
)
rmtree(base)
src = base / 'src'
xonly = src / 'xonly'
readable = xonly / 'readable'
nested_src = src / 'nested'
exact_dest = base / 'exact-dest'
tree_dest = base / 'tree-dest'
unreadable_dest = base / 'unreadable-dest'
write_only_dest = base / 'write-only-dest'
nested_dest = base / 'nested-dest'
nested_parent = nested_dest / 'nested'
for path in (
readable,
nested_src,
exact_dest,
tree_dest,
unreadable_dest,
write_only_dest,
nested_parent,
):
path.mkdir(parents=True, exist_ok=True)
(xonly / 'exact').write_text('known file beneath search-only parent\n')
(readable / 'nested').write_text('enumerated below search-only ancestor\n')
incoming = src / 'incoming'
incoming.write_text('created beneath write-search-only destination\n')
(nested_src / 'known').write_text(
'created beneath nested write-search-only parent\n'
)
if os.geteuid() == 0:
for root, dirs, files in os.walk(base):
os.chown(root, 65534, 65534)
for name in dirs + files:
os.chown(Path(root) / name, 65534, 65534)
def permission_probe(path, flag, expected, label):
proc = subprocess.run(
launcher + ['test', flag, str(path)],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
if proc.returncode != expected:
test_skipped(
f'filesystem does not enforce {label}: test {flag} returned '
f'{proc.returncode}, expected {expected}'
)
failures = []
try:
xonly.chmod(0o111)
write_only_dest.chmod(0o333)
nested_parent.chmod(0o333)
permission_probe(xonly, '-r', 1, 'search-only mode')
permission_probe(xonly, '-x', 0, 'search-only mode')
permission_probe(write_only_dest, '-r', 1, 'write-search-only mode')
permission_probe(write_only_dest, '-w', 0, 'write-search-only mode')
permission_probe(write_only_dest, '-x', 0, 'write-search-only mode')
permission_probe(nested_parent, '-r', 1, 'nested write-search-only mode')
permission_probe(nested_parent, '-w', 0, 'nested write-search-only mode')
permission_probe(nested_parent, '-x', 0, 'nested write-search-only mode')
# Keep received implied dirs usable on systems without a safe fchmodat2.
# The source remains mode 0111, so sender traversal coverage is unchanged.
exact = subprocess.run(
launcher + rsync_argv(
'-aR', '--chmod=Du+rw', 'xonly/exact', f'{exact_dest}/',
),
cwd=src,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
exact_path = exact_dest / 'xonly' / 'exact'
exact_content = exact_path.read_text() if exact_path.is_file() else None
if exact.returncode != 0 or exact_content != (
'known file beneath search-only parent\n'
):
failures.append(
'exact -R source beneath mode 0111 failed: '
f'rc={exact.returncode}, stderr={exact.stderr.strip()!r}, '
f'content={exact_content!r}'
)
tree = subprocess.run(
launcher + rsync_argv(
'-aR', '--chmod=Du+rw', 'xonly/readable/', f'{tree_dest}/',
),
cwd=src,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
tree_path = tree_dest / 'xonly' / 'readable' / 'nested'
tree_content = tree_path.read_text() if tree_path.is_file() else None
if tree.returncode != 0 or tree_content != (
'enumerated below search-only ancestor\n'
):
failures.append(
'readable directory beneath mode 0111 ancestor failed: '
f'rc={tree.returncode}, stderr={tree.stderr.strip()!r}, '
f'content={tree_content!r}'
)
unreadable = subprocess.run(
launcher + rsync_argv(
'-a', 'xonly/', f'{unreadable_dest}/',
),
cwd=src,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
if unreadable.returncode == 0:
failures.append(
'mode 0111 source directory was enumerable without read permission'
)
receiver = subprocess.run(
launcher + rsync_argv(
'-t', str(incoming), f'{write_only_dest}/',
),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
received = write_only_dest / 'incoming'
received_content = received.read_text() if received.is_file() else None
if receiver.returncode != 0 or received_content != (
'created beneath write-search-only destination\n'
):
failures.append(
'known-file creation beneath mode 0333 destination failed: '
f'rc={receiver.returncode}, stderr={receiver.stderr.strip()!r}, '
f'content={received_content!r}'
)
# Protocol 29 rejects this nested -R shape before the resolver is reached.
proto = forced_protocol()
if proto is None or proto >= 30:
nested_receiver = subprocess.run(
launcher + rsync_argv(
'-tR', '--no-implied-dirs', 'nested/known', f'{nested_dest}/',
),
cwd=src,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
nested_received = nested_parent / 'known'
nested_content = (
nested_received.read_text() if nested_received.is_file() else None
)
if nested_receiver.returncode != 0 or nested_content != (
'created beneath nested write-search-only parent\n'
):
failures.append(
'known-file creation beneath nested mode 0333 destination '
f'failed: rc={nested_receiver.returncode}, '
f'stderr={nested_receiver.stderr.strip()!r}, '
f'content={nested_content!r}'
)
finally:
xonly.chmod(0o755)
write_only_dest.chmod(0o755)
nested_parent.chmod(0o755)
for dest in (exact_dest, tree_dest):
copied_xonly = dest / 'xonly'
if copied_xonly.is_dir():
copied_xonly.chmod(0o755)
if external_base:
rmtree(base)
if failures:
test_fail('\n'.join(failures))
+1
View File
@@ -23,6 +23,7 @@ different tests merge cleanly.
| file | contents |
| --- | --- |
| `common.txt` | skipped on every platform that runs the oracle — mostly `require_tcp` / `require_asan` tests, which the default stdio-pipe `make check` cannot satisfy |
| `almalinux-8.txt` | AlmaLinux 8 container additions |
| `linux.txt` | Linux-only additions |
| `macos.txt` | macOS-only additions |
| `cygwin.txt` | Cygwin-only additions |
+9
View File
@@ -0,0 +1,9 @@
# Tests expected to SKIP. One name per line, '#' starts a comment; the file
# must stay sorted and duplicate-free (runtests.py enforces both). Referenced
# from a workflow as RSYNC_EXPECT_SKIPPED=@testsuite/skiplist/<file>[,@...].
# See testsuite/skiplist/README.md.
#
# AlmaLinux 8 container additions to common.txt and linux.txt.
pseudo-paths # Bash process substitution is unavailable in the AlmaLinux 8 container
read-batch-pipe
+1
View File
@@ -11,6 +11,7 @@
checksum-zero-blocklen # the pure-Python receiver needs a real TCP socket; run with --use-tcp
chroot-basis-forge-inner-module # the pure-Python sender needs a real TCP daemon; run with --use-tcp
daemon-access-idn # hosts allow/deny hostname matching needs a real TCP peer
daemon-access-ip # hosts allow/deny address matching needs a real TCP peer
daemon-argv-limit # raw malicious daemon client needs a real TCP daemon; run with --use-tcp
daemon-chroot # daemon chroot path needs the real start_daemon socket flow
+5 -2
View File
@@ -25,7 +25,6 @@ copy-xattrs-symlink-race
daemon-auth-group
daemon-chroot-munge-default
daemon-config-symlink
daemon-max-alloc-zero
daemon-module-chdir-symlink
daemon-module-private-parent
daemon-secrets-file-symlink
@@ -52,15 +51,19 @@ partial-protected-regular-retry-linux
partial-protected-regular-retry-policy # deterministic partial EACCES recovery uses dyld interposing
password-file-symlink
protected-regular
pseudo-paths
read-batch-pipe
rename-mixed-parent-transfer
rrsync-sender-leaf-flip
rrsync-sender-parent-pin
rrsync-symlink
rrsync-userns-procfs
search-only-destination
search-only-held-dirfd
sender-remove-source-root-anchor
simd-checksum
source-change-size-continues
symlink-dest-backupdir
symlink-exclude-xattr
symlink-race-dest
symlink-race-relative-dest
temp-dir-symlink-injection
+5 -1
View File
@@ -14,7 +14,6 @@ backup-crossdev-copy
chmod-temp-dir
copy-xattrs-symlink-race
daemon-auth-group
daemon-max-alloc-zero
dir-sgid
fake-super-acl-xattr
link-dest-symlink-enotsup # the ENOTSUP hard-link hook is an LD_PRELOAD, Linux-only
@@ -22,10 +21,15 @@ open-noatime
partial-protected-regular-retry-linux
preallocate
protected-regular
pseudo-paths # dynamically skips on runners lacking bash process substitution
read-batch-pipe
readonly-partial-abort-mode-regression #
rrsync-sender-leaf-flip
rrsync-sender-parent-pin
rrsync-symlink
rrsync-userns-procfs
search-only-destination
search-only-held-dirfd
sender-remove-source-root-anchor
simd-checksum
source-change-size-continues
+14 -2
View File
@@ -50,8 +50,20 @@ dest.mkdir(parents=True)
os.symlink(outside, dest / 'sub') # attacker-owned dest component
os.lchown(dest / 'sub', ATT_UID, ATT_UID)
subprocess.run(rsync_argv('-a', f'{src}/sub/', f'{dest}/sub/'),
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
proc = subprocess.run(
rsync_argv('-a', f'{src}/sub/', f'{dest}/sub/'),
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
text=True,
)
if proc.returncode == 0:
test_fail("attacker-owned destination symlink was not rejected")
if "refusing to follow a symlink owned by an untrusted user" not in proc.stderr:
test_fail(
"untrusted destination symlink failure omitted the actionable "
f"diagnostic: {proc.stderr!r}"
)
escaped = sorted(p.name for p in outside.iterdir())
if escaped:
+5
View File
@@ -156,6 +156,11 @@ static void print_info_flags(enum logcode f)
#endif
"crtimes",
#ifndef SUPPORT_IDN
"no "
#endif
"IDN",
"*Optimizations",
#ifndef USE_ROLL_SIMD
+102 -7
View File
@@ -24,6 +24,9 @@
#include "ifuncs.h"
#include "itypes.h"
#include "inums.h"
#ifdef SUPPORT_IDN
#include <idn2.h>
#endif
extern int dry_run;
extern int module_id;
@@ -915,16 +918,110 @@ void glob_expand_module(char *base1, char *arg, char ***argv_p, int *argc_p, int
/**
* Convert a string to lower case
*
* Only ASCII is folded. The hosts allow/deny list that calls this can hold
* UTF-8, and a per-byte fold via the locale's ctype would mangle it (in
* ISO-8859-1 the 0xC4 lead byte of "č" is an upper-case 'Ä').
**/
void strlower(char *s)
{
while (*s) {
if (isUpper(s))
if (!(*(unsigned char *)s & 0x80) && isUpper(s))
*s = toLower(s);
s++;
}
}
#ifdef SUPPORT_IDN
/* Does this label hold nothing but the [-a-z0-9] of an A-label? */
static int is_a_label(const char *s)
{
if (!*s)
return 0;
for ( ; *s; s++) {
if (!(*s >= 'a' && *s <= 'z') && !(*s >= '0' && *s <= '9') && *s != '-')
return 0;
}
return 1;
}
/**
* Convert the non-ASCII labels of a host name into their IDNA A-label
* (Punycode) form, putting the result in buf. Returns 1 if buf was filled in,
* or 0 to tell the caller to keep the name it has.
*
* A label that is already ASCII is copied verbatim, so an address, a mask, an
* xn-- name, and any wildmatch characters come out just as they went in. A
* converted label is only used if it comes back as a bare A-label: the IDNA
* mapping folds some non-ASCII characters onto ASCII ones (U+FF0A FULLWIDTH
* ASTERISK becomes '*'), and a hosts allow/deny entry must not pick up a
* wildcard that its author never typed. Anything else leaves the name alone,
* which fails to match instead of matching too much.
*
* Set from_locale for a name that came from the command line, which is in the
* user's locale encoding; the daemon's config file is read as UTF-8.
**/
int idn_to_ascii(const char *name, int from_locale, char *buf, size_t buflen)
{
const char *lab, *end;
size_t len = 0;
int converted = 0;
for (lab = name; ; lab = end + 1) {
char label[256], *idn;
size_t lablen, alen;
int is_ascii = 1;
for (end = lab; *end && *end != '.'; end++) {
if (*(unsigned char *)end & 0x80)
is_ascii = 0;
}
lablen = end - lab;
if (is_ascii) {
if (len + lablen + 2 > buflen)
return 0;
memcpy(buf + len, lab, lablen);
len += lablen;
} else {
/* IDN2_NFC_INPUT has libidn2 normalize the label, so a name
* typed with combining marks folds to the same A-label as
* its composed spelling. IDN2_NONTRANSITIONAL asks for the
* TR46 processing that everything else does these days. */
int flags = IDN2_NFC_INPUT | IDN2_NONTRANSITIONAL;
int rc;
if (lablen >= sizeof label)
return 0;
memcpy(label, lab, lablen);
label[lablen] = '\0';
rc = from_locale ? idn2_lookup_ul(label, &idn, flags)
: idn2_to_ascii_8z(label, &idn, flags);
if (rc != IDN2_OK)
return 0;
alen = strlen(idn);
if (!is_a_label(idn) || len + alen + 2 > buflen) {
idn2_free(idn);
return 0;
}
memcpy(buf + len, idn, alen);
len += alen;
idn2_free(idn);
converted = 1;
}
if (!*end)
break;
buf[len++] = '.';
}
buf[len] = '\0';
return converted;
}
#endif
/**
* Split a string into tokens based (usually) on whitespace & commas. If the
* string starts with a comma (after skipping any leading whitespace), then
@@ -1252,7 +1349,7 @@ int change_dir(const char *dir, int set_path_only)
* non-daemon receiver can opt back into the legacy plain chdir with
* --insecure-links. */
if (am_daemon && !am_chrooted) {
int dfd = open_no_attacker_symlinks(dir, O_RDONLY | O_DIRECTORY, 0);
int dfd = open_no_attacker_symlinks_dirfd(dir);
if (dfd < 0)
return 0;
if (fchdir(dfd) != 0) {
@@ -1283,7 +1380,7 @@ int change_dir(const char *dir, int set_path_only)
* another uid. A real dir is opened directly. This closes the
* destination chdir TOCTOU; --insecure-links keeps the plain
* chdir for an operator whose dest is a foreign-owned symlink. */
dfd = open_no_attacker_symlinks(nf, O_RDONLY | O_DIRECTORY, 0);
dfd = open_no_attacker_symlinks_dirfd(nf);
if (dfd < 0)
return 0;
if (fchdir(dfd) != 0) {
@@ -1344,8 +1441,7 @@ int change_dir(const char *dir, int set_path_only)
prefix[save_dir_len] = '\0';
basedir = prefix;
}
dfd = secure_relative_open(basedir, dir,
O_RDONLY | O_DIRECTORY, 0);
dfd = secure_relative_dirfd(basedir, dir);
if (dfd < 0) {
chdir_failed = 1;
} else {
@@ -1363,8 +1459,7 @@ int change_dir(const char *dir, int set_path_only)
* symlink not owned by uid 0 or our euid, closing the
* relative-dest chdir TOCTOU while still following the operator's
* own symlinks. --insecure-links keeps the plain chdir. */
int dfd = open_no_attacker_symlinks(curr_dir,
O_RDONLY | O_DIRECTORY, 0);
int dfd = open_no_attacker_symlinks_dirfd(curr_dir);
if (dfd < 0)
chdir_failed = 1;
else {