Compare commits

..

15 Commits

Author SHA1 Message Date
Andrew Tridgell
806dff20d9 tests: add clang scan-build static-analysis CI (informational)
Run the clang static analyzer over a check-progs build, publish the HTML report
as an artifact, and print the bug count to the run summary. INFORMATIONAL only:
it does not pass --status-bugs, so it surfaces new analyzer findings without
going red on the existing (overwhelmingly false-positive) reports.

Runs on push/PR to master and via workflow_dispatch. No cron: it is
informational and its output only changes with the code (push/PR) or the clang
version, so a daily run on an unchanged tree would add noise without value.
2026-06-08 20:54:57 +10:00
Andrew Tridgell
8f63c498e9 tests: add ASan+UBSan CI gate
Add a clang AddressSanitizer + UndefinedBehaviorSanitizer workflow that builds
rsync with -fsanitize=address,undefined -fno-sanitize-recover=undefined -DNDEBUG
and runs the full test suite over both the stdio-pipe and TCP daemon transports.

UBSAN_OPTIONS=halt_on_error=1 together with -fno-sanitize-recover=undefined makes
any undefined behaviour fatal, so this job gates: the tree must stay UBSan-clean.
The remaining findings are fixed in code (hashtable/mdfour shifts, xattrs, and
log.c's file_struct, kept aligned via rounding.h); only byteorder.h's intentional
unaligned accessors are suppressed, with no_sanitize. -DNDEBUG builds as a release
does (assert() compiled out) so ASan covers the production code paths.

Runs on push/PR to master and via workflow_dispatch, plus a weekly cron to
catch breakage from a moving ubuntu-latest/clang toolchain (push/PR already
cover every code change, so daily would just re-run an unchanged tree).
2026-06-08 20:54:57 +10:00
Andrew Tridgell
df2833b318 io: drop the dead/unnecessary read_varint UBSan guard
The cherry-picked #428 wrapped no_sanitize attributes on read_varint() and
read_varlong() in `#ifndef CAREFUL_ALIGNMENT`, but byteorder.h always
#defines CAREFUL_ALIGNMENT (to 0 or 1), so that guard is never true and the
attributes were dead code.

They are also unnecessary: both functions read the assembled value through
an aligned union member (union { char b[5]; int32 x; }), not an unaligned
cast, so UBSan's alignment check never fires there (verified: the ASan+UBSan
suite is clean without them).  Remove the whole block rather than fix the
guard.  (The byteorder.h annotations from #428, which are real and correctly
placed inside the !CAREFUL_ALIGNMENT branch, are kept.)
2026-06-08 20:54:57 +10:00
Sam James
7214372a8a Disable UBSAN for alignment-sensitive functions when !CAREFUL_ALIGNMENT
rsync sets CAREFUL_ALIGNMENT for architectures which do not support
unaligned access. Disable UBSAN for functions which may use unaligned
accesses when CAREFUL_ALIGNMENT is set.

Bug: https://github.com/WayneD/rsync/issues/427
Signed-off-by: Sam James <sam@gentoo.org>
(cherry picked from commit 11c1e934e8)
2026-06-08 20:54:57 +10:00
Andrew Tridgell
497357800a log: align the file_struct built in log_delete()
log_delete() builds a struct file_struct inside a char buffer offset by the
(EXTRA_LEN-granular) extra data.  The EXTRA_ROUNDING block that rounds that
offset up to the struct's alignment (exactly as flist.c does for its pool
allocations) was dead code here: log.c never included rounding.h, so
EXTRA_ROUNDING was undefined and the rounding never ran, leaving the
file_struct pointer potentially under-aligned.  That trips UBSan's alignment
check and would fault on strict-alignment arches.

Include rounding.h (and add the Makefile dependency) so the existing rounding
actually applies -- fixing the alignment at the source rather than suppressing
the sanitizer.
2026-06-08 20:54:57 +10:00
Andrew Tridgell
fa084c4ae3 xattrs: fix UBSan-detected undefined behavior
Three pre-existing issues UBSan flags during the xattr tests:

  * xattr_lookup_hash(): the summed hashlittle2() values overflow the
    signed int64 accumulator (UB).  Accumulate in uint64_t and convert back
    at return -- the key is only used for hash-table equality, so the value
    is unchanged.
  * rsync_xal_get(): for an empty list (count == 0) the loop init
    `rxa += count-1` forms `items - 1` on a NULL `items` (UB).  Guard with
    `if (count)`.
  * rsync_xal_store(): `memcpy(dst, xalp->items, 0)` passes a NULL source for
    an empty list (UB).  Guard with `if (xalp->count)`.
2026-06-08 20:54:57 +10:00
Andrew Tridgell
4148419736 hashtable, mdfour: avoid signed left-shift overflow
UBSan flags two spots that shift a value into the top bits of a word via a
signed operand:

  * lib/mdfour.c copy64(): `in[i] << 24` promotes the uchar to int, so a
    byte >= 128 overflows int (UB).  Cast each byte to uint32.
  * hashtable.c NON_ZERO_64(): `(int64)(x) << 32` overflows int64 whenever
    x's high bit is set.  Shift as uint64_t (covers all four call sites).

Behavior-preserving -- only the intermediate type changes; the resulting
bit pattern is identical.
2026-06-08 20:54:57 +10:00
Andrew Tridgell
66712a90b3 rsync-web: updates for the 3.4.4 release 2026-06-08 14:45:03 +10:00
Andrew Tridgell
b780749ffb release.py: accept a git worktree in require_top_of_checkout()
In a git worktree .git is a file (a gitdir pointer), not a directory,
so os.path.isdir('.git') wrongly aborted with "no .git dir" when the
release was run from a worktree. Use os.path.exists() so it works from
both a normal checkout and a linked worktree.
2026-06-08 14:44:37 +10:00
Andrew Tridgell
d25c5e4b11 ci: move the daily scheduled jobs to weekly
Every platform build (the BSD/Solaris/macOS/cygwin/almalinux/ubuntu jobs),
coverage, the version-mix job and the android static build ran on a daily cron
*in addition to* push and pull_request to master. Since push/PR already cover
every code change, the cron only adds drift coverage -- catching breakage from a
moving runner image or toolchain that no commit triggers. Those images do not
change daily, so a daily run mostly re-tests an unchanged tree.

Move them all to a weekly cron (Mondays, keeping each job's existing time) to
keep that drift coverage at roughly a seventh of the Actions spend and log
noise. fleettest was already weekly. Per-change CI on push/PR is unchanged, and
workflow_dispatch still allows an on-demand run.
2026-06-08 10:25:38 +10:00
Andrew Tridgell
1ddfe17d65 fleettest: --cleanup also kills stray flippers/daemons and root-owned dirs
A run killed without a parent-death backstop can strand a TOCTOU path-flipper
(a busy `python -c` rename loop that pins a CPU) and an orphaned test rsyncd
(--no-detach --address=127.0.0.1) that squats its fixed port -- the wedge the
claim_ports() bind-probe now reports and points at --cleanup. Sweep both, best
effort, before removing the run dirs.

Each sweep counts the pattern, kills it (with a `sudo -n` retry for a process a
root-running test left), then re-counts after a settle: KILLED reports what
actually died, and a process that survives (pkill blocked, no passwordless sudo,
missing/limited pkill) is reported as SURVIVED and fails the run instead of
falsely claiming success.

Run-dir removal falls back to `sudo -n rm` so a dir whose contents a root test
owns is removed instead of failing with "Permission denied" (the failure mode
seen on the ubuntu/mac targets); only a dir that survives even sudo is failed.

The kill patterns use the pgrep self-exclusion trick ('r[e]name', 'det[a]ch')
so they match a real process's "rename"/"detach" but not the literal pattern in
the cleanup shell's own argv -- run_on() passes the whole script as the remote
argv, so without it --cleanup would signal itself. The patterns are host-global
(not scoped to one run), so --cleanup is documented to run between runs, not
during one.
2026-06-08 09:41:59 +10:00
Andrew Tridgell
6e6b4135ab testsuite: verify a claimed test port is actually bindable
claim_ports() takes a POSIX byte-range lock per port, which serializes
concurrent live test runs. But the kernel drops that lock the instant the
holding process dies, even if the run left an orphaned rsync --daemon still
bound to the port -- which happens when a run is SIGKILLed on a platform with
no parent-death backstop (rsyncfns only arms PR_SET_PDEATHSIG, Linux-only, so
the BSDs/Solaris/macOS can strand a daemon). A later run then wins the freed
lock while the socket is still squatted and dies with a cryptic "bind() failed:
Address already in use" / "did not see server greeting".

After taking each lock, actually bind the port (SO_REUSEADDR, so a port merely
in TIME_WAIT is not a false positive; only a live squatter fails) and close it
immediately. On failure stop with an actionable message naming the port and the
likely orphaned daemon. Closes the gap that masked the OpenBSD daemon-auth wedge.
2026-06-08 09:41:59 +10:00
Andrew Tridgell
c2b8e4532b fleettest: require runtests.py in --testsuite-repo, not the build tree
When --testsuite-repo provides the suite, the build tree (--repo) need not
carry runtests.py -- it may be an older release whose shell testsuite predates
the Python runtests.py (e.g. a 3.4.1 backport branch built and tested with the
current suite).  Check runtests.py in TESTSUITE_REPO and only require the build
tree to be rsync source (rsync.h).
2026-06-08 06:29:49 +10:00
Andrew Tridgell
7b66c0665f fleettest: add --testsuite-repo to run another tree's suite against this build
--repo couples the built source and the test suite that exercises it.
--testsuite-repo PATH overlays runtests.py + testsuite/ from a second tree onto
the staged build tree, and sources the expected-skip workflows from it, so one
can build an older release (e.g. a 3.4.x stable branch) and run the current
comprehensive suite against that binary. Defaults to --repo, so the existing
single-tree behaviour is unchanged.
2026-06-08 06:29:49 +10:00
Andrew Tridgell
49f8dd1ca4 runtests: stop discovering obsolete *.test shell tests
The shell testsuite was removed in 1f689ec0 (rewritten in Python); only
*_test.py remain, yet collect_tests still globbed *.test and _testbase mapped
foo.test and foo_test.py to the same canonical name. Harmless on a master tree
(no .test files), but when an older tree's *.test files are present -- e.g.
fleettest --testsuite-repo building a 3.4.x release whose shell suite still
exists -- both glob to the same test name and scratch dir and race under -j,
producing spurious failures. Drop .test discovery entirely.
2026-06-08 06:29:49 +10:00
28 changed files with 429 additions and 303 deletions

View File

@@ -18,7 +18,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/almalinux-8-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -21,7 +21,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/android-static-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
workflow_dispatch:
env:

72
.github/workflows/asan-build.yml vendored Normal file
View File

@@ -0,0 +1,72 @@
name: rsync ASan+UBSan (clang)
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/asan-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/asan-build.yml'
schedule:
# Weekly (Mon 09:42 UTC): catch breakage from a moving ubuntu-latest/clang
# toolchain (a new clang can add a UBSan check, or change ASan behaviour)
# that no code push would otherwise trigger. Push/PR already gate every
# code change, so daily would just re-run an unchanged tree.
- cron: '42 9 * * 1'
workflow_dispatch:
jobs:
asan:
runs-on: ubuntu-latest
name: rsync ASan+UBSan (clang)
env:
# rsync intentionally leaks small allocations at process exit, so leak
# detection would be all noise; chase only memory-safety errors.
ASAN_OPTIONS: detect_leaks=0:abort_on_error=1
# UBSan is a gate: -fno-sanitize-recover=undefined (below) aborts on the
# first finding and halt_on_error=1 makes that fatal, so any undefined
# behaviour fails the run. This needs the tree to be UBSan-clean: the
# remaining findings are fixed in code (hashtable/mdfour shifts, xattrs,
# and log.c's file_struct, kept aligned via rounding.h); only byteorder.h's
# intentional unaligned accessors are suppressed, with no_sanitize.
UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y clang acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev openssl
echo "/usr/local/bin" >>"$GITHUB_PATH"
- name: configure
# -DNDEBUG builds as a shipped release does (assert() compiled out), so
# AddressSanitizer catches the over-reads/over-writes that an "assert()
# instead of a real bounds check" bug would cause in a production build.
# UBSan rides along on the same build; -fno-sanitize-recover=undefined
# makes any undefined behaviour abort (and thus fail the run) instead of
# merely printing it.
run: |
CC=clang \
CFLAGS="-fsanitize=address,undefined -fno-sanitize-recover=undefined -fno-omit-frame-pointer -g -O1 -DNDEBUG" \
LDFLAGS="-fsanitize=address,undefined" \
./configure --with-rrsync --disable-md2man
- name: make
# check-progs builds rsync plus the test helper programs (tls, trimslash,
# t_unsafe, ...) that runtests.py requires; plain "make" builds only rsync
# and runtests aborts on the missing helpers.
run: make check-progs
- name: info
run: ./rsync --version
- name: check (stdio-pipe transport)
# ASan+UBSan-instrumented coverage of the transfer, daemon, sender,
# receiver and metadata paths over the default stdio-pipe transport.
run: ./runtests.py --rsync-bin="$PWD/rsync" -j8
- name: check (TCP daemon transport)
# --use-tcp also exercises the loopback rsyncd listener and the client's
# TCP connection path.
run: ./runtests.py --rsync-bin="$PWD/rsync" --use-tcp -j8

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/coverage.yml'
schedule:
- cron: '42 9 * * *'
- cron: '42 9 * * 1'
workflow_dispatch:
jobs:

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/cygwin-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/freebsd-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/macos-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/netbsd-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/openbsd-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

51
.github/workflows/scan-build.yml vendored Normal file
View File

@@ -0,0 +1,51 @@
name: rsync scan-build (clang analyzer)
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/scan-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/scan-build.yml'
workflow_dispatch:
jobs:
scan-build:
runs-on: ubuntu-latest
name: rsync scan-build (clang analyzer)
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
run: |
sudo apt-get update
sudo apt-get install -y clang clang-tools acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev openssl
- name: configure (under scan-build)
# Run configure under scan-build so its analyzer compiler-wrapper is baked
# into the Makefile's $(CC); --disable-md2man avoids the doc toolchain.
run: scan-build ./configure --with-rrsync --disable-md2man
- name: scan-build (informational)
# Static analysis only -- INFORMATIONAL, not a gate. rsync currently has
# a fair number of reports that are overwhelmingly known false positives
# (e.g. unix.Chroot "no chdir after chroot", core.NonNullParamChecker
# against functions that can't actually receive NULL). We publish the
# HTML report as an artifact and print the bug count to the run summary,
# but do NOT pass --status-bugs, so this surfaces new analyzer findings
# without going red on arrival. check-progs builds rsync + the test
# helpers without needing the man-page toolchain.
run: |
scan-build -o "$PWD/scan-report" make check-progs -j"$(nproc)" 2>&1 | tee scan-build.out
echo '## scan-build summary' >>"$GITHUB_STEP_SUMMARY"
grep -E 'scan-build: .* bugs? found|scan-build: No bugs found' scan-build.out >>"$GITHUB_STEP_SUMMARY" || true
- name: upload report
if: always()
uses: actions/upload-artifact@v4
with:
name: scan-build-report
path: scan-report
if-no-files-found: ignore

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/solaris-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -16,7 +16,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-22.04-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -12,7 +12,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-build.yml'
schedule:
- cron: '42 8 * * *'
- cron: '42 8 * * 1'
jobs:
test:

View File

@@ -33,7 +33,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-version-mix.yml'
schedule:
- cron: '52 8 * * *'
- cron: '52 8 * * 1'
jobs:
version-mix:

View File

@@ -139,6 +139,7 @@ usage.o: version.h latest-year.h help-rsync.h help-rsyncd.h git-version.h defaul
loadparm.o: default-dont-compress.h daemon-parm.h
flist.o: rounding.h
log.o: rounding.h
default-cvsignore.h default-dont-compress.h: rsync.1.md define-from-md.awk
$(AWK) -f $(srcdir)/define-from-md.awk -v hfile=$@ $(srcdir)/rsync.1.md

View File

@@ -68,10 +68,26 @@ SIVAL64(char *buf, int pos, int64 val)
#else /* !CAREFUL_ALIGNMENT */
/* We don't want false positives about alignment from UBSAN, see:
https://github.com/WayneD/rsync/issues/427#issuecomment-1375132291
*/
/* From https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html */
#ifndef GCC_VERSION
#define GCC_VERSION (__GNUC__ * 10000 \
+ __GNUC_MINOR__ * 100 \
+ __GNUC_PATCHLEVEL__)
#endif
/* This handles things for architectures like the 386 that can handle alignment errors.
* WARNING: This section is dependent on the length of an int32 (and thus a uint32)
* being correct (4 bytes)! Set CAREFUL_ALIGNMENT if it is not. */
#ifdef __clang__
__attribute__((no_sanitize("undefined")))
#elif GCC_VERSION >= 409
__attribute__((no_sanitize_undefined))
#endif
static inline uint32
IVALu(const uchar *buf, int pos)
{
@@ -83,6 +99,11 @@ IVALu(const uchar *buf, int pos)
return *u.num;
}
#ifdef __clang__
__attribute__((no_sanitize("undefined")))
#elif GCC_VERSION >= 409
__attribute__((no_sanitize_undefined))
#endif
static inline void
SIVALu(uchar *buf, int pos, uint32 val)
{
@@ -94,6 +115,11 @@ SIVALu(uchar *buf, int pos, uint32 val)
*u.num = val;
}
#ifdef __clang__
__attribute__((no_sanitize("undefined")))
#elif GCC_VERSION >= 409
__attribute__((no_sanitize_undefined))
#endif
static inline int64
IVAL64(const char *buf, int pos)
{
@@ -105,6 +131,11 @@ IVAL64(const char *buf, int pos)
return *u.num;
}
#ifdef __clang__
__attribute__((no_sanitize("undefined")))
#elif GCC_VERSION >= 409
__attribute__((no_sanitize_undefined))
#endif
static inline void
SIVAL64(char *buf, int pos, int64 val)
{

View File

@@ -351,7 +351,7 @@ void *hashtable_find(struct hashtable *tbl, int64 key, void *data_when_new)
*/
#define NON_ZERO_32(x) ((x) ? (x) : (uint32_t)1)
#define NON_ZERO_64(x, y) ((x) || (y) ? (y) | (int64)(x) << 32 | (y) : (int64)1)
#define NON_ZERO_64(x, y) ((x) || (y) ? (y) | (uint64_t)(x) << 32 | (y) : (int64)1)
uint32_t hashlittle(const void *key, size_t length)
{

View File

@@ -89,8 +89,8 @@ static void copy64(uint32 *M, const uchar *in)
int i;
for (i = 0; i < MD4_DIGEST_LEN; i++) {
M[i] = (in[i*4+3] << 24) | (in[i*4+2] << 16)
| (in[i*4+1] << 8) | (in[i*4+0] << 0);
M[i] = ((uint32)in[i*4+3] << 24) | ((uint32)in[i*4+2] << 16)
| ((uint32)in[i*4+1] << 8) | ((uint32)in[i*4+0] << 0);
}
}

1
log.c
View File

@@ -22,6 +22,7 @@
#include "rsync.h"
#include "itypes.h"
#include "inums.h"
#include "rounding.h" /* EXTRA_ROUNDING, so log_delete() aligns its file_struct */
extern int dry_run;
extern int am_daemon;

View File

@@ -104,8 +104,8 @@ def require_samba_host():
def require_top_of_checkout():
if not os.path.isfile('packaging/release.py'):
die("Run this script from the top of your rsync checkout.")
if not os.path.isdir('.git'):
die("There is no .git dir in the current directory.")
if not os.path.exists('.git'):
die("There is no .git in the current directory (run from the top of a git checkout or worktree).")
def replace_or_die(regex, repl, txt, die_msg):
@@ -636,6 +636,8 @@ If you have a 'samba' remote configured (git.samba.org:/data/git/rsync.git):
Then upload the tarball + .asc to the GitHub release for {v_ver},
and announce on rsync-announce@, rsync@, and Discord.
NOTE! Also update the PPAs if needed
""")

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# This script uses openssl, gnutls, socat, or stunnel to secure an rsync daemon connection.
# This script uses openssl, gnutls, or stunnel to secure an rsync daemon connection.
# By default this script takes rsync args and hands them off to the actual
# rsync command with an --rsh option that makes it open an SSL connection to an
@@ -31,16 +31,13 @@ function rsync_ssl_run {
function rsync_ssl_helper {
if [[ -z "$RSYNC_SSL_TYPE" ]]; then
found=$(path_search openssl socat stunnel4 stunnel) || exit 1
found=`path_search openssl stunnel4 stunnel` || exit 1
if [[ "$found" == */openssl ]]; then
RSYNC_SSL_TYPE=openssl
RSYNC_SSL_OPENSSL="$found"
elif [[ "$found" == */gnutls-cli ]]; then
RSYNC_SSL_TYPE=gnutls
RSYNC_SSL_GNUTLS="$found"
elif [[ "$found" == */socat ]]; then
RSYNC_SSL_TYPE=socat
RSYNC_SSL_SOCAT="$found"
else
RSYNC_SSL_TYPE=stunnel
RSYNC_SSL_STUNNEL="$found"
@@ -50,25 +47,19 @@ function rsync_ssl_helper {
case "$RSYNC_SSL_TYPE" in
openssl)
if [[ -z "$RSYNC_SSL_OPENSSL" ]]; then
RSYNC_SSL_OPENSSL=$(path_search openssl) || exit 1
RSYNC_SSL_OPENSSL=`path_search openssl` || exit 1
fi
optsep=' '
;;
gnutls)
if [[ -z "$RSYNC_SSL_GNUTLS" ]]; then
RSYNC_SSL_GNUTLS=$(path_search gnutls-cli) || exit 1
fi
optsep=' '
;;
socat)
if [[ -z "$RSYNC_SSL_SOCAT" ]]; then
RSYNC_SSL_SOCAT=$(path_search socat) || exit 1
RSYNC_SSL_GNUTLS=`path_search gnutls-cli` || exit 1
fi
optsep=' '
;;
stunnel)
if [[ -z "$RSYNC_SSL_STUNNEL" ]]; then
RSYNC_SSL_STUNNEL=$(path_search stunnel4 stunnel) || exit 1
RSYNC_SSL_STUNNEL=`path_search stunnel4 stunnel` || exit 1
fi
optsep=' = '
;;
@@ -81,21 +72,17 @@ function rsync_ssl_helper {
if [[ -z "$RSYNC_SSL_CERT" ]]; then
certopt=""
gnutls_cert_opt=""
socat_cert_opt=""
else
certopt="-cert$optsep$RSYNC_SSL_CERT"
gnutls_cert_opt="--x509certfile=$RSYNC_SSL_CERT"
socat_cert_opt=",cert=$RSYNC_SSL_CERT"
fi
if [[ -z "$RSYNC_SSL_KEY" ]]; then
keyopt=""
gnutls_key_opt=""
socat_key_opt=""
else
keyopt="-key$optsep$RSYNC_SSL_KEY"
gnutls_key_opt="--x509keyfile=$RSYNC_SSL_KEY"
socat_key_opt=",key=$RSYNC_SSL_KEY"
fi
if [[ -z ${RSYNC_SSL_CA_CERT+x} ]]; then
@@ -104,8 +91,6 @@ function rsync_ssl_helper {
caopt="-verify_return_error -verify 4"
# gnutls:
gnutls_opts=""
# socat:
socat_opts="verify=1"
# stunnel:
# Since there is no way of using the default CA certificate collection,
# we cannot do any verification. Thus, stunnel should really only be
@@ -118,8 +103,6 @@ function rsync_ssl_helper {
caopt="-verify 1"
# gnutls:
gnutls_opts="--insecure"
# socat:
socat_opts="verify=0"
# stunnel:
cafile=""
verify="verifyChain = no"
@@ -129,8 +112,6 @@ function rsync_ssl_helper {
caopt="-CAfile $RSYNC_SSL_CA_CERT -verify_return_error -verify 4"
# gnutls:
gnutls_opts="--x509cafile=$RSYNC_SSL_CA_CERT"
# socat:
socat_opts="cafile=$RSYNC_SSL_CA_CERT,verify=1"
# stunnel:
cafile="CAfile = $RSYNC_SSL_CA_CERT"
verify="verifyChain = yes"
@@ -155,18 +136,10 @@ function rsync_ssl_helper {
exit 1
fi
if [[ "$hostname" =~ ^[0-9.]+$ || "$hostname" == *:* ]]; then
socat_sni_opt=",no-sni=1"
else
socat_sni_opt=",snihost=$hostname"
fi
if [[ $RSYNC_SSL_TYPE == openssl ]]; then
exec $RSYNC_SSL_OPENSSL s_client $caopt $certopt $keyopt -quiet -verify_quiet -servername $hostname -verify_hostname $hostname -connect $hostname:$port
elif [[ $RSYNC_SSL_TYPE == gnutls ]]; then
exec $RSYNC_SSL_GNUTLS --logfile=/dev/null $gnutls_cert_opt $gnutls_key_opt $gnutls_opts $hostname:$port
elif [[ $RSYNC_SSL_TYPE == socat ]]; then
exec $RSYNC_SSL_SOCAT - "OPENSSL:$hostname:$port,commonname=$hostname$socat_sni_opt,$socat_opts$socat_cert_opt$socat_key_opt"
else
# devzero@web.de came up with this no-tmpfile calling syntax:
exec $RSYNC_SSL_STUNNEL -fd 10 11<&0 <<EOF 10<&0 0<&11 11<&-
@@ -204,7 +177,7 @@ function path_search {
if [[ "$#" == 0 ]]; then
echo "Usage: rsync-ssl [--type=SSL_TYPE] RSYNC_ARG [...]" 1>&2
echo "The SSL_TYPE can be openssl, socat, or stunnel"
echo "The SSL_TYPE can be openssl or stunnel"
exit 1
fi

View File

@@ -27,10 +27,10 @@ rsync version to be at least 3.2.0.
If the **first** arg is a `--type=SSL_TYPE` option, the script will only use
that particular program to open an ssl connection instead of trying to find an
openssl, socat, or stunnel executable via a simple heuristic (assuming that
the `RSYNC_SSL_TYPE` environment variable is not set as well -- see below).
This option must specify one of `openssl`, `socat`, or `stunnel`. The equal
sign is required for this particular option.
openssl or stunnel executable via a simple heuristic (assuming that the
`RSYNC_SSL_TYPE` environment variable is not set as well -- see below). This
option must specify one of `openssl` or `stunnel`. The equal sign is
required for this particular option.
All the other options are passed through to the rsync command, so consult the
**rsync**(1) manpage for more information on how it works.
@@ -42,8 +42,8 @@ The ssl helper scripts are affected by the following environment variables:
0. `RSYNC_SSL_TYPE`
Specifies the program type that should be used to open the ssl connection.
It must be one of `openssl`, `socat`, or `stunnel`. The `--type=SSL_TYPE`
option overrides this, when specified.
It must be one of `openssl` or `stunnel`. The `--type=SSL_TYPE` option
overrides this, when specified.
0. `RSYNC_SSL_PORT`
@@ -78,11 +78,6 @@ The ssl helper scripts are affected by the following environment variables:
Specifies the gnutls-cli executable to run when the connection type is set
to gnutls. If unspecified, the $PATH is searched for "gnutls-cli".
0. `RSYNC_SSL_SOCAT`
Specifies the socat executable to run when the connection type is set to
socat. If unspecified, the $PATH is searched for "socat".
0. `RSYNC_SSL_STUNNEL`
Specifies the stunnel executable to run when the connection type is set to
@@ -95,8 +90,6 @@ The ssl helper scripts are affected by the following environment variables:
> rsync-ssl --type=openssl -aiv example.com::mod/ dest
> rsync-ssl --type=socat -aiv example.com::mod/ dest
> rsync-ssl -aiv --port 9874 example.com::mod/ dest
> rsync-ssl -aiv rsync://example.com:9874/mod/ dest
@@ -118,10 +111,6 @@ connection against the CA certificate collection, so it only encrypts the
connection without any cert validation unless you have specified the
certificate environment options.
The `openssl` type uses `openssl s_client`, which is retained for
compatibility. If your OpenSSL version's `s_client` has trouble handling
rsync traffic, try `--type=socat` or `--type=stunnel`.
This script also supports a `--type=gnutls` option, but at the time of this
release the gnutls-cli command was dropping output, making it unusable. If
that bug has been fixed in your version, feel free to put gnutls into an

View File

@@ -26,6 +26,58 @@ License</A> and is currently being maintained by
<img src="badge.svg">
</a></div>
<h3>Rsync version 3.4.4 released</h3>
<i class=date>June 8th, 2026</i>
<p>Rsync version 3.4.4 has been released. This is a regression fix
release for the issues that have been reported with the 3.4.3
security release. Many thanks to everyone who reported the issues
(see <a href="https://download.samba.org/pub/rsync/NEWS#3.4.4">NEWS.md</a>
for credits).<p>
The 3.4.3 release had so many issues for two main reasons:
<ul>
<li>the 3.4 testsuite did not have broad enough coverage to
catch the regressions notices by users
<li>the nature of a security release prevents wide beta testing,
resulting in not enough manual testing in disparate environments
</ul>
To fix this for future releases we have greatly expanded the test
suite for 3.5 (currently in master) and grown the development
team, especially with more people with security expertise.
Thanks for your patience!
<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.4.4">3.4.4 NEWS</a> for a detailed changelog.
The latest manpages are also available for:<ul>
<li><a href="https://download.samba.org/pub/rsync/rsync.1"><b>rsync</b>(1)</a>
<li><a href="https://download.samba.org/pub/rsync/rsync-ssl.1"><b>rsync-ssl</b>(1)</a>
<li><a href="https://download.samba.org/pub/rsync/rsyncd.conf.5"><b>rsyncd.conf</b>(5)</a>
<li><a href="https://download.samba.org/pub/rsync/rrsync.1"><b>rrsync</b>(1)</a>
</ul>
<p>The source tar is available here:
<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.4.tar.gz">rsync-3.4.4.tar.gz</a>
(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.4.tar.gz.asc">signature</a>)</b>,
and the diffs from version 3.4.3 are available here:
<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.3-3.4.4.diffs.gz">rsync-3.4.3-3.4.4.diffs.gz</a>
(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.3-3.4.4.diffs.gz.asc">signature</a>)</b>.
<p>Patch sets are also available for the older stable series, for
distributors not yet able to move to 3.4.4. Each is GPG signed. The
<i>full</i> set applies to a pristine release tarball; the <i>update</i>
set has only the patches added since the previous security release:<ul>
<li>rsync 3.2.7:
<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.2.7-full.tar.gz">rsync-3.2.7-full.tar.gz</a>
(<a href="https://download.samba.org/pub/rsync/src/rsync-3.2.7-full.tar.gz.asc">signature</a>)</b>,
<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.2.7-update.tar.gz">rsync-3.2.7-update.tar.gz</a>
(<a href="https://download.samba.org/pub/rsync/src/rsync-3.2.7-update.tar.gz.asc">signature</a>)</b>
<li>rsync 3.4.1:
<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.1-full.tar.gz">rsync-3.4.1-full.tar.gz</a>
(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.1-full.tar.gz.asc">signature</a>)</b>,
<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.1-update.tar.gz">rsync-3.4.1-update.tar.gz</a>
(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.1-update.tar.gz.asc">signature</a>)</b>
</ul>
<h3>Rsync version 3.4.3 released</h3>
<i class=date>May 20th, 2026</i>

View File

@@ -191,35 +191,31 @@ _PY_TEST_SUFFIX = '_test.py'
def _is_test_path(path):
base = os.path.basename(path)
return base.endswith('.test') or base.endswith(_PY_TEST_SUFFIX)
return os.path.basename(path).endswith(_PY_TEST_SUFFIX)
def _testbase(path):
"""Strip the test extension to get the canonical test name."""
base = os.path.basename(path)
if base.endswith('.test'):
return base[:-len('.test')]
if base.endswith(_PY_TEST_SUFFIX):
return base[:-len(_PY_TEST_SUFFIX)]
return base
def collect_tests(suitedir, patterns):
"""Collect test scripts (.test or _test.py) matching the given patterns."""
"""Collect test scripts (_test.py) matching the given patterns."""
if not patterns:
candidates = (glob.glob(os.path.join(suitedir, '*.test'))
+ glob.glob(os.path.join(suitedir, '*' + _PY_TEST_SUFFIX)))
candidates = glob.glob(os.path.join(suitedir, '*' + _PY_TEST_SUFFIX))
tests = sorted(p for p in candidates if _is_test_path(p))
else:
seen = set()
tests = []
for pat in patterns:
# Accept either bare name ("mkpath"), explicit extension, or glob.
if pat.endswith('.test') or pat.endswith('.py'):
if pat.endswith('.py'):
pats = [pat]
else:
pats = [pat + '.test', pat + _PY_TEST_SUFFIX]
pats = [pat + _PY_TEST_SUFFIX]
for p in pats:
for m in sorted(glob.glob(os.path.join(suitedir, p))):
if _is_test_path(m) and m not in seen:

View File

@@ -31,8 +31,11 @@ without interfering: each pushes, builds and tests in isolation. The run dir is
removed when the run ends -- on success or failure, and best-effort on
Ctrl-C/kill (pass --keep to retain it for inspection). A run that is hard-killed
(SIGKILL), or signalled mid-push, or whose ssh dies during cleanup can leave a
stray <builddir>-<id> behind; sweep those with `fleettest.py --cleanup`
(optionally scoped with --targets). Because each
stray <builddir>-<id> behind -- plus an orphaned path-flipper or test rsyncd on
platforms without a parent-death backstop; sweep all of those (root-owned files
included, via sudo -n) with `fleettest.py --cleanup` (optionally scoped with
--targets). Run --cleanup between runs, not during one: its process kills are
host-global and would also catch a concurrent run's flipper/daemon. Because each
run starts from a fresh dir, every build is a full configure + build.
PROVISIONING: each target must have the build toolchain its workflow's prepare
@@ -83,7 +86,11 @@ from pathlib import Path
# source tree these point at, so it must be run from inside an rsync checkout
# or given --repo PATH.
REPO = Path.cwd()
WORKFLOWS = REPO / ".github" / "workflows"
# Source tree providing the test suite (runtests.py + testsuite/). Defaults to
# REPO; --testsuite-repo decouples it so one tree is built and another's suite is
# run against the result.
TESTSUITE_REPO = REPO
WORKFLOWS = TESTSUITE_REPO / ".github" / "workflows"
# Fleet config (overridable with --fleet): ~/.fleettest.json is tried first, then
# fleettest.json next to this script. The example template sits next to the
@@ -767,33 +774,94 @@ def _on_signal(signum, frame):
os._exit(130 if signum == signal.SIGINT else 143)
# sweep() counts a pattern, kills it (best effort; sudo -n retry for processes a
# root-running test left), then RE-counts after a settle so we report what
# actually died (KILLED = before-after) and flag any survivor (SURVIVED, sets
# fail) instead of claiming success when pkill couldn't reach it. The patterns
# use the pgrep self-exclusion trick -- 'r[e]name'/'det[a]ch' match a real
# process's "rename"/"detach" but not the bracketed literal in this script's own
# argv (run_on passes the whole script as the remote argv), so we never signal
# ourselves. @BASE@ is substituted with the target's run-dir prefix.
_CLEANUP_SCRIPT = r'''fail=0
sweep() {
command -v pgrep >/dev/null 2>&1 || return 0
before=$(pgrep -f "$2" 2>/dev/null | wc -l | tr -d ' ')
[ "$before" -gt 0 ] 2>/dev/null || return 0
pkill -f "$2" 2>/dev/null
sudo -n pkill -f "$2" 2>/dev/null
sleep 1
after=$(pgrep -f "$2" 2>/dev/null | wc -l | tr -d ' ')
killed=$((before - after))
[ "$killed" -gt 0 ] 2>/dev/null && echo "KILLED $killed stray $1(s)"
if [ "$after" -gt 0 ] 2>/dev/null; then
echo "SURVIVED $after stray $1(s)"
fail=1
fi
}
sweep flipper 'r[e]name.*r[e]name.*r[e]name'
sweep daemon 'det[a]ch --address=127.0.0.1'
for d in @BASE@-*; do
[ -e "$d" ] || continue
if rm -rf -- "$d" 2>/dev/null || sudo -n rm -rf -- "$d" 2>/dev/null; then
echo "REMOVED $d"
else
echo "FAILED $d"
fail=1
fi
done
exit $fail
'''
def cleanup_remnants(targets: list[Target]) -> int:
"""--cleanup mode: remove every <base>-* run dir on each target, reporting
what each removed. Returns a process exit code. Only suffixed run dirs are
swept -- a bare <base> is left alone."""
"""--cleanup mode: on each target, kill the stray processes a killed run can
leave behind, then remove every <base>-* run dir, reporting what went.
Returns a process exit code. Only suffixed run dirs are swept -- a bare
<base> is left alone.
A run that is SIGKILLed (or whose ssh drops) can strand two kinds of process
on platforms without a parent-death backstop: the TOCTOU path-flipper (a
busy `python -c` rename loop that pins a CPU) and an orphaned test rsyncd
(`--no-detach --address=127.0.0.1`, which then squats its fixed port -- the
very wedge claim_ports()' bind-probe now reports). Both are killed best
effort (sudo -n retry for root-owned ones); a kill is verified by re-counting
afterwards, and a process that survives is reported and fails the run.
CAVEAT: the kill patterns are host-global, not scoped to a particular run, so
--cleanup assumes no *other* fleettest run is active on the target -- it
would also kill a concurrent run's flipper/daemon (and any manual `rsync
--daemon --no-detach --address=127.0.0.1`). Run it between runs, not during
one. Run dirs whose contents a root test owns are removed via a `sudo -n rm`
fallback; only a dir that survives even that is a failure."""
rc = 0
for t in targets:
base = t.builddir
if _unsafe_builddir(base):
log(f"[{t.name}] skipped (unsafe builddir {base!r})")
continue
# Echo each match before removing it so the harness can report what
# went; an unmatched glob stays literal and is skipped by the -e test.
script = (f'set -e\n'
f'for d in {base}-*; do\n'
f' [ -e "$d" ] || continue\n'
f' echo "$d"\n'
f' rm -rf -- "$d"\n'
f'done\n')
r = run_on(t, script, timeout=120)
removed = [ln for ln in r.out.splitlines() if ln.strip()]
if r.rc != 0:
# Structured markers (KILLED/SURVIVED/REMOVED/FAILED) keep the report
# clean even though run_on() folds stderr into stdout.
r = run_on(t, _CLEANUP_SCRIPT.replace("@BASE@", base), timeout=120)
lines = r.out.splitlines()
removed = [ln.split(" ", 1)[1] for ln in lines if ln.startswith("REMOVED ")]
failed = [ln.split(" ", 1)[1] for ln in lines if ln.startswith("FAILED ")]
killed = [ln.replace("KILLED ", "killed ", 1)
for ln in lines if ln.startswith("KILLED ")]
survived = [ln.replace("SURVIVED ", "still alive: ", 1)
for ln in lines if ln.startswith("SURVIVED ")]
msgs = killed[:]
if removed:
msgs.append("removed: " + " ".join(removed))
if survived:
rc = 1
log(f"[{t.name}] cleanup error (rc={r.rc}): {r.out.strip()[:200]}")
elif removed:
log(f"[{t.name}] removed: {' '.join(removed)}")
else:
log(f"[{t.name}] nothing to remove")
msgs += survived
if failed:
rc = 1
msgs.append("could not remove (even with sudo): " + " ".join(failed))
if r.rc not in (0, 1):
rc = 1
msgs.append(f"cleanup error rc={r.rc}: {r.out.strip()[:160]}")
log(f"[{t.name}] " + ("; ".join(msgs) if msgs else "nothing to remove"))
return rc
@@ -809,24 +877,44 @@ def main() -> int:
ap.add_argument("--keep", action="store_true",
help="keep each run's build dir (default: remove it at exit)")
ap.add_argument("--cleanup", action="store_true",
help="remove stray <builddir>-* run dirs on the targets, then exit")
help="kill stray flippers/test daemons and remove stray "
"<builddir>-* run dirs (root-owned via sudo -n) on the "
"targets, then exit; run between runs, not during one "
"(kills are host-global)")
ap.add_argument("--jobs", type=int, help="override -j for both transports")
ap.add_argument("--timing", action="store_true",
help="report per-target wall-clock (push/build/test) to find "
"the slowest target")
ap.add_argument("--repo", help="rsync source tree to build (default: cwd)")
ap.add_argument("--testsuite-repo",
help="rsync tree to take runtests.py + testsuite/ from "
"(default: --repo). Build one tree and run another's test "
"suite against it, e.g. --repo ../rsync-v3.4 --testsuite-repo .")
ap.add_argument("--fleet", help="fleet config JSON (default: ~/.fleettest.json, "
"else fleettest.json next to this script)")
ap.add_argument("--list", action="store_true", help="list targets and exit")
args = ap.parse_args()
global REPO, WORKFLOWS
global REPO, WORKFLOWS, TESTSUITE_REPO
REPO = Path(args.repo).resolve() if args.repo else Path.cwd()
WORKFLOWS = REPO / ".github" / "workflows"
if not args.cleanup and not (REPO / "runtests.py").is_file():
print(f"{REPO} is not an rsync source tree (no runtests.py); "
f"run from inside a checkout or pass --repo", file=sys.stderr)
return 2
TESTSUITE_REPO = Path(args.testsuite_repo).resolve() if args.testsuite_repo else REPO
# The expected-skip lists travel with the suite, so read workflows from the
# tree that provides the tests.
WORKFLOWS = TESTSUITE_REPO / ".github" / "workflows"
if not args.cleanup:
# The Python test suite (runtests.py + testsuite/) comes from
# TESTSUITE_REPO, so that is where runtests.py must live. The build tree
# (REPO) only has to be a buildable rsync source -- it may be an older
# release whose runtests.py predates the Python suite, or lacks it.
if not (TESTSUITE_REPO / "runtests.py").is_file():
print(f"{TESTSUITE_REPO} has no runtests.py; run from inside a "
f"checkout or pass --testsuite-repo a tree with the Python "
f"test suite", file=sys.stderr)
return 2
if not (REPO / "rsync.h").is_file():
print(f"{REPO} is not an rsync source tree (no rsync.h); "
f"run from inside a checkout or pass --repo", file=sys.stderr)
return 2
if args.fleet:
config_path = Path(args.fleet).resolve()
@@ -905,6 +993,19 @@ def main() -> int:
print(f"git archive failed: {ar.stderr}", file=sys.stderr)
return 2
# --testsuite-repo: overlay another tree's runtests.py + testsuite/ onto
# the built source (merge, no delete). Build REPO's rsync, but run
# TESTSUITE_REPO's suite against it. The leftover .test files from REPO
# are ignored by a Python runtests.py (it globs *_test.py).
if TESTSUITE_REPO != REPO:
ov = subprocess.run(
f"git -C {TESTSUITE_REPO} archive HEAD -- runtests.py testsuite "
f"| tar -x -C {staging}",
shell=True, capture_output=True, text=True)
if ov.returncode != 0:
print(f"testsuite overlay archive failed: {ov.stderr}", file=sys.stderr)
return 2
# Tests that opt into the non-root pass (same for every target).
args.nonroot_tests = discover_nonroot_tests(Path(staging) / "testsuite")

View File

@@ -1,190 +0,0 @@
#!/usr/bin/env python3
"""rsync-ssl socat transport anti-regression tests.
These tests exercise the wrapper/helper contract without requiring a live TLS
server. Fake helper binaries capture argv so the test can verify the intended
transport selection and OPENSSL address construction.
"""
import os
import shutil
import subprocess
import sys
from rsyncfns import SCRATCHDIR, SRCDIR, test_fail
RSYNC_SSL = SRCDIR / 'rsync-ssl'
BASH = shutil.which('bash')
HELPER_ARGV = SCRATCHDIR / 'helper.argv'
RSYNC_ARGV = SCRATCHDIR / 'rsync.argv'
OPENSSL_ARGV = SCRATCHDIR / 'openssl.argv'
FAKEBIN = SCRATCHDIR / 'fakebin'
FAKEBIN.mkdir()
if BASH is None:
test_fail('bash is required to run rsync-ssl')
def script(path, text):
path.write_text(text)
path.chmod(0o755)
return path
def argv_capture_script(path, output_path, env_name=None):
env_capture = ''
if env_name:
env_capture = (
f" out.write({env_name + '='!r} + "
f"os.environ.get({env_name!r}, '') + '\\n')\n"
)
return script(path, f'''#!{sys.executable}
import os
import sys
with open({str(output_path)!r}, 'w', encoding='utf-8') as out:
{env_capture} for arg in sys.argv[1:]:
out.write(arg + '\\n')
''')
fake_socat = argv_capture_script(FAKEBIN / 'socat', HELPER_ARGV)
FALLBACKBIN = SCRATCHDIR / 'fallbackbin'
FALLBACKBIN.mkdir()
fallback_helper_argv = SCRATCHDIR / 'fallback-helper.argv'
argv_capture_script(FALLBACKBIN / 'socat', fallback_helper_argv)
argv_capture_script(FAKEBIN / 'openssl', OPENSSL_ARGV)
argv_capture_script(FAKEBIN / 'rsync', RSYNC_ARGV, 'RSYNC_SSL_TYPE')
def clean_env(**updates):
env = os.environ.copy()
for key in list(env):
if key.startswith('RSYNC_SSL_') or key == 'RSYNC_PORT':
del env[key]
env['PATH'] = f'{FAKEBIN}:{env["PATH"]}'
for key, value in updates.items():
env[key] = value
return env
def fallback_env(**updates):
env = clean_env(**updates)
env['PATH'] = f'{FALLBACKBIN}'
return env
def run_helper(host, **env_updates):
HELPER_ARGV.unlink(missing_ok=True)
proc = subprocess.run(
[str(RSYNC_SSL), '--HELPER', host, 'rsync', '--server', '--daemon', '.'],
env=clean_env(RSYNC_SSL_TYPE='socat', RSYNC_SSL_SOCAT=str(fake_socat),
**env_updates),
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
)
if proc.returncode != 0:
test_fail(f'rsync-ssl socat helper failed: {proc.stderr}')
if not HELPER_ARGV.exists():
test_fail('fake socat helper was not executed')
return HELPER_ARGV.read_text().splitlines()
# --- --type=socat is consumed by the wrapper and passed via helper env -------
proc = subprocess.run(
[str(RSYNC_SSL), '--type=socat', 'example.com::module'],
env=clean_env(),
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
)
if proc.returncode != 0:
test_fail(f'rsync-ssl --type=socat wrapper failed: {proc.stderr}')
rsync_argv = RSYNC_ARGV.read_text().splitlines()
if rsync_argv[0] != 'RSYNC_SSL_TYPE=socat':
test_fail('--type=socat did not set RSYNC_SSL_TYPE for the rsync wrapper')
if '--type=socat' in rsync_argv:
test_fail('--type=socat leaked through to the real rsync argv')
if not any(arg.startswith('--rsh=') and arg.endswith(' --HELPER')
for arg in rsync_argv):
test_fail('rsync-ssl did not install itself as the rsync --rsh helper')
# --- socat helper uses default verification and SNI for host names -----------
argv = run_helper('example.com')
want = [
'-',
'OPENSSL:example.com:874,commonname=example.com,snihost=example.com,verify=1',
]
if argv != want:
test_fail(f'unexpected socat argv for host name: {argv!r}')
# --- explicit CA/cert/key/port are preserved and IP addresses disable SNI ----
argv = run_helper(
'127.0.0.1',
RSYNC_PORT='8873',
RSYNC_SSL_CA_CERT='/tmp/ca.pem',
RSYNC_SSL_CERT='/tmp/cert.pem',
RSYNC_SSL_KEY='/tmp/key.pem',
)
want = [
'-',
('OPENSSL:127.0.0.1:8873,commonname=127.0.0.1,no-sni=1,'
'cafile=/tmp/ca.pem,verify=1,cert=/tmp/cert.pem,key=/tmp/key.pem'),
]
if argv != want:
test_fail(f'unexpected socat argv for IP address: {argv!r}')
# --- empty RSYNC_SSL_CA_CERT deliberately disables socat verification --------
argv = run_helper('example.net', RSYNC_SSL_CA_CERT='')
want = [
'-',
'OPENSSL:example.net:874,commonname=example.net,snihost=example.net,verify=0',
]
if argv != want:
test_fail(f'unexpected socat argv for disabled verification: {argv!r}')
# --- default helper selection keeps existing openssl-first behaviour ---------
OPENSSL_ARGV.unlink(missing_ok=True)
proc = subprocess.run(
[str(RSYNC_SSL), '--HELPER', 'example.org', 'rsync', '--server',
'--daemon', '.'],
env=clean_env(),
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
)
if proc.returncode != 0:
test_fail(f'rsync-ssl default helper failed: {proc.stderr}')
if not OPENSSL_ARGV.exists():
test_fail('default rsync-ssl helper selection did not execute openssl')
openssl_argv = OPENSSL_ARGV.read_text().splitlines()
if not openssl_argv or openssl_argv[0] != 's_client':
test_fail(f'default helper selection did not use openssl s_client: {openssl_argv!r}')
# --- if openssl is unavailable, default selection prefers socat over stunnel -
fallback_helper_argv.unlink(missing_ok=True)
proc = subprocess.run(
[BASH, str(RSYNC_SSL), '--HELPER', 'fallback.example', 'rsync',
'--server', '--daemon', '.'],
env=fallback_env(),
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
)
if proc.returncode != 0:
test_fail(f'rsync-ssl fallback helper failed: {proc.stderr}')
if not fallback_helper_argv.exists():
test_fail('default rsync-ssl fallback selection did not execute socat')
fallback_argv = fallback_helper_argv.read_text().splitlines()
want = [
'-',
('OPENSSL:fallback.example:874,commonname=fallback.example,'
'snihost=fallback.example,verify=1'),
]
if fallback_argv != want:
test_fail(f'unexpected socat argv for fallback selection: {fallback_argv!r}')
print('rsync-ssl-socat: wrapper and helper transport behaviour verified')

View File

@@ -175,6 +175,42 @@ def _open_lock_file() -> int:
return fd
def _probe_bindable(port: int) -> 'None':
"""Confirm `port` is actually free once we hold its claim_ports() lock.
The byte-range lock only coordinates *live* test drivers, and the kernel
releases it the instant the holding process dies -- even if that driver left
an orphaned daemon still bound to the port. That happens when a run is
SIGKILLed (or its ssh drops) on a platform with no parent-death backstop:
rsyncfns only arms PR_SET_PDEATHSIG, which is Linux-only, so on the
BSDs/Solaris/macOS a killed fleettest run can strand its rsyncd, which then
squats the fixed test port forever. A later run wins the (now-free) lock but
the socket is still taken, and the daemon dies with a cryptic "bind() failed:
Address already in use" / the client "did not see server greeting".
So actually try to bind it. SO_REUSEADDR is used so a port merely in
TIME_WAIT (recently and cleanly closed) is NOT a false positive; only a
live bound/listening socket -- a real squatter -- makes the bind fail, and
then we stop here with an actionable message instead of failing obscurely
later. The probe socket is closed immediately, freeing the port for the
daemon that is about to bind it.
"""
s = _socket.socket(_socket.AF_INET, _socket.SOCK_STREAM)
s.setsockopt(_socket.SOL_SOCKET, _socket.SO_REUSEADDR, 1)
try:
s.bind(('127.0.0.1', port))
except OSError as e:
test_fail(
f"port {port} was claimed for this run but something is still bound "
f"to 127.0.0.1:{port} ({e.strerror}). The claim_ports() lock only "
"serializes live test runs, so a still-bound port almost always "
"means an orphaned 'rsync --daemon' from a previously killed run "
f"(find it with `fstat | grep {port}` / `netstat -an | grep {port}` "
"and kill it, or run `fleettest.py --cleanup`), then retry.")
finally:
s.close()
def claim_ports(*ports: int) -> 'None':
"""Reserve the given TCP port numbers for the rest of this process.
@@ -210,6 +246,9 @@ def claim_ports(*ports: int) -> 'None':
# F_SETLKW via fcntl.lockf(LOCK_EX, length, start): exclusive
# byte-range lock on byte `port`, blocking until acquired.
fcntl.lockf(_port_lock_fd, fcntl.LOCK_EX, 1, port)
# The lock only proves no other live test run owns the port; an orphaned
# daemon from a killed run can still squat it (see _probe_bindable).
_probe_bindable(port)
# --- standalone rsyncd helpers ---------------------------------------------

View File

@@ -295,8 +295,12 @@ static int rsync_xal_get(const char *fname, item_list *xalp)
rxa = xalp->items;
if (count > 1)
qsort(rxa, count, sizeof (rsync_xa), rsync_xal_compare_names);
for (rxa += count-1; count; count--, rxa--)
rxa->num = count;
/* Guard count==0: rxa is then xalp->items (possibly NULL) and the
* "rxa += count-1" init would form NULL-1 (undefined). */
if (count) {
for (rxa += count-1; count; count--, rxa--)
rxa->num = count;
}
return 0;
}
@@ -381,17 +385,19 @@ static int64 xattr_lookup_hash(const item_list *xalp)
{
const rsync_xa *rxas = xalp->items;
size_t i;
int64 key = hashlittle2(&xalp->count, sizeof xalp->count);
/* Accumulate unsigned: the summed hash values routinely overflow a
* signed int64 (UB), and we only care about the resulting bit pattern. */
uint64_t key = (uint64_t)hashlittle2(&xalp->count, sizeof xalp->count);
for (i = 0; i < xalp->count; i++) {
key += hashlittle2(rxas[i].name, rxas[i].name_len);
key += (uint64_t)hashlittle2(rxas[i].name, rxas[i].name_len);
if (rxas[i].datum_len > MAX_FULL_DATUM)
key += hashlittle2(rxas[i].datum, xattr_sum_len);
key += (uint64_t)hashlittle2(rxas[i].datum, xattr_sum_len);
else
key += hashlittle2(rxas[i].datum, rxas[i].datum_len);
key += (uint64_t)hashlittle2(rxas[i].datum, rxas[i].datum_len);
}
return key;
return (int64)key;
}
static int find_matching_xattr(const item_list *xalp)
@@ -460,7 +466,9 @@ static int rsync_xal_store(item_list *xalp)
* entire initial-count, not just enough space for one new item. */
*new_list = empty_xa_list;
(void)EXPAND_ITEM_LIST(&new_list->xa_items, rsync_xa, xalp->count);
memcpy(new_list->xa_items.items, xalp->items, xalp->count * sizeof (rsync_xa));
/* xalp->items is NULL for an empty list; memcpy(dst, NULL, 0) is UB. */
if (xalp->count)
memcpy(new_list->xa_items.items, xalp->items, xalp->count * sizeof (rsync_xa));
new_list->xa_items.count = xalp->count;
xalp->count = 0;