Compare commits

..

50 Commits

Author SHA1 Message Date
Andrew Tridgell
de3cc03b03 Preparing for release of 3.4.2 [buildall] 2026-04-28 14:29:48 +10:00
Andrew Tridgell
006ee327d6 packaging: new release script 2026-04-28 14:27:41 +10:00
Andrew Tridgell
9b6363fa10 update NEWS.md ready for 3.4.2 2026-04-28 12:55:38 +10:00
Andrew Tridgell
9e2f0fe9ae packaging: remove support for rsync-patches 2026-04-28 12:55:38 +10:00
Michal Ruprich
4f6e4ea64a Do not clean DISPLAY unconditionally 2026-04-22 13:05:35 +10:00
Andrew Tridgell
567c40935f call tzset() before chroot to cache timezone data
localtime/localtime_r need /etc/localtime for timezone info.
After chroot this file is inaccessible, causing log timestamps
to fall back to UTC. Calling tzset() before chroot ensures the
timezone data is cached by glibc for subsequent calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 13:02:10 +10:00
Michal Ruprich
8e11f0c169 Using a correct time in log file 2026-04-22 13:02:10 +10:00
Andrew Tridgell
e9dbc8d66d rsyncd.conf: document the temp dir parameter
The temp dir parameter was functional but undocumented in the man page.

Fixes: https://github.com/RsyncProject/rsync/issues/820

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:58 +10:00
Andrew Tridgell
bd2dbd2f32 runtests.py: preserve test-execution order in skipped list
The sorted() call reordered skipped test names alphabetically,
causing CI expected-skipped mismatches (e.g. acls,acls-default
instead of acls-default,acls). Sort by original test order instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:39 +10:00
Andrew Tridgell
350e295d1c runtests.py: add -j/--parallel option for parallel test execution
Add parallel test execution using concurrent.futures. With -j8 the
test suite completes in ~4s vs ~29s sequential (~7x speedup).

Also fix two issues that caused failures under parallel execution:
- rsync_ls_lR now prunes testtmp/ so parallel tests don't see each
  other's temp files when scanning the source tree
- clean-fname-underflow.test now uses $scratchdir instead of /tmp

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:39 +10:00
Andrew Tridgell
066156fcd9 replace runtests.sh with runtests.py
Rewrite the test runner in Python with proper command-line options
including --valgrind which directs valgrind output to per-process
log files so it doesn't interfere with test output comparisons.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:39 +10:00
Holger Hoffstätte
a5bbe859db Fix glibc-2.43 constness warnings
Glibc 2.43 added C23 const-preserving overloads to various string functions,
which change the return type depending on the constness of the argument(s).
Currently this leads to warnings from calls to strtok() or strchr().
Fix this by properly declaring the respective variable types.

Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
2026-04-22 12:10:08 +10:00
Andrew Tridgell
d046525de3 zero all new memory from allocations
Change my_alloc() to use calloc instead of malloc so all fresh
allocations return zeroed memory. Also zero the expanded portion
in expand_item_list() after realloc, since it knows both old and
new sizes. This gives more predictable behaviour in case of bugs
where uninitialised or stale memory is accidentally accessed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 11:44:10 +10:00
Andrew Tridgell
bb0a8118c2 xattrs: fixed count in qsort
this fixes the count passed to the sort of the xattr list. This issue
was reported here:

https://www.openwall.com/lists/oss-security/2026/04/16/2

the bug is not exploitable due to the fork-per-connection design of
rsync, the attack is the equivalent of the user closing the socket
themselves.
2026-04-22 10:38:14 +10:00
Andrew Tridgell
d1df0aaf70 fix signed integer overflow in proxy protocol v2 header parsing
The len field in the proxy v2 header was declared as signed char,
allowing a negative size to bypass the validation check and cause
a stack buffer overflow when passed to read_buf() as size_t.

This bug was reported by John Walker from ZeroPath, many thanks for
the clear report!

With the current code this bug does not represent a security issue as
it only results in the exit of the forked process that is specific to
the attached client, so it is equivalent to the client closing the
socket, so no CVE for this, but it is good to fix it to prevent a
future issue.
2026-04-16 13:59:52 +10:00
Andrew Tridgell
15d8e49a64 zlib: convert K&R function definitions to ANSI style
The bundled zlib 1.2.8 used K&R-style function definitions which are
rejected by clang 16+ as hard errors. Convert all 90 functions across
9 files to ANSI-style prototypes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 13:49:30 +10:00
Andrew Tridgell
b905ab23af CI: add simd-checksum to expected-skipped on macOS and Cygwin
The new simd-checksum test is skipped on platforms where SIMD
instructions are unavailable (macOS ARM, Cygwin). Add it to the
RSYNC_EXPECT_SKIPPED lists so CI doesn't fail on the mismatch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:52:01 +11:00
Andrew Tridgell
aa142f08ef fix uninitialized mul_one in AVX2 checksum and add SIMD checksum test
The AVX2 get_checksum1_avx2_64() read mul_one before initializing it,
which is undefined behavior. Replace the cmpeq/abs trick with
_mm256_set1_epi8(1) to match the SSSE3 and SSE2 versions.

Add a TEST_SIMD_CHECKSUM1 test mode that verifies all SIMD paths
(SSE2, SSSE3, AVX2, and the full dispatch chain) produce identical
results to the C reference, across multiple buffer sizes with both
aligned and unaligned buffers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:52:01 +11:00
Andrew Tridgell
236417cf35 acl: fixed ACL ID mapping for non-root
closes issue #618
2026-01-19 11:32:13 +11:00
Andrew Tridgell
2a97d81e99 CI: fixed MacOS test
fixed multiple MacOS issues
2025-12-31 11:37:27 +11:00
Andrew Tridgell
359e539a72 reject negative token values in compressed stream receivers
Validate that token numbers read from compressed streams are
non-negative. A negative token value would cause the return value
of recv_*_token() to become positive, which callers interpret as
literal data length, but no data pointer is set on this code path.

While this only causes the receiver to crash (which is process-isolated
and only affects the attacker's own connection), it's still undefined
behavior.

Reported-by: Will Sergeant <wlsergeant@gmail.com>
2025-12-31 09:31:52 +11:00
Andrew Tridgell
9e0898460d util: fixed issue in clean_fname()
fixes buffer underflow (not exploitable) in clean_fname
2025-12-30 17:49:35 +11:00
Andrew Tridgell
185520a141 testsuite: added clean-fname-underflow test 2025-12-30 17:49:35 +11:00
Andrew Tridgell
c98f9d1f68 fix uninitialized buf1 in get_checksum2() MD4 path
The static buf1 pointer was only allocated when len > len1, but on
first call with len == 0, this condition is false (0 > 0), leaving
buf1 NULL when passed to memcpy().

Fixes #673
2025-12-30 16:51:43 +11:00
Nebojša Cvetković
1f9ce2fcbe rsync: Add missing dirs long option 2025-12-30 16:48:34 +11:00
Andrew Tridgell
797e17fc4a fixed an invalid access to files array
this was found by Calum Hutton from Rapid7. It is a real bug, but
analysis shows it can't be leverged into an exploit. Worth fixing
though.

Many thanks to Calum and Rapid7 for finding and reporting this
2025-08-23 17:49:19 +10:00
Ronnie Sahlberg
c2db921890 options.c: Fix segv if poptGetContext returns NULL
If poptGetContext returns NULL, perhaps due to OOM,
a NULL pointer is passed into poptReadDefaultConfig()
which in turns SEGVs when trying to dereference it.

This was found using https://github.com/sahlberg/malloc-fail-tester.git
$ ./test_malloc_failure.sh rsync -Pav crash crosh

Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
2025-08-23 17:49:03 +10:00
Silent
77be09aaed syscall: fix a Y2038 bug by replacing Int32x32To64 with multiplication
Int32x32To64 macro internally truncates the arguments to int32,
while time_t is 64-bit on most/all modern platforms.
Therefore, usage of this macro creates a Year 2038 bug.
2025-08-23 17:32:11 +10:00
Jeremy Norris
0d0f615240 Ignore directory has vanished errors. 2025-08-23 17:31:52 +10:00
Max Kellermann
b6457bbc83 make lots of global variables const
This way, they can live in `.rodata` and the compiler is allowed to do
certain optimizations.
2025-08-23 17:31:40 +10:00
Peter Eriksson
1807ce485a Fix handling of objects with many xattrs on FreeBSD 2025-08-23 17:31:28 +10:00
Rahul Mehta
9c175ac9ef chore: gitignore MacOS debug symbols 2025-08-23 17:31:12 +10:00
Emily
a84b79ea58 Allow ls(1) to fail in test setup
This can happen when the tests are unable to `stat(2)` some files in
`/etc`, `/bin`, or `/`, due to Unix permissions or other sandboxing. We
still guard against serious errors, which use exit code 2.
2025-08-23 17:30:59 +10:00
fbuescher
d4c4f6754e fixed remove multiple leading slashes 2025-08-23 17:14:43 +10:00
Michal Ruprich
a4b926dcdc bool is a keyword in C23 2025-08-23 17:14:26 +10:00
Eli Schwartz
0973d0e380 configure.ac: check for xattr support both in libc and in -lattr
In 2015, the attr/xattr.h header was fully removed from upstream attr.

In 2020, rsync started preferring the standard header, if it exists:
https://github.com/RsyncProject/rsync/pull/22

But the fix was incomplete. We still looked for the getxattr function in
-lattr, and used it if -lattr exists. This was the case even if the
system libc was sufficient to provide the needed functions. Result:
overlinking to -lattr, if it happened to be installed for any other
reason.

```
checking whether to support extended attributes... Using Linux xattrs
checking for getxattr in -lattr... yes
```

Instead, use a different autoconf macro that first checks if the
function is available for use without any libraries (e.g. it is in
libc).

Result:

```
checking whether to support extended attributes... Using Linux xattrs
checking for library containing getxattr... none required
```

Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>
2025-08-23 17:14:06 +10:00
Ethan Halsall
e405cfc073 feat: add compress threads to man page 2025-08-23 17:13:49 +10:00
Ethan Halsall
b78a841bb0 feat: validate compress threads option 2025-08-23 17:13:49 +10:00
Ethan Halsall
f7a2b8a3fa feat: add threads to zstd compression 2025-08-23 17:13:49 +10:00
Arnaud Rebillout
d941807915 Fix flaky hardlinks test
The test was added in dc34990, it turns out that it's flaky. It failed
once on the Debian build infra, cf. [1].

The problem is that the command `rsync -aH '$fromdir/sym' '$todir'`
updates the mod time of `$todir`, so there might be a diff between the
output of `rsync_ls_lR $fromdir` and `rsync_ls_lR $todir`, if ever rsync
runs 1 second (or more) after the directories were created.

To clarify: it's easy to make the test fails 100% of the times with this
change:

```
 makepath "$fromdir/sym" "$todir"
+sleep 5
 checkit "$RSYNC -aH '$fromdir/sym' '$todir'" "$fromdir" "$todir"
```

With the fix proposed here, we don't use `checkit` anymore, instead we
just run the rsync command, then a simple `diff` to compare the two
directories. This is exactly what the other `-H` test just above does.

In case there's some doubts, `diff` fails if `sym` is missing:

```
$ mkdir -p foo/sym bar
$ diff foo bar || echo KO!
Only in foo: sym
KO!
```

I tested that, after this commit, the test still catches the `-H`
regression in rsync 3.4.0.

Fixes: https://github.com/RsyncProject/rsync/issues/735

[1]: https://buildd.debian.org/status/fetch.php?pkg=rsync&arch=ppc64el&ver=3.4.1%2Bds1-1&stamp=1741147156&raw=0
2025-08-23 17:13:28 +10:00
Krzysztof Płocharz
992e10efaf Fix --open-noatime option not working on files
atime of source files could sometimes be overwritten
even though --open-noatime option was used.

To fix that, optional O_NOATIME flag was added
to do_open_nofollow which is also used to open regular
files since fix:
  "fixed symlink race condition in sender"
Previously optional O_NOATIME flag was only in do_open.
2025-08-23 17:13:09 +10:00
Chris Lamb
1c5ebdc4e5 Make the build reproducible
From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1093201:
Whilst working on the Reproducible Builds effort [0], we noticed that
rsync could not be built reproducibly.

This is because the date in the manual page can vary depending on
whether there is a .git directory and the modification time of version.h
and Mafile, which might get modified when patching via quilt.

A patch is attached that makes this use SOURCE_DATE_EPOCH, which
will always be reliable.
2025-08-23 16:40:34 +10:00
Wayne Davison
9994933c8c Test on ubuntu-latest. 2025-02-11 13:37:12 -08:00
Alan Coopersmith
23d9ead5af popt: remove obsolete findme.c & findme.h
popt 1.14 merged these into popt.c but the import into rsync
missed removing them.

Fixes: https://github.com/RsyncProject/rsync/issues/710

Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>
2025-01-17 08:31:36 +11:00
Wayne Davison
fcfdd36054 Update MAINTAINER_TZ_OFFSET on release.
This also fixes a string with \s that wasn't a r'...' string.
2025-01-15 23:27:27 -08:00
Wayne Davison
89b847393f Fix python deprecation warning. 2025-01-15 22:36:29 -08:00
Wayne Davison
788ecbe5ea Don't edit copyright year values anymore. 2025-01-15 22:30:32 -08:00
Wayne Davison
353506bc51 Improve interior dashes in long options.
Improve the backslash-adding code in md-convert to affect dashes in the
interior of long options.  Perhaps fixes #686.
2025-01-15 22:23:30 -08:00
Wayne Davison
7cff121ec8 Start 3.4.2dev going. 2025-01-15 22:01:42 -08:00
Andrew Tridgell
14f33837dc fixed build error on ia64 NonStop
it treats missing prototype as an error, not warning
2025-01-16 15:27:21 +11:00
77 changed files with 1654 additions and 4181 deletions

View File

@@ -1,76 +0,0 @@
name: Test rsync on AlmaLinux 8
# Older-LTS coverage on the Fedora/RHEL family to help with backporting
# security fixes. AlmaLinux 8 is the RHEL 8 rebuild and is the oldest
# active LTS in this family (RHEL 8 full support runs to 2029).
# GitHub Actions has no native runner for this family, so the job runs
# inside an almalinux:8 container hosted on ubuntu-latest.
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/almalinux-8-build.yml'
pull_request:
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/almalinux-8-build.yml'
schedule:
- cron: '42 8 * * *'
jobs:
test:
runs-on: ubuntu-latest
container:
image: almalinux:8
name: Test rsync on AlmaLinux 8
steps:
- name: install git
# actions/checkout needs git in the container before the checkout step.
run: dnf -y install git
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
# PowerTools is needed for libzstd-devel etc; xxhash and lz4 dev
# headers live in EPEL on RHEL 8. The default python3 on RHEL 8
# is 3.6, which is too old for runtests.py (uses capture_output=
# / text= introduced in 3.7), so install python39 and point
# /usr/bin/python3 at it.
run: |
dnf -y install epel-release
dnf config-manager --set-enabled powertools
dnf -y install gcc gcc-c++ make autoconf automake m4 \
python39 python39-pip diffutils \
openssl openssl-devel \
attr libattr-devel acl libacl-devel \
zstd libzstd-devel \
lz4 lz4-devel \
xxhash xxhash-devel
alternatives --set python3 /usr/bin/python3.9
pip3 install commonmark
- name: configure
run: ./configure --with-rrsync
- name: make
run: make
- name: info
run: ./rsync --version
- name: check
# In the container we already run as root, so no sudo. The
# crtimes-not-supported skip matches the other Linux jobs.
run: RSYNC_EXPECT_SKIPPED=crtimes make check
- name: ssl file list
run: ./rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4
with:
name: almalinux-8-bin
path: |
rsync
rsync-ssl
rsync.1
rsync-ssl.1
rsyncd.conf.5
rrsync.1
rrsync

View File

@@ -7,6 +7,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/cygwin-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/cygwin-build.yml'
@@ -38,7 +39,7 @@ jobs:
- name: info
run: bash -c '/usr/local/bin/rsync --version'
- name: check
run: bash -c 'RSYNC_EXPECT_SKIPPED=acls-default,acls,bare-do-open-symlink-race,chdir-symlink-race,chmod-symlink-race,chown,daemon-chroot-acl,devices,dir-sgid,protected-regular,sender-flist-symlink-leak,simd-checksum,symlink-dirlink-basis make check'
run: bash -c 'RSYNC_EXPECT_SKIPPED=acls-default,acls,chown,devices,dir-sgid,open-noatime,protected-regular,simd-checksum make check'
- name: ssl file list
run: bash -c 'PATH="/usr/local/bin:$PATH" rsync-ssl --no-motd download.samba.org::rsyncftp/ || true'
- name: save artifact

View File

@@ -7,6 +7,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/freebsd-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/freebsd-build.yml'
@@ -33,7 +34,6 @@ jobs:
./configure --with-rrsync -disable-zstd --disable-md2man --disable-xxhash --disable-lz4
make
./rsync --version
make check
./rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4

View File

@@ -7,6 +7,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/macos-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/macos-build.yml'
@@ -40,7 +41,7 @@ jobs:
- name: info
run: rsync --version
- name: check
run: sudo RSYNC_EXPECT_SKIPPED=acls-default,chmod-temp-dir,chown-fake,daemon-chroot-acl,devices-fake,dir-sgid,protected-regular,simd-checksum,xattrs-hlink,xattrs make check
run: sudo RSYNC_EXPECT_SKIPPED=acls-default,chmod-temp-dir,chown-fake,devices-fake,dir-sgid,open-noatime,protected-regular,simd-checksum,xattrs-hlink,xattrs make check
- name: ssl file list
run: rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact

View File

@@ -1,51 +0,0 @@
name: Test rsync on NetBSD
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/netbsd-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/netbsd-build.yml'
schedule:
- cron: '42 8 * * *'
jobs:
test:
runs-on: ubuntu-latest
name: Test rsync on NetBSD
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Test in NetBSD VM
id: test
uses: vmactions/netbsd-vm@v1
with:
usesh: true
prepare: |
PATH=/usr/sbin:$PATH pkg_add autoconf automake python312
ln -sf /usr/pkg/bin/python3.12 /usr/pkg/bin/python3
run: |
uname -a
./configure --with-rrsync --disable-zstd --disable-md2man --disable-xxhash --disable-lz4
make
./rsync --version
make check
./rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4
with:
name: netbsd-bin
path: |
rsync
rsync-ssl
rsync.1
rsync-ssl.1
rsyncd.conf.5
rrsync.1
rrsync

View File

@@ -1,52 +0,0 @@
name: Test rsync on OpenBSD
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/openbsd-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/openbsd-build.yml'
schedule:
- cron: '42 8 * * *'
jobs:
test:
runs-on: ubuntu-latest
name: Test rsync on OpenBSD
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Test in OpenBSD VM
id: test
uses: vmactions/openbsd-vm@v1
with:
usesh: true
prepare: |
pkg_add -I bash autoconf%2.71 automake%1.16
run: |
uname -a
export AUTOCONF_VERSION=2.71
export AUTOMAKE_VERSION=1.16
./configure --with-rrsync --disable-zstd --disable-md2man --disable-xxhash --disable-lz4
make
./rsync --version
make check
./rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4
with:
name: openbsd-bin
path: |
rsync
rsync-ssl
rsync.1
rsync-ssl.1
rsyncd.conf.5
rrsync.1
rrsync

View File

@@ -7,6 +7,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/solaris-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/solaris-build.yml'
@@ -33,7 +34,6 @@ jobs:
./configure --with-rrsync -disable-zstd --disable-md2man --disable-xxhash --disable-lz4
make
./rsync --version
make check
./rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4

View File

@@ -1,59 +0,0 @@
name: Test rsync on Ubuntu 22.04
# Older-LTS coverage to help with backporting security fixes. ubuntu-22.04
# is currently the oldest GitHub Actions runner image (20.04 was retired
# in April 2025).
on:
push:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-22.04-build.yml'
pull_request:
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-22.04-build.yml'
schedule:
- cron: '42 8 * * *'
jobs:
test:
runs-on: ubuntu-22.04
name: Test rsync on Ubuntu 22.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: prep
run: |
sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev python3-cmarkgfm openssl
echo "/usr/local/bin" >>$GITHUB_PATH
- name: configure
run: ./configure --with-rrsync
- name: make
run: make
- name: install
run: sudo make install
- name: info
run: rsync --version
- name: check
run: sudo RSYNC_EXPECT_SKIPPED=crtimes make check
- name: check30
run: sudo RSYNC_EXPECT_SKIPPED=crtimes make check30
- name: check29
run: sudo RSYNC_EXPECT_SKIPPED=crtimes make check29
- name: ssl file list
run: rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v4
with:
name: ubuntu-22.04-bin
path: |
rsync
rsync-ssl
rsync.1
rsync-ssl.1
rsyncd.conf.5
rrsync.1
rrsync

View File

@@ -7,6 +7,7 @@ on:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-build.yml'
pull_request:
branches: [ master ]
paths-ignore:
- '.github/workflows/*.yml'
- '!.github/workflows/ubuntu-build.yml'

1
.gitignore vendored
View File

@@ -58,3 +58,4 @@ aclocal.m4
/auto-build-save
.deps
/*.exe
*.dSYM/

View File

@@ -49,7 +49,7 @@ OBJS2=options.o io.o compat.o hlink.o token.o uidlist.o socket.o hashtable.o \
usage.o fileio.o batch.o clientname.o chmod.o acls.o xattrs.o
OBJS3=progress.o pipe.o @MD5_ASM@ @ROLL_SIMD@ @ROLL_ASM@
DAEMON_OBJ = params.o loadparm.o clientserver.o access.o connection.o authenticate.o
popt_OBJS=popt/findme.o popt/popt.o popt/poptconfig.o \
popt_OBJS= popt/popt.o popt/poptconfig.o \
popt/popthelp.o popt/poptparse.o popt/poptint.o
OBJS=$(OBJS1) $(OBJS2) $(OBJS3) $(DAEMON_OBJ) $(LIBOBJ) @BUILD_ZLIB@ @BUILD_POPT@
@@ -57,13 +57,13 @@ TLS_OBJ = tls.o syscall.o util2.o t_stub.o lib/compat.o lib/snprintf.o lib/perms
# Programs we must have to run the test cases
CHECK_PROGS = rsync$(EXEEXT) tls$(EXEEXT) getgroups$(EXEEXT) getfsdev$(EXEEXT) \
testrun$(EXEEXT) trimslash$(EXEEXT) t_unsafe$(EXEEXT) t_chmod_secure$(EXEEXT) \
t_secure_relpath$(EXEEXT) wildtest$(EXEEXT) simdtest$(EXEEXT)
testrun$(EXEEXT) trimslash$(EXEEXT) t_unsafe$(EXEEXT) wildtest$(EXEEXT) \
simdtest$(EXEEXT)
CHECK_SYMLINKS = testsuite/chown-fake.test testsuite/devices-fake.test testsuite/xattrs-hlink.test
# Objects for CHECK_PROGS to clean
CHECK_OBJS=tls.o testrun.o getgroups.o getfsdev.o t_stub.o t_unsafe.o t_chmod_secure.o t_secure_relpath.o trimslash.o wildtest.o
CHECK_OBJS=tls.o testrun.o getgroups.o getfsdev.o t_stub.o t_unsafe.o trimslash.o wildtest.o
# note that the -I. is needed to handle config.h when using VPATH
.c.o:
@@ -179,14 +179,6 @@ T_UNSAFE_OBJ = t_unsafe.o syscall.o util1.o util2.o t_stub.o lib/compat.o lib/sn
t_unsafe$(EXEEXT): $(T_UNSAFE_OBJ)
$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(T_UNSAFE_OBJ) $(LIBS)
T_CHMOD_SECURE_OBJ = t_chmod_secure.o syscall.o util1.o util2.o t_stub.o lib/compat.o lib/snprintf.o lib/wildmatch.o lib/permstring.o
t_chmod_secure$(EXEEXT): $(T_CHMOD_SECURE_OBJ)
$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(T_CHMOD_SECURE_OBJ) $(LIBS)
T_SECURE_RELPATH_OBJ = t_secure_relpath.o syscall.o util1.o util2.o t_stub.o lib/compat.o lib/snprintf.o lib/wildmatch.o lib/permstring.o
t_secure_relpath$(EXEEXT): $(T_SECURE_RELPATH_OBJ)
$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(T_SECURE_RELPATH_OBJ) $(LIBS)
.PHONY: conf
conf: configure.sh config.h.in
@@ -321,15 +313,15 @@ test: check
.PHONY: check
check: all $(CHECK_PROGS) $(CHECK_SYMLINKS)
rsync_bin=`pwd`/rsync$(EXEEXT) $(srcdir)/runtests.sh
$(srcdir)/runtests.py --rsync-bin=`pwd`/rsync$(EXEEXT)
.PHONY: check29
check29: all $(CHECK_PROGS) $(CHECK_SYMLINKS)
rsync_bin=`pwd`/rsync$(EXEEXT) $(srcdir)/runtests.sh --protocol=29
$(srcdir)/runtests.py --rsync-bin=`pwd`/rsync$(EXEEXT) --protocol=29
.PHONY: check30
check30: all $(CHECK_PROGS) $(CHECK_SYMLINKS)
rsync_bin=`pwd`/rsync$(EXEEXT) $(srcdir)/runtests.sh --protocol=30
$(srcdir)/runtests.py --rsync-bin=`pwd`/rsync$(EXEEXT) --protocol=30
wildtest.o: wildtest.c t_stub.o lib/wildmatch.c rsync.h config.h
wildtest$(EXEEXT): wildtest.o lib/compat.o lib/snprintf.o @BUILD_POPT@
@@ -358,7 +350,7 @@ testsuite/xattrs-hlink.test:
.PHONY: installcheck
installcheck: $(CHECK_PROGS) $(CHECK_SYMLINKS)
POSIXLY_CORRECT=1 TOOLDIR=`pwd` rsync_bin="$(bindir)/rsync$(EXEEXT)" srcdir="$(srcdir)" $(srcdir)/runtests.sh
$(srcdir)/runtests.py --rsync-bin="$(bindir)/rsync$(EXEEXT)" --srcdir="$(srcdir)" --tooldir=`pwd`
# TODO: Add 'dist' target; need to know which files will be included

146
NEWS.md
View File

@@ -1,3 +1,144 @@
# NEWS for rsync 3.4.2 (28 Apr 2026)
## Changes in this version:
### SECURITY RELATED:
Several security-relevant defects were reported and fixed since 3.4.1.
None were assigned a CVE — rsync's fork-per-connection design scopes
the impact of each of these to the attacker's own connection, which is
equivalent to the client closing the socket itself — but they are
fixed here as a matter of hygiene and to reduce the chances of a
future exploitable combination. Many thanks to the external
researchers who reported these issues.
- Fixed a signed integer overflow in the PROXY protocol v2 header
parser: a negative `len` field could bypass the size check and cause
a stack buffer overflow in `read_buf()`. Reported by John Walker of
ZeroPath.
- Fixed an invalid access to the files array. Reported by Calum
Hutton of Rapid7.
- Reject negative token values in the compressed-stream token
decoder; a negative value could cause callers to misinterpret a
missing data pointer as literal data. Reported by Will Sergeant.
- Fixed the element count passed to the xattr `qsort()` (see
https://www.openwall.com/lists/oss-security/2026/04/16/2).
- Fixed a buffer underflow in `clean_fname()`, and added a regression
test.
- Fixed an uninitialized `mul_one` in the AVX2 get_checksum1 path
(undefined behaviour), and added a SIMD-checksum self-test that
cross-checks SSE2, SSSE3 and AVX2 against the C reference on both
aligned and unaligned buffers.
- Fixed an uninitialized `buf1` on the first call to
`get_checksum2()` in the MD4 path (fixes #673).
- Zero all new memory from internal allocations: `my_alloc()` now uses
`calloc`, and `expand_item_list()` zeros the expanded portion after
`realloc`. This gives more predictable behaviour if stale or
uninitialised memory is ever accidentally read.
### BUG FIXES:
- Call `tzset()` before chroot so that log timestamps continue to
reflect the configured local timezone after the daemon chroots
(glibc needs `/etc/localtime`, which is unreachable post-chroot).
- Use the correct time when writing to the log file.
- Do not clear `DISPLAY` unconditionally.
- Fixed a Y2038 bug in `syscall.c` by replacing the `Int32x32To64`
macro (which truncates its arguments to 32 bits) with a plain
64-bit multiplication.
- Fixed ACL ID mapping for non-root users (closes #618).
- Fixed handling of objects with many xattrs on FreeBSD.
- Fixed `--open-noatime` not taking effect when opening regular
files: `O_NOATIME` is now also passed to `do_open_nofollow()`, which
has been used for regular files since the CVE fix "fixed symlink
race condition in sender".
- Ignore "directory has vanished" errors.
- Fixed the removal of multiple leading slashes.
- Added the missing `--dirs` long option.
- Fixed a segfault if `poptGetContext()` returns NULL (e.g. under
OOM) by not passing NULL to `poptReadDefaultConfig()`. Reported by
Ronnie Sahlberg; found with `malloc-fail-tester`.
- Fixed a build error on ia64 NonStop (which treats missing
prototypes as an error, not a warning).
- Fixed a flaky hardlinks test (fixes #735).
### ENHANCEMENTS:
- Added multi-threaded `zstd` compression, gated by a new
`--compress-threads=N` option, with validation and man-page
coverage.
- Documented the `temp dir` parameter in the rsyncd.conf man page
(fixes #820).
- Improved rendering of interior dashes in long-option names in
`md-convert` (perhaps fixes #686).
### PORTABILITY / BUILD:
- Fixed glibc 2.43 const-preserving overloads of `strtok()`,
`strchr()` etc. by declaring the affected locals with the right
constness. Contributed by Holger Hoffstätte.
- Converted the bundled zlib 1.2.8 from K&R-style function
definitions to ANSI prototypes, so it builds with clang 16+.
- Avoid using `bool` as an identifier; it is a keyword in C23.
- `configure.ac`: check for xattr functions in libc first and only
fall back to `-lattr`, avoiding spurious overlinking when `-lattr`
happens to be installed. Contributed by Eli Schwartz.
- Made the build reproducible by honouring `SOURCE_DATE_EPOCH` for
the manpage date.
- Removed obsolete `popt/findme.c` and `popt/findme.h` that upstream
popt 1.14 folded into `popt.c` (fixes #710). Contributed by Alan
Coopersmith.
### INTERNAL:
- Made many module-global variables `const` so they can live in
`.rodata` and enable additional compiler optimization.
### DEVELOPER RELATED:
- Replaced `runtests.sh` with `runtests.py`, a Python test runner
that supports `--valgrind` (with per-process log files so valgrind
output no longer interferes with output comparisons) and
`-j/--parallel` execution for roughly a 7× speed-up on typical
hardware.
- Added a SIMD checksum self-test and a `clean-fname-underflow`
regression test.
- Various CI fixes for macOS and Cygwin (including adding
`simd-checksum` to the expected-skipped lists on platforms without
SIMD), and tests now run on `ubuntu-latest`.
- removed support for the unmaintained rsync-patches archive
------------------------------------------------------------------------------
# NEWS for rsync 3.4.1 (16 Jan 2025)
Release 3.4.1 is a fix for regressions introduced in 3.4.0
@@ -19,6 +160,7 @@ Release 3.4.1 is a fix for regressions introduced in 3.4.0
- fix to permissions handling in the developer release script
------------------------------------------------------------------------------
# NEWS for rsync 3.4.0 (15 Jan 2025)
Release 3.4.0 is a security release that fixes a number of important vulnerabilities.
@@ -73,6 +215,7 @@ to develop and test fixes.
- added FreeBSD and Solaris CI builds
------------------------------------------------------------------------------
# NEWS for rsync 3.3.0 (6 Apr 2024)
## Changes in this version:
@@ -4837,8 +4980,9 @@ to develop and test fixes.
| RELEASE DATE | VER. | DATE OF COMMIT\* | PROTOCOL |
|--------------|--------|------------------|-------------|
| 28 Apr 2026 | 3.4.2 | | 32 |
| 16 Jan 2025 | 3.4.1 | | 32 |
| 15 Jan 2025 | 3.4.0 | | 32 |
| 15 Jan 2025 | 3.4.0 | 15 Jan 2025 | 32 |
| 06 Apr 2024 | 3.3.0 | | 31 |
| 20 Oct 2022 | 3.2.7 | | 31 |
| 09 Sep 2022 | 3.2.6 | | 31 |

2
acls.c
View File

@@ -697,7 +697,7 @@ static uint32 recv_acl_access(int f, uchar *name_follows_ptr)
static uchar recv_ida_entries(int f, ida_entries *ent)
{
uchar computed_mask_bits = 0;
int i, count = read_varint_bounded(f, 0, MAX_WIRE_ACL_COUNT, "ACL count");
int i, count = read_varint(f);
ent->idas = count ? new_array(id_access, count) : NULL;
ent->count = count;

View File

@@ -39,7 +39,7 @@ static int validate_backup_dir(void)
{
STRUCT_STAT st;
if (do_lstat_at(backup_dir_buf, &st) < 0) {
if (do_lstat(backup_dir_buf, &st) < 0) {
if (errno == ENOENT)
return 0;
rsyserr(FERROR, errno, "backup lstat %s failed", backup_dir_buf);
@@ -98,7 +98,7 @@ static BOOL copy_valid_path(const char *fname)
for ( ; b; name = b + 1, b = strchr(name, '/')) {
*b = '\0';
while (do_mkdir_at(backup_dir_buf, ACCESSPERMS) < 0) {
while (do_mkdir(backup_dir_buf, ACCESSPERMS) < 0) {
if (errno == EEXIST) {
val = validate_backup_dir();
if (val > 0)
@@ -197,7 +197,7 @@ static inline int link_or_rename(const char *from, const char *to,
if (IS_SPECIAL(stp->st_mode) || IS_DEVICE(stp->st_mode))
return 0; /* Use copy code. */
#endif
if (do_link_at(from, to) == 0) {
if (do_link(from, to) == 0) {
if (DEBUG_GTE(BACKUP, 1))
rprintf(FINFO, "make_backup: HLINK %s successful.\n", from);
return 2;
@@ -207,7 +207,7 @@ static inline int link_or_rename(const char *from, const char *to,
return 0;
}
#endif
if (do_rename_at(from, to) == 0) {
if (do_rename(from, to) == 0) {
if (stp->st_nlink > 1 && !S_ISDIR(stp->st_mode)) {
/* If someone has hard-linked the file into the backup
* dir, rename() might return success but do nothing! */
@@ -246,7 +246,7 @@ int make_backup(const char *fname, BOOL prefer_rename)
goto success;
if (errno == EEXIST || errno == EISDIR) {
STRUCT_STAT bakst;
if (do_lstat_at(buf, &bakst) == 0) {
if (do_lstat(buf, &bakst) == 0) {
int flags = get_del_for_flag(bakst.st_mode) | DEL_FOR_BACKUP | DEL_RECURSE;
if (delete_item(buf, bakst.st_mode, flags) != 0)
return 0;
@@ -277,7 +277,7 @@ int make_backup(const char *fname, BOOL prefer_rename)
/* Check to see if this is a device file, or link */
if ((am_root && preserve_devices && IS_DEVICE(file->mode))
|| (preserve_specials && IS_SPECIAL(file->mode))) {
if (do_mknod_at(buf, file->mode, sx.st.st_rdev) < 0)
if (do_mknod(buf, file->mode, sx.st.st_rdev) < 0)
rsyserr(FERROR, errno, "mknod %s failed", full_fname(buf));
else if (DEBUG_GTE(BACKUP, 1))
rprintf(FINFO, "make_backup: DEVICE %s successful.\n", fname);
@@ -294,7 +294,7 @@ int make_backup(const char *fname, BOOL prefer_rename)
}
ret = 2;
} else {
if (do_symlink_at(sl, buf) < 0)
if (do_symlink(sl, buf) < 0)
rsyserr(FERROR, errno, "link %s -> \"%s\"", full_fname(buf), sl);
else if (DEBUG_GTE(BACKUP, 1))
rprintf(FINFO, "make_backup: SYMLINK %s successful.\n", fname);

View File

@@ -75,7 +75,7 @@ static int *flag_ptr[] = {
NULL
};
static char *flag_name[] = {
static const char *const flag_name[] = {
"--recurse (-r)",
"--owner (-o)",
"--group (-g)",

View File

@@ -198,7 +198,7 @@ NORETURN void _exit_cleanup(int code, const char *file, int line)
switch_step++;
if (cleanup_fname)
do_unlink_at(cleanup_fname);
do_unlink(cleanup_fname);
if (exit_code)
kill_all(SIGUSR1);
if (cleanup_pid && cleanup_pid == getpid()) {

View File

@@ -30,7 +30,6 @@ extern int list_only;
extern int am_sender;
extern int am_server;
extern int am_daemon;
extern int am_chrooted;
extern int am_root;
extern int msgs2stderr;
extern int rsync_port;
@@ -39,7 +38,6 @@ extern int ignore_errors;
extern int preserve_xattrs;
extern int kluge_around_eof;
extern int munge_symlinks;
extern int use_secure_symlinks;
extern int open_noatime;
extern int sanitize_paths;
extern int numeric_ids;
@@ -985,7 +983,6 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
io_printf(f_out, "@ERROR: chroot failed\n");
return -1;
}
am_chrooted = 1;
module_chdir = module_dir;
}
@@ -1008,15 +1005,6 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
}
}
/* Enable secure symlink handling for any non-chrooted daemon module.
* This prevents TOCTOU race attacks where an attacker could switch a
* directory to a symlink between path validation and file open.
* Match the gate used by the do_*_at() wrappers in syscall.c
* (am_daemon && !am_chrooted) -- the protection has nothing to do
* with symlink munging, so a module configured with
* "munge symlinks = false" must still get the secure-open path. */
use_secure_symlinks = am_daemon && !am_chrooted;
if (gid_list.count) {
gid_t *gid_array = gid_list.items;
if (setgid(gid_array[0])) {
@@ -1312,28 +1300,6 @@ int start_daemon(int f_in, int f_out)
if (lp_proxy_protocol() && !read_proxy_protocol_header(f_in))
return -1;
/* Do reverse DNS lookup before chroot/setuid. The result is cached,
* so the later client_name() call will use this cached value. This
* ensures hostname-based ACLs work even when DNS is unavailable
* after chroot.
*
* "reverse lookup" can be set globally OR per-module, so we also
* scan each module: a deployment with "reverse lookup = no" in the
* global section but "reverse lookup = yes" in a specific module
* still triggers a post-chroot lookup at access-check time
* (rsync_module() in this file), which would also fail in the
* chroot and turn hostname-based deny rules into silent bypasses. */
{
int need_reverse = lp_reverse_lookup(-1);
int j, num_modules = lp_num_modules();
for (j = 0; !need_reverse && j < num_modules; j++) {
if (lp_reverse_lookup(j))
need_reverse = 1;
}
if (need_reverse)
(void)client_name(client_addr(f_in));
}
p = lp_daemon_chroot();
if (*p) {
log_init(0); /* Make use we've initialized syslog before chrooting. */
@@ -1342,19 +1308,6 @@ int start_daemon(int f_in, int f_out)
rsyserr(FLOG, errno, "daemon chroot(\"%s\") failed", p);
return -1;
}
/* Deliberately do NOT set am_chrooted here. am_chrooted
* gates the per-module symlink-race defenses
* (secure_relative_open() and the do_*_at() wrappers in
* syscall.c) and means "the kernel is enforcing path
* confinement at the module boundary". The daemon chroot
* confines path resolution to the daemon-chroot directory,
* not to any individual module path -- modules sharing the
* daemon chroot are still distinguishable filesystem
* subtrees and a sender-controlled symlink in module A
* could redirect a syscall to module B (or to other files
* inside the daemon chroot) without the per-module
* defenses. Leave am_chrooted=0 here so secure_relative_open()
* still fires for "use chroot = no" modules. */
if (chdir("/") < 0) {
rsyserr(FLOG, errno, "daemon chdir(\"/\") failed");
return -1;

View File

@@ -52,6 +52,7 @@ extern int need_messages_from_generator;
extern int delete_mode, delete_before, delete_during, delete_after;
extern int do_compression;
extern int do_compression_level;
extern int do_compression_threads;
extern int saw_stderr_opt;
extern int msgs2stderr;
extern char *shell_cmd;

View File

@@ -6,7 +6,7 @@
BEGIN {
heading = "/* DO NOT EDIT THIS FILE! It is auto-generated from a list of values in " ARGV[1] "! */\n\n"
sect = psect = defines = accessors = prior_ptype = ""
parms = "\nstatic struct parm_struct parm_table[] = {"
parms = "\nstatic const struct parm_struct parm_table[] = {"
comment_fmt = "\n/********** %s **********/\n"
tdstruct = "typedef struct {"
}

View File

@@ -98,7 +98,7 @@ static enum delret delete_dir_contents(char *fname, uint16 flags)
strlcpy(p, fp->basename, remainder);
if (!(fp->mode & S_IWUSR) && !am_root && fp->flags & FLAG_OWNED_BY_US)
do_chmod_at(fname, fp->mode | S_IWUSR);
do_chmod(fname, fp->mode | S_IWUSR);
/* Save stack by recursing to ourself directly. */
if (S_ISDIR(fp->mode)) {
if (delete_dir_contents(fname, flags | DEL_RECURSE) != DR_SUCCESS)
@@ -139,7 +139,7 @@ enum delret delete_item(char *fbuf, uint16 mode, uint16 flags)
}
if (flags & DEL_NO_UID_WRITE)
do_chmod_at(fbuf, mode | S_IWUSR);
do_chmod(fbuf, mode | S_IWUSR);
if (S_ISDIR(mode) && !(flags & DEL_DIR_IS_EMPTY)) {
/* This only happens on the first call to delete_item() since
@@ -160,7 +160,7 @@ enum delret delete_item(char *fbuf, uint16 mode, uint16 flags)
if (S_ISDIR(mode)) {
what = "rmdir";
ok = do_rmdir_at(fbuf) == 0;
ok = do_rmdir(fbuf) == 0;
} else {
if (make_backups > 0 && !(flags & DEL_FOR_BACKUP) && (backup_dir || !is_backup_file(fbuf))) {
what = "make_backup";

21
flist.c
View File

@@ -840,9 +840,9 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x
}
if (xflags & XMIT_MOD_NSEC)
#ifndef CAN_SET_NSEC
(void)read_varint_bounded(f, 0, MAX_WIRE_NSEC, "modtime_nsec");
(void)read_varint(f);
#else
modtime_nsec = read_varint_bounded(f, 0, MAX_WIRE_NSEC, "modtime_nsec");
modtime_nsec = read_varint(f);
else
modtime_nsec = 0;
#endif
@@ -861,19 +861,8 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x
#endif
}
#endif
if (!(xflags & XMIT_SAME_MODE)) {
if (!(xflags & XMIT_SAME_MODE))
mode = from_wire_mode(read_int(f));
/* Reject modes whose type bits are not one of the standard
* file types; otherwise garbage mode values propagate through
* the file-type checks below unpredictably. */
if (!S_ISREG(mode) && !S_ISDIR(mode) && !S_ISLNK(mode)
&& !S_ISCHR(mode) && !S_ISBLK(mode)
&& !S_ISFIFO(mode) && !S_ISSOCK(mode)) {
rprintf(FERROR, "invalid file mode 0%o for %s [%s]\n",
(unsigned)mode, lastname, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
}
if (atimes_ndx && !S_ISDIR(mode) && !(xflags & XMIT_SAME_ATIME)) {
atime = read_varlong(f, 4);
#if SIZEOF_TIME_T < SIZEOF_INT64
@@ -3178,8 +3167,8 @@ static void output_flist(struct file_list *flist)
} else
*uidbuf = '\0';
if (gid_ndx) {
static char parens[] = "(\0)\0\0\0";
char *pp = parens + (file->flags & FLAG_SKIP_GROUP ? 0 : 3);
static const char parens[] = "(\0)\0\0\0";
const char *pp = parens + (file->flags & FLAG_SKIP_GROUP ? 0 : 3);
snprintf(gidbuf, sizeof gidbuf, " gid=%s%u%s",
pp, F_GROUP(file), pp + 2);
} else

View File

@@ -229,13 +229,11 @@ static int read_delay_line(char *buf, int *flags_p)
*flags_p = 0;
if (sscanf(bp, "%x ", &mode) != 1) {
goto invalid_data;
invalid_data:
rprintf(FERROR, "ERROR: invalid data in delete-delay file.\n");
return -1;
}
past_space = strchr(bp, ' ');
if (!past_space) {
goto invalid_data;
}
past_space++;
past_space = strchr(bp, ' ') + 1;
len = j - read_pos - (past_space - bp) + 1; /* count the '\0' */
read_pos = j + 1;
@@ -249,10 +247,6 @@ static int read_delay_line(char *buf, int *flags_p)
memcpy(buf, past_space, len);
return mode;
invalid_data:
rprintf(FERROR, "ERROR: invalid data in delete-delay file.\n");
return -1;
}
static void do_delayed_deletions(char *delbuf)
@@ -990,7 +984,7 @@ static int try_dests_reg(struct file_struct *file, char *fname, int ndx,
if (find_exact_for_existing) {
if (alt_dest_type == LINK_DEST && real_st.st_dev == sxp->st.st_dev && real_st.st_ino == sxp->st.st_ino)
return -1;
if (do_unlink_at(fname) < 0 && errno != ENOENT)
if (do_unlink(fname) < 0 && errno != ENOENT)
goto got_nothing_for_ya;
}
#ifdef SUPPORT_HARD_LINKS
@@ -1118,7 +1112,7 @@ static int try_dests_non(struct file_struct *file, char *fname, int ndx,
&& !IS_SPECIAL(file->mode) && !IS_DEVICE(file->mode)
#endif
&& !S_ISDIR(file->mode)) {
if (do_link_at(cmpbuf, fname) < 0) {
if (do_link(cmpbuf, fname) < 0) {
rsyserr(FERROR_XFER, errno,
"failed to hard-link %s with %s",
cmpbuf, fname);
@@ -1321,7 +1315,7 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx,
}
}
if (relative_paths && !implied_dirs && file->mode != 0
&& do_stat_at(dn, &sx.st) < 0) {
&& do_stat(dn, &sx.st) < 0) {
if (dry_run)
goto parent_is_dry_missing;
if (make_path(fname, MKP_DROP_NAME | MKP_SKIP_SLASH) < 0) {
@@ -1433,7 +1427,7 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx,
&& (stype == FT_DIR
|| delete_item(fname, sx.st.st_mode, del_opts | DEL_FOR_DIR) != 0))
goto cleanup; /* Any errors get reported later. */
if (do_mkdir_at(fname, (file->mode|added_perms) & 0700) == 0)
if (do_mkdir(fname, (file->mode|added_perms) & 0700) == 0)
file->flags |= FLAG_DIR_CREATED;
goto cleanup;
}
@@ -1475,10 +1469,10 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx,
itemize(fnamecmp, file, ndx, statret, &sx,
statret ? ITEM_LOCAL_CHANGE : 0, 0, NULL);
}
if (real_ret != 0 && do_mkdir_at(fname,file->mode|added_perms) < 0 && errno != EEXIST) {
if (real_ret != 0 && do_mkdir(fname,file->mode|added_perms) < 0 && errno != EEXIST) {
if (!relative_paths || errno != ENOENT
|| make_path(fname, MKP_DROP_NAME | MKP_SKIP_SLASH) < 0
|| (do_mkdir_at(fname, file->mode|added_perms) < 0 && errno != EEXIST)) {
|| (do_mkdir(fname, file->mode|added_perms) < 0 && errno != EEXIST)) {
rsyserr(FERROR_XFER, errno,
"recv_generator: mkdir %s failed",
full_fname(fname));
@@ -1505,7 +1499,7 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx,
#ifdef HAVE_CHMOD
if (!am_root && (file->mode & S_IRWXU) != S_IRWXU && dir_tweaking) {
mode_t mode = file->mode | S_IRWXU;
if (do_chmod_at(fname, mode) < 0) {
if (do_chmod(fname, mode) < 0) {
rsyserr(FERROR_XFER, errno,
"failed to modify permissions on %s",
full_fname(fname));
@@ -1814,7 +1808,7 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx,
;
else if (quick_check_ok(FT_REG, fnamecmp, file, &sx.st)) {
if (partialptr) {
do_unlink_at(partialptr);
do_unlink(partialptr);
handle_partial_dir(partialptr, PDIR_DELETE);
}
set_file_attrs(fname, file, &sx, NULL, maybe_ATTRS_REPORT | maybe_ATTRS_ACCURATE_TIME);
@@ -1902,7 +1896,7 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx,
back_file = NULL;
goto cleanup;
}
if ((f_copy = do_open_at(backupptr, O_WRONLY | O_CREAT | O_TRUNC | O_EXCL, 0600)) < 0) {
if ((f_copy = do_open(backupptr, O_WRONLY | O_CREAT | O_TRUNC | O_EXCL, 0600)) < 0) {
rsyserr(FERROR_XFER, errno, "open %s", full_fname(backupptr));
unmake_file(back_file);
back_file = NULL;
@@ -2022,7 +2016,7 @@ int atomic_create(struct file_struct *file, char *fname, const char *slnk, const
if (slnk) {
#ifdef SUPPORT_LINKS
if (do_symlink_at(slnk, create_name) < 0) {
if (do_symlink(slnk, create_name) < 0) {
rsyserr(FERROR_XFER, errno, "symlink %s -> \"%s\" failed",
full_fname(create_name), slnk);
return 0;
@@ -2038,7 +2032,7 @@ int atomic_create(struct file_struct *file, char *fname, const char *slnk, const
return 0;
#endif
} else {
if (do_mknod_at(create_name, file->mode, rdev) < 0) {
if (do_mknod(create_name, file->mode, rdev) < 0) {
rsyserr(FERROR_XFER, errno, "mknod %s failed",
full_fname(create_name));
return 0;
@@ -2046,14 +2040,14 @@ int atomic_create(struct file_struct *file, char *fname, const char *slnk, const
}
if (!skip_atomic) {
if (do_rename_at(tmpname, fname) < 0) {
if (do_rename(tmpname, fname) < 0) {
char *full_tmpname = strdup(full_fname(tmpname));
if (full_tmpname == NULL)
out_of_memory("atomic_create");
rsyserr(FERROR_XFER, errno, "rename %s -> \"%s\" failed",
full_tmpname, full_fname(fname));
free(full_tmpname);
do_unlink_at(tmpname);
do_unlink(tmpname);
return 0;
}
}
@@ -2117,7 +2111,7 @@ static void touch_up_dirs(struct file_list *flist, int ndx)
continue;
fname = f_name(file, NULL);
if (fix_dir_perms)
do_chmod_at(fname, file->mode);
do_chmod(fname, file->mode);
if (need_retouch_dir_times) {
STRUCT_STAT st;
if (link_stat(fname, &st, 0) == 0 && mtime_differs(&st, file)) {
@@ -2152,8 +2146,6 @@ void check_for_finished_files(int itemizing, enum logcode code, int check_redo)
if (send_failed)
ndx = get_hlink_num();
flist = flist_for_ndx(ndx, "check_for_finished_files.1");
if (ndx < flist->ndx_start)
exit_cleanup(RERR_PROTOCOL);
file = flist->files[ndx - flist->ndx_start];
assert(file->flags & FLAG_HLINKED);
if (send_failed)
@@ -2182,8 +2174,6 @@ void check_for_finished_files(int itemizing, enum logcode code, int check_redo)
flist = cur_flist;
cur_flist = flist_for_ndx(ndx, "check_for_finished_files.2");
if (ndx < cur_flist->ndx_start)
exit_cleanup(RERR_PROTOCOL);
file = cur_flist->files[ndx - cur_flist->ndx_start];
if (solo_file)

View File

@@ -454,7 +454,7 @@ int hard_link_check(struct file_struct *file, int ndx, char *fname,
int hard_link_one(struct file_struct *file, const char *fname,
const char *oldname, int terse)
{
if (do_link_at(oldname, fname) < 0) {
if (do_link(oldname, fname) < 0) {
enum logcode code;
if (terse) {
if (!INFO_GTE(NAME, 1))

59
io.c
View File

@@ -117,7 +117,7 @@ static int active_filecnt = 0;
static OFF_T active_bytecnt = 0;
static int first_message = 1;
static char int_byte_extra[64] = {
static const char int_byte_extra[64] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* (00 - 3F)/4 */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* (40 - 7F)/4 */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* (80 - BF)/4 */
@@ -1090,9 +1090,6 @@ static void got_flist_entry_status(enum festatus status, int ndx)
{
struct file_list *flist = flist_for_ndx(ndx, "got_flist_entry_status");
if (ndx < flist->ndx_start)
exit_cleanup(RERR_PROTOCOL);
if (remove_source_files) {
active_filecnt--;
active_bytecnt -= F_LENGTH(flist->files[ndx - flist->ndx_start]);
@@ -1868,45 +1865,6 @@ int64 read_varlong(int f, uchar min_bytes)
return u.x;
}
/* Read an int32 and verify lo <= v <= hi. On out-of-range, abort with a
* protocol error naming "what". The bound is co-located with the read so it
* cannot be forgotten by a downstream user. */
int32 read_int_bounded(int f, int32 lo, int32 hi, const char *what)
{
int32 v = read_int(f);
if (v < lo || v > hi) {
rprintf(FERROR, "wire value %s out of range: %ld not in [%ld,%ld] [%s]\n",
what, (long)v, (long)lo, (long)hi, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
return v;
}
/* As read_int_bounded but for varint-encoded values. */
int32 read_varint_bounded(int f, int32 lo, int32 hi, const char *what)
{
int32 v = read_varint(f);
if (v < lo || v > hi) {
rprintf(FERROR, "wire value %s out of range: %ld not in [%ld,%ld] [%s]\n",
what, (long)v, (long)lo, (long)hi, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
return v;
}
/* Read a varint that will be used as a size_t. Rejects negative values
* (which would wrap to ~SIZE_MAX) and values exceeding the supplied max. */
size_t read_varint_size(int f, size_t max, const char *what)
{
int32 v = read_varint(f);
if (v < 0 || (size_t)v > max) {
rprintf(FERROR, "wire size %s out of range: %ld > %lu [%s]\n",
what, (long)v, (unsigned long)max, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
return (size_t)v;
}
int64 read_longint(int f)
{
#if SIZEOF_INT64 >= 8
@@ -2013,21 +1971,6 @@ void read_sum_head(int f, struct sum_struct *sum)
(long)sum->count, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
/* Guard against integer overflow in downstream allocations sized by
* count*element_size. my_alloc uses divide-not-multiply so it is
* already wraparound-safe, but checking here gives a clearer error
* and also covers the (size_t)count * xfer_sum_len arithmetic that
* is performed *before* reaching my_alloc. */
if (xfer_sum_len > 0 && (size_t)sum->count > SIZE_MAX / (size_t)xfer_sum_len) {
rprintf(FERROR, "Invalid checksum count %ld (too large) [%s]\n",
(long)sum->count, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
if ((size_t)sum->count > SIZE_MAX / sizeof(struct sum_buf)) {
rprintf(FERROR, "Invalid checksum count %ld (sum_buf overflow) [%s]\n",
(long)sum->count, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
sum->blength = read_int(f);
if (sum->blength < 0 || sum->blength > max_blength) {
rprintf(FERROR, "Invalid block length %ld [%s]\n",

View File

@@ -1 +1 @@
#define LATEST_YEAR "2025"
#define LATEST_YEAR "2026"

View File

@@ -197,7 +197,7 @@ void md5_update(md_context *ctx, const uchar *input, uint32 length)
memcpy(ctx->buffer + left, input, length);
}
static uchar md5_padding[CSUM_CHUNK] = { 0x80 };
static const uchar md5_padding[CSUM_CHUNK] = { 0x80 };
void md5_result(md_context *ctx, uchar digest[MD5_DIGEST_LEN])
{

View File

@@ -126,9 +126,18 @@ ssize_t sys_llistxattr(const char *path, char *list, size_t size)
unsigned char keylen;
ssize_t off, len = extattr_list_link(path, EXTATTR_NAMESPACE_USER, list, size);
if (len <= 0 || (size_t)len > size)
if (len <= 0 || size == 0)
return len;
if ((size_t)len >= size) {
/* FreeBSD extattr_list_xx() returns 'size' as 'len' in case there are
more data available, truncating the output, we solve this by signalling
ERANGE in case len == size so that the code in xattrs.c will retry with
a bigger buffer */
errno = ERANGE;
return -1;
}
/* FreeBSD puts a single-byte length before each string, with no '\0'
* terminator. We need to change this into a series of null-terminted
* strings. Since the size is the same, we can simply transform the
@@ -136,7 +145,7 @@ ssize_t sys_llistxattr(const char *path, char *list, size_t size)
for (off = 0; off < len; off += keylen + 1) {
keylen = ((unsigned char*)list)[off];
if (off + keylen >= len) {
/* Should be impossible, but kernel bugs happen! */
/* Should be impossible, but bugs happen! */
errno = EINVAL;
return -1;
}

View File

@@ -65,7 +65,7 @@ typedef enum {
struct enum_list {
int value;
char *name;
const char *name;
};
struct parm_struct {
@@ -73,7 +73,7 @@ struct parm_struct {
parm_type type;
parm_class class;
void *ptr;
struct enum_list *enum_list;
const struct enum_list *enum_list;
unsigned flags;
};
@@ -95,7 +95,7 @@ static item_list section_list = EMPTY_ITEM_LIST;
static int iSectionIndex = -1;
static BOOL bInGlobalSection = True;
static struct enum_list enum_syslog_facility[] = {
static const struct enum_list enum_syslog_facility[] = {
#ifdef LOG_AUTH
{ LOG_AUTH, "auth" },
#endif

12
log.c
View File

@@ -456,17 +456,11 @@ void rsyserr(enum logcode code, int errcode, const char *format, ...)
char buf[BIGPATHBUFLEN];
size_t len;
/* snprintf returns the would-have-been length on truncation, so
* each cumulative call must be guarded; if not, sizeof buf - len
* can underflow when promoted to size_t and the next call writes
* past the buffer. */
len = snprintf(buf, sizeof buf, RSYNC_NAME ": [%s] ", who_am_i());
if (len < sizeof buf) {
va_start(ap, format);
len += vsnprintf(buf + len, sizeof buf - len, format, ap);
va_end(ap);
}
va_start(ap, format);
len += vsnprintf(buf + len, sizeof buf - len, format, ap);
va_end(ap);
if (len < sizeof buf) {
len += snprintf(buf + len, sizeof buf - len,

29
main.c
View File

@@ -239,11 +239,11 @@ void write_del_stats(int f)
void read_del_stats(int f)
{
stats.deleted_files = read_varint_bounded(f, 0, MAX_WIRE_DEL_STAT, "deleted_files");
stats.deleted_files += stats.deleted_dirs = read_varint_bounded(f, 0, MAX_WIRE_DEL_STAT, "deleted_dirs");
stats.deleted_files += stats.deleted_symlinks = read_varint_bounded(f, 0, MAX_WIRE_DEL_STAT, "deleted_symlinks");
stats.deleted_files += stats.deleted_devices = read_varint_bounded(f, 0, MAX_WIRE_DEL_STAT, "deleted_devices");
stats.deleted_files += stats.deleted_specials = read_varint_bounded(f, 0, MAX_WIRE_DEL_STAT, "deleted_specials");
stats.deleted_files = read_varint(f);
stats.deleted_files += stats.deleted_dirs = read_varint(f);
stats.deleted_files += stats.deleted_symlinks = read_varint(f);
stats.deleted_files += stats.deleted_devices = read_varint(f);
stats.deleted_files += stats.deleted_specials = read_varint(f);
}
static void become_copy_as_user()
@@ -386,7 +386,7 @@ static void handle_stats(int f)
static void output_itemized_counts(const char *prefix, int *counts)
{
static char *labels[] = { "reg", "dir", "link", "dev", "special" };
static char *const labels[] = { "reg", "dir", "link", "dev", "special" };
char buf[1024], *pre = " (";
int j, len = 0;
int total = counts[0];
@@ -394,18 +394,9 @@ static void output_itemized_counts(const char *prefix, int *counts)
counts[0] -= counts[1] + counts[2] + counts[3] + counts[4];
for (j = 0; j < 5; j++) {
if (counts[j]) {
/* snprintf can return more than its size arg
* on truncation; keep len <= sizeof buf - 2 so
* the closing ')' and trailing NUL always
* have room and the next iteration's
* sizeof buf - len - 2 cannot underflow. */
if (len >= (int)sizeof buf - 2)
break;
len += snprintf(buf+len, sizeof buf - len - 2,
"%s%s: %s",
pre, labels[j], comma_num(counts[j]));
if (len > (int)sizeof buf - 2)
len = (int)sizeof buf - 2;
pre = ", ";
}
}
@@ -1568,10 +1559,6 @@ static int start_client(int argc, char *argv[])
shell_user = shell_machine;
shell_machine = p+1;
}
if (*shell_machine == '-') {
rprintf(FERROR, "Invalid remote host: hostnames may not start with '-'.\n");
exit_cleanup(RERR_SYNTAX);
}
}
if (DEBUG_GTE(CMD, 2)) {
@@ -1756,7 +1743,9 @@ int main(int argc,char *argv[])
our_gid = MY_GID();
am_root = our_uid == ROOT_UID;
unset_env_var("DISPLAY");
// DISPLAY should not be emptied unconditionally
if (!getenv("SSH_ASKPASS"))
unset_env_var("DISPLAY");
#if defined USE_OPENSSL && defined SET_OPENSSL_CONF
#define TO_STR2(x) #x

View File

@@ -120,6 +120,7 @@ TZ_RE = re.compile(r'^#define\s+MAINTAINER_TZ_OFFSET\s+(-?\d+(\.\d+)?)', re.M)
VAR_REF_RE = re.compile(r'\$\{(\w+)\}')
VERSION_RE = re.compile(r' (\d[.\d]+)[, ]')
BIN_CHARS_RE = re.compile(r'[\1-\7]+')
LONG_OPT_DASH_RE = re.compile(r'(--\w[-\w]+)')
SPACE_DOUBLE_DASH_RE = re.compile(r'\s--(\s)')
NON_SPACE_SINGLE_DASH_RE = re.compile(r'(^|\W)-')
WHITESPACE_RE = re.compile(r'\s')
@@ -247,6 +248,9 @@ def find_man_substitutions():
env_subs['date'] = time.strftime('%d %b %Y', time.gmtime(mtime + tz_offset)).lstrip('0')
if 'SOURCE_DATE_EPOCH' in os.environ:
env_subs['date'] = time.strftime('%d %b %Y', time.gmtime(int(os.environ.get('SOURCE_DATE_EPOCH', time.time()))))
def html_via_commonmark(txt):
return commonmark.HtmlRenderer().render(commonmark.Parser().parse(txt))
@@ -540,6 +544,7 @@ class TransformHtml(HTMLParser):
if st.in_pre:
html = htmlify(txt)
else:
txt = LONG_OPT_DASH_RE.sub(lambda x: x.group(1).replace('-', NBR_DASH[0]), txt)
txt = SPACE_DOUBLE_DASH_RE.sub(NBR_SPACE[0] + r'--\1', txt).replace('--', NBR_DASH[0]*2)
txt = NON_SPACE_SINGLE_DASH_RE.sub(r'\1' + NBR_DASH[0], txt)
html = htmlify(txt)

View File

@@ -86,6 +86,7 @@ int sparse_files = 0;
int preallocate_files = 0;
int do_compression = 0;
int do_compression_level = CLVL_NOT_SPECIFIED;
int do_compression_threads = 0; /*n = 0 use rsync thread, n >= 1 spawn n threads for compression */
int am_root = 0; /* 0 = normal, 1 = root, 2 = --super, -1 = --fake-super */
int am_server = 0;
int am_sender = 0;
@@ -113,20 +114,11 @@ int mkpath_dest_arg = 0;
int allow_inc_recurse = 1;
int xfer_dirs = -1;
int am_daemon = 0;
/* Set after a successful per-module chroot ("use chroot = yes") in
* clientserver.c. NOT set for the daemon-level "daemon chroot = /X"
* chroot: that confines path resolution to /X, but module paths
* /X/modA, /X/modB, etc. are not chroot boundaries, so the per-module
* symlink-race defenses (secure_relative_open() / do_*_at() in
* syscall.c, gated by `am_daemon && !am_chrooted`) must still fire
* even when the daemon is inside a daemon chroot. */
int am_chrooted = 0;
int connect_timeout = 0;
int keep_partial = 0;
int safe_symlinks = 0;
int copy_unsafe_links = 0;
int munge_symlinks = 0;
int use_secure_symlinks = 0;
int size_only = 0;
int daemon_bwlimit = 0;
int bwlimit = 0;
@@ -234,7 +226,7 @@ char *iconv_opt =
struct chmod_mode_struct *chmod_modes = NULL;
static const char *debug_verbosity[] = {
static const char *const debug_verbosity[] = {
/*0*/ NULL,
/*1*/ NULL,
/*2*/ "BIND,CMD,CONNECT,DEL,DELTASUM,DUP,FILTER,FLIST,ICONV",
@@ -245,7 +237,7 @@ static const char *debug_verbosity[] = {
#define MAX_VERBOSITY ((int)(sizeof debug_verbosity / sizeof debug_verbosity[0]) - 1)
static const char *info_verbosity[1+MAX_VERBOSITY] = {
static const char *const info_verbosity[1+MAX_VERBOSITY] = {
/*0*/ "NONREG",
/*1*/ "COPY,DEL,FLIST,MISC,NAME,STATS,SYMSAFE",
/*2*/ "BACKUP,MISC2,MOUNT,NAME2,REMOVE,SKIP",
@@ -483,7 +475,7 @@ static void parse_output_words(struct output_struct *words, short *levels, const
static void output_item_help(struct output_struct *words)
{
short *levels = words == info_words ? info_levels : debug_levels;
const char **verbosity = words == info_words ? info_verbosity : debug_verbosity;
const char *const*verbosity = words == info_words ? info_verbosity : debug_verbosity;
char buf[128], *opt, *fmt = "%-10s %s\n";
int j;
@@ -765,6 +757,8 @@ static struct poptOption long_options[] = {
{"skip-compress", 0, POPT_ARG_STRING, &skip_compress, 0, 0, 0 },
{"compress-level", 0, POPT_ARG_INT, &do_compression_level, 0, 0, 0 },
{"zl", 0, POPT_ARG_INT, &do_compression_level, 0, 0, 0 },
{"compress-threads", 0, POPT_ARG_INT, &do_compression_threads, 0, 0, 0 },
{"zt", 0, POPT_ARG_INT, &do_compression_threads, 0, 0, 0 },
{0, 'P', POPT_ARG_NONE, 0, 'P', 0, 0 },
{"progress", 0, POPT_ARG_VAL, &do_progress, 1, 0, 0 },
{"no-progress", 0, POPT_ARG_VAL, &do_progress, 0, 0, 0 },
@@ -853,7 +847,7 @@ static struct poptOption long_options[] = {
{0,0,0,0, 0, 0, 0}
};
static struct poptOption long_daemon_options[] = {
static const struct poptOption long_daemon_options[] = {
/* longName, shortName, argInfo, argPtr, value, descrip, argDesc */
{"address", 0, POPT_ARG_STRING, &bind_address, 0, 0, 0 },
{"bwlimit", 0, POPT_ARG_INT, &daemon_bwlimit, 0, 0, 0 },
@@ -2019,6 +2013,8 @@ int parse_arguments(int *argc_p, const char ***argv_p)
create_refuse_error(refused_compress);
goto cleanup;
}
if (do_compression_threads < 0)
do_compression_threads = 0;
}
#ifdef HAVE_SETVBUF

View File

@@ -1,6 +1,6 @@
Summary: A fast, versatile, remote (and local) file-copying tool
Name: rsync
Version: 3.4.1
Version: 3.4.2
%define fullversion %{version}
Release: 1
%define srcdir src
@@ -79,5 +79,5 @@ rm -rf $RPM_BUILD_ROOT
%dir /etc/rsync-ssl/certs
%changelog
* Thu Jan 16 2025 Rsync Project <rsync.project@gmail.com>
Released 3.4.1.
* Tue Apr 28 2026 Rsync Project <rsync.project@gmail.com>
Released 3.4.2.

View File

@@ -38,12 +38,16 @@ def main():
if not os.path.isfile('packaging/release-rsync'):
die('You must run this script from the top of your rsync checkout.')
now = datetime.now()
now = datetime.now().astimezone() # Requires python 3.6 or later
cl_today = now.strftime('* %a %b %d %Y')
year = now.strftime('%Y')
ztoday = now.strftime('%d %b %Y')
today = ztoday.lstrip('0')
# The MAINTAINER_TZ_OFFSET is a float number of hours vs UTC. It can start with '-' but not '+'.
tz_now = now.strftime('%z')
tz_num = tz_now[0:1].replace('+', '') + str(float(tz_now[1:3]) + float(tz_now[3:]) / 60)
curdir = os.getcwd()
signal.signal(signal.SIGINT, signal_handler)
@@ -81,10 +85,6 @@ def main():
die('"a" must not exist in the current directory.')
if os.path.lexists('b'):
die('"b" must not exist in the current directory.')
if os.path.lexists('patches.gen'):
die('"patches.gen" must not exist in the current directory.')
check_git_state(args.master_branch, True, 'patches')
curversion = get_rsync_version()
@@ -120,8 +120,6 @@ def main():
if not re.match(r'^del', ans, flags=re.I):
die("Aborted")
cmd_chk(['git', 'tag', '-d', v_ver])
if os.path.isdir('patches/.git'):
cmd_chk(f"cd patches && git tag -d '{v_ver}'")
version = re.sub(r'[-.]*pre[-.]*', 'pre', version)
if 'pre' in version and not curversion.endswith('dev'):
@@ -213,6 +211,9 @@ About to:
x_re = re.compile(r'^(#define RSYNC_VERSION).*', re.M)
msg = f"Unable to update RSYNC_VERSION in {fn}"
txt = replace_or_die(x_re, r'\1 "%s"' % version, txt, msg)
x_re = re.compile(r'^(#define MAINTAINER_TZ_OFFSET).*', re.M)
msg = f"Unable to update MAINTAINER_TZ_OFFSET in {fn}"
txt = replace_or_die(x_re, r'\1 ' + tz_num, txt, msg)
elif '.spec' in fn:
for var, val in specvars.items():
x_re = re.compile(r'^%s .*' % re.escape(var), re.M)
@@ -220,7 +221,7 @@ About to:
x_re = re.compile(r'^\* \w\w\w \w\w\w \d\d \d\d\d\d (.*)', re.M)
txt = replace_or_die(x_re, r'%s \1' % cl_today, txt, f"Unable to update ChangeLog header in {fn}")
elif fn == 'rsync.h':
x_re = re.compile('(#define\s+SUBPROTOCOL_VERSION)\s+(\d+)')
x_re = re.compile(r'(#define\s+SUBPROTOCOL_VERSION)\s+(\d+)')
repl = lambda m: m[1] + ' ' + ('0' if not pre or not proto_changed else '1' if m[2] == '0' else m[2])
txt = replace_or_die(x_re, repl, txt, f"Unable to find SUBPROTOCOL_VERSION define in {fn}")
elif fn == 'NEWS.md':
@@ -252,7 +253,6 @@ About to:
cmd_run("git diff".split())
srctar_name = f"{rsync_ver}.tar.gz"
pattar_name = f"rsync-patches-{version}.tar.gz"
diff_name = f"{rsync_lastver}-{version}.diffs.gz"
srctar_file = os.path.join(dest, srcdir, srctar_name)
pattar_file = os.path.join(dest, srcdir, pattar_name)
@@ -266,8 +266,6 @@ About to:
- git commit all changes
- run a full build, ensuring that the manpages & configure.sh are up-to-date
- merge the {args.master_branch} branch into the patch/{args.master_branch}/* branches
- update the files in the "patches" dir and OPTIONALLY (if you type 'y') to
run patch-update with the --make option (which opens a shell on error)
""")
ans = input("<Press Enter OR 'y' to continue> ")
@@ -282,20 +280,10 @@ About to:
if s.returncode:
die('Aborting')
print('Updating files in "patches" dir ...')
s = cmd_run(f'packaging/patch-update --branch={args.master_branch}')
if s.returncode:
die('Aborting')
if re.match(r'^y', ans, re.I):
print(f'\nRunning smart-make on all "patch/{args.master_branch}/*" branches ...')
cmd_run(f"packaging/patch-update --branch={args.master_branch} --skip-check --make")
if os.path.isdir('patches/.git'):
s = cmd_run(f"cd patches && git commit -a -m 'The patches for {version}.'")
if s.returncode:
die('Aborting')
print(f"""\
{dash_line}
@@ -303,8 +291,6 @@ About to:
- create signed tag for this release: {v_ver}
- create release diffs, "{diff_name}"
- create release tar, "{srctar_name}"
- generate {rsync_ver}/patches/* files
- create patches tar, "{pattar_name}"
- update top-level README.md, NEWS.md, TODO, and ChangeLog
- update top-level rsync*.html manpages
- gpg-sign the release files
@@ -320,12 +306,6 @@ About to:
if 'bad passphrase' in out or 'failed' in out:
die('Aborting')
if os.path.isdir('patches/.git'):
out = cmd_txt(f"cd patches && git tag -s -m 'Version {version}.' {v_ver}", capture='combined').out
print(out, end='')
if 'bad passphrase' in out or 'failed' in out:
die('Aborting')
os.environ['PATH'] = ORIGINAL_PATH
# Extract the generated files from the old tar.
@@ -347,15 +327,6 @@ About to:
cmd_chk(['fakeroot', 'tar', 'czf', srctar_file, '--exclude=.github', rsync_ver])
shutil.rmtree(rsync_ver)
print(f'Updating files in "{rsync_ver}/patches" dir ...')
os.mkdir(rsync_ver, 0o755)
os.mkdir(f"{rsync_ver}/patches", 0o755)
cmd_chk(f"packaging/patch-update --skip-check --branch={args.master_branch} --gen={rsync_ver}/patches".split())
print(f"Creating {pattar_file} ...")
cmd_chk(['fakeroot', 'tar', 'chzf', pattar_file, rsync_ver + '/patches'])
shutil.rmtree(rsync_ver)
print(f"Updating the other files in {dest} ...")
md_files = 'README.md NEWS.md INSTALL.md'.split()
html_files = [ fn for fn in gen_pathnames if fn.endswith('.html') ]

703
packaging/release.py Executable file
View File

@@ -0,0 +1,703 @@
#!/usr/bin/env python3
# Step-based release script for rsync. Each step is a separate invocation
# selected by a --step-N-XX option, so the maintainer drives the release
# manually one piece at a time.
#
# All persistent state and working files live in ../release/ (a sibling of
# the rsync git checkout):
#
# ../release/rsync-ftp/ mirror of samba.org:/home/ftp/pub/rsync
# ../release/rsync-html/ git checkout of rsync-web (the html site)
# ../release/work/ scratch space for tarball / diff staging
# ../release/release-state.json info shared between steps
#
# The rsync-patches archive is no longer maintained and has been dropped.
#
# Run "packaging/release.py --list" to see the step list.
import os, sys, re, argparse, glob, shutil, json, signal, subprocess
from datetime import datetime
sys.path = ['packaging'] + sys.path
from pkglib import (
warn, die, cmd_run, cmd_chk, cmd_txt, cmd_txt_chk, cmd_pipe,
check_git_state, get_rsync_version,
get_NEWS_version_info, get_protocol_versions,
)
# ---------- Paths ----------
RELEASE_DIR = os.path.realpath('../release')
FTP_DIR = os.path.join(RELEASE_DIR, 'rsync-ftp')
HTML_DIR = os.path.join(RELEASE_DIR, 'rsync-html')
WORK_DIR = os.path.join(RELEASE_DIR, 'work')
STATE_FILE = os.path.join(RELEASE_DIR, 'release-state.json')
# Local rsync-web checkout (sibling of rsync-git) is the source-of-truth for
# the git-tracked html content. The maintainer pulls/commits/pushes there;
# step-1-fetch just snapshots it into HTML_DIR for the release flow.
HTML_SRC = os.path.realpath('../rsync-web')
FTP_REMOTE_PATH = '/home/ftp/pub/rsync'
HTML_REMOTE_PATH = '/home/httpd/html/rsync'
# Files that ./configure + make produce and that the release tarball / diff
# need to bundle alongside the git-tracked source. Mirrors the GENFILES
# definition in Makefile.in (with rrsync.1{,.html} since we always configure
# --with-rrsync in --step-4-build).
GEN_FILES = [
'configure.sh',
'aclocal.m4',
'config.h.in',
'rsync.1', 'rsync.1.html',
'rsync-ssl.1', 'rsync-ssl.1.html',
'rsyncd.conf.5', 'rsyncd.conf.5.html',
'rrsync.1', 'rrsync.1.html',
]
# ---------- Step registry ----------
STEPS = [
('step-1-fetch', 'mirror ../release/rsync-ftp from samba.org and snapshot ../release/rsync-html from ../rsync-web'),
('step-2-prepare', 'gather release info interactively and write release-state.json'),
('step-3-tweak', 'update version.h, rsync.h, NEWS.md, and packaging/*.spec'),
('step-4-build', 'run smart-make + make gen'),
('step-5-commit', 'git commit -a (commit the prepared release changes)'),
('step-6-tag', 'create the gpg-signed git tag'),
('step-7-tarball', 'build the source tarball and diffs.gz against the previous release'),
('step-8-update-ftp', 'refresh README/NEWS/INSTALL/html in the ftp dir, regen ChangeLog.gz, gpg-sign tarballs'),
('step-9-toplinks', 'hard-link top-level release files (final releases only)'),
('step-10-push-ftp', 'rsync ../release/rsync-ftp/ to samba.org'),
('step-11-push-html', 'rsync ../release/rsync-html/ to samba.org (after any manual edits)'),
('step-12-push-git', 'print the git push commands for you to run'),
]
STEP_FLAGS = [s[0] for s in STEPS]
DASH_LINE = '=' * 74
# ---------- State helpers ----------
def load_state():
if not os.path.isfile(STATE_FILE):
die(f"{STATE_FILE} not found. Run --step-2-prepare first.")
with open(STATE_FILE, 'r', encoding='utf-8') as fh:
return json.load(fh)
def save_state(state):
os.makedirs(RELEASE_DIR, exist_ok=True)
with open(STATE_FILE, 'w', encoding='utf-8') as fh:
json.dump(state, fh, indent=2, sort_keys=True)
fh.write('\n')
def require_samba_host():
host = os.environ.get('RSYNC_SAMBA_HOST', '')
if not host.endswith('.samba.org'):
die("Set RSYNC_SAMBA_HOST in your environment to the samba hostname (e.g. hr3.samba.org).")
return host
def require_top_of_checkout():
if not os.path.isfile('packaging/release.py'):
die("Run this script from the top of your rsync checkout.")
if not os.path.isdir('.git'):
die("There is no .git dir in the current directory.")
def replace_or_die(regex, repl, txt, die_msg):
m = regex.search(txt)
if not m:
die(die_msg)
return regex.sub(repl, txt, 1)
def section(title):
print(f"\n{DASH_LINE}\n== {title}\n{DASH_LINE}")
def confirm(prompt, default_no=True):
suffix = '[n] ' if default_no else '[y] '
ans = input(f"{prompt} {suffix}").strip().lower()
if default_no:
return ans.startswith('y')
return ans == '' or ans.startswith('y')
# ---------- Step 1: fetch ftp + html ----------
def step_1_fetch(args):
host = require_samba_host()
os.makedirs(RELEASE_DIR, exist_ok=True)
os.makedirs(WORK_DIR, exist_ok=True)
section(f"Fetching ftp dir into {FTP_DIR}")
if not os.path.isdir(FTP_DIR):
os.makedirs(FTP_DIR)
# The .filt file lives in the ftp dir on the server; mirror down using the
# transmitted filter, falling back to no filter on the very first pull.
filt = os.path.join(FTP_DIR, '.filt')
if os.path.exists(filt):
opts = ['-aivOHP', f'-f:_{filt}']
else:
opts = ['-aivOHP']
cmd_chk(['rsync', *opts, f'{host}:{FTP_REMOTE_PATH}/', f'{FTP_DIR}/'])
section(f"Snapshotting html dir from {HTML_SRC} into {HTML_DIR}")
if not os.path.isdir(HTML_SRC):
die(f"{HTML_SRC} not found. Clone the rsync-web repo there first.")
if not os.path.isdir(os.path.join(HTML_SRC, '.git')):
die(f"{HTML_SRC} exists but is not a git checkout.")
print(f"(Make sure {HTML_SRC} is up to date — this script does not 'git pull' for you.)")
os.makedirs(HTML_DIR, exist_ok=True)
cmd_chk(['rsync', '-aiv', '--exclude=/.git',
f'{HTML_SRC}/', f'{HTML_DIR}/'])
# Then mirror non-git html content from the server (mirroring samba-rsync's
# behavior: skip files that the html git already provides).
filt = os.path.join(HTML_DIR, 'filt')
if os.path.exists(filt):
tmp_filt = os.path.join(HTML_DIR, 'tmp-filt')
cmd_chk(f"sed -n -e 's/[-P]/H/p' '{filt}' >'{tmp_filt}'")
cmd_chk(['rsync', '-aivOHP', f'-f._{tmp_filt}',
f'{host}:{HTML_REMOTE_PATH}/', f'{HTML_DIR}/'])
os.unlink(tmp_filt)
print(f"\nFetch complete. Local dirs are now in {RELEASE_DIR}.")
# ---------- Step 2: prepare ----------
def step_2_prepare(args):
require_top_of_checkout()
os.makedirs(RELEASE_DIR, exist_ok=True)
if not os.path.isdir(FTP_DIR):
die(f"{FTP_DIR} does not exist. Run --step-1-fetch first.")
now = datetime.now().astimezone()
cl_today = now.strftime('* %a %b %d %Y')
year = now.strftime('%Y')
ztoday = now.strftime('%d %b %Y')
today = ztoday.lstrip('0')
tz_now = now.strftime('%z')
tz_num = tz_now[0:1].replace('+', '') + str(float(tz_now[1:3]) + float(tz_now[3:]) / 60)
curversion = get_rsync_version()
lastversion, last_protocol_version, pdate = get_NEWS_version_info()
protocol_version, subprotocol_version = get_protocol_versions()
# Default next version: bump preN, or move dev -> pre1.
version = curversion
m = re.search(r'pre(\d+)', version)
if m:
version = re.sub(r'pre\d+', 'pre' + str(int(m[1]) + 1), version)
else:
version = version.replace('dev', 'pre1')
print(f"\nCurrent version (version.h): {curversion}")
print(f"Last released version (NEWS.md): {lastversion}")
print(f"Current protocol version: {protocol_version} (last released: {last_protocol_version})")
ans = input(f"\nVersion to release [{version}, '.' to drop the preN suffix]: ").strip()
if ans == '.':
version = re.sub(r'pre\d+', '', version)
elif ans:
version = ans
if not re.match(r'^[\d.]+(pre\d+)?$', version):
die(f'Invalid version: "{version}"')
version = re.sub(r'[-.]*pre[-.]*', 'pre', version)
if 'pre' in version and not curversion.endswith('dev'):
lastversion = curversion
ans = input(f"Previous version to diff against [{lastversion}]: ").strip()
if ans:
lastversion = ans
lastversion = re.sub(r'[-.]*pre[-.]*', 'pre', lastversion)
m = re.search(r'(pre\d+)', version)
pre = m[1] if m else ''
finalversion = re.sub(r'pre\d+', '', version)
release = '0.1' if pre else '1'
ans = input(f"RPM release number [{release}]: ").strip()
if ans:
release = ans
if pre:
release += '.' + pre
proto_changed = protocol_version != last_protocol_version
if proto_changed:
if finalversion in pdate:
proto_change_date = pdate[finalversion]
else:
while True:
ans = input(f"Date the protocol changed to {protocol_version} (dd Mmm yyyy): ").strip()
if re.match(r'^\d\d \w\w\w \d\d\d\d$', ans):
break
proto_change_date = ans
else:
proto_change_date = ' ' * 11
if 'pre' in lastversion:
if not pre:
die("Refusing to diff a release version against a pre-release version.")
srcdir = srcdiffdir = lastsrcdir = 'src-previews'
elif pre:
srcdir = srcdiffdir = 'src-previews'
lastsrcdir = 'src'
else:
srcdir = lastsrcdir = 'src'
srcdiffdir = 'src-diffs'
state = {
'version': version,
'lastversion': lastversion,
'finalversion': finalversion,
'pre': pre,
'release': release,
'protocol_version': protocol_version,
'subprotocol_version': subprotocol_version,
'proto_changed': proto_changed,
'proto_change_date': proto_change_date,
'srcdir': srcdir,
'srcdiffdir': srcdiffdir,
'lastsrcdir': lastsrcdir,
'today': today,
'ztoday': ztoday,
'cl_today': cl_today,
'year': year,
'tz_num': tz_num,
'master_branch': args.master_branch,
}
save_state(state)
section("Release info")
for k in ('version', 'lastversion', 'release', 'srcdir', 'srcdiffdir', 'lastsrcdir',
'protocol_version', 'proto_changed', 'proto_change_date'):
print(f" {k}: {state[k]}")
print(f"\nWrote {STATE_FILE}. Re-run --step-2-prepare to change anything.")
# ---------- Step 3: tweak version files ----------
def step_3_tweak(args):
require_top_of_checkout()
state = load_state()
version = state['version']
finalversion = state['finalversion']
pre = state['pre']
release = state['release']
today = state['today']
ztoday = state['ztoday']
cl_today = state['cl_today']
year = state['year']
tz_num = state['tz_num']
proto_changed = state['proto_changed']
proto_change_date = state['proto_change_date']
protocol_version = state['protocol_version']
srcdir = state['srcdir']
specvars = {
'Version:': finalversion,
'Release:': release,
'%define fullversion': f'%{{version}}{pre}',
'Released': version + '.',
'%define srcdir': srcdir,
}
tweak_files = ['version.h', 'rsync.h', 'NEWS.md']
tweak_files += glob.glob('packaging/*.spec')
tweak_files += glob.glob('packaging/*/*.spec')
for fn in tweak_files:
with open(fn, 'r', encoding='utf-8') as fh:
old_txt = txt = fh.read()
if fn == 'version.h':
x_re = re.compile(r'^(#define RSYNC_VERSION).*', re.M)
txt = replace_or_die(x_re, r'\1 "%s"' % version, txt,
f"Unable to update RSYNC_VERSION in {fn}")
x_re = re.compile(r'^(#define MAINTAINER_TZ_OFFSET).*', re.M)
txt = replace_or_die(x_re, r'\1 ' + tz_num, txt,
f"Unable to update MAINTAINER_TZ_OFFSET in {fn}")
elif fn == 'rsync.h':
x_re = re.compile(r'(#define\s+SUBPROTOCOL_VERSION)\s+(\d+)')
repl = lambda m: m[1] + ' ' + (
'0' if not pre or not proto_changed
else '1' if m[2] == '0'
else m[2])
txt = replace_or_die(x_re, repl, txt,
f"Unable to find SUBPROTOCOL_VERSION in {fn}")
elif fn == 'NEWS.md':
efv = re.escape(finalversion)
x_re = re.compile(
r'^# NEWS for rsync %s \(UNRELEASED\)\s+## Changes in this version:\n' % efv
+ r'(\n### PROTOCOL NUMBER:\s+- The protocol number was changed to \d+\.\n)?')
rel_day = 'UNRELEASED' if pre else today
repl = (f'# NEWS for rsync {finalversion} ({rel_day})\n\n'
+ '## Changes in this version:\n')
if proto_changed:
repl += f'\n### PROTOCOL NUMBER:\n\n - The protocol number was changed to {protocol_version}.\n'
good_top = re.sub(r'\(.*?\)', '(UNRELEASED)', repl, 1)
msg = (f"The top of {fn} is not in the right format. It should be:\n" + good_top)
txt = replace_or_die(x_re, repl, txt, msg)
x_re = re.compile(
r'^(\| )(\S{2} \S{3} \d{4})(\s+\|\s+%s\s+\| ).{11}(\s+\| )\S{2}(\s+\|+)$' % efv,
re.M)
repl = lambda m: (m[1] + (m[2] if pre else ztoday) + m[3]
+ proto_change_date + m[4] + protocol_version + m[5])
txt = replace_or_die(x_re, repl, txt,
f'Unable to find "| ?? ??? {year} | {finalversion} | ... |" line in {fn}')
elif '.spec' in fn:
for var, val in specvars.items():
x_re = re.compile(r'^%s .*' % re.escape(var), re.M)
txt = replace_or_die(x_re, var + ' ' + val, txt,
f"Unable to update {var} in {fn}")
x_re = re.compile(r'^\* \w\w\w \w\w\w \d\d \d\d\d\d (.*)', re.M)
txt = replace_or_die(x_re, r'%s \1' % cl_today, txt,
f"Unable to update ChangeLog header in {fn}")
else:
die(f"Unrecognized file in tweak_files: {fn}")
if txt != old_txt:
print(f"Updating {fn}")
with open(fn, 'w', encoding='utf-8') as fh:
fh.write(txt)
cmd_chk(['packaging/year-tweak'])
section("git diff after tweaks")
cmd_run(['git', '--no-pager', 'diff'])
# ---------- Step 4: build ----------
def step_4_build(args):
require_top_of_checkout()
load_state() # just to ensure we've prepared
section("Running prepare-source + configure --prefix=/usr --with-rrsync + make + make gen")
# Always re-prepare so configure.sh is current; we run configure ourselves
# with the release-required flags rather than relying on the cached
# config.status (which may have been produced with different options).
if os.path.isfile('.fetch'):
cmd_chk(['./prepare-source', 'fetch'])
else:
cmd_chk(['./prepare-source'])
cmd_chk(['./configure', '--prefix=/usr', '--with-rrsync'])
cmd_chk(['make'])
cmd_chk(['make', 'gen'])
# ---------- Step 5: commit ----------
def step_5_commit(args):
require_top_of_checkout()
state = load_state()
version = state['version']
section("git status")
cmd_run(['git', 'status'])
if not confirm("Commit all current changes with the release message?"):
die("Aborted.")
cmd_chk(['git', 'commit', '-a', '-m', f'Preparing for release of {version} [buildall]'])
# ---------- Step 6: tag ----------
def step_6_tag(args):
require_top_of_checkout()
state = load_state()
version = state['version']
v_ver = 'v' + version
out = cmd_txt_chk(['git', 'tag', '-l', v_ver]).out
if out.strip():
if not confirm(f"Tag {v_ver} already exists. Delete and recreate?"):
die("Aborted.")
cmd_chk(['git', 'tag', '-d', v_ver])
# Prime the gpg agent so the actual tag signing won't prompt.
section("Priming gpg agent")
cmd_run("touch TeMp; gpg --sign TeMp; rm -f TeMp TeMp.gpg")
section(f"Creating signed tag {v_ver}")
out = cmd_txt(['git', 'tag', '-s', '-m', f'Version {version}.', v_ver],
capture='combined').out
print(out, end='')
if 'bad passphrase' in out.lower() or 'failed' in out.lower():
die("Tag creation failed.")
# ---------- Step 7: tarball + diff ----------
def step_7_tarball(args):
require_top_of_checkout()
state = load_state()
version = state['version']
lastversion = state['lastversion']
pre = state['pre']
srcdir = state['srcdir']
srcdiffdir = state['srcdiffdir']
lastsrcdir = state['lastsrcdir']
rsync_ver = 'rsync-' + version
rsync_lastver = 'rsync-' + lastversion
v_ver = 'v' + version
srctar_name = f"{rsync_ver}.tar.gz"
diff_name = f"{rsync_lastver}-{version}.diffs.gz"
srctar_file = os.path.join(FTP_DIR, srcdir, srctar_name)
diff_file = os.path.join(FTP_DIR, srcdiffdir, diff_name)
lasttar_file = os.path.join(FTP_DIR, lastsrcdir, rsync_lastver + '.tar.gz')
for d in (os.path.dirname(srctar_file), os.path.dirname(diff_file)):
os.makedirs(d, exist_ok=True)
if not os.path.isfile(lasttar_file):
die(f"Previous tarball not found: {lasttar_file}")
# Stage in ../release/work to keep the source checkout clean.
if os.path.isdir(WORK_DIR):
shutil.rmtree(WORK_DIR)
os.makedirs(WORK_DIR)
a_dir = os.path.join(WORK_DIR, 'a')
b_dir = os.path.join(WORK_DIR, 'b')
# Extract gen files from the previous tarball into work/a/.
tweaked_gen_files = [os.path.join(rsync_lastver, fn) for fn in GEN_FILES]
cmd_chk(['tar', '-C', WORK_DIR, '-xzf', lasttar_file, *tweaked_gen_files])
os.rename(os.path.join(WORK_DIR, rsync_lastver), a_dir)
# Copy current gen files (built in the top-level checkout) into work/b/.
os.makedirs(b_dir)
cmd_chk(['rsync', '-a', *GEN_FILES, b_dir + '/'])
section(f"Creating {diff_file}")
sed_script = r's:^((---|\+\+\+) [ab]/[^\t]+)\t.*:\1:' # no single quotes!
cmd_chk(
f"(git diff v{lastversion} {v_ver} -- ':!.github'; "
f"diff -upN {a_dir} {b_dir} | sed -r '{sed_script}') | gzip -9 >{diff_file}")
section(f"Creating {srctar_file}")
# Reuse work/b/ (which already holds the fresh gen files) as the release
# staging dir, then let "git archive" overlay the git-tracked source files
# on top. That way the tarball ends up with both gen files and source.
rsync_ver_dir = os.path.join(WORK_DIR, rsync_ver)
shutil.rmtree(a_dir)
os.rename(b_dir, rsync_ver_dir)
cmd_chk(f"git archive --format=tar --prefix={rsync_ver}/ {v_ver} | "
f"tar -C {WORK_DIR} -xf -")
cmd_chk(f"support/git-set-file-times --quiet --prefix={rsync_ver_dir}/")
cmd_chk(['fakeroot', 'tar', '-C', WORK_DIR, '-czf', srctar_file,
'--exclude=.github', rsync_ver])
# Leave staging in place; --step-8-update-ftp does its own thing.
print(f"\nCreated:\n {srctar_file}\n {diff_file}")
# ---------- Step 8: update ftp ----------
def step_8_update_ftp(args):
require_top_of_checkout()
state = load_state()
version = state['version']
lastversion = state['lastversion']
srcdir = state['srcdir']
srcdiffdir = state['srcdiffdir']
rsync_ver = 'rsync-' + version
rsync_lastver = 'rsync-' + lastversion
srctar_file = os.path.join(FTP_DIR, srcdir, f"{rsync_ver}.tar.gz")
diff_file = os.path.join(FTP_DIR, srcdiffdir,
f"{rsync_lastver}-{version}.diffs.gz")
section(f"Refreshing top-of-tree files in {FTP_DIR}")
md_files = ['README.md', 'NEWS.md', 'INSTALL.md']
html_files = [fn for fn in GEN_FILES if fn.endswith('.html')]
cmd_chk(['rsync', '-a', *md_files, *html_files, FTP_DIR + '/'])
cmd_chk(['./md-convert', '--dest', FTP_DIR, *md_files])
section(f"Regenerating {FTP_DIR}/ChangeLog.gz")
cmd_chk(f"git log --name-status | gzip -9 >{FTP_DIR}/ChangeLog.gz")
# Prime gpg agent and then sign the tar + diff.
section("Priming gpg agent")
cmd_run("touch TeMp; gpg --sign TeMp; rm -f TeMp TeMp.gpg")
for fn in (srctar_file, diff_file):
if not os.path.isfile(fn):
die(f"Missing file to sign: {fn}. Did --step-7-tarball run successfully?")
asc_fn = fn + '.asc'
if os.path.lexists(asc_fn):
os.unlink(asc_fn)
section(f"GPG-signing {fn}")
res = cmd_run(['gpg', '--batch', '-ba', fn])
if res.returncode not in (0, 2):
die("gpg signing failed.")
# ---------- Step 9: top-level hard links ----------
def step_9_toplinks(args):
require_top_of_checkout()
state = load_state()
pre = state['pre']
if pre:
print("Skipping: pre-releases do not get top-level hard links.")
return
version = state['version']
lastversion = state['lastversion']
srcdir = state['srcdir']
srcdiffdir = state['srcdiffdir']
rsync_ver = 'rsync-' + version
rsync_lastver = 'rsync-' + lastversion
srctar_file = os.path.join(FTP_DIR, srcdir, f"{rsync_ver}.tar.gz")
diff_file = os.path.join(FTP_DIR, srcdiffdir,
f"{rsync_lastver}-{version}.diffs.gz")
section("Removing stale top-level rsync-* files")
for find in [f'{FTP_DIR}/rsync-*.gz',
f'{FTP_DIR}/rsync-*.asc',
f'{FTP_DIR}/src-previews/rsync-*diffs.gz*']:
for fn in glob.glob(find):
os.unlink(fn)
top_link = [
srctar_file, srctar_file + '.asc',
diff_file, diff_file + '.asc',
]
for fn in top_link:
target = re.sub(r'/src(-\w+)?/', '/', fn)
if os.path.lexists(target):
os.unlink(target)
os.link(fn, target)
print(f" linked {target}")
# ---------- Step 10: push ftp ----------
def step_10_push_ftp(args):
host = require_samba_host()
if not os.path.isdir(FTP_DIR):
die(f"{FTP_DIR} does not exist. Run --step-1-fetch first.")
section(f"rsync ftp dir to {host}")
rsync_with_confirm(['-aivOHP', '--chown=:rsync', '--del',
f'-f._{os.path.join(FTP_DIR, ".filt")}',
f'{FTP_DIR}/', f'{host}:{FTP_REMOTE_PATH}/'])
# ---------- Step 11: push html ----------
def step_11_push_html(args):
host = require_samba_host()
if not os.path.isdir(HTML_DIR):
die(f"{HTML_DIR} does not exist. Run --step-1-fetch first.")
section(f"rsync html dir to {host}")
filt = os.path.join(HTML_DIR, 'filt')
rsync_with_confirm(['-aivOHP', '--chown=:rsync', '--del',
f'-f._{filt}',
f'{HTML_DIR}/', f'{host}:{HTML_REMOTE_PATH}/'])
# ---------- Step 12: print push-git instructions ----------
def step_12_push_git(args):
state = load_state()
version = state['version']
master_branch = state['master_branch']
v_ver = 'v' + version
print(f"""\
{DASH_LINE}
Run these from the rsync-git checkout (this script does not push for you):
git push origin {master_branch}
git push origin {v_ver}
If you have a 'samba' remote configured (git.samba.org:/data/git/rsync.git):
git push samba {master_branch}
git push samba {v_ver}
Then upload the tarball + .asc to the GitHub release for {v_ver}, run
packaging/send-news (when convenient), and announce on rsync-announce@,
rsync@, and Discord.
""")
# ---------- shared rsync-with-confirm ----------
def rsync_with_confirm(rsync_args):
"""Run an rsync command in dry-run mode, then ask before running for real."""
cmd_run(['rsync', '--dry-run', *rsync_args])
if confirm("Run without --dry-run?"):
cmd_run(['rsync', *rsync_args])
# ---------- dispatch ----------
STEP_FUNCS = {
'step-1-fetch': step_1_fetch,
'step-2-prepare': step_2_prepare,
'step-3-tweak': step_3_tweak,
'step-4-build': step_4_build,
'step-5-commit': step_5_commit,
'step-6-tag': step_6_tag,
'step-7-tarball': step_7_tarball,
'step-8-update-ftp': step_8_update_ftp,
'step-9-toplinks': step_9_toplinks,
'step-10-push-ftp': step_10_push_ftp,
'step-11-push-html': step_11_push_html,
'step-12-push-git': step_12_push_git,
}
def signal_handler(sig, frame):
die("\nAborting due to SIGINT.")
def main():
parser = argparse.ArgumentParser(
description="Step-based release script for rsync.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Run --list to see the steps. Each invocation runs exactly one --step-* option.")
parser.add_argument('--branch', '-b', dest='master_branch', default='master',
help="The branch to release (default: master).")
parser.add_argument('--list', action='store_true',
help="List all release steps and exit.")
grp = parser.add_mutually_exclusive_group()
for flag, descr in STEPS:
grp.add_argument('--' + flag, dest='step', action='store_const',
const=flag, help=descr)
args = parser.parse_args()
if args.list:
print("Release steps:")
for flag, descr in STEPS:
print(f" --{flag:18s} {descr}")
return
if not args.step:
parser.error("pick one --step-N-XX option (or --list to see them).")
signal.signal(signal.SIGINT, signal_handler)
os.environ['LESS'] = 'mqeiXR'
STEP_FUNCS[args.step](args)
if __name__ == '__main__':
main()
# vim: sw=4 et ft=python

View File

@@ -7,9 +7,6 @@
import sys, os, re, argparse, subprocess
from datetime import datetime
MAINTAINER_NAME = 'Wayne Davison'
MAINTAINER_SUF = ' ' + MAINTAINER_NAME + "\n"
def main():
latest_year = '2000'
@@ -22,10 +19,6 @@ def main():
m = argparse.Namespace(**m.groupdict())
if m.year > latest_year:
latest_year = m.year
if m.fn.startswith('zlib/') or m.fn.startswith('popt/'):
continue
if re.search(r'\.(c|h|sh|test)$', m.fn):
maybe_edit_copyright_year(m.fn, m.year)
proc.communicate()
fn = 'latest-year.h'
@@ -39,55 +32,8 @@ def main():
fh.write(txt)
def maybe_edit_copyright_year(fn, year):
opening_lines = [ ]
copyright_line = None
with open(fn, 'r', encoding='utf-8') as fh:
for lineno, line in enumerate(fh):
opening_lines.append(line)
if lineno > 3 and not re.search(r'\S', line):
break
m = re.match(r'^(?P<pre>.*Copyright\s+\S+\s+)(?P<year>\d\d\d\d(?:-\d\d\d\d)?(,\s+\d\d\d\d)*)(?P<suf>.+)', line)
if not m:
continue
copyright_line = argparse.Namespace(**m.groupdict())
copyright_line.lineno = len(opening_lines)
copyright_line.is_maintainer_line = MAINTAINER_NAME in copyright_line.suf
copyright_line.txt = line
if copyright_line.is_maintainer_line:
break
if not copyright_line:
return
if copyright_line.is_maintainer_line:
cyears = copyright_line.year.split('-')
if year == cyears[0]:
cyears = [ year ]
else:
cyears = [ cyears[0], year ]
txt = copyright_line.pre + '-'.join(cyears) + MAINTAINER_SUF
if txt == copyright_line.txt:
return
opening_lines[copyright_line.lineno - 1] = txt
else:
if fn.startswith('lib/') or fn.startswith('testsuite/'):
return
txt = copyright_line.pre + year + MAINTAINER_SUF
opening_lines[copyright_line.lineno - 1] += txt
remaining_txt = fh.read()
print(f"Updating {fn} with year {year}")
with open(fn, 'w', encoding='utf-8') as fh:
fh.write(''.join(opening_lines))
fh.write(remaining_txt)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Grab the year of last mod for our c & h files and make sure the Copyright comment is up-to-date.")
parser = argparse.ArgumentParser(description="Grab the year of the last mod for our c & h files and make sure the LATEST_YEAR value is accurate.")
args = parser.parse_args()
main()

View File

@@ -1,61 +0,0 @@
/** \ingroup popt
* \file popt/findme.c
*/
/* (C) 1998-2002 Red Hat, Inc. -- Licensing details are in the COPYING
file accompanying popt source distributions, available from
ftp://ftp.rpm.org/pub/rpm/dist. */
#include "system.h"
#include "findme.h"
const char * findProgramPath(const char * argv0)
{
char * path = getenv("PATH");
char * pathbuf;
char * start, * chptr;
char * buf;
size_t bufsize;
if (argv0 == NULL) return NULL; /* XXX can't happen */
/* If there is a / in the argv[0], it has to be an absolute path */
if (strchr(argv0, '/'))
return xstrdup(argv0);
if (path == NULL) return NULL;
bufsize = strlen(path) + 1;
start = pathbuf = malloc(bufsize);
if (pathbuf == NULL) return NULL; /* XXX can't happen */
strlcpy(pathbuf, path, bufsize);
bufsize += sizeof "/" - 1 + strlen(argv0);
buf = malloc(bufsize);
if (buf == NULL) {
free(pathbuf);
return NULL; /* XXX can't happen */
}
chptr = NULL;
/*@-branchstate@*/
do {
if ((chptr = strchr(start, ':')))
*chptr = '\0';
snprintf(buf, bufsize, "%s/%s", start, argv0);
if (!access(buf, X_OK)) {
free(pathbuf);
return buf;
}
if (chptr)
start = chptr + 1;
else
start = NULL;
} while (start && *start);
/*@=branchstate@*/
free(pathbuf);
free(buf);
return NULL;
}

View File

@@ -1,20 +0,0 @@
/** \ingroup popt
* \file popt/findme.h
*/
/* (C) 1998-2000 Red Hat, Inc. -- Licensing details are in the COPYING
file accompanying popt source distributions, available from
ftp://ftp.rpm.org/pub/rpm/dist. */
#ifndef H_FINDME
#define H_FINDME
/**
* Return absolute path to executable by searching PATH.
* @param argv0 name of executable
* @return (malloc'd) absolute path to executable (or NULL)
*/
/*@null@*/ const char * findProgramPath(/*@null@*/ const char * argv0)
/*@*/;
#endif

View File

@@ -70,7 +70,6 @@ extern int fuzzy_basis;
extern struct name_num_item *xfer_sum_nni;
extern int xfer_sum_len;
extern int use_secure_symlinks;
static struct bitbag *delayed_bits = NULL;
static int phase = 0, redoing = 0;
@@ -215,12 +214,7 @@ int open_tmpfile(char *fnametmp, const char *fname, struct file_struct *file)
* access to ensure that there is no race condition. They will be
* correctly updated after the right owner and group info is set.
* (Thanks to snabb@epipe.fi for pointing this out.) */
/* When use_secure_symlinks is on (non-chroot daemon with munge_symlinks),
* use secure_mkstemp to prevent symlink race attacks on parent directories. */
if (use_secure_symlinks)
fd = secure_mkstemp(fnametmp, (file->mode|added_perms) & INITACCESSPERMS);
else
fd = do_mkstemp(fnametmp, (file->mode|added_perms) & INITACCESSPERMS);
fd = do_mkstemp(fnametmp, (file->mode|added_perms) & INITACCESSPERMS);
#if 0
/* In most cases parent directories will already exist because their
@@ -318,12 +312,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
}
}
while (1) {
data = NULL;
i = recv_token(f_in, &data);
if (i == 0)
break;
while ((i = recv_token(f_in, &data)) != 0) {
if (INFO_GTE(PROGRESS, 1))
show_progress(offset, total_size);
@@ -331,10 +320,6 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
maybe_send_keepalive(time(NULL), MSK_ALLOW_FLUSH | MSK_ACTIVE_RECEIVER);
if (i > 0) {
if (!data) {
rprintf(FERROR, "Invalid literal token with no data [%s]\n", who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
if (DEBUG_GTE(DELTASUM, 3)) {
rprintf(FINFO,"data recv %d at %s\n",
i, big_num(offset));
@@ -352,11 +337,6 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
}
i = -(i+1);
if (i < 0 || i >= sum.count) {
rprintf(FERROR, "Invalid block index %d (count=%ld) [%s]\n",
i, (long)sum.count, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
offset2 = i * (OFF_T)sum.blength;
len = sum.blength;
if (i == (int)sum.count-1 && sum.remainder != 0)
@@ -456,7 +436,7 @@ static void handle_delayed_updates(char *local_name)
}
/* We don't use robust_rename() here because the
* partial-dir must be on the same drive. */
if (do_rename_at(partialptr, fname) < 0) {
if (do_rename(partialptr, fname) < 0) {
rsyserr(FERROR_XFER, errno,
"rename failed for %s (from %s)",
full_fname(fname), partialptr);
@@ -472,10 +452,7 @@ static void handle_delayed_updates(char *local_name)
static void no_batched_update(int ndx, BOOL is_redo)
{
struct file_list *flist = flist_for_ndx(ndx, "no_batched_update");
struct file_struct *file;
if (ndx < flist->ndx_start)
exit_cleanup(RERR_PROTOCOL);
file = flist->files[ndx - flist->ndx_start];
struct file_struct *file = flist->files[ndx - flist->ndx_start];
rprintf(FERROR_XFER, "(No batched update for%s \"%s\")\n",
is_redo ? " resend of" : "", f_name(file, NULL));
@@ -612,8 +589,6 @@ int recv_files(int f_in, int f_out, char *local_name)
if (ndx - cur_flist->ndx_start >= 0)
file = cur_flist->files[ndx - cur_flist->ndx_start];
else if (cur_flist->parent_ndx < 0)
exit_cleanup(RERR_PROTOCOL);
else
file = dir_flist->files[cur_flist->parent_ndx];
fname = local_name ? local_name : f_name(file, fbuf);
@@ -879,21 +854,11 @@ int recv_files(int f_in, int f_out, char *local_name)
/* We now check to see if we are writing the file "inplace" */
if (inplace || one_inplace) {
fnametmp = one_inplace ? partialptr : fname;
/* When use_secure_symlinks is on (non-chroot daemon),
* use secure open to prevent symlink race attacks where an
* attacker could switch a directory to a symlink between
* path validation and file open. */
if (use_secure_symlinks)
fd2 = secure_relative_open(NULL, fnametmp, O_WRONLY|O_CREAT, 0600);
else
fd2 = do_open(fnametmp, O_WRONLY|O_CREAT, 0600);
fd2 = do_open(fnametmp, O_WRONLY|O_CREAT, 0600);
#ifdef linux
if (fd2 == -1 && errno == EACCES) {
/* Maybe the error was due to protected_regular setting? */
if (use_secure_symlinks)
fd2 = secure_relative_open(NULL, fname, O_WRONLY, 0600);
else
fd2 = do_open(fname, O_WRONLY, 0600);
fd2 = do_open(fname, O_WRONLY, 0600);
}
#endif
if (fd2 == -1) {
@@ -945,7 +910,7 @@ int recv_files(int f_in, int f_out, char *local_name)
recv_ok = -1;
else if (fnamecmp == partialptr) {
if (!one_inplace)
do_unlink_at(partialptr);
do_unlink(partialptr);
handle_partial_dir(partialptr, PDIR_DELETE);
}
} else if (keep_partial && partialptr && (!one_inplace || delay_updates)) {
@@ -954,7 +919,7 @@ int recv_files(int f_in, int f_out, char *local_name)
"Unable to create partial-dir for %s -- discarding %s.\n",
local_name ? local_name : f_name(file, NULL),
recv_ok ? "completed file" : "partial file");
do_unlink_at(fnametmp);
do_unlink(fnametmp);
recv_ok = -1;
} else if (!finish_transfer(partialptr, fnametmp, fnamecmp, NULL,
file, recv_ok, !partial_dir))
@@ -965,7 +930,7 @@ int recv_files(int f_in, int f_out, char *local_name)
} else
partialptr = NULL;
} else if (!one_inplace)
do_unlink_at(fnametmp);
do_unlink(fnametmp);
cleanup_disable();

View File

@@ -513,6 +513,7 @@ has its own detailed description later in this manpage.
--compress, -z compress file data during the transfer
--compress-choice=STR choose the compression algorithm (aka --zc)
--compress-level=NUM explicitly set compression level (aka --zl)
--compress-threads=NUM explicitly set compression threads (aka --zt)
--skip-compress=LIST skip compressing files with suffix in LIST
--cvs-exclude, -C auto-ignore files in the same way CVS does
--filter=RULE, -f add a file-filtering RULE
@@ -2817,6 +2818,22 @@ expand it.
report something like "`Client compress: zstd (level 3)`" (along with the
checksum choice in effect).
0. `--compress-threads=NUM`, `--zt=NUM`
Set the number of threads to spawn when compressing data. Setting this
option to 1 or more will instruct the compression library to spawn 1 or
more threads for compression. Ideally, increasing the number of threads
will increase transfer speed if the transfer is CPU bound on the sender.
This option does not affect decompression.
Compression algorithms that allow threading:
- `zstd` (only when libzstd is compiled with threading support)
This option is ignored if one of the above alogithms is not selected as the
`--compression-choice` or if compression not enabled.
0. `--skip-compress=LIST`
**NOTE:** no compression method currently supports per-file compression

View File

@@ -547,7 +547,7 @@ int set_file_attrs(const char *fname, struct file_struct *file, stat_x *sxp,
if (am_root >= 0) {
uid_t uid = change_uid ? (uid_t)F_OWNER(file) : sxp->st.st_uid;
gid_t gid = change_gid ? (gid_t)F_GROUP(file) : sxp->st.st_gid;
if (do_lchown_at(fname, uid, gid) != 0) {
if (do_lchown(fname, uid, gid) != 0) {
/* We shouldn't have attempted to change uid
* or gid unless have the privilege. */
rsyserr(FERROR_XFER, errno, "%s %s failed",
@@ -657,7 +657,7 @@ int set_file_attrs(const char *fname, struct file_struct *file, stat_x *sxp,
#ifdef HAVE_CHMOD
if (!BITS_EQUAL(sxp->st.st_mode, new_mode, CHMOD_BITS)) {
int ret = am_root < 0 ? 0 : do_chmod_at(fname, new_mode);
int ret = am_root < 0 ? 0 : do_chmod(fname, new_mode);
if (ret < 0) {
rsyserr(FERROR_XFER, errno,
"failed to set permissions on %s",
@@ -758,7 +758,7 @@ int finish_transfer(const char *fname, const char *fnametmp,
full_fname(fnametmp), fname);
if (!partialptr || (ret == -2 && temp_copy_name)
|| robust_rename(fnametmp, partialptr, NULL, file->mode) < 0)
do_unlink_at(fnametmp);
do_unlink(fnametmp);
return 0;
}
if (ret == 0) {
@@ -774,7 +774,7 @@ int finish_transfer(const char *fname, const char *fnametmp,
ok_to_set_time ? ATTRS_ACCURATE_TIME : ATTRS_SKIP_MTIME | ATTRS_SKIP_ATIME | ATTRS_SKIP_CRTIME);
if (temp_copy_name) {
if (do_rename_at(fnametmp, fname) < 0) {
if (do_rename(fnametmp, fname) < 0) {
rsyserr(FERROR_XFER, errno, "rename %s -> \"%s\"",
full_fname(fnametmp), fname);
return 0;

23
rsync.h
View File

@@ -163,29 +163,6 @@
/* For compatibility with older rsyncs */
#define OLD_MAX_BLOCK_SIZE ((int32)1 << 29)
/* Policy ceilings on attacker-controlled wire values. Picked well above any
* legitimate filesystem / protocol traffic but well below sizes that could
* cause integer overflow or DoS-grade allocations. See input_checking.txt.
*
* Note on MAX_WIRE_XATTR_DATALEN: xattr datum size is bounded only by the
* wire-format maximum (signed int32 varint, ~2GB). macOS resource forks
* are transferred as the com.apple.ResourceFork xattr and can legitimately
* be many GB; --max-alloc (default 1GB, configurable) is the real
* allocation cap. read_varint_size() still rejects negative values so a
* hostile peer cannot wrap to ~SIZE_MAX. */
#define MAX_WIRE_XATTR_COUNT 65536
#define MAX_WIRE_XATTR_NAMELEN 4096
#define MAX_WIRE_XATTR_DATALEN ((int32)0x7fffffff)
#define MAX_WIRE_ACL_COUNT 65536
#define MAX_WIRE_NSEC 999999999
/* MAX_WIRE_DEL_STAT is the per-category cap for read_del_stats() in main.c,
* which accumulates 5 wire-supplied counts into the int32 stats.deleted_files
* accumulator. Capped at 2^28 so 5 * 2^28 = 1.34 GB stays under INT32_MAX
* (2.15 GB) with margin -- a higher cap (e.g. 2^30) would let a hostile peer
* supplying 3+ max-sized counts overflow the accumulator, which is signed-int
* UB. 2^28 is still well above any plausible real transfer's deletion count. */
#define MAX_WIRE_DEL_STAT ((int32)1 << 28)
#define ROUND_UP_1024(siz) ((siz) & (1024-1) ? ((siz) | (1024-1)) + 1 : (siz))
#define IOERR_GENERAL (1<<0) /* For backward compatibility, this must == 1 */

View File

@@ -1073,6 +1073,16 @@ in the values of parameters. See that section for details.
**system()** call's default shell), and use RSYNC_NO_XFER_EXEC to disable
both options completely.
0. `temp dir`
Specifies a directory that rsync should use for temporary files created
during the transfer of updated files. If that directory is on a different
partition, after transfer file is being copied instead of unlinked.
This parameter equals with `--temp-dir` option, so please consult rsync
manpage for further information.
## CONFIG DIRECTIVES
There are currently two config directives available that allow a config file to

468
runtests.py Executable file
View File

@@ -0,0 +1,468 @@
#!/usr/bin/env python3
# Copyright (C) 2001, 2002 by Martin Pool <mbp@samba.org>
# Copyright (C) 2003-2022 Wayne Davison
# Copyright (C) 2026 Andrew Tridgell
#
# Rewrite of runtests.sh in Python (runtests.sh is now deprecated).
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version
# 2 as published by the Free Software Foundation.
"""rsync test runner.
Invokes test scripts from testsuite/ and reports results.
Can be called by 'make check' or directly.
Usage:
./runtests.py [options] [TEST ...]
Each TEST is a test name (e.g. 'delete') or glob pattern (e.g. 'xattr*').
If no tests are specified, all tests are run.
"""
import argparse
import concurrent.futures
import glob
import os
import subprocess
import sys
import threading
def parse_args():
p = argparse.ArgumentParser(description='Run rsync test suite')
p.add_argument('tests', nargs='*', metavar='TEST',
help='Test names or patterns to run (default: all)')
p.add_argument('-j', '--parallel', type=int, default=1, metavar='N',
help='Run up to N tests in parallel (default: 1)')
p.add_argument('--valgrind', action='store_true',
help='Run rsync under valgrind (logs to per-process files)')
p.add_argument('--valgrind-opts', default='', metavar='OPTS',
help='Extra valgrind options (e.g. "--leak-check=full")')
p.add_argument('--preserve-scratch', action='store_true',
help='Keep scratch directories after tests complete')
p.add_argument('--log-level', type=int, default=1, metavar='N',
help='Verbosity level 1-10 (default: 1)')
p.add_argument('--always-log', action='store_true',
help='Show test logs even for passing tests')
p.add_argument('--stop-on-fail', action='store_true',
help='Stop after first test failure')
p.add_argument('--timeout', type=int, default=300, metavar='SECS',
help='Per-test timeout in seconds (default: 300)')
p.add_argument('--rsync-bin', default=None, metavar='PATH',
help='Path to rsync binary (default: ./rsync)')
p.add_argument('--tooldir', default=None, metavar='DIR',
help='Tool/build directory (default: cwd)')
p.add_argument('--srcdir', default=None, metavar='DIR',
help='Source directory (default: script directory)')
p.add_argument('--protocol', type=int, default=None, metavar='VER',
help='Force protocol version (adds --protocol=VER to rsync)')
p.add_argument('--expect-skipped', default=None, metavar='LIST',
help='Comma-separated list of expected-skipped tests')
return p.parse_args()
def find_setfacl_nodef(scratchbase):
"""Determine the setfacl command to remove default ACLs."""
for cmd in [
['setacl', '-k', 'u::7,g::5,o:5', scratchbase],
['setfacl', '-k', scratchbase],
['setfacl', '-s', 'u::7,g::5,o:5', scratchbase],
]:
try:
subprocess.run(cmd, capture_output=True, timeout=5)
return cmd[:2] if cmd[0] == 'setacl' else cmd[:2]
except (FileNotFoundError, subprocess.TimeoutExpired):
continue
try:
r = subprocess.run(['setfacl', '--help'], capture_output=True, text=True, timeout=5)
if '-k,' in r.stdout or '-k,' in r.stderr:
return ['setfacl', '-k']
except (FileNotFoundError, subprocess.TimeoutExpired):
pass
return None
def get_tls_args(config_h):
"""Determine TLS_ARGS from config.h."""
args = ''
try:
with open(config_h) as f:
text = f.read()
if '#define HAVE_LUTIMES 1' in text:
args += ' -l'
if '#undef CHOWN_MODIFIES_SYMLINK' in text:
args += ' -L'
except FileNotFoundError:
pass
return args.strip()
def read_shconfig(path):
"""Read shell config variables from shconfig."""
env = {}
try:
with open(path) as f:
for line in f:
line = line.strip()
if line.startswith('#') or line.startswith('export') or not line:
continue
if '=' in line:
k, _, v = line.partition('=')
env[k.strip()] = v.strip().strip('"')
except FileNotFoundError:
pass
return env
def get_testuser():
"""Determine the current test user."""
for cmd in ['/usr/bin/whoami', '/usr/ucb/whoami', '/bin/whoami']:
if os.path.isfile(cmd):
try:
return subprocess.check_output([cmd], text=True).strip()
except subprocess.CalledProcessError:
pass
try:
return subprocess.check_output(['id', '-un'], text=True).strip()
except (FileNotFoundError, subprocess.CalledProcessError):
return os.environ.get('LOGNAME', os.environ.get('USER', 'UNKNOWN'))
def prep_scratch(scratchdir, srcdir, tooldir, setfacl_nodef):
"""Prepare a scratch directory for a test."""
if os.path.isdir(scratchdir):
subprocess.run(['chmod', '-R', 'u+rwX', scratchdir], capture_output=True)
subprocess.run(['rm', '-rf', scratchdir], capture_output=True)
os.makedirs(scratchdir, exist_ok=True)
if setfacl_nodef:
subprocess.run(setfacl_nodef + [scratchdir], capture_output=True)
try:
os.chmod(scratchdir, os.stat(scratchdir).st_mode & ~0o2000) # clear setgid
except OSError:
pass
src_link = os.path.join(scratchdir, 'src')
if not os.path.exists(src_link):
if os.path.isabs(srcdir):
os.symlink(srcdir, src_link)
else:
os.symlink(os.path.join(tooldir, srcdir), src_link)
def collect_tests(suitedir, patterns):
"""Collect test scripts matching the given patterns."""
if not patterns:
tests = sorted(glob.glob(os.path.join(suitedir, '*.test')))
else:
tests = []
for pat in patterns:
if not pat.endswith('.test'):
pat = pat + '.test'
matches = sorted(glob.glob(os.path.join(suitedir, pat)))
tests.extend(matches)
return tests
def build_rsync_cmd(rsync_bin, args, scratchbase):
"""Build the RSYNC command string for tests."""
parts = []
if args.valgrind:
vlog = os.path.join(scratchbase, 'valgrind.%p.log')
vopts = f'--log-file={vlog}'
if args.valgrind_opts:
vopts += ' ' + args.valgrind_opts
parts.append(f'valgrind {vopts}')
parts.append(rsync_bin)
if args.protocol is not None:
parts.append(f'--protocol={args.protocol}')
return ' '.join(parts)
class TestResult:
"""Result of a single test execution."""
__slots__ = ('testbase', 'result', 'output', 'skipped_reason')
def __init__(self, testbase, result, output='', skipped_reason=''):
self.testbase = testbase
self.result = result
self.output = output
self.skipped_reason = skipped_reason
def run_one_test(testscript, testbase, scratchdir, base_env, timeout,
srcdir, tooldir, setfacl_nodef, always_log):
"""Run a single test. Returns a TestResult.
This function is safe to call from multiple threads — it uses only
per-test state (unique scratchdir, copy of env).
"""
prep_scratch(scratchdir, srcdir, tooldir, setfacl_nodef)
env = base_env.copy()
env['scratchdir'] = scratchdir
logfile = os.path.join(scratchdir, 'test.log')
try:
with open(logfile, 'w') as log:
proc = subprocess.run(
['sh', '-e', testscript],
stdout=log, stderr=subprocess.STDOUT,
env=env, timeout=timeout,
cwd=env.get('TOOLDIR', '.')
)
result = proc.returncode
except subprocess.TimeoutExpired:
result = 1
with open(logfile, 'a') as log:
log.write(f"\nTIMEOUT: test took over {timeout} seconds\n")
# Build output text
output_parts = []
show_log = always_log or (result not in (0, 77, 78))
if show_log:
output_parts.append(f'----- {testbase} log follows')
try:
with open(logfile) as f:
output_parts.append(f.read().rstrip())
except FileNotFoundError:
pass
output_parts.append(f'----- {testbase} log ends')
rsyncd_log = os.path.join(scratchdir, 'rsyncd.log')
if os.path.isfile(rsyncd_log):
output_parts.append(f'----- {testbase} rsyncd.log follows')
with open(rsyncd_log) as f:
output_parts.append(f.read().rstrip())
output_parts.append(f'----- {testbase} rsyncd.log ends')
skipped_reason = ''
if result == 0:
output_parts.append(f'PASS {testbase}')
elif result == 77:
whyfile = os.path.join(scratchdir, 'whyskipped')
try:
with open(whyfile) as f:
skipped_reason = f.read().strip()
except FileNotFoundError:
pass
output_parts.append(f'SKIP {testbase} ({skipped_reason})')
elif result == 78:
output_parts.append(f'XFAIL {testbase}')
else:
output_parts.append(f'FAIL {testbase}')
return TestResult(testbase, result, '\n'.join(output_parts), skipped_reason)
# Lock for serializing output in parallel mode
_print_lock = threading.Lock()
def main():
args = parse_args()
# Also accept legacy environment variables
if args.preserve_scratch or os.environ.get('preserve_scratch') == 'yes':
args.preserve_scratch = True
if args.log_level == 1:
args.log_level = int(os.environ.get('loglevel', '1'))
if args.expect_skipped is None:
args.expect_skipped = os.environ.get('RSYNC_EXPECT_SKIPPED', 'IGNORE')
if os.environ.get('whichtests'):
args.tests = [os.environ['whichtests']]
# Determine directories
tooldir = args.tooldir or os.environ.get('TOOLDIR') or os.getcwd()
script_path = os.path.dirname(os.path.abspath(__file__))
srcdir = args.srcdir or script_path
if not srcdir or srcdir == '.':
srcdir = tooldir
rsync_bin = args.rsync_bin or os.environ.get('rsync_bin') or os.path.join(tooldir, 'rsync')
suitedir = os.path.join(srcdir, 'testsuite')
scratchbase = os.path.join(os.environ.get('scratchbase', tooldir), 'testtmp')
os.makedirs(scratchbase, exist_ok=True)
shconfig = read_shconfig(os.path.join(tooldir, 'shconfig'))
tls_args = get_tls_args(os.path.join(tooldir, 'config.h'))
setfacl_nodef = find_setfacl_nodef(scratchbase)
rsync_cmd = build_rsync_cmd(rsync_bin, args, scratchbase)
if not os.path.isfile(rsync_bin):
sys.stderr.write(f"rsync_bin {rsync_bin} is not a file\n")
sys.exit(2)
if not os.path.isdir(srcdir):
sys.stderr.write(f"srcdir {srcdir} is not a directory\n")
sys.exit(2)
testuser = get_testuser()
# Print header
print('=' * 60)
print(f'{sys.argv[0]} running in {tooldir}')
print(f' rsync_bin={rsync_cmd}')
print(f' srcdir={srcdir}')
print(f' TLS_ARGS={tls_args}')
print(f' testuser={testuser}')
print(f' os={subprocess.check_output(["uname", "-a"], text=True).strip()}')
print(f' preserve_scratch={"yes" if args.preserve_scratch else "no"}')
if args.valgrind:
print(f' valgrind=enabled (logs in valgrind.*.log)')
if args.parallel > 1:
print(f' parallel={args.parallel}')
print(f' scratchbase={scratchbase}')
# Build base environment for test scripts
path = os.environ.get('PATH', '')
if os.path.isdir('/usr/xpg4/bin'):
path = '/usr/xpg4/bin:' + path
base_env = os.environ.copy()
base_env.update({
'PATH': path,
'POSIXLY_CORRECT': '1',
'TOOLDIR': tooldir,
'srcdir': srcdir,
'RSYNC': rsync_cmd,
'TLS_ARGS': tls_args,
'RUNSHFLAGS': '-e',
'scratchbase': scratchbase,
'suitedir': suitedir,
'TESTRUN_TIMEOUT': str(args.timeout),
'HOME': scratchbase,
})
for k, v in shconfig.items():
if v:
base_env[k] = v
if setfacl_nodef:
base_env['setfacl_nodef'] = ' '.join(setfacl_nodef)
else:
base_env['setfacl_nodef'] = 'true'
if args.log_level > 8:
base_env['RUNSHFLAGS'] = '-e -x'
# Collect tests
tests = collect_tests(suitedir, args.tests)
full_run = len(args.tests) == 0
# Record test order for consistent skipped-list output
test_order = {os.path.basename(t).replace('.test', ''): i for i, t in enumerate(tests)}
passed = 0
failed = 0
skipped = 0
skipped_list = []
def process_result(tr):
"""Process a TestResult and update counters. Returns True if test failed."""
nonlocal passed, failed, skipped
with _print_lock:
if tr.output:
print(tr.output)
scratchdir = os.path.join(scratchbase, tr.testbase)
if tr.result == 0:
passed += 1
if not args.preserve_scratch and os.path.isdir(scratchdir):
subprocess.run(['rm', '-rf', scratchdir], capture_output=True)
return False
elif tr.result == 77:
skipped_list.append(tr.testbase)
skipped += 1
if not args.preserve_scratch and os.path.isdir(scratchdir):
subprocess.run(['rm', '-rf', scratchdir], capture_output=True)
return False
elif tr.result == 78:
failed += 1
return True
else:
failed += 1
return True
if args.parallel > 1:
# Parallel execution
with concurrent.futures.ThreadPoolExecutor(max_workers=args.parallel) as executor:
futures = {}
for testscript in tests:
testbase = os.path.basename(testscript).replace('.test', '')
scratchdir = os.path.join(scratchbase, testbase)
timeout = 600 if 'hardlinks' in testbase else args.timeout
f = executor.submit(
run_one_test, testscript, testbase, scratchdir,
base_env, timeout, srcdir, tooldir, setfacl_nodef,
args.always_log
)
futures[f] = testbase
for f in concurrent.futures.as_completed(futures):
tr = f.result()
is_fail = process_result(tr)
if is_fail and args.stop_on_fail:
# Cancel pending futures
for pending in futures:
pending.cancel()
break
else:
# Sequential execution
for testscript in tests:
testbase = os.path.basename(testscript).replace('.test', '')
scratchdir = os.path.join(scratchbase, testbase)
timeout = 600 if 'hardlinks' in testbase else args.timeout
tr = run_one_test(
testscript, testbase, scratchdir,
base_env, timeout, srcdir, tooldir, setfacl_nodef,
args.always_log
)
is_fail = process_result(tr)
if is_fail and args.stop_on_fail:
break
# Check valgrind logs for errors
vg_errors = 0
if args.valgrind:
for vlog in sorted(glob.glob(os.path.join(scratchbase, 'valgrind.*.log'))):
try:
with open(vlog) as f:
content = f.read()
for line in content.splitlines():
if 'ERROR SUMMARY:' in line and 'ERROR SUMMARY: 0 errors' not in line:
vg_errors += 1
print(f'----- valgrind errors in {os.path.basename(vlog)}:')
print(content)
break
except FileNotFoundError:
pass
# Summary
print('-' * 60)
print('----- overall results:')
print(f' {passed} passed')
if failed > 0:
print(f' {failed} failed')
if skipped > 0:
print(f' {skipped} skipped')
if vg_errors > 0:
print(f' {vg_errors} valgrind error(s) found (see logs in {scratchbase})')
skipped_str = ','.join(sorted(skipped_list, key=lambda x: test_order.get(x, 0)))
if full_run and args.expect_skipped != 'IGNORE':
print('----- skipped results:')
print(f' expected: {args.expect_skipped}')
print(f' got: {skipped_str}')
else:
skipped_str = ''
args.expect_skipped = ''
print('-' * 60)
exit_code = failed + vg_errors
if exit_code == 0 and skipped_str != args.expect_skipped:
exit_code = 1
print(f'overall result is {exit_code}')
sys.exit(exit_code)
if __name__ == '__main__':
main()

View File

@@ -1,360 +0,0 @@
#! /bin/sh
# Copyright (C) 2001, 2002 by Martin Pool <mbp@samba.org>
# Copyright (C) 2003-2022 Wayne Davison
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version
# 2 as published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
# -------------------------------------------------------------------------
# rsync top-level test script -- this invokes all the other more
# detailed tests in order. This script can either be called by `make
# check' or `make installcheck'. `check' runs against the copies of
# the program and other files in the build directory, and
# `installcheck' against the installed copy of the program.
# It can also be called on a single test file using a run like this:
#
# preserve_scratch=yes whichtests=itemize.test ./runtests.sh
# In either case we need to also be able to find the source directory,
# since we read test scripts and possibly other information from
# there.
# Whenever possible, informational messages are written to stdout and
# error messages to stderr. They're separated out by the build farm
# display scripts.
# According to the GNU autoconf manual, the only valid place to set up
# directory locations is through Make, since users are allowed to (try
# to) change their mind on the Make command line. So, Make has to
# pass in all the values we need.
# For other configured settings we read ./config.sh, which tells us
# about shell commands on this machine and similar things.
# rsync_bin gives the location of the rsync binary. This is either
# builddir/rsync if we're testing an uninstalled copy, or
# install_prefix/bin/rsync if we're testing an installed copy. On the
# build farm rsync will be installed, but into a scratch /usr.
# srcdir gives the location of the source tree, which lets us find the
# build scripts. At the moment we assume we are invoked from the
# source directory.
# This script must be invoked from the build directory.
# A scratch directory, 'testtmp', is used in the build directory to
# hold per-test subdirectories.
# This script also uses the $loglevel environment variable. 1 is the
# default value, and 10 the most verbose. You can set this from the
# Make command line. It's also set by the build farm to give more
# detail for failing builds.
# -------------------------------------------------------------------------
# NOTES FOR TEST CASES:
# Each test case runs in its own shell.
# Exit codes from tests:
# 1 tests failed
# 2 error in starting tests
# 77 this test skipped (random value unlikely to happen by chance, same as
# automake)
# HOWEVER, the overall exit code to the farm is different: we return
# the *number of tests that failed*, so that it will show up nicely in
# the overall summary.
# rsync.fns contains some general setup functions and definitions.
# -------------------------------------------------------------------------
# NOTES ON PORTABILITY:
# Both this script and the Makefile have to be pretty conservative
# about which Unix features they use.
# We cannot count on Make exporting variables to commands, unless
# they're explicitly given on the command line.
# Also, we can't count on 'cp -a' or 'mkdir -p', although they're
# pretty handy (see function makepath for the latter).
# I think some of the GNU documentation suggests that we shouldn't
# rely on shell functions. However, the Bash manual seems to say that
# they're in POSIX 1003.2, and since the build farm relies on them
# they're probably working on most machines we really care about.
# You cannot use "function foo {" syntax, but must instead say "foo()
# {", or it breaks on FreeBSD.
# BSD machines tend not to have "head" or "seq".
# You cannot do "export VAR=VALUE" all on one line; the export must be
# separate from the assignment. (SCO SysV)
# Don't rely on grep -q, as that doesn't work everywhere -- just redirect
# stdout to /dev/null to keep it quiet.
# -------------------------------------------------------------------------
# STILL TO DO:
# We need a good protection against tests that hang indefinitely.
# Perhaps some combination of starting them in the background, wait,
# and kill?
# Perhaps we need a common way to cleanup tests. At the moment just
# clobbering the directory when we're done should be enough.
# If any of the targets fail, then (GNU?) Make returns 2, instead of
# the return code from the failing command. This is fine, but it
# means that the build farm just shows "2" for failed tests, not the
# number of tests that actually failed. For more details we might
# need to grovel through the log files to find a line saying how many
# failed.
set -e
. "./shconfig"
RUNSHFLAGS='-e'
export RUNSHFLAGS
# for Solaris
if [ -d /usr/xpg4/bin ]; then
PATH="/usr/xpg4/bin/:$PATH"
export PATH
fi
if [ "x$loglevel" != x ] && [ "$loglevel" -gt 8 ]; then
if set -x; then
# If it doesn't work the first time, don't keep trying.
RUNSHFLAGS="$RUNSHFLAGS -x"
fi
fi
POSIXLY_CORRECT=1
if test x"$TOOLDIR" = x; then
TOOLDIR=`pwd`
fi
srcdir=`dirname $0`
if test x"$srcdir" = x || test x"$srcdir" = x.; then
srcdir="$TOOLDIR"
fi
if test x"$rsync_bin" = x; then
rsync_bin="$TOOLDIR/rsync"
fi
# This allows the user to specify extra rsync options -- use carefully!
RSYNC="$rsync_bin $*"
#RSYNC="valgrind $rsync_bin $*"
TLS_ARGS=''
if grep -E '^#define HAVE_LUTIMES 1' config.h >/dev/null; then
TLS_ARGS="$TLS_ARGS -l"
fi
if grep -E '#undef CHOWN_MODIFIES_SYMLINK' config.h >/dev/null; then
TLS_ARGS="$TLS_ARGS -L"
fi
export POSIXLY_CORRECT TOOLDIR srcdir RSYNC TLS_ARGS
echo "============================================================"
echo "$0 running in $TOOLDIR"
echo " rsync_bin=$RSYNC"
echo " srcdir=$srcdir"
echo " TLS_ARGS=$TLS_ARGS"
if [ -f /usr/bin/whoami ]; then
testuser=`/usr/bin/whoami`
elif [ -f /usr/ucb/whoami ]; then
testuser=`/usr/ucb/whoami`
elif [ -f /bin/whoami ]; then
testuser=`/bin/whoami`
else
testuser=`id -un 2>/dev/null || echo ${LOGNAME:-${USERNAME:-${USER:-'UNKNOWN'}}}`
fi
echo " testuser=$testuser"
echo " os=`uname -a`"
# It must be "yes", not just nonnull
if [ "x$preserve_scratch" = xyes ]; then
echo " preserve_scratch=yes"
else
echo " preserve_scratch=no"
fi
# Check if setacl/setfacl is around and if it supports the -k or -s option.
if setacl -k u::7,g::5,o:5 testsuite 2>/dev/null; then
setfacl_nodef='setacl -k'
elif setfacl --help 2>&1 | grep ' -k,\|\[-[a-z]*k' >/dev/null; then
setfacl_nodef='setfacl -k'
elif setfacl -s u::7,g::5,o:5 testsuite 2>/dev/null; then
setfacl_nodef='setfacl -s u::7,g::5,o:5'
else
# The "true" command runs successfully, but does nothing.
setfacl_nodef=true
fi
export setfacl_nodef
if [ ! -f "$rsync_bin" ]; then
echo "rsync_bin $rsync_bin is not a file" >&2
exit 2
fi
if [ ! -d "$srcdir" ]; then
echo "srcdir $srcdir is not a directory" >&2
exit 2
fi
expect_skipped="${RSYNC_EXPECT_SKIPPED-IGNORE}"
skipped_list=''
skipped=0
missing=0
passed=0
failed=0
# Directory that holds the other test subdirs. We create separate dirs
# inside for each test case, so that they can be left behind in case of
# failure to aid investigation. We don't remove the testtmp subdir at
# the end so that it can be configured as a symlink to a filesystem that
# has ACLs and xattr support enabled (if desired).
scratchbase="${scratchbase:-$TOOLDIR}"/testtmp
echo " scratchbase=$scratchbase"
[ -d "$scratchbase" ] || mkdir "$scratchbase"
suitedir="$srcdir/testsuite"
TESTRUN_TIMEOUT=300
export scratchdir suitedir TESTRUN_TIMEOUT
prep_scratch() {
[ -d "$scratchdir" ] && chmod -R u+rwX "$scratchdir" && rm -rf "$scratchdir"
mkdir "$scratchdir"
# Get rid of default ACLs and dir-setgid to avoid confusing some tests.
$setfacl_nodef "$scratchdir" 2>/dev/null || true
chmod g-s "$scratchdir"
case "$srcdir" in
/*) ln -s "$srcdir" "$scratchdir/src" ;;
*) ln -s "$TOOLDIR/$srcdir" "$scratchdir/src" ;;
esac
return 0
}
maybe_discard_scratch() {
[ x"$preserve_scratch" != xyes ] && [ -d "$scratchdir" ] && rm -rf "$scratchdir"
return 0
}
if [ "x$whichtests" = x ]; then
whichtests="*.test"
full_run=yes
else
full_run=no
fi
for testscript in $suitedir/$whichtests; do
testbase=`echo $testscript | sed -e 's!.*/!!' -e 's/.test\$//'`
scratchdir="$scratchbase/$testbase"
prep_scratch
case "$testscript" in
*hardlinks*) TESTRUN_TIMEOUT=600 ;;
*) TESTRUN_TIMEOUT=300 ;;
esac
set +e
"$TOOLDIR/"testrun $RUNSHFLAGS "$testscript" >"$scratchdir/test.log" 2>&1
result=$?
set -e
if [ "x$always_log" = xyes ] || ( [ $result != 0 ] && [ $result != 77 ] && [ $result != 78 ] )
then
echo "----- $testbase log follows"
cat "$scratchdir/test.log"
echo "----- $testbase log ends"
if [ -f "$scratchdir/rsyncd.log" ]; then
echo "----- $testbase rsyncd.log follows"
cat "$scratchdir/rsyncd.log"
echo "----- $testbase rsyncd.log ends"
fi
fi
case $result in
0)
echo "PASS $testbase"
passed=`expr $passed + 1`
maybe_discard_scratch
;;
77)
# backticks will fill the whole file onto one line, which is a feature
whyskipped=`cat "$scratchdir/whyskipped"`
echo "SKIP $testbase ($whyskipped)"
skipped_list="$skipped_list,$testbase"
skipped=`expr $skipped + 1`
maybe_discard_scratch
;;
78)
# It failed, but we expected that. don't dump out error logs,
# because most users won't want to see them. But do leave
# the working directory around.
echo "XFAIL $testbase"
failed=`expr $failed + 1`
;;
*)
echo "FAIL $testbase"
failed=`expr $failed + 1`
if [ "x$nopersist" = xyes ]; then
exit 1
fi
esac
done
echo '------------------------------------------------------------'
echo "----- overall results:"
echo " $passed passed"
[ "$failed" -gt 0 ] && echo " $failed failed"
[ "$skipped" -gt 0 ] && echo " $skipped skipped"
[ "$missing" -gt 0 ] && echo " $missing missing"
if [ "$full_run" = yes ] && [ "$expect_skipped" != IGNORE ]; then
skipped_list=`echo "$skipped_list" | sed 's/^,//'`
echo "----- skipped results:"
echo " expected: $expect_skipped"
echo " got: $skipped_list"
else
skipped_list=''
expect_skipped=''
fi
echo '------------------------------------------------------------'
# OK, so expr exits with 0 if the result is neither null nor zero; and
# 1 if the expression is null or zero. This is the opposite of what
# we want, and if we just call expr then this script will always fail,
# because -e is set.
result=`expr $failed + $missing || true`
if [ "$result" = 0 ] && [ "$skipped_list" != "$expect_skipped" ]; then
result=1
fi
echo "overall result is $result"
exit $result

View File

@@ -48,8 +48,6 @@ extern int make_backups;
extern int inplace;
extern int inplace_partial;
extern int batch_fd;
extern int use_secure_symlinks;
extern char *module_dir;
extern int write_batch;
extern int file_old_total;
extern BOOL want_progress_now;
@@ -140,8 +138,6 @@ void successful_send(int ndx)
return;
flist = flist_for_ndx(ndx, "successful_send");
if (ndx < flist->ndx_start)
exit_cleanup(RERR_PROTOCOL);
file = flist->files[ndx - flist->ndx_start];
if (!change_pathname(file, NULL, 0))
return;
@@ -356,25 +352,7 @@ void send_files(int f_in, int f_out)
exit_cleanup(RERR_PROTOCOL);
}
if (use_secure_symlinks) {
/* Open from module root to prevent TOCTOU race where
* change_pathname's chdir follows a directory symlink.
* Reconstruct the full path relative to module_dir
* from F_PATHNAME (path) and f_name (fname). */
char secure_path[MAXPATHLEN];
int slen = snprintf(secure_path, sizeof secure_path, "%s%s%s", path, slash, fname);
if (slen >= (int)sizeof secure_path) {
io_error |= IOERR_GENERAL;
rprintf(FERROR_XFER, "path too long: %s%s%s\n", path, slash, fname);
free_sums(s);
if (protocol_version >= 30)
send_msg_int(MSG_NO_SEND, ndx);
continue;
}
fd = secure_relative_open(module_dir, secure_path, O_RDONLY, 0);
} else {
fd = do_open_checklinks(fname);
}
fd = do_open_checklinks(fname);
if (fd == -1) {
if (errno == ENOENT) {
enum logcode c = am_daemon && protocol_version < 28 ? FERROR : FWARNING;

View File

@@ -47,23 +47,21 @@ static struct sigaction sigact;
static int sock_exec(const char *prog);
#define PROXY_BUF_SIZE 1024
/* Establish a proxy connection on an open socket to a web proxy by using the
* CONNECT method. If proxy_user and proxy_pass are not NULL, they are used to
* authenticate to the proxy using the "Basic" proxy-authorization protocol. */
static int establish_proxy_connection(int fd, char *host, int port, char *proxy_user, char *proxy_pass)
{
char *cp, buffer[PROXY_BUF_SIZE + 1];
char *authhdr, authbuf[PROXY_BUF_SIZE + 1];
char *cp, buffer[1024];
char *authhdr, authbuf[1024];
int len;
if (proxy_user && proxy_pass) {
stringjoin(buffer, PROXY_BUF_SIZE,
stringjoin(buffer, sizeof buffer,
proxy_user, ":", proxy_pass, NULL);
len = strlen(buffer);
if ((len*8 + 5) / 6 >= PROXY_BUF_SIZE - 3) {
if ((len*8 + 5) / 6 >= (int)sizeof authbuf - 3) {
rprintf(FERROR,
"authentication information is too long\n");
return -1;
@@ -76,14 +74,14 @@ static int establish_proxy_connection(int fd, char *host, int port, char *proxy_
authhdr = "";
}
len = snprintf(buffer, PROXY_BUF_SIZE, "CONNECT %s:%d HTTP/1.0%s%s\r\n\r\n", host, port, authhdr, authbuf);
assert(len > 0 && len < PROXY_BUF_SIZE);
len = snprintf(buffer, sizeof buffer, "CONNECT %s:%d HTTP/1.0%s%s\r\n\r\n", host, port, authhdr, authbuf);
assert(len > 0 && len < (int)sizeof buffer);
if (write(fd, buffer, len) != len) {
rsyserr(FERROR, errno, "failed to write to proxy");
return -1;
}
for (cp = buffer; cp < &buffer[PROXY_BUF_SIZE - 1]; cp++) {
for (cp = buffer; cp < &buffer[sizeof buffer - 1]; cp++) {
if (read(fd, cp, 1) != 1) {
rsyserr(FERROR, errno, "failed to read from proxy");
return -1;
@@ -92,13 +90,11 @@ static int establish_proxy_connection(int fd, char *host, int port, char *proxy_
break;
}
if (cp == &buffer[PROXY_BUF_SIZE - 1]) {
rprintf(FERROR, "proxy response line too long\n");
return -1;
}
*cp = '\0';
if (cp > buffer && cp[-1] == '\r')
cp[-1] = '\0';
if (*cp != '\n')
cp++;
*cp-- = '\0';
if (*cp == '\r')
*cp = '\0';
if (strncmp(buffer, "HTTP/", 5) != 0) {
rprintf(FERROR, "bad response from proxy -- %s\n",
buffer);
@@ -114,7 +110,7 @@ static int establish_proxy_connection(int fd, char *host, int port, char *proxy_
}
/* throw away the rest of the HTTP header */
while (1) {
for (cp = buffer; cp < &buffer[PROXY_BUF_SIZE]; cp++) {
for (cp = buffer; cp < &buffer[sizeof buffer - 1]; cp++) {
if (read(fd, cp, 1) != 1) {
rsyserr(FERROR, errno,
"failed to read from proxy");

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python3
import os, re, argparse, subprocess
from datetime import datetime
from datetime import datetime, UTC
NULL_COMMIT_RE = re.compile(r'\0\0commit [a-f0-9]{40}$|\0$')
@@ -74,7 +74,7 @@ def print_line(fn, mtime, commit_time):
if args.list > 1:
ts = str(commit_time).rjust(10)
else:
ts = datetime.utcfromtimestamp(commit_time).strftime("%Y-%m-%d %H:%M:%S")
ts = datetime.fromtimestamp(commit_time, UTC).strftime("%Y-%m-%d %H:%M:%S")
chg = '.' if mtime == commit_time else '*'
print(chg, ts, fn)

View File

@@ -46,6 +46,7 @@ long_opts = {
'compare-dest': 2,
'compress-choice': 1,
'compress-level': 1,
'compress-threads': 1,
'copy-dest': 2,
'copy-devices': -1,
'copy-unsafe-links': 0,
@@ -59,6 +60,7 @@ long_opts = {
'delete-during': 0,
'delete-excluded': 0,
'delete-missing-args': 0,
'dirs': 0,
'existing': 0,
'fake-super': 0,
'files-from': 3,
@@ -300,6 +302,7 @@ def validated_arg(opt, arg, typ=3, wild=False):
if arg.startswith('./'):
arg = arg[1:]
arg = arg.replace('//', '/')
arg = arg.lstrip('/')
if args.dir != '/':
if HAS_DOT_DOT_RE.search(arg):
die("do not use .. in", opt, "(anchor the path at the root of your restricted dir)")

View File

@@ -2,7 +2,7 @@
REAL_RSYNC=/usr/bin/rsync
IGNOREEXIT=24
IGNOREOUT='^(file has vanished: |rsync warning: some files vanished before they could be transferred)'
IGNOREOUT='^((file|directory) has vanished: |rsync warning: some files vanished before they could be transferred)'
# If someone installs this as "rsync", make sure we don't affect a server run.
for arg in "${@}"; do

1271
syscall.c
View File

File diff suppressed because it is too large Load Diff

View File

@@ -1,117 +0,0 @@
/*
* Test harness for do_chmod_at(). Confirms the symlink-TOCTOU
* primitive used by CVE-2026-29518 (and its incomplete-fix follow-up
* for chmod) is closed by do_chmod_at(): a parent directory component
* being a symlink that escapes the receiver's confinement must be
* rejected, while a parent symlink that resolves *within* the tree
* must still work (so legitimate dir-symlinks are not regressed).
*
* Not linked into rsync itself.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#include "rsync.h"
#include <sys/stat.h>
int dry_run = 0;
int am_root = 0;
int am_sender = 0;
int read_only = 0;
int list_only = 0;
int copy_links = 0;
int copy_unsafe_links = 0;
extern int am_daemon, am_chrooted;
short info_levels[COUNT_INFO], debug_levels[COUNT_DEBUG];
static int errs = 0;
static void check(const char *label, int actual_rc, int expect_ok,
const char *path, mode_t expected_mode)
{
struct stat st;
int got_ok = (actual_rc == 0);
if (got_ok != expect_ok) {
fprintf(stderr, "FAIL [%s]: rc=%d errno=%d (%s), expected %s\n",
label, actual_rc, errno, strerror(errno),
expect_ok ? "success" : "rejection");
errs++;
return;
}
if (path && stat(path, &st) < 0) {
fprintf(stderr, "FAIL [%s]: stat(%s) failed: %s\n",
label, path, strerror(errno));
errs++;
return;
}
if (path && (st.st_mode & 07777) != expected_mode) {
fprintf(stderr,
"FAIL [%s]: %s mode is 0%o, expected 0%o\n",
label, path, st.st_mode & 07777, expected_mode);
errs++;
return;
}
fprintf(stderr, "OK [%s]\n", label);
}
int main(int argc, char **argv)
{
if (argc != 2) {
fprintf(stderr, "usage: %s <module-dir>\n", argv[0]);
return 2;
}
if (chdir(argv[1]) < 0) {
perror("chdir");
return 2;
}
/* Simulate the daemon-without-chroot deployment that do_chmod_at()
* defends. With am_daemon=0 or am_chrooted=1 the wrapper falls
* through to plain do_chmod() and the symlink-race test would be
* meaningless. */
am_daemon = 1;
am_chrooted = 0;
/* Test layout (all inside the directory we just chdir'd to):
*
* ./realdir/sentinel -- regular target file
* ./inside_link -> realdir -- legitimate dir-symlink within the tree
* ./escape_link -> ../trap -- attacker swap, target outside tree
* ../trap/sentinel -- the file the attacker wants to alter
*
* The shell wrapper that calls this helper has set both sentinel
* files to mode 0600 so we have a clean baseline to compare.
*/
/* Scenario A: legitimate parent dir-symlink, chmod must succeed. */
int rc = do_chmod_at("inside_link/sentinel", 0640);
check("A: legit dir-symlink within tree",
rc, 1, "realdir/sentinel", 0640);
/* Scenario B: parent symlink escapes the tree -- chmod must be
* rejected and the outside file's mode must be unchanged. */
rc = do_chmod_at("escape_link/sentinel", 0666);
check("B: parent symlink escapes tree (the attack)",
rc, 0, "../trap/sentinel", 0600);
/* Scenario C: plain relative path with no symlink components,
* regression check that the safe wrapper doesn't break the
* normal case. */
rc = do_chmod_at("realdir/sentinel", 0644);
check("C: plain relative path (regression check)",
rc, 1, "realdir/sentinel", 0644);
/* Scenario D: top-level file, no parent directory component.
* Falls back to do_chmod(); should succeed. */
rc = do_chmod_at("topfile", 0640);
check("D: top-level file, no parent component",
rc, 1, "topfile", 0640);
if (errs)
fprintf(stderr, "%d failure(s)\n", errs);
return errs ? 1 : 0;
}

View File

@@ -1,151 +0,0 @@
/*
* Test harness for secure_relative_open()'s front-door input
* validation. Codex audit Finding 5 noted that the existing check
*
* if (strncmp(relpath, "../", 3) == 0 || strstr(relpath, "/../"))
*
* catches "../foo" and "foo/../bar" but misses bare ".." (an actual
* one-level escape on platforms that fall back to the per-component
* walk), as well as "a/..", "foo/..", and any other form that
* decomposes to a ".." component when split on "/". The kernel-
* enforced RESOLVE_BENEATH (Linux 5.6+) and O_RESOLVE_BENEATH
* (FreeBSD 13+, macOS 15+) reject these in-kernel; the per-
* component fallback used on NetBSD, OpenBSD, Solaris, Cygwin and
* pre-5.6 Linux does not, so the validation must happen at the
* front door.
*
* This helper invokes secure_relative_open() with each suspect
* input and checks both the failure (rc < 0) and the errno
* (EINVAL means "rejected at the front door"). Pre-fix, the kernel
* may reject with a different errno (EXDEV from RESOLVE_BENEATH);
* post-fix, the front-door check catches every variant up front
* with a consistent EINVAL across platforms.
*
* Not linked into rsync itself.
*/
#include "rsync.h"
#include <sys/stat.h>
int dry_run = 0;
int am_root = 0;
int am_sender = 0;
int read_only = 0;
int list_only = 0;
int copy_links = 0;
int copy_unsafe_links = 0;
extern int am_daemon, am_chrooted;
short info_levels[COUNT_INFO], debug_levels[COUNT_DEBUG];
static int errs = 0;
static void check_relpath(const char *relpath)
{
int fd;
int saved_errno;
errno = 0;
fd = secure_relative_open(NULL, relpath, O_RDONLY | O_DIRECTORY, 0);
saved_errno = errno;
if (fd >= 0) {
fprintf(stderr,
"FAIL [relpath=%-12s]: returned valid fd %d (escape) -- expected -1 EINVAL\n",
relpath, fd);
close(fd);
errs++;
return;
}
if (saved_errno != EINVAL) {
fprintf(stderr,
"FAIL [relpath=%-12s]: rejected but errno=%d (%s), expected EINVAL\n",
relpath, saved_errno, strerror(saved_errno));
errs++;
return;
}
fprintf(stderr, "OK [relpath=%-12s]: rejected with EINVAL\n", relpath);
}
static void check_basedir(const char *basedir)
{
int fd;
int saved_errno;
errno = 0;
fd = secure_relative_open(basedir, "ok", O_RDONLY | O_DIRECTORY, 0);
saved_errno = errno;
if (fd >= 0) {
fprintf(stderr,
"FAIL [basedir=%-12s]: returned valid fd %d -- expected -1 EINVAL\n",
basedir, fd);
close(fd);
errs++;
return;
}
if (saved_errno != EINVAL) {
fprintf(stderr,
"FAIL [basedir=%-12s]: rejected but errno=%d (%s), expected EINVAL\n",
basedir, saved_errno, strerror(saved_errno));
errs++;
return;
}
fprintf(stderr, "OK [basedir=%-12s]: rejected with EINVAL\n", basedir);
}
int main(int argc, char **argv)
{
if (argc != 2) {
fprintf(stderr, "usage: %s <test-dir>\n", argv[0]);
return 2;
}
if (chdir(argv[1]) < 0) {
perror("chdir");
return 2;
}
/* secure_relative_open's daemon-only confinement protections only
* fire when am_daemon && !am_chrooted (the threat model is the
* daemon-no-chroot deployment), but the front-door input
* validation runs unconditionally. We set am_daemon anyway so the
* helper exercises the same code shape the receiver does. */
am_daemon = 1;
am_chrooted = 0;
mkdir("subdir", 0755);
/* Each of these relpaths must be rejected with EINVAL at the
* secure_relative_open() front door. ".." is the actual one-level
* escape; the others ("subdir/..", "subdir/../subdir") resolve
* back to the start dir on systems that allow them, but we still
* reject them as defence-in-depth: a path containing a ".." token
* is suspicious and the caller should normalise before passing
* it in. The "../foo" / "foo/../bar" / "/foo" / "/" cases are
* regression checks for the existing checks. */
check_relpath("..");
check_relpath("../foo");
check_relpath("subdir/..");
check_relpath("subdir/../subdir");
check_relpath("foo/../bar");
check_relpath("/foo");
check_relpath("/");
/* Same checks against basedir (which the codex Finding 2 fix
* routes through the same RESOLVE_BENEATH-equivalent). Absolute
* basedirs are operator-trusted and intentionally not validated
* here. */
check_basedir("..");
check_basedir("../subdir");
check_basedir("subdir/..");
check_basedir("foo/../bar");
if (errs)
fprintf(stderr, "\n%d failure(s)\n", errs);
return errs ? 1 : 0;
}

View File

@@ -23,8 +23,6 @@
int do_fsync = 0;
int inplace = 0;
int am_daemon = 0;
int am_chrooted = 0;
int modify_window = 0;
int preallocate_files = 0;
int protect_args = 0;

View File

@@ -1,113 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for the basedir-confinement gap in
# secure_relative_open(). The function opens basedir with a plain
# openat(AT_FDCWD, basedir, O_RDONLY | O_DIRECTORY), without
# RESOLVE_BENEATH or a per-component O_NOFOLLOW walk, so a parent
# symlink ON basedir is followed unrestrictedly. RESOLVE_BENEATH is
# then applied only to relpath, anchored at the wrong directory.
#
# The receiver's basis-file lookup at receiver.c passes
# basis_dir[fnamecmp_type] (from --copy-dest / --link-dest /
# --compare-dest -- all sender-controllable in daemon mode) as
# basedir. A daemon-module attacker with write access can plant a
# symlink at module/cd -> /outside, then run --link-dest=cd to
# make the daemon's basis-file lookup resolve into /outside,
# leaking the contents of daemon-readable files via the rsync
# delta-rolling read-disclosure primitive.
#
# We detect the escape by leveraging --link-dest: when basis
# matches source exactly (content + mtime + mode), --link-dest
# hard-links the destination to the basis file. With the bug, the
# destination ends up as a hard link to the outside-the-module
# file (same inode). With the fix, no basis is found and the
# destination is a fresh copy (different inode).
#
# The vulnerable code path is the same on every platform
# (including the per-component fallback on systems without
# RESOLVE_BENEATH), so this test is not platform-gated.
. "$suitedir/rsync.fns"
mod="$scratchdir/module"
outside="$scratchdir/outside"
src="$scratchdir/src"
conf="$scratchdir/test-rsyncd.conf"
rm -rf "$mod" "$outside" "$src"
mkdir -p "$mod" "$outside" "$src"
# Portable inode-number helper (GNU coreutils stat -c, BSD stat -f).
file_inode() {
stat -c %i "$1" 2>/dev/null || stat -f %i "$1"
}
# Outside-the-module file an attacker would like the daemon to
# treat as a basis.
echo "OUTSIDE_SECRET_DATA" > "$outside/target.txt"
chmod 0644 "$outside/target.txt"
# The symlink trap planted in the module by the local attacker.
ln -s "$outside" "$mod/cd"
# Source file matches outside/target.txt exactly (content + mtime
# + mode) so --link-dest will hard-link the destination to the
# basis file iff the daemon's basedir lookup reaches outside/.
echo "OUTSIDE_SECRET_DATA" > "$src/target.txt"
touch -r "$outside/target.txt" "$src/target.txt"
chmod 0644 "$src/target.txt"
# When running as root the daemon would drop to "nobody" by
# default, which can't write into the test scratch dir. Force the
# daemon to keep our uid/gid in that case so the basis-link
# transfer can actually create the destination file. (Non-root
# can't specify uid/gid in rsyncd.conf -- comment them out then.)
my_uid=`get_testuid`
root_uid=`get_rootuid`
root_gid=`get_rootgid`
uid_setting="uid = $root_uid"
gid_setting="gid = $root_gid"
if test x"$my_uid" != x"$root_uid"; then
uid_setting="#$uid_setting"
gid_setting="#$gid_setting"
fi
cat > "$conf" <<EOF
use chroot = no
$uid_setting
$gid_setting
log file = $scratchdir/rsyncd.log
[upload]
path = $mod
use chroot = no
read only = no
EOF
# Recursive --link-dest push directly into the module root. We
# avoid pushing into a destination subdir because the receiver
# would chdir into it before resolving --link-dest, making the
# relative basedir "cd" resolve in the wrong CWD and masking the
# bug. The realistic attack pushes into the module root (or the
# attacker uses a basedir path that resolves correctly from
# whichever subdir the receiver chdirs into).
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC -rtp --link-dest=cd "$src/" rsync://localhost/upload/ \
>/dev/null 2>&1 || true
if [ ! -f "$mod/target.txt" ]; then
test_fail "destination file was not created -- daemon transfer failed before the test could observe the basedir behaviour"
fi
outside_inode=$(file_inode "$outside/target.txt")
dst_inode=$(file_inode "$mod/target.txt")
if [ "$outside_inode" = "$dst_inode" ]; then
test_fail "basedir-escape: --link-dest hard-linked module/target.txt to outside/target.txt (inode $outside_inode); daemon's basis-file lookup followed the parent symlink on the basedir"
fi
exit 0

View File

@@ -1,206 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for codex audit Findings 3b and 3c:
#
# 3b: generator.c:1905 -- the in-place backup creation opens
# backupptr via bare do_open(O_WRONLY|O_CREAT|O_TRUNC|O_EXCL).
# With --backup-dir set to an attacker-planted parent symlink,
# the backup file is written outside the module under the
# daemon's authority.
#
# 3c-symlink: syscall.c:207 -- do_symlink_at falls through to bare
# do_symlink for am_root < 0 (fake-super), which then opens
# the destination path with bare open() (final-component
# fake-super file). A parent symlink on the destination path
# redirects the file creation outside the module.
#
# 3c-mknod: syscall.c:506 -- do_mknod_at falls through to bare
# do_mknod for am_root < 0, same path-based open(). For
# FIFOs/sockets/devices the bare path is also used.
#
# Each scenario plants a "secret" file outside the module at a
# location the symlink trap points to. The check is that the
# outside file's content and mode are unchanged after the attack
# attempt.
. "$suitedir/rsync.fns"
# All three scenarios depend on receiver-side daemon code paths
# that are only secured on platforms with a working
# secure_relative_open. The chdir/chmod tests already skip the
# same set; mirror that.
case "$(uname -s)" in
SunOS|OpenBSD|NetBSD|CYGWIN*)
test_skipped "secure_relative_open relies on RESOLVE_BENEATH-equivalent kernel support not available on $(uname -s)"
;;
esac
mod="$scratchdir/module"
outside="$scratchdir/outside"
src="$scratchdir/src"
conf="$scratchdir/test-rsyncd.conf"
# Portable inode-and-mode helpers.
file_mode() {
stat -c %a "$1" 2>/dev/null || stat -f %Lp "$1"
}
setup() {
rm -rf "$mod" "$outside" "$src"
mkdir -p "$mod" "$outside" "$src"
echo "OUTSIDE_PROTECTED_DATA" > "$outside/target.txt"
chmod 0644 "$outside/target.txt"
outside_pristine="$scratchdir/outside-pristine.txt"
cp -p "$outside/target.txt" "$outside_pristine"
ln -s "$outside" "$mod/cd"
}
verify_outside_unchanged() {
label="$1"
mode=$(file_mode "$outside/target.txt")
case "$mode" in
644|0644) ;;
*) test_fail "$label: outside/target.txt mode changed from 644 to $mode" ;;
esac
if ! cmp -s "$outside/target.txt" "$outside_pristine"; then
test_fail "$label: outside/target.txt content changed -- daemon followed the cd symlink"
fi
}
verify_outside_unchanged_or_absent() {
label="$1"
target="$2" # specific file under outside/ to check absence of
if [ -e "$outside/$target" ]; then
test_fail "$label: outside/$target was created -- daemon followed the cd symlink"
fi
}
# When running as root the daemon would drop to "nobody" by default
# and fail to write into the test scratch dir. Force it to keep our
# uid/gid in that case so the receiver actually runs the code paths
# we want to test.
my_uid=`get_testuid`
root_uid=`get_rootuid`
root_gid=`get_rootgid`
uid_setting="uid = $root_uid"
gid_setting="gid = $root_gid"
if test x"$my_uid" != x"$root_uid"; then
uid_setting="#$uid_setting"
gid_setting="#$gid_setting"
fi
############################################################
# Scenario 3b: --inplace --backup --backup-dir=cd
#
# Pre-create module/target.txt so the receiver enters the in-place
# update path; a backup of the existing content must be made
# before the update. With --backup-dir=cd, backupptr resolves to
# "cd/target.txt"; with the bug, robust_unlink and the bare
# do_open at generator.c:1905 both follow the cd symlink, the
# unlink deletes outside/target.txt and the create writes the
# pre-existing module/target.txt content there.
############################################################
setup
echo "EXISTING_MODULE_DATA" > "$mod/target.txt"
chmod 0666 "$mod/target.txt"
echo "NEW_DATA_FROM_SENDER" > "$src/target.txt"
chmod 0644 "$src/target.txt"
cat > "$conf" <<EOF
use chroot = no
$uid_setting
$gid_setting
log file = $scratchdir/rsyncd.log
[upload]
path = $mod
use chroot = no
read only = no
EOF
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC --inplace --backup --backup-dir=cd "$src/target.txt" \
rsync://localhost/upload/target.txt >/dev/null 2>&1 || true
verify_outside_unchanged "3b inplace+backup-dir=cd"
############################################################
# Scenario 3c-symlink: fake-super symlink push to a path with a
# symlinked parent
#
# With "fake super = yes" set on the module, the receiver
# represents symlinks as fake-super files (regular files with the
# link target written to them). The path-based open() in
# do_symlink's fake-super branch follows parent symlinks. We push
# a single symlink to the destination path "cd/sym" so the
# receiver's create-file call lands at "cd/sym" relative to the
# module root, where cd is the symlink trap.
############################################################
setup
mkdir -p "$src/cd"
ln -s /etc/passwd "$src/cd/sym"
cat > "$conf" <<EOF
use chroot = no
$uid_setting
$gid_setting
log file = $scratchdir/rsyncd.log
[upload_fake]
path = $mod
use chroot = no
read only = no
fake super = yes
EOF
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC -rl "$src/" rsync://localhost/upload_fake/ >/dev/null 2>&1 || true
verify_outside_unchanged_or_absent "3c-symlink fake-super symlink push" "sym"
############################################################
# Scenario 3c-mknod: fake-super FIFO push to a path with a
# symlinked parent
#
# Similar to 3c-symlink but for special files. mkfifo works
# without root; we push a FIFO and verify the receiver doesn't
# create a fake-super file at outside/fifo.
############################################################
setup
mkdir -p "$src/cd"
mkfifo "$src/cd/fifo" 2>/dev/null
if [ ! -p "$src/cd/fifo" ]; then
test_skipped "mkfifo unavailable; cannot exercise 3c-mknod"
fi
cat > "$conf" <<EOF
use chroot = no
$uid_setting
$gid_setting
log file = $scratchdir/rsyncd.log
[upload_fake]
path = $mod
use chroot = no
read only = no
fake super = yes
EOF
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC -rD "$src/" rsync://localhost/upload_fake/ >/dev/null 2>&1 || true
verify_outside_unchanged_or_absent "3c-mknod fake-super FIFO push" "fifo"
exit 0

View File

@@ -1,135 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for the symlink-TOCTOU class of bug at the receiver's
# chdir(). After the CVE-2026-29518 fix to secure_relative_open(), an
# attack remained where the receiver's chdir() into a destination
# subdirectory followed an attacker-planted symlink, escaping the
# module. Every subsequent path-relative syscall (open, chmod, lchown,
# utimes, etc.) inherited the escape -- secure_relative_open's
# RESOLVE_BENEATH anchor itself was outside the module by then, so it
# stopped protecting against anything.
#
# This test runs an actual rsync daemon (via RSYNC_CONNECT_PROG to
# avoid the network) configured with "use chroot = no", plants a
# symlink at module/subdir -> ../outside, and runs four flavours of
# rsync transfer that previously all reached files in ../outside:
#
# 1. single-file dest = subdir/target.txt (the original poc_chmod)
# 2. -r src/subdir/ to upload/subdir/ (the chdir-escape case)
# 3. -r src/subdir/ to upload/subdir/ (no --size-only: forces basis read+write)
# 4. -r src/ to upload/ (was already protected by the
# original CVE-2026-29518 fix;
# regression-checked here)
#
# All four must leave the outside-the-module sentinel file's mode AND
# content unchanged.
. "$suitedir/rsync.fns"
case "$(uname -s)" in
SunOS|OpenBSD|NetBSD|CYGWIN*)
test_skipped "secure chdir relies on RESOLVE_BENEATH-equivalent kernel support not available on $(uname -s)"
;;
esac
mod="$scratchdir/module"
outside="$scratchdir/outside"
src="$scratchdir/src"
conf="$scratchdir/test-rsyncd.conf"
rm -rf "$mod" "$outside" "$src"
mkdir -p "$mod" "$outside" "$src" "$src/subdir"
# Portable octal-mode helper -- macOS and FreeBSD's stat use -f, GNU
# coreutils stat uses -c.
file_mode() {
stat -c %a "$1" 2>/dev/null || stat -f %Lp "$1"
}
# The "secret" file outside the module the attacker is trying to alter.
# Save a pristine copy alongside it so we can compare with cmp(1) rather
# than depending on sha1sum/shasum/sha1, which differ across platforms.
echo "OUTSIDE_SECRET_DATA" > "$outside/target.txt"
chmod 0600 "$outside/target.txt"
outside_pristine="$scratchdir/outside-pristine.txt"
cp -p "$outside/target.txt" "$outside_pristine"
# Symlink trap planted in the module by the local attacker.
ln -s "$outside" "$mod/subdir"
# Source files the sender will push: same size as the outside target,
# different content, mode 0666 (the perms the attacker tries to push).
SIZE=$(stat -c %s "$outside/target.txt" 2>/dev/null \
|| stat -f %z "$outside/target.txt")
head -c "$SIZE" /dev/urandom > "$src/target.txt"
head -c "$SIZE" /dev/urandom > "$src/subdir/target.txt"
chmod 0666 "$src/target.txt" "$src/subdir/target.txt"
cat > "$conf" <<EOF
use chroot = no
log file = $scratchdir/rsyncd.log
[upload]
path = $mod
use chroot = no
read only = no
EOF
reset_outside() {
chmod 0600 "$outside/target.txt"
echo "OUTSIDE_SECRET_DATA" > "$outside/target.txt"
}
verify_unchanged() {
label="$1"
mode=$(file_mode "$outside/target.txt")
case "$mode" in
600|0600) ;;
*) test_fail "$label: outside file mode changed from 600 to $mode (chmod escape)" ;;
esac
if ! cmp -s "$outside/target.txt" "$outside_pristine"; then
test_fail "$label: outside file content changed (write escape)"
fi
}
run_attack() {
label="$1"; shift
reset_outside
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC "$@" >/dev/null 2>&1 || true
verify_unchanged "$label"
}
# 1. The original poc_chmod scenario: single file, dest path with
# the symlinked subdir as a path component. With --size-only the
# receiver normally skips the basis open and goes straight to chmod
# -- only the chdir-escape blocks the chmod from reaching outside.
run_attack "single-file --size-only" \
-tp --size-only \
"$src/target.txt" rsync://localhost/upload/subdir/target.txt
# 2. -r push into the symlinked subdir: receiver chdir's into "subdir",
# follows the symlink, ends up in outside.
run_attack "-r --size-only into subdir/" \
-rtp --size-only \
"$src/subdir/" rsync://localhost/upload/subdir/
# 3. Same but no --size-only -- forces the basis-file open and a real
# rename, so this exercises the read-disclosure and write-escape
# paths together.
run_attack "-r without --size-only into subdir/" \
-rtp \
"$src/subdir/" rsync://localhost/upload/subdir/
# 4. -r src/ to upload/ -- this case was already covered by the
# original CVE-2026-29518 fix because the receiver stays at module
# root and operates on slashed paths. Regression check.
run_attack "-r --size-only into upload/ root" \
-rtp --size-only \
"$src/" rsync://localhost/upload/
exit 0

View File

@@ -1,68 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for the symlink-TOCTOU class of bug applied to
# chmod() on the receiver side. The CVE-2026-29518 fix used
# secure_relative_open() for the basis-file open, but every other
# path-based syscall the receiver runs on sender-controllable paths
# is vulnerable to the same primitive: a local attacker swaps a
# symlink into one of the parent directory components between the
# receiver's check and its act, and the syscall escapes the module.
#
# This test exercises the new do_chmod_at() wrapper via the
# t_chmod_secure helper. The helper sets up two scenarios:
# - a parent dir-symlink that resolves WITHIN the module tree
# (legitimate -K-style use, must continue to work)
# - a parent dir-symlink that escapes the module tree (the
# attack, must be rejected)
# plus two regression scenarios (plain relative path, top-level
# file) that just confirm the safe wrapper doesn't break the
# normal case.
#
# The kernel-enforced "stay below dirfd" path resolution is
# only available on Linux 5.6+, FreeBSD 13+, and macOS 15+.
# Skip on platforms that fall back to per-component O_NOFOLLOW
# (Solaris, OpenBSD, NetBSD, Cygwin); the per-component fallback
# would also reject the attack but the legitimate dir-symlink
# scenario would fail there.
. "$suitedir/rsync.fns"
case "$(uname -s)" in
SunOS|OpenBSD|NetBSD|CYGWIN*)
test_skipped "do_chmod_at relies on RESOLVE_BENEATH-equivalent kernel support not available on $(uname -s)"
;;
esac
mod="$scratchdir/module"
trap_outside="$scratchdir/trap"
rm -rf "$mod" "$trap_outside"
mkdir -p "$mod/realdir" "$trap_outside"
# Set up the four file-system objects the helper expects:
echo bystander > "$mod/realdir/sentinel"
chmod 0600 "$mod/realdir/sentinel"
echo target > "$trap_outside/sentinel"
chmod 0600 "$trap_outside/sentinel"
ln -s realdir "$mod/inside_link"
ln -s ../trap "$mod/escape_link"
echo top > "$mod/topfile"
chmod 0600 "$mod/topfile"
"$TOOLDIR/t_chmod_secure" "$mod" || \
test_fail "t_chmod_secure reported failures (see stderr above)"
# Sanity-check from the shell side too: the outside file's mode must
# still be 0600 -- the helper checked this, but a second look from
# the shell guards against a helper-internal stat() bug.
mode=$(stat -c '%a' "$trap_outside/sentinel" 2>/dev/null \
|| stat -f '%Lp' "$trap_outside/sentinel" 2>/dev/null)
if [ "$mode" != "600" ]; then
test_fail "outside sentinel mode changed from 600 to $mode -- chmod escaped the module"
fi
exit 0

View File

@@ -2,67 +2,26 @@
# clean-fname-underflow.test
# Ensure clean_fname() does not read-before-buffer when collapsing "..".
# This exercises the --server path where a crafted merge filename hits clean_fname().
#
# Usage:
# ./configure && make
# make check TESTS='clean-fname-underflow.test'
set -eu
. "$suitedir/rsync.fns"
# Try to find the just-built rsync binary if RSYNC_BIN isn't set.
if [ -z "${RSYNC_BIN:-}" ]; then
if [ -x "./rsync" ]; then
RSYNC_BIN=./rsync
elif [ -x "../rsync" ]; then
RSYNC_BIN=../rsync
else
RSYNC_BIN=rsync
fi
fi
workdir="${TMPDIR:-/tmp}/rsync-clean-fname.$$"
mkdir -p "$workdir"
# Solaris's rm refuses to delete a directory in the path of the cwd,
# so cd out before the trap runs.
trap 'cd /; rm -rf "$workdir"' EXIT INT TERM
workdir="$scratchdir/workdir"
mkdir -p "$workdir/mod"
cd "$workdir"
# Minimal rsyncd.conf using chroot so the crafted path reaches the server parser.
cat > rsyncd.conf <<'EOF'
pid file = rsyncd.pid
use chroot = true
[mod]
path = ./mod
read only = false
EOF
mkdir -p mod
# Start daemon on a random high port.
PORT=$(awk 'BEGIN{srand(); printf "%d", 20000+int(rand()*20000)}')
"$RSYNC_BIN" --daemon --no-detach --config=rsyncd.conf --port="$PORT" >/dev/null 2>&1 &
DAEMON_PID=$!
# Give the daemon a moment to come up.
sleep 0.3
rsync_bin=`echo $RSYNC | sed 's/ .*//'`
# Invoke the server-side path. We don't need a real transfer; we just want to
# ensure clean_fname() doesn't crash when given "a/../test" via --filter=merge.
EXIT_OK=0
if "$RSYNC_BIN" --server --sender -vlr --filter='merge a/../test' . mod/ >/dev/null 2>&1; then
EXIT_OK=1
if $rsync_bin --server --sender -vlr --filter='merge a/../test' . mod/ >/dev/null 2>&1; then
: # success
else
status=$?
# Non-zero exit is expected for bogus input; ensure it wasn't a signal/crash.
if [ $status -lt 128 ]; then
EXIT_OK=1
if [ $status -ge 128 ]; then
test_fail "rsync exited due to a signal (status=$status)"
fi
fi
kill "$DAEMON_PID" >/dev/null 2>&1 || true
if [ "$EXIT_OK" -ne 1 ]; then
echo "clean-fname-underflow.test: rsync exited due to a signal or unexpected status"
exit 1
fi
echo "OK: clean_fname() handled 'a/../test' without crashing"
exit 0

View File

@@ -1,98 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for codex audit Finding 3a: copy_file()'s source
# open in copy_altdest_file() is via do_open_nofollow(), which only
# refuses a final-component symlink. Parent components are still
# resolved with normal symlink-following. A daemon module attacker
# who plants a parent symlink at module/cd -> /outside, then runs
# --copy-dest=cd against a source file matching the size+mtime of
# /outside/target.txt, drives the receiver to:
#
# 1. Find a match-level >= 2 basis at "cd/target.txt"
# 2. Call copy_altdest_file -> copy_file(src="cd/target.txt", ...)
# 3. do_open_nofollow follows the "cd" parent symlink and reads
# the contents of /outside/target.txt under the daemon's
# authority
# 4. Copy that content into the module destination
#
# Result: outside/target.txt content lands at module/target.txt,
# accessible to the attacker on a subsequent pull.
#
# We detect by content: src/target.txt and outside/target.txt have
# identical metadata (size + mtime + mode) but different content.
# After the transfer, module/target.txt should match src (no
# basedir escape) -- if it matches outside, the bug copied across
# the symlink boundary.
. "$suitedir/rsync.fns"
mod="$scratchdir/module"
outside="$scratchdir/outside"
src="$scratchdir/src"
conf="$scratchdir/test-rsyncd.conf"
rm -rf "$mod" "$outside" "$src"
mkdir -p "$mod" "$outside" "$src"
# Outside-the-module file the daemon should not read on the
# attacker's behalf.
echo "OUTSIDE_LEAKED_DATA!" > "$outside/target.txt"
chmod 0644 "$outside/target.txt"
# The symlink trap.
ln -s "$outside" "$mod/cd"
# Source: same size, same mtime, same mode as outside -- so the
# generator's link_stat + quick_check_ok finds a match-level >= 2
# basis and calls copy_altdest_file.
echo "ATTACKER_KNOWN_DATA!" > "$src/target.txt"
touch -r "$outside/target.txt" "$src/target.txt"
chmod 0644 "$src/target.txt"
# When running as root the daemon would drop to "nobody" by
# default and fail to mkstemp in the scratch dir; force it to
# keep our uid/gid in that case.
my_uid=`get_testuid`
root_uid=`get_rootuid`
root_gid=`get_rootgid`
uid_setting="uid = $root_uid"
gid_setting="gid = $root_gid"
if test x"$my_uid" != x"$root_uid"; then
uid_setting="#$uid_setting"
gid_setting="#$gid_setting"
fi
cat > "$conf" <<EOF
use chroot = no
$uid_setting
$gid_setting
log file = $scratchdir/rsyncd.log
[upload]
path = $mod
use chroot = no
read only = no
EOF
# --copy-dest push to module root.
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC -rtp --copy-dest=cd "$src/" rsync://localhost/upload/ \
>/dev/null 2>&1 || true
if [ ! -f "$mod/target.txt" ]; then
test_fail "destination file was not created -- daemon transfer failed before the test could observe the basedir behaviour"
fi
if cmp -s "$mod/target.txt" "$outside/target.txt"; then
test_fail "basedir-escape via copy_file source: module/target.txt now contains the contents of outside/target.txt -- daemon read /outside via the cd symlink and copied it into the module"
fi
if ! cmp -s "$mod/target.txt" "$src/target.txt"; then
test_fail "destination doesn't match source content (and isn't outside content either): unexpected state"
fi
exit 0

View File

@@ -1,111 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for GHSA-rjfm-3w2m-jf4f: a hostname-based "hosts deny"
# rule must still match when the daemon performs a 'daemon chroot' and
# the chroot does not contain the NSS files glibc needs for reverse DNS.
#
# Pre-fix, reverse DNS happened *after* the daemon chroot. With an empty
# chroot the NSS lookup failed, client_name() returned "UNKNOWN", and a
# deny rule referring to the connecting hostname silently failed to
# match.
#
# Two scenarios are exercised so we can distinguish the case the fix
# definitely covers from the per-module path that may still be
# vulnerable:
# A. global "reverse lookup = yes" (covered by b6abdb4c)
# B. only module "reverse lookup = yes" (gap to verify)
. "$suitedir/rsync.fns"
case `uname -s` in
Linux*) ;;
*) test_skipped "test is Linux-specific (uses chroot+unshare)" ;;
esac
# We need CAP_SYS_CHROOT. Re-exec under a user namespace if not root.
if ! chroot / /bin/true 2>/dev/null; then
if [ -z "$RSYNC_UNSHARED" ] && unshare --user --map-root-user true 2>/dev/null; then
echo "Re-running under unshare --user --map-root-user..."
RSYNC_UNSHARED=1 exec unshare --user --map-root-user "$SHELL_PATH" $RUNSHFLAGS "$0"
fi
test_skipped "need CAP_SYS_CHROOT (root or unshare --user --map-root-user)"
fi
# We need 127.0.0.1 to reverse-resolve to a real hostname while NSS is
# still working (i.e. before the daemon's chroot). The daemon will
# look that name up itself as part of its hostname-based ACL check;
# we then deny that name and assert the connection is rejected.
client_hostname=`getent hosts 127.0.0.1 2>/dev/null | awk 'NR==1 {print $2}'`
if [ -z "$client_hostname" ] || [ "$client_hostname" = "127.0.0.1" ]; then
test_skipped "no reverse DNS for 127.0.0.1"
fi
chrootdir="$scratchdir/chroot"
rm -rf "$chrootdir"
mkdir -p "$chrootdir/modroot"
echo "from chroot" > "$chrootdir/modroot/file1"
conf="$scratchdir/test-rsyncd.conf"
logfile="$scratchdir/rsyncd.log"
write_conf() {
cat >"$conf" <<EOF
use chroot = no
log file = $logfile
daemon chroot = $chrootdir
reverse lookup = $1
hosts deny = $client_hostname
max verbosity = 4
[chrootmod]
path = /modroot
read only = yes
reverse lookup = $2
EOF
}
# Run a transfer and return 0 if the daemon refused with @ERROR access
# denied (the expected outcome when the deny rule matches).
run_check() {
label="$1"
rm -f "$logfile"
rm -rf "$todir"
mkdir -p "$todir"
out="$scratchdir/run.out"
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC -av localhost::chrootmod/ "$todir/" >"$out" 2>&1
rc=$?
echo "----- $label (rsync exit $rc):"
cat "$out"
echo "----- daemon log:"
[ -f "$logfile" ] && cat "$logfile"
echo "-----"
grep -q '@ERROR.*access denied' "$out"
}
# Scenario A: global reverse lookup. Covered by b6abdb4c.
write_conf yes yes
if ! run_check "Scenario A (global reverse lookup = yes)"; then
test_fail "Scenario A: hostname deny rule was bypassed"
fi
# Scenario B: only the per-module reverse-lookup setting is enabled.
# The b6abdb4c fix only pre-warms client_name()'s cache when the
# global setting is on, so the post-chroot lookup in this path may
# still produce "UNKNOWN" and bypass the deny rule.
write_conf no yes
if ! run_check "Scenario B (per-module reverse lookup only)"; then
test_fail "Scenario B: hostname deny rule was bypassed (per-module reverse lookup with daemon chroot still has the bypass)"
fi
exit 0

View File

@@ -1,51 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Test that a daemon module configured with "refuse options = compress"
# rejects clients that ask for compression and still serves the same
# transfer when the client does not.
. "$suitedir/rsync.fns"
build_rsyncd_conf
# Append a module that refuses --compress (-z).
cat >>"$conf" <<EOF
[no-compress]
path = $fromdir
read only = yes
refuse options = compress
EOF
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon"
export RSYNC_CONNECT_PROG
hands_setup
# Build a reference tree mirroring the daemon's global exclude rule.
$RSYNC -av --exclude=foobar.baz "$fromdir/" "$chkdir/"
# A compressed transfer must be refused.
errlog="$scratchdir/refuse.err"
if $RSYNC -avz localhost::no-compress/ "$todir/" >/dev/null 2>"$errlog"; then
cat "$errlog" >&2
test_fail "compressed transfer was not refused"
fi
grep -- '--compress' "$errlog" >/dev/null || {
cat "$errlog" >&2
test_fail "expected refuse error mentioning --compress"
}
# The same transfer without -z must succeed.
rm -rf "$todir"
mkdir "$todir"
checkit "$RSYNC -av localhost::no-compress/ '$todir/'" "$chkdir" "$todir"
# The script would have aborted on error, so getting here means we've won.
exit 0

View File

@@ -16,9 +16,9 @@ makepath "$longdir" || test_skipped "unable to create long directory"
touch "$longdir/1" || test_skipped "unable to create files in long directory"
date > "$longdir/1"
if [ -r /etc ]; then
ls -la /etc >"$longdir/2"
ls -la /etc >"$longdir/2" || [ $? -eq 1 ]
else
ls -la / >"$longdir/2"
ls -la / >"$longdir/2" || [ $? -eq 1 ]
fi
checkit "$RSYNC --delete -avH '$fromdir/' '$todir'" "$fromdir/" "$todir"

View File

@@ -0,0 +1,32 @@
#!/bin/sh
# Test rsync --open-noatime option keeps source atimes intact
. "$suitedir/rsync.fns"
$RSYNC -VV | grep '"atimes": true' >/dev/null || test_skipped "Rsync is configured without atimes support"
# O_NOATIME is Linux-specific; skip on other platforms
case `uname` in
Linux) ;;
*) test_skipped "O_NOATIME is only supported on Linux" ;;
esac
mkdir "$fromdir"
# --open-noatime did not work properly on files with size > 0
echo content > "$fromdir/foo"
touch -a -t 200102031717.42 "$fromdir/foo"
TLS_ARGS=--atimes
"$TOOLDIR/tls" $TLS_ARGS "$fromdir/foo" > "$tmpdir/atime-from-before"
# Do not use checkit because it uses "diff" which breaks atimes
$RSYNC --open-noatime --archive --recursive --times --atimes -vvv "$fromdir/" "$todir/"
"$TOOLDIR/tls" $TLS_ARGS "$fromdir/foo" > "$tmpdir/atime-from-after"
diff "$tmpdir/atime-from-before" "$tmpdir/atime-from-after"
# The script would have aborted on error, so getting here means we've won.
exit 0

View File

@@ -1,128 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for the off-by-one stack OOB write in
# establish_proxy_connection() in socket.c when a malicious or
# man-in-the-middle HTTP proxy returns a first response line of
# 1023+ bytes without a '\n' terminator.
#
# Pre-fix: the read loop walked buffer[0..sizeof-2] one byte at a
# time, then post-loop logic did "if (*cp != '\n') cp++; *cp-- =
# '\0';". If no newline arrived before the loop filled the buffer,
# cp was left at &buffer[sizeof-1] (never written by the loop),
# *cp held stale stack bytes, and cp++ pushed cp one past the array.
# The null-termination then wrote one byte out of bounds on the
# stack. AddressSanitizer reports stack-buffer-overflow at the
# null-termination site.
#
# Post-fix: the bound-exhaustion case is detected by position and
# rejected with an "proxy response line too long" message, so no
# OOB write occurs and rsync exits with a non-signal status.
. "$suitedir/rsync.fns"
command -v python3 >/dev/null 2>&1 || test_skipped "python3 not available"
workdir="$scratchdir/workdir"
mkdir -p "$workdir"
cd "$workdir"
port_file="$workdir/port"
proxy_log="$workdir/proxy.log"
# A minimal TCP listener: binds to an ephemeral port on 127.0.0.1,
# writes the chosen port to $port_file *before* accept() so the test
# can synchronise without a sleep, accepts one connection, reads
# until end-of-headers or 64 KiB, sends exactly 1023 bytes of 'X'
# with no '\n', then closes.
python3 - "$port_file" >"$proxy_log" 2>&1 <<'PYEOF' &
import socket, sys, os
port_file = sys.argv[1]
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("127.0.0.1", 0))
port = s.getsockname()[1]
tmp = port_file + ".tmp"
with open(tmp, "w") as fp:
fp.write("%d\n" % port)
os.rename(tmp, port_file) # atomic visibility to the shell side
s.listen(1)
conn, _ = s.accept()
conn.settimeout(5)
try:
data = b""
while b"\r\n\r\n" not in data and len(data) < 65536:
chunk = conn.recv(8192)
if not chunk:
break
data += chunk
except socket.timeout:
pass
conn.sendall(b"X" * 1023) # exactly the buffer-1 trigger size
try:
conn.shutdown(socket.SHUT_RDWR)
except OSError:
pass
conn.close()
s.close()
PYEOF
proxy_pid=$!
# Wait up to ~10s for the listener to publish its port.
i=0
while [ ! -s "$port_file" ] && [ $i -lt 10 ]; do
sleep 1
i=$((i + 1))
done
if [ ! -s "$port_file" ]; then
kill "$proxy_pid" 2>/dev/null
cat "$proxy_log" >&2 2>/dev/null
test_fail "proxy listener never published a port"
fi
port=`cat "$port_file"`
case "$port" in
*[!0-9]*|"") kill "$proxy_pid" 2>/dev/null; test_fail "bogus port from listener: '$port'" ;;
esac
# Run rsync through the malicious proxy. Any rsync:// URL works:
# the proxy intercepts the CONNECT and never forwards anywhere.
rsync_err="$workdir/rsync.err"
# rsync MUST exit non-zero here (the proxy is misbehaving).
# Use `|| status=$?` so we capture the real exit code under `sh -e`;
# `if ! cmd; then status=$?` would only ever see 0 because the `!`
# is the last command before `$?`.
status=0
RSYNC_PROXY="127.0.0.1:$port" \
$RSYNC rsync://example.invalid:873/whatever/ "$workdir/out/" \
>/dev/null 2>"$rsync_err" || status=$?
# Reap the listener.
wait "$proxy_pid" 2>/dev/null || true
# 1. rsync must not have crashed (SIGSEGV/SIGABRT report >= 128).
if [ "$status" -ge 128 ]; then
cat "$rsync_err" >&2
test_fail "rsync killed by signal (status=$status) -- possible stack OOB regression"
fi
# 2. rsync must have actually exited non-zero (i.e. saw the bad proxy).
if [ "$status" -eq 0 ]; then
cat "$rsync_err" >&2
test_fail "rsync returned success despite malformed proxy response"
fi
# 3. The new error message must appear.
if ! grep -q "proxy response line too long" "$rsync_err"; then
cat "$rsync_err" >&2
test_fail "expected 'proxy response line too long' in rsync stderr"
fi
echo "OK: over-long proxy response line rejected cleanly without crashing"
exit 0

View File

@@ -97,7 +97,7 @@ printmsg() {
}
rsync_ls_lR() {
find "$@" -name .git -prune -o -name auto-build-save -prune -o -print | \
find "$@" -name .git -prune -o -name auto-build-save -prune -o -name testtmp -prune -o -print | \
sort | sed 's/ /\\ /g' | xargs "$TOOLDIR/tls" $TLS_ARGS
}
@@ -195,15 +195,15 @@ hands_setup() {
echo some data > "$fromdir/dir/subdir/foobar.baz"
mkdir "$fromdir/dir/subdir/subsubdir"
if [ -r /etc ]; then
ls -ltr /etc > "$fromdir/dir/subdir/subsubdir/etc-ltr-list"
ls -ltr /etc > "$fromdir/dir/subdir/subsubdir/etc-ltr-list" || [ $? -eq 1 ]
else
ls -ltr / > "$fromdir/dir/subdir/subsubdir/etc-ltr-list"
ls -ltr / > "$fromdir/dir/subdir/subsubdir/etc-ltr-list" || [ $? -eq 1 ]
fi
mkdir "$fromdir/dir/subdir/subsubdir2"
if [ -r /bin ]; then
ls -lt /bin > "$fromdir/dir/subdir/subsubdir2/bin-lt-list"
ls -lt /bin > "$fromdir/dir/subdir/subsubdir2/bin-lt-list" || [ $? -eq 1 ]
else
ls -lt / > "$fromdir/dir/subdir/subsubdir2/bin-lt-list"
ls -lt / > "$fromdir/dir/subdir/subsubdir2/bin-lt-list" || [ $? -eq 1 ]
fi
# echo testing head:

View File

@@ -1,34 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for codex audit Finding 5: secure_relative_open()'s
# front-door input check rejects "../foo" and "foo/../bar" but
# misses bare "..", "subdir/..", and other variants whose "/"-split
# components contain a literal "..". The kernel-enforced
# RESOLVE_BENEATH (Linux 5.6+) and O_RESOLVE_BENEATH
# (FreeBSD 13+, macOS 15+) reject these in-kernel; the per-component
# walk fallback used on NetBSD, OpenBSD, Solaris, Cygwin and pre-5.6
# Linux does not -- so the validation must happen at the front door.
#
# This test invokes the t_secure_relpath helper, which calls
# secure_relative_open() with each suspect input and verifies the
# return value is -1 with errno == EINVAL. EINVAL is the marker
# that the front-door rejected the input, not the kernel; pre-fix
# the kernel returns -1 with EXDEV (or, on the per-component
# fallback, may return a valid fd at all -- "escape").
. "$suitedir/rsync.fns"
testdir="$scratchdir/relpath-test"
rm -rf "$testdir"
mkdir -p "$testdir"
if ! "$TOOLDIR/t_secure_relpath" "$testdir"; then
test_fail "t_secure_relpath rejected one or more inputs incorrectly (see stderr above for the specific case)"
fi
exit 0

View File

@@ -1,90 +0,0 @@
#!/bin/sh
# Copyright (C) 2026 by Andrew Tridgell
# This program is distributable under the terms of the GNU GPL (see
# COPYING).
# Regression test for codex re-check finding: the sender-side file-
# list generator can still follow an attacker-planted symlink out of
# the module via change_pathname() -> change_dir(...,CD_SKIP_CHDIR)
# followed by change_dir(...,CD_NORMAL). The CD_SKIP_CHDIR sets
# skipped_chdir=1, and the next CD_NORMAL call's secure-branch in
# util1.c is gated on `!skipped_chdir`, so the secure path is
# bypassed and a raw chdir(curr_dir) follows attacker-controlled
# symlinks during flist generation.
#
# Reach: rsync daemon module with `use chroot = no`. A local
# attacker plants module/cd -> /outside. A client (innocent or
# malicious) pulls rsync://<daemon>/<module>/cd/. The daemon, as
# sender, enumerates files in /outside and ships their metadata
# (names, sizes, modes, mtimes) to the client. The actual content
# transfer fails later at the secure_relative_open step with EXDEV,
# but by then the metadata has already leaked.
#
# We detect by running a dry-run pull of the symlinked subdir and
# checking whether the client's --list-only output mentions any
# file from /outside. With the bug, /outside/secret.txt appears in
# the list with its size; with the fix, the daemon's chdir into
# the symlinked subdir is rejected and no /outside file is listed.
. "$suitedir/rsync.fns"
case "$(uname -s)" in
SunOS|OpenBSD|NetBSD|CYGWIN*)
test_skipped "secure change_dir relies on RESOLVE_BENEATH-equivalent kernel support not available on $(uname -s)"
;;
esac
mod="$scratchdir/module"
outside="$scratchdir/outside"
listfile="$scratchdir/listed.txt"
conf="$scratchdir/test-rsyncd.conf"
rm -rf "$mod" "$outside"
mkdir -p "$mod" "$outside"
# Outside-the-module file the daemon should NOT enumerate to clients.
# A distinctive name + non-trivial size makes the leak easy to spot.
echo "OUTSIDE_PROTECTED_FILE_USED_AS_LEAK_DETECTOR" > "$outside/leak_marker.txt"
chmod 0644 "$outside/leak_marker.txt"
# The symlink trap planted by the local attacker.
ln -s "$outside" "$mod/cd"
my_uid=`get_testuid`
root_uid=`get_rootuid`
root_gid=`get_rootgid`
uid_setting="uid = $root_uid"
gid_setting="gid = $root_gid"
if test x"$my_uid" != x"$root_uid"; then
uid_setting="#$uid_setting"
gid_setting="#$gid_setting"
fi
cat > "$conf" <<EOF
use chroot = no
$uid_setting
$gid_setting
log file = $scratchdir/rsyncd.log
[upload]
path = $mod
use chroot = no
read only = no
EOF
# Pull recursively into the symlinked subdir with dry-run + verbose,
# capturing the daemon's flist (file list) on stdout. If the daemon
# enumerates /outside, leak_marker.txt will appear in the listing.
RSYNC_CONNECT_PROG="$RSYNC --config=$conf --daemon" \
$RSYNC -nrv rsync://localhost/upload/cd/ "$scratchdir/dst/" \
> "$listfile" 2>&1 || true
if grep -q "leak_marker\.txt" "$listfile"; then
echo "----- leaked listing follows" >&2
sed 's/^/ /' "$listfile" >&2
echo "----- leaked listing ends" >&2
test_fail "sender flist leak: outside/leak_marker.txt was enumerated to the client (daemon's chdir followed the cd symlink during flist generation)"
fi
exit 0

View File

@@ -6,7 +6,7 @@
# This program is distributable under the terms of the GNU GPL (see
# COPYING)
# This script tests ssh, if possible. It's called by runtests.sh
# This script tests ssh, if possible. It's called by runtests.py
. "$suitedir/rsync.fns"

View File

@@ -1,259 +0,0 @@
#!/bin/sh
# Test that updating a file through a directory symlink works when using
# -K (--copy-dirlinks). This is a regression test for:
# https://github.com/RsyncProject/rsync/issues/715
#
# The CVE fix in commit c35e283 introduced secure_relative_open() which
# uses O_NOFOLLOW on all path components, breaking legitimate directory
# symlinks on the receiver side. The fix splits the path into basedir
# (dirname, symlinks followed) and basename (O_NOFOLLOW) so that
# directory symlinks are traversed while the final file component is
# still protected.
#
# The regression only manifests when delta matching is triggered (i.e.,
# the sender finds matching blocks in the old file). Small files with
# completely different content are transferred in full and don't trigger
# the bug. We use a large file with a small modification to ensure
# delta transfer is used.
#
# In addition to the original regression, this test covers edge cases
# in the fix itself:
# - --backup with directory symlinks (finish_transfer pointer identity)
# - --partial-dir with protocol < 29 (fnamecmp != partialptr guard)
# - --inplace with directory symlinks (updating_basis_or_equiv check)
# - Files without a dirname (top-level files, no split needed)
. "$suitedir/rsync.fns"
# secure_relative_open() uses kernel-enforced "stay below dirfd" via
# openat2(RESOLVE_BENEATH) on Linux 5.6+ and openat(O_RESOLVE_BENEATH)
# on FreeBSD 13+. Other platforms fall back to a per-component
# O_NOFOLLOW walk that rejects every symlink including legitimate
# directory symlinks -- the very case this test exercises. Skip on
# those rather than report a known failure.
case "$(uname -s)" in
SunOS|OpenBSD|NetBSD|CYGWIN*)
test_skipped "secure_relative_open lacks RESOLVE_BENEATH equivalent on $(uname -s); issue #715 still affects this platform"
;;
esac
RSYNC_RSH="$scratchdir/src/support/lsh.sh"
export RSYNC_RSH
# $HOME is set to $scratchdir by rsync.fns
# localhost: destination will cd to $HOME (i.e., $scratchdir)
# Helper: create a large file suitable for delta transfers.
# ~32KB is large enough for rsync's block matching to find matches.
make_testfile() {
dd if=/dev/urandom of="$1" bs=1024 count=32 2>/dev/null \
|| test_fail "failed to create test file $1"
}
# Set up source tree
srcbase="$tmpdir/src"
######################################################################
# Test 1: Basic directory symlink update (the original issue #715)
######################################################################
mkdir -p "$HOME/real-dir"
ln -s real-dir "$HOME/dir"
mkdir -p "$srcbase/dir"
make_testfile "$srcbase/dir/file"
# First transfer (initial): should create the file through the symlink
(cd "$srcbase" && $RSYNC -KRlptv --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 1: initial transfer failed"
if [ ! -f "$HOME/real-dir/file" ]; then
test_fail "test 1: initial transfer did not create file through symlink"
fi
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 1: initial transfer content mismatch"
# Small modification to trigger delta transfer
echo "appended update" >> "$srcbase/dir/file"
sleep 1
touch "$srcbase/dir/file"
# Second transfer (update): was failing with "failed verification"
(cd "$srcbase" && $RSYNC -KRlptv --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 1: update through directory symlink failed"
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 1: update transfer content mismatch"
######################################################################
# Test 2: Compression (-z) as in the original reproducer
######################################################################
echo "another line" >> "$srcbase/dir/file"
sleep 1
touch "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptzv --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 2: compressed update through directory symlink failed"
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 2: compressed update content mismatch"
######################################################################
# Test 3: Nested directory symlinks (nested/sub/data.txt where
# "nested" is a symlink to "nested_real")
######################################################################
mkdir -p "$HOME/nested_real/sub"
ln -s nested_real "$HOME/nested"
mkdir -p "$srcbase/nested/sub"
make_testfile "$srcbase/nested/sub/data.txt"
(cd "$srcbase" && $RSYNC -KRlptv --rsync-path="$RSYNC" nested/sub/data.txt localhost:) \
|| test_fail "test 3: initial nested transfer failed"
echo "appended nested" >> "$srcbase/nested/sub/data.txt"
sleep 1
touch "$srcbase/nested/sub/data.txt"
(cd "$srcbase" && $RSYNC -KRlptv --rsync-path="$RSYNC" nested/sub/data.txt localhost:) \
|| test_fail "test 3: update through nested directory symlink failed"
diff "$srcbase/nested/sub/data.txt" "$HOME/nested_real/sub/data.txt" >/dev/null \
|| test_fail "test 3: nested update content mismatch"
######################################################################
# Test 4: --backup with directory symlinks
#
# Exercises the finish_transfer() "fnamecmp == fname" pointer
# comparison that determines whether to update fnamecmp to the
# backup name. If broken, --backup would reference a renamed file
# for xattr handling.
######################################################################
# Reset destination
rm -f "$HOME/real-dir/file" "$HOME/real-dir/file~"
make_testfile "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 4: initial transfer for backup test failed"
echo "backup update" >> "$srcbase/dir/file"
sleep 1
touch "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --backup --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 4: update with --backup through directory symlink failed"
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 4: backup update content mismatch"
if [ ! -f "$HOME/real-dir/file~" ]; then
test_fail "test 4: backup file was not created"
fi
######################################################################
# Test 5: --inplace with directory symlinks
#
# Exercises the updating_basis_or_equiv check which uses
# "fnamecmp == fname". With --inplace, rsync writes directly to
# the destination file instead of a temp file.
######################################################################
rm -f "$HOME/real-dir/file" "$HOME/real-dir/file~"
make_testfile "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --inplace --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 5: initial inplace transfer failed"
echo "inplace update" >> "$srcbase/dir/file"
sleep 1
touch "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --inplace --rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 5: inplace update through directory symlink failed"
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 5: inplace update content mismatch"
######################################################################
# Test 6: Top-level file (no dirname, no split needed)
#
# Ensures the dirname/basename split is not attempted for files
# at the top level (file->dirname is NULL).
######################################################################
make_testfile "$srcbase/topfile"
mkdir -p "$HOME"
(cd "$srcbase" && $RSYNC -Rlptv --rsync-path="$RSYNC" topfile localhost:) \
|| test_fail "test 6: initial top-level transfer failed"
echo "toplevel update" >> "$srcbase/topfile"
sleep 1
touch "$srcbase/topfile"
(cd "$srcbase" && $RSYNC -Rlptv --rsync-path="$RSYNC" topfile localhost:) \
|| test_fail "test 6: top-level update failed"
diff "$srcbase/topfile" "$HOME/topfile" >/dev/null \
|| test_fail "test 6: top-level update content mismatch"
######################################################################
# Test 7: --partial-dir with protocol < 29
#
# For protocol < 29, fnamecmp_type stays FNAMECMP_FNAME even when
# fnamecmp is set to partialptr. The dirname/basename split must
# NOT trigger in this case (guarded by "fnamecmp == fname").
######################################################################
rm -f "$HOME/real-dir/file"
make_testfile "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --protocol=28 --partial-dir=.rsync-partial \
--rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 7: initial proto28 partial-dir transfer failed"
echo "partial-dir update" >> "$srcbase/dir/file"
sleep 1
touch "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --protocol=28 --partial-dir=.rsync-partial \
--rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 7: proto28 partial-dir update through dirlink failed"
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 7: proto28 partial-dir update content mismatch"
######################################################################
# Test 8: Protocol < 29 basic directory symlink update
#
# Exercises the protocol < 29 code path and its fallback logic
# (clearing basedir on retry).
######################################################################
rm -f "$HOME/real-dir/file"
make_testfile "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --protocol=28 \
--rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 8: initial proto28 transfer failed"
echo "proto28 update" >> "$srcbase/dir/file"
sleep 1
touch "$srcbase/dir/file"
(cd "$srcbase" && $RSYNC -KRlptv --protocol=28 \
--rsync-path="$RSYNC" dir/file localhost:) \
|| test_fail "test 8: proto28 update through directory symlink failed"
diff "$srcbase/dir/file" "$HOME/real-dir/file" >/dev/null \
|| test_fail "test 8: proto28 update content mismatch"
# The script would have aborted on error, so getting here means we've won.
exit 0

View File

@@ -38,10 +38,7 @@ EOF
xls() {
for fn in "${@}"; do
runat "$fn" "$SHELL_PATH" <<EOF
for x in *; do
case "\$x" in SUNWattr_*) continue;; esac
echo "\$x=\`cat \$x\`"
done
for x in *; do echo "\$x=\`cat \$x\`"; done
EOF
done
}

2
tls.c
View File

@@ -230,7 +230,7 @@ static void list_file(const char *fname)
mtimebuf, atimebuf, crtimebuf, fname, linkbuf);
}
static struct poptOption long_options[] = {
static const struct poptOption long_options[] = {
/* longName, shortName, argInfo, argPtr, value, descrip, argDesc */
{"atimes", 'U', POPT_ARG_NONE, &display_atimes, 0, 0, 0},
#ifdef SUPPORT_CRTIMES

138
token.c
View File

@@ -33,6 +33,7 @@ extern int do_compression;
extern int protocol_version;
extern int module_id;
extern int do_compression_level;
extern int do_compression_threads;
extern char *skip_compress;
#ifndef Z_INSERT_ONLY
@@ -291,14 +292,6 @@ static int32 simple_recv_token(int f, char **data)
int32 i = read_int(f);
if (i <= 0)
return i;
/* simple_send_token caps each literal chunk at CHUNK_SIZE;
* reject anything larger so a hostile peer cannot drive the
* read_buf below past our static CHUNK_SIZE buffer. */
if (i > CHUNK_SIZE) {
rprintf(FERROR, "invalid uncompressed token length %ld [%s]\n",
(long)i, who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
residue = i;
}
@@ -501,52 +494,9 @@ static char *cbuf;
static char *dbuf;
/* for decoding runs of tokens */
#define MAX_TOKEN_INDEX ((int32)0x7ffffffe)
static int32 rx_token;
static int32 rx_run;
static NORETURN void invalid_compressed_token(void)
{
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
static int32 recv_compressed_token_num(int f, int32 flag)
{
if (flag & TOKEN_REL) {
int32 incr = flag & 0x3f;
if (rx_token > MAX_TOKEN_INDEX - incr)
invalid_compressed_token();
rx_token += incr;
flag >>= 6;
} else {
rx_token = read_int(f);
if (rx_token < 0 || rx_token > MAX_TOKEN_INDEX)
invalid_compressed_token();
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;
if (rx_run <= 0 || rx_token > MAX_TOKEN_INDEX - rx_run)
invalid_compressed_token();
recv_state = r_running;
}
return -1 - rx_token;
}
static int32 recv_compressed_token_run(void)
{
if (rx_run <= 0 || rx_token >= MAX_TOKEN_INDEX)
invalid_compressed_token();
++rx_token;
if (--rx_run == 0)
recv_state = r_idle;
return -1 - rx_token;
}
/* Receive a deflated token and inflate it */
static int32 recv_deflated_token(int f, char **data)
{
@@ -637,7 +587,22 @@ static int32 recv_deflated_token(int f, char **data)
}
/* here we have a token of some kind */
return recv_compressed_token_num(f, flag);
if (flag & TOKEN_REL) {
rx_token += flag & 0x3f;
flag >>= 6;
} else {
rx_token = read_int(f);
if (rx_token < 0) {
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;
recv_state = r_running;
}
return -1 - rx_token;
case r_inflating:
rx_strm.next_out = (Bytef *)dbuf;
@@ -657,7 +622,10 @@ static int32 recv_deflated_token(int f, char **data)
break;
case r_running:
return recv_compressed_token_run();
++rx_token;
if (--rx_run == 0)
recv_state = r_idle;
return -1 - rx_token;
}
}
}
@@ -730,6 +698,8 @@ static void send_zstd_token(int f, int32 token, struct map_struct *buf, OFF_T of
obuf = new_array(char, OBUF_SIZE);
ZSTD_CCtx_setParameter(zstd_cctx, ZSTD_c_compressionLevel, do_compression_level);
ZSTD_CCtx_setParameter(zstd_cctx, ZSTD_c_nbWorkers, do_compression_threads);
zstd_out_buff.dst = obuf + 2;
comp_init_done = 1;
@@ -767,12 +737,11 @@ static void send_zstd_token(int f, int32 token, struct map_struct *buf, OFF_T of
zstd_in_buff.src = map_ptr(buf, offset, nb);
zstd_in_buff.size = nb;
zstd_in_buff.pos = 0;
int finished;
do {
if (zstd_out_buff.size == 0) {
zstd_out_buff.size = MAX_DATA_COUNT;
zstd_out_buff.pos = 0;
}
zstd_out_buff.size = MAX_DATA_COUNT;
zstd_out_buff.pos = 0;
/* File ended, flush */
if (token != -2)
@@ -790,20 +759,21 @@ static void send_zstd_token(int f, int32 token, struct map_struct *buf, OFF_T of
* state and send a smaller buffer so that the remote side can
* finish the file.
*/
if (zstd_out_buff.pos == zstd_out_buff.size || flush == ZSTD_e_flush) {
finished = (flush == ZSTD_e_flush) ? (r == 0) : (zstd_in_buff.pos == zstd_in_buff.size);
if (zstd_out_buff.pos != 0) {
n = zstd_out_buff.pos;
obuf[0] = DEFLATED_DATA + (n >> 8);
obuf[1] = n;
write_buf(f, obuf, n+2);
zstd_out_buff.size = 0;
}
/*
* Loop while the input buffer isn't full consumed or the
* internal state isn't fully flushed.
*/
} while (zstd_in_buff.pos < zstd_in_buff.size || r > 0);
} while (!finished);
flush_pending = token == -2;
}
@@ -866,7 +836,22 @@ static int32 recv_zstd_token(int f, char **data)
return 0;
}
/* here we have a token of some kind */
return recv_compressed_token_num(f, flag);
if (flag & TOKEN_REL) {
rx_token += flag & 0x3f;
flag >>= 6;
} else {
rx_token = read_int(f);
if (rx_token < 0) {
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;
recv_state = r_running;
}
return -1 - rx_token;
case r_inflated: /* zstd doesn't get into this state */
break;
@@ -897,7 +882,10 @@ static int32 recv_zstd_token(int f, char **data)
break;
case r_running:
return recv_compressed_token_run();
++rx_token;
if (--rx_run == 0)
recv_state = r_idle;
return -1 - rx_token;
}
}
}
@@ -1017,7 +1005,22 @@ static int32 recv_compressed_token(int f, char **data)
}
/* here we have a token of some kind */
return recv_compressed_token_num(f, flag);
if (flag & TOKEN_REL) {
rx_token += flag & 0x3f;
flag >>= 6;
} else {
rx_token = read_int(f);
if (rx_token < 0) {
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;
recv_state = r_running;
}
return -1 - rx_token;
case r_inflating:
avail_out = LZ4_decompress_safe(next_in, dbuf, avail_in, size);
@@ -1033,7 +1036,10 @@ static int32 recv_compressed_token(int f, char **data)
break;
case r_running:
return recv_compressed_token_run();
++rx_token;
if (--rx_run == 0)
recv_state = r_idle;
return -1 - rx_token;
}
}
}

104
util1.c
View File

@@ -141,7 +141,7 @@ int set_times(const char *fname, STRUCT_STAT *stp)
#ifdef HAVE_UTIMENSAT
#include "case_N.h"
if (do_utimensat_at(fname, stp) == 0)
if (do_utimensat(fname, stp) == 0)
break;
if (errno != ENOSYS)
return -1;
@@ -336,13 +336,7 @@ static int unlink_and_reopen(const char *dest, mode_t mode)
mode |= S_IWUSR;
#endif
mode &= INITACCESSPERMS;
/* Use do_open_at so the create/truncate goes through a secure
* parent dirfd in the daemon-no-chroot deployment. Otherwise
* an attacker could swap a parent component with a symlink in
* the window between robust_unlink (which uses do_unlink_at,
* already secure) and the create here, and redirect the new
* file outside the module. */
if ((ofd = do_open_at(dest, O_WRONLY | O_CREAT | O_TRUNC | O_EXCL, mode)) < 0) {
if ((ofd = do_open(dest, O_WRONLY | O_CREAT | O_TRUNC | O_EXCL, mode)) < 0) {
int save_errno = errno;
rsyserr(FERROR_XFER, save_errno, "open %s", full_fname(dest));
errno = save_errno;
@@ -366,23 +360,12 @@ static int unlink_and_reopen(const char *dest, mode_t mode)
* --copy-dest options. */
int copy_file(const char *source, const char *dest, int tmpfilefd, mode_t mode)
{
extern int am_daemon, am_chrooted;
int ifd, ofd;
char buf[1024 * 8];
int len; /* Number of bytes read into `buf'. */
OFF_T prealloc_len = 0, offset = 0;
/* On a daemon without chroot, route the source open through
* secure_relative_open so a parent-symlink on the source path
* (e.g. --copy-dest=cd where cd is a symlink to an outside
* directory) cannot redirect the read to a file the daemon can
* see but the attacker should not. Plain do_open_nofollow only
* refuses a final-component symlink; parents are still followed. */
if (am_daemon && !am_chrooted && source && *source && source[0] != '/')
ifd = secure_relative_open(NULL, source, O_RDONLY | O_NOFOLLOW, 0);
else
ifd = do_open_nofollow(source, O_RDONLY);
if (ifd < 0) {
if ((ifd = do_open_nofollow(source, O_RDONLY)) < 0) {
int save_errno = errno;
rsyserr(FERROR_XFER, errno, "open %s", full_fname(source));
errno = save_errno;
@@ -496,13 +479,13 @@ int copy_file(const char *source, const char *dest, int tmpfilefd, mode_t mode)
int robust_unlink(const char *fname)
{
#ifndef ETXTBSY
return do_unlink_at(fname);
return do_unlink(fname);
#else
static int counter = 1;
int rc, pos, start;
char path[MAXPATHLEN];
rc = do_unlink_at(fname);
rc = do_unlink(fname);
if (rc == 0 || errno != ETXTBSY)
return rc;
@@ -532,7 +515,7 @@ int robust_unlink(const char *fname)
}
/* maybe we should return rename()'s exit status? Nah. */
if (do_rename_at(fname, path) != 0) {
if (do_rename(fname, path) != 0) {
errno = ETXTBSY;
return -1;
}
@@ -555,7 +538,7 @@ int robust_rename(const char *from, const char *to, const char *partialptr,
return 0;
while (tries--) {
if (do_rename_at(from, to) == 0)
if (do_rename(from, to) == 0)
return 0;
switch (errno) {
@@ -576,7 +559,7 @@ int robust_rename(const char *from, const char *to, const char *partialptr,
}
if (copy_file(from, to, -1, mode) != 0)
return -2;
do_unlink_at(from);
do_unlink(from);
return 1;
default:
return -1;
@@ -1133,7 +1116,6 @@ char *sanitize_path(char *dest, const char *p, const char *rootdir, int depth, i
* Also cleans the path using the clean_fname() function. */
int change_dir(const char *dir, int set_path_only)
{
extern int am_daemon, am_chrooted;
static int initialised, skipped_chdir;
unsigned int len;
@@ -1172,57 +1154,10 @@ int change_dir(const char *dir, int set_path_only)
curr_dir[curr_dir_len++] = '/';
memcpy(curr_dir + curr_dir_len, dir, len + 1);
if (!set_path_only) {
int chdir_failed;
/* In the daemon-without-chroot deployment we must not
* follow a symlink in any component of the chdir
* target -- otherwise CWD escapes the module and
* every subsequent path-relative syscall (open,
* chmod, lchown, ...) inherits the escape, which
* defeats secure_relative_open's RESOLVE_BENEATH
* anchor and re-opens the CVE-2026-29518 class of
* symlink TOCTOU attacks. Use the secure resolver
* to get a confined dirfd, then fchdir() to it.
*
* If skipped_chdir is set, a previous CD_SKIP_CHDIR
* call buffered an absolute prefix in curr_dir
* (e.g. change_pathname's CD_SKIP_CHDIR to orig_dir)
* without syncing the kernel's CWD. Resolve `dir`
* relative to that prefix as basedir so the secure
* branch still anchors at the operator-trusted
* directory rather than wherever the kernel CWD
* happens to be. */
if (am_daemon && !am_chrooted) {
const char *basedir = NULL;
char prefix[MAXPATHLEN];
int dfd;
if (skipped_chdir) {
if (save_dir_len >= sizeof prefix) {
errno = ENAMETOOLONG;
chdir_failed = 1;
goto chdir_cleanup;
}
memcpy(prefix, curr_dir, save_dir_len);
prefix[save_dir_len] = '\0';
basedir = prefix;
}
dfd = secure_relative_open(basedir, dir,
O_RDONLY | O_DIRECTORY, 0);
if (dfd < 0) {
chdir_failed = 1;
} else {
chdir_failed = fchdir(dfd) != 0;
close(dfd);
}
} else {
chdir_failed = chdir(curr_dir) != 0;
}
chdir_cleanup:
if (chdir_failed) {
curr_dir_len = save_dir_len;
curr_dir[curr_dir_len] = '\0';
return 0;
}
if (!set_path_only && chdir(curr_dir)) {
curr_dir_len = save_dir_len;
curr_dir[curr_dir_len] = '\0';
return 0;
}
skipped_chdir = set_path_only;
}
@@ -1350,20 +1285,20 @@ int handle_partial_dir(const char *fname, int create)
dir = partial_fname;
if (create) {
STRUCT_STAT st;
int statret = do_lstat_at(dir, &st);
int statret = do_lstat(dir, &st);
if (statret == 0 && !S_ISDIR(st.st_mode)) {
if (do_unlink_at(dir) < 0) {
if (do_unlink(dir) < 0) {
*fn = '/';
return 0;
}
statret = -1;
}
if (statret < 0 && do_mkdir_at(dir, 0700) < 0) {
if (statret < 0 && do_mkdir(dir, 0700) < 0) {
*fn = '/';
return 0;
}
} else
do_rmdir_at(dir);
do_rmdir(dir);
*fn = '/';
return 1;
@@ -1459,12 +1394,7 @@ char *timestring(time_t t)
static char buffers[4][20]; /* We support 4 simultaneous timestring results. */
char *TimeBuf = buffers[ndx = (ndx + 1) % 4];
struct tm tmp, *tm = localtime_r(&t, &tmp);
int len;
if (!tm) {
strlcpy(TimeBuf, "(time out of range)", sizeof buffers[0]);
return TimeBuf;
}
len = snprintf(TimeBuf, sizeof buffers[0], "%4d/%02d/%02d %02d:%02d:%02d",
int len = snprintf(TimeBuf, sizeof buffers[0], "%4d/%02d/%02d %02d:%02d:%02d",
(int)tm->tm_year + 1900, (int)tm->tm_mon + 1, (int)tm->tm_mday,
(int)tm->tm_hour, (int)tm->tm_min, (int)tm->tm_sec);
assert(len > 0); /* Silence gcc warning */

View File

@@ -1,2 +1,2 @@
#define RSYNC_VERSION "3.4.1"
#define MAINTAINER_TZ_OFFSET -7.0
#define RSYNC_VERSION "3.4.2"
#define MAINTAINER_TZ_OFFSET 10.0

View File

@@ -42,7 +42,7 @@ int empties_mod = 0;
int empty_at_start = 0;
int empty_at_end = 0;
static struct poptOption long_options[] = {
static const struct poptOption long_options[] = {
/* longName, shortName, argInfo, argPtr, value, descrip, argDesc */
{"iterations", 'i', POPT_ARG_NONE, &output_iterations, 0, 0, 0},
{"empties", 'e', POPT_ARG_STRING, 0, 'e', 0, 0},

View File

@@ -697,13 +697,6 @@ int recv_xattr_request(struct file_struct *file, int f_in)
rxa = lst->items;
num = 0;
while ((rel_pos = read_varint(f_in)) != 0) {
/* Detect signed overflow before the accumulating add. A hostile
* peer could otherwise wrap 'num' to land on an arbitrary value. */
if ((rel_pos > 0 && num > INT_MAX - rel_pos)
|| (rel_pos < 0 && num < INT_MIN - rel_pos)) {
rprintf(FERROR, "xattr rel_pos accumulation overflow [%s]\n", who_am_i());
exit_cleanup(RERR_PROTOCOL);
}
num += rel_pos;
if (am_sender) {
/* The sender-related num values are only in order on the sender.
@@ -749,7 +742,7 @@ int recv_xattr_request(struct file_struct *file, int f_in)
}
old_datum = rxa->datum;
rxa->datum_len = read_varint_size(f_in, MAX_WIRE_XATTR_DATALEN, "xattr datum_len");
rxa->datum_len = read_varint(f_in);
if (SIZE_MAX - rxa->name_len < rxa->datum_len)
overflow_exit("recv_xattr_request");
@@ -790,8 +783,7 @@ void receive_xattr(int f, struct file_struct *file)
return;
}
count = read_varint_bounded(f, 0, MAX_WIRE_XATTR_COUNT, "xattr count");
if (count != 0) {
if ((count = read_varint(f)) != 0) {
(void)EXPAND_ITEM_LIST(&temp_xattr, rsync_xa, count);
temp_xattr.count = 0;
}
@@ -799,8 +791,8 @@ void receive_xattr(int f, struct file_struct *file)
for (num = 1; num <= count; num++) {
char *ptr, *name;
rsync_xa *rxa;
size_t name_len = read_varint_size(f, MAX_WIRE_XATTR_NAMELEN, "xattr name_len");
size_t datum_len = read_varint_size(f, MAX_WIRE_XATTR_DATALEN, "xattr datum_len");
size_t name_len = read_varint(f);
size_t datum_len = read_varint(f);
size_t dget_len = datum_len > MAX_FULL_DATUM ? 1 + (size_t)xattr_sum_len : datum_len;
size_t extra_len = MIGHT_NEED_RPRE ? RPRE_LEN : 0;
if (SIZE_MAX - dget_len < extra_len || SIZE_MAX - dget_len - extra_len < name_len)
@@ -1094,7 +1086,7 @@ int set_xattr(const char *fname, const struct file_struct *file, const char *fna
&& !S_ISLNK(sxp->st.st_mode)
#endif
&& access(fname, W_OK) < 0
&& do_chmod_at(fname, (sxp->st.st_mode & CHMOD_BITS) | S_IWUSR) == 0)
&& do_chmod(fname, (sxp->st.st_mode & CHMOD_BITS) | S_IWUSR) == 0)
added_write_perm = 1;
ndx = F_XATTR(file);
@@ -1102,7 +1094,7 @@ int set_xattr(const char *fname, const struct file_struct *file, const char *fna
lst = &glst->xa_items;
int return_value = rsync_xal_set(fname, lst, fnamecmp, sxp);
if (added_write_perm) /* remove the temporary write permission */
do_chmod_at(fname, sxp->st.st_mode);
do_chmod(fname, sxp->st.st_mode);
return return_value;
}
@@ -1219,7 +1211,7 @@ int set_stat_xattr(const char *fname, struct file_struct *file, mode_t new_mode)
mode = (fst.st_mode & _S_IFMT) | (fmode & ACCESSPERMS)
| (S_ISDIR(fst.st_mode) ? 0700 : 0600);
if (fst.st_mode != mode)
do_chmod_at(fname, mode);
do_chmod(fname, mode);
if (!IS_DEVICE(fst.st_mode))
fst.st_rdev = 0; /* just in case */
@@ -1257,12 +1249,7 @@ int set_stat_xattr(const char *fname, struct file_struct *file, mode_t new_mode)
int x_stat(const char *fname, STRUCT_STAT *fst, STRUCT_STAT *xst)
{
/* Use the *_at variants so that on a daemon-no-chroot deployment
* the metadata read goes through a secure parent dirfd instead
* of bare path resolution. The *_at wrappers fall through to
* plain do_stat outside the daemon-no-chroot context, so this
* change is transparent for non-daemon use. */
int ret = do_stat_at(fname, fst);
int ret = do_stat(fname, fst);
if ((ret < 0 || get_stat_xattr(fname, -1, fst, xst) < 0) && xst)
xst->st_mode = 0;
return ret;
@@ -1270,7 +1257,7 @@ int x_stat(const char *fname, STRUCT_STAT *fst, STRUCT_STAT *xst)
int x_lstat(const char *fname, STRUCT_STAT *fst, STRUCT_STAT *xst)
{
int ret = do_lstat_at(fname, fst);
int ret = do_lstat(fname, fst);
if ((ret < 0 || get_stat_xattr(fname, -1, fst, xst) < 0) && xst)
xst->st_mode = 0;
return ret;