Compare commits

..

50 Commits

Author SHA1 Message Date
Andrew Tridgell
de3cc03b03 Preparing for release of 3.4.2 [buildall] 2026-04-28 14:29:48 +10:00
Andrew Tridgell
006ee327d6 packaging: new release script 2026-04-28 14:27:41 +10:00
Andrew Tridgell
9b6363fa10 update NEWS.md ready for 3.4.2 2026-04-28 12:55:38 +10:00
Andrew Tridgell
9e2f0fe9ae packaging: remove support for rsync-patches 2026-04-28 12:55:38 +10:00
Michal Ruprich
4f6e4ea64a Do not clean DISPLAY unconditionally 2026-04-22 13:05:35 +10:00
Andrew Tridgell
567c40935f call tzset() before chroot to cache timezone data
localtime/localtime_r need /etc/localtime for timezone info.
After chroot this file is inaccessible, causing log timestamps
to fall back to UTC. Calling tzset() before chroot ensures the
timezone data is cached by glibc for subsequent calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 13:02:10 +10:00
Michal Ruprich
8e11f0c169 Using a correct time in log file 2026-04-22 13:02:10 +10:00
Andrew Tridgell
e9dbc8d66d rsyncd.conf: document the temp dir parameter
The temp dir parameter was functional but undocumented in the man page.

Fixes: https://github.com/RsyncProject/rsync/issues/820

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:58 +10:00
Andrew Tridgell
bd2dbd2f32 runtests.py: preserve test-execution order in skipped list
The sorted() call reordered skipped test names alphabetically,
causing CI expected-skipped mismatches (e.g. acls,acls-default
instead of acls-default,acls). Sort by original test order instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:39 +10:00
Andrew Tridgell
350e295d1c runtests.py: add -j/--parallel option for parallel test execution
Add parallel test execution using concurrent.futures. With -j8 the
test suite completes in ~4s vs ~29s sequential (~7x speedup).

Also fix two issues that caused failures under parallel execution:
- rsync_ls_lR now prunes testtmp/ so parallel tests don't see each
  other's temp files when scanning the source tree
- clean-fname-underflow.test now uses $scratchdir instead of /tmp

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:39 +10:00
Andrew Tridgell
066156fcd9 replace runtests.sh with runtests.py
Rewrite the test runner in Python with proper command-line options
including --valgrind which directs valgrind output to per-process
log files so it doesn't interfere with test output comparisons.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 12:34:39 +10:00
Holger Hoffstätte
a5bbe859db Fix glibc-2.43 constness warnings
Glibc 2.43 added C23 const-preserving overloads to various string functions,
which change the return type depending on the constness of the argument(s).
Currently this leads to warnings from calls to strtok() or strchr().
Fix this by properly declaring the respective variable types.

Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
2026-04-22 12:10:08 +10:00
Andrew Tridgell
d046525de3 zero all new memory from allocations
Change my_alloc() to use calloc instead of malloc so all fresh
allocations return zeroed memory. Also zero the expanded portion
in expand_item_list() after realloc, since it knows both old and
new sizes. This gives more predictable behaviour in case of bugs
where uninitialised or stale memory is accidentally accessed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 11:44:10 +10:00
Andrew Tridgell
bb0a8118c2 xattrs: fixed count in qsort
this fixes the count passed to the sort of the xattr list. This issue
was reported here:

https://www.openwall.com/lists/oss-security/2026/04/16/2

the bug is not exploitable due to the fork-per-connection design of
rsync, the attack is the equivalent of the user closing the socket
themselves.
2026-04-22 10:38:14 +10:00
Andrew Tridgell
d1df0aaf70 fix signed integer overflow in proxy protocol v2 header parsing
The len field in the proxy v2 header was declared as signed char,
allowing a negative size to bypass the validation check and cause
a stack buffer overflow when passed to read_buf() as size_t.

This bug was reported by John Walker from ZeroPath, many thanks for
the clear report!

With the current code this bug does not represent a security issue as
it only results in the exit of the forked process that is specific to
the attached client, so it is equivalent to the client closing the
socket, so no CVE for this, but it is good to fix it to prevent a
future issue.
2026-04-16 13:59:52 +10:00
Andrew Tridgell
15d8e49a64 zlib: convert K&R function definitions to ANSI style
The bundled zlib 1.2.8 used K&R-style function definitions which are
rejected by clang 16+ as hard errors. Convert all 90 functions across
9 files to ANSI-style prototypes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 13:49:30 +10:00
Andrew Tridgell
b905ab23af CI: add simd-checksum to expected-skipped on macOS and Cygwin
The new simd-checksum test is skipped on platforms where SIMD
instructions are unavailable (macOS ARM, Cygwin). Add it to the
RSYNC_EXPECT_SKIPPED lists so CI doesn't fail on the mismatch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:52:01 +11:00
Andrew Tridgell
aa142f08ef fix uninitialized mul_one in AVX2 checksum and add SIMD checksum test
The AVX2 get_checksum1_avx2_64() read mul_one before initializing it,
which is undefined behavior. Replace the cmpeq/abs trick with
_mm256_set1_epi8(1) to match the SSSE3 and SSE2 versions.

Add a TEST_SIMD_CHECKSUM1 test mode that verifies all SIMD paths
(SSE2, SSSE3, AVX2, and the full dispatch chain) produce identical
results to the C reference, across multiple buffer sizes with both
aligned and unaligned buffers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:52:01 +11:00
Andrew Tridgell
236417cf35 acl: fixed ACL ID mapping for non-root
closes issue #618
2026-01-19 11:32:13 +11:00
Andrew Tridgell
2a97d81e99 CI: fixed MacOS test
fixed multiple MacOS issues
2025-12-31 11:37:27 +11:00
Andrew Tridgell
359e539a72 reject negative token values in compressed stream receivers
Validate that token numbers read from compressed streams are
non-negative. A negative token value would cause the return value
of recv_*_token() to become positive, which callers interpret as
literal data length, but no data pointer is set on this code path.

While this only causes the receiver to crash (which is process-isolated
and only affects the attacker's own connection), it's still undefined
behavior.

Reported-by: Will Sergeant <wlsergeant@gmail.com>
2025-12-31 09:31:52 +11:00
Andrew Tridgell
9e0898460d util: fixed issue in clean_fname()
fixes buffer underflow (not exploitable) in clean_fname
2025-12-30 17:49:35 +11:00
Andrew Tridgell
185520a141 testsuite: added clean-fname-underflow test 2025-12-30 17:49:35 +11:00
Andrew Tridgell
c98f9d1f68 fix uninitialized buf1 in get_checksum2() MD4 path
The static buf1 pointer was only allocated when len > len1, but on
first call with len == 0, this condition is false (0 > 0), leaving
buf1 NULL when passed to memcpy().

Fixes #673
2025-12-30 16:51:43 +11:00
Nebojša Cvetković
1f9ce2fcbe rsync: Add missing dirs long option 2025-12-30 16:48:34 +11:00
Andrew Tridgell
797e17fc4a fixed an invalid access to files array
this was found by Calum Hutton from Rapid7. It is a real bug, but
analysis shows it can't be leverged into an exploit. Worth fixing
though.

Many thanks to Calum and Rapid7 for finding and reporting this
2025-08-23 17:49:19 +10:00
Ronnie Sahlberg
c2db921890 options.c: Fix segv if poptGetContext returns NULL
If poptGetContext returns NULL, perhaps due to OOM,
a NULL pointer is passed into poptReadDefaultConfig()
which in turns SEGVs when trying to dereference it.

This was found using https://github.com/sahlberg/malloc-fail-tester.git
$ ./test_malloc_failure.sh rsync -Pav crash crosh

Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
2025-08-23 17:49:03 +10:00
Silent
77be09aaed syscall: fix a Y2038 bug by replacing Int32x32To64 with multiplication
Int32x32To64 macro internally truncates the arguments to int32,
while time_t is 64-bit on most/all modern platforms.
Therefore, usage of this macro creates a Year 2038 bug.
2025-08-23 17:32:11 +10:00
Jeremy Norris
0d0f615240 Ignore directory has vanished errors. 2025-08-23 17:31:52 +10:00
Max Kellermann
b6457bbc83 make lots of global variables const
This way, they can live in `.rodata` and the compiler is allowed to do
certain optimizations.
2025-08-23 17:31:40 +10:00
Peter Eriksson
1807ce485a Fix handling of objects with many xattrs on FreeBSD 2025-08-23 17:31:28 +10:00
Rahul Mehta
9c175ac9ef chore: gitignore MacOS debug symbols 2025-08-23 17:31:12 +10:00
Emily
a84b79ea58 Allow ls(1) to fail in test setup
This can happen when the tests are unable to `stat(2)` some files in
`/etc`, `/bin`, or `/`, due to Unix permissions or other sandboxing. We
still guard against serious errors, which use exit code 2.
2025-08-23 17:30:59 +10:00
fbuescher
d4c4f6754e fixed remove multiple leading slashes 2025-08-23 17:14:43 +10:00
Michal Ruprich
a4b926dcdc bool is a keyword in C23 2025-08-23 17:14:26 +10:00
Eli Schwartz
0973d0e380 configure.ac: check for xattr support both in libc and in -lattr
In 2015, the attr/xattr.h header was fully removed from upstream attr.

In 2020, rsync started preferring the standard header, if it exists:
https://github.com/RsyncProject/rsync/pull/22

But the fix was incomplete. We still looked for the getxattr function in
-lattr, and used it if -lattr exists. This was the case even if the
system libc was sufficient to provide the needed functions. Result:
overlinking to -lattr, if it happened to be installed for any other
reason.

```
checking whether to support extended attributes... Using Linux xattrs
checking for getxattr in -lattr... yes
```

Instead, use a different autoconf macro that first checks if the
function is available for use without any libraries (e.g. it is in
libc).

Result:

```
checking whether to support extended attributes... Using Linux xattrs
checking for library containing getxattr... none required
```

Signed-off-by: Eli Schwartz <eschwartz@gentoo.org>
2025-08-23 17:14:06 +10:00
Ethan Halsall
e405cfc073 feat: add compress threads to man page 2025-08-23 17:13:49 +10:00
Ethan Halsall
b78a841bb0 feat: validate compress threads option 2025-08-23 17:13:49 +10:00
Ethan Halsall
f7a2b8a3fa feat: add threads to zstd compression 2025-08-23 17:13:49 +10:00
Arnaud Rebillout
d941807915 Fix flaky hardlinks test
The test was added in dc34990, it turns out that it's flaky. It failed
once on the Debian build infra, cf. [1].

The problem is that the command `rsync -aH '$fromdir/sym' '$todir'`
updates the mod time of `$todir`, so there might be a diff between the
output of `rsync_ls_lR $fromdir` and `rsync_ls_lR $todir`, if ever rsync
runs 1 second (or more) after the directories were created.

To clarify: it's easy to make the test fails 100% of the times with this
change:

```
 makepath "$fromdir/sym" "$todir"
+sleep 5
 checkit "$RSYNC -aH '$fromdir/sym' '$todir'" "$fromdir" "$todir"
```

With the fix proposed here, we don't use `checkit` anymore, instead we
just run the rsync command, then a simple `diff` to compare the two
directories. This is exactly what the other `-H` test just above does.

In case there's some doubts, `diff` fails if `sym` is missing:

```
$ mkdir -p foo/sym bar
$ diff foo bar || echo KO!
Only in foo: sym
KO!
```

I tested that, after this commit, the test still catches the `-H`
regression in rsync 3.4.0.

Fixes: https://github.com/RsyncProject/rsync/issues/735

[1]: https://buildd.debian.org/status/fetch.php?pkg=rsync&arch=ppc64el&ver=3.4.1%2Bds1-1&stamp=1741147156&raw=0
2025-08-23 17:13:28 +10:00
Krzysztof Płocharz
992e10efaf Fix --open-noatime option not working on files
atime of source files could sometimes be overwritten
even though --open-noatime option was used.

To fix that, optional O_NOATIME flag was added
to do_open_nofollow which is also used to open regular
files since fix:
  "fixed symlink race condition in sender"
Previously optional O_NOATIME flag was only in do_open.
2025-08-23 17:13:09 +10:00
Chris Lamb
1c5ebdc4e5 Make the build reproducible
From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1093201:
Whilst working on the Reproducible Builds effort [0], we noticed that
rsync could not be built reproducibly.

This is because the date in the manual page can vary depending on
whether there is a .git directory and the modification time of version.h
and Mafile, which might get modified when patching via quilt.

A patch is attached that makes this use SOURCE_DATE_EPOCH, which
will always be reliable.
2025-08-23 16:40:34 +10:00
Wayne Davison
9994933c8c Test on ubuntu-latest. 2025-02-11 13:37:12 -08:00
Alan Coopersmith
23d9ead5af popt: remove obsolete findme.c & findme.h
popt 1.14 merged these into popt.c but the import into rsync
missed removing them.

Fixes: https://github.com/RsyncProject/rsync/issues/710

Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>
2025-01-17 08:31:36 +11:00
Wayne Davison
fcfdd36054 Update MAINTAINER_TZ_OFFSET on release.
This also fixes a string with \s that wasn't a r'...' string.
2025-01-15 23:27:27 -08:00
Wayne Davison
89b847393f Fix python deprecation warning. 2025-01-15 22:36:29 -08:00
Wayne Davison
788ecbe5ea Don't edit copyright year values anymore. 2025-01-15 22:30:32 -08:00
Wayne Davison
353506bc51 Improve interior dashes in long options.
Improve the backslash-adding code in md-convert to affect dashes in the
interior of long options.  Perhaps fixes #686.
2025-01-15 22:23:30 -08:00
Wayne Davison
7cff121ec8 Start 3.4.2dev going. 2025-01-15 22:01:42 -08:00
Andrew Tridgell
14f33837dc fixed build error on ia64 NonStop
it treats missing prototype as an error, not warning
2025-01-16 15:27:21 +11:00
64 changed files with 1808 additions and 948 deletions

View File

@@ -39,7 +39,7 @@ jobs:
- name: info
run: bash -c '/usr/local/bin/rsync --version'
- name: check
run: bash -c 'RSYNC_EXPECT_SKIPPED=acls-default,acls,chown,devices,dir-sgid,protected-regular make check'
run: bash -c 'RSYNC_EXPECT_SKIPPED=acls-default,acls,chown,devices,dir-sgid,open-noatime,protected-regular,simd-checksum make check'
- name: ssl file list
run: bash -c 'PATH="/usr/local/bin:$PATH" rsync-ssl --no-motd download.samba.org::rsyncftp/ || true'
- name: save artifact

View File

@@ -25,10 +25,15 @@ jobs:
- name: prep
run: |
brew install automake openssl xxhash zstd lz4
sudo pip3 install commonmark
echo "/usr/local/bin" >>$GITHUB_PATH
pip3 install --user --break-system-packages commonmark
echo "$(brew --prefix)/bin" >>$GITHUB_PATH
- name: configure
run: CPPFLAGS=-I/usr/local/opt/openssl/include/ LDFLAGS=-L/usr/local/opt/openssl/lib/ ./configure --with-rrsync
run: |
BREW_PREFIX=$(brew --prefix)
OPENSSL_PREFIX=$(brew --prefix openssl)
CPPFLAGS="-I${BREW_PREFIX}/include -I${OPENSSL_PREFIX}/include" \
LDFLAGS="-L${BREW_PREFIX}/lib -L${OPENSSL_PREFIX}/lib" \
./configure --with-rrsync
- name: make
run: make
- name: install
@@ -36,11 +41,11 @@ jobs:
- name: info
run: rsync --version
- name: check
run: sudo RSYNC_EXPECT_SKIPPED=acls-default,chmod-temp-dir,chown-fake,devices-fake,dir-sgid,protected-regular,xattrs-hlink,xattrs make check
run: sudo RSYNC_EXPECT_SKIPPED=acls-default,chmod-temp-dir,chown-fake,devices-fake,dir-sgid,open-noatime,protected-regular,simd-checksum,xattrs-hlink,xattrs make check
- name: ssl file list
run: rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
- name: save artifact
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: macos-bin
path: |

View File

@@ -16,7 +16,7 @@ on:
jobs:
test:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
name: Test rsync on Ubuntu
steps:
- uses: actions/checkout@v4

1
.gitignore vendored
View File

@@ -58,3 +58,4 @@ aclocal.m4
/auto-build-save
.deps
/*.exe
*.dSYM/

View File

@@ -49,7 +49,7 @@ OBJS2=options.o io.o compat.o hlink.o token.o uidlist.o socket.o hashtable.o \
usage.o fileio.o batch.o clientname.o chmod.o acls.o xattrs.o
OBJS3=progress.o pipe.o @MD5_ASM@ @ROLL_SIMD@ @ROLL_ASM@
DAEMON_OBJ = params.o loadparm.o clientserver.o access.o connection.o authenticate.o
popt_OBJS=popt/findme.o popt/popt.o popt/poptconfig.o \
popt_OBJS= popt/popt.o popt/poptconfig.o \
popt/popthelp.o popt/poptparse.o popt/poptint.o
OBJS=$(OBJS1) $(OBJS2) $(OBJS3) $(DAEMON_OBJ) $(LIBOBJ) @BUILD_ZLIB@ @BUILD_POPT@
@@ -57,7 +57,8 @@ TLS_OBJ = tls.o syscall.o util2.o t_stub.o lib/compat.o lib/snprintf.o lib/perms
# Programs we must have to run the test cases
CHECK_PROGS = rsync$(EXEEXT) tls$(EXEEXT) getgroups$(EXEEXT) getfsdev$(EXEEXT) \
testrun$(EXEEXT) trimslash$(EXEEXT) t_unsafe$(EXEEXT) wildtest$(EXEEXT)
testrun$(EXEEXT) trimslash$(EXEEXT) t_unsafe$(EXEEXT) wildtest$(EXEEXT) \
simdtest$(EXEEXT)
CHECK_SYMLINKS = testsuite/chown-fake.test testsuite/devices-fake.test testsuite/xattrs-hlink.test
@@ -312,20 +313,28 @@ test: check
.PHONY: check
check: all $(CHECK_PROGS) $(CHECK_SYMLINKS)
rsync_bin=`pwd`/rsync$(EXEEXT) $(srcdir)/runtests.sh
$(srcdir)/runtests.py --rsync-bin=`pwd`/rsync$(EXEEXT)
.PHONY: check29
check29: all $(CHECK_PROGS) $(CHECK_SYMLINKS)
rsync_bin=`pwd`/rsync$(EXEEXT) $(srcdir)/runtests.sh --protocol=29
$(srcdir)/runtests.py --rsync-bin=`pwd`/rsync$(EXEEXT) --protocol=29
.PHONY: check30
check30: all $(CHECK_PROGS) $(CHECK_SYMLINKS)
rsync_bin=`pwd`/rsync$(EXEEXT) $(srcdir)/runtests.sh --protocol=30
$(srcdir)/runtests.py --rsync-bin=`pwd`/rsync$(EXEEXT) --protocol=30
wildtest.o: wildtest.c t_stub.o lib/wildmatch.c rsync.h config.h
wildtest$(EXEEXT): wildtest.o lib/compat.o lib/snprintf.o @BUILD_POPT@
$(CC) $(CFLAGS) $(LDFLAGS) -o $@ wildtest.o lib/compat.o lib/snprintf.o @BUILD_POPT@ $(LIBS)
simdtest$(EXEEXT): simd-checksum-x86_64.cpp $(HEADERS)
@if test x"@ROLL_SIMD@" != x; then \
$(CXX) -I. $(CXXFLAGS) $(CPPFLAGS) $(LDFLAGS) -DTEST_SIMD_CHECKSUM1 \
-o $@ $(srcdir)/simd-checksum-x86_64.cpp @ROLL_ASM@ $(LIBS); \
else \
touch $@; \
fi
testsuite/chown-fake.test:
ln -s chown.test $(srcdir)/testsuite/chown-fake.test
@@ -341,7 +350,7 @@ testsuite/xattrs-hlink.test:
.PHONY: installcheck
installcheck: $(CHECK_PROGS) $(CHECK_SYMLINKS)
POSIXLY_CORRECT=1 TOOLDIR=`pwd` rsync_bin="$(bindir)/rsync$(EXEEXT)" srcdir="$(srcdir)" $(srcdir)/runtests.sh
$(srcdir)/runtests.py --rsync-bin="$(bindir)/rsync$(EXEEXT)" --srcdir="$(srcdir)" --tooldir=`pwd`
# TODO: Add 'dist' target; need to know which files will be included

146
NEWS.md
View File

@@ -1,3 +1,144 @@
# NEWS for rsync 3.4.2 (28 Apr 2026)
## Changes in this version:
### SECURITY RELATED:
Several security-relevant defects were reported and fixed since 3.4.1.
None were assigned a CVE — rsync's fork-per-connection design scopes
the impact of each of these to the attacker's own connection, which is
equivalent to the client closing the socket itself — but they are
fixed here as a matter of hygiene and to reduce the chances of a
future exploitable combination. Many thanks to the external
researchers who reported these issues.
- Fixed a signed integer overflow in the PROXY protocol v2 header
parser: a negative `len` field could bypass the size check and cause
a stack buffer overflow in `read_buf()`. Reported by John Walker of
ZeroPath.
- Fixed an invalid access to the files array. Reported by Calum
Hutton of Rapid7.
- Reject negative token values in the compressed-stream token
decoder; a negative value could cause callers to misinterpret a
missing data pointer as literal data. Reported by Will Sergeant.
- Fixed the element count passed to the xattr `qsort()` (see
https://www.openwall.com/lists/oss-security/2026/04/16/2).
- Fixed a buffer underflow in `clean_fname()`, and added a regression
test.
- Fixed an uninitialized `mul_one` in the AVX2 get_checksum1 path
(undefined behaviour), and added a SIMD-checksum self-test that
cross-checks SSE2, SSSE3 and AVX2 against the C reference on both
aligned and unaligned buffers.
- Fixed an uninitialized `buf1` on the first call to
`get_checksum2()` in the MD4 path (fixes #673).
- Zero all new memory from internal allocations: `my_alloc()` now uses
`calloc`, and `expand_item_list()` zeros the expanded portion after
`realloc`. This gives more predictable behaviour if stale or
uninitialised memory is ever accidentally read.
### BUG FIXES:
- Call `tzset()` before chroot so that log timestamps continue to
reflect the configured local timezone after the daemon chroots
(glibc needs `/etc/localtime`, which is unreachable post-chroot).
- Use the correct time when writing to the log file.
- Do not clear `DISPLAY` unconditionally.
- Fixed a Y2038 bug in `syscall.c` by replacing the `Int32x32To64`
macro (which truncates its arguments to 32 bits) with a plain
64-bit multiplication.
- Fixed ACL ID mapping for non-root users (closes #618).
- Fixed handling of objects with many xattrs on FreeBSD.
- Fixed `--open-noatime` not taking effect when opening regular
files: `O_NOATIME` is now also passed to `do_open_nofollow()`, which
has been used for regular files since the CVE fix "fixed symlink
race condition in sender".
- Ignore "directory has vanished" errors.
- Fixed the removal of multiple leading slashes.
- Added the missing `--dirs` long option.
- Fixed a segfault if `poptGetContext()` returns NULL (e.g. under
OOM) by not passing NULL to `poptReadDefaultConfig()`. Reported by
Ronnie Sahlberg; found with `malloc-fail-tester`.
- Fixed a build error on ia64 NonStop (which treats missing
prototypes as an error, not a warning).
- Fixed a flaky hardlinks test (fixes #735).
### ENHANCEMENTS:
- Added multi-threaded `zstd` compression, gated by a new
`--compress-threads=N` option, with validation and man-page
coverage.
- Documented the `temp dir` parameter in the rsyncd.conf man page
(fixes #820).
- Improved rendering of interior dashes in long-option names in
`md-convert` (perhaps fixes #686).
### PORTABILITY / BUILD:
- Fixed glibc 2.43 const-preserving overloads of `strtok()`,
`strchr()` etc. by declaring the affected locals with the right
constness. Contributed by Holger Hoffstätte.
- Converted the bundled zlib 1.2.8 from K&R-style function
definitions to ANSI prototypes, so it builds with clang 16+.
- Avoid using `bool` as an identifier; it is a keyword in C23.
- `configure.ac`: check for xattr functions in libc first and only
fall back to `-lattr`, avoiding spurious overlinking when `-lattr`
happens to be installed. Contributed by Eli Schwartz.
- Made the build reproducible by honouring `SOURCE_DATE_EPOCH` for
the manpage date.
- Removed obsolete `popt/findme.c` and `popt/findme.h` that upstream
popt 1.14 folded into `popt.c` (fixes #710). Contributed by Alan
Coopersmith.
### INTERNAL:
- Made many module-global variables `const` so they can live in
`.rodata` and enable additional compiler optimization.
### DEVELOPER RELATED:
- Replaced `runtests.sh` with `runtests.py`, a Python test runner
that supports `--valgrind` (with per-process log files so valgrind
output no longer interferes with output comparisons) and
`-j/--parallel` execution for roughly a 7× speed-up on typical
hardware.
- Added a SIMD checksum self-test and a `clean-fname-underflow`
regression test.
- Various CI fixes for macOS and Cygwin (including adding
`simd-checksum` to the expected-skipped lists on platforms without
SIMD), and tests now run on `ubuntu-latest`.
- removed support for the unmaintained rsync-patches archive
------------------------------------------------------------------------------
# NEWS for rsync 3.4.1 (16 Jan 2025)
Release 3.4.1 is a fix for regressions introduced in 3.4.0
@@ -19,6 +160,7 @@ Release 3.4.1 is a fix for regressions introduced in 3.4.0
- fix to permissions handling in the developer release script
------------------------------------------------------------------------------
# NEWS for rsync 3.4.0 (15 Jan 2025)
Release 3.4.0 is a security release that fixes a number of important vulnerabilities.
@@ -73,6 +215,7 @@ to develop and test fixes.
- added FreeBSD and Solaris CI builds
------------------------------------------------------------------------------
# NEWS for rsync 3.3.0 (6 Apr 2024)
## Changes in this version:
@@ -4837,8 +4980,9 @@ to develop and test fixes.
| RELEASE DATE | VER. | DATE OF COMMIT\* | PROTOCOL |
|--------------|--------|------------------|-------------|
| 28 Apr 2026 | 3.4.2 | | 32 |
| 16 Jan 2025 | 3.4.1 | | 32 |
| 15 Jan 2025 | 3.4.0 | | 32 |
| 15 Jan 2025 | 3.4.0 | 15 Jan 2025 | 32 |
| 06 Apr 2024 | 3.3.0 | | 31 |
| 20 Oct 2022 | 3.2.7 | | 31 |
| 09 Sep 2022 | 3.2.6 | | 31 |

View File

@@ -99,7 +99,7 @@ static void make_mask(char *mask, int plen, int addrlen)
return;
}
static int match_address(const char *addr, const char *tok)
static int match_address(const char *addr, char *tok)
{
char *p;
struct addrinfo hints, *resa, *rest;

2
acls.c
View File

@@ -713,7 +713,7 @@ static uchar recv_ida_entries(int f, ida_entries *ent)
else
id = recv_group_name(f, id, NULL);
} else if (access & NAME_IS_USER) {
if (inc_recurse && am_root && !numeric_ids)
if (inc_recurse && !numeric_ids)
id = match_uid(id);
} else {
if (inc_recurse && (!am_root || !numeric_ids))

View File

@@ -75,7 +75,7 @@ static int *flag_ptr[] = {
NULL
};
static char *flag_name[] = {
static const char *const flag_name[] = {
"--recurse (-r)",
"--owner (-o)",
"--group (-g)",

View File

@@ -176,7 +176,7 @@ void parse_checksum_choice(int final_call)
if (valid_checksums.negotiated_nni)
xfer_sum_nni = file_sum_nni = valid_checksums.negotiated_nni;
else {
char *cp = checksum_choice ? strchr(checksum_choice, ',') : NULL;
const char *cp = checksum_choice ? strchr(checksum_choice, ',') : NULL;
if (cp) {
xfer_sum_nni = parse_csum_name(checksum_choice, cp - checksum_choice);
file_sum_nni = parse_csum_name(cp+1, -1);
@@ -366,9 +366,8 @@ void get_checksum2(char *buf, int32 len, char *sum)
mdfour_begin(&m);
if (len > len1) {
if (buf1)
free(buf1);
if (len > len1 || !buf1) {
free(buf1);
buf1 = new_array(char, len+4);
len1 = len;
}

View File

@@ -167,7 +167,7 @@ int read_proxy_protocol_header(int fd)
char sig[PROXY_V2_SIG_SIZE];
char ver_cmd;
char fam;
char len[2];
unsigned char len[2];
union {
struct {
char src_addr[4];

View File

@@ -976,6 +976,8 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
}
if (use_chroot) {
/* Cache timezone data before chroot makes /etc/localtime inaccessible */
tzset();
if (chroot(module_chdir)) {
rsyserr(FLOG, errno, "chroot(\"%s\") failed", module_chdir);
io_printf(f_out, "@ERROR: chroot failed\n");
@@ -1301,6 +1303,7 @@ int start_daemon(int f_in, int f_out)
p = lp_daemon_chroot();
if (*p) {
log_init(0); /* Make use we've initialized syslog before chrooting. */
tzset();
if (chroot(p) < 0) {
rsyserr(FLOG, errno, "daemon chroot(\"%s\") failed", p);
return -1;

View File

@@ -52,6 +52,7 @@ extern int need_messages_from_generator;
extern int delete_mode, delete_before, delete_during, delete_after;
extern int do_compression;
extern int do_compression_level;
extern int do_compression_threads;
extern int saw_stderr_opt;
extern int msgs2stderr;
extern char *shell_cmd;
@@ -131,7 +132,7 @@ static const char *client_info;
* of that protocol for it to be advertised as available. */
static void check_sub_protocol(void)
{
char *dot;
const char *dot;
int their_protocol, their_sub;
int our_sub = get_subprotocol_version();
@@ -414,7 +415,7 @@ static const char *getenv_nstr(int ntype)
env_str = ntype == NSTR_COMPRESS ? "zlib" : protocol_version >= 30 ? "md5" : "md4";
if (am_server && env_str) {
char *cp = strchr(env_str, '&');
const char *cp = strchr(env_str, '&');
if (cp)
env_str = cp + 1;
}

View File

@@ -1392,7 +1392,7 @@ else
AC_DEFINE(HAVE_LINUX_XATTRS, 1, [True if you have Linux xattrs (or equivalent)])
AC_DEFINE(SUPPORT_XATTRS, 1)
AC_DEFINE(NO_SYMLINK_USER_XATTRS, 1, [True if symlinks do not support user xattrs])
AC_CHECK_LIB(attr,getxattr)
AC_SEARCH_LIBS(getxattr,attr)
;;
darwin*)
AC_MSG_RESULT(Using OS X xattrs)

View File

@@ -6,7 +6,7 @@
BEGIN {
heading = "/* DO NOT EDIT THIS FILE! It is auto-generated from a list of values in " ARGV[1] "! */\n\n"
sect = psect = defines = accessors = prior_ptype = ""
parms = "\nstatic struct parm_struct parm_table[] = {"
parms = "\nstatic const struct parm_struct parm_table[] = {"
comment_fmt = "\n/********** %s **********/\n"
tdstruct = "typedef struct {"
}

View File

@@ -904,7 +904,7 @@ static int rule_matches(const char *fname, filter_rule *ex, int name_flags)
{
int slash_handling, str_cnt = 0, anchored_match = 0;
int ret_match = ex->rflags & FILTRULE_NEGATE ? 0 : 1;
char *p, *pattern = ex->pattern;
const char *p, *pattern = ex->pattern;
const char *strings[16]; /* more than enough */
const char *name = fname + (*fname == '/');

View File

@@ -3167,8 +3167,8 @@ static void output_flist(struct file_list *flist)
} else
*uidbuf = '\0';
if (gid_ndx) {
static char parens[] = "(\0)\0\0\0";
char *pp = parens + (file->flags & FLAG_SKIP_GROUP ? 0 : 3);
static const char parens[] = "(\0)\0\0\0";
const char *pp = parens + (file->flags & FLAG_SKIP_GROUP ? 0 : 3);
snprintf(gidbuf, sizeof gidbuf, " gid=%s%u%s",
pp, F_GROUP(file), pp + 2);
} else

6
io.c
View File

@@ -117,7 +117,7 @@ static int active_filecnt = 0;
static OFF_T active_bytecnt = 0;
static int first_message = 1;
static char int_byte_extra[64] = {
static const char int_byte_extra[64] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* (00 - 3F)/4 */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* (40 - 7F)/4 */
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* (80 - BF)/4 */
@@ -1158,8 +1158,8 @@ void set_io_timeout(int secs)
static void check_for_d_option_error(const char *msg)
{
static char rsync263_opts[] = "BCDHIKLPRSTWabceghlnopqrtuvxz";
char *colon;
static const char rsync263_opts[] = "BCDHIKLPRSTWabceghlnopqrtuvxz";
const char *colon;
int saw_d = 0;
if (*msg != 'r'

View File

@@ -1 +1 @@
#define LATEST_YEAR "2025"
#define LATEST_YEAR "2026"

View File

@@ -197,7 +197,7 @@ void md5_update(md_context *ctx, const uchar *input, uint32 length)
memcpy(ctx->buffer + left, input, length);
}
static uchar md5_padding[CSUM_CHUNK] = { 0x80 };
static const uchar md5_padding[CSUM_CHUNK] = { 0x80 };
void md5_result(md_context *ctx, uchar digest[MD5_DIGEST_LEN])
{

View File

@@ -126,9 +126,18 @@ ssize_t sys_llistxattr(const char *path, char *list, size_t size)
unsigned char keylen;
ssize_t off, len = extattr_list_link(path, EXTATTR_NAMESPACE_USER, list, size);
if (len <= 0 || (size_t)len > size)
if (len <= 0 || size == 0)
return len;
if ((size_t)len >= size) {
/* FreeBSD extattr_list_xx() returns 'size' as 'len' in case there are
more data available, truncating the output, we solve this by signalling
ERANGE in case len == size so that the code in xattrs.c will retry with
a bigger buffer */
errno = ERANGE;
return -1;
}
/* FreeBSD puts a single-byte length before each string, with no '\0'
* terminator. We need to change this into a series of null-terminted
* strings. Since the size is the same, we can simply transform the
@@ -136,7 +145,7 @@ ssize_t sys_llistxattr(const char *path, char *list, size_t size)
for (off = 0; off < len; off += keylen + 1) {
keylen = ((unsigned char*)list)[off];
if (off + keylen >= len) {
/* Should be impossible, but kernel bugs happen! */
/* Should be impossible, but bugs happen! */
errno = EINVAL;
return -1;
}

View File

@@ -65,7 +65,7 @@ typedef enum {
struct enum_list {
int value;
char *name;
const char *name;
};
struct parm_struct {
@@ -73,7 +73,7 @@ struct parm_struct {
parm_type type;
parm_class class;
void *ptr;
struct enum_list *enum_list;
const struct enum_list *enum_list;
unsigned flags;
};
@@ -95,7 +95,7 @@ static item_list section_list = EMPTY_ITEM_LIST;
static int iSectionIndex = -1;
static BOOL bInGlobalSection = True;
static struct enum_list enum_syslog_facility[] = {
static const struct enum_list enum_syslog_facility[] = {
#ifdef LOG_AUTH
{ LOG_AUTH, "auth" },
#endif
@@ -178,7 +178,7 @@ static char *expand_vars(const char *str)
for (t = buf, f = str; bufsize && *f; ) {
if (*f == '%' && isUpper(f+1)) {
char *percent = strchr(f+1, '%');
const char *percent = strchr(f+1, '%');
if (percent && percent - f < bufsize) {
char *val;
strlcpy(t, f+1, percent - f);

6
main.c
View File

@@ -386,7 +386,7 @@ static void handle_stats(int f)
static void output_itemized_counts(const char *prefix, int *counts)
{
static char *labels[] = { "reg", "dir", "link", "dev", "special" };
static char *const labels[] = { "reg", "dir", "link", "dev", "special" };
char buf[1024], *pre = " (";
int j, len = 0;
int total = counts[0];
@@ -1743,7 +1743,9 @@ int main(int argc,char *argv[])
our_gid = MY_GID();
am_root = our_uid == ROOT_UID;
unset_env_var("DISPLAY");
// DISPLAY should not be emptied unconditionally
if (!getenv("SSH_ASKPASS"))
unset_env_var("DISPLAY");
#if defined USE_OPENSSL && defined SET_OPENSSL_CONF
#define TO_STR2(x) #x

View File

@@ -120,6 +120,7 @@ TZ_RE = re.compile(r'^#define\s+MAINTAINER_TZ_OFFSET\s+(-?\d+(\.\d+)?)', re.M)
VAR_REF_RE = re.compile(r'\$\{(\w+)\}')
VERSION_RE = re.compile(r' (\d[.\d]+)[, ]')
BIN_CHARS_RE = re.compile(r'[\1-\7]+')
LONG_OPT_DASH_RE = re.compile(r'(--\w[-\w]+)')
SPACE_DOUBLE_DASH_RE = re.compile(r'\s--(\s)')
NON_SPACE_SINGLE_DASH_RE = re.compile(r'(^|\W)-')
WHITESPACE_RE = re.compile(r'\s')
@@ -247,6 +248,9 @@ def find_man_substitutions():
env_subs['date'] = time.strftime('%d %b %Y', time.gmtime(mtime + tz_offset)).lstrip('0')
if 'SOURCE_DATE_EPOCH' in os.environ:
env_subs['date'] = time.strftime('%d %b %Y', time.gmtime(int(os.environ.get('SOURCE_DATE_EPOCH', time.time()))))
def html_via_commonmark(txt):
return commonmark.HtmlRenderer().render(commonmark.Parser().parse(txt))
@@ -540,6 +544,7 @@ class TransformHtml(HTMLParser):
if st.in_pre:
html = htmlify(txt)
else:
txt = LONG_OPT_DASH_RE.sub(lambda x: x.group(1).replace('-', NBR_DASH[0]), txt)
txt = SPACE_DOUBLE_DASH_RE.sub(NBR_SPACE[0] + r'--\1', txt).replace('--', NBR_DASH[0]*2)
txt = NON_SPACE_SINGLE_DASH_RE.sub(r'\1' + NBR_DASH[0], txt)
html = htmlify(txt)

View File

@@ -86,6 +86,7 @@ int sparse_files = 0;
int preallocate_files = 0;
int do_compression = 0;
int do_compression_level = CLVL_NOT_SPECIFIED;
int do_compression_threads = 0; /*n = 0 use rsync thread, n >= 1 spawn n threads for compression */
int am_root = 0; /* 0 = normal, 1 = root, 2 = --super, -1 = --fake-super */
int am_server = 0;
int am_sender = 0;
@@ -225,7 +226,7 @@ char *iconv_opt =
struct chmod_mode_struct *chmod_modes = NULL;
static const char *debug_verbosity[] = {
static const char *const debug_verbosity[] = {
/*0*/ NULL,
/*1*/ NULL,
/*2*/ "BIND,CMD,CONNECT,DEL,DELTASUM,DUP,FILTER,FLIST,ICONV",
@@ -236,7 +237,7 @@ static const char *debug_verbosity[] = {
#define MAX_VERBOSITY ((int)(sizeof debug_verbosity / sizeof debug_verbosity[0]) - 1)
static const char *info_verbosity[1+MAX_VERBOSITY] = {
static const char *const info_verbosity[1+MAX_VERBOSITY] = {
/*0*/ "NONREG",
/*1*/ "COPY,DEL,FLIST,MISC,NAME,STATS,SYMSAFE",
/*2*/ "BACKUP,MISC2,MOUNT,NAME2,REMOVE,SKIP",
@@ -474,7 +475,7 @@ static void parse_output_words(struct output_struct *words, short *levels, const
static void output_item_help(struct output_struct *words)
{
short *levels = words == info_words ? info_levels : debug_levels;
const char **verbosity = words == info_words ? info_verbosity : debug_verbosity;
const char *const*verbosity = words == info_words ? info_verbosity : debug_verbosity;
char buf[128], *opt, *fmt = "%-10s %s\n";
int j;
@@ -756,6 +757,8 @@ static struct poptOption long_options[] = {
{"skip-compress", 0, POPT_ARG_STRING, &skip_compress, 0, 0, 0 },
{"compress-level", 0, POPT_ARG_INT, &do_compression_level, 0, 0, 0 },
{"zl", 0, POPT_ARG_INT, &do_compression_level, 0, 0, 0 },
{"compress-threads", 0, POPT_ARG_INT, &do_compression_threads, 0, 0, 0 },
{"zt", 0, POPT_ARG_INT, &do_compression_threads, 0, 0, 0 },
{0, 'P', POPT_ARG_NONE, 0, 'P', 0, 0 },
{"progress", 0, POPT_ARG_VAL, &do_progress, 1, 0, 0 },
{"no-progress", 0, POPT_ARG_VAL, &do_progress, 0, 0, 0 },
@@ -844,7 +847,7 @@ static struct poptOption long_options[] = {
{0,0,0,0, 0, 0, 0}
};
static struct poptOption long_daemon_options[] = {
static const struct poptOption long_daemon_options[] = {
/* longName, shortName, argInfo, argPtr, value, descrip, argDesc */
{"address", 0, POPT_ARG_STRING, &bind_address, 0, 0, 0 },
{"bwlimit", 0, POPT_ARG_INT, &daemon_bwlimit, 0, 0, 0 },
@@ -1156,7 +1159,7 @@ static time_t parse_time(const char *arg)
{
const char *cp;
time_t val, now = time(NULL);
struct tm t, *today = localtime(&now);
struct tm t, tmp, *today = localtime_r(&now, &tmp);
int in_date, old_mday, n;
memset(&t, 0, sizeof t);
@@ -1369,6 +1372,10 @@ int parse_arguments(int *argc_p, const char ***argv_p)
/* TODO: Call poptReadDefaultConfig; handle errors. */
pc = poptGetContext(RSYNC_NAME, argc, argv, long_options, 0);
if (pc == NULL) {
strlcpy(err_buf, "poptGetContext returned NULL\n", sizeof err_buf);
return 0;
}
if (!am_server) {
poptReadDefaultConfig(pc, 0);
popt_unalias(pc, "--daemon");
@@ -2006,6 +2013,8 @@ int parse_arguments(int *argc_p, const char ***argv_p)
create_refuse_error(refused_compress);
goto cleanup;
}
if (do_compression_threads < 0)
do_compression_threads = 0;
}
#ifdef HAVE_SETVBUF

View File

@@ -1,6 +1,6 @@
Summary: A fast, versatile, remote (and local) file-copying tool
Name: rsync
Version: 3.4.1
Version: 3.4.2
%define fullversion %{version}
Release: 1
%define srcdir src
@@ -79,5 +79,5 @@ rm -rf $RPM_BUILD_ROOT
%dir /etc/rsync-ssl/certs
%changelog
* Thu Jan 16 2025 Rsync Project <rsync.project@gmail.com>
Released 3.4.1.
* Tue Apr 28 2026 Rsync Project <rsync.project@gmail.com>
Released 3.4.2.

View File

@@ -38,12 +38,16 @@ def main():
if not os.path.isfile('packaging/release-rsync'):
die('You must run this script from the top of your rsync checkout.')
now = datetime.now()
now = datetime.now().astimezone() # Requires python 3.6 or later
cl_today = now.strftime('* %a %b %d %Y')
year = now.strftime('%Y')
ztoday = now.strftime('%d %b %Y')
today = ztoday.lstrip('0')
# The MAINTAINER_TZ_OFFSET is a float number of hours vs UTC. It can start with '-' but not '+'.
tz_now = now.strftime('%z')
tz_num = tz_now[0:1].replace('+', '') + str(float(tz_now[1:3]) + float(tz_now[3:]) / 60)
curdir = os.getcwd()
signal.signal(signal.SIGINT, signal_handler)
@@ -81,10 +85,6 @@ def main():
die('"a" must not exist in the current directory.')
if os.path.lexists('b'):
die('"b" must not exist in the current directory.')
if os.path.lexists('patches.gen'):
die('"patches.gen" must not exist in the current directory.')
check_git_state(args.master_branch, True, 'patches')
curversion = get_rsync_version()
@@ -120,8 +120,6 @@ def main():
if not re.match(r'^del', ans, flags=re.I):
die("Aborted")
cmd_chk(['git', 'tag', '-d', v_ver])
if os.path.isdir('patches/.git'):
cmd_chk(f"cd patches && git tag -d '{v_ver}'")
version = re.sub(r'[-.]*pre[-.]*', 'pre', version)
if 'pre' in version and not curversion.endswith('dev'):
@@ -213,6 +211,9 @@ About to:
x_re = re.compile(r'^(#define RSYNC_VERSION).*', re.M)
msg = f"Unable to update RSYNC_VERSION in {fn}"
txt = replace_or_die(x_re, r'\1 "%s"' % version, txt, msg)
x_re = re.compile(r'^(#define MAINTAINER_TZ_OFFSET).*', re.M)
msg = f"Unable to update MAINTAINER_TZ_OFFSET in {fn}"
txt = replace_or_die(x_re, r'\1 ' + tz_num, txt, msg)
elif '.spec' in fn:
for var, val in specvars.items():
x_re = re.compile(r'^%s .*' % re.escape(var), re.M)
@@ -220,7 +221,7 @@ About to:
x_re = re.compile(r'^\* \w\w\w \w\w\w \d\d \d\d\d\d (.*)', re.M)
txt = replace_or_die(x_re, r'%s \1' % cl_today, txt, f"Unable to update ChangeLog header in {fn}")
elif fn == 'rsync.h':
x_re = re.compile('(#define\s+SUBPROTOCOL_VERSION)\s+(\d+)')
x_re = re.compile(r'(#define\s+SUBPROTOCOL_VERSION)\s+(\d+)')
repl = lambda m: m[1] + ' ' + ('0' if not pre or not proto_changed else '1' if m[2] == '0' else m[2])
txt = replace_or_die(x_re, repl, txt, f"Unable to find SUBPROTOCOL_VERSION define in {fn}")
elif fn == 'NEWS.md':
@@ -252,7 +253,6 @@ About to:
cmd_run("git diff".split())
srctar_name = f"{rsync_ver}.tar.gz"
pattar_name = f"rsync-patches-{version}.tar.gz"
diff_name = f"{rsync_lastver}-{version}.diffs.gz"
srctar_file = os.path.join(dest, srcdir, srctar_name)
pattar_file = os.path.join(dest, srcdir, pattar_name)
@@ -266,8 +266,6 @@ About to:
- git commit all changes
- run a full build, ensuring that the manpages & configure.sh are up-to-date
- merge the {args.master_branch} branch into the patch/{args.master_branch}/* branches
- update the files in the "patches" dir and OPTIONALLY (if you type 'y') to
run patch-update with the --make option (which opens a shell on error)
""")
ans = input("<Press Enter OR 'y' to continue> ")
@@ -282,20 +280,10 @@ About to:
if s.returncode:
die('Aborting')
print('Updating files in "patches" dir ...')
s = cmd_run(f'packaging/patch-update --branch={args.master_branch}')
if s.returncode:
die('Aborting')
if re.match(r'^y', ans, re.I):
print(f'\nRunning smart-make on all "patch/{args.master_branch}/*" branches ...')
cmd_run(f"packaging/patch-update --branch={args.master_branch} --skip-check --make")
if os.path.isdir('patches/.git'):
s = cmd_run(f"cd patches && git commit -a -m 'The patches for {version}.'")
if s.returncode:
die('Aborting')
print(f"""\
{dash_line}
@@ -303,8 +291,6 @@ About to:
- create signed tag for this release: {v_ver}
- create release diffs, "{diff_name}"
- create release tar, "{srctar_name}"
- generate {rsync_ver}/patches/* files
- create patches tar, "{pattar_name}"
- update top-level README.md, NEWS.md, TODO, and ChangeLog
- update top-level rsync*.html manpages
- gpg-sign the release files
@@ -320,12 +306,6 @@ About to:
if 'bad passphrase' in out or 'failed' in out:
die('Aborting')
if os.path.isdir('patches/.git'):
out = cmd_txt(f"cd patches && git tag -s -m 'Version {version}.' {v_ver}", capture='combined').out
print(out, end='')
if 'bad passphrase' in out or 'failed' in out:
die('Aborting')
os.environ['PATH'] = ORIGINAL_PATH
# Extract the generated files from the old tar.
@@ -347,15 +327,6 @@ About to:
cmd_chk(['fakeroot', 'tar', 'czf', srctar_file, '--exclude=.github', rsync_ver])
shutil.rmtree(rsync_ver)
print(f'Updating files in "{rsync_ver}/patches" dir ...')
os.mkdir(rsync_ver, 0o755)
os.mkdir(f"{rsync_ver}/patches", 0o755)
cmd_chk(f"packaging/patch-update --skip-check --branch={args.master_branch} --gen={rsync_ver}/patches".split())
print(f"Creating {pattar_file} ...")
cmd_chk(['fakeroot', 'tar', 'chzf', pattar_file, rsync_ver + '/patches'])
shutil.rmtree(rsync_ver)
print(f"Updating the other files in {dest} ...")
md_files = 'README.md NEWS.md INSTALL.md'.split()
html_files = [ fn for fn in gen_pathnames if fn.endswith('.html') ]

703
packaging/release.py Executable file
View File

@@ -0,0 +1,703 @@
#!/usr/bin/env python3
# Step-based release script for rsync. Each step is a separate invocation
# selected by a --step-N-XX option, so the maintainer drives the release
# manually one piece at a time.
#
# All persistent state and working files live in ../release/ (a sibling of
# the rsync git checkout):
#
# ../release/rsync-ftp/ mirror of samba.org:/home/ftp/pub/rsync
# ../release/rsync-html/ git checkout of rsync-web (the html site)
# ../release/work/ scratch space for tarball / diff staging
# ../release/release-state.json info shared between steps
#
# The rsync-patches archive is no longer maintained and has been dropped.
#
# Run "packaging/release.py --list" to see the step list.
import os, sys, re, argparse, glob, shutil, json, signal, subprocess
from datetime import datetime
sys.path = ['packaging'] + sys.path
from pkglib import (
warn, die, cmd_run, cmd_chk, cmd_txt, cmd_txt_chk, cmd_pipe,
check_git_state, get_rsync_version,
get_NEWS_version_info, get_protocol_versions,
)
# ---------- Paths ----------
RELEASE_DIR = os.path.realpath('../release')
FTP_DIR = os.path.join(RELEASE_DIR, 'rsync-ftp')
HTML_DIR = os.path.join(RELEASE_DIR, 'rsync-html')
WORK_DIR = os.path.join(RELEASE_DIR, 'work')
STATE_FILE = os.path.join(RELEASE_DIR, 'release-state.json')
# Local rsync-web checkout (sibling of rsync-git) is the source-of-truth for
# the git-tracked html content. The maintainer pulls/commits/pushes there;
# step-1-fetch just snapshots it into HTML_DIR for the release flow.
HTML_SRC = os.path.realpath('../rsync-web')
FTP_REMOTE_PATH = '/home/ftp/pub/rsync'
HTML_REMOTE_PATH = '/home/httpd/html/rsync'
# Files that ./configure + make produce and that the release tarball / diff
# need to bundle alongside the git-tracked source. Mirrors the GENFILES
# definition in Makefile.in (with rrsync.1{,.html} since we always configure
# --with-rrsync in --step-4-build).
GEN_FILES = [
'configure.sh',
'aclocal.m4',
'config.h.in',
'rsync.1', 'rsync.1.html',
'rsync-ssl.1', 'rsync-ssl.1.html',
'rsyncd.conf.5', 'rsyncd.conf.5.html',
'rrsync.1', 'rrsync.1.html',
]
# ---------- Step registry ----------
STEPS = [
('step-1-fetch', 'mirror ../release/rsync-ftp from samba.org and snapshot ../release/rsync-html from ../rsync-web'),
('step-2-prepare', 'gather release info interactively and write release-state.json'),
('step-3-tweak', 'update version.h, rsync.h, NEWS.md, and packaging/*.spec'),
('step-4-build', 'run smart-make + make gen'),
('step-5-commit', 'git commit -a (commit the prepared release changes)'),
('step-6-tag', 'create the gpg-signed git tag'),
('step-7-tarball', 'build the source tarball and diffs.gz against the previous release'),
('step-8-update-ftp', 'refresh README/NEWS/INSTALL/html in the ftp dir, regen ChangeLog.gz, gpg-sign tarballs'),
('step-9-toplinks', 'hard-link top-level release files (final releases only)'),
('step-10-push-ftp', 'rsync ../release/rsync-ftp/ to samba.org'),
('step-11-push-html', 'rsync ../release/rsync-html/ to samba.org (after any manual edits)'),
('step-12-push-git', 'print the git push commands for you to run'),
]
STEP_FLAGS = [s[0] for s in STEPS]
DASH_LINE = '=' * 74
# ---------- State helpers ----------
def load_state():
if not os.path.isfile(STATE_FILE):
die(f"{STATE_FILE} not found. Run --step-2-prepare first.")
with open(STATE_FILE, 'r', encoding='utf-8') as fh:
return json.load(fh)
def save_state(state):
os.makedirs(RELEASE_DIR, exist_ok=True)
with open(STATE_FILE, 'w', encoding='utf-8') as fh:
json.dump(state, fh, indent=2, sort_keys=True)
fh.write('\n')
def require_samba_host():
host = os.environ.get('RSYNC_SAMBA_HOST', '')
if not host.endswith('.samba.org'):
die("Set RSYNC_SAMBA_HOST in your environment to the samba hostname (e.g. hr3.samba.org).")
return host
def require_top_of_checkout():
if not os.path.isfile('packaging/release.py'):
die("Run this script from the top of your rsync checkout.")
if not os.path.isdir('.git'):
die("There is no .git dir in the current directory.")
def replace_or_die(regex, repl, txt, die_msg):
m = regex.search(txt)
if not m:
die(die_msg)
return regex.sub(repl, txt, 1)
def section(title):
print(f"\n{DASH_LINE}\n== {title}\n{DASH_LINE}")
def confirm(prompt, default_no=True):
suffix = '[n] ' if default_no else '[y] '
ans = input(f"{prompt} {suffix}").strip().lower()
if default_no:
return ans.startswith('y')
return ans == '' or ans.startswith('y')
# ---------- Step 1: fetch ftp + html ----------
def step_1_fetch(args):
host = require_samba_host()
os.makedirs(RELEASE_DIR, exist_ok=True)
os.makedirs(WORK_DIR, exist_ok=True)
section(f"Fetching ftp dir into {FTP_DIR}")
if not os.path.isdir(FTP_DIR):
os.makedirs(FTP_DIR)
# The .filt file lives in the ftp dir on the server; mirror down using the
# transmitted filter, falling back to no filter on the very first pull.
filt = os.path.join(FTP_DIR, '.filt')
if os.path.exists(filt):
opts = ['-aivOHP', f'-f:_{filt}']
else:
opts = ['-aivOHP']
cmd_chk(['rsync', *opts, f'{host}:{FTP_REMOTE_PATH}/', f'{FTP_DIR}/'])
section(f"Snapshotting html dir from {HTML_SRC} into {HTML_DIR}")
if not os.path.isdir(HTML_SRC):
die(f"{HTML_SRC} not found. Clone the rsync-web repo there first.")
if not os.path.isdir(os.path.join(HTML_SRC, '.git')):
die(f"{HTML_SRC} exists but is not a git checkout.")
print(f"(Make sure {HTML_SRC} is up to date — this script does not 'git pull' for you.)")
os.makedirs(HTML_DIR, exist_ok=True)
cmd_chk(['rsync', '-aiv', '--exclude=/.git',
f'{HTML_SRC}/', f'{HTML_DIR}/'])
# Then mirror non-git html content from the server (mirroring samba-rsync's
# behavior: skip files that the html git already provides).
filt = os.path.join(HTML_DIR, 'filt')
if os.path.exists(filt):
tmp_filt = os.path.join(HTML_DIR, 'tmp-filt')
cmd_chk(f"sed -n -e 's/[-P]/H/p' '{filt}' >'{tmp_filt}'")
cmd_chk(['rsync', '-aivOHP', f'-f._{tmp_filt}',
f'{host}:{HTML_REMOTE_PATH}/', f'{HTML_DIR}/'])
os.unlink(tmp_filt)
print(f"\nFetch complete. Local dirs are now in {RELEASE_DIR}.")
# ---------- Step 2: prepare ----------
def step_2_prepare(args):
require_top_of_checkout()
os.makedirs(RELEASE_DIR, exist_ok=True)
if not os.path.isdir(FTP_DIR):
die(f"{FTP_DIR} does not exist. Run --step-1-fetch first.")
now = datetime.now().astimezone()
cl_today = now.strftime('* %a %b %d %Y')
year = now.strftime('%Y')
ztoday = now.strftime('%d %b %Y')
today = ztoday.lstrip('0')
tz_now = now.strftime('%z')
tz_num = tz_now[0:1].replace('+', '') + str(float(tz_now[1:3]) + float(tz_now[3:]) / 60)
curversion = get_rsync_version()
lastversion, last_protocol_version, pdate = get_NEWS_version_info()
protocol_version, subprotocol_version = get_protocol_versions()
# Default next version: bump preN, or move dev -> pre1.
version = curversion
m = re.search(r'pre(\d+)', version)
if m:
version = re.sub(r'pre\d+', 'pre' + str(int(m[1]) + 1), version)
else:
version = version.replace('dev', 'pre1')
print(f"\nCurrent version (version.h): {curversion}")
print(f"Last released version (NEWS.md): {lastversion}")
print(f"Current protocol version: {protocol_version} (last released: {last_protocol_version})")
ans = input(f"\nVersion to release [{version}, '.' to drop the preN suffix]: ").strip()
if ans == '.':
version = re.sub(r'pre\d+', '', version)
elif ans:
version = ans
if not re.match(r'^[\d.]+(pre\d+)?$', version):
die(f'Invalid version: "{version}"')
version = re.sub(r'[-.]*pre[-.]*', 'pre', version)
if 'pre' in version and not curversion.endswith('dev'):
lastversion = curversion
ans = input(f"Previous version to diff against [{lastversion}]: ").strip()
if ans:
lastversion = ans
lastversion = re.sub(r'[-.]*pre[-.]*', 'pre', lastversion)
m = re.search(r'(pre\d+)', version)
pre = m[1] if m else ''
finalversion = re.sub(r'pre\d+', '', version)
release = '0.1' if pre else '1'
ans = input(f"RPM release number [{release}]: ").strip()
if ans:
release = ans
if pre:
release += '.' + pre
proto_changed = protocol_version != last_protocol_version
if proto_changed:
if finalversion in pdate:
proto_change_date = pdate[finalversion]
else:
while True:
ans = input(f"Date the protocol changed to {protocol_version} (dd Mmm yyyy): ").strip()
if re.match(r'^\d\d \w\w\w \d\d\d\d$', ans):
break
proto_change_date = ans
else:
proto_change_date = ' ' * 11
if 'pre' in lastversion:
if not pre:
die("Refusing to diff a release version against a pre-release version.")
srcdir = srcdiffdir = lastsrcdir = 'src-previews'
elif pre:
srcdir = srcdiffdir = 'src-previews'
lastsrcdir = 'src'
else:
srcdir = lastsrcdir = 'src'
srcdiffdir = 'src-diffs'
state = {
'version': version,
'lastversion': lastversion,
'finalversion': finalversion,
'pre': pre,
'release': release,
'protocol_version': protocol_version,
'subprotocol_version': subprotocol_version,
'proto_changed': proto_changed,
'proto_change_date': proto_change_date,
'srcdir': srcdir,
'srcdiffdir': srcdiffdir,
'lastsrcdir': lastsrcdir,
'today': today,
'ztoday': ztoday,
'cl_today': cl_today,
'year': year,
'tz_num': tz_num,
'master_branch': args.master_branch,
}
save_state(state)
section("Release info")
for k in ('version', 'lastversion', 'release', 'srcdir', 'srcdiffdir', 'lastsrcdir',
'protocol_version', 'proto_changed', 'proto_change_date'):
print(f" {k}: {state[k]}")
print(f"\nWrote {STATE_FILE}. Re-run --step-2-prepare to change anything.")
# ---------- Step 3: tweak version files ----------
def step_3_tweak(args):
require_top_of_checkout()
state = load_state()
version = state['version']
finalversion = state['finalversion']
pre = state['pre']
release = state['release']
today = state['today']
ztoday = state['ztoday']
cl_today = state['cl_today']
year = state['year']
tz_num = state['tz_num']
proto_changed = state['proto_changed']
proto_change_date = state['proto_change_date']
protocol_version = state['protocol_version']
srcdir = state['srcdir']
specvars = {
'Version:': finalversion,
'Release:': release,
'%define fullversion': f'%{{version}}{pre}',
'Released': version + '.',
'%define srcdir': srcdir,
}
tweak_files = ['version.h', 'rsync.h', 'NEWS.md']
tweak_files += glob.glob('packaging/*.spec')
tweak_files += glob.glob('packaging/*/*.spec')
for fn in tweak_files:
with open(fn, 'r', encoding='utf-8') as fh:
old_txt = txt = fh.read()
if fn == 'version.h':
x_re = re.compile(r'^(#define RSYNC_VERSION).*', re.M)
txt = replace_or_die(x_re, r'\1 "%s"' % version, txt,
f"Unable to update RSYNC_VERSION in {fn}")
x_re = re.compile(r'^(#define MAINTAINER_TZ_OFFSET).*', re.M)
txt = replace_or_die(x_re, r'\1 ' + tz_num, txt,
f"Unable to update MAINTAINER_TZ_OFFSET in {fn}")
elif fn == 'rsync.h':
x_re = re.compile(r'(#define\s+SUBPROTOCOL_VERSION)\s+(\d+)')
repl = lambda m: m[1] + ' ' + (
'0' if not pre or not proto_changed
else '1' if m[2] == '0'
else m[2])
txt = replace_or_die(x_re, repl, txt,
f"Unable to find SUBPROTOCOL_VERSION in {fn}")
elif fn == 'NEWS.md':
efv = re.escape(finalversion)
x_re = re.compile(
r'^# NEWS for rsync %s \(UNRELEASED\)\s+## Changes in this version:\n' % efv
+ r'(\n### PROTOCOL NUMBER:\s+- The protocol number was changed to \d+\.\n)?')
rel_day = 'UNRELEASED' if pre else today
repl = (f'# NEWS for rsync {finalversion} ({rel_day})\n\n'
+ '## Changes in this version:\n')
if proto_changed:
repl += f'\n### PROTOCOL NUMBER:\n\n - The protocol number was changed to {protocol_version}.\n'
good_top = re.sub(r'\(.*?\)', '(UNRELEASED)', repl, 1)
msg = (f"The top of {fn} is not in the right format. It should be:\n" + good_top)
txt = replace_or_die(x_re, repl, txt, msg)
x_re = re.compile(
r'^(\| )(\S{2} \S{3} \d{4})(\s+\|\s+%s\s+\| ).{11}(\s+\| )\S{2}(\s+\|+)$' % efv,
re.M)
repl = lambda m: (m[1] + (m[2] if pre else ztoday) + m[3]
+ proto_change_date + m[4] + protocol_version + m[5])
txt = replace_or_die(x_re, repl, txt,
f'Unable to find "| ?? ??? {year} | {finalversion} | ... |" line in {fn}')
elif '.spec' in fn:
for var, val in specvars.items():
x_re = re.compile(r'^%s .*' % re.escape(var), re.M)
txt = replace_or_die(x_re, var + ' ' + val, txt,
f"Unable to update {var} in {fn}")
x_re = re.compile(r'^\* \w\w\w \w\w\w \d\d \d\d\d\d (.*)', re.M)
txt = replace_or_die(x_re, r'%s \1' % cl_today, txt,
f"Unable to update ChangeLog header in {fn}")
else:
die(f"Unrecognized file in tweak_files: {fn}")
if txt != old_txt:
print(f"Updating {fn}")
with open(fn, 'w', encoding='utf-8') as fh:
fh.write(txt)
cmd_chk(['packaging/year-tweak'])
section("git diff after tweaks")
cmd_run(['git', '--no-pager', 'diff'])
# ---------- Step 4: build ----------
def step_4_build(args):
require_top_of_checkout()
load_state() # just to ensure we've prepared
section("Running prepare-source + configure --prefix=/usr --with-rrsync + make + make gen")
# Always re-prepare so configure.sh is current; we run configure ourselves
# with the release-required flags rather than relying on the cached
# config.status (which may have been produced with different options).
if os.path.isfile('.fetch'):
cmd_chk(['./prepare-source', 'fetch'])
else:
cmd_chk(['./prepare-source'])
cmd_chk(['./configure', '--prefix=/usr', '--with-rrsync'])
cmd_chk(['make'])
cmd_chk(['make', 'gen'])
# ---------- Step 5: commit ----------
def step_5_commit(args):
require_top_of_checkout()
state = load_state()
version = state['version']
section("git status")
cmd_run(['git', 'status'])
if not confirm("Commit all current changes with the release message?"):
die("Aborted.")
cmd_chk(['git', 'commit', '-a', '-m', f'Preparing for release of {version} [buildall]'])
# ---------- Step 6: tag ----------
def step_6_tag(args):
require_top_of_checkout()
state = load_state()
version = state['version']
v_ver = 'v' + version
out = cmd_txt_chk(['git', 'tag', '-l', v_ver]).out
if out.strip():
if not confirm(f"Tag {v_ver} already exists. Delete and recreate?"):
die("Aborted.")
cmd_chk(['git', 'tag', '-d', v_ver])
# Prime the gpg agent so the actual tag signing won't prompt.
section("Priming gpg agent")
cmd_run("touch TeMp; gpg --sign TeMp; rm -f TeMp TeMp.gpg")
section(f"Creating signed tag {v_ver}")
out = cmd_txt(['git', 'tag', '-s', '-m', f'Version {version}.', v_ver],
capture='combined').out
print(out, end='')
if 'bad passphrase' in out.lower() or 'failed' in out.lower():
die("Tag creation failed.")
# ---------- Step 7: tarball + diff ----------
def step_7_tarball(args):
require_top_of_checkout()
state = load_state()
version = state['version']
lastversion = state['lastversion']
pre = state['pre']
srcdir = state['srcdir']
srcdiffdir = state['srcdiffdir']
lastsrcdir = state['lastsrcdir']
rsync_ver = 'rsync-' + version
rsync_lastver = 'rsync-' + lastversion
v_ver = 'v' + version
srctar_name = f"{rsync_ver}.tar.gz"
diff_name = f"{rsync_lastver}-{version}.diffs.gz"
srctar_file = os.path.join(FTP_DIR, srcdir, srctar_name)
diff_file = os.path.join(FTP_DIR, srcdiffdir, diff_name)
lasttar_file = os.path.join(FTP_DIR, lastsrcdir, rsync_lastver + '.tar.gz')
for d in (os.path.dirname(srctar_file), os.path.dirname(diff_file)):
os.makedirs(d, exist_ok=True)
if not os.path.isfile(lasttar_file):
die(f"Previous tarball not found: {lasttar_file}")
# Stage in ../release/work to keep the source checkout clean.
if os.path.isdir(WORK_DIR):
shutil.rmtree(WORK_DIR)
os.makedirs(WORK_DIR)
a_dir = os.path.join(WORK_DIR, 'a')
b_dir = os.path.join(WORK_DIR, 'b')
# Extract gen files from the previous tarball into work/a/.
tweaked_gen_files = [os.path.join(rsync_lastver, fn) for fn in GEN_FILES]
cmd_chk(['tar', '-C', WORK_DIR, '-xzf', lasttar_file, *tweaked_gen_files])
os.rename(os.path.join(WORK_DIR, rsync_lastver), a_dir)
# Copy current gen files (built in the top-level checkout) into work/b/.
os.makedirs(b_dir)
cmd_chk(['rsync', '-a', *GEN_FILES, b_dir + '/'])
section(f"Creating {diff_file}")
sed_script = r's:^((---|\+\+\+) [ab]/[^\t]+)\t.*:\1:' # no single quotes!
cmd_chk(
f"(git diff v{lastversion} {v_ver} -- ':!.github'; "
f"diff -upN {a_dir} {b_dir} | sed -r '{sed_script}') | gzip -9 >{diff_file}")
section(f"Creating {srctar_file}")
# Reuse work/b/ (which already holds the fresh gen files) as the release
# staging dir, then let "git archive" overlay the git-tracked source files
# on top. That way the tarball ends up with both gen files and source.
rsync_ver_dir = os.path.join(WORK_DIR, rsync_ver)
shutil.rmtree(a_dir)
os.rename(b_dir, rsync_ver_dir)
cmd_chk(f"git archive --format=tar --prefix={rsync_ver}/ {v_ver} | "
f"tar -C {WORK_DIR} -xf -")
cmd_chk(f"support/git-set-file-times --quiet --prefix={rsync_ver_dir}/")
cmd_chk(['fakeroot', 'tar', '-C', WORK_DIR, '-czf', srctar_file,
'--exclude=.github', rsync_ver])
# Leave staging in place; --step-8-update-ftp does its own thing.
print(f"\nCreated:\n {srctar_file}\n {diff_file}")
# ---------- Step 8: update ftp ----------
def step_8_update_ftp(args):
require_top_of_checkout()
state = load_state()
version = state['version']
lastversion = state['lastversion']
srcdir = state['srcdir']
srcdiffdir = state['srcdiffdir']
rsync_ver = 'rsync-' + version
rsync_lastver = 'rsync-' + lastversion
srctar_file = os.path.join(FTP_DIR, srcdir, f"{rsync_ver}.tar.gz")
diff_file = os.path.join(FTP_DIR, srcdiffdir,
f"{rsync_lastver}-{version}.diffs.gz")
section(f"Refreshing top-of-tree files in {FTP_DIR}")
md_files = ['README.md', 'NEWS.md', 'INSTALL.md']
html_files = [fn for fn in GEN_FILES if fn.endswith('.html')]
cmd_chk(['rsync', '-a', *md_files, *html_files, FTP_DIR + '/'])
cmd_chk(['./md-convert', '--dest', FTP_DIR, *md_files])
section(f"Regenerating {FTP_DIR}/ChangeLog.gz")
cmd_chk(f"git log --name-status | gzip -9 >{FTP_DIR}/ChangeLog.gz")
# Prime gpg agent and then sign the tar + diff.
section("Priming gpg agent")
cmd_run("touch TeMp; gpg --sign TeMp; rm -f TeMp TeMp.gpg")
for fn in (srctar_file, diff_file):
if not os.path.isfile(fn):
die(f"Missing file to sign: {fn}. Did --step-7-tarball run successfully?")
asc_fn = fn + '.asc'
if os.path.lexists(asc_fn):
os.unlink(asc_fn)
section(f"GPG-signing {fn}")
res = cmd_run(['gpg', '--batch', '-ba', fn])
if res.returncode not in (0, 2):
die("gpg signing failed.")
# ---------- Step 9: top-level hard links ----------
def step_9_toplinks(args):
require_top_of_checkout()
state = load_state()
pre = state['pre']
if pre:
print("Skipping: pre-releases do not get top-level hard links.")
return
version = state['version']
lastversion = state['lastversion']
srcdir = state['srcdir']
srcdiffdir = state['srcdiffdir']
rsync_ver = 'rsync-' + version
rsync_lastver = 'rsync-' + lastversion
srctar_file = os.path.join(FTP_DIR, srcdir, f"{rsync_ver}.tar.gz")
diff_file = os.path.join(FTP_DIR, srcdiffdir,
f"{rsync_lastver}-{version}.diffs.gz")
section("Removing stale top-level rsync-* files")
for find in [f'{FTP_DIR}/rsync-*.gz',
f'{FTP_DIR}/rsync-*.asc',
f'{FTP_DIR}/src-previews/rsync-*diffs.gz*']:
for fn in glob.glob(find):
os.unlink(fn)
top_link = [
srctar_file, srctar_file + '.asc',
diff_file, diff_file + '.asc',
]
for fn in top_link:
target = re.sub(r'/src(-\w+)?/', '/', fn)
if os.path.lexists(target):
os.unlink(target)
os.link(fn, target)
print(f" linked {target}")
# ---------- Step 10: push ftp ----------
def step_10_push_ftp(args):
host = require_samba_host()
if not os.path.isdir(FTP_DIR):
die(f"{FTP_DIR} does not exist. Run --step-1-fetch first.")
section(f"rsync ftp dir to {host}")
rsync_with_confirm(['-aivOHP', '--chown=:rsync', '--del',
f'-f._{os.path.join(FTP_DIR, ".filt")}',
f'{FTP_DIR}/', f'{host}:{FTP_REMOTE_PATH}/'])
# ---------- Step 11: push html ----------
def step_11_push_html(args):
host = require_samba_host()
if not os.path.isdir(HTML_DIR):
die(f"{HTML_DIR} does not exist. Run --step-1-fetch first.")
section(f"rsync html dir to {host}")
filt = os.path.join(HTML_DIR, 'filt')
rsync_with_confirm(['-aivOHP', '--chown=:rsync', '--del',
f'-f._{filt}',
f'{HTML_DIR}/', f'{host}:{HTML_REMOTE_PATH}/'])
# ---------- Step 12: print push-git instructions ----------
def step_12_push_git(args):
state = load_state()
version = state['version']
master_branch = state['master_branch']
v_ver = 'v' + version
print(f"""\
{DASH_LINE}
Run these from the rsync-git checkout (this script does not push for you):
git push origin {master_branch}
git push origin {v_ver}
If you have a 'samba' remote configured (git.samba.org:/data/git/rsync.git):
git push samba {master_branch}
git push samba {v_ver}
Then upload the tarball + .asc to the GitHub release for {v_ver}, run
packaging/send-news (when convenient), and announce on rsync-announce@,
rsync@, and Discord.
""")
# ---------- shared rsync-with-confirm ----------
def rsync_with_confirm(rsync_args):
"""Run an rsync command in dry-run mode, then ask before running for real."""
cmd_run(['rsync', '--dry-run', *rsync_args])
if confirm("Run without --dry-run?"):
cmd_run(['rsync', *rsync_args])
# ---------- dispatch ----------
STEP_FUNCS = {
'step-1-fetch': step_1_fetch,
'step-2-prepare': step_2_prepare,
'step-3-tweak': step_3_tweak,
'step-4-build': step_4_build,
'step-5-commit': step_5_commit,
'step-6-tag': step_6_tag,
'step-7-tarball': step_7_tarball,
'step-8-update-ftp': step_8_update_ftp,
'step-9-toplinks': step_9_toplinks,
'step-10-push-ftp': step_10_push_ftp,
'step-11-push-html': step_11_push_html,
'step-12-push-git': step_12_push_git,
}
def signal_handler(sig, frame):
die("\nAborting due to SIGINT.")
def main():
parser = argparse.ArgumentParser(
description="Step-based release script for rsync.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Run --list to see the steps. Each invocation runs exactly one --step-* option.")
parser.add_argument('--branch', '-b', dest='master_branch', default='master',
help="The branch to release (default: master).")
parser.add_argument('--list', action='store_true',
help="List all release steps and exit.")
grp = parser.add_mutually_exclusive_group()
for flag, descr in STEPS:
grp.add_argument('--' + flag, dest='step', action='store_const',
const=flag, help=descr)
args = parser.parse_args()
if args.list:
print("Release steps:")
for flag, descr in STEPS:
print(f" --{flag:18s} {descr}")
return
if not args.step:
parser.error("pick one --step-N-XX option (or --list to see them).")
signal.signal(signal.SIGINT, signal_handler)
os.environ['LESS'] = 'mqeiXR'
STEP_FUNCS[args.step](args)
if __name__ == '__main__':
main()
# vim: sw=4 et ft=python

View File

@@ -7,9 +7,6 @@
import sys, os, re, argparse, subprocess
from datetime import datetime
MAINTAINER_NAME = 'Wayne Davison'
MAINTAINER_SUF = ' ' + MAINTAINER_NAME + "\n"
def main():
latest_year = '2000'
@@ -22,10 +19,6 @@ def main():
m = argparse.Namespace(**m.groupdict())
if m.year > latest_year:
latest_year = m.year
if m.fn.startswith('zlib/') or m.fn.startswith('popt/'):
continue
if re.search(r'\.(c|h|sh|test)$', m.fn):
maybe_edit_copyright_year(m.fn, m.year)
proc.communicate()
fn = 'latest-year.h'
@@ -39,55 +32,8 @@ def main():
fh.write(txt)
def maybe_edit_copyright_year(fn, year):
opening_lines = [ ]
copyright_line = None
with open(fn, 'r', encoding='utf-8') as fh:
for lineno, line in enumerate(fh):
opening_lines.append(line)
if lineno > 3 and not re.search(r'\S', line):
break
m = re.match(r'^(?P<pre>.*Copyright\s+\S+\s+)(?P<year>\d\d\d\d(?:-\d\d\d\d)?(,\s+\d\d\d\d)*)(?P<suf>.+)', line)
if not m:
continue
copyright_line = argparse.Namespace(**m.groupdict())
copyright_line.lineno = len(opening_lines)
copyright_line.is_maintainer_line = MAINTAINER_NAME in copyright_line.suf
copyright_line.txt = line
if copyright_line.is_maintainer_line:
break
if not copyright_line:
return
if copyright_line.is_maintainer_line:
cyears = copyright_line.year.split('-')
if year == cyears[0]:
cyears = [ year ]
else:
cyears = [ cyears[0], year ]
txt = copyright_line.pre + '-'.join(cyears) + MAINTAINER_SUF
if txt == copyright_line.txt:
return
opening_lines[copyright_line.lineno - 1] = txt
else:
if fn.startswith('lib/') or fn.startswith('testsuite/'):
return
txt = copyright_line.pre + year + MAINTAINER_SUF
opening_lines[copyright_line.lineno - 1] += txt
remaining_txt = fh.read()
print(f"Updating {fn} with year {year}")
with open(fn, 'w', encoding='utf-8') as fh:
fh.write(''.join(opening_lines))
fh.write(remaining_txt)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Grab the year of last mod for our c & h files and make sure the Copyright comment is up-to-date.")
parser = argparse.ArgumentParser(description="Grab the year of the last mod for our c & h files and make sure the LATEST_YEAR value is accurate.")
args = parser.parse_args()
main()

View File

@@ -1,61 +0,0 @@
/** \ingroup popt
* \file popt/findme.c
*/
/* (C) 1998-2002 Red Hat, Inc. -- Licensing details are in the COPYING
file accompanying popt source distributions, available from
ftp://ftp.rpm.org/pub/rpm/dist. */
#include "system.h"
#include "findme.h"
const char * findProgramPath(const char * argv0)
{
char * path = getenv("PATH");
char * pathbuf;
char * start, * chptr;
char * buf;
size_t bufsize;
if (argv0 == NULL) return NULL; /* XXX can't happen */
/* If there is a / in the argv[0], it has to be an absolute path */
if (strchr(argv0, '/'))
return xstrdup(argv0);
if (path == NULL) return NULL;
bufsize = strlen(path) + 1;
start = pathbuf = malloc(bufsize);
if (pathbuf == NULL) return NULL; /* XXX can't happen */
strlcpy(pathbuf, path, bufsize);
bufsize += sizeof "/" - 1 + strlen(argv0);
buf = malloc(bufsize);
if (buf == NULL) {
free(pathbuf);
return NULL; /* XXX can't happen */
}
chptr = NULL;
/*@-branchstate@*/
do {
if ((chptr = strchr(start, ':')))
*chptr = '\0';
snprintf(buf, bufsize, "%s/%s", start, argv0);
if (!access(buf, X_OK)) {
free(pathbuf);
return buf;
}
if (chptr)
start = chptr + 1;
else
start = NULL;
} while (start && *start);
/*@=branchstate@*/
free(pathbuf);
free(buf);
return NULL;
}

View File

@@ -1,20 +0,0 @@
/** \ingroup popt
* \file popt/findme.h
*/
/* (C) 1998-2000 Red Hat, Inc. -- Licensing details are in the COPYING
file accompanying popt source distributions, available from
ftp://ftp.rpm.org/pub/rpm/dist. */
#ifndef H_FINDME
#define H_FINDME
/**
* Return absolute path to executable by searching PATH.
* @param argv0 name of executable
* @return (malloc'd) absolute path to executable (or NULL)
*/
/*@null@*/ const char * findProgramPath(/*@null@*/ const char * argv0)
/*@*/;
#endif

View File

@@ -513,6 +513,7 @@ has its own detailed description later in this manpage.
--compress, -z compress file data during the transfer
--compress-choice=STR choose the compression algorithm (aka --zc)
--compress-level=NUM explicitly set compression level (aka --zl)
--compress-threads=NUM explicitly set compression threads (aka --zt)
--skip-compress=LIST skip compressing files with suffix in LIST
--cvs-exclude, -C auto-ignore files in the same way CVS does
--filter=RULE, -f add a file-filtering RULE
@@ -2817,6 +2818,22 @@ expand it.
report something like "`Client compress: zstd (level 3)`" (along with the
checksum choice in effect).
0. `--compress-threads=NUM`, `--zt=NUM`
Set the number of threads to spawn when compressing data. Setting this
option to 1 or more will instruct the compression library to spawn 1 or
more threads for compression. Ideally, increasing the number of threads
will increase transfer speed if the transfer is CPU bound on the sender.
This option does not affect decompression.
Compression algorithms that allow threading:
- `zstd` (only when libzstd is compiled with threading support)
This option is ignored if one of the above alogithms is not selected as the
`--compression-choice` or if compression not enabled.
0. `--skip-compress=LIST`
**NOTE:** no compression method currently supports per-file compression

View File

@@ -1073,6 +1073,16 @@ in the values of parameters. See that section for details.
**system()** call's default shell), and use RSYNC_NO_XFER_EXEC to disable
both options completely.
0. `temp dir`
Specifies a directory that rsync should use for temporary files created
during the transfer of updated files. If that directory is on a different
partition, after transfer file is being copied instead of unlinked.
This parameter equals with `--temp-dir` option, so please consult rsync
manpage for further information.
## CONFIG DIRECTIVES
There are currently two config directives available that allow a config file to

468
runtests.py Executable file
View File

@@ -0,0 +1,468 @@
#!/usr/bin/env python3
# Copyright (C) 2001, 2002 by Martin Pool <mbp@samba.org>
# Copyright (C) 2003-2022 Wayne Davison
# Copyright (C) 2026 Andrew Tridgell
#
# Rewrite of runtests.sh in Python (runtests.sh is now deprecated).
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version
# 2 as published by the Free Software Foundation.
"""rsync test runner.
Invokes test scripts from testsuite/ and reports results.
Can be called by 'make check' or directly.
Usage:
./runtests.py [options] [TEST ...]
Each TEST is a test name (e.g. 'delete') or glob pattern (e.g. 'xattr*').
If no tests are specified, all tests are run.
"""
import argparse
import concurrent.futures
import glob
import os
import subprocess
import sys
import threading
def parse_args():
p = argparse.ArgumentParser(description='Run rsync test suite')
p.add_argument('tests', nargs='*', metavar='TEST',
help='Test names or patterns to run (default: all)')
p.add_argument('-j', '--parallel', type=int, default=1, metavar='N',
help='Run up to N tests in parallel (default: 1)')
p.add_argument('--valgrind', action='store_true',
help='Run rsync under valgrind (logs to per-process files)')
p.add_argument('--valgrind-opts', default='', metavar='OPTS',
help='Extra valgrind options (e.g. "--leak-check=full")')
p.add_argument('--preserve-scratch', action='store_true',
help='Keep scratch directories after tests complete')
p.add_argument('--log-level', type=int, default=1, metavar='N',
help='Verbosity level 1-10 (default: 1)')
p.add_argument('--always-log', action='store_true',
help='Show test logs even for passing tests')
p.add_argument('--stop-on-fail', action='store_true',
help='Stop after first test failure')
p.add_argument('--timeout', type=int, default=300, metavar='SECS',
help='Per-test timeout in seconds (default: 300)')
p.add_argument('--rsync-bin', default=None, metavar='PATH',
help='Path to rsync binary (default: ./rsync)')
p.add_argument('--tooldir', default=None, metavar='DIR',
help='Tool/build directory (default: cwd)')
p.add_argument('--srcdir', default=None, metavar='DIR',
help='Source directory (default: script directory)')
p.add_argument('--protocol', type=int, default=None, metavar='VER',
help='Force protocol version (adds --protocol=VER to rsync)')
p.add_argument('--expect-skipped', default=None, metavar='LIST',
help='Comma-separated list of expected-skipped tests')
return p.parse_args()
def find_setfacl_nodef(scratchbase):
"""Determine the setfacl command to remove default ACLs."""
for cmd in [
['setacl', '-k', 'u::7,g::5,o:5', scratchbase],
['setfacl', '-k', scratchbase],
['setfacl', '-s', 'u::7,g::5,o:5', scratchbase],
]:
try:
subprocess.run(cmd, capture_output=True, timeout=5)
return cmd[:2] if cmd[0] == 'setacl' else cmd[:2]
except (FileNotFoundError, subprocess.TimeoutExpired):
continue
try:
r = subprocess.run(['setfacl', '--help'], capture_output=True, text=True, timeout=5)
if '-k,' in r.stdout or '-k,' in r.stderr:
return ['setfacl', '-k']
except (FileNotFoundError, subprocess.TimeoutExpired):
pass
return None
def get_tls_args(config_h):
"""Determine TLS_ARGS from config.h."""
args = ''
try:
with open(config_h) as f:
text = f.read()
if '#define HAVE_LUTIMES 1' in text:
args += ' -l'
if '#undef CHOWN_MODIFIES_SYMLINK' in text:
args += ' -L'
except FileNotFoundError:
pass
return args.strip()
def read_shconfig(path):
"""Read shell config variables from shconfig."""
env = {}
try:
with open(path) as f:
for line in f:
line = line.strip()
if line.startswith('#') or line.startswith('export') or not line:
continue
if '=' in line:
k, _, v = line.partition('=')
env[k.strip()] = v.strip().strip('"')
except FileNotFoundError:
pass
return env
def get_testuser():
"""Determine the current test user."""
for cmd in ['/usr/bin/whoami', '/usr/ucb/whoami', '/bin/whoami']:
if os.path.isfile(cmd):
try:
return subprocess.check_output([cmd], text=True).strip()
except subprocess.CalledProcessError:
pass
try:
return subprocess.check_output(['id', '-un'], text=True).strip()
except (FileNotFoundError, subprocess.CalledProcessError):
return os.environ.get('LOGNAME', os.environ.get('USER', 'UNKNOWN'))
def prep_scratch(scratchdir, srcdir, tooldir, setfacl_nodef):
"""Prepare a scratch directory for a test."""
if os.path.isdir(scratchdir):
subprocess.run(['chmod', '-R', 'u+rwX', scratchdir], capture_output=True)
subprocess.run(['rm', '-rf', scratchdir], capture_output=True)
os.makedirs(scratchdir, exist_ok=True)
if setfacl_nodef:
subprocess.run(setfacl_nodef + [scratchdir], capture_output=True)
try:
os.chmod(scratchdir, os.stat(scratchdir).st_mode & ~0o2000) # clear setgid
except OSError:
pass
src_link = os.path.join(scratchdir, 'src')
if not os.path.exists(src_link):
if os.path.isabs(srcdir):
os.symlink(srcdir, src_link)
else:
os.symlink(os.path.join(tooldir, srcdir), src_link)
def collect_tests(suitedir, patterns):
"""Collect test scripts matching the given patterns."""
if not patterns:
tests = sorted(glob.glob(os.path.join(suitedir, '*.test')))
else:
tests = []
for pat in patterns:
if not pat.endswith('.test'):
pat = pat + '.test'
matches = sorted(glob.glob(os.path.join(suitedir, pat)))
tests.extend(matches)
return tests
def build_rsync_cmd(rsync_bin, args, scratchbase):
"""Build the RSYNC command string for tests."""
parts = []
if args.valgrind:
vlog = os.path.join(scratchbase, 'valgrind.%p.log')
vopts = f'--log-file={vlog}'
if args.valgrind_opts:
vopts += ' ' + args.valgrind_opts
parts.append(f'valgrind {vopts}')
parts.append(rsync_bin)
if args.protocol is not None:
parts.append(f'--protocol={args.protocol}')
return ' '.join(parts)
class TestResult:
"""Result of a single test execution."""
__slots__ = ('testbase', 'result', 'output', 'skipped_reason')
def __init__(self, testbase, result, output='', skipped_reason=''):
self.testbase = testbase
self.result = result
self.output = output
self.skipped_reason = skipped_reason
def run_one_test(testscript, testbase, scratchdir, base_env, timeout,
srcdir, tooldir, setfacl_nodef, always_log):
"""Run a single test. Returns a TestResult.
This function is safe to call from multiple threads — it uses only
per-test state (unique scratchdir, copy of env).
"""
prep_scratch(scratchdir, srcdir, tooldir, setfacl_nodef)
env = base_env.copy()
env['scratchdir'] = scratchdir
logfile = os.path.join(scratchdir, 'test.log')
try:
with open(logfile, 'w') as log:
proc = subprocess.run(
['sh', '-e', testscript],
stdout=log, stderr=subprocess.STDOUT,
env=env, timeout=timeout,
cwd=env.get('TOOLDIR', '.')
)
result = proc.returncode
except subprocess.TimeoutExpired:
result = 1
with open(logfile, 'a') as log:
log.write(f"\nTIMEOUT: test took over {timeout} seconds\n")
# Build output text
output_parts = []
show_log = always_log or (result not in (0, 77, 78))
if show_log:
output_parts.append(f'----- {testbase} log follows')
try:
with open(logfile) as f:
output_parts.append(f.read().rstrip())
except FileNotFoundError:
pass
output_parts.append(f'----- {testbase} log ends')
rsyncd_log = os.path.join(scratchdir, 'rsyncd.log')
if os.path.isfile(rsyncd_log):
output_parts.append(f'----- {testbase} rsyncd.log follows')
with open(rsyncd_log) as f:
output_parts.append(f.read().rstrip())
output_parts.append(f'----- {testbase} rsyncd.log ends')
skipped_reason = ''
if result == 0:
output_parts.append(f'PASS {testbase}')
elif result == 77:
whyfile = os.path.join(scratchdir, 'whyskipped')
try:
with open(whyfile) as f:
skipped_reason = f.read().strip()
except FileNotFoundError:
pass
output_parts.append(f'SKIP {testbase} ({skipped_reason})')
elif result == 78:
output_parts.append(f'XFAIL {testbase}')
else:
output_parts.append(f'FAIL {testbase}')
return TestResult(testbase, result, '\n'.join(output_parts), skipped_reason)
# Lock for serializing output in parallel mode
_print_lock = threading.Lock()
def main():
args = parse_args()
# Also accept legacy environment variables
if args.preserve_scratch or os.environ.get('preserve_scratch') == 'yes':
args.preserve_scratch = True
if args.log_level == 1:
args.log_level = int(os.environ.get('loglevel', '1'))
if args.expect_skipped is None:
args.expect_skipped = os.environ.get('RSYNC_EXPECT_SKIPPED', 'IGNORE')
if os.environ.get('whichtests'):
args.tests = [os.environ['whichtests']]
# Determine directories
tooldir = args.tooldir or os.environ.get('TOOLDIR') or os.getcwd()
script_path = os.path.dirname(os.path.abspath(__file__))
srcdir = args.srcdir or script_path
if not srcdir or srcdir == '.':
srcdir = tooldir
rsync_bin = args.rsync_bin or os.environ.get('rsync_bin') or os.path.join(tooldir, 'rsync')
suitedir = os.path.join(srcdir, 'testsuite')
scratchbase = os.path.join(os.environ.get('scratchbase', tooldir), 'testtmp')
os.makedirs(scratchbase, exist_ok=True)
shconfig = read_shconfig(os.path.join(tooldir, 'shconfig'))
tls_args = get_tls_args(os.path.join(tooldir, 'config.h'))
setfacl_nodef = find_setfacl_nodef(scratchbase)
rsync_cmd = build_rsync_cmd(rsync_bin, args, scratchbase)
if not os.path.isfile(rsync_bin):
sys.stderr.write(f"rsync_bin {rsync_bin} is not a file\n")
sys.exit(2)
if not os.path.isdir(srcdir):
sys.stderr.write(f"srcdir {srcdir} is not a directory\n")
sys.exit(2)
testuser = get_testuser()
# Print header
print('=' * 60)
print(f'{sys.argv[0]} running in {tooldir}')
print(f' rsync_bin={rsync_cmd}')
print(f' srcdir={srcdir}')
print(f' TLS_ARGS={tls_args}')
print(f' testuser={testuser}')
print(f' os={subprocess.check_output(["uname", "-a"], text=True).strip()}')
print(f' preserve_scratch={"yes" if args.preserve_scratch else "no"}')
if args.valgrind:
print(f' valgrind=enabled (logs in valgrind.*.log)')
if args.parallel > 1:
print(f' parallel={args.parallel}')
print(f' scratchbase={scratchbase}')
# Build base environment for test scripts
path = os.environ.get('PATH', '')
if os.path.isdir('/usr/xpg4/bin'):
path = '/usr/xpg4/bin:' + path
base_env = os.environ.copy()
base_env.update({
'PATH': path,
'POSIXLY_CORRECT': '1',
'TOOLDIR': tooldir,
'srcdir': srcdir,
'RSYNC': rsync_cmd,
'TLS_ARGS': tls_args,
'RUNSHFLAGS': '-e',
'scratchbase': scratchbase,
'suitedir': suitedir,
'TESTRUN_TIMEOUT': str(args.timeout),
'HOME': scratchbase,
})
for k, v in shconfig.items():
if v:
base_env[k] = v
if setfacl_nodef:
base_env['setfacl_nodef'] = ' '.join(setfacl_nodef)
else:
base_env['setfacl_nodef'] = 'true'
if args.log_level > 8:
base_env['RUNSHFLAGS'] = '-e -x'
# Collect tests
tests = collect_tests(suitedir, args.tests)
full_run = len(args.tests) == 0
# Record test order for consistent skipped-list output
test_order = {os.path.basename(t).replace('.test', ''): i for i, t in enumerate(tests)}
passed = 0
failed = 0
skipped = 0
skipped_list = []
def process_result(tr):
"""Process a TestResult and update counters. Returns True if test failed."""
nonlocal passed, failed, skipped
with _print_lock:
if tr.output:
print(tr.output)
scratchdir = os.path.join(scratchbase, tr.testbase)
if tr.result == 0:
passed += 1
if not args.preserve_scratch and os.path.isdir(scratchdir):
subprocess.run(['rm', '-rf', scratchdir], capture_output=True)
return False
elif tr.result == 77:
skipped_list.append(tr.testbase)
skipped += 1
if not args.preserve_scratch and os.path.isdir(scratchdir):
subprocess.run(['rm', '-rf', scratchdir], capture_output=True)
return False
elif tr.result == 78:
failed += 1
return True
else:
failed += 1
return True
if args.parallel > 1:
# Parallel execution
with concurrent.futures.ThreadPoolExecutor(max_workers=args.parallel) as executor:
futures = {}
for testscript in tests:
testbase = os.path.basename(testscript).replace('.test', '')
scratchdir = os.path.join(scratchbase, testbase)
timeout = 600 if 'hardlinks' in testbase else args.timeout
f = executor.submit(
run_one_test, testscript, testbase, scratchdir,
base_env, timeout, srcdir, tooldir, setfacl_nodef,
args.always_log
)
futures[f] = testbase
for f in concurrent.futures.as_completed(futures):
tr = f.result()
is_fail = process_result(tr)
if is_fail and args.stop_on_fail:
# Cancel pending futures
for pending in futures:
pending.cancel()
break
else:
# Sequential execution
for testscript in tests:
testbase = os.path.basename(testscript).replace('.test', '')
scratchdir = os.path.join(scratchbase, testbase)
timeout = 600 if 'hardlinks' in testbase else args.timeout
tr = run_one_test(
testscript, testbase, scratchdir,
base_env, timeout, srcdir, tooldir, setfacl_nodef,
args.always_log
)
is_fail = process_result(tr)
if is_fail and args.stop_on_fail:
break
# Check valgrind logs for errors
vg_errors = 0
if args.valgrind:
for vlog in sorted(glob.glob(os.path.join(scratchbase, 'valgrind.*.log'))):
try:
with open(vlog) as f:
content = f.read()
for line in content.splitlines():
if 'ERROR SUMMARY:' in line and 'ERROR SUMMARY: 0 errors' not in line:
vg_errors += 1
print(f'----- valgrind errors in {os.path.basename(vlog)}:')
print(content)
break
except FileNotFoundError:
pass
# Summary
print('-' * 60)
print('----- overall results:')
print(f' {passed} passed')
if failed > 0:
print(f' {failed} failed')
if skipped > 0:
print(f' {skipped} skipped')
if vg_errors > 0:
print(f' {vg_errors} valgrind error(s) found (see logs in {scratchbase})')
skipped_str = ','.join(sorted(skipped_list, key=lambda x: test_order.get(x, 0)))
if full_run and args.expect_skipped != 'IGNORE':
print('----- skipped results:')
print(f' expected: {args.expect_skipped}')
print(f' got: {skipped_str}')
else:
skipped_str = ''
args.expect_skipped = ''
print('-' * 60)
exit_code = failed + vg_errors
if exit_code == 0 and skipped_str != args.expect_skipped:
exit_code = 1
print(f'overall result is {exit_code}')
sys.exit(exit_code)
if __name__ == '__main__':
main()

View File

@@ -1,360 +0,0 @@
#! /bin/sh
# Copyright (C) 2001, 2002 by Martin Pool <mbp@samba.org>
# Copyright (C) 2003-2022 Wayne Davison
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version
# 2 as published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
# -------------------------------------------------------------------------
# rsync top-level test script -- this invokes all the other more
# detailed tests in order. This script can either be called by `make
# check' or `make installcheck'. `check' runs against the copies of
# the program and other files in the build directory, and
# `installcheck' against the installed copy of the program.
# It can also be called on a single test file using a run like this:
#
# preserve_scratch=yes whichtests=itemize.test ./runtests.sh
# In either case we need to also be able to find the source directory,
# since we read test scripts and possibly other information from
# there.
# Whenever possible, informational messages are written to stdout and
# error messages to stderr. They're separated out by the build farm
# display scripts.
# According to the GNU autoconf manual, the only valid place to set up
# directory locations is through Make, since users are allowed to (try
# to) change their mind on the Make command line. So, Make has to
# pass in all the values we need.
# For other configured settings we read ./config.sh, which tells us
# about shell commands on this machine and similar things.
# rsync_bin gives the location of the rsync binary. This is either
# builddir/rsync if we're testing an uninstalled copy, or
# install_prefix/bin/rsync if we're testing an installed copy. On the
# build farm rsync will be installed, but into a scratch /usr.
# srcdir gives the location of the source tree, which lets us find the
# build scripts. At the moment we assume we are invoked from the
# source directory.
# This script must be invoked from the build directory.
# A scratch directory, 'testtmp', is used in the build directory to
# hold per-test subdirectories.
# This script also uses the $loglevel environment variable. 1 is the
# default value, and 10 the most verbose. You can set this from the
# Make command line. It's also set by the build farm to give more
# detail for failing builds.
# -------------------------------------------------------------------------
# NOTES FOR TEST CASES:
# Each test case runs in its own shell.
# Exit codes from tests:
# 1 tests failed
# 2 error in starting tests
# 77 this test skipped (random value unlikely to happen by chance, same as
# automake)
# HOWEVER, the overall exit code to the farm is different: we return
# the *number of tests that failed*, so that it will show up nicely in
# the overall summary.
# rsync.fns contains some general setup functions and definitions.
# -------------------------------------------------------------------------
# NOTES ON PORTABILITY:
# Both this script and the Makefile have to be pretty conservative
# about which Unix features they use.
# We cannot count on Make exporting variables to commands, unless
# they're explicitly given on the command line.
# Also, we can't count on 'cp -a' or 'mkdir -p', although they're
# pretty handy (see function makepath for the latter).
# I think some of the GNU documentation suggests that we shouldn't
# rely on shell functions. However, the Bash manual seems to say that
# they're in POSIX 1003.2, and since the build farm relies on them
# they're probably working on most machines we really care about.
# You cannot use "function foo {" syntax, but must instead say "foo()
# {", or it breaks on FreeBSD.
# BSD machines tend not to have "head" or "seq".
# You cannot do "export VAR=VALUE" all on one line; the export must be
# separate from the assignment. (SCO SysV)
# Don't rely on grep -q, as that doesn't work everywhere -- just redirect
# stdout to /dev/null to keep it quiet.
# -------------------------------------------------------------------------
# STILL TO DO:
# We need a good protection against tests that hang indefinitely.
# Perhaps some combination of starting them in the background, wait,
# and kill?
# Perhaps we need a common way to cleanup tests. At the moment just
# clobbering the directory when we're done should be enough.
# If any of the targets fail, then (GNU?) Make returns 2, instead of
# the return code from the failing command. This is fine, but it
# means that the build farm just shows "2" for failed tests, not the
# number of tests that actually failed. For more details we might
# need to grovel through the log files to find a line saying how many
# failed.
set -e
. "./shconfig"
RUNSHFLAGS='-e'
export RUNSHFLAGS
# for Solaris
if [ -d /usr/xpg4/bin ]; then
PATH="/usr/xpg4/bin/:$PATH"
export PATH
fi
if [ "x$loglevel" != x ] && [ "$loglevel" -gt 8 ]; then
if set -x; then
# If it doesn't work the first time, don't keep trying.
RUNSHFLAGS="$RUNSHFLAGS -x"
fi
fi
POSIXLY_CORRECT=1
if test x"$TOOLDIR" = x; then
TOOLDIR=`pwd`
fi
srcdir=`dirname $0`
if test x"$srcdir" = x || test x"$srcdir" = x.; then
srcdir="$TOOLDIR"
fi
if test x"$rsync_bin" = x; then
rsync_bin="$TOOLDIR/rsync"
fi
# This allows the user to specify extra rsync options -- use carefully!
RSYNC="$rsync_bin $*"
#RSYNC="valgrind $rsync_bin $*"
TLS_ARGS=''
if grep -E '^#define HAVE_LUTIMES 1' config.h >/dev/null; then
TLS_ARGS="$TLS_ARGS -l"
fi
if grep -E '#undef CHOWN_MODIFIES_SYMLINK' config.h >/dev/null; then
TLS_ARGS="$TLS_ARGS -L"
fi
export POSIXLY_CORRECT TOOLDIR srcdir RSYNC TLS_ARGS
echo "============================================================"
echo "$0 running in $TOOLDIR"
echo " rsync_bin=$RSYNC"
echo " srcdir=$srcdir"
echo " TLS_ARGS=$TLS_ARGS"
if [ -f /usr/bin/whoami ]; then
testuser=`/usr/bin/whoami`
elif [ -f /usr/ucb/whoami ]; then
testuser=`/usr/ucb/whoami`
elif [ -f /bin/whoami ]; then
testuser=`/bin/whoami`
else
testuser=`id -un 2>/dev/null || echo ${LOGNAME:-${USERNAME:-${USER:-'UNKNOWN'}}}`
fi
echo " testuser=$testuser"
echo " os=`uname -a`"
# It must be "yes", not just nonnull
if [ "x$preserve_scratch" = xyes ]; then
echo " preserve_scratch=yes"
else
echo " preserve_scratch=no"
fi
# Check if setacl/setfacl is around and if it supports the -k or -s option.
if setacl -k u::7,g::5,o:5 testsuite 2>/dev/null; then
setfacl_nodef='setacl -k'
elif setfacl --help 2>&1 | grep ' -k,\|\[-[a-z]*k' >/dev/null; then
setfacl_nodef='setfacl -k'
elif setfacl -s u::7,g::5,o:5 testsuite 2>/dev/null; then
setfacl_nodef='setfacl -s u::7,g::5,o:5'
else
# The "true" command runs successfully, but does nothing.
setfacl_nodef=true
fi
export setfacl_nodef
if [ ! -f "$rsync_bin" ]; then
echo "rsync_bin $rsync_bin is not a file" >&2
exit 2
fi
if [ ! -d "$srcdir" ]; then
echo "srcdir $srcdir is not a directory" >&2
exit 2
fi
expect_skipped="${RSYNC_EXPECT_SKIPPED-IGNORE}"
skipped_list=''
skipped=0
missing=0
passed=0
failed=0
# Directory that holds the other test subdirs. We create separate dirs
# inside for each test case, so that they can be left behind in case of
# failure to aid investigation. We don't remove the testtmp subdir at
# the end so that it can be configured as a symlink to a filesystem that
# has ACLs and xattr support enabled (if desired).
scratchbase="${scratchbase:-$TOOLDIR}"/testtmp
echo " scratchbase=$scratchbase"
[ -d "$scratchbase" ] || mkdir "$scratchbase"
suitedir="$srcdir/testsuite"
TESTRUN_TIMEOUT=300
export scratchdir suitedir TESTRUN_TIMEOUT
prep_scratch() {
[ -d "$scratchdir" ] && chmod -R u+rwX "$scratchdir" && rm -rf "$scratchdir"
mkdir "$scratchdir"
# Get rid of default ACLs and dir-setgid to avoid confusing some tests.
$setfacl_nodef "$scratchdir" 2>/dev/null || true
chmod g-s "$scratchdir"
case "$srcdir" in
/*) ln -s "$srcdir" "$scratchdir/src" ;;
*) ln -s "$TOOLDIR/$srcdir" "$scratchdir/src" ;;
esac
return 0
}
maybe_discard_scratch() {
[ x"$preserve_scratch" != xyes ] && [ -d "$scratchdir" ] && rm -rf "$scratchdir"
return 0
}
if [ "x$whichtests" = x ]; then
whichtests="*.test"
full_run=yes
else
full_run=no
fi
for testscript in $suitedir/$whichtests; do
testbase=`echo $testscript | sed -e 's!.*/!!' -e 's/.test\$//'`
scratchdir="$scratchbase/$testbase"
prep_scratch
case "$testscript" in
*hardlinks*) TESTRUN_TIMEOUT=600 ;;
*) TESTRUN_TIMEOUT=300 ;;
esac
set +e
"$TOOLDIR/"testrun $RUNSHFLAGS "$testscript" >"$scratchdir/test.log" 2>&1
result=$?
set -e
if [ "x$always_log" = xyes ] || ( [ $result != 0 ] && [ $result != 77 ] && [ $result != 78 ] )
then
echo "----- $testbase log follows"
cat "$scratchdir/test.log"
echo "----- $testbase log ends"
if [ -f "$scratchdir/rsyncd.log" ]; then
echo "----- $testbase rsyncd.log follows"
cat "$scratchdir/rsyncd.log"
echo "----- $testbase rsyncd.log ends"
fi
fi
case $result in
0)
echo "PASS $testbase"
passed=`expr $passed + 1`
maybe_discard_scratch
;;
77)
# backticks will fill the whole file onto one line, which is a feature
whyskipped=`cat "$scratchdir/whyskipped"`
echo "SKIP $testbase ($whyskipped)"
skipped_list="$skipped_list,$testbase"
skipped=`expr $skipped + 1`
maybe_discard_scratch
;;
78)
# It failed, but we expected that. don't dump out error logs,
# because most users won't want to see them. But do leave
# the working directory around.
echo "XFAIL $testbase"
failed=`expr $failed + 1`
;;
*)
echo "FAIL $testbase"
failed=`expr $failed + 1`
if [ "x$nopersist" = xyes ]; then
exit 1
fi
esac
done
echo '------------------------------------------------------------'
echo "----- overall results:"
echo " $passed passed"
[ "$failed" -gt 0 ] && echo " $failed failed"
[ "$skipped" -gt 0 ] && echo " $skipped skipped"
[ "$missing" -gt 0 ] && echo " $missing missing"
if [ "$full_run" = yes ] && [ "$expect_skipped" != IGNORE ]; then
skipped_list=`echo "$skipped_list" | sed 's/^,//'`
echo "----- skipped results:"
echo " expected: $expect_skipped"
echo " got: $skipped_list"
else
skipped_list=''
expect_skipped=''
fi
echo '------------------------------------------------------------'
# OK, so expr exits with 0 if the result is neither null nor zero; and
# 1 if the expression is null or zero. This is the opposite of what
# we want, and if we just call expr then this script will always fail,
# because -e is set.
result=`expr $failed + $missing || true`
if [ "$result" = 0 ] && [ "$skipped_list" != "$expect_skipped" ]; then
result=1
fi
echo "overall result is $result"
exit $result

View File

@@ -262,6 +262,8 @@ void send_files(int f_in, int f_out)
if (ndx - cur_flist->ndx_start >= 0)
file = cur_flist->files[ndx - cur_flist->ndx_start];
else if (cur_flist->parent_ndx < 0)
exit_cleanup(RERR_PROTOCOL);
else
file = dir_flist->files[cur_flist->parent_ndx];
if (F_PATHNAME(file)) {

View File

@@ -347,8 +347,7 @@ __attribute__ ((target("avx2"))) MVSTATIC int32 get_checksum1_avx2_64(schar* buf
__m128i tmp = _mm_load_si128((__m128i*) mul_t1_buf);
__m256i mul_t1 = _mm256_cvtepu8_epi16(tmp);
__m256i mul_const = _mm256_broadcastd_epi32(_mm_cvtsi32_si128(4 | (3 << 8) | (2 << 16) | (1 << 24)));
__m256i mul_one;
mul_one = _mm256_abs_epi8(_mm256_cmpeq_epi16(mul_one,mul_one)); // set all vector elements to 1
__m256i mul_one = _mm256_set1_epi8(1);
for (; i < (len-64); i+=64) {
// Load ... 4*[int8*16]
@@ -548,6 +547,118 @@ int main() {
#pragma clang optimize on
#endif /* BENCHMARK_SIMD_CHECKSUM1 */
#ifdef TEST_SIMD_CHECKSUM1
static uint32 checksum_via_default(char *buf, int32 len)
{
uint32 s1 = 0, s2 = 0;
get_checksum1_default_1((schar*)buf, len, 0, &s1, &s2);
return (s1 & 0xffff) + (s2 << 16);
}
static uint32 checksum_via_sse2(char *buf, int32 len)
{
int32 i;
uint32 s1 = 0, s2 = 0;
i = get_checksum1_sse2_32((schar*)buf, len, 0, &s1, &s2);
get_checksum1_default_1((schar*)buf, len, i, &s1, &s2);
return (s1 & 0xffff) + (s2 << 16);
}
static uint32 checksum_via_ssse3(char *buf, int32 len)
{
int32 i;
uint32 s1 = 0, s2 = 0;
i = get_checksum1_ssse3_32((schar*)buf, len, 0, &s1, &s2);
get_checksum1_default_1((schar*)buf, len, i, &s1, &s2);
return (s1 & 0xffff) + (s2 << 16);
}
static uint32 checksum_via_avx2(char *buf, int32 len)
{
int32 i;
uint32 s1 = 0, s2 = 0;
#ifdef USE_ROLL_ASM
i = get_checksum1_avx2_asm((schar*)buf, len, 0, &s1, &s2);
#else
i = get_checksum1_avx2_64((schar*)buf, len, 0, &s1, &s2);
#endif
get_checksum1_default_1((schar*)buf, len, i, &s1, &s2);
return (s1 & 0xffff) + (s2 << 16);
}
int main()
{
static const int sizes[] = {1, 4, 31, 32, 33, 63, 64, 65, 128, 129, 256, 700, 1024, 4096, 65536};
int num_sizes = sizeof(sizes) / sizeof(sizes[0]);
int max_size = sizes[num_sizes - 1];
int failures = 0;
/* Allocate with extra bytes for unaligned test */
unsigned char *raw = (unsigned char *)malloc(max_size + 64 + 1);
if (!raw) {
fprintf(stderr, "malloc failed\n");
return 1;
}
/* Fill with deterministic data */
for (int i = 0; i < max_size + 64 + 1; i++)
raw[i] = (i + (i % 3) + (i % 11)) % 256;
/* Test with aligned buffer (64-byte aligned) */
unsigned char *aligned = raw + (64 - ((uintptr_t)raw % 64));
/* Test with unaligned buffer (+1 byte offset) */
unsigned char *unaligned = aligned + 1;
struct { const char *name; unsigned char *buf; } buffers[] = {
{"aligned", aligned},
{"unaligned", unaligned},
};
for (int b = 0; b < 2; b++) {
char *buf = (char *)buffers[b].buf;
const char *bname = buffers[b].name;
for (int s = 0; s < num_sizes; s++) {
int32 len = sizes[s];
uint32 ref = checksum_via_default(buf, len);
uint32 cs_sse2 = checksum_via_sse2(buf, len);
uint32 cs_ssse3 = checksum_via_ssse3(buf, len);
uint32 cs_avx2 = checksum_via_avx2(buf, len);
uint32 cs_auto = get_checksum1(buf, len);
if (cs_sse2 != ref) {
printf("FAIL %-9s size=%5d: SSE2=%08x ref=%08x\n", bname, len, cs_sse2, ref);
failures++;
}
if (cs_ssse3 != ref) {
printf("FAIL %-9s size=%5d: SSSE3=%08x ref=%08x\n", bname, len, cs_ssse3, ref);
failures++;
}
if (cs_avx2 != ref) {
printf("FAIL %-9s size=%5d: AVX2=%08x ref=%08x\n", bname, len, cs_avx2, ref);
failures++;
}
if (cs_auto != ref) {
printf("FAIL %-9s size=%5d: auto=%08x ref=%08x\n", bname, len, cs_auto, ref);
failures++;
}
}
}
free(raw);
if (failures) {
printf("%d checksum mismatches!\n", failures);
return 1;
}
printf("All SIMD checksum tests passed.\n");
return 0;
}
#endif /* TEST_SIMD_CHECKSUM1 */
#endif /* } USE_ROLL_SIMD */
#endif /* } __cplusplus */
#endif /* } __x86_64__ */

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python3
import os, re, argparse, subprocess
from datetime import datetime
from datetime import datetime, UTC
NULL_COMMIT_RE = re.compile(r'\0\0commit [a-f0-9]{40}$|\0$')
@@ -74,7 +74,7 @@ def print_line(fn, mtime, commit_time):
if args.list > 1:
ts = str(commit_time).rjust(10)
else:
ts = datetime.utcfromtimestamp(commit_time).strftime("%Y-%m-%d %H:%M:%S")
ts = datetime.fromtimestamp(commit_time, UTC).strftime("%Y-%m-%d %H:%M:%S")
chg = '.' if mtime == commit_time else '*'
print(chg, ts, fn)

View File

@@ -46,6 +46,7 @@ long_opts = {
'compare-dest': 2,
'compress-choice': 1,
'compress-level': 1,
'compress-threads': 1,
'copy-dest': 2,
'copy-devices': -1,
'copy-unsafe-links': 0,
@@ -59,6 +60,7 @@ long_opts = {
'delete-during': 0,
'delete-excluded': 0,
'delete-missing-args': 0,
'dirs': 0,
'existing': 0,
'fake-super': 0,
'files-from': 3,
@@ -300,6 +302,7 @@ def validated_arg(opt, arg, typ=3, wild=False):
if arg.startswith('./'):
arg = arg[1:]
arg = arg.replace('//', '/')
arg = arg.lstrip('/')
if args.dir != '/':
if HAS_DOT_DOT_RE.search(arg):
die("do not use .. in", opt, "(anchor the path at the root of your restricted dir)")

View File

@@ -2,7 +2,7 @@
REAL_RSYNC=/usr/bin/rsync
IGNOREEXIT=24
IGNOREOUT='^(file has vanished: |rsync warning: some files vanished before they could be transferred)'
IGNOREOUT='^((file|directory) has vanished: |rsync warning: some files vanished before they could be transferred)'
# If someone installs this as "rsync", make sure we don't affect a server run.
for arg in "${@}"; do

View File

@@ -480,7 +480,7 @@ int do_SetFileTime(const char *path, time_t crtime)
free(pathw);
if (handle == INVALID_HANDLE_VALUE)
return -1;
int64 temp_time = Int32x32To64(crtime, 10000000) + 116444736000000000LL;
int64 temp_time = (crtime * 10000000LL) + 116444736000000000LL;
FILETIME birth_time;
birth_time.dwLowDateTime = (DWORD)temp_time;
birth_time.dwHighDateTime = (DWORD)(temp_time >> 32);
@@ -683,6 +683,11 @@ int do_open_nofollow(const char *pathname, int flags)
#endif
}
#ifdef O_NOATIME
if (open_noatime)
flags |= O_NOATIME;
#endif
#ifdef O_NOFOLLOW
fd = open(pathname, flags|O_NOFOLLOW);
#else

View File

@@ -0,0 +1,27 @@
#!/bin/sh
# clean-fname-underflow.test
# Ensure clean_fname() does not read-before-buffer when collapsing "..".
# This exercises the --server path where a crafted merge filename hits clean_fname().
. "$suitedir/rsync.fns"
workdir="$scratchdir/workdir"
mkdir -p "$workdir/mod"
cd "$workdir"
rsync_bin=`echo $RSYNC | sed 's/ .*//'`
# Invoke the server-side path. We don't need a real transfer; we just want to
# ensure clean_fname() doesn't crash when given "a/../test" via --filter=merge.
if $rsync_bin --server --sender -vlr --filter='merge a/../test' . mod/ >/dev/null 2>&1; then
: # success
else
status=$?
# Non-zero exit is expected for bogus input; ensure it wasn't a signal/crash.
if [ $status -ge 128 ]; then
test_fail "rsync exited due to a signal (status=$status)"
fi
fi
echo "OK: clean_fname() handled 'a/../test' without crashing"
exit 0

View File

@@ -81,7 +81,8 @@ diff $diffopt "$name1" "$todir" || test_fail "solo copy of name1 failed"
# enabled (this has broken in 3.4.0 so far, so we need this test).
rm -rf "$fromdir" "$todir"
makepath "$fromdir/sym" "$todir"
checkit "$RSYNC -aH '$fromdir/sym' '$todir'" "$fromdir" "$todir"
$RSYNC -aH "$fromdir/sym" "$todir"
diff $diffopt "$fromdir" "$todir" || test_fail "solo copy of sym failed"
# The script would have aborted on error, so getting here means we've won.
exit 0

View File

@@ -16,9 +16,9 @@ makepath "$longdir" || test_skipped "unable to create long directory"
touch "$longdir/1" || test_skipped "unable to create files in long directory"
date > "$longdir/1"
if [ -r /etc ]; then
ls -la /etc >"$longdir/2"
ls -la /etc >"$longdir/2" || [ $? -eq 1 ]
else
ls -la / >"$longdir/2"
ls -la / >"$longdir/2" || [ $? -eq 1 ]
fi
checkit "$RSYNC --delete -avH '$fromdir/' '$todir'" "$fromdir/" "$todir"

View File

@@ -0,0 +1,32 @@
#!/bin/sh
# Test rsync --open-noatime option keeps source atimes intact
. "$suitedir/rsync.fns"
$RSYNC -VV | grep '"atimes": true' >/dev/null || test_skipped "Rsync is configured without atimes support"
# O_NOATIME is Linux-specific; skip on other platforms
case `uname` in
Linux) ;;
*) test_skipped "O_NOATIME is only supported on Linux" ;;
esac
mkdir "$fromdir"
# --open-noatime did not work properly on files with size > 0
echo content > "$fromdir/foo"
touch -a -t 200102031717.42 "$fromdir/foo"
TLS_ARGS=--atimes
"$TOOLDIR/tls" $TLS_ARGS "$fromdir/foo" > "$tmpdir/atime-from-before"
# Do not use checkit because it uses "diff" which breaks atimes
$RSYNC --open-noatime --archive --recursive --times --atimes -vvv "$fromdir/" "$todir/"
"$TOOLDIR/tls" $TLS_ARGS "$fromdir/foo" > "$tmpdir/atime-from-after"
diff "$tmpdir/atime-from-before" "$tmpdir/atime-from-after"
# The script would have aborted on error, so getting here means we've won.
exit 0

View File

@@ -97,7 +97,7 @@ printmsg() {
}
rsync_ls_lR() {
find "$@" -name .git -prune -o -name auto-build-save -prune -o -print | \
find "$@" -name .git -prune -o -name auto-build-save -prune -o -name testtmp -prune -o -print | \
sort | sed 's/ /\\ /g' | xargs "$TOOLDIR/tls" $TLS_ARGS
}
@@ -195,15 +195,15 @@ hands_setup() {
echo some data > "$fromdir/dir/subdir/foobar.baz"
mkdir "$fromdir/dir/subdir/subsubdir"
if [ -r /etc ]; then
ls -ltr /etc > "$fromdir/dir/subdir/subsubdir/etc-ltr-list"
ls -ltr /etc > "$fromdir/dir/subdir/subsubdir/etc-ltr-list" || [ $? -eq 1 ]
else
ls -ltr / > "$fromdir/dir/subdir/subsubdir/etc-ltr-list"
ls -ltr / > "$fromdir/dir/subdir/subsubdir/etc-ltr-list" || [ $? -eq 1 ]
fi
mkdir "$fromdir/dir/subdir/subsubdir2"
if [ -r /bin ]; then
ls -lt /bin > "$fromdir/dir/subdir/subsubdir2/bin-lt-list"
ls -lt /bin > "$fromdir/dir/subdir/subsubdir2/bin-lt-list" || [ $? -eq 1 ]
else
ls -lt / > "$fromdir/dir/subdir/subsubdir2/bin-lt-list"
ls -lt / > "$fromdir/dir/subdir/subsubdir2/bin-lt-list" || [ $? -eq 1 ]
fi
# echo testing head:

11
testsuite/simd-checksum.test Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/sh
# Test SIMD checksum implementations against the C reference
. "$suitedir/rsync.fns"
if ! test -x "$TOOLDIR/simdtest"; then
test_skipped "simdtest not built (SIMD not available)"
fi
"$TOOLDIR/simdtest"

View File

@@ -6,7 +6,7 @@
# This program is distributable under the terms of the GNU GPL (see
# COPYING)
# This script tests ssh, if possible. It's called by runtests.sh
# This script tests ssh, if possible. It's called by runtests.py
. "$suitedir/rsync.fns"

4
tls.c
View File

@@ -127,7 +127,7 @@ static void storetime(char *dest, size_t destsize, time_t t, int nsecs)
{
if (t) {
int len;
struct tm *mt = gmtime(&t);
struct tm tmp, *mt = gmtime_r(&t, &tmp);
len = snprintf(dest, destsize,
" %04d-%02d-%02d %02d:%02d:%02d",
@@ -230,7 +230,7 @@ static void list_file(const char *fname)
mtimebuf, atimebuf, crtimebuf, fname, linkbuf);
}
static struct poptOption long_options[] = {
static const struct poptOption long_options[] = {
/* longName, shortName, argInfo, argPtr, value, descrip, argDesc */
{"atimes", 'U', POPT_ARG_NONE, &display_atimes, 0, 0, 0},
#ifdef SUPPORT_CRTIMES

42
token.c
View File

@@ -33,6 +33,7 @@ extern int do_compression;
extern int protocol_version;
extern int module_id;
extern int do_compression_level;
extern int do_compression_threads;
extern char *skip_compress;
#ifndef Z_INSERT_ONLY
@@ -589,8 +590,13 @@ static int32 recv_deflated_token(int f, char **data)
if (flag & TOKEN_REL) {
rx_token += flag & 0x3f;
flag >>= 6;
} else
} else {
rx_token = read_int(f);
if (rx_token < 0) {
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;
@@ -692,6 +698,8 @@ static void send_zstd_token(int f, int32 token, struct map_struct *buf, OFF_T of
obuf = new_array(char, OBUF_SIZE);
ZSTD_CCtx_setParameter(zstd_cctx, ZSTD_c_compressionLevel, do_compression_level);
ZSTD_CCtx_setParameter(zstd_cctx, ZSTD_c_nbWorkers, do_compression_threads);
zstd_out_buff.dst = obuf + 2;
comp_init_done = 1;
@@ -729,12 +737,11 @@ static void send_zstd_token(int f, int32 token, struct map_struct *buf, OFF_T of
zstd_in_buff.src = map_ptr(buf, offset, nb);
zstd_in_buff.size = nb;
zstd_in_buff.pos = 0;
int finished;
do {
if (zstd_out_buff.size == 0) {
zstd_out_buff.size = MAX_DATA_COUNT;
zstd_out_buff.pos = 0;
}
zstd_out_buff.size = MAX_DATA_COUNT;
zstd_out_buff.pos = 0;
/* File ended, flush */
if (token != -2)
@@ -752,20 +759,21 @@ static void send_zstd_token(int f, int32 token, struct map_struct *buf, OFF_T of
* state and send a smaller buffer so that the remote side can
* finish the file.
*/
if (zstd_out_buff.pos == zstd_out_buff.size || flush == ZSTD_e_flush) {
finished = (flush == ZSTD_e_flush) ? (r == 0) : (zstd_in_buff.pos == zstd_in_buff.size);
if (zstd_out_buff.pos != 0) {
n = zstd_out_buff.pos;
obuf[0] = DEFLATED_DATA + (n >> 8);
obuf[1] = n;
write_buf(f, obuf, n+2);
zstd_out_buff.size = 0;
}
/*
* Loop while the input buffer isn't full consumed or the
* internal state isn't fully flushed.
*/
} while (zstd_in_buff.pos < zstd_in_buff.size || r > 0);
} while (!finished);
flush_pending = token == -2;
}
@@ -831,8 +839,13 @@ static int32 recv_zstd_token(int f, char **data)
if (flag & TOKEN_REL) {
rx_token += flag & 0x3f;
flag >>= 6;
} else
} else {
rx_token = read_int(f);
if (rx_token < 0) {
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;
@@ -995,8 +1008,13 @@ static int32 recv_compressed_token(int f, char **data)
if (flag & TOKEN_REL) {
rx_token += flag & 0x3f;
flag >>= 6;
} else
} else {
rx_token = read_int(f);
if (rx_token < 0) {
rprintf(FERROR, "invalid token number in compressed stream\n");
exit_cleanup(RERR_PROTOCOL);
}
}
if (flag & 1) {
rx_run = read_byte(f);
rx_run += read_byte(f) << 8;

16
util1.c
View File

@@ -942,7 +942,7 @@ int count_dir_elements(const char *p)
* resulting name would be empty, returns ".". */
int clean_fname(char *name, int flags)
{
char *limit = name - 1, *t = name, *f = name;
char *limit = name, *t = name, *f = name;
int anchored;
if (!name)
@@ -987,9 +987,13 @@ int clean_fname(char *name, int flags)
f += 2;
continue;
}
while (s > limit && *--s != '/') {}
if (s != t - 1 && (s < name || *s == '/')) {
t = s + 1;
/* backing up for ".." — avoid reading before 'name' */
while (s > limit && s[-1] != '/')
s--;
/* If found prior '/', or we reached the start, adjust t. */
if (s != t - 1 && (s <= name || *s == '/')) {
t = (s == name) ? name : s + 1;
f += 2;
continue;
}
@@ -1389,7 +1393,7 @@ char *timestring(time_t t)
static int ndx = 0;
static char buffers[4][20]; /* We support 4 simultaneous timestring results. */
char *TimeBuf = buffers[ndx = (ndx + 1) % 4];
struct tm *tm = localtime(&t);
struct tm tmp, *tm = localtime_r(&t, &tmp);
int len = snprintf(TimeBuf, sizeof buffers[0], "%4d/%02d/%02d %02d:%02d:%02d",
(int)tm->tm_year + 1900, (int)tm->tm_mon + 1, (int)tm->tm_mday,
(int)tm->tm_hour, (int)tm->tm_min, (int)tm->tm_sec);
@@ -1714,6 +1718,8 @@ void *expand_item_list(item_list *lp, size_t item_size, const char *desc, int in
new_ptr == lp->items ? " not" : "");
}
memset((char *)new_ptr + lp->malloced * item_size, 0,
(expand_size - lp->malloced) * item_size);
lp->items = new_ptr;
lp->malloced = expand_size;
}

View File

@@ -79,9 +79,7 @@ void *my_alloc(void *ptr, size_t num, size_t size, const char *file, int line)
who_am_i(), do_big_num(max_alloc, 0, NULL), src_file(file), line);
exit_cleanup(RERR_MALLOC);
}
if (!ptr)
ptr = malloc(num * size);
else if (ptr == do_calloc)
if (!ptr || ptr == do_calloc)
ptr = calloc(num, size);
else
ptr = realloc(ptr, num * size);

View File

@@ -1,2 +1,2 @@
#define RSYNC_VERSION "3.4.1"
#define MAINTAINER_TZ_OFFSET -7.0
#define RSYNC_VERSION "3.4.2"
#define MAINTAINER_TZ_OFFSET 10.0

View File

@@ -32,7 +32,9 @@ int fnmatch_errors = 0;
int wildmatch_errors = 0;
#if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 202311L
typedef char bool;
#endif
int output_iterations = 0;
int explode_mod = 0;
@@ -40,7 +42,7 @@ int empties_mod = 0;
int empty_at_start = 0;
int empty_at_end = 0;
static struct poptOption long_options[] = {
static const struct poptOption long_options[] = {
/* longName, shortName, argInfo, argPtr, value, descrip, argDesc */
{"iterations", 'i', POPT_ARG_NONE, &output_iterations, 0, 0, 0},
{"empties", 'e', POPT_ARG_STRING, 0, 'e', 0, 0},

View File

@@ -860,8 +860,8 @@ void receive_xattr(int f, struct file_struct *file)
rxa->num = num;
}
if (need_sort && count > 1)
qsort(temp_xattr.items, count, sizeof (rsync_xa), rsync_xal_compare_names);
if (need_sort && temp_xattr.count > 1)
qsort(temp_xattr.items, temp_xattr.count, sizeof (rsync_xa), rsync_xal_compare_names);
ndx = rsync_xal_store(&temp_xattr); /* adds item to rsync_xal_l */

View File

@@ -62,10 +62,7 @@ local uLong adler32_combine_ OF((uLong adler1, uLong adler2, z_off64_t len2));
#endif
/* ========================================================================= */
uLong ZEXPORT adler32(adler, buf, len)
uLong adler;
const Bytef *buf;
uInt len;
uLong ZEXPORT adler32(uLong adler, const Bytef *buf, uInt len)
{
unsigned long sum2;
unsigned n;
@@ -133,10 +130,7 @@ uLong ZEXPORT adler32(adler, buf, len)
}
/* ========================================================================= */
local uLong adler32_combine_(adler1, adler2, len2)
uLong adler1;
uLong adler2;
z_off64_t len2;
local uLong adler32_combine_(uLong adler1, uLong adler2, z_off64_t len2)
{
unsigned long sum1;
unsigned long sum2;
@@ -162,18 +156,12 @@ local uLong adler32_combine_(adler1, adler2, len2)
}
/* ========================================================================= */
uLong ZEXPORT adler32_combine(adler1, adler2, len2)
uLong adler1;
uLong adler2;
z_off_t len2;
uLong ZEXPORT adler32_combine(uLong adler1, uLong adler2, z_off_t len2)
{
return adler32_combine_(adler1, adler2, len2);
}
uLong ZEXPORT adler32_combine64(adler1, adler2, len2)
uLong adler1;
uLong adler2;
z_off64_t len2;
uLong ZEXPORT adler32_combine64(uLong adler1, uLong adler2, z_off64_t len2)
{
return adler32_combine_(adler1, adler2, len2);
}

View File

@@ -19,12 +19,7 @@
memory, Z_BUF_ERROR if there was not enough room in the output buffer,
Z_STREAM_ERROR if the level parameter is invalid.
*/
int ZEXPORT compress2 (dest, destLen, source, sourceLen, level)
Bytef *dest;
uLongf *destLen;
const Bytef *source;
uLong sourceLen;
int level;
int ZEXPORT compress2 (Bytef *dest, uLongf *destLen, const Bytef *source, uLong sourceLen, int level)
{
z_stream stream;
int err;
@@ -59,11 +54,7 @@ int ZEXPORT compress2 (dest, destLen, source, sourceLen, level)
/* ===========================================================================
*/
int ZEXPORT compress (dest, destLen, source, sourceLen)
Bytef *dest;
uLongf *destLen;
const Bytef *source;
uLong sourceLen;
int ZEXPORT compress (Bytef *dest, uLongf *destLen, const Bytef *source, uLong sourceLen)
{
return compress2(dest, destLen, source, sourceLen, Z_DEFAULT_COMPRESSION);
}
@@ -72,8 +63,7 @@ int ZEXPORT compress (dest, destLen, source, sourceLen)
If the default memLevel or windowBits for deflateInit() is changed, then
this function needs to be updated.
*/
uLong ZEXPORT compressBound (sourceLen)
uLong sourceLen;
uLong ZEXPORT compressBound (uLong sourceLen)
{
return sourceLen + (sourceLen >> 12) + (sourceLen >> 14) +
(sourceLen >> 25) + 13;

View File

@@ -87,7 +87,7 @@ local void make_crc_table OF((void));
allow for word-at-a-time CRC calculation for both big-endian and little-
endian machines, where a word is four bytes.
*/
local void make_crc_table()
local void make_crc_table(void)
{
z_crc_t c;
int n, k;
@@ -164,9 +164,7 @@ local void make_crc_table()
}
#ifdef MAKECRCH
local void write_table(out, table)
FILE *out;
const z_crc_t FAR *table;
local void write_table(FILE *out, const z_crc_t FAR *table)
{
int n;
@@ -187,7 +185,7 @@ local void write_table(out, table)
/* =========================================================================
* This function can be used by asm versions of crc32()
*/
const z_crc_t FAR * ZEXPORT get_crc_table()
const z_crc_t FAR * ZEXPORT get_crc_table(void)
{
#ifdef DYNAMIC_CRC_TABLE
if (crc_table_empty)
@@ -201,10 +199,7 @@ const z_crc_t FAR * ZEXPORT get_crc_table()
#define DO8 DO1; DO1; DO1; DO1; DO1; DO1; DO1; DO1
/* ========================================================================= */
unsigned long ZEXPORT crc32(crc, buf, len)
unsigned long crc;
const unsigned char FAR *buf;
uInt len;
unsigned long ZEXPORT crc32(unsigned long crc, const unsigned char FAR *buf, uInt len)
{
if (buf == Z_NULL) return 0UL;
@@ -244,10 +239,7 @@ unsigned long ZEXPORT crc32(crc, buf, len)
#define DOLIT32 DOLIT4; DOLIT4; DOLIT4; DOLIT4; DOLIT4; DOLIT4; DOLIT4; DOLIT4
/* ========================================================================= */
local unsigned long crc32_little(crc, buf, len)
unsigned long crc;
const unsigned char FAR *buf;
unsigned len;
local unsigned long crc32_little(unsigned long crc, const unsigned char FAR *buf, unsigned len)
{
register z_crc_t c;
register const z_crc_t FAR *buf4;
@@ -284,10 +276,7 @@ local unsigned long crc32_little(crc, buf, len)
#define DOBIG32 DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4
/* ========================================================================= */
local unsigned long crc32_big(crc, buf, len)
unsigned long crc;
const unsigned char FAR *buf;
unsigned len;
local unsigned long crc32_big(unsigned long crc, const unsigned char FAR *buf, unsigned len)
{
register z_crc_t c;
register const z_crc_t FAR *buf4;
@@ -322,9 +311,7 @@ local unsigned long crc32_big(crc, buf, len)
#define GF2_DIM 32 /* dimension of GF(2) vectors (length of CRC) */
/* ========================================================================= */
local unsigned long gf2_matrix_times(mat, vec)
unsigned long *mat;
unsigned long vec;
local unsigned long gf2_matrix_times(unsigned long *mat, unsigned long vec)
{
unsigned long sum;
@@ -339,9 +326,7 @@ local unsigned long gf2_matrix_times(mat, vec)
}
/* ========================================================================= */
local void gf2_matrix_square(square, mat)
unsigned long *square;
unsigned long *mat;
local void gf2_matrix_square(unsigned long *square, unsigned long *mat)
{
int n;
@@ -350,10 +335,7 @@ local void gf2_matrix_square(square, mat)
}
/* ========================================================================= */
local uLong crc32_combine_(crc1, crc2, len2)
uLong crc1;
uLong crc2;
z_off64_t len2;
local uLong crc32_combine_(uLong crc1, uLong crc2, z_off64_t len2)
{
int n;
unsigned long row;
@@ -406,18 +388,12 @@ local uLong crc32_combine_(crc1, crc2, len2)
}
/* ========================================================================= */
uLong ZEXPORT crc32_combine(crc1, crc2, len2)
uLong crc1;
uLong crc2;
z_off_t len2;
uLong ZEXPORT crc32_combine(uLong crc1, uLong crc2, z_off_t len2)
{
return crc32_combine_(crc1, crc2, len2);
}
uLong ZEXPORT crc32_combine64(crc1, crc2, len2)
uLong crc1;
uLong crc2;
z_off64_t len2;
uLong ZEXPORT crc32_combine64(uLong crc1, uLong crc2, z_off64_t len2)
{
return crc32_combine_(crc1, crc2, len2);
}

View File

@@ -200,11 +200,8 @@ struct static_tree_desc_s {int dummy;}; /* for buggy compilers */
zmemzero((Bytef *)s->head, (unsigned)(s->hash_size-1)*sizeof(*s->head));
/* ========================================================================= */
int ZEXPORT deflateInit_(strm, level, version, stream_size)
z_streamp strm;
int level;
const char *version;
int stream_size;
int ZEXPORT deflateInit_(z_streamp strm, int level, const char *version,
int stream_size)
{
return deflateInit2_(strm, level, Z_DEFLATED, MAX_WBITS, DEF_MEM_LEVEL,
Z_DEFAULT_STRATEGY, version, stream_size);
@@ -212,16 +209,8 @@ int ZEXPORT deflateInit_(strm, level, version, stream_size)
}
/* ========================================================================= */
int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
version, stream_size)
z_streamp strm;
int level;
int method;
int windowBits;
int memLevel;
int strategy;
const char *version;
int stream_size;
int ZEXPORT deflateInit2_(z_streamp strm, int level, int method, int windowBits,
int memLevel, int strategy, const char *version, int stream_size)
{
deflate_state *s;
int wrap = 1;
@@ -359,10 +348,8 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
}
/* ========================================================================= */
int ZEXPORT deflateSetDictionary (strm, dictionary, dictLength)
z_streamp strm;
const Bytef *dictionary;
uInt dictLength;
int ZEXPORT deflateSetDictionary (z_streamp strm, const Bytef *dictionary,
uInt dictLength)
{
deflate_state *s;
uInt str, n;
@@ -428,8 +415,7 @@ int ZEXPORT deflateSetDictionary (strm, dictionary, dictLength)
}
/* ========================================================================= */
int ZEXPORT deflateResetKeep (strm)
z_streamp strm;
int ZEXPORT deflateResetKeep (z_streamp strm)
{
deflate_state *s;
@@ -463,8 +449,7 @@ int ZEXPORT deflateResetKeep (strm)
}
/* ========================================================================= */
int ZEXPORT deflateReset (strm)
z_streamp strm;
int ZEXPORT deflateReset (z_streamp strm)
{
int ret;
@@ -475,9 +460,7 @@ int ZEXPORT deflateReset (strm)
}
/* ========================================================================= */
int ZEXPORT deflateSetHeader (strm, head)
z_streamp strm;
gz_headerp head;
int ZEXPORT deflateSetHeader (z_streamp strm, gz_headerp head)
{
if (strm == Z_NULL || strm->state == Z_NULL) return Z_STREAM_ERROR;
if (strm->state->wrap != 2) return Z_STREAM_ERROR;
@@ -486,10 +469,7 @@ int ZEXPORT deflateSetHeader (strm, head)
}
/* ========================================================================= */
int ZEXPORT deflatePending (strm, pending, bits)
unsigned *pending;
int *bits;
z_streamp strm;
int ZEXPORT deflatePending (z_streamp strm, unsigned *pending, int *bits)
{
if (strm == Z_NULL || strm->state == Z_NULL) return Z_STREAM_ERROR;
if (pending != Z_NULL)
@@ -500,10 +480,7 @@ int ZEXPORT deflatePending (strm, pending, bits)
}
/* ========================================================================= */
int ZEXPORT deflatePrime (strm, bits, value)
z_streamp strm;
int bits;
int value;
int ZEXPORT deflatePrime (z_streamp strm, int bits, int value)
{
deflate_state *s;
int put;
@@ -526,10 +503,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
}
/* ========================================================================= */
int ZEXPORT deflateParams(strm, level, strategy)
z_streamp strm;
int level;
int strategy;
int ZEXPORT deflateParams(z_streamp strm, int level, int strategy)
{
deflate_state *s;
compress_func func;
@@ -567,12 +541,8 @@ int ZEXPORT deflateParams(strm, level, strategy)
}
/* ========================================================================= */
int ZEXPORT deflateTune(strm, good_length, max_lazy, nice_length, max_chain)
z_streamp strm;
int good_length;
int max_lazy;
int nice_length;
int max_chain;
int ZEXPORT deflateTune(z_streamp strm, int good_length, int max_lazy,
int nice_length, int max_chain)
{
deflate_state *s;
@@ -602,9 +572,7 @@ int ZEXPORT deflateTune(strm, good_length, max_lazy, nice_length, max_chain)
* upper bound of about 14% expansion does not seem onerous for output buffer
* allocation.
*/
uLong ZEXPORT deflateBound(strm, sourceLen)
z_streamp strm;
uLong sourceLen;
uLong ZEXPORT deflateBound(z_streamp strm, uLong sourceLen)
{
deflate_state *s;
uLong complen, wraplen;
@@ -664,9 +632,7 @@ uLong ZEXPORT deflateBound(strm, sourceLen)
* IN assertion: the stream state is correct and there is enough room in
* pending_buf.
*/
local void putShortMSB (s, b)
deflate_state *s;
uInt b;
local void putShortMSB (deflate_state *s, uInt b)
{
put_byte(s, (Byte)(b >> 8));
put_byte(s, (Byte)(b & 0xff));
@@ -678,8 +644,7 @@ local void putShortMSB (s, b)
* to avoid allocating a large strm->next_out buffer and copying into it.
* (See also read_buf()).
*/
local void flush_pending(strm)
z_streamp strm;
local void flush_pending(z_streamp strm)
{
unsigned len;
deflate_state *s = strm->state;
@@ -701,9 +666,7 @@ local void flush_pending(strm)
}
/* ========================================================================= */
int ZEXPORT deflate (strm, flush)
z_streamp strm;
int flush;
int ZEXPORT deflate (z_streamp strm, int flush)
{
int old_flush; /* value of flush param for previous deflate call */
deflate_state *s;
@@ -1015,8 +978,7 @@ int ZEXPORT deflate (strm, flush)
}
/* ========================================================================= */
int ZEXPORT deflateEnd (strm)
z_streamp strm;
int ZEXPORT deflateEnd (z_streamp strm)
{
int status;
@@ -1050,9 +1012,7 @@ int ZEXPORT deflateEnd (strm)
* To simplify the source, this is not supported for 16-bit MSDOS (which
* doesn't have enough memory anyway to duplicate compression states).
*/
int ZEXPORT deflateCopy (dest, source)
z_streamp dest;
z_streamp source;
int ZEXPORT deflateCopy (z_streamp dest, z_streamp source)
{
#ifdef MAXSEG_64K
return Z_STREAM_ERROR;
@@ -1109,10 +1069,7 @@ int ZEXPORT deflateCopy (dest, source)
* allocating a large strm->next_in buffer and copying from it.
* (See also flush_pending()).
*/
local int read_buf(strm, buf, size)
z_streamp strm;
Bytef *buf;
unsigned size;
local int read_buf(z_streamp strm, Bytef *buf, unsigned size)
{
unsigned len = strm->avail_in;
@@ -1139,8 +1096,7 @@ local int read_buf(strm, buf, size)
/* ===========================================================================
* Initialize the "longest match" routines for a new zlib stream
*/
local void lm_init (s)
deflate_state *s;
local void lm_init (deflate_state *s)
{
s->window_size = (ulg)2L*s->w_size;
@@ -1181,9 +1137,7 @@ local void lm_init (s)
/* For 80x86 and 680x0, an optimized version will be provided in match.asm or
* match.S. The code will be functionally equivalent.
*/
local uInt longest_match(s, cur_match)
deflate_state *s;
IPos cur_match; /* current match */
local uInt longest_match(deflate_state *s, IPos cur_match)
{
unsigned chain_length = s->max_chain_length;/* max hash chain length */
register Bytef *scan = s->window + s->strstart; /* current string */
@@ -1330,9 +1284,7 @@ local uInt longest_match(s, cur_match)
/* ---------------------------------------------------------------------------
* Optimized version for FASTEST only
*/
local uInt longest_match(s, cur_match)
deflate_state *s;
IPos cur_match; /* current match */
local uInt longest_match(deflate_state *s, IPos cur_match)
{
register Bytef *scan = s->window + s->strstart; /* current string */
register Bytef *match; /* matched string */
@@ -1389,10 +1341,7 @@ local uInt longest_match(s, cur_match)
/* ===========================================================================
* Check that the match at match_start is indeed a match.
*/
local void check_match(s, start, match, length)
deflate_state *s;
IPos start, match;
int length;
local void check_match(deflate_state *s, IPos start, IPos match, int length)
{
/* check that the match is indeed a match */
if (zmemcmp(s->window + match,
@@ -1423,8 +1372,7 @@ local void check_match(s, start, match, length)
* performed for at least two bytes (required for the zip translate_eol
* option -- not supported here).
*/
local void fill_window(s)
deflate_state *s;
local void fill_window(deflate_state *s)
{
register unsigned n, m;
register Posf *p;
@@ -1597,9 +1545,7 @@ local void fill_window(s)
* NOTE: this function should be optimized to avoid extra copying from
* window to pending_buf.
*/
local block_state deflate_stored(s, flush)
deflate_state *s;
int flush;
local block_state deflate_stored(deflate_state *s, int flush)
{
/* Stored blocks are limited to 0xffff bytes, pending_buf is limited
* to pending_buf_size, and each stored block has a 5 byte header:
@@ -1670,9 +1616,7 @@ local block_state deflate_stored(s, flush)
* new strings in the dictionary only for unmatched strings or for short
* matches. It is used only for the fast compression options.
*/
local block_state deflate_fast(s, flush)
deflate_state *s;
int flush;
local block_state deflate_fast(deflate_state *s, int flush)
{
IPos hash_head; /* head of the hash chain */
int bflush; /* set if current block must be flushed */
@@ -1782,9 +1726,7 @@ local block_state deflate_fast(s, flush)
* evaluation for matches: a match is finally adopted only if there is
* no better match at the next window position.
*/
local block_state deflate_slow(s, flush)
deflate_state *s;
int flush;
local block_state deflate_slow(deflate_state *s, int flush)
{
IPos hash_head; /* head of hash chain */
int bflush; /* set if current block must be flushed */
@@ -1923,9 +1865,7 @@ local block_state deflate_slow(s, flush)
* one. Do not maintain a hash table. (It will be regenerated if this run of
* deflate switches away from Z_RLE.)
*/
local block_state deflate_rle(s, flush)
deflate_state *s;
int flush;
local block_state deflate_rle(deflate_state *s, int flush)
{
int bflush; /* set if current block must be flushed */
uInt prev; /* byte at distance one to match */
@@ -1996,9 +1936,7 @@ local block_state deflate_rle(s, flush)
* For Z_HUFFMAN_ONLY, do not look for matches. Do not maintain a hash table.
* (It will be regenerated if this run of deflate switches away from Huffman.)
*/
local block_state deflate_huff(s, flush)
deflate_state *s;
int flush;
local block_state deflate_huff(deflate_state *s, int flush)
{
int bflush; /* set if current block must be flushed */

View File

@@ -45,9 +45,7 @@
requires strm->avail_out >= 258 for each loop to avoid checking for
output space.
*/
void ZLIB_INTERNAL inflate_fast(strm, start)
z_streamp strm;
unsigned start; /* inflate()'s starting value for strm->avail_out */
void ZLIB_INTERNAL inflate_fast(z_streamp strm, unsigned start)
{
struct inflate_state FAR *state;
z_const unsigned char FAR *in; /* local strm->next_in */

View File

@@ -101,8 +101,7 @@ local int updatewindow OF((z_streamp strm, const unsigned char FAR *end,
local unsigned syncsearch OF((unsigned FAR *have, const unsigned char FAR *buf,
unsigned len));
int ZEXPORT inflateResetKeep(strm)
z_streamp strm;
int ZEXPORT inflateResetKeep(z_streamp strm)
{
struct inflate_state FAR *state;
@@ -126,8 +125,7 @@ z_streamp strm;
return Z_OK;
}
int ZEXPORT inflateReset(strm)
z_streamp strm;
int ZEXPORT inflateReset(z_streamp strm)
{
struct inflate_state FAR *state;
@@ -139,9 +137,7 @@ z_streamp strm;
return inflateResetKeep(strm);
}
int ZEXPORT inflateReset2(strm, windowBits)
z_streamp strm;
int windowBits;
int ZEXPORT inflateReset2(z_streamp strm, int windowBits)
{
int wrap;
struct inflate_state FAR *state;
@@ -177,11 +173,7 @@ int windowBits;
return inflateReset(strm);
}
int ZEXPORT inflateInit2_(strm, windowBits, version, stream_size)
z_streamp strm;
int windowBits;
const char *version;
int stream_size;
int ZEXPORT inflateInit2_(z_streamp strm, int windowBits, const char *version, int stream_size)
{
int ret;
struct inflate_state FAR *state;
@@ -219,18 +211,12 @@ int stream_size;
return ret;
}
int ZEXPORT inflateInit_(strm, version, stream_size)
z_streamp strm;
const char *version;
int stream_size;
int ZEXPORT inflateInit_(z_streamp strm, const char *version, int stream_size)
{
return inflateInit2_(strm, DEF_WBITS, version, stream_size);
}
int ZEXPORT inflatePrime(strm, bits, value)
z_streamp strm;
int bits;
int value;
int ZEXPORT inflatePrime(z_streamp strm, int bits, int value)
{
struct inflate_state FAR *state;
@@ -258,8 +244,7 @@ int value;
used for threaded applications, since the rewriting of the tables and virgin
may not be thread-safe.
*/
local void fixedtables(state)
struct inflate_state FAR *state;
local void fixedtables(struct inflate_state FAR *state)
{
#ifdef BUILDFIXED
static int virgin = 1;
@@ -376,10 +361,7 @@ void makefixed()
output will fall in the output data, making match copies simpler and faster.
The advantage may be dependent on the size of the processor's data caches.
*/
local int updatewindow(strm, end, copy)
z_streamp strm;
const Bytef *end;
unsigned copy;
local int updatewindow(z_streamp strm, const Bytef *end, unsigned copy)
{
struct inflate_state FAR *state;
unsigned dist;
@@ -602,9 +584,7 @@ unsigned copy;
will return Z_BUF_ERROR if it has not reached the end of the stream.
*/
int ZEXPORT inflate(strm, flush)
z_streamp strm;
int flush;
int ZEXPORT inflate(z_streamp strm, int flush)
{
struct inflate_state FAR *state;
z_const unsigned char FAR *next; /* next input */
@@ -1274,8 +1254,7 @@ int flush;
return ret;
}
int ZEXPORT inflateEnd(strm)
z_streamp strm;
int ZEXPORT inflateEnd(z_streamp strm)
{
struct inflate_state FAR *state;
if (strm == Z_NULL || strm->state == Z_NULL || strm->zfree == (free_func)0)
@@ -1288,10 +1267,7 @@ z_streamp strm;
return Z_OK;
}
int ZEXPORT inflateGetDictionary(strm, dictionary, dictLength)
z_streamp strm;
Bytef *dictionary;
uInt *dictLength;
int ZEXPORT inflateGetDictionary(z_streamp strm, Bytef *dictionary, uInt *dictLength)
{
struct inflate_state FAR *state;
@@ -1311,10 +1287,7 @@ uInt *dictLength;
return Z_OK;
}
int ZEXPORT inflateSetDictionary(strm, dictionary, dictLength)
z_streamp strm;
const Bytef *dictionary;
uInt dictLength;
int ZEXPORT inflateSetDictionary(z_streamp strm, const Bytef *dictionary, uInt dictLength)
{
struct inflate_state FAR *state;
unsigned long dictid;
@@ -1346,9 +1319,7 @@ uInt dictLength;
return Z_OK;
}
int ZEXPORT inflateGetHeader(strm, head)
z_streamp strm;
gz_headerp head;
int ZEXPORT inflateGetHeader(z_streamp strm, gz_headerp head)
{
struct inflate_state FAR *state;
@@ -1374,10 +1345,7 @@ gz_headerp head;
called again with more data and the *have state. *have is initialized to
zero for the first call.
*/
local unsigned syncsearch(have, buf, len)
unsigned FAR *have;
const unsigned char FAR *buf;
unsigned len;
local unsigned syncsearch(unsigned FAR *have, const unsigned char FAR *buf, unsigned len)
{
unsigned got;
unsigned next;
@@ -1397,8 +1365,7 @@ unsigned len;
return next;
}
int ZEXPORT inflateSync(strm)
z_streamp strm;
int ZEXPORT inflateSync(z_streamp strm)
{
unsigned len; /* number of bytes to look at or looked at */
unsigned long in, out; /* temporary to save total_in and total_out */
@@ -1448,8 +1415,7 @@ z_streamp strm;
block. When decompressing, PPP checks that at the end of input packet,
inflate is waiting for these length bytes.
*/
int ZEXPORT inflateSyncPoint(strm)
z_streamp strm;
int ZEXPORT inflateSyncPoint(z_streamp strm)
{
struct inflate_state FAR *state;
@@ -1458,9 +1424,7 @@ z_streamp strm;
return state->mode == STORED && state->bits == 0;
}
int ZEXPORT inflateCopy(dest, source)
z_streamp dest;
z_streamp source;
int ZEXPORT inflateCopy(z_streamp dest, z_streamp source)
{
struct inflate_state FAR *state;
struct inflate_state FAR *copy;
@@ -1505,9 +1469,7 @@ z_streamp source;
return Z_OK;
}
int ZEXPORT inflateUndermine(strm, subvert)
z_streamp strm;
int subvert;
int ZEXPORT inflateUndermine(z_streamp strm, int subvert)
{
struct inflate_state FAR *state;
@@ -1522,8 +1484,7 @@ int subvert;
#endif
}
long ZEXPORT inflateMark(strm)
z_streamp strm;
long ZEXPORT inflateMark(z_streamp strm)
{
struct inflate_state FAR *state;

View File

@@ -29,13 +29,9 @@ const char inflate_copyright[] =
table index bits. It will differ if the request is greater than the
longest code or if it is less than the shortest code.
*/
int ZLIB_INTERNAL inflate_table(type, lens, codes, table, bits, work)
codetype type;
unsigned short FAR *lens;
unsigned codes;
code FAR * FAR *table;
unsigned FAR *bits;
unsigned short FAR *work;
int ZLIB_INTERNAL inflate_table(codetype type, unsigned short FAR *lens,
unsigned codes, code FAR * FAR *table, unsigned FAR *bits,
unsigned short FAR *work)
{
unsigned len; /* a code's length in bits */
unsigned sym; /* index of code symbols */

View File

@@ -185,10 +185,7 @@ local void gen_trees_header OF((void));
#ifdef DEBUG
local void send_bits OF((deflate_state *s, int value, int length));
local void send_bits(s, value, length)
deflate_state *s;
int value; /* value to send */
int length; /* number of bits */
local void send_bits(deflate_state *s, int value, int length)
{
Tracevv((stderr," l %2d v %4x ", length, value));
Assert(length > 0 && length <= 15, "invalid length");
@@ -231,7 +228,7 @@ local void send_bits(s, value, length)
/* ===========================================================================
* Initialize the various 'constant' tables.
*/
local void tr_static_init()
local void tr_static_init(void)
{
#if defined(GEN_TREES_H) || !defined(STDC)
static int static_init_done = 0;
@@ -325,7 +322,7 @@ local void tr_static_init()
((i) == (last)? "\n};\n\n" : \
((i) % (width) == (width)-1 ? ",\n" : ", "))
void gen_trees_header()
void gen_trees_header(void)
{
FILE *header = fopen("trees.h", "w");
int i;
@@ -378,8 +375,7 @@ void gen_trees_header()
/* ===========================================================================
* Initialize the tree data structures for a new zlib stream.
*/
void ZLIB_INTERNAL _tr_init(s)
deflate_state *s;
void ZLIB_INTERNAL _tr_init(deflate_state *s)
{
tr_static_init();
@@ -406,8 +402,7 @@ void ZLIB_INTERNAL _tr_init(s)
/* ===========================================================================
* Initialize a new block.
*/
local void init_block(s)
deflate_state *s;
local void init_block(deflate_state *s)
{
int n; /* iterates over tree elements */
@@ -450,10 +445,7 @@ local void init_block(s)
* when the heap property is re-established (each father smaller than its
* two sons).
*/
local void pqdownheap(s, tree, k)
deflate_state *s;
ct_data *tree; /* the tree to restore */
int k; /* node to move down */
local void pqdownheap(deflate_state *s, ct_data *tree, int k)
{
int v = s->heap[k];
int j = k << 1; /* left son of k */
@@ -485,9 +477,7 @@ local void pqdownheap(s, tree, k)
* The length opt_len is updated; static_len is also updated if stree is
* not null.
*/
local void gen_bitlen(s, desc)
deflate_state *s;
tree_desc *desc; /* the tree descriptor */
local void gen_bitlen(deflate_state *s, tree_desc *desc)
{
ct_data *tree = desc->dyn_tree;
int max_code = desc->max_code;
@@ -572,10 +562,7 @@ local void gen_bitlen(s, desc)
* OUT assertion: the field code is set for all tree elements of non
* zero code length.
*/
local void gen_codes (tree, max_code, bl_count)
ct_data *tree; /* the tree to decorate */
int max_code; /* largest code with non zero frequency */
ushf *bl_count; /* number of codes at each bit length */
local void gen_codes(ct_data *tree, int max_code, ushf *bl_count)
{
ush next_code[MAX_BITS+1]; /* next code value for each bit length */
ush code = 0; /* running code value */
@@ -614,9 +601,7 @@ local void gen_codes (tree, max_code, bl_count)
* and corresponding code. The length opt_len is updated; static_len is
* also updated if stree is not null. The field max_code is set.
*/
local void build_tree(s, desc)
deflate_state *s;
tree_desc *desc; /* the tree descriptor */
local void build_tree(deflate_state *s, tree_desc *desc)
{
ct_data *tree = desc->dyn_tree;
const ct_data *stree = desc->stat_desc->static_tree;
@@ -702,10 +687,7 @@ local void build_tree(s, desc)
* Scan a literal or distance tree to determine the frequencies of the codes
* in the bit length tree.
*/
local void scan_tree (s, tree, max_code)
deflate_state *s;
ct_data *tree; /* the tree to be scanned */
int max_code; /* and its largest code of non zero frequency */
local void scan_tree(deflate_state *s, ct_data *tree, int max_code)
{
int n; /* iterates over all tree elements */
int prevlen = -1; /* last emitted length */
@@ -747,10 +729,7 @@ local void scan_tree (s, tree, max_code)
* Send a literal or distance tree in compressed form, using the codes in
* bl_tree.
*/
local void send_tree (s, tree, max_code)
deflate_state *s;
ct_data *tree; /* the tree to be scanned */
int max_code; /* and its largest code of non zero frequency */
local void send_tree(deflate_state *s, ct_data *tree, int max_code)
{
int n; /* iterates over all tree elements */
int prevlen = -1; /* last emitted length */
@@ -798,8 +777,7 @@ local void send_tree (s, tree, max_code)
* Construct the Huffman tree for the bit lengths and return the index in
* bl_order of the last bit length code to send.
*/
local int build_bl_tree(s)
deflate_state *s;
local int build_bl_tree(deflate_state *s)
{
int max_blindex; /* index of last bit length code of non zero freq */
@@ -833,9 +811,7 @@ local int build_bl_tree(s)
* lengths of the bit length codes, the literal tree and the distance tree.
* IN assertion: lcodes >= 257, dcodes >= 1, blcodes >= 4.
*/
local void send_all_trees(s, lcodes, dcodes, blcodes)
deflate_state *s;
int lcodes, dcodes, blcodes; /* number of codes for each tree */
local void send_all_trees(deflate_state *s, int lcodes, int dcodes, int blcodes)
{
int rank; /* index in bl_order */
@@ -862,11 +838,7 @@ local void send_all_trees(s, lcodes, dcodes, blcodes)
/* ===========================================================================
* Send a stored block
*/
void ZLIB_INTERNAL _tr_stored_block(s, buf, stored_len, last)
deflate_state *s;
charf *buf; /* input block */
ulg stored_len; /* length of input block */
int last; /* one if this is the last block for a file */
void ZLIB_INTERNAL _tr_stored_block(deflate_state *s, charf *buf, ulg stored_len, int last)
{
send_bits(s, (STORED_BLOCK<<1)+last, 3); /* send block type */
#ifdef DEBUG
@@ -879,8 +851,7 @@ void ZLIB_INTERNAL _tr_stored_block(s, buf, stored_len, last)
/* ===========================================================================
* Flush the bits in the bit buffer to pending output (leaves at most 7 bits)
*/
void ZLIB_INTERNAL _tr_flush_bits(s)
deflate_state *s;
void ZLIB_INTERNAL _tr_flush_bits(deflate_state *s)
{
bi_flush(s);
}
@@ -889,8 +860,7 @@ void ZLIB_INTERNAL _tr_flush_bits(s)
* Send one empty static block to give enough lookahead for inflate.
* This takes 10 bits, of which 7 may remain in the bit buffer.
*/
void ZLIB_INTERNAL _tr_align(s)
deflate_state *s;
void ZLIB_INTERNAL _tr_align(deflate_state *s)
{
send_bits(s, STATIC_TREES<<1, 3);
send_code(s, END_BLOCK, static_ltree);
@@ -904,11 +874,7 @@ void ZLIB_INTERNAL _tr_align(s)
* Determine the best encoding for the current block: dynamic trees, static
* trees or store, and output the encoded block to the zip file.
*/
void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
deflate_state *s;
charf *buf; /* input block, or NULL if too old */
ulg stored_len; /* length of input block */
int last; /* one if this is the last block for a file */
void ZLIB_INTERNAL _tr_flush_block(deflate_state *s, charf *buf, ulg stored_len, int last)
{
ulg opt_lenb, static_lenb; /* opt_len and static_len in bytes */
int max_blindex = 0; /* index of last bit length code of non zero freq */
@@ -1007,10 +973,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
* Save the match info and tally the frequency counts. Return true if
* the current block must be flushed.
*/
int ZLIB_INTERNAL _tr_tally (s, dist, lc)
deflate_state *s;
unsigned dist; /* distance of matched string */
unsigned lc; /* match length-MIN_MATCH or unmatched char (if dist==0) */
int ZLIB_INTERNAL _tr_tally(deflate_state *s, unsigned dist, unsigned lc)
{
s->sym_buf[s->sym_next++] = dist;
s->sym_buf[s->sym_next++] = dist >> 8;
@@ -1035,10 +998,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
/* ===========================================================================
* Send the block data compressed using the given Huffman trees
*/
local void compress_block(s, ltree, dtree)
deflate_state *s;
const ct_data *ltree; /* literal tree */
const ct_data *dtree; /* distance tree */
local void compress_block(deflate_state *s, const ct_data *ltree, const ct_data *dtree)
{
unsigned dist; /* distance of matched string */
int lc; /* match length or unmatched char (if dist == 0) */
@@ -1095,8 +1055,7 @@ local void compress_block(s, ltree, dtree)
* (7 {BEL}, 8 {BS}, 11 {VT}, 12 {FF}, 26 {SUB}, 27 {ESC}).
* IN assertion: the fields Freq of dyn_ltree are set.
*/
local int detect_data_type(s)
deflate_state *s;
local int detect_data_type(deflate_state *s)
{
/* black_mask is the bit mask of black-listed bytes
* set bits 0..6, 14..25, and 28..31
@@ -1129,9 +1088,7 @@ local int detect_data_type(s)
* method would use a table)
* IN assertion: 1 <= len <= 15
*/
local unsigned bi_reverse(code, len)
unsigned code; /* the value to invert */
int len; /* its bit length */
local unsigned bi_reverse(unsigned code, int len)
{
register unsigned res = 0;
do {
@@ -1144,8 +1101,7 @@ local unsigned bi_reverse(code, len)
/* ===========================================================================
* Flush the bit buffer, keeping at most 7 bits in it.
*/
local void bi_flush(s)
deflate_state *s;
local void bi_flush(deflate_state *s)
{
if (s->bi_valid == 16) {
put_short(s, s->bi_buf);
@@ -1161,8 +1117,7 @@ local void bi_flush(s)
/* ===========================================================================
* Flush the bit buffer and align the output on a byte boundary
*/
local void bi_windup(s)
deflate_state *s;
local void bi_windup(deflate_state *s)
{
if (s->bi_valid > 8) {
put_short(s, s->bi_buf);
@@ -1180,11 +1135,7 @@ local void bi_windup(s)
* Copy a stored block, storing first the length and its
* one's complement if requested.
*/
local void copy_block(s, buf, len, header)
deflate_state *s;
charf *buf; /* the input data */
unsigned len; /* its length */
int header; /* true if block header must be written */
local void copy_block(deflate_state *s, charf *buf, unsigned len, int header)
{
bi_windup(s); /* align on byte boundary */

View File

@@ -27,12 +27,12 @@ z_const char * const z_errmsg[10] = {
""};
const char * ZEXPORT zlibVersion()
const char * ZEXPORT zlibVersion(void)
{
return ZLIB_VERSION;
}
uLong ZEXPORT zlibCompileFlags()
uLong ZEXPORT zlibCompileFlags(void)
{
uLong flags;
@@ -122,8 +122,7 @@ uLong ZEXPORT zlibCompileFlags()
# endif
int ZLIB_INTERNAL z_verbose = verbose;
void ZLIB_INTERNAL z_error (m)
char *m;
void ZLIB_INTERNAL z_error (char *m)
{
fprintf(stderr, "%s\n", m);
exit(1);
@@ -133,8 +132,7 @@ void ZLIB_INTERNAL z_error (m)
/* exported to allow conversion of error code to string for compress() and
* uncompress()
*/
const char * ZEXPORT zError(err)
int err;
const char * ZEXPORT zError(int err)
{
return ERR_MSG(err);
}
@@ -149,10 +147,7 @@ const char * ZEXPORT zError(err)
#ifndef HAVE_MEMCPY
void ZLIB_INTERNAL zmemcpy(dest, source, len)
Bytef* dest;
const Bytef* source;
uInt len;
void ZLIB_INTERNAL zmemcpy(Bytef* dest, const Bytef* source, uInt len)
{
if (len == 0) return;
do {
@@ -160,10 +155,7 @@ void ZLIB_INTERNAL zmemcpy(dest, source, len)
} while (--len != 0);
}
int ZLIB_INTERNAL zmemcmp(s1, s2, len)
const Bytef* s1;
const Bytef* s2;
uInt len;
int ZLIB_INTERNAL zmemcmp(const Bytef* s1, const Bytef* s2, uInt len)
{
uInt j;
@@ -173,9 +165,7 @@ int ZLIB_INTERNAL zmemcmp(s1, s2, len)
return 0;
}
void ZLIB_INTERNAL zmemzero(dest, len)
Bytef* dest;
uInt len;
void ZLIB_INTERNAL zmemzero(Bytef* dest, uInt len)
{
if (len == 0) return;
do {
@@ -301,19 +291,14 @@ extern voidp calloc OF((uInt items, uInt size));
extern void free OF((voidpf ptr));
#endif
voidpf ZLIB_INTERNAL zcalloc (opaque, items, size)
voidpf opaque;
unsigned items;
unsigned size;
voidpf ZLIB_INTERNAL zcalloc (voidpf opaque, unsigned items, unsigned size)
{
if (opaque) items += size - size; /* make compiler happy */
return sizeof(uInt) > 2 ? (voidpf)malloc(items * size) :
(voidpf)calloc(items, size);
}
void ZLIB_INTERNAL zcfree (opaque, ptr)
voidpf opaque;
voidpf ptr;
void ZLIB_INTERNAL zcfree (voidpf opaque, voidpf ptr)
{
free(ptr);
if (opaque) return; /* make compiler happy */