Commit Graph

6558 Commits

Author SHA1 Message Date
Wayne Davison
cb0fe9e195 Improve CI setup. 2020-05-25 21:17:51 -07:00
Wayne Davison
87f2984df0 Improve how negotiated info affects batch files. 2020-05-25 19:19:59 -07:00
Wayne Davison
24ce3e9d54 Tweak the --zz option to --zc (aka --compress-choice). 2020-05-25 16:57:47 -07:00
Wayne Davison
98cddfaf7a Rename a couple files to .md 2020-05-25 15:05:06 -07:00
Wayne Davison
d1fcb8ce9d Add some extra indent. 2020-05-25 14:59:51 -07:00
Wayne Davison
b4ace35304 Create ccpp.yml 2020-05-25 14:43:25 -07:00
Wayne Davison
888ce058d8 Two sparse fixes from Yuxuan Shui.
- Make "len" parameter of do_punch_hole an OFF_T.
- Clear sparse_past_write in sparse_end(), otherwise when write_sparse()
  is called for the next file, do_punch_hole() will be called with a pos
  that's not actually the current position in file, causing it to fail.
2020-05-25 14:01:52 -07:00
Wayne Davison
c394e86184 Include lz4 compression support.
Based on a patch that was emailed to me without a valid return address.
2020-05-25 13:45:56 -07:00
Wayne Davison
4aaadc2f29 Include zstd compression support.
Based on a patch by Sebastian A. Siewior. Fixes bug #14338.
2020-05-25 13:44:48 -07:00
Wayne Davison
abef92c037 Fix handling of a compressor that has no off_level. 2020-05-25 13:02:56 -07:00
Wayne Davison
87019d7721 Output the default checksum & compress lists in the --version output. 2020-05-25 13:02:12 -07:00
Wayne Davison
fc265c5a92 A couple minor configure.ac tweaks. 2020-05-25 11:50:44 -07:00
Wayne Davison
d999efe6e5 Make compression-level handling generic. 2020-05-25 11:18:51 -07:00
Wayne Davison
97e8c55ee8 Some minor tweaks & tidying up. 2020-05-24 22:50:51 -07:00
Wayne Davison
739fa96737 Change odd-ball map_ptr() to use remainder like the others. 2020-05-24 20:38:48 -07:00
Jorrit Jongma
d474f2986e Improve performance of file_checksum()
Previously files were hashed in blocks of CSUM_CHUNK (64) bytes. This
causes significant overhead. The CSUM_CHUNK define cannot be changed as
md5.c depends on it, but there is no obvious reason to use it in
file_checksum(). By using CHUNK_SIZE (32 kB) instead, in some test
cases throughput more than doubles.
2020-05-24 20:33:33 -07:00
Wayne Davison
a863c62cd1 More NEWS updates. 2020-05-24 20:19:15 -07:00
Wayne Davison
5ac353d845 Prefer zlibx compression consistently instead of having 2 possible default preference orders. 2020-05-24 19:52:08 -07:00
Wayne Davison
faecd066a6 Don't auto-foward debug options to the server side anymore. 2020-05-24 19:37:15 -07:00
Wayne Davison
6efc43cc0a Fix -z choice with older rsyncs. 2020-05-24 19:16:05 -07:00
Wayne Davison
4496e0e8e7 A few more compression tweaks. 2020-05-24 18:43:03 -07:00
Wayne Davison
64d5ea39c0 More compress changes
- Add the zlibx (external-code compatible) compression name.
- Re-enable zlib support with the external library so it can be
  tried as a fallback if zlibx isn't available.
- Add --compress-choice=STR (aka -zz=STR) option.
- Make --cc=STR an alias for --checksum-choice=STR.
- Hook up the new compression negotiation logic.
2020-05-24 17:24:42 -07:00
Wayne Davison
4af8403aa2 Fix negotiation of none & improve NSTR debug msgs. 2020-05-24 13:49:06 -07:00
Wayne Davison
2f84a6bd73 Add support for negotiated checksum names. 2020-05-24 13:22:19 -07:00
Wayne Davison
eda15d52a8 Make xxh64 the "main_name" for the current xxhash. 2020-05-24 02:10:05 -07:00
Wayne Davison
741d5f10c6 Fix some warnings. 2020-05-24 02:04:14 -07:00
Wayne Davison
4f92fd8ddd Some more checksum improvements
- Add/improve --debug=CSUM2 messages.
- Add an "xxh64" alias for "xxhash" name because we should be
  getting a few more xxhash variations in the future.
- Tweak the matching code to handle entries that have multiple
  names.
- Tweak some of the vars/defines.
2020-05-24 01:00:53 -07:00
Wayne Davison
7f2359a5cc Improve some early debug-message newlines.
Avoid a newline issue during the output of --DEBUG=CSUM info from
both the server and the client -- we need to output the full message
with its newline as much as possible.
2020-05-23 21:51:28 -07:00
Wayne Davison
6e942e5898 Avoid re-evaluating the args of SIVAL* w/CAREFUL_ALIGNMENT. 2020-05-23 21:43:53 -07:00
Wayne Davison
1c9bb168bb Unify the checksum context memory, since we only use one at a time. 2020-05-23 18:52:03 -07:00
Wayne Davison
799de21af6 Fixed the use of openssl MD4 for transfer checksums. 2020-05-23 16:22:36 -07:00
Wayne Davison
1cb1edeb68 Optional openssl support for MD4 pre-transfer checksums (but, sadly, not transfer checksums). 2020-05-23 12:30:58 -07:00
Wayne Davison
15c1162b24 Add optional use of the openssl crypto lib for MD5. 2020-05-23 10:06:59 -07:00
Wayne Davison
a7175ee029 Mention a few more news items. 2020-05-22 23:26:25 -07:00
Wayne Davison
68516f91be Add "input" handling for cmd_txt_*() pkglib.py. 2020-05-22 22:49:02 -07:00
Jorrit Jongma
531ffa8104 Optimized assembler version of md5_process() for x86-64
Originally created by Marc Bevand and placed in the public domain.
Enable/disabled via the same --enable-simd configure switch as
the rolling checksum optimizations.
2020-05-22 22:37:21 -07:00
Wayne Davison
d7212df0f1 A little more safety in negotiate_checksum(). 2020-05-22 19:29:05 -07:00
Wayne Davison
a28bc3ebf6 Promoting xxhash support. 2020-05-22 17:59:12 -07:00
Wayne Davison
55bb4dab7a Some checksum improvements
- Improve csum negotation logic.
- Define the csum names in a single structure.
- Add --debug=CSUM.
2020-05-22 17:59:12 -07:00
Jorrit Jongma
5fa4209ca0 AVX2 optimized version of get_checksum1() for x86-64
Additionally restructures build switches and defines from SSE2 to SIMD,
to allow potential reuse should patches become available with SIMD
instructions for other processor architectures.

(Some minor tweaks of Jorrit's patch to avoid requiring GNU make and to
avoid C++ comments in .c files.)
2020-05-22 11:31:31 -07:00
Wayne Davison
4f6c8c6652 Checksum negotiation & more bits for compat_flags
- Add checksum negotiation to the protocol so that we can easily add new
  checksum algorithms and each will be used when both sides support it.
- Increase the size of the compat_flags value in the protocol from a
  byte to an int.
2020-05-22 09:52:14 -07:00
Wayne Davison
a7303a3d3d Fix a bug in the writing of the batch.sh file
Fix the code that writes the options and the default destination path
into the batch.sh file to be able to handle options being specified
after source/dest args.
2020-05-22 08:27:07 -07:00
Jorrit Jongma
70c6b408dc SSE2/SSSE3 optimized version of get_checksum1() for x86-64
Requires compilation using GCC C++ front end, build scripts have been
modified accordingly. C++ is only used when the optimization is enabled
(g++ as compiler, x86-64 build target, --enable-sse2 is passed to
configure).

(Wayne made a few tweaks, including making it disabled by default.)
2020-05-21 14:41:55 -07:00
Wayne Davison
be7af36c51 Tweak the accept/refuse strings a bit. 2020-05-18 00:06:06 -07:00
Wayne Davison
3b36bde953 Add back a lost "*" and document the refusing of log-file* opts. 2020-05-17 23:28:35 -07:00
Wayne Davison
c3986d4c5a More manpage improvements for "refuse options". 2020-05-17 22:19:25 -07:00
Wayne Davison
b3a1a0ca9d Add the ability to negate matches for the daemon's "refuse options". 2020-05-17 21:29:11 -07:00
Wayne Davison
e448d31d63 Need to flush early errors before we exit. 2020-05-17 21:20:15 -07:00
Wayne Davison
37de48979e Some pkglib improvements. 2020-05-17 20:32:43 -07:00
Wayne Davison
08f955e17b A couple more manpage fixes. 2020-05-13 00:20:03 -07:00