Preparing for release of 3.2.7pre1

- The sanitize_paths variable was set too often. It only needs to be set
  when the "inner" path is not "/".  This change avoids sanitizing &
  munging things for a path=/ module just because chroot is off.
- The default for "use chroot" is now "unset" instead of "true".  When
  unset it checks if chrooting works, and if not, it proceeds with a
  sanitized copy instead of totally failing to work.  This makes it
  easier to setup a non-root rsync daemon, for instance.  It will have
  no effect on a typical Linux root-run daemon where the default will
  continue to use chroot (because chrooting works).  A config file can
  explicitly set "use chroot = true | false" to force the choice.
- Try to improve the "use chroot" manpage.

.cirrus.yml

@@ -1,7 +1,7 @@
 freebsd_task:
   name: FreeBSD
   freebsd_instance:
-    image_family: freebsd-13-0
+    image_family: freebsd-13-1
   env:
     PATH: /usr/local/bin:$PATH
   prep_script:

NEWS.md

@@ -1,3 +1,88 @@
+# NEWS for rsync 3.2.7 (UNRELEASED)
+
+## Changes in this version:
+
+### BUG FIXES:
+
+- Fixed the client-side validating of the remote sender's filtering behavior.
+
+- More fixes for the "unrequested file-list name" name, including a copy of
+  "/" with `--relative` enabled and a copy with a lot of related paths with
+  `--relative` enabled (often derived from a `--files-from` list).
+
+- When rsync gets an unpack error on an ACL, mention the filename.
+
+- Avoid over-setting sanitize_paths when a daemon is serving "/" (even if
+  "use chroot" is false).
+
+### ENHANCEMENTS:
+
+- Added negotiated daemon-auth support that allows a stronger checksum digest
+  to be used to validate a user's login to the daemon.  Added SHA512, SHA256,
+  and SHA1 digests to MD5 & MD4.  These new digests are at the highest priority
+  in the new daemon-auth negotiation list.
+
+- Added support for the SHA1 digest in file checksums.  While this tends to be
+  overkill, it is available if someone really needs it.  This overly-long
+  checksum is at the lowest priority in the normal checksum negotiation list.
+  See `--checksum-choice` (`--cc`) and the `RSYNC_CHECKSUM_LIST` environment
+  var for how to customize this.
+
+- Improved the xattr hash table to use a 64-bit key without slowing down the
+  key's computation.  This should make extra sure that a collision doesn't
+  happen.
+
+- If the `--version` option is repeated (e.g. `-VV`) then the information is
+  output in a (still fairly readable) JSON format.  Client side only.
+
+- The script `support/json-rsync-version` is available to get the JSON style
+  version output from any rsync.  The script accepts either text on stdin
+  **or** an arg that specifies an rsync executable to run with a doubled
+  `--version` option.  If the text we get isn't already in JSON format, it is
+  converted. Newer rsync versions will provide more complete json info than
+  older rsync versions.
+
+- The [`use chroot`](rsyncd.conf.5#use_chroot) daemon parameter now defaults to
+  "unset" so that rsync can use chroot when it works and a sanitized copy when
+  chroot is not supported (e.g., for a non-root daemon).  Explicitly setting
+  the parameter to true or false (on or off) behaves the same way as before.
+
+- The `--fuzzy` option was optimized a bit to try to cut down on the amount of
+  computations when considering a big pool of files. The simple heuristic from
+  Kenneth Finnegan resuled in about a 2x speedup.
+
+### PACKAGING RELATED:
+
+- The checksum code now uses openssl's EVP methods, which gets rid of various
+  deprecation warnings and makes it easy to support more digest methods.  On
+  newer systems, the MD4 digest is marked as legacy in the openssl code, which
+  makes openssl refuse to support it via EVP.  You can choose to ignore this
+  and allow the included MD4 code to be used for older rsync connections (when
+  talking to an rsync prior to 3.0.0) or you can choose to configure rsync to
+  tell openssl to enable legacy algorithms (see below).
+
+- A simple openssl config file is supplied that can be installed for rsync to
+  use.  If you install packaging/openssl-rsync.cnf to a public spot (such as
+  `/etc/ssl/openssl-rsync.cnf`) and then run configure with the option
+  `--with-openssl-conf=/path/name.cnf`, this will cause rsync to export the
+  configured path in the OPENSSL_CONF environment variable (when the variable
+  is not already set).  This will enable openssl's MD4 code for rsync to use.
+
+- The packager may wish to include an explicit "use chroot = true" in the top
+  section of their supplied /etc/rsyncd.conf file if the daemon is being
+  installed to run as the root user (though rsync should behave the same even
+  with the value unset, a little extra paranoia doesn't hurt).
+
+- I've noticed that some packagers haven't installed support/nameconvert for
+  users to use in their chrooted rsync configs.  Even if it is not installed
+  as an executable script (to avoid a python3 dependency) it would be good to
+  install it with the other rsync-related support scripts.
+
+- It would be good to add support/json-rsync-version to the list of installed
+  support scripts.
+
+------------------------------------------------------------------------------
+
 # NEWS for rsync 3.2.6 (9 Sep 2022)
 
 ## Changes in this version:
@@ -4592,6 +4677,7 @@
 
 | RELEASE DATE | VER.   | DATE OF COMMIT\* | PROTOCOL    |
 |--------------|--------|------------------|-------------|
+| ?? Dec 2022  | 3.2.7  |                  | 31          |
 | 09 Sep 2022  | 3.2.6  |                  | 31          |
 | 14 Aug 2022  | 3.2.5  |                  | 31          |
 | 15 Apr 2022  | 3.2.4  |                  | 31          |

acls.c

@@ -519,6 +519,7 @@ static int get_rsync_acl(const char *fname, rsync_acl *racl,
 
 		sys_acl_free_acl(sacl);
 		if (!ok) {
+			rsyserr(FERROR_XFER, errno, "get_acl: unpack_smb_acl(%s)", fname);
 			return -1;
 		}
 	} else if (no_acl_syscall_error(errno)) {

authenticate.c

@@ -2,7 +2,7 @@
  * Support rsync daemon authentication.
  *
  * Copyright (C) 1998-2000 Andrew Tridgell
- * Copyright (C) 2002-2020 Wayne Davison
+ * Copyright (C) 2002-2022 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -24,6 +24,7 @@
 
 extern int read_only;
 extern char *password_file;
+extern struct name_num_obj valid_auth_checksums;
 
 /***************************************************************************
 encode a buffer using base64 - simple and slow algorithm. null terminates
@@ -72,9 +73,9 @@ static void gen_challenge(const char *addr, char *challenge)
 	SIVAL(input, 20, tv.tv_usec);
 	SIVAL(input, 24, getpid());
 
-	sum_init(-1, 0);
+	len = sum_init(valid_auth_checksums.negotiated_nni, 0);
 	sum_update(input, sizeof input);
-	len = sum_end(digest);
+	sum_end(digest);
 
 	base64_encode(digest, len, challenge, 0);
 }
@@ -86,10 +87,10 @@ static void generate_hash(const char *in, const char *challenge, char *out)
 	char buf[MAX_DIGEST_LEN];
 	int len;
 
-	sum_init(-1, 0);
+	len = sum_init(valid_auth_checksums.negotiated_nni, 0);
 	sum_update(in, strlen(in));
 	sum_update(challenge, strlen(challenge));
-	len = sum_end(buf);
+	sum_end(buf);
 
 	base64_encode(buf, len, out, 0);
 }
@@ -238,6 +239,7 @@ char *auth_server(int f_in, int f_out, int module, const char *host,
 	if (!users || !*users)
 		return "";
 
+	negotiate_daemon_auth(f_out, 0);
 	gen_challenge(addr, challenge);
 
 	io_printf(f_out, "%s%s\n", leader, challenge);
@@ -350,6 +352,7 @@ void auth_client(int fd, const char *user, const char *challenge)
 
 	if (!user || !*user)
 		user = "nobody";
+	negotiate_daemon_auth(-1, 1);
 
 	if (!(pass = getpassf(password_file))
 	 && !(pass = getenv("RSYNC_PASSWORD"))) {

checksum.c

@@ -42,49 +42,95 @@ extern int protocol_version;
 extern int proper_seed_order;
 extern const char *checksum_choice;
 
+#define NNI_BUILTIN (1<<0)
+#define NNI_EVP (1<<1)
+#define NNI_EVP_OK (1<<2)
+
 struct name_num_item valid_checksums_items[] = {
 #ifdef SUPPORT_XXH3
-	{ CSUM_XXH3_128, "xxh128", NULL },
-	{ CSUM_XXH3_64, "xxh3", NULL },
+	{ CSUM_XXH3_128, 0, "xxh128", NULL },
+	{ CSUM_XXH3_64, 0, "xxh3", NULL },
 #endif
 #ifdef SUPPORT_XXHASH
-	{ CSUM_XXH64, "xxh64", NULL },
-	{ CSUM_XXH64, "xxhash", NULL },
+	{ CSUM_XXH64, 0, "xxh64", NULL },
+	{ CSUM_XXH64, 0, "xxhash", NULL },
 #endif
-	{ CSUM_MD5, "md5", NULL },
-	{ CSUM_MD4, "md4", NULL },
-	{ CSUM_NONE, "none", NULL },
-	{ 0, NULL, NULL }
+	{ CSUM_MD5, NNI_BUILTIN|NNI_EVP, "md5", NULL },
+	{ CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL },
+#ifdef SHA_DIGEST_LENGTH
+	{ CSUM_SHA1, NNI_EVP, "sha1", NULL },
+#endif
+	{ CSUM_NONE, 0, "none", NULL },
+	{ 0, 0, NULL, NULL }
 };
 
 struct name_num_obj valid_checksums = {
-	"checksum", NULL, NULL, 0, 0, valid_checksums_items
+	"checksum", NULL, 0, 0, valid_checksums_items
 };
 
-int xfersum_type = 0; /* used for the file transfer checksums */
-int checksum_type = 0; /* used for the pre-transfer (--checksum) checksums */
+struct name_num_item valid_auth_checksums_items[] = {
+#ifdef SHA512_DIGEST_LENGTH
+	{ CSUM_SHA512, NNI_EVP, "sha512", NULL },
+#endif
+#ifdef SHA256_DIGEST_LENGTH
+	{ CSUM_SHA256, NNI_EVP, "sha256", NULL },
+#endif
+#ifdef SHA_DIGEST_LENGTH
+	{ CSUM_SHA1, NNI_EVP, "sha1", NULL },
+#endif
+	{ CSUM_MD5, NNI_BUILTIN|NNI_EVP, "md5", NULL },
+	{ CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL },
+	{ 0, 0, NULL, NULL }
+};
+
+struct name_num_obj valid_auth_checksums = {
+	"daemon auth checksum", NULL, 0, 0, valid_auth_checksums_items
+};
+
+/* These cannot make use of openssl, so they're marked just as built-in */
+struct name_num_item implied_checksum_md4 =
+    { CSUM_MD4, NNI_BUILTIN, "md4", NULL };
+struct name_num_item implied_checksum_md5 =
+    { CSUM_MD5, NNI_BUILTIN, "md5", NULL };
+
+struct name_num_item *xfer_sum_nni; /* used for the transfer checksum2 computations */
+int xfer_sum_len;
+struct name_num_item *file_sum_nni; /* used for the pre-transfer --checksum computations */
+int file_sum_len, file_sum_extra_cnt;
+
+#ifdef USE_OPENSSL
+const EVP_MD *xfer_sum_evp_md;
+const EVP_MD *file_sum_evp_md;
+EVP_MD_CTX *ctx_evp = NULL;
+#endif
 
 static int initialized_choices = 0;
 
-int parse_csum_name(const char *name, int len)
+struct name_num_item *parse_csum_name(const char *name, int len)
 {
 	struct name_num_item *nni;
 
 	if (len < 0 && name)
 		len = strlen(name);
 
-	if (!name || (len == 4 && strncasecmp(name, "auto", 4) == 0)) {
-		if (protocol_version >= 30)
-			return CSUM_MD5;
-		if (protocol_version >= 27)
-			return CSUM_MD4_OLD;
-		if (protocol_version >= 21)
-			return CSUM_MD4_BUSTED;
-		return CSUM_MD4_ARCHAIC;
-	}
+	init_checksum_choices();
 
-	if (!initialized_choices)
-		init_checksum_choices();
+	if (!name || (len == 4 && strncasecmp(name, "auto", 4) == 0)) {
+		if (protocol_version >= 30) {
+			if (!proper_seed_order)
+				return &implied_checksum_md5;
+			name = "md5";
+			len = 3;
+		} else {
+			if (protocol_version >= 27)
+				implied_checksum_md4.num = CSUM_MD4_OLD;
+			else if (protocol_version >= 21)
+				implied_checksum_md4.num = CSUM_MD4_BUSTED;
+			else
+				implied_checksum_md4.num = CSUM_MD4_ARCHAIC;
+			return &implied_checksum_md4;
+		}
+	}
 
 	nni = get_nni_by_name(&valid_checksums, name, len);
 
@@ -93,44 +139,74 @@ int parse_csum_name(const char *name, int len)
 		exit_cleanup(RERR_UNSUPPORTED);
 	}
 
-	return nni->num;
+	return nni;
 }
 
-static const char *checksum_name(int num)
+#ifdef USE_OPENSSL
+static const EVP_MD *csum_evp_md(struct name_num_item *nni)
 {
-	struct name_num_item *nni = get_nni_by_num(&valid_checksums, num);
+	const EVP_MD *emd;
+	if (!(nni->flags & NNI_EVP))
+		return NULL;
 
-	return nni ? nni->name : num < CSUM_MD4 ? "md4" : "UNKNOWN";
+#ifdef USE_MD5_ASM
+	if (nni->num == CSUM_MD5)
+		emd = NULL;
+	else
+#endif
+		emd = EVP_get_digestbyname(nni->name);                                               
+	if (emd && !(nni->flags & NNI_EVP_OK)) { /* Make sure it works before we advertise it */
+		if (!ctx_evp && !(ctx_evp = EVP_MD_CTX_create()))
+			out_of_memory("csum_evp_md");
+		/* Some routines are marked as legacy and are not enabled in the openssl.cnf file.
+		 * If we can't init the emd, we'll fall back to our built-in code. */
+		if (EVP_DigestInit_ex(ctx_evp, emd, NULL) == 0)
+			emd = NULL;
+		else
+			nni->flags = (nni->flags & ~NNI_BUILTIN) | NNI_EVP_OK;
+	}
+	if (!emd)
+		nni->flags &= ~NNI_EVP;
+	return emd;
 }
+#endif
 
 void parse_checksum_choice(int final_call)
 {
-	if (valid_checksums.negotiated_name)
-		xfersum_type = checksum_type = valid_checksums.negotiated_num;
+	if (valid_checksums.negotiated_nni)
+		xfer_sum_nni = file_sum_nni = valid_checksums.negotiated_nni;
 	else {
 		char *cp = checksum_choice ? strchr(checksum_choice, ',') : NULL;
 		if (cp) {
-			xfersum_type = parse_csum_name(checksum_choice, cp - checksum_choice);
-			checksum_type = parse_csum_name(cp+1, -1);
+			xfer_sum_nni = parse_csum_name(checksum_choice, cp - checksum_choice);
+			file_sum_nni = parse_csum_name(cp+1, -1);
 		} else
-			xfersum_type = checksum_type = parse_csum_name(checksum_choice, -1);
+			xfer_sum_nni = file_sum_nni = parse_csum_name(checksum_choice, -1);
 		if (am_server && checksum_choice)
-			validate_choice_vs_env(NSTR_CHECKSUM, xfersum_type, checksum_type);
+			validate_choice_vs_env(NSTR_CHECKSUM, xfer_sum_nni->num, file_sum_nni->num);
 	}
+	xfer_sum_len = csum_len_for_type(xfer_sum_nni->num, 0);
+	file_sum_len = csum_len_for_type(file_sum_nni->num, 0);
+#ifdef USE_OPENSSL
+	xfer_sum_evp_md = csum_evp_md(xfer_sum_nni);
+	file_sum_evp_md = csum_evp_md(file_sum_nni);
+#endif
 
-	if (xfersum_type == CSUM_NONE)
+	file_sum_extra_cnt = (file_sum_len + EXTRA_LEN - 1) / EXTRA_LEN;
+
+	if (xfer_sum_nni->num == CSUM_NONE)
 		whole_file = 1;
 
 	/* Snag the checksum name for both write_batch's option output & the following debug output. */
-	if (valid_checksums.negotiated_name)
-		checksum_choice = valid_checksums.negotiated_name;
+	if (valid_checksums.negotiated_nni)
+		checksum_choice = valid_checksums.negotiated_nni->name;
 	else if (checksum_choice == NULL)
-		checksum_choice = checksum_name(xfersum_type);
+		checksum_choice = xfer_sum_nni->name;
 
 	if (final_call && DEBUG_GTE(NSTR, am_server ? 3 : 1)) {
 		rprintf(FINFO, "%s%s checksum: %s\n",
 			am_server ? "Server" : "Client",
-			valid_checksums.negotiated_name ? " negotiated" : "",
+			valid_checksums.negotiated_nni ? " negotiated" : "",
 			checksum_choice);
 	}
 }
@@ -150,6 +226,18 @@ int csum_len_for_type(int cst, BOOL flist_csum)
 		return MD4_DIGEST_LEN;
 	  case CSUM_MD5:
 		return MD5_DIGEST_LEN;
+#ifdef SHA_DIGEST_LENGTH
+	  case CSUM_SHA1:
+		return SHA_DIGEST_LENGTH;
+#endif
+#ifdef SHA256_DIGEST_LENGTH
+	  case CSUM_SHA256:
+		return SHA256_DIGEST_LENGTH;
+#endif
+#ifdef SHA512_DIGEST_LENGTH
+	  case CSUM_SHA512:
+		return SHA512_DIGEST_LENGTH;
+#endif
 	  case CSUM_XXH64:
 	  case CSUM_XXH3_64:
 		return 64/8;
@@ -175,6 +263,9 @@ int canonical_checksum(int csum_type)
 		break;
 	  case CSUM_MD4:
 	  case CSUM_MD5:
+	  case CSUM_SHA1:
+	  case CSUM_SHA256:
+	  case CSUM_SHA512:
 		return -1;
 	  case CSUM_XXH64:
 	  case CSUM_XXH3_64:
@@ -211,7 +302,22 @@ uint32 get_checksum1(char *buf1, int32 len)
 
 void get_checksum2(char *buf, int32 len, char *sum)
 {
-	switch (xfersum_type) {
+#ifdef USE_OPENSSL
+	if (xfer_sum_evp_md) {
+		static EVP_MD_CTX *evp = NULL;
+		uchar seedbuf[4];
+		if (!evp && !(evp = EVP_MD_CTX_create()))
+			out_of_memory("get_checksum2");
+		EVP_DigestInit_ex(evp, xfer_sum_evp_md, NULL);
+		if (checksum_seed) {
+			SIVALu(seedbuf, 0, checksum_seed);
+			EVP_DigestUpdate(evp, seedbuf, 4);
+		}
+		EVP_DigestUpdate(evp, (uchar *)buf, len);
+		EVP_DigestFinal_ex(evp, (uchar *)sum, NULL);
+	} else
+#endif
+	switch (xfer_sum_nni->num) {
 #ifdef SUPPORT_XXHASH
 	  case CSUM_XXH64:
 		SIVAL64(sum, 0, XXH64(buf, len, checksum_seed));
@@ -229,7 +335,7 @@ void get_checksum2(char *buf, int32 len, char *sum)
 	  }
 #endif
 	  case CSUM_MD5: {
-		md5_context m5;
+		md_context m5;
 		uchar seedbuf[4];
 		md5_begin(&m5);
 		if (proper_seed_order) {
@@ -249,20 +355,6 @@ void get_checksum2(char *buf, int32 len, char *sum)
 		break;
 	  }
 	  case CSUM_MD4:
-#ifdef USE_OPENSSL
-	  {
-		MD4_CTX m4;
-		MD4_Init(&m4);
-		MD4_Update(&m4, (uchar *)buf, len);
-		if (checksum_seed) {
-			uchar seedbuf[4];
-			SIVALu(seedbuf, 0, checksum_seed);
-			MD4_Update(&m4, seedbuf, 4);
-		}
-		MD4_Final((uchar *)sum, &m4);
-		break;
-	  }
-#endif
 	  case CSUM_MD4_OLD:
 	  case CSUM_MD4_BUSTED:
 	  case CSUM_MD4_ARCHAIC: {
@@ -295,7 +387,7 @@ void get_checksum2(char *buf, int32 len, char *sum)
 		 * are multiples of 64.  This is fixed by calling mdfour_update()
 		 * even when there are no more bytes.
 		 */
-		if (len - i > 0 || xfersum_type > CSUM_MD4_BUSTED)
+		if (len - i > 0 || xfer_sum_nni->num > CSUM_MD4_BUSTED)
 			mdfour_update(&m, (uchar *)(buf1+i), len-i);
 
 		mdfour_result(&m, (uchar *)sum);
@@ -313,15 +405,33 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum)
 	int32 remainder;
 	int fd;
 
-	memset(sum, 0, MAX_DIGEST_LEN);
-
 	fd = do_open(fname, O_RDONLY, 0);
-	if (fd == -1)
+	if (fd == -1) {
+		memset(sum, 0, file_sum_len);
 		return;
+	}
 
 	buf = map_file(fd, len, MAX_MAP_SIZE, CHUNK_SIZE);
 
-	switch (checksum_type) {
+#ifdef USE_OPENSSL
+	if (file_sum_evp_md) {
+		static EVP_MD_CTX *evp = NULL;
+		if (!evp && !(evp = EVP_MD_CTX_create()))
+			out_of_memory("file_checksum");
+
+		EVP_DigestInit_ex(evp, file_sum_evp_md, NULL);
+
+		for (i = 0; i + CHUNK_SIZE <= len; i += CHUNK_SIZE)
+			EVP_DigestUpdate(evp, (uchar *)map_ptr(buf, i, CHUNK_SIZE), CHUNK_SIZE);
+
+		remainder = (int32)(len - i);
+		if (remainder > 0)
+			EVP_DigestUpdate(evp, (uchar *)map_ptr(buf, i, remainder), remainder);
+
+		EVP_DigestFinal_ex(evp, (uchar *)sum, NULL);
+	} else
+#endif
+	switch (file_sum_nni->num) {
 #ifdef SUPPORT_XXHASH
 	  case CSUM_XXH64: {
 		static XXH64_state_t* state = NULL;
@@ -381,7 +491,7 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum)
 	  }
 #endif
 	  case CSUM_MD5: {
-		md5_context m5;
+		md_context m5;
 
 		md5_begin(&m5);
 
@@ -396,23 +506,6 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum)
 		break;
 	  }
 	  case CSUM_MD4:
-#ifdef USE_OPENSSL
-	  {
-		MD4_CTX m4;
-
-		MD4_Init(&m4);
-
-		for (i = 0; i + CHUNK_SIZE <= len; i += CHUNK_SIZE)
-			MD4_Update(&m4, (uchar *)map_ptr(buf, i, CHUNK_SIZE), CHUNK_SIZE);
-
-		remainder = (int32)(len - i);
-		if (remainder > 0)
-			MD4_Update(&m4, (uchar *)map_ptr(buf, i, remainder), remainder);
-
-		MD4_Final((uchar *)sum, &m4);
-		break;
-	  }
-#endif
 	  case CSUM_MD4_OLD:
 	  case CSUM_MD4_BUSTED:
 	  case CSUM_MD4_ARCHAIC: {
@@ -428,7 +521,7 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum)
 		 * are multiples of 64.  This is fixed by calling mdfour_update()
 		 * even when there are no more bytes. */
 		remainder = (int32)(len - i);
-		if (remainder > 0 || checksum_type > CSUM_MD4_BUSTED)
+		if (remainder > 0 || file_sum_nni->num > CSUM_MD4_BUSTED)
 			mdfour_update(&m, (uchar *)map_ptr(buf, i, remainder), remainder);
 
 		mdfour_result(&m, (uchar *)sum);
@@ -436,7 +529,7 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum)
 	  }
 	  default:
 		rprintf(FERROR, "Invalid checksum-choice for --checksum: %s (%d)\n",
-			checksum_name(checksum_type), checksum_type);
+			file_sum_nni->name, file_sum_nni->num);
 		exit_cleanup(RERR_UNSUPPORTED);
 	}
 
@@ -445,30 +538,43 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum)
 }
 
 static int32 sumresidue;
-static union {
-	md_context md;
-#ifdef USE_OPENSSL
-	MD4_CTX m4;
-#endif
-	md5_context m5;
-} ctx;
+static md_context ctx_md;
 #ifdef SUPPORT_XXHASH
 static XXH64_state_t* xxh64_state;
 #endif
 #ifdef SUPPORT_XXH3
 static XXH3_state_t* xxh3_state;
 #endif
-static int cursum_type;
+static struct name_num_item *cur_sum_nni;
+int cur_sum_len;
 
-void sum_init(int csum_type, int seed)
+#ifdef USE_OPENSSL
+static const EVP_MD *cur_sum_evp_md;
+#endif
+
+/* Initialize a hash digest accumulator.  Data is supplied via
+ * sum_update() and the resulting binary digest is retrieved via
+ * sum_end().  This only supports one active sum at a time. */
+int sum_init(struct name_num_item *nni, int seed)
 {
 	char s[4];
 
-	if (csum_type < 0)
-		csum_type = parse_csum_name(NULL, 0);
-	cursum_type = csum_type;
+	if (!nni)
+		nni = parse_csum_name(NULL, 0);
+	cur_sum_nni = nni;
+	cur_sum_len = csum_len_for_type(nni->num, 0);
+#ifdef USE_OPENSSL
+	cur_sum_evp_md = csum_evp_md(nni);
+#endif
 
-	switch (csum_type) {
+#ifdef USE_OPENSSL
+	if (cur_sum_evp_md) {
+		if (!ctx_evp && !(ctx_evp = EVP_MD_CTX_create()))
+			out_of_memory("file_checksum");
+		EVP_DigestInit_ex(ctx_evp, cur_sum_evp_md, NULL);
+	} else
+#endif
+	switch (cur_sum_nni->num) {
 #ifdef SUPPORT_XXHASH
 	  case CSUM_XXH64:
 		if (!xxh64_state && !(xxh64_state = XXH64_createState()))
@@ -489,20 +595,16 @@ void sum_init(int csum_type, int seed)
 		break;
 #endif
 	  case CSUM_MD5:
-		md5_begin(&ctx.m5);
+		md5_begin(&ctx_md);
 		break;
 	  case CSUM_MD4:
-#ifdef USE_OPENSSL
-		MD4_Init(&ctx.m4);
-#else
-		mdfour_begin(&ctx.md);
+		mdfour_begin(&ctx_md);
 		sumresidue = 0;
-#endif
 		break;
 	  case CSUM_MD4_OLD:
 	  case CSUM_MD4_BUSTED:
 	  case CSUM_MD4_ARCHAIC:
-		mdfour_begin(&ctx.md);
+		mdfour_begin(&ctx_md);
 		sumresidue = 0;
 		SIVAL(s, 0, seed);
 		sum_update(s, 4);
@@ -512,19 +614,19 @@ void sum_init(int csum_type, int seed)
 	  default: /* paranoia to prevent missing case values */
 		exit_cleanup(RERR_UNSUPPORTED);
 	}
+
+	return cur_sum_len;
 }
 
-/**
- * Feed data into an MD4 accumulator, md.  The results may be
- * retrieved using sum_end().  md is used for different purposes at
- * different points during execution.
- *
- * @todo Perhaps get rid of md and just pass in the address each time.
- * Very slightly clearer and slower.
- **/
+/* Feed data into a hash digest accumulator. */
 void sum_update(const char *p, int32 len)
 {
-	switch (cursum_type) {
+#ifdef USE_OPENSSL
+	if (cur_sum_evp_md) {
+		EVP_DigestUpdate(ctx_evp, (uchar *)p, len);
+	} else
+#endif
+	switch (cur_sum_nni->num) {
 #ifdef SUPPORT_XXHASH
 	  case CSUM_XXH64:
 		XXH64_update(xxh64_state, p, len);
@@ -539,39 +641,35 @@ void sum_update(const char *p, int32 len)
 		break;
 #endif
 	  case CSUM_MD5:
-		md5_update(&ctx.m5, (uchar *)p, len);
+		md5_update(&ctx_md, (uchar *)p, len);
 		break;
 	  case CSUM_MD4:
-#ifdef USE_OPENSSL
-		MD4_Update(&ctx.m4, (uchar *)p, len);
-		break;
-#endif
 	  case CSUM_MD4_OLD:
 	  case CSUM_MD4_BUSTED:
 	  case CSUM_MD4_ARCHAIC:
 		if (len + sumresidue < CSUM_CHUNK) {
-			memcpy(ctx.md.buffer + sumresidue, p, len);
+			memcpy(ctx_md.buffer + sumresidue, p, len);
 			sumresidue += len;
 			break;
 		}
 
 		if (sumresidue) {
 			int32 i = CSUM_CHUNK - sumresidue;
-			memcpy(ctx.md.buffer + sumresidue, p, i);
-			mdfour_update(&ctx.md, (uchar *)ctx.md.buffer, CSUM_CHUNK);
+			memcpy(ctx_md.buffer + sumresidue, p, i);
+			mdfour_update(&ctx_md, (uchar *)ctx_md.buffer, CSUM_CHUNK);
 			len -= i;
 			p += i;
 		}
 
 		while (len >= CSUM_CHUNK) {
-			mdfour_update(&ctx.md, (uchar *)p, CSUM_CHUNK);
+			mdfour_update(&ctx_md, (uchar *)p, CSUM_CHUNK);
 			len -= CSUM_CHUNK;
 			p += CSUM_CHUNK;
 		}
 
 		sumresidue = len;
 		if (sumresidue)
-			memcpy(ctx.md.buffer, p, sumresidue);
+			memcpy(ctx_md.buffer, p, sumresidue);
 		break;
 	  case CSUM_NONE:
 		break;
@@ -580,13 +678,18 @@ void sum_update(const char *p, int32 len)
 	}
 }
 
-/* NOTE: all the callers of sum_end() pass in a pointer to a buffer that is
- * MAX_DIGEST_LEN in size, so even if the csum-len is shorter than that (i.e.
- * CSUM_MD4_ARCHAIC), we don't have to worry about limiting the data we write
- * into the "sum" buffer. */
-int sum_end(char *sum)
+/* The sum buffer only needs to be as long as the current checksum's digest
+ * len, not MAX_DIGEST_LEN. Note that for CSUM_MD4_ARCHAIC that is the full
+ * MD4_DIGEST_LEN even if the file-list code is going to ignore all but the
+ * first 2 bytes of it. */
+void sum_end(char *sum)
 {
-	switch (cursum_type) {
+#ifdef USE_OPENSSL
+	if (cur_sum_evp_md) {
+		EVP_DigestFinal_ex(ctx_evp, (uchar *)sum, NULL);
+	} else
+#endif
+	switch (cur_sum_nni->num) {
 #ifdef SUPPORT_XXHASH
 	  case CSUM_XXH64:
 		SIVAL64(sum, 0, XXH64_digest(xxh64_state));
@@ -604,22 +707,18 @@ int sum_end(char *sum)
 	  }
 #endif
 	  case CSUM_MD5:
-		md5_result(&ctx.m5, (uchar *)sum);
+		md5_result(&ctx_md, (uchar *)sum);
 		break;
 	  case CSUM_MD4:
-#ifdef USE_OPENSSL
-		MD4_Final((uchar *)sum, &ctx.m4);
-		break;
-#endif
 	  case CSUM_MD4_OLD:
-		mdfour_update(&ctx.md, (uchar *)ctx.md.buffer, sumresidue);
-		mdfour_result(&ctx.md, (uchar *)sum);
+		mdfour_update(&ctx_md, (uchar *)ctx_md.buffer, sumresidue);
+		mdfour_result(&ctx_md, (uchar *)sum);
 		break;
 	  case CSUM_MD4_BUSTED:
 	  case CSUM_MD4_ARCHAIC:
 		if (sumresidue)
-			mdfour_update(&ctx.md, (uchar *)ctx.md.buffer, sumresidue);
-		mdfour_result(&ctx.md, (uchar *)sum);
+			mdfour_update(&ctx_md, (uchar *)ctx_md.buffer, sumresidue);
+		mdfour_result(&ctx_md, (uchar *)sum);
 		break;
 	  case CSUM_NONE:
 		*sum = '\0';
@@ -627,34 +726,74 @@ int sum_end(char *sum)
 	  default: /* paranoia to prevent missing case values */
 		exit_cleanup(RERR_UNSUPPORTED);
 	}
-
-	return csum_len_for_type(cursum_type, 0);
 }
 
+#if defined SUPPORT_XXH3 || defined USE_OPENSSL
+static void verify_digest(struct name_num_item *nni, BOOL check_auth_list)
+{
+#ifdef SUPPORT_XXH3
+	static int xxh3_result = 0;
+#endif
+#ifdef USE_OPENSSL
+	static int prior_num = 0, prior_flags = 0, prior_result = 0;
+#endif
+
+#ifdef SUPPORT_XXH3
+	if (nni->num == CSUM_XXH3_64 || nni->num == CSUM_XXH3_128) {
+		if (!xxh3_result) {
+			char buf[32816];
+			int j;
+			for (j = 0; j < (int)sizeof buf; j++)
+				buf[j] = ' ' + (j % 96);
+			sum_init(nni, 0);
+			sum_update(buf, 32816);
+			sum_update(buf, 31152);
+			sum_update(buf, 32474);
+			sum_update(buf, 9322);
+			xxh3_result = XXH3_64bits_digest(xxh3_state) != 0xadbcf16d4678d1de ? -1 : 1;
+		}
+		if (xxh3_result < 0)
+			nni->num = CSUM_gone;
+		return;
+	}
+#endif
+
+#ifdef USE_OPENSSL
+	if (BITS_SETnUNSET(nni->flags, NNI_EVP, NNI_BUILTIN|NNI_EVP_OK)) {
+		if (nni->num == prior_num && nni->flags == prior_flags) {
+			nni->flags = prior_result;
+			if (!(nni->flags & NNI_EVP))
+				nni->num = CSUM_gone;
+		} else {
+			prior_num = nni->num;
+			prior_flags = nni->flags;
+			if (!csum_evp_md(nni))
+				nni->num = CSUM_gone;
+			prior_result = nni->flags;
+			if (check_auth_list && (nni = get_nni_by_num(&valid_auth_checksums, prior_num)) != NULL)
+				verify_digest(nni, False);
+		}
+	}
+#endif
+}
+#endif
+
 void init_checksum_choices()
 {
-#ifdef SUPPORT_XXH3
-	char buf[32816];
-	int j;
-	for (j = 0; j < (int)sizeof buf; j++) {
-		buf[j] = ' ' + (j % 96);
-	}
-	sum_init(CSUM_XXH3_64, 0);
-	sum_update(buf, 32816);
-	sum_update(buf, 31152);
-	sum_update(buf, 32474);
-	sum_update(buf, 9322);
-	if (XXH3_64bits_digest(xxh3_state) != 0xadbcf16d4678d1de) {
-		int t, f;
-		struct name_num_item *nni = valid_checksums.list;
-		for (t = f = 0; nni[f].name; f++) {
-			if (nni[f].num == CSUM_XXH3_64 || nni[f].num == CSUM_XXH3_128)
-				continue;
-			if (t != f)
-				nni[t++] = nni[f];
-		}
-		nni[t].name = NULL;
-	}
+#if defined SUPPORT_XXH3 || defined USE_OPENSSL
+	struct name_num_item *nni;
 #endif
+
+	if (initialized_choices)
+		return;
+
+#if defined SUPPORT_XXH3 || defined USE_OPENSSL
+	for (nni = valid_checksums.list; nni->name; nni++)
+		verify_digest(nni, True);
+
+	for (nni = valid_auth_checksums.list; nni->name; nni++)
+		verify_digest(nni, False);
+#endif
+
 	initialized_choices = 1;
 }

clientserver.c

@@ -67,6 +67,7 @@ extern uid_t our_uid;
 extern gid_t our_gid;
 
 char *auth_user;
+char *daemon_auth_choices;
 int read_only = 0;
 int module_id = -1;
 int pid_file_fd = -1;
@@ -149,13 +150,9 @@ int start_socket_client(char *host, int remote_argc, char *remote_argv[],
 static int exchange_protocols(int f_in, int f_out, char *buf, size_t bufsiz, int am_client)
 {
 	int remote_sub = -1;
-#if SUBPROTOCOL_VERSION != 0
-	int our_sub = protocol_version < PROTOCOL_VERSION ? 0 : SUBPROTOCOL_VERSION;
-#else
-	int our_sub = 0;
-#endif
+	int our_sub = get_subprotocol_version();
 
-	io_printf(f_out, "@RSYNCD: %d.%d\n", protocol_version, our_sub);
+	output_daemon_greeting(f_out, am_client);
 	if (!am_client) {
 		char *motd = lp_motd_file();
 		if (motd && *motd) {
@@ -187,16 +184,30 @@ static int exchange_protocols(int f_in, int f_out, char *buf, size_t bufsiz, int
 	}
 
 	if (remote_sub < 0) {
-		if (remote_protocol == 30) {
+		if (remote_protocol >= 30) {
 			if (am_client)
-				rprintf(FERROR, "rsync: server is speaking an incompatible beta of protocol 30\n");
+				rprintf(FERROR, "rsync: the server omitted the subprotocol value: %s\n", buf);
 			else
-				io_printf(f_out, "@ERROR: your client is speaking an incompatible beta of protocol 30\n");
+				io_printf(f_out, "@ERROR: your client omitted the subprotocol value: %s\n", buf);
 			return -1;
 		}
 		remote_sub = 0;
 	}
 
+	daemon_auth_choices = strchr(buf + 9, ' ');
+	if (daemon_auth_choices) {
+		char *cp;
+		daemon_auth_choices = strdup(daemon_auth_choices + 1);
+		if ((cp = strchr(daemon_auth_choices, '\n')) != NULL)
+			*cp = '\0';
+	} else if (remote_protocol > 31) {
+		if (am_client)
+			rprintf(FERROR, "rsync: the server omitted the digest name list: %s\n", buf);
+		else
+			io_printf(f_out, "@ERROR: your client omitted the digest name list: %s\n", buf);
+		return -1;
+	}
+
 	if (protocol_version > remote_protocol) {
 		protocol_version = remote_protocol;
 		if (remote_sub)
@@ -429,7 +440,7 @@ static int read_arg_from_pipe(int fd, char *buf, int limit)
 }
 #endif
 
-static void set_env_str(const char *var, const char *str)
+void set_env_str(const char *var, const char *str)
 {
 #ifdef HAVE_SETENV
 	if (setenv(var, str, 1) < 0)
@@ -690,7 +701,7 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
 	int set_uid;
 	char *p, *err_msg = NULL;
 	char *name = lp_name(i);
-	int use_chroot = lp_use_chroot(i);
+	int use_chroot = lp_use_chroot(i); /* might be 1 (yes), 0 (no), or -1 (unset) */
 	int ret, pre_exec_arg_fd = -1, pre_exec_error_fd = -1;
 	int save_munge_symlinks;
 	pid_t pre_exec_pid = 0;
@@ -815,6 +826,20 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
 		io_printf(f_out, "@ERROR: no path setting.\n");
 		return -1;
 	}
+	if (use_chroot < 0) {
+		if (strstr(module_dir, "/./") != NULL)
+			use_chroot = 1; /* The module is expecting a chroot inner & outer path. */
+		else if (chroot("/") < 0) {
+			rprintf(FLOG, "chroot test failed: %s. "
+				      "Switching 'use chroot' from unset to false.\n",
+				      strerror(errno));
+			use_chroot = 0;
+		} else {
+			if (chdir("/") < 0)
+			    rsyserr(FLOG, errno, "chdir(\"/\") failed");
+			use_chroot = 1;
+		}
+	}
 	if (use_chroot) {
 		if ((p = strstr(module_dir, "/./")) != NULL) {
 			*p = '\0'; /* Temporary... */
@@ -951,20 +976,8 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
 	}
 
 	if (use_chroot) {
-		/*
-		 * XXX: The 'use chroot' flag is a fairly reliable
-		 * source of confusion, because it fails under two
-		 * important circumstances: running as non-root,
-		 * running on Win32 (or possibly others).  On the
-		 * other hand, if you are running as root, then it
-		 * might be better to always use chroot.
-		 *
-		 * So, perhaps if we can't chroot we should just issue
-		 * a warning, unless a "require chroot" flag is set,
-		 * in which case we fail.
-		 */
 		if (chroot(module_chdir)) {
-			rsyserr(FLOG, errno, "chroot %s failed", module_chdir);
+			rsyserr(FLOG, errno, "chroot(\"%s\") failed", module_chdir);
 			io_printf(f_out, "@ERROR: chroot failed\n");
 			return -1;
 		}
@@ -973,7 +986,7 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char
 
 	if (!change_dir(module_chdir, CD_NORMAL))
 		return path_failure(f_out, module_chdir, True);
-	if (module_dirlen || (!use_chroot && !*lp_daemon_chroot()))
+	if (module_dirlen)
 		sanitize_paths = 1;
 
 	if ((munge_symlinks = lp_munge_symlinks(module_id)) < 0)
@@ -1288,8 +1301,12 @@ int start_daemon(int f_in, int f_out)
 	p = lp_daemon_chroot();
 	if (*p) {
 		log_init(0); /* Make use we've initialized syslog before chrooting. */
-		if (chroot(p) < 0 || chdir("/") < 0) {
-			rsyserr(FLOG, errno, "daemon chroot %s failed", p);
+		if (chroot(p) < 0) {
+			rsyserr(FLOG, errno, "daemon chroot(\"%s\") failed", p);
+			return -1;
+		}
+		if (chdir("/") < 0) {
+			rsyserr(FLOG, errno, "daemon chdir(\"/\") failed");
 			return -1;
 		}
 	}

compat.c

@@ -60,13 +60,16 @@ extern char *files_from;
 extern char *filesfrom_host;
 extern const char *checksum_choice;
 extern const char *compress_choice;
+extern char *daemon_auth_choices;
 extern filter_rule_list filter_list;
 extern int need_unsorted_flist;
 #ifdef ICONV_OPTION
 extern iconv_t ic_send, ic_recv;
 extern char *iconv_opt;
 #endif
-extern struct name_num_obj valid_checksums;
+extern struct name_num_obj valid_checksums, valid_auth_checksums;
+
+extern struct name_num_item *xfer_sum_nni;
 
 int remote_protocol = 0;
 int file_extra_cnt = 0; /* count of file-list extras that everyone gets */
@@ -79,6 +82,9 @@ int inplace_partial = 0;
 int do_negotiated_strings = 0;
 int xmit_id0_names = 0;
 
+struct name_num_item *xattr_sum_nni;
+int xattr_sum_len = 0;
+
 /* These index values are for the file-list's extra-attribute array. */
 int pathname_ndx, depth_ndx, atimes_ndx, crtimes_ndx, uid_ndx, gid_ndx, acls_ndx, xattrs_ndx, unsort_ndx;
 
@@ -93,19 +99,19 @@ int filesfrom_convert = 0;
 
 struct name_num_item valid_compressions_items[] = {
 #ifdef SUPPORT_ZSTD
-	{ CPRES_ZSTD, "zstd", NULL },
+	{ CPRES_ZSTD, 0, "zstd", NULL },
 #endif
 #ifdef SUPPORT_LZ4
-	{ CPRES_LZ4, "lz4", NULL },
+	{ CPRES_LZ4, 0, "lz4", NULL },
 #endif
-	{ CPRES_ZLIBX, "zlibx", NULL },
-	{ CPRES_ZLIB, "zlib", NULL },
-	{ CPRES_NONE, "none", NULL },
-	{ 0, NULL, NULL }
+	{ CPRES_ZLIBX, 0, "zlibx", NULL },
+	{ CPRES_ZLIB, 0, "zlib", NULL },
+	{ CPRES_NONE, 0, "none", NULL },
+	{ 0, 0, NULL, NULL }
 };
 
 struct name_num_obj valid_compressions = {
-	"compress", NULL, NULL, 0, 0, valid_compressions_items
+	"compress", NULL, 0, 0, valid_compressions_items
 };
 
 #define CF_INC_RECURSE	 (1<<0)
@@ -127,11 +133,7 @@ static void check_sub_protocol(void)
 {
 	char *dot;
 	int their_protocol, their_sub;
-#if SUBPROTOCOL_VERSION != 0
-	int our_sub = protocol_version < PROTOCOL_VERSION ? 0 : SUBPROTOCOL_VERSION;
-#else
-	int our_sub = 0;
-#endif
+	int our_sub = get_subprotocol_version();
 
 	/* client_info starts with VER.SUB string if client is a pre-release. */
 	if (!(their_protocol = atoi(client_info))
@@ -178,8 +180,8 @@ void set_allow_inc_recurse(void)
 
 void parse_compress_choice(int final_call)
 {
-	if (valid_compressions.negotiated_name)
-		do_compression = valid_compressions.negotiated_num;
+	if (valid_compressions.negotiated_nni)
+		do_compression = valid_compressions.negotiated_nni->num;
 	else if (compress_choice) {
 		struct name_num_item *nni = get_nni_by_name(&valid_compressions, compress_choice, -1);
 		if (!nni) {
@@ -201,8 +203,8 @@ void parse_compress_choice(int final_call)
 		compress_choice = NULL;
 
 	/* Snag the compression name for both write_batch's option output & the following debug output. */
-	if (valid_compressions.negotiated_name)
-		compress_choice = valid_compressions.negotiated_name;
+	if (valid_compressions.negotiated_nni)
+		compress_choice = valid_compressions.negotiated_nni->name;
 	else if (compress_choice == NULL) {
 		struct name_num_item *nni = get_nni_by_num(&valid_compressions, do_compression);
 		compress_choice = nni ? nni->name : "UNKNOWN";
@@ -212,7 +214,7 @@ void parse_compress_choice(int final_call)
 	 && (do_compression != CPRES_NONE || do_compression_level != CLVL_NOT_SPECIFIED)) {
 		rprintf(FINFO, "%s%s compress: %s (level %d)\n",
 			am_server ? "Server" : "Client",
-			valid_compressions.negotiated_name ? " negotiated" : "",
+			valid_compressions.negotiated_nni ? " negotiated" : "",
 			compress_choice, do_compression_level);
 	}
 }
@@ -225,6 +227,8 @@ struct name_num_item *get_nni_by_name(struct name_num_obj *nno, const char *name
 		len = strlen(name);
 
 	for (nni = nno->list; nni->name; nni++) {
+		if (nni->num == CSUM_gone)
+			continue;
 		if (strncasecmp(name, nni->name, len) == 0 && nni->name[len] == '\0')
 			return nni;
 	}
@@ -259,10 +263,12 @@ static void init_nno_saw(struct name_num_obj *nno, int val)
 	if (!nno->saw) {
 		nno->saw = new_array0(uchar, nno->saw_len);
 
-		/* We'll take this opportunity to make sure that the main_name values are set right. */
+		/* We'll take this opportunity to set the main_nni values for duplicates. */
 		for (cnt = 1, nni = nno->list; nni->name; nni++, cnt++) {
+			if (nni->num == CSUM_gone)
+				continue;
 			if (nno->saw[nni->num])
-				nni->main_name = nno->list[nno->saw[nni->num]-1].name;
+				nni->main_nni = &nno->list[nno->saw[nni->num]-1];
 			else
 				nno->saw[nni->num] = cnt;
 		}
@@ -288,8 +294,8 @@ static int parse_nni_str(struct name_num_obj *nno, const char *from, char *tobuf
 				struct name_num_item *nni = get_nni_by_name(nno, tok, to - tok);
 				if (nni && !nno->saw[nni->num]) {
 					nno->saw[nni->num] = ++cnt;
-					if (nni->main_name) {
-						to = tok + strlcpy(tok, nni->main_name, tobuf_len - (tok - tobuf));
+					if (nni->main_nni) {
+						to = tok + strlcpy(tok, nni->main_nni->name, tobuf_len - (tok - tobuf));
 						if (to - tobuf >= tobuf_len) {
 							to = tok - 1;
 							break;
@@ -323,13 +329,44 @@ static int parse_nni_str(struct name_num_obj *nno, const char *from, char *tobuf
 	return to - tobuf;
 }
 
+static int parse_negotiate_str(struct name_num_obj *nno, char *tmpbuf)
+{
+	struct name_num_item *nni, *ret = NULL;
+	int best = nno->saw_len; /* We want best == 1 from the client list, so start with a big number. */
+	char *space, *tok = tmpbuf;
+	while (tok) {
+		while (*tok == ' ') tok++; /* Should be unneeded... */
+		if (!*tok)
+			break;
+		if ((space = strchr(tok, ' ')) != NULL)
+			*space = '\0';
+		nni = get_nni_by_name(nno, tok, -1);
+		if (space) {
+			*space = ' ';
+			tok = space + 1;
+		} else
+			tok = NULL;
+		if (!nni || !nno->saw[nni->num] || best <= nno->saw[nni->num])
+			continue;
+		ret = nni;
+		best = nno->saw[nni->num];
+		if (best == 1 || am_server) /* The server side stops at the first acceptable client choice */
+			break;
+	}
+	if (ret) {
+		free(nno->saw);
+		nno->saw = NULL;
+		nno->negotiated_nni = ret->main_nni ? ret->main_nni : ret;
+		return 1;
+	}
+	return 0;
+}
+
 /* This routine is always called with a tmpbuf of MAX_NSTR_STRLEN length, but the
  * buffer may be pre-populated with a "len" length string to use OR a len of -1
  * to tell us to read a string from the fd. */
 static void recv_negotiate_str(int f_in, struct name_num_obj *nno, char *tmpbuf, int len)
 {
-	struct name_num_item *ret = NULL;
-
 	if (len < 0)
 		len = read_vstring(f_in, tmpbuf, MAX_NSTR_STRLEN);
 
@@ -340,37 +377,8 @@ static void recv_negotiate_str(int f_in, struct name_num_obj *nno, char *tmpbuf,
 			rprintf(FINFO, "Server %s list (on client): %s\n", nno->type, tmpbuf);
 	}
 
-	if (len > 0) {
-		struct name_num_item *nni;
-		int best = nno->saw_len; /* We want best == 1 from the client list, so start with a big number. */
-		char *space, *tok = tmpbuf;
-		while (tok) {
-			while (*tok == ' ') tok++; /* Should be unneeded... */
-			if (!*tok)
-				break;
-			if ((space = strchr(tok, ' ')) != NULL)
-				*space = '\0';
-			nni = get_nni_by_name(nno, tok, -1);
-			if (space) {
-				*space = ' ';
-				tok = space + 1;
-			} else
-				tok = NULL;
-			if (!nni || !nno->saw[nni->num] || best <= nno->saw[nni->num])
-				continue;
-			ret = nni;
-			best = nno->saw[nni->num];
-			if (best == 1 || am_server) /* The server side stops at the first acceptable client choice */
-				break;
-		}
-		if (ret) {
-			free(nno->saw);
-			nno->saw = NULL;
-			nno->negotiated_name = ret->main_name ? ret->main_name : ret->name;
-			nno->negotiated_num = ret->num;
-			return;
-		}
-	}
+	if (len > 0 && parse_negotiate_str(nno, tmpbuf))
+		return;
 
 	if (!am_server || !do_negotiated_strings) {
 		char *cp = tmpbuf;
@@ -466,8 +474,10 @@ int get_default_nno_list(struct name_num_obj *nno, char *to_buf, int to_buf_len,
 	init_nno_saw(nno, 0);
 
 	for (nni = nno->list, len = 0; nni->name; nni++) {
-		if (nni->main_name) {
-			if (!dup_markup)
+		if (nni->num == CSUM_gone)
+			continue;
+		if (nni->main_nni) {
+			if (!dup_markup || nni->main_nni->num == CSUM_gone)
 				continue;
 			delim = dup_markup;
 		}
@@ -556,7 +566,7 @@ static void negotiate_the_strings(int f_in, int f_out)
 	/* If the other side is too old to negotiate, the above steps just made sure that
 	 * the env didn't disallow the old algorithm. Mark things as non-negotiated. */
 	if (!do_negotiated_strings)
-		valid_checksums.negotiated_name = valid_compressions.negotiated_name = NULL;
+		valid_checksums.negotiated_nni = valid_compressions.negotiated_nni = NULL;
 }
 
 void setup_protocol(int f_out,int f_in)
@@ -805,11 +815,73 @@ void setup_protocol(int f_out,int f_in)
 		checksum_seed = read_int(f_in);
 	}
 
-	parse_checksum_choice(1); /* Sets checksum_type & xfersum_type */
+	parse_checksum_choice(1); /* Sets file_sum_nni & xfer_sum_nni */
 	parse_compress_choice(1); /* Sets do_compression */
 
+	/* TODO in the future allow this algorithm to be chosen somehow, but it can't get too
+	 * long or the size starts to cause a problem in the xattr abbrev/non-abbrev code. */
+	xattr_sum_nni = parse_csum_name(NULL, 0);
+	xattr_sum_len = csum_len_for_type(xattr_sum_nni->num, 0);
+
 	if (write_batch && !am_server)
 		write_batch_shell_file();
 
 	init_flist();
 }
+
+void output_daemon_greeting(int f_out, int am_client)
+{
+	char tmpbuf[MAX_NSTR_STRLEN];
+	int our_sub = get_subprotocol_version();
+
+	get_default_nno_list(&valid_auth_checksums, tmpbuf, MAX_NSTR_STRLEN, '\0');
+
+	io_printf(f_out, "@RSYNCD: %d.%d %s\n", protocol_version, our_sub, tmpbuf);
+
+	if (am_client && DEBUG_GTE(NSTR, 2))
+		rprintf(FINFO, "Client %s list (on client): %s\n", valid_auth_checksums.type, tmpbuf);
+}
+
+void negotiate_daemon_auth(int f_out, int am_client)
+{
+	char tmpbuf[MAX_NSTR_STRLEN];
+	int save_am_server = am_server;
+	int md4_is_old = 0;
+
+	if (!am_client)
+		am_server = 1;
+
+	if (daemon_auth_choices)
+		strlcpy(tmpbuf, daemon_auth_choices, MAX_NSTR_STRLEN);
+	else {
+		strlcpy(tmpbuf, protocol_version >= 30 ? "md5" : "md4", MAX_NSTR_STRLEN);
+		md4_is_old = 1;
+	}
+
+	if (am_client) {
+		recv_negotiate_str(-1, &valid_auth_checksums, tmpbuf, strlen(tmpbuf));
+		if (DEBUG_GTE(NSTR, 1)) {
+			rprintf(FINFO, "Client negotiated %s: %s\n", valid_auth_checksums.type,
+				valid_auth_checksums.negotiated_nni->name);
+		}
+	} else {
+		if (!parse_negotiate_str(&valid_auth_checksums, tmpbuf)) {
+			get_default_nno_list(&valid_auth_checksums, tmpbuf, MAX_NSTR_STRLEN, '\0');
+			io_printf(f_out, "@ERROR: your client does not support one of our daemon-auth checksums: %s\n",
+				  tmpbuf);
+			exit_cleanup(RERR_UNSUPPORTED);
+		}
+	}
+	am_server = save_am_server;
+	if (md4_is_old && valid_auth_checksums.negotiated_nni->num == CSUM_MD4)
+		valid_auth_checksums.negotiated_nni->num = CSUM_MD4_OLD;
+}
+
+int get_subprotocol_version()
+{
+#if SUBPROTOCOL_VERSION != 0
+	return protocol_version < PROTOCOL_VERSION ? 0 : SUBPROTOCOL_VERSION;
+#else
+	return 0;
+#endif
+}

configure.ac

@@ -136,6 +136,16 @@ if test x"$GCC" = x"yes"; then
 	CFLAGS="$CFLAGS -Wall -W"
 fi
 
+AC_ARG_WITH(openssl-conf,
+	AS_HELP_STRING([--with-openssl-conf=PATH],[set default OPENSSL_CONF path for rsync]))
+case "$with_openssl_conf" in
+    *[^-/a-zA-Z0-9.,=@+_]*) AC_MSG_ERROR([Invalid path given to --with-openssl-conf]) ;;
+    /*) CFLAGS="$CFLAGS -DSET_OPENSSL_CONF=$with_openssl_conf" ;;
+    no|'') ;;
+    yes) AC_MSG_ERROR([No path given to --with-openssl-conf]) ;;
+    *) AC_MSG_ERROR([Non absolute path given to --with-openssl-conf]) ;;
+esac
+
 AC_ARG_WITH(rrsync,
         AS_HELP_STRING([--with-rrsync],[also install the rrsync script and its manpage]))
 if test x"$with_rrsync" != x"yes"; then

csprotocol.txt

@@ -7,39 +7,54 @@ basically a summary of clientserver.c and authenticate.c.
 This is the protocol used for rsync --daemon; i.e. connections to port
 873 rather than invocations over a remote shell.
 
-When the server accepts a connection, it prints a greeting
+When the server accepts a connection, it prints a newline-terminated
+greeting line:
 
-  @RSYNCD: <version>.<subprotocol>
+  @RSYNCD: <version>.<subprotocol> <digest1> <digestN>
 
-where <version> is the numeric version (see PROTOCOL_VERSION in rsync.h)
-'.' is a literal period, and <subprotocol> is the numeric subprotocol
-version (see SUBPROTOCOL_VERSION -- it will be 0 for final releases).
-Protocols prior to 30 only output <version> alone.  The daemon expects
-to see a similar greeting back from the client.  For protocols prior to
-30, an absent ".<subprotocol>" value is assumed to be 0.  For protocol
-30, an absent value is a fatal error.  The daemon then follows this line
-with a free-format text message-of-the-day (if any is defined).
+The <version> is the numeric version (see PROTOCOL_VERSION in rsync.h)
+The <subprotocol> is the numeric subprotocol version (which is 0 for a
+final protocol version, as the SUBPROTOCOL_VERSION define discusses).
+The <digestN> names are the authentication digest algorithms that the
+daemon supports, listed in order of preference.
+
+An rsync prior to 3.2.7 omits the digest names.  An rsync prior to 3.0.0
+also omits the period and the <subprotocol> value.  Since a final
+protocol has a subprotocol value of 0, a missing subprotocol value is
+assumed to be 0 for any protocol prior to 30.  It is considered a fatal
+error for protocol 30 and above to omit it.  It is considered a fatal
+error for protocol 32 and above to omit the digest name list (currently
+31 is the newest protocol).
+
+The daemon expects to see a similar greeting line back from the client.
+Once received, the daemon follows the opening line with a free-format
+text message-of-the-day (if any is defined).
 
 The server is now in the connected state.  The client can either send
-the command
+the command:
 
   #list
 
-to get a listing of modules, or the name of a module.  After this, the
+(to get a listing of modules) or the name of a module.  After this, the
 connection is now bound to a particular module.  Access per host for
 this module is now checked, as is per-module connection limits.
 
-If authentication is required to use this module, the server will say
+If authentication is required to use this module, the server will say:
 
   @RSYNCD: AUTHREQD <challenge>
 
 where <challenge> is a random string of base64 characters.  The client
-must respond with
+must respond with:
 
   <user> <response>
 
-where <user> is the username they claim to be, and <response> is the
-base64 form of the MD4 hash of challenge+password.
+The <user> is the username they claim to be. The <response> is the
+base64 form of the digest hash of the challenge+password string. The
+chosen digest method is the most preferred client method that is also in
+the server's list.  If no digest list was explicitly provided, the side
+expecting a list assumes the other side provided either the single name
+"md5" (for a negotiated protocol 30 or 31), or the single name "md4"
+(for an older protocol).
 
 At this point the server applies all remaining constraints before
 handing control to the client, including switching uid/gid, setting up
@@ -76,6 +91,13 @@ stay tuned (or write it yourself!).
 ------------
 Protocol version changes
 
+31	(2013-09-28, 3.1.0)
+
+	Initial release of protocol 31 had no changes.  Rsync 3.2.7
+	introduced the suffixed list of digest names on the greeting
+	line.  The presence of the list is allowed even if the greeting
+	indicates an older protocol version number.
+
 30	(2007-10-04, 3.0.0pre1)
 
 	The use of a ".<subprotocol>" number was added to

daemon-parm.txt

@@ -60,9 +60,9 @@ BOOL	read_only		True
 BOOL	reverse_lookup		True
 BOOL	strict_modes		True
 BOOL	transfer_logging	False
-BOOL	use_chroot		True
 BOOL	write_only		False
 
 BOOL3	munge_symlinks		Unset
 BOOL3	numeric_ids		Unset
 BOOL3	open_noatime		Unset
+BOOL3	use_chroot		Unset

exclude.c

@@ -78,6 +78,10 @@ static filter_rule **mergelist_parents;
 static int mergelist_cnt = 0;
 static int mergelist_size = 0;
 
+#define LOCAL_RULE   1
+#define REMOTE_RULE  2
+static uchar cur_elide_value = REMOTE_RULE;
+
 /* Each filter_list_struct describes a singly-linked list by keeping track
  * of both the head and tail pointers.  The list is slightly unusual in that
  * a parent-dir's content can be appended to the end of the local list in a
@@ -220,6 +224,7 @@ static void add_rule(filter_rule_list *listp, const char *pat, unsigned int pat_
 				slash_cnt++;
 		}
 	}
+	rule->elide = 0;
 	strlcpy(rule->pattern + pre_len, pat, pat_len + 1);
 	pat_len += pre_len;
 	if (suf_len) {
@@ -488,10 +493,11 @@ void add_implied_include(const char *arg, int skip_daemon_module)
 			if (saw_live_open_brkt)
 				maybe_add_literal_brackets_rule(rule, arg_len);
 			if (relative_paths && slash_cnt) {
-				filter_rule const *ent;
-				int found = 0;
-				slash_cnt = 1;
-				for (p = new_pat + 1; (p = strchr(p, '/')) != NULL; p++) {
+				int sub_slash_cnt = slash_cnt;
+				while ((p = strrchr(new_pat, '/')) != NULL && p != new_pat) {
+					filter_rule const *ent;
+					filter_rule *R_rule;
+					int found = 0;
 					*p = '\0';
 					for (ent = implied_filter_list.head; ent; ent = ent->next) {
 						if (ent != rule && strcmp(ent->pattern, new_pat) == 0) {
@@ -499,25 +505,29 @@ void add_implied_include(const char *arg, int skip_daemon_module)
 							break;
 						}
 					}
-					if (!found) {
-						filter_rule *R_rule = new0(filter_rule);
-						R_rule->rflags = FILTRULE_INCLUDE | FILTRULE_DIRECTORY;
-						/* Check if our sub-path has wildcards or escaped backslashes */
-						if (saw_wild && strpbrk(rule->pattern, "*[?\\"))
-							R_rule->rflags |= FILTRULE_WILD;
-						R_rule->pattern = strdup(new_pat);
-						R_rule->u.slash_cnt = slash_cnt;
-						R_rule->next = implied_filter_list.head;
-						implied_filter_list.head = R_rule;
-						if (DEBUG_GTE(FILTER, 3)) {
-							rprintf(FINFO, "[%s] add_implied_include(%s/)\n",
-								who_am_i(), R_rule->pattern);
-						}
-						if (saw_live_open_brkt)
-							maybe_add_literal_brackets_rule(R_rule, -1);
+					if (found) {
+						*p = '/';
+						break; /* We added all parent dirs already */
 					}
+					R_rule = new0(filter_rule);
+					R_rule->rflags = FILTRULE_INCLUDE | FILTRULE_DIRECTORY;
+					/* Check if our sub-path has wildcards or escaped backslashes */
+					if (saw_wild && strpbrk(new_pat, "*[?\\"))
+						R_rule->rflags |= FILTRULE_WILD;
+					R_rule->pattern = strdup(new_pat);
+					R_rule->u.slash_cnt = --sub_slash_cnt;
+					R_rule->next = implied_filter_list.head;
+					implied_filter_list.head = R_rule;
+					if (DEBUG_GTE(FILTER, 3)) {
+						rprintf(FINFO, "[%s] add_implied_include(%s/)\n",
+							who_am_i(), R_rule->pattern);
+					}
+					if (saw_live_open_brkt)
+						maybe_add_literal_brackets_rule(R_rule, -1);
+				}
+				for (p = new_pat; sub_slash_cnt < slash_cnt; sub_slash_cnt++) {
+					p += strlen(p);
 					*p = '/';
-					slash_cnt++;
 				}
 			}
 		}
@@ -545,15 +555,12 @@ void add_implied_include(const char *arg, int skip_daemon_module)
 				p += arg_len;
 			}
 		}
-		if (p[-1] != '/') {
-			*p++ = '/';
-			slash_cnt++;
-		}
+		*p++ = '/';
 		*p++ = '*';
 		if (recurse)
 			*p++ = '*';
 		*p = '\0';
-		rule->u.slash_cnt = slash_cnt;
+		rule->u.slash_cnt = slash_cnt + 1;
 		rule->next = implied_filter_list.head;
 		implied_filter_list.head = rule;
 		if (DEBUG_GTE(FILTER, 3))
@@ -900,7 +907,7 @@ static int rule_matches(const char *fname, filter_rule *ex, int name_flags)
 	const char *strings[16]; /* more than enough */
 	const char *name = fname + (*fname == '/');
 
-	if (!*name)
+	if (!*name || ex->elide == cur_elide_value)
 		return 0;
 
 	if (!(name_flags & NAME_IS_XATTR) ^ !(ex->rflags & FILTRULE_XATTR))
@@ -1016,6 +1023,15 @@ int name_is_excluded(const char *fname, int name_flags, int filter_level)
 	return 0;
 }
 
+int check_server_filter(filter_rule_list *listp, enum logcode code, const char *name, int name_flags)
+{
+	int ret;
+	cur_elide_value = LOCAL_RULE;
+	ret = check_filter(listp, code, name, name_flags);
+	cur_elide_value = REMOTE_RULE;
+	return ret;
+}
+
 /* Return -1 if file "name" is defined to be excluded by the specified
  * exclude list, 1 if it is included, and 0 if it was not matched. */
 int check_filter(filter_rule_list *listp, enum logcode code,
@@ -1571,7 +1587,7 @@ char *get_rule_prefix(filter_rule *rule, const char *pat, int for_xfer,
 
 static void send_rules(int f_out, filter_rule_list *flp)
 {
-	filter_rule *ent, *prev = NULL;
+	filter_rule *ent;
 
 	for (ent = flp->head; ent; ent = ent->next) {
 		unsigned int len, plen, dlen;
@@ -1586,21 +1602,15 @@ static void send_rules(int f_out, filter_rule_list *flp)
 		 * merge files as an optimization (since they can only have
 		 * include/exclude rules). */
 		if (ent->rflags & FILTRULE_SENDER_SIDE)
-			elide = am_sender ? 1 : -1;
+			elide = am_sender ? LOCAL_RULE : REMOTE_RULE;
 		if (ent->rflags & FILTRULE_RECEIVER_SIDE)
-			elide = elide ? 0 : am_sender ? -1 : 1;
+			elide = elide ? 0 : am_sender ? REMOTE_RULE : LOCAL_RULE;
 		else if (delete_excluded && !elide
 		 && (!(ent->rflags & FILTRULE_PERDIR_MERGE)
 		  || ent->rflags & FILTRULE_NO_PREFIXES))
-			elide = am_sender ? 1 : -1;
-		if (elide < 0) {
-			if (prev)
-				prev->next = ent->next;
-			else
-				flp->head = ent->next;
-		} else
-			prev = ent;
-		if (elide > 0)
+			elide = am_sender ? LOCAL_RULE : REMOTE_RULE;
+		ent->elide = elide;
+		if (elide == LOCAL_RULE)
 			continue;
 		if (ent->rflags & FILTRULE_CVS_IGNORE
 		    && !(ent->rflags & FILTRULE_MERGE_FILE)) {
@@ -1628,7 +1638,6 @@ static void send_rules(int f_out, filter_rule_list *flp)
 		if (dlen)
 			write_byte(f_out, '/');
 	}
-	flp->tail = prev;
 }
 
 /* This is only called by the client. */

flist.c

@@ -33,7 +33,6 @@ extern int am_sender;
 extern int am_generator;
 extern int inc_recurse;
 extern int always_checksum;
-extern int checksum_type;
 extern int module_id;
 extern int ignore_errors;
 extern int numeric_ids;
@@ -80,6 +79,8 @@ extern struct stats stats;
 extern char *filesfrom_host;
 extern char *usermap, *groupmap;
 
+extern struct name_num_item *file_sum_nni;
+
 extern char curr_dir[MAXPATHLEN];
 
 extern struct chmod_mode_struct *chmod_modes;
@@ -145,7 +146,8 @@ void init_flist(void)
 		rprintf(FINFO, "FILE_STRUCT_LEN=%d, EXTRA_LEN=%d\n",
 			(int)FILE_STRUCT_LEN, (int)EXTRA_LEN);
 	}
-	flist_csum_len = csum_len_for_type(checksum_type, 1);
+	/* Note that this isn't identical to file_sum_len in the case of CSUM_MD4_ARCHAIC: */
+	flist_csum_len = csum_len_for_type(file_sum_nni->num, 1);
 
 	show_filelist_progress = INFO_GTE(FLIST, 1) && xfer_dirs && !am_server && !inc_recurse;
 }
@@ -754,7 +756,7 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x
 	if (*thisname
 	 && (clean_fname(thisname, CFN_REFUSE_DOT_DOT_DIRS) < 0 || (!relative_paths && *thisname == '/'))) {
 		rprintf(FERROR, "ABORTING due to unsafe pathname from sender: %s\n", thisname);
-		exit_cleanup(RERR_PROTOCOL);
+		exit_cleanup(RERR_UNSUPPORTED);
 	}
 
 	if (sanitize_paths)
@@ -986,16 +988,16 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x
 		exit_cleanup(RERR_UNSUPPORTED);
 	}
 
-	if (*thisname != '.' || thisname[1] != '\0') {
+	if (*thisname == '/' ? thisname[1] != '.' || thisname[2] != '\0' : *thisname != '.' || thisname[1] != '\0') {
 		int filt_flags = S_ISDIR(mode) ? NAME_IS_DIR : NAME_IS_FILE;
 		if (!trust_sender_filter /* a per-dir filter rule means we must trust the sender's filtering */
-		 && filter_list.head && check_filter(&filter_list, FINFO, thisname, filt_flags) < 0) {
+		 && filter_list.head && check_server_filter(&filter_list, FINFO, thisname, filt_flags) < 0) {
 			rprintf(FERROR, "ERROR: rejecting excluded file-list name: %s\n", thisname);
-			exit_cleanup(RERR_PROTOCOL);
+			exit_cleanup(RERR_UNSUPPORTED);
 		}
 		if (implied_filter_list.head && check_filter(&implied_filter_list, FINFO, thisname, filt_flags) <= 0) {
 			rprintf(FERROR, "ERROR: rejecting unrequested file-list name: %s\n", thisname);
-			exit_cleanup(RERR_PROTOCOL);
+			exit_cleanup(RERR_UNSUPPORTED);
 		}
 	}
 
@@ -2640,7 +2642,7 @@ struct file_list *recv_file_list(int f, int dir_ndx)
 					rprintf(FERROR,
 						"ABORTING due to invalid path from sender: %s/%s\n",
 						cur_dir, file->basename);
-					exit_cleanup(RERR_PROTOCOL);
+					exit_cleanup(RERR_UNSUPPORTED);
 				}
 				good_dirname = cur_dir;
 			}

generator.c

@@ -875,9 +875,12 @@ static struct file_struct *find_fuzzy(struct file_struct *file, struct file_list
 			len = strlen(name);
 			suf = find_filename_suffix(name, len, &suf_len);
 
-			dist = fuzzy_distance(name, len, fname, fname_len);
-			/* Add some extra weight to how well the suffixes match. */
-			dist += fuzzy_distance(suf, suf_len, fname_suf, fname_suf_len) * 10;
+			dist = fuzzy_distance(name, len, fname, fname_len, lowest_dist);
+			/* Add some extra weight to how well the suffixes match unless we've already disqualified
+			 * this file based on a heuristic. */
+			if (dist < 0xFFFF0000U) {
+				dist += fuzzy_distance(suf, suf_len, fname_suf, fname_suf_len, 0xFFFF0000U) * 10;
+			}
 			if (DEBUG_GTE(FUZZY, 2)) {
 				rprintf(FINFO, "fuzzy distance for %s = %d.%05d\n",
 					f_name(fp, NULL), (int)(dist>>16), (int)(dist&0xFFFF));

hashtable.c

@@ -1,7 +1,7 @@
 /*
  * Routines to provide a memory-efficient hashtable.
  *
- * Copyright (C) 2007-2020 Wayne Davison
+ * Copyright (C) 2007-2022 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -350,6 +350,9 @@ void *hashtable_find(struct hashtable *tbl, int64 key, void *data_when_new)
  -------------------------------------------------------------------------------
 */
 
+#define NON_ZERO_32(x) ((x) ? (x) : (uint32_t)1)
+#define NON_ZERO_64(x, y) ((x) || (y) ? (y) | (int64)(x) << 32 | (y) : (int64)1)
+
 uint32_t hashlittle(const void *key, size_t length)
 {
   uint32_t a,b,c;                                          /* internal state */
@@ -390,7 +393,7 @@ uint32_t hashlittle(const void *key, size_t length)
     case 3 : a+=((uint32_t)k8[2])<<16;   /* fall through */
     case 2 : a+=((uint32_t)k8[1])<<8;    /* fall through */
     case 1 : a+=k8[0]; break;
-    case 0 : return c;
+    case 0 : return NON_ZERO_32(c);
     }
   } else if (HASH_LITTLE_ENDIAN && ((u.i & 0x1) == 0)) {
     const uint16_t *k = (const uint16_t *)key;         /* read 16-bit chunks */
@@ -436,7 +439,7 @@ uint32_t hashlittle(const void *key, size_t length)
              break;
     case 1 : a+=k8[0];
              break;
-    case 0 : return c;                     /* zero length requires no mixing */
+    case 0 : return NON_ZERO_32(c);         /* zero length requires no mixing */
     }
 
   } else {                        /* need to read the key one byte at a time */
@@ -489,10 +492,171 @@ uint32_t hashlittle(const void *key, size_t length)
 	     /* FALLTHROUGH */
     case 1 : a+=k[0];
              break;
-    case 0 : return c;
+    case 0 : return NON_ZERO_32(c);
     }
   }
 
   final(a,b,c);
-  return c;
+  return NON_ZERO_32(c);
 }
+
+#if SIZEOF_INT64 >= 8
+/*
+ * hashlittle2: return 2 32-bit hash values joined into an int64.
+ *
+ * This is identical to hashlittle(), except it returns two 32-bit hash
+ * values instead of just one.  This is good enough for hash table
+ * lookup with 2^^64 buckets, or if you want a second hash if you're not
+ * happy with the first, or if you want a probably-unique 64-bit ID for
+ * the key.  *pc is better mixed than *pb, so use *pc first.  If you want
+ * a 64-bit value do something like "*pc + (((uint64_t)*pb)<<32)".
+ */
+int64 hashlittle2(const void *key, size_t length)
+{
+  uint32_t a,b,c;                                          /* internal state */
+  union { const void *ptr; size_t i; } u;     /* needed for Mac Powerbook G4 */
+
+  /* Set up the internal state */
+  a = b = c = 0xdeadbeef + ((uint32_t)length);
+
+  u.ptr = key;
+  if (HASH_LITTLE_ENDIAN && ((u.i & 0x3) == 0)) {
+    const uint32_t *k = (const uint32_t *)key;         /* read 32-bit chunks */
+    const uint8_t  *k8;
+
+    /*------ all but last block: aligned reads and affect 32 bits of (a,b,c) */
+    while (length > 12)
+    {
+      a += k[0];
+      b += k[1];
+      c += k[2];
+      mix(a,b,c);
+      length -= 12;
+      k += 3;
+    }
+
+    /*----------------------------- handle the last (probably partial) block */
+    k8 = (const uint8_t *)k;
+    switch(length)
+    {
+    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;
+    case 11: c+=((uint32_t)k8[10])<<16;  /* fall through */
+    case 10: c+=((uint32_t)k8[9])<<8;    /* fall through */
+    case 9 : c+=k8[8];                   /* fall through */
+    case 8 : b+=k[1]; a+=k[0]; break;
+    case 7 : b+=((uint32_t)k8[6])<<16;   /* fall through */
+    case 6 : b+=((uint32_t)k8[5])<<8;    /* fall through */
+    case 5 : b+=k8[4];                   /* fall through */
+    case 4 : a+=k[0]; break;
+    case 3 : a+=((uint32_t)k8[2])<<16;   /* fall through */
+    case 2 : a+=((uint32_t)k8[1])<<8;    /* fall through */
+    case 1 : a+=k8[0]; break;
+    case 0 : return NON_ZERO_64(b, c);
+    }
+  } else if (HASH_LITTLE_ENDIAN && ((u.i & 0x1) == 0)) {
+    const uint16_t *k = (const uint16_t *)key;         /* read 16-bit chunks */
+    const uint8_t  *k8;
+
+    /*--------------- all but last block: aligned reads and different mixing */
+    while (length > 12)
+    {
+      a += k[0] + (((uint32_t)k[1])<<16);
+      b += k[2] + (((uint32_t)k[3])<<16);
+      c += k[4] + (((uint32_t)k[5])<<16);
+      mix(a,b,c);
+      length -= 12;
+      k += 6;
+    }
+
+    /*----------------------------- handle the last (probably partial) block */
+    k8 = (const uint8_t *)k;
+    switch(length)
+    {
+    case 12: c+=k[4]+(((uint32_t)k[5])<<16);
+             b+=k[2]+(((uint32_t)k[3])<<16);
+             a+=k[0]+(((uint32_t)k[1])<<16);
+             break;
+    case 11: c+=((uint32_t)k8[10])<<16;     /* fall through */
+    case 10: c+=k[4];
+             b+=k[2]+(((uint32_t)k[3])<<16);
+             a+=k[0]+(((uint32_t)k[1])<<16);
+             break;
+    case 9 : c+=k8[8];                      /* fall through */
+    case 8 : b+=k[2]+(((uint32_t)k[3])<<16);
+             a+=k[0]+(((uint32_t)k[1])<<16);
+             break;
+    case 7 : b+=((uint32_t)k8[6])<<16;      /* fall through */
+    case 6 : b+=k[2];
+             a+=k[0]+(((uint32_t)k[1])<<16);
+             break;
+    case 5 : b+=k8[4];                      /* fall through */
+    case 4 : a+=k[0]+(((uint32_t)k[1])<<16);
+             break;
+    case 3 : a+=((uint32_t)k8[2])<<16;      /* fall through */
+    case 2 : a+=k[0];
+             break;
+    case 1 : a+=k8[0];
+             break;
+    case 0 : return NON_ZERO_64(b, c);  /* zero length strings require no mixing */
+    }
+
+  } else {                        /* need to read the key one byte at a time */
+    const uint8_t *k = (const uint8_t *)key;
+
+    /*--------------- all but the last block: affect some 32 bits of (a,b,c) */
+    while (length > 12)
+    {
+      a += k[0];
+      a += ((uint32_t)k[1])<<8;
+      a += ((uint32_t)k[2])<<16;
+      a += ((uint32_t)k[3])<<24;
+      b += k[4];
+      b += ((uint32_t)k[5])<<8;
+      b += ((uint32_t)k[6])<<16;
+      b += ((uint32_t)k[7])<<24;
+      c += k[8];
+      c += ((uint32_t)k[9])<<8;
+      c += ((uint32_t)k[10])<<16;
+      c += ((uint32_t)k[11])<<24;
+      mix(a,b,c);
+      length -= 12;
+      k += 12;
+    }
+
+    /*-------------------------------- last block: affect all 32 bits of (c) */
+    switch(length)                   /* all the case statements fall through */
+    {
+    case 12: c+=((uint32_t)k[11])<<24;
+	     /* FALLTHROUGH */
+    case 11: c+=((uint32_t)k[10])<<16;
+	     /* FALLTHROUGH */
+    case 10: c+=((uint32_t)k[9])<<8;
+	     /* FALLTHROUGH */
+    case 9 : c+=k[8];
+	     /* FALLTHROUGH */
+    case 8 : b+=((uint32_t)k[7])<<24;
+	     /* FALLTHROUGH */
+    case 7 : b+=((uint32_t)k[6])<<16;
+	     /* FALLTHROUGH */
+    case 6 : b+=((uint32_t)k[5])<<8;
+	     /* FALLTHROUGH */
+    case 5 : b+=k[4];
+	     /* FALLTHROUGH */
+    case 4 : a+=((uint32_t)k[3])<<24;
+	     /* FALLTHROUGH */
+    case 3 : a+=((uint32_t)k[2])<<16;
+	     /* FALLTHROUGH */
+    case 2 : a+=((uint32_t)k[1])<<8;
+	     /* FALLTHROUGH */
+    case 1 : a+=k[0];
+             break;
+    case 0 : return NON_ZERO_64(b, c);
+    }
+  }
+
+  final(a,b,c);
+  return NON_ZERO_64(b, c);
+}
+#else
+#define hashlittle2(key, len) hashlittle(key, len)
+#endif

lib/md-defines.h

@@ -1,11 +1,28 @@
 /* Keep this simple so both C and ASM can use it */
 
+/* These allow something like CFLAGS=-DDISABLE_SHA512_DIGEST */
+#ifdef DISABLE_SHA256_DIGEST
+#undef SHA256_DIGEST_LENGTH
+#endif
+#ifdef DISABLE_SHA512_DIGEST
+#undef SHA512_DIGEST_LENGTH
+#endif
+
 #define MD4_DIGEST_LEN 16
 #define MD5_DIGEST_LEN 16
+#if defined SHA512_DIGEST_LENGTH
+#define MAX_DIGEST_LEN SHA512_DIGEST_LENGTH
+#elif defined SHA256_DIGEST_LENGTH
+#define MAX_DIGEST_LEN SHA256_DIGEST_LENGTH
+#elif defined SHA_DIGEST_LENGTH
+#define MAX_DIGEST_LEN SHA_DIGEST_LENGTH
+#else
 #define MAX_DIGEST_LEN MD5_DIGEST_LEN
+#endif
 
 #define CSUM_CHUNK 64
 
+#define CSUM_gone -1
 #define CSUM_NONE 0
 #define CSUM_MD4_ARCHAIC 1
 #define CSUM_MD4_BUSTED 2
@@ -15,3 +32,6 @@
 #define CSUM_XXH64 6
 #define CSUM_XXH3_64 7
 #define CSUM_XXH3_128 8
+#define CSUM_SHA1 9
+#define CSUM_SHA256 10
+#define CSUM_SHA512 11

lib/md5.c

@@ -20,7 +20,6 @@
 
 #include "rsync.h"
 
-#if !defined USE_OPENSSL || USE_MD5_ASM /* { */
 void md5_begin(md_context *ctx)
 {
 	ctx->A = 0x67452301;
@@ -224,7 +223,6 @@ void md5_result(md_context *ctx, uchar digest[MD5_DIGEST_LEN])
 	SIVALu(digest, 8, ctx->C);
 	SIVALu(digest, 12, ctx->D);
 }
-#endif /* } */
 
 #ifdef TEST_MD5 /* { */
 

lib/mdigest.h

@@ -1,8 +1,8 @@
 /* The include file for both the MD4 and MD5 routines. */
 
 #ifdef USE_OPENSSL
-#include "openssl/md4.h"
-#include "openssl/md5.h"
+#include <openssl/sha.h>
+#include <openssl/evp.h>
 #endif
 #include "md-defines.h"
 
@@ -17,14 +17,6 @@ void mdfour_begin(md_context *md);
 void mdfour_update(md_context *md, const uchar *in, uint32 length);
 void mdfour_result(md_context *md, uchar digest[MD4_DIGEST_LEN]);
 
-#if defined USE_OPENSSL && !defined USE_MD5_ASM
-#define md5_context MD5_CTX
-#define md5_begin MD5_Init
-#define md5_update MD5_Update
-#define md5_result(cptr, digest) MD5_Final(digest, cptr)
-#else
-#define md5_context md_context
 void md5_begin(md_context *ctx);
 void md5_update(md_context *ctx, const uchar *input, uint32 length);
 void md5_result(md_context *ctx, uchar digest[MD5_DIGEST_LEN]);
-#endif

log.c

@@ -36,8 +36,6 @@ extern int protocol_version;
 extern int always_checksum;
 extern int preserve_mtimes;
 extern int msgs2stderr;
-extern int xfersum_type;
-extern int checksum_type;
 extern int stdout_format_has_i;
 extern int stdout_format_has_o_or_i;
 extern int logfile_format_has_i;
@@ -62,6 +60,8 @@ extern unsigned int module_dirlen;
 extern char sender_file_sum[MAX_DIGEST_LEN];
 extern const char undetermined_hostname[];
 
+extern struct name_num_item *xfer_sum_nni, *file_sum_nni;
+
 static int log_initialised;
 static int logfile_was_closed;
 static FILE *logfile_fp;
@@ -680,12 +680,12 @@ static void log_formatted(enum logcode code, const char *format, const char *op,
 			n = NULL;
 			if (S_ISREG(file->mode)) {
 				if (always_checksum)
-					n = sum_as_hex(checksum_type, F_SUM(file), 1);
+					n = sum_as_hex(file_sum_nni->num, F_SUM(file), 1);
 				else if (iflags & ITEM_TRANSFER)
-					n = sum_as_hex(xfersum_type, sender_file_sum, 0);
+					n = sum_as_hex(xfer_sum_nni->num, sender_file_sum, 0);
 			}
 			if (!n) {
-				int sum_len = csum_len_for_type(always_checksum ? checksum_type : xfersum_type,
+				int sum_len = csum_len_for_type(always_checksum ? file_sum_nni->num : xfer_sum_nni->num,
 								always_checksum);
 				memset(buf2, ' ', sum_len*2);
 				buf2[sum_len*2] = '\0';

main.c

@@ -1743,6 +1743,17 @@ int main(int argc,char *argv[])
 
 	unset_env_var("DISPLAY");
 
+#if defined USE_OPENSSL && defined SET_OPENSSL_CONF
+#define TO_STR2(x) #x
+#define TO_STR(x) TO_STR2(x)
+	/* ./configure --with-openssl-conf=/etc/ssl/openssl-rsync.cnf
+	 * defines SET_OPENSSL_CONF as that unquoted pathname. */
+	if (!getenv("OPENSSL_CONF")) /* Don't override it if it's already set. */
+		set_env_str("OPENSSL_CONF", TO_STR(SET_OPENSSL_CONF));
+#undef TO_STR
+#undef TO_STR2
+#endif
+
 	memset(&stats, 0, sizeof(stats));
 
 	/* Even a non-daemon runs needs the default config values to be set, e.g.

match.c

@@ -3,7 +3,7 @@
  *
  * Copyright (C) 1996 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
- * Copyright (C) 2003-2020 Wayne Davison
+ * Copyright (C) 2003-2022 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -24,7 +24,9 @@
 
 extern int checksum_seed;
 extern int append_mode;
-extern int xfersum_type;
+
+extern struct name_num_item *xfer_sum_nni;
+extern int xfer_sum_len;
 
 int updating_basis_file;
 char sender_file_sum[MAX_DIGEST_LEN];
@@ -356,15 +358,13 @@ static void hash_search(int f,struct sum_struct *s,
  **/
 void match_sums(int f, struct sum_struct *s, struct map_struct *buf, OFF_T len)
 {
-	int sum_len;
-
 	last_match = 0;
 	false_alarms = 0;
 	hash_hits = 0;
 	matches = 0;
 	data_transfer = 0;
 
-	sum_init(xfersum_type, checksum_seed);
+	sum_init(xfer_sum_nni, checksum_seed);
 
 	if (append_mode > 0) {
 		if (append_mode == 2) {
@@ -405,22 +405,22 @@ void match_sums(int f, struct sum_struct *s, struct map_struct *buf, OFF_T len)
 		matched(f, s, buf, len, -1);
 	}
 
-	sum_len = sum_end(sender_file_sum);
+	sum_end(sender_file_sum);
 
 	/* If we had a read error, send a bad checksum.  We use all bits
 	 * off as long as the checksum doesn't happen to be that, in
 	 * which case we turn the last 0 bit into a 1. */
 	if (buf && buf->status != 0) {
 		int i;
-		for (i = 0; i < sum_len && sender_file_sum[i] == 0; i++) {}
-		memset(sender_file_sum, 0, sum_len);
-		if (i == sum_len)
+		for (i = 0; i < xfer_sum_len && sender_file_sum[i] == 0; i++) {}
+		memset(sender_file_sum, 0, xfer_sum_len);
+		if (i == xfer_sum_len)
 			sender_file_sum[i-1]++;
 	}
 
 	if (DEBUG_GTE(DELTASUM, 2))
 		rprintf(FINFO,"sending file_sum\n");
-	write_buf(f, sender_file_sum, sum_len);
+	write_buf(f, sender_file_sum, xfer_sum_len);
 
 	if (DEBUG_GTE(DELTASUM, 2)) {
 		rprintf(FINFO, "false_alarms=%d hash_hits=%d matches=%d\n",

mkgitver

@@ -6,7 +6,7 @@ if [ ! -f git-version.h ]; then
     touch git-version.h
 fi
 
-if [ -e "$srcdir/.git" ]; then
+if test -d "$srcdir/.git" || test -f "$srcdir/.git"; then
     gitver=`git describe --abbrev=8 2>/dev/null | sed -n '/^v3\.[0-9][0-9]*\.[0-9][0-9]*\(-\|$\)/p'`
     if [ -n "$gitver" ]; then
 	echo "#define RSYNC_GITVER \"$gitver\"" >git-version.h.new

options.c

@@ -1926,7 +1926,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		saw_stderr_opt = 1;
 
 	if (version_opt_cnt) {
-		print_rsync_version(FINFO);
+		print_rsync_version(version_opt_cnt > 1 && !am_server ? FNONE : FINFO);
 		exit_cleanup(0);
 	}
 

packaging/lsb/rsync.spec

@@ -1,9 +1,9 @@
 Summary: A fast, versatile, remote (and local) file-copying tool
 Name: rsync
-Version: 3.2.6
-%define fullversion %{version}
-Release: 1
-%define srcdir src
+Version: 3.2.7
+%define fullversion %{version}pre1
+Release: 0.1.pre1
+%define srcdir src-previews
 Group: Applications/Internet
 License: GPL
 Source0: https://rsync.samba.org/ftp/rsync/%{srcdir}/rsync-%{fullversion}.tar.gz
@@ -79,8 +79,8 @@ rm -rf $RPM_BUILD_ROOT
 %dir /etc/rsync-ssl/certs
 
 %changelog
-* Fri Sep 09 2022 Wayne Davison <wayne@opencoder.net>
-Released 3.2.6.
+* Fri Sep 30 2022 Wayne Davison <wayne@opencoder.net>
+Released 3.2.7pre1.
 
 * Fri Mar 21 2008 Wayne Davison <wayne@opencoder.net>
 Added installation of /etc/xinetd.d/rsync file and some commented-out

packaging/openssl-rsync.cnf

@@ -0,0 +1,18 @@
+# This config file can be used with rsync to enable legacy digests
+# (such as MD4) by using the OPENSSL_CONF environment variable.
+# See rsync's configure --with-openssl-conf=/path/name option.
+
+openssl_conf = openssl_init
+
+[openssl_init]
+providers = provider_sect
+
+[provider_sect]
+default = default_sect
+legacy = legacy_sect
+
+[default_sect]
+activate = 1
+
+[legacy_sect]
+activate = 1

packaging/release-rsync

@@ -105,6 +105,8 @@ def main():
         if not re.match(r'^del', ans, flags=re.I):
             die("Aborted")
         cmd_chk(['git', 'tag', '-d', v_ver])
+        if os.path.isdir('patches/.git'):
+            cmd_chk(f"cd patches && git tag -d '{v_ver}'")
 
     version = re.sub(r'[-.]*pre[-.]*', 'pre', version)
     if 'pre' in version and not curversion.endswith('dev'):

receiver.c

@@ -56,7 +56,6 @@ extern int inplace;
 extern int inplace_partial;
 extern int allowed_lull;
 extern int delay_updates;
-extern int xfersum_type;
 extern BOOL want_progress_now;
 extern mode_t orig_umask;
 extern struct stats stats;
@@ -68,6 +67,9 @@ extern struct file_list *cur_flist, *first_flist, *dir_flist;
 extern filter_rule_list daemon_filter_list;
 extern OFF_T preallocated_len;
 
+extern struct name_num_item *xfer_sum_nni;
+extern int xfer_sum_len;
+
 static struct bitbag *delayed_bits = NULL;
 static int phase = 0, redoing = 0;
 static flist_ndx_list batch_redo_list;
@@ -240,7 +242,6 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 	static char file_sum1[MAX_DIGEST_LEN];
 	struct map_struct *mapbuf;
 	struct sum_struct sum;
-	int sum_len;
 	int32 len;
 	OFF_T total_size = F_LENGTH(file);
 	OFF_T offset = 0;
@@ -280,7 +281,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 	} else
 		mapbuf = NULL;
 
-	sum_init(xfersum_type, checksum_seed);
+	sum_init(xfer_sum_nni, checksum_seed);
 
 	if (append_mode > 0) {
 		OFF_T j;
@@ -393,7 +394,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 	if (INFO_GTE(PROGRESS, 1))
 		end_progress(total_size);
 
-	sum_len = sum_end(file_sum1);
+	sum_end(file_sum1);
 
 	if (do_fsync && fd != -1 && fsync(fd) != 0) {
 		rsyserr(FERROR, errno, "fsync failed on %s", full_fname(fname));
@@ -403,10 +404,10 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 	if (mapbuf)
 		unmap_file(mapbuf);
 
-	read_buf(f_in, sender_file_sum, sum_len);
+	read_buf(f_in, sender_file_sum, xfer_sum_len);
 	if (DEBUG_GTE(DELTASUM, 2))
 		rprintf(FINFO,"got file_sum\n");
-	if (fd != -1 && memcmp(file_sum1, sender_file_sum, sum_len) != 0)
+	if (fd != -1 && memcmp(file_sum1, sender_file_sum, xfer_sum_len) != 0)
 		return 0;
 	return 1;
 }

rsync.1.md

@@ -580,11 +580,14 @@ expand it.
 
 0.  `--version`, `-V`
 
-    Print the rsync version plus other info and exit.
+    Print the rsync version plus other info and exit.  When repeated, the
+    information is output is a JSON format that is still fairly readable
+    (client side only).
 
-    The output includes the default list of checksum algorithms, the default
-    list of compression algorithms, a list of compiled-in capabilities, a link
-    to the rsync web site, and some license/copyright info.
+    The output includes a list of compiled-in capabilities, a list of
+    optimizations, the default list of checksum algorithms, the default list of
+    compression algorithms, the default list of daemon auth digests, a link to
+    the rsync web site, and a few other items.
 
 0.  `--verbose`, `-v`
 
@@ -1727,6 +1730,7 @@ expand it.
     - `xxh64` (aka `xxhash`)
     - `md5`
     - `md4`
+    - `sha1`
     - `none`
 
     Run `rsync --version` to see the default checksum list compiled into your

rsync.h

@@ -826,6 +826,7 @@ extern int uid_ndx;
 extern int gid_ndx;
 extern int acls_ndx;
 extern int xattrs_ndx;
+extern int file_sum_extra_cnt;
 
 #ifdef USE_FLEXIBLE_ARRAY
 #define FILE_STRUCT_LEN (sizeof (struct file_struct))
@@ -836,7 +837,7 @@ extern int xattrs_ndx;
 #define DEV_EXTRA_CNT 2
 #define DIRNODE_EXTRA_CNT 3
 #define EXTRA64_CNT ((sizeof (union file_extras64) + EXTRA_LEN - 1) / EXTRA_LEN)
-#define SUM_EXTRA_CNT ((MAX_DIGEST_LEN + EXTRA_LEN - 1) / EXTRA_LEN)
+#define SUM_EXTRA_CNT file_sum_extra_cnt
 
 #define REQ_EXTRA(f,ndx) ((union file_extras*)(f) - (ndx))
 #define OPT_EXTRA(f,bump) ((union file_extras*)(f) - file_extra_cnt - 1 - (bump))
@@ -1023,6 +1024,7 @@ typedef struct filter_struct {
 		int slash_cnt;
 		struct filter_list_struct *mergelist;
 	} u;
+	uchar elide;
 } filter_rule;
 
 typedef struct filter_list_struct {
@@ -1162,16 +1164,16 @@ typedef struct {
 #define NSTR_COMPRESS 1
 
 struct name_num_item {
-	int num;
-	const char *name, *main_name;
+	int num, flags;
+	const char *name;
+	struct name_num_item *main_nni;
 };
 
 struct name_num_obj {
 	const char *type;
-	const char *negotiated_name;
+	struct name_num_item *negotiated_nni;
 	uchar *saw;
 	int saw_len;
-	int negotiated_num;
 	struct name_num_item *list;
 };
 

rsyncd.conf.5.md

@@ -164,6 +164,16 @@ the values of parameters.  See the GLOBAL PARAMETERS section for more details.
     available in this module.  You must specify this parameter for each module
     in `rsyncd.conf`.
 
+    If the value contains a "/./" element then the path will be divided at that
+    point into a chroot dir and an inner-chroot subdir.  If [`use chroot`](#)
+    is set to false, though, the extraneous dot dir is just cleaned out of the
+    path.  An example of this idiom is:
+
+    >     path = /var/rsync/./module1
+
+    This will (when chrooting) chroot to "/var/rsync" and set the inside-chroot
+    path to "/module1".
+
     You may base the path's value off of an environment variable by surrounding
     the variable name with percent signs.  You can even reference a variable
     that is set by rsync when the user connects.  For example, this would use
@@ -187,29 +197,47 @@ the values of parameters.  See the GLOBAL PARAMETERS section for more details.
     path, and of complicating the preservation of users and groups by name (see
     below).
 
-    As an additional safety feature, you can specify a dot-dir in the module's
-    "[path](#)" to indicate the point where the chroot should occur.  This allows
-    rsync to run in a chroot with a non-"/" path for the top of the transfer
-    hierarchy.  Doing this guards against unintended library loading (since
-    those absolute paths will not be inside the transfer hierarchy unless you
-    have used an unwise pathname), and lets you setup libraries for the chroot
-    that are outside of the transfer.  For example, specifying
-    "/var/rsync/./module1" will chroot to the "/var/rsync" directory and set
-    the inside-chroot path to "/module1".  If you had omitted the dot-dir, the
-    chroot would have used the whole path, and the inside-chroot path would
-    have been "/".
+    If `use chroot` is not set, it defaults to trying to enable a chroot but
+    allows the daemon to continue (after logging a warning) if it fails. The
+    one exception to this is when a module's [`path`](#) has a "/./" chroot
+    divider in it -- this causes an unset value to be treated as true for that
+    module.
 
-    When both "use chroot" and "[daemon chroot](#)" are false, OR the inside-chroot
-    path of "use chroot" is not "/", rsync will: (1) munge symlinks by default
-    for security reasons (see "[munge symlinks](#)" for a way to turn this off, but
-    only if you trust your users), (2) substitute leading slashes in absolute
-    paths with the module's path (so that options such as `--backup-dir`,
-    `--compare-dest`, etc. interpret an absolute path as rooted in the module's
-    "[path](#)" dir), and (3) trim ".." path elements from args if rsync believes
-    they would escape the module hierarchy.  The default for "use chroot" is
-    true, and is the safer choice (especially if the module is not read-only).
+    Prior to rsync 3.2.7, the default value was "true".  The new "unset"
+    default makes it easier to setup an rsync daemon as a non-root user or to
+    run a daemon on a system where chroot fails.  Explicitly setting the value
+    to "true" in rsyncd.conf will always require the chroot to succeed.
 
-    When this parameter is enabled *and* the "[name converter](#)" parameter is
+    It is also possible to specify a dot-dir in the module's "[path](#)" to
+    indicate that you want to chdir to the earlier part of the path and then
+    serve files from inside the latter part of the path (with sanitizing and
+    default symlink munging).  This can be useful if you need some library dirs
+    inside the chroot (typically for uid & gid lookups) but don't want to put
+    the lib dir into the top of the served path (even though they can be hidden
+    with an [`exclude`](#) directive).  However, a better choice for a modern
+    rsync setup is to use a [`name converter`](#)" and try to avoid inner lib
+    dirs altogether.  See also the [`daemon chroot`](#) parameter, which causes
+    rsync to chroot into its own chroot area before doing any path-related
+    chrooting.
+
+    If the daemon is serving the "/" dir (either directly or due to being
+    chrooted to the module's path), rsync does not do any path sanitizing or
+    (default) munging.
+
+    When it has to limit access to a particular subdir (either due to chroot
+    being disabled or having an inside-chroot path set), rsync will munge
+    symlinks (by default) and sanitize paths.  Those that dislike munged
+    symlinks (and really, really trust their users to not break out of the
+    subdir) can disable the symlink munging via the "[munge symlinks](#)"
+    parameter.
+
+    When rsync is sanitizing paths, it trims ".." path elements from args that
+    it believes would escape the module hierarchy. It also substitutes leading
+    slashes in absolute paths with the module's path (so that options such as
+    `--backup-dir` & `--compare-dest` interpret an absolute path as rooted in
+    the module's "[path](#)" dir).
+
+    When a chroot is in effect *and* the "[name converter](#)" parameter is
     *not* set, the "[numeric ids](#)" parameter will default to being enabled
     (disabling name lookups).  This means that if you manually setup
     name-lookup libraries in your chroot (instead of using a name converter)

support/json-rsync-version

@@ -0,0 +1,76 @@
+#!/usr/bin/python3
+
+import sys, argparse, subprocess, json
+
+def main():
+    if not args.rsync or args.rsync == '-':
+        ver_out = sys.stdin.read().strip()
+    else:
+        ver_out = subprocess.check_output([args.rsync, '--version', '--version'], encoding='utf-8').strip()
+    if ver_out.startswith('{'):
+        print(ver_out)
+        return
+    info = { }
+    for line in ver_out.splitlines():
+        if line.startswith('rsync '):
+            prog, vstr, ver, pstr, vstr2, proto = line.split()
+            info['program'] = prog
+            if ver.startswith('v'):
+                ver = ver[1:]
+            info[vstr] = ver
+            if '.' not in proto:
+                proto += '.0'
+            else:
+                proto = proto.replace('.PR', '.')
+            info[pstr] = proto
+        elif line.startswith('Copyright '):
+            info['copyright'] = line[10:]
+        elif line.startswith('Web site: '):
+            info['url'] = line[10:]
+        elif line.startswith('  '):
+            if not saw_comma and ',' in line:
+                saw_comma = True
+                info[sect_name] = { }
+            if saw_comma:
+                for x in line.strip(' ,').split(', '):
+                    if ' ' in x:
+                        val, var = x.split(' ', 1)
+                        if val == 'no':
+                            val = False
+                        elif val.endswith('-bit'):
+                            var = var[:-1] + '_bits'
+                            val = int(val.split('-')[0])
+                        if var == 'protect-args':
+                            var = 'secluded-args'
+                    else:
+                        var = x
+                        val = True
+                    var = var.replace(' ', '_').replace('-', '_')
+                    info[sect_name][var] = val
+            else:
+                info[sect_name] += [ x for x in line.split() if not x.startswith('(') ]
+        elif line == '':
+            break
+        else:
+            sect_name = line.strip(' :').replace(' ', '_').lower()
+            info[sect_name] = [ ]
+            saw_comma = False
+    for chk in 'capabilities optimizations'.split():
+        if chk not in info:
+            info[chk] = { }
+    for chk in 'checksum_list compress_list daemon_auth_list'.split():
+        if chk not in info:
+            info[chk] = [ ]
+    info['license'] = 'GPL3'
+    info['caveat'] = 'rsync comes with ABSOLUTELY NO WARRANTY'
+    print(json.dumps(info))
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description="Output rsync's version data in JSON format, even if the rsync doesn't support a native json-output method.", add_help=False)
+    parser.add_argument('rsync', nargs='?', help="Specify an rsync command to run. Otherwise stdin is consumed.")
+    parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.")
+    args = parser.parse_args()
+    main()
+
+# vim: sw=4 et

usage.c

@@ -22,9 +22,9 @@
 #include "latest-year.h"
 #include "git-version.h"
 #include "default-cvsignore.h"
+#include "itypes.h"
 
-extern struct name_num_obj valid_checksums;
-extern struct name_num_obj valid_compressions;
+extern struct name_num_obj valid_checksums, valid_compressions, valid_auth_checksums;
 
 static char *istring(const char *fmt, int val)
 {
@@ -37,7 +37,8 @@ static char *istring(const char *fmt, int val)
 static void print_info_flags(enum logcode f)
 {
 	STRUCT_STAT *dumstat;
-	char line_buf[75];
+	BOOL as_json = f == FNONE ? 1 : 0; /* We use 1 == first attribute, 2 == need closing array */
+	char line_buf[75], item_buf[32];
 	int line_len, j;
 	char *info_flags[] = {
 
@@ -164,46 +165,136 @@ static void print_info_flags(enum logcode f)
 
 	for (line_len = 0, j = 0; ; j++) {
 		char *str = info_flags[j], *next_nfo = str ? info_flags[j+1] : NULL;
-		int str_len = str && *str != '*' ? strlen(str) : 1000;
 		int need_comma = next_nfo && *next_nfo != '*' ? 1 : 0;
-		if (line_len && line_len + 1 + str_len + need_comma >= (int)sizeof line_buf) {
-			rprintf(f, "   %s\n", line_buf);
+		int item_len;
+		if (!str || *str == '*')
+			item_len = 1000;
+		else if (as_json) {
+			char *space = strchr(str, ' ');
+			int is_no = space && strncmp(str, "no ", 3) == 0;
+			int is_bits = space && isDigit(str);
+			char *quot = space && !is_no && !is_bits ? "\"" : "";
+			char *item = space ? space + 1 : str;
+			char *val = !space ? "true" : is_no ? "false" : str;
+			int val_len = !space ? 4 : is_no ? 5 : space - str;
+			if (is_bits && (space = strchr(val, '-')) != NULL)
+			    val_len = space - str;
+			item_len = snprintf(item_buf, sizeof item_buf,
+					   " \"%s%s\": %s%.*s%s%s", item, is_bits ? "bits" : "",
+					   quot, val_len, val, quot, need_comma ? "," : "");
+			if (is_bits)
+				item_buf[strlen(item)+2-1] = '_'; /* Turn the 's' into a '_' */
+			for (space = item; (space = strpbrk(space, " -")) != NULL; space++)
+				item_buf[space - item + 2] = '_';
+		} else
+			item_len = snprintf(item_buf, sizeof item_buf, " %s%s", str, need_comma ? "," : "");
+		if (line_len && line_len + item_len >= (int)sizeof line_buf) {
+			if (as_json)
+				printf("   %s\n", line_buf);
+			else
+				rprintf(f, "   %s\n", line_buf);
 			line_len = 0;
 		}
 		if (!str)
 			break;
 		if (*str == '*') {
-			rprintf(f, "%s:\n", str+1);
-			continue;
+			if (as_json) {
+				if (as_json == 2)
+					printf("  }");
+				else
+					as_json = 2;
+				printf(",\n  \"%c%s\": {\n", toLower(str+1), str+2);
+			} else
+				rprintf(f, "%s:\n", str+1);
+		} else {
+			strlcpy(line_buf + line_len, item_buf, sizeof line_buf - line_len);
+			line_len += item_len;
 		}
-		line_len += snprintf(line_buf+line_len, sizeof line_buf - line_len, " %s%s", str, need_comma ? "," : "");
 	}
+	if (as_json == 2)
+		printf("  }");
 }
 
+static void output_nno_list(enum logcode f, const char *name, struct name_num_obj *nno)
+{
+	char namebuf[64], tmpbuf[256];
+	char *tok, *next_tok, *comma = ",";
+	char *cp;
+
+	/* Using '(' ensures that we get a trailing "none" but also includes aliases. */
+	get_default_nno_list(nno, tmpbuf, sizeof tmpbuf - 1, '(');
+	if (f != FNONE) {
+		rprintf(f, "%s:\n", name);
+		rprintf(f, "    %s\n", tmpbuf);
+		return;
+	}
+
+	strlcpy(namebuf, name, sizeof namebuf);
+	for (cp = namebuf; *cp; cp++) {
+		if (*cp == ' ')
+			*cp = '_';
+		else if (isUpper(cp))
+			*cp = toLower(cp);
+	}
+
+	printf(",\n  \"%s\": [\n   ", namebuf);
+
+	for (tok = strtok(tmpbuf, " "); tok; tok = next_tok) {
+		next_tok = strtok(NULL, " ");
+		if (*tok != '(') /* Ignore the alises in the JSON output */
+			printf(" \"%s\"%s", tok, comma + (next_tok ? 0 : 1));
+	}
+
+	printf("\n  ]");
+}
+
+/* A request of f == FNONE wants json on stdout. */
 void print_rsync_version(enum logcode f)
 {
-	char tmpbuf[256], *subprotocol = "";
+	char copyright[] = "(C) 1996-" LATEST_YEAR " by Andrew Tridgell, Wayne Davison, and others.";
+	char url[] = "https://rsync.samba.org/";
+	BOOL first_line = 1;
 
+#define json_line(name, value) \
+	do { \
+		printf("%c\n  \"%s\": \"%s\"", first_line ? '{' : ',', name, value); \
+		first_line = 0; \
+	} while (0)
+
+	if (f == FNONE) {
+		char verbuf[32];
+		json_line("program", RSYNC_NAME);
+		json_line("version", rsync_version());
+		(void)snprintf(verbuf, sizeof verbuf, "%d.%d", PROTOCOL_VERSION, SUBPROTOCOL_VERSION);
+		json_line("protocol", verbuf);
+		json_line("copyright", copyright);
+		json_line("url", url);
+	} else {
 #if SUBPROTOCOL_VERSION != 0
-	subprotocol = istring(".PR%d", SUBPROTOCOL_VERSION);
+		char *subprotocol = istring(".PR%d", SUBPROTOCOL_VERSION);
+#else
+		char *subprotocol = "";
 #endif
-	rprintf(f, "%s  version %s  protocol version %d%s\n",
-		RSYNC_NAME, rsync_version(), PROTOCOL_VERSION, subprotocol);
-
-	rprintf(f, "Copyright (C) 1996-" LATEST_YEAR " by Andrew Tridgell, Wayne Davison, and others.\n");
-	rprintf(f, "Web site: https://rsync.samba.org/\n");
+		rprintf(f, "%s  version %s  protocol version %d%s\n",
+			RSYNC_NAME, rsync_version(), PROTOCOL_VERSION, subprotocol);
+		rprintf(f, "Copyright %s\n", copyright);
+		rprintf(f, "Web site: %s\n", url);
+	}
 
 	print_info_flags(f);
 
 	init_checksum_choices();
 
-	rprintf(f, "Checksum list:\n");
-	get_default_nno_list(&valid_checksums, tmpbuf, sizeof tmpbuf, '(');
-	rprintf(f, "    %s\n", tmpbuf);
+	output_nno_list(f, "Checksum list", &valid_checksums);
+	output_nno_list(f, "Compress list", &valid_compressions);
+	output_nno_list(f, "Daemon auth list", &valid_auth_checksums);
 
-	rprintf(f, "Compress list:\n");
-	get_default_nno_list(&valid_compressions, tmpbuf, sizeof tmpbuf, '(');
-	rprintf(f, "    %s\n", tmpbuf);
+	if (f == FNONE) {
+		json_line("license", "GPL3");
+		json_line("caveat", "rsync comes with ABSOLUTELY NO WARRANTY");
+		printf("\n}\n");
+		return;
+	}
 
 #ifdef MAINTAINER_MODE
 	rprintf(f, "Panic Action: \"%s\"\n", get_panic_action());
@@ -265,11 +356,13 @@ void daemon_usage(enum logcode F)
 
 const char *rsync_version(void)
 {
+	char *ver;
 #ifdef RSYNC_GITVER
-	return RSYNC_GITVER;
+	ver = RSYNC_GITVER;
 #else
-	return RSYNC_VERSION;
+	ver = RSYNC_VERSION;
 #endif
+	return *ver == 'v' ? ver+1 : ver;
 }
 
 const char *default_cvsignore(void)

util1.c

@@ -1487,12 +1487,19 @@ const char *find_filename_suffix(const char *fn, int fn_len, int *len_ptr)
 
 #define UNIT (1 << 16)
 
-uint32 fuzzy_distance(const char *s1, unsigned len1, const char *s2, unsigned len2)
+uint32 fuzzy_distance(const char *s1, unsigned len1, const char *s2, unsigned len2, uint32 upperlimit)
 {
 	uint32 a[MAXPATHLEN], diag, above, left, diag_inc, above_inc, left_inc;
 	int32 cost;
 	unsigned i1, i2;
 
+	/* Check to see if the Levenshtein distance must be greater than the
+	 * upper limit defined by the previously found lowest distance using
+	 * the heuristic that the Levenshtein distance is greater than the
+	 * difference in length of the two strings */
+	if ((len1 > len2 ? len1 - len2 : len2 - len1) * UNIT > upperlimit)
+		return 0xFFFFU * UNIT + 1;
+
 	if (!len1 || !len2) {
 		if (!len1) {
 			s1 = s2;

version.h

@@ -1,2 +1,2 @@
-#define RSYNC_VERSION "3.2.6"
+#define RSYNC_VERSION "3.2.7pre1"
 #define MAINTAINER_TZ_OFFSET -7.0

xattrs.c

@@ -39,9 +39,13 @@ extern int preserve_specials;
 extern int checksum_seed;
 extern int saw_xattr_filter;
 
+extern struct name_num_item *xattr_sum_nni;
+extern int xattr_sum_len;
+
 #define RSYNC_XAL_INITIAL 5
 #define RSYNC_XAL_LIST_INITIAL 100
 
+#define MAX_XATTR_DIGEST_LEN MD5_DIGEST_LEN
 #define MAX_FULL_DATUM 32
 
 #define HAS_PREFIX(str, prfx) (*(str) == *(prfx) && strncmp(str, prfx, sizeof (prfx) - 1) == 0)
@@ -269,8 +273,8 @@ static int rsync_xal_get(const char *fname, item_list *xalp)
 
 		if (datum_len > MAX_FULL_DATUM) {
 			/* For large datums, we store a flag and a checksum. */
-			name_offset = 1 + MAX_DIGEST_LEN;
-			sum_init(-1, checksum_seed);
+			name_offset = 1 + MAX_XATTR_DIGEST_LEN;
+			sum_init(xattr_sum_nni, checksum_seed);
 			sum_update(ptr, datum_len);
 			free(ptr);
 
@@ -377,20 +381,14 @@ static int64 xattr_lookup_hash(const item_list *xalp)
 {
 	const rsync_xa *rxas = xalp->items;
 	size_t i;
-	int64 key = hashlittle(&xalp->count, sizeof xalp->count);
+	int64 key = hashlittle2(&xalp->count, sizeof xalp->count);
 
 	for (i = 0; i < xalp->count; i++) {
-		key += hashlittle(rxas[i].name, rxas[i].name_len);
+		key += hashlittle2(rxas[i].name, rxas[i].name_len);
 		if (rxas[i].datum_len > MAX_FULL_DATUM)
-			key += hashlittle(rxas[i].datum, MAX_DIGEST_LEN);
+			key += hashlittle2(rxas[i].datum, xattr_sum_len);
 		else
-			key += hashlittle(rxas[i].datum, rxas[i].datum_len);
-	}
-
-	if (key == 0) {
-		/* This is very unlikely, but we should never
-		 * return 0 as hashtable_find() doesn't like it. */
-		return 1;
+			key += hashlittle2(rxas[i].datum, rxas[i].datum_len);
 	}
 
 	return key;
@@ -435,7 +433,7 @@ static int find_matching_xattr(const item_list *xalp)
 			if (rxas1[j].datum_len > MAX_FULL_DATUM) {
 				if (memcmp(rxas1[j].datum + 1,
 					   rxas2[j].datum + 1,
-					   MAX_DIGEST_LEN) != 0)
+					   xattr_sum_len) != 0)
 					break;
 			} else {
 				if (memcmp(rxas1[j].datum, rxas2[j].datum,
@@ -471,8 +469,6 @@ static int rsync_xal_store(item_list *xalp)
 
 	if (rsync_xal_h == NULL)
 		rsync_xal_h = hashtable_create(512, HT_KEY64);
-	if (rsync_xal_h == NULL)
-		out_of_memory("rsync_xal_h hashtable_create()");
 
 	new_ref = new0(rsync_xa_list_ref);
 	new_ref->ndx = ndx;
@@ -535,7 +531,7 @@ int send_xattr(int f, stat_x *sxp)
 #endif
 			write_buf(f, name, name_len);
 			if (rxa->datum_len > MAX_FULL_DATUM)
-				write_buf(f, rxa->datum + 1, MAX_DIGEST_LEN);
+				write_buf(f, rxa->datum + 1, xattr_sum_len);
 			else
 				write_bigbuf(f, rxa->datum, rxa->datum_len);
 		}
@@ -588,7 +584,7 @@ int xattr_diff(struct file_struct *file, stat_x *sxp, int find_all)
 		else if (snd_rxa->datum_len > MAX_FULL_DATUM) {
 			same = cmp == 0 && snd_rxa->datum_len == rec_rxa->datum_len
 			    && memcmp(snd_rxa->datum + 1, rec_rxa->datum + 1,
-				      MAX_DIGEST_LEN) == 0;
+				      xattr_sum_len) == 0;
 			/* Flag unrequested items that we need. */
 			if (!same && find_all && snd_rxa->datum[0] == XSTATE_ABBREV)
 				snd_rxa->datum[0] = XSTATE_TODO;
@@ -797,7 +793,7 @@ void receive_xattr(int f, struct file_struct *file)
 		rsync_xa *rxa;
 		size_t name_len = read_varint(f);
 		size_t datum_len = read_varint(f);
-		size_t dget_len = datum_len > MAX_FULL_DATUM ? 1 + MAX_DIGEST_LEN : datum_len;
+		size_t dget_len = datum_len > MAX_FULL_DATUM ? 1 + (size_t)xattr_sum_len : datum_len;
 		size_t extra_len = MIGHT_NEED_RPRE ? RPRE_LEN : 0;
 		if (SIZE_MAX - dget_len < extra_len || SIZE_MAX - dget_len - extra_len < name_len)
 			overflow_exit("receive_xattr");
@@ -812,7 +808,7 @@ void receive_xattr(int f, struct file_struct *file)
 			read_buf(f, ptr, dget_len);
 		else {
 			*ptr = XSTATE_ABBREV;
-			read_buf(f, ptr + 1, MAX_DIGEST_LEN);
+			read_buf(f, ptr + 1, xattr_sum_len);
 		}
 
 		if (saw_xattr_filter) {
@@ -943,7 +939,7 @@ static int rsync_xal_set(const char *fname, item_list *xalp,
 	rsync_xa *rxas = xalp->items;
 	ssize_t list_len;
 	size_t i, len;
-	char *name, *ptr, sum[MAX_DIGEST_LEN];
+	char *name, *ptr, sum[MAX_XATTR_DIGEST_LEN];
 #ifdef HAVE_LINUX_XATTRS
 	int user_only = am_root <= 0;
 #endif
@@ -958,7 +954,6 @@ static int rsync_xal_set(const char *fname, item_list *xalp,
 		name = rxas[i].name;
 
 		if (XATTR_ABBREV(rxas[i])) {
-			int sum_len;
 			/* See if the fnamecmp version is identical. */
 			len = name_len = rxas[i].name_len;
 			if ((ptr = get_xattr_data(fnamecmp, name, &len, 1)) == NULL) {
@@ -975,10 +970,10 @@ static int rsync_xal_set(const char *fname, item_list *xalp,
 				goto still_abbrev;
 			}
 
-			sum_init(-1, checksum_seed);
+			sum_init(xattr_sum_nni, checksum_seed);
 			sum_update(ptr, len);
-			sum_len = sum_end(sum);
-			if (memcmp(sum, rxas[i].datum + 1, sum_len) != 0) {
+			sum_end(sum);
+			if (memcmp(sum, rxas[i].datum + 1, xattr_sum_len) != 0) {
 				free(ptr);
 				goto still_abbrev;
 			}