Preparing for release of 3.3.0 [buildall]

Fortified (-D_FORTIFY_SOURCE=2 for gcc) builds make strlcpy() crash when
its third parameter (size) is larger than the buffer:
  $ rsync -FFXHav '--filter=merge global-rsync-filter' Align-37-43/ xxx
  sending incremental file list
  *** buffer overflow detected ***: terminated

It's in the exclude code in setup_merge_file():
  strlcpy(y, save, MAXPATHLEN);

Note the 'y' pointer was incremented, so it no longer points to memory
with MAXPATHLEN "owned" bytes.

Fix it by remembering the number of copied bytes into the 'save' buffer
and use that instead of MAXPATHLEN which is clearly incorrect.

Fixes #511.

.github/workflows/build.yml

@@ -15,7 +15,7 @@ jobs:
   ubuntu-build:
     runs-on: ubuntu-20.04
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: prep
       run: |
         sudo apt-get install acl libacl1-dev attr libattr1-dev liblz4-dev libzstd-dev libxxhash-dev python3-cmarkgfm openssl wget
@@ -38,7 +38,7 @@ jobs:
     - name: ssl file list
       run: rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
     - name: save artifact
-      uses: actions/upload-artifact@v2
+      uses: actions/upload-artifact@v3
       with:
         name: ubuntu-bin
         path: |
@@ -53,7 +53,7 @@ jobs:
   macos-build:
     runs-on: macos-latest
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: prep
       run: |
         brew install automake openssl xxhash zstd lz4 wget
@@ -73,7 +73,7 @@ jobs:
     - name: ssl file list
       run: rsync-ssl --no-motd download.samba.org::rsyncftp/ || true
     - name: save artifact
-      uses: actions/upload-artifact@v2
+      uses: actions/upload-artifact@v3
       with:
         name: macos-bin
         path: |
@@ -89,7 +89,7 @@ jobs:
     runs-on: windows-2022
     if: (github.event_name == 'schedule' || contains(github.event.head_commit.message, '[buildall]'))
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: cygwin
       run: choco install -y --no-progress cygwin cyg-get
     - name: prep
@@ -112,7 +112,7 @@ jobs:
     - name: ssl file list
       run: bash -c 'PATH="/usr/local/bin:$PATH" rsync-ssl --no-motd download.samba.org::rsyncftp/ || true'
     - name: save artifact
-      uses: actions/upload-artifact@v2
+      uses: actions/upload-artifact@v3
       with:
         name: cygwin-bin
         path: |

NEWS.md

@@ -1,3 +1,68 @@
+# NEWS for rsync 3.3.0 (6 Apr 2024)
+
+## Changes in this version:
+
+### BUG FIXES:
+
+- Fixed a bug with `--sparse --inplace` where a trailing gap in the source
+  file would not clear out the trailing data in the destination file.
+
+- Fixed an buffer overflow in the checksum2 code if SHA1 is being used for
+  the checksum2 algorithm.
+
+- Fixed an issue when rsync is compiled using `_FORTIFY_SOURCE` so that the
+  extra tests don't complain about a strlcpy() limit value (which was too
+  large, even though it wasn't possible for the larger value to cause an
+  overflow).
+
+- Add a backtick to the list of characters that the filename quoting needs to
+  escape using backslashes.
+
+- Fixed a string-comparison issue in the internal handling of `--progress` (a
+  locale such as tr_TR.utf-8 needed the internal triggering of `--info` options
+  to use upper-case flag names to ensure that they match).
+
+- Make sure that a local transfer marks the sender side as trusted.
+
+- Change the argv handling to work with a newer popt library -- one that likes
+  to free more data than it used to.
+
+- Rsync now calls `OpenSSL_add_all_algorithms()` when compiled against an older
+  openssl library.
+
+- Fixed a problem in the daemon auth for older protocols (29 and before) if the
+  openssl library is being used to compute MD4 checksums.
+
+- Fixed `rsync -VV` on Cygwin -- it needed a flush of stdout.
+
+- Fixed an old stats bug that counted devices as symlinks.
+
+### ENHANCEMENTS:
+
+- Enhanced rrsync with the `-no-overwrite` option that allows you to ensure
+  that existing files on your restricted but writable directory can't be
+  modified.
+
+- Enhanced the manpages to mark links with .UR & .UE. If your nroff doesn't
+  support these idioms, touch the file `.md2man-force` in the source directory
+  so that `md-convert` gets called with the `--force-link-text` option, and
+  that should ensure that your manpages are still readable even with the
+  ignored markup.
+
+- Some manpage improvements on the handling of [global] modules.
+
+- Changed the mapfrom & mapto perl scripts (in the support dir) into a single
+  python script named idmap.  Converted a couple more perl scripts into python.
+
+- Changed the mnt-excl perl script (in the support dir) into a python script.
+
+### DEVELOPER RELATED:
+
+ - Updated config.guess (timestamp 2023-01-01) and config.sub (timestamp
+   2023-01-21).
+
+------------------------------------------------------------------------------
+
 # NEWS for rsync 3.2.7 (20 Oct 2022)
 
 ## Changes in this version:
@@ -50,7 +115,7 @@
 
 - The `--fuzzy` option was optimized a bit to try to cut down on the amount of
   computations when considering a big pool of files. The simple heuristic from
-  Kenneth Finnegan resuled in about a 2x speedup.
+  Kenneth Finnegan resulted in about a 2x speedup.
 
 - If rsync is forced to use protocol 29 or before (perhaps due to talking to an
   rsync before 3.0.0), the modify time of a file is limited to 4-bytes.  Rsync
@@ -217,9 +282,10 @@
  - A new form of arg protection was added that works similarly to the older
    `--protect-args` ([`-s`](rsync.1#opt)) option but in a way that avoids
    breaking things like rrsync (the restricted rsync script): rsync now uses
-   backslash escaping for sending "shell-active" characters to the remote
-   shell. This includes spaces, so fetching a remote file via a simple quoted
-   filename value now works by default without any extra quoting:
+   backslash escaping for sending "shell-active" characters to the remote shell
+   (such as `$(){}<>#&` and others). This includes spaces, so fetching a remote
+   file via a quoted filename value now works by default without any extra
+   quoting:
 
    ```shell
        rsync -aiv host:'a simple file.pdf' .
@@ -227,10 +293,14 @@
 
    Wildcards are not escaped in filename args, but they are escaped in options
    like the [`--suffix`](rsync.1#opt) and [`--usermap`](rsync.1#opt) values.
-   If your rsync script depends on the old arg-splitting behavior, either run
-   it with the [`--old-args`](rsync.1#opt) option or `export RSYNC_OLD_ARGS=1`
-   in the script's environment.  See also the [ADVANCED USAGE](rsync.1#)
-   section of rsync's manpage for how to use a more modern arg style.
+
+   If a script depends on the old arg behavior (perhaps because it quotes or
+   protects the args already, or perhaps because it expects arg splitting),
+   there are two easy ways to get things going with a modern rsync: either
+   `export RSYNC_OLD_ARGS=1` in the script's environment (perhaps in the script
+   itself) or add the option [`--old-args`](rsync.1#opt) to the rsync commands
+   that are run.  See also the [ADVANCED USAGE](rsync.1#) section of rsync's
+   manpage for how to use a more modern arg style.
 
  - A long-standing bug was preventing rsync from figuring out the current
    locale's decimal point character, which made rsync always output numbers
@@ -4692,6 +4762,7 @@
 
 | RELEASE DATE | VER.   | DATE OF COMMIT\* | PROTOCOL    |
 |--------------|--------|------------------|-------------|
+| 06 Apr 2024  | 3.3.0  |                  | 31          |
 | 20 Oct 2022  | 3.2.7  |                  | 31          |
 | 09 Sep 2022  | 3.2.6  |                  | 31          |
 | 14 Aug 2022  | 3.2.5  |                  | 31          |

checksum.c

@@ -3,7 +3,7 @@
  *
  * Copyright (C) 1996 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
- * Copyright (C) 2004-2022 Wayne Davison
+ * Copyright (C) 2004-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -154,7 +154,7 @@ static const EVP_MD *csum_evp_md(struct name_num_item *nni)
 		emd = NULL;
 	else
 #endif
-		emd = EVP_get_digestbyname(nni->name);                                               
+		emd = EVP_get_digestbyname(nni->name);
 	if (emd && !(nni->flags & NNI_EVP_OK)) { /* Make sure it works before we advertise it */
 		if (!ctx_evp && !(ctx_evp = EVP_MD_CTX_create()))
 			out_of_memory("csum_evp_md");
@@ -300,6 +300,7 @@ uint32 get_checksum1(char *buf1, int32 len)
 }
 #endif
 
+/* The "sum" buffer must be at least MAX_DIGEST_LEN bytes! */
 void get_checksum2(char *buf, int32 len, char *sum)
 {
 #ifdef USE_OPENSSL
@@ -787,6 +788,10 @@ void init_checksum_choices()
 	if (initialized_choices)
 		return;
 
+#if defined USE_OPENSSL && OPENSSL_VERSION_NUMBER < 0x10100000L
+	OpenSSL_add_all_algorithms();
+#endif
+
 #if defined SUPPORT_XXH3 || defined USE_OPENSSL
 	for (nni = valid_checksums.list; nni->name; nni++)
 		verify_digest(nni, True);

compat.c

@@ -834,6 +834,8 @@ void output_daemon_greeting(int f_out, int am_client)
 	char tmpbuf[MAX_NSTR_STRLEN];
 	int our_sub = get_subprotocol_version();
 
+	init_checksum_choices();
+
 	get_default_nno_list(&valid_auth_checksums, tmpbuf, MAX_NSTR_STRLEN, '\0');
 
 	io_printf(f_out, "@RSYNCD: %d.%d %s\n", protocol_version, our_sub, tmpbuf);
@@ -873,8 +875,10 @@ void negotiate_daemon_auth(int f_out, int am_client)
 		}
 	}
 	am_server = save_am_server;
-	if (md4_is_old && valid_auth_checksums.negotiated_nni->num == CSUM_MD4)
+	if (md4_is_old && valid_auth_checksums.negotiated_nni->num == CSUM_MD4) {
 		valid_auth_checksums.negotiated_nni->num = CSUM_MD4_OLD;
+		valid_auth_checksums.negotiated_nni->flags = 0;
+	}
 }
 
 int get_subprotocol_version()

configure.ac

@@ -432,10 +432,10 @@ AH_TEMPLATE([USE_OPENSSL],
 if test x"$enable_openssl" != x"no"; then
     if test x"$ac_cv_header_openssl_md4_h" = x"yes" && test x"$ac_cv_header_openssl_md5_h" = x"yes"; then
       AC_MSG_RESULT(yes)
-      AC_SEARCH_LIBS(MD5_Init, crypto,
+      AC_SEARCH_LIBS(EVP_MD_CTX_copy, crypto,
           [AC_DEFINE(USE_OPENSSL)
 	   enable_openssl=yes],
-          [err_msg="$err_msg$nl- Failed to find MD5_Init function in openssl crypto lib.";
+          [err_msg="$err_msg$nl- Failed to find EVP_MD_CTX_copy function in openssl crypto lib.";
 	   no_lib="$no_lib openssl"])
     else
         AC_MSG_RESULT(no)

delete.c

@@ -4,7 +4,7 @@
  * Copyright (C) 1996-2000 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
  * Copyright (C) 2002 Martin Pool <mbp@samba.org>
- * Copyright (C) 2003-2020 Wayne Davison
+ * Copyright (C) 2003-2024 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -188,7 +188,7 @@ enum delret delete_item(char *fbuf, uint16 mode, uint16 flags)
 				stats.deleted_symlinks++;
 #endif
 			else if (IS_DEVICE(mode))
-				stats.deleted_symlinks++;
+				stats.deleted_devices++;
 			else
 				stats.deleted_specials++;
 		}

exclude.c

@@ -4,7 +4,7 @@
  * Copyright (C) 1996-2001 Andrew Tridgell <tridge@samba.org>
  * Copyright (C) 1996 Paul Mackerras
  * Copyright (C) 2002 Martin Pool
- * Copyright (C) 2003-2022 Wayne Davison
+ * Copyright (C) 2003-2024 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -720,7 +720,8 @@ static BOOL setup_merge_file(int mergelist_num, filter_rule *ex,
 	parent_dirscan = True;
 	while (*y) {
 		char save[MAXPATHLEN];
-		strlcpy(save, y, MAXPATHLEN);
+		/* copylen is strlen(y) which is < MAXPATHLEN. +1 for \0 */
+		size_t copylen = strlcpy(save, y, MAXPATHLEN) + 1;
 		*y = '\0';
 		dirbuf_len = y - dirbuf;
 		strlcpy(x, ex->pattern, MAXPATHLEN - (x - buf));
@@ -734,7 +735,7 @@ static BOOL setup_merge_file(int mergelist_num, filter_rule *ex,
 			lp->head = NULL;
 		}
 		lp->tail = NULL;
-		strlcpy(y, save, MAXPATHLEN);
+		strlcpy(y, save, copylen);
 		while ((*x++ = *y++) != '/') {}
 	}
 	parent_dirscan = False;

fileio.c

@@ -3,7 +3,7 @@
  *
  * Copyright (C) 1998 Andrew Tridgell
  * Copyright (C) 2002 Martin Pool
- * Copyright (C) 2004-2020 Wayne Davison
+ * Copyright (C) 2004-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -40,30 +40,34 @@ OFF_T preallocated_len = 0;
 static OFF_T sparse_seek = 0;
 static OFF_T sparse_past_write = 0;
 
-int sparse_end(int f, OFF_T size)
+int sparse_end(int f, OFF_T size, int updating_basis_or_equiv)
 {
-	int ret;
+	int ret = 0;
 
-	sparse_past_write = 0;
-
-	if (!sparse_seek)
-		return 0;
-
-#ifdef HAVE_FTRUNCATE
-	ret = do_ftruncate(f, size);
-#else
-	if (do_lseek(f, sparse_seek-1, SEEK_CUR) != size-1)
-		ret = -1;
-	else {
-		do {
-			ret = write(f, "", 1);
-		} while (ret < 0 && errno == EINTR);
-
-		ret = ret <= 0 ? -1 : 0;
-	}
+	if (updating_basis_or_equiv) {
+		if (sparse_seek && do_punch_hole(f, sparse_past_write, sparse_seek) < 0)
+			ret = -1;
+#ifdef HAVE_FTRUNCATE /* A compilation formality -- in-place requires ftruncate() */
+		else /* Just in case the original file was longer */
+			ret = do_ftruncate(f, size);
 #endif
+	} else if (sparse_seek) {
+#ifdef HAVE_FTRUNCATE
+		ret = do_ftruncate(f, size);
+#else
+		if (do_lseek(f, sparse_seek-1, SEEK_CUR) != size-1)
+			ret = -1;
+		else {
+			do {
+				ret = write(f, "", 1);
+			} while (ret < 0 && errno == EINTR);
 
-	sparse_seek = 0;
+			ret = ret <= 0 ? -1 : 0;
+		}
+#endif
+	}
+
+	sparse_past_write = sparse_seek = 0;
 
 	return ret;
 }

flist.c

@@ -4,7 +4,7 @@
  * Copyright (C) 1996 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
  * Copyright (C) 2001, 2002 Martin Pool <mbp@samba.org>
- * Copyright (C) 2002-2022 Wayne Davison
+ * Copyright (C) 2002-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -2367,7 +2367,7 @@ struct file_list *send_file_list(int f, int argc, char *argv[])
 		}
 
 		dirlen = dir ? strlen(dir) : 0;
-		if (dirlen != lastdir_len || memcmp(lastdir, dir, dirlen) != 0) {
+		if (dirlen != lastdir_len || (dirlen && memcmp(lastdir, dir, dirlen) != 0)) {
 			if (!change_pathname(NULL, dir, -dirlen))
 				goto bad_path;
 			lastdir = pathname;
@@ -2659,7 +2659,7 @@ struct file_list *recv_file_list(int f, int dir_ndx)
 		} else if (S_ISLNK(file->mode))
 			stats.num_symlinks++;
 		else if (IS_DEVICE(file->mode))
-			stats.num_symlinks++;
+			stats.num_devices++;
 		else
 			stats.num_specials++;
 

generator.c

@@ -4,7 +4,7 @@
  * Copyright (C) 1996-2000 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
  * Copyright (C) 2002 Martin Pool <mbp@samba.org>
- * Copyright (C) 2003-2022 Wayne Davison
+ * Copyright (C) 2003-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -783,7 +783,7 @@ static int generate_and_send_sums(int fd, OFF_T len, int f_out, int f_copy)
 	for (i = 0; i < sum.count; i++) {
 		int32 n1 = (int32)MIN(len, (OFF_T)sum.blength);
 		char *map = map_ptr(mapbuf, offset, n1);
-		char sum2[SUM_LENGTH];
+		char sum2[MAX_DIGEST_LEN];
 		uint32 sum1;
 
 		len -= n1;

latest-year.h

@@ -1 +1 @@
-#define LATEST_YEAR "2022"
+#define LATEST_YEAR "2024"

main.c

@@ -89,6 +89,8 @@ extern int backup_dir_len;
 extern int basis_dir_cnt;
 extern int default_af_hint;
 extern int stdout_format_has_i;
+extern int trust_sender_filter;
+extern int trust_sender_args;
 extern struct stats stats;
 extern char *stdout_format;
 extern char *logfile_format;
@@ -1381,15 +1383,6 @@ int client_run(int f_in, int f_out, pid_t pid, int argc, char *argv[])
 	return MAX(exit_code, exit_code2);
 }
 
-static void dup_argv(char *argv[])
-{
-	int i;
-
-	for (i = 0; argv[i]; i++)
-		argv[i] = strdup(argv[i]);
-}
-
-
 /* Start a client for either type of remote connection.  Work out
  * whether the arguments request a remote shell or rsyncd connection,
  * and call the appropriate connection function, then run_client.
@@ -1405,10 +1398,6 @@ static int start_client(int argc, char *argv[])
 	int ret;
 	pid_t pid;
 
-	/* Don't clobber argv[] so that ps(1) can still show the right
-	 * command line. */
-	dup_argv(argv);
-
 	if (!read_batch) { /* for read_batch, NO source is specified */
 		char *path = check_for_hostspec(argv[0], &shell_machine, &rsync_port);
 		if (path) { /* source is remote */
@@ -1484,8 +1473,10 @@ static int start_client(int argc, char *argv[])
 	}
 
 	/* A local transfer doesn't unbackslash anything, so leave the args alone. */
-	if (local_server)
+	if (local_server) {
 		old_style_args = 2;
+		trust_sender_args = trust_sender_filter = 1;
+	}
 
 	if (!rsync_port && remote_argc && !**remote_argv) /* Turn an empty arg into a dot dir. */
 		*remote_argv = ".";

match.c

@@ -3,7 +3,7 @@
  *
  * Copyright (C) 1996 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
- * Copyright (C) 2003-2022 Wayne Davison
+ * Copyright (C) 2003-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -142,7 +142,7 @@ static void hash_search(int f,struct sum_struct *s,
 {
 	OFF_T offset, aligned_offset, end;
 	int32 k, want_i, aligned_i, backup;
-	char sum2[SUM_LENGTH];
+	char sum2[MAX_DIGEST_LEN];
 	uint32 s1, s2, sum;
 	int more;
 	schar *map;

maybe-make-man

@@ -8,6 +8,7 @@ fi
 inname="$1"
 srcdir=`dirname "$0"`
 flagfile="$srcdir/.md2man-works"
+force_flagfile="$srcdir/.md2man-force"
 
 if [ ! -f "$flagfile" ]; then
     # We test our smallest manpage just to see if the python setup works.
@@ -32,4 +33,10 @@ if [ ! -f "$flagfile" ]; then
     fi
 fi
 
-"$srcdir/md-convert" "$srcdir/$inname"
+if [ -f "$force_flagfile" ]; then
+    opt='--force-link-text'
+else
+    opt=''
+fi
+
+"$srcdir/md-convert" $opt "$srcdir/$inname"

md-convert

@@ -276,7 +276,10 @@ class TransformHtml(HTMLParser):
                 bad_hashtags = set(),
                 latest_targets = [ ],
                 opt_prefix = 'opt',
+                a_href = None,
+                a_href_external = False,
                 a_txt_start = None,
+                after_a_tag = False,
                 target_suf = '',
                 )
 
@@ -315,6 +318,13 @@ class TransformHtml(HTMLParser):
         for bad in st.referenced_hashtags - st.created_hashtags:
             warn('Unknown hashtag link in', self.fn + ':', '#' + bad)
 
+    def handle_UE(self):
+        st = self.state
+        if st.txt.startswith(('.', ',', '!', '?', ';', ':')):
+            st.man_out[-1] = ".UE " + st.txt[0] + "\n"
+            st.txt = st.txt[1:]
+        st.after_a_tag = False
+
     def handle_starttag(self, tag, attrs_list):
         st = self.state
         if args.debug:
@@ -387,13 +397,20 @@ class TransformHtml(HTMLParser):
             for var, val in attrs_list:
                 if var == 'href':
                     if val.startswith(('https://', 'http://', 'mailto:', 'ftp:')):
-                        pass # nothing to check
+                        if st.after_a_tag:
+                            self.handle_UE()
+                        st.man_out.append(manify(st.txt.strip()) + "\n")
+                        st.man_out.append(".UR " + val + "\n")
+                        st.txt = ''
+                        st.a_href = val
+                        st.a_href_external = True
                     elif '#' in val:
                         pg, tgt = val.split('#', 1)
                         if pg and pg not in VALID_PAGES or '#' in tgt:
                             st.bad_hashtags.add(val)
                         elif tgt in ('', 'opt', 'dopt'):
                             st.a_href = val
+                            st.a_href_external = False
                         elif pg == '':
                             st.referenced_hashtags.add(tgt)
                             if tgt in st.latest_targets:
@@ -409,6 +426,8 @@ class TransformHtml(HTMLParser):
         st = self.state
         if args.debug:
             self.output_debug('END', (tag,))
+        if st.after_a_tag:
+            self.handle_UE()
         if tag in CONSUMES_TXT or st.dt_from == tag:
             txt = st.txt.strip()
             st.txt = ''
@@ -473,7 +492,15 @@ class TransformHtml(HTMLParser):
         elif tag == 'hr':
             return
         elif tag == 'a':
-            if st.a_href:
+            if st.a_href_external:
+                st.txt = st.txt.strip()
+                if args.force_link_text or st.a_href != st.txt:
+                    st.man_out.append(manify(st.txt) + "\n")
+                st.man_out.append(".UE\n") # This might get replaced with a punctuation version in handle_UE()
+                st.after_a_tag = True
+                st.a_href_external = False
+                st.txt = ''
+            elif st.a_href:
                 atxt = st.txt[st.a_txt_start:]
                 find = 'href="' + st.a_href + '"'
                 for j in range(len(st.html_out)-1, 0, -1):
@@ -612,6 +639,7 @@ if __name__ == '__main__':
     parser = argparse.ArgumentParser(description="Convert markdown into html and (optionally) nroff. Each input filename must have a .md suffix, which is changed to .html for the output filename. If the input filename ends with .num.md (e.g. foo.1.md) then a nroff file is also output with the input filename's .md suffix removed (e.g. foo.1).", add_help=False)
     parser.add_argument('--test', action='store_true', help="Just test the parsing without outputting any files.")
     parser.add_argument('--dest', metavar='DIR', help="Create files in DIR instead of the current directory.")
+    parser.add_argument('--force-link-text', action='store_true', help="Don't remove the link text if it matches the link href. Useful when nroff doesn't understand .UR and .UE.")
     parser.add_argument('--debug', '-D', action='count', default=0, help='Output copious info on the html parsing. Repeat for even more.')
     parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.")
     parser.add_argument("mdfiles", metavar='FILE.md', nargs='+', help="One or more .md files to convert.")

options.c

@@ -3,7 +3,7 @@
  *
  * Copyright (C) 1998-2001 Andrew Tridgell <tridge@samba.org>
  * Copyright (C) 2000, 2001, 2002 Martin Pool <mbp@samba.org>
- * Copyright (C) 2002-2022 Wayne Davison
+ * Copyright (C) 2002-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -200,6 +200,7 @@ int remote_option_cnt = 0;
 const char **remote_options = NULL;
 const char *checksum_choice = NULL;
 const char *compress_choice = NULL;
+static const char *empty_argv[1];
 
 int quiet = 0;
 int output_motd = 1;
@@ -1347,7 +1348,7 @@ char *alt_dest_opt(int type)
  **/
 int parse_arguments(int *argc_p, const char ***argv_p)
 {
-	static poptContext pc;
+	poptContext pc;
 	const char *arg, **argv = *argv_p;
 	int argc = *argc_p;
 	int opt, want_dest_type;
@@ -1367,10 +1368,6 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 
 	/* TODO: Call poptReadDefaultConfig; handle errors. */
 
-	/* The context leaks in case of an error, but if there's a
-	 * problem we always exit anyhow. */
-	if (pc)
-		poptFreeContext(pc);
 	pc = poptGetContext(RSYNC_NAME, argc, argv, long_options, 0);
 	if (!am_server) {
 		poptReadDefaultConfig(pc, 0);
@@ -1413,7 +1410,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				strlcpy(err_buf,
 					"Attempt to hack rsync thwarted!\n",
 					sizeof err_buf);
-				return 0;
+				goto cleanup;
 			}
 #ifdef ICONV_OPTION
 			iconv_opt = NULL;
@@ -1459,7 +1456,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			if (tmpdir && strlen(tmpdir) >= MAXPATHLEN - 10) {
 				snprintf(err_buf, sizeof err_buf,
 					 "the --temp-dir path is WAY too long.\n");
-				return 0;
+				goto cleanup;
 			}
 
 			if (!daemon_opt) {
@@ -1469,8 +1466,16 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				exit_cleanup(RERR_SYNTAX);
 			}
 
-			*argv_p = argv = poptGetArgs(pc);
-			*argc_p = argc = count_args(argv);
+			argv = poptGetArgs(pc);
+			argc = count_args(argv);
+			if (!argc) {
+				*argv_p = empty_argv;
+				*argc_p = 0;
+			} else if (poptDupArgv(argc, argv, argc_p, argv_p) != 0)
+				out_of_memory("parse_arguments");
+			argv = *argv_p;
+			poptFreeContext(pc);
+
 			am_starting_up = 0;
 			daemon_opt = 0;
 			am_daemon = 1;
@@ -1525,7 +1530,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		case 'a':
 			if (refused_archive_part) {
 				create_refuse_error(refused_archive_part);
-				return 0;
+				goto cleanup;
 			}
 			if (!recurse) /* preserve recurse == 2 */
 				recurse = 1;
@@ -1595,7 +1600,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		case 'P':
 			if (refused_partial || refused_progress) {
 				create_refuse_error(refused_partial ? refused_partial : refused_progress);
-				return 0;
+				goto cleanup;
 			}
 			do_progress = 1;
 			keep_partial = 1;
@@ -1630,7 +1635,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			if (*arg != '-') {
 				snprintf(err_buf, sizeof err_buf,
 					"Remote option must start with a dash: %s\n", arg);
-				return 0;
+				goto cleanup;
 			}
 			if (remote_option_cnt+2 >= remote_option_alloc) {
 				remote_option_alloc += 16;
@@ -1672,27 +1677,27 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			ssize_t size;
 			arg = poptGetOptArg(pc);
 			if ((size = parse_size_arg(arg, 'b', "block-size", 0, max_blength, False)) < 0)
-				return 0;
+				goto cleanup;
 			block_size = (int32)size;
 			break;
 		}
 
 		case OPT_MAX_SIZE:
 			if ((max_size = parse_size_arg(max_size_arg, 'b', "max-size", 0, -1, False)) < 0)
-				return 0;
+				goto cleanup;
 			max_size_arg = strdup(do_big_num(max_size, 0, NULL));
 			break;
 
 		case OPT_MIN_SIZE:
 			if ((min_size = parse_size_arg(min_size_arg, 'b', "min-size", 0, -1, False)) < 0)
-				return 0;
+				goto cleanup;
 			min_size_arg = strdup(do_big_num(min_size, 0, NULL));
 			break;
 
 		case OPT_BWLIMIT: {
 			ssize_t size = parse_size_arg(bwlimit_arg, 'K', "bwlimit", 512, -1, True);
 			if (size < 0)
-				return 0;
+				goto cleanup;
 			bwlimit_arg = strdup(do_big_num(size, 0, NULL));
 			bwlimit = (size + 512) / 1024;
 			break;
@@ -1721,7 +1726,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				snprintf(err_buf, sizeof err_buf,
 					"ERROR: the %s option conflicts with the %s option\n",
 					alt_dest_opt(want_dest_type), alt_dest_opt(0));
-				return 0;
+				goto cleanup;
 			}
 			alt_dest_type = want_dest_type;
 
@@ -1729,7 +1734,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				snprintf(err_buf, sizeof err_buf,
 					"ERROR: at most %d %s args may be specified\n",
 					MAX_BASIS_DIRS, alt_dest_opt(0));
-				return 0;
+				goto cleanup;
 			}
 			/* We defer sanitizing this arg until we know what
 			 * our destination directory is going to be. */
@@ -1742,7 +1747,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				snprintf(err_buf, sizeof err_buf,
 					"Invalid argument passed to --chmod (%s)\n",
 					arg);
-				return 0;
+				goto cleanup;
 			}
 			break;
 
@@ -1761,11 +1766,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				if (usermap_via_chown) {
 					snprintf(err_buf, sizeof err_buf,
 						"--usermap conflicts with prior --chown.\n");
-					return 0;
+					goto cleanup;
 				}
 				snprintf(err_buf, sizeof err_buf,
 					"You can only specify --usermap once.\n");
-				return 0;
+				goto cleanup;
 			}
 			usermap = (char *)poptGetOptArg(pc);
 			usermap_via_chown = False;
@@ -1777,11 +1782,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				if (groupmap_via_chown) {
 					snprintf(err_buf, sizeof err_buf,
 						"--groupmap conflicts with prior --chown.\n");
-					return 0;
+					goto cleanup;
 				}
 				snprintf(err_buf, sizeof err_buf,
 					"You can only specify --groupmap once.\n");
-				return 0;
+				goto cleanup;
 			}
 			groupmap = (char *)poptGetOptArg(pc);
 			groupmap_via_chown = False;
@@ -1800,11 +1805,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 					if (!usermap_via_chown) {
 						snprintf(err_buf, sizeof err_buf,
 							"--chown conflicts with prior --usermap.\n");
-						return 0;
+						goto cleanup;
 					}
 					snprintf(err_buf, sizeof err_buf,
 						"You can only specify a user-affecting --chown once.\n");
-					return 0;
+					goto cleanup;
 				}
 				if (asprintf(&usermap, "*:%.*s", len, chown) < 0)
 					out_of_memory("parse_arguments");
@@ -1816,11 +1821,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 					if (!groupmap_via_chown) {
 						snprintf(err_buf, sizeof err_buf,
 							"--chown conflicts with prior --groupmap.\n");
-						return 0;
+						goto cleanup;
 					}
 					snprintf(err_buf, sizeof err_buf,
 						"You can only specify a group-affecting --chown once.\n");
-					return 0;
+					goto cleanup;
 				}
 				if (asprintf(&groupmap, "*:%s", arg) < 0)
 					out_of_memory("parse_arguments");
@@ -1848,7 +1853,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			snprintf(err_buf,sizeof(err_buf),
 				 "ACLs are not supported on this %s\n",
 				 am_server ? "server" : "client");
-			return 0;
+			goto cleanup;
 #endif
 
 		case 'X':
@@ -1859,7 +1864,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			snprintf(err_buf,sizeof(err_buf),
 				 "extended attributes are not supported on this %s\n",
 				 am_server ? "server" : "client");
-			return 0;
+			goto cleanup;
 #endif
 
 		case OPT_STOP_AFTER: {
@@ -1868,7 +1873,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			stop_at_utime = time(NULL);
 			if ((val = atol(arg) * 60) <= 0 || LONG_MAX - val < stop_at_utime || (long)(time_t)val != val) {
 				snprintf(err_buf, sizeof err_buf, "invalid --stop-after value: %s\n", arg);
-				return 0;
+				goto cleanup;
 			}
 			stop_at_utime += val;
 			break;
@@ -1879,11 +1884,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			arg = poptGetOptArg(pc);
 			if ((stop_at_utime = parse_time(arg)) == (time_t)-1) {
 				snprintf(err_buf, sizeof err_buf, "invalid --stop-at format: %s\n", arg);
-				return 0;
+				goto cleanup;
 			}
 			if (stop_at_utime <= time(NULL)) {
 				snprintf(err_buf, sizeof err_buf, "--stop-at time is not in the future: %s\n", arg);
-				return 0;
+				goto cleanup;
 			}
 			break;
 #endif
@@ -1901,7 +1906,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			else {
 				snprintf(err_buf, sizeof err_buf,
 					"--stderr mode \"%s\" is not one of errors, all, or client\n", arg);
-				return 0;
+				goto cleanup;
 			}
 			saw_stderr_opt = 1;
 			break;
@@ -1912,13 +1917,13 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			 * turned this option off. */
 			if (opt >= OPT_REFUSED_BASE) {
 				create_refuse_error(opt);
-				return 0;
+				goto cleanup;
 			}
 			snprintf(err_buf, sizeof err_buf, "%s%s: %s\n",
 				 am_server ? "on remote machine: " : "",
 				 poptBadOption(pc, POPT_BADOPTION_NOALIAS),
 				 poptStrerror(opt));
-			return 0;
+			goto cleanup;
 		}
 	}
 
@@ -1938,9 +1943,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (max_alloc_arg) {
 		ssize_t size = parse_size_arg(max_alloc_arg, 'B', "max-alloc", 1024*1024, -1, True);
 		if (size < 0)
-			return 0;
+			goto cleanup;
 		max_alloc = size;
 	}
+	if (!max_alloc)
+		max_alloc = SIZE_MAX;
 
 	if (old_style_args < 0) {
 		if (!am_server && protect_args <= 0 && (arg = getenv("RSYNC_OLD_ARGS")) != NULL && *arg) {
@@ -1952,7 +1959,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		if (protect_args > 0) {
 			snprintf(err_buf, sizeof err_buf,
 				 "--secluded-args conflicts with --old-args.\n");
-			return 0;
+			goto cleanup;
 		}
 		protect_args = 0;
 	}
@@ -1997,7 +2004,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			do_compression = CPRES_AUTO;
 		if (do_compression && refused_compress) {
 			create_refuse_error(refused_compress);
-			return 0;
+			goto cleanup;
 		}
 	}
 
@@ -2022,7 +2029,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		default:
 			snprintf(err_buf, sizeof err_buf,
 				"Invalid --outbuf setting -- specify N, L, or B.\n");
-			return 0;
+			goto cleanup;
 		}
 		setvbuf(stdout, (char *)NULL, mode, 0);
 	}
@@ -2050,7 +2057,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	}
 	if (refused_no_iconv && !iconv_opt) {
 		create_refuse_error(refused_no_iconv);
-		return 0;
+		goto cleanup;
 	}
 #endif
 
@@ -2061,18 +2068,30 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (orig_protect_args == 2 && am_server)
 		protect_args = orig_protect_args;
 
-	if (protect_args == 1 && am_server)
+	if (protect_args == 1 && am_server) {
+		poptFreeContext(pc);
 		return 1;
+	}
 
-	*argv_p = argv = poptGetArgs(pc);
-	*argc_p = argc = count_args(argv);
+	/* Because popt 1.19 has started to free the returned args data, we now
+	 * make a copy of the array and then do an immediate cleanup. */
+	argv = poptGetArgs(pc);
+	argc = count_args(argv);
+	if (!argc) {
+		*argv_p = empty_argv;
+		*argc_p = 0;
+	} else if (poptDupArgv(argc, argv, argc_p, argv_p) != 0)
+		out_of_memory("parse_arguments");
+	argv = *argv_p;
+	poptFreeContext(pc);
+	pc = NULL;
 
 #ifndef SUPPORT_LINKS
 	if (preserve_links && !am_sender) {
 		snprintf(err_buf, sizeof err_buf,
 			 "symlinks are not supported on this %s\n",
 			 am_server ? "server" : "client");
-		return 0;
+		goto cleanup;
 	}
 #endif
 
@@ -2081,7 +2100,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		snprintf(err_buf, sizeof err_buf,
 			 "hard links are not supported on this %s\n",
 			 am_server ? "server" : "client");
-		return 0;
+		goto cleanup;
 	}
 #endif
 
@@ -2089,20 +2108,20 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (am_root < 0 && preserve_xattrs > 1) {
 		snprintf(err_buf, sizeof err_buf,
 			 "--fake-super conflicts with -XX\n");
-		return 0;
+		goto cleanup;
 	}
 #else
 	if (am_root < 0) {
 		snprintf(err_buf, sizeof err_buf,
 			 "--fake-super requires an rsync with extended attributes enabled\n");
-		return 0;
+		goto cleanup;
 	}
 #endif
 
 	if (write_batch && read_batch) {
 		snprintf(err_buf, sizeof err_buf,
 			"--write-batch and --read-batch can not be used together\n");
-		return 0;
+		goto cleanup;
 	}
 	if (write_batch > 0 || read_batch) {
 		if (am_server) {
@@ -2121,25 +2140,25 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (read_batch && files_from) {
 		snprintf(err_buf, sizeof err_buf,
 			"--read-batch cannot be used with --files-from\n");
-		return 0;
+		goto cleanup;
 	}
 	if (read_batch && remove_source_files) {
 		snprintf(err_buf, sizeof err_buf,
 			"--read-batch cannot be used with --remove-%s-files\n",
 			remove_source_files == 1 ? "source" : "sent");
-		return 0;
+		goto cleanup;
 	}
 	if (batch_name && strlen(batch_name) > MAX_BATCH_NAME_LEN) {
 		snprintf(err_buf, sizeof err_buf,
 			"the batch-file name must be %d characters or less.\n",
 			MAX_BATCH_NAME_LEN);
-		return 0;
+		goto cleanup;
 	}
 
 	if (tmpdir && strlen(tmpdir) >= MAXPATHLEN - 10) {
 		snprintf(err_buf, sizeof err_buf,
 			 "the --temp-dir path is WAY too long.\n");
-		return 0;
+		goto cleanup;
 	}
 
 	if (max_delete < 0 && max_delete != INT_MIN) {
@@ -2173,7 +2192,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (delete_before + !!delete_during + delete_after > 1) {
 		snprintf(err_buf, sizeof err_buf,
 			"You may not combine multiple --delete-WHEN options.\n");
-		return 0;
+		goto cleanup;
 	}
 	if (delete_before || delete_during || delete_after)
 		delete_mode = 1;
@@ -2184,7 +2203,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				delete_during = 1;
 			else {
 				create_refuse_error(refused_delete_before);
-				return 0;
+				goto cleanup;
 			}
 		} else if (refused_delete_during)
 			delete_before = 1;
@@ -2193,14 +2212,14 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (!xfer_dirs && delete_mode) {
 		snprintf(err_buf, sizeof err_buf,
 			"--delete does not work without --recursive (-r) or --dirs (-d).\n");
-		return 0;
+		goto cleanup;
 	}
 
 	if (missing_args == 3) /* simplify if both options were specified */
 		missing_args = 2;
 	if (refused_delete && (delete_mode || missing_args == 2)) {
 		create_refuse_error(refused_delete);
-		return 0;
+		goto cleanup;
 	}
 
 	if (remove_source_files) {
@@ -2209,7 +2228,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		 * options. */
 		if (refused_delete && am_sender) {
 			create_refuse_error(refused_delete);
-			return 0;
+			goto cleanup;
 		}
 		need_messages_from_generator = 1;
 	}
@@ -2263,7 +2282,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		snprintf(err_buf, sizeof err_buf,
 			"--suffix cannot contain slashes: %s\n",
 			backup_suffix);
-		return 0;
+		goto cleanup;
 	}
 	if (backup_dir) {
 		size_t len;
@@ -2276,7 +2295,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		if (len > sizeof backup_dir_buf - 128) {
 			snprintf(err_buf, sizeof err_buf,
 				"the --backup-dir path is WAY too long.\n");
-			return 0;
+			goto cleanup;
 		}
 		backup_dir_len = (int)len;
 		if (!backup_dir_len) {
@@ -2295,7 +2314,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			"--suffix cannot be empty %s\n", backup_dir_len < 0
 			? "when --backup-dir is the same as the dest dir"
 			: "without a --backup-dir");
-		return 0;
+		goto cleanup;
 	} else if (make_backups && delete_mode && !delete_excluded && !am_server) {
 		snprintf(backup_dir_buf, sizeof backup_dir_buf,
 			"P *%s", backup_suffix);
@@ -2323,7 +2342,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (do_progress && !am_server) {
 		if (!log_before_transfer && INFO_EQ(NAME, 0))
 			parse_output_words(info_words, info_levels, "name", DEFAULT_PRIORITY);
-		parse_output_words(info_words, info_levels, "flist2,progress", DEFAULT_PRIORITY);
+		parse_output_words(info_words, info_levels, "FLIST2,PROGRESS", DEFAULT_PRIORITY);
 	}
 
 	if (dry_run)
@@ -2364,11 +2383,11 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 		if (whole_file > 0) {
 			snprintf(err_buf, sizeof err_buf,
 				 "--append cannot be used with --whole-file\n");
-			return 0;
+			goto cleanup;
 		}
 		if (refused_inplace) {
 			create_refuse_error(refused_inplace);
-			return 0;
+			goto cleanup;
 		}
 		inplace = 1;
 	}
@@ -2376,7 +2395,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 	if (write_devices) {
 		if (refused_inplace) {
 			create_refuse_error(refused_inplace);
-			return 0;
+			goto cleanup;
 		}
 		inplace = 1;
 	}
@@ -2391,13 +2410,13 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				 "--%s cannot be used with --%s\n",
 				 append_mode ? "append" : "inplace",
 				 delay_updates ? "delay-updates" : "partial-dir");
-			return 0;
+			goto cleanup;
 		}
 		/* --inplace implies --partial for refusal purposes, but we
 		 * clear the keep_partial flag for internal logic purposes. */
 		if (refused_partial) {
 			create_refuse_error(refused_partial);
-			return 0;
+			goto cleanup;
 		}
 		keep_partial = 0;
 #else
@@ -2405,7 +2424,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			 "--%s is not supported on this %s\n",
 			 append_mode ? "append" : "inplace",
 			 am_server ? "server" : "client");
-		return 0;
+		goto cleanup;
 #endif
 	} else {
 		if (keep_partial && !partial_dir && !am_server) {
@@ -2419,7 +2438,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				partial_dir = NULL;
 			if (!partial_dir && refused_partial) {
 				create_refuse_error(refused_partial);
-				return 0;
+				goto cleanup;
 			}
 			keep_partial = 1;
 		}
@@ -2440,14 +2459,14 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 			if (am_server) {
 				snprintf(err_buf, sizeof err_buf,
 					"The --files-from sent to the server cannot specify a host.\n");
-				return 0;
+				goto cleanup;
 			}
 			files_from = p;
 			filesfrom_host = h;
 			if (strcmp(files_from, "-") == 0) {
 				snprintf(err_buf, sizeof err_buf,
 					"Invalid --files-from remote filename\n");
-				return 0;
+				goto cleanup;
 			}
 		} else {
 			if (sanitize_paths)
@@ -2466,7 +2485,7 @@ int parse_arguments(int *argc_p, const char ***argv_p)
 				snprintf(err_buf, sizeof err_buf,
 					"failed to open files-from file %s: %s\n",
 					files_from, strerror(errno));
-				return 0;
+				goto cleanup;
 			}
 		}
 	}
@@ -2483,6 +2502,9 @@ int parse_arguments(int *argc_p, const char ***argv_p)
   options_rejected:
 	snprintf(err_buf, sizeof err_buf,
 		"Your options have been rejected by the server.\n");
+  cleanup:
+	if (pc)
+		poptFreeContext(pc);
 	return 0;
 }
 
@@ -2498,7 +2520,7 @@ static char SPLIT_ARG_WHEN_OLD[1];
  **/
 char *safe_arg(const char *opt, const char *arg)
 {
-#define SHELL_CHARS "!#$&;|<>(){}\"' \t\\"
+#define SHELL_CHARS "!#$&;|<>(){}\"'` \t\\"
 #define WILD_CHARS  "*?[]" /* We don't allow remote brace expansion */
 	BOOL is_filename_arg = !opt;
 	char *escapes = is_filename_arg ? SHELL_CHARS : WILD_CHARS SHELL_CHARS;
@@ -2510,7 +2532,7 @@ char *safe_arg(const char *opt, const char *arg)
 	char *ret;
 	if (!protect_args && old_style_args < 2 && (!old_style_args || (!is_filename_arg && opt != SPLIT_ARG_WHEN_OLD))) {
 		const char *f;
-		if (!trust_sender_args && *arg == '~'
+		if (*arg == '~' && is_filename_arg && !am_sender && !trust_sender_args
 		 && ((relative_paths && !strstr(arg, "/./"))
 		  || !strchr(arg, '/'))) {
 			extras++;

packaging/lsb/rsync.spec

@@ -1,6 +1,6 @@
 Summary: A fast, versatile, remote (and local) file-copying tool
 Name: rsync
-Version: 3.2.7
+Version: 3.3.0
 %define fullversion %{version}
 Release: 1
 %define srcdir src
@@ -79,8 +79,8 @@ rm -rf $RPM_BUILD_ROOT
 %dir /etc/rsync-ssl/certs
 
 %changelog
-* Thu Oct 20 2022 Wayne Davison <wayne@opencoder.net>
-Released 3.2.7.
+* Sat Apr 06 2024 Wayne Davison <wayne@opencoder.net>
+Released 3.3.0.
 
 * Fri Mar 21 2008 Wayne Davison <wayne@opencoder.net>
 Added installation of /etc/xinetd.d/rsync file and some commented-out

receiver.c

@@ -3,7 +3,7 @@
  *
  * Copyright (C) 1996-2000 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
- * Copyright (C) 2003-2022 Wayne Davison
+ * Copyright (C) 2003-2023 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -372,7 +372,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 
 	if (fd != -1 && offset > 0) {
 		if (sparse_files > 0) {
-			if (sparse_end(fd, offset) != 0)
+			if (sparse_end(fd, offset, updating_basis_or_equiv) != 0)
 				goto report_write_error;
 		} else if (flush_write_file(fd) < 0) {
 		    report_write_error:

rsync.1.md

@@ -245,7 +245,7 @@ to be copied to different destination directories using more than one copy.
 
 While a copy of a case-ignoring filesystem to a case-ignoring filesystem can
 work out fairly well, if no `--delete-during` or `--delete-before` option is
-active, rsync can potentially update an existing file on the receiveing side
+active, rsync can potentially update an existing file on the receiving side
 without noticing that the upper-/lower-case of the filename should be changed
 to match the sender.
 
@@ -1636,7 +1636,9 @@ expand it.
 0.  `--crtimes`, `-N,`
 
     This tells rsync to set the create times (newness) of the destination
-    files to the same value as the source files.
+    files to the same value as the source files. Your OS & filesystem must
+    support the setting of arbitrary creation (birth) times for this option
+    to be supported.
 
 0.  `--omit-dir-times`, `-O`
 
@@ -2104,7 +2106,8 @@ expand it.
     See the [`--max-size`](#opt) option for a description of how SIZE can be
     specified.  The default suffix if none is given is bytes.
 
-    Beginning in 3.2.3, a value of 0 specifies no limit.
+    Beginning in 3.2.7, a value of 0 is an easy way to specify SIZE_MAX (the
+    largest limit possible).
 
     You can set a default value using the environment variable
     [`RSYNC_MAX_ALLOC`](#) using the same SIZE values as supported by this

rsyncd.conf.5.md

@@ -74,25 +74,7 @@ reread the `rsyncd.conf` file. The file is re-read on each client connection.
 ## GLOBAL PARAMETERS
 
 The first parameters in the file (before a [module] header) are the global
-parameters.  Rsync also allows for the use of a "[global]" module name to
-indicate the start of one or more global-parameter sections (the name must be
-lower case).
-
-You may also include any module parameters in the global part of the config
-file in which case the supplied value will override the default for that
-parameter.
-
-You may use references to environment variables in the values of parameters.
-String parameters will have %VAR% references expanded as late as possible (when
-the string is first used in the program), allowing for the use of variables
-that rsync sets at connection time, such as RSYNC_USER_NAME.  Non-string
-parameters (such as true/false settings) are expanded when read from the config
-file.  If a variable does not exist in the environment, or if a sequence of
-characters is not a valid reference (such as an un-paired percent sign), the
-raw characters are passed through unchanged.  This helps with backward
-compatibility and safety (e.g. expanding a non-existent %VAR% to an empty
-string in a path could result in a very unsafe path).  The safest way to insert
-a literal % into a value is to use %%.
+parameters:
 
 [comment]: # (An OL starting at 0 is converted into a DL by the parser.)
 
@@ -138,6 +120,22 @@ a literal % into a value is to use %%.
     You can override the default backlog value when the daemon listens for
     connections.  It defaults to 5.
 
+You may also include any [MODULE PARAMETERS](#) in the global part of the
+config file, in which case the supplied value will override the default for
+that parameter.
+
+You may use references to environment variables in the values of parameters.
+String parameters will have %VAR% references expanded as late as possible (when
+the string is first used in the program), allowing for the use of variables
+that rsync sets at connection time, such as RSYNC_USER_NAME.  Non-string
+parameters (such as true/false settings) are expanded when read from the config
+file.  If a variable does not exist in the environment, or if a sequence of
+characters is not a valid reference (such as an un-paired percent sign), the
+raw characters are passed through unchanged.  This helps with backward
+compatibility and safety (e.g. expanding a non-existent %VAR% to an empty
+string in a path could result in a very unsafe path).  The safest way to insert
+a literal % into a value is to use %%.
+
 ## MODULE PARAMETERS
 
 After the global parameters you should define a number of modules, each module
@@ -146,11 +144,17 @@ a module name in square brackets [module] followed by the parameters for that
 module.  The module name cannot contain a slash or a closing square bracket.
 If the name contains whitespace, each internal sequence of whitespace will be
 changed into a single space, while leading or trailing whitespace will be
-discarded.  Also, the name cannot be "global" as that exact name indicates that
-global parameters follow (see above).
+discarded.
 
-As with GLOBAL PARAMETERS, you may use references to environment variables in
-the values of parameters.  See the GLOBAL PARAMETERS section for more details.
+There is also a special module name of "[global]" that does not define a module
+but instead switches back to the global settings context where default
+parameters can be specified.  Because each defined module gets its full set of
+parameters as a combination of the default values that are set at that position
+in the config file plus its own parameter list, the use of a "[global]" section
+can help to maintain shared config values for multiple modules.
+
+As with [GLOBAL PARAMETERS](#), you may use references to environment variables
+in the values of parameters.  See that section for details.
 
 0.  `comment`
 
@@ -1019,7 +1023,7 @@ the values of parameters.  See the GLOBAL PARAMETERS section for more details.
     _not_ displayed if the script returns success.  The other programs cannot
     send any text to the user.  All output except for the `pre-xfer exec`
     stdout goes to the corresponding daemon's stdout/stderr, which is typically
-    discarded.  See the `--no-detatch` option for a way to see the daemon's
+    discarded.  See the `--no-detach` option for a way to see the daemon's
     output, which can assist with debugging.
 
     Note that the `early exec` command runs before any part of the transfer

support/cvs2includes

@@ -1,42 +1,63 @@
-#!/usr/bin/env perl
-#
+#!/usr/bin/env python3
 # This script finds all CVS/Entries files in the current directory and below
 # and creates a local .cvsinclude file with non-inherited rules including each
 # checked-in file.  Then, use this option whenever using --cvs-exclude (-C):
 #
-# 	-f ': .cvsinclude'
+#    -f ': .cvsinclude'
 #
 # That ensures that all checked-in files/dirs are included in the transfer.
 # (You could alternately put ": .cvsinclude" into an .rsync-filter file and
 # use the -F option, which is easier to type.)
 #
 # The downside is that you need to remember to re-run cvs2includes whenever
-# you add a new file to the project.
-use strict;
+# CVS gets an added or removed file. Maybe just run it before every copy.
 
-open(FIND, 'find . -name CVS -type d |') or die $!;
-while (<FIND>) {
-    chomp;
-    s#^\./##;
+import os, argparse
 
-    my $entries = "$_/Entries";
-    s/CVS$/.cvsinclude/;
-    my $filter = $_;
+INC_NAME = '.cvsinclude'
 
-    open(ENTRIES, $entries) or die "Unable to open $entries: $!\n";
-    my @includes;
-    while (<ENTRIES>) {
-	push(@includes, $1) if m#/(.+?)/#;
-    }
-    close ENTRIES;
-    if (@includes) {
-	open(FILTER, ">$filter") or die "Unable to write $filter: $!\n";
-	print FILTER map "+ /$_\n", @includes;
-	close FILTER;
-	print "Updated $filter\n";
-    } elsif (-f $filter) {
-	unlink($filter);
-	print "Removed $filter\n";
-    }
-}
-close FIND;
+def main():
+    if args.dir:
+        os.chdir(args.dir)
+
+    cvs_includes = set()
+    for root, dirs, files in os.walk('.'):
+        if INC_NAME in files:
+            cvs_includes.add((root + '/' + INC_NAME)[2:])
+        if root.endswith('/CVS') and 'Entries' in files:
+            entries = root[2:] + '/Entries'
+            includes = [ ]
+            with open(entries) as fh:
+                for line in fh:
+                    if line.startswith(('/', 'D/')):
+                        includes.append(line.split('/', 2)[1])
+            if includes:
+                inc = root[2:-3] + INC_NAME
+                cvs_includes.discard(inc)
+                try:
+                    with open(inc) as fh:
+                        old_txt = fh.read()
+                except OSError:
+                    old_txt = ''
+                txt = ''.join(f"+ /{x}\n" for x in includes)
+                if txt == old_txt:
+                    print("Unchanged", inc)
+                else:
+                    print("Updating", inc)
+                    with open(inc, 'w') as fh:
+                        fh.write(txt)
+        dirs.sort()
+
+    for inc in sorted(cvs_includes):
+        print("Removing", inc)
+        os.unlink(inc)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=f"Transform CVS/Entries into {INC_NAME} files.", add_help=False)
+    parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.")
+    parser.add_argument("dir", nargs='?', help="The top CVS dir. Defaults to the current directory.")
+    args = parser.parse_args()
+    main()
+
+# vim: sw=4 et

support/files-to-excludes

@@ -1,27 +1,37 @@
-#!/usr/bin/env perl
-# This script takes an input of filenames and outputs a set of
-# include/exclude directives that can be used by rsync to copy
-# just the indicated files using an --exclude-from=FILE option.
-use strict;
+#!/usr/bin/env python3
+# This script takes an input of filenames and outputs a set of include/exclude
+# directives that can be used by rsync to copy just the indicated files using
+# an --exclude-from=FILE or -f'. FILE' option. To be able to delete files on
+# the receiving side, either use --delete-excluded or change the exclude (-)
+# rules to hide filter rules (H) that only affect the sending side.
 
-my %hash;
+import os, fileinput, argparse
 
-while (<>) {
-    chomp;
-    s#^/+##;
-    my $path = '/';
-    while (m#([^/]+/)/*#g) {
-	$path .= $1;
-	print "+ $path\n" unless $hash{$path}++;
-    }
-    if (m#([^/]+)$#) {
-	print "+ $path$1\n";
-    } else {
-	delete $hash{$path};
-    }
-}
+def main():
+    paths = set()
+    for line in fileinput.input(args.files):
+        dirs = line.strip().lstrip('/').split('/')
+        if not dirs:
+            continue
+        for j in range(1, len(dirs)):
+            if dirs[j] == '':
+                continue
+            path = '/' + '/'.join(dirs[:j]) + '/'
+            if path not in paths:
+                print('+', path)
+                paths.add(path)
+        print('+', '/' + '/'.join(dirs))
 
-foreach (sort keys %hash) {
-    print "- $_*\n";
-}
-print "- /*\n";
+    for path in sorted(paths):
+        print('-', path + '*')
+    print('-', '/*')
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description="Transform a list of files into a set of include/exclude rules.", add_help=False)
+    parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.")
+    parser.add_argument("files", metavar="FILE", default='-', nargs='*', help="The file(s) that hold the pathnames to translate. Defaults to stdin.")
+    args = parser.parse_args()
+    main()
+
+# vim: sw=4 et

support/idmap

@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+# This helper script makes it easy to use a passwd or group file to map values
+# in a LOCAL transfer.  For instance, if you mount a backup that does not have
+# the same passwd setup as the local machine, you can do a copy to/from the
+# backup area as follows and get the differing ID values mapped just like a
+# remote transfer to/from the backed-up machine would do:
+#
+# rsync -av --usermap=`idmap --to /mnt/backup/etc/passwd` \
+#           --groupmap=`idmap --to /mnt/backup/etc/group` \
+#           /some/src/ /mnt/backup/some/dest/
+#
+# rsync -av --usermap=`idmap --from /mnt/backup/etc/passwd` \
+#           --groupmap=`idmap --from /mnt/backup/etc/group` \
+#           /mnt/backup/some/src/ /some/dest/
+
+import re, fileinput, argparse
+
+NAME_ID_RE = re.compile(r'^(\w+):[^:]+:(\d+)')
+
+def main():
+    maps = [ ]
+    for line in fileinput.input(args.files):
+        m = NAME_ID_RE.match(line)
+        if not m:
+            continue
+        if args.to:
+            pair = (m[1], m[2])
+        else:
+            pair = (m[2], m[1])
+        maps.append(':'.join(pair))
+    print(','.join(maps))
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description="Output usermap or groupmap args for rsync.", add_help=False)
+    action = parser.add_argument_group()
+    action = parser.add_mutually_exclusive_group(required=True)
+    action.add_argument("--from", action="store_true", help="Output the map for use on the sending side.")
+    action.add_argument("--to", action="store_true", help="Output the map for use on the receiving side.")
+    parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.")
+    parser.add_argument("files", metavar="FILE", default='-', nargs='*', help="The file(s) that hold the name & id pairs. Defaults to stdin.")
+    args = parser.parse_args()
+    main()
+
+# vim: sw=4 et

support/json-rsync-version

@@ -1,4 +1,4 @@
-#!/usr/bin/python3
+#!/usr/bin/env python3
 
 import sys, argparse, subprocess, json
 

support/mapfrom

@@ -1,15 +0,0 @@
-#!/usr/bin/env perl
-# This helper script makes it easy to use a passwd or group file to map
-# values in a LOCAL transfer.  For instance, if you mount a backup that
-# does not have the same passwd setup as the local machine, you can do
-# a copy FROM the backup area as follows and get the differing ID values
-# mapped just like a remote transfer FROM the backed-up machine would do:
-#
-# rsync -av --usermap=`mapfrom /mnt/backup/etc/passwd` \
-#           --groupmap=`mapfrom /mnt/backup/etc/group` \
-#           /mnt/backup/some/src/ /some/dest/
-
-while (<>) {
-    push @_, "$2:$1" if /^(\w+):[^:]+:(\d+)/;
-}
-print join(',', @_), "\n";

support/mapto

@@ -1,15 +0,0 @@
-#!/usr/bin/env perl
-# This helper script makes it easy to use a passwd or group file to map
-# values in a LOCAL transfer.  For instance, if you mount a backup that
-# does not have the same passwd setup as the local machine, you can do
-# a copy TO the backup area as follows and get the differing ID values
-# mapped just like a remote transfer TO the backed-up machine would do:
-#
-# rsync -av --usermap=`mapto /mnt/backup/etc/passwd` \
-#           --groupmap=`mapto /mnt/backup/etc/group` \
-#           /some/src/ /mnt/backup/some/dest/
-
-while (<>) {
-    push @_, "$1:$2" if /^(\w+):[^:]+:(\d+)/;
-}
-print join(',', @_), "\n";

support/mnt-excl

@@ -1,4 +1,4 @@
-#!/usr/bin/env perl
+#!/usr/bin/env python3
 # This script takes a command-line arg of a source directory
 # that will be passed to rsync, and generates a set of excludes
 # that will exclude all mount points from the list.  This is
@@ -27,23 +27,33 @@
 # awk '{print $2}' /proc/mounts | grep -v '^/$' | \
 #   rsync -avf 'merge,/- -' /dir host:/dest/
 
-use strict;
-use warnings;
-use Cwd 'abs_path';
+import os, argparse
 
-my $file = '/proc/mounts';
-my $dir = shift || '/';
-my $trailing_slash = $dir =~ m{./$} ? '/' : '';
-$dir = abs_path($dir) . $trailing_slash;
-$dir =~ s{([^/]*)$}{};
-my $trailing = $1;
-$trailing = '' if $trailing eq '.' || !-d "$dir$trailing";
-$trailing .= '/' if $trailing ne '';
+MNT_FILE = '/proc/mounts';
 
-open(IN, $file) or die "Unable to open $file: $!\n";
-while (<IN>) {
-    $_ = (split)[1];
-    next unless s{^\Q$dir$trailing\E}{}o && $_ ne '';
-    print "- /$trailing$_\n";
-}
-close IN;
+def main():
+    trailing_slash = '/' if args.path.endswith(('/', '/.')) and args.path != '/' else ''
+    args.path = os.path.realpath(args.path) + trailing_slash
+    parent_dir = os.path.dirname(args.path)
+    trailing = os.path.basename(args.path)
+    if not os.path.isdir(args.path):
+        trailing = ''
+    elif trailing != '':
+        trailing += '/'
+    want_path = os.path.join(parent_dir, trailing)
+    wp_len = len(want_path)
+
+    with open(MNT_FILE) as fh:
+        for line in fh:
+            mnt_path = line.split()[1]
+            if mnt_path.startswith(want_path) and mnt_path != want_path:
+                print(f"- /{trailing}{mnt_path[wp_len:]}")
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description="Output mount points as rsync excludes.", add_help=False)
+    parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.")
+    parser.add_argument('path', metavar='PATH', nargs='?', default='/', help="Limit output to those within the PATH hierarchy.")
+    args = parser.parse_args()
+    main()
+
+# vim: sw=4 et

support/rrsync

@@ -258,6 +258,9 @@ def main():
 
     if args.munge:
         rsync_opts.append('--munge-links')
+    
+    if args.no_overwrite:
+      rsync_opts.append('--ignore-existing')
 
     if not rsync_args:
         rsync_args = [ '.' ]
@@ -364,6 +367,7 @@ if __name__ == '__main__':
     arg_parser.add_argument('-munge', action='store_true', help="Enable rsync's --munge-links on the server side.")
     arg_parser.add_argument('-no-del', action='store_true', help="Disable rsync's --delete* and --remove* options.")
     arg_parser.add_argument('-no-lock', action='store_true', help="Avoid the single-run (per-user) lock check.")
+    arg_parser.add_argument('-no-overwrite', action='store_true', help="Prevent overwriting existing files by enforcing --ignore-existing")
     arg_parser.add_argument('-help', '-h', action='help', help="Output this help message and exit.")
     arg_parser.add_argument('dir', metavar='DIR', help="The restricted directory to use.")
     args = arg_parser.parse_args()

support/rrsync.1.md

@@ -5,7 +5,7 @@ rrsync - a script to setup restricted rsync users via ssh logins
 ## SYNOPSIS
 
 ```
-rrsync [-ro|-rw] [-munge] [-no-del] [-no-lock] DIR
+rrsync [-ro|-rw] [-munge] [-no-del] [-no-lock] [-no-overwrite]  DIR
 ```
 
 The single non-option argument specifies the restricted _DIR_ to use. It can be
@@ -85,6 +85,11 @@ The remainder of this manpage is dedicated to using the rrsync script.
 
     Avoid the single-run (per-user) lock check.  Useful with [`-munge`](#opt).
 
+0.  `-no-overwrite`
+
+    Enforce `--ignore-existing` on the server. Prevents overwriting existing
+    files when the server is the receiver.
+
 0.  `-help`, `-h`
 
     Output this help message and exit.

usage.c

@@ -293,6 +293,7 @@ void print_rsync_version(enum logcode f)
 		json_line("license", "GPLv3");
 		json_line("caveat", "rsync comes with ABSOLUTELY NO WARRANTY");
 		printf("\n}\n");
+		fflush(stdout);
 		return;
 	}
 

util2.c

@@ -4,7 +4,7 @@
  * Copyright (C) 1996-2000 Andrew Tridgell
  * Copyright (C) 1996 Paul Mackerras
  * Copyright (C) 2001, 2002 Martin Pool <mbp@samba.org>
- * Copyright (C) 2003-2020 Wayne Davison
+ * Copyright (C) 2003-2024 Wayne Davison
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -72,7 +72,7 @@ int msleep(int t)
 
 void *my_alloc(void *ptr, size_t num, size_t size, const char *file, int line)
 {
-	if (max_alloc && num >= max_alloc/size) {
+	if (num >= max_alloc/size) {
 		if (!file)
 			return NULL;
 		rprintf(FERROR, "[%s] exceeded --max-alloc=%s setting (file=%s, line=%d)\n",

version.h

@@ -1,2 +1,2 @@
-#define RSYNC_VERSION "3.2.7"
+#define RSYNC_VERSION "3.3.0"
 #define MAINTAINER_TZ_OFFSET -7.0