rrsync: pin receiver option directories the peer can pivot

A checked receiver-side directory option whose leaf does not exist can
be created as attacker-controlled transfer content and then consumed by
the same transfer: --backup-dir=a, with "a" arriving in-band as a
symlink pointing out of the restricted directory.  The same shape works
against --copy-dest, where it reads an outside file and delivers it to
the client.  Passing the leaf under the parent's /proc/self/fd pin is
not enough: the peer's symlink wins the race to the name, and rsync
creates or reads through it.

The answer is to hand rsync an inode rather than a name -- but what
inode depends on what rsync does with the option, so the policy is per
option rather than per type ("type 2" means "check when receiving", not
"is a directory"):

  --backup-dir, --partial-dir   rsync creates them on demand.  rrsync
                                creates them instead, walking down from
                                the restricted dir one component at a
                                time with O_NOFOLLOW (nested names too:
                                make_bak_dir() builds a hierarchy), and
                                pins the result.  The partial dir is
                                made 0700, as rsync makes it.
  --temp-dir                    rsync requires it to exist, so a missing
                                one stays an error.
  --link-dest, --compare-dest,  rsync only reads through these, and a
  --copy-dest                   missing one is the ordinary first-run
                                case that must keep working.  rrsync
                                pins an empty directory it then unlinks:
                                the transfer behaves as with a missing
                                one, and there is no name left for the
                                peer to take over.  Not quite identical:
                                rsync prints "--link-dest arg does not
                                exist" for a genuinely missing basis and
                                the placeholder suppresses that.

If the transfer later replaces a created name, the held inode is merely
detached -- the backup fails, it does not escape.

Without /proc/self/fd there is no way to name an inode, so on those
platforms every missing type-2 option path is refused instead -- the six
above, not the type-3 paths covered at the end.
That is the same fail-closed behaviour this change originally had
everywhere; the pinning is what buys back first use where it can.

An earlier version refused every missing type-2 leaf on every platform.
That closed the pivot but broke first-use --backup-dir and --partial-dir
-- including --partial-dir=.rsync-partial, the documented resumable-
upload idiom -- and would have broken first-run --link-dest, which is
how every rotating-snapshot script starts.

The regressions run their security assertions BEFORE their controls, so
an environment where a control fails for an unrelated reason cannot mask
the escape check by aborting first -- which is exactly what happened to
one reviewer.  The alt-dest one also has to beat a race: the generator
runs ahead of the receiver, so sorting the pivot symlink first does not
guarantee it is installed before the basis lookup, and a run where the
generator won would pass vacuously.  A few thousand files in between
give the receiver the head start, and the test asserts the symlink was
really installed rather than trusting the ordering.  Measured 5/5 RED on
the parent, 3/3 GREEN here.

The regressions cover each branch of the policy: the backup-dir pivot,
the --copy-dest read escape (its own test), first-use --backup-dir,
nested first-use --backup-dir, first-use --partial-dir including its
mode, a refused missing --temp-dir, an accepted first-run --link-dest
with no placeholder left behind, and an existing basis still working.
Each requires the specific mechanism rather than just "the outside file
was left alone", which any unrelated failure would satisfy, and each
takes the refusal branch where the pin primitive is unavailable.

Behaviour worth knowing about, since rrsync now creates these rather
than rsync: they are created while the ARGUMENTS are parsed, so they
appear even under --dry-run, where rsync's own make_path() deliberately
does not mkdir; and one is left behind if the transfer then fails.  A
pinned --partial-dir is also one directory rather than one per
destination directory, and rsync will not auto-remove it -- which is not
new, rrsync already rewrote an existing relative partial dir to its pin.

A nested --partial-dir now behaves differently from plain rsync, which
documents creating "just the last directory -- not the whole path".  With
--partial-dir=a/b and "a" missing, rsync creates nothing and rrsync
creates both.  Containment is unaffected -- each component is made
beneath the fd already held, O_NOFOLLOW -- but rrsync accepts a shape
rsync would not honour, and since rsync only removes the last component
of a relative partial dir, an empty parent is left behind.

Not fixed here, and NOT claimed to be: this covers the six type-2
directory options.  --files-from, --log-file and receiver positional
paths are type 3; where the leaf and its parent are both missing, or on
a platform without /proc/self/fd, those still reach rsync unpinned, as
SECURITY.md describes.  That is the rest of the issue, not this one.

The wrapper is handed a shim rather than RSYNC directly: RSYNC is a
multi-word command whenever the runner forces --protocol=N, and rrsync
execlp()s its RSYNC as a single executable name, so this test died
before reaching the policy under test.  A fleet run caught it on the
protocol columns of three targets.

(cherry picked from commit 41f411d6e9)

support/rrsync

@@ -257,6 +257,124 @@ def pin_dir(path, orig_arg):
     pinned_dirs[key] = fd
     return fd
 
+# Checked receiver-side directory options, split by what rsync does with a
+# missing one.  It creates these itself on demand, so rrsync creates and pins
+# them (0700 for the partial dir, which is what rsync uses -- partial files are
+# incomplete copies of the peer's data and rsync deliberately does not publish
+# them to the rest of the machine).
+CREATE_DIR_MODE = {'--backup-dir': 0o777, '--partial-dir': 0o700}
+# It requires this one to exist already, so a missing one stays an error.
+MUST_EXIST_DIR_OPTS = ('--temp-dir',)
+# And it only READS through these: a missing one is the ordinary first-run case
+# and must keep working, so stand in an empty directory rather than refusing.
+EMPTY_BASIS_OPTS = ('--link-dest', '--compare-dest', '--copy-dest')
+
+def pinned_empty_dir(orig_arg):
+    """Pin an empty unlinked directory to stand in for a missing basis dir.
+
+    A basis lookup cannot tell an empty directory from a missing one, so this
+    preserves "first run has no basis" exactly -- but without leaving a name
+    the peer can win: the directory is created inside the restricted dir,
+    opened, then unlinked while we keep the fd, so what rsync is handed has no
+    path at all for an in-band symlink to take over.
+    """
+    name = '.rrsync-empty-basis.%d' % os.getpid()
+    rootfd = os.open('.', os.O_RDONLY | os.O_DIRECTORY)
+    try:
+        os.mkdir(name, 0o700, dir_fd=rootfd)
+    except FileExistsError:
+        pass   # our own leftover, or something planted; the open decides
+    except OSError as e:
+        os.close(rootfd)
+        die('unable to create basis placeholder:', orig_arg, e.strerror)
+    try:
+        fd = os.open(name, os.O_RDONLY | os.O_NOFOLLOW | os.O_DIRECTORY,
+                     dir_fd=rootfd)
+    except OSError as e:
+        os.close(rootfd)
+        die('unable to pin basis placeholder:', orig_arg, e.strerror)
+    try:
+        os.rmdir(name, dir_fd=rootfd)
+    except FileNotFoundError:
+        pass   # already gone; we hold the inode either way
+    except OSError as e:
+        # The whole point is that rsync gets an inode with no name.  If the
+        # name survives, the peer can still reach and fill the directory, so
+        # this is not a placeholder we can safely hand over.
+        os.close(rootfd)
+        os.close(fd)
+        die('unable to detach basis placeholder:', orig_arg, e.strerror)
+    os.close(rootfd)
+    if os.listdir(fd):
+        os.close(fd)
+        die('basis placeholder is not empty:', orig_arg)
+    make_inheritable(fd)
+    pinned_fds.append(fd)
+    return '/proc/self/fd/%d' % fd
+
+def create_pinned_dir(path, orig_arg, mode):
+    """Create a missing receiver-option directory and return a pin of it.
+
+    rsync makes --backup-dir/--partial-dir itself and then works inside it, so
+    the parent-pinned /proc/self/fd/<parent>/<leaf> spelling is not enough: the
+    same transfer can plant a symlink at <leaf> first and rsync would create
+    through it, outside the restricted dir.  Creating it here, beneath the
+    already-pinned parent, means the name is a real directory before rsync ever
+    looks at it, and the O_NOFOLLOW reopen proves we hold what we made rather
+    than something that raced in between.
+    """
+    # Walk down from the restricted dir a component at a time, creating what
+    # is missing.  rsync builds a whole missing --backup-dir hierarchy itself
+    # (backup.c make_bak_dir()), so stopping at "the immediate parent must
+    # exist" would refuse a first-use dated/nested name that works today.
+    #
+    # O_NOFOLLOW on every component costs nothing and rules out a symlink
+    # anywhere along the way: `path` is a realpath, so none of its components
+    # is legitimately a symlink, and each open is relative to the fd we are
+    # already holding rather than to a name that could be reshaped underneath.
+    fd = os.open('.', os.O_RDONLY | os.O_DIRECTORY)   # the chdir'd restricted dir
+    for comp in os.path.relpath(path, args.dir).split(os.sep):
+        if comp in ('', '.', '..'):
+            os.close(fd)
+            die('bad receiver option path:', orig_arg)
+        try:
+            nfd = os.open(comp, os.O_RDONLY | os.O_NOFOLLOW | os.O_DIRECTORY,
+                          dir_fd=fd)
+        except FileNotFoundError:
+            try:
+                os.mkdir(comp, mode, dir_fd=fd)
+            except FileExistsError:
+                pass   # raced in; the open below decides if it is usable
+            except OSError as e:
+                os.close(fd)
+                die('unable to create receiver option dir:',
+                    orig_arg, e.strerror)
+            try:
+                nfd = os.open(comp,
+                              os.O_RDONLY | os.O_NOFOLLOW | os.O_DIRECTORY,
+                              dir_fd=fd)
+            except OSError as e:
+                os.close(fd)
+                die('receiver option path is not a usable directory:',
+                    orig_arg, e.strerror)
+        except OSError as e:
+            os.close(fd)
+            die('receiver option path is not a usable directory:',
+                orig_arg, e.strerror)
+        os.close(fd)
+        fd = nfd
+    try:
+        dpath = os.readlink('/proc/self/fd/%d' % fd)
+    except OSError as e:
+        os.close(fd)
+        die('post-pin readlink failed (race?):', orig_arg, e.strerror)
+    if dpath != args.dir and not dpath.startswith(args.dir_slash):
+        os.close(fd)
+        die('post-pin path escaped tree (race?):', orig_arg, dpath)
+    make_inheritable(fd)
+    pinned_fds.append(fd)
+    return '/proc/self/fd/%d' % fd
+
 # sender_pinned_arg() verdicts that are not a rewritten argument.
 KEEP_LEAF_PIN = 'keep'          # hand rsync the leaf's own /proc/self/fd/N
 LEAF_PIN_UNUSABLE = 'unusable'  # no pin for this shape; keep the plain name
@@ -657,6 +775,8 @@ def validated_arg(opt, arg, typ=3, wild=False):
                 if am_sender:
                     die('post-realpath open failed (race detected):',
                         orig_arg, 'No such file or directory')
+                if opt in MUST_EXIST_DIR_OPTS:
+                    die('receiver option path does not exist:', orig_arg)
                 # Receiver-side new destination: the leaf has no inode to pin
                 # yet, but pin its existing PARENT directory and route the
                 # exec'd rsync's creation through /proc/self/fd/<parent>/<leaf>,
@@ -666,7 +786,24 @@ def validated_arg(opt, arg, typ=3, wild=False):
                 # (a deeper -R new path), fall through unpinned as before.
                 fd = None
                 leaf = os.path.basename(real_arg)
-                if HAVE_PROC_SELF_FD and leaf and leaf not in ('.', '..'):
+                if opt in CREATE_DIR_MODE or opt in EMPTY_BASIS_OPTS:
+                    # An auxiliary directory the peer can supply mid-transfer.
+                    # The parent-pinned spelling below is not enough for these:
+                    # the same transfer can plant a symlink at <leaf> first,
+                    # and rsync would create or read through it, outside the
+                    # tree.  Both answers below hand rsync an inode instead of
+                    # a name, so without that primitive there is no safe way to
+                    # proceed -- refuse rather than pass the name through.
+                    if not HAVE_PROC_SELF_FD:
+                        die('receiver option path does not exist:', orig_arg)
+                    if opt in CREATE_DIR_MODE:
+                        if not leaf or leaf in ('.', '..'):
+                            die('bad receiver option path:', orig_arg)
+                        arg = create_pinned_dir(real_arg, orig_arg,
+                                                CREATE_DIR_MODE[opt])
+                    else:
+                        arg = pinned_empty_dir(orig_arg)
+                elif HAVE_PROC_SELF_FD and leaf and leaf not in ('.', '..'):
                     try:
                         pfd = os.open(os.path.dirname(real_arg) or '/',
                                       os.O_RDONLY | os.O_NOFOLLOW | os.O_DIRECTORY)