Preparing for release of 2.6.5

addition to FLAG_TOP_DIR), so that the unduplication of a real
dir and an implied dir (implied by --relative) works right.

COPYING

@@ -2,7 +2,7 @@
 		       Version 2, June 1991
 
  Copyright (C) 1989, 1991 Free Software Foundation, Inc.
-                          675 Mass Ave, Cambridge, MA 02139, USA
+                       59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
  Everyone is permitted to copy and distribute verbatim copies
  of this license document, but changing it is not allowed.
 
@@ -279,7 +279,7 @@ POSSIBILITY OF SUCH DAMAGES.
 
 		     END OF TERMS AND CONDITIONS
 
-	Appendix: How to Apply These Terms to Your New Programs
+	    How to Apply These Terms to Your New Programs
 
   If you develop a new program, and you want it to be of the greatest
 possible use to the public, the best way to achieve this is to make it
@@ -291,7 +291,7 @@ convey the exclusion of warranty; and each file should have at least
 the "copyright" line and a pointer to where the full notice is found.
 
     <one line to give the program's name and a brief idea of what it does.>
-    Copyright (C) 19yy  <name of author>
+    Copyright (C) <year>  <name of author>
 
     This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
@@ -305,14 +305,15 @@ the "copyright" line and a pointer to where the full notice is found.
 
     You should have received a copy of the GNU General Public License
     along with this program; if not, write to the Free Software
-    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
 
 Also add information on how to contact you by electronic and paper mail.
 
 If the program is interactive, make it output a short notice like this
 when it starts in an interactive mode:
 
-    Gnomovision version 69, Copyright (C) 19yy name of author
+    Gnomovision version 69, Copyright (C) year name of author
     Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
     This is free software, and you are welcome to redistribute it
     under certain conditions; type `show c' for details.

NEWS

@@ -1,342 +1,181 @@
-NEWS for rsync 2.6.4 (UNRELEASED)
-Protocol: 29 (changed)
-Changes since 2.6.3:
+NEWS for rsync 2.6.5 (1 Jun 2005)
+Protocol: 29 (unchanged)
+Changes since 2.6.4:
 
   OUTPUT CHANGES:
 
-    - When rsync deletes a directory and outputs a verbose message about
-      it, it now appends a trailing slash to the name instead of (only
-      sometimes) outputting a preceding "directory " string.
+    - Non-printable chars in filenames are now output using backslash-
+      escaped characters rather than '?'s.  Any non-printable character is
+      output using 3 digits of octal (e.g. "\n" -> "\012"), and a backslash
+      is now output as "\\".  Rsync also uses your locale setting, which
+      can make it treat fewer high-bit characters as non-printable.
 
-    - The --stats output will contain file-list time-statistics if both
-      sides are 2.6.4, or if the local side is 2.6.4 and the files are
-      being pushed (since the stats come from the sending side).
-      (Requires protocol 29 for a pull.)
+    - If rsync received an empty file-list when pulling files, it would
+      output a "nothing to do" message and exit with a 0 (success) exit
+      status, even if the remote rsync returned an error (it did not do
+      this under the same conditions when pushing files).  This was changed
+      to make the pulling behavior the same as the pushing behavior:  we
+      now do the normal end-of-run outputting (depending on options) and
+      exit with the appropriate exit status.
 
-    - The "%o" (operation) log-format escape now has a third value (besides
-      "send" and "recv"):  "del." (with trailing dot to make it 4 chars).
-      This changes the way deletions are logged in the daemon's log file.
-
-    - When the --log-format option is combined with --verbose, rsync now
-      avoids outputting the name of the file twice in most circumstances.
-      As long as the --log-format item does not refer to any post-transfer
-      items (such as %b or %c), the --log-format message is output prior to
-      the transfer with --verbose being the equivalent of a --log-format of
-      '%n%L' (which outputs the name and any symlink info).  If the log
-      output must occur after the transfer to be complete, the only time
-      the name is also output prior to the transfer is when --progress was
-      specified (so that the name will precede the progress stats, and the
-      full --log-format output will come after).
-  
   BUG FIXES:
 
-    - Restore the list-clearing behavior of "!" in a .cvsignore file (2.6.3
-      was only treating it as a special token in an rsync include/exclude
-      file).
+    - A crash bug was fixed when a daemon had its "path" set to "/", did
+      not have chroot enabled, and used some anchored excludes in the
+      rsyncd.conf file.
 
-    - The combination of --verbose and --dry-run now mentions the full list
-      of changes that would be output without --dry-run.
+    - Fixed a bug in the transfer of a single file when -H is specified
+      (rsync would either infinite loop or perhaps crash).
 
-    - Avoid a mkdir warning when removing a directory in the destination
-      that already exists in the --backup-dir.
+    - Fixed a case where the generator might try (and fail) to tweak the
+      write-permissions of a read-only directory in list-only mode (this
+      only caused an annoying warning message).
 
-    - An OS that has a binary mode for its files (such as cygwin) needed
-      setmode(fd, O_BINARY) called on the temp-file we opened with
-      mkstemp().  (Fix derived from the cygwin's 2.6.3 rsync package.)
+    - If --compare-dest or --link-dest uses a locally-copied file as the
+      basis for an updated version, log this better when --verbose or -i
+      is in effect.
 
-    - Fixed a potential hang when verbosity is high, the client side is
-      the sender, and the file-list is large.
+    - Fixed the accidental disabling of --backup during the --delete-after
+      processing.
 
-    - We now check if the OS doesn't support using mknod() for creating
-      FIFOs and sockets, and compile-in using mkfifo() and socket() when
-      necessary.
+    - Restored the ability to use the --address option in client mode (in
+      addition to its use in daemon mode).
 
-    - Fixed an off-by-one error in the handling of --max-delete=N.
+    - Make sure that some temporary progress information from the delete
+      processing does not get left on the screen when it is followed by a
+      newline.
 
-    - One place in the code wasn't checking if fork() failed.
+    - When --existing skips a directory with extra verbosity, refer to it
+      as a "directory", not a "file".
 
-    - The "ignore nonreadable" daemon parameter used to erroneously affect
-      symlinks that pointed to a non-existent file.  This has been fixed.
+    - When transferring a single file to a different-named file, any
+      generator messages that are source-file related no longer refer to
+      the file by the destination filename.
 
-    - If the OS does not have lchown() and a chown() of a symlink will
-      affect the referent of a symlink (as it should), we no longer try
-      to set the user and group of a symlink.
+    - Fixed a bug where hard-linking a group of files might fail if the
+      generator hasn't created a needed destination directory yet.
 
-    - The generator now properly runs the hard-link loop and the dir-time
-      rewriting loop after we're sure that the redo phase is complete.
+    - Fixed a bug where a hard-linked group of files that is newly-linked
+      to a file in a --link-dest dir doesn't link the files from the rest
+      of the cluster.
 
-    - When --backup was specified with --partial-dir=DIR (where DIR is a
-      relative path), the backup code was erroneously trying to backup a
-      file that was put into the partial-dir.
+    - When deleting files with the --one-file-system (-x) option set, rsync
+      no longer tries to remove files from inside a mount-point on the
+      receiving side.  Also, we don't complain about being unable to remove
+      the mount-point dir.
 
-    - If a file gets resent in a single transfer and the --backup option is
-      enabled along with --inplace, rsync no longer performs a duplicate
-      backup (it used to overwrite the first backup with the failed file).
+    - Fixed a compatibility problem when using --cvs-ignore (-C) and
+      sending files to an older rsync without using --delete.
 
-    - One call to flush_write_file() was not being checked for an error.
+    - Make sure that a "- !" or "+ !" include/exclude pattern does not
+      trigger the list-clearing action that is reserved for "!".
 
-    - The --no-relative option was not being sent from the client to a
-      server sender.
+    - Avoid a timeout in the generator when the sender/receiver aren't
+      handling the generator's checksum output quickly enough.
 
-    - If an rsync daemon specified "dont compress = ..." for a file and the
-      client tried to specify --compress, the libz code was not handling a
-      compression level of 0 properly.  This could cause a transfer failure
-      if the block-size for a file was large enough (e.g. rsync might have
-      exited with an error for large files).
+    - Fixed the omission of some directories in the delete processing when
+      --relative (-R) was combined with a source path that had a trailing
+      slash.
 
-    - Fixed a bug that would sometimes surface when using --compress and
-      sending a file with a block-size larger than 64K (either manually
-      specified, or computed due to the file being really large).  Prior
-      versions of rsync would sometimes fail to decompress the data
-      properly, and thus the transferred file would fail its verification.
+    - Fixed a case where rsync would erroneously delete some files and then
+      re-transfer them when the options --relative (-R) and --recursive
+      (-r) were both enabled (along with --delete) and a source path had a
+      trailing slash.
 
-    - If a daemon can't open the specified log file (i.e. syslog is not
-      being used), die without crashing.  We also output an error about
-      the failure on stderr (which will only be seen if --no-detach was
-      specified).
+    - Make sure that --max-size doesn't affect a device or a symlink.
 
-    - A local transfer no longer duplicates all its include/exclude options
-      (since the forked process already has a copy of the exclude list,
-      there's no need to send them a set of duplicates).
+    - Make sure that a system with a really small MAXPATHLEN does not cause
+      the buffers in readfd_unbuffered() to be too small to receive normal
+      messages.  (This mainly affected Cygwin.)
 
-    - When --progress is specified, the output of items that the generator
-      is creating (e.g. dirs, symlinks) is now integrated into the progress
-      output without overlapping it.  (Requires protocol 29.)
+    - If a source pathname ends with a filename of "..", treat it as if
+      "../" had been specified (so that we don't copy files to the parent
+      dir of the destination).
 
-    - When --timeout is specified, lulls that occur in the transfer while
-      the generator is doing work that does not generate socket traffic
-      (looking for changed files, deleting files, doing directory-time
-      touch-ups, etc.) will cause a new keep-alive packet to be sent that
-      should keep the transfer going as long as the generator continues to
-      make progress.  (Requires protocol 29.)
+    - If --delete is combined with a file-listing rsync command (i.e. no
+      transfer is happening), avoid outputting a warning that we couldn't
+      delete anything.
 
-    - The stat size of a device is not added to the total file size of the
-      items in the transfer since the size might be undefined on some OSes.
+    - If --stats is specified with --delete-after, ensure that all the
+      "deleting" messages are output before the statistics.
 
-    - Fixed a problem with refused-option messages sometimes not making it
-      back to the client side when a remote --files-from was in effect and
-      the daemon was the receiver.
+    - Improved one "if" in the deletion code that was only checking errno
+      for ENOTEMPTY when it should have also been checking for EEXIST (for
+      compatibility with OS variations).
 
   ENHANCEMENTS:
 
-    - Rsync now supports popt's option aliases, which means that you can
-      use /etc/popt and/or ~/.popt to create your own option aliases.
+    - Added the --only-write-batch=FILE option that may be used (instead
+      of --write-batch=FILE) to create a batch file without doing any
+      actual updating of the destination.  This allows you to divert all
+      the file-updating data away from a slow data link (as long as you
+      are pushing the data to the remote server when creating the batch).
 
-    - Added the --delete-during (--del) option which will delete files
-      from the receiving side incrementally as each directory in the
-      transfer is being processed.  This makes it more efficient than the
-      default, before-the-transfer behavior, which is now available as
-      --delete-before (and is still the default --delete-WHEN option that
-      will be chosen if --delete or --delete-excluded is specified without
-      a --delete-WHEN choice).  All the --del* options infer --delete, so
-      an rsync daemon that refuses "delete" will still refuse to allow any
-      file-deleting options.
+    - When the generator is taking a long time to fill up its output buffer
+      (e.g. if the transferred files are few, small, or missing), it now
+      periodically flushes the output buffer so that the sender/receiver
+      can get started on the files sooner rather than later.
 
-    - All the --delete-WHEN options are now more memory efficient:
-      Previously an entire duplicate set of file-list objects was created
-      on the receiving side for the entire destination hierarchy.  The new
-      algorithm only creates one directory of objects at a time (for files
-      inside the transfer).
+    - Improved the keep-alive code to handle a long silence between the
+      sender and the receiver that can occur when the sender is receiving
+      the checksum data for a large file.
 
-    - Added support for specifying multiple --compare-dest or --link-dest
-      options, but only of a single type. (Promoted from the patches dir
-      and enhanced.)
+    - Improved the auth-errors that are logged by the daemon to include
+      some information on why the authorization failed:  wrong user,
+      password mismatch, etc.  (The client-visible message is unchanged!)
 
-    - Added the --max-size option. (Promoted from the patches dir.)
+    - Improved the client's handling of an "@ERROR" from a daemon so that
+      it does not complain about an unexpectedly closed socket (since we
+      really did expect the socket to close).
 
-    - The daemon-mode options were separated from the normal rsync options
-      so that they can't be mixed together.  This makes it impossible to
-      start a daemon that had improper default option values that could
-      cause problems when a client connects (e.g. a hang or an abort).
+    - If the daemon can't open the log-file specified in rsyncd.conf, fall
+      back to using syslog and log an appropriate warning.  This is better
+      than what was typically a totally silent (and fatal) failure (since a
+      daemon is not usually run with the --no-detach option that was
+      necessary to see the error on stderr).
 
-    - The --bwlimit option may now be used in combination with --daemon
-      to specify both a default value for the daemon side and a value
-      that cannot be exceeded by a user-specified --bwlimit option.
+    - The man pages now consistently refer to an rsync daemon as a "daemon"
+      instead of a "server" (to distinguish it from the server process in a
+      non-daemon transfer).
 
-    - Added the "port" parameter to the rsyncd.conf file. (Promoted from
-      the patches dir.)  Also added "address".  A command-line option
-      will take precedence over a config-file option, as expected.
-
-    - In _exit_cleanup(): when we are exiting with a partially-received
-      file, we now flush any data in the write-cache before closing the
-      partial file.
-
-    - The --inplace support was enhanced to work with --compare-dest and
-      --link-dest. (Requires protocol 29.)
-
-    - Added the --dirs (-d) option for an easier way to copy directories
-      without recursion.
-
-    - Added the --list-only option, which is mainly a way for the client to
-      put the server into listing mode without needing to resort to any
-      internal option kluges (e.g. the age-old use of "-r --exclude="/*/*"
-      for a non-recursive listing).  This option is used automatically
-      (behind the scenes) when a modern rsync speaks to a modern daemon,
-      but may also be specified manually if you want to force the use of
-      the --list-only option over a remote-shell connection.
-
-    - Added the --omit-dir-times (-O) option, which will avoid updating
-      the modified time for directories when --times was specified.  This
-      option will avoid an extra pass through the file-list at the end of
-      the transfer (to tweak all the directory times), which can result in
-      an appreciable speedup for a really large transfer. (Promoted from
-      the patches dir.)
-
-    - Added the --filter (-f) option and its helper option, -F.  Filter
-      rules are an extension to the existing include/exclude handling
-      that also supports nested filter files as well as per-directory
-      filter files (like .cvsignore, but with full filter-rule parsing).
-      This new option was chosen in order to ensure that all existing
-      include/exclude processing remained 100% compatible with older
-      versions.  Protocol 29 is needed for full filter-rule support, but
-      backward-compatible rules work with earlier protocol versions.
-      (Promoted from the patches dir and enhanced.)
-
-    - Added the --delay-updates option that puts all updated files into
-      a temporary directory (by default ".~tmp~", but settable via the
-      --partial-dir=DIR option) until the end of the transfer.  This
-      makes the updates a little more atomic for a large transfer.
-
-    - If rsync is put into the background, any output from --progress is
-      reduced.
-
-    - Documented the "max verbosity" setting for rsyncd.conf.  (This
-      setting was added a couple releases ago, but left undocumented.)
-
-    - The sender and the generator now double-check the file-list index
-      they are given, and refuse to try to do a file transfer on a
-      non-file index (since that would indicate that something had gone
-      very wrong).
-
-    - Added the --itemize-changes (-i) option, which is a way to output a
-      more detailed list of what files changed in any way and how they
-      changed.  The effect is the same as specifying a --log-format of
-      "%i %n%L" (see the rsyncd.conf manpage).  Works with --dry-run too.
-
-    - Added the --fuzzy option, which attempts to find a basis file for a
-      file that is being created from scratch.  The current algorithm
-      only looks in the destination directory for the created file, but
-      it does attempt to find a match based on size/mod-time (in case the
-      file was renamed with no other changes) as well as based on a fuzzy
-      name-matching algorithm.  This option requires protocol 29 because
-      it needs the new file-sorting order.  (Promoted from patches dir
-      and enhanced.)
-
-    - Added the --remove-sent-files option, which lets you move files
-      between systems.
-
-    - The hostname in HOST:PATH or HOST::PATH may now be an IPv6 literal
-      enclosed in '[' and ']' (e.g. "[::1]").  (We already allowed IPv6
-      literals in the rsync://HOST:PORT/PATH format.)
-
-    - When building under windows, the default for --daemon is now to
-      avoid detaching, requiring the new --detach option to force rsync
-      to detach.
-
-    - Improved the option descriptions in the --help text.
-
-  SUPPORT FILES:
-
-    - Added atomic-rsync to the support dir: a perl script that will
-      transfer some files using rsync, and then move the updated files into
-      place all at once at the end of the transfer.  Only works when
-      pulling, and uses --link-dest and a parallel hierarchy of files to
-      effect its update.
-
-    - Added mnt-excl to the support dir: a perl script that takes the
-      /proc/mounts file and translates it into a set of excludes that will
-      exclude all mount points (even mapped mounts to the same disk).  The
-      excludes are made relative to the specified source dir and properly
-      anchored.
-
-    - Added savetransfer.c to the support dir: a C program that can make
-      a copy of all the data that flows over the wire.  This lets you test
-      for data corruption (by saving the data on both the sending side and
-      the receiving side) or provides a way to help debug a protocol error.
-
-    - Added rrsync to the support dir: this is my version of Joe Smith's
-      restricted rsync perl script.  This helps to ensure that only certain
-      rsync commands can be run by an ssh invocation.
+    - Made a small change to the rrsync script (restricted rsync -- in the
+      support dir) to make a read-only server reject all --remove-* options
+      when sending files (to future-proof it against the possibility of
+      other similar options being added at some point).
 
   INTERNAL:
 
-    - Added better checking of the checksum-header values that come over
-      the socket.
+    - Rsync now calls setlocale(LC_CTYPE, "").  This enables isprint() to
+      better discern which filename characters need to be escaped in
+      messages (which should result in fewer escaped characters in some
+      locales).
 
-    - Merged a variety of file-deleting functions into a single function so
-      that it is easier to maintain.
+    - Improved the naming of the log-file open/reopen/close functions.
 
-    - Improved the type of some variables (particularly blocksize vars) for
-      consistency and proper size.
-
-    - Got rid of the uint64 type (which we didn't need).
-
-    - Use a slightly more compatible set of core #include directives.
-
-    - Defined int32 in a way that ensures that the build dies if we can't
-      find a variable with at least 32 bits.
-
-    - The daemon's "read only" config item now sets an internal read_only
-      variable that makes extra sure that no write/delete calls on the
-      read-only side can succeed.
-
-  PROTOCOL DIFFERENCES FOR VERSION 29:
-
-    - A 16-bit flag-word is transmitted after every file-list index.  This
-      indicates what is changing between the sender and the receiver.  The
-      generator now transmits an index and a flag-word to indicate when
-      dirs and symlinks have changed (instead of producing a message),
-      which makes the outputting of the information more consistent and
-      less prone to screen corruption (because either the receiver or the
-      sender is now outputting all the file-change info).
-
-    - If --inplace is specified, the generator flags any transfer that is
-      using an alternate basis file so that the sender can use the entire
-      basis file in the rsync algorithm (unlike a normal --inplace update).
-
-    - The sending of exclude names is done using filter-rule syntax.  This
-      means that all names have a prefixed rule indicator, even excludes
-      (which used to be sent as a bare pattern, when possible).  The -C
-      option will include the per-dir .cvsignore merge file in the list of
-      filter rules so it is positioned correctly (unlike in some older
-      transfer scenarios).
-
-    - Rsync sorts the filename list in a different way: it sorts the subdir
-      names after the non-subdir names for each dir's contents, and it
-      always puts a dir's contents immediately after the dir's name in the
-      list.  (Previously an item named "foo.txt" would sort in between
-      directory "foo/" and "foo/bar".)
-
-    - When talking to a protocol 29 rsync daemon, a list-only request
-      is able to note this before the options are sent over the wire and
-      the new --list-only option is included in the options.
-
-    - When the --stats bytes are sent over the wire (or stored in a batch),
-      they now include two elapsed-time values: one for how long it took to
-      build the file-list, and one for how long it took to send it over the
-      wire (each expressed in thousandths of a second).
-
-    - When --delete-excluded is specified with some filter excludes, a
-      client sender will now initiate a send of the filter rules to the
-      receiver (older protocols used to omit the sending of excludes in
-      this situation since there were no receiver-specific rules that
-      survived --delete-excluded back then).  Note that, as with all the
-      filter-list sending, only items that are significant to the other
-      side will actually be sent over the wire, so the filter-rule list
-      is often empty in this scenario.
-
-    - A protocol-29 batch file includes a bit for the setting of the --dirs
-      option.  Also, the shell script created by --write-batch will use the
-      --filter option instead of --exclude-from to capture any filter rules.
-
-    - An index equal to the file-list count is sent as a keep-alive packet
-      from the generator to the sender, which then forwards it on to the
-      receiver.  This normally invalid index is only a valid keep-alive
-      packet if the 16-bit flag-word that follows it contains a single bit
-      (ITEM_IS_NEW, which is normally an illegal flag to appear alone).
+    - Removed some protocol-compatibility code that was only needed to help
+      someone running a pre-release of 2.6.4.
 
   BUILD CHANGES:
 
-    - Handle an operating system that use mkdev() in place of makedev().
+    - Added configure option "--disable-locale" to disable any use of
+      setlocale() in the binary.
 
-    - Improved configure to better handle cross-compiling.
+    - Fixed a bug in the SUPPORT{,_HARD}_LINKS #defines which prevented
+      rsync from being built without symlink or hard-link support.
+
+    - Only #define HAVE_REMSH if it is going to be set to 1.
+
+    - Configure now disables the use of mkstemp() under HP-UX (since they
+      refuse to fix its broken handling of large files).
+
+    - Configure now explicitly checks for the lseek64() function so that
+      the code can use HAVE_LSEEK64 instead of inferring lseek64()'s
+      presence based on the presence of the off64_t type.
+
+    - Configure no longer mentions the change in the default remote-shell
+      (from rsh to ssh) that occurred for the 2.6.0 release.
+
+    - Some minor enhancements to the test scripts.
+
+    - Added a few new *.diff files to the patches dir, including a patch
+      that enables the optional copying of extended attributes.

OLDNEWS

@@ -1,3 +1,394 @@
+NEWS for rsync 2.6.4 (30 March 2005)
+Protocol: 29 (changed)
+Changes since 2.6.3:
+
+  OUTPUT CHANGES:
+
+    - When rsync deletes a directory and outputs a verbose message about
+      it, it now appends a trailing slash to the name instead of (only
+      sometimes) outputting a preceding "directory " string.
+
+    - The --stats output will contain file-list time-statistics if both
+      sides are 2.6.4, or if the local side is 2.6.4 and the files are
+      being pushed (since the stats come from the sending side).
+      (Requires protocol 29 for a pull.)
+
+    - The "%o" (operation) log-format escape now has a third value (besides
+      "send" and "recv"):  "del." (with trailing dot to make it 4 chars).
+      This changes the way deletions are logged in the daemon's log file.
+
+    - When the --log-format option is combined with --verbose, rsync now
+      avoids outputting the name of the file twice in most circumstances.
+      As long as the --log-format item does not refer to any post-transfer
+      items (such as %b or %c), the --log-format message is output prior to
+      the transfer, so --verbose is now the equivalent of a --log-format of
+      '%n%L' (which outputs the name and any link info).  If the log output
+      must occur after the transfer to be complete, the only time the name
+      is also output prior to the transfer is when --progress was specified
+      (so that the name will precede the progress stats, and the full
+      --log-format output will come after).
+
+    - Non-printable characters in filenames are replaced with a '?' to
+      avoid corrupting the screen or generating empty lines in the output.
+  
+  BUG FIXES:
+
+    - Restore the list-clearing behavior of "!" in a .cvsignore file (2.6.3
+      was only treating it as a special token in an rsync include/exclude
+      file).
+
+    - The combination of --verbose and --dry-run now mentions the full list
+      of changes that would be output without --dry-run.
+
+    - Avoid a mkdir warning when removing a directory in the destination
+      that already exists in the --backup-dir.
+
+    - An OS that has a binary mode for its files (such as cygwin) needed
+      setmode(fd, O_BINARY) called on the temp-file we opened with
+      mkstemp().  (Fix derived from the cygwin's 2.6.3 rsync package.)
+
+    - Fixed a potential hang when verbosity is high, the client side is
+      the sender, and the file-list is large.
+
+    - Fixed a potential protocol-corrupting bug where the generator could
+      merge a message from the receiver into the middle of a multiplexed
+      packet of data if only part of that data had been written out to the
+      socket when the message from the generator arrived.
+
+    - We now check if the OS doesn't support using mknod() for creating
+      FIFOs and sockets, and compile-in some compatibility code using
+      mkfifo() and socket() when necessary.
+
+    - Fixed an off-by-one error in the handling of --max-delete=N.  Also,
+      if the --max-delete limit is exceeded during a run, we now output a
+      warning about this at the end of the run and exit with a new error
+      code (25).
+
+    - One place in the code wasn't checking if fork() failed.
+
+    - The "ignore nonreadable" daemon parameter used to erroneously affect
+      readable symlinks that pointed to a non-existent file.
+
+    - If the OS does not have lchown() and a chown() of a symlink will
+      affect the referent of a symlink (as it should), we no longer try
+      to set the user and group of a symlink.
+
+    - The generator now properly runs the hard-link loop and the dir-time
+      rewriting loop after we're sure that the redo phase is complete.
+
+    - When --backup was specified with --partial-dir=DIR, where DIR is a
+      relative path, the backup code was erroneously trying to backup a
+      file that was put into the partial-dir.
+
+    - If a file gets resent in a single transfer and the --backup option is
+      enabled along with --inplace, rsync no longer performs a duplicate
+      backup (it used to overwrite the first backup with the failed file).
+
+    - One call to flush_write_file() was not being checked for an error.
+
+    - The --no-relative option was not being sent from the client to a
+      server sender.
+
+    - If an rsync daemon specified "dont compress = ..." for a file and the
+      client tried to specify --compress, the libz code was not handling a
+      compression level of 0 properly.  This could cause a transfer failure
+      if the block-size for a file was large enough (e.g. rsync might have
+      exited with an error for large files).
+
+    - Fixed a bug that would sometimes surface when using --compress and
+      sending a file with a block-size larger than 64K (either manually
+      specified, or computed due to the file being really large).  Prior
+      versions of rsync would sometimes fail to decompress the data
+      properly, and thus the transferred file would fail its verification.
+
+    - If a daemon can't open the specified log file (i.e. syslog is not
+      being used), die without crashing.  We also output an error about
+      the failure on stderr (which will only be seen if --no-detach was
+      specified) and exit with a new error code (6).
+
+    - A local transfer no longer duplicates all its include/exclude options
+      (since the forked process already has a copy of the exclude list,
+      there's no need to send them a set of duplicates).
+
+    - When --progress is specified, the output of items that the generator
+      is creating (e.g. dirs, symlinks) is now integrated into the progress
+      output without overlapping it.  (Requires protocol 29.)
+
+    - When --timeout is specified, lulls that occur in the transfer while
+      the generator is doing work that does not generate socket traffic
+      (looking for changed files, deleting files, doing directory-time
+      touch-ups, etc.) will cause a new keep-alive packet to be sent that
+      should keep the transfer going as long as the generator continues to
+      make progress.  (Requires protocol 29.)
+
+    - The stat size of a device is not added to the total file size of the
+      items in the transfer (the size might be undefined on some OSes).
+
+    - Fixed a problem with refused-option messages sometimes not making it
+      back to the client side when a remote --files-from was in effect and
+      the daemon was the receiver.
+
+    - The --compare-dest option was not updating a file that differed in
+      (the preserved) attributes from the version in the compare-dest DIR.
+
+    - When rsync is copying files into a write-protected directory, fixed
+      the change-report output for the directory so that we don't report
+      an identical directory as changed.
+
+  ENHANCEMENTS:
+
+    - Rsync now supports popt's option aliases, which means that you can
+      use /etc/popt and/or ~/.popt to create your own option aliases.
+
+    - Added the --delete-during (--del) option which will delete files
+      from the receiving side incrementally as each directory in the
+      transfer is being processed.  This makes it more efficient than the
+      default, before-the-transfer behavior, which is now also available as
+      --delete-before (and is still the default --delete-WHEN option that
+      will be chosen if --delete or --delete-excluded is specified without
+      a --delete-WHEN choice).  All the --del* options infer --delete, so
+      an rsync daemon that refuses "delete" will still refuse to allow any
+      file-deleting options (including the new --remove-sent-files option).
+
+    - All the --delete-WHEN options are now more memory efficient:
+      Previously an duplicate set of file-list objects was created on the
+      receiving side for the entire destination hierarchy.  The new
+      algorithm only creates one directory of objects at a time (for files
+      inside the transfer).
+
+    - Added the --copy-dest option, which works like --link-dest except
+      that it locally copies identical files instead of hard-linking them.
+
+    - Added support for specifying multiple --compare-dest, --copy-dest, or
+      --link-dest options, but only of a single type. (Promoted from the
+      patches dir and enhanced.) (Requires protocol 29.)
+
+    - Added the --max-size option. (Promoted from the patches dir.)
+
+    - The daemon-mode options are now separated from the normal rsync
+      options so that they can't be mixed together.  This makes it
+      impossible to start a daemon that has improper default option values
+      (which could cause problems when a client connects, such as hanging
+      or crashing).
+
+    - The --bwlimit option may now be used in combination with --daemon
+      to specify both a default value for the daemon side and a value
+      that cannot be exceeded by a user-specified --bwlimit option.
+
+    - Added the "port" parameter to the rsyncd.conf file. (Promoted from
+      the patches dir.)  Also added "address".  The command-line options
+      take precedence over a config-file option, as expected.
+
+    - In _exit_cleanup(): when we are exiting with a partially-received
+      file, we now flush any data in the write-cache before closing the
+      partial file.
+
+    - The --inplace support was enhanced to work with --compare-dest,
+      --link-dest, and (the new) --copy-dest options. (Requires protocol
+      29.)
+
+    - Added the --dirs (-d) option for an easier way to copy directories
+      without recursion.  Any directories that are encountered are created
+      on the destination.  Specifying a directory with a trailing slash
+      copies its immediate contents to the destination.
+
+    - The --files-from option now implies --dirs (-d).
+
+    - Added the --list-only option, which is mainly a way for the client to
+      put the server into listing mode without needing to resort to any
+      internal option kluges (e.g. the age-old use of "-r --exclude="/*/*"
+      for a non-recursive listing).  This option is used automatically
+      (behind the scenes) when a modern rsync speaks to a modern daemon,
+      but may also be specified manually if you want to force the use of
+      the --list-only option over a remote-shell connection.
+
+    - Added the --omit-dir-times (-O) option, which will avoid updating
+      the modified time for directories when --times was specified.  This
+      option will avoid an extra pass through the file-list at the end of
+      the transfer (to tweak all the directory times), which may provide
+      an appreciable speedup for a really large transfer. (Promoted from
+      the patches dir.)
+
+    - Added the --filter (-f) option and its helper option, -F.  Filter
+      rules are an extension to the existing include/exclude handling
+      that also supports nested filter files as well as per-directory
+      filter files (like .cvsignore, but with full filter-rule parsing).
+      This new option was chosen in order to ensure that all existing
+      include/exclude processing remained 100% compatible with older
+      versions.  Protocol 29 is needed for full filter-rule support, but
+      backward-compatible rules work with earlier protocol versions.
+      (Promoted from the patches dir and enhanced.)
+
+    - Added the --delay-updates option that puts all updated files into
+      a temporary directory (by default ".~tmp~", but settable via the
+      --partial-dir=DIR option) until the end of the transfer.  This
+      makes the updates a little more atomic for a large transfer.
+
+    - If rsync is put into the background, any output from --progress is
+      reduced.
+
+    - Documented the "max verbosity" setting for rsyncd.conf.  (This
+      setting was added a couple releases ago, but left undocumented.)
+
+    - The sender and the generator now double-check the file-list index
+      they are given, and refuse to try to do a file transfer on a
+      non-file index (since that would indicate that something had gone
+      very wrong).
+
+    - Added the --itemize-changes (-i) option, which is a way to output a
+      more detailed list of what files changed and in what way.  The effect
+      is the same as specifying a --log-format of "%i %n%L" (see both the
+      rsync and rsyncd.conf manpages).  Works with --dry-run too.
+
+    - Added the --fuzzy (-y) option, which attempts to find a basis file
+      for a file that is being created from scratch.  The current algorithm
+      only looks in the destination directory for the created file, but it
+      does attempt to find a match based on size/mod-time (in case the file
+      was renamed with no other changes) as well as based on a fuzzy
+      name-matching algorithm.  This option requires protocol 29 because it
+      needs the new file-sorting order.  (Promoted from patches dir and
+      enhanced.) (Requires protocol 29.)
+
+    - Added the --remove-sent-files option, which lets you move files
+      between systems.
+
+    - The hostname in HOST:PATH or HOST::PATH may now be an IPv6 literal
+      enclosed in '[' and ']' (e.g. "[::1]").  (We already allowed IPv6
+      literals in the rsync://HOST:PORT/PATH format.)
+
+    - When rsync recurses to build the file list, it no longer keeps open
+      one or more directory handles from the dir's parent dirs.
+
+    - When building under windows, the default for --daemon is now to
+      avoid detaching, requiring the new --detach option to force rsync
+      to detach.
+
+    - The --dry-run option can now be combined with either --write-batch or
+      --read-batch, allowing you to run a do-nothing test command to see
+      what would happen without --dry-run.
+
+    - The daemon's "read only" config item now sets an internal read_only
+      variable that makes extra sure that no write/delete calls on the
+      read-only side can succeed.
+
+    - The log-format % escapes can now have a numeric field width in
+      between the % and the escape letter (e.g. "%-40n %08p").
+
+    - Improved the option descriptions in the --help text.
+
+  SUPPORT FILES:
+
+    - Added atomic-rsync to the support dir: a perl script that will
+      transfer some files using rsync, and then move the updated files into
+      place all at once at the end of the transfer.  Only works when
+      pulling, and uses --link-dest and a parallel hierarchy of files to
+      effect its update.
+
+    - Added mnt-excl to the support dir: a perl script that takes the
+      /proc/mounts file and translates it into a set of excludes that will
+      exclude all mount points (even mapped mounts to the same disk).  The
+      excludes are made relative to the specified source dir and properly
+      anchored.
+
+    - Added savetransfer.c to the support dir: a C program that can make
+      a copy of all the data that flows over the wire.  This lets you test
+      for data corruption (by saving the data on both the sending side and
+      the receiving side) and provides one way to debug a protocol error.
+
+    - Added rrsync to the support dir: this is an updated version of Joe
+      Smith's restricted rsync perl script.  This helps to ensure that only
+      certain rsync commands can be run by an ssh invocation.
+
+  INTERNAL:
+
+    - Added better checking of the checksum-header values that come over
+      the socket.
+
+    - Merged a variety of file-deleting functions into a single function so
+      that it is easier to maintain.
+
+    - Improved the type of some variables (particularly blocksize vars) for
+      consistency and proper size.
+
+    - Got rid of the uint64 type (which we didn't need).
+
+    - Use a slightly more compatible set of core #include directives.
+
+    - Defined int32 in a way that ensures that the build dies if we can't
+      find a variable with at least 32 bits.
+
+  PROTOCOL DIFFERENCES FOR VERSION 29:
+
+    - A 16-bit flag-word is transmitted after every file-list index.  This
+      indicates what is changing between the sender and the receiver.  The
+      generator now transmits an index and a flag-word to indicate when
+      dirs and symlinks have changed (instead of producing a message),
+      which makes the outputting of the information more consistent and
+      less prone to screen corruption (because the local receiver/sender is
+      now outputting all the file-change info messages).
+
+    - If a file is being hard-linked, the ITEM_XNAME_FOLLOWS bit is enabled
+      in the flag-word and the name of the file that was linked immediately
+      follows in vstring format (see below).
+
+    - If a file is being transferred with an alternate-basis file, the
+      ITEM_BASIS_TYPE_FOLLOWS bit is enabled in the flag-word and a single
+      byte follows, indicating what type of basis file was chosen.  If that
+      indicates that a fuzzy-match was selected, the ITEM_XNAME_FOLLOWS bit
+      is set in the flag-word and the name of the match in vstring format
+      follows the basis byte.  A vstring is a variable length string that
+      has its size written prior to the string, and no terminating null.
+      If the string is from 1-127 bytes, the length is a single byte.  If
+      it is from 128-32767 bytes, the length is written as ((len >> 8) |
+      0x80) followed by (len % 0x100).
+
+    - The sending of exclude names is done using filter-rule syntax.  This
+      means that all names have a prefixed rule indicator, even excludes
+      (which used to be sent as a bare pattern, when possible).  The -C
+      option will include the per-dir .cvsignore merge file in the list of
+      filter rules so it is positioned correctly (unlike in some older
+      transfer scenarios).
+
+    - Rsync sorts the filename list in a different way: it sorts the subdir
+      names after the non-subdir names for each dir's contents, and it
+      always puts a dir's contents immediately after the dir's name in the
+      list.  (Previously an item named "foo.txt" would sort in between
+      directory "foo/" and "foo/bar".)
+
+    - When talking to a protocol 29 rsync daemon, a list-only request
+      is able to note this before the options are sent over the wire and
+      the new --list-only option is included in the options.
+
+    - When the --stats bytes are sent over the wire (or stored in a batch),
+      they now include two elapsed-time values: one for how long it took to
+      build the file-list, and one for how long it took to send it over the
+      wire (each expressed in thousandths of a second).
+
+    - When --delete-excluded is specified with some filter rules (AKA
+      excludes), a client sender will now initiate a send of the rules to
+      the receiver (older protocols used to omit the sending of excludes in
+      this situation since there were no receiver-specific rules that
+      survived --delete-excluded back then).  Note that, as with all the
+      filter-list sending, only items that are significant to the other
+      side will actually be sent over the wire, so the filter-rule list
+      that is sent in this scenario is often empty.
+
+    - An index equal to the file-list count is sent as a keep-alive packet
+      from the generator to the sender, which then forwards it on to the
+      receiver.  This normally invalid index is only a valid keep-alive
+      packet if the 16-bit flag-word that follows it contains a single bit
+      (ITEM_IS_NEW, which is normally an illegal flag to appear alone).
+
+    - A protocol-29 batch file includes a bit for the setting of the --dirs
+      option and for the setting of the --compress option.  Also, the shell
+      script created by --write-batch will use the --filter option instead
+      of --exclude-from to capture any filter rules.
+
+  BUILD CHANGES:
+
+    - Handle an operating system that use mkdev() in place of makedev().
+
+    - Improved configure to better handle cross-compiling.
 NEWS for rsync 2.6.3 (30 Sep 2004)
 Protocol: 28 (unchanged)
 Changes since 2.6.2:
@@ -1042,7 +1433,9 @@ Changes since 2.4.6:
       build farm.
 
 Partial Protocol History
-	RELEASE DATE	VER.	DATE OF COMMIT	PROTOCOL
+	RELEASE DATE	VER.	DATE OF COMMIT*	PROTOCOL
+	01 Jun 2005	2.6.5			29
+	30 Mar 2005	2.6.4	17 Jan 2005	29
 	30 Sep 2004	2.6.3			28
 	30 Apr 2004	2.6.2			28
 	26 Apr 2004	2.6.1	08 Jan 2004	28

README

@@ -89,6 +89,9 @@ mailing list archives at
 To send a bug report, follow the instructions on the bug-tracking
 page of the web site.
 
+If you don't have web access, email your bug report to
+rsync@lists.samba.org.
+
 
 CVS TREE
 --------

authenticate.c

@@ -51,10 +51,11 @@ void base64_encode(char *buf, int len, char *out)
 	}
 }
 
-/* create a 16 byte challenge buffer */
+/* Generate a challenge buffer and return it base64-encoded. */
 static void gen_challenge(char *addr, char *challenge)
 {
 	char input[32];
+	char md4_out[MD4_SUM_LENGTH];
 	struct timeval tv;
 
 	memset(input, 0, sizeof input);
@@ -67,7 +68,9 @@ static void gen_challenge(char *addr, char *challenge)
 
 	sum_init(0);
 	sum_update(input, sizeof input);
-	sum_end(challenge);
+	sum_end(md4_out);
+
+	base64_encode(md4_out, MD4_SUM_LENGTH, challenge);
 }
 
 
@@ -195,17 +198,18 @@ static char *getpassf(char *filename)
 	return NULL;
 }
 
-/* generate a 16 byte hash from a password and challenge */
+/* Generate an MD4 hash created from the combination of the password
+ * and the challenge string and return it base64-encoded. */
 static void generate_hash(char *in, char *challenge, char *out)
 {
-	char buf[16];
+	char buf[MD4_SUM_LENGTH];
 
 	sum_init(0);
 	sum_update(in, strlen(in));
 	sum_update(challenge, strlen(challenge));
 	sum_end(buf);
 
-	base64_encode(buf, 16, out);
+	base64_encode(buf, MD4_SUM_LENGTH, out);
 }
 
 /* Possibly negotiate authentication with the client.  Use "leader" to
@@ -214,17 +218,15 @@ static void generate_hash(char *in, char *challenge, char *out)
  * Return NULL if authentication failed.  Return "" if anonymous access.
  * Otherwise return username.
  */
-char *auth_server(int f_in, int f_out, int module, char *addr, char *leader)
+char *auth_server(int f_in, int f_out, int module, char *host, char *addr,
+		  char *leader)
 {
 	char *users = lp_auth_users(module);
-	char challenge[16];
-	char b64_challenge[30];
+	char challenge[MD4_SUM_LENGTH*2];
 	char line[MAXPATHLEN];
-	static char user[100];
-	char secret[100];
-	char pass[30];
-	char pass2[30];
-	char *tok;
+	char secret[512];
+	char pass2[MD4_SUM_LENGTH*2];
+	char *tok, *pass;
 
 	/* if no auth list then allow anyone in! */
 	if (!users || !*users)
@@ -232,52 +234,60 @@ char *auth_server(int f_in, int f_out, int module, char *addr, char *leader)
 
 	gen_challenge(addr, challenge);
 
-	base64_encode(challenge, 16, b64_challenge);
+	io_printf(f_out, "%s%s\n", leader, challenge);
 
-	io_printf(f_out, "%s%s\n", leader, b64_challenge);
-
-	if (!read_line(f_in, line, sizeof line - 1))
+	if (!read_line(f_in, line, sizeof line - 1)
+	 || (pass = strchr(line, ' ')) == NULL) {
+		rprintf(FLOG, "auth failed on module %s from %s (%s): "
+			"invalid challenge response\n",
+			lp_name(module), host, addr);
 		return NULL;
+	}
+	*pass++ = '\0';
 
-	memset(user, 0, sizeof user);
-	memset(pass, 0, sizeof pass);
+	if (!(users = strdup(users)))
+		out_of_memory("auth_server");
 
-	if (sscanf(line,"%99s %29s", user, pass) != 2)
-		return NULL;
-
-	users = strdup(users);
-	if (!users)
-		return NULL;
-
-	for (tok=strtok(users," ,\t"); tok; tok = strtok(NULL," ,\t")) {
-		if (wildmatch(tok, user))
+	for (tok = strtok(users, " ,\t"); tok; tok = strtok(NULL, " ,\t")) {
+		if (wildmatch(tok, line))
 			break;
 	}
 	free(users);
 
-	if (!tok)
-		return NULL;
-
-	memset(secret, 0, sizeof secret);
-	if (!get_secret(module, user, secret, sizeof secret - 1)) {
-		memset(secret, 0, sizeof secret);
+	if (!tok) {
+		rprintf(FLOG, "auth failed on module %s from %s (%s): "
+			"unauthorized user\n",
+			lp_name(module), host, addr);
 		return NULL;
 	}
 
-	generate_hash(secret, b64_challenge, pass2);
+	memset(secret, 0, sizeof secret);
+	if (!get_secret(module, line, secret, sizeof secret - 1)) {
+		memset(secret, 0, sizeof secret);
+		rprintf(FLOG, "auth failed on module %s from %s (%s): "
+			"missing secret for user \"%s\"\n",
+			lp_name(module), host, addr, line);
+		return NULL;
+	}
+
+	generate_hash(secret, challenge, pass2);
 	memset(secret, 0, sizeof secret);
 
-	if (strcmp(pass, pass2) == 0)
-		return user;
+	if (strcmp(pass, pass2) != 0) {
+		rprintf(FLOG, "auth failed on module %s from %s (%s): "
+			"password mismatch\n",
+			lp_name(module), host, addr);
+		return NULL;
+	}
 
-	return NULL;
+	return strdup(line);
 }
 
 
 void auth_client(int fd, char *user, char *challenge)
 {
 	char *pass;
-	char pass2[30];
+	char pass2[MD4_SUM_LENGTH*2];
 
 	if (!user || !*user)
 		user = "nobody";
@@ -302,5 +312,3 @@ void auth_client(int fd, char *user, char *challenge)
 	generate_hash(pass, challenge, pass2);
 	io_printf(fd, "%s %s\n", user, pass2);
 }
-
-

batch.c

@@ -18,21 +18,22 @@ extern int preserve_devices;
 extern int preserve_uid;
 extern int preserve_gid;
 extern int always_checksum;
+extern int do_compression;
 extern int protocol_version;
 extern char *batch_name;
 
 extern struct filter_list_struct filter_list;
 
-static int fudged_recurse;
-
 static int *flag_ptr[] = {
-	&fudged_recurse,
-	&preserve_uid,
-	&preserve_gid,
-	&preserve_links,
-	&preserve_devices,
-	&preserve_hard_links,
-	&always_checksum,
+	&recurse,		/* 0 */
+	&preserve_uid,		/* 1 */
+	&preserve_gid,		/* 2 */
+	&preserve_links,	/* 3 */
+	&preserve_devices,	/* 4 */
+	&preserve_hard_links,	/* 5 */
+	&always_checksum,	/* 6 */
+	&xfer_dirs,		/* 7 (protocol 29) */
+	&do_compression,	/* 8 (protocol 29) */
 	NULL
 };
 
@@ -45,6 +46,7 @@ static char *flag_name[] = {
 	"--hard-links (-H)",
 	"--checksum (-c)",
 	"--dirs (-d)",
+	"--compress (-z)",
 	NULL
 };
 
@@ -54,7 +56,8 @@ void write_stream_flags(int fd)
 
 	/* Start the batch file with a bitmap of data-stream-affecting
 	 * flags. */
-	fudged_recurse = recurse < 0;
+	if (protocol_version < 29)
+		flag_ptr[7] = NULL;
 	for (i = 0, flags = 0; flag_ptr[i]; i++) {
 		if (*flag_ptr[i])
 			flags |= 1 << i;
@@ -66,9 +69,8 @@ void read_stream_flags(int fd)
 {
 	int i, flags;
 
-	fudged_recurse = recurse < 0;
 	if (protocol_version < 29)
-		xfer_dirs = 0;
+		flag_ptr[7] = NULL;
 	for (i = 0, flags = read_int(fd); flag_ptr[i]; i++) {
 		int set = flags & (1 << i) ? 1 : 0;
 		if (*flag_ptr[i] != set) {
@@ -80,9 +82,12 @@ void read_stream_flags(int fd)
 			*flag_ptr[i] = set;
 		}
 	}
-	recurse = fudged_recurse ? -1 : 0;
-	if (protocol_version < 29)
-		xfer_dirs = recurse ? 1 : 0;
+	if (protocol_version < 29) {
+		if (recurse)
+			xfer_dirs |= 1;
+		else if (xfer_dirs < 2)
+			xfer_dirs = 0;
+	}
 }
 
 static void write_arg(int fd, char *arg)
@@ -134,7 +139,7 @@ static void write_filter_rules(int fd)
  * (hopefully) work. */
 void write_batch_shell_file(int argc, char *argv[], int file_arg_cnt)
 {
-	int fd, i;
+	int fd, i, len;
 	char *p, filename[MAXPATHLEN];
 
 	stringjoin(filename, sizeof filename,
@@ -170,11 +175,12 @@ void write_batch_shell_file(int argc, char *argv[], int file_arg_cnt)
 			continue;
 		}
 		write(fd, " ", 1);
-		if (strncmp(p, "--write-batch", 13) == 0) {
+		if (strncmp(p, "--write-batch", len = 13) == 0
+		 || strncmp(p, "--only-write-batch", len = 18) == 0) {
 			write(fd, "--read-batch", 12);
-			if (p[13] == '=') {
+			if (p[len] == '=') {
 				write(fd, "=", 1);
-				write_arg(fd, p + 14);
+				write_arg(fd, p + len + 1);
 			}
 		} else
 			write_arg(fd, p);

cleanup.c

@@ -136,10 +136,12 @@ void _exit_cleanup(int code, const char *file, int line)
 	}
 
 	if (code == 0) {
-		if ((io_error & ~IOERR_VANISHED) || log_got_error)
-			code = RERR_PARTIAL;
-		else if (io_error)
+		if (io_error & IOERR_DEL_LIMIT)
+			code = RERR_DEL_LIMIT;
+		if (io_error & IOERR_VANISHED)
 			code = RERR_VANISHED;
+		if (io_error & IOERR_GENERAL || log_got_error)
+			code = RERR_PARTIAL;
 	}
 
 	if (code)

clientserver.c

@@ -41,7 +41,6 @@ extern int filesfrom_fd;
 extern int remote_protocol;
 extern int protocol_version;
 extern int io_timeout;
-extern int select_timeout;
 extern int orig_umask;
 extern int no_detach;
 extern int default_af_hint;
@@ -100,7 +99,7 @@ int start_socket_client(char *host, char *path, int argc, char *argv[])
 
 	ret = start_inband_exchange(user, path, fd, fd, argc);
 
-	return ret < 0? ret : client_run(fd, fd, -1, argc, argv);
+	return ret ? ret : client_run(fd, fd, -1, argc, argv);
 }
 
 int start_inband_exchange(char *user, char *path, int f_in, int f_out, 
@@ -196,10 +195,10 @@ int start_inband_exchange(char *user, char *path, int f_in, int f_out,
 			rprintf(FERROR, "%s\n", line);
 			/* This is always fatal; the server will now
 			 * close the socket. */
-			return RERR_STARTCLIENT;
-		} else {
-			rprintf(FINFO,"%s\n", line);
+			return -1;
 		}
+
+		rprintf(FINFO, "%s\n", line);
 	}
 	kluge_around_eof = 0;
 
@@ -268,11 +267,9 @@ static int rsync_module(int f_in, int f_out, int i)
 		return -1;
 	}
 
-	auth_user = auth_server(f_in, f_out, i, addr, "@RSYNCD: AUTHREQD ");
+	auth_user = auth_server(f_in, f_out, i, host, addr, "@RSYNCD: AUTHREQD ");
 
 	if (!auth_user) {
-		rprintf(FLOG, "auth failed on module %s from %s (%s)\n",
-			name, host, addr);
 		io_printf(f_out, "@ERROR: auth failed on module %s\n", name);
 		return -1;
 	}
@@ -283,10 +280,10 @@ static int rsync_module(int f_in, int f_out, int i)
 		read_only = 1;
 
 	if (lp_transfer_logging(i)) {
-		if (strstr(lp_log_format(i), "%i") != NULL)
+		if (log_format_has(lp_log_format(i), 'i'))
 			daemon_log_format_has_i = 1;
 		if (daemon_log_format_has_i
-		    || strstr(lp_log_format(i), "%o") != NULL)
+		    || log_format_has(lp_log_format(i), 'o'))
 			daemon_log_format_has_o_or_i = 1;
 	}
 
@@ -321,13 +318,11 @@ static int rsync_module(int f_in, int f_out, int i)
 	/* TODO: Perhaps take a list of gids, and make them into the
 	 * supplementary groups. */
 
-	if (use_chroot) {
+	if (use_chroot || (module_dirlen = strlen(lp_path(i))) == 1) {
 		module_dirlen = 0;
 		set_filter_dir("/", 1);
-	} else {
-		module_dirlen = strlen(lp_path(i));
+	} else
 		set_filter_dir(lp_path(i), module_dirlen);
-	}
 
 	p = lp_filter(i);
 	parse_rule(&server_filter_list, p, MATCHFLG_WORD_SPLIT,
@@ -522,11 +517,8 @@ static int rsync_module(int f_in, int f_out, int i)
 		exit_cleanup(RERR_UNSUPPORTED);
 	}
 
-	if (lp_timeout(i)) {
-		io_timeout = lp_timeout(i);
-		if (io_timeout < select_timeout)
-			select_timeout = io_timeout;
-	}
+	if (lp_timeout(i) && lp_timeout(i) > io_timeout)
+		set_io_timeout(lp_timeout(i));
 
 	start_server(f_in, f_out, argc, argp);
 

compat.c

@@ -30,10 +30,13 @@ int remote_protocol = 0;
 extern int verbose;
 extern int am_server;
 extern int am_sender;
+extern int inplace;
 extern int fuzzy_basis;
 extern int read_batch;
 extern int checksum_seed;
+extern int basis_dir_cnt;
 extern int protocol_version;
+extern char *dest_option;
 
 void setup_protocol(int f_out,int f_in)
 {
@@ -77,11 +80,25 @@ void setup_protocol(int f_out,int f_in)
 
 	if (fuzzy_basis && protocol_version < 29) {
 		rprintf(FERROR,
-			"--fuzzy requres protocol 29 or higher (negotiated %d).\n",
+			"--fuzzy requires protocol 29 or higher (negotiated %d).\n",
 			protocol_version);
 		exit_cleanup(RERR_PROTOCOL);
 	}
 
+	if (basis_dir_cnt && inplace && protocol_version < 29) {
+		rprintf(FERROR,
+			"%s with --inplace requires protocol 29 or higher (negotiated %d).\n",
+			dest_option, protocol_version);
+		exit_cleanup(RERR_PROTOCOL);
+	}
+
+	if (basis_dir_cnt > 1 && protocol_version < 29) {
+		rprintf(FERROR,
+			"Multiple %s options requires protocol 29 or higher (negotiated %d).\n",
+			dest_option, protocol_version);
+		exit_cleanup(RERR_PROTOCOL);
+	}
+
 	if (am_server) {
 		if (!checksum_seed)
 			checksum_seed = time(NULL);

configure.in

@@ -5,7 +5,7 @@ AC_CONFIG_SRCDIR([byteorder.h])
 AC_CONFIG_HEADER(config.h)
 AC_PREREQ(2.59)
 
-RSYNC_VERSION=2.6.4pre2
+RSYNC_VERSION=2.6.5
 AC_SUBST(RSYNC_VERSION)
 AC_MSG_NOTICE([Configuring rsync $RSYNC_VERSION])
 
@@ -122,7 +122,9 @@ AC_ARG_WITH(rsh,
 	AC_HELP_STRING([--with-rsh=CMD], [set remote shell command to CMD (default: ssh)]))
 
 AC_CHECK_PROG(HAVE_REMSH, remsh, 1, 0)
-AC_DEFINE_UNQUOTED(HAVE_REMSH, $HAVE_REMSH, [remote shell is remsh not rsh])
+if test x$HAVE_REMSH = x1; then
+	AC_DEFINE(HAVE_REMSH, 1, [Define to 1 if remote shell is remsh, not rsh])
+fi
 
 if test x"$with_rsh" != x
 then
@@ -181,6 +183,17 @@ ipv6trylibc=yes
 AC_ARG_ENABLE(ipv6,
 	AC_HELP_STRING([--disable-ipv6], [don't even try to use IPv6]))
 
+dnl Do you want to disable use of locale functions
+AH_TEMPLATE([CONFIG_LOCALE],
+[Undefine if you don't want locale features.  By default this is defined.])
+AC_ARG_ENABLE([locale],
+	AC_HELP_STRING([--disable-locale], [turn off locale features]),
+[if test x$enableval = xyes; then
+  AC_DEFINE(CONFIG_LOCALE)
+fi],
+AC_DEFINE(CONFIG_LOCALE)
+)
+
 if test "x$enable_ipv6" != xno
 then
 	AC_MSG_CHECKING([ipv6 stack type])
@@ -287,7 +300,7 @@ AC_HEADER_SYS_WAIT
 AC_CHECK_HEADERS(sys/fcntl.h sys/select.h fcntl.h sys/time.h sys/unistd.h \
     unistd.h utime.h grp.h compat.h sys/param.h ctype.h sys/wait.h \
     sys/ioctl.h sys/filio.h string.h stdlib.h sys/socket.h sys/mode.h \
-    sys/un.h glob.h mcheck.h arpa/inet.h arpa/nameser.h \
+    sys/un.h glob.h mcheck.h arpa/inet.h arpa/nameser.h locale.h \
     netdb.h malloc.h float.h)
 AC_HEADER_MAJOR
 
@@ -478,7 +491,7 @@ AC_CHECK_FUNCS(waitpid wait4 getcwd strdup strerror chown chmod mknod mkfifo \
     fchmod fstat strchr readlink link utime utimes strftime mtrace ftruncate \
     memmove lchown vsnprintf snprintf vasprintf asprintf setsid glob strpbrk \
     strlcat strlcpy strtol mallinfo getgroups setgroups geteuid getegid \
-    setmode open64 mkstemp64 va_copy __va_copy)
+    setlocale setmode open64 lseek64 mkstemp64 va_copy __va_copy)
 
 AC_CHECK_FUNCS(getpgrp tcgetpgrp)
 if test $ac_cv_func_getpgrp = yes; then
@@ -621,7 +634,17 @@ rsync_cv_HAVE_SECURE_MKSTEMP=yes,
 rsync_cv_HAVE_SECURE_MKSTEMP=no,
 rsync_cv_HAVE_SECURE_MKSTEMP=cross)])
 if test x"$rsync_cv_HAVE_SECURE_MKSTEMP" = x"yes"; then
-    AC_DEFINE(HAVE_SECURE_MKSTEMP, 1, [Define to 1 if mkstemp() is available and works right])
+    case $target_os in
+    hpux*)
+	dnl HP-UX has a broken mkstemp() implementation they refuse to fix,
+	dnl so we noisily skip using it.  See HP change request JAGaf34426
+	dnl for details. (sbonds)
+	AC_MSG_WARN(Skipping broken HP-UX mkstemp() -- using mktemp() instead)
+	;;
+    *)
+	AC_DEFINE(HAVE_SECURE_MKSTEMP, 1, [Define to 1 if mkstemp() is available and works right])
+	;;
+    esac
 fi
 
 
@@ -716,21 +739,6 @@ AC_SUBST(BUILD_POPT)
 AC_CONFIG_FILES([Makefile lib/dummy zlib/dummy popt/dummy shconfig])
 AC_OUTPUT
 
-if test x"$with_rsh" = x; then
-	if test x"$HAVE_REMSH" = x1; then
-		rmsh1='remsh:'
-		rmsh2='=remsh'
-	else
-		rmsh1='rsh:  '
-		rmsh2='=rsh  '
-	fi
-	AC_MSG_RESULT()
-	AC_MSG_RESULT([    **********************************************************************])
-	AC_MSG_RESULT([    *   As of v2.6.0, the default remote shell is ssh instead of rsh!!   *])
-	AC_MSG_RESULT([    *   To use previous default of $rmsh1 ./configure --with-rsh$rmsh2   *])
-	AC_MSG_RESULT([    **********************************************************************])
-fi
-
 AC_MSG_RESULT()
 AC_MSG_RESULT([    rsync ${RSYNC_VERSION} configuration successful])
 AC_MSG_RESULT()

errcode.h

@@ -40,6 +40,7 @@
 #define RERR_MALLOC     22      /* error allocating core memory buffers */
 #define RERR_PARTIAL    23      /* partial transfer */
 #define RERR_VANISHED   24      /* file(s) vanished on sender side */
+#define RERR_DEL_LIMIT  25      /* skipped some deletes due to --max-delete */
 
 #define RERR_TIMEOUT    30      /* timeout in data send/receive */
 

exclude.c

@@ -674,8 +674,7 @@ static const char *parse_rule_tok(const char *p, uint32 mflags, int xflags,
 		} else if (*s == '+' && s[1] == ' ') {
 			new_mflags |= MATCHFLG_INCLUDE;
 			s += 2;
-		}
-		if (*s == '!')
+		} else if (*s == '!')
 			new_mflags |= MATCHFLG_CLEAR_LIST; /* Tentative! */
 	} else {
 		char ch = 0, *mods = "";
@@ -716,7 +715,6 @@ static const char *parse_rule_tok(const char *p, uint32 mflags, int xflags,
 			if ((s = RULE_STRCMP(s, "show")) != NULL)
 				ch = 'S';
 			break;
-
 		default:
 			ch = *s;
 			if (s[1] == ',')
@@ -1107,9 +1105,9 @@ static void send_rules(int f_out, struct filter_list_struct *flp)
 			continue;
 		if (ent->match_flags & MATCHFLG_CVS_IGNORE
 		    && !(ent->match_flags & MATCHFLG_MERGE_FILE)) {
-			int f = am_sender || protocol_version < 29 ? f_out : -1;
+			int f = am_sender || protocol_version < 29 ? f_out : -2;
 			send_rules(f, &cvs_filter_list);
-			if (f >= 0)
+			if (f == f_out)
 				continue;
 		}
 		p = get_rule_prefix(ent->match_flags, ent->pattern, 1, &plen);

flist.c

@@ -27,8 +27,6 @@
 
 #include "rsync.h"
 
-extern struct stats stats;
-
 extern int verbose;
 extern int dry_run;
 extern int list_only;
@@ -59,6 +57,8 @@ extern int copy_unsafe_links;
 extern int protocol_version;
 extern int sanitize_paths;
 extern int orig_umask;
+extern struct stats stats;
+extern struct file_list *the_file_list;
 
 extern char curr_dir[MAXPATHLEN];
 
@@ -69,8 +69,9 @@ int io_error;
 dev_t filesystem_dev; /* used to implement -x */
 
 static char empty_sum[MD4_SUM_LENGTH];
+static int flist_count_offset;
 static unsigned int file_struct_len;
-static struct file_list *received_flist, *sorting_flist;
+static struct file_list *sorting_flist;
 
 static void clean_flist(struct file_list *flist, int strip_root, int no_dups);
 static void output_flist(struct file_list *flist);
@@ -98,16 +99,16 @@ static void start_filelist_progress(char *kind)
 }
 
 
-static void emit_filelist_progress(const struct file_list *flist)
+static void emit_filelist_progress(int count)
 {
-	rprintf(FINFO, " %d files...\r", flist->count);
+	rprintf(FINFO, " %d files...\r", count);
 }
 
 
-static void maybe_emit_filelist_progress(const struct file_list *flist)
+static void maybe_emit_filelist_progress(int count)
 {
-	if (do_progress && show_filelist_p() && (flist->count % 100) == 0)
-		emit_filelist_progress(flist);
+	if (do_progress && show_filelist_p() && (count % 100) == 0)
+		emit_filelist_progress(count);
 }
 
 
@@ -654,6 +655,9 @@ static struct file_struct *receive_file_entry(struct file_list *flist,
 		if (flags & XMIT_TOP_DIR) {
 			in_del_hier = 1;
 			del_hier_name_len = file->dir.depth == 0 ? 0 : l1 + l2;
+			if (relative_paths && del_hier_name_len > 2
+			    && basename_len == 1+1 && *basename == '.')
+				del_hier_name_len -= 2;
 			file->flags |= FLAG_TOP_DIR | FLAG_DEL_HERE;
 		} else if (in_del_hier) {
 			if (!relative_paths || !del_hier_name_len
@@ -943,7 +947,7 @@ skip_filters:
 		STRUCT_STAT st2;
 		int save_mode = file->mode;
 		file->mode = S_IFDIR; /* find a directory w/our name */
-		if (flist_find(received_flist, file) >= 0
+		if (flist_find(the_file_list, file) >= 0
 		    && do_stat(thisname, &st2) == 0 && S_ISDIR(st2.st_mode)) {
 			file->modtime = st2.st_mtime;
 			file->length = st2.st_size;
@@ -962,17 +966,16 @@ skip_filters:
 }
 
 
-void send_file_name(int f, struct file_list *flist, char *fname,
-		    int recursive, unsigned short base_flags)
+static struct file_struct *send_file_name(int f, struct file_list *flist,
+					  char *fname, unsigned short base_flags)
 {
 	struct file_struct *file;
-	char fbuf[MAXPATHLEN];
 
 	file = make_file(fname, flist, f == -2 ? SERVER_FILTERS : ALL_FILTERS);
 	if (!file)
-		return;
+		return NULL;
 
-	maybe_emit_filelist_progress(flist);
+	maybe_emit_filelist_progress(flist->count + flist_count_offset);
 
 	flist_expand(flist);
 
@@ -980,8 +983,15 @@ void send_file_name(int f, struct file_list *flist, char *fname,
 		flist->files[flist->count++] = file;
 		send_file_entry(file, f, base_flags);
 	}
+	return file;
+}
 
-	if (recursive && S_ISDIR(file->mode)
+static void send_if_directory(int f, struct file_list *flist,
+			      struct file_struct *file)
+{
+	char fbuf[MAXPATHLEN];
+
+	if (S_ISDIR(file->mode)
 	    && !(file->flags & FLAG_MOUNT_POINT) && f_name_to(file, fbuf)) {
 		void *save_filters;
 		unsigned int len = strlen(fbuf);
@@ -1000,13 +1010,11 @@ void send_file_name(int f, struct file_list *flist, char *fname,
 }
 
 
-/* Note that the "recurse" value either contains -1, for infinite recursion, or
- * a number >= 0 indicating how many levels of recursion we will allow.  This
- * function is normally called by the sender, but the receiving side also calls
- * it from delete_in_dir() with f set to -1 so that we just construct the file
- * list in memory without sending it over the wire.  Also, get_dirlist() might
- * call this with f set to -2, which indicates that local filter rules should
- * be ignored. */
+/* This function is normally called by the sender, but the receiving side also
+ * calls it from get_dirlist() with f set to -1 so that we just construct the
+ * file list in memory without sending it over the wire.  Also, get_dirlist()
+ * might call this with f set to -2, which also indicates that local filter
+ * rules should be ignored. */
 static void send_directory(int f, struct file_list *flist,
 			   char *fbuf, int len)
 {
@@ -1014,6 +1022,7 @@ static void send_directory(int f, struct file_list *flist,
 	unsigned remainder;
 	char *p;
 	DIR *d;
+	int start = flist->count;
 
 	if (!(d = opendir(fbuf))) {
 		io_error |= IOERR_GENERAL;
@@ -1032,10 +1041,9 @@ static void send_directory(int f, struct file_list *flist,
 		if (dname[0] == '.' && (dname[1] == '\0'
 		    || (dname[1] == '.' && dname[2] == '\0')))
 			continue;
-		if (strlcpy(p, dname, remainder) < remainder) {
-			int do_subdirs = recurse >= 1 ? recurse-- : recurse;
-			send_file_name(f, flist, fbuf, do_subdirs, 0);
-		} else {
+		if (strlcpy(p, dname, remainder) < remainder)
+			send_file_name(f, flist, fbuf, 0);
+		else {
 			io_error |= IOERR_GENERAL;
 			rprintf(FINFO,
 				"cannot send long-named file %s\n",
@@ -1051,6 +1059,12 @@ static void send_directory(int f, struct file_list *flist,
 	}
 
 	closedir(d);
+
+	if (recurse) {
+		int i, end = flist->count - 1;
+		for (i = start; i <= end; i++)
+			send_if_directory(f, flist, flist->files[i]);
+	}
 }
 
 
@@ -1084,9 +1098,10 @@ struct file_list *send_file_list(int f, int argc, char *argv[])
 	}
 
 	while (1) {
+		struct file_struct *file;
 		char fname2[MAXPATHLEN];
 		char *fname = fname2;
-		int do_subdirs;
+		int is_dot_dir;
 
 		if (use_ff_fd) {
 			if (read_filesfrom_line(filesfrom_fd, fname) == 0)
@@ -1105,16 +1120,25 @@ struct file_list *send_file_list(int f, int argc, char *argv[])
 			if (l == 2 && fname[0] == '.') {
 				/* Turn "./" into just "." rather than "./." */
 				fname[1] = '\0';
-			} else if (l < MAXPATHLEN) {
+			} else {
+				if (l + 1 >= MAXPATHLEN)
+					overflow("send_file_list");
 				fname[l++] = '.';
 				fname[l] = '\0';
 			}
+			is_dot_dir = 1;
+		} else if (l > 1 && fname[l-1] == '.' && fname[l-2] == '.'
+		    && (l == 2 || fname[l-3] == '/')) {
+			if (l + 2 >= MAXPATHLEN)
+				overflow("send_file_list");
+			fname[l++] = '/';
+			fname[l++] = '.';
+			fname[l] = '\0';
+			is_dot_dir = 1;
+		} else {
+			is_dot_dir = fname[l-1] == '.'
+				   && (l == 1 || fname[l-2] == '/');
 		}
-		if (fname[l-1] == '.' && (l == 1 || fname[l-2] == '/')) {
-			if (!recurse && xfer_dirs)
-				recurse = 1; /* allow one level */
-		} else if (recurse > 0)
-			recurse = 0;
 
 		if (link_stat(fname, &st, keep_dirlinks) != 0) {
 			io_error |= IOERR_GENERAL;
@@ -1162,7 +1186,7 @@ struct file_list *send_file_list(int f, int argc, char *argv[])
 				xfer_dirs = 1;
 				while ((slash = strchr(slash+1, '/')) != 0) {
 					*slash = 0;
-					send_file_name(f, flist, fname, 0, 0);
+					send_file_name(f, flist, fname, 0);
 					*slash = '/';
 				}
 				copy_links = save_copy_links;
@@ -1201,8 +1225,10 @@ struct file_list *send_file_list(int f, int argc, char *argv[])
 		if (one_file_system)
 			filesystem_dev = st.st_dev;
 
-		do_subdirs = recurse >= 1 ? recurse-- : recurse;
-		send_file_name(f, flist, fname, do_subdirs, XMIT_TOP_DIR);
+		if ((file = send_file_name(f, flist, fname, XMIT_TOP_DIR))) {
+			if (recurse || (xfer_dirs && is_dot_dir))
+				send_if_directory(f, flist, file);
+		}
 
 		if (olddir[0]) {
 			flist_dir = NULL;
@@ -1275,7 +1301,6 @@ struct file_list *recv_file_list(int f)
 	start_read = stats.total_read;
 
 	flist = flist_new(WITH_HLINK, "recv_file_list");
-	received_flist = flist;
 
 	flist->count = 0;
 	flist->malloced = 1000;
@@ -1298,7 +1323,7 @@ struct file_list *recv_file_list(int f)
 
 		flist->files[flist->count++] = file;
 
-		maybe_emit_filelist_progress(flist);
+		maybe_emit_filelist_progress(flist->count);
 
 		if (verbose > 2) {
 			rprintf(FINFO, "recv_file_name(%s)\n",
@@ -1499,8 +1524,8 @@ static void clean_flist(struct file_list *flist, int strip_root, int no_dups)
 			}
 			/* Make sure that if we unduplicate '.', that we don't
 			 * lose track of a user-specified top directory. */
-			if (flist->files[drop]->flags & FLAG_TOP_DIR)
-				flist->files[keep]->flags |= FLAG_TOP_DIR;
+			flist->files[keep]->flags |= flist->files[drop]->flags
+						   & (FLAG_TOP_DIR|FLAG_DEL_HERE);
 
 			clear_file(drop, flist);
 
@@ -1655,8 +1680,13 @@ int f_name_cmp(struct file_struct *f1, struct file_struct *f2)
 				break;
 			case s_SLASH:
 				type1 = S_ISDIR(f1->mode) ? t_path : t_ITEM;
-				state1 = s_BASE;
 				c1 = (uchar*)f1->basename;
+				if (type1 == t_PATH && *c1 == '.' && !c1[1]) {
+					type1 = t_ITEM;
+					state1 = s_TRAILING;
+					c1 = (uchar*)"";
+				} else
+					state1 = s_BASE;
 				break;
 			case s_BASE:
 				state1 = s_TRAILING;
@@ -1675,25 +1705,18 @@ int f_name_cmp(struct file_struct *f1, struct file_struct *f2)
 		if (!*c2) {
 			switch (state2) {
 			case s_DIR:
-				if (state1 == s_SLASH && sorting_flist) {
-					int j;
-					/* Optimize for future comparisons. */
-					for (j = 0;
-					     j < sorting_flist->count;
-					     j++) {
-						struct file_struct *fp
-						    = sorting_flist->files[j];
-						if (fp->dirname == f2->dirname)
-							fp->dirname = f1->dirname;
-					}
-				}
 				state2 = s_SLASH;
 				c2 = (uchar*)"/";
 				break;
 			case s_SLASH:
 				type2 = S_ISDIR(f2->mode) ? t_path : t_ITEM;
-				state2 = s_BASE;
 				c2 = (uchar*)f2->basename;
+				if (type2 == t_PATH && *c2 == '.' && !c2[1]) {
+					type2 = t_ITEM;
+					state2 = s_TRAILING;
+					c2 = (uchar*)"";
+				} else
+					state2 = s_BASE;
 				break;
 			case s_BASE:
 				state2 = s_TRAILING;
@@ -1773,6 +1796,8 @@ struct file_list *get_dirlist(char *dirname, int dlen,
 	recurse = 0;
 	send_directory(ignore_filter_rules ? -2 : -1, dirlist, dirname, dlen);
 	recurse = save_recurse;
+	if (do_progress)
+		flist_count_offset += dirlist->count;
 
 	clean_flist(dirlist, 0, 0);
 

hlink.c

@@ -23,12 +23,21 @@
 extern int dry_run;
 extern int verbose;
 extern int make_backups;
+extern struct file_list *the_file_list;
 
 #ifdef SUPPORT_HARD_LINKS
-static int hlink_compare(struct file_struct **file1, struct file_struct **file2)
+
+#define SKIPPED_LINK (-1)
+#define FINISHED_LINK (-2)
+
+#define FPTR(i) (the_file_list->files[i])
+#define LINKED(p1,p2) (FPTR(p1)->F_DEV == FPTR(p2)->F_DEV \
+		    && FPTR(p1)->F_INODE == FPTR(p2)->F_INODE)
+
+static int hlink_compare(int *int1, int *int2)
 {
-	struct file_struct *f1 = *file1;
-	struct file_struct *f2 = *file2;
+	struct file_struct *f1 = FPTR(*int1);
+	struct file_struct *f2 = FPTR(*int2);
 
 	if (f1->F_DEV != f2->F_DEV)
 		return (int) (f1->F_DEV > f2->F_DEV ? 1 : -1);
@@ -39,51 +48,48 @@ static int hlink_compare(struct file_struct **file1, struct file_struct **file2)
 	return f_name_cmp(f1, f2);
 }
 
-static struct file_struct **hlink_list;
+static int *hlink_list;
 static int hlink_count;
 
-#define LINKED(p1,p2) ((p1)->F_DEV == (p2)->F_DEV \
-		    && (p1)->F_INODE == (p2)->F_INODE)
-
 /* Analyze the data in the hlink_list[], remove items that aren't multiply
  * linked, and replace the dev+inode data with the hlindex+next linked list. */
-static void link_idev_data(struct file_list *flist)
+static void link_idev_data(void)
 {
-	struct file_struct *head;
-	int from, to, start;
+	int cur, from, to, start;
 
 	alloc_pool_t hlink_pool;
-	alloc_pool_t idev_pool = flist->hlink_pool;
+	alloc_pool_t idev_pool = the_file_list->hlink_pool;
 
 	hlink_pool = pool_create(128 * 1024, sizeof (struct hlink),
 	    out_of_memory, POOL_INTERN);
 
 	for (from = to = 0; from < hlink_count; from++) {
 		start = from;
-		head = hlink_list[start];
-		while (from < hlink_count-1
-		    && LINKED(hlink_list[from], hlink_list[from+1])) {
-			pool_free(idev_pool, 0, hlink_list[from]->link_u.idev);
-			hlink_list[from]->link_u.links = pool_talloc(hlink_pool,
+		while (1) {
+			cur = hlink_list[from];
+			if (from == hlink_count-1
+			    || !LINKED(cur, hlink_list[from+1]))
+				break;
+			pool_free(idev_pool, 0, FPTR(cur)->link_u.idev);
+			FPTR(cur)->link_u.links = pool_talloc(hlink_pool,
 			    struct hlink, 1, "hlink_list");
 
-			hlink_list[from]->F_HLINDEX = to;
-			hlink_list[from]->F_NEXT = hlink_list[from+1];
-			from++;
+			FPTR(cur)->F_HLINDEX = to;
+			FPTR(cur)->F_NEXT = hlink_list[++from];
 		}
+		pool_free(idev_pool, 0, FPTR(cur)->link_u.idev);
 		if (from > start) {
-			pool_free(idev_pool, 0, hlink_list[from]->link_u.idev);
-			hlink_list[from]->link_u.links = pool_talloc(hlink_pool,
+			int head = hlink_list[start];
+			FPTR(cur)->link_u.links = pool_talloc(hlink_pool,
 			    struct hlink, 1, "hlink_list");
 
-			hlink_list[from]->F_HLINDEX = to;
-			hlink_list[from]->F_NEXT = head;
-			hlink_list[from]->flags |= FLAG_HLINK_EOL;
+			FPTR(head)->flags |= FLAG_HLINK_TOL;
+			FPTR(cur)->F_HLINDEX = to;
+			FPTR(cur)->F_NEXT = head;
+			FPTR(cur)->flags |= FLAG_HLINK_EOL;
 			hlink_list[to++] = head;
-		} else {
-			pool_free(idev_pool, 0, head->link_u.idev);
-			head->link_u.idev = NULL;
-		}
+		} else
+			FPTR(cur)->link_u.links = NULL;
 	}
 
 	if (!to) {
@@ -93,33 +99,30 @@ static void link_idev_data(struct file_list *flist)
 		hlink_pool = NULL;
 	} else {
 		hlink_count = to;
-		if (!(hlink_list = realloc_array(hlink_list,
-		    struct file_struct *, hlink_count)))
+		hlink_list = realloc_array(hlink_list, int, hlink_count);
+		if (!hlink_list)
 			out_of_memory("init_hard_links");
 	}
-	flist->hlink_pool = hlink_pool;
+	the_file_list->hlink_pool = hlink_pool;
 	pool_destroy(idev_pool);
 }
 #endif
 
-void init_hard_links(struct file_list *flist)
+void init_hard_links(void)
 {
 #ifdef SUPPORT_HARD_LINKS
 	int i;
 
-	if (flist->count < 2)
-		return;
-
 	if (hlink_list)
 		free(hlink_list);
 
-	if (!(hlink_list = new_array(struct file_struct *, flist->count)))
+	if (!(hlink_list = new_array(int, the_file_list->count)))
 		out_of_memory("init_hard_links");
 
 	hlink_count = 0;
-	for (i = 0; i < flist->count; i++) {
-		if (flist->files[i]->link_u.idev)
-			hlink_list[hlink_count++] = flist->files[i];
+	for (i = 0; i < the_file_list->count; i++) {
+		if (FPTR(i)->link_u.idev)
+			hlink_list[hlink_count++] = i;
 	}
 
 	qsort(hlink_list, hlink_count,
@@ -129,22 +132,71 @@ void init_hard_links(struct file_list *flist)
 		free(hlink_list);
 		hlink_list = NULL;
 	} else
-		link_idev_data(flist);
+		link_idev_data();
 #endif
 }
 
-int hard_link_check(struct file_struct *file, int skip)
+#ifdef SUPPORT_HARD_LINKS
+static int maybe_hard_link(struct file_struct *file, int ndx,
+			   char *fname, int statret, STRUCT_STAT *st,
+			   char *toname, STRUCT_STAT *to_st,
+			   int itemizing, enum logcode code)
+{
+	if (statret == 0) {
+		if (st->st_dev == to_st->st_dev
+		 && st->st_ino == to_st->st_ino) {
+			if (itemizing) {
+				itemize(file, ndx, statret, st,
+					ITEM_LOCAL_CHANGE | ITEM_XNAME_FOLLOWS,
+					0, "");
+			}
+			return 0;
+		}
+		if (make_backups) {
+			if (!make_backup(fname))
+				return -1;
+		} else if (robust_unlink(fname)) {
+			rsyserr(FERROR, errno, "unlink %s failed",
+				full_fname(fname));
+			return -1;
+		}
+	}
+	return hard_link_one(file, ndx, fname, statret, st, toname,
+			     0, itemizing, code);
+}
+#endif
+
+int hard_link_check(struct file_struct *file, int ndx, char *fname,
+		    int statret, STRUCT_STAT *st, int itemizing,
+		    enum logcode code, int skip)
 {
 #ifdef SUPPORT_HARD_LINKS
-	if (!hlink_list || !file->link_u.links)
+	int head;
+	if (!file->link_u.links)
 		return 0;
 	if (skip && !(file->flags & FLAG_HLINK_EOL))
-		hlink_list[file->F_HLINDEX] = file->F_NEXT;
-	if (hlink_list[file->F_HLINDEX] != file) {
-		if (verbose > 1) {
+		head = hlink_list[file->F_HLINDEX] = file->F_NEXT;
+	else
+		head = hlink_list[file->F_HLINDEX];
+	if (ndx != head) {
+		struct file_struct *head_file = FPTR(head);
+		if (verbose > 2) {
 			rprintf(FINFO, "\"%s\" is a hard link\n",
 				safe_fname(f_name(file)));
 		}
+		if (head_file->F_HLINDEX == FINISHED_LINK) {
+			STRUCT_STAT st2;
+			char *toname = f_name(head_file);
+			if (link_stat(toname, &st2, 0) < 0) {
+				rsyserr(FERROR, errno, "stat %s failed",
+					full_fname(toname));
+				return -1;
+			}
+			maybe_hard_link(file, ndx, fname, statret, st,
+					toname, &st2, itemizing, code);
+			file->F_HLINDEX = FINISHED_LINK;
+		} else
+			file->F_HLINDEX = SKIPPED_LINK;
 		return 1;
 	}
 #endif
@@ -152,63 +204,64 @@ int hard_link_check(struct file_struct *file, int skip)
 }
 
 #ifdef SUPPORT_HARD_LINKS
-static void hard_link_one(char *hlink1, char *hlink2)
+int hard_link_one(struct file_struct *file, int ndx, char *fname,
+		  int statret, STRUCT_STAT *st, char *toname, int terse,
+		  int itemizing, enum logcode code)
 {
-	if (do_link(hlink1, hlink2)) {
-		if (verbose) {
-			rsyserr(FINFO, errno, "link %s => %s failed",
-				full_fname(hlink2), safe_fname(hlink1));
-		}
+	if (do_link(toname, fname)) {
+		if (terse) {
+			if (!verbose)
+				return -1;
+			code = FINFO;
+		} else
+			code = FERROR;
+		rsyserr(code, errno, "link %s => %s failed",
+			full_fname(fname), safe_fname(toname));
+		return -1;
 	}
-	else if (verbose)
-		rprintf(FINFO, "%s => %s\n", safe_fname(hlink2), safe_fname(hlink1));
+
+	if (itemizing) {
+		itemize(file, ndx, statret, st,
+			ITEM_LOCAL_CHANGE | ITEM_XNAME_FOLLOWS, 0,
+			terse ? "" : toname);
+	}
+	if (code && verbose && !terse) {
+		rprintf(code, "%s => %s\n",
+			safe_fname(fname), safe_fname(toname));
+	}
+	return 0;
 }
 #endif
 
 
-
-/**
- * Create any hard links in the global hlink_list.  They were put
- * there by running init_hard_links on the filelist.
- **/
-void do_hard_links(int allowed_lull, int flist_count)
+void hard_link_cluster(struct file_struct *file, int master, int itemizing,
+		       enum logcode code)
 {
 #ifdef SUPPORT_HARD_LINKS
-	struct file_struct *file, *first;
 	char hlink1[MAXPATHLEN];
 	char *hlink2;
 	STRUCT_STAT st1, st2;
-	int i;
+	int statret, ndx = master;
 
-	if (!hlink_list)
+	file->F_HLINDEX = FINISHED_LINK;
+	if (link_stat(f_name_to(file, hlink1), &st1, 0) < 0)
 		return;
-
-	for (i = 0; i < hlink_count; i++) {
-		first = file = hlink_list[i];
-		if (link_stat(f_name_to(first, hlink1), &st1, 0) < 0)
-			continue;
-		while ((file = file->F_NEXT) != first) {
-			hlink2 = f_name(file);
-			if (link_stat(hlink2, &st2, 0) == 0) {
-				if (st2.st_dev == st1.st_dev
-				    && st2.st_ino == st1.st_ino)
-					continue;
-				if (make_backups) {
-					if (!make_backup(hlink2))
-						continue;
-				} else if (robust_unlink(hlink2)) {
-					if (verbose > 0) {
-						rsyserr(FINFO, errno,
-							"unlink %s failed",
-							full_fname(hlink2));
-					}
-					continue;
-				}
-			}
-			hard_link_one(hlink1, hlink2);
+	if (!(file->flags & FLAG_HLINK_TOL)) {
+		while (!(file->flags & FLAG_HLINK_EOL)) {
+			ndx = file->F_NEXT;
+			file = FPTR(ndx);
 		}
-		if (allowed_lull)
-			maybe_send_keepalive(allowed_lull, flist_count);
 	}
+	do {
+		ndx = file->F_NEXT;
+		file = FPTR(ndx);
+		if (file->F_HLINDEX != SKIPPED_LINK)
+			continue;
+		hlink2 = f_name(file);
+		statret = link_stat(hlink2, &st2, 0);
+		maybe_hard_link(file, ndx, hlink2, statret, &st2,
+				hlink1, &st1, itemizing, code);
+		file->F_HLINDEX = FINISHED_LINK;
+	} while (!(file->flags & FLAG_HLINK_EOL));
 #endif
 }

io.c

@@ -43,19 +43,23 @@ extern int bwlimit;
 extern size_t bwlimit_writemax;
 extern int verbose;
 extern int io_timeout;
+extern int allowed_lull;
 extern int am_server;
 extern int am_daemon;
 extern int am_sender;
 extern int am_generator;
 extern int eol_nulls;
+extern int read_batch;
 extern int csum_length;
 extern int checksum_seed;
 extern int protocol_version;
+extern int remove_sent_files;
+extern int preserve_hard_links;
 extern char *filesfrom_host;
 extern struct stats stats;
+extern struct file_list *the_file_list;
 
 const char phase_unknown[] = "unknown";
-int select_timeout = SELECT_TIMEOUT;
 int ignore_timeout = 0;
 int batch_fd = -1;
 int batch_gen_fd = -1;
@@ -81,12 +85,13 @@ int kluge_around_eof = 0;
 
 int msg_fd_in = -1;
 int msg_fd_out = -1;
+int sock_f_in = -1;
+int sock_f_out = -1;
 
 static int io_multiplexing_out;
 static int io_multiplexing_in;
-static int sock_f_in = -1;
-static int sock_f_out = -1;
-static time_t last_io;
+static time_t last_io_in;
+static time_t last_io_out;
 static int no_flush;
 
 static int write_batch_monitor_in = -1;
@@ -98,16 +103,21 @@ static char io_filesfrom_buf[2048];
 static char *io_filesfrom_bp;
 static char io_filesfrom_lastchar;
 static int io_filesfrom_buflen;
+static size_t contiguous_write_len = 0;
+static int select_timeout = SELECT_TIMEOUT;
 
 static void read_loop(int fd, char *buf, size_t len);
 
-struct redo_list {
-	struct redo_list *next;
-	int num;
+struct flist_ndx_item {
+	struct flist_ndx_item *next;
+	int ndx;
 };
 
-static struct redo_list *redo_list_head;
-static struct redo_list *redo_list_tail;
+struct flist_ndx_list {
+	struct flist_ndx_item *head, *tail;
+};
+
+static struct flist_ndx_list redo_list, hlink_list;
 
 struct msg_list {
 	struct msg_list *next;
@@ -118,19 +128,37 @@ struct msg_list {
 static struct msg_list *msg_list_head;
 static struct msg_list *msg_list_tail;
 
-static void redo_list_add(int num)
+static void flist_ndx_push(struct flist_ndx_list *lp, int ndx)
 {
-	struct redo_list *rl;
+	struct flist_ndx_item *item;
 
-	if (!(rl = new(struct redo_list)))
-		exit_cleanup(RERR_MALLOC);
-	rl->next = NULL;
-	rl->num = num;
-	if (redo_list_tail)
-		redo_list_tail->next = rl;
+	if (!(item = new(struct flist_ndx_item)))
+		out_of_memory("flist_ndx_push");
+	item->next = NULL;
+	item->ndx = ndx;
+	if (lp->tail)
+		lp->tail->next = item;
 	else
-		redo_list_head = rl;
-	redo_list_tail = rl;
+		lp->head = item;
+	lp->tail = item;
+}
+
+static int flist_ndx_pop(struct flist_ndx_list *lp)
+{
+	struct flist_ndx_item *next;
+	int ndx;
+
+	if (!lp->head)
+		return -1;
+
+	ndx = lp->head->ndx;
+	next = lp->head->next;
+	free(lp->head);
+	lp->head = next;
+	if (!next)
+		lp->tail = NULL;
+
+	return ndx;
 }
 
 static void check_timeout(void)
@@ -140,17 +168,17 @@ static void check_timeout(void)
 	if (!io_timeout || ignore_timeout)
 		return;
 
-	if (!last_io) {
-		last_io = time(NULL);
+	if (!last_io_in) {
+		last_io_in = time(NULL);
 		return;
 	}
 
 	t = time(NULL);
 
-	if (t - last_io >= io_timeout) {
+	if (t - last_io_in >= io_timeout) {
 		if (!am_server && !am_daemon) {
 			rprintf(FERROR, "io timeout after %d seconds -- exiting\n",
-				(int)(t-last_io));
+				(int)(t-last_io_in));
 		}
 		exit_cleanup(RERR_TIMEOUT);
 	}
@@ -164,6 +192,18 @@ void io_set_sock_fds(int f_in, int f_out)
 	sock_f_out = f_out;
 }
 
+void set_io_timeout(int secs)
+{
+	io_timeout = secs;
+
+	if (!io_timeout || io_timeout > SELECT_TIMEOUT)
+		select_timeout = SELECT_TIMEOUT;
+	else
+		select_timeout = io_timeout;
+
+	allowed_lull = read_batch ? 0 : (io_timeout + 1) / 2;
+}
+
 /* Setup the fd used to receive MSG_* messages.  Only needed during the
  * early stages of being a local sender (up through the sending of the
  * file list) or when we're the generator (to fetch the messages from
@@ -187,10 +227,10 @@ static void msg_list_add(int code, char *buf, int len)
 	struct msg_list *ml;
 
 	if (!(ml = new(struct msg_list)))
-		exit_cleanup(RERR_MALLOC);
+		out_of_memory("msg_list_add");
 	ml->next = NULL;
 	if (!(ml->buf = new_array(char, len+4)))
-		exit_cleanup(RERR_MALLOC);
+		out_of_memory("msg_list_add");
 	SIVAL(ml->buf, 0, ((code+MPLEX_BASE)<<24) | len);
 	memcpy(ml->buf+4, buf, len);
 	ml->len = len+4;
@@ -223,7 +263,7 @@ static void read_msg_fd(void)
 	int tag, len;
 
 	/* Temporarily disable msg_fd_in.  This is needed to avoid looping back
-	 * to this routine from read_timeout() and writefd_unbuffered(). */
+	 * to this routine from writefd_unbuffered(). */
 	msg_fd_in = -1;
 
 	read_loop(fd, buf, 4);
@@ -238,7 +278,7 @@ static void read_msg_fd(void)
 			rprintf(FERROR, "invalid message %d:%d\n", tag, len);
 			exit_cleanup(RERR_STREAMIO);
 		}
-		redo_list_add(-1);
+		flist_ndx_push(&redo_list, -1);
 		break;
 	case MSG_REDO:
 		if (len != 4 || !am_generator) {
@@ -246,7 +286,7 @@ static void read_msg_fd(void)
 			exit_cleanup(RERR_STREAMIO);
 		}
 		read_loop(fd, buf, 4);
-		redo_list_add(IVAL(buf,0));
+		flist_ndx_push(&redo_list, IVAL(buf,0));
 		break;
 	case MSG_DELETED:
 		if (len >= (int)sizeof buf || !am_generator) {
@@ -262,7 +302,10 @@ static void read_msg_fd(void)
 			exit_cleanup(RERR_STREAMIO);
 		}
 		read_loop(fd, buf, len);
-		io_multiplex_write(MSG_SUCCESS, buf, len);
+		if (remove_sent_files)
+			io_multiplex_write(MSG_SUCCESS, buf, len);
+		if (preserve_hard_links)
+			flist_ndx_push(&hlink_list, IVAL(buf,0));
 		break;
 	case MSG_INFO:
 	case MSG_ERROR:
@@ -324,22 +367,22 @@ int msg_list_push(int flush_it_all)
 	return 1;
 }
 
-int get_redo_num(void)
+int get_redo_num(int itemizing, enum logcode code)
 {
-	struct redo_list *next;
-	int num;
-
-	while (!redo_list_head)
+	while (1) {
+		if (hlink_list.head)
+			check_for_finished_hlinks(itemizing, code);
+		if (redo_list.head)
+			break;
 		read_msg_fd();
+	}
 
-	num = redo_list_head->num;
-	next = redo_list_head->next;
-	free(redo_list_head);
-	redo_list_head = next;
-	if (!next)
-		redo_list_tail = NULL;
+	return flist_ndx_pop(&redo_list);
+}
 
-	return num;
+int get_hlink_num(void)
+{
+	return flist_ndx_pop(&hlink_list);
 }
 
 /**
@@ -419,11 +462,7 @@ static int read_timeout(int fd, char *buf, size_t len)
 		FD_ZERO(&r_fds);
 		FD_ZERO(&w_fds);
 		FD_SET(fd, &r_fds);
-		if (msg_fd_in >= 0) {
-			FD_SET(msg_fd_in, &r_fds);
-			if (msg_fd_in > maxfd)
-				maxfd = msg_fd_in;
-		} else if (msg_list_head) {
+		if (msg_list_head) {
 			FD_SET(msg_fd_out, &w_fds);
 			if (msg_fd_out > maxfd)
 				maxfd = msg_fd_out;
@@ -460,9 +499,7 @@ static int read_timeout(int fd, char *buf, size_t len)
 			continue;
 		}
 
-		if (msg_fd_in >= 0 && FD_ISSET(msg_fd_in, &r_fds))
-			read_msg_fd();
-		else if (msg_list_head && FD_ISSET(msg_fd_out, &w_fds))
+		if (msg_list_head && FD_ISSET(msg_fd_out, &w_fds))
 			msg_list_push(NORMAL_FLUSH);
 
 		if (io_filesfrom_f_out >= 0) {
@@ -551,8 +588,8 @@ static int read_timeout(int fd, char *buf, size_t len)
 		len -= n;
 		ret += n;
 
-		if (io_timeout && fd == sock_f_in)
-			last_io = time(NULL);
+		if (fd == sock_f_in && io_timeout)
+			last_io_in = time(NULL);
 	}
 
 	return ret;
@@ -642,13 +679,20 @@ void io_end_buffering(void)
 }
 
 
-void maybe_send_keepalive(int allowed_lull, int ndx)
+void maybe_flush_socket(void)
 {
-	if (time(NULL) - last_io >= allowed_lull) {
+	if (iobuf_out && iobuf_out_cnt && time(NULL) - last_io_out >= 5)
+		io_flush(NORMAL_FLUSH);
+}
+
+
+void maybe_send_keepalive(void)
+{
+	if (time(NULL) - last_io_out >= allowed_lull) {
 		if (!iobuf_out || !iobuf_out_cnt) {
 			if (protocol_version < 29)
 				return; /* there's nothing we can do */
-			write_int(sock_f_out, ndx);
+			write_int(sock_f_out, the_file_list->count);
 			write_shortint(sock_f_out, ITEM_IS_NEW);
 		}
 		if (iobuf_out)
@@ -682,8 +726,13 @@ static int readfd_unbuffered(int fd, char *buf, size_t len)
 {
 	static size_t remaining;
 	static size_t iobuf_in_ndx;
+	size_t msg_bytes;
 	int tag, ret = 0;
-	char line[MAXPATHLEN+1];
+#if MAXPATHLEN < 4096
+	char line[4096+1024];
+#else
+	char line[MAXPATHLEN+1024];
+#endif
 
 	if (!iobuf_in || fd != sock_f_in)
 		return read_timeout(fd, buf, len);
@@ -706,60 +755,56 @@ static int readfd_unbuffered(int fd, char *buf, size_t len)
 		read_loop(fd, line, 4);
 		tag = IVAL(line, 0);
 
-		remaining = tag & 0xFFFFFF;
+		msg_bytes = tag & 0xFFFFFF;
 		tag = (tag >> 24) - MPLEX_BASE;
 
 		switch (tag) {
 		case MSG_DATA:
-			if (remaining > iobuf_in_siz) {
+			if (msg_bytes > iobuf_in_siz) {
 				if (!(iobuf_in = realloc_array(iobuf_in, char,
-							       remaining)))
+							       msg_bytes)))
 					out_of_memory("readfd_unbuffered");
-				iobuf_in_siz = remaining;
+				iobuf_in_siz = msg_bytes;
 			}
-			read_loop(fd, iobuf_in, remaining);
+			read_loop(fd, iobuf_in, msg_bytes);
+			remaining = msg_bytes;
 			iobuf_in_ndx = 0;
 			break;
 		case MSG_DELETED:
-			if (remaining >= sizeof line) {
-				rprintf(FERROR, "invalid multi-message %d:%ld\n",
-					tag, (long)remaining);
-				exit_cleanup(RERR_STREAMIO);
-			}
-			read_loop(fd, line, remaining);
-			line[remaining] = '\0';
+			if (msg_bytes >= sizeof line)
+				goto overflow;
+			read_loop(fd, line, msg_bytes);
+			line[msg_bytes] = '\0';
 			/* A directory name was sent with the trailing null */
-			if (remaining > 0 && !line[remaining-1])
+			if (msg_bytes > 0 && !line[msg_bytes-1])
 				log_delete(line, S_IFDIR);
 			else
 				log_delete(line, S_IFREG);
-			remaining = 0;
 			break;
 		case MSG_SUCCESS:
-			if (remaining != 4) {
-				rprintf(FERROR, "invalid multi-message %d:%ld\n",
-					tag, (long)remaining);
+			if (msg_bytes != 4) {
+				rprintf(FERROR, "invalid multi-message %d:%ld [%s]\n",
+					tag, (long)msg_bytes, who_am_i());
 				exit_cleanup(RERR_STREAMIO);
 			}
-			read_loop(fd, line, remaining);
+			read_loop(fd, line, msg_bytes);
 			successful_send(IVAL(line, 0));
-			remaining = 0;
 			break;
 		case MSG_INFO:
 		case MSG_ERROR:
-			if (remaining >= sizeof line) {
+			if (msg_bytes >= sizeof line) {
+			    overflow:
 				rprintf(FERROR,
-					"[%s] multiplexing overflow %d:%ld\n\n",
-					who_am_i(), tag, (long)remaining);
+					"multiplexing overflow %d:%ld [%s]\n",
+					tag, (long)msg_bytes, who_am_i());
 				exit_cleanup(RERR_STREAMIO);
 			}
-			read_loop(fd, line, remaining);
-			rwrite((enum logcode)tag, line, remaining);
-			remaining = 0;
+			read_loop(fd, line, msg_bytes);
+			rwrite((enum logcode)tag, line, msg_bytes);
 			break;
 		default:
-			rprintf(FERROR, "[%s] unexpected tag %d\n",
-				who_am_i(), tag);
+			rprintf(FERROR, "unexpected tag %d [%s]\n",
+				tag, who_am_i());
 			exit_cleanup(RERR_STREAMIO);
 		}
 	}
@@ -845,7 +890,7 @@ void read_buf(int f,char *buf,size_t len)
 void read_sbuf(int f,char *buf,size_t len)
 {
 	readfd(f, buf, len);
-	buf[len] = 0;
+	buf[len] = '\0';
 }
 
 uchar read_byte(int f)
@@ -855,6 +900,25 @@ uchar read_byte(int f)
 	return c;
 }
 
+int read_vstring(int f, char *buf, int bufsize)
+{
+	int len = read_byte(f);
+
+	if (len & 0x80)
+		len = (len & ~0x80) * 0x100 + read_byte(f);
+
+	if (len >= bufsize) {
+		rprintf(FERROR, "over-long vstring received (%d > %d)\n",
+			len, bufsize - 1);
+		return -1;
+	}
+
+	if (len)
+		readfd(f, buf, len);
+	buf[len] = '\0';
+	return len;
+}
+
 /* Populate a sum_struct with values from the socket.  This is
  * called by both the sender and the receiver. */
 void read_sum_head(int f, struct sum_struct *sum)
@@ -862,20 +926,20 @@ void read_sum_head(int f, struct sum_struct *sum)
 	sum->count = read_int(f);
 	sum->blength = read_int(f);
 	if (sum->blength < 0 || sum->blength > MAX_BLOCK_SIZE) {
-		rprintf(FERROR, "[%s] Invalid block length %ld\n",
-			who_am_i(), (long)sum->blength);
+		rprintf(FERROR, "Invalid block length %ld [%s]\n",
+			(long)sum->blength, who_am_i());
 		exit_cleanup(RERR_PROTOCOL);
 	}
 	sum->s2length = protocol_version < 27 ? csum_length : (int)read_int(f);
 	if (sum->s2length < 0 || sum->s2length > MD4_SUM_LENGTH) {
-		rprintf(FERROR, "[%s] Invalid checksum length %d\n",
-			who_am_i(), sum->s2length);
+		rprintf(FERROR, "Invalid checksum length %d [%s]\n",
+			sum->s2length, who_am_i());
 		exit_cleanup(RERR_PROTOCOL);
 	}
 	sum->remainder = read_int(f);
 	if (sum->remainder < 0 || sum->remainder > sum->blength) {
-		rprintf(FERROR, "[%s] Invalid remainder length %ld\n",
-			who_am_i(), (long)sum->remainder);
+		rprintf(FERROR, "Invalid remainder length %ld [%s]\n",
+			(long)sum->remainder, who_am_i());
 		exit_cleanup(RERR_PROTOCOL);
 	}
 }
@@ -955,8 +1019,8 @@ static void sleep_for_bwlimit(int bytes_written)
 
 
 /* Write len bytes to the file descriptor fd, looping as necessary to get
- * the job done and also (in the generator) reading any data on msg_fd_in
- * (to avoid deadlock).
+ * the job done and also (in certain circumstnces) reading any data on
+ * msg_fd_in to avoid deadlock.
  *
  * This function underlies the multiplexing system.  The body of the
  * application never calls this function directly. */
@@ -964,7 +1028,7 @@ static void writefd_unbuffered(int fd,char *buf,size_t len)
 {
 	size_t n, total = 0;
 	fd_set w_fds, r_fds;
-	int maxfd, count, ret;
+	int maxfd, count, ret, using_r_fds;
 	struct timeval tv;
 
 	no_flush++;
@@ -974,23 +1038,20 @@ static void writefd_unbuffered(int fd,char *buf,size_t len)
 		FD_SET(fd,&w_fds);
 		maxfd = fd;
 
-		if (msg_fd_in >= 0) {
+		if (msg_fd_in >= 0 && len-total >= contiguous_write_len) {
 			FD_ZERO(&r_fds);
 			FD_SET(msg_fd_in,&r_fds);
 			if (msg_fd_in > maxfd)
 				maxfd = msg_fd_in;
-		}
-		if (fd != sock_f_out && iobuf_out_cnt && no_flush == 1) {
-			FD_SET(sock_f_out, &w_fds);
-			if (sock_f_out > maxfd)
-				maxfd = sock_f_out;
-		}
+			using_r_fds = 1;
+		} else
+			using_r_fds = 0;
 
 		tv.tv_sec = select_timeout;
 		tv.tv_usec = 0;
 
 		errno = 0;
-		count = select(maxfd + 1, msg_fd_in >= 0 ? &r_fds : NULL,
+		count = select(maxfd + 1, using_r_fds ? &r_fds : NULL,
 			       &w_fds, NULL, &tv);
 
 		if (count <= 0) {
@@ -1000,17 +1061,11 @@ static void writefd_unbuffered(int fd,char *buf,size_t len)
 			continue;
 		}
 
-		if (msg_fd_in >= 0 && FD_ISSET(msg_fd_in, &r_fds))
+		if (using_r_fds && FD_ISSET(msg_fd_in, &r_fds))
 			read_msg_fd();
 
-		if (!FD_ISSET(fd, &w_fds)) {
-			if (fd != sock_f_out && iobuf_out_cnt) {
-				no_flush--;
-				io_flush(NORMAL_FLUSH);
-				no_flush++;
-			}
+		if (!FD_ISSET(fd, &w_fds))
 			continue;
-		}
 
 		n = len - total;
 		if (bwlimit && n > bwlimit_writemax)
@@ -1036,7 +1091,7 @@ static void writefd_unbuffered(int fd,char *buf,size_t len)
 			/* If the other side is sending us error messages, try
 			 * to grab any messages they sent before they died. */
 			while (fd == sock_f_out && io_multiplexing_in) {
-				io_timeout = select_timeout = 30;
+				set_io_timeout(30);
 				ignore_timeout = 0;
 				readfd_unbuffered(sock_f_in, io_filesfrom_buf,
 						  sizeof io_filesfrom_buf);
@@ -1047,8 +1102,8 @@ static void writefd_unbuffered(int fd,char *buf,size_t len)
 		total += ret;
 
 		if (fd == sock_f_out) {
-			if (io_timeout)
-				last_io = time(NULL);
+			if (io_timeout || am_generator)
+				last_io_out = time(NULL);
 			sleep_for_bwlimit(ret);
 		}
 	}
@@ -1068,6 +1123,13 @@ static void mplex_write(enum msgcode code, char *buf, size_t len)
 
 	SIVAL(buffer, 0, ((MPLEX_BASE + (int)code)<<24) + len);
 
+	/* When the generator reads messages from the msg_fd_in pipe, it can
+	 * cause output to occur down the socket.  Setting contiguous_write_len
+	 * prevents the reading of msg_fd_in once we actually start to write
+	 * this sequence of data (though we might read it before the start). */
+	if (am_generator && msg_fd_in >= 0)
+		contiguous_write_len = len + 4;
+
 	if (n > sizeof buffer - 4)
 		n = sizeof buffer - 4;
 
@@ -1079,6 +1141,9 @@ static void mplex_write(enum msgcode code, char *buf, size_t len)
 
 	if (len)
 		writefd_unbuffered(sock_f_out, buf, len);
+
+	if (am_generator && msg_fd_in >= 0)
+		contiguous_write_len = 0;
 }
 
 
@@ -1187,19 +1252,37 @@ void write_buf(int f,char *buf,size_t len)
 	writefd(f,buf,len);
 }
 
-
 /** Write a string to the connection */
 void write_sbuf(int f, char *buf)
 {
 	writefd(f, buf, strlen(buf));
 }
 
-
 void write_byte(int f, uchar c)
 {
 	writefd(f, (char *)&c, 1);
 }
 
+void write_vstring(int f, char *str, int len)
+{
+	uchar lenbuf[3], *lb = lenbuf;
+
+	if (len > 0x7F) {
+		if (len > 0x7FFF) {
+			rprintf(FERROR,
+				"attempting to send over-long vstring (%d > %d)\n",
+				len, 0x7FFF);
+			exit_cleanup(RERR_PROTOCOL);
+		}
+		*lb++ = len / 0x100 + 0x80;
+	}
+	*lb = len;
+
+	writefd(f, (char*)lenbuf, lb - lenbuf + 1);
+	if (len)
+		writefd(f, str, len);
+}
+
 
 /**
  * Read a line of up to @p maxlen characters into @p buf (not counting

log.c

@@ -32,6 +32,7 @@ extern int dry_run;
 extern int am_daemon;
 extern int am_server;
 extern int am_sender;
+extern int local_server;
 extern int quiet;
 extern int module_id;
 extern int msg_fd_out;
@@ -43,6 +44,7 @@ extern char *auth_user;
 extern char *log_format;
 
 static int log_initialised;
+static int logfile_was_closed;
 static char *logfname;
 static FILE *logfile;
 struct stats stats;
@@ -71,8 +73,9 @@ struct {
 	{ RERR_TIMEOUT    , "timeout in data send/receive" },
 	{ RERR_CMD_FAILED , "remote shell failed" },
 	{ RERR_CMD_KILLED , "remote shell killed" },
-	{ RERR_CMD_RUN,     "remote command could not be run" },
-	{ RERR_CMD_NOTFOUND, "remote command not found" },
+	{ RERR_CMD_RUN    , "remote command could not be run" },
+	{ RERR_CMD_NOTFOUND,"remote command not found" },
+	{ RERR_DEL_LIMIT  , "the --max-delete limit stopped deletions" },
 	{ 0, NULL }
 };
 
@@ -94,9 +97,9 @@ static char const *rerr_name(int code)
 
 static void logit(int priority, char *buf)
 {
-	if (logfname) {
-		if (!logfile)
-			log_open();
+	if (logfile_was_closed)
+		logfile_reopen();
+	if (logfile) {
 		fprintf(logfile,"%s [%d] %s",
 			timestring(time(NULL)), (int)getpid(), buf);
 		fflush(logfile);
@@ -105,9 +108,48 @@ static void logit(int priority, char *buf)
 	}
 }
 
+static void syslog_init()
+{
+	static int been_here = 0;
+	int options = LOG_PID;
+
+	if (been_here)
+		return;
+	been_here = 1;
+
+#ifdef LOG_NDELAY
+	options |= LOG_NDELAY;
+#endif
+
+#ifdef LOG_DAEMON
+	openlog("rsyncd", options, lp_syslog_facility());
+#else
+	openlog("rsyncd", options);
+#endif
+
+#ifndef LOG_NDELAY
+	logit(LOG_INFO, "rsyncd started\n");
+#endif
+}
+
+static void logfile_open(void)
+{
+	extern int orig_umask;
+	int old_umask = umask(022 | orig_umask);
+	logfile = fopen(logfname, "a");
+	umask(old_umask);
+	if (!logfile) {
+		int fopen_errno = errno;
+		/* Rsync falls back to using syslog on failure. */
+		syslog_init();
+		rsyserr(FERROR, fopen_errno,
+			"failed to open log-file %s", logfname);
+		rprintf(FINFO, "Ignoring \"log file\" setting.\n");
+	}
+}
+
 void log_init(void)
 {
-	int options = LOG_PID;
 	time_t t;
 
 	if (log_initialised)
@@ -122,52 +164,29 @@ void log_init(void)
 
 	/* optionally use a log file instead of syslog */
 	logfname = lp_log_file();
-	if (logfname) {
-		if (*logfname) {
-			log_open();
-			return;
-		}
-		logfname = NULL;
-	}
-
-#ifdef LOG_NDELAY
-	options |= LOG_NDELAY;
-#endif
-
-#ifdef LOG_DAEMON
-	openlog("rsyncd", options, lp_syslog_facility());
-#else
-	openlog("rsyncd", options);
-#endif
-
-#ifndef LOG_NDELAY
-	logit(LOG_INFO,"rsyncd started\n");
-#endif
+	if (logfname && *logfname)
+		logfile_open();
+	else
+		syslog_init();
 }
 
-void log_open(void)
-{
-	if (logfname && !logfile) {
-		extern int orig_umask;
-		int old_umask = umask(022 | orig_umask);
-		logfile = fopen(logfname, "a");
-		umask(old_umask);
-		if (!logfile) {
-			am_daemon = 0; /* avoid trying to log again */
-			rsyserr(FERROR, errno, "fopen() of log-file failed");
-			exit_cleanup(RERR_FILESELECT);
-		}
-	}
-}
-
-void log_close(void)
+void logfile_close(void)
 {
 	if (logfile) {
+		logfile_was_closed = 1;
 		fclose(logfile);
 		logfile = NULL;
 	}
 }
 
+void logfile_reopen(void)
+{
+	if (logfile_was_closed) {
+		logfile_was_closed = 0;
+		logfile_open();
+	}
+}
+
 /* this is the underlying (unformatted) rsync debugging function. Call
  * it with FINFO, FERROR or FLOG */
 void rwrite(enum logcode code, char *buf, int len)
@@ -237,7 +256,7 @@ void rwrite(enum logcode code, char *buf, int len)
 	if (buf[len-1] == '\r' || buf[len-1] == '\n')
 		fflush(f);
 }
-		
+
 
 /* This is the rsync debugging function. Call it with FINFO, FERROR or
  * FLOG. */
@@ -317,7 +336,7 @@ void rsyserr(enum logcode code, int errcode, const char *format, ...)
 void rflush(enum logcode code)
 {
 	FILE *f = NULL;
-	
+
 	if (am_daemon) {
 		return;
 	}
@@ -345,48 +364,74 @@ void rflush(enum logcode code)
 
 /* a generic logging routine for send/recv, with parameter
  * substitiution */
-static void log_formatted(enum logcode code,
-			  char *format, char *op, struct file_struct *file,
-			  struct stats *initial_stats, int iflags)
+static void log_formatted(enum logcode code, char *format, char *op,
+			  struct file_struct *file, struct stats *initial_stats,
+			  int iflags, char *hlink)
 {
-	char buf[MAXPATHLEN+1024];
-	char buf2[MAXPATHLEN];
-	char *p, *n;
+	char buf[MAXPATHLEN+1024], buf2[MAXPATHLEN], fmt[32];
+	char *p, *s, *n;
 	size_t len, total;
 	int64 b;
 
+	*fmt = '%';
+
 	/* We expand % codes one by one in place in buf.  We don't
-	 * copy in the terminating nul of the inserted strings, but
-	 * rather keep going until we reach the nul of the format. */
+	 * copy in the terminating null of the inserted strings, but
+	 * rather keep going until we reach the null of the format. */
 	total = strlcpy(buf, format, sizeof buf);
-	
-	for (p = buf; (p = strchr(p, '%')) != NULL && p[1]; ) {
+	if (total > MAXPATHLEN) {
+		rprintf(FERROR, "log-format string is WAY too long!\n");
+		exit_cleanup(RERR_MESSAGEIO);
+	}
+	buf[total++] = '\n';
+	buf[total] = '\0';
+
+	for (p = buf; (p = strchr(p, '%')) != NULL; ) {
+		s = p++;
+		n = fmt + 1;
+		if (*p == '-')
+			*n++ = *p++;
+		while (isdigit(*(uchar*)p) && n - fmt < (int)(sizeof fmt) - 8)
+			*n++ = *p++;
+		if (!*p)
+			break;
+		*n = '\0';
 		n = NULL;
 
-		switch (p[1]) {
+		switch (*p) {
 		case 'h': if (am_daemon) n = client_name(0); break;
 		case 'a': if (am_daemon) n = client_addr(0); break;
 		case 'l':
-			snprintf(buf2, sizeof buf2, "%.0f",
+			strlcat(fmt, ".0f", sizeof fmt);
+			snprintf(buf2, sizeof buf2, fmt,
 				 (double)file->length);
 			n = buf2;
 			break;
 		case 'p':
-			snprintf(buf2, sizeof buf2, "%d",
-				 (int)getpid());
+			strlcat(fmt, "ld", sizeof fmt);
+			snprintf(buf2, sizeof buf2, fmt,
+				 (long)getpid());
 			n = buf2;
 			break;
 		case 'o': n = op; break;
 		case 'f':
-			pathjoin(buf2, sizeof buf2,
-			    am_sender && file->dir.root ? file->dir.root : "",
-			    safe_fname(f_name(file)));
-			clean_fname(buf2, 0);
-			n = buf2;
-			if (*n == '/') n++;
+			n = safe_fname(f_name(file));
+			if (am_sender && file->dir.root) {
+				pathjoin(buf2, sizeof buf2,
+					 file->dir.root, n);
+				/* The buffer from safe_fname() has more
+				 * room than MAXPATHLEN, so this is safe. */
+				if (fmt[1])
+					strcpy(n, buf2);
+				else
+					n = buf2;
+			}
+			clean_fname(n, 0);
+			if (*n == '/')
+				n++;
 			break;
 		case 'n':
-			n = (char*)safe_fname(f_name(file));
+			n = safe_fname(f_name(file));
 			if (S_ISDIR(file->mode)) {
 				/* The buffer from safe_fname() has more
 				 * room than MAXPATHLEN, so this is safe. */
@@ -394,12 +439,21 @@ static void log_formatted(enum logcode code,
 			}
 			break;
 		case 'L':
-			if (S_ISLNK(file->mode) && file->u.link) {
-				snprintf(buf2, sizeof buf2, " -> %s",
-					 safe_fname(file->u.link));
-				n = buf2;
-			} else
+			if (hlink && *hlink) {
+				n = safe_fname(hlink);
+				strcpy(buf2, " => ");
+			} else if (S_ISLNK(file->mode) && file->u.link) {
+				n = safe_fname(file->u.link);
+				strcpy(buf2, " -> ");
+			} else {
 				n = "";
+				if (!fmt[1])
+					break;
+				strcpy(buf2, "    ");
+			}
+			strlcat(fmt, "s", sizeof fmt);
+			snprintf(buf2 + 4, sizeof buf2 - 4, fmt, n);
+			n = buf2;
 			break;
 		case 'm': n = lp_name(module_id); break;
 		case 't': n = timestring(time(NULL)); break;
@@ -413,7 +467,8 @@ static void log_formatted(enum logcode code,
 				b = stats.total_read -
 					initial_stats->total_read;
 			}
-			snprintf(buf2, sizeof buf2, "%.0f", (double)b);
+			strlcat(fmt, ".0f", sizeof fmt);
+			snprintf(buf2, sizeof buf2, fmt, (double)b);
 			n = buf2;
 			break;
 		case 'c':
@@ -424,17 +479,20 @@ static void log_formatted(enum logcode code,
 				b = stats.total_read -
 					initial_stats->total_read;
 			}
-			snprintf(buf2, sizeof buf2, "%.0f", (double)b);
+			strlcat(fmt, ".0f", sizeof fmt);
+			snprintf(buf2, sizeof buf2, fmt, (double)b);
 			n = buf2;
 			break;
 		case 'i':
 			if (iflags & ITEM_DELETED) {
-				n = "deleting";
+				n = "*deleting";
 				break;
 			}
-			n = buf2;
-			n[0] = !(iflags & ITEM_UPDATING) ? '.'
-			     : *op == 's' ? '>' : '<';
+			n = buf2 + MAXPATHLEN - 32;
+			n[0] = iflags & ITEM_LOCAL_CHANGE
+			      ? iflags & ITEM_XNAME_FOLLOWS ? 'h' : 'c'
+			     : !(iflags & ITEM_TRANSFER) ? '.'
+			     : !local_server && *op == 's' ? '<' : '>';
 			n[1] = S_ISDIR(file->mode) ? 'd'
 			     : IS_DEVICE(file->mode) ? 'D'
 			     : S_ISLNK(file->mode) ? 'L' : 'f';
@@ -446,14 +504,15 @@ static void log_formatted(enum logcode code,
 			n[5] = !(iflags & ITEM_REPORT_PERMS) ? '.' : 'p';
 			n[6] = !(iflags & ITEM_REPORT_OWNER) ? '.' : 'o';
 			n[7] = !(iflags & ITEM_REPORT_GROUP) ? '.' : 'g';
-			n[8] = '\0';
+			n[8] = !(iflags & ITEM_REPORT_XATTRS) ? '.' : 'a';
+			n[9] = '\0';
 
 			if (iflags & (ITEM_IS_NEW|ITEM_MISSING_DATA)) {
 				char ch = iflags & ITEM_IS_NEW ? '+' : '?';
 				int i;
 				for (i = 2; n[i]; i++)
 					n[i] = ch;
-			} else if (!(iflags & ITEM_UPDATING)) {
+			} else if (!(iflags & (ITEM_TRANSFER|ITEM_LOCAL_CHANGE))) {
 				int i;
 				for (i = 2; n[i]; i++) {
 					if (n[i] != '.')
@@ -462,23 +521,25 @@ static void log_formatted(enum logcode code,
 				if (!n[i]) {
 					for (i = 2; n[i]; i++)
 						n[i] = ' ';
-					n[0] = '=';
 				}
 			}
 			break;
 		}
 
-		/* n is the string to be inserted in place of this %
-		 * code; len is its length not including the trailing
-		 * NUL */
-		if (!n) {
-			p += 2;
+		/* "n" is the string to be inserted in place of this % code. */
+		if (!n)
 			continue;
+		if (n != buf2 && fmt[1]) {
+			strlcat(fmt, "s", sizeof fmt);
+			snprintf(buf2, sizeof buf2, fmt, n);
+			n = buf2;
 		}
-
 		len = strlen(n);
 
-		if (len + total - 2 >= sizeof buf) {
+		/* Subtract the length of the escape from the string's size. */
+		total -= p - s + 1;
+
+		if (len + total >= (size_t)sizeof buf) {
 			rprintf(FERROR,
 				"buffer overflow expanding %%%c -- exiting\n",
 				p[0]);
@@ -486,45 +547,69 @@ static void log_formatted(enum logcode code,
 		}
 
 		/* Shuffle the rest of the string along to make space for n */
-		if (len != 2)
-			memmove(p + len, p + 2, total - (p + 2 - buf) + 1);
-		total += len - 2;
+		if (len != (size_t)(p - s + 1))
+			memmove(s + len, p + 1, total - (s - buf) + 1);
+		total += len;
 
-		/* Insert the contents of string "n", but NOT its nul. */
+		/* Insert the contents of string "n", but NOT its null. */
 		if (len)
-			memcpy(p, n, len);
+			memcpy(s, n, len);
 
 		/* Skip over inserted string; continue looking */
-		p += len;
+		p = s + len;
 	}
 
-	rprintf(code, "%s\n", buf);
+	rwrite(code, buf, total);
 }
 
-/* log the outgoing transfer of a file */
-void log_send(struct file_struct *file, struct stats *initial_stats, int iflags)
+/* Return 1 if the format escape is in the log-format string (e.g. look for
+ * the 'b' in the "%9b" format escape). */
+int log_format_has(const char *format, char esc)
 {
-	if (lp_transfer_logging(module_id)) {
-		log_formatted(FLOG, lp_log_format(module_id), "send",
-			      file, initial_stats, iflags);
-	} else if (log_format && !am_server) {
-		log_formatted(FINFO, log_format, "send",
-			      file, initial_stats, iflags);
+	const char *p;
+
+	if (!format)
+		return 0;
+
+	for (p = format; (p = strchr(p, '%')) != NULL; ) {
+		if (*++p == '-')
+			p++;
+		while (isdigit(*(uchar*)p))
+			p++;
+		if (!*p)
+			break;
+		if (*p == esc)
+			return 1;
 	}
+	return 0;
 }
 
-/* log the incoming transfer of a file */
-void log_recv(struct file_struct *file, struct stats *initial_stats, int iflags)
+/* log the transfer of a file */
+void log_item(struct file_struct *file, struct stats *initial_stats,
+	      int iflags, char *hlink)
 {
+	char *s_or_r = am_sender ? "send" : "recv";
+
 	if (lp_transfer_logging(module_id)) {
-		log_formatted(FLOG, lp_log_format(module_id), "recv",
-			      file, initial_stats, iflags);
+		log_formatted(FLOG, lp_log_format(module_id), s_or_r,
+			      file, initial_stats, iflags, hlink);
 	} else if (log_format && !am_server) {
-		log_formatted(FINFO, log_format, "recv",
-			      file, initial_stats, iflags);
+		log_formatted(FINFO, log_format, s_or_r,
+			      file, initial_stats, iflags, hlink);
 	}
 }
 
+void maybe_log_item(struct file_struct *file, int iflags, int itemizing,
+		    char *buf)
+{
+	int see_item = itemizing && (iflags || verbose > 1);
+	if (am_server) {
+		if (am_daemon && !dry_run && see_item)
+			log_item(file, &stats, iflags, buf);
+	} else if (see_item || iflags & ITEM_LOCAL_CHANGE || *buf
+	    || (S_ISDIR(file->mode) && iflags & SIGNIFICANT_ITEM_FLAGS))
+		log_item(file, &stats, iflags, buf);
+}
 
 void log_delete(char *fname, int mode)
 {
@@ -542,15 +627,16 @@ void log_delete(char *fname, int mode)
 			len++; /* directories include trailing null */
 		send_msg(MSG_DELETED, fname, len);
 	} else {
-		fmt = log_format_has_o_or_i ? log_format : "%i %n";
-		log_formatted(FCLIENT, fmt, "del.", &file, &stats, ITEM_DELETED);
+		fmt = log_format_has_o_or_i ? log_format : "deleting %n";
+		log_formatted(FCLIENT, fmt, "del.", &file, &stats,
+			      ITEM_DELETED, NULL);
 	}
 
 	if (!am_daemon || dry_run || !lp_transfer_logging(module_id))
 		return;
 
-	fmt = daemon_log_format_has_o_or_i ? lp_log_format(module_id) : "%i %n";
-	log_formatted(FLOG, fmt, "del.", &file, &stats, ITEM_DELETED);
+	fmt = daemon_log_format_has_o_or_i ? lp_log_format(module_id) : "deleting %n";
+	log_formatted(FLOG, fmt, "del.", &file, &stats, ITEM_DELETED, NULL);
 }
 
 

main.c

@@ -20,8 +20,9 @@
 */
 
 #include "rsync.h"
-
-time_t starttime = 0;
+#if defined CONFIG_LOCALE && defined HAVE_LOCALE_H
+#include <locale.h>
+#endif
 
 extern int verbose;
 extern int dry_run;
@@ -66,6 +67,7 @@ extern char *shell_cmd;
 extern char *batch_name;
 
 int local_server = 0;
+struct file_list *the_file_list;
 
 /* There's probably never more than at most 2 outstanding child processes,
  * but set it higher, just in case. */
@@ -76,6 +78,9 @@ struct pid_status {
 	int   status;
 } pid_stat_table[MAXCHILDPROCS];
 
+static time_t starttime, endtime;
+static int64 total_read, total_written;
+
 static void show_malloc_stats(void);
 
 /****************************************************************************
@@ -120,12 +125,13 @@ void wait_process(pid_t pid, int *status)
  * the report.  All processes might also generate a set of debug stats, if
  * the verbose level is high enough (this is the only thing that the
  * generator process and the server receiver ever do here). */
-static void report(int f)
+static void handle_stats(int f)
 {
+	endtime = time(NULL);
+
 	/* Cache two stats because the read/write code can change it. */
-	int64 total_read = stats.total_read;
-	int64 total_written = stats.total_written;
-	time_t t = time(NULL);
+	total_read = stats.total_read;
+	total_written = stats.total_written;
 
 	if (do_stats && verbose > 1) {
 		/* These come out from every process */
@@ -157,7 +163,9 @@ static void report(int f)
 
 	/* this is the client */
 
-	if (!am_sender) {
+	if (f < 0 && !am_sender) /* e.g. when we got an empty file list. */
+		; 
+	else if (!am_sender) {
 		/* Read the first two in opposite order because the meaning of
 		 * read/write swaps when switching from sender to receiver. */
 		total_written = read_longint(f);
@@ -178,7 +186,10 @@ static void report(int f)
 			write_longint(batch_fd, stats.flist_xfertime);
 		}
 	}
+}
 
+static void output_summary(void)
+{
 	if (do_stats) {
 		rprintf(FINFO,"\nNumber of files: %d\n", stats.num_files);
 		rprintf(FINFO,"Number of files transferred: %d\n",
@@ -210,7 +221,7 @@ static void report(int f)
 		rprintf(FINFO,
 			"\nsent %.0f bytes  received %.0f bytes  %.2f bytes/sec\n",
 			(double)total_written, (double)total_read,
-			(total_written + total_read)/(0.5 + (t - starttime)));
+			(total_written + total_read)/(0.5 + (endtime - starttime)));
 		rprintf(FINFO, "total size is %.0f  speedup is %.2f\n",
 			(double)stats.total_size,
 			(double)stats.total_size / (total_written+total_read));
@@ -282,7 +293,7 @@ static pid_t do_cmd(char *cmd, char *machine, char *user, char *path,
 
 		for (tok = strtok(cmd, " "); tok; tok = strtok(NULL, " ")) {
 			/* Comparison leaves rooms for server_options(). */
-			if (argc >= MAX_ARGS - 100) {
+			if (argc >= MAX_ARGS - MAX_SERVER_ARGS) {
 				rprintf(FERROR, "internal: args[] overflowed in do_cmd()\n");
 				exit_cleanup(RERR_SYNTAX);
 			}
@@ -429,23 +440,24 @@ static char *get_local_name(struct file_list *flist,char *name)
 
 
 /* This is only called by the sender. */
-static void read_final_goodbye(int f_in, int f_out, int flist_count)
+static void read_final_goodbye(int f_in, int f_out)
 {
 	int i;
 
 	if (protocol_version < 29)
 		i = read_int(f_in);
 	else {
-		while ((i = read_int(f_in)) == flist_count
+		while ((i = read_int(f_in)) == the_file_list->count
 		    && read_shortint(f_in) == ITEM_IS_NEW) {
 			/* Forward the keep-alive (no-op) to the receiver. */
-			write_int(f_out, flist_count);
+			write_int(f_out, the_file_list->count);
 			write_shortint(f_out, ITEM_IS_NEW);
 		}
 	}
 
 	if (i != -1) {
-		rprintf(FERROR, "Invalid packet from generator at end of run.\n");
+		rprintf(FERROR, "Invalid packet at end of run (%d) [%s]\n",
+			i, who_am_i());
 		exit_cleanup(RERR_PROTOCOL);
 	}
 }
@@ -500,15 +512,16 @@ static void do_server_sender(int f_in, int f_out, int argc,char *argv[])
 	if (!flist || flist->count == 0) {
 		exit_cleanup(0);
 	}
+	the_file_list = flist;
 
 	io_start_buffering_in();
 	io_start_buffering_out();
 
 	send_files(flist,f_out,f_in);
 	io_flush(FULL_FLUSH);
-	report(f_out);
+	handle_stats(f_out);
 	if (protocol_version >= 24)
-		read_final_goodbye(f_in, f_out, flist->count);
+		read_final_goodbye(f_in, f_out);
 	io_flush(FULL_FLUSH);
 	exit_cleanup(0);
 }
@@ -518,19 +531,16 @@ static int do_recv(int f_in,int f_out,struct file_list *flist,char *local_name)
 {
 	int pid;
 	int status = 0;
-	int error_pipe[2], name_pipe[2];
-	BOOL need_name_pipe = (basis_dir[0] || partial_dir || fuzzy_basis
-			    || (inplace && make_backups)) && !dry_run;
+	int error_pipe[2];
 
 	/* The receiving side mustn't obey this, or an existing symlink that
 	 * points to an identical file won't be replaced by the referent. */
 	copy_links = 0;
 
 	if (preserve_hard_links)
-		init_hard_links(flist);
+		init_hard_links();
 
-	if (fd_pair(error_pipe) < 0
-	    || (need_name_pipe && fd_pair(name_pipe) < 0)) {
+	if (fd_pair(error_pipe) < 0) {
 		rsyserr(FERROR, errno, "pipe failed in do_recv");
 		exit_cleanup(RERR_IPC);
 	}
@@ -544,11 +554,6 @@ static int do_recv(int f_in,int f_out,struct file_list *flist,char *local_name)
 
 	if (pid == 0) {
 		close(error_pipe[0]);
-		if (need_name_pipe) {
-			close(name_pipe[1]);
-			set_blocking(name_pipe[0]);
-		} else
-			name_pipe[0] = -1;
 		if (f_in != f_out)
 			close(f_out);
 
@@ -558,9 +563,9 @@ static int do_recv(int f_in,int f_out,struct file_list *flist,char *local_name)
 		/* set place to send errors */
 		set_msg_fd_out(error_pipe[1]);
 
-		recv_files(f_in, flist, local_name, name_pipe[0]);
+		recv_files(f_in, flist, local_name);
 		io_flush(FULL_FLUSH);
-		report(f_in);
+		handle_stats(f_in);
 
 		send_msg(MSG_DONE, "", 0);
 		io_flush(FULL_FLUSH);
@@ -574,7 +579,8 @@ static int do_recv(int f_in,int f_out,struct file_list *flist,char *local_name)
 			while (read_int(f_in) == flist->count
 			    && read_shortint(f_in) == ITEM_IS_NEW) {}
 
-			rprintf(FERROR, "Invalid packet from server at end of run.\n");
+			rprintf(FERROR, "Invalid packet at end of run [%s]\n",
+				who_am_i());
 			exit_cleanup(RERR_PROTOCOL);
 		}
 
@@ -587,15 +593,10 @@ static int do_recv(int f_in,int f_out,struct file_list *flist,char *local_name)
 
 	am_generator = 1;
 	close_multiplexing_in();
-	if (write_batch)
+	if (write_batch && !am_server)
 		stop_write_batch();
 
 	close(error_pipe[1]);
-	if (need_name_pipe) {
-		close(name_pipe[0]);
-		set_nonblocking(name_pipe[1]);
-	} else
-		name_pipe[1] = -1;
 	if (f_in != f_out)
 		close(f_in);
 
@@ -603,9 +604,9 @@ static int do_recv(int f_in,int f_out,struct file_list *flist,char *local_name)
 
 	set_msg_fd_in(error_pipe[0]);
 
-	generate_files(f_out, flist, local_name, name_pipe[1]);
+	generate_files(f_out, flist, local_name);
 
-	report(-1);
+	handle_stats(-1);
 	io_flush(FULL_FLUSH);
 	if (protocol_version >= 24) {
 		/* send a final goodbye message */
@@ -676,6 +677,7 @@ static void do_server_recv(int f_in, int f_out, int argc,char *argv[])
 		rprintf(FERROR,"server_recv: recv_file_list error\n");
 		exit_cleanup(RERR_FILESELECT);
 	}
+	the_file_list = flist;
 
 	if (argc > 0) {
 		if (strcmp(dir,".")) {
@@ -764,26 +766,27 @@ int client_run(int f_in, int f_out, pid_t pid, int argc, char *argv[])
 		if (filesfrom_host)
 			filesfrom_fd = f_in;
 
-		if (write_batch)
+		if (write_batch && !am_server)
 			start_write_batch(f_out);
-		if (!read_batch) /* don't write to pipe */
-			flist = send_file_list(f_out,argc,argv);
+		flist = send_file_list(f_out, argc, argv);
 		set_msg_fd_in(-1);
 		if (verbose > 3)
 			rprintf(FINFO,"file list sent\n");
+		the_file_list = flist;
 
 		io_flush(NORMAL_FLUSH);
 		send_files(flist,f_out,f_in);
 		io_flush(FULL_FLUSH);
+		handle_stats(-1);
 		if (protocol_version >= 24)
-			read_final_goodbye(f_in, f_out, flist->count);
+			read_final_goodbye(f_in, f_out);
 		if (pid != -1) {
 			if (verbose > 3)
 				rprintf(FINFO,"client_run waiting on %d\n", (int) pid);
 			io_flush(FULL_FLUSH);
 			wait_process(pid, &status);
 		}
-		report(-1);
+		output_summary();
 		io_flush(FULL_FLUSH);
 		exit_cleanup(status);
 	}
@@ -801,20 +804,20 @@ int client_run(int f_in, int f_out, pid_t pid, int argc, char *argv[])
 		filesfrom_fd = -1;
 	}
 
-	if (write_batch)
+	if (write_batch && !am_server)
 		start_write_batch(f_in);
 	flist = recv_file_list(f_in);
-	if (!flist || flist->count == 0) {
-		rprintf(FINFO, "client: nothing to do: "
-			"perhaps you need to specify some filenames or "
-			"the --recursive option?\n");
-		exit_cleanup(0);
+	the_file_list = flist;
+
+	if (flist && flist->count > 0) {
+		local_name = get_local_name(flist, argv[0]);
+
+		status2 = do_recv(f_in, f_out, flist, local_name);
+	} else {
+		handle_stats(-1);
+		output_summary();
 	}
 
-	local_name = get_local_name(flist,argv[0]);
-
-	status2 = do_recv(f_in,f_out,flist,local_name);
-
 	if (pid != -1) {
 		if (verbose > 3)
 			rprintf(FINFO,"client_run2 waiting on %d\n", (int) pid);
@@ -984,8 +987,11 @@ static RETSIGTYPE sigusr1_handler(UNUSED(int val))
 
 static RETSIGTYPE sigusr2_handler(UNUSED(int val))
 {
+	if (!am_server)
+		output_summary();
 	close_all();
-	if (log_got_error) _exit(RERR_PARTIAL);
+	if (log_got_error)
+		_exit(RERR_PARTIAL);
 	_exit(0);
 }
 
@@ -1110,6 +1116,10 @@ int main(int argc,char *argv[])
 	 * see the EPIPE. */
 	signal(SIGPIPE, SIG_IGN);
 
+#if defined CONFIG_LOCALE && defined HAVE_SETLOCALE
+	setlocale(LC_CTYPE, "");
+#endif
+
 	/* Initialize push_dir here because on some old systems getcwd
 	 * (implemented by forking "pwd" and reading its output) doesn't
 	 * work when there are other child processes.  Also, on all systems
@@ -1118,7 +1128,7 @@ int main(int argc,char *argv[])
 
 	init_flist();
 
-	if (write_batch || read_batch) {
+	if ((write_batch || read_batch) && !am_server) {
 		if (write_batch)
 			write_batch_shell_file(orig_argc, orig_argv, argc);
 
@@ -1137,6 +1147,8 @@ int main(int argc,char *argv[])
 		if (read_batch)
 			read_stream_flags(batch_fd);
 	}
+	if (write_batch < 0)
+		dry_run = 1;
 
 	if (am_daemon && !am_server)
 		return daemon_main();

match.c

@@ -133,12 +133,8 @@ static void matched(int f, struct sum_struct *s, struct map_struct *buf,
 	else
 		last_match = offset;
 
-	if (buf && do_progress) {
+	if (buf && do_progress)
 		show_progress(last_match, buf->file_size);
-
-		if (i == -1)
-			end_progress(buf->file_size);
-	}
 }
 
 

options.c

@@ -23,7 +23,6 @@
 
 extern int module_id;
 extern int sanitize_paths;
-extern int select_timeout;
 extern struct filter_list_struct filter_list;
 extern struct filter_list_struct server_filter_list;
 
@@ -53,6 +52,7 @@ int omit_dir_times = 0;
 int update_only = 0;
 int cvs_exclude = 0;
 int dry_run = 0;
+int do_xfers = 1;
 int ignore_times = 0;
 int delete_mode = 0;
 int delete_during = 0;
@@ -75,6 +75,7 @@ int implied_dirs = 1;
 int numeric_ids = 0;
 int force_delete = 0;
 int io_timeout = 0;
+int allowed_lull = 0;
 char *files_from = NULL;
 int filesfrom_fd = -1;
 char *filesfrom_host = NULL;
@@ -143,12 +144,13 @@ char *backup_dir = NULL;
 char backup_dir_buf[MAXPATHLEN];
 int rsync_port = 0;
 int compare_dest = 0;
+int copy_dest = 0;
 int link_dest = 0;
 int basis_dir_cnt = 0;
+char *dest_option = NULL;
 
 int verbose = 0;
 int quiet = 0;
-int itemize_changes = 0;
 int log_before_transfer = 0;
 int log_format_has_i = 0;
 int log_format_has_o_or_i = 0;
@@ -161,9 +163,9 @@ char *batch_name = NULL;
 static int daemon_opt;   /* sets am_daemon after option error-reporting */
 static int F_option_cnt = 0;
 static int modify_window_set;
+static int itemize_changes = 0;
 static int refused_delete, refused_archive_part;
 static int refused_partial, refused_progress, refused_delete_before;
-static char *dest_option = NULL;
 static char *max_size_arg;
 static char partialdir_for_delayupdate[] = ".~tmp~";
 
@@ -249,7 +251,7 @@ void usage(enum logcode F)
   rprintf(F,"\nrsync is a file transfer program capable of efficient remote update\nvia a fast differencing algorithm.\n\n");
 
   rprintf(F,"Usage: rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST\n");
-  rprintf(F,"  or   rsync [OPTION]... [USER@]HOST:SRC DEST\n");
+  rprintf(F,"  or   rsync [OPTION]... [USER@]HOST:SRC [DEST]\n");
   rprintf(F,"  or   rsync [OPTION]... SRC [SRC]... DEST\n");
   rprintf(F,"  or   rsync [OPTION]... [USER@]HOST::SRC [DEST]\n");
   rprintf(F,"  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST\n");
@@ -292,7 +294,7 @@ void usage(enum logcode F)
   rprintf(F," -x, --one-file-system       don't cross filesystem boundaries\n");
   rprintf(F," -B, --block-size=SIZE       force a fixed checksum block-size\n");
   rprintf(F," -e, --rsh=COMMAND           specify the remote shell to use\n");
-  rprintf(F,"     --rsync-path=PATH       specify path to rsync on the remote machine\n");
+  rprintf(F,"     --rsync-path=PROGRAM    specify the rsync to run on the remote machine\n");
   rprintf(F,"     --existing              only update files that already exist on receiver\n");
   rprintf(F,"     --ignore-existing       ignore files that already exist on receiving side\n");
   rprintf(F,"     --remove-sent-files     sent files/symlinks are removed from sending side\n");
@@ -317,6 +319,7 @@ void usage(enum logcode F)
   rprintf(F," -T, --temp-dir=DIR          create temporary files in directory DIR\n");
   rprintf(F," -y, --fuzzy                 find similar file for basis if no dest file\n");
   rprintf(F,"     --compare-dest=DIR      also compare destination files relative to DIR\n");
+  rprintf(F,"     --copy-dest=DIR         ... and include copies of unchanged files\n");
   rprintf(F,"     --link-dest=DIR         hardlink to files in DIR when unchanged\n");
   rprintf(F," -z, --compress              compress file data during the transfer\n");
   rprintf(F," -C, --cvs-exclude           auto-ignore files the same way CVS does\n");
@@ -328,8 +331,8 @@ void usage(enum logcode F)
   rprintf(F,"     --include=PATTERN       don't exclude files matching PATTERN\n");
   rprintf(F,"     --include-from=FILE     read include patterns from FILE\n");
   rprintf(F,"     --files-from=FILE       read list of source-file names from FILE\n");
-  rprintf(F," -0, --from0                 all *-from file lists are delimited by nulls\n");
-  rprintf(F,"     --version               print version number\n");
+  rprintf(F," -0, --from0                 all *-from/filter files are delimited by 0s\n");
+  rprintf(F,"     --address=ADDRESS       bind address for outgoing socket to daemon\n");
   rprintf(F,"     --port=PORT             specify double-colon alternate port number\n");
   rprintf(F,"     --blocking-io           use blocking I/O for the remote shell\n");
   rprintf(F,"     --no-blocking-io        turn off blocking I/O when it is the default\n");
@@ -337,16 +340,19 @@ void usage(enum logcode F)
   rprintf(F,"     --progress              show progress during transfer\n");
   rprintf(F," -P                          same as --partial --progress\n");
   rprintf(F," -i, --itemize-changes       output a change-summary for all updates\n");
-  rprintf(F,"     --log-format=FORMAT     log file-transfers using specified format\n");
+  rprintf(F,"     --log-format=FORMAT     output filenames using the specified format\n");
   rprintf(F,"     --password-file=FILE    read password from FILE\n");
   rprintf(F,"     --list-only             list the files instead of copying them\n");
   rprintf(F,"     --bwlimit=KBPS          limit I/O bandwidth; KBytes per second\n");
   rprintf(F,"     --write-batch=FILE      write a batched update to FILE\n");
+  rprintf(F,"     --only-write-batch=FILE like --write-batch but w/o updating destination\n");
   rprintf(F,"     --read-batch=FILE       read a batched update from FILE\n");
+  rprintf(F,"     --protocol=NUM          force an older protocol version to be used\n");
 #ifdef INET6
   rprintf(F," -4, --ipv4                  prefer IPv4\n");
   rprintf(F," -6, --ipv6                  prefer IPv6\n");
 #endif
+  rprintf(F,"     --version               print version number\n");
   rprintf(F," -h, --help                  show this help screen\n");
 
   rprintf(F,"\nUse \"rsync --daemon --help\" to see the daemon-mode command-line options.\n");
@@ -355,9 +361,9 @@ void usage(enum logcode F)
 }
 
 enum {OPT_VERSION = 1000, OPT_DAEMON, OPT_SENDER, OPT_EXCLUDE, OPT_EXCLUDE_FROM,
-      OPT_FILTER, OPT_COMPARE_DEST, OPT_LINK_DEST,
+      OPT_FILTER, OPT_COMPARE_DEST, OPT_COPY_DEST, OPT_LINK_DEST,
       OPT_INCLUDE, OPT_INCLUDE_FROM, OPT_MODIFY_WINDOW,
-      OPT_READ_BATCH, OPT_WRITE_BATCH, OPT_TIMEOUT, OPT_MAX_SIZE,
+      OPT_READ_BATCH, OPT_WRITE_BATCH, OPT_ONLY_WRITE_BATCH, OPT_MAX_SIZE,
       OPT_REFUSED_BASE = 9000};
 
 static struct poptOption long_options[] = {
@@ -413,7 +419,7 @@ static struct poptOption long_options[] = {
   {"archive",         'a', POPT_ARG_NONE,   &archive_mode, 0, 0, 0 },
   {"server",           0,  POPT_ARG_NONE,   &am_server, 0, 0, 0 },
   {"sender",           0,  POPT_ARG_NONE,   0, OPT_SENDER, 0, 0 },
-  {"recursive",       'r', POPT_ARG_VAL,    &recurse, -1, 0, 0 },
+  {"recursive",       'r', POPT_ARG_NONE,   &recurse, 0, 0, 0 },
   {"list-only",        0,  POPT_ARG_VAL,    &list_only, 2, 0, 0 },
   {"relative",        'R', POPT_ARG_VAL,    &relative_paths, 1, 0, 0 },
   {"no-relative",      0,  POPT_ARG_VAL,    &relative_paths, 0, 0, 0 },
@@ -421,9 +427,10 @@ static struct poptOption long_options[] = {
   {"block-size",      'B', POPT_ARG_LONG,   &block_size, 0, 0, 0 },
   {"max-delete",       0,  POPT_ARG_INT,    &max_delete, 0, 0, 0 },
   {"max-size",         0,  POPT_ARG_STRING, &max_size_arg,  OPT_MAX_SIZE, 0, 0 },
-  {"timeout",          0,  POPT_ARG_INT,    &io_timeout, OPT_TIMEOUT, 0, 0 },
+  {"timeout",          0,  POPT_ARG_INT,    &io_timeout, 0, 0, 0 },
   {"temp-dir",        'T', POPT_ARG_STRING, &tmpdir, 0, 0, 0 },
   {"compare-dest",     0,  POPT_ARG_STRING, 0, OPT_COMPARE_DEST, 0, 0 },
+  {"copy-dest",        0,  POPT_ARG_STRING, 0, OPT_COPY_DEST, 0, 0 },
   {"link-dest",        0,  POPT_ARG_STRING, 0, OPT_LINK_DEST, 0, 0 },
   {"fuzzy",           'y', POPT_ARG_NONE,   &fuzzy_basis, 0, 0, 0 },
   /* TODO: Should this take an optional int giving the compression level? */
@@ -438,6 +445,7 @@ static struct poptOption long_options[] = {
   {"no-blocking-io",   0,  POPT_ARG_VAL,    &blocking_io, 0, 0, 0 },
   {0,                 'F', POPT_ARG_NONE,   0, 'F', 0, 0 },
   {0,                 'P', POPT_ARG_NONE,   0, 'P', 0, 0 },
+  {"address",          0,  POPT_ARG_STRING, &bind_address, 0, 0, 0 },
   {"port",             0,  POPT_ARG_INT,    &rsync_port, 0, 0, 0 },
   {"log-format",       0,  POPT_ARG_STRING, &log_format, 0, 0, 0 },
   {"itemize-changes", 'i', POPT_ARG_NONE,   &itemize_changes, 0, 0, 0 },
@@ -446,6 +454,7 @@ static struct poptOption long_options[] = {
   {"hard-links",      'H', POPT_ARG_NONE,   &preserve_hard_links, 0, 0, 0 },
   {"read-batch",       0,  POPT_ARG_STRING, &batch_name, OPT_READ_BATCH, 0, 0 },
   {"write-batch",      0,  POPT_ARG_STRING, &batch_name, OPT_WRITE_BATCH, 0, 0 },
+  {"only-write-batch", 0,  POPT_ARG_STRING, &batch_name, OPT_ONLY_WRITE_BATCH, 0, 0 },
   {"files-from",       0,  POPT_ARG_STRING, &files_from, 0, 0, 0 },
   {"from0",           '0', POPT_ARG_NONE,   &eol_nulls, 0, 0, 0},
   {"no-implied-dirs",  0,  POPT_ARG_VAL,    &implied_dirs, 0, 0, 0 },
@@ -456,7 +465,6 @@ static struct poptOption long_options[] = {
   {"ipv6",            '6', POPT_ARG_VAL,    &default_af_hint, AF_INET6, 0, 0 },
 #endif
   /* All these options switch us into daemon-mode option-parsing. */
-  {"address",          0,  POPT_ARG_STRING, 0, OPT_DAEMON, 0, 0 },
   {"config",           0,  POPT_ARG_STRING, 0, OPT_DAEMON, 0, 0 },
   {"daemon",           0,  POPT_ARG_NONE,   0, OPT_DAEMON, 0, 0 },
   {"detach",           0,  POPT_ARG_NONE,   0, OPT_DAEMON, 0, 0 },
@@ -613,6 +621,35 @@ static int count_args(const char **argv)
 }
 
 
+static OFF_T parse_size_arg(const char *size_arg)
+{
+	const char *arg;
+	OFF_T size;
+
+	for (arg = size_arg; isdigit(*(uchar*)arg); arg++) {}
+	if (*arg == '.')
+		for (arg++; isdigit(*(uchar*)arg); arg++) {}
+	switch (*arg) {
+	case 'k': case 'K':
+		size = atof(size_arg) * 1024;
+		break;
+	case 'm': case 'M':
+		size = atof(size_arg) * 1024*1024;
+		break;
+	case 'g': case 'G':
+		size = atof(size_arg) * 1024*1024*1024;
+		break;
+	case '\0':
+		size = atof(size_arg);
+		break;
+	default:
+		size = 0;
+		break;
+	}
+	return size;
+}
+
+
 static void create_refuse_error(int which)
 {
 	/* The "which" value is the index + OPT_REFUSED_BASE. */
@@ -787,33 +824,18 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 			write_batch = 1;
 			break;
 
+		case OPT_ONLY_WRITE_BATCH:
+			/* batch_name is already set */
+			write_batch = -1;
+			break;
+
 		case OPT_READ_BATCH:
 			/* batch_name is already set */
 			read_batch = 1;
 			break;
 
 		case OPT_MAX_SIZE:
-			for (arg = max_size_arg; isdigit(*(uchar*)arg); arg++) {}
-			if (*arg == '.')
-				for (arg++; isdigit(*(uchar*)arg); arg++) {}
-			switch (*arg) {
-			case 'k': case 'K':
-				max_size = atof(max_size_arg) * 1024;
-				break;
-			case 'm': case 'M':
-				max_size = atof(max_size_arg) * 1024*1024;
-				break;
-			case 'g': case 'G':
-				max_size = atof(max_size_arg) * 1024*1024*1024;
-				break;
-			case '\0':
-				max_size = atof(max_size_arg);
-				break;
-			default:
-				max_size = 0;
-				break;
-			}
-			if (max_size <= 0) {
+			if ((max_size = parse_size_arg(max_size_arg)) <= 0) {
 				snprintf(err_buf, sizeof err_buf,
 					"--max-size value is invalid: %s\n",
 					max_size_arg);
@@ -821,11 +843,6 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 			}
 			break;
 
-		case OPT_TIMEOUT:
-			if (io_timeout && io_timeout < select_timeout)
-				select_timeout = io_timeout;
-			break;
-
 		case OPT_LINK_DEST:
 #ifdef HAVE_LINK
 			link_dest = 1;
@@ -838,6 +855,11 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 			return 0;
 #endif
 
+		case OPT_COPY_DEST:
+			copy_dest = 1;
+			dest_option = "--copy-dest";
+			goto set_dest_dir;
+
 		case OPT_COMPARE_DEST:
 			compare_dest = 1;
 			dest_option = "--compare-dest";
@@ -892,13 +914,7 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 			"--write-batch and --read-batch can not be used together\n");
 		return 0;
 	}
-	if (write_batch || read_batch) {
-		if (dry_run) {
-			snprintf(err_buf, sizeof err_buf,
-				"--%s-batch cannot be used with --dry_run (-n)\n",
-				write_batch ? "write" : "read");
-			return 0;
-		}
+	if (write_batch > 0 || read_batch) {
 		if (am_server) {
 			rprintf(FINFO,
 				"ignoring --%s-batch option sent to server\n",
@@ -908,7 +924,8 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 			 * batch args to server. */
 			read_batch = write_batch = 0;
 			batch_name = NULL;
-		}
+		} else if (dry_run)
+			write_batch = 0;
 	}
 	if (read_batch && files_from) {
 		snprintf(err_buf, sizeof err_buf,
@@ -928,9 +945,9 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 		return 0;
 	}
 
-	if (compare_dest + link_dest > 1) {
+	if (compare_dest + copy_dest + link_dest > 1) {
 		snprintf(err_buf, sizeof err_buf,
-			"You may not mix --compare-dest and --link-dest.\n");
+			"You may not mix --compare-dest, --copy-dest, and --link-dest.\n");
 		return 0;
 	}
 
@@ -940,7 +957,7 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 			return 0;
 		}
 		if (!files_from)
-			recurse = -1; /* infinite recursion */
+			recurse = 1;
 #ifdef SUPPORT_LINKS
 		preserve_links = 1;
 #endif
@@ -1074,10 +1091,10 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 		omit_dir_times = 1;
 
 	if (log_format) {
-		if (strstr(log_format, "%i") != NULL)
+		if (log_format_has(log_format, 'i'))
 			log_format_has_i = 1;
-		if (strstr(log_format, "%b") == NULL
-		 && strstr(log_format, "%c") == NULL)
+		if (!log_format_has(log_format, 'b')
+		 && !log_format_has(log_format, 'c'))
 			log_before_transfer = !am_server;
 	} else if (itemize_changes) {
 		log_format = "%i %n%L";
@@ -1089,12 +1106,16 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
 	    && !am_server)
 		verbose = 1;
 
+	if (dry_run)
+		do_xfers = 0;
+
+	set_io_timeout(io_timeout);
+
 	if (verbose && !log_format) {
 		log_format = "%n%L";
 		log_before_transfer = !am_server;
 	}
-	if (log_format_has_i
-	    || (log_format && strstr(log_format, "%o") != NULL))
+	if (log_format_has_i || log_format_has(log_format, 'o'))
 		log_format_has_o_or_i = 1;
 
 	if (daemon_bwlimit && (!bwlimit || bwlimit > daemon_bwlimit))
@@ -1211,7 +1232,7 @@ int parse_arguments(int *argc, const char ***argv, int frommain)
  **/
 void server_options(char **args,int *argc)
 {
-	static char argstr[50+MAX_BASIS_DIRS*2];
+	static char argstr[64];
 	int ac = *argc;
 	char *arg;
 
@@ -1242,7 +1263,7 @@ void server_options(char **args,int *argc)
 		argstr[x++] = 'b';
 	if (update_only)
 		argstr[x++] = 'u';
-	if (dry_run)
+	if (!do_xfers) /* NOT "dry_run"! */
 		argstr[x++] = 'n';
 	if (preserve_links)
 		argstr[x++] = 'l';
@@ -1273,7 +1294,7 @@ void server_options(char **args,int *argc)
 		argstr[x++] = 'O';
 	if (preserve_perms)
 		argstr[x++] = 'p';
-	if (recurse < 0)
+	if (recurse)
 		argstr[x++] = 'r';
 	if (always_checksum)
 		argstr[x++] = 'c';
@@ -1293,7 +1314,7 @@ void server_options(char **args,int *argc)
 	/* This is a complete hack - blame Rusty.  FIXME!
 	 * This hack is only needed for older rsync versions that
 	 * don't understand the --list-only option. */
-	if (list_only == 1 && recurse >= 0)
+	if (list_only == 1 && !recurse)
 		argstr[x++] = 'r';
 
 	argstr[x] = 0;
@@ -1370,6 +1391,8 @@ void server_options(char **args,int *argc)
 			args[ac++] = "--delete-after";
 		if (force_delete)
 			args[ac++] = "--force";
+		if (write_batch < 0)
+			args[ac++] = "--only-write-batch=X";
 	}
 
 	if (size_only)

packaging/lsb/rsync.spec

@@ -1,24 +1,46 @@
-Summary: Program for efficient remote updates of files.
+Summary: A program for synchronizing files over a network.
 Name: rsync
-Version: 2.6.4pre2
+Version: 2.6.5
 Release: 1
-Copyright: GPL
-Group: Applications/Networking
-Source:	ftp://samba.anu.edu.au/pub/rsync/rsync-%{version}.tar.gz
-URL: http://samba.anu.edu.au/rsync/
-Packager: Andrew Tridgell <tridge@samba.anu.edu.au>
-BuildRoot: /tmp/rsync
+Group: Applications/Internet
+Source:	ftp://rsync.samba.org/pub/rsync/rsync-%{version}.tar.gz
+URL: http://rsync.samba.org/
+
+Prefix: %{_prefix}
+BuildRoot: /var/tmp/%{name}-root
+License: GPL
 
 %description
-rsync is a replacement for rcp that has many more features.
+Rsync uses a reliable algorithm to bring remote and host files into
+sync very quickly. Rsync is fast because it just sends the differences
+in the files over the network instead of sending the complete
+files. Rsync is often used as a very powerful mirroring process or
+just as a more capable replacement for the rcp command. A technical
+report which describes the rsync algorithm is included in this
+package.
 
-rsync uses the "rsync algorithm" which provides a very fast method for
-bringing remote files into sync. It does this by sending just the
-differences in the files across the link, without requiring that both
-sets of files are present at one of the ends of the link beforehand.
+%prep
+%setup -q
 
-A technical report describing the rsync algorithm is included with
-this package. 
+%build
+%configure
+
+make
+
+%install
+rm -rf $RPM_BUILD_ROOT
+
+%makeinstall
+
+%clean
+rm -rf $RPM_BUILD_ROOT
+
+%files
+%defattr(-,root,root)
+%doc COPYING README tech_report.tex
+%{_prefix}/bin/rsync
+%{_mandir}/man1/rsync.1*
+%{_mandir}/man5/rsyncd.conf.5*
 
 %changelog
 * Thu Jan 30 2003 Horst von Brand <vonbrand@inf.utfsm.cl>
@@ -64,30 +86,3 @@ to '%build', removed '%prefix'.
 
 rsync-1.6.2-1 packaged.  (This entry by jam to credit Michael for the
 previous package(s).)
-
-%prep
-%setup
-
-%build
-./configure --prefix=/usr --mandir=%{_mandir}
-make CFLAGS="$RPM_OPT_FLAGS"
-strip rsync
-
-%install
-mkdir -p $RPM_BUILD_ROOT/usr/bin
-mkdir -p $RPM_BUILD_ROOT/%{_mandir}/man{1,5}
-install -m755 rsync $RPM_BUILD_ROOT/usr/bin
-install -m644 rsync.1 $RPM_BUILD_ROOT/%{_mandir}/man1
-install -m644 rsyncd.conf.5 $RPM_BUILD_ROOT/%{_mandir}/man5
-
-%clean
-rm -rf $RPM_BUILD_ROOT
-
-%files
-%attr(-,root,root) /usr/bin/rsync
-%attr(-,root,root) %{_mandir}/man1/rsync.1*
-%attr(-,root,root) %{_mandir}/man5/rsyncd.conf.5*
-%attr(-,root,root) %doc tech_report.tex
-%attr(-,root,root) %doc README
-%attr(-,root,root) %doc COPYING
-%attr(-,root,root) %doc doc/README-SGML doc/rsync.sgml

pipe.c

@@ -25,7 +25,6 @@ extern int am_sender;
 extern int am_server;
 extern int blocking_io;
 extern int orig_umask;
-extern int write_batch;
 extern int filesfrom_fd;
 
 /**
@@ -128,10 +127,6 @@ pid_t local_child(int argc, char **argv, int *f_in, int *f_out,
 		am_sender = !am_sender;
 		am_server = 1;
 
-		/* The server side never writes the batch, even if it
-		 * is local (it makes the logic easier elsewhere). */
-		write_batch = 0;
-
 		if (!am_sender)
 			filesfrom_fd = -1;
 

progress.c

@@ -67,17 +67,18 @@ static void rprint_progress(OFF_T ofs, OFF_T size, struct timeval *now,
 
 	if (is_last) {
 		/* Compute stats based on the starting info. */
-		diff = msdiff(&ph_start.time, now);
-		if (!diff)
+		if (!ph_start.time.tv_sec
+		    || !(diff = msdiff(&ph_start.time, now)))
 			diff = 1;
 		rate = (double) (ofs - ph_start.ofs) * 1000.0 / diff / 1024.0;
 		/* Switch to total time taken for our last update. */
 		remain = (double) diff / 1000.0;
 	} else {
 		/* Compute stats based on recent progress. */
-		diff = msdiff(&ph_list[oldest_hpos].time, now);
-		rate = diff ? (double) (ofs - ph_list[oldest_hpos].ofs) * 1000.0
-		    / diff / 1024.0 : 0;
+		if (!(diff = msdiff(&ph_list[oldest_hpos].time, now)))
+			diff = 1;
+		rate = (double) (ofs - ph_list[oldest_hpos].ofs) * 1000.0
+		     / diff / 1024.0;
 		remain = rate ? (double) (size - ofs) / rate / 1000.0 : 0.0;
 	}
 

receiver.c

@@ -21,7 +21,7 @@
 #include "rsync.h"
 
 extern int verbose;
-extern int dry_run;
+extern int do_xfers;
 extern int am_daemon;
 extern int am_server;
 extern int do_progress;
@@ -30,6 +30,7 @@ extern int log_format_has_i;
 extern int daemon_log_format_has_i;
 extern int csum_length;
 extern int read_batch;
+extern int write_batch;
 extern int batch_gen_fd;
 extern int protocol_version;
 extern int relative_paths;
@@ -53,7 +54,7 @@ extern char *log_format;
 extern char *tmpdir;
 extern char *partial_dir;
 extern char *basis_dir[];
-
+extern struct file_list *the_file_list;
 extern struct filter_list_struct server_filter_list;
 
 #define SLOT_SIZE	(16*1024)	/* Desired size in bytes */
@@ -62,6 +63,7 @@ extern struct filter_list_struct server_filter_list;
 
 static uint32 **delayed_bits = NULL;
 static int delayed_slot_cnt = 0;
+static int phase = 0;
 
 static void init_delayed_bits(int max_ndx)
 {
@@ -266,7 +268,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 				continue;
 			}
 		}
-		if (fd != -1 && write_file(fd, map, len) != (int)len)
+		if (fd != -1 && map && write_file(fd, map, len) != (int)len)
 			goto report_write_error;
 		offset += len;
 	}
@@ -303,64 +305,86 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r,
 }
 
 
-static void read_gen_name(int fd, char *dirname, char *buf)
-{
-	int dlen;
-	int len = read_byte(fd);
-
-	if (len & 0x80) {
-#if MAXPATHLEN > 32767
-		uchar lenbuf[2];
-		read_buf(fd, (char *)lenbuf, 2);
-		len = (len & ~0x80) * 0x10000 + lenbuf[0] * 0x100 + lenbuf[1];
-#else
-		len = (len & ~0x80) * 0x100 + read_byte(fd);
-#endif
-	}
-
-	if (dirname) {
-		dlen = strlcpy(buf, dirname, MAXPATHLEN);
-		buf[dlen++] = '/';
-	} else
-		dlen = 0;
-
-	if (dlen + len >= MAXPATHLEN) {
-		rprintf(FERROR, "bogus data on generator name pipe\n");
-		exit_cleanup(RERR_PROTOCOL);
-	}
-
-	read_sbuf(fd, buf + dlen, len);
-}
-
-
 static void discard_receive_data(int f_in, OFF_T length)
 {
 	receive_data(f_in, NULL, -1, 0, NULL, -1, length);
 }
 
+static void handle_delayed_updates(struct file_list *flist, char *local_name)
+{
+	char *fname, *partialptr, numbuf[4];
+	int i;
+
+	for (i = -1; (i = next_delayed_bit(i)) >= 0; ) {
+		struct file_struct *file = flist->files[i];
+		fname = local_name ? local_name : f_name(file);
+		if ((partialptr = partial_dir_fname(fname)) != NULL) {
+			if (make_backups && !make_backup(fname))
+				continue;
+			if (verbose > 2) {
+				rprintf(FINFO, "renaming %s to %s\n",
+					safe_fname(partialptr),
+					safe_fname(fname));
+			}
+			if (do_rename(partialptr, fname) < 0) {
+				rsyserr(FERROR, errno,
+					"rename failed for %s (from %s)",
+					full_fname(fname),
+					safe_fname(partialptr));
+			} else {
+				if (remove_sent_files
+				    || (preserve_hard_links
+				     && file->link_u.links)) {
+					SIVAL(numbuf, 0, i);
+					send_msg(MSG_SUCCESS,numbuf,4);
+				}
+				handle_partial_dir(partialptr,
+						   PDIR_DELETE);
+			}
+		}
+	}
+}
+
+static int get_next_gen_i(int batch_gen_fd, int next_gen_i, int desired_i)
+{
+	while (next_gen_i < desired_i) {
+		if (next_gen_i >= 0) {
+			rprintf(FINFO,
+				"(No batched update for%s \"%s\")\n",
+				phase ? " resend of" : "",
+				safe_fname(f_name(the_file_list->files[next_gen_i])));
+		}
+		next_gen_i = read_int(batch_gen_fd);
+		if (next_gen_i == -1)
+			next_gen_i = the_file_list->count;
+	}
+	return next_gen_i;
+}
+
 
 /**
  * main routine for receiver process.
  *
  * Receiver process runs on the same host as the generator process. */
-int recv_files(int f_in, struct file_list *flist, char *local_name,
-	       int f_in_name)
+int recv_files(int f_in, struct file_list *flist, char *local_name)
 {
 	int next_gen_i = -1;
 	int fd1,fd2;
 	STRUCT_STAT st;
-	int iflags;
+	int iflags, xlen;
 	char *fname, fbuf[MAXPATHLEN];
-	char template[MAXPATHLEN];
+	char xname[MAXPATHLEN];
 	char fnametmp[MAXPATHLEN];
 	char *fnamecmp, *partialptr, numbuf[4];
 	char fnamecmpbuf[MAXPATHLEN];
+	uchar fnamecmp_type;
 	struct file_struct *file;
 	struct stats initial_stats;
 	int save_make_backups = make_backups;
 	int itemizing = am_daemon ? daemon_log_format_has_i
 		      : !am_server && log_format_has_i;
-	int i, recv_ok, phase = 0;
+	int max_phase = protocol_version >= 29 ? 2 : 1;
+	int i, recv_ok;
 
 	if (verbose > 2)
 		rprintf(FINFO,"recv_files(%d) starting\n",flist->count);
@@ -379,38 +403,27 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 		i = read_int(f_in);
 		if (i == -1) {
 			if (read_batch) {
-				if (next_gen_i != flist->count) {
-					do {
-						if (f_in_name >= 0
-						    && next_gen_i >= 0)
-							read_byte(f_in_name);
-					} while (read_int(batch_gen_fd) != -1);
-				}
+				get_next_gen_i(batch_gen_fd, next_gen_i,
+					       flist->count);
 				next_gen_i = -1;
 			}
-
-			if (phase)
+			if (++phase > max_phase)
 				break;
-
-			phase = 1;
 			csum_length = SUM_LENGTH;
 			if (verbose > 2)
 				rprintf(FINFO, "recv_files phase=%d\n", phase);
+			if (phase == 2 && delay_updates)
+				handle_delayed_updates(flist, local_name);
 			send_msg(MSG_DONE, "", 0);
 			if (keep_partial && !partial_dir)
 				make_backups = 0; /* prevents double backup */
 			continue;
 		}
 
-		if (i < 0 || i >= flist->count) {
-			/* Handle the new keep-alive (no-op) packet. */
-			if (i == flist->count && protocol_version >= 29
-	 		    && read_shortint(f_in) == ITEM_IS_NEW)
-				continue;
-			rprintf(FERROR,"Invalid file index %d in recv_files (count=%d)\n",
-				i, flist->count);
-			exit_cleanup(RERR_PROTOCOL);
-		}
+		iflags = read_item_attrs(f_in, -1, i, &fnamecmp_type,
+					 xname, &xlen);
+		if (iflags == ITEM_IS_NEW) /* no-op packet */
+			continue;
 
 		file = flist->files[i];
 		fname = local_name ? local_name : f_name_to(file, fbuf);
@@ -418,25 +431,14 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 		if (verbose > 2)
 			rprintf(FINFO, "recv_files(%s)\n", safe_fname(fname));
 
-		if (protocol_version >= 29) {
-			iflags = read_shortint(f_in);
-			if (!(iflags & ITEM_UPDATING) || !S_ISREG(file->mode)) {
-				int see_item = itemizing && (iflags || verbose > 1);
-				if (am_server) {
-					if (am_daemon && !dry_run && see_item)
-						log_recv(file, &stats, iflags);
-				} else if (see_item || iflags & ITEM_UPDATING
-				    || (S_ISDIR(file->mode)
-				     && iflags & ITEM_REPORT_TIME))
-					log_recv(file, &stats, iflags);
-				continue;
-			}
-		} else
-			iflags = ITEM_UPDATING | ITEM_MISSING_DATA;
-
-		if (!S_ISREG(file->mode)) {
-			rprintf(FERROR, "[%s] got index of non-regular file: %d\n",
-				who_am_i(), i);
+		if (!(iflags & ITEM_TRANSFER)) {
+			maybe_log_item(file, iflags, itemizing, xname);
+			continue;
+		}
+		if (phase == 2) {
+			rprintf(FERROR,
+				"got transfer request in phase 2 [%s]\n",
+				who_am_i());
 			exit_cleanup(RERR_PROTOCOL);
 		}
 
@@ -451,22 +453,24 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 			exit_cleanup(RERR_PROTOCOL);
 		}
 
-		if (dry_run) { /* log the transfer */
+		if (!do_xfers) { /* log the transfer */
 			if (!am_server && log_format)
-				log_recv(file, &stats, iflags);
+				log_item(file, &stats, iflags, NULL);
+			if (read_batch)
+				discard_receive_data(f_in, file->length);
+			continue;
+		}
+		if (write_batch < 0) {
+			log_item(file, &stats, iflags, NULL);
+			if (!am_server)
+				discard_receive_data(f_in, file->length);
 			continue;
 		}
 
 		if (read_batch) {
-			while (i > next_gen_i) {
-				if (f_in_name >= 0 && next_gen_i >= 0)
-					read_byte(f_in_name);
-				next_gen_i = read_int(batch_gen_fd);
-				if (next_gen_i == -1)
-					next_gen_i = flist->count;
-			}
+			next_gen_i = get_next_gen_i(batch_gen_fd, next_gen_i, i);
 			if (i < next_gen_i) {
-				rprintf(FINFO, "skipping update for \"%s\"\n",
+				rprintf(FINFO, "(Skipping batched update for \"%s\")\n",
 					safe_fname(fname));
 				discard_receive_data(f_in, file->length);
 				continue;
@@ -476,42 +480,72 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 
 		partialptr = partial_dir ? partial_dir_fname(fname) : fname;
 
-		if (f_in_name >= 0) {
-			uchar j;
-			switch (j = read_byte(f_in_name)) {
+		if (protocol_version >= 29) {
+			switch (fnamecmp_type) {
 			case FNAMECMP_FNAME:
 				fnamecmp = fname;
 				break;
 			case FNAMECMP_PARTIAL_DIR:
-				fnamecmp = partialptr ? partialptr : fname;
+				fnamecmp = partialptr;
 				break;
 			case FNAMECMP_BACKUP:
 				fnamecmp = get_backup_name(fname);
 				break;
 			case FNAMECMP_FUZZY:
-				read_gen_name(f_in_name, file->dirname, fnamecmpbuf);
-				fnamecmp = fnamecmpbuf;
+				if (file->dirname) {
+					pathjoin(fnamecmpbuf, MAXPATHLEN,
+						 file->dirname, xname);
+					fnamecmp = fnamecmpbuf;
+				} else
+					fnamecmp = xname;
 				break;
 			default:
-				if (j >= basis_dir_cnt) {
+				if (fnamecmp_type >= basis_dir_cnt) {
 					rprintf(FERROR,
 						"invalid basis_dir index: %d.\n",
-						j);
+						fnamecmp_type);
 					exit_cleanup(RERR_PROTOCOL);
 				}
 				pathjoin(fnamecmpbuf, sizeof fnamecmpbuf,
-					 basis_dir[j], fname);
+					 basis_dir[fnamecmp_type], fname);
 				fnamecmp = fnamecmpbuf;
 				break;
 			}
-		} else
-			fnamecmp = fname;
+			if (!fnamecmp || (server_filter_list.head
+			  && check_filter(&server_filter_list, fname, 0) < 0))
+				fnamecmp = fname;
+		} else {
+			/* Reminder: --inplace && --partial-dir are never
+			 * enabled at the same time. */
+			if (inplace && make_backups) {
+				if (!(fnamecmp = get_backup_name(fname)))
+					fnamecmp = fname;
+			} else if (partial_dir && partialptr)
+				fnamecmp = partialptr;
+			else
+				fnamecmp = fname;
+		}
 
 		initial_stats = stats;
 
 		/* open the file */
 		fd1 = do_open(fnamecmp, O_RDONLY, 0);
 
+		if (fd1 == -1 && protocol_version < 29) {
+			if (fnamecmp != fname) {
+				fnamecmp = fname;
+				fd1 = do_open(fnamecmp, O_RDONLY, 0);
+			}
+
+			if (fd1 == -1 && basis_dir[0]) {
+				/* pre-29 allowed only one alternate basis */
+				pathjoin(fnamecmpbuf, sizeof fnamecmpbuf,
+					 basis_dir[0], fname);
+				fnamecmp = fnamecmpbuf;
+				fd1 = do_open(fnamecmp, O_RDONLY, 0);
+			}
+		}
+
 		if (fd1 != -1 && do_fstat(fd1,&st) != 0) {
 			rsyserr(FERROR, errno, "fstat %s failed",
 				full_fname(fnamecmp));
@@ -564,8 +598,6 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 				continue;
 			}
 
-			strlcpy(template, fnametmp, sizeof template);
-
 			/* we initially set the perms without the
 			 * setuid/setgid bits to ensure that there is no race
 			 * condition. They are then correctly updated after
@@ -579,7 +611,8 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 			 * transferred, but that may not be the case with -R */
 			if (fd2 == -1 && relative_paths && errno == ENOENT
 			    && create_directory_path(fnametmp, orig_umask) == 0) {
-				strlcpy(fnametmp, template, sizeof fnametmp);
+				/* Get back to name with XXXXXX in it. */
+				get_tmpname(fnametmp, fname);
 				fd2 = do_mkstemp(fnametmp, file->mode & INITACCESSPERMS);
 			}
 			if (fd2 == -1) {
@@ -597,7 +630,7 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 
 		/* log the transfer */
 		if (log_before_transfer)
-			log_recv(file, &initial_stats, iflags);
+			log_item(file, &initial_stats, iflags, NULL);
 		else if (!am_server && verbose && do_progress)
 			rprintf(FINFO, "%s\n", safe_fname(fname));
 
@@ -606,7 +639,7 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 				       fname, fd2, file->length);
 
 		if (!log_before_transfer)
-			log_recv(file, &initial_stats, iflags);
+			log_item(file, &initial_stats, iflags, NULL);
 
 		if (fd1 != -1)
 			close(fd1);
@@ -637,14 +670,14 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 
 		cleanup_disable();
 
-		if (recv_ok) {
-			if (remove_sent_files && recv_ok > 0) {
+		if (recv_ok > 0) {
+			if (remove_sent_files
+			    || (preserve_hard_links && file->link_u.links)) {
 				SIVAL(numbuf, 0, i);
 				send_msg(MSG_SUCCESS, numbuf, 4);
 			}
-		} else {
-			int msgtype = csum_length == SUM_LENGTH || read_batch ?
-				FERROR : FINFO;
+		} else if (!recv_ok) {
+			int msgtype = phase || read_batch ? FERROR : FINFO;
 			if (msgtype == FERROR || verbose) {
 				char *errstr, *redostr, *keptstr;
 				if (!(keep_partial && partialptr) && !inplace)
@@ -665,7 +698,7 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 					errstr, safe_fname(fname),
 					keptstr, redostr);
 			}
-			if (csum_length != SUM_LENGTH) {
+			if (!phase) {
 				SIVAL(numbuf, 0, i);
 				send_msg(MSG_REDO, numbuf, 4);
 			}
@@ -673,35 +706,8 @@ int recv_files(int f_in, struct file_list *flist, char *local_name,
 	}
 	make_backups = save_make_backups;
 
-	if (delay_updates) {
-		for (i = -1; (i = next_delayed_bit(i)) >= 0; ) {
-			struct file_struct *file = flist->files[i];
-			fname = local_name ? local_name : f_name(file);
-			partialptr = partial_dir_fname(fname);
-			if (partialptr) {
-				if (make_backups && !make_backup(fname))
-					continue;
-				if (verbose > 2) {
-					rprintf(FINFO, "renaming %s to %s\n",
-						safe_fname(partialptr),
-						safe_fname(fname));
-				}
-				if (do_rename(partialptr, fname) < 0) {
-					rsyserr(FERROR, errno,
-						"rename failed for %s (from %s)",
-						full_fname(fname),
-						safe_fname(partialptr));
-				} else {
-					if (remove_sent_files) {
-						SIVAL(numbuf, 0, i);
-						send_msg(MSG_SUCCESS,numbuf,4);
-					}
-					handle_partial_dir(partialptr,
-							   PDIR_DELETE);
-				}
-			}
-		}
-	}
+	if (phase == 2 && delay_updates) /* for protocol_version < 29 */
+		handle_delayed_updates(flist, local_name);
 
 	if (verbose > 2)
 		rprintf(FINFO,"recv_files finished\n");

rsync.c

@@ -35,7 +35,6 @@ extern int am_starting_up;
 extern int preserve_uid;
 extern int preserve_gid;
 extern int inplace;
-extern int recurse;
 extern int keep_dirlinks;
 extern int make_backups;
 extern struct stats stats;

rsync.h

@@ -58,11 +58,12 @@
 /* These flags are used in the live flist data. */
 
 #define FLAG_TOP_DIR (1<<0)
-#define FLAG_HLINK_EOL (1<<1)	/* generator only */
-#define FLAG_MOUNT_POINT (1<<2)	/* sender only */
-#define FLAG_NO_FUZZY (1<<2)	/* generator only */
+#define FLAG_HLINK_EOL (1<<1)	/* receiver/generator */
+#define FLAG_MOUNT_POINT (1<<2)	/* sender */
+#define FLAG_NO_FUZZY (1<<2)	/* generator */
 #define FLAG_DEL_HERE (1<<3)	/* receiver/generator */
-#define FLAG_SENT (1<<3)	/* sender only */
+#define FLAG_SENT (1<<3)	/* sender */
+#define FLAG_HLINK_TOL (1<<4)	/* receiver/generator */
 
 /* update this if you make incompatible changes */
 #define PROTOCOL_VERSION 29
@@ -99,10 +100,11 @@
 
 #define IOERR_GENERAL	(1<<0) /* For backward compatibility, this must == 1 */
 #define IOERR_VANISHED	(1<<1)
+#define IOERR_DEL_LIMIT (1<<2)
 
 #define MAX_ARGS 1000
-
 #define MAX_BASIS_DIRS 20
+#define MAX_SERVER_ARGS (MAX_BASIS_DIRS*2 + 100)
 
 #define MPLEX_BASE 7
 
@@ -137,18 +139,26 @@
 #define DEL_TERSE		(1<<3)
 
 /* For use by the itemize_changes code */
-#define ITEM_UPDATING (1<<0)
 #define ITEM_REPORT_CHECKSUM (1<<1)
 #define ITEM_REPORT_SIZE (1<<2)
 #define ITEM_REPORT_TIME (1<<3)
 #define ITEM_REPORT_PERMS (1<<4)
 #define ITEM_REPORT_OWNER (1<<5)
 #define ITEM_REPORT_GROUP (1<<6)
-#define ITEM_IS_NEW (1<<7)
-#define ITEM_USING_ALT_BASIS (1<<8)
+#define ITEM_REPORT_XATTRS (1<<7)
+#define ITEM_BASIS_TYPE_FOLLOWS (1<<11)
+#define ITEM_XNAME_FOLLOWS (1<<12)
+#define ITEM_IS_NEW (1<<13)
+#define ITEM_LOCAL_CHANGE (1<<14)
+#define ITEM_TRANSFER (1<<15)
+/* These are outside the range of the transmitted flags. */
+#define ITEM_NO_DEST_AND_NO_UPDATE (1<<16) /* used by itemize() */
+#define ITEM_MISSING_DATA (1<<16)	   /* used by log_formatted() */
+#define ITEM_DELETED (1<<17)		   /* used by log_formatted() */
+
+#define SIGNIFICANT_ITEM_FLAGS (~(\
+	ITEM_BASIS_TYPE_FOLLOWS | ITEM_XNAME_FOLLOWS | ITEM_LOCAL_CHANGE))
 
-#define ITEM_MISSING_DATA (1<<16) /* these are outside the transmitted flags */
-#define ITEM_DELETED (1<<17)
 
 /* Log-message categories.  FLOG and FCLIENT are only used on the daemon
  * side for custom logging -- they don't get sent over the socket. */
@@ -471,7 +481,7 @@ struct idev {
 #define HL_SKIP		1
 
 struct hlink {
-	struct file_struct *next;
+	int next;
 	int hlindex;
 };
 
@@ -664,8 +674,12 @@ extern char *sys_errlist[];
 extern int errno;
 #endif
 
-#define SUPPORT_LINKS HAVE_READLINK
-#define SUPPORT_HARD_LINKS HAVE_LINK
+#ifdef HAVE_READLINK
+#define SUPPORT_LINKS 1
+#endif
+#ifdef HAVE_LINK
+#define SUPPORT_HARD_LINKS 1
+#endif
 
 #define SIGNAL_CAST (RETSIGTYPE (*)())
 

rsync.yo

@@ -1,11 +1,11 @@
 mailto(rsync-bugs@samba.org)
-manpage(rsync)(1)(28 Feb 2005)()()
+manpage(rsync)(1)(1 Jun 2005)()()
 manpagename(rsync)(faster, flexible replacement for rcp)
 manpagesynopsis()
 
 rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
 
-rsync [OPTION]... [USER@]HOST:SRC DEST
+rsync [OPTION]... [USER@]HOST:SRC [DEST]
 
 rsync [OPTION]... SRC [SRC]... DEST
 
@@ -38,47 +38,39 @@ itemize(
   it() can use any transparent remote shell, including ssh or rsh
   it() does not require root privileges
   it() pipelining of file transfers to minimize latency costs
-  it() support for anonymous or authenticated rsync servers (ideal for
+  it() support for anonymous or authenticated rsync daemons (ideal for
        mirroring)
 )
 
 manpagesection(GENERAL)
 
-There are eight different ways of using rsync. They are:
+Rsync copies files either to or from a remote host, or locally on the
+current host (it does not support copying files between two remote hosts).
 
-itemize(
-	it() for copying local files. This is invoked when neither
-	     source nor destination path contains a : separator
-	it() for copying from the local machine to a remote machine using
-	a remote shell program as the transport (such as ssh or
-	rsh). This is invoked when the destination path contains a
-	single : separator.
-	it() for copying from a remote machine to the local machine
-	using a remote shell program. This is invoked when the source
-	contains a : separator.
-	it() for copying from a remote rsync server to the local
-	machine. This is invoked when the source path contains a ::
-	separator or an rsync:// URL.
-	it() for copying from the local machine to a remote rsync
-	server. This is invoked when the destination path contains a ::
-	separator or an rsync:// URL.
-	it() for copying from a remote machine using a remote shell
-	program as the transport, using rsync server on the remote
-	machine.  This is invoked when the source path contains a ::
-	separator and the bf(--rsh=COMMAND) (aka "bf(-e COMMAND)") option is
-	also provided.
-	it() for copying from the local machine to a remote machine
-	using a remote shell program as the transport, using rsync
-	server on the remote machine.  This is invoked when the
-	destination path contains a :: separator and the
-	bf(--rsh=COMMAND) option is also provided.
-	it() for listing files on a remote machine. This is done the
-	same way as rsync transfers except that you leave off the
-	local destination.
-)
+There are two different ways for rsync to contact a remote system: using a
+remote-shell program as the transport (such as ssh or rsh) or contacting an
+rsync daemon directly via TCP.  The remote-shell transport is used whenever
+the source or destination path contains a single colon (:) separator after
+a host specification.  Contacting an rsync daemon directly happens when the
+source or destination path contains a double colon (::) separator after a
+host specification, OR when an rsync:// URL is specified.
 
-Note that in all cases (other than listing) at least one of the source
-and destination paths must be local.
+As a special case, if a remote source is specified without a destination,
+the remote files are listed in an output format similar to "ls -l".
+
+As expected, if neither the source or destination path specify a remote
+host, the copy occurs locally (see also the bf(--list-only) option).
+
+Finally, it is possible to use a remote-shell transport to contact a remote
+host and then to spawn a single-use rsync daemon.  This allows the use of
+some of the daemon features (such as named modules) without having to run a
+daemon as a service.  To achieve this, invoke rsync with an explicit
+bf(--rsh=COMMAND) (aka "bf(-e COMMAND)") option combined with either the
+source or destination path specified as an rsync daemon (i.e. either a ::
+separator or an rsync:// URL).  In this case, rsync contacts the remote
+host specified using the specified remote shell, and then starts a
+single-use rsync daemon to deal with that copy request.  See the section
+"CONNECTING TO AN RSYNC DAEMON OVER A REMOTE SHELL PROGRAM" below.
 
 manpagesection(SETUP)
 
@@ -139,6 +131,15 @@ tt(rsync -av /src/foo /dest)nl()
 tt(rsync -av /src/foo/ /dest/foo)nl()
 )
 
+Note also that host and module references don't require a trailing slash to
+copy the contents of the default directory.  For example, both of these
+copy the remote directory's contents into "/dest":
+
+quote(
+tt(rsync -av host: /dest)nl()
+tt(rsync -av host::module /dest)nl()
+)
+
 You can also use rsync in local-only mode, where both the source and
 destination don't have a ':' in the name. In this case it behaves like
 an improved copy command.
@@ -178,10 +179,10 @@ tt(rsync -av host:file?name?with?spaces /dest)nl()
 This latter example assumes that your shell passes through unmatched
 wildcards.  If it complains about "no match", put the name in quotes.
 
-manpagesection(CONNECTING TO AN RSYNC SERVER)
+manpagesection(CONNECTING TO AN RSYNC DAEMON)
 
 It is also possible to use rsync without a remote shell as the
-transport. In this case you will connect to a remote rsync server
+transport. In this case you will connect to a remote rsync daemon
 running on TCP port 873.
 
 You may establish the connection via a web proxy by setting the
@@ -195,15 +196,15 @@ that:
 itemize(
 	it() you either use a double colon :: instead of a single colon to
 	separate the hostname from the path, or you use an rsync:// URL.
-	it() the remote server may print a message of the day when you
+	it() the remote daemon may print a message of the day when you
 	connect.
-	it() if you specify no path name on the remote server then the
-	list of accessible paths on the server will be shown.
+	it() if you specify no path name on the remote daemon then the
+	list of accessible paths on the daemon will be shown.
 	it() if you specify no local destination then a listing of the
-	specified files on the remote server is provided.
+	specified files on the remote daemon is provided.
 )
 
-Some paths on the remote server may require authentication. If so then
+Some paths on the remote daemon may require authentication. If so then
 you will receive a password prompt when you connect. You can avoid the
 password prompt by setting the environment variable RSYNC_PASSWORD to
 the password you want to use or using the bf(--password-file) option. This
@@ -212,24 +213,24 @@ may be useful when scripting rsync.
 WARNING: On some systems environment variables are visible to all
 users. On those systems using bf(--password-file) is recommended.
 
-manpagesection(CONNECTING TO AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM)
+manpagesection(CONNECTING TO AN RSYNC DAEMON OVER A REMOTE SHELL PROGRAM)
 
 It is sometimes useful to be able to set up file transfers using rsync
-server capabilities on the remote machine, while still using ssh or
+daemon capabilities on the remote machine, while still using ssh or
 rsh for transport.  This is especially useful when you want to connect
 to a remote machine via ssh (for encryption or to get through a
-firewall), but you still want to have access to the rsync server
-features (see RUNNING AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM,
+firewall), but you still want to have access to the rsync daemon
+features (see RUNNING AN RSYNC DAEMON OVER A REMOTE SHELL PROGRAM,
 below).
 
 From the user's perspective, using rsync in this way is the same as
-using it to connect to an rsync server, except that you must
+using it to connect to an rsync daemon, except that you must
 explicitly set the remote shell program on the command line with
 bf(--rsh=COMMAND).  (Setting RSYNC_RSH in the environment will not turn on
 this functionality.)
 
 In order to distinguish between the remote-shell user and the rsync
-server user, you can use '-l user' on your remote-shell command:
+daemon user, you can use '-l user' on your remote-shell command:
 
 verb(    rsync -av --rsh="ssh -l ssh-user" \ 
         rsync-user@host::module[/path] local-path)
@@ -237,26 +238,26 @@ verb(    rsync -av --rsh="ssh -l ssh-user" \
 The "ssh-user" will be used at the ssh level; the "rsync-user" will be
 used to check against the rsyncd.conf on the remote host.
 
-manpagesection(RUNNING AN RSYNC SERVER)
+manpagesection(RUNNING AN RSYNC DAEMON)
 
-An rsync server is configured using a configuration file.  Please see the
+An rsync daemon is configured using a configuration file.  Please see the
 rsyncd.conf(5) man page for more information.  By default the configuration
 file is called /etc/rsyncd.conf, unless rsync is running over a remote
 shell program and is not running as root; in that case, the default name
 is rsyncd.conf in the current directory on the remote computer
 (typically $HOME).
 
-manpagesection(RUNNING AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM)
+manpagesection(RUNNING AN RSYNC DAEMON OVER A REMOTE SHELL PROGRAM)
 
 See the rsyncd.conf(5) man page for full information on the rsync
-server configuration file.
+daemon configuration file.
 
 Several configuration options will not be available unless the remote
 user is root (e.g. chroot, setuid/setgid, etc.).  There is no need to
-configure inetd or the services map to include the rsync server port
-if you run an rsync server only via a remote shell program.
+configure inetd or the services map to include the rsync daemon port
+if you run an rsync daemon only via a remote shell program.
 
-To run an rsync server out of a single-use ssh key, see this section
+To run an rsync daemon out of a single-use ssh key, see this section
 in the rsyncd.conf(5) man page.
 
 manpagesection(EXAMPLES)
@@ -297,7 +298,6 @@ Here is a short summary of the options available in rsync. Please refer
 to the detailed description below for a complete description.  verb(
  -v, --verbose               increase verbosity
  -q, --quiet                 suppress non-error messages
- -c, --checksum              always checksum
  -c, --checksum              skip based on checksum, not mod-time & size
  -a, --archive               archive mode; same as -rlptgoD (no -H)
  -r, --recursive             recurse into directories
@@ -329,7 +329,7 @@ to the detailed description below for a complete description.  verb(
  -x, --one-file-system       don't cross filesystem boundaries
  -B, --block-size=SIZE       force a fixed checksum block-size
  -e, --rsh=COMMAND           specify the remote shell to use
-     --rsync-path=PATH       specify path to rsync on the remote machine
+     --rsync-path=PROGRAM    specify the rsync to run on remote machine
      --existing              only update files that already exist
      --ignore-existing       ignore files that already exist on receiver
      --remove-sent-files     sent files/symlinks are removed from sender
@@ -354,6 +354,7 @@ to the detailed description below for a complete description.  verb(
  -T, --temp-dir=DIR          create temporary files in directory DIR
  -y, --fuzzy                 find similar file for basis if no dest file
      --compare-dest=DIR      also compare received files relative to DIR
+     --copy-dest=DIR         ... and include copies of unchanged files
      --link-dest=DIR         hardlink to files in DIR when unchanged
  -z, --compress              compress file data during the transfer
  -C, --cvs-exclude           auto-ignore files in the same way CVS does
@@ -365,8 +366,8 @@ to the detailed description below for a complete description.  verb(
      --include=PATTERN       don't exclude files matching PATTERN
      --include-from=FILE     read include patterns from FILE
      --files-from=FILE       read list of source-file names from FILE
- -0, --from0                 all *from file lists are delimited by nulls
-     --version               print version number
+ -0, --from0                 all *from/filter files are delimited by 0s
+     --address=ADDRESS       bind address for outgoing socket to daemon
      --port=PORT             specify double-colon alternate port number
      --blocking-io           use blocking I/O for the remote shell
      --no-blocking-io        turn off blocking I/O when it is default
@@ -374,15 +375,18 @@ to the detailed description below for a complete description.  verb(
      --progress              show progress during transfer
  -P                          same as --partial --progress
  -i, --itemize-changes       output a change-summary for all updates
-     --log-format=FORMAT     log file-transfers using specified format
+     --log-format=FORMAT     output filenames using the specified format
      --password-file=FILE    read password from FILE
      --list-only             list the files instead of copying them
      --bwlimit=KBPS          limit I/O bandwidth; KBytes per second
      --write-batch=FILE      write a batched update to FILE
+     --only-write-batch=FILE like --write-batch but w/o updating dest
      --read-batch=FILE       read a batched update from FILE
+     --protocol=NUM          force an older protocol version to be used
      --checksum-seed=NUM     set block/file checksum seed (advanced)
  -4, --ipv4                  prefer IPv4
  -6, --ipv6                  prefer IPv6
+     --version               print version number
  -h, --help                  show this help screen)
 
 Rsync can also be run as a daemon, in which case the following options are
@@ -422,7 +426,7 @@ you are debugging rsync.
 
 Note that the names of the transferred files that are output are done using
 a default bf(--log-format) of "%n%L", which tells you just the name of the
-file and, if the item is a symlink, where it points.  At the single bf(-v)
+file and, if the item is a link, where it points.  At the single bf(-v)
 level of verbosity, this does not mention when a file gets its attributes
 changed.  If you ask for an itemized list of changed attributes (either
 bf(--itemize-changes) or adding "%i" to the bf(--log-format) setting), the
@@ -619,9 +623,10 @@ default.
 dit(bf(-p, --perms)) This option causes rsync to set the destination
 permissions to be the same as the source permissions.
 
-Without this option, each new file gets its permissions set based on the
-source file's permissions and the umask at the receiving end, while all
-other files (including updated files) retain their existing permissions
+Without this option, all existing files (including updated files) retain
+their existing permissions, while each new file gets its permissions set
+based on the source file's permissions, but masked by the receiving end's
+umask setting
 (which is the same behavior as other file-copy utilities, such as cp).
 
 dit(bf(-o, --owner)) This option causes rsync to set the owner of the
@@ -750,8 +755,8 @@ is only relevant without bf(--delete) because deletions are now done depth-first
 Requires the bf(--recursive) option (which is implied by bf(-a)) to have any effect.
 
 dit(bf(--max-delete=NUM)) This tells rsync not to delete more than NUM
-files or directories. This is useful when mirroring very large trees
-to prevent disasters.
+files or directories (NUM must be non-zero).
+This is useful when mirroring very large trees to prevent disasters.
 
 dit(bf(--max-size=SIZE)) This tells rsync to avoid transferring any
 file that is larger than the specified SIZE. The SIZE value can be
@@ -768,11 +773,11 @@ remote copies of rsync. Typically, rsync is configured to use ssh by
 default, but you may prefer to use rsh on a local network.
 
 If this option is used with bf([user@]host::module/path), then the
-remote shell em(COMMAND) will be used to run an rsync server on the
+remote shell em(COMMAND) will be used to run an rsync daemon on the
 remote host, and all data will be transmitted through that remote
 shell connection, rather than through a direct socket connection to a
-running rsync server on the remote host.  See the section "CONNECTING
-TO AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM" above.
+running rsync daemon on the remote host.  See the section "CONNECTING
+TO AN RSYNC DAEMON OVER A REMOTE SHELL PROGRAM" above.
 
 Command-line arguments are permitted in COMMAND provided that COMMAND is
 presented to rsync as a single argument.  For example:
@@ -787,10 +792,18 @@ environment variable, which accepts the same range of values as bf(-e).
 
 See also the bf(--blocking-io) option which is affected by this option.
 
-dit(bf(--rsync-path=PATH)) Use this to specify the path to the copy of
-rsync on the remote machine. Useful when it's not in your path. Note
-that this is the full path to the binary, not just the directory that
-the binary is in.
+dit(bf(--rsync-path=PROGRAM)) Use this to specify what program is to be run
+on the remote machine to start-up rsync.  Often used when rsync is not in
+the default remote-shell's path (e.g. --rsync-path=/usr/local/bin/rsync).
+Note that PROGRAM is run with the help of a shell, so it can be any
+program, script, or command sequence you'd care to run, so long as it does
+not corrupt the standard-in & standard-out that rsync is using to
+communicate.
+
+One tricky example is to set a different default directory on the remote
+machine for use with the bf(--relative) option.  For instance:
+
+quote(tt(    rsync -avR --rsync-path="cd /a/b && rsync" hst:c/d /e/))
 
 dit(bf(-C, --cvs-exclude)) This is a useful shorthand for excluding a
 broad range of files that you often don't want to transfer between
@@ -814,7 +827,7 @@ See the bf(cvs(1)) manual for more information.
 
 If you're combining bf(-C) with your own bf(--filter) rules, you should
 note that these CVS excludes are appended at the end of your own rules,
-regardless of where the -C was placed on the command-line.  This makes them
+regardless of where the bf(-C) was placed on the command-line.  This makes them
 a lower priority than any rules you specified explicitly.  If you want to
 control where these CVS excludes get inserted into your filter rules, you
 should omit the bf(-C) as a command-line option and use a combination of
@@ -896,9 +909,14 @@ command:
 quote(tt(   rsync -a --files-from=/tmp/foo /usr remote:/backup))
 
 If /tmp/foo contains the string "bin" (or even "/bin"), the /usr/bin
-directory will be created as /backup/bin on the remote host (but the
-contents of the /usr/bin dir would not be sent unless you specified bf(-r)
-or the names were explicitly listed in /tmp/foo).  Also keep in mind
+directory will be created as /backup/bin on the remote host.  If it
+contains "bin/" (note the trailing slash), the immediate contents of
+the directory would also be sent (without needing to be explicitly
+mentioned in the file -- this began in version 2.6.4).  In both cases,
+if the bf(-r) option was enabled, that dir's entire hierarchy would
+also be transferred (keep in mind that bf(-r) needs to be specified
+explicitly with bf(--files-from), since it is not implied by bf(-a)).
+Also note
 that the effect of the (enabled by default) bf(--relative) option is to
 duplicate only the path info that is read from the file -- it does not
 force the duplication of the source-spec path (/usr in this case).
@@ -914,7 +932,7 @@ quote(tt(   rsync -a --files-from=:/path/file-list src:/ /tmp/copy))
 This would copy all the files specified in the /path/file-list file that
 was located on the remote "src" host.
 
-dit(bf(-0, --from0)) This tells rsync that the filenames it reads from a
+dit(bf(-0, --from0)) This tells rsync that the rules/filenames it reads from a
 file are terminated by a null ('\0') character, not a NL, CR, or CR+LF.
 This affects bf(--exclude-from), bf(--include-from), bf(--files-from), and any
 merged files specified in a bf(--filter) rule.
@@ -945,14 +963,32 @@ directory.  This is useful for creating a sparse backup of just files that
 have changed from an earlier backup.
 
 Beginning in version 2.6.4, multiple bf(--compare-dest) directories may be
-provided and rsync will search the list in the order specified until it
-finds an existing file.  That first discovery is used as the basis file,
-and also determines if the transfer needs to happen.
+provided, which will cause rsync to search the list in the order specified
+for an exact match.
+If a match is found that differs only in attributes, a local copy is made
+and the attributes updated.
+If a match is not found, a basis file from one of the em(DIR)s will be
+selected to try to speed up the transfer.
 
 If em(DIR) is a relative path, it is relative to the destination directory.
-See also bf(--link-dest).
+See also bf(--copy-dest) and bf(--link-dest).
 
-dit(bf(--link-dest=DIR)) This option behaves like bf(--compare-dest), but
+dit(bf(--copy-dest=DIR)) This option behaves like bf(--compare-dest), but
+rsync will also copy unchanged files found in em(DIR) to the destination
+directory using a local copy.
+This is useful for doing transfers to a new destination while leaving
+existing files intact, and then doing a flash-cutover when all files have
+been successfully transferred.
+
+Multiple bf(--copy-dest) directories may be provided, which will cause
+rsync to search the list in the order specified for an unchanged file.
+If a match is not found, a basis file from one of the em(DIR)s will be
+selected to try to speed up the transfer.
+
+If em(DIR) is a relative path, it is relative to the destination directory.
+See also bf(--compare-dest) and bf(--link-dest).
+
+dit(bf(--link-dest=DIR)) This option behaves like bf(--copy-dest), but
 unchanged files are hard linked from em(DIR) to the destination directory.
 The files must be identical in all preserved attributes (e.g. permissions,
 possibly ownership) in order for the files to be linked together.
@@ -960,13 +996,16 @@ An example:
 
 quote(tt(  rsync -av --link-dest=$PWD/prior_dir host:src_dir/ new_dir/))
 
-Beginning with version 2.6.4, if more than one bf(--link-dest) option is
-specified, rsync will try to find an exact match to link with (searching
-the list in the order specified), and if not found, a basis file from one
-of the em(DIR)s will be selected to try to speed up the transfer.
+Beginning in version 2.6.4, multiple bf(--link-dest) directories may be
+provided, which will cause rsync to search the list in the order specified
+for an exact match.
+If a match is found that differs only in attributes, a local copy is made
+and the attributes updated.
+If a match is not found, a basis file from one of the em(DIR)s will be
+selected to try to speed up the transfer.
 
 If em(DIR) is a relative path, it is relative to the destination directory.
-See also bf(--compare-dest).
+See also bf(--compare-dest) and bf(--copy-dest).
 
 Note that rsync versions prior to 2.6.1 had a bug that could prevent
 bf(--link-dest) from working properly for a non-root user when bf(-o) was specified
@@ -1002,6 +1041,11 @@ dit(bf(--timeout=TIMEOUT)) This option allows you to set a maximum I/O
 timeout in seconds. If no data is transferred for the specified time
 then rsync will exit. The default is 0, which means no timeout.
 
+dit(bf(--address)) By default rsync will bind to the wildcard address when
+connecting to an rsync daemon.  The bf(--address) option allows you to
+specify a specific IP address (or hostname) to bind to.  See also this
+option in the bf(--daemon) mode section.
+
 dit(bf(--port=PORT)) This specifies an alternate TCP port number to use
 rather than the default of 873.  This is only needed if you are using the
 double-colon (::) syntax to connect with an rsync daemon (since the URL
@@ -1021,49 +1065,61 @@ dit(bf(-i, --itemize-changes)) Requests a simple itemized list of the
 changes that are being made to each file, including attribute changes.
 This is exactly the same as specifying bf(--log-format='%i %n%L').
 
-The "%i" escape has a cryptic output that is 8 letters long.  The general
-format is as follows:
+The "%i" escape has a cryptic output that is 9 letters long.  The general
+format is like the string bf(UXcstpoga)), where bf(U) is replaced by the
+kind of update being done, bf(X) is replaced by the file-type, and the
+other letters represent attributes that may be output if they are being
+modified.
 
-quote(tt(  =Xcstpog))
+The update types that replace the bf(U) are as follows:
 
-The bf(=) is output as either a bf(<) (receive) or a bf(>) (send) if the
-item is being transferred, a bf(.) if only the attributes are being
-updated, or a bf(=) if the items are identical.  Note that when a symlink
-or a device gets its value changed, that is considered to be a transfer (as
-opposed to a change in permissions, ownership, etc.).
+quote(itemize(
+  it() A bf(<) means that a file is being transferred to the remote host
+  (sent).
+  it() A bf(>) means that a file is being transferred to the local host
+  (received).
+  it() A bf(c) means that a local change/creation is occurring for the item
+  (such as the creation of a directory or the changing of a symlink, etc.).
+  it() A bf(h) means that the item is a hard-link to another item (requires
+  bf(--hard-links)).
+  it() A bf(.) means that the item is not being updated (though it might
+  have attributes that are being modified).
+))
 
-The bf(X) will be replaced by one of the following: an "f" for a file, a
-"d" for a dir, an "L" for a symlink, or a "D" for a device.
+The file-types that replace the bf(X) are: bf(f) for a file, a bf(d) for a
+directory, an bf(L) for a symlink, and a bf(D) for a device.
 
-The rest of the letters in the string above are the actual letters that
+The other letters in the string above are the actual letters that
 will be output if the associated attribute for the item is being updated or
 a "." for no change.  Three exceptions to this are: (1) a newly created
-item replaces each letter with a "+", (2) an identical item replaces each
-letter with a space, and (3) an unknown attribute replaces each letter with
-a "?" (this happens when talking to an older rsync).
+item replaces each letter with a "+", (2) an identical item replaces the
+dots with spaces, and (3) an unknown attribute replaces each letter with
+a "?" (this can happen when talking to an older rsync).
 
 The attribute that is associated with each letter is as follows:
 
 quote(itemize(
   it() A bf(c) means the checksum of the file is different and will be
-  updated by the file transfer (requries bf(--checksum)).
+  updated by the file transfer (requires bf(--checksum)).
   it() A bf(s) means the size of the file is different and will be updated
   by the file transfer.
   it() A bf(t) means the modification time is different and is being updated
-  to the server's value (requires bf(--times)).  An alternate value of bf(T)
+  to the sender's value (requires bf(--times)).  An alternate value of bf(T)
   means that the time will be set to the transfer time, which happens
   anytime a symlink is transferred, or when a file or device is transferred
   without bf(--times).
   it() A bf(p) means the permissions are different and are being updated to
-  the server's value (requires bf(--perms)).
+  the sender's value (requires bf(--perms)).
   it() An bf(o) means the owner is different and is being updated to the
-  server's value (requires bf(--owner) and root privileges).
+  sender's value (requires bf(--owner) and root privileges).
   it() A bf(g) means the group is different and is being updated to the
-  server's value (requires bf(--group) and the authority to set the group).
+  sender's value (requires bf(--group) and the authority to set the group).
+  it() The bf(a) is reserved for a future enhanced version that supports
+  extended file attributes, such as ACLs.
 ))
 
 One other output is possible:  when deleting files, the "%i" will output
-the string "deleting" for each item that is being removed (assuming that
+the string "*deleting" for each item that is being removed (assuming that
 you are talking to a recent enough rsync that it logs deletions instead of
 outputting them as a verbose message).
 
@@ -1078,7 +1134,7 @@ Specifying this option will mention each file, dir, etc. that gets updated
 in a significant way (a transferred file, a recreated symlink/device, or a
 touched directory) unless the itemized-changes escape (%i) is included in
 the string, in which case the logging of names increases to mention any
-item that is updated in any way (as long as the receiving side is version
+item that is changed in any way (as long as the receiving side is at least
 2.6.4).  See the bf(--itemized-changes) option for a description of the
 output of "%i".
 
@@ -1146,7 +1202,7 @@ option does not look for this environment value is (1) when bf(--inplace) was
 specified (since bf(--inplace) conflicts with bf(--partial-dir)), or (2) when
 bf(--delay-updates) was specified (see below).
 
-For the purposes of the server-config's "refuse options" setting,
+For the purposes of the daemon-config's "refuse options" setting,
 bf(--partial-dir) does em(not) imply bf(--partial).  This is so that a
 refusal of the bf(--partial) option can be used to disallow the overwriting
 of destination files with a partial transfer, while still allowing the
@@ -1160,7 +1216,7 @@ atomic.  By default the files are placed into a directory named ".~tmp~" in
 each file's destination directory, but you can override this by specifying
 the bf(--partial-dir) option.  (Note that RSYNC_PARTIAL_DIR has no effect
 on this value, nor is bf(--partial-dir) considered to be implied for the
-purposes of the server-config's "refuse options" setting.)
+purposes of the daemon-config's "refuse options" setting.)
 Conflicts with bf(--inplace).
 
 This option uses more memory on the receiving side (one bit per file
@@ -1204,8 +1260,8 @@ purpose is to make it much easier to specify these two options for a long
 transfer that may be interrupted.
 
 dit(bf(--password-file)) This option allows you to provide a password
-in a file for accessing a remote rsync server. Note that this option
-is only useful when accessing an rsync server using the built in
+in a file for accessing a remote rsync daemon. Note that this option
+is only useful when accessing an rsync daemon using the built in
 transport, not when using a remote shell as the transport. The file
 must not be world readable. It should contain just the password as a
 single line.
@@ -1213,9 +1269,11 @@ single line.
 dit(bf(--list-only)) This option will cause the source files to be listed
 instead of transferred.  This option is inferred if there is no destination
 specified, so you don't usually need to use it explicitly.  However, it can
-come in handy for a power user that wants to avoid the "bf(-r --exclude='/*/*')"
+come in handy for a user that wants to avoid the "bf(-r --exclude='/*/*')"
 options that rsync might use as a compatibility kluge when generating a
-non-recursive listing.
+non-recursive listing, or to list the files that are involved in a local
+copy (since the destination path is not optional for a local copy, you
+must specify this option explicitly and still include a destination).
 
 dit(bf(--bwlimit=KBPS)) This option allows you to specify a maximum
 transfer rate in kilobytes per second. This option is most effective when
@@ -1227,13 +1285,38 @@ of zero specifies no limit.
 
 dit(bf(--write-batch=FILE)) Record a file that can later be applied to
 another identical destination with bf(--read-batch). See the "BATCH MODE"
-section for details.
+section for details, and also the bf(--only-write-batch) option.
+
+dit(bf(--only-write-batch=FILE)) Works like bf(--write-batch), except that
+no updates are made on the destination system when creating the batch.
+This lets you transport the changes to the destination system via some
+other means and then apply the changes via bf(--read-batch).
+
+Note that you can feel free to write the batch directly to some portable
+media: if this media fills to capacity before the end of the transfer, you
+can just apply that partial transfer to the destination and repeat the
+whole process to get the rest of the changes (as long as you don't mind a
+partially updated destination system while the multi-update cycle is
+happening).
+
+Also note that you only save bandwidth when pushing changes to a remote
+system because this allows the batched data to be diverted from the sender
+into the batch file without having to flow over the wire to the receiver
+(when pulling, the sender is remote, and thus can't write the batch).
 
 dit(bf(--read-batch=FILE)) Apply all of the changes stored in FILE, a
 file previously generated by bf(--write-batch).
 If em(FILE) is "-" the batch data will be read from standard input.
 See the "BATCH MODE" section for details.
 
+dit(bf(--protocol=NUM)) Force an older protocol version to be used.  This
+is useful for creating a batch file that is compatible with an older
+version of rsync.  For instance, if rsync 2.6.4 is being used with the
+bf(--write-batch) option, but rsync 2.6.3 is what will be used to run the
+bf(--read-batch) option, you should use "--protocol=28" when creating the
+batch file to force the older protocol version to be used in the batch
+file (assuming you can't upgrade the rsync on the reading system).
+
 dit(bf(-4, --ipv4) or bf(-6, --ipv6)) Tells rsync to prefer IPv4/IPv6
 when creating sockets.  This only affects sockets that rsync has direct
 control over, such as the outgoing socket when directly contacting an
@@ -1266,12 +1349,11 @@ become a background daemon.  The daemon will read the config file
 requests accordingly.  See the rsyncd.conf(5) man page for more
 details.
 
-dit(bf(--address)) By default rsync will bind to the wildcard address
-when run as a daemon with the bf(--daemon) option or when connecting to a
-rsync server. The bf(--address) option allows you to specify a specific IP
-address (or hostname) to bind to. This makes virtual hosting possible
-in conjunction with the bf(--config) option.  See also the "address" global
-option in the rsyncd.conf manpage.
+dit(bf(--address)) By default rsync will bind to the wildcard address when
+run as a daemon with the bf(--daemon) option.  The bf(--address) option
+allows you to specify a specific IP address (or hostname) to bind to.  This
+makes virtual hosting possible in conjunction with the bf(--config) option.
+See also the "address" global option in the rsyncd.conf manpage.
 
 dit(bf(--bwlimit=KBPS)) This option allows you to specify a maximum
 transfer rate in kilobytes per second for the data the daemon sends.
@@ -1517,7 +1599,7 @@ itemize(
   default to having that modifier set.  For instance, "merge,-/ .excl" would
   treat the contents of .excl as absolute-path excludes,
   while "dir-merge,s .filt" and ":sC" would each make all their
-  per-directory rules apply only on the server side.
+  per-directory rules apply only on the sending side.
 )
 
 The following modifiers are accepted after a "+" or "-":
@@ -1538,7 +1620,7 @@ itemize(
   being transferred.  The default is for a rule to affect both sides
   unless bf(--delete-excluded) was specified, in which case default rules
   become sender-side only.  See also the hide (H) and show (S) rules,
-  which are an alternate way to specify server-side includes/excludes.
+  which are an alternate way to specify sending-side includes/excludes.
   it() An bf(r) is used to indicate that the rule applies to the receiving
   side.  When a rule affects the receiving side, it prevents files from
   being deleted.  See the bf(s) modifier for more info.  See also the
@@ -1813,7 +1895,7 @@ Caveats:
 The read-batch option expects the destination tree that it is updating
 to be identical to the destination tree that was used to create the
 batch update fileset.  When a difference between the destination trees
-is encountered the update might be discarded with no error (if the file
+is encountered the update might be discarded with a warning (if the file
 appears to be up-to-date already) or the file-update may be attempted
 and then, if the file fails to verify, the update discarded with an
 error.  This means that it should be safe to re-run a read-batch operation
@@ -1828,10 +1910,10 @@ destination tree.
 The rsync version used on all destinations must be at least as new as the
 one used to generate the batch file.  Rsync will die with an error if the
 protocol version in the batch file is too new for the batch-reading rsync
-to handle.
-
-The bf(--dry-run) (bf(-n)) option does not work in batch mode and yields a runtime
-error.
+to handle.  See also the bf(--protocol) option for a way to have the
+creating rsync generate a batch file that an older rsync can understand.
+(Note that batch files changed format in version 2.6.3, so mixing versions
+older than that with newer versions will not work.)
 
 When reading a batch file, rsync will force the value of certain options
 to match the data in the batch file if you didn't set them to the same
@@ -1872,12 +1954,31 @@ ensure the rsync module they copy does not include symbolic links to
 bf(/etc/passwd) in the public section of the site.  Using
 bf(--copy-unsafe-links) will cause any links to be copied as the file
 they point to on the destination.  Using bf(--safe-links) will cause
-unsafe links to be omitted altogether.
+unsafe links to be omitted altogether.  (Note that you must specify
+bf(--links) for bf(--safe-links) to have any effect.)
 
 Symbolic links are considered unsafe if they are absolute symlinks
 (start with bf(/)), empty, or if they contain enough bf("..")
 components to ascend from the directory being copied.
 
+Here's a summary of how the symlink options are interpreted.  The list is
+in order of precedence, so if your combination of options isn't mentioned,
+use the first line that is a complete subset of your options:
+
+dit(bf(--copy-links)) Turn all symlinks into normal files (leaving no
+symlinks for any other options to affect).
+
+dit(bf(--links --copy-unsafe-links)) Turn all unsafe symlinks into files
+and duplicate all safe symlinks.
+
+dit(bf(--copy-unsafe-links)) Turn all unsafe symlinks into files, noisily
+skip all safe symlinks.
+
+dit(bf(--links --safe-links))  Duplicate safe symlinks and skip unsafe
+ones.
+
+dit(bf(--links)) Duplicate all symlinks.
+
 manpagediagnostics()
 
 rsync occasionally produces error messages that may seem a little
@@ -1915,6 +2016,7 @@ was made to manipulate 64-bit files on a platform that cannot support
 them; or an option was specified that is supported by the client and
 not by the server.
 dit(bf(5)) Error starting client-server protocol
+dit(bf(6)) Daemon unable to append to log-file
 dit(bf(10)) Error in socket I/O
 dit(bf(11)) Error in file I/O
 dit(bf(12)) Error in rsync protocol data stream
@@ -1925,6 +2027,7 @@ dit(bf(21)) Some error returned by waitpid()
 dit(bf(22)) Error allocating core memory buffers
 dit(bf(23)) Partial transfer due to error
 dit(bf(24)) Partial transfer due to vanished source files
+dit(bf(25)) The --max-delete limit stopped deletions
 dit(bf(30)) Timeout in data send/receive
 enddit()
 
@@ -1945,7 +2048,7 @@ password allows you to run authenticated rsync connections to an rsync
 daemon without user intervention. Note that this does not supply a
 password to a shell transport such as ssh.
 dit(bf(USER) or bf(LOGNAME)) The USER or LOGNAME environment variables
-are used to determine the default username sent to an rsync server.
+are used to determine the default username sent to an rsync daemon.
 If neither is set, the username defaults to "nobody".
 dit(bf(HOME)) The HOME environment variable is used to find the user's
 default .cvsignore file.
@@ -1975,6 +2078,10 @@ see also the comments on the bf(--delete) option
 Please report bugs! See the website at
 url(http://rsync.samba.org/)(http://rsync.samba.org/)
 
+manpagesection(VERSION)
+
+This man page is current for version 2.6.5 of rsync.
+
 manpagesection(CREDITS)
 
 rsync is distributed under the GNU public license.  See the file

rsyncd.conf.yo

@@ -1,6 +1,6 @@
 mailto(rsync-bugs@samba.org)
-manpage(rsyncd.conf)(5)(28 Feb 2005)()()
-manpagename(rsyncd.conf)(configuration file for rsync server)
+manpage(rsyncd.conf)(5)(1 Jun 2005)()()
+manpagename(rsyncd.conf)(configuration file for rsync in daemon mode)
 manpagesynopsis()
 
 rsyncd.conf
@@ -8,7 +8,7 @@ rsyncd.conf
 manpagedescription()
 
 The rsyncd.conf file is the runtime configuration file for rsync when
-run as an rsync server. 
+run as an rsync daemon. 
 
 The rsyncd.conf file controls authentication, access, logging and
 available modules.
@@ -53,7 +53,7 @@ You can launch it either via inetd, as a stand-alone daemon, or from
 an rsync client via a remote shell.  If run as a stand-alone daemon then
 just run the command "bf(rsync --daemon)" from a suitable startup script.
 If run from an rsync client via a remote shell (by specifying both the
-bf(--rsh) (bf(-e)) option and server mode with "::" or "rsync://"), the bf(--daemon)
+bf(--rsh) (bf(-e)) option and daemon mode with "::" or "rsync://"), the bf(--daemon)
 option is automatically passed to the remote side.
 
 When run via inetd you should add a line like this to /etc/services:
@@ -68,7 +68,7 @@ Replace "/usr/bin/rsync" with the path to where you have rsync installed on
 your system.  You will then need to send inetd a HUP signal to tell it to
 reread its config file.
 
-Note that you should not send the rsync server a HUP signal to force
+Note that you should bf(not) send the rsync daemon a HUP signal to force
 it to reread the tt(rsyncd.conf) file. The file is re-read on each client
 connection. 
 
@@ -90,19 +90,30 @@ is no motd file.
 dit(bf(log file)) The "log file" option tells the rsync daemon to log
 messages to that file rather than using syslog. This is particularly
 useful on systems (such as AIX) where syslog() doesn't work for
-chrooted programs.
+chrooted programs.  If the daemon fails to open to specified file, it
+will fall back to using syslog and output an error about the failure.
+(Note that a failure to open the specified log file used to be a fatal
+error.)
 
 dit(bf(pid file)) The "pid file" option tells the rsync daemon to write
 its process ID to that file.
 
 dit(bf(syslog facility)) The "syslog facility" option allows you to
 specify the syslog facility name to use when logging messages from the
-rsync server. You may use any standard syslog facility name which is
+rsync daemon. You may use any standard syslog facility name which is
 defined on your system. Common names are auth, authpriv, cron, daemon,
 ftp, kern, lpr, mail, news, security, syslog, user, uucp, local0,
 local1, local2, local3, local4, local5, local6 and local7. The default
 is daemon. 
 
+dit(bf(port)) You can override the default port the daemon will listen on
+by specifying this value (defaults to 873).  This is ignored if the daemon
+is being run by inetd, and is superseded by the bf(--port) command-line option.
+
+dit(bf(address)) You can override the default IP address the daemon
+will listen on by specifying this value.  This is ignored if the daemon is
+being run by inetd, and is superseded by the bf(--address) command-line option.
+
 dit(bf(socket options)) This option can provide endless fun for people
 who like to tune their systems to the utmost degree. You can set all
 sorts of socket options which may make transfers faster (or
@@ -126,11 +137,11 @@ dit(bf(comment)) The "comment" option specifies a description string
 that is displayed next to the module name when clients obtain a list
 of available modules. The default is no comment.
 
-dit(bf(path)) The "path" option specifies the directory in the servers
+dit(bf(path)) The "path" option specifies the directory in the daemon's
 filesystem to make available in this module.  You must specify this option
 for each module in tt(rsyncd.conf).
 
-dit(bf(use chroot)) If "use chroot" is true, the rsync server will chroot
+dit(bf(use chroot)) If "use chroot" is true, the rsync daemon will chroot
 to the "path" before starting the file transfer with the client.  This has
 the advantage of extra protection against possible implementation security
 holes, but it has the disadvantages of requiring super-user privileges, 
@@ -156,17 +167,14 @@ Note that you are free to setup user/group information in the chroot area
 differently from your normal system.  For example, you could abbreviate
 the list of users and groups.  Also, you can protect this information from
 being downloaded/uploaded by adding an exclude rule to the rsync.conf file
-(e.g. "exclude = /etc/").  Note that having the exclusion affect uploads
-is a relatively new feature in rsync, so make sure your server is running
-at least 2.6.3 to effect this.
-
-dit(bf(port)) You can override the default port the daemon will listen on
-by specifying this value (defaults to 873).  This is ignored if the daemon
-is being run by inetd, and is superseded by the bf(--port) command-line option.
-
-dit(bf(address)) You can override the default IP address the daemon
-will listen on by specifying this value.  This is ignored if the daemon is
-being run by inetd, and is superseded by the bf(--address) command-line option.
+(e.g. "exclude = /etc/**").  Note that having the exclusion affect uploads
+is a relatively new feature in rsync, so make sure your daemon is
+at least 2.6.3 to effect this.  Also note that it is safest to exclude a
+directory and all its contents combining the rule "/some/dir/" with the
+rule "/some/dir/**" just to be sure that rsync will not allow deeper
+access to some of the excluded files inside the directory (rsync tries to
+do this automatically, but you might as well specify both to be extra
+sure).
 
 dit(bf(max connections)) The "max connections" option allows you to
 specify the maximum number of simultaneous connections you will allow.
@@ -180,7 +188,7 @@ generate (since the information goes into the log file). The default is 1,
 which allows the client to request one level of verbosity.
 
 dit(bf(lock file)) The "lock file" option specifies the file to use to
-support the "max connections" option. The rsync server uses record
+support the "max connections" option. The rsync daemon uses record
 locking on this file to ensure that the max connections limit is not
 exceeded for the modules sharing the lock file. 
 The default is tt(/var/run/rsyncd.lock).
@@ -188,13 +196,13 @@ The default is tt(/var/run/rsyncd.lock).
 dit(bf(read only)) The "read only" option determines whether clients
 will be able to upload files or not. If "read only" is true then any
 attempted uploads will fail. If "read only" is false then uploads will
-be possible if file permissions on the server allow them. The default
+be possible if file permissions on the daemon side allow them. The default
 is for all modules to be read only.
 
 dit(bf(write only)) The "write only" option determines whether clients
 will be able to download files or not. If "write only" is true then any
 attempted downloads will fail. If "write only" is false then downloads
-will be possible if file permissions on the server allow them.  The
+will be possible if file permissions on the daemon side allow them.  The
 default is for this option to be disabled.
 
 dit(bf(list)) The "list" option determines if this module should be
@@ -214,30 +222,30 @@ was run as root. This complements the "uid" option. The default is gid -2,
 which is normally the group "nobody".
 
 dit(bf(filter)) The "filter" option allows you to specify a space-separated
-list of filter rules that the server will not allow to be read or written.
+list of filter rules that the daemon will not allow to be read or written.
 This is only superficially equivalent to the client specifying these
 patterns with the bf(--filter) option.  Only one "filter" option may be
 specified, but it may contain as many rules as you like, including
 merge-file rules.  Note that per-directory merge-file rules do not provide
 as much protection as global rules, but they can be used to make bf(--delete)
-work better when a client downloads the server's files (if the per-dir
+work better when a client downloads the daemon's files (if the per-dir
 merge files are included in the transfer).
 
 dit(bf(exclude)) The "exclude" option allows you to specify a
-space-separated list of patterns that the server will not allow to be read
+space-separated list of patterns that the daemon will not allow to be read
 or written.  This is only superficially equivalent to the client
 specifying these patterns with the bf(--exclude) option.  Only one "exclude"
 option may be specified, but you can use "-" and "+" before patterns to
 specify exclude/include.
 
 Because this exclude list is not passed to the client it only applies on
-the server: that is, it excludes files received by a client when receiving
-from a server and files deleted on a server when sending to a server, but
+the daemon: that is, it excludes files received by a client when receiving
+from a daemon and files deleted on a daemon when sending to a daemon, but
 it doesn't exclude files from being deleted on a client when receiving
-from a server.  
+from a daemon.  
 
 dit(bf(exclude from)) The "exclude from" option specifies a filename
-on the server that contains exclude patterns, one per line.
+on the daemon that contains exclude patterns, one per line.
 This is only superficially equivalent
 to the client specifying the bf(--exclude-from) option with an equivalent file.
 See the "exclude" option above.
@@ -245,14 +253,14 @@ See the "exclude" option above.
 dit(bf(include)) The "include" option allows you to specify a
 space-separated list of patterns which rsync should not exclude. This is
 only superficially equivalent to the client specifying these patterns with
-the bf(--include) option because it applies only on the server.  This is
+the bf(--include) option because it applies only on the daemon.  This is
 useful as it allows you to build up quite complex exclude/include rules.
 Only one "include" option may be specified, but you can use "+" and "-"
 before patterns to switch include/exclude.  See the "exclude" option
 above.
 
 dit(bf(include from)) The "include from" option specifies a filename
-on the server that contains include patterns, one per line. This is
+on the daemon that contains include patterns, one per line. This is
 only superficially equivalent to the client specifying the
 bf(--include-from) option with a equivalent file.
 See the "exclude" option above.
@@ -268,10 +276,10 @@ usernames are passwords are stored in the file specified by the
 "secrets file" option. The default is for all users to be able to
 connect without a password (this is called "anonymous rsync").
 
-See also the bf(CONNECTING TO AN RSYNC SERVER OVER A REMOTE SHELL
-PROGRAM) section in rsync(1) for information on how handle an
+See also the "CONNECTING TO AN RSYNC DAEMON OVER A REMOTE SHELL
+PROGRAM" section in rsync(1) for information on how handle an
 rsyncd.conf-level username that differs from the remote-shell-level
-username when using a remote shell to connect to an rsync server.
+username when using a remote shell to connect to an rsync daemon.
 
 dit(bf(secrets file)) The "secrets file" option specifies the name of
 a file that contains the username:password pairs used for
@@ -346,21 +354,21 @@ rejected. See the "hosts allow" option for more information.
 The default is no "hosts deny" option, which means all hosts can connect.
 
 dit(bf(ignore errors)) The "ignore errors" option tells rsyncd to
-ignore I/O errors on the server when deciding whether to run the delete
+ignore I/O errors on the daemon when deciding whether to run the delete
 phase of the transfer. Normally rsync skips the bf(--delete) step if any
 I/O errors have occurred in order to prevent disastrous deletion due
 to a temporary resource shortage or other I/O error. In some cases this
 test is counter productive so you can use this option to turn off this
 behavior. 
 
-dit(bf(ignore nonreadable)) This tells the rsync server to completely
+dit(bf(ignore nonreadable)) This tells the rsync daemon to completely
 ignore files that are not readable by the user. This is useful for
 public archives that may have some non-readable files among the
 directories, and the sysadmin doesn't want those files to be seen at all.
 
 dit(bf(transfer logging)) The "transfer logging" option enables per-file 
 logging of downloads and uploads in a format somewhat similar to that
-used by ftp daemons.  The server always logs the transfer at the end, so
+used by ftp daemons.  The daemon always logs the transfer at the end, so
 if a transfer is aborted, no mention will be made in the log file.
 
 If you want to customize the log lines, see the "log format" option.
@@ -368,7 +376,9 @@ If you want to customize the log lines, see the "log format" option.
 dit(bf(log format)) The "log format" option allows you to specify the
 format used for logging file transfers when transfer logging is enabled.
 The format is a text string containing embedded single-character escape
-sequences prefixed with a percent (%) character.
+sequences prefixed with a percent (%) character.  An optional numeric
+field width may also be specified between the percent and the escape
+letter (e.g. "%-50n %8l %07p").
 
 The default log format is "%o %h [%a] %m (%u) %f %l", and a "%t [%p] "
 is always prefixed when using the "log file" option.
@@ -384,9 +394,11 @@ quote(itemize(
   it() %l for the length of the file in bytes
   it() %p for the process ID of this rsync session
   it() %o for the operation, which is "send", "recv", or "del."
+  (the latter includes the trailing period)
   it() %f for the filename (long form on sender; no trailing "/")
   it() %n for the filename (short form; trailing "/" on dir)
-  it() %L either the string " -> SYMLINK" or "" if not a symlink
+  it() %L either the string " -> SYMLINK", or " => HARDLINK" or an
+  empty string (where bf(SYMLINK) or bf(HARDLINK) is a filename)
   it() %P for the module path
   it() %m for the module name
   it() %t for the current date time
@@ -401,19 +413,19 @@ For a list of what the characters mean that are output by "%i", see the
 bf(--itemize-changes) option in the rsync manpage.
 
 Note that some of the logged output changes when talking with older
-rsync versions.  For instance, deleted files were only logged as verbose
+rsync versions.  For instance, deleted files were only output as verbose
 messages prior to rsync 2.6.4.
 
 dit(bf(timeout)) The "timeout" option allows you to override the
 clients choice for I/O timeout for this module. Using this option you
 can ensure that rsync won't wait on a dead client forever. The timeout
 is specified in seconds. A value of zero means no timeout and is the
-default. A good choice for anonymous rsync servers may be 600 (giving
+default. A good choice for anonymous rsync daemons may be 600 (giving
 a 10 minute timeout).
 
 dit(bf(refuse options)) The "refuse options" option allows you to
 specify a space-separated list of rsync command line options that will
-be refused by your rsync server.
+be refused by your rsync daemon.
 You may specify the full option name, its one-letter abbreviation, or a
 wild-card string that matches multiple options.
 For example, this would refuse bf(--checksum) (bf(-c)) and all the various
@@ -428,7 +440,7 @@ bf(remove-sent-files) when the daemon is the sender; if you want the latter
 without the former, instead refuse "delete-*" -- that refuses all the
 delete modes without affecting bf(--remove-sent-files).
 
-When an option is refused, the server prints an error message and exits.
+When an option is refused, the daemon prints an error message and exits.
 To prevent all compression, you can use "dont compress = *" (see below)
 instead of "refuse options = compress" to avoid returning an error to a
 client that requests compression.
@@ -456,7 +468,7 @@ realize that this is not a "military strength" authentication system.
 It should be good enough for most purposes but if you want really top
 quality security then I recommend that you run rsync over ssh.
 
-Also note that the rsync server protocol does not currently provide any
+Also note that the rsync daemon protocol does not currently provide any
 encryption of the data that is transferred over the connection. Only
 authentication is provided. Use ssh as the transport if you want
 encryption.
@@ -464,16 +476,16 @@ encryption.
 Future versions of rsync may support SSL for better authentication and
 encryption, but that is still being investigated.
 
-manpagesection(RUNNING AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM)
+manpagesection(RUNNING AN RSYNC DAEMON OVER A REMOTE SHELL PROGRAM)
 
 If rsync is run with both the bf(--daemon) and bf(--rsh) (bf(-e)) options, it will
 spawn an rsync daemon using a remote shell connection.  Several
 configuration options will not be available unless the remote user is
 root (e.g. chroot, setuid/setgid, etc.).  There is no need to configure
-inetd or the services map to include the rsync server port if you run an
-rsync server only via a remote shell program.
+inetd or the services map to include the rsync daemon port if you run an
+rsync daemon only via a remote shell program.
 
-ADVANCED: To run an rsync server out of a single-use ssh key, use the
+ADVANCED: To run an rsync daemon out of a single-use ssh key, use the
 "command=em(COMMAND)" syntax in the remote user's authorized_keys entry,
 where command would be
 
@@ -553,15 +565,12 @@ manpagediagnostics()
 
 manpagebugs()
 
-The rsync server does not send all types of error messages to the
-client. this means a client may be mystified as to why a transfer
-failed. The error will have been logged by syslog on the server.
-
 Please report bugs! The rsync bug tracking system is online at
 url(http://rsync.samba.org/)(http://rsync.samba.org/)
 
 manpagesection(VERSION)
-This man page is current for version 2.x of rsync.
+
+This man page is current for version 2.6.5 of rsync.
 
 manpagesection(CREDITS)
 
@@ -582,7 +591,7 @@ Gailly and Mark Adler.
 manpagesection(THANKS)
 
 Thanks to Warren Stanley for his original idea and patch for the rsync
-server. Thanks to Karsten Thygesen for his many suggestions and
+daemon. Thanks to Karsten Thygesen for his many suggestions and
 documentation! 
 
 manpageauthor()

sender.c

@@ -20,7 +20,7 @@
 #include "rsync.h"
 
 extern int verbose;
-extern int dry_run;
+extern int do_xfers;
 extern int am_server;
 extern int am_daemon;
 extern int log_before_transfer;
@@ -28,13 +28,17 @@ extern int log_format_has_i;
 extern int daemon_log_format_has_i;
 extern int csum_length;
 extern int io_error;
+extern int allowed_lull;
 extern int protocol_version;
 extern int remove_sent_files;
 extern int updating_basis_file;
 extern int make_backups;
 extern int do_progress;
 extern int inplace;
+extern int batch_fd;
+extern int write_batch;
 extern struct stats stats;
+extern struct file_list *the_file_list;
 extern char *log_format;
 
 
@@ -53,6 +57,7 @@ static struct sum_struct *receive_sums(int f)
 {
 	struct sum_struct *s;
 	int32 i;
+	int lull_mod = allowed_lull * 5;
 	OFF_T offset = 0;
 
 	if (!(s = new(struct sum_struct)))
@@ -86,6 +91,9 @@ static struct sum_struct *receive_sums(int f)
 			s->sums[i].len = s->blength;
 		offset += s->sums[i].len;
 
+		if (allowed_lull && !(i % lull_mod))
+			maybe_send_keepalive();
+
 		if (verbose > 3) {
 			rprintf(FINFO,
 				"chunk[%d] len=%d offset=%.0f sum1=%08x\n",
@@ -99,18 +107,16 @@ static struct sum_struct *receive_sums(int f)
 	return s;
 }
 
-static struct file_list *the_flist;
-
-void successful_send(int i)
+void successful_send(int ndx)
 {
 	char fname[MAXPATHLEN];
 	struct file_struct *file;
 	unsigned int offset;
 
-	if (!the_flist || i < 0 || i >= the_flist->count)
+	if (ndx < 0 || ndx >= the_file_list->count)
 		return;
 
-	file = the_flist->files[i];
+	file = the_file_list->files[ndx];
 	/* The generator might tell us about symlinks we didn't send. */
 	if (!(file->flags & FLAG_SENT) && !S_ISLNK(file->mode))
 		return;
@@ -126,6 +132,69 @@ void successful_send(int i)
 	}
 }
 
+static void write_ndx_and_attrs(int f_out, int ndx, int iflags,
+				uchar fnamecmp_type, char *buf, int len)
+{
+	write_int(f_out, ndx);
+	if (protocol_version < 29)
+		return;
+	write_shortint(f_out, iflags);
+	if (iflags & ITEM_BASIS_TYPE_FOLLOWS)
+		write_byte(f_out, fnamecmp_type);
+	if (iflags & ITEM_XNAME_FOLLOWS)
+		write_vstring(f_out, buf, len);
+}
+
+/* This is also used by receive.c with f_out = -1. */
+int read_item_attrs(int f_in, int f_out, int ndx, uchar *type_ptr,
+		    char *buf, int *len_ptr)
+{
+	int len;
+	uchar fnamecmp_type = FNAMECMP_FNAME;
+	int iflags = protocol_version >= 29 ? read_shortint(f_in)
+		   : ITEM_TRANSFER | ITEM_MISSING_DATA;
+
+	/* Handle the new keep-alive (no-op) packet. */
+	if (ndx == the_file_list->count && iflags == ITEM_IS_NEW)
+		;
+	else if (ndx < 0 || ndx >= the_file_list->count) {
+		rprintf(FERROR, "Invalid file index: %d (count=%d) [%s]\n",
+			ndx, the_file_list->count, who_am_i());
+		exit_cleanup(RERR_PROTOCOL);
+	} else if (iflags == ITEM_IS_NEW) {
+		rprintf(FERROR, "Invalid itemized flag word: %x [%s]\n",
+			iflags, who_am_i());
+		exit_cleanup(RERR_PROTOCOL);
+	}
+
+	if (iflags & ITEM_BASIS_TYPE_FOLLOWS)
+		fnamecmp_type = read_byte(f_in);
+	*type_ptr = fnamecmp_type;
+
+	if (iflags & ITEM_XNAME_FOLLOWS) {
+		if ((len = read_vstring(f_in, buf, MAXPATHLEN)) < 0)
+			exit_cleanup(RERR_PROTOCOL);
+	} else {
+		*buf = '\0';
+		len = -1;
+	}
+	*len_ptr = len;
+
+	if (iflags & ITEM_TRANSFER) {
+		if (!S_ISREG(the_file_list->files[ndx]->mode)) {
+			rprintf(FERROR,
+				"received request to transfer non-regular file: %d [%s]\n",
+				ndx, who_am_i());
+			exit_cleanup(RERR_PROTOCOL);
+		}
+	} else if (f_out >= 0) {
+		write_ndx_and_attrs(f_out, ndx, iflags,
+				    fnamecmp_type, buf, len);
+	}
+
+	return iflags;
+}
+
 void send_files(struct file_list *flist, int f_out, int f_in)
 {
 	int fd = -1;
@@ -133,51 +202,42 @@ void send_files(struct file_list *flist, int f_out, int f_in)
 	struct map_struct *mbuf = NULL;
 	STRUCT_STAT st;
 	char *fname2, fname[MAXPATHLEN];
-	int iflags;
+	char xname[MAXPATHLEN];
+	uchar fnamecmp_type;
+	int iflags, xlen;
 	struct file_struct *file;
-	int phase = 0;
+	int phase = 0, max_phase = protocol_version >= 29 ? 2 : 1;
 	struct stats initial_stats;
 	int save_make_backups = make_backups;
 	int itemizing = am_daemon ? daemon_log_format_has_i
 		      : !am_server && log_format_has_i;
+	int f_xfer = write_batch < 0 ? batch_fd : f_out;
 	int i, j;
 
 	if (verbose > 2)
 		rprintf(FINFO, "send_files starting\n");
 
-	the_flist = flist;
-
 	while (1) {
 		unsigned int offset;
 
 		i = read_int(f_in);
 		if (i == -1) {
-			if (phase == 0) {
-				phase++;
-				csum_length = SUM_LENGTH;
-				write_int(f_out, -1);
-				if (verbose > 2)
-					rprintf(FINFO, "send_files phase=%d\n", phase);
-				/* For inplace: redo phase turns off the backup
-				 * flag so that we do a regular inplace send. */
-				make_backups = 0;
-				continue;
-			}
-			break;
+			if (++phase > max_phase)
+				break;
+			csum_length = SUM_LENGTH;
+			if (verbose > 2)
+				rprintf(FINFO, "send_files phase=%d\n", phase);
+			write_int(f_out, -1);
+			/* For inplace: redo phase turns off the backup
+			 * flag so that we do a regular inplace send. */
+			make_backups = 0;
+			continue;
 		}
 
-		if (i < 0 || i >= flist->count) {
-			/* Handle the new keep-alive (no-op) packet. */
-			if (i == flist->count && protocol_version >= 29
-	 		    && read_shortint(f_in) == ITEM_IS_NEW) {
-				write_int(f_out, i);
-				write_shortint(f_out, ITEM_IS_NEW);
-				continue;
-			}
-			rprintf(FERROR, "Invalid file index %d (count=%d)\n",
-				i, flist->count);
-			exit_cleanup(RERR_PROTOCOL);
-		}
+		iflags = read_item_attrs(f_in, f_out, i, &fnamecmp_type,
+					 xname, &xlen);
+		if (iflags == ITEM_IS_NEW) /* no-op packet */
+			continue;
 
 		file = flist->files[i];
 		if (file->dir.root) {
@@ -192,45 +252,29 @@ void send_files(struct file_list *flist, int f_out, int f_in)
 		if (verbose > 2)
 			rprintf(FINFO, "send_files(%d, %s)\n", i, fname);
 
-		if (protocol_version >= 29) {
-			iflags = read_shortint(f_in);
-			if (!(iflags & ITEM_UPDATING) || !S_ISREG(file->mode)) {
-				int see_item = itemizing && (iflags || verbose > 1);
-				write_int(f_out, i);
-				write_shortint(f_out, iflags);
-				if (am_server) {
-					if (am_daemon && !dry_run && see_item)
-						log_send(file, &stats, iflags);
-				} else if (see_item || iflags & ITEM_UPDATING
-				    || (S_ISDIR(file->mode)
-				     && iflags & ITEM_REPORT_TIME))
-					log_send(file, &stats, iflags);
-				continue;
-			}
-		} else
-			iflags = ITEM_UPDATING | ITEM_MISSING_DATA;
-
-		if (inplace && protocol_version >= 29) {
-			updating_basis_file = !(iflags & ITEM_USING_ALT_BASIS);
-		} else
-			updating_basis_file = inplace && !make_backups;
-
-		if (!S_ISREG(file->mode)) {
-			rprintf(FERROR, "[%s] got index of non-regular file: %d\n",
-				who_am_i(), i);
+		if (!(iflags & ITEM_TRANSFER)) {
+			maybe_log_item(file, iflags, itemizing, xname);
+			continue;
+		}
+		if (phase == 2) {
+			rprintf(FERROR,
+				"got transfer request in phase 2 [%s]\n",
+				who_am_i());
 			exit_cleanup(RERR_PROTOCOL);
 		}
 
+		updating_basis_file = inplace && (protocol_version >= 29
+			? fnamecmp_type == FNAMECMP_FNAME : !make_backups);
+
 		stats.current_file_index = i;
 		stats.num_transferred_files++;
 		stats.total_transferred_size += file->length;
 
-		if (dry_run) { /* log the transfer */
+		if (!do_xfers) { /* log the transfer */
 			if (!am_server && log_format)
-				log_send(file, &stats, iflags);
-			write_int(f_out, i);
-			if (protocol_version >= 29)
-				write_shortint(f_out, iflags);
+				log_item(file, &stats, iflags, NULL);
+			write_ndx_and_attrs(f_out, i, iflags, fnamecmp_type,
+					    xname, xlen);
 			continue;
 		}
 
@@ -281,10 +325,9 @@ void send_files(struct file_list *flist, int f_out, int f_in)
 				safe_fname(fname), (double)st.st_size);
 		}
 
-		write_int(f_out, i);
-		if (protocol_version >= 29)
-			write_shortint(f_out, iflags);
-		write_sum_head(f_out, s);
+		write_ndx_and_attrs(f_out, i, iflags, fnamecmp_type,
+				    xname, xlen);
+		write_sum_head(f_xfer, s);
 
 		if (verbose > 2) {
 			rprintf(FINFO, "calling match_sums %s\n",
@@ -292,15 +335,18 @@ void send_files(struct file_list *flist, int f_out, int f_in)
 		}
 
 		if (log_before_transfer)
-			log_send(file, &initial_stats, iflags);
+			log_item(file, &initial_stats, iflags, NULL);
 		else if (!am_server && verbose && do_progress)
 			rprintf(FINFO, "%s\n", safe_fname(fname2));
 
 		set_compression(fname);
 
-		match_sums(f_out, s, mbuf, st.st_size);
+		match_sums(f_xfer, s, mbuf, st.st_size);
+		if (do_progress)
+			end_progress(st.st_size);
+
 		if (!log_before_transfer)
-			log_send(file, &initial_stats, iflags);
+			log_item(file, &initial_stats, iflags, NULL);
 
 		if (mbuf) {
 			j = unmap_file(mbuf);

socket.c

@@ -127,7 +127,7 @@ static int establish_proxy_connection(int fd, char *host, int port,
  * if this fails.
  **/
 int try_bind_local(int s, int ai_family, int ai_socktype,
-		   const char *bind_address)
+		   const char *bind_addr)
 {
 	int error;
 	struct addrinfo bhints, *bres_all, *r;
@@ -136,9 +136,9 @@ int try_bind_local(int s, int ai_family, int ai_socktype,
 	bhints.ai_family = ai_family;
 	bhints.ai_socktype = ai_socktype;
 	bhints.ai_flags = AI_PASSIVE;
-	if ((error = getaddrinfo(bind_address, NULL, &bhints, &bres_all))) {
+	if ((error = getaddrinfo(bind_addr, NULL, &bhints, &bres_all))) {
 		rprintf(FERROR, RSYNC_NAME ": getaddrinfo %s: %s\n",
-			bind_address, gai_strerror(error));
+			bind_addr, gai_strerror(error));
 		return -1;
 	}
 
@@ -172,12 +172,12 @@ int try_bind_local(int s, int ai_family, int ai_socktype,
  * reachable, perhaps because we can't e.g. route ipv6 to that network
  * but we can get ip4 packets through.
  *
- * @param bind_address Local address to use.  Normally NULL to bind
+ * @param bind_addr Local address to use.  Normally NULL to bind
  * the wildcard address.
  *
  * @param af_hint Address family, e.g. AF_INET or AF_INET6.
  **/
-int open_socket_out(char *host, int port, const char *bind_address,
+int open_socket_out(char *host, int port, const char *bind_addr,
 		    int af_hint)
 {
 	int type = SOCK_STREAM;
@@ -253,9 +253,9 @@ int open_socket_out(char *host, int port, const char *bind_address,
 		if (s < 0)
 			continue;
 
-		if (bind_address
+		if (bind_addr
 		 && try_bind_local(s, res->ai_family, type,
-				   bind_address) == -1) {
+				   bind_addr) == -1) {
 			close(s);
 			s = -1;
 			continue;
@@ -293,9 +293,9 @@ int open_socket_out(char *host, int port, const char *bind_address,
  *
  * This is based on the Samba LIBSMB_PROG feature.
  *
- * @param bind_address Local address to use.  Normally NULL to get the stack default.
+ * @param bind_addr Local address to use.  Normally NULL to get the stack default.
  **/
-int open_socket_out_wrapped(char *host, int port, const char *bind_address,
+int open_socket_out_wrapped(char *host, int port, const char *bind_addr,
 			    int af_hint)
 {
 	char *prog = getenv("RSYNC_CONNECT_PROG");
@@ -307,7 +307,7 @@ int open_socket_out_wrapped(char *host, int port, const char *bind_address,
 	}
 	if (prog)
 		return sock_exec(prog);
-	return open_socket_out(host, port, bind_address, af_hint);
+	return open_socket_out(host, port, bind_addr, af_hint);
 }
 
 
@@ -322,10 +322,10 @@ int open_socket_out_wrapped(char *host, int port, const char *bind_address,
  * We return an array of file-descriptors to the sockets, with a trailing
  * -1 value to indicate the end of the list.
  *
- * @param bind_address Local address to bind, or NULL to allow it to
+ * @param bind_addr Local address to bind, or NULL to allow it to
  * default.
  **/
-static int *open_socket_in(int type, int port, const char *bind_address,
+static int *open_socket_in(int type, int port, const char *bind_addr,
 			   int af_hint)
 {
 	int one = 1;
@@ -339,10 +339,10 @@ static int *open_socket_in(int type, int port, const char *bind_address,
 	hints.ai_socktype = type;
 	hints.ai_flags = AI_PASSIVE;
 	snprintf(portbuf, sizeof portbuf, "%d", port);
-	error = getaddrinfo(bind_address, portbuf, &hints, &all_ai);
+	error = getaddrinfo(bind_addr, portbuf, &hints, &all_ai);
 	if (error) {
 		rprintf(FERROR, RSYNC_NAME ": getaddrinfo: bind address %s: %s\n",
-			bind_address, gai_strerror(error));
+			bind_addr, gai_strerror(error));
 		return NULL;
 	}
 
@@ -478,7 +478,7 @@ void start_accept_loop(int port, int (*fn)(int, int))
 		/* close log file before the potentially very long select so
 		 * file can be trimmed by another process instead of growing
 		 * forever */
-		log_close();
+		logfile_close();
 
 #ifdef FD_COPY
 		FD_COPY(&deffds, &fds);
@@ -507,8 +507,8 @@ void start_accept_loop(int port, int (*fn)(int, int))
 			for (i = 0; sp[i] >= 0; i++)
 				close(sp[i]);
 			/* Re-open log file in child before possibly giving
-			 * up privileges (see log_close() above). */
-			log_open();
+			 * up privileges (see logfile_close() above). */
+			logfile_reopen();
 			ret = fn(fd, fd);
 			close_all();
 			_exit(ret);

support/atomic-rsync

@@ -1,9 +1,9 @@
 #!/usr/bin/perl
 #
 # This script lets you update a hierarchy of files in an atomic way by
-# first creating a new hierarchy using --link-dest to rsync, and then
-# swapping the hierarchy into place.  See the usage message for more
-# details and some important caveats!
+# first creating a new hierarchy using rsync's --link-dest option, and
+# then swapping the hierarchy into place.  **See the usage message for
+# more details and some important caveats!**
 
 use strict;
 use Cwd 'abs_path';

support/cvs2includes

@@ -0,0 +1,42 @@
+#!/usr/bin/perl
+#
+# This script finds all CVS/Entries files in the current directory and below
+# and creates a local .cvsinclude file with non-inherited rules including each
+# checked-in file.  Then, use this option whenever using --cvs-exclude (-C):
+#
+# 	-f ': .cvsinclude'
+#
+# That ensures that all checked-in files/dirs are included in the transfer.
+# (You could alternately put ": .cvsinclude" into an .rsync-filter file and
+# use the -F option, which is easier to type.)
+#
+# The downside is that you need to remember to re-run cvs2includes whenever
+# You add a new file to the project.
+use strict;
+
+open(FIND, 'find . -name CVS -type d |') or die $!;
+while (<FIND>) {
+    chomp;
+    s#^\./##;
+
+    my $entries = "$_/Entries";
+    s/CVS$/.cvsinclude/;
+    my $filter = $_;
+
+    open(ENTRIES, $entries) or die "Unable to open $entries: $!\n";
+    my @includes;
+    while (<ENTRIES>) {
+	push(@includes, $1) if m#/(.+?)/#;
+    }
+    close ENTRIES;
+    if (@includes) {
+	open(FILTER, ">$filter") or die "Unable to write $filter: $!\n";
+	print FILTER '+ /', join("\n+ /", @includes), "\n";
+	close FILTER;
+	print "Updated $filter\n";
+    } elsif (-f $filter) {
+	unlink($filter);
+	print "Removed $filter\n";
+    }
+}
+close FIND;

support/mnt-excl

@@ -20,7 +20,7 @@
 # ADDENDUM:  The addition of the --filter option (which has support for
 # absolute-anchored excludes) has made this script less useful than it
 # was.  Beginning with 2.6.4, you can achieve the effect of this script
-# though this command:
+# through this command:
 #
 # awk '{print $2}' /proc/mounts | rsync -f 'merge,/- -' host:/dir /dest/
 

support/rrsync

@@ -31,8 +31,8 @@ die "Not invoked via sshd\n$Usage"	unless defined $command;
 die "SSH_ORIGINAL_COMMAND='$command' is not rsync\n" unless $command =~ /^rsync\s/;
 die "$0 -ro: sending to read-only server not allowed\n"
 	if $ro and $command !~ /^rsync --server --sender /;
-die "$0 -ro: use of --remove-sent-files with read-only server not allowed\n"
-	if $ro and $command =~ /\s--remove-sent-files/;
+die "$0 -ro: use of $1 with read-only server not allowed\n"
+	if $ro and $command =~ /\s(--remove-\S+)/;
 
 my ($cmd,$dir) = $command =~ /^(rsync\s+(?:-[-a-zA-Z]+\s+)+\.) ?("[^"]*"|[^\s"]*)$/;
 die "$0: invalid rsync-command syntax or options\n" if !defined $cmd;

syscall.c

@@ -243,7 +243,7 @@ int do_fstat(int fd, STRUCT_STAT *st)
 
 OFF_T do_lseek(int fd, OFF_T offset, int whence)
 {
-#if SIZEOF_OFF64_T
+#ifdef HAVE_LSEEK64
 	off64_t lseek64();
 	return lseek64(fd, offset, whence);
 #else

testsuite/backup.test

@@ -35,15 +35,20 @@ for fn in name1 name2; do
     mv "$todir/$fn~" "$todir/$fn"
 done
 
-checkit "$RSYNC -avv --no-whole-file --backup --backup-dir=\"$bakdir\" \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir" \
+echo deleted-file >"$todir/dname"
+cp -p "$todir/dname" "$chkdir"
+
+checkit "$RSYNC -avv --no-whole-file --delete-after \
+    --backup --backup-dir=\"$bakdir\" \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir" \
     | tee "$outfile"
 
 for fn in name1 name2; do
     grep "backed up $fn to .*/$fn$" "$outfile" >/dev/null || test_fail "no backup message output for $fn"
 done
 diff -r $diffopt "$chkdir" "$bakdir" || test_fail "backup dir contents are bogus"
+rm "$bakdir/dname"
 
-checkit "$RSYNC -avv \"$fromdir/\" \"$chkdir/\"" "$fromdir" "$chkdir"
+checkit "$RSYNC -avv --del \"$fromdir/\" \"$chkdir/\"" "$fromdir" "$chkdir"
 cat $srcdir/[efgr]*.[ch] > "$name1"
 cat $srcdir/[ew]*.[ch] > "$name2"
 

testsuite/chown.test

@@ -15,6 +15,16 @@
 
 set -x
 
+case `id -u` in
+'') ;; # If "id" failed, try to continue...
+0)  ;;
+*)  if [ -f /usr/bin/fakeroot ]; then
+	echo "Let's try re-running the script under fakeroot..."
+	exec /usr/bin/fakeroot /bin/sh "$0"
+    fi
+    ;;
+esac
+
 # Build some hardlinks
 
 mkdir "$fromdir"

testsuite/compare-dest.test

@@ -31,9 +31,9 @@ $RSYNC -av --exclude=/text --exclude=etc-ltr-list "$fromdir/" "$chkdir/"
 checkit "$RSYNC -avv --no-whole-file \
     --compare-dest=\"$alt1dir\" --compare-dest=\"$alt2dir\" \
     \"$fromdir/\" \"$todir/\"" "$chkdir" "$todir"
-#checkit "$RSYNC -avv --no-whole-file \
-#    --copy-dest=\"$alt1dir\" --copy-dest=\"$alt2dir\" \
-#    \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
+checkit "$RSYNC -avv --no-whole-file \
+    --copy-dest=\"$alt1dir\" --copy-dest=\"$alt2dir\" \
+    \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
 
 # The script would have aborted on error, so getting here means we've won.
 exit 0

testsuite/devices.test

@@ -13,8 +13,13 @@
 
 case `id -u` in
 '') ;; # If "id" failed, try to continue...
-0) ;;
-*) test_skipped "Rsync won't copy devices unless we're root" ;;
+0)  ;;
+*)  if [ -f /usr/bin/fakeroot ]; then
+	echo "Let's try re-running the script under fakeroot..."
+	exec /usr/bin/fakeroot /bin/sh "$0"
+    fi
+    test_skipped "Rsync won't copy devices unless we're root"
+    ;;
 esac
 
 # TODO: Need to test whether hardlinks are possible on this OS/filesystem

testsuite/fuzzy.test

@@ -0,0 +1,25 @@
+#! /bin/sh
+
+# Copyright (C) 2005 by Wayne Davison <wayned@samba.org>
+
+# This program is distributable under the terms of the GNU GPL see
+# COPYING).
+
+# Test rsync handling of the --fuzzy option.
+
+. "$suitedir/rsync.fns"
+
+set -x
+
+mkdir "$fromdir"
+mkdir "$todir"
+
+cp -p "$srcdir"/rsync.c "$fromdir"/rsync.c
+cp -p "$fromdir"/rsync.c "$todir"/rsync2.c
+
+# Let's do it!
+checkit "$RSYNC -avvi --no-whole-file --fuzzy --delete-after \
+    \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
+
+# The script would have aborted on error, so getting here means we've won.
+exit 0

testsuite/hands.test

@@ -20,7 +20,7 @@ rm "$todir/text"
 runtest "one file" 'checkit "$RSYNC -avH \"$fromdir/\" \"$todir\"" "$fromdir/" "$todir"'
 
 echo "extra line" >> "$todir/text"
-runtest "extra data" 'checkit "$RSYNC -avH \"$fromdir/\" \"$todir\"" "$fromdir/" "$todir"'
+runtest "extra data" 'checkit "$RSYNC -avH --no-whole-file \"$fromdir/\" \"$todir\"" "$fromdir/" "$todir"'
 
 cp "$fromdir/text" "$todir/ThisShouldGo"
 runtest " --delete" 'checkit "$RSYNC --delete -avH \"$fromdir/\" \"$todir\"" "$fromdir/" "$todir"'

testsuite/hardlinks.test

@@ -30,8 +30,34 @@ echo "This is the file" > "$name1"
 ln "$name1" "$name2" || fail "Can't create hardlink"
 ln "$name2" "$name3" || fail "Can't create hardlink"
 cp "$name2" "$name4" || fail "Can't copy file"
+cat $srcdir/*.c >"$fromdir/text"
 
-checkit "$RSYNC -aHvv \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
+checkit "$RSYNC -aHivv \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
+
+echo "extra extra" >>"$todir/name1"
+
+checkit "$RSYNC -aHivv --no-whole-file \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
+
+# Add a new link in a new subdirectory to test that we don't try to link
+# the files before the directory gets created.
+mkdir "$fromdir/subdir"
+ln "$name1" "$fromdir/subdir/new-file"
+rm "$todir/text"
+
+checkit "$RSYNC -aHivv \"$fromdir/\" \"$todir/\"" "$fromdir" "$todir"
+
+# Do some duplicate copies using --link-dest and --copy-dest to test that
+# we hard-link all locally-inherited items.
+checkit "$RSYNC -aHivv --link-dest=\"$todir\" \"$fromdir/\" \"$chkdir/\"" "$todir" "$chkdir"
+
+rm -rf "$chkdir"
+checkit "$RSYNC -aHivv --copy-dest=\"$todir\" \"$fromdir/\" \"$chkdir/\"" "$fromdir" "$chkdir"
+
+# Make sure there's nothing wrong with sending a single file with -H
+# enabled (this has broken twice so far, so we need this test).
+rm -rf "$todir"
+$RSYNC -aHivv "$name1" "$todir/"
+diff $diffopt "$name1" "$todir" || test_fail "solo copy of name1 failed"
 
 # The script would have aborted on error, so getting here means we've won.
 exit 0

testsuite/longdir.test

@@ -17,7 +17,11 @@ longdir="$fromdir/$longname/$longname/$longname"
 makepath "$longdir" || test_skipped "unable to create long directory"
 touch "$longdir/1" || test_skipped "unable to create files in long directory"
 date > "$longdir/1"
-ls -la / > "$longdir/2"
+if [ -r /etc ]; then
+    ls -la /etc >"$longdir/2"
+else
+    ls -la / >"$longdir/2"
+fi
 checkit "$RSYNC --delete -avH \"$fromdir/\" \"$todir\"" "$fromdir/" "$todir"
 
 # The script would have aborted on error, so getting here means we've won.

testsuite/relative.test

@@ -0,0 +1,39 @@
+#!/bin/sh
+
+# Copyright (C) 2005 by Wayne Davison <wayned@samba.org>
+#
+# This program is distributable under the terms of the GNU GPL (see COPYING)
+
+. "$suitedir/rsync.fns"
+
+deepstr='down/3/deep'
+deepdir="$fromdir/$deepstr"
+mkdir -p "$deepdir"
+fromdir="$deepdir"
+hands_setup
+fromdir="$tmpdir/from"
+
+outfile="$scratchdir/rsync.out"
+
+cd "$fromdir"
+
+# Main script starts here
+
+echo "$fromdir"
+runtest "basic relative" 'checkit "$RSYNC -avR ./$deepstr \"$todir\"" "$fromdir/" "$todir"'
+
+ln "$deepdir/filelist" "$deepdir/dir"
+runtest "hard links" 'checkit "$RSYNC -avHR ./$deepstr/ \"$todir\"" "$fromdir/" "$todir"'
+
+cp "$deepdir/text" "$todir/$deepstr/ThisShouldGo"
+cp "$deepdir/text" "$todir/$deepstr/dir/ThisShouldGoToo"
+runtest "deletion" 'checkit "$RSYNC -avHR --delete ./$deepstr/ \"$todir\"" "$fromdir/" "$todir"'
+
+runtest "non-deletion" 'checkit "$RSYNC -aiHR --delete ./$deepstr/ \"$todir\"" "$fromdir/" "$todir"' \
+    | tee "$outfile"
+
+# Make sure no files were deleted
+grep 'deleting ' "$outfile" && test_fail "Erroneous deletions occurred!"
+
+# The script would have aborted on error, so getting here means we've won.
+exit 0

util.c

@@ -245,7 +245,8 @@ static int safe_read(int desc, char *ptr, size_t len)
 
 /** Copy a file.
  *
- * This is used in conjunction with the --temp-dir and --backup options */
+ * This is used in conjunction with the --temp-dir, --backup, and
+ * --copy-dest options. */
 int copy_file(char *source, char *dest, mode_t mode)
 {
 	int ifd;
@@ -876,12 +877,13 @@ int pop_dir(char *dir)
 	return 1;
 }
 
-/* Return the filename, turning any non-printable characters into '?'s.
- * This ensures that outputting it on a line of its own cannot generate an
- * empty line.  This function can return only MAX_SAFE_NAMES values at a
- * time!  The returned value can be longer than MAXPATHLEN (because we
- * may be trying to output an error about a too-long filename)! */
-const char *safe_fname(const char *fname)
+/* Return the filename, turning any non-printable characters into escaped
+ * characters (e.g. \n -> \012, \ -> \\).  This ensures that outputting it
+ * cannot generate an empty line nor corrupt the screen.  This function can
+ * return only MAX_SAFE_NAMES values at a time!  The returned value can be
+ * longer than MAXPATHLEN (because we may be trying to output an error about
+ * a too-long filename)! */
+char *safe_fname(const char *fname)
 {
 #define MAX_SAFE_NAMES 4
 	static char fbuf[MAX_SAFE_NAMES][MAXPATHLEN*2];
@@ -891,12 +893,21 @@ const char *safe_fname(const char *fname)
 
 	ndx = (ndx + 1) % MAX_SAFE_NAMES;
 	for (t = fbuf[ndx]; *fname; fname++) {
-		if (!isprint(*(uchar*)fname))
-			*t++ = '?';
-		else
+		if (*fname == '\\') {
+			if ((limit -= 2) < 0)
+				break;
+			*t++ = '\\';
+			*t++ = '\\';
+		} else if (!isprint(*(uchar*)fname)) {
+			if ((limit -= 4) < 0)
+				break;
+			sprintf(t, "\\%03o", *(uchar*)fname);
+			t += 4;
+		} else {
+			if (--limit < 0)
+				break;
 			*t++ = *fname;
-		if (--limit == 0)
-			break;
+		}
 	}
 	*t = '\0';
 
@@ -1250,11 +1261,11 @@ const char *find_filename_suffix(const char *fn, int fn_len, int *len_ptr)
 			break;
 		s_len = fn_len - (s - fn);
 		fn_len = s - fn;
-		if (s_len == 3) {
+		if (s_len == 4) {
 			if (strcmp(s+1, "bak") == 0
 			 || strcmp(s+1, "old") == 0)
 				continue;
-		} else if (s_len == 4) {
+		} else if (s_len == 5) {
 			if (strcmp(s+1, "orig") == 0)
 				continue;
 		} else if (s_len > 2 && had_tilde