Cygwin/rsync Hang Problem Testing Results

Cygwin/rsync Hang Problem Testing Results

[Paul Thompson]

People of cygwin & rsync,

I recently attempted to get cygwin and rsync working to solve a 
backup/mirroring need in my computer life. Well, as you might guess, I 
ran into a little but of trouble.

Strangely enough, rsync seemed to be regularly hanging when I attempted 
to do a "get" (sycronize a remote to a local dir). Well, considering I 
want to automate this, that was not going to work. So I searched the 
web, mailing lists, etc, and came across all of the posts on the 
subject, but no solution.

Then I stumbled across an idea - to test previous versions of rsync to 
if any of them worked better with cygwin. Here are the (hopefully 
helpful) results of those tests. I did three tests for each version of 
cygwin:

  locallist:
    rsync -n --stats --recursive --rsh=ssh sync/ /home/drsync/

  get:
    RSYNC_CMD="rsync --verbose --progress --stats -auz -e ssh"
    $RSYNC_CMD myuser@myserver.com:sync/ sync/

  put:
    RSYNC_CMD="rsync --verbose --progress --stats -auz -e ssh"
    $RSYNC_CMD sync/ myuser@myserver.com:sync/

When rsync was hanging, it did so usually by the third time I ran the 
command. So I figured running each 10ish times should probably show the 
error.

So I downloaded the last pile of rsync releases and started testing at 
intervals, starting back at 2.3.0 and splitting the difference as I 
zeroed in on the offending release. Here are the results:

    Version   locallist   get       put
    ---------------------------------------
    2.3.0     success     success   success
    2.4.0     success     success   success
    2.4.1     success     success   failure
    2.4.2     success     success   failure
    2.4.3     success     success   failure
    2.4.4     success     success   failure
    2.4.5     success     success   failure
    2.4.6     failure     failure   failure
    2.5.0     failure     failure   success
    2.5.6     failure     failure   success

The failures of the put test (v. 2.4.1 - 2.4.6) happened EVERY time I 
ran the test and returned an error of:

    read error: Connection reset by peer

As those did not occur after 2.4.6, I assume that was a fixed bug, 
unrelated to the hanging problem that was the reason for my tests.

Anyhow, when the locallist and get tests failed, rsync would just plain 
hang - no explanation, error, I could leave it there for an hour and it 
wouldn't have changed. And then, when I ctrl-c killed the process (in my 
cygwin/bash shell), the actual rsync process would stick around and 
require killing via the windows task manager.

So, based on my testing, I would conclude that the infamous cygwin rsync 
hang bug was introduced somewhere in the coding for 2.4.6. As for 
myself, I plan to go ahead and just use version 2.4.0, as it is the most 
recent version of rsync that worked for all three tests. If I can be of 
any more help, including testing patches, please let me know.

Thanks for all of the hard work on cygwin and rsync, folks.

Sincerely,
Paul Thompson
captbunzo@squirrelmail.org


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Cygwin/rsync Hang Problem Testing Results

[Bill C. Riemers]

I have found rsync runs slighty longer when you run ssh -e 'noescape'
option, as in the following script:
#!/bin/bash
exec ssh -e 'none' "$@"

In fact, I find the above script is absolutely essential with cvs.  (I've
yet to platform where it wasn't required.)
One would expect the -T option to also improve things, but it doesn't...

I find it helps with rsync, but doesn't solve the problem.

Worse, each time rsync hangs, after killing it, the rsync process continues
on the remote machine.  If I don't manually kill -9 rsync on the remote
machine, the next time I run rsync it will hang sooner and run slower.
After five or ten times without killing the remote rsync command, rsync
never gets past update the filelist before I grow too impatient to see what
will happen.

                                                 Bill



----- Original Message ----- 
From: "Paul Thompson" <captbunzo@squirrelmail.org>
To: <rsync@lists.samba.org>; <cygwin@cygwin.com>
Sent: Thursday, October 02, 2003 7:28 PM
Subject: Cygwin/rsync Hang Problem Testing Results


> People of cygwin & rsync,
>
> I recently attempted to get cygwin and rsync working to solve a
> backup/mirroring need in my computer life. Well, as you might guess, I
> ran into a little but of trouble.
>
> Strangely enough, rsync seemed to be regularly hanging when I attempted
> to do a "get" (sycronize a remote to a local dir). Well, considering I
> want to automate this, that was not going to work. So I searched the
> web, mailing lists, etc, and came across all of the posts on the
> subject, but no solution.
>
> Then I stumbled across an idea - to test previous versions of rsync to
> if any of them worked better with cygwin. Here are the (hopefully
> helpful) results of those tests. I did three tests for each version of
> cygwin:
>
>   locallist:
>     rsync -n --stats --recursive --rsh=ssh sync/ /home/drsync/
>
>   get:
>     RSYNC_CMD="rsync --verbose --progress --stats -auz -e ssh"
>     $RSYNC_CMD myuser@myserver.com:sync/ sync/
>
>   put:
>     RSYNC_CMD="rsync --verbose --progress --stats -auz -e ssh"
>     $RSYNC_CMD sync/ myuser@myserver.com:sync/
>
> When rsync was hanging, it did so usually by the third time I ran the
> command. So I figured running each 10ish times should probably show the
> error.
>
> So I downloaded the last pile of rsync releases and started testing at
> intervals, starting back at 2.3.0 and splitting the difference as I
> zeroed in on the offending release. Here are the results:
>
>     Version   locallist   get       put
>     ---------------------------------------
>     2.3.0     success     success   success
>     2.4.0     success     success   success
>     2.4.1     success     success   failure
>     2.4.2     success     success   failure
>     2.4.3     success     success   failure
>     2.4.4     success     success   failure
>     2.4.5     success     success   failure
>     2.4.6     failure     failure   failure
>     2.5.0     failure     failure   success
>     2.5.6     failure     failure   success
>
> The failures of the put test (v. 2.4.1 - 2.4.6) happened EVERY time I
> ran the test and returned an error of:
>
>     read error: Connection reset by peer
>
> As those did not occur after 2.4.6, I assume that was a fixed bug,
> unrelated to the hanging problem that was the reason for my tests.
>
> Anyhow, when the locallist and get tests failed, rsync would just plain
> hang - no explanation, error, I could leave it there for an hour and it
> wouldn't have changed. And then, when I ctrl-c killed the process (in my
> cygwin/bash shell), the actual rsync process would stick around and
> require killing via the windows task manager.
>
> So, based on my testing, I would conclude that the infamous cygwin rsync
> hang bug was introduced somewhere in the coding for 2.4.6. As for
> myself, I plan to go ahead and just use version 2.4.0, as it is the most
> recent version of rsync that worked for all three tests. If I can be of
> any more help, including testing patches, please let me know.
>
> Thanks for all of the hard work on cygwin and rsync, folks.
>
> Sincerely,
> Paul Thompson
> captbunzo@squirrelmail.org
>
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Cygwin/rsync Hang Problem Testing Results

[Lapo Luchini]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Thompson wrote:

| 2.4.5     success     success 2.4.6     failure     failure

Funny this is: 2.4.6 is exactly the release where many cygwin-related
patches landed in rsync code, as I'm seeing from a diff.
(I began packaging rsync on CygWin at 2.4.6 and I never had your good
idea to test older versions too, what a chanche it is that 2.4.5 is the
first working one!)

I attach an purposedly incomplete diff between sources of 2.4.5 and
2.4.6 (I removed 'uninteresting' changes such as configure script
changing line numbers...)

[Most notable difference is the addiction of shutdown and the lowering
of the sleep time.]

- --
Lapo 'Raist' Luchini
lapo@lapo.it (PGP & X.509 keys available)
http://www.lapo.it (ICQ UIN: 529796)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAj9+atwACgkQaJiCLMjyUvuHMACeJjK59CeZrpe1/eWuiUloZd5Q
Y7MAoLr3BjhtJXRNfaYAMD6ReAZ/1mVp
=712L
-----END PGP SIGNATURE-----


[-- Attachment #2: rsync-2.4.5-6.diff --]
[-- Type: text/plain, Size: 22366 bytes --]

--- rsync-2.4.5/authenticate.c	2000-01-23 03:16:51.000000000 +0100
+++ rsync-2.4.6/authenticate.c	2000-08-19 17:25:05.000000000 +0200
@@ -135,7 +135,6 @@
 static char *getpassf(char *filename)
 {
 	char buffer[100];
-	int len=0;
 	int fd=0;
 	STRUCT_STAT st;
 	int ok = 1;
@@ -170,7 +169,7 @@
 	if (envpw) rprintf(FERROR,"RSYNC_PASSWORD environment variable ignored\n");
 
 	buffer[sizeof(buffer)-1]='\0';
-	if ( (len=read(fd,buffer,sizeof(buffer)-1)) > 0)
+	if (read(fd,buffer,sizeof(buffer)-1) > 0)
 	{
 		char *p = strtok(buffer,"\n\r");
 		close(fd);
diff -bu rsync-2.4.5/clientserver.c rsync-2.4.6/clientserver.c
--- rsync-2.4.5/clientserver.c	2000-08-04 23:26:17.000000000 +0200
+++ rsync-2.4.6/clientserver.c	2000-08-29 06:47:08.000000000 +0200
@@ -38,6 +38,11 @@
 	extern int am_sender;
 	extern struct in_addr socket_address;
 
+	if (argc == 0 && !am_sender) {
+		extern int list_only;
+		list_only = 1;
+	}
+
 	if (*path == '/') {
 		rprintf(FERROR,"ERROR: The remote path must start with a module name\n");
 		return -1;
diff -bu rsync-2.4.5/configure rsync-2.4.6/configure
--- rsync-2.4.5/configure	2000-04-19 07:33:39.000000000 +0200
+++ rsync-2.4.6/configure	2000-09-06 01:21:27.000000000 +0200
@@ -525,6 +525,7 @@
 
 # compile with optimisation and without debugging by default
 CFLAGS=${CFLAGS-"-O"}
+LDFLAGS=${LDFLAGS-""}
 
 ac_aux_dir=
 for ac_dir in $srcdir $srcdir/.. $srcdir/../..; do
@@ -654,10 +655,46 @@
     ac_cv_target_system_type="$target"
 
 
+# look for getconf early as this affects just about everything
+# Extract the first word of "getconf", so it can be a program name with args.
+set dummy getconf; ac_word=$2
+echo $ac_n "checking for $ac_word""... $ac_c" 1>&6
+echo "configure:663: checking for $ac_word" >&5
+if eval "test \"`echo '$''{'ac_cv_prog_HAVE_GETCONF'+set}'`\" = set"; then
+  echo $ac_n "(cached) $ac_c" 1>&6
+else
+  if test -n "$HAVE_GETCONF"; then
+  ac_cv_prog_HAVE_GETCONF="$HAVE_GETCONF" # Let the user override the test.
+else
+  IFS="${IFS= 	}"; ac_save_ifs="$IFS"; IFS=":"
+  ac_dummy="$PATH"
+  for ac_dir in $ac_dummy; do
+    test -z "$ac_dir" && ac_dir=.
+    if test -f $ac_dir/$ac_word; then
+      ac_cv_prog_HAVE_GETCONF="1"
+      break
+    fi
+  done
+  IFS="$ac_save_ifs"
+  test -z "$ac_cv_prog_HAVE_GETCONF" && ac_cv_prog_HAVE_GETCONF="0"
+fi
+fi
+HAVE_GETCONF="$ac_cv_prog_HAVE_GETCONF"
+if test -n "$HAVE_GETCONF"; then
+  echo "$ac_t""$HAVE_GETCONF" 1>&6
+else
+  echo "$ac_t""no" 1>&6
+fi
+
+if test $HAVE_GETCONF = 1; then
+	CFLAGS=$CFLAGS" "`getconf LFS_CFLAGS`
+	LDFLAGS=$LDFLAGS" "`getconf LFS_LDFLAGS`
+fi
+
 # Extract the first word of "gcc", so it can be a program name with args.
 set dummy gcc; ac_word=$2
 echo $ac_n "checking for $ac_word""... $ac_c" 1>&6
-echo "configure:661: checking for $ac_word" >&5
+echo "configure:698: checking for $ac_word" >&5
 if eval "test \"`echo '$''{'ac_cv_prog_CC'+set}'`\" = set"; then
   echo $ac_n "(cached) $ac_c" 1>&6
 else
@@ -3502,6 +3540,7 @@
 s%@build_cpu@%$build_cpu%g
 s%@build_vendor@%$build_vendor%g
 s%@build_os@%$build_os%g
+s%@HAVE_GETCONF@%$HAVE_GETCONF%g
 s%@CC@%$CC%g
 s%@INSTALL_PROGRAM@%$INSTALL_PROGRAM%g
 s%@INSTALL_SCRIPT@%$INSTALL_SCRIPT%g
diff -bu rsync-2.4.5/configure.in rsync-2.4.6/configure.in
--- rsync-2.4.5/configure.in	2000-04-19 07:33:39.000000000 +0200
+++ rsync-2.4.6/configure.in	2000-09-06 01:21:27.000000000 +0200
@@ -4,15 +4,24 @@
 
 # compile with optimisation and without debugging by default
 CFLAGS=${CFLAGS-"-O"}
+LDFLAGS=${LDFLAGS-""}
 
 AC_CANONICAL_SYSTEM
 AC_VALIDATE_CACHE_SYSTEM_TYPE
 
+# look for getconf early as this affects just about everything
+AC_CHECK_PROG(HAVE_GETCONF, getconf, 1, 0)
+if test $HAVE_GETCONF = 1; then
+	CFLAGS=$CFLAGS" "`getconf LFS_CFLAGS`
+	LDFLAGS=$LDFLAGS" "`getconf LFS_LDFLAGS`
+fi
+
 dnl Checks for programs.
 AC_PROG_CC
 AC_PROG_INSTALL
 AC_SUBST(SHELL)
 
+
 AC_CHECK_PROG(HAVE_REMSH, remsh, 1, 0)
 AC_DEFINE_UNQUOTED(HAVE_REMSH, $HAVE_REMSH)
 
diff -bu rsync-2.4.5/exclude.c rsync-2.4.6/exclude.c
--- rsync-2.4.5/exclude.c	2000-01-29 03:35:01.000000000 +0100
+++ rsync-2.4.6/exclude.c	2000-08-29 06:45:49.000000000 +0200
@@ -222,6 +222,12 @@
 {
 	int i;
 	extern int remote_version;
+	extern int list_only, recurse;
+
+	/* this is a complete hack - blame Rusty */
+	if (list_only && !recurse) {
+		add_exclude("/*/*", 0);
+	}
 
 	if (!exclude_list) {
 		write_int(f,0);
diff -bu rsync-2.4.5/flist.c rsync-2.4.6/flist.c
--- rsync-2.4.5/flist.c	2000-08-19 14:52:39.000000000 +0200
+++ rsync-2.4.6/flist.c	2000-09-01 01:01:28.000000000 +0200
@@ -55,7 +55,7 @@
 
 static void clean_flist(struct file_list *flist, int strip_root);
 
-struct string_area *string_area_new(int size)
+static struct string_area *string_area_new(int size)
 {
 	struct string_area *a;
 
@@ -65,12 +65,12 @@
 	a->current = a->base = malloc(size);
 	if (!a->current) out_of_memory("string_area_new buffer");
 	a->end = a->base + size;
-	a->next = 0;
+	a->next = NULL;
 
 	return a;
 }
 
-void string_area_free(struct string_area *a)
+static void string_area_free(struct string_area *a)
 {
 	struct string_area *next;
 
@@ -80,7 +80,7 @@
 	}
 }
 
-char *string_area_malloc(struct string_area **ap, int size)
+static char *string_area_malloc(struct string_area **ap, int size)
 {
 	char *p;
 	struct string_area *a;
@@ -100,7 +100,7 @@
 	return p;
 }
 
-char *string_area_strdup(struct string_area **ap, const char *src)
+static char *string_area_strdup(struct string_area **ap, const char *src)
 {
 	char* dest = string_area_malloc(ap, strlen(src) + 1);
 	return strcpy(dest, src);
@@ -534,7 +534,7 @@
 		if (lastdir && strcmp(fname, lastdir)==0) {
 			file->dirname = lastdir;
 		} else {
-			file->dirname = STRDUP(ap, fname);
+			file->dirname = strdup(fname);
 			lastdir = file->dirname;
 		}
 		file->basename = STRDUP(ap, p+1);
@@ -579,7 +579,7 @@
 		if (lastdir && strcmp(lastdir, flist_dir)==0) {
 			file->basedir = lastdir;
 		} else {
-			file->basedir = STRDUP(ap, flist_dir);
+			file->basedir = strdup(flist_dir);
 			lastdir = file->basedir;
 		}
 	} else {
@@ -1015,7 +1015,7 @@
 #if ARENA_SIZE > 0
 	flist->string_area = string_area_new(0);
 #else
-	flist->string_area = 0;
+	flist->string_area = NULL;
 #endif
 	return flist;
 }
diff -bu rsync-2.4.5/generator.c rsync-2.4.6/generator.c
--- rsync-2.4.5/generator.c	2000-08-04 23:11:46.000000000 +0200
+++ rsync-2.4.6/generator.c	2000-09-06 04:12:13.000000000 +0200
@@ -35,6 +35,7 @@
 extern int io_timeout;
 extern int remote_version;
 extern int always_checksum;
+extern int modify_window;
 extern char *compare_dest;
 
 
@@ -75,7 +76,7 @@
 		return 0;
 	}
 
-	return (st->st_mtime == file->modtime);
+	return (cmp_modtime(st->st_mtime,file->modtime) == 0);
 }
 
 
@@ -343,7 +344,7 @@
 		return;
 	}
 
-	if (update_only && st.st_mtime > file->modtime && fnamecmp == fname) {
+	if (update_only && cmp_modtime(st.st_mtime,file->modtime)>0 && fnamecmp == fname) {
 		if (verbose > 1)
 			rprintf(FINFO,"%s is newer\n",fname);
 		return;
diff -bu rsync-2.4.5/io.c rsync-2.4.6/io.c
--- rsync-2.4.5/io.c	2000-06-06 23:13:05.000000000 +0200
+++ rsync-2.4.6/io.c	2000-08-29 07:07:08.000000000 +0200
@@ -39,16 +39,10 @@
 extern int io_timeout;
 extern struct stats stats;
 
-static int buffer_f_in = -1;
 static int io_error_fd = -1;
 
 static void read_loop(int fd, char *buf, int len);
 
-void setup_readbuffer(int f_in)
-{
-	buffer_f_in = f_in;
-}
-
 static void check_timeout(void)
 {
 	extern int am_server, am_daemon;
@@ -323,11 +317,7 @@
 	return c;
 }
 
-
-
-/* write len bytes to fd, possibly reading from buffer_f_in if set
-   in order to unclog the pipe. don't return until all len
-   bytes have been written */
+/* write len bytes to fd */
 static void writefd_unbuffered(int fd,char *buf,int len)
 {
 	int total = 0;
@@ -382,6 +372,7 @@
 
 			if (ret == -1 && 
 			    (errno == EWOULDBLOCK || errno == EAGAIN)) {
+				msleep(1);
 				continue;
 			}
 
@@ -473,6 +464,17 @@
 	}
 }
 
+/* some OSes have a bug where an exit causes the pending writes on
+   a socket to be flushed. Do an explicit shutdown to try to prevent this */
+void io_shutdown(void)
+{
+	if (multiplex_out_fd != -1) close(multiplex_out_fd);
+	if (io_error_fd != -1) close(io_error_fd);
+	multiplex_out_fd = -1;
+	io_error_fd = -1;
+}
+
+
 static void writefd(int fd,char *buf,int len)
 {
 	stats.total_written += len;
@@ -623,8 +625,3 @@
 	io_multiplexing_out = 0;
 }
 
-void io_close_input(int fd)
-{
-	buffer_f_in = -1;
-}
-
Common subdirectories: rsync-2.4.5/lib and rsync-2.4.6/lib
diff -bu rsync-2.4.5/loadparm.c rsync-2.4.6/loadparm.c
--- rsync-2.4.5/loadparm.c	2000-08-19 14:53:00.000000000 +0200
+++ rsync-2.4.6/loadparm.c	2000-08-19 17:25:05.000000000 +0200
@@ -77,8 +77,6 @@
 	unsigned flags;
 };
 
-static BOOL bLoaded = False;
-
 #ifndef GLOBAL_NAME
 #define GLOBAL_NAME "global"
 #endif
@@ -734,8 +732,6 @@
 	iServiceIndex = -1;
 	bRetval = pm_process(n2, globals_only?NULL:do_section, do_parameter);
   
-	bLoaded = True;
-
 	return (bRetval);
 }
 
diff -bu rsync-2.4.5/main.c rsync-2.4.6/main.c
--- rsync-2.4.5/main.c	2000-08-19 14:53:24.000000000 +0200
+++ rsync-2.4.6/main.c	2000-08-29 06:46:50.000000000 +0200
@@ -349,14 +349,13 @@
 		close(recv_pipe[1]);
 		io_flush();
 		/* finally we go to sleep until our parent kills us
-		   with a USR2 signal. We sleepp for a short time as on
+		   with a USR2 signal. We sleep for a short time as on
 		   some OSes a signal won't interrupt a sleep! */
-		while (1) sleep(1);
+		while (1) msleep(20);
 	}
 
 	close(recv_pipe[1]);
 	close(error_pipe[1]);
-	io_close_input(f_in);
 	if (f_in != f_out) close(f_in);
 
 	io_start_buffering(f_out);
@@ -465,7 +464,6 @@
 	int status = 0, status2 = 0;
 	char *local_name = NULL;
 	extern int am_sender;
-	extern int list_only;
 	extern int remote_version;
 
 	set_nonblocking(f_in);
@@ -503,7 +501,10 @@
 		exit_cleanup(status);
 	}
 
-	if (argc == 0) list_only = 1;
+	if (argc == 0) {
+		extern int list_only;
+		list_only = 1;
+	}
 	
 	send_exclude_list(f_out);
 	
@@ -642,6 +643,11 @@
 		exit_cleanup(RERR_SYNTAX);
 	}
 	
+	if (argc == 0 && !am_sender) {
+		extern int list_only;
+		list_only = 1;
+	}
+	
 	pid = do_cmd(shell_cmd,shell_machine,shell_user,shell_path,&f_in,&f_out);
 	
 	ret = client_run(f_in, f_out, pid, argc, argv);
diff -bu rsync-2.4.5/options.c rsync-2.4.6/options.c
--- rsync-2.4.5/options.c	2000-06-23 15:50:18.000000000 +0200
+++ rsync-2.4.6/options.c	2000-09-06 04:12:13.000000000 +0200
@@ -67,6 +67,11 @@
 int only_existing=0;
 int max_delete=0;
 int ignore_errors=0;
+#ifdef _WIN32
+int modify_window=2;
+#else
+int modify_window=0;
+#endif
 int blocking_io=0;
 
 char *backup_suffix = BACKUP_SUFFIX;
@@ -85,6 +90,9 @@
 int always_checksum = 0;
 int list_only = 0;
 
+static int modify_window_set;
+
+
 struct in_addr socket_address = {INADDR_ANY};
 
 void usage(enum logcode F)
@@ -144,6 +152,7 @@
   rprintf(F,"     --timeout=TIME          set IO timeout in seconds\n");
   rprintf(F," -I, --ignore-times          don't exclude files that match length and time\n");
   rprintf(F,"     --size-only             only use file size when determining if a file should be transferred\n");
+  rprintf(F,"     --modify-window=NUM     Timestamp window (seconds) for file match (default=%d)\n",modify_window);
   rprintf(F," -T  --temp-dir=DIR          create temporary files in directory DIR\n");
   rprintf(F,"     --compare-dest=DIR      also compare destination files relative to DIR\n");
   rprintf(F," -P                          equivalent to --partial --progress\n");
@@ -178,7 +187,8 @@
       OPT_COPY_UNSAFE_LINKS, OPT_SAFE_LINKS, OPT_COMPARE_DEST,
       OPT_LOG_FORMAT, OPT_PASSWORD_FILE, OPT_SIZE_ONLY, OPT_ADDRESS,
       OPT_DELETE_AFTER, OPT_EXISTING, OPT_MAX_DELETE, OPT_BACKUP_DIR, 
-      OPT_IGNORE_ERRORS, OPT_BWLIMIT, OPT_BLOCKING_IO};
+      OPT_IGNORE_ERRORS, OPT_BWLIMIT, OPT_BLOCKING_IO,
+      OPT_MODIFY_WINDOW};
 
 static char *short_options = "oblLWHpguDCtcahvqrRIxnSe:B:T:zP";
 
@@ -200,6 +210,7 @@
   {"one-file-system",0,  0,    'x'},
   {"ignore-times",0,     0,    'I'},
   {"size-only",   0,     0,    OPT_SIZE_ONLY},
+  {"modify-window",1,    0,    OPT_MODIFY_WINDOW},
   {"help",        0,     0,    'h'},
   {"dry-run",     0,     0,    'n'},
   {"sparse",      0,     0,    'S'},
@@ -331,6 +342,11 @@
 			size_only = 1;
 			break;
 
+		case OPT_MODIFY_WINDOW:
+			modify_window = atoi(optarg);
+			modify_window_set = 1;
+			break;
+			
 		case 'x':
 			one_file_system=1;
 			break;
@@ -598,6 +614,7 @@
 	static char bsize[30];
 	static char iotime[30];
 	static char mdelete[30];
+	static char mwindow[30];
 	static char bw[50];
 
 	int i, x;
@@ -611,6 +628,7 @@
 	argstr[0] = '-';
 	for (i=0;i<verbose;i++)
 		argstr[x++] = 'v';
+
 	/* the -q option is intentionally left out */
 	if (make_backups)
 		argstr[x++] = 'b';
@@ -652,6 +670,14 @@
 		argstr[x++] = 'S';
 	if (do_compression)
 		argstr[x++] = 'z';
+
+	/* this is a complete hack - blame Rusty 
+
+	   this is a hack to make the list_only (remote file list)
+	   more useful */
+	if (list_only && !recurse) 
+		argstr[x++] = 'r';
+
 	argstr[x] = 0;
 
 	if (x != 1) args[ac++] = argstr;
@@ -690,6 +716,12 @@
 	if (size_only)
 		args[ac++] = "--size-only";
 
+	if (modify_window_set) {
+	        slprintf(mwindow,sizeof(mwindow),"--modify-window=%d",
+			 modify_window);
+		args[ac++] = mwindow;
+	}
+
 	if (keep_partial)
 		args[ac++] = "--partial";
 
Common subdirectories: rsync-2.4.5/packaging and rsync-2.4.6/packaging
diff -bu rsync-2.4.5/proto.h rsync-2.4.6/proto.h
--- rsync-2.4.5/proto.h	2000-08-19 14:53:38.000000000 +0200
+++ rsync-2.4.6/proto.h	2000-09-06 04:12:13.000000000 +0200
@@ -39,10 +39,6 @@
 struct map_struct *map_file(int fd,OFF_T len);
 char *map_ptr(struct map_struct *map,OFF_T offset,int len);
 void unmap_file(struct map_struct *map);
-struct string_area *string_area_new(int size);
-void string_area_free(struct string_area *a);
-char *string_area_malloc(struct string_area **ap, int size);
-char *string_area_strdup(struct string_area **ap, const char *src);
 int readlink_stat(const char *Path, STRUCT_STAT *Buffer, char *Linkbuf) ;
 int link_stat(const char *Path, STRUCT_STAT *Buffer) ;
 struct file_struct *make_file(int f, char *fname, struct string_area **ap,
@@ -62,7 +58,6 @@
 void init_hard_links(struct file_list *flist);
 int check_hard_link(struct file_struct *file);
 void do_hard_links(struct file_list *flist);
-void setup_readbuffer(int f_in);
 void io_set_error_fd(int fd);
 int32 read_int(int f);
 int64 read_longint(int f);
@@ -72,6 +67,7 @@
 void io_start_buffering(int fd);
 void io_flush(void);
 void io_end_buffering(int fd);
+void io_shutdown(void);
 void write_int(int f,int32 x);
 void write_longint(int f, int64 x);
 void write_buf(int f,char *buf,int len);
@@ -83,7 +79,6 @@
 int io_multiplex_write(enum logcode code, char *buf, int len);
 int io_error_write(int f, enum logcode code, char *buf, int len);
 void io_multiplexing_close(void);
-void io_close_input(int fd);
 char *lp_motd_file(void);
 char *lp_log_file(void);
 char *lp_pid_file(void);
@@ -214,5 +209,6 @@
 int unsafe_symlink(char *dest, char *src);
 char *timestring(time_t t);
 void msleep(int t);
+int cmp_modtime(time_t file1, time_t file2);
 int _Insure_trap_error(int a1, int a2, int a3, int a4, int a5, int a6);
 int sys_gettimeofday(struct timeval *tv);
diff -bu rsync-2.4.5/rsync.1 rsync-2.4.6/rsync.1
--- rsync-2.4.5/rsync.1	2000-08-19 15:04:48.000000000 +0200
+++ rsync-2.4.6/rsync.1	2000-09-06 04:12:13.000000000 +0200
@@ -252,7 +252,7 @@
  -r, --recursive             recurse into directories
  -R, --relative              use relative path names
  -b, --backup                make backups (default ~ suffix)
-     --backup-dir=DIR        put backups in the specified directory
+     --backup-dir            make backups into this directory
      --suffix=SUFFIX         override backup suffix
  -u, --update                update only (don\'t overwrite newer files)
  -l, --links                 preserve soft links
@@ -285,6 +285,7 @@
      --timeout=TIME          set IO timeout in seconds
  -I, --ignore-times          don\'t exclude files that match length and time
      --size-only             only use file size when determining if a file should be transferred
+     --modify-window=NUM     Timestamp window (seconds) for file match (default=0)
  -T  --temp-dir=DIR          create temporary files in directory DIR
      --compare-dest=DIR      also compare destination files relative to DIR
  -P                          equivalent to --partial --progress
@@ -353,6 +354,14 @@
 after using another mirroring system which may not preserve timestamps
 exactly\&.
 .IP 
+.IP "\fB--modify-window\fP" 
+When comparing two timestamps rsync treats
+the timestamps as being equal if they are within the value of
+modify_window\&. This is normally zero, but you may find it useful to
+set this to a larger value in some situations\&. In particular, when
+transferring to/from FAT filesystems which cannot represent times with
+a 1 second resolution this option is useful\&.
+.IP 
 .IP "\fB-c, --checksum\fP" 
 This forces the sender to checksum all files using
 a 128-bit MD4 checksum before transfer\&. The checksum is then
@@ -579,9 +588,9 @@
 .IP 
 .IP "\fB--exclude-from=FILE\fP" 
 This option is similar to the --exclude
-option, but instead it adds all filenames listed in the file FILE to
-the exclude list\&.  Blank lines in FILE and lines starting with \';\' or \'#\'
-are ignored\&.
+option, but instead it adds all exclude patterns listed in the file
+FILE to the exclude list\&.  Blank lines in FILE and lines starting with
+\';\' or \'#\' are ignored\&.
 .IP 
 .IP "\fB--include=PATTERN\fP" 
 This option tells rsync to not exclude the
diff -bu rsync-2.4.5/rsync.c rsync-2.4.6/rsync.c
--- rsync-2.4.5/rsync.c	2000-01-24 10:13:39.000000000 +0100
+++ rsync-2.4.6/rsync.c	2000-09-06 04:12:13.000000000 +0200
@@ -43,7 +43,7 @@
 
 
 /*
- * delete a file or directory. If force_delet is set then delete 
+ * delete a file or directory. If force_delete is set then delete 
  * recursively 
  */
 int delete_file(char *fname)
@@ -62,7 +62,6 @@
 	ret = do_stat(fname, &st);
 #endif
 	if (ret) {
-		rprintf(FERROR,"stat(%s) : %s\n", fname, strerror(errno));
 		return -1;
 	}
 
@@ -163,7 +162,7 @@
 	}
 
 	if (preserve_times && !S_ISLNK(st->st_mode) &&
-	    st->st_mtime != file->modtime) {
+	    cmp_modtime(st->st_mtime, file->modtime) != 0) {
 		/* don't complain about not setting times on directories
 		   because some filesystems can't do it */
 		if (set_modtime(fname,file->modtime) != 0 &&
diff -bu rsync-2.4.5/rsync.yo rsync-2.4.6/rsync.yo
--- rsync-2.4.5/rsync.yo	2000-07-29 06:41:19.000000000 +0200
+++ rsync-2.4.6/rsync.yo	2000-09-06 04:12:13.000000000 +0200
@@ -223,7 +223,7 @@
  -r, --recursive             recurse into directories
  -R, --relative              use relative path names
  -b, --backup                make backups (default ~ suffix)
-     --backup-dir=DIR        put backups in the specified directory
+     --backup-dir            make backups into this directory
      --suffix=SUFFIX         override backup suffix
  -u, --update                update only (don't overwrite newer files)
  -l, --links                 preserve soft links
@@ -256,6 +256,7 @@
      --timeout=TIME          set IO timeout in seconds
  -I, --ignore-times          don't exclude files that match length and time
      --size-only             only use file size when determining if a file should be transferred
+     --modify-window=NUM     Timestamp window (seconds) for file match (default=0)
  -T  --temp-dir=DIR          create temporary files in directory DIR
      --compare-dest=DIR      also compare destination files relative to DIR
  -P                          equivalent to --partial --progress
@@ -316,6 +317,13 @@
 after using another mirroring system which may not preserve timestamps
 exactly.
 
+dit(bf(--modify-window)) When comparing two timestamps rsync treats
+the timestamps as being equal if they are within the value of
+modify_window. This is normally zero, but you may find it useful to
+set this to a larger value in some situations. In particular, when
+transferring to/from FAT filesystems which cannot represent times with
+a 1 second resolution this option is useful.
+
 dit(bf(-c, --checksum)) This forces the sender to checksum all files using
 a 128-bit MD4 checksum before transfer. The checksum is then
 explicitly checked on the receiver and any files of the same name
@@ -497,9 +505,9 @@
 this option.
 
 dit(bf(--exclude-from=FILE)) This option is similar to the --exclude
-option, but instead it adds all filenames listed in the file FILE to
-the exclude list.  Blank lines in FILE and lines starting with ';' or '#'
-are ignored.
+option, but instead it adds all exclude patterns listed in the file
+FILE to the exclude list.  Blank lines in FILE and lines starting with
+';' or '#' are ignored.
 
 dit(bf(--include=PATTERN)) This option tells rsync to not exclude the
 specified pattern of filenames. This is useful as it allows you to
diff -bu rsync-2.4.5/sender.c rsync-2.4.6/sender.c
--- rsync-2.4.5/sender.c	2000-01-23 13:30:35.000000000 +0100
+++ rsync-2.4.6/sender.c	2000-08-19 17:25:05.000000000 +0200
@@ -97,8 +97,6 @@
 	if (verbose > 2)
 		rprintf(FINFO,"send_files starting\n");
 
-	setup_readbuffer(f_in);
-
 	while (1) {
 		int offset=0;
 
diff -bu rsync-2.4.5/util.c rsync-2.4.6/util.c
--- rsync-2.4.5/util.c	2000-08-19 14:53:51.000000000 +0200
+++ rsync-2.4.6/util.c	2000-09-06 04:12:13.000000000 +0200
@@ -955,7 +955,27 @@
 }
 
 
-#ifdef __INSURE__
+/*******************************************************************
+ Determine if two file modification times are equivalent (either exact 
+ or in the modification timestamp window established by --modify-window) 
+ Returns 0 if the times should be treated as the same, 1 if the 
+ first is later and -1 if the 2nd is later
+ *******************************************************************/
+int cmp_modtime(time_t file1, time_t file2)
+{
+	time_t diff;
+	extern int modify_window;
+
+	if (file2 > file1) {
+		if (file2 - file1 <= modify_window) return 0;
+		return -1;
+	}
+	if (file1 - file2 <= modify_window) return 0;
+	return 1;
+}
+
+
+#ifdef __INSURE__XX
 #include <dlfcn.h>
 
 /*******************************************************************


[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Cygwin/rsync Hang Problem Testing Results

[Mark Thornton]

Paul Thompson wrote:

> People of cygwin & rsync,
>
> I recently attempted to get cygwin and rsync working to solve a 
> backup/mirroring need in my computer life. Well, as you might guess, I 
> ran into a little but of trouble.

...

> So, based on my testing, I would conclude that the infamous cygwin 
> rsync hang bug was introduced somewhere in the coding for 2.4.6. As 
> for myself, I plan to go ahead and just use version 2.4.0, as it is 
> the most recent version of rsync that worked for all three tests. If I 
> can be of any more help, including testing patches, please let me know.
>

Can you suggest a download source for 2.4.0? I have only been able to 
find 2.4.6, 2.5.6-1 and 2.5.6-2. By the way were either of your machines 
dual processors (or hyperthreaded)?

Mark Thornton

> Thanks for all of the hard work on cygwin and rsync, folks.
>
> Sincerely,
> Paul Thompson
> captbunzo@squirrelmail.org
>
>
> -- 
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>
>



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Cygwin/rsync Hang Problem Testing Results

[Paul Thompson]

Mark Thornton wrote:

>> So, based on my testing, I would conclude that the infamous cygwin 
>> rsync hang bug was introduced somewhere in the coding for 2.4.6. As 
>> for myself, I plan to go ahead and just use version 2.4.0, as it is 
>> the most recent version of rsync that worked for all three tests. If 
>> I can be of any more help, including testing patches, please let me 
>> know.
>>
>
> Can you suggest a download source for 2.4.0? I have only been able to 
> find 2.4.6, 2.5.6-1 and 2.5.6-2. By the way were either of your 
> machines dual processors (or hyperthreaded)?


rsync ftp site -

    http://rsync.samba.org/ftp/rsync/old-versions/rsync-2.4.0.tar.gz
    http://rsync.samba.org/ftp/rsync/old-versions/

And one of my machines is dual processored. However, the machine that I 
did all of my testing on was a normal Pentium-4 laptop. No 
hyperthreading, I think...

Paul


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Cygwin/rsync Hang Problem Testing Results

[Mark Thornton]

Paul Thompson wrote:

> People of cygwin & rsync,
>
> I recently attempted to get cygwin and rsync working to solve a 
> backup/mirroring need in my computer life. Well, as you might guess, I 
> ran into a little but of trouble.
>
> Strangely enough, rsync seemed to be regularly hanging when I 
> attempted to do a "get" (sycronize a remote to a local dir). Well, 
> considering I want to automate this, that was not going to work. So I 
> searched the web, mailing lists, etc, and came across all of the posts 
> on the subject, but no solution.
>
> Then I stumbled across an idea - to test previous versions of rsync to 
> if any of them worked better with cygwin. Here are the (hopefully 
> helpful) results of those tests. I did three tests for each version of 
> cygwin:
>
>
<snip>

> So, based on my testing, I would conclude that the infamous cygwin 
> rsync hang bug was introduced somewhere in the coding for 2.4.6. As 
> for myself, I plan to go ahead and just use version 2.4.0, as it is 
> the most recent version of rsync that worked for all three tests. If I 
> can be of any more help, including testing patches, please let me know.


I tried 2.4.0 and seemed to work for a while, but now with more 
differences to transfer I have encountered the "connection reset by 
peer" error with this version also. Perhaps the subsequent changes 
didn't introduce the bug but merely changed the circumstances in which 
it appears (very easy if it is threading related).

Regards,
Mark Thornton


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/