* Faster rsync? @ 2023-08-29 12:32 Adam Kessel 2023-08-29 13:17 ` Mario Emmenlauer 0 siblings, 1 reply; 4+ messages in thread From: Adam Kessel @ 2023-08-29 12:32 UTC (permalink / raw) To: cygwin I've found rsync to be painfully slow on large folders -- hours to sync thousands of files, even when they already match size and --size-only is used. It's much faster between native Linux boxes. Is there any trick to improving performance? I've been looking for a native version for Windows, but it doesn't appear to exist -- as best I can tell, https://acrosync.com/windows.html is just the cygwin version packaged up. From past experience, WSL rsync is much faster, but I don't want to install WSL on this box. Any ideas? ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Faster rsync? 2023-08-29 12:32 Faster rsync? Adam Kessel @ 2023-08-29 13:17 ` Mario Emmenlauer 2023-08-29 13:43 ` Eliot Moss 0 siblings, 1 reply; 4+ messages in thread From: Mario Emmenlauer @ 2023-08-29 13:17 UTC (permalink / raw) To: Adam Kessel, cygwin On 29.08.23 14:32, Adam Kessel via Cygwin wrote: > I've found rsync to be painfully slow on large folders -- hours to sync thousands of files, even when they already match size and --size-only is used. It's much > faster between native Linux boxes. I've been using rsync, unison and similar tools on Windows and Linux since basically forever. In my humble opinion, the problem is the Windows file system performance, not the synchronization tools. As a separate example, try to download the boost source code, and extract the archive. I can do the extraction in way under a minute on Linux, but have to wait many many minutes on a similarly equipped Windows machine. Just my two cents. Mario ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Faster rsync? 2023-08-29 13:17 ` Mario Emmenlauer @ 2023-08-29 13:43 ` Eliot Moss 2023-08-29 17:55 ` Eliot Moss 0 siblings, 1 reply; 4+ messages in thread From: Eliot Moss @ 2023-08-29 13:43 UTC (permalink / raw) To: Mario Emmenlauer, Adam Kessel, cygwin On 8/29/2023 9:17 AM, Mario Emmenlauer via Cygwin wrote: > On 29.08.23 14:32, Adam Kessel via Cygwin wrote: >> I've found rsync to be painfully slow on large folders -- hours to sync thousands of files, even >> when they already match size and --size-only is used. It's much faster between native Linux boxes. I've been told (on this list) that how you mount the Windows drive in Cygwin matters. I use this in my /etc/fstab: d:/ /cygdrive/d ntfs binary,posix=0,user,auto,notexec 0 0 Unfortunately, it's still not as quick as I think it ought to be. (If someone out there can recommend better settings, I'm open!) I regularly rsync a collection of hundreds of files, each about 2Gb in size. Windows describes the partition as an NTFS Basic Data Partition. It takes rsync a long time to get started (i.e., to figure out which files need transferring). My rsync uses these flags: rsync -avuP --partial --partial-dir=.rsync-partial --log-file=... --log-file-format=... > I've been using rsync, unison and similar tools on Windows and Linux since basically forever. In my > humble opinion, the problem is the Windows file system performance, not the synchronization tools. > As a separate example, try to download the boost source code, and extract the archive. I can do the > extraction in way under a minute on Linux, but have to wait many many minutes on a similarly > equipped Windows machine. > > Just my two cents. > > Mario > Regards - Eliot Moss ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Faster rsync? 2023-08-29 13:43 ` Eliot Moss @ 2023-08-29 17:55 ` Eliot Moss 0 siblings, 0 replies; 4+ messages in thread From: Eliot Moss @ 2023-08-29 17:55 UTC (permalink / raw) To: Mario Emmenlauer, Adam Kessel, cygwin On 8/29/2023 9:43 AM, Eliot Moss wrote: > On 8/29/2023 9:17 AM, Mario Emmenlauer via Cygwin wrote: >> On 29.08.23 14:32, Adam Kessel via Cygwin wrote: >>> I've found rsync to be painfully slow on large folders -- hours to sync thousands of files, even >>> when they already match size and --size-only is used. It's much faster between native Linux boxes. > > I've been told (on this list) that how you mount the Windows drive in Cygwin matters. > > I use this in my /etc/fstab: > > d:/ /cygdrive/d ntfs binary,posix=0,user,auto,notexec 0 0 > > Unfortunately, it's still not as quick as I think it ought to be. > (If someone out there can recommend better settings, I'm open!) > > I regularly rsync a collection of hundreds of files, each about 2Gb in size. > > Windows describes the partition as an NTFS Basic Data Partition. It takes > rsync a long time to get started (i.e., to figure out which files need > transferring). > > My rsync uses these flags: > > rsync -avuP --partial --partial-dir=.rsync-partial --log-file=... --log-file-format=... These flags seems to give speedy directory scanning: rsync -r --chmod=D2700,F600 -t -z -p -v --size-only --partial-dir=.rsync-partial Regards - EM ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-08-29 17:55 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-08-29 12:32 Faster rsync? Adam Kessel 2023-08-29 13:17 ` Mario Emmenlauer 2023-08-29 13:43 ` Eliot Moss 2023-08-29 17:55 ` Eliot Moss
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).