* rsync performance degradation? Could be a windows issue? @ 2016-03-31 20:49 Greg Freemyer 2016-04-11 16:58 ` Greg Freemyer 2016-04-11 23:20 ` Vincent Rivière 0 siblings, 2 replies; 6+ messages in thread From: Greg Freemyer @ 2016-03-31 20:49 UTC (permalink / raw) To: cygwin All, I'm not sure this is on-topic, but at least I'm in cygwin when I notice the below: I do a lot of large data transfers between USB drives. Often I get great speeds (70MB/sec or more). Sometimes it falls way off to closer to 20MB/sec with the same class of hardware. I experienced the "slow" transfer speed today, so I thought I'd ask if anyone knew a way to resolve it? The PC I'm running on is spec'ed for data processing so a 6 core (12 thread) setup with 128GB or RAM. No significant data processing is going on. Basically the machine is currently dedicated to running rsync (as below). I have 2 USB drives hooked up. Both are USB-3 and should be able to do 70MB/sec with no issue. Neither drive has any unrelated I/O going on that I know of. (I don't see any in the Windows Performance Monitor.) I have a folder with almost 200 4GB files ( 750GB total and all the files are large). I'm using "rsync -av" to copy files from the source to the destination and the folder on the destination started out empty Using Windows "Resource Monitor" I normally would see both drives hitting over 70 MB/sec. Today I'm seeing the source drive jump all the way up to 125 MB/sec or so, but the destination is down around 20 MB/sec When the 4GB source file completes reading, the source drive basically goes idle for a while until the file is written out. Then the source drive jumps back to 125 MB/sec until it finishes reading the next file. The destination drive on the other hand is stuck down around 20 MB/sec. I really don't think this is caused by slow hardware. Could the destination drive be dropping down to USB-2 mode? === And now that I've written up the above, the system has returned to a more normal 75 MB/sec consistently on the output drive. I'm still curious what might be going on. As I say I move a lot of data around as part of my work, and much of the time it is big groups of either 1.5GB or 4GB files. Thanks Greg -- Greg Freemyer www.IntelligentAvatar.net -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rsync performance degradation? Could be a windows issue? 2016-03-31 20:49 rsync performance degradation? Could be a windows issue? Greg Freemyer @ 2016-04-11 16:58 ` Greg Freemyer 2016-04-11 17:35 ` Marco Atzeri 2016-04-11 23:20 ` Vincent Rivière 1 sibling, 1 reply; 6+ messages in thread From: Greg Freemyer @ 2016-04-11 16:58 UTC (permalink / raw) To: cygwin On Thu, Mar 31, 2016 at 4:48 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote: > All, > > I'm not sure this is on-topic, but at least I'm in cygwin when I > notice the below: > > I do a lot of large data transfers between USB drives. Often I get > great speeds (70MB/sec or more). > > Sometimes it falls way off to closer to 20MB/sec with the same class > of hardware. > > I experienced the "slow" transfer speed today, so I thought I'd ask if > anyone knew a way to resolve it? I'm trying rsync of a bunch of large files again today. Initially I saw the same slow copy speed (about 20 MB/sec). My destination was connected to a USB 3.1 port, but it seemed to be the bottleneck so I moved it to a USB 3 port (3.1 should be 2x the speed of 3.0, so it should not have been the bottleneck). After getting my throughput up above 50 MB/sec I notice in the windows "performance monitor" that my source disk is hitting 100% utilization, then a few seconds later my destination disk is. And back and forth. It seems I'm only reading or writing for a few seconds, then alternating. I assume the issue is that too much data is being read / cached by rsync prior to it being written out so I'm getting no advantage of reading and writing in parallel. (I gather iostat isn't available for cygwin?) Is there some rsync (or cygwin) option / feature that would encourage parallel reading/writing? fyi: I did some linux testing with "dd" over the weekend and I hit 140 MB/sec if I used a 100MB blocksize. That was to / from the raw disk (/dev/sda => /dev/sdb). If I bumped my blocksize to 1GB for dd in linux, my throughput dropped to 70 MB/sec just as I see right now with rsync in cygwin. Thanks Greg -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rsync performance degradation? Could be a windows issue? 2016-04-11 16:58 ` Greg Freemyer @ 2016-04-11 17:35 ` Marco Atzeri 2016-04-11 18:21 ` Greg Freemyer 0 siblings, 1 reply; 6+ messages in thread From: Marco Atzeri @ 2016-04-11 17:35 UTC (permalink / raw) To: cygwin On 11/04/2016 18:57, Greg Freemyer wrote: > On Thu, Mar 31, 2016 at 4:48 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote: >> All, >> >> I'm not sure this is on-topic, but at least I'm in cygwin when I >> notice the below: >> >> I do a lot of large data transfers between USB drives. Often I get >> great speeds (70MB/sec or more). >> >> Sometimes it falls way off to closer to 20MB/sec with the same class >> of hardware. >> >> I experienced the "slow" transfer speed today, so I thought I'd ask if >> anyone knew a way to resolve it? > > I'm trying rsync of a bunch of large files again today. > > Initially I saw the same slow copy speed (about 20 MB/sec). My > destination was connected to a USB 3.1 port, but it seemed to be the > bottleneck so I moved it to a USB 3 port (3.1 should be 2x the speed > of 3.0, so it should not have been the bottleneck). > > After getting my throughput up above 50 MB/sec I notice in the windows > "performance monitor" that my source disk is hitting 100% utilization, > then a few seconds later my destination disk is. And back and forth. > It seems I'm only reading or writing for a few seconds, then > alternating. > > I assume the issue is that too much data is being read / cached by > rsync prior to it being written out so I'm getting no advantage of > reading and writing in parallel. > > (I gather iostat isn't available for cygwin?) > > Is there some rsync (or cygwin) option / feature that would encourage > parallel reading/writing? > > fyi: I did some linux testing with "dd" over the weekend and I hit 140 > MB/sec if I used a 100MB blocksize. That was to / from the raw disk > (/dev/sda => /dev/sdb). > > If I bumped my blocksize to 1GB for dd in linux, my throughput dropped > to 70 MB/sec just as I see right now with rsync in cygwin. > > Thanks > Greg > Is windows robocopy faster ? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rsync performance degradation? Could be a windows issue? 2016-04-11 17:35 ` Marco Atzeri @ 2016-04-11 18:21 ` Greg Freemyer 2016-04-11 23:08 ` Greg Freemyer 0 siblings, 1 reply; 6+ messages in thread From: Greg Freemyer @ 2016-04-11 18:21 UTC (permalink / raw) To: cygwin On Mon, Apr 11, 2016 at 1:34 PM, Marco Atzeri <marco.atzeri@gmail.com> wrote: > > > On 11/04/2016 18:57, Greg Freemyer wrote: >> >> On Thu, Mar 31, 2016 at 4:48 PM, Greg Freemyer <greg.freemyer@gmail.com> >> wrote: >>> >>> All, >>> >>> I'm not sure this is on-topic, but at least I'm in cygwin when I >>> notice the below: >>> >>> I do a lot of large data transfers between USB drives. Often I get >>> great speeds (70MB/sec or more). >>> >>> Sometimes it falls way off to closer to 20MB/sec with the same class >>> of hardware. >>> >>> I experienced the "slow" transfer speed today, so I thought I'd ask if >>> anyone knew a way to resolve it? >> >> >> I'm trying rsync of a bunch of large files again today. >> >> Initially I saw the same slow copy speed (about 20 MB/sec). My >> destination was connected to a USB 3.1 port, but it seemed to be the >> bottleneck so I moved it to a USB 3 port (3.1 should be 2x the speed >> of 3.0, so it should not have been the bottleneck). >> >> After getting my throughput up above 50 MB/sec I notice in the windows >> "performance monitor" that my source disk is hitting 100% utilization, >> then a few seconds later my destination disk is. And back and forth. >> It seems I'm only reading or writing for a few seconds, then >> alternating. >> >> I assume the issue is that too much data is being read / cached by >> rsync prior to it being written out so I'm getting no advantage of >> reading and writing in parallel. >> >> (I gather iostat isn't available for cygwin?) >> >> Is there some rsync (or cygwin) option / feature that would encourage >> parallel reading/writing? >> >> fyi: I did some linux testing with "dd" over the weekend and I hit 140 >> MB/sec if I used a 100MB blocksize. That was to / from the raw disk >> (/dev/sda => /dev/sdb). >> >> If I bumped my blocksize to 1GB for dd in linux, my throughput dropped >> to 70 MB/sec just as I see right now with rsync in cygwin. >> >> Thanks >> Greg >> > > > Is windows robocopy faster ? As I test, I just copied 30 GB of 1.5 GB files via robocopy. None of the files had been accessed since a reboot, so none should have been in cache. According to Resource Monitor, yes. I'm getting about 105 MB/sec for read and 105 MB/sec for write vs 70 MB/sec for rsync. Significantly, with robocopy Resource Monitor is showing a consistent read/write speed. With rsync read and write fluctuate back and forth. As I stated before, I believe I should be able to get 140 MB/sec, but I wouldn't complain about 105 MB/sec. 70 MB/sec just seems to highlight a flaw in how rsync manages the data flow since it is 50% of theoretical max. ie. this seems to be what rsync in cygwin is doing: while (files) { read 1.5 GB file to ram write 1.5 GB file from ram fsync() ensure 1.5 GB file is on disk } endwhile I haven't tested in linux. Maybe rsync just isn't as efficient as I expect? There is also the problem that rsync often slows down to below 50 MB/sec I've tried it with no other activity on the PC. Today I was only getting 20 MB/sec at first. (I moved the USB cable and it came up to 70 MB/sec). Thanks Greg -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rsync performance degradation? Could be a windows issue? 2016-04-11 18:21 ` Greg Freemyer @ 2016-04-11 23:08 ` Greg Freemyer 0 siblings, 0 replies; 6+ messages in thread From: Greg Freemyer @ 2016-04-11 23:08 UTC (permalink / raw) To: cygwin On Mon, Apr 11, 2016 at 2:20 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote: > As I test, I just copied 30 GB of 1.5 GB files via robocopy. None of > the files had been accessed since a reboot, so none should have been > in cache. > > According to Resource Monitor, yes. > > I'm getting about 105 MB/sec for read and 105 MB/sec for write vs 70 > MB/sec for rsync. I just moved both drives to a Linux box. Using the same rsync command I'm getting 85 - 90 MB/sec. That linux box is a 5-year old laptop, where as the windows box is a pretty beefy box. (6 cores, 64 GB ram) Clearly the WIndows box should be better performing and when running robocopy it is. So it seems something about the combination of a Windows kernel, cygwin, and rsync is dropping my performance at least a factor 20% and maybe as much as 35%. If I were to run rsync in strace, would anyone want to try and work on the performance issue? Or should it just be chalked up to rsync being optimized for a linux kernel? Thanks Greg -- Greg Freemyer www.IntelligentAvatar.net -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rsync performance degradation? Could be a windows issue? 2016-03-31 20:49 rsync performance degradation? Could be a windows issue? Greg Freemyer 2016-04-11 16:58 ` Greg Freemyer @ 2016-04-11 23:20 ` Vincent Rivière 1 sibling, 0 replies; 6+ messages in thread From: Vincent Rivière @ 2016-04-11 23:20 UTC (permalink / raw) To: cygwin On 31/03/2016 22:48, Greg Freemyer wrote: > I experienced the "slow" transfer speed today, I also experience similar slowness when using the current Cygwin rsync to backup my internal hard drives to external USB3 hard drive. Because of that, I continue using the old official Cygwin rsync 3.0.8 binary. I get correct speed with this one. If that could matter, I use Windows XP and Cygwin 32. -- Vincent Rivière -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-04-11 23:20 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-03-31 20:49 rsync performance degradation? Could be a windows issue? Greg Freemyer 2016-04-11 16:58 ` Greg Freemyer 2016-04-11 17:35 ` Marco Atzeri 2016-04-11 18:21 ` Greg Freemyer 2016-04-11 23:08 ` Greg Freemyer 2016-04-11 23:20 ` Vincent Rivière
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).