From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 55654 invoked by alias); 11 Apr 2016 16:58:35 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 55642 invoked by uid 89); 11 Apr 2016 16:58:34 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_40,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=on-topic, ontopic, transfers, experienced X-HELO: mail-lf0-f49.google.com Received: from mail-lf0-f49.google.com (HELO mail-lf0-f49.google.com) (209.85.215.49) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Mon, 11 Apr 2016 16:58:33 +0000 Received: by mail-lf0-f49.google.com with SMTP id j11so165446684lfb.1 for ; Mon, 11 Apr 2016 09:58:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Glhi4o2uTRhk7hMDGT8InrkEKYZINGVQKRrJIClVjGQ=; b=YMo/ntrqjpory0oKtrgdi31ToDwkZ0RAi9rnqAEPgYOip3JpJf7VO0G6NAXBF1386Z B+OfFTNRpwozhkfJ0DQnZ3jXxhhu0yjkGdE/onhkeKPKcj78/VEbUGYFdagK8wZ3YvQs N+pcvEuNsN9IxtuRmF1IhIuj/rgdmmv0VgGgMnLsfhX4p88z9SEtqHRxl3wH863M0CTF pdMzAiNyUoOoiBh7s4Pt/e56BJrudbpNlUwp8fjTbNmVm1wsBGL8BcDbx+0TVFwJlDoX 94P6H1f0+Uwi+p4zR6EyMpAOwhD5oSzFDGV8mEBN0sz5AEHH8vh2PS7dceZt4yuHi1Xt 4PIg== X-Gm-Message-State: AD7BkJL0PJIUY2g0xyWkd0+u3di+dUkQMSnN8mdDO4OD9Uzn0uodX4hKcN0AjIIuVM28fAeO1fegThyPvLLKHA== X-Received: by 10.112.188.134 with SMTP id ga6mr8825315lbc.77.1460393910234; Mon, 11 Apr 2016 09:58:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.21.67 with HTTP; Mon, 11 Apr 2016 09:57:50 -0700 (PDT) In-Reply-To: References: From: Greg Freemyer Date: Mon, 11 Apr 2016 16:58:00 -0000 Message-ID: Subject: Re: rsync performance degradation? Could be a windows issue? To: cygwin@cygwin.com Content-Type: text/plain; charset=UTF-8 X-IsSubscribed: yes X-SW-Source: 2016-04/txt/msg00208.txt.bz2 On Thu, Mar 31, 2016 at 4:48 PM, Greg Freemyer wrote: > All, > > I'm not sure this is on-topic, but at least I'm in cygwin when I > notice the below: > > I do a lot of large data transfers between USB drives. Often I get > great speeds (70MB/sec or more). > > Sometimes it falls way off to closer to 20MB/sec with the same class > of hardware. > > I experienced the "slow" transfer speed today, so I thought I'd ask if > anyone knew a way to resolve it? I'm trying rsync of a bunch of large files again today. Initially I saw the same slow copy speed (about 20 MB/sec). My destination was connected to a USB 3.1 port, but it seemed to be the bottleneck so I moved it to a USB 3 port (3.1 should be 2x the speed of 3.0, so it should not have been the bottleneck). After getting my throughput up above 50 MB/sec I notice in the windows "performance monitor" that my source disk is hitting 100% utilization, then a few seconds later my destination disk is. And back and forth. It seems I'm only reading or writing for a few seconds, then alternating. I assume the issue is that too much data is being read / cached by rsync prior to it being written out so I'm getting no advantage of reading and writing in parallel. (I gather iostat isn't available for cygwin?) Is there some rsync (or cygwin) option / feature that would encourage parallel reading/writing? fyi: I did some linux testing with "dd" over the weekend and I hit 140 MB/sec if I used a 100MB blocksize. That was to / from the raw disk (/dev/sda => /dev/sdb). If I bumped my blocksize to 1GB for dd in linux, my throughput dropped to 70 MB/sec just as I see right now with rsync in cygwin. Thanks Greg -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple