From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 118810 invoked by alias); 4 Jan 2020 22:15:57 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 118802 invoked by uid 89); 4 Jan 2020 22:15:57 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=1.6 required=5.0 tests=AWL,BAYES_20,FOREIGN_BODY,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=no version=3.3.1 spammy=videos, diese, wurde, mit X-HELO: ms94429.hosts.iptox.net Received: from ms94429.hosts.iptox.net (HELO ms94429.hosts.iptox.net) (95.128.200.216) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Sat, 04 Jan 2020 22:15:55 +0000 Received: from ms94429.hosts.iptox.net (unknown [127.0.0.1]) by ms94429.hosts.iptox.net (Postfix) with ESMTP id A3383EE4F55 for ; Sat, 4 Jan 2020 22:15:53 +0000 (UTC) Received: from [100.101.50.80] (ip-109-42-2-224.web.vodafone.de [109.42.2.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ms94429.hosts.iptox.net (Postfix) with ESMTPSA id 73FE4EE297A for ; Sat, 4 Jan 2020 23:15:53 +0100 (CET) Date: Sat, 04 Jan 2020 22:15:00 -0000 User-Agent: K-9 Mail for Android MIME-Version: 1.0 Subject: rsync and ls -lR slow for directories with many files To: cygwin@cygwin.com From: muell@plutonium24.de Message-ID: <8582CD6F-C872-41FB-9425-2CBD1126AE33@plutonium24.de> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2020-01/txt/msg00039.txt.bz2 I am running rsync on a small linux server to synchronize files in one dire= ctory and its subdirectories from Windows (using sshd from Cygwin) to this = server for backup purposes. The directory contains almost 1 TB of images an= d videos in about 160k files on a slow disk (Seagate Archive 8TB with SMR) = with NTFS. Even if there are no changes and whith whole file transfers rsync takes abo= ut 45 minutes to come to this conclusion. I am using the following command line on the linux server: rsync -avx --stats --whole-file --no-perms --no-owner --no-group @: As rsync was only transferring a small number of bytes and gave no clue to = the cause for being so slow and as rsync should only need filenames, dates = and sizes I did a "ls -lR|wc" on both systems. On the linux server this too= k about 1 minute (only slightly faster magnetic disk, empty read cache at s= tart) and doing the same on cygwin took almost as long as rsync (over 40 mi= nutes). Using Windows Explorer (after a reboot to guarantee that the cache = is empty) to get the total number of files and the total size took only a f= ew seconds. Reading all file sizes with Treesize also took less than one mi= nute. As ls -lR needs the same information I would have expected it to take= the same time. Runnin "ls -lR" a second time on Cygwin is fast as lightning as it only tak= es less than 30s. Is there any way to get ls -lR or better rsync as fast as listing the direc= tory with Windows tools? Frank --=20 Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. --=20 Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple