rsync and ls -lR slow for directories with many files

[muell@plutonium24.de]

I am running rsync on a small linux server to synchronize files in one directory and its subdirectories from Windows (using sshd from Cygwin) to this server for backup purposes. The directory contains almost 1 TB of images and videos in about 160k files on a slow disk (Seagate Archive 8TB with SMR) with NTFS.

Even if there are no changes and whith whole file transfers rsync takes about 45 minutes to come to this conclusion.
I am using the following command line on the linux server:

rsync -avx --stats --whole-file --no-perms --no-owner --no-group <user>@<server>:<source directory> <local destination directory>

As rsync was only transferring a small number of bytes and gave no clue to the cause for being so slow and as rsync should only need filenames, dates and sizes I did a "ls -lR|wc" on both systems. On the linux server this took about 1 minute (only slightly faster magnetic disk, empty read cache at start) and doing the same on cygwin took almost as long as rsync (over 40 minutes). Using Windows Explorer (after a reboot to guarantee that the cache is empty) to get the total number of files and the total size took only a few seconds. Reading all file sizes with Treesize also took less than one minute. As ls -lR needs the same information I would have expected it to take the same time.

Runnin "ls -lR" a second time on Cygwin is fast as lightning as it only takes less than 30s.

Is there any way to get ls -lR or better rsync as fast as listing the directory with Windows tools?

Frank

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple