public inbox for cygwin@cygwin.com
 help / color / mirror / Atom feed
From: Eliot Moss <moss@cs.umass.edu>
To: cygwin@kosowsky.org, cygwin@cygwin.com
Subject: Re: Renaming (with 'mv') very large files is SLOW
Date: Mon, 31 Jan 2022 09:59:17 -0500	[thread overview]
Message-ID: <c1e4529a-d119-1662-6eeb-2d42950802c7@cs.umass.edu> (raw)
In-Reply-To: <25079.63376.349893.86395@consult.pretender>

On 1/31/2022 9:52 AM, cygwin@kosowsky.org wrote:
 > I tried renaming some very large files (20-40 GB) using:
 >     mv <oldname> <newname>
 > without changing the directory of course.
 >
 > The process took about 10-20 minutes with Task Manager showing disk
 > activity of 100+ MB/s.
 >
 > Is there something about such large 'renaming' that actually results
 > in the file being really moved (aka copied) rather than just renamed?

The two places are probably on different volumes (loosely, different disks).
That requires a physical move, even under Linux.  Your volumes seem a bit slow
to access - is one perhaps across a slow network?  The rates you cite suggest
movement of 50Mb/s (50Mb read + 50Mb write = 100Mb overall).  For 40 Gb that
should take 40Gb / 50Mb = about 820 secs = a little under 14 mins.

(When I say your volumes are slow, I speak from the luxury of having a 2Tb
solid state drive!  Actually, those speeds may be reasonable depending on the
nature of your system.)

If the two locations are on different drives, there's no real avoiding this.

Regards - Eliot Moss

  reply	other threads:[~2022-01-31 14:59 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-31 14:52 cygwin
2022-01-31 14:59 ` Eliot Moss [this message]
2022-01-31 15:13   ` René Berber
2022-01-31 15:20     ` cygwin
2022-01-31 15:51       ` [EXTERNAL] " Lavrentiev, Anton (NIH/NLM/NCBI) [C]
2022-01-31 15:17   ` cygwin
2022-01-31 21:36     ` Adam Dinwoodie
2022-02-02 17:09       ` L A Walsh
2022-02-01  8:47     ` Andrey Repin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c1e4529a-d119-1662-6eeb-2d42950802c7@cs.umass.edu \
    --to=moss@cs.umass.edu \
    --cc=cygwin@cygwin.com \
    --cc=cygwin@kosowsky.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).