From: Brian Inglis <Brian.Inglis@SystematicSw.ab.ca>
To: cygwin-apps@cygwin.com
Subject: Re: scallywag / cygport not pulling lzip
Date: Mon, 11 Jan 2021 15:05:05 -0700 [thread overview]
Message-ID: <6bbcf804-593b-16be-dfd1-e1430c1c4bf8@SystematicSw.ab.ca> (raw)
In-Reply-To: <871rev9rbx.fsf@Rainer.invalid>
On 2021-01-08 12:11, Achim Gratz wrote:
> Brian Inglis writes:
>> Do we know what the frequency weighted difference is on bandwidth of
>> packages actually downloaded?
>
> Not that I know of, as everything goes through mirrors. But I happen to
> have a complete Cygwin mirror on disk at the moment plus another one
> that only has the packages for my install and that's a fairly large
> installation, but without a desktop environment:
>
> 30G /mnt/mirror/cygwin
> 149G /mnt/fullmirror/cygwin
>
> So you can probably assume that only about 20% of the files are
> frequently accessed (likely significantly less since most folks would
> not install the debuginfo or source packages that are included in the
> above figure).
>
>> I am more concerned with mirror providers (and also the lack of them)
>> especially those with limited resources, and those in marginal
>> locations and circumstances, for whom download time and charges may
>> override other considerations, and perhaps prevent them (or many) from
>> accessing or taking full advantage of available software.
>
> We could save way more space than that by de-duplicating the noarch
> parts into their own archives as I have already demonstrated before.
> The last time I did that I was cutting out around 30GiB IIRC.
>
>> I doubt the unarchiving time difference is more than a blip in the
>> total time required to *download* *AND* install any package, greatly
>> outweighed by the download time difference, unless you are on a big
>> pipe to a nearby mirror.
>
> It is not, with a typical VDSL connection I'd be able to download faster
> than I can install on a more typical machine, I need only about 5MiB/s
> to saturate the filesystem for small files and around 20…40MiB/s for
> large ones (to an NVMe drive, a spinning disk or some of the slower SSD
> can't sustain that). But that point is somewhat moot since setup will
> always mirror to disk first and that's not easy to change since we read
> the file twice: once for the SHA512 check (which can use up to around
> 300MiB/s input bandwidth somewhat higher in peaks and then the actual
> installation).
Some setup phase stats from my own most recent upgrade of about 130MB of
downloads, and stats since 2013 (nearly 8 years) since I last cleared setup.log:
$ cyg-setup-phase-times.awk /var/log/setup.log.full
sv 00:04:26 dl 00:01:28 pr 00:00:02 ui 00:00:36 ex 00:03:35 pi 00:07:12
$ cyg-setup-phase-times.awk /var/log/setup.log
sv 04:32:46 dl 02:31:27 pr 00:16:08 ui 00:51:10 ex 06:06:49 pi 00:00:41
phases are:
sv - solve formerly Adding required packages - high times are interaction delays
dl - download
pr - preremove
ui - uninstall
ex - extract
pi - postinstall
so your comments about extracts are validated, taking about 3 times as long as
downloads even on a currently 2MByte/s medium speed cable modem link to a nearby
(7.5km direct, 11km drive, 15 hop 10ms round trip) university campus mirror in
recent years.
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
next prev parent reply other threads:[~2021-01-11 22:05 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-08 7:38 Marco Atzeri
2021-01-08 13:23 ` ASSI
2021-01-08 13:58 ` Marco Atzeri
2021-01-08 14:21 ` Jon Turney
2021-01-08 16:52 ` Brian Inglis
2021-01-08 17:13 ` Achim Gratz
2021-01-08 18:09 ` Brian Inglis
2021-01-08 19:11 ` Achim Gratz
2021-01-11 22:05 ` Brian Inglis [this message]
2021-01-12 16:28 ` Brian Inglis
2021-01-12 19:46 ` ASSI
2021-01-09 19:08 ` Achim Gratz
2022-08-14 19:07 Hannes Mueller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6bbcf804-593b-16be-dfd1-e1430c1c4bf8@SystematicSw.ab.ca \
--to=brian.inglis@systematicsw.ab.ca \
--cc=cygwin-apps@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).