Re: scallywag / cygport not pulling lzip

[Achim Gratz <Stromeko@nexgo.de>]

Brian Inglis writes:
> Do we know what the frequency weighted difference is on bandwidth of
> packages actually downloaded?

Not that I know of, as everything goes through mirrors.  But I happen to
have a complete Cygwin mirror on disk at the moment plus another one
that only has the packages for my install and that's a fairly large
installation, but without a desktop environment:

30G     /mnt/mirror/cygwin
149G    /mnt/fullmirror/cygwin

So you can probably assume that only about 20% of the files are
frequently accessed (likely significantly less since most folks would
not install the debuginfo or source packages that are included in the
above figure).

> I am more concerned with mirror providers (and also the lack of them)
> especially those with limited resources, and those in marginal
> locations and circumstances, for whom download time and charges may
> override other considerations, and perhaps prevent them (or many) from
> accessing or taking full advantage of available software.

We could save way more space than that by de-duplicating the noarch
parts into their own archives as I have already demonstrated before.
The last time I did that I was cutting out around 30GiB IIRC.

> I doubt the unarchiving time difference is more than a blip in the
> total time required to *download* *AND* install any package, greatly
> outweighed by the download time difference, unless you are on a big
> pipe to a nearby mirror.

It is not, with a typical VDSL connection I'd be able to download faster
than I can install on a more typical machine, I need only about 5MiB/s
to saturate the filesystem for small files and around 20…40MiB/s for
large ones (to an NVMe drive, a spinning disk or some of the slower SSD
can't sustain that).  But that point is somewhat moot since setup will
always mirror to disk first and that's not easy to change since we read
the file twice: once for the SHA512 check (which can use up to around
300MiB/s input bandwidth somewhat higher in peaks and then the actual
installation).

I have a large base of internal installations that I feed from a
(single) local repo and some of those machines are behind rather slow
links (not quite modem speed, but still slow by todays standards) and
using zstd still makes quite the difference there.  The more typical
install time was reduced by a bit under 50% for both slow and fast
connections, so I no longer recommend that folks reserve time
specifically for the Cygwin installation as I had done before.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada