public inbox for cygwin-apps@cygwin.com
 help / color / mirror / Atom feed
* Zstandard support for setup
@ 2018-08-11 19:52 Achim Gratz
  2018-08-12 13:28 ` Achim Gratz
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Achim Gratz @ 2018-08-11 19:52 UTC (permalink / raw)
  To: cygwin-apps


I've finally implemented ZStandard support for setup and tested it today
with a fresh install on a Celeron 1037U dual-core machine from and to
local SSD.  The archive extraction part of setup went from taking 23
minutes to about 14 minutes.  A (much) faster machine at work did the
same install yesterday in under 12 minutes, limited mainly by the
latency of requesting those files from a remote filer (found a defect
trunk NIC the admin didn't know about while doing that) and writing it
out to iSCSI block storage.  For large writes to disk I've seen it peek
in the 200MiB/s range today, which is close to the performance limit for
my SSD through the file system.  However, for unpacking lots of smaller
files the latency of the SSD limits the average write speed to around
35MiB/s (up from 20MiB/s).  It would take a major reorganization of
setup to allow for overlapped / multithreaded extraction in order to
improve significantly on that, minor improvements might still be had
with more aggressive buffering in memory to reduce read/write turnaround
and perhaps by using zero-copy data streaming.

The current code is available here:

http://repo.or.cz/w/cygwin-setup/local.git

The debug output statements are still in the code (although commented),
I'll let that sit a while and see if I find something else I want to
clean up before I submit it for the upstream repo.

Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2018-10-14 11:29 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-11 19:52 Zstandard support for setup Achim Gratz
2018-08-12 13:28 ` Achim Gratz
2018-08-17 18:49 ` Achim Gratz
2018-08-20 18:11   ` Marco Atzeri
2018-08-20 19:54     ` Achim Gratz
2018-08-22 19:12   ` Achim Gratz
2018-08-23 20:40 ` Achim Gratz
2018-08-29 16:42 ` Jon Turney
2018-09-02  8:31   ` Achim Gratz
2018-09-02 17:03     ` Achim Gratz
2018-10-07 19:46       ` Jon Turney
2018-10-09 18:04         ` Achim Gratz
2018-10-07 22:09     ` Jon Turney
2018-10-09 18:05       ` Achim Gratz
2018-10-10 18:37   ` Achim Gratz
2018-10-13 17:19     ` Jon Turney
2018-10-14 11:29       ` Achim Gratz

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).