From: Mark Geisert <mark@maxrnd.com>
To: Cygwin-Apps <cygwin-apps@cygwin.com>
Subject: Re: Extreme slowdown due to malloc?
Date: Mon, 21 Dec 2020 20:37:14 -0800 [thread overview]
Message-ID: <012a9e3c-ec24-f307-a3c4-9f2589d54e34@maxrnd.com> (raw)
In-Reply-To: <87mty66fw5.fsf@Rainer.invalid>
Hi Achim,
Achim Gratz wrote:
> I've been experimenting a bit with ZStandard dictionaries. The
> dictionary builder is probably not the most optimized piece of software
Is this what leads you to suspect malloc? Really heavy use of malloc?
> and if you feed it large amounts of data it needs quite a lot of
> cycles. So I thought I run some of this on Cygwin since that machine is
> faster and has more threads than my Linux box. Unfortunately that plan
> shattered due to extreme slowness of the first (single-threaded) part of
> the dictionary builder that sets up the partial suffix array.
>
> |------+---------------+---------------|
> | | E3-1225v3 | E3-1276v3 |
> | | 4C/4T | 4C/8T |
> | | 3.2/3.6GHz | 3.6/4.0GHz |
> |------+---------------+---------------|
> | 100 | 00:14 / 55s | 00:23 / 126s |
> | 200 | 00:39 / 145s | 01:10 / 241s |
> | 400 | 01:12 / 266s | 01:25 / 322s |
> | 800 | 02:06 / 466s | 11:12 / 1245s |
> | 1600 | 03:57 / 872s | > 2hr |
> | 3200 | 08:03 / 1756s | n/a |
> | 6400 | 16:17 / 3581s | n/a |
> |------+---------------+---------------|
>
> The obvious difference is that I/O takes a lot longer on Cygwin (roughly
> a minute for reading all the data) and that I have an insane amount of
> page faults on Windows (as reported by time) vs. none on Linux.
How much RAM does the Windows machine have? Do you have a paging file? Is it
fixed size or "let Windows manage"? How big is it?
> While doing that I also noticed that top shows the program taking 100%
> CPU in the multithreaded portion of the program, while it should show
> close to 800% at that time. I'm not sure if that information just isn't
> available on Windows or if procps-ng needs to look someplace else for
> that to be shown as expected.
No offense, but are you sure it's actually running multi-threaded on Windows?
I have a Cygwin malloc speedup patch that *might* help the m-t part. I'll prepare
and submit that to cygwin-patches shortly.
Cheers,
..mark
next prev parent reply other threads:[~2020-12-22 4:37 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-21 20:52 Achim Gratz
2020-12-22 4:37 ` Mark Geisert [this message]
2020-12-22 6:34 ` ASSI
2021-01-02 14:17 ` Achim Gratz
2021-01-18 7:07 ` Mark Geisert
2021-01-18 20:06 ` Achim Gratz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=012a9e3c-ec24-f307-a3c4-9f2589d54e34@maxrnd.com \
--to=mark@maxrnd.com \
--cc=cygwin-apps@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).