From: Jon Turney <jon.turney@dronecode.org.uk>
To: cygwin-developers@cygwin.com
Subject: Re: Maybe consider rpmalloc
Date: Wed, 14 Apr 2021 19:53:22 +0100 [thread overview]
Message-ID: <e82d01ea-e56b-e949-1188-ffac6d04875a@dronecode.org.uk> (raw)
In-Reply-To: <93809c4f-7747-3611-0d20-bde09e091f1d@maxrnd.com>
On 14/04/2021 09:19, Mark Geisert wrote:
> Teemu Nätkinniemi wrote:
>> Thanks a lot for looking into this issue! I wonder if there are any
>> other applications affected by this?
>
> We have several examples by now. All are (relatively) long-lasting
> apps, with high to very high memory allocation churn, often
> multi-threaded. I believe some specific rsync operations hit this.
> Achim reported a zstd operation that exhibited the symptoms. And I've
> been attempting to get a working replacement for Cygwin's malloc for
> some time but every malloc I've tested, several of them, exhibits
> similar symptoms: excessive time being spent in ntdll.dll presumably
> supporting the memory operations.
>
> Your rpmalloc "hack" is interesting in that you aren't using Cygwin's
> mmap() underneath the malloc routines; you're calling Windows VM ops
> directly. Not sure yet what all the implications are.
>
> I need to identify what's being hit within ntdll.dll. Is it one or two
> routines, or just hot locks. So that means getting the correct PDB file
> from the MS Symbol Server and working with Windows tools I'm unfamiliar
> with. Sigh, in an earlier life I had a gdb that we'd taught how to work
Yes, this would indeed be a very useful thing to have in gdb.
I'm not aware of any public work in that direction, though.
> with PDB files; dunno if I could resurrect that. Profiling the Cygwin
> DLL itself, call profiling I mean, might lead somewhere as well.
In the past I've had some success with using the Very Sleepy profiler
([1]), which can use both PDB and DWARF symbols, on cygwin executables.
[1] https://github.com/VerySleepy/verysleepy
next prev parent reply other threads:[~2021-04-14 18:54 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 6:01 Cygwin malloc tune-up status Mark Geisert
2020-09-27 16:54 ` Johannes Schindelin
2020-09-29 2:22 ` Mark Geisert
2021-04-01 9:19 ` Teemu Nätkinniemi
2021-04-02 5:45 ` Maybe consider rpmalloc -- Was: " Mark Geisert
2021-04-03 2:53 ` Maybe consider rpmalloc Mark Geisert
2021-04-03 6:46 ` Teemu Nätkinniemi
2021-04-03 6:48 ` Teemu Nätkinniemi
2021-04-11 9:28 ` Mark Geisert
2021-04-12 8:48 ` Teemu Nätkinniemi
2021-04-13 8:24 ` Mark Geisert
2021-04-13 13:05 ` Teemu Nätkinniemi
2021-04-14 8:19 ` Mark Geisert
2021-04-14 18:36 ` Teemu Nätkinniemi
2021-04-14 18:53 ` Jon Turney [this message]
2021-04-19 5:16 ` Mark Geisert
2021-04-20 19:34 ` Jon Turney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e82d01ea-e56b-e949-1188-ffac6d04875a@dronecode.org.uk \
--to=jon.turney@dronecode.org.uk \
--cc=cygwin-developers@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).