Re: Maybe consider rpmalloc

["Teemu Nätkinniemi" <tnatkinn@gmail.com>]

ke 14. huhtik. 2021 klo 12.20 Mark Geisert (mark@maxrnd.com) kirjoitti:
>
> Teemu Nätkinniemi wrote:
> > Thanks a lot for looking into this issue! I wonder if there are any
> > other applications affected by this?
>
> We have several examples by now.  All are (relatively) long-lasting apps, with
> high to very high memory allocation churn, often multi-threaded.  I believe some
> specific rsync operations hit this.  Achim reported a zstd operation that
> exhibited the symptoms.  And I've been attempting to get a working replacement for
> Cygwin's malloc for some time but every malloc I've tested, several of them,
> exhibits similar symptoms: excessive time being spent in ntdll.dll presumably
> supporting the memory operations.

Bwa's author has a more recent program called bwa-mem2 which seems to
have exact same problem as bwa. I have tried the rpmalloc trick but it
did not work or I could not identify the correct routine(s).

https://github.com/bwa-mem2/bwa-mem2

> Your rpmalloc "hack" is interesting in that you aren't using Cygwin's mmap()
> underneath the malloc routines; you're calling Windows VM ops directly.  Not sure
> yet what all the implications are.

It was not my hack. Just a third party who managed to fix the problem
as bwa is actually a useful program and there was a need to get it
running on Windows/Cygwin.

> Lots of plausible directions to go...

I am thankful that you are looking into this. Does anyone else have
any input that could be useful? Corinna?

Teemu