Re: More (?) steps toward jemalloc within Cygwin DLL

[Corinna Vinschen <corinna-cygwin@cygwin.com>]

Hi Mark,

On Jul 21 01:50, Mark Geisert wrote:
> Corinna Vinschen wrote:
> > [...]
> > ...the big problem are dependencies on malloc during Cygwin startup,
> > especially in fork/exec, so the real challenge is to get the new malloc
> > still let Cygwin processes start up correctly first time and especially
> > in fork/exec situations, and to make sure the malloc bookkeeping
> > survives fork/exec.
> 
> O.K., understood.
> 
> > These malloc dependencies sometimes crop up in the weirdest situations,
> > so that's something to look out for.  For instance, using pthread
> > functions may call malloc as well.  If a problem can be solved by
> > changing another part of Cygwin, don't hesitate to discuss this!
> 
> Yes, a couple of the malloc packages I'm testing want to allocate locks and
> TLS slots right off the bat so there's nasty recursion possible.

Given these locks are process-only, it's probably a good idea to overload
the functions with equivalent WinAPI function calls using, for instance,
slim R/W Locks.  If these locks are stored as global NO_COPY objects, they
don't even have to be initialized at process startup explicitely.  If they
have to be created dynamically, they should probably go into the cygheap,
so they are duplicated automagically.

> [...]
> Here's a question I didn't expect to come up: If it turns out a home-grown
> wrapper on the Win32 HeapXXX functions performs better (hint: it does, 2.5
> to 3 times better) than any malloc package derived from dlmalloc, is there
> any reason why we ought not use it?  Assuming it can be made to work for all
> those cases you mentioned above, of course.

It won't work with fork.  Malloc'd memory has to be duplicated in the exact
same spot during fork (think application pointers to allocated memory).
Windows Heaps are ASLR'ed and the mechanism the heap is allocating and
freeing memory is not built for reproducability in another process.

That's why we have our own process heap given away via sbrk, as well as
diligent bookkeeping of mmap'ed memory.

> > The only danger here is this: If you manage to get dlmalloc replaced
> > reliably, you *will* get a pink plush hippo!
> 
> Oh, gee, that sounds like a really nice reward... Wow, I'm gonna have to do
> this project now for sure!

I'm really looking forward to it!


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer