[Bug malloc/26663] malloc() doesn't handle MALLOC_MMAP_THRESHOLD_=0 well

["carlos at redhat dot com" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=26663

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carlos at redhat dot com
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2020-09-24
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to Topi Miettinen from comment #0)
> Using environment variable MALLOC_MMAP_THRESHOLD_=0 may cause malloc() to
> fail very easily. A case in point is startup of systemd-cryptsetup, which
> fails when lvm2 library can't malloc() something
> (https://github.com/lvmteam/lvm2/issues/39). This is probably a bug in
> kernel, there's plenty of RAM available and I suppose the memory shouldn't
> ever be too fragmented for page sized items.

In the upstream ticket I've asked "Why?" It's important we understand your use
case so we can give better feedback about solutions.

In this bug we'll talk about the specific behaviour of malloc.

Firstly you need to be aware that you'll hit MALLOC_MMAP_MAX_ at some point
which is 65,536 mappings before malloc stops allocating anything.

Secondly, the MALLOC_MMAP_THRESHOLD_ value is the threshold at which requests
to extend the heap will go directly to mmap() instead of growing the current
arena (also done via mmap()).

Thus even if you set MALLOC_MMAP_THRESHOLD_ to a low value, you will still use
the process heap, it is only when you run out of heap that you will switch to
trying to mmap() all subsequent allocations via single mmap() calls (instead of
trying to extend the main arena with another larger single mmap to service
current and future requests).

So the first thing to check is the value of MALLOC_MMAP_MAX_ and see if that is
set high enough for your expectations.

I would also use `strace -ff -ttt` on the process to see if mmap() is actually
failing or not, and with what values.

If you want deeper insight into your application behaviour you can use a
trace/dump tooling for the malloc API calls:
https://pagure.io/glibc-malloc-trace-utils
(not yet integrated upstream)

> But perhaps malloc() should use a better allocation strategy when mmap()ing,
> for example mmap() larger areas at once, use them as arenas and deal them to
> malloc() users as needed.

This suggestion is exactly what it does *normally*, but the setting of
MALLOC_MMAP_THRESHOLD_=0 bypasses that and attempts to *always* service
extension requirements with single mmap() calls.

> Maybe also when mmap() specifically fails with EAGAIN it could be retried a
> few times. It looks like sysmalloc() only checks for MAP_FAILED
> (https://sourceware.org/git?p=glibc.git;a=blob;f=malloc/malloc.c;
> h=cd9933b4e580a58a694ebf34e76ac6fecee29c14;hb=HEAD#l2330) but not errno.
> Perhaps also some unused memory could be released by munmap()ping unused
> parts of arenas etc. before retrying.

Calls to malloc() should not block retrying mmap(), they should fail
immediately and report that failure to higher software layers that have better
visbility into overall allocation strategy. The retry should happen at higher
levels. From the allocators perspective there is no difference between EAGAIN
and ENOMEM.

There is one place where we can improve though, and it's in tracking the
currently cached amount of memory that the process is using, and keeping that
at a constant percentage by walking the cached chunk lists and freeing down to
an acceptable percentage e.g. calling malloc_trim(0) but terminating when a
certain percentage has been freed.

Does this answer your questions?

If it does then I'll mark this RESOLVED/NOTABUG

-- 
You are receiving this mail because:
You are on the CC list for the bug.