Re: [PATCH v2] Remove upper limit on tunable MALLOC_MMAP_THRESHOLD

[DJ Delorie <dj@redhat.com>]

Patrick McGehearty <patrick.mcgehearty@oracle.com> writes:
> If a chunk smaller than the mmap_threshold is requested,
> then MORECORE [typically sbrk()] is called and HEAP_MAX
> is not considered by the malloc code.  Heaps are only used
> for mmap()ed allocations, not sbrk()'ed allocations,
> so far as I can tell in reading the code.

There are two types of arenas: the sbrk-based arena (limited in size by
ulimit), and zero or more mmap-based arenas (limited by HEAP_MAX).  The
sbrk-based one is used when the program is single threaded; the
mmap-based ones are used when the program is multi-threaded.  Your
original email made it sound like you were concerned with the
multi-threaded case, where the mmap-based heaps are used.

In either case, a malloc() request may be satisfied by pulling a free
chunk out of either type of arena (possibly growing the arena if needed
and possible), or by calling mmap() directly to satisfy that one
request.

I would think mmap_threshold should still apply if you're using the
mmap'd heaps, so you can reserve the heaps for smaller chunks, but that
is meaningless if mmap_threshold is larger than the heap size.  I could
not find an obvious place in the code where mmap_threshold is used to
bypass the mmap'd heaps, though.

So while I have no problems with allowing larger mmap_threshold settings
for the sbrk-based arena, I still wonder what happens to requests that
go through an mmap-based arena that are larger than HEAP_MAX but still
under the mmap_threshold.

Of course, I've spent more time typing this response than it would take
to write a test program and see what happens ;-)

> It might be desirable to also allow HEAP_MAX to be set by
> the user before the first call to malloc, but I see that
> as a separate task.

Our current implementation requires that the heap size be a compile-time
constant, but... yeah.