[Bug malloc/26969] A common malloc pattern can make memory not given back to OS

["carlos at redhat dot com" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=26969

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carlos at redhat dot com
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |NOTABUG

--- Comment #1 from Carlos O'Donell <carlos at redhat dot com> ---
The glibc implementation of malloc is a heap-based allocator and in that design
the heap must be logically freed back down in the order that it was originally
allocated or the heap will continue to grow to keep a maximum working set of
chunks for application.

If you want to free back down to zero at the last deallocation you must tune
the allocator by disabling fastbins and tcache.

For example:
- Allocate A
- Allocate B
- Allocate C
- Free A
- Free B
- Free C

Consider A, B and C are all the same size.

Until "Free C" happens the entire stack is held at 3 objects deep.

This can happen because tcache or fastbins holds the most recently freed chunk
for re-use. There is nothing wrong with this strategy because the C library,
apriori, doesn't know if you'll carry out this entire workload again.

The worse-case degenerate situation for tcache is a sequence of allocations
which cause tcache to always hold the top-of-heap chunks as in-use. In a real
program those chunks are refilled into the tcache much more randomly via malloc
from the unsorted bin or small bin refill strategy. Thus tcache should not keep
the top-of-heap from freeing down in those cases. It's only in synthetic test
cases like this where I think you see tcache being the blocker to freeing down
from the top of heap.

If you need to free pages between workloads and while idle you can call
malloc_trim() to release page-sized consolidated parts of the heaps.

If you need a minimal working set, then you need to turn off fastbins and
tcache.

One possible enhancement we can make is to split the heaps by pool sizes, and
that's something I've talked about a bit with DJ Delorie. As it stands though
that would be a distinct enhancement.

I'm marking this as RESOLVED/NOTABUG since the algorithm is working as intended
but doesn't meet your specific synthetic workload. If you have a real
non-synthetic workload that exhibits problems please open a bug and we can talk
about it and review performance and capture an API trace.

-- 
You are receiving this mail because:
You are on the CC list for the bug.