[Bug malloc/26969] A common malloc pattern can make memory not given back to OS

["keyid.w at qq dot com" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=26969

--- Comment #4 from XY Wen <keyid.w at qq dot com> ---
(In reply to Dmitry from comment #3)
> This is a really common problem especially for long-running multithreaded
> processes with a lot of arenas. Calling malloc_trim - not always an option,
> if a service using C bindings and is written on high level languages. Also,
> malloc_trim would cause unnecessary overhead, since it will be trimming and
> locking all arenas. Additional confusion comes from man page for malloc_trim
> which says it's called sometimes during free.
> 
> I was looking at the code and have the following suggestion for improvement:
> the main idea is to call mtrim for arena in _int_free. To amortize
> performance overhead: call it only if free is called for a chunk with size >
> than FASTBIN_CONSOLIDATION_THRESHOLD, and we have a chance to free more
> memory than let's say 3*TRIM_THRESHOLD. To understand how much we can return
> to OS we can add 1 bit flag to chunk, set it to 0 after returning to OS, set
> to 1 otherwise. This would not give 100% accurate result, but should give a
> good estimate.
> 
> I'd be glad to work on this functionality, if you think it makes sense. If
> you have other suggestions also happy to discuss this. The main point -
> mtrim should be called during free some times, otherwise current malloc is
> unfortunately hard to use for long-running multithreaded workload, since
> it's using 2-3x more RSS in comparison with jemalloc or tcmalloc.

I think your solution will be helpful in this situation. I have no better idea
right now(and I think it's hard to design a trade-off strategy since it
probably need lots of tests). But jemalloc may have some different methods to
solve this problem, to which you can refer, because I noticed that the virtual
memory is also reduced using jemalloc. By the way, tcmalloc doesn't do better
in my usage.

-- 
You are receiving this mail because:
You are on the CC list for the bug.