From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <sourceware-bugzilla@sourceware.org>
Received: by sourceware.org (Postfix, from userid 48)
 id D4F59385782B; Fri, 29 Jan 2021 16:08:53 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D4F59385782B
From: "dimahabr at gmail dot com" <sourceware-bugzilla@sourceware.org>
To: glibc-bugs@sourceware.org
Subject: [Bug malloc/26969] A common malloc pattern can make memory not given
 back to OS
Date: Fri, 29 Jan 2021 16:08:53 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: glibc
X-Bugzilla-Component: malloc
X-Bugzilla-Version: 2.27
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: minor
X-Bugzilla-Who: dimahabr at gmail dot com
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Resolution: 
X-Bugzilla-Priority: P1
X-Bugzilla-Assigned-To: unassigned at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags: security-
X-Bugzilla-Changed-Fields: cc
Message-ID: <bug-26969-131-twtEatOYEm@http.sourceware.org/bugzilla/>
In-Reply-To: <bug-26969-131@http.sourceware.org/bugzilla/>
References: <bug-26969-131@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-BeenThere: glibc-bugs@sourceware.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Glibc-bugs mailing list <glibc-bugs.sourceware.org>
List-Unsubscribe: <https://sourceware.org/mailman/options/glibc-bugs>,
 <mailto:glibc-bugs-request@sourceware.org?subject=unsubscribe>
List-Archive: <https://sourceware.org/pipermail/glibc-bugs/>
List-Help: <mailto:glibc-bugs-request@sourceware.org?subject=help>
List-Subscribe: <https://sourceware.org/mailman/listinfo/glibc-bugs>,
 <mailto:glibc-bugs-request@sourceware.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Jan 2021 16:08:53 -0000

https://sourceware.org/bugzilla/show_bug.cgi?id=3D26969

Dmitry <dimahabr at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dimahabr at gmail dot com

--- Comment #3 from Dmitry <dimahabr at gmail dot com> ---
This is a really common problem especially for long-running multithreaded
processes with a lot of arenas. Calling malloc_trim - not always an option,=
 if
a service using C bindings and is written on high level languages. Also,
malloc_trim would cause unnecessary overhead, since it will be trimming and
locking all arenas. Additional confusion comes from man page for malloc_trim
which says it's called sometimes during free.

I was looking at the code and have the following suggestion for improvement:
the main idea is to call mtrim for arena in _int_free. To amortize performa=
nce
overhead: call it only if free is called for a chunk with size > than
FASTBIN_CONSOLIDATION_THRESHOLD, and we have a chance to free more memory t=
han
let's say 3*TRIM_THRESHOLD. To understand how much we can return to OS we c=
an
add 1 bit flag to chunk, set it to 0 after returning to OS, set to 1 otherw=
ise.
This would not give 100% accurate result, but should give a good estimate.

I'd be glad to work on this functionality, if you think it makes sense. If =
you
have other suggestions also happy to discuss this. The main point - mtrim
should be called during free some times, otherwise current malloc is
unfortunately hard to use for long-running multithreaded workload, since it=
's
using 2-3x more RSS in comparison with jemalloc or tcmalloc.

--=20
You are receiving this mail because:
You are on the CC list for the bug.=