From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 85CB83858410; Wed, 28 Jun 2023 07:56:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 85CB83858410 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1687938976; bh=wok+yj/LYOG83lNLCQ0s4A2EwSWBRJ6gE3sUy0xrLy4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=JIGA5jPl0fI8w+yUiJiDCsiGMuaeClLxfhtYjr8TJAD73hKJnV/7R9pEMoklqV+hD tnqTdez70OussC+oJiBljdQBUnJTt3+/CGZ+CtE7AJwuwX8IFdLNlotGUCHn0EdDcW RD7F2yVDckqPBFlksd1Fau3psGjZutxL5bUNT3w0= From: "nicolas at freedelity dot be" To: glibc-bugs@sourceware.org Subject: [Bug malloc/30579] trim_threshold in realloc lead to high memory usage Date: Wed, 28 Jun 2023 07:56:15 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: glibc X-Bugzilla-Component: malloc X-Bugzilla-Version: 2.37 X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: nicolas at freedelity dot be X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: unassigned at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 List-Id: https://sourceware.org/bugzilla/show_bug.cgi?id=3D30579 --- Comment #3 from Nicolas Dusart --- I'm concerned that this issue may have been considered resolved for you as = it may seem that it originated from a misunderstanding of my part. But I have to stress out that this change may not be appropriate. It has significantly impacted some of our processes which were not usable anymore = and a lot of debugging were necessary to spot the root cause. I suspect it could similarly affect many other project using glibc. It may = take some time before production environment use new version of glibc, so I'd ex= pect these concern to pop out in a few month or years. I believe we could both benefit from further discussion on this matter, and= I would greatly appreciate gaining insight into the reasoning behind this cha= nge. With that in mind, I'd like to respectfully re-express my concerns about the changes observed in the recent behavior of realloc: - First, as I've mentioned before, it appears that the new behavior will t= end to accumulate small blocks of less than `M_TRIM_THRESHOLD` bytes inside the heap. Total size of all these blocks may largely exceed `M_TRIM_THRESHOLD` bytes. - Secondly, this new behavior of realloc seems to be misaligned with the documentation of M_TRIM_THRESHOLD. According to the documentation, the "M_TRIM_THRESHOLD parameter specifies the minimum size (in bytes) that this block of memory [at the top of the heap] must reach before sbrk is used to = trim the heap." The current behavior, however, doesn't seem to be adhering to this definiti= on. - I acknowledge that mallopt might be a solution to adapt the behavior throughout the code based on specific needs. However, it seems to be a feas= ible solution only in a single-threaded process. In a multi-threaded process, so= me threads might benefit from a large M_TRIM_THRESHOLD while others might need= a smaller one. Unfortunately, mallopt doesn't seem to provide a solution in t= his context. These points bring me to the argument that this shift in behavior can be considered a breaking change as it might have a significant impact on exist= ing processes, particularly those that rely on the current documentation of your library. Previously, we could consider that at maximum of `M_TRIM_THRESHOLD` unused bytes could be reserved indefinitely to our process even if we tried to deallocate this memory. That was a limit that can be controlled. With the new behavior, the amount of unused memory reserved for our process= is unbounded. Furthermore, the unused slots cannot be reused by our process. T= his implies that this space is lost, not just to the rest of the system, but al= so to the process itself until the next free() call is made. In some processes, this newly realloced space is meant to be kept indefinitely. Could you provide some insight into the reasons why these arguments might n= ot be considered strong enough to avoid such a breaking change? Even if the decision is to stick with the new behavior, it would seem neces= sary to update the documentation to reflect these changes accurately. This would ensure that developers are made aware of this new behavior and can take necessary steps to mitigate any impact on their processes. --=20 You are receiving this mail because: You are on the CC list for the bug.=