From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) by sourceware.org (Postfix) with ESMTPS id 8A3003858D39 for ; Fri, 26 Nov 2021 18:10:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8A3003858D39 X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.178.34] ([77.180.21.227]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MRCK6-1nDrOX1yUe-00N8MG; Fri, 26 Nov 2021 19:10:33 +0100 Subject: Re: Excessive memory consumption when using malloc() To: Adhemerval Zanella , Carlos O'Donell , Konstantin Kharlamov , libc-help@sourceware.org References: <560ed6888a62b21362cda5385655c3a84fd354b9.camel@yandex.ru> <56522c8f847ddd27fdffedecb516f778837f9e92.camel@yandex.ru> From: Christian Hoff Message-ID: <3f88fb7c-672a-da09-89d7-240e9991369d@gmx.net> Date: Fri, 26 Nov 2021 19:10:32 +0100 User-Agent: Mozilla/5.0 (X11; Linux armv7l; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Language: en-GB X-Provags-ID: V03:K1:tEzj8XnvjLa/bCPW3FcmcjYxq1uf8ylTvkliIeioOGHH0joQ6aT YGixIwvVKLoEHx1Uuj6JPlBnR8kI4tLLQynlnbDuC5o7SkrP6DDURE3AZqcygU1mfPGm+tH gHXEA03kPUY9ZHPVLwjWwfSs7tmPg8PfG8hMVV+1WD9yRVGcPHrHdtrdA4fHnGqG+2a8h6P Z2o3yWuhHD5hH2v2DFNCg== X-UI-Out-Filterresults: notjunk:1;V03:K0:JEBsXpb71lk=:u2WrWuFnZZLYTa2gu9QPQF 5hBrkjivsg/uAqaxViW+94L30gGjSbcgXooVeszfYBDRUr6fj9OZG4RjgaNKhT/Y1WjuLa/WF S4DSKnnAADPOfoI6DZ0G+3c+AWqi6uQmdw2ZQdJ2k3r5B/F9fi7kmjCS04L63wfRv2T5kZ9tx y5yfsh0IxRo5o+olaS89jPmWSp2QosBpzoXCkmL2y7nICnQc7DBrLbjt4ht1GRU4aVHmbVupK iLcGO987IDxaakIlhJD49SxHaHk1U6zwoZD6WkZlo/GI5tA3km5JhWeE5mk0lRY6ytPhS7x12 LsliLR9sav3FiT/2HW9nlhD6SryKESLrEB2KUMhVgxC9i4NR7pF4caoYJ0FRq08SN4ydMvvpk kgs8dAt5FY/9Tae1k8K5PJaUB8soUxG1P7zUF+7roSPPXmftD33bi8WWTe8tL2l0fs3F32Slf 93N2pdQ11c9lsufgjV2p73cBx0OB1fJBY94dXzPuZFgL9OR4OT94xxwBvXmI66n8aQM/UX8LK g/2nS2YPm1AcWAvnqfWM3GxTX/FuOjXJUHLxSmziMgMApdrTKVih5vVCnl7hTrqkNjeBf4KIJ CN4SEgeSq2hW9GabuB63Zro1nV8bdzjC9sOrS9o54cH+N6PI2LCYqvoikayJU7ah83mAtvKBp ZP5YMrOkhyz9+qT4pBMHXbOb4PKHE4EUV5MPpkTJ+X0UHjqjdY7DgTTqtjKonPQ5pKb0qjZMh F6g0p6luH2dCTKCa5oW1MZ0u0WzqYdm+k19ZQNS0LLRGKV8yuY0Jj4DwKksV0PnQZJ17AhL01 SqXszJwz2vM67xXB7K6Wgnoh/kD3d81pTCXsYzlxTv8WI3o3v7LHnxB8xw1DiabUZS7iXUATb H2dRFtZLNGSF2wCURV+qQAiKbHz34O+r76yZ+QDnK7UMMNZz+ddkXxFJAZmbRUfwyAJeY2Y1s vagLXv6n7UPLkpchv4wGcd5p6sfCz0lAU55WOQYpj7CU//ewrxh4+gHYaH6hChGehbP+Ln64R a/gFuxrhsovK70zu6QRmdUlJSKuIBQCZGwEKMzqSoJL1Zc+XAgteHrAl2bSdO2qIujHl6bE7W 7Cs4bxFtu/NQ6Y= X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, NICE_REPLY_A, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Nov 2021 18:10:39 -0000 Hello Adhemerval, On 11/25/21 9:56 PM, Adhemerval Zanella wrote: > What I think we might improve is to maybe add an heuristic to call > malloc_trim once a certain level of fragmentation in the main_arena is f= ound. > The question is which metric and threshold to use. The trimming does ha= ve > a cost, however I think it worth to decrease fragmentation and memory ut= ilization. Yes, I think this is a very good idea. It is difficult for us to tell our customers that our application consumes so much memory even while it is not running any computations. Of course, we can call malloc_trim() as a workaround, but even this is difficult as this is a Java application, so calling system functions is not so straightforward. Any glibc improvements in this area would be highly welcome. Best regards, =C2=A0=C2=A0 Christian