From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from forward103o.mail.yandex.net (forward103o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::606]) by sourceware.org (Postfix) with ESMTPS id 70A943857C4F for ; Mon, 21 Dec 2020 22:59:51 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 70A943857C4F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=yandex.ru Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=hi-angel@yandex.ru Received: from sas1-d4a7cb224acf.qloud-c.yandex.net (sas1-d4a7cb224acf.qloud-c.yandex.net [IPv6:2a02:6b8:c14:494:0:640:d4a7:cb22]) by forward103o.mail.yandex.net (Yandex) with ESMTP id 7868E5F82550; Tue, 22 Dec 2020 01:59:48 +0300 (MSK) Received: from sas1-e00c2743cdb8.qloud-c.yandex.net (sas1-e00c2743cdb8.qloud-c.yandex.net [2a02:6b8:c14:3a22:0:640:e00c:2743]) by sas1-d4a7cb224acf.qloud-c.yandex.net (mxback/Yandex) with ESMTP id GZ25AnFo1q-xmDe65ri; Tue, 22 Dec 2020 01:59:48 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1608591588; bh=JByu7MzomcQoS72MYF3c5lKU3LtGcYy+ITIUfowpkPc=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=ZpX2q62V1T9bDbY2X3i0Qi+RwK44oe1mZ7LPeKVx0ZnGB4djAKOQ4VaPH1IckB5xt 7wi8KuvfSqyyjLpw3Kdto5PEgVWOdsVWvjE+5GBECrMo0WurSvWe+f/rQpvX0bh678 mmlRLV9dM5Yrp4v/aFqTW8Ea+eP53/zgi5HZAIWE= Authentication-Results: sas1-d4a7cb224acf.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-e00c2743cdb8.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id mXzU18dGlf-xlI01B9i; Tue, 22 Dec 2020 01:59:47 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <453d41b6d3f13dac758eee840a2d5693ac032bbb.camel@yandex.ru> Subject: Re: `free` fails to reclaim memory From: Konstantin Kharlamov To: tomas@tuxteam.de Cc: libc-help@sourceware.org Date: Tue, 22 Dec 2020 01:59:46 +0300 In-Reply-To: <20201213123241.GC30670@tuxteam.de> References: <83k0tmeq6f.fsf@gnu.org> <9f49e5542f303736d2e53ce3dc53c1374969e6b4.camel@yandex.ru> <3142ea6128492d89e81c58a3d0f2450f1a583f53.camel@yandex.ru> <20201213115724.GA30670@tuxteam.de> <20201213123241.GC30670@tuxteam.de> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.2 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Dec 2020 22:59:54 -0000 On Sun, 2020-12-13 at 13:32 +0100, tomas@tuxteam.de wrote: > On Sun, Dec 13, 2020 at 03:14:39PM +0300, Konstantin Kharlamov wrote: > > [...] > > > Thanks, I might look at tunables, but I'd also like to point out that this > > doesn't make it less of a glibc problem. Because you can't expect everyone > > to > > tune app up specifically for glibc. There're various libc libs on different > > platforms, and it would take a lot of effort from developers to tune an app > > for > > every existing libc out there. > > That's right. On the other hand, what we are seeing as the "libc defaults" > is most probably the result of a compromise. The library has to behave > sanely for a wide range of malloc/free behaviours. Returning space to the > OS for every 16 byte free will quickly become prohibitive (it involves > a syscall, after all). > > I don't think the libc authors will see that as a bug. For these usecases Glibc already has a per-thread malloc cache, to which, if I correctly understand, the discussed behavior is irrelevant. It's just failure of libc to figure out it conserved too much memory, and that it needs to free some up. So, I see no reason why it would not be considered a bug. Either way, since this discussion doesn't seem to be too popular, I just went ahead and reported it: https://sourceware.org/bugzilla/show_bug.cgi?id=27103 > If the generic behaviour doesn't fit your application's usage pattern, > that's what the tunables are for. Or the explicit call to malloc_trim. > > In most applications, you'll be calling malloc and free in some interleaved > pattern, so the virtual size will stabilise at some point corresponding > to the maximum used space. In those cases, it seems a "good thing" that > this space isn't returned to the OS "too early" -- that would slow down > memory management. > > That's what one should see in the Emacs case, I think. Well, in my case it won't stabilize. The usecase is that I disabled memory-threshold for Emacs garbage-collector (for performance reasons), and instead I hooked-up running GC to idle time. Now, there can easily appear short-burst workloads which would allocate lots of memory before GC gets a chance to run. And then what? The memory Emacs just took will hardly be used again, but glibc isn't gonna return memory either.