Re: `free` fails to reclaim memory

[Konstantin Kharlamov <hi-angel@yandex.ru>]

On Sun, 2020-12-13 at 13:32 +0100, tomas@tuxteam.de wrote:
> On Sun, Dec 13, 2020 at 03:14:39PM +0300, Konstantin Kharlamov wrote:
>
> [...]
>
> > Thanks, I might look at tunables, but I'd also like to point out that this
> > doesn't make it less of a glibc problem. Because you can't expect everyone
> > to
> > tune app up specifically for glibc. There're various libc libs on different
> > platforms, and it would take a lot of effort from developers to tune an app
> > for
> > every existing libc out there.
>
> That's right. On the other hand, what we are seeing as the "libc defaults"
> is most probably the result of a compromise. The library has to behave
> sanely for a wide range of malloc/free behaviours. Returning space to the
> OS for every 16 byte free will quickly become prohibitive (it involves
> a syscall, after all).
>
> I don't think the libc authors will see that as a bug.

For these usecases Glibc already has a per-thread malloc cache, to which, if I correctly understand, the discussed behavior is irrelevant. It's just failure of libc to figure out it conserved too much memory, and that it needs to free some up.

So, I see no reason why it would not be considered a bug. Either way, since this discussion doesn't seem to be too popular, I just went ahead and reported it: https://sourceware.org/bugzilla/show_bug.cgi?id=27103

> If the generic behaviour doesn't fit your application's usage pattern,
> that's what the tunables are for. Or the explicit call to malloc_trim.
>
> In most applications, you'll be calling malloc and free in some interleaved
> pattern, so the virtual size will stabilise at some point corresponding
> to the maximum used space. In those cases, it seems a "good thing" that
> this space isn't returned to the OS "too early" -- that would slow down
> memory management.
>
> That's what one should see in the Emacs case, I think.

Well, in my case it won't stabilize. The usecase is that I disabled memory-threshold for Emacs garbage-collector (for performance reasons), and instead I hooked-up running GC to idle time.

Now, there can easily appear short-burst workloads which would allocate lots of memory before GC gets a chance to run. And then what? The memory Emacs just took will hardly be used again, but glibc isn't gonna return memory either.