[Bug malloc/27103] New: glibc fails to free memory

[Bug malloc/27103] New: glibc fails to free memory

[hi-angel at yandex dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27103

            Bug ID: 27103
           Summary: glibc fails to free memory
           Product: glibc
           Version: 2.32
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: malloc
          Assignee: unassigned at sourceware dot org
          Reporter: hi-angel at yandex dot ru
  Target Milestone: ---

This is suspected to be the reason for this bugreport¹ and was discussed in
glibc ML here².

# Steps to reproduce (in terms of terminal commands)

     λ cat test.cpp
    // credits to
https://stackoverflow.com/a/48652734/2388257#glibc-application-holding-onto-unused-memory-until-just-before-exit
    #include <cstdlib>
    #include <cstdio>
    #include <unistd.h>

    int main() {
        static const int N = 50000;
        static void *arr[N];

        for (int i = 0; i < N; i++)
            arr[i] = std::malloc(1024);

        // reverse to simplify allocators job
        for (int i = N - 1; i >= 0; i--)
            std::free(arr[i]);
        puts("Measure");
        sleep(1e5);
    }
     λ g++ test.cpp -o a
     λ ./a &
    [1] 647847
    Measure
     λ smem -kc "name pid pss" | grep -P "\ba\b"
    a                        647847    50.4M

## Expected

Memory doesn't take up 50M because these 50M were freed in the point of measure

## Actual

Memory takes up 50M despite that memory have been freed.

# Links

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45200
2: https://sourceware.org/pipermail/libc-help/2020-December/thread.html#005592

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/27103] glibc fails to free memory

[hi-angel at yandex dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27103

--- Comment #1 from Hi-Angel <hi-angel at yandex dot ru> ---
To add to the list of affected apps: Ruby¹, glib².

And yeah, calling `malloc_trim(0);` next to each `free()` resolves the problem,
but it is merely a workaround. Because it is glibc-specific function and one
can't reasonably expect every developer of each existing app to be able to
write code for each existing libc-implementation. Glibc `free()` should call
`malloc_trime()` themselves whenever they deem appropriate.

1:
https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#investigating-fragmentation-at-the-memory-allocator-level
2: https://gitlab.gnome.org/GNOME/glib/-/issues/2057

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/27103] glibc fails to free memory

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27103

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=14581
         Resolution|---                         |NOTABUG
                 CC|                            |carlos at redhat dot com

--- Comment #2 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to Hi-Angel from comment #1)
> To add to the list of affected apps: Ruby¹, glib².
> 
> And yeah, calling `malloc_trim(0);` next to each `free()` resolves the
> problem, but it is merely a workaround. Because it is glibc-specific
> function and one can't reasonably expect every developer of each existing
> app to be able to write code for each existing libc-implementation. Glibc
> `free()` should call `malloc_trime()` themselves whenever they deem
> appropriate.
> 
> 1:
> https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-
> bloat.html#investigating-fragmentation-at-the-memory-allocator-level
> 2: https://gitlab.gnome.org/GNOME/glib/-/issues/2057

The ruby case is a known issue with aligned allocations.
https://sourceware.org/bugzilla/show_bug.cgi?id=14581

We have a patch to improve the ruby case significantly, but it needs more work.

The gnome case is unrelated and is just about freeing back unused pages under
memory pressure which is exactly what malloc_trim() will do when called.

This particular issue is not a bug, the allocator is under no requirement to
free back the 50MiB to the OS if it believes that the memory will be used again
and that performance is improved by caching it in userspace.

-- 
You are receiving this mail because:
You are on the CC list for the bug.