[Bug malloc/14581] New: glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[Bug malloc/14581] New: glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[dev at parallels dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

             Bug #: 14581
           Summary: glibc leaks memory and do not reuse after free
                    (leading to unlimited RSS growth)
           Product: glibc
           Version: 2.15
            Status: NEW
          Severity: normal
          Priority: P2
         Component: malloc
        AssignedTo: unassigned@sourceware.org
        ReportedBy: dev@parallels.com
    Classification: Unclassified


If you run the program below you will find that it never really allocates more
then 100MB of RAM, while it's RSS grows >700MB (and higher if the internal loop
made longer):

--------------------------
Arena 0:
system bytes     =  759681024
in use bytes     =   75144000
Total (incl. mmap):
system bytes     =  759681024
in use bytes     =   75144000
max mmap regions =          0
max mmap bytes   =          0
---------------------------

The same is confirmed by kernel RSS if allocated memory is touched (below only
VSZ grows as memory is not touched).

Please note, that this program does a VERY simple thing it allocates/frees
objects of 2 fixed sizes only and number of objects is strictly limited, there
is no heap fragmentation if you print virtual addresses of allocated objects
and kernel VMAs, i.e. you will find a huge unused memory extents which are
never reused by glibc. So there is no place for excuses for such behavior like
fragmentation.

Tested on RHEL6.3 (glibc-2.12-1.80.el6_3.5) and on FC16
(glibc-2.14.90-24.fc16.9), FC17 (glibc-2.15).

--------------------------------------
#include <stdio.h>
#include <malloc.h>

#define LSIZE (65536+4096)
#define SSIZE 556
#define NL 500
#define NS 70000

int main(int argc, char **argv)
{
        void * bigalloc[NL];
        void * smallalloc[NS];
        int bptr = 0;
        int sptr = 0;
        int i;

        memset(bigalloc, 0, sizeof(bigalloc));
        memset(smallalloc, 0, sizeof(smallalloc));

        for (i = 0; i < (16*1024*1024*1024ULL)/65536; i++) {
                free(bigalloc[i % NL]);
                free(smallalloc[i % NS]);
                smallalloc[i % NS] = malloc(SSIZE);
                bigalloc[i % NL] = memalign(4096, LSIZE);
        }

        malloc_stats();
        system("ps axv|fgrep stressalloc");
}

--------------------------------------

Linked from https://bugzilla.redhat.com/show_bug.cgi?id=843478

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

Rich Felker <bugdal at aerifal dot cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugdal at aerifal dot cx

--- Comment #1 from Rich Felker <bugdal at aerifal dot cx> 2012-09-14 19:26:39 UTC ---
Is this supposed to be a bug report or a trick question for CS students? :) You
claimed "there is no heap fragmentation", but the allocation/free pattern
you're performing seems like a classic fragmentation stress test pattern. There
is no general-purpose allocation strategy that can avoid all instances of
pathological fragmentation, and this pattern is one that would be especially
hard to avoid without having special-cased it (and probably making malloc
behavior much worse for common real-world cases). Did you run into this issue
in a real-world application, or did you construct this test as a worst-case?

By the way, the pathological fragmentation observed here does not seem specific
to glibc. It seems like it will occur with any allocator utilizing the basic
dlmalloc strategy. I just confirmed that the same issue happens with ours in
musl libc.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[dev at parallels dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #2 from Kirill Korotaev <dev at parallels dot com> 2012-09-15 10:40:39 UTC ---
> Is this supposed to be a bug report or a trick question for CS students? :) You
> claimed "there is no heap fragmentation", but the allocation/free pattern
> you're performing seems like a classic fragmentation stress test pattern.

I guess you haven't looked closely to the code and result... :(
1. Just print object addresses and allocated VMAs allocated using
/proc/self/maps and you will find HUGE holes of not used RAM.
2. Do you realize that test NEVER has more then 100MB allocated, while RSS
grows infinitely and to unbounded values?

> There is no general-purpose allocation strategy that can avoid all instances of
> pathological fragmentation, and this pattern is one that would be especially
> hard to avoid without having special-cased it (and probably making malloc
> behavior much worse for common real-world cases).

No comments, see above.

> Did you run into this issue in a real-world application, or did you construct this test as a worst-case?

Yes, we hit this in real-life application. Very simple one, an application
processing external requests from network. When a message arrives you allocate
556 bytes for message content to read from socket. Then you need 68KB to
process the request. And then you free both. See the result.

> By the way, the pathological fragmentation observed here does not seem specific
> to glibc. It seems like it will occur with any allocator utilizing the basic
> dlmalloc strategy. I just confirmed that the same issue happens with ours in musl libc.

This pattern doesn't lead to unbound usage on TCmalloc, buddy allocator and
original ptmalloc.
See my next comment. Bug is in memalign.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[dev at parallels dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #3 from Kirill Korotaev <dev at parallels dot com> 2012-09-15 10:43:46 UTC ---
one more finding:  not reproducible if replace memalign with malloc() (even if
add paddings as memalign does).

Comment from ANK on this:
Interesting.

memalign() in glibc is very simple: it is just malloc(size + alignment +
some_small_number)
then tail and head returned to pool.

Probably, this is the culprit. Long shot: when result of memalign is freed,
malloc
does not merge fragments, so hole will be of three free parts: head, body, tail
and it is unusable for future memalign.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #4 from Rich Felker <bugdal at aerifal dot cx> 2012-09-15 12:39:28 UTC ---
Very interesting. I'll have to look at the issue in more detail. It doesn't
seem to be the memalign fragment merging issue alone causing this, because it
also happens on at least one dlmalloc-style allocator that never allows
adjacent non-merged chunks to exist.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #5 from Rich Felker <bugdal at aerifal dot cx> 2012-09-15 14:08:25 UTC ---
Created attachment 6634
  --> http://sourceware.org/bugzilla/attachment.cgi?id=6634
improved test that reports final heap

This version of the test semi-portably reports the state of the heap after the
fragmentation loop. Most of the gaps are too small to ever be used (natural
fragmentation, but there are a number of larger gaps with sizes in the megabyte
range. I believe they're also just an artifact of normal fragmentation, but I
haven't investigated enough yet to be confident in that claim.

Running the program multiple times with ASLR disabled and different iteration
counts would allow one to diff the output between any iterations N and N+K to
get a clearer idea of what's happening.

I'm still not convinced that memory growth is unbounded, just that
fragmentation is very bad...

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #6 from Rich Felker <bugdal at aerifal dot cx> 2012-09-15 21:00:22 UTC ---
Could you explain what you mean by "if you print virtual addresses of allocated
objects and kernel VMAs, i.e. you will find a huge unused memory extents which
are never reused by glibc"? I'm not aware of any way the kernel VMA data could
inform you about heap utilization, which is entirely under userspace control.

I did a simulation of your test loop with much smaller sizes using pen and
paper, and 

With SSIZE=2, ALIGN=8, LSIZE=5, NS=[many], NL=4:

3z: SS LLLLLSS LLLLLSS LLLLLSS LLLLL
4a: SS      SS LLLLLSS LLLLLSS LLLLL
4z: SSSS    SS LLLLLSS LLLLLSS LLLLLLLLLL
5a: SSSS    SS      SS LLLLLSS LLLLLLLLLL
5z: SSSSSS  SS LLLLLSS LLLLLSS LLLLLLLLLL
6a: SSSSSS  SS LLLLLSS      SS LLLLLLLLLL
6z: SSSSSSSSSS LLLLLSS LLLLLSS LLLLLLLLLL
7a: SSSSSSSSSS LLLLLSS LLLLLSS      LLLLL
7z: SSSSSSSSSS LLLLLSS LLLLLSSSS    LLLLLLLLLL
...

where 3z means "at the end of iteration 3" and 4a means "after the free steps
of iteration 4", etc. I might have gotten some details wrong, but it seems this
pattern necessarily enforces fragmentation by destroying the alignment
potential of each LSIZE-sized free range.

Obviously there are some allocation strategies that avoid the issue, but I
don't see it being avoidable in dlmalloc-type schemes in general. If you have
an idea for how to avoid it without destroying the good properties of the
allocator strategy, please share.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[dev at parallels dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #7 from Kirill Korotaev <dev at parallels dot com> 2012-09-16 09:44:02 UTC ---
(In reply to comment #6)
> Could you explain what you mean by "if you print virtual addresses of allocated
> objects and kernel VMAs, i.e. you will find a huge unused memory extents which
> are never reused by glibc"? I'm not aware of any way the kernel VMA data could
> inform you about heap utilization, which is entirely under userspace control.

It's very simple. There is RSS and VSZ in /proc/pid/status.
RSS tells you how much physical memory was really allocated by kernel. If you
add memset() of objects after being allocated you will find that it's really
700MB which corresponds to VSZ as well. i.e. this memory is committed.

> I did a simulation of your test loop with much smaller sizes using pen and
> paper, and 
> 
> With SSIZE=2, ALIGN=8, LSIZE=5, NS=[many], NL=4:
> 
> 3z: SS LLLLLSS LLLLLSS LLLLLSS LLLLL
> 4a: SS      SS LLLLLSS LLLLLSS LLLLL
> 4z: SSSS    SS LLLLLSS LLLLLSS LLLLLLLLLL
> 5a: SSSS    SS      SS LLLLLSS LLLLLLLLLL
> 5z: SSSSSS  SS LLLLLSS LLLLLSS LLLLLLLLLL
> 6a: SSSSSS  SS LLLLLSS      SS LLLLLLLLLL
> 6z: SSSSSSSSSS LLLLLSS LLLLLSS LLLLLLLLLL
> 7a: SSSSSSSSSS LLLLLSS LLLLLSS      LLLLL
> 7z: SSSSSSSSSS LLLLLSS LLLLLSSSS    LLLLLLLLLL
> ...
> 
> where 3z means "at the end of iteration 3" and 4a means "after the free steps
> of iteration 4", etc. I might have gotten some details wrong, but it seems this
> pattern necessarily enforces fragmentation by destroying the alignment
> potential of each LSIZE-sized free range.

First 500 iterations are not interesting that much, cause they do not free any
previously allocated objects.
Have you noticed that array index wraps after NL and NS iterations passed and
then most interesting begins?


> Obviously there are some allocation strategies that avoid the issue, but I
> don't see it being avoidable in dlmalloc-type schemes in general. If you have
> an idea for how to avoid it without destroying the good properties of the
> allocator strategy, please share.

Looks like I start to realize what you mean...

Actually, theoretically any allocator should not ever allocate physical RAM
more then 2*allocated_size due to fragmentation on pattern like this, right?
(it's simple: if you allocated more then 2x times, this means you have unused
holes bigger then any single object and could allocate from it...). In our case
we see about 10x times ratio...

And there are many which behave like that: TCMalloc, buddy etc.
What is not natural in this test is that memalign replaced with malloc() fixes
the problem...

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #8 from Rich Felker <bugdal at aerifal dot cx> 2012-09-16 12:46:01 UTC ---
> It's very simple. There is RSS and VSZ in /proc/pid/status.
> RSS tells you how much physical memory was really allocated by kernel. If you
> add memset() of objects after being allocated you will find that it's really
> 700MB which corresponds to VSZ as well. i.e. this memory is committed.

Of course, but it can't show you gaps in the heap, only the total size of the
heap.

> First 500 iterations are not interesting that much, cause they do not free any
> previously allocated objects.
> Have you noticed that array index wraps after NL and NS iterations passed and
> then most interesting begins?

That's why my experiment on paper had NL=4, to see quickly what happens after
the index wraps.

> Actually, theoretically any allocator should not ever allocate physical RAM
> more then 2*allocated_size due to fragmentation on pattern like this, right?

No, the theoretical limit is many orders of magnitude worse, especially with
alignment constraints. Picture wanting to allocate an object of size K, but
with N objects of size 1 spaced evenly every K-1 units. In this case you have
N*(K-1) units of memory "free", but unable to accommodate the size-K object,
thus requiring new heap space for it. This fragmentation can grow unboundedly;
N can be made arbitrarily large. Also, the size-1 objects can be spaced even
farther apart and still block out allocation of a size-K object if the latter
has alignment requirements. I think your test case is one situation where the
alignment issue matters.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[siddhesh at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

Siddhesh Poyarekar <siddhesh at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |siddhesh at redhat dot com
         Resolution|                            |INVALID

--- Comment #9 from Siddhesh Poyarekar <siddhesh at redhat dot com> 2013-05-13 09:30:23 UTC ---
This is not a bug; the allocator is working as designed and does not claim to
be immune to fragmentation.  If there are better strategies to reduce
fragmentation, then patches to implement them are welcome on libc-alpha.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[ondra at iuuk dot mff.cuni.cz]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

OndrejBilka <ondra at iuuk dot mff.cuni.cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |ondra at iuuk dot
                   |                            |mff.cuni.cz
         Resolution|INVALID                     |

--- Comment #10 from OndrejBilka <ondra at iuuk dot mff.cuni.cz> 2013-05-20 15:12:23 UTC ---
It is valid corner case so it will be open until resolved.

Rich Allocation happen to be 65536 bytes with 4096 alignment.
This is not pathologic case as when you allocate 65536+4096 bytes and pick
aligned address a modified bound mentioned should hold.

The bound says that we must not allocate more than twice usable memory but
usable memory is taken as maximum over program history. 

With alignment this example factor should be less than 2*(65536+4096)/65536

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[siddhesh at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14581

Siddhesh Poyarekar <siddhesh at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|siddhesh at redhat dot com  |
           Severity|normal                      |enhancement

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/14581] glibc leaks memory and do not reuse after free (leading to unlimited RSS growth)

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[mail at nh2 dot me]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

Niklas Hambüchen <mail at nh2 dot me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mail at nh2 dot me

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=27103

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[mirh at protonmail dot ch]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

mirh <mirh at protonmail dot ch> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mirh at protonmail dot ch

--- Comment #18 from mirh <mirh at protonmail dot ch> ---
The Fedora patch was backed out hours after the last comment here, but there's
now new steam behind it. 
https://patchwork.sourceware.org/project/glibc/patch/xn4jz19fts.fsf@greed.delorie.com/

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #19 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to mirh from comment #18)
> The Fedora patch was backed out hours after the last comment here, but
> there's now new steam behind it. 
> https://patchwork.sourceware.org/project/glibc/patch/xn4jz19fts.fsf@greed.
> delorie.com/

Yes, the original Fedora patch was incomplete. I reverted it and set it aside
as we worked on other things. We're coming back to this work again to improve
the reuse of aligned allocations.

Just a note that Ruby, which was a driving use case for fixing the aligned
allocation reuse, has entirely switched to mmap to allocate aligned pages that
it needs for internal uses.

While we still want to fix this issue, it's really bad external fragmentation,
but Ruby is no longer a specific problem.

If you get a chance, please feel free to try out DJ's latest patch and comment.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[nsz at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

Szabolcs Nagy <nsz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nsz at gcc dot gnu.org

--- Comment #20 from Szabolcs Nagy <nsz at gcc dot gnu.org> ---
i think i ran into this or a similar issue, but the proposed patch "[v4,1/1]
memalign: Support scanning for aligned chunks." does not seem to help.

example: 100 iterations where each iteration leaks 16+64 bytes, everything else
is freed, but every iteration seems to increase the heap by 3231744 bytes at
least. with GLIBC_TUNABLES=glibc.malloc.mmap_threshold=600000 there is no issue
(large allocations turn into mmap/munmap pairs).

#include <stdlib.h>
#include <string.h>
static void *volatile q;
static void use(void *p){ q = p; }
int main(int argc, char *argv[])
{
        void *p1;
        void *p2;
        void *p3;
        for (int i=0; i < 100; i++) {
                p1 = aligned_alloc(512, 1176064);
                memset(p1, 0, 1176064);
                p2 = malloc(16);
                p3 = malloc(64);
                use(p1);
                use(p2);
                use(p3);
                free(p1);
                p1 = aligned_alloc(256, 646656);
                p2 = aligned_alloc(1024, 3232768);
                memset(p2, 0, 3232768);
                p3 = malloc(128);
                use(p1);
                use(p2);
                use(p3);
                free(p2);
                free(p3);
                free(p1);
        }
}

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[acoplan at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

Alex Coplan <acoplan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |acoplan at gcc dot gnu.org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|carlos at redhat dot com           |dj at redhat dot com

--- Comment #21 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to Szabolcs Nagy from comment #20)
> i think i ran into this or a similar issue, but the proposed patch "[v4,1/1]
> memalign: Support scanning for aligned chunks." does not seem to help.

The patch in question is here:
https://patchwork.sourceware.org/project/glibc/patch/xn4jz19fts.fsf@greed.delorie.com/

I pinged DJ to look at the test case. Thanks for putting something together.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[dj at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #22 from dj at redhat dot com <dj at redhat dot com> ---
(In reply to Szabolcs Nagy from comment #20)
> example: 100 iterations where each iteration leaks 16+64 bytes, everything
> else is freed, but every iteration seems to increase the heap by 3231744
> bytes at least. with GLIBC_TUNABLES=glibc.malloc.mmap_threshold=600000 there
> is no issue (large allocations turn into mmap/munmap pairs).

I did a bunch of testing with this example, but didn't see continued growth. 
Typically, after a few loops, malloc learned to use one of the large chunks for
the 16/64 and to reuse all the others.  I tried with and without tcache and
fastbins in case those made a difference.  Note: this is *without* my memalign
patch.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[nsz at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #23 from Szabolcs Nagy <nsz at gcc dot gnu.org> ---
i tried on both aarch64 and x86_64 upstream master and if i add vmstat() at the
very end of main with

static void vmstat(void)
{
        char buf[1000];
        FILE *f = fopen("/proc/self/status","r");
        while (fgets(buf, sizeof buf, f))
                if (buf[0] == 'V') fputs(buf, stdout);
        fclose(f);
}

then it prints

VmPeak:   316888 kB
VmSize:   316888 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:    314636 kB
VmRSS:    314636 kB
VmData:   314652 kB
VmStk:       132 kB
VmExe:         8 kB
VmLib:      1480 kB
VmPTE:       660 kB
VmSwap:        0 kB

i.e. >300MB is used (if compiled with -O1 then memset is avoided so RSS is
small but VmPeak is still big). i also see >100 brk calls in strace. i dont
know what might be different in my environment.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[nsz at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #24 from Szabolcs Nagy <nsz at gcc dot gnu.org> ---
(In reply to dj@redhat.com from comment #22)
> I did a bunch of testing with this example, but didn't see continued growth.
> Typically, after a few loops, malloc learned to use one of the large chunks

ok after some tries it seems i don't get large vm usage on x86_64 with static
linking, but if i add

use(malloc(12345));

at the begining of main then i do. so it seems the initial heap state matters
(which can be different between static/dynamic linking or depend on the
environment).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/14581] memalign allocations are often not reused after free

[dj at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14581

--- Comment #25 from dj at redhat dot com <dj at redhat dot com> ---
Hmmm... so it seems this is reproducible if tcache+fastbins are enabled, and
partially reproducible if only fastbins are.  If both are disabled via
tunables, it shows no growth.  I think this is a general side effect of things
in tcache/fastbins/unsorted not allowing coalescing.

-- 
You are receiving this mail because:
You are on the CC list for the bug.