[Bug libc/13137] New: malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly

[Bug libc/13137] New: malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly

[heuler at infosim dot net]

http://sourceware.org/bugzilla/show_bug.cgi?id=13137

             Bug #: 13137
           Summary: malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not
                    working correctly
           Product: glibc
           Version: 2.12
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper.fsp@gmail.com
        ReportedBy: heuler@infosim.net
    Classification: Unclassified


We have an huge application which makes very heavy use of threads (typical 500
- 2000 threads running in parallel for task handling). In one setup around 5 GB
of physical memory are used. 
The same application on Windows (using MS VC++) uses only 60% with exactly the
same configuration. The reason for that is the glibc malloc() implementation
using multiple arenas to prevent thread contention. Using malloc_info() we
found out that 2 - 3 GB of memory are unused (free) in the memory management.
As that free memory is distributed between the areans it can't be reclaimed.
The application uses 64 arenas (maximum is 8 per core? we have 8 core test
system) and our threads  depending on the task need sometimes a lot of memory
(several 100MB). Since the thread if it has bad luck gets uses only one arena
it can't use the free memory of the others.

Our solution was to try to limit the number of arenas in the malloc
implementation. For that the M_ARENA_MAX / MALLOC_ARENA_MAX parameters
(environment or mallopt() are available. According to the documentation they
define the maximum number of arenas used.
In our tests we configured for only 1 arena. But the application tends to use
around 10 arenas which is more than the specified maximum. It is of course less
than the 64 arenas if the parameters not sed and works also better in terms of
free memory fragmentation.
But the limit is exceed which seems to be a bug since the documentation also in
Ulrich Dreppers blog defines that as a hard limit.

We tested e.g. on RHEL 6 with latest updates.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[heuler at infosim dot net]

http://sourceware.org/bugzilla/show_bug.cgi?id=13137

Marius Heuler <heuler at infosim dot net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|malloc M_ARENA_MAX /        |malloc M_ARENA_MAX /
                   |MALLOC_ARENA_MAX limit not  |MALLOC_ARENA_MAX limit not
                   |working correctly           |working correctly (limit
                   |                            |exceeded)

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13137

--- Comment #1 from Andreas Schwab <schwab@linux-m68k.org> 2011-11-09 14:54:35 UTC ---
The minimum arena limit is 8.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[jsm28 at gcc dot gnu.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13137

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libc                        |malloc

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[siddhesh at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13137

Siddhesh Poyarekar <siddhesh at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING
                 CC|                            |siddhesh at redhat dot com

--- Comment #2 from Siddhesh Poyarekar <siddhesh at redhat dot com> 2012-10-03 18:05:04 UTC ---
This should have been fixed by the following commit:

commit 41b81892f11fe1353123e892158b53de73863d62
Author: Ulrich Drepper <drepper@gmail.com>
Date:   Tue Jan 31 14:42:34 2012 -0500

    Handle ARENA_TEST correctly

If you can verify with glibc-2.16, then we can close this. Otherwise please
provide a reproducer so that we can analyze and fix the problem.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[siddhesh at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13137

Siddhesh Poyarekar <siddhesh at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |FIXED

--- Comment #3 from Siddhesh Poyarekar <siddhesh at redhat dot com> 2012-10-17 09:17:47 UTC ---
Closing as fixed. Please reopen if you're able to reproduce the problem.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[jackie.rosen at hushmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=13137

Jackie Rosen <jackie.rosen at hushmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jackie.rosen at hushmail dot com

--- Comment #4 from Jackie Rosen <jackie.rosen at hushmail dot com> ---
*** Bug 260998 has been marked as a duplicate of this bug. ***
Seen from the domain http://volichat.com
Page where seen: http://volichat.com/adult-chat-rooms
Marked for reference. Resolved as fixed @bugzilla.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[schwab at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=13137

Andreas Schwab <schwab at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|jackie.rosen at hushmail dot com   |

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=13137

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/13137] malloc M_ARENA_MAX / MALLOC_ARENA_MAX limit not working correctly (limit exceeded)

[namboru at jasaseo dot one]

https://sourceware.org/bugzilla/show_bug.cgi?id=13137

--- Comment #5 from namboru <namboru at jasaseo dot one> ---
Kami adalah jasa seo anda https://seohandal.id

-- 
You are receiving this mail because:
You are on the CC list for the bug.