[Bug libc/11261] malloc uses excessive memory for multi-threaded applications

["rich at testardi dot com" <sourceware-bugzilla@sourceware.org>]

http://sourceware.org/bugzilla/show_bug.cgi?id=11261

--- Comment #9 from Rich Testardi <rich at testardi dot com> 2011-08-27 22:02:03 UTC ---
Hi,

We ended up building our own memory allocator -- it's faster and more efficient
than glibc, and it works equally fast with threads and wihout.

We used the "small block allocator" concept from HP-UX where we only allocate
huge (32MB) allocations from the system (after setting M_MMAP_THRESHOLD
suitably small).

We then carve out large *naturally aligned* 1MB blocks from the huge allocation
(accepting 3% waste, since the allocation was page alogned to begin with, not
naturally aligned).

And we carve each one of those large blocks into small fixed size buckets
(which are fractional powers of 2 -- like 16 bytes, 20, 24, 28, 32, 40, 48, 56,
64, 80, etc.).

Then we put the aligned addresses into a very fast hash and have a linked list
for each bucket size.

This means our allocate routine is just a lock, linked list remove, unlock, on
average, and our free routine is just a hash lookup, lock, linked list insert,
unlock on average.

The trick here is that from any address being freed, you can get back to the
naturally aligned 1MB block that contains it with just a pointer mask, and from
there you can get the allocation's size as well as the head of the linked list
of free entries to which it should be returned...

-- Rich

  ----- Original Message ----- 
  From: heuler at infosim dot net 
  To: rich@testardi.com 
  Sent: Saturday, August 27, 2011 3:45 PM
  Subject: [Bug libc/11261] malloc uses excessive memory for multi-threaded
applications


  http://sourceware.org/bugzilla/show_bug.cgi?id=11261

  Marius Heuler <heuler at infosim dot net> changed:

             What    |Removed                     |Added
  ----------------------------------------------------------------------------
               Status|RESOLVED                    |REOPENED
                   CC|                            |heuler at infosim dot net
           Resolution|WONTFIX                     |

  --- Comment #8 from Marius Heuler <heuler at infosim dot net> 2011-08-27
21:45:04 UTC ---
  We have exactly the same problem with the current implementation of malloc.

  The suggested solutions by Ulrich using M_ARENA_MAX does not work since the
  check for number of arenas is not thread safe. In fact the limit is not
working
  for heay threading applications where that would be needed! 

  Since the number of cores and usage of threads will increase strongly there
  should be a solution for that kind of applications! If the arena limit would
  work as described we would have no problem.

  -- 
  Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
  ------- You are receiving this mail because: -------
  You reported the bug.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.