public inbox for glibc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug libc/11261] New: malloc uses excessive memory for multi-threaded applications
@ 2010-02-08 20:23 rich at testardi dot com
  2010-02-09 15:28 ` [Bug libc/11261] " drepper at redhat dot com
                   ` (6 more replies)
  0 siblings, 7 replies; 26+ messages in thread
From: rich at testardi dot com @ 2010-02-08 20:23 UTC (permalink / raw)
  To: glibc-bugs

malloc uses excessive memory for multi-threaded applications

The following program demonstrates malloc(3) using in excess of 600 megabytes 
of system memory while the program has never allocated more than 100 megabytes 
at any given time.  This results from the use of thread-specific "preferred 
arenas" for memory allocations.

The program first starts by contending a number of threads doing simple 
malloc/frees, with no net memory allocations.  This establishes preferred 
arenas for each thread as a result of USE_ARENAS and PER_THREADS.  Once 
preferred arenas are established, the program then has each thread, in turn, 
allocate 100 megabytes and then free all but 20 kilobytes, for a net memory 
allocation of 200 kilobytes.  The resulting malloc_stats() show 600 megabytes 
of allocated memory that cannot be returned to the system.

Over time, fragmentation of the heap can cause excessive paging when actual 
memory allocation never exceeded system capacity.  With the use of preferred 
arenas in this way, multi-threaded program memory usage is essentially 
unbounded (or bounded to the number of threads times the actual memory usage).

The program run and source code is below, as well as the glibc version from my 
RHEL5 system.  Thank you for your consideration.

[root@lab2-160 test_heap]# ./memx
creating 10 threads
allowing threads to contend to create preferred arenas
display preferred arenas
Arena 0:
system bytes     =     135168
in use bytes     =       2880
Arena 1:
system bytes     =     135168
in use bytes     =       2224
Arena 2:
system bytes     =     135168
in use bytes     =       2224
Arena 3:
system bytes     =     135168
in use bytes     =       2224
Arena 4:
system bytes     =     135168
in use bytes     =       2224
Arena 5:
system bytes     =     135168
in use bytes     =       2224
Total (incl. mmap):
system bytes     =     811008
in use bytes     =      14000
max mmap regions =          0
max mmap bytes   =          0
allowing threads to allocate 100MB each, sequentially in turn
thread 3 alloc 100MB
thread 3 free 100MB-20kB
thread 5 alloc 100MB
thread 5 free 100MB-20kB
thread 7 alloc 100MB
thread 7 free 100MB-20kB
thread 2 alloc 100MB
thread 2 free 100MB-20kB
thread 0 alloc 100MB
thread 0 free 100MB-20kB
thread 8 alloc 100MB
thread 8 free 100MB-20kB
thread 4 alloc 100MB
thread 4 free 100MB-20kB
thread 6 alloc 100MB
thread 6 free 100MB-20kB
thread 9 alloc 100MB
thread 9 free 100MB-20kB
thread 1 alloc 100MB
thread 1 free 100MB-20kB
Arena 0:
system bytes     =  100253696
in use bytes     =      40928
Arena 1:
system bytes     =  100184064
in use bytes     =      42352
Arena 2:
system bytes     =  100163584
in use bytes     =      22320
Arena 3:
system bytes     =  100163584
in use bytes     =      22320
Arena 4:
system bytes     =  100163584
in use bytes     =      22320
Arena 5:
system bytes     =  100204544
in use bytes     =      62384
Total (incl. mmap):
system bytes     =  601133056
in use bytes     =     212624
max mmap regions =          0
max mmap bytes   =          0
[root@lab2-160 test_heap]# rpm -q glibc
glibc-2.5-42.el5_4.2
glibc-2.5-42.el5_4.2
[root@lab2-160 test_heap]# 

====================================================================

[root@lab2-160 test_heap]# cat memx.c
// ****************************************************************************

#include <stdio.h>
#include <errno.h>
#include <assert.h>
#include <stdlib.h>
#include <pthread.h>
#include <inttypes.h>

#define NTHREADS  10
#define NALLOCS  10000
#define ALLOCSIZE  10000

static volatile int go;
static volatile int die;
static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

static void *ps[NALLOCS];  // allocations that are freed in turn by each thread
static void *pps1[NTHREADS];  // straggling allocations to prevent arena free
static void *pps2[NTHREADS];  // straggling allocations to prevent arena free

void
my_sleep(
    int ms
    )
{
    int rv;
    struct timespec ts;
    struct timespec rem;

    ts.tv_sec  = ms / 1000;
    ts.tv_nsec = (ms % 1000) * 1000000;
    for (;;) {
        rv = nanosleep(&ts, &rem);
        if (! rv) {
            break;
        }
        assert(errno == EINTR);
        ts = rem;
    }
}

void *
my_thread(
    void *context
    )
{
    int i;
    int rv;
    void *p;

    // first we spin to get our own arena
    while (go == 0) {
        p = malloc(ALLOCSIZE);
        assert(p);
        if (rand()%20000 == 0) {
            my_sleep(10);
        }
        free(p);
    }

    // then we give main a chance to print stats
    while (go == 1) {
        my_sleep(100);
    }
    assert(go == 2);

    // then one thread at a time, do our big allocs
    rv = pthread_mutex_lock(&mutex);
    assert(! rv);
    printf("thread %d alloc 100MB\n", (int)(intptr_t)context);
    for (i = 0; i < NALLOCS; i++) {
        ps[i] = malloc(ALLOCSIZE);
        assert(ps[i]);
    }
    printf("thread %d free 100MB-20kB\n", (int)(intptr_t)context);
    // N.B. we leave two allocations straggling
    pps1[(int)(intptr_t)context] = ps[0];
    for (i = 1; i < NALLOCS-1; i++) {
        free(ps[i]);
    }
    pps2[(int)(intptr_t)context] = ps[i];
    rv = pthread_mutex_unlock(&mutex);
    assert(! rv);
}

int
main()
{
    int i;
    int rv;
    pthread_t thread;

    printf("creating %d threads\n", NTHREADS);
    for (i = 0; i < NTHREADS; i++) {
        rv = pthread_create(&thread, NULL, my_thread, (void *)(intptr_t)i);
        assert(! rv);
        rv = pthread_detach(thread);
        assert(! rv);
    }

    printf("allowing threads to contend to create preferred arenas\n");
    my_sleep(20000);

    printf("display preferred arenas\n");
    go = 1;
    my_sleep(1000);
    malloc_stats();

    printf("allowing threads to allocate 100MB each, sequentially in turn\n");
    go = 2;
    my_sleep(5000);
    malloc_stats();

    // free the stragglers
    for (i = 0; i < NTHREADS; i++) {
        free(pps1[i]);
        free(pps2[i]);
    }

    return 0;
}
[root@lab2-160 test_heap]#

-- 
           Summary: malloc uses excessive memory for multi-threaded
                    applications
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper at redhat dot com
        ReportedBy: rich at testardi dot com
                CC: glibc-bugs at sources dot redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=11261

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


^ permalink raw reply	[flat|nested] 26+ messages in thread
end of thread, other threads:[~2015-02-12 20:04 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <bug-11261-131@http.sourceware.org/bugzilla/>
2011-08-27 21:45 ` [Bug libc/11261] malloc uses excessive memory for multi-threaded applications heuler at infosim dot net
2011-08-27 22:02 ` rich at testardi dot com
2011-09-02  7:39 ` heuler at infosim dot net
2011-09-02  7:45 ` heuler at infosim dot net
2011-09-11 15:46 ` drepper.fsp at gmail dot com
2011-09-11 21:32 ` rich at testardi dot com
2012-07-29 10:10 ` zhannk at gmail dot com
2012-12-19 10:47 ` schwab@linux-m68k.org
2013-03-14 19:03 ` carlos at redhat dot com
2013-12-12  0:22 ` neleai at seznam dot cz
2013-12-12  3:32 ` siddhesh at redhat dot com
2013-12-12  8:41 ` neleai at seznam dot cz
2013-12-12 10:48 ` siddhesh at redhat dot com
2014-02-07  3:01 ` [Bug malloc/11261] " jsm28 at gcc dot gnu.org
2014-02-16 19:42 ` jackie.rosen at hushmail dot com
2014-05-28 19:46 ` schwab at sourceware dot org
2014-05-28 19:46 ` schwab at sourceware dot org
2014-06-30 18:50 ` fweimer at redhat dot com
2015-02-12 20:04 ` carlos at redhat dot com
2010-02-08 20:23 [Bug libc/11261] New: " rich at testardi dot com
2010-02-09 15:28 ` [Bug libc/11261] " drepper at redhat dot com
2010-02-09 16:02 ` rich at testardi dot com
2010-02-10 13:10 ` rich at testardi dot com
2010-02-10 13:21 ` drepper at redhat dot com
2010-02-10 13:42 ` rich at testardi dot com
2010-02-10 14:29 ` rich at testardi dot com
2010-02-10 15:52 ` rich at testardi dot com

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).