[Bug libc/3455] New: ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

[Bug libc/3455] New: ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

[cseddon at sequencedesign dot com]

While ensuring that a program built on RHL72 would run on RHEL3, I discovered
that the program would run out of memory prematurely.

I obtained a version of ptmalloc, and, while debugging, determined that a call
to _int_free() from sYSMALLOc() was being made to free up old_top().  This
results in a call to sYSTRIm() to trim off the any unused memory.  However, in
that flow, it ends up clearing out all of the newly-allocated memory that was
just requested.

(The MMAP base note is due to an internal requirement, and the 65535 value is
 one taken from a third-party's sbrk() call, which caused our problem)

Here is a sample program:
#include <malloc.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h> // sbrk

const int oneMeg = 1024*1024;
//const int oneMeg = 1024;
void* ourMalloc(size_t size) {
    void* retVal = malloc(size);
    if (retVal > (void*)0x80000000) {
        printf("MMAP base exceeded\n");
    }
    return retVal;
}

main() {
    void* p;
    void* firstP = 0;
    void* lastP;
    int totalAllocs = 0;
    mallopt(M_MMAP_MAX, -1);
    while (p = malloc(oneMeg)) {
        if (!firstP) {
            firstP = p;
            // introduce a 'foreign sbrk'
            void* brk = sbrk(0);
            printf("b4 sbrk: %p\n", brk);
            sbrk(65535);
            brk = sbrk(0);
            printf("after sbrk: %p\n", brk);
        }
        lastP = p; 
        totalAllocs++;
        memset(p, 'a', oneMeg);
    }
    printf("started @ %p\n", firstP);
    printf("ended @ %p\n", lastP);
    printf("total: %u\n", totalAllocs * oneMeg);
}

This programs runs fine on RHL72:

cseddon@power-linux-01:~ % ./a.out
b4 sbrk: 0x804988c
after sbrk: 0x805988b
started @ 0x40164008
ended @ 0x3fe5b488
total: 3075473408

On RHEL3, however:

cseddon@platform-rhel3-01:~ % ~/a.out
b4 sbrk: 0x816b000
after sbrk: 0x817afff
started @ 0x8049898
ended @ 0x8049898
total: 1048576

And, when run with trimming shut off it again allocates the max memory:

cseddon@platform-rhel3-01:~ % setenv MALLOC_TRIM_THRESHOLD_ -1
cseddon@platform-rhel3-01:~ % a.out
b4 sbrk: 0x816b000
after sbrk: 0x817afff
started @ 0x8049898
ended @ 0xbfe9a008
total: 3068133376


Our workaround is to shut off trimming usin mallopt(M_TRIM_THRESHOLD, -1)

-charlie

-- 
           Summary: ptmalloc fails if foreign sbrk and no mmap and size is
                    greater than  trim threshold
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper at redhat dot com
        ReportedBy: cseddon at sequencedesign dot com
                CC: glibc-bugs at sources dot redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=3455

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/3455] ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2006-11-03 17:44 -------
You are not allowed to use sbrk.  The heap belongs to the runtime.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID


http://sourceware.org/bugzilla/show_bug.cgi?id=3455

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/3455] ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

[cseddon at sequencedesign dot com]

------- Additional Comments From cseddon at sequencedesign dot com  2006-11-03 19:22 -------
_I_ did not use sbrk().  A third-party package did.

My test program does, but only to provide a demostration of the problem.

malloc has all sorts of special-casing in it for handling 'foreign sbrk'.  It
appears to be programmed to handle the foreign sbrk(), but fails to do so.

If it is impossible to use sbrk(), then shouldn't it's prototype be removed?

-charlie

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |


http://sourceware.org/bugzilla/show_bug.cgi?id=3455

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/3455] ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2006-11-03 20:58 -------
Don't reopen.  Any program using sbrk is broken.  Period.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID


http://sourceware.org/bugzilla/show_bug.cgi?id=3455

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/3455] ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

[cseddon at sequencedesign dot com]

------- Additional Comments From cseddon at sequencedesign dot com  2006-11-04 13:40 -------
Why, then, is there so much code in ptmalloc to handle foreign sbrk()'s?

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=3455

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.