[Bug libc/1065] New: Inefficient malloc / realloc behavior

[Bug libc/1065] New: Inefficient malloc / realloc behavior

[fritz at intrinsity dot com]

I'm seeing some inefficient behavior on the part of malloc when comparing a
Scientific Linux 3.0.2 box when compared to a Red Hat 8.0 box.  In addition
to the inefficient behavior, I'm also seeing a bogus message when using the
MALLOC_CHECK_ environment variable.

I'm sorry to leave the "triplet" boxes above blank - I have no idea what to
enter there.  Hopefully the information I've included about gcc version, kernel
version, etc, will answer those questions.  I'd be more than willing to take
action to fill in those blanks, I just don't know what to do there.

Below I've got information listed for the SL box versus the RH box, along
with the simple short program that shows the behavior.  Let me know if there
is any other info needed to understand the problem.

Scientific Linux box
--------------------
CPU speed as listed in /proc/cpuinfo:
cpu MHz         : 1468.511

info listed when telnet'ing into box:
Scientific Linux SL Release 3.0.2 (SL)
Kernel 2.4.21-15.0.2.ELsmp on an i686

% uname -a
Linux sim010.farm.intrinsity.com 2.4.21-15.0.2.ELsmp #1 SMP Fri Jun 18 12:05:01
CDT 2004 i686 athlon i386 GNU/Linux

% gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux
Thread model: posix
gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-34)

% g++ -o prob prob.cc

% ./prob
out of memory at i 46674

Last info from a top run (redirected to a file) and examined after the
prob executable ended:
27504 fritz     25   0  168M 168M   604 R    93.5 16.7   1:49 prob


% setenv MALLOC_CHECK_ 1
% ./prob
malloc: using debugging hooks
malloc: top chunk is corrupt
out of memory at i 5901
27456 fritz     21   0 22920  22M   636 R    89.4  2.2   0:00 prob

Red Hat 8 machine
-----------------
CPU speed as listed in /proc/cpuinfo:
cpu MHz         : 1195.354

info listed when telnet'ing into the box:
Red Hat Linux release 8.0 (Psyche)
Kernel 2.4.18-14 on an i686

% uname -a
Linux schrems.eng.intrinsity.com 2.4.18-14 #1 Wed Sep 4 12:13:11 EDT 2002 i686
athlon i386 GNU/Linux

% g++ -o prob.rh8 prob.cc

% gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --host=i386-redhat-linux --with-system-zlib --enable-__cxa_atexit
Thread model: posix
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)



./prob.rh8
28220 fritz     15   0 2925M 777M   252 D     0.4 77.1   0:20 prob.rh8
out of memory at i 1882141


% setenv MALLOC_CHECK_ 1
% ./prob.rh8
out of memory at i 1872841
28287 fritz     17   0 2940M 766M   368 R    11.0 76.1   0:20 prob.rh8

----------------------------------------------------------------------

/* start of prob.cc */
#include <stdlib.h>
#include <stdio.h>

int main()
{
    for (int i = 0; i < 3000000; i++)
        {
        char *p = (char*)malloc(500000);
        if (p == NULL)
            {
            printf("out of memory at i %d\n", i);
            exit(1);
            }
        realloc(p, 500);
        malloc(1000);
        }
}
/* end of prob.cc */

Some notes:

1) The Scientific Linux box is a faster machine than the Red Hat 8.0 box.
Despite this, the executable takes 1:49 of cpu time on the SL box compared to
0:20 cpu time on the RH box.
2) The SL box has malloc report back via a NULL return that no more memory
is available, despite the image of the executable being 168 MB (on a 1 GB box).
The RH box has malloc fail via a NULL return when the executable is
at an image size of 2925 MB (also on a 1 GB box).  Seems like a bug that with
800+ MB of RAM available (not including swap) that the malloc would fail.
3) When the MALLOC_CHECK_ environment variable is used, the SL box informs
me that the "top chunk is corrupt" before returning NULL.  The image size only
makes it to 22 MB.  The RH box when using MALLOC_CHECK_ makes it to a nearly
identical sized image as when the MALLOC_CHECK_ is not used and in nearly
the same amount of time.  Also, very importantly, it does not complain that
the top chunk is corrupt.
4) I can't be sure that the above program captures the problem I'm seeing in
my executable.  I took a wild guess to arrive at the above program which shows
the same symptoms.  I was able to get my executable to run by using large areas
of stack space instead of large areas of heap space, thus avoiding the
malloc/realloc behavior I'm seeing.

-- 
           Summary: Inefficient malloc / realloc behavior
           Product: glibc
           Version: 2.3.2
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: gotom at debian dot or dot jp
        ReportedBy: fritz at intrinsity dot com
                CC: glibc-bugs at sources dot redhat dot com


http://sources.redhat.com/bugzilla/show_bug.cgi?id=1065

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/1065] Inefficient malloc / realloc behavior

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2005-09-27 15:26 -------
Bring this up with your vendor (whoever produces "Scientific Linux").

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID


http://sourceware.org/bugzilla/show_bug.cgi?id=1065

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/1065] Inefficient malloc / realloc behavior

[fritz at intrinsity dot com]

------- Additional Comments From fritz at intrinsity dot com  2005-09-27 15:54 -------
(In reply to comment #1)
> Bring this up with your vendor (whoever produces "Scientific Linux").

Hey Ulrich,

I had originally contacted the Scientific Linux folks and filed it as a
bug on them.  Here's what came of that:

----------------------------------------

Fritz,

Scientific Linux is a rebuild of RedHat Enterprise Linux.  We do not
have the resources to fix problems not associated with our rebuild.

So in your case the problem is with a GNU product I would suggest you go
to GNU.

In some cases the problem should be reported to RedHat but this gets to be
sticky as RedHat has absolutely nothing to do with Scientific Linux.  So
if a bug is reported to them then it should be verified that the problem
exists on their product and the problem then should be reported against
their product.

We will add more info to the bug page to clarify these issues.

If you really need issues such as what you reported fixed then I suggest
you purchase a Linux Distribution that includes support from the vendor.

-Connie Sieh

----------------------------------------

I had also tried the GNU site.  After many bounced messages to multiple
addresses, I finally got it to stick at bug-glibc@gnu.org.  Then I got this
response:

To: Fritz Boehm <fritz@intrinsity.com>
From: Petter Reinholdtsen <pere@hungry.com>
You might want to know that the glibc developers no longer reads this
list, due to the amount of spam on it.  You will have better success
reporting your bug to <URL:http://sources.redhat.com/bugzilla/>.

----------------------------------------

I don't have the resources to obtain RedHat Enterprise.  I already spent 6 days
here at work isolating the problem.  I'm trying to do my part in feeding back
into the system the bugs I find.  I can see where things are sticky with all
these releases, but that is beyond my control.  It would be a shame if this bug
was lost, but I'm not going to push it any more.  If RedHat wants to summarily
drop this, I can't blame them or you, but this is as far as I can go with
this matter.

Fritz

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=1065

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.