[Bug libc/2531] New: missing call or documentation for malloc_trim() ...

[Bug libc/2531] New: missing call or documentation for malloc_trim() ...

[sakovacs at freemail dot hu]

Let me describe my issue and you can decide whether it is a bug in libc or just     
missing documentation on the man pages. I've spent a couple of weeks    
debugging my program, it processed big files and allocated lots of small memory    
blocks in the process, free()'d the memory afterwards and RSS still didn't    
decrease. I obviously thought it was a memory leak, valgrinded, etc, until I    
came down to this very simple program:    
    
#include <unistd.h>    
#include <malloc.h>    
int main(int argc, char *argv[]) {    
    int max = 10000000;    
    char** p1 = (char**) malloc(max * sizeof(char*));    
    for (int x = 0; x < max; x++) {    
        p1[x] = (char*) malloc(29);    
    }    
    for (int x = 0; x < max; x++) {    
        free(p1[x]);    
    }    
    free(p1);    
    usleep(20000 * 1000);   // so that we have time to check the RSS.. 
    return 0;    
}    
    
This program runs up to ~500Mb on my system, and even after free()'ing the RSS   
does not shrink.   
   
I think for most of you it is obvious now, that the issue is that free() does    
not return memory to the system. Unfortunately on the man page there is no   
reference to this behaviour, also no reference to malloc_trim() call that can   
return some of the memory (depending on the fragmentation). In this example 
code, if you call malloc_trim(0) right before the usleep() the RSS goes back to 
less than 1M (this is a simple case, no fregmentation, all memory could be 
returned to the OS).  
  
Now, I know that malloc_trim() does not guarantee the return of any memory, but  
I would think that  
A) the linux free() implementation should call it from time to time (based on  
the allocated/free memory ratio?.. I'm not the proper person to define such an  
algorithm)  
B) the man pages should contain the fact that free() never returns allocated  
pages to the OS and that on linux malloc_trim() call can be used to try to 
return pages. 
 
I just would like to avoid others spending hours debugging their programs while 
there is nothing to debug. Looking forward to hear your opinion, 
 
Cheers, 
Sandor

-- 
           Summary: missing call or documentation for malloc_trim() ...
           Product: glibc
           Version: 2.3.5
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper at redhat dot com
        ReportedBy: sakovacs at freemail dot hu
                CC: glibc-bugs at sources dot redhat dot com
 GCC build triplet: x86_64-pc-linux-gnu
  GCC host triplet: x86_64-pc-linux-gnu
GCC target triplet: x86_64-pc-linux-gnu


http://sourceware.org/bugzilla/show_bug.cgi?id=2531

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/2531] missing call or documentation for malloc_trim() ...

[sakovacs at freemail dot hu]

------- Additional Comments From sakovacs at freemail dot hu  2006-04-09 08:55 -------
*** Bug 2530 has been marked as a duplicate of this bug. ***

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2531

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/2531] missing call or documentation for malloc_trim() ...

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2006-04-26 02:15 -------
Why don't you make your little program write out the data:

#include <unistd.h>
#include <stdlib.h>
#include <malloc.h>
#include <stdio.h>
int
main (int argc, char *argv[])
{
  char cmd[100];
  snprintf (cmd, sizeof (cmd), "grep heap /proc/%d/maps || echo no heap",
            getpid ());
  void *before = sbrk (0);
  system (cmd);
  int max = 10000000;
  char **p1 = (char **) malloc (max * sizeof (char *));
  for (int x = 0; x < max; x++)
    {
      p1[x] = (char *) malloc (29);
    }
  for (int x = 0; x < max; x++)
    {
      free (p1[x]);
    }
  free (p1);
  void *middle = sbrk (0);
  system (cmd);
  malloc_trim (0);
  void *after = sbrk (0);
  system (cmd);
  printf ("before = %p, middle = %p, after = %p\n", before, middle, after);
  return 0;
}


When I run this I see

no heap
00501000-1cee1000 rw-p 00501000 00:00 0                                  [heap]
00501000-00502000 rw-p 00501000 00:00 0                                  [heap]
before = 0x501000, middle = 0x1cee1000, after = 0x502000


This clearly shows that

a) libc returns all memory if malloc_trim is used
b) the kernel frees all memory.


So, what's the problem?

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID


http://sourceware.org/bugzilla/show_bug.cgi?id=2531

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/2531] missing call or documentation for malloc_trim() ...

[sakovacs at freemail dot hu]

------- Additional Comments From sakovacs at freemail dot hu  2006-04-26 11:35 -------
  
> a) libc returns all memory if malloc_trim is used  
  
I agree. That's what I said too:  
>... if you call malloc_trim(0) right before the usleep() the RSS goes back to   
> less than 1M (this is a simple case, no fregmentation, all memory could be   
> returned to the OS).  
  
b) the kernel frees all memory.  
  
I agree with this as well.  
  
> So, what's the problem?  
  
All I was saying is that either put malloc_trim() should be referenced in the  
malloc/free man page or free() should call it from time to time -- I since  
read that this should be the case. So a simple "see also" entry would help a 
lot for many guys having similar issues.  
  
Cheers,  
Sandor  
  
  

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2531

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/2531] missing call or documentation for malloc_trim() ...

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2006-05-01 18:41 -------
The man pages are not maintained in glibc.  Tell this to the man page maintainer.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2531

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.