Malloc not trimming sbrk space once it fills up

Malloc not trimming sbrk space once it fills up

[Jakub Jelinek]

[-- Attachment #1: Type: text/plain, Size: 206 bytes --]

Hi!

The attached testcase shows that once morecore fails for the first time
and malloc needs to fall back into mmap arena, no more sbrk trimming
happens even if all chunks in mmap areas are freed.

	Jakub

[-- Attachment #2: tst-malloc-trim.c --]
[-- Type: text/plain, Size: 1933 bytes --]

/* Copyright (C) 2004 Free Software Foundation, Inc.
   This file is part of the GNU C Library.
   Contributed by Jakub Jelinek <jakub@redhat.com>, 2004.

   The GNU C Library is free software; you can redistribute it and/or
   modify it under the terms of the GNU Lesser General Public
   License as published by the Free Software Foundation; either
   version 2.1 of the License, or (at your option) any later version.

   The GNU C Library is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
   Lesser General Public License for more details.

   You should have received a copy of the GNU Lesser General Public
   License along with the GNU C Library; if not, write to the Free
   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
   02111-1307 USA.  */

#include <malloc.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/mman.h>
#include <unistd.h>

ptrdiff_t sum;

__malloc_ptr_t
counting_morecore (ptrdiff_t size)
{
  sum += size;
  return __default_morecore (size);
}

int
main (void)
{
  size_t pagesize = sysconf (_SC_PAGESIZE);

  char *p = sbrk (0);
  p += (128 * pagesize) & ~(pagesize - 1);
  if (mmap (p, pagesize, PROT_NONE, MAP_PRIVATE|MAP_ANON, -1, 0) != p)
    {
      puts ("Couldn't test");
      return 0;
    }

  if (mallopt (M_TRIM_THRESHOLD, 32 * pagesize) == 0)
    {
      puts ("Couldn't test");
      return 0;
    }

  __morecore = counting_morecore;

  char *array[512];
  int i;
  for (i = 0; i < 512; ++i)
    array[i] = malloc (pagesize / 4);

  ptrdiff_t midsum = sum;

  for (i = 511; i >= 0; --i)
    free (array[i]);

  __morecore = __default_morecore;

  if (midsum == sum && midsum > 32 * pagesize)
    {
      printf ("free did not trim sbrk area at all: midsum %td sum %td\n",
	      midsum, sum);
      return 1;
    }
  return 0;
}

Re: Malloc not trimming sbrk space once it fills up

[wmglo]

Hi,

> The attached testcase shows that once morecore fails for the first time
> and malloc needs to fall back into mmap arena, no more sbrk trimming
> happens even if all chunks in mmap areas are freed.

Thanks for the testcase, but this is to be expected.

Do you think this course of events is common enough to warrant a
change?

I'll take a look whether it can be changed easily.

Regards,
Wolfram.

Re: Malloc not trimming sbrk space once it fills up

[Jakub Jelinek]

On Wed, Jan 07, 2004 at 06:28:58PM -0000, wmglo@dent.med.uni-muenchen.de wrote:
> Hi,
> 
> > The attached testcase shows that once morecore fails for the first time
> > and malloc needs to fall back into mmap arena, no more sbrk trimming
> > happens even if all chunks in mmap areas are freed.
> 
> Thanks for the testcase, but this is to be expected.
> 
> Do you think this course of events is common enough to warrant a
> change?

We've got a bug report about it, so probably some people ran into it.
I can imagine some programs which normally don't use too much memory
but at peak times need a few gigs.
It is less urgent with the kernel changes in RHEL3/Fedora Core 1
where mmap allocates from top to bottom, so most of the memory
is usable as brk area unless mmap is used with non-NULL first argument.

> I'll take a look whether it can be changed easily.

Thanks.

	Jakub