[Bug nptl/18988] New: pthread wastes memory with mlockall(MCL_FUTURE)

[Bug nptl/18988] New: pthread wastes memory with mlockall(MCL_FUTURE)

[rlblaster at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18988

            Bug ID: 18988
           Summary: pthread wastes memory with mlockall(MCL_FUTURE)
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: nptl
          Assignee: unassigned at sourceware dot org
          Reporter: rlblaster at gmail dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

For each new thread pthread allocates the stack for it and some guard empty
pages before the stack to avoid stack overflows. The problem is that on Linux
with mlockall(MCL_FUTURE) these guard pages are actually taking up real
physical memory leading to a lot of memory wasted if you have a lot of threads
and large guardsize (either for safety or you just have large pages).

Here is a simple application demonstrating the problem:

/* gcc threads.c -lphreads */
#include <assert.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>

void *sleeper(void *arg)
{
  (void) arg;
  sleep(100);
  return NULL;
}

int main(void)
{
  int i, r;
  pthread_attr_t a;
  r = mlockall(MCL_CURRENT | MCL_FUTURE);
  assert(r == 0);
  r = pthread_attr_init(&a);
  assert(r == 0);
  r = pthread_attr_setguardsize(&a, 1<<20);
  assert(r == 0);

  puts("memory before threads");
  system("grep MemAvailable /proc/meminfo");
  for (i = 0; i < 1000; i++) {
    pthread_t t;
    r = pthread_create(&t, &a, sleeper, NULL);
    assert(r == 0);
  }
  puts("memory after threads");
  system("grep MemAvailable /proc/meminfo");
  return 0;
}


Now let's run it on a x86_64, linux kernel version 4.1.6:
$ gcc threads.c -lpthread && ./a.out
memory before threads
MemAvailable:   31960328 kB
memory after threads
MemAvailable:   23729272 kB

The problem is that glibc allocates memory via mmap(..., PROT_READ|PROT_WRITE,
...) and then does a mprotect(PROT_NONE) on the guard pages. This needs to be
switched to mmap(..., PROT_NONE, ...) and then mprotect(PROT_READ|PROT_WRITE)
on the stack pages. musl does this and thus does not have this problem:

$ musl-gcc threads.c -lpthread && ./a.out
memory before threads
MemAvailable:   31718320 kB
memory after threads
MemAvailable:   31693888 kB

There is some further discussion at
https://sourceware.org/ml/libc-alpha/2015-09/msg00500.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.