[RFC] Stack allocation, hugepages and RSS implications

[Cupertino Miranda <cupertino.miranda@oracle.com>]

Hi everyone,

For performance purposes, one of ours in-house applications requires to enable
TRANSPARENT_HUGEPAGES_ALWAYS option in linux kernel, actually making the
kernel to force all of the big enough and alligned memory allocations to
reside in hugepages.  I believe the reason behind this decision is to
have more control on data location.

For stack allocation, it seems that hugepages make resident set size
(RSS) increase significantly, and without any apparent benefit, as the
huge page will be split in small pages even before leaving glibc stack
allocation code.

As an example, this is what happens in case of a pthread_create with 2MB
stack size:
 1. mmap request for the 2MB allocation with PROT_NONE;
      a huge page is "registered" by the kernel
 2. the thread descriptor is writen in the end of the stack.
      this will trigger a page exception in the kernel which will make the actual
      memory allocation of the 2MB.
 3. an mprotect changes protection on the guard (one of the small pages of the
    allocated space):
      at this point the kernel needs to break the 2MB page into many small pages
      in order to change the protection on that memory region.
      This will eliminate any benefit of having small pages for stack allocation,
      but also makes RSS to be increaded by 2MB even though nothing was
      written to most of the small pages.

As an exercise I added __madvise(..., MADV_NOHUGEPAGE) right after
the __mmap in nptl/allocatestack.c. As expected, RSS was significantly reduced for
the application.

At this point I am very much confident that there is a real benefit in our
particular use case to enforce stacks not ever to use hugepages.

This RFC is to understand if I have missed some option in glibc that would
allow to better control stack allocation.
If not, I am tempted to propose/submit a change, in the form of a tunable, to
enforce NOHUGEPAGES for stacks.

In any case, I wonder if there is an actual use case where an hugepage would
survive glibc stack allocation and will bring an actual benefit.

Looking forward for your comments.

Best regards,
Cupertino