Silent stack-heap collision under GNU/Linux

[Vincent Lefevre <vincent+gcc@vinc17.org>]

It appears that GCC can generate code that yields silent stack-heap
collision under GNU/Linux. I mean, the program doesn't crash (at least
not immediately), the memory just gets corrupted. At the same time,
this overrides the stack-size limit defined at the kernel level
(getrlimit system call / RLIMIT_STACK) because the kernel has no
chance to detect the collision (no page fault); thus this limit
doesn't protect the user, and the problem seems to be on GCC's side.

Why aren't such collisions detected by default?

How can one tell GCC to detect them?

Here's a test case:

------------------------------------------------------------------------
#include <stdio.h>

static char a = 0;
static unsigned long pa, pb, pc;

#define GETADDR(V) \
  do { p##V = (unsigned long) &V; printf ("&" #V " = %016lx\n", p##V); } \
  while (0)

void foo (unsigned long s)
{
  char c[s];

  GETADDR(c);
  if (pa >= pc)
    c[pa - pc] = 1;
  else
    printf ("Cannot test.\n");
}

int main (int argc, char **argv)
{
  char b;

  GETADDR(a);
  GETADDR(b);

  printf ("a = %d\n", a);
  if (pb > pa)
    foo (pb - pa);
  else
    printf ("Cannot test.\n");
  printf ("a = %d\n", a);

  return 0;
}
------------------------------------------------------------------------

I get something like:

&a = 0000000000600b20
&b = 00007fffbfea7d2f
a = 0
&c = 0000000000600ac0
a = 1

Same problem with the 32-bit ABI (-m32).

With GCC 4.9.1 (Debian/unstable), the program terminates successfully.
With the 4.10.0 snapshot, I also get a segmentation fault at the end,
but that's too late.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)