Segfault in MAP_NORESERVE mmap above ~4GB

Segfault in MAP_NORESERVE mmap above ~4GB

[Erik Bray]

Hi all,

This issue pertains to Cygwin 64-bit.  The following example program
demonstrates the issue:

$ cat mmap_test.c
#include <sys/mman.h>
#include <stdio.h>
#include <windows.h>


#define VSIZE 0x100001000
#define SIZE 0x1000


void foo() {
    void *top, *bot, *c;

    c = mmap(NULL, VSIZE, PROT_READ|PROT_WRITE,
             MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0);

    top = c + VSIZE;
    bot = top - SIZE;

    printf("     c = 0x%016lx\n", c);
    printf("   top = 0x%016lx\n", top);
    printf("   bot = 0x%016lx\n", bot);

    printf("  c[0] = %ul\n", *((unsigned long *)c));
    printf("bot[0] = %ul\n", *((unsigned long *)bot));
}


int main(void) {
    foo();
    return 0;
}

$ gcc mmap_test.c -o mmap_test

$ ./mmap_test.exe
     c = 0x000006feffff0000
   top = 0x000006ffffff1000
   bot = 0x000006ffffff0000
  c[0] = 0l
Segmentation fault (core dumped)

--------------------------------------------------

As you can see, the address stored in `bot` is within the mmap'd
region, but trying to access it results in an access violation, while
addresses low in the region can be accessed.  This is only an issue
because of MAP_NORESERVE, so when the mmap is created it only reserves
an address range for it but does not commit any resources.

Instead, when Cygwin's exception handler receives a
STATUS_ACCESS_VIOLATION [1] it calls into
mmap_is_attached_or_noreserve [2] with the address associated with
access violation.  This searches the records of existing mmap'd
regions until it finds a matching region, and commits enough of that
region to be able to service the memory access.

This search works for lower addresses in the region, but fails for
higher addresses.  From a bit of debugging it seems this is occurring
because the constructor for mmap_record is squeezing the mmap length
into a DWORD [3], and the rest of the problems stem from there.  There
are a few other places related to mmap_record that seem to be treating
the mmap length as a DWORD instead of a size_t.  I would offer a patch
but I've never built Cygwin before so I haven't tested this yet.

Thanks,
Erik

[1] https://github.com/openunix/cygwin/blob/99590589326b5537d549cdd41ca4177ce7051d4a/winsup/cygwin/exceptions.cc#L722
[2] https://github.com/openunix/cygwin/blob/99590589326b5537d549cdd41ca4177ce7051d4a/winsup/cygwin/mmap.cc#L704
[3] https://github.com/openunix/cygwin/blob/99590589326b5537d549cdd41ca4177ce7051d4a/winsup/cygwin/mmap.cc#L275

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Segfault in MAP_NORESERVE mmap above ~4GB

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1719 bytes --]

Hi Erik,

On May 12 15:30, Erik Bray wrote:
> Hi all,
> 
> This issue pertains to Cygwin 64-bit.  The following example program
> demonstrates the issue:
> 
> $ cat mmap_test.c
> #include <sys/mman.h>
> #include <stdio.h>
> #include <windows.h>
> 
> 
> #define VSIZE 0x100001000
> #define SIZE 0x1000
> 
> 
> void foo() {
>     void *top, *bot, *c;
> 
>     c = mmap(NULL, VSIZE, PROT_READ|PROT_WRITE,
>              MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0);
> 
>     top = c + VSIZE;
>     bot = top - SIZE;
> 
>     printf("     c = 0x%016lx\n", c);
>     printf("   top = 0x%016lx\n", top);
>     printf("   bot = 0x%016lx\n", bot);
> 
>     printf("  c[0] = %ul\n", *((unsigned long *)c));
>     printf("bot[0] = %ul\n", *((unsigned long *)bot));
> }
> 
> 
> int main(void) {
>     foo();
>     return 0;
> }
> 
> $ gcc mmap_test.c -o mmap_test
> 
> $ ./mmap_test.exe
>      c = 0x000006feffff0000
>    top = 0x000006ffffff1000
>    bot = 0x000006ffffff0000
>   c[0] = 0l
> Segmentation fault (core dumped)
> 
> --------------------------------------------------

Thanks especially for the testcase.  I just applied a fix for this
which, hopefully, catches all problems with too small length/size
variables and parameters.

I've uploaded a new developer snapshot to https://cygwin.com/snapshots/

Would you mind to inspect the patch(*) critically?


Thanks a lot,
Corinna


(*) https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=commitdiff;h=0aa738220bb9dea2ad479e484560767b36701947

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]