Re: how to get mmap page size?

Re: how to get mmap page size?

[Jay Foad]

Corinna Vinschen wrote:

> mmap always allocates in 64K chunks.

That doesn't seem to be true in practice. In the test below I mmap a
1080K file (this is a multiple of 4K but not a multiple of 64K). The
first byte after the end of the file isn't readable.

$ cat mmaptest.c
#include <fcntl.h>
#include <stdio.h>
#include <sys/mman.h>
#include <sys/stat.h>
int main(int argc, char *argv[])
{
        int fd;
        struct stat st;
        void *p;
        if (argc != 2) return 1;
        if ((fd = open(argv[1], O_RDONLY, 0)) < 0) return 1;
        if (fstat(fd, &st) != 0) return 1;
        if ((p = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd,
0)) == MAP_FAILED) return 1;
        printf("last byte 0x%02X\n", ((unsigned char *)p)[st.st_size - 1]);
        fflush(stdout);
        printf("next byte 0x%02X\n", ((unsigned char *)p)[st.st_size]);
        return 0;
}

$ gcc -o mmaptest mmaptest.c

$ dd bs=1080K count=1 if=/dev/random of=foo
1+0 records in
1+0 records out
1105920 bytes (1.1 MB) copied, 0.077 s, 14.4 MB/s

$ ./mmaptest foo
last byte 0x5F
      9 [main] mmaptest 3040 _cygtls::handle_exceptions: Error while dumping sta
te (probably corrupted stack)
Segmentation fault (core dumped)

This is a recent cygwin installation on XP Pro SP2.

Thanks,
Jay.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: how to get mmap page size?

[Corinna Vinschen]

On Jan 16 15:10, Jay Foad wrote:
> Corinna Vinschen wrote:
> 
> > mmap always allocates in 64K chunks.
> 
> That doesn't seem to be true in practice. In the test below I mmap a
> 1080K file (this is a multiple of 4K but not a multiple of 64K). The
> first byte after the end of the file isn't readable.

Right.  The reason is that mmap started to map memory top down at one
point to fix a problem with Windows Vista.  The mechanism to align
the map always to 64K is still in Cygwin, but it currently doesn't
work for files.  I'll have a look into fixing that for Cygwin 1.7.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: how to get mmap page size?

[Corinna Vinschen]

On Jan 16 16:45, Jay Foad wrote:
> Corinna Vinschen wrote:
> 
> > Right.  The reason is that mmap started to map memory top down at one
> > point to fix a problem with Windows Vista.  The mechanism to align
> > the map always to 64K is still in Cygwin, but it currently doesn't
> > work for files.  I'll have a look into fixing that for Cygwin 1.7.
> 
> OK, thanks. In the mean time I guess I'll just use a hard coded value
> of 4K for the page size in my application.

I fixed that in CVS.  Actually it turned out that it never quite worked
as intended due to a erronous condition.  Please note that this can't
work under WOW64 since WOW64 does not support the AT_ROUND_TO_PAGE flag
to NtMapViewOfSection for some reason.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: how to get mmap page size?

[Jay Foad]

Corinna Vinschen wrote:

> Right.  The reason is that mmap started to map memory top down at one
> point to fix a problem with Windows Vista.  The mechanism to align
> the map always to 64K is still in Cygwin, but it currently doesn't
> work for files.  I'll have a look into fixing that for Cygwin 1.7.

OK, thanks. In the mean time I guess I'll just use a hard coded value
of 4K for the page size in my application.

Thanks,
Jay.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: how to get mmap page size?

[Phil Betts]

Jay Foad wrote on Friday, January 16, 2009 12:24 PM::

> I have an application that wants to use mmap() to read a file, but
> only if it can guarantee that this will leave one or more zero bytes
> after the end of the contents of the file in memory:
> 
>   if ((filesize & (pagesize - 1) != 0)
>     use_mmap();
>   else
>     use_read();
> 

FYI, this is not good practice (although I fully understand why you
might want to do it).  From the mmap man page on an OSF1 UNIX box at 
work:

| If the len parameter is not a multiple of the page size returned by
| the sysconf(_SC_PAGE_SIZE) call, then the result of any reference to
| an address between the end of the region and the end of the page 
| containing the end of the region is undefined.

and:

|  +  If the end of the mapped file region is beyond the end of the 
|     file, the result of any reference to an address in the mapped 
|     file region corresponding to an offset beyond the end of the 
|     file is unspecified.

In other words, you're relying on system dependent behaviour if you
reference memory beyond the end of the file.  Your requirement
"only if it can guarantee that this will leave one or more zero bytes
after the end of the contents" cannot be satisfied if system
independence is required (or may be in future).  If you ever build 
your code on other systems, any bugs due to this would probably be 
really tough to track down, may be quite subtle, and may pose a
security risk.


Phil
-- 


This email has been scanned by Ascribe PLC using Microsoft Antigen for Exchange.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: how to get mmap page size?

[Corinna Vinschen]

On Jan 16 12:24, Jay Foad wrote:
> I have an application that wants to use mmap() to read a file, but
> only if it can guarantee that this will leave one or more zero bytes
> after the end of the contents of the file in memory:
> 
>   if ((filesize & (pagesize - 1) != 0)
>     use_mmap();
>   else
>     use_read();
> 
> How do I get the appropriate page size on cygwin?
> 
> If I call getpagesize() it returns 64k, the "allocation granularity".

That's what you want.  mmap always allocates in 64K chunks.

> Internally cygwin has a getsystempagesize() (introduced here:
> http://cygwin.com/ml/cygwin-cvs/2005-q4/msg00099.html) that returns
> the appropriate page size, 4k, but that doesn't seem to be exposed to
> applications.

And it's not required.  The system pagesize is really only used
internally.  Applications should rely on getpagesize().


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

how to get mmap page size?

[Jay Foad]

I have an application that wants to use mmap() to read a file, but
only if it can guarantee that this will leave one or more zero bytes
after the end of the contents of the file in memory:

  if ((filesize & (pagesize - 1) != 0)
    use_mmap();
  else
    use_read();

How do I get the appropriate page size on cygwin?

If I call getpagesize() it returns 64k, the "allocation granularity".
Internally cygwin has a getsystempagesize() (introduced here:
http://cygwin.com/ml/cygwin-cvs/2005-q4/msg00099.html) that returns
the appropriate page size, 4k, but that doesn't seem to be exposed to
applications.

I know that this has been discussed before, e.g. here:

http://www.cygwin.com/ml/cygwin/2002-01/msg00747.html

but I haven't found an answer to this particular question.

Thanks,
Jay.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/