page_size vs allocation_granularity

page_size vs allocation_granularity

[Ken Brown]

Hi Corinna,

I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to 
return wincap.allocation_granularity() rather than wincap.page_size().  Changing 
this would improve Linux compatibility, I think, but maybe it would have some 
bad consequences that I'm not aware of.

I'm asking because in my recent fooling around with php, I noticed that Yaakov 
had to apply the following Cygwin-specific patch to avoid a crash:

--- origsrc/php-7.1.13/Zend/zend_stream.c	2018-01-02 20:32:22.000000000 -0600
+++ src/php-7.1.13/Zend/zend_stream.c	2018-01-09 01:42:30.871472500 -0600
@@ -29,7 +29,9 @@
  #if HAVE_MMAP
  # if HAVE_UNISTD_H
  #  include <unistd.h>
-#  if defined(_SC_PAGESIZE)
+#  if defined(__CYGWIN__)
+#    define REAL_PAGE_SIZE 4096
+#  elif defined(_SC_PAGESIZE)
  #    define REAL_PAGE_SIZE sysconf(_SC_PAGESIZE);
  #  elif defined(_SC_PAGE_SIZE)
  #    define REAL_PAGE_SIZE sysconf(_SC_PAGE_SIZE);

Ken

Re: page_size vs allocation_granularity

[Corinna Vinschen]

On Jul 21 18:40, Ken Brown via Cygwin-developers wrote:
> Hi Corinna,
> 
> I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to
> return wincap.allocation_granularity() rather than wincap.page_size().
> Changing this would improve Linux compatibility, I think, but maybe it would
> have some bad consequences that I'm not aware of.

It was a long and hard process to move from 4K to 64K pagesize, with
lots of loaded discussions.  The Cygwin mailing list archives will
show a lot of this in the 200X years.

It was the only way to make mmap 99% POSIX-conformant.  Consider, for
instance this:

  pagesize = sysconf(_SC_PAGESIZE);
  addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE,
	       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
  addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE,
		MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

On Windows, this fails with pagesize = 4K, but it works with pagesize =
64K, because of that idiotic Windows allocation granularity.  Almost
all POSIX expectations are automagically fixed by using the granularity
as pagesize in a POSIX sense.

There's only one problem left:  While you can only allocate usefully in
64K steps, the size of the memory area allocated for a file is only 4K
aligned, thus leaving the remainder of the 64K block unmapped. 

This problem could be fixed back in 32 bit times by adding the
AT_ROUND_TO_PAGE mapping.  Very unfortunately, the 64 bit Windows
designer decided to keep the braindead 64K allocation granularity
but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance
to make this single situation POSIX-compatible as well.

> I'm asking because in my recent fooling around with php, I noticed that
> Yaakov had to apply the following Cygwin-specific patch to avoid a crash:

It would be nice to learn what kind of crash that was.

If php reads or writes in the remainder of the block constituting EOF,
or tries to change page protection, shit happens.  Every time, a process
stabs into the EOF block following the last valid 4K block, it results
in a STATUS_ACCESS_VIOLATION which in turn calls
mmap_is_attached_or_noreserve().  While this situation can be
recognized, I don't see a way to fix this from the processes POV.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

Re: page_size vs allocation_granularity

[Corinna Vinschen]

On Jul 22 10:33, Corinna Vinschen wrote:
> On Jul 21 18:40, Ken Brown via Cygwin-developers wrote:
> > Hi Corinna,
> > 
> > I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to
> > return wincap.allocation_granularity() rather than wincap.page_size().
> > Changing this would improve Linux compatibility, I think, but maybe it would
> > have some bad consequences that I'm not aware of.
> 
> It was a long and hard process to move from 4K to 64K pagesize, with
> lots of loaded discussions.  The Cygwin mailing list archives will
> show a lot of this in the 200X years.
> 
> It was the only way to make mmap 99% POSIX-conformant.  Consider, for
> instance this:
> 
>   pagesize = sysconf(_SC_PAGESIZE);
>   addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE,
> 	       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>   addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE,
> 		MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> 
> On Windows, this fails with pagesize = 4K, but it works with pagesize =
> 64K, because of that idiotic Windows allocation granularity.  Almost
> all POSIX expectations are automagically fixed by using the granularity
> as pagesize in a POSIX sense.
> 
> There's only one problem left:  While you can only allocate usefully in
> 64K steps, the size of the memory area allocated for a file is only 4K
> aligned, thus leaving the remainder of the 64K block unmapped. 
> 
> This problem could be fixed back in 32 bit times by adding the
> AT_ROUND_TO_PAGE mapping.  Very unfortunately, the 64 bit Windows
> designer decided to keep the braindead 64K allocation granularity
> but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance
> to make this single situation POSIX-compatible as well.

Oh, and there was a short hope a few months back when I discovered the
new memory management PLACEHOLDER flags.  Unfortunately Mappings within
placeholders are only allowed to be anonymous mappings, so there's no
relief for file backed mappings.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

Re: page_size vs allocation_granularity

[Ken Brown]

On 7/22/2020 4:33 AM, Corinna Vinschen wrote:
> On Jul 21 18:40, Ken Brown via Cygwin-developers wrote:
>> Hi Corinna,
>>
>> I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to
>> return wincap.allocation_granularity() rather than wincap.page_size().
>> Changing this would improve Linux compatibility, I think, but maybe it would
>> have some bad consequences that I'm not aware of.
> 
> It was a long and hard process to move from 4K to 64K pagesize, with
> lots of loaded discussions.  The Cygwin mailing list archives will
> show a lot of this in the 200X years.
> 
> It was the only way to make mmap 99% POSIX-conformant.  Consider, for
> instance this:
> 
>    pagesize = sysconf(_SC_PAGESIZE);
>    addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE,
> 	       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>    addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE,
> 		MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> 
> On Windows, this fails with pagesize = 4K, but it works with pagesize =
> 64K, because of that idiotic Windows allocation granularity.  Almost
> all POSIX expectations are automagically fixed by using the granularity
> as pagesize in a POSIX sense.
> 
> There's only one problem left:  While you can only allocate usefully in
> 64K steps, the size of the memory area allocated for a file is only 4K
> aligned, thus leaving the remainder of the 64K block unmapped.
> 
> This problem could be fixed back in 32 bit times by adding the
> AT_ROUND_TO_PAGE mapping.  Very unfortunately, the 64 bit Windows
> designer decided to keep the braindead 64K allocation granularity
> but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance
> to make this single situation POSIX-compatible as well.
> 
>> I'm asking because in my recent fooling around with php, I noticed that
>> Yaakov had to apply the following Cygwin-specific patch to avoid a crash:
> 
> It would be nice to learn what kind of crash that was.

Yaakov's patch cites

   https://cygwin.com/ml/cygwin/2017-05/msg00350.html

I'll take a look.

Ken

Re: page_size vs allocation_granularity

[Ken Brown]

On 7/22/2020 4:33 AM, Corinna Vinschen wrote:
> On Jul 21 18:40, Ken Brown via Cygwin-developers wrote:
>> Hi Corinna,
>>
>> I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to
>> return wincap.allocation_granularity() rather than wincap.page_size().
>> Changing this would improve Linux compatibility, I think, but maybe it would
>> have some bad consequences that I'm not aware of.
> 
> It was a long and hard process to move from 4K to 64K pagesize, with
> lots of loaded discussions.  The Cygwin mailing list archives will
> show a lot of this in the 200X years.
> 
> It was the only way to make mmap 99% POSIX-conformant.  Consider, for
> instance this:
> 
>    pagesize = sysconf(_SC_PAGESIZE);
>    addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE,
> 	       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>    addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE,
> 		MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> 
> On Windows, this fails with pagesize = 4K, but it works with pagesize =
> 64K, because of that idiotic Windows allocation granularity.  Almost
> all POSIX expectations are automagically fixed by using the granularity
> as pagesize in a POSIX sense.
> 
> There's only one problem left:  While you can only allocate usefully in
> 64K steps, the size of the memory area allocated for a file is only 4K
> aligned, thus leaving the remainder of the 64K block unmapped.
> 
> This problem could be fixed back in 32 bit times by adding the
> AT_ROUND_TO_PAGE mapping.  Very unfortunately, the 64 bit Windows
> designer decided to keep the braindead 64K allocation granularity
> but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance
> to make this single situation POSIX-compatible as well.
> 
>> I'm asking because in my recent fooling around with php, I noticed that
>> Yaakov had to apply the following Cygwin-specific patch to avoid a crash:
> 
> It would be nice to learn what kind of crash that was.

Here's a better reference than the one I gave in my previous reply, which 
actually explains what's going on:

   https://sourceware.org/pipermail/cygwin/2017-May/232562.html

> If php reads or writes in the remainder of the block constituting EOF,
> or tries to change page protection, shit happens.  Every time, a process
> stabs into the EOF block following the last valid 4K block, it results
> in a STATUS_ACCESS_VIOLATION which in turn calls
> mmap_is_attached_or_noreserve().  While this situation can be
> recognized, I don't see a way to fix this from the processes POV.

So that's exactly what happens when php maps a file whose size is a multiple of 
4K but not a multiple of 64K.  It expects that there is a zero-filled region 
beyond EOF that it can safely read from.

Ken

Re: page_size vs allocation_granularity

[Ken Brown]

On 7/22/2020 12:42 PM, Ken Brown via Cygwin-developers wrote:
> On 7/22/2020 4:33 AM, Corinna Vinschen wrote:
>> On Jul 21 18:40, Ken Brown via Cygwin-developers wrote:
>>> Hi Corinna,
>>>
>>> I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to
>>> return wincap.allocation_granularity() rather than wincap.page_size().
>>> Changing this would improve Linux compatibility, I think, but maybe it would
>>> have some bad consequences that I'm not aware of.
>>
>> It was a long and hard process to move from 4K to 64K pagesize, with
>> lots of loaded discussions.  The Cygwin mailing list archives will
>> show a lot of this in the 200X years.
>>
>> It was the only way to make mmap 99% POSIX-conformant.  Consider, for
>> instance this:
>>
>>    pagesize = sysconf(_SC_PAGESIZE);
>>    addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE,
>>            MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>>    addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE,
>>         MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>>
>> On Windows, this fails with pagesize = 4K, but it works with pagesize =
>> 64K, because of that idiotic Windows allocation granularity.  Almost
>> all POSIX expectations are automagically fixed by using the granularity
>> as pagesize in a POSIX sense.
>>
>> There's only one problem left:  While you can only allocate usefully in
>> 64K steps, the size of the memory area allocated for a file is only 4K
>> aligned, thus leaving the remainder of the 64K block unmapped.
>>
>> This problem could be fixed back in 32 bit times by adding the
>> AT_ROUND_TO_PAGE mapping.  Very unfortunately, the 64 bit Windows
>> designer decided to keep the braindead 64K allocation granularity
>> but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance
>> to make this single situation POSIX-compatible as well.
>>
>>> I'm asking because in my recent fooling around with php, I noticed that
>>> Yaakov had to apply the following Cygwin-specific patch to avoid a crash:
>>
>> It would be nice to learn what kind of crash that was.
> 
> Here's a better reference than the one I gave in my previous reply, which 
> actually explains what's going on:
> 
>    https://sourceware.org/pipermail/cygwin/2017-May/232562.html
> 
>> If php reads or writes in the remainder of the block constituting EOF,
>> or tries to change page protection, shit happens.  Every time, a process
>> stabs into the EOF block following the last valid 4K block, it results
>> in a STATUS_ACCESS_VIOLATION which in turn calls
>> mmap_is_attached_or_noreserve().  While this situation can be
>> recognized, I don't see a way to fix this from the processes POV.
> 
> So that's exactly what happens when php maps a file whose size is a multiple of 
> 4K but not a multiple of 64K.  It expects that there is a zero-filled region 
> beyond EOF that it can safely read from.

Interestingly, you mentioned exactly this scenario in 2002 as a reason for 
keeping the pagesize at 4K rather than 64K:

   https://cygwin.com/pipermail/cygwin/2002-January/068154.html

I have nothing new to contribute, so we should probably just drop this.  My 
curiosity has been satisfied.

Ken

Re: page_size vs allocation_granularity

[Corinna Vinschen]

On Jul 22 14:35, Ken Brown via Cygwin-developers wrote:
> On 7/22/2020 12:42 PM, Ken Brown via Cygwin-developers wrote:
> > On 7/22/2020 4:33 AM, Corinna Vinschen wrote:
> > > If php reads or writes in the remainder of the block constituting EOF,
> > > or tries to change page protection, shit happens.  Every time, a process
> > > stabs into the EOF block following the last valid 4K block, it results
> > > in a STATUS_ACCESS_VIOLATION which in turn calls
> > > mmap_is_attached_or_noreserve().  While this situation can be
> > > recognized, I don't see a way to fix this from the processes POV.
> > 
> > So that's exactly what happens when php maps a file whose size is a
> > multiple of 4K but not a multiple of 64K.  It expects that there is a
> > zero-filled region beyond EOF that it can safely read from.
> 
> Interestingly, you mentioned exactly this scenario in 2002 as a reason for
> keeping the pagesize at 4K rather than 64K:
> 
>   https://cygwin.com/pipermail/cygwin/2002-January/068154.html

Yeah, it took quite some time for me to realize that a 64K pagesize is
usually the better approach for POSIX compatibility.  And the fact that
AT_ROUND_TO_PAGE worked nicely on 32 bit (and 64 bit way off) helped,
too.  I was pretty stubborn back then... I hope that changed.

> I have nothing new to contribute, so we should probably just drop this.  My
> curiosity has been satisfied.

I toyed around with Windows mappings today, all the stuff I already
tried since 2000 over and over again.  Still no joy.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer