RE: Cygwin PHP (all available versions) has a hard 4MB memory limit

RE: Cygwin PHP (all available versions) has a hard 4MB memory limit

[km2z7kca0oge]

> You told us nothing about what versions of Windows, Cygwin, and PHP you are running, so WAG, either:
Wow Brian, what a rude response. I definitely followed the problem reporting guide, and didn't realise you'd need information overload that probably doesn't relate to the case at hand. When reporting bugs I always give as much info as I believe is needed in helping, there's no need to snap.

I gave lots of information, such as:
  - It applies to all bundled versions of PHP from the `setup-x86-64.exe`.
  - That I've reproduced it on multiple (two) machines, including one of those being a machine that has never had (and so a fresh install of) Cygwin with just PHP added.
  - Compiling PHP from source doesn't produce this issue so it's something to do with the bundled version only

For extra information, both machines tested are:
  - Windows 10 64-bit (10.0.18363)
  - One machine is 16GB, the other 8GB RAM.
  - All PHP versions from the `setup-x86-64.exe` (7.3.4-1 and 7.1.16-1)

> - you have defaulted to or specified a PHP configuration limit of 4MB memory for PHP tasks, or
Nope, as shown in the output of my example, the memory limit is set to 128MB:
> Output:
> $ php test.php
> 128M <--- This here shows the configured memory limit
> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5

> including copying verbatim all error messages seen
Here, the message was included in my initial email: > PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5

> - if you're running 32 bit Cygwin, possibly under 32 bit Windows, you have probably run out of heap space from installing too many packages requiring too many DLLs.
Nope all 64-bit, I would've mentioned if not.

> and PHP build configurations and logs.
See the output below for some more info, either way this is a pre-packaged version of PHP with very little changed from default configuration.

------------------------------
> Greetings, km2z7kca0oge--- via Cygwin!
Hey Andrey,

I've tried your script and I hit the 128MB limit, as expected. So maybe it's to do with the `http` wrapper. Could you try my version of the script please and see how you get on?

I first bumped into this problem when I rolled back from composer 2.X to 1.X which uses more memory.

I generated an 800MB file using: ` fsutil file createnew 800mega 838860800` and then ran your script you provided substituting your backup for the 800 MB file I generated.

The output of the script is below:
```
$ ./test-mailing-list.php
#!/usr/bin/env php
<?= file_get_contents(__FILE__);
phpinfo(1);
echo ini_get('memory_limit'), "\n";
print number_format(strlen(file_get_contents('800mega')));
phpinfo();
phpinfo()
PHP Version => 7.3.7

System => CYGWIN_NT-10.0-18363 AML0147 3.1.6-340.x86_64 2020-07-09 08:20 UTC x86_64
Build Date => Jul 21 2019 16:57:32
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc
Loaded Configuration File => /etc/php.ini
Scan this dir for additional .ini files => /etc/php.d
Additional .ini files parsed => /etc/php.d/bcmath.ini,
/etc/php.d/bz2.ini,
/etc/php.d/curl.ini,
/etc/php.d/fileinfo.ini,
/etc/php.d/gd.ini,
/etc/php.d/gmp.ini,
/etc/php.d/intl.ini,
/etc/php.d/json.ini,
/etc/php.d/ldap.ini,
/etc/php.d/mbstring.ini,
/etc/php.d/pdo_mysql.ini,
/etc/php.d/pdo_pgsql.ini,
/etc/php.d/pdo_sqlite.ini,
/etc/php.d/phar.ini,
/etc/php.d/posix.ini,
/etc/php.d/simplexml.ini,
/etc/php.d/sockets.ini,
/etc/php.d/sodium.ini,
/etc/php.d/sqlite3.ini,
/etc/php.d/tokenizer.ini,
/etc/php.d/vld.ini,
/etc/php.d/xmlwriter.ini,
/etc/php.d/zip.ini,
/etc/php.d/zlib.ini

PHP API => 20180731
PHP Extension => 20180731
Zend Extension => 320180731
Zend Extension Build => API320180731,NTS
PHP Extension Build => API20180731,NTS
Debug Build => no
Thread Safety => disabled
Zend Signal Handling => enabled
Zend Memory Manager => enabled
Zend Multibyte Support => provided by mbstring
IPv6 Support => enabled
DTrace Support => disabled

Registered PHP Streams => https, ftps, php, file, glob, data, http, ftp, compress.bzip2, compress.zlib, zip, phar
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, tls, tlsv1.0, tlsv1.1, tlsv1.2
Registered Stream Filters => string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, bzip2.*, zlib.*

This program makes use of the Zend Scripting Language Engine:
Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies
128M
PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 838869024 bytes) in /c/Users/JackBlower/tmp-safe/test-mailing-list.php on line 5
```

Notice how this time it's running out of memory at 128MB and has the "Allowed memory size" error instead of the "Out of memory" error from before.

Many Thanks,

Jack

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Brian Inglis]

On 2020-07-18 09:33, km2z7kca0oge--- via Cygwin wrote:
>> You told us nothing about what versions of Windows, Cygwin, and PHP you are running, so WAG, either:
> Wow Brian, what a rude response. I definitely followed the problem reporting guide, and didn't realise you'd need information overload that probably doesn't relate to the case at hand. When reporting bugs I always give as much info as I believe is needed in helping, there's no need to snap.

You did not specify Windows, Cygwin, Web server type, PHP, or Zend versions or
include the *required* cygcheck.out attachment which shows all this.

Many problems are because Cygwin and/or package releases have not been upgraded
for weeks, months, or even years, or reports do not include verbatim commands
and output, which may contain or clearly show symptoms of obvious errors, or
suggest a reproducible test case.

Most problems are unique to the reporter's system, otherwise others would have
reported it, and it would be known, and possibly fixed by volunteer maintainers.

More information than one might think is often required to diagnose the problem
with your unique situation and suggest a solution or provide a fix.

Many of the most knowledgeable and experienced volunteers providing support will
not even look at emails which don't attach a plain text cygcheck.out, from which
they can often diagnose a problem, suggest a workaround, further diagnostic
steps, or provide a fix.

Others of us try to nudge people into providing sufficient information so that
those with the most relevant knowledge and experience may take a look at the
issue, if they have time available, and suggest a further diagnostic,
configuration change, or provide a fix.

> I gave lots of information, such as:
>   - It applies to all bundled versions of PHP from the `setup-x86-64.exe`.
>   - That I've reproduced it on multiple (two) machines, including one of those being a machine that has never had (and so a fresh install of) Cygwin with just PHP added.
>   - Compiling PHP from source doesn't produce this issue so it's something to do with the bundled version only
> 
> For extra information, both machines tested are:
>   - Windows 10 64-bit (10.0.18363)
>   - One machine is 16GB, the other 8GB RAM.
>   - All PHP versions from the `setup-x86-64.exe` (7.3.4-1 and 7.1.16-1)

From below it appears you have also tested current PHP 7.3.7 with Zend 3.3.7.

One common issue with Cygwin installations causing memory problems is when
autorebase has not run or completed properly, so please try the following
command on a failing Cygwin installation:

$ awk '/0p_000_autorebase.dash"/,/zp_/' /var/log/setup.log.full

and if you see the message:

"The following DLLs couldn't be rebased because they were in use:"

followed by a number of DLLs, please run:

$ rebase-trigger full

then ensure *ALL* Cygwin services and process are shut down:
Win-R/taskmgr /7 OR ctrl-shift-esc - run Task Manager/select Details tab,
right-click on column headings/Select columns/check Image path name/OK if required,
sort by Image path name and see nothing starts with Cygwin installation root path,
[possibly select menu Options/Set default tab/Details for next time,] and Exit),
then rerun the Cygwin Setup program, and ensure that autorebase runs
successfully to completion, as above.

Another issue can be Windows memory management, so sometimes a system restart
will resolve memory related problems, or if your system is under extreme memory
loading at times (too little available/free) and you experience unexpected
issues under load, you need to change Virtual memory paging management from
Automatic to Custom by following:
press Win-Break keys/select Advanced tab/select Performance Settings... button/
/select Advanced tab/select Virtual memory Change... button/
/Uncheck Automatically manage paging file size.../
/select a drive from the list/select Custom size/
/Initial size (MB)/8192 (or 16384 depending on memory size)/
/Maximum size (MB)/16384 (or 32768 depending on memory size)/
/select *Set* button [essential!]/repeat with other drive(s)/select OK button (3
times) to close dialogue boxes, then Exit System control panel.

>> - you have defaulted to or specified a PHP configuration limit of 4MB memory for PHP tasks, or
> Nope, as shown in the output of my example, the memory limit is set to 128MB:
>> Output:
>> $ php test.php
>> 128M <--- This here shows the configured memory limit
>> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5
> 
>> including copying verbatim all error messages seen
> Here, the message was included in my initial email: > PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5
> 
>> - if you're running 32 bit Cygwin, possibly under 32 bit Windows, you have probably run out of heap space from installing too many packages requiring too many DLLs.
> Nope all 64-bit, I would've mentioned if not.
> 
>> and PHP build configurations and logs.
> See the output below for some more info, either way this is a pre-packaged version of PHP with very little changed from default configuration.
> 
> ------------------------------
>> Greetings, km2z7kca0oge--- via Cygwin!
> Hey Andrey,
> 
> I've tried your script and I hit the 128MB limit, as expected. So maybe it's to do with the `http` wrapper. Could you try my version of the script please and see how you get on?
> 
> I first bumped into this problem when I rolled back from composer 2.X to 1.X which uses more memory.
> 
> I generated an 800MB file using: ` fsutil file createnew 800mega 838860800` and then ran your script you provided substituting your backup for the 800 MB file I generated.
> 
> The output of the script is below:
> ```
> $ ./test-mailing-list.php
> #!/usr/bin/env php
> <?= file_get_contents(__FILE__);
> phpinfo(1);
> echo ini_get('memory_limit'), "\n";
> print number_format(strlen(file_get_contents('800mega')));
> phpinfo();
> phpinfo()
> PHP Version => 7.3.7
> 
> System => CYGWIN_NT-10.0-18363 AML0147 3.1.6-340.x86_64 2020-07-09 08:20 UTC x86_64
> Build Date => Jul 21 2019 16:57:32
> Server API => Command Line Interface
> Virtual Directory Support => disabled
> Configuration File (php.ini) Path => /etc
> Loaded Configuration File => /etc/php.ini
> Scan this dir for additional .ini files => /etc/php.d
> Additional .ini files parsed => /etc/php.d/bcmath.ini,
> /etc/php.d/bz2.ini,
> /etc/php.d/curl.ini,
> /etc/php.d/fileinfo.ini,
> /etc/php.d/gd.ini,
> /etc/php.d/gmp.ini,
> /etc/php.d/intl.ini,
> /etc/php.d/json.ini,
> /etc/php.d/ldap.ini,
> /etc/php.d/mbstring.ini,
> /etc/php.d/pdo_mysql.ini,
> /etc/php.d/pdo_pgsql.ini,
> /etc/php.d/pdo_sqlite.ini,
> /etc/php.d/phar.ini,
> /etc/php.d/posix.ini,
> /etc/php.d/simplexml.ini,
> /etc/php.d/sockets.ini,
> /etc/php.d/sodium.ini,
> /etc/php.d/sqlite3.ini,
> /etc/php.d/tokenizer.ini,
> /etc/php.d/vld.ini,
> /etc/php.d/xmlwriter.ini,
> /etc/php.d/zip.ini,
> /etc/php.d/zlib.ini
> 
> PHP API => 20180731
> PHP Extension => 20180731
> Zend Extension => 320180731
> Zend Extension Build => API320180731,NTS
> PHP Extension Build => API20180731,NTS
> Debug Build => no
> Thread Safety => disabled
> Zend Signal Handling => enabled
> Zend Memory Manager => enabled
> Zend Multibyte Support => provided by mbstring
> IPv6 Support => enabled
> DTrace Support => disabled
> 
> Registered PHP Streams => https, ftps, php, file, glob, data, http, ftp, compress.bzip2, compress.zlib, zip, phar
> Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, tls, tlsv1.0, tlsv1.1, tlsv1.2
> Registered Stream Filters => string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, bzip2.*, zlib.*
> 
> This program makes use of the Zend Scripting Language Engine:
> Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies
> 128M
> PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 838869024 bytes) in /c/Users/JackBlower/tmp-safe/test-mailing-list.php on line 5
> ```
> 
> Notice how this time it's running out of memory at 128MB and has the "Allowed memory size" error instead of the "Out of memory" error from before.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[km2z7kca0oge]

Hey Ken,

Thanks so much for looking into this further. I thought it was the malloc patch too, in fact I can compile a PHP out of the box without any of the patches Cygwin applies and it appears at a cursory glance to run fine and so I'm kinda unsure as to why there are so many patches for Cygwin.


> I'll follow up on the cygwin-apps list to see if we can get a rebuilt php into the distro.
Excellent thanks! Bear in mind this affects both versions that Cygwin offers and so it'd need both packages rebuilt and new versions pushed.

Many Thanks,

Jack

(P.S. Now I know how to use this mailing list I can start a new thread asking why PHP is compiled with libedit instead of readline and why it ignores `~/.editrc` haha.)

RE: Cygwin PHP (all available versions) has a hard 4MB memory limit

[km2z7kca0oge]

Oh, to add one more thing here, I've noticed there's a patch in the Cygwin Source Package called `7.1.9-malloc-cygwin.patch` which references something to do with Page sizes and 4096 which makes me wonder if this is interfering somehow.

> +#    define REAL_PAGE_SIZE 4096

--Jack

Cygwin PHP (all available versions) has a hard 4MB memory limit

[km2z7kca0oge]

Hi there,

Recently I've noticed that PHP seems have to hard 4MB memory limit, even when overridden in the settings. For whatever reason the bundled PHP versions with Cygwin have this problem.

The failing message is `Out of memory` which indicates PHP thinks the system has exhausted all RAM (the usual out of memory message is "Fatal error: Allowed memory size of XXX bytes exhausted" when you hit the cap).

If I compile a fresh PHP from source (for example 7.4.8) https://github.com/php/php-src/tree/php-7.4.8 it works absolutely fine.

I've reproduced this on another PC with a fresh install of Cygwin only installing the base PHP as the only extra package.

Example script:
```
<?php

echo ini_get('memory_limit'), "\n";

file_get_contents('http://mirror.cwcs.co.uk/centos/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso'); // A large file such as an ISO.
```

Output:
$ php test.php
128M
PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5

Any help would be great thanks.

Many Thanks,

Jack

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Brian Inglis]

On 2020-07-17 15:39, km2z7kca0oge--- via Cygwin wrote:
> Recently I've noticed that PHP seems have to hard 4MB memory limit, even when
> overridden in the settings. For whatever reason the bundled PHP versions
> with Cygwin have this problem.
> The failing message is `Out of memory` which indicates PHP thinks the system 
> has exhausted all RAM (the usual out of memory message is "Fatal error:
> Allowed memory size of XXX bytes exhausted" when you hit the cap).
> If I compile a fresh PHP from source (for example 7.4.8) 
> https://github.com/php/php-src/tree/php-7.4.8 it works absolutely fine.
> I've reproduced this on another PC with a fresh install of Cygwin only 
> installing the base PHP as the only extra package.
> > Example script:
> <?php
> echo ini_get('memory_limit'), "\n";
> file_get_contents('http://mirror.cwcs.co.uk/centos/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso'); // A large file such as an ISO.
> 
> Output:
> $ php test.php
> 128M
> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5
> Any help would be great thanks.

You told us nothing about what versions of Windows, Cygwin, and PHP you are
running, so WAG, either:

- you have defaulted to or specified a PHP configuration limit of 4MB memory for
PHP tasks, or

- if you're running 32 bit Cygwin, possibly under 32 bit Windows, you have
probably run out of heap space from installing too many packages requiring too
many DLLs.

If you are running something else, you need to tell us about those, following
the problem reporting guidelines linked below, including copying verbatim all
error messages seen, and PHP build configurations and logs.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Andrey Repin]

Greetings, km2z7kca0oge--- via Cygwin!

> Recently I've noticed that PHP seems have to hard 4MB memory limit, even
> when overridden in the settings. For whatever reason the bundled PHP
> versions with Cygwin have this problem.

> The failing message is `Out of memory` which indicates PHP thinks the
> system has exhausted all RAM (the usual out of memory message is "Fatal
> error: Allowed memory size of XXX bytes exhausted" when you hit the cap).

> If I compile a fresh PHP from source (for example 7.4.8)
> https://github.com/php/php-src/tree/php-7.4.8 it works absolutely fine.

> I've reproduced this on another PC with a fresh install of Cygwin only
> installing the base PHP as the only extra package.

> Example script:
> ```
> <?php

> echo ini_get('memory_limit'), "\n";

> file_get_contents('http://mirror.cwcs.co.uk/centos/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso'); // A large file such as an ISO.
> ```

#!/usr/bin/env php
<?= file_get_contents(__FILE__);
phpinfo(1);
echo ini_get('memory_limit'), "\n";
print number_format(strlen(file_get_contents('/c/var/backup/backup-2020-07-18.rar')));
phpinfo()
PHP Version => 7.3.7

System => CYGWIN_NT-6.1-7601 daemon2 3.1.5-340.x86_64 2020-06-01 08:59 UTC x86_64
Build Date => Jul 21 2019 16:57:32
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc
Loaded Configuration File => /etc/php-cli.ini
Scan this dir for additional .ini files => /etc/php.d
Additional .ini files parsed => /etc/php.d/00-core.ini,
/etc/php.d/00-date.ini,
/etc/php.d/00-mail.ini,
/etc/php.d/10-Cygwin.ini,
/etc/php.d/10-mbstring.ini,
/etc/php.d/20-intl.ini,
/etc/php.d/20-openssl.ini,
/etc/php.d/20-phar.ini,
/etc/php.d/20-session.ini,
/etc/php.d/20-tidy.ini,
/etc/php.d/20-timezonedb.ini,
/etc/php.d/bcmath.ini,
/etc/php.d/bz2.ini,
/etc/php.d/calendar.ini,
/etc/php.d/ctype.ini,
/etc/php.d/curl.ini,
/etc/php.d/exif.ini,
/etc/php.d/fileinfo.ini,
/etc/php.d/ftp.ini,
/etc/php.d/gd.ini,
/etc/php.d/gettext.ini,
/etc/php.d/gmp.ini,
/etc/php.d/iconv.ini,
/etc/php.d/imap.ini,
/etc/php.d/intl.ini,
/etc/php.d/json.ini,
/etc/php.d/ldap.ini,
/etc/php.d/opcache.ini,
/etc/php.d/pdo_mysql.ini,
/etc/php.d/pdo_odbc.ini,
/etc/php.d/pdo_sqlite.ini,
/etc/php.d/phar.ini,
/etc/php.d/posix.ini,
/etc/php.d/shmop.ini,
/etc/php.d/simplexml.ini,
/etc/php.d/soap.ini,
/etc/php.d/sockets.ini,
/etc/php.d/sqlite3.ini,
/etc/php.d/sysvmsg.ini,
/etc/php.d/sysvsem.ini,
/etc/php.d/sysvshm.ini,
/etc/php.d/tidy.ini,
/etc/php.d/tokenizer.ini,
/etc/php.d/wddx.ini,
/etc/php.d/xmlreader.ini,
/etc/php.d/xmlrpc.ini,
/etc/php.d/xmlwriter.ini,
/etc/php.d/xsl.ini,
/etc/php.d/zip.ini,
/etc/php.d/zlib.ini

PHP API => 20180731
PHP Extension => 20180731
Zend Extension => 320180731
Zend Extension Build => API320180731,NTS
PHP Extension Build => API20180731,NTS
Debug Build => no
Thread Safety => disabled
Zend Signal Handling => enabled
Zend Memory Manager => enabled
Zend Multibyte Support => provided by mbstring
IPv6 Support => enabled
DTrace Support => disabled

Registered PHP Streams => https, ftps, php, file, glob, data, http, ftp, compress.bzip2, compress.zlib, zip, phar
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, tls, tlsv1.0, tlsv1.1, tlsv1.2
Registered Stream Filters => string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, bzip2.*, convert.iconv.*, zlib.*

This program makes use of the Zend Scripting Language Engine:
Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.7, Copyright (c) 1999-2018, by Zend Technologies
-1
642,477,327

> Output:
> $ php test.php
> 128M
> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate
> 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5

> Any help would be great thanks.

More info is needed.


-- 
With best regards,
Andrey Repin
Saturday, July 18, 2020 10:28:03

Sorry for my terrible english...

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Ken Brown]

On 7/17/2020 5:39 PM, km2z7kca0oge--- via Cygwin wrote:
> Hi there,
> 
> Recently I've noticed that PHP seems have to hard 4MB memory limit,

[...]

> Example script:
> ```
> <?php
> 
> echo ini_get('memory_limit'), "\n";
> 
> file_get_contents('http://mirror.cwcs.co.uk/centos/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso'); // A large file such as an ISO.
> ```
> 
> Output:
> $ php test.php
> 128M
> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5

Thanks for the simple reproduction steps.  I've run this under gdb, and the 
function zend_mm_alloc_huge, which tries to get a chunk of memory from the 
system, seems to be failing when it tries to increase the zend heap from 4MB to 
6MB.  I'm in the process of building an unoptimized version of php to make 
debugging easier, in the hope of figuring out why the function is failing.

Ken

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Ken Brown]

On 7/18/2020 10:11 PM, Ken Brown via Cygwin wrote:
> On 7/17/2020 5:39 PM, km2z7kca0oge--- via Cygwin wrote:
>> Hi there,
>>
>> Recently I've noticed that PHP seems have to hard 4MB memory limit,
> 
> [...]
> 
>> Example script:
>> ```
>> <?php
>>
>> echo ini_get('memory_limit'), "\n";
>>
>> file_get_contents('http://mirror.cwcs.co.uk/centos/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso'); 
>> // A large file such as an ISO.
>> ```
>>
>> Output:
>> $ php test.php
>> 128M
>> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate 2097184 
>> bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5
> 
> Thanks for the simple reproduction steps.  I've run this under gdb, and the 
> function zend_mm_alloc_huge, which tries to get a chunk of memory from the 
> system, seems to be failing when it tries to increase the zend heap from 4MB to 
> 6MB.  I'm in the process of building an unoptimized version of php to make 
> debugging easier, in the hope of figuring out why the function is failing.

The problem is that some mmap calls are failing, for two reasons.  The first 
(and less important) is a Cygwin bug, which is now fixed:

https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=d8a8d2ce5953af3383aff019596bdf7f7aef6c41

The second is that mmap is sometimes called with MAP_FIXED, and these calls may 
fail.  But these calls occur only because of the following hunk of the patch 
7.1.9-malloc-cygwin.patch, which is used in the Cygwin build of php:

--- origsrc/php-7.1.9/Zend/zend_alloc.c	2017-08-30 11:13:20.000000000 -0500
+++ src/php-7.1.9/Zend/zend_alloc.c	2017-09-26 21:43:38.087650400 -0500
@@ -719,7 +719,7 @@ static void *zend_mm_chunk_alloc_int(siz
  		/* chunk has to be aligned */
  		zend_mm_munmap(ptr, size);
  		ptr = zend_mm_mmap(size + alignment - REAL_PAGE_SIZE);
-#ifdef _WIN32
+#if defined(_WIN32) || defined(__CYGWIN__)
  		offset = ZEND_MM_ALIGNED_OFFSET(ptr, alignment);
  		zend_mm_munmap(ptr, size + alignment - REAL_PAGE_SIZE);
  		ptr = zend_mm_mmap_fixed((void*)((char*)ptr + (alignment - offset)), size);

I don't know the reason for this hunk of the patch.  I suspect it was to 
compensate for some deficiencies in Cygwin's mmap that have since been fixed, 
but only Yaakov (the php maintainer) can confirm this.

After rebuilding php with that hunk of the patch removed, I get the following 
with your test case:

$ php test.php
128M
PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to 
allocate 65011744 bytes) in /tmp/test.php on line 5

This seems to be what's expected, since 134217728 == 0x8000000 == 128M.  Here's 
another example:

$ php -d "memory_limit=256M" test.php
256M
PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to 
allocate 132120608 bytes) in /tmp/test.php on line 5

I'll follow up on the cygwin-apps list to see if we can get a rebuilt php into 
the distro.

Ken

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Cary Lewis]

I am running php on two different Windows 10 computers, and on one machine
I can run composer with no issues, but on other I get our of memory issues.

Both report:

 php -v
PHP 7.3.7 (cli) (built: Jul 21 2019 18:10:35) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies

but trying to execute:

composer require twilio/sdk

on one of the machines results in this:

PHP Fatal error:  Out of memory (allocated 38273024) (tried to allocate
2098524 bytes) in
phar:///usr/bin/composer/src/Composer/Util/RemoteFilesystem.php on line 459

Fatal error: Out of memory (allocated 38273024) (tried to allocate 2098524
bytes) in phar:///usr/bin/composer/src/Composer/Util/RemoteFilesystem.php
on line 459

md5sum of /us/bin//php is the same on both computers.



On Tue, Jul 21, 2020 at 4:45 PM Ken Brown via Cygwin <cygwin@cygwin.com>
wrote:

> On 7/18/2020 10:11 PM, Ken Brown via Cygwin wrote:
> > On 7/17/2020 5:39 PM, km2z7kca0oge--- via Cygwin wrote:
> >> Hi there,
> >>
> >> Recently I've noticed that PHP seems have to hard 4MB memory limit,
> >
> > [...]
> >
> >> Example script:
> >> ```
> >> <?php
> >>
> >> echo ini_get('memory_limit'), "\n";
> >>
> >> file_get_contents('
> http://mirror.cwcs.co.uk/centos/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso');
>
> >> // A large file such as an ISO.
> >> ```
> >>
> >> Output:
> >> $ php test.php
> >> 128M
> >> PHP Fatal error:  Out of memory (allocated 4194304) (tried to allocate
> 2097184
> >> bytes) in /c/Users/JackBlower/tmp-safe/test.php on line 5
> >
> > Thanks for the simple reproduction steps.  I've run this under gdb, and
> the
> > function zend_mm_alloc_huge, which tries to get a chunk of memory from
> the
> > system, seems to be failing when it tries to increase the zend heap from
> 4MB to
> > 6MB.  I'm in the process of building an unoptimized version of php to
> make
> > debugging easier, in the hope of figuring out why the function is
> failing.
>
> The problem is that some mmap calls are failing, for two reasons.  The
> first
> (and less important) is a Cygwin bug, which is now fixed:
>
>
> https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=d8a8d2ce5953af3383aff019596bdf7f7aef6c41
>
> The second is that mmap is sometimes called with MAP_FIXED, and these
> calls may
> fail.  But these calls occur only because of the following hunk of the
> patch
> 7.1.9-malloc-cygwin.patch, which is used in the Cygwin build of php:
>
> --- origsrc/php-7.1.9/Zend/zend_alloc.c 2017-08-30 11:13:20.000000000 -0500
> +++ src/php-7.1.9/Zend/zend_alloc.c     2017-09-26 21:43:38.087650400 -0500
> @@ -719,7 +719,7 @@ static void *zend_mm_chunk_alloc_int(siz
>                 /* chunk has to be aligned */
>                 zend_mm_munmap(ptr, size);
>                 ptr = zend_mm_mmap(size + alignment - REAL_PAGE_SIZE);
> -#ifdef _WIN32
> +#if defined(_WIN32) || defined(__CYGWIN__)
>                 offset = ZEND_MM_ALIGNED_OFFSET(ptr, alignment);
>                 zend_mm_munmap(ptr, size + alignment - REAL_PAGE_SIZE);
>                 ptr = zend_mm_mmap_fixed((void*)((char*)ptr + (alignment -
> offset)), size);
>
> I don't know the reason for this hunk of the patch.  I suspect it was to
> compensate for some deficiencies in Cygwin's mmap that have since been
> fixed,
> but only Yaakov (the php maintainer) can confirm this.
>
> After rebuilding php with that hunk of the patch removed, I get the
> following
> with your test case:
>
> $ php test.php
> 128M
> PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried
> to
> allocate 65011744 bytes) in /tmp/test.php on line 5
>
> This seems to be what's expected, since 134217728 == 0x8000000 == 128M.
> Here's
> another example:
>
> $ php -d "memory_limit=256M" test.php
> 256M
> PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried
> to
> allocate 132120608 bytes) in /tmp/test.php on line 5
>
> I'll follow up on the cygwin-apps list to see if we can get a rebuilt php
> into
> the distro.
>
> Ken
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Ken Brown]

[Please don't top-post on this list.]

On 8/3/2020 9:57 PM, Cary Lewis wrote:
> I am running php on two different Windows 10 computers, and on one machine I can 
> run composer with no issues, but on other I get our of memory issues.
> 
> Both report:
> 
>   php -v
> PHP 7.3.7 (cli) (built: Jul 21 2019 18:10:35) ( NTS )
> Copyright (c) 1997-2018 The PHP Group
> Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies
> 
> but trying to execute:
> 
> composer require twilio/sdk
> 
> on one of the machines results in this:
> 
> PHP Fatal error:  Out of memory (allocated 38273024) (tried to allocate 2098524 
> bytes) in phar:///usr/bin/composer/src/Composer/Util/RemoteFilesystem.php on 
> line 459
> 
> Fatal error: Out of memory (allocated 38273024) (tried to allocate 2098524 
> bytes) in phar:///usr/bin/composer/src/Composer/Util/RemoteFilesystem.php on 
> line 459

I sent my proposed fix for the originally reported problem to the cygwin-apps list:

   https://cygwin.com/pipermail/cygwin-apps/2020-July/040350.html

But no one has stepped forward to take over maintainership.  If you want to try 
my build to see if it fixes your problem too, you can install it from my 
personal Cygwin repo by using setup-x86_64.exe.  Follow the instructions here:

   http://sanibeltranquility.com/cygwin/

If this works for you, I'll try again to get the fix into the Cygwin distro.

Ken

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Cary Lewis]

Thank you Ken, I followed your instructions and I am now able to use https
and composer with your versions of php.

Did you have to apply any of the cygport patches?

I hope that the offical cygwin repos and mirros get updated.

Thanks again.

On Tue, Aug 4, 2020 at 6:45 AM Ken Brown <kbrown@cornell.edu> wrote:

> [Please don't top-post on this list.]
>
> On 8/3/2020 9:57 PM, Cary Lewis wrote:
> > I am running php on two different Windows 10 computers, and on one
> machine I can
> > run composer with no issues, but on other I get our of memory issues.
> >
> > Both report:
> >
> >   php -v
> > PHP 7.3.7 (cli) (built: Jul 21 2019 18:10:35) ( NTS )
> > Copyright (c) 1997-2018 The PHP Group
> > Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies
> >
> > but trying to execute:
> >
> > composer require twilio/sdk
> >
> > on one of the machines results in this:
> >
> > PHP Fatal error:  Out of memory (allocated 38273024) (tried to allocate
> 2098524
> > bytes) in
> phar:///usr/bin/composer/src/Composer/Util/RemoteFilesystem.php on
> > line 459
> >
> > Fatal error: Out of memory (allocated 38273024) (tried to allocate
> 2098524
> > bytes) in
> phar:///usr/bin/composer/src/Composer/Util/RemoteFilesystem.php on
> > line 459
>
> I sent my proposed fix for the originally reported problem to the
> cygwin-apps list:
>
>    https://cygwin.com/pipermail/cygwin-apps/2020-July/040350.html
>
> But no one has stepped forward to take over maintainership.  If you want
> to try
> my build to see if it fixes your problem too, you can install it from my
> personal Cygwin repo by using setup-x86_64.exe.  Follow the instructions
> here:
>
>    http://sanibeltranquility.com/cygwin/
>
> If this works for you, I'll try again to get the fix into the Cygwin
> distro.
>
> Ken
>

Re: Cygwin PHP (all available versions) has a hard 4MB memory limit

[Ken Brown]

[Once again, please don't top-post on this list.]

On 8/4/2020 9:35 AM, Cary Lewis wrote:
> Thank you Ken, I followed your instructions and I am now able to use https and 
> composer with your versions of php.
> 
> Did you have to apply any of the cygport patches?

I applied all of them except for one hunk of one of them.  Yaakov, the 
(retiring) php maintainer, had reasons for applying the patches, and I'm not 
going to second-guess him.  The next maintainer (who won't be me) can look into 
it more thoroughly if s/he wants.

> I hope that the offical cygwin repos and mirros get updated.

I'll see what I can do.

Ken