Cygwin Shell Fails to Start when Enabling High Address

Cygwin Shell Fails to Start when Enabling High Address

[Benjamin Cao]

Hi,

I had to enable high address for some machines for testing. I set the AllocationPreference registry key to be 0X100000.

To start by Cygwin shell, I have a script that does this:

@echo off
set CYGWIN="nodosfilewarning"
sh --login

The error I will get when trying to start the shell is this:

1 [main] sh 5648 c:\cygwin\bin\sh.exe: *** fatal error - internal error reading the windows environment - too many environment variables?

If I leave the CYGWIN variable unset, it will work fine. My concern is that when and if I come across an instance in my testing where I NEED the CYGWIN variable set, this problem will come up again. Are there any tips I can user to further investigate this issue?

Thanks,
Ben Cao

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin Shell Fails to Start when Enabling High Address

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

Hi Benjamin,

On Feb 24 19:58, Benjamin Cao wrote:
> Hi,
> 
> I had to enable high address for some machines for testing. I set the
> AllocationPreference registry key to be 0X100000.

Cygwin doesn't support this AllocationPreference registry setting.

Cygwin and thus Cygwin applications(*) are already large address aware.
Cygwn uses large address regions automatically if available, but it's
essential that Cygwin itself can decide how and what to use them for.

Cygwin will also use MEM_TOP_DOWN allocations in certain system calls
(e.g. mmap) but this, too, is crafted to avoid collisions, and Cygwin
needs full control over this behaviour.

In general, Cygwin needs as much control over memory allocations as
possible in Windows for several reasons, mainly for the sake of fork and
exec calls.

So, having said that, AllocationPreference 0x100000 breaks Cygwin's
memory handling.  Therefore, don't use it in conjunction with Cygwin
applications.


Corinna


(*) There are exceptions, of course.  32 bit applications using
    the high bit of addresses for dubious reasons exist.  Of course
    they deserve to be broken.

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: Cygwin Shell Fails to Start when Enabling High Address

[Benjamin Cao]

Hi,

We are revisiting our issue with running Cygwin by enabling the AllocationPreference registry to run in high address. We’re hoping some clarification from our colleague could help shed some light on our situation.

“A major part of the work we did in the NT64 port was making sure that 64-bit pointers are not truncated to 32 bits. Most of the time you can get away with a truncated pointer because the address is in the lower 4GB of the address space and the upper 32 bits of the 64-bit pointer are not used. The AllocationPreference=0x100000 setting forces memory allocations to use addresses above 4GB so that the upper 32 bits of 64-bit pointers are used. This can uncover bugs that would otherwise not be found until a process has used a lot of memory. Rather than stressing the machines by having every test allocate a ton of memory we can use this setting to accomplish the same thing. This testing has found several bugs in existing code and it helps guard against developers making the mistake of truncating pointers in new code.”

This is a setting supported by Microsoft (see below).
The AllocationPreference registry setting is documented on Microsoft’s website:   https://msdn.microsoft.com/en-us/library/windows/hardware/dn613975(v=vs.85).aspx

Usually, VirtualAlloc returns virtual addresses in low -> high order. So, unless your process allocates a lot of memory or it has a very fragmented virtual address space, it will never get back very high addresses. This is possibly hiding bugs related to high addresses. There is a simple way to force allocations in top -> down order in Windows Server 2003, Datacenter Edition and Enterprise Edition operating systems and this can reveal important bugs.

It’s important that we have this registry enabled for our work and if your answer is still the same as before, will getting a support contract enable us to get this capability?

Thanks,
Ben Cao

-----Original Message-----
From: cygwin-owner@cygwin.com [mailto:cygwin-owner@cygwin.com] On Behalf Of Corinna Vinschen
Sent: Tuesday, February 24, 2015 4:34 PM
To: cygwin@cygwin.com
Subject: Re: Cygwin Shell Fails to Start when Enabling High Address

Hi Benjamin,

On Feb 24 19:58, Benjamin Cao wrote:
> Hi,
> 
> I had to enable high address for some machines for testing. I set the 
> AllocationPreference registry key to be 0X100000.

Cygwin doesn't support this AllocationPreference registry setting.

Cygwin and thus Cygwin applications(*) are already large address aware.
Cygwn uses large address regions automatically if available, but it's essential that Cygwin itself can decide how and what to use them for.

Cygwin will also use MEM_TOP_DOWN allocations in certain system calls (e.g. mmap) but this, too, is crafted to avoid collisions, and Cygwin needs full control over this behaviour.

In general, Cygwin needs as much control over memory allocations as possible in Windows for several reasons, mainly for the sake of fork and exec calls.

So, having said that, AllocationPreference 0x100000 breaks Cygwin's memory handling.  Therefore, don't use it in conjunction with Cygwin applications.


Corinna


(*) There are exceptions, of course.  32 bit applications using
    the high bit of addresses for dubious reasons exist.  Of course
    they deserve to be broken.

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: Cygwin Shell Fails to Start when Enabling High Address

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3143 bytes --]

On Jul 27 18:41, Benjamin Cao wrote:
> Hi,
> 
> We are revisiting our issue with running Cygwin by enabling the
> AllocationPreference registry to run in high address. We’re hoping
> some clarification from our colleague could help shed some light on
> our situation.
> 
> “A major part of the work we did in the NT64 port was making sure that
> 64-bit pointers are not truncated to 32 bits. Most of the time you can
> get away with a truncated pointer because the address is in the lower
> 4GB of the address space and the upper 32 bits of the 64-bit pointer
> are not used. The AllocationPreference=0x100000 setting forces memory
> allocations to use addresses above 4GB so that the upper 32 bits of
> 64-bit pointers are used. This can uncover bugs that would otherwise
> not be found until a process has used a lot of memory. Rather than
> stressing the machines by having every test allocate a ton of memory
> we can use this setting to accomplish the same thing. This testing has
> found several bugs in existing code and it helps guard against
> developers making the mistake of truncating pointers in new code.”
> 
> This is a setting supported by Microsoft (see below).
> The AllocationPreference registry setting is documented on Microsoft’s website:   https://msdn.microsoft.com/en-us/library/windows/hardware/dn613975(v=vs.85).aspx
> 
> Usually, VirtualAlloc returns virtual addresses in low -> high order.
> So, unless your process allocates a lot of memory or it has a very
> fragmented virtual address space, it will never get back very high
> addresses. This is possibly hiding bugs related to high addresses.
> There is a simple way to force allocations in top -> down order in
> Windows Server 2003, Datacenter Edition and Enterprise Edition
> operating systems and this can reveal important bugs.
> 
> It’s important that we have this registry enabled for our work and if
> your answer is still the same as before, will getting a support
> contract enable us to get this capability?

So you're talking about a 64 bit application?  You didn't say so in your
OP.

Anyway, I'm very much puzzled.

Either your application is a native Windows, non-Cygwin application
using VirtualAlloc.  Then it doesn't matter how Cygwin handles memory
for it's own processes to you since the memory allocations in your
process have nothing to do with Cygwin's allocation style.

Or it's a POSIX application ported to Cygwin.  In this case you
shouldn't use VirtualAlloc but malloc or mmap, and the way memory gets
allocated is not in your realm, but in Cygwin's.  Also, on 64 bit Cygwin
all memory (except thread stacks) is allocated beyond 0x1:0000000
anyway.  On 32 bit with 3gb/4gt setting or on WOW64, the heap used for
malloc allocations < 256K is beyond 0x80000000 and mmap s are allocated
top down, so they start beyond 0x80000000 as well.
So a Cygwin application is either high memory clean or it will crash.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: Cygwin Shell Fails to Start when Enabling High Address

[Benjamin Cao]

Corinna,

When I start Cygwin, with the AllocationPreference registry enable for VirtualAlloc, the shell immediate closes. I get the below error.

2 [main] sh 5240 c:\cygwin\bin\sh.exe: *** fatal error - internal error reading the windows environment - too many environment variables?

I guess you have answered the question to when we could get our testing running on Cygwin, but I'm unable to get to that point if the shell cannot start. Any advice on what I'm seeing now?

-----Original Message-----
From: cygwin-owner@cygwin.com [mailto:cygwin-owner@cygwin.com] On Behalf Of Corinna Vinschen
Sent: Monday, July 27, 2015 4:15 PM
To: cygwin@cygwin.com
Subject: Re: Cygwin Shell Fails to Start when Enabling High Address

On Jul 27 18:41, Benjamin Cao wrote:
> Hi,
> 
> We are revisiting our issue with running Cygwin by enabling the 
> AllocationPreference registry to run in high address. We’re hoping 
> some clarification from our colleague could help shed some light on 
> our situation.
> 
> “A major part of the work we did in the NT64 port was making sure that 
> 64-bit pointers are not truncated to 32 bits. Most of the time you can 
> get away with a truncated pointer because the address is in the lower 
> 4GB of the address space and the upper 32 bits of the 64-bit pointer 
> are not used. The AllocationPreference=0x100000 setting forces memory 
> allocations to use addresses above 4GB so that the upper 32 bits of 
> 64-bit pointers are used. This can uncover bugs that would otherwise 
> not be found until a process has used a lot of memory. Rather than 
> stressing the machines by having every test allocate a ton of memory 
> we can use this setting to accomplish the same thing. This testing has 
> found several bugs in existing code and it helps guard against 
> developers making the mistake of truncating pointers in new code.”
> 
> This is a setting supported by Microsoft (see below).
> The AllocationPreference registry setting is documented on Microsoft’s website:   https://msdn.microsoft.com/en-us/library/windows/hardware/dn613975(v=vs.85).aspx
> 
> Usually, VirtualAlloc returns virtual addresses in low -> high order.
> So, unless your process allocates a lot of memory or it has a very 
> fragmented virtual address space, it will never get back very high 
> addresses. This is possibly hiding bugs related to high addresses.
> There is a simple way to force allocations in top -> down order in 
> Windows Server 2003, Datacenter Edition and Enterprise Edition 
> operating systems and this can reveal important bugs.
> 
> It’s important that we have this registry enabled for our work and if 
> your answer is still the same as before, will getting a support 
> contract enable us to get this capability?

So you're talking about a 64 bit application?  You didn't say so in your OP.

Anyway, I'm very much puzzled.

Either your application is a native Windows, non-Cygwin application using VirtualAlloc.  Then it doesn't matter how Cygwin handles memory for it's own processes to you since the memory allocations in your process have nothing to do with Cygwin's allocation style.

Or it's a POSIX application ported to Cygwin.  In this case you shouldn't use VirtualAlloc but malloc or mmap, and the way memory gets allocated is not in your realm, but in Cygwin's.  Also, on 64 bit Cygwin all memory (except thread stacks) is allocated beyond 0x1:0000000 anyway.  On 32 bit with 3gb/4gt setting or on WOW64, the heap used for malloc allocations < 256K is beyond 0x80000000 and mmap s are allocated top down, so they start beyond 0x80000000 as well.
So a Cygwin application is either high memory clean or it will crash.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: Cygwin Shell Fails to Start when Enabling High Address

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1169 bytes --]

On Jul 28 17:26, Benjamin Cao wrote:
> Corinna,
> 
> When I start Cygwin, with the AllocationPreference registry enable for
> VirtualAlloc, the shell immediate closes. I get the below error.
> 
> 2 [main] sh 5240 c:\cygwin\bin\sh.exe: *** fatal error - internal
> error reading the windows environment - too many environment
> variables?
> 
> I guess you have answered the question to when we could get our
> testing running on Cygwin, but I'm unable to get to that point if the
> shell cannot start. Any advice on what I'm seeing now?

A bug in Cygwin.  Actually, 2 bugs, one generic bug and one in 64 bit
Cygwin only.

I applied patches to the git repo and I just uploaded new developer
snapshots to https://cygwin.com/snapshots/ which contain the fixes.

I'm also creating new test releases 2.2.0-0.4 and upload and announce
them later today.

Please give any of them a try, preferredly today, so I may be able
to create the offical Cygwin 2.2.0 release today as well.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: Cygwin Shell Fails to Start when Enabling High Address

[Benjamin Cao]

Hi Corinna,

I have tried the latest snapshot and have confirmed that with AllocationPreference enabled, I can start Cygwin without any crashes.

Thanks,
Ben Cao

-----Original Message-----
From: cygwin-owner@cygwin.com [mailto:cygwin-owner@cygwin.com] On Behalf Of Corinna Vinschen
Sent: Wednesday, July 29, 2015 7:49 AM
To: cygwin@cygwin.com
Subject: Re: Cygwin Shell Fails to Start when Enabling High Address

On Jul 28 17:26, Benjamin Cao wrote:
> Corinna,
> 
> When I start Cygwin, with the AllocationPreference registry enable for 
> VirtualAlloc, the shell immediate closes. I get the below error.
> 
> 2 [main] sh 5240 c:\cygwin\bin\sh.exe: *** fatal error - internal 
> error reading the windows environment - too many environment 
> variables?
> 
> I guess you have answered the question to when we could get our 
> testing running on Cygwin, but I'm unable to get to that point if the 
> shell cannot start. Any advice on what I'm seeing now?

A bug in Cygwin.  Actually, 2 bugs, one generic bug and one in 64 bit Cygwin only.

I applied patches to the git repo and I just uploaded new developer snapshots to https://cygwin.com/snapshots/ which contain the fixes.

I'm also creating new test releases 2.2.0-0.4 and upload and announce them later today.

Please give any of them a try, preferredly today, so I may be able to create the offical Cygwin 2.2.0 release today as well.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: Cygwin Shell Fails to Start when Enabling High Address

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

On Jul 29 13:10, Benjamin Cao wrote:
> Hi Corinna,
> 
> I have tried the latest snapshot and have confirmed that with
> AllocationPreference enabled, I can start Cygwin without any crashes.

Thanks for testing.  Unfortunately another bug showed up, so I push the
offical release into next week.


Thanks again,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]