Re: Inefficient use of 64-bit addresses in Clang

Re: Inefficient use of 64-bit addresses in Clang

[falk.tannhauser]

References: <578eb489-9391-9009-82ad-676eeb4c1c92@agner.org>
In-Reply-To: <578eb489-9391-9009-82ad-676eeb4c1c92@agner.org>

Could the different behaviour between Cygwin and Linux simply be due to different Clang versions?
The current version under Cygwin is 5.0.1, while the latest version available under Linux
appears to be 8.0.1 .

Falk

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Inefficient use of 64-bit addresses in Clang

[Agner Fog]

It's a difference in memory model.

clang 6.0.0 under ubuntu with --target=x86_64-pc-cygwin gives relative 
addresses, unless you specify -mcmodel=large.

Cygwin clang with -mcmodel=small does the right thing: use relative 
addresses.

The -mcmodel=small option appears to work differently for Linux and for 
Windows targets. I cannot find any documentation of this difference. See:

https://bugs.llvm.org/show_bug.cgi?id=42983


On 12/08/2019 11.45, falk.tannhauser@free.fr wrote:
> References: <578eb489-9391-9009-82ad-676eeb4c1c92@agner.org>
> In-Reply-To: <578eb489-9391-9009-82ad-676eeb4c1c92@agner.org>
>
> Could the different behaviour between Cygwin and Linux simply be due to different Clang versions?
> The current version under Cygwin is 5.0.1, while the latest version available under Linux
> appears to be 8.0.1 .
>
> Falk
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Clang is using the wrong memory model

[Agner Fog]

Cygwin Clang is using -mcmodel=medium as default for Win64, according to 
my tests, while the right model is -mcmodel=small

Linux Clang with --target=x86_64-pc-cygwin gives the small memory model.

I took this to the LLVM Bugzilla as you asked me to: 
https://bugs.llvm.org/show_bug.cgi?id=42983

This gave the following conclusion:

-mcmodel=small does something different when the target is Windows. This 
difference appears to be undocumented. The small memory model with a 
Linux target puts everything below the 2GB limit so that 32-bit absolute 
addresses can be used. The small memory model with a Windows target is 
using 32-bit relative addresses instead, which is the correct thing to 
do in Windows.

I told the LLVM guy that this difference needs to be documented so that 
you can rely on it, but this request has so far been ignored.

He says that you must have modified the source code to change the 
default memory model.

The medium memory model gives inefficient code because it uses an extra 
instruction to load a 64-bit absolute address into a register before 
every access to static data.

I cannot blame you Cygwin people for not using the small memory model as 
long as it is undocumented.

I will ask you to please join the discussion at 
https://bugs.llvm.org/show_bug.cgi?id=42983 so that we can clarify how 
to solve this problem.


On 14/08/2019 07.51, Agner Fog wrote:
> It's a difference in memory model.
>
> clang 6.0.0 under ubuntu with --target=x86_64-pc-cygwin gives relative 
> addresses, unless you specify -mcmodel=large.
>
> Cygwin clang with -mcmodel=small does the right thing: use relative 
> addresses.
>
> The -mcmodel=small option appears to work differently for Linux and 
> for Windows targets. I cannot find any documentation of this 
> difference. See:
>
> https://bugs.llvm.org/show_bug.cgi?id=42983
>
>
> On 12/08/2019 11.45, falk.tannhauser@free.fr wrote:
>> References: <578eb489-9391-9009-82ad-676eeb4c1c92@agner.org>
>> In-Reply-To: <578eb489-9391-9009-82ad-676eeb4c1c92@agner.org>
>>
>> Could the different behaviour between Cygwin and Linux simply be due 
>> to different Clang versions?
>> The current version under Cygwin is 5.0.1, while the latest version 
>> available under Linux
>> appears to be 8.0.1 .
>>
>> Falk
>>
>> -- 
>> Problem reports:       http://cygwin.com/problems.html
>> FAQ:                   http://cygwin.com/faq/
>> Documentation:         http://cygwin.com/docs.html
>> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>>
>>
>
> -- 
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Mark Geisert]

Hi, please don't start a new Subject: by replying on another thread.  It fouls 
up mail threading for folks trying to follow multiple threads.  Just start a new 
thread with your new Subject:.  Thanks.

Agner Fog wrote:
> Cygwin Clang is using -mcmodel=medium as default for Win64, according to my 
> tests, while the right model is -mcmodel=small
> 
> Linux Clang with --target=x86_64-pc-cygwin gives the small memory model.
> 
> I took this to the LLVM Bugzilla as you asked me to: 
> https://bugs.llvm.org/show_bug.cgi?id=42983

Disclaimer: I am not a Cygwin maintainer or the Cygwin Clang maintainer.  This 
is a general Cygwin users' mailing list, not a private support channel.  I'm 
just some guy on the list, albeit somebody interested in solving your issue.

Cygwin is not Windows.  Cygwin's shared libraries are more than 32 bits distant 
from executables you build.  -mcmodel=small won't work.  There may be a better 
-mcmodel setting, TBD, but small isn't it, unfortunately.

The reasons are longstanding and technical.  The quick version is that there are 
thousands of different DLLs that a Cygwin user might want installed on their 
system.  Cygwin is a Posix environment, and to support fork() the address space 
for those DLLs is chosen to avoid Windows interference, so much as possible.

> This gave the following conclusion:
> 
> -mcmodel=small does something different when the target is Windows. This 
> difference appears to be undocumented. The small memory model with a Linux 
> target puts everything below the 2GB limit so that 32-bit absolute addresses can 
> be used. The small memory model with a Windows target is using 32-bit relative 
> addresses instead, which is the correct thing to do in Windows.
> 
> I told the LLVM guy that this difference needs to be documented so that you can 
> rely on it, but this request has so far been ignored.
> 
> He says that you must have modified the source code to change the default memory 
> model.

Now that I've downloaded and unpacked the Clang source tree, I can confirm we 
have a patch to set -mcmodel=medium on 64-bit Cygwin; it's left at small for 
32-bit Cygwin.

> The medium memory model gives inefficient code because it uses an extra 
> instruction to load a 64-bit absolute address into a register before every 
> access to static data.

My understanding is that Clang makes an attempt to accept all the same options 
as gcc.  Could you please try your Clang build with option "-mcmodel=small"?  I 
suspect the link phase won't work because the Clang runtime library is more than 
32 bits away from your executable's code.  You might try another -mcmodel 
setting.  I don't know what all of them are.

First step is just getting the thing to link.  Then testing whether it works 
properly.  Then code efficiency :-).

> I cannot blame you Cygwin people for not using the small memory model as long as 
> it is undocumented.

The necessary reason for our patch was explained above.

> I will ask you to please join the discussion at 
> https://bugs.llvm.org/show_bug.cgi?id=42983 so that we can clarify how to solve 
> this problem.

This issue does now appear to be an issue with Cygwin's Clang build, so let's 
keep discussing here.  We can sync up with the LLVM bugtracker later on.

BTW is it a goal to be building for Cygwin?  Or is it that you want 
Windows-native executables built with Cygwin tools?  If the latter, there are 
cross-compilers provided by Cygwin that can do that.  I don't think there's a 
Clang for that situation yet, though.

..mark

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1996 bytes --]

On Aug 16 08:06, Agner Fog wrote:
> Cygwin Clang is using -mcmodel=medium as default for Win64, according to my
> tests, while the right model is -mcmodel=small

-mcmodel=small is *only* the right model if the target is native
Windows.  If the target is a Cygwin application it *must* at least
be compiled with -mcmodel=medium.  The reason is the standarized
memory layout of Cygwin application and DLLs.

> Linux Clang with --target=x86_64-pc-cygwin gives the small memory model.

Which is wrong.

> I took this to the LLVM Bugzilla as you asked me to:
> https://bugs.llvm.org/show_bug.cgi?id=42983
> 
> This gave the following conclusion:
> 
> -mcmodel=small does something different when the target is Windows. This
> difference appears to be undocumented. The small memory model with a Linux
> target puts everything below the 2GB limit so that 32-bit absolute addresses
> can be used. The small memory model with a Windows target is using 32-bit
> relative addresses instead, which is the correct thing to do in Windows.

Yes, but not for Cygwin applications and DLLs.  The reason is that Cygwin
apps and DLLs reside in the memory beyond the first 2 Gigs in a standarized
way so as not to collide with Windows code and datastructures.  And that
in turn was necessary to make fork() more reliable on 64 bit.

Here's the memory layout:

0000:00000000	Windows
0000:80000000	Thread stacks
0001:00400000	Executable
0001:80000000	*The* Cygwin DLL
0002:00000000	Rebased Cygwin DLLs
0004:00000000	Unrebased Cygwin DLLs(*)

0006:00000000	Heap (up to Cygwin 3.0)
0008:00000000	Heap (starting with Cygwin 3.1)

0700:00000000	Top-down start address for mmaps up to Windows 8
		or up to Cygwin 3.0
7000:00000000	Top-down start address for mmaps starting with Windows 8.1
		and Cygwin 3.1

So there's a difference between non-Cygwin (-mcmodel=small as default)
and Cygwin (at least -mcmodel=medium).


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Clang is using the wrong memory model

[Agner Fog]

Thanks for your replies.

A Cygwin application with -mcmodel=small appears to work fine.

As I explained, -mcmodel=small does something else when the target is 
Windows. It does not require addresses to be below 2GB, it only requires 
the distance between a code section and it's data section to be below 2GB.

Your concern would be right if -mcmodel=small was doing the same thing 
as it does in Linux, but this is not the case.

It does not matter where the application and the Cygwin DLL are placed 
because Windows allows EXEs and DLLs to be placed at any address. There 
is no static linking between an application and the Cygwin DLL, so the 
distance between application and DLL does not matter.

I don't see any problem with fork() either, because there is no static 
link between parent and child process. The OS puts everything in place 
using virtual addresses.

Agner


On 16/08/2019 10.26, Corinna Vinschen wrote:
> On Aug 16 08:06, Agner Fog wrote:
>> Cygwin Clang is using -mcmodel=medium as default for Win64, according to my
>> tests, while the right model is -mcmodel=small
> -mcmodel=small is *only* the right model if the target is native
> Windows.  If the target is a Cygwin application it *must* at least
> be compiled with -mcmodel=medium.  The reason is the standarized
> memory layout of Cygwin application and DLLs.
>
>> Linux Clang with --target=x86_64-pc-cygwin gives the small memory model.
> Which is wrong.
>
>> I took this to the LLVM Bugzilla as you asked me to:
>> https://bugs.llvm.org/show_bug.cgi?id=42983
>>
>> This gave the following conclusion:
>>
>> -mcmodel=small does something different when the target is Windows. This
>> difference appears to be undocumented. The small memory model with a Linux
>> target puts everything below the 2GB limit so that 32-bit absolute addresses
>> can be used. The small memory model with a Windows target is using 32-bit
>> relative addresses instead, which is the correct thing to do in Windows.
> Yes, but not for Cygwin applications and DLLs.  The reason is that Cygwin
> apps and DLLs reside in the memory beyond the first 2 Gigs in a standarized
> way so as not to collide with Windows code and datastructures.  And that
> in turn was necessary to make fork() more reliable on 64 bit.
>
> Here's the memory layout:
>
> 0000:00000000	Windows
> 0000:80000000	Thread stacks
> 0001:00400000	Executable
> 0001:80000000	*The* Cygwin DLL
> 0002:00000000	Rebased Cygwin DLLs
> 0004:00000000	Unrebased Cygwin DLLs(*)
>
> 0006:00000000	Heap (up to Cygwin 3.0)
> 0008:00000000	Heap (starting with Cygwin 3.1)
>
> 0700:00000000	Top-down start address for mmaps up to Windows 8
> 		or up to Cygwin 3.0
> 7000:00000000	Top-down start address for mmaps starting with Windows 8.1
> 		and Cygwin 3.1
>
> So there's a difference between non-Cygwin (-mcmodel=small as default)
> and Cygwin (at least -mcmodel=medium).
>
>
> Corinna
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Agner Fog]

Thanks for your replies.

A Cygwin application with -mcmodel=small appears to work fine.

As I explained, -mcmodel=small does something else when the target is 
Windows. It does not require addresses to be below 2GB, it only requires 
the distance between a code section and it's data section to be below 2GB.

Your concern would be right if -mcmodel=small was doing the same thing 
as it does in Linux, but this is not the case.

It does not matter where the application and the Cygwin DLL are placed 
because Windows allows EXEs and DLLs to be placed at any address. There 
is no static linking between an application and the Cygwin DLL, so the 
distance between application and DLL does not matter.

I don't see any problem with fork() either, because there is no static 
link between parent and child process. The OS puts everything in place 
using virtual addresses.

Agner


On 16/08/2019 10.26, Corinna Vinschen wrote:
> On Aug 16 08:06, Agner Fog wrote:
>> Cygwin Clang is using -mcmodel=medium as default for Win64, according to my
>> tests, while the right model is -mcmodel=small
> -mcmodel=small is *only* the right model if the target is native
> Windows.  If the target is a Cygwin application it *must* at least
> be compiled with -mcmodel=medium.  The reason is the standarized
> memory layout of Cygwin application and DLLs.
>
>> Linux Clang with --target=x86_64-pc-cygwin gives the small memory model.
> Which is wrong.
>
>> I took this to the LLVM Bugzilla as you asked me to:
>> https://bugs.llvm.org/show_bug.cgi?id=42983
>>
>> This gave the following conclusion:
>>
>> -mcmodel=small does something different when the target is Windows. This
>> difference appears to be undocumented. The small memory model with a Linux
>> target puts everything below the 2GB limit so that 32-bit absolute addresses
>> can be used. The small memory model with a Windows target is using 32-bit
>> relative addresses instead, which is the correct thing to do in Windows.
> Yes, but not for Cygwin applications and DLLs.  The reason is that Cygwin
> apps and DLLs reside in the memory beyond the first 2 Gigs in a standarized
> way so as not to collide with Windows code and datastructures.  And that
> in turn was necessary to make fork() more reliable on 64 bit.
>
> Here's the memory layout:
>
> 0000:00000000	Windows
> 0000:80000000	Thread stacks
> 0001:00400000	Executable
> 0001:80000000	*The* Cygwin DLL
> 0002:00000000	Rebased Cygwin DLLs
> 0004:00000000	Unrebased Cygwin DLLs(*)
>
> 0006:00000000	Heap (up to Cygwin 3.0)
> 0008:00000000	Heap (starting with Cygwin 3.1)
>
> 0700:00000000	Top-down start address for mmaps up to Windows 8
> 		or up to Cygwin 3.0
> 7000:00000000	Top-down start address for mmaps starting with Windows 8.1
> 		and Cygwin 3.1
>
> So there's a difference between non-Cygwin (-mcmodel=small as default)
> and Cygwin (at least -mcmodel=medium).
>
>
> Corinna
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

On Aug 16 11:27, Agner Fog wrote:
> Thanks for your replies.
> 
> A Cygwin application with -mcmodel=small appears to work fine.
> 
> As I explained, -mcmodel=small does something else when the target is
> Windows. It does not require addresses to be below 2GB, it only requires the
> distance between a code section and it's data section to be below 2GB.

And that's the point.  The distances are not guranteed to be within
2 GB.  Think errno accessed from another DLL.  Your application works
only by chance.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Clang is using the wrong memory model

[Agner Fog]

On 16/08/2019 11.52, Corinna Vinschen wrote:
> 2 GB. Think errno accessed from another DLL. Your application works 
> only by chance.

Good example.

errno appears to be a global variable for historical reasons, but errno 
is implemented as a macro that translates to a call to the imported 
function __errno.

This function call goes through a 64-bit import table from cygwin1.dll

Works fine with -mcmodel=small

Static sharing of a global variable between exe and dll, or between 
different dll's is not possible.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

On Aug 16 12:38, Agner Fog wrote:
> 
> On 16/08/2019 11.52, Corinna Vinschen wrote:
> > 2 GB. Think errno accessed from another DLL. Your application works only
> > by chance.
> 
> Good example.
> 
> errno appears to be a global variable for historical reasons, but errno is
> implemented as a macro that translates to a call to the imported function
> __errno.
> 
> This function call goes through a 64-bit import table from cygwin1.dll
> 
> Works fine with -mcmodel=small
> 
> Static sharing of a global variable between exe and dll, or between
> different dll's is not possible.

So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
__progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
for kicks.

Just because lots of stuff works with the small mode doesn't mean it's
the right thing to do.  Mcmodels medium and large have been introduced
into GCC during the Cygwin 64 bit port for a reason.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Clang is using the wrong memory model

[Kai Tietz]

Hey,

Just my 5 cents to this.  As Corinna pointed out, is the case, that a
"small" memory model application works for you, no valid prove that
all application will work on such a model.  Another thing, which
cygwin depends heavily on is the pseudo-relocation stuff.  It is not
guaranteed that code distance between DSOs are in valid range here to
fit in a single instruction to be relocated.  So the distance allowed
for a "small model instruction" might be in some cases even less 4 GB.
For pseudo-relocation the large memory model is absolutely mandatory
for bigger application.

Kai

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Agner Fog]

> So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
> __progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
> for kicks.
__ctype_ptr__ is a function

h_errno works like errno with an imported function

FE_DFL_ENV is a macro

__progname and optarg are local variables to each exe or dll


Note that the 2GB limit applies to_static linking_, not to_dynamic linking_.

Allow me to explain the difference. Static linking is the linking of object files together into an EXE or DLL file. Dynamic linking is the linking between different EXE or DLL files.

Static linking is done by the linker. If function A() in a.cpp is calling function B() in b.cpp, then you can compile and link a.cpp and b.cpp together into ab.exe, using the linker. The linker will insert the relative address of B by directly modifying the call B instruction in a.o. The linker will give an error message if the address overflows beyond the 2GB limit. The same process applies when linking to a global variable in b.cpp. The call instruction always uses a 32-bit relative address. The link to a variable can use different addressing modes: 32-bit absolute address, 32-bit relative address, and 64-bit absolute address. The 32-bit absolute address can be used in Linux, but not in Win64 and MacOS. This is why the small memory model is different in Linux and Windows (64 bit).

Dynamic linking is done by the loader, not the linker. If a function in ab.exe calls function C() in C.DLL, then the process is different. D.DLL is loaded by the loader, and the address of C() is inserted into an import table. The import table contains 64-bit absolute addresses.

It is not possible to insert the address of a function or variable from a different EXE or DLL directly into a code instruction. The loader simply cannot do this, only the linker can.

This is the reason why the link to errno and other seemingly global variables are replaced by functions. errno was not a bad example, it shows how a variable in a different DLL is accessed.

gcc is using the small memory model by default in Cygwin64, and it works.

clang is using the small memory by default when cross-compiling for a Cygwin64 target from Linux, and it works.

Kai Tietz wrote:
>   Another thing, which
> cygwin depends heavily on is the pseudo-relocation stuff.  It is not
> guaranteed that code distance between DSOs are in valid range here to
> fit in a single instruction to be relocated.  So the distance allowed
> for a "small model instruction" might be in some cases even less 4 GB.
> For pseudo-relocation the large memory model is absolutely mandatory
> for bigger application.
Kai, can you please point me to a description of the DSO and pseudo-relocation. I cannot find it. But I suppose it inserts 64-bit addresses into some sort of GOT or PLT without using the traditional linker.

Agner


On 16/08/2019 13.11, Corinna Vinschen wrote:
> On Aug 16 12:38, Agner Fog wrote:
>> On 16/08/2019 11.52, Corinna Vinschen wrote:
>>> 2 GB. Think errno accessed from another DLL. Your application works only
>>> by chance.
>> Good example.
>>
>> errno appears to be a global variable for historical reasons, but errno is
>> implemented as a macro that translates to a call to the imported function
>> __errno.
>>
>> This function call goes through a 64-bit import table from cygwin1.dll
>>
>> Works fine with -mcmodel=small
>>
>> Static sharing of a global variable between exe and dll, or between
>> different dll's is not possible.
> So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
> __progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
> for kicks.
>
> Just because lots of stuff works with the small mode doesn't mean it's
> the right thing to do.  Mcmodels medium and large have been introduced
> into GCC during the Cygwin 64 bit port for a reason.
>
>
> Corinna
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1725 bytes --]

Oe Aug 17 07:31, Agner Fog wrote:
> > So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
> > __progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
> > for kicks.
> __ctype_ptr__ is a function
> 
> h_errno works like errno with an imported function
> 
> FE_DFL_ENV is a macro
> 
> __progname and optarg are local variables to each exe or dll

That would contradict what, e.g., __progname is for.  Here's a test:

$ cat > dll.c <<EOF
#include <stdio.h>

extern char *__progname;

void
printprog ()
{
  printf ("progname: %s\n", __progname);
}
EOF
$ cat > main.c <<EOF
extern void printprog();

int
main ()
{
  printprog ();
}
EOF
$ uname -a
CYGWIN_NT-10.0 vmbert10 3.1.0(0.340/5/3) 2019-08-16 14:36 x86_64 Cygwin

Lets try the medium model first:

  $ gcc -g -shared -mcmodel=medium -o dll.dll dll.c
  $ gcc -g -mcmodel=medium -o main main.c dll.dll
  $ ./main
  progname: main

Now let's try the small model:

  $ gcc -g -shared -mcmodel=small -o dll.dll dll.c
  $ gcc -g -mcmodel=small -o main main.c dll.dll
  $ ./main
  Cygwin runtime failure: /home/corinna/main.exe: Invalid relocation.  Offset
  0xfffffffd80348989 at address 0x40000103b doesn't fit into 32 bits

Now let's try without explicit mcmodel on the CLI:

  $ gcc -g -shared -o dll.dll dll.c
  $ gcc -g -o main main.c dll.dll
  $ ./main
  progname: main

> gcc is using the small memory model by default in Cygwin64, and it works.

No, it's not, see above.

> clang is using the small memory by default when cross-compiling for a Cygwin64 target from Linux, and it works.

...in *your* example code.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Clang is using the wrong memory model

[Agner Fog]

Thanks a lot for your help in clarifying this.

When I complained here about the wasteful 64-bit addresses you said that 
it was an LLVM issue. When I complained to LLVM they said it was a 
Cygwin issue, and that you were using the wrong memory model.

All this confusion is due to a terrible lack of documentation of everything.
I had to do a lot of reverse engineering to figure out what is 
happening. What I have found out so far is listed below. Much of this is 
undocumented. Obviously, I would like to know if any or this is wrong or 
if specific documentation is available other than the SysV ABI and 
Windows ABI:

* Cygwin is using its own loader which is different from the Windows 
loader.
* The Cygwin loader emulates the behavior of Linux shared objects. This 
includes the ability to directly access a variable inside a DLL
* Access to a variable in a different DLL requires a 64-bit address. 
This is obtained by using the medium memory model with a gcc or Clang 
compiler.
* The small memory model works differently on different targets. A 
-mcmodel=small with a Linux target puts everything below 2GB addresses. 
32-bit absolute addresses are allowed. -mcmodel=small with a Windows or 
Mac target allows addresses above 2GB, but limits the distance between 
code and data in the same executable to 2GB. 32-bit absolute addresses 
are not allowed. 32-bit relative addresses are used instead.
* The memory models work differently in gcc an Clang. Gcc with a medium 
or large memory model is using 64-bit address tables to access a 
variable in a different C/CPP file. Clang with a medium or large memory 
model is using 64-bit addresses not only for external variables, but 
also for local static data. This includes floating point constants, 
string constants, array initializers, jump tables, global variables, and 
more.
* Cygwin uses a medium memory model by default. The medium memory model 
is necessary only for a program that makes direct access to a variable 
in a different DLL. The medium memory model is wasteful, and more so 
with Clang than with gcc.

Now I am speculating what we can do to avoid the wasteful 64-bit 
address-load instructions to improve the performance of Cygwin programs.

We can improve performance by using the small memory model when 
possible. The medium memory model is needed only for programs that link 
to a variable in a different DLL. The DLL that contains the link target 
does not need the medium memory model.

Direct access to a variable in a different DLL is considered bad 
programming practice by modern standards. This should occur only in old 
Linux code.

A link to a variable in a different DLL may be replaced by function 
calls (this is done with errno). In some cases, static linking can be an 
efficient alternative.

It would be helpful if the Cygwin loader could print the name of the 
offending variable when relocation fails with the small memory model. 
This could help programmers remove any obstacles to using the more 
efficient small memory model.


Agner


On 17/08/2019 10.16, Corinna Vinschen wrote:
> Oe Aug 17 07:31, Agner Fog wrote:
>>> So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
>>> __progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
>>> for kicks.
>> __ctype_ptr__ is a function
>>
>> h_errno works like errno with an imported function
>>
>> FE_DFL_ENV is a macro
>>
>> __progname and optarg are local variables to each exe or dll
> That would contradict what, e.g., __progname is for.  Here's a test:
>
> $ cat > dll.c <<EOF
> #include <stdio.h>
>
> extern char *__progname;
>
> void
> printprog ()
> {
>    printf ("progname: %s\n", __progname);
> }
> EOF
> $ cat > main.c <<EOF
> extern void printprog();
>
> int
> main ()
> {
>    printprog ();
> }
> EOF
> $ uname -a
> CYGWIN_NT-10.0 vmbert10 3.1.0(0.340/5/3) 2019-08-16 14:36 x86_64 Cygwin
>
> Lets try the medium model first:
>
>    $ gcc -g -shared -mcmodel=medium -o dll.dll dll.c
>    $ gcc -g -mcmodel=medium -o main main.c dll.dll
>    $ ./main
>    progname: main
>
> Now let's try the small model:
>
>    $ gcc -g -shared -mcmodel=small -o dll.dll dll.c
>    $ gcc -g -mcmodel=small -o main main.c dll.dll
>    $ ./main
>    Cygwin runtime failure: /home/corinna/main.exe: Invalid relocation.  Offset
>    0xfffffffd80348989 at address 0x40000103b doesn't fit into 32 bits
>
> Now let's try without explicit mcmodel on the CLI:
>
>    $ gcc -g -shared -o dll.dll dll.c
>    $ gcc -g -o main main.c dll.dll
>    $ ./main
>    progname: main
>
>> gcc is using the small memory model by default in Cygwin64, and it works.
> No, it's not, see above.
>
>> clang is using the small memory by default when cross-compiling for a Cygwin64 target from Linux, and it works.
> ...in *your* example code.
>
>
> Corinna
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1941 bytes --]

On Aug 18 08:04, Agner Fog wrote:
> Thanks a lot for your help in clarifying this.
> 
> When I complained here about the wasteful 64-bit addresses you said that it
> was an LLVM issue.

I never said anything like that.  The issue is that your clang
linux->cygwin cross compiler uses the wrong model, that's all.  That's
a bug in clang or whatever it's using under the hood.  Clang should
follow what GCC does for years, using the medium model on Cygwin.

> When I complained to LLVM they said it was a Cygwin
> issue, and that you were using the wrong memory model.
> 
> All this confusion is due to a terrible lack of documentation of everything.
> I had to do a lot of reverse engineering to figure out what is happening.
> What I have found out so far is listed below. Much of this is undocumented.
> Obviously, I would like to know if any or this is wrong or if specific
> documentation is available other than the SysV ABI and Windows ABI:
> 
> * Cygwin is using its own loader which is different from the Windows loader.

Nope, Cygwin uses the Windows loader.

> * The Cygwin loader emulates the behavior of Linux shared objects. This
> includes the ability to directly access a variable inside a DLL

See above.

> * Access to a variable in a different DLL requires a 64-bit address. This is
> obtained by using the medium memory model with a gcc or Clang compiler.

ACK

[...]

To me, the only interesting thing is that clang continues to use the
medium memory model *by default*.  It doesn't make sense if package
maintainers and devs building something on Cygwin have to care for
memory models all of a sudden.  The default should just work.

If you want to use the small model in your own projects, great, if it
works for you.  If the medium model is wasteful in clang, that's a clang
optimization problem, not a Cygwin problem.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Clang is using the wrong memory model

[Agner Fog]

On 18/08/2019 13.57, Corinna Vinschen wrote:
> Nope, Cygwin uses the Windows loader. 

Then, how do you do the extra linking? What is producing the "Cygwin 
runtime failure" message when loading/linking a DLL fails?

>   If the medium model is wasteful in clang, that's a clang
> optimization problem, not a Cygwin problem.

The medium model in Clang is not wasteful. It does exactly what it is 
designed to do. It was never designed with Cygwin in mind. The program 
build with a medium model is wasteful because it makes all addresses 64 
bits when few or no addresses actually need to be 64 bits.

> If you want to use the small model in your own projects, great, if it
> works for you.
It is not for my own project. I am writing manuals on how to optimize 
software.

Agner


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Clang is using the wrong memory model

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1329 bytes --]

On Aug 18 18:24, Agner Fog wrote:
> On 18/08/2019 13.57, Corinna Vinschen wrote:
> > Nope, Cygwin uses the Windows loader.
> 
> Then, how do you do the extra linking? What is producing the "Cygwin runtime
> failure" message when loading/linking a DLL fails?
> 
> >   If the medium model is wasteful in clang, that's a clang
> > optimization problem, not a Cygwin problem.
> 
> The medium model in Clang is not wasteful.

Your words:

* The memory models work differently in gcc an Clang. Gcc with a medium or
large memory model is using 64-bit address tables to access a variable in a
different C/CPP file. Clang with a medium or large memory model is using
64-bit addresses not only for external variables, but also for local static
data. This includes floating point constants, string constants, array
initializers, jump tables, global variables, and more.

> designed to do. It was never designed with Cygwin in mind.

That's not Cygwin's fault, is it?  You're asking to change the system to
pamper the compiler, rather than asking the compiler to work better for
a given system.  That's not how this usually works.  

Make sure that clang uses the medium memory model by default.  If clang
is doing this inefficiently, clang needs fixing.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Inefficient use of 64-bit addresses in Clang

[Agner Fog]

Clang is using 64-bit absolute addresses when accessing static data in 
64-bit mode. This is inefficient because it requires an extra 10-bytes 
long instruction for loading an address into a register every time it 
needs to access static data. All other compilers use relative addresses.

Example:

> #include <immintrin.h>
>
> __m128d test (__m128d a) {
>     __m128d b = _mm_add_pd(a, _mm_set1_pd(1.5));
>     __m128d c = _mm_mul_pd(b, _mm_set1_pd(2.5));
>     return c;
> }

Assembly output:

> .LCPI0_0:
>     .quad    4609434218613702656     # double 1.5
>     .quad    4609434218613702656     # double 1.5
> .LCPI0_1:
>     .quad    4612811918334230528     # double 2.5
>     .quad    4612811918334230528     # double 2.5
>     .text
>     .globl    _Z4testDv2_d
>     .p2align    4, 0x90
> _Z4testDv2_d:                           # @_Z4testDv2_d
> # BB#0:
>     vmovapd    (%rcx), %xmm0
>     movabsq    $.LCPI0_0, %rax
>     vaddpd    (%rax), %xmm0, %xmm0
>     movabsq    $.LCPI0_1, %rax
>     vmulpd    (%rax), %xmm0, %xmm0
>     retq

Linux Clang uses 32-bit relative addresses:

>     vaddpd    .LCPI0_0(%rip), %xmm0, %xmm0
>     vmulpd    .LCPI0_1(%rip), %xmm0, %xmm0
>     retq


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Inefficient use of 64-bit addresses in Clang

[Mark Geisert]

Agner Fog wrote:
> Clang is using 64-bit absolute addresses when accessing static data in 64-bit 
> mode. This is inefficient because it requires an extra 10-bytes long instruction 
> for loading an address into a register every time it needs to access static 
> data. All other compilers use relative addresses.
> 
> Example:
> 
>> #include <immintrin.h>
>>
>> __m128d test (__m128d a) {
>>     __m128d b = _mm_add_pd(a, _mm_set1_pd(1.5));
>>     __m128d c = _mm_mul_pd(b, _mm_set1_pd(2.5));
>>     return c;
>> }
> 
> Assembly output:
> 
>> .LCPI0_0:
>>     .quad    4609434218613702656     # double 1.5
>>     .quad    4609434218613702656     # double 1.5
>> .LCPI0_1:
>>     .quad    4612811918334230528     # double 2.5
>>     .quad    4612811918334230528     # double 2.5
>>     .text
>>     .globl    _Z4testDv2_d
>>     .p2align    4, 0x90
>> _Z4testDv2_d:                           # @_Z4testDv2_d
>> # BB#0:
>>     vmovapd    (%rcx), %xmm0
>>     movabsq    $.LCPI0_0, %rax
>>     vaddpd    (%rax), %xmm0, %xmm0
>>     movabsq    $.LCPI0_1, %rax
>>     vmulpd    (%rax), %xmm0, %xmm0
>>     retq
> 
> Linux Clang uses 32-bit relative addresses:
> 
>>     vaddpd    .LCPI0_0(%rip), %xmm0, %xmm0
>>     vmulpd    .LCPI0_1(%rip), %xmm0, %xmm0
>>     retq

This kind of bug report should probably go to the Clang bugtracker at 
https://bugs.llvm.org .  On that page you could enter "clang cygwin" in the 
Quick Search field and see what known issues there are.

Clang is packaged for Cygwin by a very industrious and busy maintainer, but no 
development on Clang is being done here (to my knowledge).  So something like a 
Cygwin packaging error that breaks Clang on Cygwin would be on-topic for this 
list, but a problem at code generation level should go to the Clang developers.

..mark

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple