gcc and 128-bit compare/exchange

gcc and 128-bit compare/exchange

[Eliot Moss]

This is probably to the gcc maintainer ...

I am running on a processor that has compare/exchange 128-bit (cx16 capability),
and I compiler with -mcx16 and -latomic.  I'm on the latest release cygwin gcc
(9.2.0-3, I believe) and the corresponding libatomic.  I have a program with
this in it:

__atomic_compare_exchange((__int128 *)&s1, (__int128 *)&z, (__int128 *)&s2, 0, __ATOMIC_SEQ_CST, 
__ATOMIC_SEQ_CST);

This compiles to a call (nice if it would inline, but ...) to
__atomic_compare_exchange_16, which uses mutex's, not the CMPXCHG16B
instruction I was hoping for.  Note I am doing dynamic linking,
which on at least one other platform results in dynamic selection
of a lib_at implementation of the compare/exchange, which does use
the desired instruction.

Is this a limitation of cygwin gcc, or should I be doing something
different to achieve the desired effect?

Obviously it would be best not to going an asm inline if I can avoid it,
but I suppose I can dig into the libatomic source to get the right
incantation for it if need be ...

Regards - Eliot Moss

Re: gcc and 128-bit compare/exchange

[Eliot Moss]

On 3/8/2020 10:29 PM, Eliot Moss wrote:
> This is probably to the gcc maintainer ...
> 
> I am running on a processor that has compare/exchange 128-bit (cx16 capability),
> and I compiler with -mcx16 and -latomic.  I'm on the latest release cygwin gcc
> (9.2.0-3, I believe) and the corresponding libatomic.  I have a program with
> this in it:
> 
> __atomic_compare_exchange((__int128 *)&s1, (__int128 *)&z, (__int128 *)&s2, 0, __ATOMIC_SEQ_CST, 
> __ATOMIC_SEQ_CST);
> 
> This compiles to a call (nice if it would inline, but ...) to
> __atomic_compare_exchange_16, which uses mutex's, not the CMPXCHG16B
> instruction I was hoping for.  Note I am doing dynamic linking,
> which on at least one other platform results in dynamic selection
> of a lib_at implementation of the compare/exchange, which does use
> the desired instruction.
> 
> Is this a limitation of cygwin gcc, or should I be doing something
> different to achieve the desired effect?
> 
> Obviously it would be best not to going an asm inline if I can avoid it,
> but I suppose I can dig into the libatomic source to get the right
> incantation for it if need be ...

A quick followup: I was able to get __sync_val_compare_and_swap_16 to work
(and its bool form).  That will do for now, though of course it's deprecated.

Regards - EM

Re: gcc and 128-bit compare/exchange

[Brian Inglis]

On 2020-03-08 20:59, Eliot Moss wrote:
> On 3/8/2020 10:29 PM, Eliot Moss wrote:
>> This is probably to the gcc maintainer ...
>>
>> I am running on a processor that has compare/exchange 128-bit (cx16 capability),
>> and I compiler with -mcx16 and -latomic.  I'm on the latest release cygwin gcc
>> (9.2.0-3, I believe) and the corresponding libatomic.  I have a program with
>> this in it:
>>
>> __atomic_compare_exchange((__int128 *)&s1, (__int128 *)&z, (__int128 *)&s2, 0,
>> __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
>>
>> This compiles to a call (nice if it would inline, but ...) to
>> __atomic_compare_exchange_16, which uses mutex's, not the CMPXCHG16B
>> instruction I was hoping for.  Note I am doing dynamic linking,
>> which on at least one other platform results in dynamic selection
>> of a lib_at implementation of the compare/exchange, which does use
>> the desired instruction.
>>
>> Is this a limitation of cygwin gcc, or should I be doing something
>> different to achieve the desired effect?
>>
>> Obviously it would be best not to going an asm inline if I can avoid it,
>> but I suppose I can dig into the libatomic source to get the right
>> incantation for it if need be ...
> 
> A quick followup: I was able to get __sync_val_compare_and_swap_16 to work
> (and its bool form).  That will do for now, though of course it's deprecated.

You just needed to go to the next info page:

	$ info gcc __atomic

and use:

	#include <stdatomic.h>
	extern bool __atomic_compare_exchange ( TYPE *ptr,
						TYPE *expected,
						TYPE *desired,
						bool weak,
						int success_memorder,
						int failure_memorder);

or higher level macro:

	atomic_compare_exchange_strong(PTR, VAL, DES)

defined in /lib/gcc/x86_64-pc-cygwin/*/include/stdatomic.h

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

Re: gcc and 128-bit compare/exchange

[Eliot Moss]

On 3/10/2020 4:35 PM, Brian Inglis wrote:
> On 2020-03-08 20:59, Eliot Moss wrote:
>> On 3/8/2020 10:29 PM, Eliot Moss wrote:
>>> This is probably to the gcc maintainer ...
>>>
>>> I am running on a processor that has compare/exchange 128-bit (cx16 capability),
>>> and I compiler with -mcx16 and -latomic.  I'm on the latest release cygwin gcc
>>> (9.2.0-3, I believe) and the corresponding libatomic.  I have a program with
>>> this in it:
>>>
>>> __atomic_compare_exchange((__int128 *)&s1, (__int128 *)&z, (__int128 *)&s2, 0,
>>> __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
>>>
>>> This compiles to a call (nice if it would inline, but ...) to
>>> __atomic_compare_exchange_16, which uses mutex's, not the CMPXCHG16B
>>> instruction I was hoping for.  Note I am doing dynamic linking,
>>> which on at least one other platform results in dynamic selection
>>> of a lib_at implementation of the compare/exchange, which does use
>>> the desired instruction.
>>>
>>> Is this a limitation of cygwin gcc, or should I be doing something
>>> different to achieve the desired effect?
>>>
>>> Obviously it would be best not to going an asm inline if I can avoid it,
>>> but I suppose I can dig into the libatomic source to get the right
>>> incantation for it if need be ...
>>
>> A quick followup: I was able to get __sync_val_compare_and_swap_16 to work
>> (and its bool form).  That will do for now, though of course it's deprecated.
> 
> You just needed to go to the next info page:
> 
> 	$ info gcc __atomic
> 
> and use:
> 
> 	#include <stdatomic.h>
> 	extern bool __atomic_compare_exchange ( TYPE *ptr,
> 						TYPE *expected,
> 						TYPE *desired,
> 						bool weak,
> 						int success_memorder,
> 						int failure_memorder);
> 
> or higher level macro:
> 
> 	atomic_compare_exchange_strong(PTR, VAL, DES)
> 
> defined in /lib/gcc/x86_64-pc-cygwin/*/include/stdatomic.h

That is what I was already doing :-) .... it compiles and runs,
landing at libat_compare_exchange_16, which uses mutexes, not the
16-byte compare-exchange instruction.  So how do I get the compare-
exchange instruction to be used in the library?

Regards - EM

Re: gcc and 128-bit compare/exchange

[Brian Inglis]

On 2020-03-10 15:13, Eliot Moss wrote:
> On 3/10/2020 4:35 PM, Brian Inglis wrote:
>> On 2020-03-08 20:59, Eliot Moss wrote:
>>> On 3/8/2020 10:29 PM, Eliot Moss wrote:
>>>> This is probably to the gcc maintainer ...
>>>>
>>>> I am running on a processor that has compare/exchange 128-bit (cx16
>>>> capability),
>>>> and I compiler with -mcx16 and -latomic.  I'm on the latest release cygwin gcc
>>>> (9.2.0-3, I believe) and the corresponding libatomic.  I have a program with
>>>> this in it:
>>>>
>>>> __atomic_compare_exchange((__int128 *)&s1, (__int128 *)&z, (__int128 *)&s2, 0,
>>>> __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
>>>>
>>>> This compiles to a call (nice if it would inline, but ...) to
>>>> __atomic_compare_exchange_16, which uses mutex's, not the CMPXCHG16B
>>>> instruction I was hoping for.  Note I am doing dynamic linking,
>>>> which on at least one other platform results in dynamic selection
>>>> of a lib_at implementation of the compare/exchange, which does use
>>>> the desired instruction.
>>>>
>>>> Is this a limitation of cygwin gcc, or should I be doing something
>>>> different to achieve the desired effect?
>>>>
>>>> Obviously it would be best not to going an asm inline if I can avoid it,
>>>> but I suppose I can dig into the libatomic source to get the right
>>>> incantation for it if need be ...
>>>
>>> A quick followup: I was able to get __sync_val_compare_and_swap_16 to work
>>> (and its bool form).  That will do for now, though of course it's deprecated.
>>
>> You just needed to go to the next info page:
>>
>>     $ info gcc __atomic
>>
>> and use:
>>
>>     #include <stdatomic.h>
>>     extern bool __atomic_compare_exchange ( TYPE *ptr,
>>                         TYPE *expected,
>>                         TYPE *desired,
>>                         bool weak,
>>                         int success_memorder,
>>                         int failure_memorder);
>>
>> or higher level macro:
>>
>>     atomic_compare_exchange_strong(PTR, VAL, DES)
>>
>> defined in /lib/gcc/x86_64-pc-cygwin/*/include/stdatomic.h
> 
> That is what I was already doing :-) .... it compiles and runs,
> landing at libat_compare_exchange_16, which uses mutexes, not the
> 16-byte compare-exchange instruction.  So how do I get the compare-
> exchange instruction to be used in the library?

Digging further into the murk where a simple builtin inline cmpxchg16b isn't.

Doing what you're doing now seems easier and better supported than alternatives.
You can drop stdatomic.h and -latomic as the low level functions are builtins.

You could also write your own inline function using cmpxchg16b directly in asm.

Looks like because of cpu and library requirements, you would have to enable
indirect inline functions in gcc, write libatomic library functions which
support gcc indirect inline functions, to test cpu cx16 feature, then setup
indirection, to bypass mutexes.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

Re: gcc and 128-bit compare/exchange

[Eliot Moss]

On 3/11/2020 1:31 AM, Brian Inglis wrote:

>>>> A quick followup: I was able to get __sync_val_compare_and_swap_16 to work
>>>> (and its bool form).  That will do for now, though of course it's deprecated.

This does get me the inlined asm code.

> Digging further into the murk where a simple builtin inline cmpxchg16b isn't.
> 
> Doing what you're doing now seems easier and better supported than alternatives.
> You can drop stdatomic.h and -latomic as the low level functions are builtins.
> 
> You could also write your own inline function using cmpxchg16b directly in asm.
> 
> Looks like because of cpu and library requirements, you would have to enable
> indirect inline functions in gcc, write libatomic library functions which
> support gcc indirect inline functions, to test cpu cx16 feature, then setup
> indirection, to bypass mutexes.

There are two issues here:

1) Whether the call is inlined.

2) Whether I get the compare-exchange instruction or a block of code protected
    by a pthread mutex.

The __atomic functions do not inline, as far as I know, from my testing on true
Linux platforms.  But Linux _does_ give me the compare-exchange version of
the function.

What I am really reporting is that Cygwin is giving the pthread mutex form
when it should not be.  My CPU clearly has the capability, and the compiler clearly
knows how to emit the instruction, since the __sync form does it.  What is
mysterious and tangled is why libatomic / libc are not delivering the desired
version of the atomic compare-exchange function.

My _guess_ is that having that as an option depends somehow on how gcc is built,
e.g., its build configuration.  I don't know enough about how those ifunc mechanisms
work to know if that is it, or if it's something else, but the behavior seems to
be Cygwin-specific.  Hence my report.

Regards - Eliot

Re: gcc and 128-bit compare/exchange

[Brian Inglis]

On 2020-03-11 00:13, Eliot Moss wrote:
> On 3/11/2020 1:31 AM, Brian Inglis wrote:
> 
>>>>> A quick followup: I was able to get __sync_val_compare_and_swap_16 to work
>>>>> (and its bool form).  That will do for now, though of course it's deprecated.
> 
> This does get me the inlined asm code.
> 
>> Digging further into the murk where a simple builtin inline cmpxchg16b isn't.
>>
>> Doing what you're doing now seems easier and better supported than alternatives.
>> You can drop stdatomic.h and -latomic as the low level functions are builtins.
>>
>> You could also write your own inline function using cmpxchg16b directly in asm.
>>
>> Looks like because of cpu and library requirements, you would have to enable
>> indirect inline functions in gcc, write libatomic library functions which
>> support gcc indirect inline functions, to test cpu cx16 feature, then setup
>> indirection, to bypass mutexes.
> 
> There are two issues here:
> 
> 1) Whether the call is inlined.
> 
> 2) Whether I get the compare-exchange instruction or a block of code protected
>    by a pthread mutex.
> 
> The __atomic functions do not inline, as far as I know, from my testing on true
> Linux platforms.  But Linux _does_ give me the compare-exchange version of
> the function.
> 
> What I am really reporting is that Cygwin is giving the pthread mutex form
> when it should not be.  My CPU clearly has the capability, and the compiler clearly
> knows how to emit the instruction, since the __sync form does it.  What is
> mysterious and tangled is why libatomic / libc are not delivering the desired
> version of the atomic compare-exchange function.
> 
> My _guess_ is that having that as an option depends somehow on how gcc is
> built, e.g., its build configuration.  I don't know enough about how those
> ifunc mechanisms work to know if that is it, or if it's something else, but
> the behavior seems to be Cygwin-specific. Hence my report.
There are gcc bugzilla comments about requiring gcc to be built with glibc
libatomic to guarantee indirect inline functions support, and presumably glibc
detecting gcc indirect inline functions support, and not supporting other libc
variants including musl, newlib, uclibc, etc.

The problem is that newlib is BSD licensed and glibc is GPL and you can not
contaminate newlib by looking at or including GPL code, although you may be able
to do so in the Cygwin winsup library.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

Re: gcc and 128-bit compare/exchange

[Achim Gratz]

Eliot Moss writes:
> What I am really reporting is that Cygwin is giving the pthread mutex form
> when it should not be.  My CPU clearly has the capability, and the compiler clearly
> knows how to emit the instruction, since the __sync form does it.  What is
> mysterious and tangled is why libatomic / libc are not delivering the desired
> version of the atomic compare-exchange function.

Assuming you want to use a library, it would have to detect this
capability at runtime (unless you force-build for a single CPU
architecture) and dispatch different code based on the result.  It's
probably the latter part that's missing on Cygwin.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: gcc and 128-bit compare/exchange

[Eliot Moss]

On 3/11/2020 12:30 PM, Brian Inglis wrote:
> On 2020-03-11 00:13, Eliot Moss wrote:
>> On 3/11/2020 1:31 AM, Brian Inglis wrote:

> There are gcc bugzilla comments about requiring gcc to be built with glibc
> libatomic to guarantee indirect inline functions support, and presumably glibc
> detecting gcc indirect inline functions support, and not supporting other libc
> variants including musl, newlib, uclibc, etc.
> 
> The problem is that newlib is BSD licensed and glibc is GPL and you can not
> contaminate newlib by looking at or including GPL code, although you may be able
> to do so in the Cygwin winsup library.

Hmm.  Well, I just install standard stuff on Linux and then on Cygwin, and
I see different behavior.  I don't know how licenses come into that (I'm not
saying they don't, only that it exceeds my knowledge).  Are you saying that
Cygwin's build of gcc is intended to work with other libraries in addition
to glibc, and hence Cygwin's gcc might have been built without some stuff
to avoid license contamination?

It is probably not worth my while to do my own build of gcc just for this.
I can just write my own wrapper for the __sync function.  But it seemed
wrong / broken to me that the __atomic builtin did not do what was expected.

(Brian, are you the maintainer, or is there someone else with whom the
conversation would be taken up?

Regards - Eliot

Re: gcc and 128-bit compare/exchange

[Brian Inglis]

On 2020-03-11 21:36, Eliot Moss wrote:
> On 3/11/2020 12:30 PM, Brian Inglis wrote:
>> On 2020-03-11 00:13, Eliot Moss wrote:
>>> On 3/11/2020 1:31 AM, Brian Inglis wrote:
> 
>> There are gcc bugzilla comments about requiring gcc to be built with glibc
>> libatomic to guarantee indirect inline functions support, and presumably glibc
>> detecting gcc indirect inline functions support, and not supporting other libc
>> variants including musl, newlib, uclibc, etc.
>>
>> The problem is that newlib is BSD licensed and glibc is GPL and you can not
>> contaminate newlib by looking at or including GPL code, although you may be able
>> to do so in the Cygwin winsup library.
> 
> Hmm.  Well, I just install standard stuff on Linux and then on Cygwin, and
> I see different behavior.  I don't know how licenses come into that (I'm not
> saying they don't, only that it exceeds my knowledge).  Are you saying that
> Cygwin's build of gcc is intended to work with other libraries in addition
> to glibc, and hence Cygwin's gcc might have been built without some stuff
> to avoid license contamination?

All gccs allow building and working with any adequate libc including musl,
newlib, uclibc, etc. but on Cygwin it is winsup for system stuff with newlib for
generic stuff.

All I was pointing out was that while you could not copy LGPLed glibc libatomic
code to BSD licensed newlib, you should be able to copy LGPLed glibc libatomic
code to LGPLed Cygwin winsup libc, if you want to enable it with ifuncs,
assuming they work under Windows.

> It is probably not worth my while to do my own build of gcc just for this.
> I can just write my own wrapper for the __sync function.  But it seemed
> wrong / broken to me that the __atomic builtin did not do what was expected.
> 
> (Brian, are you the maintainer, or is there someone else with whom the
> conversation would be taken up?

It's gcc maintainer (see announcement but post here), and base project
committers for newlib (via newlib AT sourceware DOT org) and winsup (via cygwin
DASH patches AT cygwin DOT com), ml archives in same place as Cygwin lists.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

Re: gcc and 128-bit compare/exchange

[Eliot Moss]

On 3/12/2020 5:13 PM, Brian Inglis wrote:
> On 2020-03-11 21:36, Eliot Moss wrote:
>> On 3/11/2020 12:30 PM, Brian Inglis wrote:
>>> On 2020-03-11 00:13, Eliot Moss wrote:
>>>> On 3/11/2020 1:31 AM, Brian Inglis wrote:
>>
>>> There are gcc bugzilla comments about requiring gcc to be built with glibc
>>> libatomic to guarantee indirect inline functions support, and presumably glibc
>>> detecting gcc indirect inline functions support, and not supporting other libc
>>> variants including musl, newlib, uclibc, etc.
>>>
>>> The problem is that newlib is BSD licensed and glibc is GPL and you can not
>>> contaminate newlib by looking at or including GPL code, although you may be able
>>> to do so in the Cygwin winsup library.
>>
>> Hmm.  Well, I just install standard stuff on Linux and then on Cygwin, and
>> I see different behavior.  I don't know how licenses come into that (I'm not
>> saying they don't, only that it exceeds my knowledge).  Are you saying that
>> Cygwin's build of gcc is intended to work with other libraries in addition
>> to glibc, and hence Cygwin's gcc might have been built without some stuff
>> to avoid license contamination?
> 
> All gccs allow building and working with any adequate libc including musl,
> newlib, uclibc, etc. but on Cygwin it is winsup for system stuff with newlib for
> generic stuff.
> 
> All I was pointing out was that while you could not copy LGPLed glibc libatomic
> code to BSD licensed newlib, you should be able to copy LGPLed glibc libatomic
> code to LGPLed Cygwin winsup libc, if you want to enable it with ifuncs,
> assuming they work under Windows.
> 
>> It is probably not worth my while to do my own build of gcc just for this.
>> I can just write my own wrapper for the __sync function.  But it seemed
>> wrong / broken to me that the __atomic builtin did not do what was expected.
>>
>> (Brian, are you the maintainer, or is there someone else with whom the
>> conversation would be taken up?
> 
> It's gcc maintainer (see announcement but post here), and base project
> committers for newlib (via newlib AT sourceware DOT org) and winsup (via cygwin
> DASH patches AT cygwin DOT com), ml archives in same place as Cygwin lists.

Thank you for the additional information.  At this point, I remain a little
unclear as to what you are suggesting my next step should be:

- Are you suggesting I do my own private build?
- Are you suggesting that I email those other lists?
- Are you thinking that the relevant maintainer would be reading this and that,
   for the moment, I should just wait for further response?

Regards - Eliot

Re: gcc and 128-bit compare/exchange

[Brian Inglis]

On 2020-03-12 16:49, Eliot Moss wrote:
> On 3/12/2020 5:13 PM, Brian Inglis wrote:
>> On 2020-03-11 21:36, Eliot Moss wrote:
>>> On 3/11/2020 12:30 PM, Brian Inglis wrote:
>>>> On 2020-03-11 00:13, Eliot Moss wrote:
>>>>> On 3/11/2020 1:31 AM, Brian Inglis wrote:
>>>
>>>> There are gcc bugzilla comments about requiring gcc to be built with glibc
>>>> libatomic to guarantee indirect inline functions support, and presumably glibc
>>>> detecting gcc indirect inline functions support, and not supporting other libc
>>>> variants including musl, newlib, uclibc, etc.
>>>>
>>>> The problem is that newlib is BSD licensed and glibc is GPL and you can not
>>>> contaminate newlib by looking at or including GPL code, although you may be
>>>> able
>>>> to do so in the Cygwin winsup library.
>>>
>>> Hmm.  Well, I just install standard stuff on Linux and then on Cygwin, and
>>> I see different behavior.  I don't know how licenses come into that (I'm not
>>> saying they don't, only that it exceeds my knowledge).  Are you saying that
>>> Cygwin's build of gcc is intended to work with other libraries in addition
>>> to glibc, and hence Cygwin's gcc might have been built without some stuff
>>> to avoid license contamination?
>>
>> All gccs allow building and working with any adequate libc including musl,
>> newlib, uclibc, etc. but on Cygwin it is winsup for system stuff with newlib for
>> generic stuff.
>>
>> All I was pointing out was that while you could not copy LGPLed glibc libatomic
>> code to BSD licensed newlib, you should be able to copy LGPLed glibc libatomic
>> code to LGPLed Cygwin winsup libc, if you want to enable it with ifuncs,
>> assuming they work under Windows.
>>
>>> It is probably not worth my while to do my own build of gcc just for this.
>>> I can just write my own wrapper for the __sync function.  But it seemed
>>> wrong / broken to me that the __atomic builtin did not do what was expected.
>>>
>>> (Brian, are you the maintainer, or is there someone else with whom the
>>> conversation would be taken up?
>>
>> It's gcc maintainer (see announcement but post here), and base project
>> committers for newlib (via newlib AT sourceware DOT org) and winsup (via cygwin
>> DASH patches AT cygwin DOT com), ml archives in same place as Cygwin lists.
> 
> Thank you for the additional information.  At this point, I remain a little
> unclear as to what you are suggesting my next step should be:
> 
> - Are you suggesting I do my own private build?
> - Are you suggesting that I email those other lists?
> - Are you thinking that the relevant maintainer would be reading this and that,
>   for the moment, I should just wait for further response?

Just putting the background and options out there.

Everyone makes their own choices for their own reasons.
Many may choose like you did and I would do the same.

Someone may need it enough to make it work sometime.
The nature of volunteer projects is that when something itches badly enough, it
gets scratched.

Some student may think it's worth some effort to learn, contribute, do a paper,
to add to resume, or propose for a Google Summer of Coronavirus project or
similar, with university classes going online or pausing, as spring break and
summer jobs may now be questionable.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.