AVX on Cygwin

AVX on Cygwin

[Achim Gratz]

You may have noted that the recent gmp update makes problems on some
machines and two of the three reports come from Broadwell CPU.  There is
one thing that did indeed change with the update and that is use of the
AVX ADC instruction on Broadwell/Skylake.  Is it possible that somehow
the stack model or some register save/restore is different on Cygwin
that would produce that problem?  I can only test on SandyBridge and
IvyBridge for Intel and these have no problem.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: AVX on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2002 bytes --]

Hi Achim,

On Nov 26 19:17, Achim Gratz wrote:
> 
> You may have noted that the recent gmp update makes problems on some
> machines and two of the three reports come from Broadwell CPU.  There is
> one thing that did indeed change with the update and that is use of the
> AVX ADC instruction on Broadwell/Skylake.  Is it possible that somehow
> the stack model or some register save/restore is different on Cygwin
> that would produce that problem?  I can only test on SandyBridge and
> IvyBridge for Intel and these have no problem.

Prior to the invention of the AVX stuff, a thread context structure had
a fixed size.  To hold AVX registers, the context structure became a
variable sized struct which has to be initialized with the function
InitializeContext().  It also requires an additional call to
SetXStateFeaturesMask() before being able to call GetThreadContext().
None of that is implemanted in Cygwin.

The affected scenarios are:

- sigdelayed, the function handling delayed signals, does not handle
  AVX registers,  This potentially affects almost all system calls.

- setjmp/longjmp

- Saving and restoring thread contexts during signal handling or
  in the setcontext/getcontext/swapcontext/makecontext calls
  use the CONTEXT_FULL flag which omits AVX registers.

- The mcontext_t structure has no provisions to hold AVX registers
  so they are not available in SA_SIGINFO signal handlers.

The problem with all of that is that the exposed structures are fixed
sized and have no room for extension without breaking the ABI.

So adding AVX register support is quite some work, especially in the
assembler code in the gendef script.  For that, I'd need help.
Fortunately there are not so many calls exposing contexts, only the
aforemetioned ones.

I hope this affects 64 bit only?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: AVX on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2201 bytes --]

On Nov 27 10:32, Corinna Vinschen wrote:
> Hi Achim,
> 
> On Nov 26 19:17, Achim Gratz wrote:
> > 
> > You may have noted that the recent gmp update makes problems on some
> > machines and two of the three reports come from Broadwell CPU.  There is
> > one thing that did indeed change with the update and that is use of the
> > AVX ADC instruction on Broadwell/Skylake.  Is it possible that somehow
> > the stack model or some register save/restore is different on Cygwin
> > that would produce that problem?  I can only test on SandyBridge and
> > IvyBridge for Intel and these have no problem.
> 
> Prior to the invention of the AVX stuff, a thread context structure had
> a fixed size.  To hold AVX registers, the context structure became a
> variable sized struct which has to be initialized with the function
> InitializeContext().  It also requires an additional call to
> SetXStateFeaturesMask() before being able to call GetThreadContext().
> None of that is implemanted in Cygwin.
> 
> The affected scenarios are:
> 
> - sigdelayed, the function handling delayed signals, does not handle
>   AVX registers,  This potentially affects almost all system calls.
> 
> - setjmp/longjmp
> 
> - Saving and restoring thread contexts during signal handling or
>   in the setcontext/getcontext/swapcontext/makecontext calls
>   use the CONTEXT_FULL flag which omits AVX registers.
> 
> - The mcontext_t structure has no provisions to hold AVX registers
>   so they are not available in SA_SIGINFO signal handlers.
> 
> The problem with all of that is that the exposed structures are fixed
> sized and have no room for extension without breaking the ABI.
> 
> So adding AVX register support is quite some work, especially in the
> assembler code in the gendef script.  For that, I'd need help.
> Fortunately there are not so many calls exposing contexts, only the
> aforemetioned ones.
> 
> I hope this affects 64 bit only?

Btw., for the time being it might be prudent to disable AVX in gmp...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: AVX on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

On Nov 27 10:37, Corinna Vinschen wrote:
> On Nov 27 10:32, Corinna Vinschen wrote:
> > Hi Achim,
> > 
> > On Nov 26 19:17, Achim Gratz wrote:
> > > 
> > > You may have noted that the recent gmp update makes problems on some
> > > machines and two of the three reports come from Broadwell CPU.  There is
> > > one thing that did indeed change with the update and that is use of the
> > > AVX ADC instruction on Broadwell/Skylake.  Is it possible that somehow
> > > the stack model or some register save/restore is different on Cygwin
> > > that would produce that problem?  I can only test on SandyBridge and
> > > IvyBridge for Intel and these have no problem.
> > 
> > Prior to the invention of the AVX stuff, a thread context structure had
> > a fixed size.  To hold AVX registers, the context structure became a
> > variable sized struct which has to be initialized with the function
> > InitializeContext().  It also requires an additional call to
> > SetXStateFeaturesMask() before being able to call GetThreadContext().
> > None of that is implemanted in Cygwin.
> > 
> > The affected scenarios are:
> > 
> > - sigdelayed, the function handling delayed signals, does not handle
> >   AVX registers,  This potentially affects almost all system calls.
> > 
> > - setjmp/longjmp
> > 
> > - Saving and restoring thread contexts during signal handling or
> >   in the setcontext/getcontext/swapcontext/makecontext calls
> >   use the CONTEXT_FULL flag which omits AVX registers.
> > 
> > - The mcontext_t structure has no provisions to hold AVX registers
> >   so they are not available in SA_SIGINFO signal handlers.
> > 
> > The problem with all of that is that the exposed structures are fixed
> > sized and have no room for extension without breaking the ABI.
> > 
> > So adding AVX register support is quite some work, especially in the
> > assembler code in the gendef script.  For that, I'd need help.
> > Fortunately there are not so many calls exposing contexts, only the
> > aforemetioned ones.
> > 
> > I hope this affects 64 bit only?
> 
> Btw., for the time being it might be prudent to disable AVX in gmp...

...and never mind setjmp/longjmp.  Only the functions utilizing
ucontext_t stuff and the internal save/restores will have to be
changed.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: AVX on Cygwin

[Achim Gratz]

Corinna Vinschen writes:
>> I hope this affects 64 bit only?

I don't know if it would even be possible to run AVX code from 32bit,
but yes, the three reports so far have been from 64bit systems.

> Btw., for the time being it might be prudent to disable AVX in gmp...

There is no configure option for that when producing a fat binary.  I
guess the GMP folks didn't test it on Cygwin on a system where that code
is actually used.  Haswell and IvyBridge also use some AVX bits, but
apparently nothing that bugs out (or nobody has found it yet).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: AVX on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1271 bytes --]

On Nov 27 20:33, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> I hope this affects 64 bit only?
> 
> I don't know if it would even be possible to run AVX code from 32bit,
> but yes, the three reports so far have been from 64bit systems.
> 
> > Btw., for the time being it might be prudent to disable AVX in gmp...
> 
> There is no configure option for that when producing a fat binary.  I
> guess the GMP folks didn't test it on Cygwin on a system where that code
> is actually used.  Haswell and IvyBridge also use some AVX bits, but
> apparently nothing that bugs out (or nobody has found it yet).

On second thought, what I'm wondering about is what exactly *is* the
problem with AVX?  While the AVX context isn't saved when running signal
handlers or getcontext, it's very unlikely that the AVX state changes at
all when running a system function or signal handler.  No system function
in Cygwin affects the AVX registers.  And a signal handler running AVX
functions seems unlikely, too.

Before changing Cygwin it would be nice to know exactly where/when the
problem occurs.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: AVX on Cygwin

[Achim Gratz]

Corinna Vinschen writes:
> On second thought, what I'm wondering about is what exactly *is* the
> problem with AVX?  While the AVX context isn't saved when running signal
> handlers or getcontext, it's very unlikely that the AVX state changes at
> all when running a system function or signal handler.  No system function
> in Cygwin affects the AVX registers.  And a signal handler running AVX
> functions seems unlikely, too.

I've tried to figure something out from the strace that was posted on
the main list, but I don't see anything (it's big and I may have missed
it).

> Before changing Cygwin it would be nice to know exactly where/when the
> problem occurs.

Please don't change anything yet.  I've read the release notes again,
the thing specific to Broadwell/Skylake are the two ADX instructions,
which should be using plain 64bit registers.  For whatever reason these
two architectures have special treatment only on 64bit and are ignored
for 32bit if I read things correctly in the source.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: AVX on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]

On Nov 27 22:28, Achim Gratz wrote:
> Corinna Vinschen writes:
> > On second thought, what I'm wondering about is what exactly *is* the
> > problem with AVX?  While the AVX context isn't saved when running signal
> > handlers or getcontext, it's very unlikely that the AVX state changes at
> > all when running a system function or signal handler.  No system function
> > in Cygwin affects the AVX registers.  And a signal handler running AVX
> > functions seems unlikely, too.
> 
> I've tried to figure something out from the strace that was posted on
> the main list, but I don't see anything (it's big and I may have missed
> it).
> 
> > Before changing Cygwin it would be nice to know exactly where/when the
> > problem occurs.
> 
> Please don't change anything yet.  I've read the release notes again,
> the thing specific to Broadwell/Skylake are the two ADX instructions,
> which should be using plain 64bit registers.  For whatever reason these
> two architectures have special treatment only on 64bit and are ignored
> for 32bit if I read things correctly in the source.

Ok, but I applied a small change nevertheless.  While looking into
this issue I realized that the MXCSR register wasn't preserved
during the call to sigdelayed.  This might or might not influence
the result.  I bet the latter, but better safe than sorry...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]