[Bug target/54412] New: Request for 32-byte stack alignment with -mavx on Windows

[Bug target/54412] New: Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

             Bug #: 54412
           Summary: Request for 32-byte stack alignment with -mavx on
                    Windows
    Classification: Unclassified
           Product: gcc
           Version: 4.7.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: rcopley@gmail.com


Created attachment 28103
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28103
Self-contained C source, with AVX alignment bug on Windows

Code generated by GCC 4.7.1 for the Windows x86_64-w64-mingw32 target, with
"-mavx", can segfault due to alignment errors when the 32-byte ymm registers
are spilled onto the stack. May I please submit a feature request for 32-byte
stack alignment on this target where necessary?

Compiled for Windows with "gcc -O0 -m64 -mavx bug.c" using GCC 4.7.1 with the
MingGW W64 toolchain, the attached program segfaults. Specifically, it uses
vmovapd to copy the value of %ymm0 to a location on the stack before calling
f(), but doesn't align the location to 32 bytes as required by that
instruction. In contrast, the generated code for Linux (using GCC 4.6.3 from
Ubuntu) does explicitly align the stack to 32 bytes.

The lack of stack alignment on Windows has been noted before; see for
example http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49001 and
http://stackoverflow.com/questions/5983389.

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[ktietz at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Kai Tietz <ktietz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ktietz at gcc dot gnu.org

--- Comment #1 from Kai Tietz <ktietz at gcc dot gnu.org> ---
MS' abi doesn't allow this.  So I doubt we will be able to implement that for
this target.  If we want to re-align stack on function-base we will run into
troubles with SEH-information.
Doesn't it work to align explicit the variable itself?

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #2 from R Copley <rcopley at gmail dot com> ---
Created attachment 30793
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=30793&action=edit
As before, but with explicitly 32-byte aligned variables

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #4 from R Copley <rcopley at gmail dot com> ---
(In reply to Kai Tietz from comment #1)
> MS' abi doesn't allow this.  So I doubt we will be able to implement that
> for this target.  If we want to re-align stack on function-base we will run
> into troubles with SEH-information.

You might be right, I'm not sure. Are you aware that on 64-bit Windows, SEH is
table-based, not frame-based (see, e.g.,
http://www.osronline.com/article.cfm?article=469)?

> Doesn't it work to align explicit the variable itself?

No (see attachments 2 and 3). If I understand correctly, the alignment
specification is redundant anyway, because the variables are supposed to be
naturally aligned, on their size.

Assembling attachment 3 with "-g" and running it in gdb gives:

Program received signal SIGSEGV, Segmentation fault.
main () at bug1.s:46
46              vmovapd %ymm0, -96(%rbp)

Thanks.

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[roland at rschulz dot eu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #5 from Roland Schulz <roland at rschulz dot eu> ---
This seems to me to be a duplicate of 49001.

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #6 from R Copley <rcopley at gmail dot com> ---
As I mentioned in the description, this request was indeed related to that bug.
The test-case no longer crashes with recent MinGW-W64 toolchains (GCC 4.9.1).

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[roland at rschulz dot eu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #7 from Roland Schulz <roland at rschulz dot eu> ---
For me the problem isn't fixed with gcc 4.9.1. I tried two build a)
http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/installer/mingw-w64-install.exe/download
and b) http://nuwen.net/mingw.html. Did you use a special distribution or
special flags if you compiled gcc yourself?

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #8 from R Copley <rcopley at gmail dot com> ---
No, I use the mingw-builds distro too.

gcc --version
gcc (x86_64-win32-seh-rev0, Built by MinGW-W64 project) 4.9.1

Bizarrely, the attached program exits with a random error code unless I add a
"return 0;" statement to the end of the main function.
But it doesn't segfault.

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #9 from R Copley <rcopley at gmail dot com> ---
Heh, sorry. I don't really know C, I assumed it had an implicit "return 0;"
like C++. Apparently C99 has this but earlier C standards do not. So, not
bizarre at all.

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[rcopley at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #11 from R Copley <rcopley at gmail dot com> ---
On 20 September 2014 07:08, roland at rschulz dot eu
<gcc-bugzilla@gcc.gnu.org> wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412
>
> --- Comment #10 from Roland Schulz <roland at rschulz dot eu> ---
> Created attachment 33520
>   --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=33520&action=edit
> Slightly modified testcase
>
> This slightly modified testcase in which the return value isn't stored, still
> segfaults for me. With the 32bit mingw64 binary ((i686-win32-dwarf-rev1, Built
> by MinGW-W64 project) 4.9.1) it is OK, but with the 64bit binary
> ((x86_64-win32-seh-rev1, Built by MinGW-W64 project) 4.9.1) it segfaults.

Confirmed (with the same compiler, in the mingw-builds toolchain). I
compiled your testcase with command "gcc -O0 -g -ggdb -m64 -mavx
bug.c". It segfaults on the instruction marked "=>" below.

(gdb) disassemble /m
Dump of assembler code for function f:
6       {
   0x00000000004014f0 <+0>:     push   %rbp
   0x00000000004014f1 <+1>:     mov    %rsp,%rbp
   0x00000000004014f4 <+4>:     mov    %rcx,0x10(%rbp)
   0x00000000004014f8 <+8>:     sub    $0x40,%rsp
   0x00000000004014fc <+12>:    mov    %rsp,%rax
   0x00000000004014ff <+15>:    add    $0x1f,%rax
   0x0000000000401503 <+19>:    shr    $0x5,%rax
   0x0000000000401507 <+23>:    shl    $0x5,%rax

7         v4d x __attribute__ ((aligned (32))) = { 1.0, 2.0, 3.0, 4.0, };
   0x000000000040150b <+27>:    vmovapd 0x2aed(%rip),%ymm0        # 0x404000
   0x0000000000401513 <+35>:    vmovapd %ymm0,(%rax)

8         return x;
   0x0000000000401517 <+39>:    mov    0x10(%rbp),%rdx
   0x000000000040151b <+43>:    vmovapd (%rax),%ymm0
=> 0x000000000040151f <+47>:    vmovapd %ymm0,(%rdx)

9       }
   0x0000000000401523 <+51>:    mov    0x10(%rbp),%rax
   0x0000000000401527 <+55>:    mov    %rbp,%rsp
   0x000000000040152a <+58>:    pop    %rbp
   0x000000000040152b <+59>:    retq

End of assembler dump.
(gdb) print $rdx % 32
$1 = 16

[Bug target/54412] Request for 32-byte stack alignment with -mavx on Windows

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Kai Tietz <ktietz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |SUSPENDED
   Last reconfirmed|                            |2015-09-22
     Ever confirmed|0                           |1

--- Comment #12 from Kai Tietz <ktietz at gcc dot gnu.org> ---
It is good to hear that issue is fixed for 32-bit.  But for 64-bit - as I
already explained in comment above - this issue isn't fixable for
stack-variables.

The problem is that for x64 ABI we are tighten bound to SEH-prologue
information, and this can't express alignment-operations.  The x64 ABI
guarantee 16 byte alignment on function entry, therefore sse 128-bit operations
are possible to be placed fully aligned on stack, but higher alignment is
simply not expressible.

Therefore I will need to set this bug to suspended.  If this information gets
in future extended to allow such prologue-information we need for alignment,
then we will be able to fix that.  So long it is suspended.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[arthur200126 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Mingye Wang <arthur200126 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |arthur200126 at gmail dot com

--- Comment #30 from Mingye Wang <arthur200126 at gmail dot com> ---
One of the weird probably SEH-related things is that the lack-of-alignment
behavior of comment 28 and attachment 1 is not reproduced on a "normal" Linux
GCC with __attribute__((ms_abi)) sprinkled all over to get the right calling
convention. The code takes the same shape, uses mostly the same registers, but
the `and rsp, -32` is just either not there or placed wrong.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[dimula73 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Dmitry Kazakov <dimula73 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dimula73 at gmail dot com

--- Comment #31 from Dmitry Kazakov <dimula73 at gmail dot com> ---
Hi, all!

Just wanted to note that the bug is still present in GCC 10.3.0 on Windows
(from MSYS-MinGW64 packages).

> gcc (Rev5, Built by MSYS2 project) 10.3.0

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[mehdi.chinoune at hotmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Chinoune <mehdi.chinoune at hotmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mehdi.chinoune at hotmail dot com

--- Comment #32 from Chinoune <mehdi.chinoune at hotmail dot com> ---
Still present in GCC 11.2.0

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[lists at coryfields dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Cory Fields <lists at coryfields dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lists at coryfields dot com

--- Comment #33 from Cory Fields <lists at coryfields dot com> ---
Adding another +1. Still present in 10.3.0.

Bitcoin Core's sha2 code uses avx2 when possible. We ran into this bug when
bumping our toolchain:
https://github.com/bitcoin/bitcoin/pull/24736

and opted to take Debian's patch:
https://salsa.debian.org/mingw-w64-team/gcc-mingw-w64/-/blob/master/debian/patches/vmov-alignment.patch

It's unfortunate that the best and most common advice for using avx2 with
gcc/mingw is to use a patched compiler. Might it be possible to accept Debian's
patch upstream?

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #34 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
> It's unfortunate that the best and most common advice for using avx2 with
> gcc/mingw is to use a patched compiler. Might it be possible to accept
> Debian's patch upstream?

Sure, but they need to submit it first, we cannot do it for them.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[steve at sk2 dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Stephen Kitt <steve at sk2 dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |steve at sk2 dot org

--- Comment #35 from Stephen Kitt <steve at sk2 dot org> ---
Created attachment 52737
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52737&action=edit
Use unaligned VMOV instructions (for Windows targets)

The reason I didn't submit the Debian patch is that it unconditionally replaces
V...{U,A} with V...U instructions. That's fine when we know the target needs
something like that, which is the case when we're building a Windows
cross-compiler; but I don't think it's suitable for general use as-is. It would
need a build-time conditional at the very least.

Anyway, I'll add it as an attachment here; I'll try to find time to make it
generally applicable. I haven't filed copyright assignment paperwork for me
personally; if the patch needs it, consider it submitted by skitt@redhat.com
under the corporate copyright assignment agreement.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #36 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
That patch is certainly unacceptable, not only because it affects non-Windows
too, but even on Windows it will unnecessarily pessimize e.g. accesses to data
sections or heap that can be aligned.
If the Windows ABI doesn't align stack or not as much as gcc assumes, then a
fix would ensure only automatic vars on Windows are accessed always using
unaligned vector instructions provided dynamic stack realignment is not an
option.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #37 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
> If the Windows ABI doesn't align stack or not as much as gcc assumes, then a
> fix would ensure only automatic vars on Windows are accessed always using
> unaligned vector instructions provided dynamic stack realignment is not an
> option.

It's classical double-word alignment, i.e. 16 bytes, and AVX requires 32 bytes.
The implementation of dynamic stack realignment is too much of a kludge to be
safely used on Windows IMO so, yes, the way out is probably unaligned vector
instructions.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[rcopley at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #38 from R Copley <rcopley at gmail dot com> ---
(A patch to emit unaligned instructions should probably resolve bug 49001
instead of this one, 54412.)

Could dynamic alignment be achieved, not for automatic variables and function
parameters, but for registers spilled to the stack (due to register exhaustion,
or because they may be clobbered)? So that users can write code that stores
over-aligned objects on the heap only.

If SEH is the problem, can alignment be accounted for in cases where SEH is not
in use (if there are any such cases)? I'm thinking of -fno-exceptions, and
dwarf (on x86) or setjump/longjump exceptions.

Sorry if those are stupid questions.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #39 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
> If SEH is the problem, can alignment be accounted for in cases where SEH is
> not in use (if there are any such cases)? I'm thinking of -fno-exceptions,
> and dwarf (on x86) or setjump/longjump exceptions.

The hitch is that Setjmp/Longjmp is implemented on top of SEH on 64-bit
Windows, which means that SEH information must always be generated, even in
plain C.

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #40 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Eric Botcazou from comment #37)
> > If the Windows ABI doesn't align stack or not as much as gcc assumes, then a
> > fix would ensure only automatic vars on Windows are accessed always using
> > unaligned vector instructions provided dynamic stack realignment is not an
> > option.
> 
> It's classical double-word alignment, i.e. 16 bytes, and AVX requires 32
> bytes.
> The implementation of dynamic stack realignment is too much of a kludge to
> be safely used on Windows IMO so, yes, the way out is probably unaligned
> vector instructions.

Assembler in binutils 2.38 supports:

 -muse-unaligned-vector-move
                          encode aligned vector move as unaligned vector move

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|ebotcazou at gcc dot gnu.org       |unassigned at gcc dot gnu.org
             Status|ASSIGNED                    |NEW

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[avraham.adler at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #41 from Avraham Adler <avraham.adler at gmail dot com> ---
It has been a few years since the last comment. I recently got hit by this bug
for the first time in about a decade and a half of compiling R for Windows 64
using GCC 13.2.0 as packaged in Rtools44 [1].

Does it remain true that the only option to get around this bug without killing
all AVX2 is to pass "-Wa,-muse-unaligned-vector-move" when compiling using GCC
on Windows 64? Thank you.

[1] https://stat.ethz.ch/pipermail/r-sig-windows/2024q1/000113.html

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[dimula73 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #42 from Dmitry Kazakov <dimula73 at gmail dot com> ---
Hi, Avraham!

> Does it remain true that the only option to get around this bug without killing all AVX2 is to pass "-Wa,-muse-unaligned-vector-move" when compiling using GCC on Windows 64? Thank you

I'm not sure about your particular issue, but in our case we used to manage to
workaround this issue by passing AVX2-related structures by reference (or
const-reference, when possible).

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[avraham.adler at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #43 from Avraham Adler <avraham.adler at gmail dot com> ---
Thank you, Dmitry, but that particular solution may not be possible for me.
When I try compiling with -mstackrealign -mpreferred-stack-boundary=5
-mincoming-stack-boundary=5 instead of forcing unaligned moves I get "cc1.exe:
error: '-mpreferred-stack-boundary=5' is not between 3 and 4". Is that this bug
in a different form, something that should be filed separately, or known and
intended behavior?

[Bug target/54412] minimal 32-byte stack alignment with -mavx on 64-bit Windows

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

--- Comment #44 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
> Thank you, Dmitry, but that particular solution may not be possible for me.
> When I try compiling with -mstackrealign -mpreferred-stack-boundary=5
> -mincoming-stack-boundary=5 instead of forcing unaligned moves I get
> "cc1.exe: error: '-mpreferred-stack-boundary=5' is not between 3 and 4". Is
> that this bug in a different form, something that should be filed
> separately, or known and intended behavior?

No, it's the same issue: 32-byte stack alignment is not supported with SEH.