[Bug c/98161] New: [11 Regression] Incorrect stack realignment on __force_align_arg_pointer__+-mavx

[Bug c/98161] New: [11 Regression] Incorrect stack realignment on __force_align_arg_pointer__+-mavx

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98161

            Bug ID: 98161
           Summary: [11 Regression] Incorrect stack realignment on
                    __force_align_arg_pointer__+-mavx
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: slyfox at gcc dot gnu.org
  Target Milestone: ---

The bug was initially observed as a miscompilation of wine-5.22 built with
gcc-11 -march=sandybridge. gcc-10 seems to generate something that works.

Here is the extracted executable reproducer. It should return 12, but returns
56:

// $ cat bug.c.c
typedef unsigned short u16;
typedef unsigned int   u32;
typedef unsigned char  u8;

u32
    __attribute__((__force_align_arg_pointer__))
unreach(
    const u16 * pu16,
    u16 *dst, u32 dstlen,
    const u8 *src, u32 srclen
  )
{
    for (u32 i = dstlen; srclen && i; i--, srclen--, src++, dst++)
    {
        u16 off = pu16[*src];
        if (off)
        {
            src++; srclen--;
            *dst = pu16[off + *src];
        }
    }
    return 56;
}

u32
    __attribute__((__force_align_arg_pointer__))
    __attribute__((noipa))
bug(
    const u16 * pu16,
    u16 *dst, u32 dstlen,
    const u8 *src, u32 srclen
  )
{
    if (pu16)
       /* Branch should not execute, but stack realignment
        * reads wrong 'pu16' value from stack. */
        return unreach(pu16, dst, dstlen, src, srclen);

    return (srclen < dstlen) ? srclen : dstlen;
}

int main() {
    /* Should return 12 */
    return bug(0, 0, 12, 0, 34);
}


Running the example:
$ x86_64-pc-linux-gnu-gcc -m32 -fno-PIC -fno-builtin -pipe -fcf-protection=none
-fno-stack-protector -fno-omit-frame-pointer -O1 -mavx -o bug bug.c.c; ./bug ;
echo $?
12

$ x86_64-pc-linux-gnu-gcc -m32 -fno-PIC -fno-builtin -pipe -fcf-protection=none
-fno-stack-protector -fno-omit-frame-pointer -O2 -mavx -o bug bug.c.c; ./bug ;
echo $?
56

Looking at generated code %ebp and %ecx are confused as a pointer to arguments
on stack:

bug:
        leal    4(%esp), %ecx ; argument pointer
        andl    $-16, %esp    ; %esp is realigned
        pushl   -4(%ecx)
        pushl   %ebp
        movl    %esp, %ebp    ; %ebp points to realigned location
        pushl   %ebx
        vmovd   16(%ebp), %xmm1 ; arg3(pu16) BUG: arguments are read
                                ;                 related to %ebp, not %ecx
        vmovd   24(%ebp), %xmm2 ; arg5(dstlen)
        movl    8(%ebp), %edx   ; arg1(srclen)
        pushl   %ecx
        vpminud %xmm2, %xmm1, %xmm0
        movl    12(%ebp), %ecx
        movl    20(%ebp), %ebx
        vmovd   %xmm0, %eax
        testl   %edx, %edx
        je      .L21
        subl    $4, %esp
        vmovd   %xmm2, (%esp)
        pushl   %ebx
        subl    $4, %esp
        vmovd   %xmm1, (%esp)
        pushl   %ecx
        pushl   %edx
        call    unreach
        addl    $20, %esp
.L21:
        leal    -8(%ebp), %esp
        popl    %ecx
        popl    %ebx
        popl    %ebp
        leal    -4(%ecx), %esp
        ret

[Bug c/98161] [11 Regression] Incorrect stack realignment on __force_align_arg_pointer__ with -msse4 by r11-446

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98161

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[11 Regression] Incorrect   |[11 Regression] Incorrect
                   |stack realignment on        |stack realignment on
                   |__force_align_arg_pointer__ |__force_align_arg_pointer__
                   |+-mavx                      |with -msse4 by r11-446
   Target Milestone|---                         |11.0
                 CC|                            |crazylht at gmail dot com,
                   |                            |hjl.tools at gmail dot com,
                   |                            |wwwhhhyyy333 at gmail dot com
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2020-12-06
     Ever confirmed|0                           |1

--- Comment #1 from H.J. Lu <hjl.tools at gmail dot com> ---
It is triggered by r11-446. -m32 -msse2 -O2 is OK.  But -m32 -msse4 -O2 is
not.  Hongyu, can you take a look?

[Bug target/98161] [11 Regression] Incorrect stack realignment on __force_align_arg_pointer__ with -msse4 by r11-446

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98161

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch
          Component|c                           |target
                URL|                            |https://gcc.gnu.org/piperma
                   |                            |il/gcc-patches/2020-Decembe
                   |                            |r/561224.html

--- Comment #2 from H.J. Lu <hjl.tools at gmail dot com> ---
A patch is posted at

https://gcc.gnu.org/pipermail/gcc-patches/2020-December/561224.html

[Bug target/98161] [11 Regression] Incorrect stack realignment on __force_align_arg_pointer__ with -msse4 by r11-446

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98161

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by H.J. Lu <hjl@gcc.gnu.org>:

https://gcc.gnu.org/g:6643ca0be6f34786b686415e457de96d0d9fbd2d

commit r11-5806-g6643ca0be6f34786b686415e457de96d0d9fbd2d
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Sun Dec 6 10:43:16 2020 -0800

    x86: Check mode of pseudo register push

    commit 266f44a91c0c9705d3d18e82d7c5bab32927a18f
    Author: H.J. Lu <hjl.tools@gmail.com>
    Date:   Sun May 17 10:10:34 2020 -0700

        x86: Allow V1TI vector register pushes

        Add V1TI vector register push and split it after reload to a sequence
        of:

        (set (reg:P SP_REG) (plus:P SP_REG) (const_int -8)))
        (set (match_dup 0) (match_dup 1))

    added a pseudo register push check.  But

    (insn 13 12 14 3 (set (mem:SI (pre_dec:SI (reg/f:SI 7 sp)) [0  S4 A32])
            (reg/v:SI 87 [ srclen ])) "x.c":37:16 54 {*pushsi2}
         (expr_list:REG_DEAD (reg/v:SI 87 [ srclen ])
            (expr_list:REG_ARGS_SIZE (const_int 4 [0x4])
                (nil))))

    is not a pseudo register push.  In 64-bit mode, mode of pseudo register
    push is TImode.  In 32-bit mode, it is DImode.  Add pseudo register push
    mode check to pseudo_reg_set.

    gcc/

            PR target/98161
            * config/i386/i386-features.c (pseudo_reg_set): Check mode of
            pseudo register push.

    gcc/testsuite/

            * gcc.target/i386/pr98161.c: New test.

[Bug target/98161] [11 Regression] Incorrect stack realignment on __force_align_arg_pointer__ with -msse4 by r11-446

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98161

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #4 from H.J. Lu <hjl.tools at gmail dot com> ---
Fixed for GCC 11.