[Bug target/110386] New: [14 Regression] ICE when optimizing VPABSD/VPMULLD since 07b86ab138bf8be8cb331015cd2b9775c6856ac6

[Bug target/110386] New: [14 Regression] ICE when optimizing VPABSD/VPMULLD since 07b86ab138bf8be8cb331015cd2b9775c6856ac6

[benjsith at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

            Bug ID: 110386
           Summary: [14 Regression] ICE when optimizing VPABSD/VPMULLD
                    since 07b86ab138bf8be8cb331015cd2b9775c6856ac6
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: benjsith at gmail dot com
  Target Milestone: ---

Created attachment 55394
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55394&action=edit
A preprocessed minimal repro of the VPABSD/VPMULLD issue

The following code is a minimal repro of the issue when compiled using `gcc
-mavx -O1`:

#include <immintrin.h>

__m128i do_stuff(__m128i XMM0) {
        __m128i ABS0 = _mm_abs_epi32(XMM0);
        __m128i MUL0 = _mm_mullo_epi32(ABS0, XMM0);
        __m128i MUL1 = _mm_mullo_epi32(MUL0, MUL0);
        return MUL1;
}

Godbolt link: https://godbolt.org/z/c6dK5jzco

The ICE/crash prints:

min_repro.c: In function ‘do_stuff’:
min_repro.c:3:9: error: type mismatch in binary expression
    3 | __m128i do_stuff(__m128i XMM0) {
      |         ^~~~~~~~
vector(4) unsigned int

vector(4) unsigned int

vector(4) int

_6 = _5 * _7;
during GIMPLE pass: backprop
min_repro.c:3:9: internal compiler error: verify_gimple failed
0x102003d verify_gimple_in_cfg(function*, bool, bool)
        .././../gcc/gcc/tree-cfg.cc:5646
0xebf8d0 execute_function_todo
        .././../gcc/gcc/passes.cc:2098
0xebfe3e execute_todo
        .././../gcc/gcc/passes.cc:2152

A bisect shows it started in 07b86ab138bf8be8cb331015cd2b9775c6856ac6, which is
related to a previous bug https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110108

I have confirmed that this still repros in the latest trunk,
6b724427aa1ec9b690e40f5206572f57e1adeda8

For triage/priority purposes: this issue was not from code I manually wrote,
but was found by a fuzzer meant to test SIMD codegen

[Bug target/110386] [14 Regression] ICE when optimizing VPABSD/VPMULLD since g:07b86ab138bf8be8cb331015cd2b9775c6856ac6

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
-O1 -mavx

[Bug tree-optimization/110386] [14 Regression] ICE when optimizing VPABSD/VPMULLD since g:07b86ab138bf8be8cb331015cd2b9775c6856ac6

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|target                      |tree-optimization
   Target Milestone|---                         |14.0
           Keywords|                            |ice-checking,
                   |                            |ice-on-valid-code
   Last reconfirmed|                            |2023-06-24
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
This is a bug in backprop rather than a target issue.

We start with:
  _7 = VIEW_CONVERT_EXPR<vector(4) intD.6>(XMM0_2(D));
  _8 = ABSU_EXPR <_7>;
  _5 = VIEW_CONVERT_EXPR<vector(4) unsigned intD.9>(XMM0_2(D));
  _6 = _5 * _8;
  _3 = _6 * _6;
  _4 = VIEW_CONVERT_EXPR<__m128iD.5706>(_3);


And backprop changes it into:
  _7 = VIEW_CONVERT_EXPR<vector(4) intD.6>(XMM0_2(D));
  _8 = ABSU_EXPR <_7>;
  _5 = VIEW_CONVERT_EXPR<vector(4) unsigned intD.9>(XMM0_2(D));
  _6 = _5 * _7;
  _3 = _6 * _6;
  _4 = VIEW_CONVERT_EXPR<__m128iD.5706>(_3);

But misses the conversion

[Bug tree-optimization/110386] [12/13/14 Regression] ICE with ABSU in backprop

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[14 Regression] ICE when    |[12/13/14 Regression] ICE
                   |optimizing VPABSD/VPMULLD   |with ABSU in backprop
                   |since                       |
                   |g:07b86ab138bf8be8cb331015c |
                   |d2b9775c6856ac6             |
      Known to fail|                            |12.1.0
   Target Milestone|14.0                        |12.4

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Here is a generic non-vector based testcase which shows the same issue with
backprop:
```
int f(int a)
{
        unsigned b = a;
        b =  ((unsigned)a)*(-(a<0)|1);
        unsigned t = b*a;
        return t*t;
}
```

[Bug tree-optimization/110386] [10/11/12/13/14 Regression] ICE with ABSU in backprop

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to fail|                            |10.1.0, 11.4.0, 9.5.0
   Target Milestone|12.4                        |10.5
      Known to work|                            |9.1.0
            Summary|[12/13/14 Regression] ICE   |[10/11/12/13/14 Regression]
                   |with ABSU in backprop       |ICE with ABSU in backprop

--- Comment #4 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
here is one that is reproduced even in GCC 10.1.0:
```
int f(int a)
{
        int c = c < 0 ? c : -c;
        c = -c;
        unsigned b =  c;
        unsigned t = b*a;
        return t*t;
}
```
I suspect it was introduced when ABSU_EXPR was created.

[Bug tree-optimization/110386] [10/11/12/13/14 Regression] ICE with ABSU in backprop

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P2

[Bug tree-optimization/110386] [11/12/13/14 Regression] ICE with ABSU in backprop

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|10.5                        |11.5

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 10 branch is being closed.

[Bug tree-optimization/110386] [11/12/13/14 Regression] ICE with ABSU in backprop

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |pinskia at gcc dot gnu.org
             Status|NEW                         |ASSIGNED
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=111285

--- Comment #6 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I am going to fix this.

[Bug tree-optimization/110386] [11/12/13/14 Regression] ICE with ABSU in backprop

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch
                URL|                            |https://gcc.gnu.org/piperma
                   |                            |il/gcc-patches/2023-Septemb
                   |                            |er/631257.html

--- Comment #7 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Patch posted:
https://gcc.gnu.org/pipermail/gcc-patches/2023-September/631257.html

[Bug tree-optimization/110386] [11/12/13/14 Regression] ICE with ABSU in backprop

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The trunk branch has been updated by Andrew Pinski <pinskia@gcc.gnu.org>:

https://gcc.gnu.org/g:2bbac12ea7bd8a3eef5382e1b13f6019df4ec03f

commit r14-4249-g2bbac12ea7bd8a3eef5382e1b13f6019df4ec03f
Author: Andrew Pinski <apinski@marvell.com>
Date:   Sat Sep 23 21:53:09 2023 -0700

    Fix PR 110386: backprop vs ABSU_EXPR

    The issue here is that when backprop tries to go
    and strip sign ops, it skips over ABSU_EXPR but
    ABSU_EXPR not only does an ABS, it also changes the
    type to unsigned.
    Since strip_sign_op_1 is only supposed to strip off
    sign changing operands and not ones that change types,
    removing ABSU_EXPR here is correct. We don't handle
    nop conversions so this does cause any missed optimizations either.

    OK? Bootstrapped and tested on x86_64-linux-gnu with no regressions.

            PR tree-optimization/110386

    gcc/ChangeLog:

            * gimple-ssa-backprop.cc (strip_sign_op_1): Remove ABSU_EXPR.

    gcc/testsuite/ChangeLog:

            * gcc.c-torture/compile/pr110386-1.c: New test.
            * gcc.c-torture/compile/pr110386-2.c: New test.

[Bug tree-optimization/110386] [11/12/13 Regression] ICE with ABSU in backprop

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[11/12/13/14 Regression]    |[11/12/13 Regression] ICE
                   |ICE with ABSU in backprop   |with ABSU in backprop
      Known to work|                            |14.0

--- Comment #9 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Fixed on the trunk so far.

[Bug tree-optimization/110386] [11/12/13 Regression] ICE with ABSU in backprop

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Andrew Pinski
<pinskia@gcc.gnu.org>:

https://gcc.gnu.org/g:b5fa95a3569f6ee66697876a3a380fef1b333f3d

commit r13-7927-gb5fa95a3569f6ee66697876a3a380fef1b333f3d
Author: Andrew Pinski <apinski@marvell.com>
Date:   Sat Sep 23 21:53:09 2023 -0700

    Fix PR 110386: backprop vs ABSU_EXPR

    The issue here is that when backprop tries to go
    and strip sign ops, it skips over ABSU_EXPR but
    ABSU_EXPR not only does an ABS, it also changes the
    type to unsigned.
    Since strip_sign_op_1 is only supposed to strip off
    sign changing operands and not ones that change types,
    removing ABSU_EXPR here is correct. We don't handle
    nop conversions so this does cause any missed optimizations either.

    Committed to the GCC 13 branch after bootstrapped and
    tested on x86_64-linux-gnu with no regressions.

            PR tree-optimization/110386

    gcc/ChangeLog:

            * gimple-ssa-backprop.cc (strip_sign_op_1): Remove ABSU_EXPR.

    gcc/testsuite/ChangeLog:

            * gcc.c-torture/compile/pr110386-1.c: New test.
            * gcc.c-torture/compile/pr110386-2.c: New test.

    (cherry picked from commit 2bbac12ea7bd8a3eef5382e1b13f6019df4ec03f)

[Bug tree-optimization/110386] [11/12 Regression] ICE with ABSU in backprop

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

--- Comment #11 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Andrew Pinski
<pinskia@gcc.gnu.org>:

https://gcc.gnu.org/g:f5c7306d7f039e5c74c5e82cf06610f0ae07a0e8

commit r12-10434-gf5c7306d7f039e5c74c5e82cf06610f0ae07a0e8
Author: Andrew Pinski <apinski@marvell.com>
Date:   Sat Sep 23 21:53:09 2023 -0700

    Fix PR 110386: backprop vs ABSU_EXPR

    The issue here is that when backprop tries to go
    and strip sign ops, it skips over ABSU_EXPR but
    ABSU_EXPR not only does an ABS, it also changes the
    type to unsigned.
    Since strip_sign_op_1 is only supposed to strip off
    sign changing operands and not ones that change types,
    removing ABSU_EXPR here is correct. We don't handle
    nop conversions so this does cause any missed optimizations either.

    OK? Bootstrapped and tested on x86_64-linux-gnu with no regressions.

            PR tree-optimization/110386

    gcc/ChangeLog:

            * gimple-ssa-backprop.cc (strip_sign_op_1): Remove ABSU_EXPR.

    gcc/testsuite/ChangeLog:

            * gcc.c-torture/compile/pr110386-1.c: New test.
            * gcc.c-torture/compile/pr110386-2.c: New test.

    (cherry picked from commit 2bbac12ea7bd8a3eef5382e1b13f6019df4ec03f)

[Bug tree-optimization/110386] [11/12 Regression] ICE with ABSU in backprop

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

--- Comment #12 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Andrew Pinski
<pinskia@gcc.gnu.org>:

https://gcc.gnu.org/g:dbfc2d075f10149bd94e16c1210ffe4bac7e60c3

commit r11-11422-gdbfc2d075f10149bd94e16c1210ffe4bac7e60c3
Author: Andrew Pinski <apinski@marvell.com>
Date:   Sat Sep 23 21:53:09 2023 -0700

    Fix PR 110386: backprop vs ABSU_EXPR

    The issue here is that when backprop tries to go
    and strip sign ops, it skips over ABSU_EXPR but
    ABSU_EXPR not only does an ABS, it also changes the
    type to unsigned.
    Since strip_sign_op_1 is only supposed to strip off
    sign changing operands and not ones that change types,
    removing ABSU_EXPR here is correct. We don't handle
    nop conversions so this does cause any missed optimizations either.

    OK? Bootstrapped and tested on x86_64-linux-gnu with no regressions.

            PR tree-optimization/110386

    gcc/ChangeLog:

            * gimple-ssa-backprop.c (strip_sign_op_1): Remove ABSU_EXPR.

    gcc/testsuite/ChangeLog:

            * gcc.c-torture/compile/pr110386-1.c: New test.
            * gcc.c-torture/compile/pr110386-2.c: New test.

    (cherry picked from commit 2bbac12ea7bd8a3eef5382e1b13f6019df4ec03f)

[Bug tree-optimization/110386] [11/12 Regression] ICE with ABSU in backprop

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110386

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
      Known to work|                            |11.4.1, 12.3.1
         Resolution|---                         |FIXED
      Known to fail|                            |12.3.0

--- Comment #13 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Fixed everywhere.