[Bug tree-optimization/105650] New: [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[Bug tree-optimization/105650] New: [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

            Bug ID: 105650
           Summary: [13 Regression] Possibly wrong code on fontforge
                    -fvect-cost-model=unlimited
           Product: gcc
           Version: 13.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: slyfox at gcc dot gnu.org
  Target Milestone: ---

Initially encountered wrong code on fontforge-2022-03-08 where it failed to
generate djvu fonts and crashed. I spent some time extracting example and I
hope I got it without without breaking it too much:

// $ cat a.c
// Extracted from fontforge-2022-03-08:
//   $ gcc-13/bin/gcc -O2 a.c -o a && ./a
//   ok
//   $ gcc-13/bin/gcc -O2 -fvect-cost-model=unlimited a.c -o a && ./a
//   SIGSEGV
#include <stdlib.h>

typedef struct SplineChar2 {
    int * p1;
    int * p2;
} SplineChar2;

volatile SplineChar2 gp;

static void dumpgposkernclass(void) __attribute__((noipa));
static void dumpgposkernclass(void) {
    volatile SplineChar2 * a[6] = {&gp, &gp, 0, 0, 0, &gp, };
    volatile int l = 6;

    volatile SplineChar2 **gs = a;
    int numGlyphs = l;

    int i, cnt;
    volatile int v;
    volatile SplineChar2 **glyphs__;

    // count non-null
    for ( i=cnt=0; i<numGlyphs; ++i )
        if ( gs[i]!=NULL ) ++cnt;

    size_t bl = (cnt+1)*sizeof(volatile SplineChar2 *);
    v = cnt+1;
    glyphs__ = malloc(bl);

    // store non-nulls
    for ( i=cnt=0; i<numGlyphs; ++i )
      if ( gs[i]!=NULL )
        glyphs__[cnt++] = gs[i];

    // null-terminate
    glyphs__[cnt++] = NULL;

    // corruption happens here
    if (v != cnt) __builtin_trap();
}

int main(void) {
    dumpgposkernclass();
}

Triggering:

$ gcc-13/bin/gcc -O2 a.c -o a && ./a
$ gcc-13/bin/gcc -O2 -fvect-cost-model=unlimited a.c -o a && ./a
Illegal instruction (core dumped)

It's a weekly gcc-13 from this week:

$ gcc-13/bin/gcc -v |& unnix
Using built-in specs.
COLLECT_GCC=/<<NIX>>/gcc-13.0.0/bin/gcc
COLLECT_LTO_WRAPPER=/<<NIX>>/gcc-13.0.0/libexec/gcc/x86_64-unknown-linux-gnu/13.0.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with:
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.0.0 20220515 (experimental) (GCC)

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[crazylht at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

Hongtao.liu <crazylht at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |crazylht at gmail dot com

--- Comment #1 from Hongtao.liu <crazylht at gmail dot com> ---
What's the platform are you running on, does it with SSE2?
GCC default arch is SSE2, and I didn't see any suspicious obove-sse2
instruction.

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |13.0
             Target|                            |x86_64-*-*
           Keywords|                            |needs-bisection

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
I can't reproduce with r13-565-g941efd87e22b1b

It might be a __builtin_unreachable () falling through to an illegal
instruction.

Maybe somebody can bisect to a failing revision.

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

--- Comment #3 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
(In reply to Hongtao.liu from comment #1)
> What's the platform are you running on, does it with SSE2?
> GCC default arch is SSE2, and I didn't see any suspicious obove-sse2
> instruction.

It's a znver3 CPU:

$ LANG=C lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  32
  On-line CPU(s) list:   0-15
  Off-line CPU(s) list:  16-31
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 9 5950X 16-Core Processor
    CPU family:          25
    Model:               33
    Thread(s) per core:  1
    Core(s) per socket:  16
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU max MHz:         5083.3979
    CPU min MHz:         0.0000
    BogoMIPS:            6787.31
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt
pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpu
                         id extd_apicid aperfmperf rapl pni pclmulqdq monitor
ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm
cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dn
                         owprefetch osvw ibs skinit wdt tce topoext
perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate
ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms i
                         nvpcid cqm rdt_a rdseed adx smap clflushopt clwb
sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total
cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lb
                         rv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid
decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl
umip pku ospke vaes vpclmulqdq rdpid overflow_recov succo
                         r smca fsrm
Virtualization features:
  Virtualization:        AMD-V
Caches (sum of all):
  L1d:                   512 KiB (16 instances)
  L1i:                   512 KiB (16 instances)
  L2:                    8 MiB (16 instances)
  L3:                    64 MiB (2 instances)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-15
Vulnerabilities:
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Spec store bypass:     Vulnerable
  Spectre v1:            Vulnerable: __user pointer sanitization and usercopy
barriers only; no swapgs barriers
  Spectre v2:            Vulnerable, IBPB: disabled, STIBP: disabled
  Srbds:                 Not affected
  Tsx async abort:       Not affected


I think the problem for me is value mismatch in compare of `if (v != cnt)
__builtin_trap();`. Invalid instruction is generated by `__builtin_trap()`.

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[crazylht at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

--- Comment #4 from Hongtao.liu <crazylht at gmail dot com> ---

> 
> I think the problem for me is value mismatch in compare of `if (v != cnt)
> __builtin_trap();`. Invalid instruction is generated by `__builtin_trap()`.

Oh, it's ud2.

But still can't reproduce the error with gcc version 13.0.0 20220518
(experimental) (GCC)

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

--- Comment #5 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #2)
> I can't reproduce with r13-565-g941efd87e22b1b
> 
> It might be a __builtin_unreachable () falling through to an illegal
> instruction.
> 
> Maybe somebody can bisect to a failing revision.

Aha, that's slightly fresher gcc than most recent weekly snapshot. r13-550 and
r13-552 both look like plausible fixes. I'll test both.

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

Sergei Trofimovich <slyfox at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at redhat dot com

--- Comment #6 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
r13-552 "i386: Fix up V2DI and V1TI inequality comparisons [PR105613]" did fix
it for me. Looks plausible? Or just a side-effect? I think you can revert it
locally to get the crash.

r13-550 has no effect on this crash.

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
(In reply to Sergei Trofimovich from comment #6)
> r13-552 "i386: Fix up V2DI and V1TI inequality comparisons [PR105613]" did
> fix it for me. Looks plausible? Or just a side-effect? I think you can
> revert it locally to get the crash.
> 
> r13-550 has no effect on this crash.

The code contains various comparisons of pointers in loops which might be
vectorized into V2DImode non-equality comparisons, so r13-552 is most likely
the fix you're looking for.  Especially if the problem doesn't exist before
r13-458.

[Bug tree-optimization/105650] [13 Regression] Possibly wrong code on fontforge -fvect-cost-model=unlimited

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105650

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> ---
So fixed.