[Bug target/112677] New: ASAN reports stack-buffer-overflow in tree-vect-loop.cc vect_is_simple_use when compiling with -mavx512

[Bug target/112677] New: ASAN reports stack-buffer-overflow in tree-vect-loop.cc vect_is_simple_use when compiling with -mavx512

[fkastl at suse dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112677

            Bug ID: 112677
           Summary: ASAN reports stack-buffer-overflow in
                    tree-vect-loop.cc vect_is_simple_use when compiling
                    with -mavx512
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Keywords: needs-bisection
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: fkastl at suse dot cz
  Target Milestone: ---
              Host: x86_64-linux
            Target: x86_64-linux

Created attachment 56670
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=56670&action=edit
A list of testcases triggering this error

On many testcases from the GCC testsuite an ASAN-instrumented GCC reports
stack-buffer-overflow error in vect_is_simple_use at tree-vect-loop.cc:13584
file. All of the errors happen when compiling with some kind of -mavx512 option
or with -march=skylake-avx512.

I'm attaching a list of testcases that trigger this error.

Compiler configured with:

--enable-languages=default,jit,lto,go,d --enable-host-shared
--enable-checking=release --disable-multilib --with-build-config=bootstrap-asan


One example of a testcase where this error occurs is g++.dg/opt/pr112374.C.
Running

gcc src/gcc/testsuite/g++.dg/opt/pr112374.C  -O2 -march=skylake-avx512

results in

==46365==ERROR: AddressSanitizer: stack-buffer-overflow on address
0x7f41ef71c6f8 at pc 0x00000562f3ab bp 0x7ffee76484d0 sp 0x7ffee76484c8
WRITE of size 8 at 0x7f41ef71c6f8 thread T0
    #0 0x562f3aa in vect_is_simple_use(tree_node*, vec_info*, vect_def_type*,
tree_node**, _stmt_vec_info**, gimple**)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-stmts.cc:13584
    #1 0x2c708ad in vectorizable_reduction(_loop_vec_info*, _stmt_vec_info*,
_slp_tree*, _slp_instance*, vec<stmt_info_for_cost, va_heap, vl_ptr>*)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-loop.cc:7632
    #2 0x2c971b5 in vect_analyze_loop_operations
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-loop.cc:2149
    #3 0x2c971b5 in vect_analyze_loop_2
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-loop.cc:3011
    #4 0x2c9dc43 in vect_analyze_loop_1
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-loop.cc:3450
    #5 0x2ca037e in vect_analyze_loop(loop*, vec_info_shared*)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-loop.cc:3604
    #6 0x2d9f495 in try_vectorize_loop_1
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vectorizer.cc:1066
    #7 0x2da0cd9 in execute
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vectorizer.cc:1298
    #8 0x1f4a262 in execute_one_pass(opt_pass*)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/passes.cc:2641
    #9 0x1f4bb8c in execute_pass_list_1
/home/worker/buildworker/tiber-gcc-asan/build/gcc/passes.cc:2750
    #10 0x1f4bbb2 in execute_pass_list_1
/home/worker/buildworker/tiber-gcc-asan/build/gcc/passes.cc:2751
    #11 0x1f4bbb2 in execute_pass_list_1
/home/worker/buildworker/tiber-gcc-asan/build/gcc/passes.cc:2751
    #12 0x1f4bc25 in execute_pass_list(function*, opt_pass*)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/passes.cc:2761
    #13 0x130a814 in cgraph_node::expand()
/home/worker/buildworker/tiber-gcc-asan/build/gcc/cgraphunit.cc:1841
    #14 0x130a814 in cgraph_node::expand()
/home/worker/buildworker/tiber-gcc-asan/build/gcc/cgraphunit.cc:1794
    #15 0x131004d in expand_all_functions
/home/worker/buildworker/tiber-gcc-asan/build/gcc/cgraphunit.cc:2024
    #16 0x131004d in symbol_table::compile()
/home/worker/buildworker/tiber-gcc-asan/build/gcc/cgraphunit.cc:2398
    #17 0x131004d in symbol_table::compile()
/home/worker/buildworker/tiber-gcc-asan/build/gcc/cgraphunit.cc:2309
    #18 0x1316999 in symbol_table::finalize_compilation_unit()
/home/worker/buildworker/tiber-gcc-asan/build/gcc/cgraphunit.cc:2583
    #19 0x23492cf in compile_file
/home/worker/buildworker/tiber-gcc-asan/build/gcc/toplev.cc:473
    #20 0x7e26dd in do_compile
/home/worker/buildworker/tiber-gcc-asan/build/gcc/toplev.cc:2129
    #21 0x7e26dd in toplev::main(int, char**)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/toplev.cc:2285
    #22 0x7ed873 in main
/home/worker/buildworker/tiber-gcc-asan/build/gcc/main.cc:39
    #23 0x7f41f10281af in __libc_start_call_main (/lib64/libc.so.6+0x281af)
(BuildId: bbeee08e5f56966e641c4f3ba4ea1da9d730d0ab)
    #24 0x7f41f1028278 in __libc_start_main@@GLIBC_2.34
(/lib64/libc.so.6+0x28278) (BuildId: bbeee08e5f56966e641c4f3ba4ea1da9d730d0ab)
    #25 0x7ef1d4 in _start ../sysdeps/x86_64/start.S:115

Address 0x7f41ef71c6f8 is located in stack of thread T0 at offset 1784 in frame
    #0 0x2c6e69f in vectorizable_reduction(_loop_vec_info*, _stmt_vec_info*,
_slp_tree*, _slp_instance*, vec<stmt_info_for_cost, va_heap, vl_ptr>*)
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-loop.cc:7385

  This frame has 145 object(s):
    [48, 50) '<unknown>'
    [64, 66) '<unknown>'
    [80, 84) 'dt' (line 7631)
    [96, 100) '<unknown>'
    [112, 116) '<unknown>'
    [128, 132) '<unknown>'
    [144, 148) 'cond_initial_dt' (line 7770)
    [160, 164) '<unknown>'
    [176, 180) 'orig_code' (line 7848)
    [192, 196) '<unknown>'
    [208, 212) 'reduc_fn' (line 7911)
    [224, 228) '<unknown>'
    [240, 244) '<unknown>'
    [256, 260) '<unknown>'
    [272, 276) '<unknown>'
    [288, 292) '<unknown>'
    [304, 308) '<unknown>'
    [320, 324) '<unknown>'
    [336, 340) '<unknown>'
    [352, 356) '<unknown>'
    [368, 372) '<unknown>'
    [384, 388) '<unknown>'
    [400, 404) '<unknown>'
    [416, 420) '<unknown>'
    [432, 436) '<unknown>'
    [448, 452) '<unknown>'
    [464, 468) 'overflow' (line 7238)
    [480, 484) '<unknown>'
    [496, 500) '<unknown>'
    [512, 520) 'use_p' (line 7456)
    [544, 552) 'use_stmt' (line 7457)
    [576, 584) 'def_stmt_info' (line 7630)
    [608, 616) '<unknown>'
    [640, 648) '<unknown>'
    [672, 680) 'nunits_out' (line 7804)
    [704, 712) '<unknown>'
    [736, 744) 'r'
    [768, 776) '<unknown>'
    [800, 808) '<unknown>'
    [832, 840) '<unknown>'
    [864, 872) '<unknown>'
    [896, 904) '<unknown>'
    [928, 936) '<unknown>'
    [960, 968) '<unknown>'
    [992, 1000) '<unknown>'
    [1024, 1032) '<unknown>'
    [1056, 1064) '<unknown>'
    [1088, 1096) '<unknown>'
    [1120, 1128) '<unknown>'
    [1152, 1160) '<unknown>'
    [1184, 1192) '<unknown>'
    [1216, 1224) '<unknown>'
    [1248, 1256) '<unknown>'
    [1280, 1288) '<unknown>'
    [1312, 1320) '<unknown>'
    [1344, 1352) '<unknown>'
    [1376, 1384) '<unknown>'
    [1408, 1416) '<unknown>'
    [1440, 1448) '<unknown>'
    [1472, 1480) '<unknown>'
    [1504, 1520) '<unknown>'
    [1536, 1552) '<unknown>'
    [1568, 1584) '<unknown>'
    [1600, 1616) '<unknown>'
    [1632, 1648) '<unknown>'
    [1664, 1680) '<unknown>'
    [1696, 1712) '<unknown>'
    [1728, 1744) '<unknown>'
    [1760, 1784) 'vectype_op' (line 7387) <== Memory access at offset 1784
overflows this variable
    [1824, 1848) '<unknown>'
    [1888, 1912) '<unknown>'
    [1952, 1976) '<unknown>'
    [2016, 2040) '<unknown>'
    [2080, 2104) '<unknown>'
    [2144, 2168) '<unknown>'
    [2208, 2232) '<unknown>'
    [2272, 2296) '<unknown>'
    [2336, 2360) '<unknown>'
    [2400, 2424) '<unknown>'
    [2464, 2488) '<unknown>'
    [2528, 2552) '<unknown>'
    [2592, 2616) '<unknown>'
    [2656, 2680) '<unknown>'
    [2720, 2744) '<unknown>'
    [2784, 2808) '<unknown>'
    [2848, 2872) '<unknown>'
    [2912, 2936) '<unknown>'
    [2976, 3000) '<unknown>'
    [3040, 3064) '<unknown>'
    [3104, 3128) '<unknown>'
    [3168, 3192) '<unknown>'
    [3232, 3256) '<unknown>'
    [3296, 3320) '<unknown>'
    [3360, 3384) '<unknown>'
    [3424, 3448) '<unknown>'
    [3488, 3512) '<unknown>'
    [3552, 3576) '<unknown>'
    [3616, 3640) '<unknown>'
    [3680, 3712) '<unknown>'
    [3744, 3776) '<unknown>'
    [3808, 3840) '<unknown>'
    [3872, 3904) '<unknown>'
    [3936, 3968) '<unknown>'
    [4000, 4032) '<unknown>'
    [4064, 4096) '<unknown>'
    [4128, 4160) '<unknown>'
    [4192, 4224) '<unknown>'
    [4256, 4288) '<unknown>'
    [4320, 4352) '<unknown>'
    [4384, 4416) '<unknown>'
    [4448, 4480) '<unknown>'
    [4512, 4544) '<unknown>'
    [4576, 4608) '<unknown>'
    [4640, 4672) '<unknown>'
    [4704, 4736) '<unknown>'
    [4768, 4800) '<unknown>'
    [4832, 4864) '<unknown>'
    [4896, 4928) '<unknown>'
    [4960, 4992) '<unknown>'
    [5024, 5056) '<unknown>'
    [5088, 5120) '<unknown>'
    [5152, 5184) '<unknown>'
    [5216, 5248) '<unknown>'
    [5280, 5312) '<unknown>'
    [5344, 5376) '<unknown>'
    [5408, 5440) '<unknown>'
    [5472, 5504) 'xi'
    [5536, 5568) 'yi'
    [5600, 5632) 'xi'
    [5664, 5696) 'yi'
    [5728, 5760) '<unknown>'
    [5792, 5824) '<unknown>'
    [5856, 5888) 'xi'
    [5920, 5952) 'yi'
    [5984, 6016) '<unknown>'
    [6048, 6128) 'ni' (line 8071)
    [6160, 6240) 'ni' (line 7237)
    [6272, 6352) 'max_loop_value' (line 7237)
    [6384, 6464) 'lhs_max' (line 7237)
    [6496, 6576) '<unknown>'
    [6608, 6688) '<unknown>'
    [6720, 6800) '<unknown>'
    [6832, 6944) 'op' (line 7507)
    [6976, 7088) 'op' (line 7587)
    [7120, 7232) 'op' (line 5268)
HINT: this may be a false positive if your program uses some custom stack
unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow
/home/worker/buildworker/tiber-gcc-asan/build/gcc/tree-vect-stmts.cc:13584 in
vect_is_simple_use(tree_node*, vec_info*, vect_def_type*, tree_node**,
_stmt_vec_info**, gimple**)
Shadow bytes around the buggy address:
  0x7f41ef71c400: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
  0x7f41ef71c480: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
  0x7f41ef71c500: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
  0x7f41ef71c580: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 00 f2 f2
  0x7f41ef71c600: 00 00 f2 f2 00 00 f2 f2 00 00 f2 f2 00 00 f2 f2
=>0x7f41ef71c680: 00 00 f2 f2 00 00 f2 f2 00 00 f2 f2 00 00 00[f2]
  0x7f41ef71c700: f2 f2 f2 f2 00 00 00 f2 f2 f2 f2 f2 00 00 00 f2
  0x7f41ef71c780: f2 f2 f2 f2 00 00 00 f2 f2 f2 f2 f2 00 00 00 f2
  0x7f41ef71c800: f2 f2 f2 f2 00 00 00 f2 f2 f2 f2 f2 00 00 00 f2
  0x7f41ef71c880: f2 f2 f2 f2 00 00 00 f2 f2 f2 f2 f2 00 00 00 f2
  0x7f41ef71c900: f2 f2 f2 f2 00 00 00 f2 f2 f2 f2 f2 00 00 00 f2
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==46365==ABORTING

=================================================================

[Bug tree-optimization/112677] [14 Regression] ASAN reports stack-buffer-overflow in tree-vect-loop.cc vect_is_simple_use when compiling with -mavx512

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112677

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |14.0
           Keywords|needs-bisection             |ice-on-valid-code
            Summary|ASAN reports                |[14 Regression] ASAN
                   |stack-buffer-overflow in    |reports
                   |tree-vect-loop.cc           |stack-buffer-overflow in
                   |vect_is_simple_use when     |tree-vect-loop.cc
                   |compiling with -mavx512     |vect_is_simple_use when
                   |                            |compiling with -mavx512
                 CC|                            |rdapp at gcc dot gnu.org,
                   |                            |rguenth at gcc dot gnu.org
   Last reconfirmed|                            |2023-11-23
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Confirmed.
We are processing:
  _ifc__35 = .COND_IOR (_23, mask_25, _18, mask_25);

Which has 4 operations but vectype_op is only declared for 3:
```
  tree vectype_op[3] = { NULL_TREE, NULL_TREE, NULL_TREE };
```

And it is accessed by:
```
  for (i = 0; i < (int) op.num_ops; i++)
    {
      /* The condition of COND_EXPR is checked in vectorizable_condition().  */
      if (i == 0 && op.code == COND_EXPR)
        continue;

      stmt_vec_info def_stmt_info;
      enum vect_def_type dt;
      if (!vect_is_simple_use (loop_vinfo, stmt_info, slp_for_stmt_info,
                               i + opno_adjust, &op.ops[i], &slp_op[i], &dt,
                               &vectype_op[i], &def_stmt_info))

```

We definitely should increase it to at least 4 but I am not sure if it needs to
increased more.

[Bug tree-optimization/112677] [14 Regression] ASAN reports stack-buffer-overflow in tree-vect-loop.cc vect_is_simple_use when compiling with -mavx512

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112677

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Mine.

[Bug tree-optimization/112677] [14 Regression] ASAN reports stack-buffer-overflow in tree-vect-loop.cc vect_is_simple_use when compiling with -mavx512

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112677

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:9f63a8898154473f7b773c3e2ed71e4959719b71

commit r14-5817-g9f63a8898154473f7b773c3e2ed71e4959719b71
Author: Richard Biener <rguenther@suse.de>
Date:   Fri Nov 24 10:04:15 2023 +0100

    tree-optimization/112677 - stack corruption with .COND_* reduction

    The following makes sure to allocate enough space for vectype_op
    in vectorizable_reduction.

            PR tree-optimization/112677
            * tree-vect-loop.cc (vectorizable_reduction): Use alloca
            to allocate vectype_op.

[Bug tree-optimization/112677] [14 Regression] ASAN reports stack-buffer-overflow in tree-vect-loop.cc vect_is_simple_use when compiling with -mavx512

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112677

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed.