[Bug tree-optimization/105532] New: UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[Bug tree-optimization/105532] New: UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

            Bug ID: 105532
           Summary: UBSAN: gcc/hwint.h:293:61: runtime error: shift
                    exponent 64 is too large for 64-bit type 'long
                    unsigned int'
           Product: gcc
           Version: 13.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu

Created attachment 52942
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52942&action=edit
reduced testcase

Compiler output:
$ x86_64-pc-linux-gnu-gcc -O testcase.c -c
testcase.c: In function 'foo':
testcase.c:7:15: warning: right shift count >= width of vector element
[-Wshift-count-overflow]
    7 |   return (~(u >> 255));
      |               ^~
/repo/gcc-trunk/gcc/hwint.h:293:61: runtime error: shift exponent 64 is too
large for 64-bit type 'long unsigned int'
/repo/gcc-trunk/gcc/hwint.h:293:72: runtime error: shift exponent 64 is too
large for 64-bit type 'long int'


(gdb) bt
#0  __ubsan::Diag::~Diag (this=this@entry=0x7fffffffbf80, __in_chrg=<optimized
out>) at /repo/gcc-trunk/libsanitizer/ubsan/ubsan_diag.cpp:343
#1  0x00000000074e3aa5 in handleShiftOutOfBoundsImpl
(Data=Data@entry=0x86de5c0, LHS=LHS@entry=18446744073709551615,
RHS=RHS@entry=64, Opts=...) at
/repo/gcc-trunk/libsanitizer/ubsan/ubsan_handlers.cpp:353
#2  0x00000000074e49bf in __ubsan::__ubsan_handle_shift_out_of_bounds
(Data=Data@entry=0x86de5c0, LHS=LHS@entry=18446744073709551615,
RHS=RHS@entry=64)
    at /repo/gcc-trunk/libsanitizer/ubsan/ubsan_handlers.cpp:370
#3  0x00000000006b796e in sext_hwi (prec=<optimized out>, src=<optimized out>)
at /repo/gcc-trunk/gcc/hwint.h:293
#4  wi::hwi_with_prec::hwi_with_prec (s=<optimized out>, p=<optimized out>,
v=<optimized out>, this=<optimized out>) at /repo/gcc-trunk/gcc/wide-int.h:1604
#5  wi::shwi (precision=<optimized out>, val=<optimized out>) at
/repo/gcc-trunk/gcc/wide-int.h:1613
#6  tree_nonzero_bits (t=t@entry=0x7ffff78334f8) at
/repo/gcc-trunk/gcc/fold-const.cc:16607
#7  0x000000000426322f in gimple_simplify_BIT_NOT_EXPR (res_op=0x7fffffffc550,
seq=0x0, valueize=0x2bbee60 <valueize_op_1(tree)>, type=0x7ffff784a000,
_p0=0x7ffff76f3d80, code=...) at gimple-match.cc:67410
#8  0x0000000003e599eb in gimple_simplify (res_op=res_op@entry=0x7fffffffc550,
seq=seq@entry=0x0, valueize=valueize@entry=0x2bbee60 <valueize_op_1(tree)>,
code=..., type=0x7ffff784a000, _p0=0x7ffff76f3d80)
    at gimple-match.cc:90609
#9  0x0000000003e5e779 in gimple_resimplify1 (seq=0x0, res_op=0x7fffffffc740,
valueize=0x2bbee60 <valueize_op_1(tree)>) at
/repo/gcc-trunk/gcc/gimple-match-head.cc:244
#10 0x0000000003f2590c in gimple_simplify (stmt=stmt@entry=0x7ffff7838410,
res_op=<optimized out>, res_op@entry=0x7fffffffc740, seq=<optimized out>,
seq@entry=0x0, valueize=<optimized out>, 
    valueize@entry=0x2bbee60 <valueize_op_1(tree)>, top_valueize=<optimized
out>, top_valueize@entry=0x2bbf480 <valueize_op(tree)>) at
/repo/gcc-trunk/gcc/gimple-match-head.cc:1113
#11 0x00000000016258c0 in gimple_fold_stmt_to_constant_1
(stmt=stmt@entry=0x7ffff7838410, valueize=valueize@entry=0x2bbf480
<valueize_op(tree)>, gvalueize=gvalueize@entry=0x2bbee60 <valueize_op_1(tree)>)
    at /repo/gcc-trunk/gcc/gimple-fold.cc:7391
#12 0x0000000002bd7a52 in ccp_fold (stmt=0x7ffff7838410) at
/repo/gcc-trunk/gcc/tree-ssa-ccp.cc:1289
#13 evaluate_stmt (stmt=0x7ffff7838410) at
/repo/gcc-trunk/gcc/tree-ssa-ccp.cc:2208
#14 0x0000000002bdbfad in visit_assignment (stmt=0x7ffff7838410,
output_p=0x7fffffffcd58) at /repo/gcc-trunk/gcc/tree-ssa-ccp.cc:2841
#15 0x0000000002edd1dc in ssa_propagation_engine::simulate_stmt
(this=this@entry=0x7fffffffcf88, stmt=stmt@entry=0x7ffff7838410) at
/repo/gcc-trunk/gcc/tree-ssa-propagate.cc:230
#16 0x0000000002ede165 in ssa_propagation_engine::simulate_block
(this=0x7fffffffcf88, block=0x7ffff76f4270) at
/repo/gcc-trunk/gcc/tree-ssa-propagate.cc:337
#17 0x0000000002edf73e in ssa_propagation_engine::ssa_propagate
(this=this@entry=0x7fffffffcf88) at
/repo/gcc-trunk/gcc/tree-ssa-propagate.cc:504
#18 0x0000000002bbeccd in do_ssa_ccp (nonzero_p=false) at
/repo/gcc-trunk/gcc/tree-ssa-ccp.cc:2960
#19 (anonymous namespace)::pass_ccp::execute (this=<optimized out>) at
/repo/gcc-trunk/gcc/tree-ssa-ccp.cc:3004
#20 0x00000000021af318 in execute_one_pass (pass=0xc66bc70) at
/repo/gcc-trunk/gcc/passes.cc:2638
#21 0x00000000021b2902 in execute_pass_list_1 (pass=0xc66bc70) at
/repo/gcc-trunk/gcc/passes.cc:2738
#22 0x00000000021b2945 in execute_pass_list_1 (pass=0xc66bb50) at
/repo/gcc-trunk/gcc/passes.cc:2739
#23 0x00000000021b29f9 in execute_pass_list (fn=0x7ffff784c000, pass=<optimized
out>) at /repo/gcc-trunk/gcc/passes.cc:2749
#24 0x00000000021a1bc2 in do_per_function_toporder (callback=0x21b29e0
<execute_pass_list(function*, opt_pass*)>, data=0xc66b950) at
/repo/gcc-trunk/gcc/passes.cc:1774
#25 0x00000000021b3d7a in do_per_function_toporder (data=<optimized out>,
callback=0x21b29e0 <execute_pass_list(function*, opt_pass*)>) at
/repo/gcc-trunk/gcc/plugin.h:191
#26 execute_ipa_pass_list (pass=0xc66b8f0) at
/repo/gcc-trunk/gcc/passes.cc:3083
#27 0x0000000000f92b54 in ipa_passes () at
/repo/gcc-trunk/gcc/cgraphunit.cc:2161
#28 symbol_table::compile (this=0x7ffff76ec000) at
/repo/gcc-trunk/gcc/cgraphunit.cc:2282
#29 0x0000000000fa02ad in symbol_table::compile (this=0x7ffff76ec000) at
/repo/gcc-trunk/gcc/cgraphunit.cc:2262
#30 symbol_table::finalize_compilation_unit (this=0x7ffff76ec000) at
/repo/gcc-trunk/gcc/cgraphunit.cc:2530
#31 0x00000000026d34a3 in compile_file () at /repo/gcc-trunk/gcc/toplev.cc:479
#32 0x0000000000917d45 in do_compile (no_backend=false) at
/repo/gcc-trunk/gcc/toplev.cc:2168
#33 toplev::main (this=this@entry=0x7fffffffd35e, argc=<optimized out>,
argc@entry=15, argv=<optimized out>, argv@entry=0x7fffffffd4a8) at
/repo/gcc-trunk/gcc/toplev.cc:2320
#34 0x000000000091bb6b in main (argc=15, argv=0x7fffffffd4a8) at
/repo/gcc-trunk/gcc/main.cc:39

$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/repo/gcc-trunk/binary-latest-amd64/bin/x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/repo/gcc-trunk/binary-trunk-r13-167-20220507103325-g0c723bb4be2-checking-release-bootstrap-ubsan-amd64/bin/../libexec/gcc/x86_64-pc-linux-gnu/13.0.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /repo/gcc-trunk//configure --enable-languages=c,c++
--enable-valgrind-annotations --disable-nls --enable-checking=release
--with-build-config=bootstrap-ubsan --with-cloog --with-ppl --with-isl
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--target=x86_64-pc-linux-gnu --with-ld=/usr/bin/x86_64-pc-linux-gnu-ld
--with-as=/usr/bin/x86_64-pc-linux-gnu-as --disable-libstdcxx-pch
--prefix=/repo/gcc-trunk//binary-trunk-r13-167-20220507103325-g0c723bb4be2-checking-release-bootstrap-ubsan-amd64
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.0.0 20220507 (experimental) (GCC)

[Bug tree-optimization/105532] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Hmm, so prec is 0 here.
      gcc_checking_assert (prec < HOST_BITS_PER_WIDE_INT);
      int shift = HOST_BITS_PER_WIDE_INT - prec;
      return ((HOST_WIDE_INT) ((unsigned HOST_WIDE_INT) src << shift)) >>
shift;


That is the only way to get a shift of 64.

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
            Summary|UBSAN: gcc/hwint.h:293:61:  |[11/12/13 Regression]
                   |runtime error: shift        |UBSAN: gcc/hwint.h:293:61:
                   |exponent 64 is too large    |runtime error: shift
                   |for 64-bit type 'long       |exponent 64 is too large
                   |unsigned int'               |for 64-bit type 'long
                   |                            |unsigned int'
   Target Milestone|---                         |11.4
           Keywords|                            |ice-on-valid-code
   Last reconfirmed|                            |2022-11-02
             Status|UNCONFIRMED                 |NEW
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=96688

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Confirmed.
The bug is either in tree_nonzero_bits and should not be doing:
  return wi::shwi (-1, TYPE_PRECISION (TREE_TYPE (t)));

For vector types or is in match.pd where it should not be calling
tree_nonzero_bits for vector types.
/* ~(X >> Y) -> ~X >> Y if ~X can be simplified.  */
(simplify
 (bit_not (rshift:s @0 @1))
  (if (!TYPE_UNSIGNED (TREE_TYPE (@0)))
   (rshift (bit_not! @0) @1)
   /* For logical right shifts, this is possible only if @0 doesn't
      have MSB set and the logical right shift is changed into
      arithmetic shift.  */
   (if (!wi::neg_p (tree_nonzero_bits (@0)))
    (with { tree stype = signed_type_for (TREE_TYPE (@0)); }
     (convert (rshift (bit_not! (convert:stype @0)) @1))))))

I almost want to say we should put an gcc_assert in tree_nonzero_bits for when
it gets called with vector or complex types (maybe even real types).

The match.pd pattern was added with r11-6663-g8f8762a2e8659c .

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |pinskia at gcc dot gnu.org
             Status|NEW                         |ASSIGNED

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I am going to take care of this at least for GCC 13.

There is another invalid use here too:
/* Common POPCOUNT/PARITY simplifications.  */
/* popcount(X&C1) is (X>>C2)&1 when C1 == 1<<C2.  Same for parity(X&C1).  */
(for pfun (POPCOUNT PARITY)
  (simplify
    (pfun @0)
    (with { wide_int nz = tree_nonzero_bits (@0); }
      (switch
        (if (nz == 1)
          (convert @0))
        (if (wi::popcount (nz) == 1)
          (with { tree utype = unsigned_type_for (TREE_TYPE (@0)); }
            (convert (rshift:utype (convert:utype @0)
                                   { build_int_cst (integer_type_node,
                                                    wi::ctz (nz)); }))))))))

Found while running the testsuite on x86_64.

Another one:
/* POPCOUNT simplifications.  */
/* popcount(X) + popcount(Y) is popcount(X|Y) when X&Y must be zero.  */
(simplify
  (plus (POPCOUNT:s @0) (POPCOUNT:s @1))
  (if (wi::bit_and (tree_nonzero_bits (@0), tree_nonzero_bits (@1)) == 0)
    (POPCOUNT (bit_ior @0 @1))))


Those 3 are only ones I see in match.pd (the other uses of tree_nonzero_bits
checks the type before the call).

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

--- Comment #4 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Created attachment 53820
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=53820&action=edit
Patch which is under test

This is patch 1 of 2. The 2nd patch adds the assert to catch this earlier on.

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

--- Comment #5 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #2)
> Confirmed.
> The bug is either in tree_nonzero_bits and should not be doing:
>   return wi::shwi (-1, TYPE_PRECISION (TREE_TYPE (t)));

Just FYI, TYPE_PRECISION for vectors is a log2 of the number of elements, see
TYPE_VECTOR_SUBPARTS. Which is why you get 0 here since we have 1 element.

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

--- Comment #6 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Patch submitted here:
https://gcc.gnu.org/pipermail/gcc-patches/2022-November/604914.html

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P2
           Keywords|                            |patch

--- Comment #7 from Richard Biener <rguenth at gcc dot gnu.org> ---
Patch posted & approved.

[Bug tree-optimization/105532] [11/12/13 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The trunk branch has been updated by Andrew Pinski <pinskia@gcc.gnu.org>:

https://gcc.gnu.org/g:193fccaa5c3525e979a989835c47c76d2c49d10c

commit r13-4834-g193fccaa5c3525e979a989835c47c76d2c49d10c
Author: Andrew Pinski <apinski@marvell.com>
Date:   Wed Nov 2 15:56:31 2022 +0000

    Fix PR 105532: match.pd patterns calling tree_nonzero_bits with vector
types

    Even though this PR was reported with an ubsan issue, the problem is
    tree_nonzero_bits is being called with an expression which is a vector
type.
    This fixes three patterns I noticed which does that.
    And adds a testcase for one of the patterns.

    OK? Bootstrapped and tested on x86_64-linux-gnu with no regressions

    gcc/ChangeLog:

            PR tree-optimization/105532
            * match.pd (~(X >> Y) -> ~X >> Y): Check if it is an integral
            type before calling tree_nonzero_bits.
            (popcount(X) + popcount(Y)): Likewise.
            (popcount(X&C1)): Likewise.

    gcc/testsuite/ChangeLog:

            * gcc.c-torture/compile/vector-shift-1.c: New test.

[Bug tree-optimization/105532] [11/12 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[11/12/13 Regression]       |[11/12 Regression] UBSAN:
                   |UBSAN: gcc/hwint.h:293:61:  |gcc/hwint.h:293:61: runtime
                   |runtime error: shift        |error: shift exponent 64 is
                   |exponent 64 is too large    |too large for 64-bit type
                   |for 64-bit type 'long       |'long unsigned int'
                   |unsigned int'               |
      Known to fail|13.0                        |
      Known to work|                            |13.0

--- Comment #9 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Fixed at least for GCC 13 (trunk). Still open for backporting.

[Bug tree-optimization/105532] [11/12 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Andrew Pinski
<pinskia@gcc.gnu.org>:

https://gcc.gnu.org/g:67608f7a2cf3eff99654ab3f44d5183049e3b36c

commit r12-9241-g67608f7a2cf3eff99654ab3f44d5183049e3b36c
Author: Andrew Pinski <apinski@marvell.com>
Date:   Wed Nov 2 15:56:31 2022 +0000

    Fix PR 105532: match.pd patterns calling tree_nonzero_bits with vector
types

    Even though this PR was reported with an ubsan issue, the problem is
    tree_nonzero_bits is being called with an expression which is a vector
type.
    This fixes three patterns I noticed which does that.
    And adds a testcase for one of the patterns.

    Committed after a bootstrapped and tested on x86_64-linux-gnu with no
regressions

    gcc/ChangeLog:

            PR tree-optimization/105532
            * match.pd (~(X >> Y) -> ~X >> Y): Check if it is an integral
            type before calling tree_nonzero_bits.
            (popcount(X) + popcount(Y)): Likewise.
            (popcount(X&C1)): Likewise.

    gcc/testsuite/ChangeLog:

            * gcc.c-torture/compile/vector-shift-1.c: New test.

    (cherry picked from commit 193fccaa5c3525e979a989835c47c76d2c49d10c)

[Bug tree-optimization/105532] [11 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to fail|                            |10.1.0, 12.1.0, 12.2.0
            Summary|[11/12 Regression] UBSAN:   |[11 Regression] UBSAN:
                   |gcc/hwint.h:293:61: runtime |gcc/hwint.h:293:61: runtime
                   |error: shift exponent 64 is |error: shift exponent 64 is
                   |too large for 64-bit type   |too large for 64-bit type
                   |'long unsigned int'         |'long unsigned int'
      Known to work|                            |12.2.1

--- Comment #11 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Fixed for GCC 12.3.0 also.

[Bug tree-optimization/105532] [11 Regression] UBSAN: gcc/hwint.h:293:61: runtime error: shift exponent 64 is too large for 64-bit type 'long unsigned int'

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105532

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|11.4                        |11.5

--- Comment #12 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 11.4 is being released, retargeting bugs to GCC 11.5.