[gcc r12-8158] simplify-rtx: Don't assume shift count has the same mode as the shift [PR105247]

[gcc r12-8158] simplify-rtx: Don't assume shift count has the same mode as the shift [PR105247]

[Jakub Jelinek]

https://gcc.gnu.org/g:ba2f60499dd4a3bc1bb4e99fa12dda3bc1548519

commit r12-8158-gba2f60499dd4a3bc1bb4e99fa12dda3bc1548519
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Thu Apr 14 13:47:34 2022 +0200

    simplify-rtx: Don't assume shift count has the same mode as the shift [PR105247]
    
    The following testcase ICEs on ia64.  It is UB at runtime, but we shouldn't
    ICE on it...
    The problem is that on ia64, the shift count (last operand of ASHIFT etc.)
    is promoted to DImode (using zero-extension), while most other targets
    use much narrower modes (say QImode).  If we try to simplify a shift
    and the shift count is CONST_INT or other VOIDmode integer constant
    which isn't properly sign extended for the first operand's mode
    (in the testcase the shift count is 0xfffffff8U and it is a SImode shift),
    then we ICE during wide_int wop1 = pop1; in the first hunk, INTVAL == 0xfffffff8U
    is not valid for SImode.  I think in theory we could run into this even
    on other targets, say if they use SImode or HImode shift counts for e.g.
    QImode shifts.  I hope word size is the upper bound of what a reasonable
    target should use, using e.g. multiple registers for the shift count is
    insane, so the following patch if op1 has VOIDmode and int_mode
    is narrower than word uses word_mode for extraction of the value.
    
    2022-04-14  Jakub Jelinek  <jakub@redhat.com>
    
            PR target/105247
            * simplify-rtx.cc (simplify_const_binary_operation): For shifts
            or rotates by VOIDmode constant integer shift count use word_mode
            for the operand if int_mode is narrower than word.
    
            * gcc.c-torture/compile/pr105247.c: New test.

Diff:
---
 gcc/simplify-rtx.cc                            | 24 +++++++++++++++++++++++-
 gcc/testsuite/gcc.c-torture/compile/pr105247.c | 10 ++++++++++
 2 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/gcc/simplify-rtx.cc b/gcc/simplify-rtx.cc
index 5d4939c9206..e152918b0f1 100644
--- a/gcc/simplify-rtx.cc
+++ b/gcc/simplify-rtx.cc
@@ -5066,6 +5066,15 @@ simplify_const_binary_operation (enum rtx_code code, machine_mode mode,
 	case SS_ASHIFT:
 	case US_ASHIFT:
 	  {
+	    /* The shift count might be in SImode while int_mode might
+	       be narrower.  On IA-64 it is even DImode.  If the shift
+	       count is too large and doesn't fit into int_mode, we'd
+	       ICE.  So, if int_mode is narrower than word, use
+	       word_mode for the shift count.  */
+	    if (GET_MODE (op1) == VOIDmode
+		&& GET_MODE_PRECISION (int_mode) < BITS_PER_WORD)
+	      pop1 = rtx_mode_t (op1, word_mode);
+
 	    wide_int wop1 = pop1;
 	    if (SHIFT_COUNT_TRUNCATED)
 	      wop1 = wi::umod_trunc (wop1, GET_MODE_PRECISION (int_mode));
@@ -5112,6 +5121,15 @@ simplify_const_binary_operation (enum rtx_code code, machine_mode mode,
 	case ROTATE:
 	case ROTATERT:
 	  {
+	    /* The rotate count might be in SImode while int_mode might
+	       be narrower.  On IA-64 it is even DImode.  If the shift
+	       count is too large and doesn't fit into int_mode, we'd
+	       ICE.  So, if int_mode is narrower than word, use
+	       word_mode for the shift count.  */
+	    if (GET_MODE (op1) == VOIDmode
+		&& GET_MODE_PRECISION (int_mode) < BITS_PER_WORD)
+	      pop1 = rtx_mode_t (op1, word_mode);
+
 	    if (wi::neg_p (pop1))
 	      return NULL_RTX;
 
@@ -5208,7 +5226,11 @@ simplify_const_binary_operation (enum rtx_code code, machine_mode mode,
 	case ASHIFT:
 	  if (CONST_SCALAR_INT_P (op1))
 	    {
-	      wide_int shift = rtx_mode_t (op1, mode);
+	      wide_int shift
+		= rtx_mode_t (op1,
+			      GET_MODE (op1) == VOIDmode
+			      && GET_MODE_PRECISION (int_mode) < BITS_PER_WORD
+			      ? word_mode : mode);
 	      if (SHIFT_COUNT_TRUNCATED)
 		shift = wi::umod_trunc (shift, GET_MODE_PRECISION (int_mode));
 	      else if (wi::geu_p (shift, GET_MODE_PRECISION (int_mode)))
diff --git a/gcc/testsuite/gcc.c-torture/compile/pr105247.c b/gcc/testsuite/gcc.c-torture/compile/pr105247.c
new file mode 100644
index 00000000000..88892d98453
--- /dev/null
+++ b/gcc/testsuite/gcc.c-torture/compile/pr105247.c
@@ -0,0 +1,10 @@
+/* PR target/105247 */
+
+int a;
+
+void
+foo (void)
+{
+  int y = -8;
+  a = 1 << y;
+}