[PATCH] simplify-rtx: Don't assume shift count has the same mode as the shift [PR105247]

[Jakub Jelinek <jakub@redhat.com>]

Hi!

The following testcase ICEs on ia64.  It is UB at runtime, but we shouldn't
ICE on it...
The problem is that on ia64, the shift count (last operand of ASHIFT etc.)
is promoted to DImode (using zero-extension), while most other targets
use much narrower modes (say QImode).  If we try to simplify a shift
and the shift count is CONST_INT or other VOIDmode integer constant
which isn't properly sign extended for the first operand's mode
(in the testcase the shift count is 0xfffffff8U and it is a SImode shift),
then we ICE during wide_int wop1 = pop1; in the first hunk, INTVAL == 0xfffffff8U
is not valid for SImode.  I think in theory we could run into this even
on other targets, say if they use SImode or HImode shift counts for e.g.
QImode shifts.  I hope word size is the upper bound of what a reasonable
target should use, using e.g. multiple registers for the shift count is
insane, so the following patch if op1 has VOIDmode and int_mode
is narrower than word uses word_mode for extraction of the value.

Bootstrapped/regtested on x86_64-linux and i686-linux and tested on the
testcase with cross to ia64-linux, ok for trunk?

2022-04-13  Jakub Jelinek  <jakub@redhat.com>

	PR target/105247
	* simplify-rtx.cc (simplify_const_binary_operation): For shifts
	or rotates by VOIDmode constant integer shift count use word_mode
	for the operand if int_mode is narrower than word.

	* gcc.c-torture/compile/pr105247.c: New test.

--- gcc/simplify-rtx.cc.jj	2022-03-09 09:12:30.546678018 +0100
+++ gcc/simplify-rtx.cc	2022-04-13 13:13:01.920941427 +0200
@@ -5066,6 +5066,15 @@ simplify_const_binary_operation (enum rt
 	case SS_ASHIFT:
 	case US_ASHIFT:
 	  {
+	    /* The shift count might be in SImode while int_mode might
+	       be narrower.  On IA-64 it is even DImode.  If the shift
+	       count is too large and doesn't fit into int_mode, we'd
+	       ICE.  So, if int_mode is narrower than word, use
+	       word_mode for the shift count.  */
+	    if (GET_MODE (op1) == VOIDmode
+		&& GET_MODE_PRECISION (int_mode) < BITS_PER_WORD)
+	      pop1 = rtx_mode_t (op1, word_mode);
+
 	    wide_int wop1 = pop1;
 	    if (SHIFT_COUNT_TRUNCATED)
 	      wop1 = wi::umod_trunc (wop1, GET_MODE_PRECISION (int_mode));
@@ -5112,6 +5121,15 @@ simplify_const_binary_operation (enum rt
 	case ROTATE:
 	case ROTATERT:
 	  {
+	    /* The rotate count might be in SImode while int_mode might
+	       be narrower.  On IA-64 it is even DImode.  If the shift
+	       count is too large and doesn't fit into int_mode, we'd
+	       ICE.  So, if int_mode is narrower than word, use
+	       word_mode for the shift count.  */
+	    if (GET_MODE (op1) == VOIDmode
+		&& GET_MODE_PRECISION (int_mode) < BITS_PER_WORD)
+	      pop1 = rtx_mode_t (op1, word_mode);
+
 	    if (wi::neg_p (pop1))
 	      return NULL_RTX;
 
@@ -5208,7 +5226,11 @@ simplify_const_binary_operation (enum rt
 	case ASHIFT:
 	  if (CONST_SCALAR_INT_P (op1))
 	    {
-	      wide_int shift = rtx_mode_t (op1, mode);
+	      wide_int shift
+		= rtx_mode_t (op1,
+			      GET_MODE (op1) == VOIDmode
+			      && GET_MODE_PRECISION (int_mode) < BITS_PER_WORD
+			      ? word_mode : mode);
 	      if (SHIFT_COUNT_TRUNCATED)
 		shift = wi::umod_trunc (shift, GET_MODE_PRECISION (int_mode));
 	      else if (wi::geu_p (shift, GET_MODE_PRECISION (int_mode)))
--- gcc/testsuite/gcc.c-torture/compile/pr105247.c.jj	2022-04-13 13:12:40.496239939 +0200
+++ gcc/testsuite/gcc.c-torture/compile/pr105247.c	2022-04-13 13:12:28.152411929 +0200
@@ -0,0 +1,10 @@
+/* PR target/105247 */
+
+int a;
+
+void
+foo (void)
+{
+  int y = -8;
+  a = 1 << y;
+}

	Jakub