[gcc r12-9475] powerpc: Fix up *branch_anddi3_dot for -m32 -mpowerpc64 [PR109566]

[Jakub Jelinek <jakub@gcc.gnu.org>]

https://gcc.gnu.org/g:2e5e72488b108e5d75049179ef91a093e5fedc49

commit r12-9475-g2e5e72488b108e5d75049179ef91a093e5fedc49
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Tue Apr 25 14:20:51 2023 +0200

    powerpc: Fix up *branch_anddi3_dot for -m32 -mpowerpc64 [PR109566]
    
    The following testcase reduced from newlib ICEs on powerpc-linux,
    with -O2 -m32 -mpowerpc64 since r12-6433 PR102239 optimization was
    added and on the original testcase since some ranger improvements in
    GCC 13 made it no longer latent on newlib.
    The problem is that the *branch_anddi3_dot define_insn_and_split
    relies on the *rotldi3_mask_dot define_insn_and_split being recognized
    during splitting.  The rs6000_is_valid_rotate_dot_mask function checks whether
    the mask is a CONST_INT which is a valid mask, but *rotl<mode>3_mask_dot in
    addition to checking that it is a valid mask also has
      (<MODE>mode == Pmode || UINTVAL (operands[3]) <= 0x7fffffff)
    test in the condition.  For TARGET_64BIT that doesn't add any further
    requirements, but for !TARGET_64BIT && TARGET_POWERPC64 if the AND
    second operand is larger than INT_MAX it will not be recognized.
    
    The rs6000_is_valid_rotate_dot_mask function is used solely in one spot,
    condition of *branch_anddi3_dot, so the following patch adjusts it
    to check for that as well.
    
    2023-04-25  Jakub Jelinek  <jakub@redhat.com>
    
            PR target/109566
            * config/rs6000/rs6000.cc (rs6000_is_valid_rotate_dot_mask): For
            !TARGET_64BIT, don't return true if UINTVAL (mask) << (63 - nb)
            is larger than signed int maximum.
    
            * gcc.target/powerpc/pr109566.c: New test.
    
    (cherry picked from commit 97f8f2d0a0384d377ca46da88495f9a3d18d4415)

Diff:
---
 gcc/config/rs6000/rs6000.cc                 | 11 ++++++++++-
 gcc/testsuite/gcc.target/powerpc/pr109566.c | 18 ++++++++++++++++++
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/gcc/config/rs6000/rs6000.cc b/gcc/config/rs6000/rs6000.cc
index 5654a245452..f678561071a 100644
--- a/gcc/config/rs6000/rs6000.cc
+++ b/gcc/config/rs6000/rs6000.cc
@@ -11287,7 +11287,16 @@ bool
 rs6000_is_valid_rotate_dot_mask (rtx mask, machine_mode mode)
 {
   int nb, ne;
-  return rs6000_is_valid_mask (mask, &nb, &ne, mode) && nb >= ne && ne > 0;
+  if (rs6000_is_valid_mask (mask, &nb, &ne, mode) && nb >= ne && ne > 0)
+    {
+      if (TARGET_64BIT)
+	return true;
+      /* *rotldi3_mask_dot requires for -m32 -mpowerpc64 that the mask is
+	 <= 0x7fffffff.  */
+      return (UINTVAL (mask) << (63 - nb)) <= 0x7fffffff;
+    }
+
+  return false;
 }
 
 /* Return whether MASK (a CONST_INT) is a valid mask for any rlwinm, rldicl,
diff --git a/gcc/testsuite/gcc.target/powerpc/pr109566.c b/gcc/testsuite/gcc.target/powerpc/pr109566.c
new file mode 100644
index 00000000000..bfbf9224755
--- /dev/null
+++ b/gcc/testsuite/gcc.target/powerpc/pr109566.c
@@ -0,0 +1,18 @@
+/* PR target/109566 */
+/* Skip this on aix, otherwise it emits the error message like "64-bit
+   computation with 32-bit addressing not yet supported" on aix.  */
+/* { dg-skip-if "" { powerpc*-*-aix* } } */
+/* { dg-do compile } */
+/* { dg-options "-O2 -mpowerpc64" } */
+
+void
+foo (double x)
+{
+  union { double d; unsigned i; } u;
+  u.d = x;
+  if (u.i & 0x7ff00000)
+    return;
+  else
+    for (;;)
+      ;
+}