[gcc/devel/omp/gcc-12] amdgcn: Fix register size bug

[gcc/devel/omp/gcc-12] amdgcn: Fix register size bug

[Andrew Stubbs]

https://gcc.gnu.org/g:9d0ea7a55ac815c59dd3b004008aa2195c541d12

commit 9d0ea7a55ac815c59dd3b004008aa2195c541d12
Author: Andrew Stubbs <ams@codesourcery.com>
Date:   Fri Mar 17 11:04:12 2023 +0000

    amdgcn: Fix register size bug
    
    Fix an issue in which "vectors" of duplicate entries placed in scalar
    registers caused the following 63 registers to be marked live, for the
    purpose of prologue generation, which resulted in stack corruption.
    
    gcc/ChangeLog:
    
            * config/gcn/gcn.cc (gcn_class_max_nregs): Handle vectors in SGPRs.
            (move_callee_saved_registers): Detect the bug condition early.

Diff:
---
 gcc/ChangeLog.omp     |  5 +++++
 gcc/config/gcn/gcn.cc | 13 +++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/gcc/ChangeLog.omp b/gcc/ChangeLog.omp
index fdccdabf72b..c724308eba9 100644
--- a/gcc/ChangeLog.omp
+++ b/gcc/ChangeLog.omp
@@ -1,3 +1,8 @@
+2023-03-17  Andrew Stubbs  <ams@codesourcery.com>
+
+	* config/gcn/gcn.cc (gcn_class_max_nregs): Handle vectors in SGPRs.
+	(move_callee_saved_registers): Detect the bug condition early.
+
 2023-03-17  Andrew Stubbs  <ams@codesourcery.com>
 
 	Backport from mainline:
diff --git a/gcc/config/gcn/gcn.cc b/gcc/config/gcn/gcn.cc
index 7403dfebcea..2f48405f06f 100644
--- a/gcc/config/gcn/gcn.cc
+++ b/gcc/config/gcn/gcn.cc
@@ -503,6 +503,15 @@ gcn_class_max_nregs (reg_class_t rclass, machine_mode mode)
     }
   else if (rclass == VCC_CONDITIONAL_REG && mode == BImode)
     return 2;
+
+  /* Vector modes in SGPRs are not supposed to happen (disallowed by
+     gcn_hard_regno_mode_ok), but there are some patterns that have an "Sv"
+     constraint and are used by splitters, post-reload.
+     This ensures that we don't accidentally mark the following 63 scalar
+     registers as "live".  */
+  if (rclass == SGPR_REGS && VECTOR_MODE_P (mode))
+    return CEIL (GET_MODE_SIZE (GET_MODE_INNER (mode)), 4);
+
   return CEIL (GET_MODE_SIZE (mode), 4);
 }
 
@@ -3270,6 +3279,10 @@ move_callee_saved_registers (rtx sp, machine_function *offsets,
       emit_insn (move_vectors);
       emit_insn (move_scalars);
     }
+
+  /* This happens when a new register becomes "live" after reload.
+     Check your splitters!  */
+  gcc_assert (offset <= offsets->callee_saves);
 }
 
 /* Generate prologue.  Called from gen_prologue during pro_and_epilogue pass.