[PATCH 00/12] Tweaks and extensions to the mode-switching pass

[PATCH 00/12] Tweaks and extensions to the mode-switching pass

[Richard Sandiford]

This series of patches extends the mode-switching pass so that it
can be used for AArch64's SME.  I wondered about including a detailed
description of how the SME mode changes work, but it'd probably be
a distraction.  The system is quite complex and target-specific, and
hopefully the details aren't necessary to understand the motivation.

One of the main requirements for one of the mode-switched "entities" is
that the current mode must always be known at compile time.  It would be
too cumbersome to work out the current mode at runtime and make a dynamic
choice about what to do.  The entity therefore wants the usual LCM
placement where possible, but would rather have redundant mode
transitions than transitions from unknown modes.

In many cases, the modified pass seems to generate optimal or near-optimal
mode-switching code, even with these additional requirements.  Tests are
included with the SME work.

Bootstrapped & regression-tested on aarch64-linux-gnu and
x86_64-linux-gnu, although only the latter is useful since
AArch64 doesn't yet use the pass.  Also tested by building crosses
for epiphany-elf, riscv64-elf and sh-linux-gnu, to pick one triplet
per other target that uses mode switching.

OK to install?

Thanks,
Richard

Richard Sandiford (12):
  mode-switching: Tweak the macro/hook documentation
  mode-switching: Add note problem
  mode-switching: Avoid quadractic list operation
  mode-switching: Fix the mode passed to the emit hook
  mode-switching: Simplify recording of transparency
  mode-switching: Tweak entry/exit handling
  mode-switching: Allow targets to set the mode for EH handlers
  mode-switching: Pass set of live registers to the needed hook
  mode-switching: Pass the set of live registers to the after hook
  mode-switching: Use 1-based edge aux fields
  mode-switching: Add a target-configurable confluence operator
  mode-switching: Add a backprop hook

 gcc/config/epiphany/epiphany-protos.h  |   7 +-
 gcc/config/epiphany/epiphany.cc        |   7 +-
 gcc/config/epiphany/mode-switch-use.cc |   2 +-
 gcc/config/i386/i386.cc                |   4 +-
 gcc/config/riscv/riscv.cc              |   4 +-
 gcc/config/sh/sh.cc                    |   9 +-
 gcc/doc/tm.texi                        | 126 ++++--
 gcc/doc/tm.texi.in                     |  32 +-
 gcc/mode-switching.cc                  | 582 +++++++++++++++++++++----
 gcc/target.def                         | 103 ++++-
 10 files changed, 714 insertions(+), 162 deletions(-)

-- 
2.25.1

[PATCH 01/12] mode-switching: Tweak the macro/hook documentation

[Richard Sandiford]

I found the documentation for the mode-switching macros/hooks
a bit hard to follow at first.  This patch tries to add the
information that I think would have made it easier to understand.

Of course, documentation preferences are personal, and so I could
be changing something that others understood to something that
seems impenetrable.

Some notes on specific changes:

- "in an optimizing compilation" didn't seem accurate; the pass
  is run even at -O0, and often needs to be for correctness.

- "at run time" meant when the compiler was run, rather than when
  the compiled code was run.

- Removing the list of optional macros isn't a clarification,
  but it means that upcoming patches don't create an absurdly
  long list.

- I don't really understand the purpose of TARGET_MODE_PRIORITY,
  so I mostly left that alone.

gcc/
	* target.def: Tweak documentation of mode-switching hooks.
	* doc/tm.texi.in (OPTIMIZE_MODE_SWITCHING): Tweak documentation.
	(NUM_MODES_FOR_MODE_SWITCHING): Likewise.
	* doc/tm.texi: Regenerate.
---
 gcc/doc/tm.texi    | 69 ++++++++++++++++++++++++++++------------------
 gcc/doc/tm.texi.in | 26 +++++++++--------
 gcc/target.def     | 43 ++++++++++++++++++-----------
 3 files changed, 84 insertions(+), 54 deletions(-)

diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index f7ac806ff15..759331a2c96 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10368,7 +10368,7 @@ The following macros control mode switching optimizations:
 
 @defmac OPTIMIZE_MODE_SWITCHING (@var{entity})
 Define this macro if the port needs extra instructions inserted for mode
-switching in an optimizing compilation.
+switching.
 
 For an example, the SH4 can perform both single and double precision
 floating point operations, but to perform a single precision operation,
@@ -10378,73 +10378,88 @@ purpose register as a scratch register, hence these FPSCR sets have to
 be inserted before reload, i.e.@: you cannot put this into instruction emitting
 or @code{TARGET_MACHINE_DEPENDENT_REORG}.
 
-You can have multiple entities that are mode-switched, and select at run time
-which entities actually need it.  @code{OPTIMIZE_MODE_SWITCHING} should
-return nonzero for any @var{entity} that needs mode-switching.
+You can have multiple entities that are mode-switched, some of which might
+only be needed conditionally.  The entities are identified by their index
+into the @code{NUM_MODES_FOR_MODE_SWITCHING} initializer, with the length
+of the initializer determining the number of entities.
+
+@code{OPTIMIZE_MODE_SWITCHING} should return nonzero for any @var{entity}
+that needs mode-switching.
+
 If you define this macro, you also have to define
 @code{NUM_MODES_FOR_MODE_SWITCHING}, @code{TARGET_MODE_NEEDED},
 @code{TARGET_MODE_PRIORITY} and @code{TARGET_MODE_EMIT}.
-@code{TARGET_MODE_AFTER}, @code{TARGET_MODE_ENTRY}, and @code{TARGET_MODE_EXIT}
-are optional.
+The other macros in this section are optional.
 @end defmac
 
 @defmac NUM_MODES_FOR_MODE_SWITCHING
 If you define @code{OPTIMIZE_MODE_SWITCHING}, you have to define this as
 initializer for an array of integers.  Each initializer element
 N refers to an entity that needs mode switching, and specifies the number
-of different modes that might need to be set for this entity.
-The position of the initializer in the initializer---starting counting at
+of different modes that are defined for that entity.
+The position of the element in the initializer---starting counting at
 zero---determines the integer that is used to refer to the mode-switched
 entity in question.
-In macros that take mode arguments / yield a mode result, modes are
-represented as numbers 0 @dots{} N @minus{} 1.  N is used to specify that no mode
-switch is needed / supplied.
+Modes are represented as numbers 0 @dots{} N @minus{} 1.
+In mode arguments and return values, N either represents an unknown
+mode or ``no mode'', depending on context.
 @end defmac
 
 @deftypefn {Target Hook} void TARGET_MODE_EMIT (int @var{entity}, int @var{mode}, int @var{prev_mode}, HARD_REG_SET @var{regs_live})
 Generate one or more insns to set @var{entity} to @var{mode}.
 @var{hard_reg_live} is the set of hard registers live at the point where
 the insn(s) are to be inserted. @var{prev_moxde} indicates the mode
-to switch from. Sets of a lower numbered entity will be emitted before
+to switch from, or is the number of modes if the previous mode is not
+known.  Sets of a lower numbered entity will be emitted before
 sets of a higher numbered entity to a mode of the same or lower priority.
 @end deftypefn
 
 @deftypefn {Target Hook} int TARGET_MODE_NEEDED (int @var{entity}, rtx_insn *@var{insn})
 @var{entity} is an integer specifying a mode-switched entity.
-If @code{OPTIMIZE_MODE_SWITCHING} is defined, you must define this macro
-to return an integer value not larger than the corresponding element
-in @code{NUM_MODES_FOR_MODE_SWITCHING}, to denote the mode that @var{entity}
-must be switched into prior to the execution of @var{insn}.
+If @code{OPTIMIZE_MODE_SWITCHING} is defined, you must define this hook
+to return the mode that @var{entity} must be switched into prior to the
+execution of @var{insn}, or the number of modes if @var{insn} has no
+such requirement.
 @end deftypefn
 
 @deftypefn {Target Hook} int TARGET_MODE_AFTER (int @var{entity}, int @var{mode}, rtx_insn *@var{insn})
 @var{entity} is an integer specifying a mode-switched entity.
-If this macro is defined, it is evaluated for every @var{insn} during mode
-switching.  It determines the mode that an insn results
-in (if different from the incoming mode).
+If this hook is defined, it is evaluated for every @var{insn} during mode
+switching.  It returns the mode that @var{entity} is in after @var{insn}
+has been executed.  @var{mode} is the mode that @var{entity} was in
+before @var{insn} was executed, taking account of @var{TARGET_MODE_NEEDED}.
+
+@var{mode} is equal to the number of modes defined for @var{entity}
+if the mode before @var{insn} is unknown.  The hook should likewise return
+the number of modes if it does not know what mode @var{entity} has after
+@var{insn}.
+
+Not defining the hook is equivalent to returning @var{mode}.
 @end deftypefn
 
 @deftypefn {Target Hook} int TARGET_MODE_ENTRY (int @var{entity})
-If this macro is defined, it is evaluated for every @var{entity} that
-needs mode switching.  It should evaluate to an integer, which is a mode
-that @var{entity} is assumed to be switched to at function entry.
+If this hook is defined, it is evaluated for every @var{entity} that
+needs mode switching.  It should return the mode that @var{entity} is
+guaranteed to be in on entry to the function, or the number of modes
+if there is no such guarantee.
 If @code{TARGET_MODE_ENTRY} is defined then @code{TARGET_MODE_EXIT}
 must be defined.
 @end deftypefn
 
 @deftypefn {Target Hook} int TARGET_MODE_EXIT (int @var{entity})
-If this macro is defined, it is evaluated for every @var{entity} that
-needs mode switching.  It should evaluate to an integer, which is a mode
-that @var{entity} is assumed to be switched to at function exit.
+If this hook is defined, it is evaluated for every @var{entity} that
+needs mode switching.  It should return the mode that @var{entity} must
+be in on return from the function, or the number of modes if there is no
+such requirement.
 If @code{TARGET_MODE_EXIT} is defined then @code{TARGET_MODE_ENTRY}
 must be defined.
 @end deftypefn
 
 @deftypefn {Target Hook} int TARGET_MODE_PRIORITY (int @var{entity}, int @var{n})
-This macro specifies the order in which modes for @var{entity}
+This hook specifies the order in which modes for @var{entity}
 are processed. 0 is the highest priority,
 @code{NUM_MODES_FOR_MODE_SWITCHING[@var{entity}] - 1} the lowest.
-The value of the macro should be an integer designating a mode
+The hook returns an integer designating a mode
 for @var{entity}.  For any fixed @var{entity}, @code{mode_priority}
 (@var{entity}, @var{n}) shall be a bijection in 0 @dots{}
 @code{num_modes_for_mode_switching[@var{entity}] - 1}.
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index 141027e0bb4..a7b7aa289d8 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -6932,7 +6932,7 @@ The following macros control mode switching optimizations:
 
 @defmac OPTIMIZE_MODE_SWITCHING (@var{entity})
 Define this macro if the port needs extra instructions inserted for mode
-switching in an optimizing compilation.
+switching.
 
 For an example, the SH4 can perform both single and double precision
 floating point operations, but to perform a single precision operation,
@@ -6942,27 +6942,31 @@ purpose register as a scratch register, hence these FPSCR sets have to
 be inserted before reload, i.e.@: you cannot put this into instruction emitting
 or @code{TARGET_MACHINE_DEPENDENT_REORG}.
 
-You can have multiple entities that are mode-switched, and select at run time
-which entities actually need it.  @code{OPTIMIZE_MODE_SWITCHING} should
-return nonzero for any @var{entity} that needs mode-switching.
+You can have multiple entities that are mode-switched, some of which might
+only be needed conditionally.  The entities are identified by their index
+into the @code{NUM_MODES_FOR_MODE_SWITCHING} initializer, with the length
+of the initializer determining the number of entities.
+
+@code{OPTIMIZE_MODE_SWITCHING} should return nonzero for any @var{entity}
+that needs mode-switching.
+
 If you define this macro, you also have to define
 @code{NUM_MODES_FOR_MODE_SWITCHING}, @code{TARGET_MODE_NEEDED},
 @code{TARGET_MODE_PRIORITY} and @code{TARGET_MODE_EMIT}.
-@code{TARGET_MODE_AFTER}, @code{TARGET_MODE_ENTRY}, and @code{TARGET_MODE_EXIT}
-are optional.
+The other macros in this section are optional.
 @end defmac
 
 @defmac NUM_MODES_FOR_MODE_SWITCHING
 If you define @code{OPTIMIZE_MODE_SWITCHING}, you have to define this as
 initializer for an array of integers.  Each initializer element
 N refers to an entity that needs mode switching, and specifies the number
-of different modes that might need to be set for this entity.
-The position of the initializer in the initializer---starting counting at
+of different modes that are defined for that entity.
+The position of the element in the initializer---starting counting at
 zero---determines the integer that is used to refer to the mode-switched
 entity in question.
-In macros that take mode arguments / yield a mode result, modes are
-represented as numbers 0 @dots{} N @minus{} 1.  N is used to specify that no mode
-switch is needed / supplied.
+Modes are represented as numbers 0 @dots{} N @minus{} 1.
+In mode arguments and return values, N either represents an unknown
+mode or ``no mode'', depending on context.
 @end defmac
 
 @hook TARGET_MODE_EMIT
diff --git a/gcc/target.def b/gcc/target.def
index 42622177ef9..3dae33522f1 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7020,51 +7020,62 @@ DEFHOOK
  "Generate one or more insns to set @var{entity} to @var{mode}.\n\
 @var{hard_reg_live} is the set of hard registers live at the point where\n\
 the insn(s) are to be inserted. @var{prev_moxde} indicates the mode\n\
-to switch from. Sets of a lower numbered entity will be emitted before\n\
+to switch from, or is the number of modes if the previous mode is not\n\
+known.  Sets of a lower numbered entity will be emitted before\n\
 sets of a higher numbered entity to a mode of the same or lower priority.",
  void, (int entity, int mode, int prev_mode, HARD_REG_SET regs_live), NULL)
 
 DEFHOOK
 (needed,
  "@var{entity} is an integer specifying a mode-switched entity.\n\
-If @code{OPTIMIZE_MODE_SWITCHING} is defined, you must define this macro\n\
-to return an integer value not larger than the corresponding element\n\
-in @code{NUM_MODES_FOR_MODE_SWITCHING}, to denote the mode that @var{entity}\n\
-must be switched into prior to the execution of @var{insn}.",
+If @code{OPTIMIZE_MODE_SWITCHING} is defined, you must define this hook\n\
+to return the mode that @var{entity} must be switched into prior to the\n\
+execution of @var{insn}, or the number of modes if @var{insn} has no\n\
+such requirement.",
  int, (int entity, rtx_insn *insn), NULL)
 
 DEFHOOK
 (after,
  "@var{entity} is an integer specifying a mode-switched entity.\n\
-If this macro is defined, it is evaluated for every @var{insn} during mode\n\
-switching.  It determines the mode that an insn results\n\
-in (if different from the incoming mode).",
+If this hook is defined, it is evaluated for every @var{insn} during mode\n\
+switching.  It returns the mode that @var{entity} is in after @var{insn}\n\
+has been executed.  @var{mode} is the mode that @var{entity} was in\n\
+before @var{insn} was executed, taking account of @var{TARGET_MODE_NEEDED}.\n\
+\n\
+@var{mode} is equal to the number of modes defined for @var{entity}\n\
+if the mode before @var{insn} is unknown.  The hook should likewise return\n\
+the number of modes if it does not know what mode @var{entity} has after\n\
+@var{insn}.\n\
+\n\
+Not defining the hook is equivalent to returning @var{mode}.",
  int, (int entity, int mode, rtx_insn *insn), NULL)
 
 DEFHOOK
 (entry,
- "If this macro is defined, it is evaluated for every @var{entity} that\n\
-needs mode switching.  It should evaluate to an integer, which is a mode\n\
-that @var{entity} is assumed to be switched to at function entry.\n\
+ "If this hook is defined, it is evaluated for every @var{entity} that\n\
+needs mode switching.  It should return the mode that @var{entity} is\n\
+guaranteed to be in on entry to the function, or the number of modes\n\
+if there is no such guarantee.\n\
 If @code{TARGET_MODE_ENTRY} is defined then @code{TARGET_MODE_EXIT}\n\
 must be defined.",
  int, (int entity), NULL)
 
 DEFHOOK
 (exit,
- "If this macro is defined, it is evaluated for every @var{entity} that\n\
-needs mode switching.  It should evaluate to an integer, which is a mode\n\
-that @var{entity} is assumed to be switched to at function exit.\n\
+ "If this hook is defined, it is evaluated for every @var{entity} that\n\
+needs mode switching.  It should return the mode that @var{entity} must\n\
+be in on return from the function, or the number of modes if there is no\n\
+such requirement.\n\
 If @code{TARGET_MODE_EXIT} is defined then @code{TARGET_MODE_ENTRY}\n\
 must be defined.",
  int, (int entity), NULL)
 
 DEFHOOK
 (priority,
- "This macro specifies the order in which modes for @var{entity}\n\
+ "This hook specifies the order in which modes for @var{entity}\n\
 are processed. 0 is the highest priority,\n\
 @code{NUM_MODES_FOR_MODE_SWITCHING[@var{entity}] - 1} the lowest.\n\
-The value of the macro should be an integer designating a mode\n\
+The hook returns an integer designating a mode\n\
 for @var{entity}.  For any fixed @var{entity}, @code{mode_priority}\n\
 (@var{entity}, @var{n}) shall be a bijection in 0 @dots{}\n\
 @code{num_modes_for_mode_switching[@var{entity}] - 1}.",
-- 
2.25.1

[PATCH 02/12] mode-switching: Add note problem

[Richard Sandiford]

optimize_mode_switching uses REG_DEAD notes to track register
liveness, but it failed to tell DF to calculate up-to-date notes.

Noticed by inspection.  I don't have a testcase that fails
because of this.

gcc/
	* mode-switching.cc (optimize_mode_switching): Call
	df_note_add_problem.
---

I was tempted to apply this as obvious, but wasn't sure if I was
missing something.

 gcc/mode-switching.cc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index c3e4d24de9b..8577069bde1 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -541,6 +541,7 @@ optimize_mode_switching (void)
       pre_exit = create_pre_exit (n_entities, entity_map, num_modes);
     }
 
+  df_note_add_problem ();
   df_analyze ();
 
   /* Create the bitmap vectors.  */
-- 
2.25.1

[PATCH 03/12] mode-switching: Avoid quadractic list operation

[Richard Sandiford]

add_seginfo chained insn information to the end of a list
by starting at the head of the list.  This patch avoids the
quadraticness by keeping track of the tail pointer.

gcc/
	* mode-switching.cc (add_seginfo): Replace head pointer with
	a pointer to the tail pointer.
	(optimize_mode_switching): Update calls accordingly.
---
 gcc/mode-switching.cc | 24 ++++++++----------------
 1 file changed, 8 insertions(+), 16 deletions(-)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 8577069bde1..bebe89d5fd2 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -162,23 +162,14 @@ new_seginfo (int mode, rtx_insn *insn, const HARD_REG_SET &regs_live)
 }
 
 /* Add a seginfo element to the end of a list.
-   HEAD is a pointer to the list beginning.
+   TAIL is a pointer to the list's null terminator.
    INFO is the structure to be linked in.  */
 
 static void
-add_seginfo (struct bb_info *head, struct seginfo *info)
+add_seginfo (struct seginfo ***tail_ptr, struct seginfo *info)
 {
-  struct seginfo *ptr;
-
-  if (head->seginfo == NULL)
-    head->seginfo = info;
-  else
-    {
-      ptr = head->seginfo;
-      while (ptr->next != NULL)
-	ptr = ptr->next;
-      ptr->next = info;
-    }
+  **tail_ptr = info;
+  *tail_ptr = &info->next;
 }
 
 /* Record in LIVE that register REG died.  */
@@ -574,6 +565,7 @@ optimize_mode_switching (void)
 	 Also compute the initial transparency settings.  */
       FOR_EACH_BB_FN (bb, cfun)
 	{
+	  struct seginfo **tail_ptr = &info[bb->index].seginfo;
 	  struct seginfo *ptr;
 	  int last_mode = no_mode;
 	  bool any_set_required = false;
@@ -599,7 +591,7 @@ optimize_mode_switching (void)
 		if (ins_pos != BB_END (bb))
 		  ins_pos = NEXT_INSN (ins_pos);
 		ptr = new_seginfo (no_mode, ins_pos, live_now);
-		add_seginfo (info + bb->index, ptr);
+		add_seginfo (&tail_ptr, ptr);
 		for (i = 0; i < no_mode; i++)
 		  clear_mode_bit (transp[bb->index], j, i);
 	      }
@@ -617,7 +609,7 @@ optimize_mode_switching (void)
 		      any_set_required = true;
 		      last_mode = mode;
 		      ptr = new_seginfo (mode, insn, live_now);
-		      add_seginfo (info + bb->index, ptr);
+		      add_seginfo (&tail_ptr, ptr);
 		      for (i = 0; i < no_mode; i++)
 			clear_mode_bit (transp[bb->index], j, i);
 		    }
@@ -646,7 +638,7 @@ optimize_mode_switching (void)
 	  if (!any_set_required)
 	    {
 	      ptr = new_seginfo (no_mode, BB_END (bb), live_now);
-	      add_seginfo (info + bb->index, ptr);
+	      add_seginfo (&tail_ptr, ptr);
 	      if (last_mode != no_mode)
 		for (i = 0; i < no_mode; i++)
 		  clear_mode_bit (transp[bb->index], j, i);
-- 
2.25.1

[PATCH 04/12] mode-switching: Fix the mode passed to the emit hook

[Richard Sandiford]

optimize_mode_switching passes an entity's current mode (if known)
to the emit hook.  However, the mode that it passed ignored the
effect of the after hook.  Instead, the mode for the first emit
call in a block was taken from the incoming mode, whereas the
mode for each subsequent emit call was taken from the result
of the previous call.

The previous pass through the insns already calculated the
correct mode, so this patch records it in the seginfo structure.
(There was a 32-bit hole on 64-bit hosts, so this doesn't increase
the size of the structure for them.)

gcc/
	* mode-switching.cc (seginfo): Add a prev_mode field.
	(new_seginfo): Take and initialize the prev_mode.
	(optimize_mode_switching): Update calls accordingly.
	Use the recorded modes during the emit phase, rather than
	computing one on the fly.
---
 gcc/mode-switching.cc | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index bebe89d5fd2..12ddbd6adfa 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -68,6 +68,7 @@ along with GCC; see the file COPYING3.  If not see
    NEXT is the next insn in the same basic block.  */
 struct seginfo
 {
+  int prev_mode;
   int mode;
   rtx_insn *insn_ptr;
   struct seginfo *next;
@@ -140,20 +141,22 @@ commit_mode_sets (struct edge_list *edge_list, int e, struct bb_info *info)
   return need_commit;
 }
 
-/* Allocate a new BBINFO structure, initialized with the MODE, INSN,
-   and REGS_LIVE parameters.
+/* Allocate a new BBINFO structure, initialized with the PREV_MODE, MODE,
+   INSN, and REGS_LIVE parameters.
    INSN may not be a NOTE_INSN_BASIC_BLOCK, unless it is an empty
    basic block; that allows us later to insert instructions in a FIFO-like
    manner.  */
 
 static struct seginfo *
-new_seginfo (int mode, rtx_insn *insn, const HARD_REG_SET &regs_live)
+new_seginfo (int prev_mode, int mode, rtx_insn *insn,
+	     const HARD_REG_SET &regs_live)
 {
   struct seginfo *ptr;
 
   gcc_assert (!NOTE_INSN_BASIC_BLOCK_P (insn)
 	      || insn == BB_END (NOTE_BASIC_BLOCK (insn)));
   ptr = XNEW (struct seginfo);
+  ptr->prev_mode = prev_mode;
   ptr->mode = mode;
   ptr->insn_ptr = insn;
   ptr->next = NULL;
@@ -590,7 +593,7 @@ optimize_mode_switching (void)
 		gcc_assert (NOTE_INSN_BASIC_BLOCK_P (ins_pos));
 		if (ins_pos != BB_END (bb))
 		  ins_pos = NEXT_INSN (ins_pos);
-		ptr = new_seginfo (no_mode, ins_pos, live_now);
+		ptr = new_seginfo (no_mode, no_mode, ins_pos, live_now);
 		add_seginfo (&tail_ptr, ptr);
 		for (i = 0; i < no_mode; i++)
 		  clear_mode_bit (transp[bb->index], j, i);
@@ -606,12 +609,12 @@ optimize_mode_switching (void)
 
 		  if (mode != no_mode && mode != last_mode)
 		    {
-		      any_set_required = true;
-		      last_mode = mode;
-		      ptr = new_seginfo (mode, insn, live_now);
+		      ptr = new_seginfo (last_mode, mode, insn, live_now);
 		      add_seginfo (&tail_ptr, ptr);
 		      for (i = 0; i < no_mode; i++)
 			clear_mode_bit (transp[bb->index], j, i);
+		      any_set_required = true;
+		      last_mode = mode;
 		    }
 
 		  if (targetm.mode_switching.after)
@@ -637,7 +640,7 @@ optimize_mode_switching (void)
 	     mark the block as nontransparent.  */
 	  if (!any_set_required)
 	    {
-	      ptr = new_seginfo (no_mode, BB_END (bb), live_now);
+	      ptr = new_seginfo (last_mode, no_mode, BB_END (bb), live_now);
 	      add_seginfo (&tail_ptr, ptr);
 	      if (last_mode != no_mode)
 		for (i = 0; i < no_mode; i++)
@@ -778,9 +781,9 @@ optimize_mode_switching (void)
       FOR_EACH_BB_FN (bb, cfun)
 	{
 	  struct seginfo *ptr, *next;
-	  int cur_mode = bb_info[j][bb->index].mode_in;
+	  struct seginfo *first = bb_info[j][bb->index].seginfo;
 
-	  for (ptr = bb_info[j][bb->index].seginfo; ptr; ptr = next)
+	  for (ptr = first; ptr; ptr = next)
 	    {
 	      next = ptr->next;
 	      if (ptr->mode != no_mode)
@@ -790,14 +793,15 @@ optimize_mode_switching (void)
 		  rtl_profile_for_bb (bb);
 		  start_sequence ();
 
+		  int cur_mode = (ptr == first && ptr->prev_mode == no_mode
+				  ? bb_info[j][bb->index].mode_in
+				  : ptr->prev_mode);
+
 		  targetm.mode_switching.emit (entity_map[j], ptr->mode,
 					       cur_mode, ptr->regs_live);
 		  mode_set = get_insns ();
 		  end_sequence ();
 
-		  /* modes kill each other inside a basic block.  */
-		  cur_mode = ptr->mode;
-
 		  /* Insert MODE_SET only if it is nonempty.  */
 		  if (mode_set != NULL_RTX)
 		    {
-- 
2.25.1

[PATCH 05/12] mode-switching: Simplify recording of transparency

[Richard Sandiford]

For a given block, an entity is either transparent for
all modes or for none.  Each update to the transparency set
therefore used a loop like:

		for (i = 0; i < no_mode; i++)
		  clear_mode_bit (transp[bb->index], j, i);

This patch instead starts out with a bit-per-block bitmap
and updates the main bitmap at the end.

This isn't much of a simplification on its own.  The main
purpose is to simplify later patches.

gcc/
	* mode-switching.cc (optimize_mode_switching): Initially
	compute transparency in a bit-per-block bitmap.
---
 gcc/mode-switching.cc | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 12ddbd6adfa..03dd4c1ebe4 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -556,6 +556,8 @@ optimize_mode_switching (void)
   bitmap_vector_clear (antic, last_basic_block_for_fn (cfun));
   bitmap_vector_clear (comp, last_basic_block_for_fn (cfun));
 
+  auto_sbitmap transp_all (last_basic_block_for_fn (cfun));
+
   for (j = n_entities - 1; j >= 0; j--)
     {
       int e = entity_map[j];
@@ -563,6 +565,8 @@ optimize_mode_switching (void)
       struct bb_info *info = bb_info[j];
       rtx_insn *insn;
 
+      bitmap_ones (transp_all);
+
       /* Determine what the first use (if any) need for a mode of entity E is.
 	 This will be the mode that is anticipatable for this block.
 	 Also compute the initial transparency settings.  */
@@ -595,8 +599,7 @@ optimize_mode_switching (void)
 		  ins_pos = NEXT_INSN (ins_pos);
 		ptr = new_seginfo (no_mode, no_mode, ins_pos, live_now);
 		add_seginfo (&tail_ptr, ptr);
-		for (i = 0; i < no_mode; i++)
-		  clear_mode_bit (transp[bb->index], j, i);
+		bitmap_clear_bit (transp_all, bb->index);
 	      }
 	  }
 
@@ -611,8 +614,7 @@ optimize_mode_switching (void)
 		    {
 		      ptr = new_seginfo (last_mode, mode, insn, live_now);
 		      add_seginfo (&tail_ptr, ptr);
-		      for (i = 0; i < no_mode; i++)
-			clear_mode_bit (transp[bb->index], j, i);
+		      bitmap_clear_bit (transp_all, bb->index);
 		      any_set_required = true;
 		      last_mode = mode;
 		    }
@@ -643,8 +645,7 @@ optimize_mode_switching (void)
 	      ptr = new_seginfo (last_mode, no_mode, BB_END (bb), live_now);
 	      add_seginfo (&tail_ptr, ptr);
 	      if (last_mode != no_mode)
-		for (i = 0; i < no_mode; i++)
-		  clear_mode_bit (transp[bb->index], j, i);
+		bitmap_clear_bit (transp_all, bb->index);
 	    }
 	}
       if (targetm.mode_switching.entry && targetm.mode_switching.exit)
@@ -667,8 +668,7 @@ optimize_mode_switching (void)
 		 an extra check in make_preds_opaque.  We also
 		 need this to avoid confusing pre_edge_lcm when
 		 antic is cleared but transp and comp are set.  */
-	      for (i = 0; i < no_mode; i++)
-		clear_mode_bit (transp[bb->index], j, i);
+	      bitmap_clear_bit (transp_all, bb->index);
 
 	      /* Insert a fake computing definition of MODE into entry
 		 blocks which compute no mode. This represents the mode on
@@ -688,6 +688,9 @@ optimize_mode_switching (void)
 
 	  FOR_EACH_BB_FN (bb, cfun)
 	    {
+	      if (!bitmap_bit_p (transp_all, bb->index))
+		clear_mode_bit (transp[bb->index], j, m);
+
 	      if (info[bb->index].seginfo->mode == m)
 		set_mode_bit (antic[bb->index], j, m);
 
-- 
2.25.1

[PATCH 06/12] mode-switching: Tweak entry/exit handling

[Richard Sandiford]

An entity isn't transparent in a block that requires a specific mode.
optimize_mode_switching took that into account for normal insns,
but didn't for the exit block.  Later patches misbehaved because
of this.

In contrast, an entity was correctly marked as non-transparent
in the entry block, but the reasoning seemed a bit convoluted.
It also referred to a function that no longer exists.
Since KILL = ~TRANSP, the entity is by definition not transparent
in a block that defines the entity, so I think we can make it so
without comment.

Finally, the exit handling was nested in the entry handling,
but that doesn't seem necessary.  A target could say that an
entity is undefined on entry but must be defined on return,
on a "be liberal in what you accept, be conservative in what
you do" principle.

gcc/
	* mode-switching.cc (optimize_mode_switching): Mark the exit
	block as nontransparent if it requires a specific mode.
	Handle the entry and exit mode as sibling rather than nested
	concepts.  Remove outdated comment.
---
 gcc/mode-switching.cc | 34 +++++++++++++++-------------------
 1 file changed, 15 insertions(+), 19 deletions(-)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 03dd4c1ebe4..1145350ca26 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -650,34 +650,30 @@ optimize_mode_switching (void)
 	}
       if (targetm.mode_switching.entry && targetm.mode_switching.exit)
 	{
-	  int mode = targetm.mode_switching.entry (e);
-
 	  info[post_entry->index].mode_out =
 	    info[post_entry->index].mode_in = no_mode;
-	  if (pre_exit)
-	    {
-	      info[pre_exit->index].mode_out =
-		info[pre_exit->index].mode_in = no_mode;
-	    }
 
+	  int mode = targetm.mode_switching.entry (e);
 	  if (mode != no_mode)
 	    {
-	      bb = post_entry;
-
-	      /* By always making this nontransparent, we save
-		 an extra check in make_preds_opaque.  We also
-		 need this to avoid confusing pre_edge_lcm when
-		 antic is cleared but transp and comp are set.  */
-	      bitmap_clear_bit (transp_all, bb->index);
-
 	      /* Insert a fake computing definition of MODE into entry
 		 blocks which compute no mode. This represents the mode on
 		 entry.  */
-	      info[bb->index].computing = mode;
+	      info[post_entry->index].computing = mode;
+	      bitmap_clear_bit (transp_all, post_entry->index);
+	    }
 
-	      if (pre_exit)
-		info[pre_exit->index].seginfo->mode =
-		  targetm.mode_switching.exit (e);
+	  if (pre_exit)
+	    {
+	      info[pre_exit->index].mode_out =
+		info[pre_exit->index].mode_in = no_mode;
+
+	      int mode = targetm.mode_switching.exit (e);
+	      if (mode != no_mode)
+		{
+		  info[pre_exit->index].seginfo->mode = mode;
+		  bitmap_clear_bit (transp_all, pre_exit->index);
+		}
 	    }
 	}
 
-- 
2.25.1

[PATCH 07/12] mode-switching: Allow targets to set the mode for EH handlers

[Richard Sandiford]

The mode-switching pass already had hooks to say what mode
an entity is in on entry to a function and what mode it must
be in on return.  For SME, we also want to say what mode an
entity is guaranteed to be in on entry to an exception handler.

gcc/
	* target.def (mode_switching.eh_handler): New hook.
	* doc/tm.texi.in (TARGET_MODE_EH_HANDLER): New @hook.
	* doc/tm.texi: Regenerate.
	* mode-switching.cc (optimize_mode_switching): Use eh_handler
	to get the mode on entry to an exception handler.
---
 gcc/doc/tm.texi       | 6 ++++++
 gcc/doc/tm.texi.in    | 2 ++
 gcc/mode-switching.cc | 5 ++++-
 gcc/target.def        | 7 +++++++
 4 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index 759331a2c96..1a825c5004e 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10455,6 +10455,12 @@ If @code{TARGET_MODE_EXIT} is defined then @code{TARGET_MODE_ENTRY}
 must be defined.
 @end deftypefn
 
+@deftypefn {Target Hook} int TARGET_MODE_EH_HANDLER (int @var{entity})
+If this hook is defined, it should return the mode that @var{entity} is
+guaranteed to be in on entry to an exception handler, or the number of modes
+if there is no such guarantee.
+@end deftypefn
+
 @deftypefn {Target Hook} int TARGET_MODE_PRIORITY (int @var{entity}, int @var{n})
 This hook specifies the order in which modes for @var{entity}
 are processed. 0 is the highest priority,
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index a7b7aa289d8..5360c1bb2d8 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -6979,6 +6979,8 @@ mode or ``no mode'', depending on context.
 
 @hook TARGET_MODE_EXIT
 
+@hook TARGET_MODE_EH_HANDLER
+
 @hook TARGET_MODE_PRIORITY
 
 @node Target Attributes
diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 1145350ca26..b8a887d81f7 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -597,7 +597,10 @@ optimize_mode_switching (void)
 		gcc_assert (NOTE_INSN_BASIC_BLOCK_P (ins_pos));
 		if (ins_pos != BB_END (bb))
 		  ins_pos = NEXT_INSN (ins_pos);
-		ptr = new_seginfo (no_mode, no_mode, ins_pos, live_now);
+		if (bb_has_eh_pred (bb)
+		    && targetm.mode_switching.eh_handler)
+		  last_mode = targetm.mode_switching.eh_handler (e);
+		ptr = new_seginfo (no_mode, last_mode, ins_pos, live_now);
 		add_seginfo (&tail_ptr, ptr);
 		bitmap_clear_bit (transp_all, bb->index);
 	      }
diff --git a/gcc/target.def b/gcc/target.def
index 3dae33522f1..a70275b8abd 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7070,6 +7070,13 @@ If @code{TARGET_MODE_EXIT} is defined then @code{TARGET_MODE_ENTRY}\n\
 must be defined.",
  int, (int entity), NULL)
 
+DEFHOOK
+(eh_handler,
+ "If this hook is defined, it should return the mode that @var{entity} is\n\
+guaranteed to be in on entry to an exception handler, or the number of modes\n\
+if there is no such guarantee.",
+ int, (int entity), NULL)
+
 DEFHOOK
 (priority,
  "This hook specifies the order in which modes for @var{entity}\n\
-- 
2.25.1

[PATCH 08/12] mode-switching: Pass set of live registers to the needed hook

[Richard Sandiford]

The emit hook already takes the set of live hard registers as input.
This patch passes it to the needed hook too.  SME uses this to
optimise the mode choice based on whether state is live or dead.

The main caller already had access to the required info, but the
special handling of return values did not.

gcc/
	* target.def (mode_switching.needed): Add a regs_live parameter.
	* doc/tm.texi: Regenerate.
	* config/epiphany/epiphany-protos.h (epiphany_mode_needed): Update
	accordingly.
	* config/epiphany/epiphany.cc (epiphany_mode_needed): Likewise.
	* config/epiphany/mode-switch-use.cc (insert_uses): Likewise.
	* config/i386/i386.cc (ix86_mode_needed): Likewise.
	* config/riscv/riscv.cc (riscv_mode_needed): Likewise.
	* config/sh/sh.cc (sh_mode_needed): Likewise.
	* mode-switching.cc (optimize_mode_switching): Likewise.
	(create_pre_exit): Likewise, using the DF simulate functions
	to calculate the required information.
---
 gcc/config/epiphany/epiphany-protos.h  |  4 +++-
 gcc/config/epiphany/epiphany.cc        |  2 +-
 gcc/config/epiphany/mode-switch-use.cc |  2 +-
 gcc/config/i386/i386.cc                |  2 +-
 gcc/config/riscv/riscv.cc              |  2 +-
 gcc/config/sh/sh.cc                    |  4 ++--
 gcc/doc/tm.texi                        |  5 +++--
 gcc/mode-switching.cc                  | 14 ++++++++++++--
 gcc/target.def                         |  5 +++--
 9 files changed, 27 insertions(+), 13 deletions(-)

diff --git a/gcc/config/epiphany/epiphany-protos.h b/gcc/config/epiphany/epiphany-protos.h
index 72c141c1a6d..ef49a1e06a4 100644
--- a/gcc/config/epiphany/epiphany-protos.h
+++ b/gcc/config/epiphany/epiphany-protos.h
@@ -44,7 +44,9 @@ extern void emit_set_fp_mode (int entity, int mode, int prev_mode,
 #endif
 extern void epiphany_insert_mode_switch_use (rtx_insn *insn, int, int);
 extern void epiphany_expand_set_fp_mode (rtx *operands);
-extern int epiphany_mode_needed (int entity, rtx_insn *insn);
+#ifdef HARD_CONST
+extern int epiphany_mode_needed (int entity, rtx_insn *insn, HARD_REG_SET);
+#endif
 extern int epiphany_mode_after (int entity, int last_mode, rtx_insn *insn);
 extern bool epiphany_epilogue_uses (int regno);
 extern bool epiphany_optimize_mode_switching (int entity);
diff --git a/gcc/config/epiphany/epiphany.cc b/gcc/config/epiphany/epiphany.cc
index a5460dbf97f..60a9b49d8a4 100644
--- a/gcc/config/epiphany/epiphany.cc
+++ b/gcc/config/epiphany/epiphany.cc
@@ -2400,7 +2400,7 @@ epiphany_mode_priority (int entity, int priority)
 }
 
 int
-epiphany_mode_needed (int entity, rtx_insn *insn)
+epiphany_mode_needed (int entity, rtx_insn *insn, HARD_REG_SET)
 {
   enum attr_fp_mode mode;
 
diff --git a/gcc/config/epiphany/mode-switch-use.cc b/gcc/config/epiphany/mode-switch-use.cc
index 71530612658..183b9b7a394 100644
--- a/gcc/config/epiphany/mode-switch-use.cc
+++ b/gcc/config/epiphany/mode-switch-use.cc
@@ -58,7 +58,7 @@ insert_uses (void)
 	{
 	  if (!INSN_P (insn))
 	    continue;
-	  mode = epiphany_mode_needed (e, insn);
+	  mode = epiphany_mode_needed (e, insn, {});
 	  if (mode == no_mode)
 	    continue;
 	  if (target_insert_mode_switch_use)
diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index fdc9362cf5b..7a5a9a966e8 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -15061,7 +15061,7 @@ ix86_i387_mode_needed (int entity, rtx_insn *insn)
    prior to the execution of insn.  */
 
 static int
-ix86_mode_needed (int entity, rtx_insn *insn)
+ix86_mode_needed (int entity, rtx_insn *insn, HARD_REG_SET)
 {
   switch (entity)
     {
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 08ff05dcc3f..f915de7ed56 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -9413,7 +9413,7 @@ riscv_frm_mode_needed (rtx_insn *cur_insn, int code)
    prior to the execution of insn.  */
 
 static int
-riscv_mode_needed (int entity, rtx_insn *insn)
+riscv_mode_needed (int entity, rtx_insn *insn, HARD_REG_SET)
 {
   int code = recog_memoized (insn);
 
diff --git a/gcc/config/sh/sh.cc b/gcc/config/sh/sh.cc
index 294faf7c0c3..c363490e852 100644
--- a/gcc/config/sh/sh.cc
+++ b/gcc/config/sh/sh.cc
@@ -195,7 +195,7 @@ static int calc_live_regs (HARD_REG_SET *);
 static HOST_WIDE_INT rounded_frame_size (int);
 static bool sh_frame_pointer_required (void);
 static void sh_emit_mode_set (int, int, int, HARD_REG_SET);
-static int sh_mode_needed (int, rtx_insn *);
+static int sh_mode_needed (int, rtx_insn *, HARD_REG_SET);
 static int sh_mode_after (int, int, rtx_insn *);
 static int sh_mode_entry (int);
 static int sh_mode_exit (int);
@@ -12531,7 +12531,7 @@ sh_emit_mode_set (int entity ATTRIBUTE_UNUSED, int mode,
 }
 
 static int
-sh_mode_needed (int entity ATTRIBUTE_UNUSED, rtx_insn *insn)
+sh_mode_needed (int entity ATTRIBUTE_UNUSED, rtx_insn *insn, HARD_REG_SET)
 {
   return recog_memoized (insn) >= 0  ? get_attr_fp_mode (insn) : FP_MODE_NONE;
 }
diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index 1a825c5004e..144b3f88c37 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10414,12 +10414,13 @@ known.  Sets of a lower numbered entity will be emitted before
 sets of a higher numbered entity to a mode of the same or lower priority.
 @end deftypefn
 
-@deftypefn {Target Hook} int TARGET_MODE_NEEDED (int @var{entity}, rtx_insn *@var{insn})
+@deftypefn {Target Hook} int TARGET_MODE_NEEDED (int @var{entity}, rtx_insn *@var{insn}, HARD_REG_SET @var{regs_live})
 @var{entity} is an integer specifying a mode-switched entity.
 If @code{OPTIMIZE_MODE_SWITCHING} is defined, you must define this hook
 to return the mode that @var{entity} must be switched into prior to the
 execution of @var{insn}, or the number of modes if @var{insn} has no
-such requirement.
+such requirement.  @var{regs_live} contains the set of hard registers
+that are live before @var{insn}.
 @end deftypefn
 
 @deftypefn {Target Hook} int TARGET_MODE_AFTER (int @var{entity}, int @var{mode}, rtx_insn *@var{insn})
diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index b8a887d81f7..c5fe90ba449 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -254,6 +254,9 @@ create_pre_exit (int n_entities, int *entity_map, const int *num_modes)
 	    && GET_CODE (PATTERN (last_insn)) == USE
 	    && GET_CODE ((ret_reg = XEXP (PATTERN (last_insn), 0))) == REG)
 	  {
+	    auto_bitmap live;
+	    df_simulate_initialize_backwards (src_bb, live);
+
 	    int ret_start = REGNO (ret_reg);
 	    int nregs = REG_NREGS (ret_reg);
 	    int ret_end = ret_start + nregs;
@@ -262,6 +265,8 @@ create_pre_exit (int n_entities, int *entity_map, const int *num_modes)
 	    bool forced_late_switch = false;
 	    rtx_insn *before_return_copy;
 
+	    df_simulate_one_insn_backwards (src_bb, last_insn, live);
+
 	    do
 	      {
 		rtx_insn *return_copy = PREV_INSN (last_insn);
@@ -269,6 +274,8 @@ create_pre_exit (int n_entities, int *entity_map, const int *num_modes)
 		int copy_start, copy_num;
 		int j;
 
+		df_simulate_one_insn_backwards (src_bb, return_copy, live);
+
 		if (NONDEBUG_INSN_P (return_copy))
 		  {
 		    /* When using SJLJ exceptions, the call to the
@@ -368,11 +375,14 @@ create_pre_exit (int n_entities, int *entity_map, const int *num_modes)
 		       the case for floating point on SH4 - then it might
 		       be set by an arithmetic operation that needs a
 		       different mode than the exit block.  */
+		    HARD_REG_SET hard_regs_live;
+		    REG_SET_TO_HARD_REG_SET (hard_regs_live, live);
 		    for (j = n_entities - 1; j >= 0; j--)
 		      {
 			int e = entity_map[j];
 			int mode =
-			  targetm.mode_switching.needed (e, return_copy);
+			  targetm.mode_switching.needed (e, return_copy,
+							 hard_regs_live);
 
 			if (mode != num_modes[e]
 			    && mode != targetm.mode_switching.exit (e))
@@ -610,7 +620,7 @@ optimize_mode_switching (void)
 	    {
 	      if (INSN_P (insn))
 		{
-		  int mode = targetm.mode_switching.needed (e, insn);
+		  int mode = targetm.mode_switching.needed (e, insn, live_now);
 		  rtx link;
 
 		  if (mode != no_mode && mode != last_mode)
diff --git a/gcc/target.def b/gcc/target.def
index a70275b8abd..50bad184aca 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7031,8 +7031,9 @@ DEFHOOK
 If @code{OPTIMIZE_MODE_SWITCHING} is defined, you must define this hook\n\
 to return the mode that @var{entity} must be switched into prior to the\n\
 execution of @var{insn}, or the number of modes if @var{insn} has no\n\
-such requirement.",
- int, (int entity, rtx_insn *insn), NULL)
+such requirement.  @var{regs_live} contains the set of hard registers\n\
+that are live before @var{insn}.",
+ int, (int entity, rtx_insn *insn, HARD_REG_SET regs_live), NULL)
 
 DEFHOOK
 (after,
-- 
2.25.1

[PATCH 09/12] mode-switching: Pass the set of live registers to the after hook

[Richard Sandiford]

This patch passes the set of live hard registers to the after hook,
like the previous one did for the needed hook.

gcc/
	* target.def (mode_switching.after): Add a regs_live parameter.
	* doc/tm.texi: Regenerate.
	* config/epiphany/epiphany-protos.h (epiphany_mode_after): Update
	accordingly.
	* config/epiphany/epiphany.cc (epiphany_mode_needed): Likewise.
	(epiphany_mode_after): Likewise.
	* config/i386/i386.cc (ix86_mode_after): Likewise.
	* config/riscv/riscv.cc (riscv_mode_after): Likewise.
	* config/sh/sh.cc (sh_mode_after): Likewise.
	* mode-switching.cc (optimize_mode_switching): Likewise.
---
 gcc/config/epiphany/epiphany-protos.h | 3 ++-
 gcc/config/epiphany/epiphany.cc       | 5 +++--
 gcc/config/i386/i386.cc               | 2 +-
 gcc/config/riscv/riscv.cc             | 2 +-
 gcc/config/sh/sh.cc                   | 5 +++--
 gcc/doc/tm.texi                       | 4 +++-
 gcc/mode-switching.cc                 | 8 ++++----
 gcc/target.def                        | 4 +++-
 8 files changed, 20 insertions(+), 13 deletions(-)

diff --git a/gcc/config/epiphany/epiphany-protos.h b/gcc/config/epiphany/epiphany-protos.h
index ef49a1e06a4..ff8987ea99e 100644
--- a/gcc/config/epiphany/epiphany-protos.h
+++ b/gcc/config/epiphany/epiphany-protos.h
@@ -46,8 +46,9 @@ extern void epiphany_insert_mode_switch_use (rtx_insn *insn, int, int);
 extern void epiphany_expand_set_fp_mode (rtx *operands);
 #ifdef HARD_CONST
 extern int epiphany_mode_needed (int entity, rtx_insn *insn, HARD_REG_SET);
+extern int epiphany_mode_after (int entity, int last_mode, rtx_insn *insn,
+				HARD_REG_SET);
 #endif
-extern int epiphany_mode_after (int entity, int last_mode, rtx_insn *insn);
 extern bool epiphany_epilogue_uses (int regno);
 extern bool epiphany_optimize_mode_switching (int entity);
 extern bool epiphany_is_interrupt_p (tree);
diff --git a/gcc/config/epiphany/epiphany.cc b/gcc/config/epiphany/epiphany.cc
index 60a9b49d8a4..68e748c688e 100644
--- a/gcc/config/epiphany/epiphany.cc
+++ b/gcc/config/epiphany/epiphany.cc
@@ -2437,7 +2437,7 @@ epiphany_mode_needed (int entity, rtx_insn *insn, HARD_REG_SET)
     return 2;
   case EPIPHANY_MSW_ENTITY_ROUND_KNOWN:
     if (recog_memoized (insn) == CODE_FOR_set_fp_mode)
-      mode = (enum attr_fp_mode) epiphany_mode_after (entity, mode, insn);
+      mode = (enum attr_fp_mode) epiphany_mode_after (entity, mode, insn, {});
     /* Fall through.  */
   case EPIPHANY_MSW_ENTITY_NEAREST:
   case EPIPHANY_MSW_ENTITY_TRUNC:
@@ -2498,7 +2498,8 @@ epiphany_mode_entry_exit (int entity, bool exit)
 }
 
 int
-epiphany_mode_after (int entity, int last_mode, rtx_insn *insn)
+epiphany_mode_after (int entity, int last_mode, rtx_insn *insn,
+		     HARD_REG_SET)
 {
   /* We have too few call-saved registers to hope to keep the masks across
      calls.  */
diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index 7a5a9a966e8..7b72aabf0da 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -15110,7 +15110,7 @@ ix86_avx_u128_mode_after (int mode, rtx_insn *insn)
 /* Return the mode that an insn results in.  */
 
 static int
-ix86_mode_after (int entity, int mode, rtx_insn *insn)
+ix86_mode_after (int entity, int mode, rtx_insn *insn, HARD_REG_SET)
 {
   switch (entity)
     {
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index f915de7ed56..e36b5fb9bd0 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -9514,7 +9514,7 @@ riscv_frm_mode_after (rtx_insn *insn, int mode)
 /* Return the mode that an insn results in.  */
 
 static int
-riscv_mode_after (int entity, int mode, rtx_insn *insn)
+riscv_mode_after (int entity, int mode, rtx_insn *insn, HARD_REG_SET)
 {
   switch (entity)
     {
diff --git a/gcc/config/sh/sh.cc b/gcc/config/sh/sh.cc
index c363490e852..6ec2eecf754 100644
--- a/gcc/config/sh/sh.cc
+++ b/gcc/config/sh/sh.cc
@@ -196,7 +196,7 @@ static HOST_WIDE_INT rounded_frame_size (int);
 static bool sh_frame_pointer_required (void);
 static void sh_emit_mode_set (int, int, int, HARD_REG_SET);
 static int sh_mode_needed (int, rtx_insn *, HARD_REG_SET);
-static int sh_mode_after (int, int, rtx_insn *);
+static int sh_mode_after (int, int, rtx_insn *, HARD_REG_SET);
 static int sh_mode_entry (int);
 static int sh_mode_exit (int);
 static int sh_mode_priority (int entity, int n);
@@ -12537,7 +12537,8 @@ sh_mode_needed (int entity ATTRIBUTE_UNUSED, rtx_insn *insn, HARD_REG_SET)
 }
 
 static int
-sh_mode_after (int entity ATTRIBUTE_UNUSED, int mode, rtx_insn *insn)
+sh_mode_after (int entity ATTRIBUTE_UNUSED, int mode, rtx_insn *insn,
+	       HARD_REG_SET)
 {
   if (TARGET_HITACHI && recog_memoized (insn) >= 0 &&
       get_attr_fp_set (insn) != FP_SET_NONE)
diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index 144b3f88c37..b730b5bf658 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10423,12 +10423,14 @@ such requirement.  @var{regs_live} contains the set of hard registers
 that are live before @var{insn}.
 @end deftypefn
 
-@deftypefn {Target Hook} int TARGET_MODE_AFTER (int @var{entity}, int @var{mode}, rtx_insn *@var{insn})
+@deftypefn {Target Hook} int TARGET_MODE_AFTER (int @var{entity}, int @var{mode}, rtx_insn *@var{insn}, HARD_REG_SET @var{regs_live})
 @var{entity} is an integer specifying a mode-switched entity.
 If this hook is defined, it is evaluated for every @var{insn} during mode
 switching.  It returns the mode that @var{entity} is in after @var{insn}
 has been executed.  @var{mode} is the mode that @var{entity} was in
 before @var{insn} was executed, taking account of @var{TARGET_MODE_NEEDED}.
+@var{regs_live} is the set of hard registers that are live after @var{insn}
+has been executed.
 
 @var{mode} is equal to the number of modes defined for @var{entity}
 if the mode before @var{insn} is unknown.  The hook should likewise return
diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index c5fe90ba449..7a5c4993d65 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -632,10 +632,6 @@ optimize_mode_switching (void)
 		      last_mode = mode;
 		    }
 
-		  if (targetm.mode_switching.after)
-		    last_mode = targetm.mode_switching.after (e, last_mode,
-							      insn);
-
 		  /* Update LIVE_NOW.  */
 		  for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
 		    if (REG_NOTE_KIND (link) == REG_DEAD)
@@ -645,6 +641,10 @@ optimize_mode_switching (void)
 		  for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
 		    if (REG_NOTE_KIND (link) == REG_UNUSED)
 		      reg_dies (XEXP (link, 0), &live_now);
+
+		  if (targetm.mode_switching.after)
+		    last_mode = targetm.mode_switching.after (e, last_mode,
+							      insn, live_now);
 		}
 	    }
 
diff --git a/gcc/target.def b/gcc/target.def
index 50bad184aca..9b14c037d3f 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7042,6 +7042,8 @@ If this hook is defined, it is evaluated for every @var{insn} during mode\n\
 switching.  It returns the mode that @var{entity} is in after @var{insn}\n\
 has been executed.  @var{mode} is the mode that @var{entity} was in\n\
 before @var{insn} was executed, taking account of @var{TARGET_MODE_NEEDED}.\n\
+@var{regs_live} is the set of hard registers that are live after @var{insn}\n\
+has been executed.\n\
 \n\
 @var{mode} is equal to the number of modes defined for @var{entity}\n\
 if the mode before @var{insn} is unknown.  The hook should likewise return\n\
@@ -7049,7 +7051,7 @@ the number of modes if it does not know what mode @var{entity} has after\n\
 @var{insn}.\n\
 \n\
 Not defining the hook is equivalent to returning @var{mode}.",
- int, (int entity, int mode, rtx_insn *insn), NULL)
+ int, (int entity, int mode, rtx_insn *insn, HARD_REG_SET regs_live), NULL)
 
 DEFHOOK
 (entry,
-- 
2.25.1

[PATCH 10/12] mode-switching: Use 1-based edge aux fields

[Richard Sandiford]

The pass used the edge aux field to record which mode change
should happen on the edge, with -1 meaning "none".  It's more
convenient for later patches to leave aux zero for "none",
and use numbers based at 1 to record a change.

gcc/
	* mode-switching.cc (commit_mode_sets): Use 1-based edge aux values.
---
 gcc/mode-switching.cc | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 7a5c4993d65..1815b397dd0 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -106,10 +106,10 @@ commit_mode_sets (struct edge_list *edge_list, int e, struct bb_info *info)
   for (int ed = NUM_EDGES (edge_list) - 1; ed >= 0; ed--)
     {
       edge eg = INDEX_EDGE (edge_list, ed);
-      int mode;
 
-      if ((mode = (int)(intptr_t)(eg->aux)) != -1)
+      if (eg->aux)
 	{
+	  int mode = (int) (intptr_t) eg->aux - 1;
 	  HARD_REG_SET live_at_edge;
 	  basic_block src_bb = eg->src;
 	  int cur_mode = info[src_bb->index].mode_out;
@@ -728,14 +728,12 @@ optimize_mode_switching (void)
 	{
 	  edge eg = INDEX_EDGE (edge_list, ed);
 
-	  eg->aux = (void *)(intptr_t)-1;
-
 	  for (i = 0; i < no_mode; i++)
 	    {
 	      int m = targetm.mode_switching.priority (entity_map[j], i);
 	      if (mode_bit_p (insert[ed], j, m))
 		{
-		  eg->aux = (void *)(intptr_t)m;
+		  eg->aux = (void *) (intptr_t) (m + 1);
 		  break;
 		}
 	    }
-- 
2.25.1

[PATCH 11/12] mode-switching: Add a target-configurable confluence operator

[Richard Sandiford]

The mode-switching pass assumed that all of an entity's modes
were mutually exclusive.  However, the upcoming SME changes
have an entity with some overlapping modes, so that there is
sometimes a "superunion" mode that contains two given modes.
We can use this relationship to pass something more helpful than
"don't know" to the emit hook.

This patch adds a new hook that targets can use to specify
a mode confluence operator.

With mutually exclusive modes, it's possible to compute a block's
incoming and outgoing modes by looking at its availability sets.
With the confluence operator, we instead need to solve a full
dataflow problem.

However, when emitting a mode transition, the upcoming SME use of
mode-switching benefits from having as much information as possible
about the starting mode.  Calculating this information is definitely
worth the compile time.

The dataflow problem is written to work before and after the LCM
problem has been solved.  A later patch makes use of this.

While there (since git blame would ping me for the reindented code),
I used a lambda to avoid the cut-&-pasted loops.

gcc/
	* target.def (mode_switching.confluence): New hook.
	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
	* doc/tm.texi.in: Regenerate.
	* mode-switching.cc (confluence_info): New variable.
	(mode_confluence, forward_confluence_n, forward_transfer): New
	functions.
	(optimize_mode_switching): Use them to calculate mode_in when
	TARGET_MODE_CONFLUENCE is defined.
---
 gcc/doc/tm.texi       |  16 ++++
 gcc/doc/tm.texi.in    |   2 +
 gcc/mode-switching.cc | 179 +++++++++++++++++++++++++++++++++++-------
 gcc/target.def        |  17 ++++
 4 files changed, 186 insertions(+), 28 deletions(-)

diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index b730b5bf658..cd346538fe2 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10440,6 +10440,22 @@ the number of modes if it does not know what mode @var{entity} has after
 Not defining the hook is equivalent to returning @var{mode}.
 @end deftypefn
 
+@deftypefn {Target Hook} int TARGET_MODE_CONFLUENCE (int @var{entity}, int @var{mode1}, int @var{mode2})
+By default, the mode-switching pass assumes that a given entity's modes
+are mutually exclusive.  This means that the pass can only tell
+@code{TARGET_MODE_EMIT} about an entity's previous mode if all
+incoming paths of execution leave the entity in the same state.
+
+However, some entities might have overlapping, non-exclusive modes,
+so that it is sometimes possible to represent ``mode @var{mode1} or mode
+@var{mode2}'' with something more specific than ``mode not known''.
+If this is true for at least one entity, you should define this hook
+and make it return a mode that includes @var{mode1} and @var{mode2}
+as possibilities.  (The mode can include other possibilities too.)
+The hook should return the number of modes if no suitable mode exists
+for the given arguments.
+@end deftypefn
+
 @deftypefn {Target Hook} int TARGET_MODE_ENTRY (int @var{entity})
 If this hook is defined, it is evaluated for every @var{entity} that
 needs mode switching.  It should return the mode that @var{entity} is
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index 5360c1bb2d8..ae23241ea1c 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -6975,6 +6975,8 @@ mode or ``no mode'', depending on context.
 
 @hook TARGET_MODE_AFTER
 
+@hook TARGET_MODE_CONFLUENCE
+
 @hook TARGET_MODE_ENTRY
 
 @hook TARGET_MODE_EXIT
diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 1815b397dd0..87b23d2c050 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -485,6 +485,101 @@ create_pre_exit (int n_entities, int *entity_map, const int *num_modes)
   return pre_exit;
 }
 
+/* Return the confluence of modes MODE1 and MODE2 for entity ENTITY,
+   using NO_MODE to represent an unknown mode if nothing more precise
+   is available.  */
+
+int
+mode_confluence (int entity, int mode1, int mode2, int no_mode)
+{
+  if (mode1 == mode2)
+    return mode1;
+
+  if (mode1 != no_mode
+      && mode2 != no_mode
+      && targetm.mode_switching.confluence)
+    return targetm.mode_switching.confluence (entity, mode1, mode2);
+
+  return no_mode;
+}
+
+/* Information for the dataflow problems below.  */
+struct
+{
+  /* Information about each basic block, indexed by block id.  */
+  struct bb_info *bb_info;
+
+  /* The entity that we're processing.  */
+  int entity;
+
+  /* The number of modes defined for the entity, and thus the identifier
+     of the "don't know" mode.  */
+  int no_mode;
+} confluence_info;
+
+/* Propagate information about any mode change on edge E to the
+   destination block's mode_in.  Return true if something changed.
+
+   The mode_in and mode_out fields use no_mode + 1 to mean "not yet set".  */
+
+static bool
+forward_confluence_n (edge e)
+{
+  /* The entry and exit blocks have no useful mode information.  */
+  if (e->src->index == ENTRY_BLOCK || e->dest->index == EXIT_BLOCK)
+    return false;
+
+  /* We don't control mode changes across abnormal edges.  */
+  if (e->flags & EDGE_ABNORMAL)
+    return false;
+
+  /* E->aux is nonzero if we have computed the LCM problem and scheduled
+     E to change the mode to E->aux - 1.  Otherwise model the change
+     from the source to the destination.  */
+  struct bb_info *bb_info = confluence_info.bb_info;
+  int no_mode = confluence_info.no_mode;
+  int src_mode = bb_info[e->src->index].mode_out;
+  if (e->aux)
+    src_mode = (int) (intptr_t) e->aux - 1;
+  if (src_mode == no_mode + 1)
+    return false;
+
+  int dest_mode = bb_info[e->dest->index].mode_in;
+  if (dest_mode == no_mode + 1)
+    {
+      bb_info[e->dest->index].mode_in = src_mode;
+      return true;
+    }
+
+  int entity = confluence_info.entity;
+  int new_mode = mode_confluence (entity, src_mode, dest_mode, no_mode);
+  if (dest_mode == new_mode)
+    return false;
+
+  bb_info[e->dest->index].mode_in = new_mode;
+  return true;
+}
+
+/* Update block BB_INDEX's mode_out based on its mode_in.  Return true if
+   something changed.  */
+
+static bool
+forward_transfer (int bb_index)
+{
+  /* The entry and exit blocks have no useful mode information.  */
+  if (bb_index == ENTRY_BLOCK || bb_index == EXIT_BLOCK)
+    return false;
+
+  /* Only propagate through a block if the entity is transparent.  */
+  struct bb_info *bb_info = confluence_info.bb_info;
+  if (bb_info[bb_index].computing != confluence_info.no_mode
+      || bb_info[bb_index].mode_out == bb_info[bb_index].mode_in)
+    return false;
+
+  bb_info[bb_index].mode_out = bb_info[bb_index].mode_in;
+  return true;
+}
+
 /* Find all insns that need a particular mode setting, and insert the
    necessary mode switches.  Return true if we did work.  */
 
@@ -568,6 +663,39 @@ optimize_mode_switching (void)
 
   auto_sbitmap transp_all (last_basic_block_for_fn (cfun));
 
+  auto_bitmap blocks;
+
+  /* Forward-propagate mode information through blocks where the entity
+     is transparent, so that mode_in describes the mode on entry to each
+     block and mode_out describes the mode on exit from each block.  */
+  auto forwprop_mode_info = [&](struct bb_info *info,
+				int entity, int no_mode)
+    {
+      /* Use no_mode + 1 to mean "not yet set".  */
+      FOR_EACH_BB_FN (bb, cfun)
+	{
+	  if (bb_has_abnormal_pred (bb))
+	    info[bb->index].mode_in = info[bb->index].seginfo->mode;
+	  else
+	    info[bb->index].mode_in = no_mode + 1;
+	  if (info[bb->index].computing != no_mode)
+	    info[bb->index].mode_out = info[bb->index].computing;
+	  else
+	    info[bb->index].mode_out = no_mode + 1;
+	}
+
+      confluence_info.bb_info = info;
+      confluence_info.entity = entity;
+      confluence_info.no_mode = no_mode;
+
+      bitmap_set_range (blocks, 0, last_basic_block_for_fn (cfun));
+      df_simple_dataflow (DF_FORWARD, NULL, NULL, forward_confluence_n,
+			  forward_transfer, blocks,
+			  df_get_postorder (DF_FORWARD),
+			  df_get_n_blocks (DF_FORWARD));
+
+    };
+
   for (j = n_entities - 1; j >= 0; j--)
     {
       int e = entity_map[j];
@@ -721,6 +849,7 @@ optimize_mode_switching (void)
   for (j = n_entities - 1; j >= 0; j--)
     {
       int no_mode = num_modes[entity_map[j]];
+      struct bb_info *info = bb_info[j];
 
       /* Insert all mode sets that have been inserted by lcm.  */
 
@@ -739,39 +868,33 @@ optimize_mode_switching (void)
 	    }
 	}
 
+      /* mode_in and mode_out can be calculated directly from avin and
+	 avout if all the modes are mutually exclusive.  Use the target-
+	 provided confluence function otherwise.  */
+      if (targetm.mode_switching.confluence)
+	forwprop_mode_info (info, entity_map[j], no_mode);
+
       FOR_EACH_BB_FN (bb, cfun)
 	{
-	  struct bb_info *info = bb_info[j];
-	  int last_mode = no_mode;
-
-	  /* intialize mode in availability for bb.  */
-	  for (i = 0; i < no_mode; i++)
-	    if (mode_bit_p (avout[bb->index], j, i))
-	      {
-		if (last_mode == no_mode)
-		  last_mode = i;
-		if (last_mode != i)
+	  auto modes_confluence = [&](sbitmap *av)
+	    {
+	      for (int i = 0; i < no_mode; ++i)
+		if (mode_bit_p (av[bb->index], j, i))
 		  {
-		    last_mode = no_mode;
-		    break;
+		    for (int i2 = i + 1; i2 < no_mode; ++i2)
+		      if (mode_bit_p (av[bb->index], j, i2))
+			return no_mode;
+		    return i;
 		  }
-	      }
-	  info[bb->index].mode_out = last_mode;
+	      return no_mode;
+	    };
 
-	  /* intialize mode out availability for bb.  */
-	  last_mode = no_mode;
-	  for (i = 0; i < no_mode; i++)
-	    if (mode_bit_p (avin[bb->index], j, i))
-	      {
-		if (last_mode == no_mode)
-		  last_mode = i;
-		if (last_mode != i)
-		  {
-		    last_mode = no_mode;
-		    break;
-		  }
-	      }
-	  info[bb->index].mode_in = last_mode;
+	  /* intialize mode in/out availability for bb.  */
+	  if (!targetm.mode_switching.confluence)
+	    {
+	      info[bb->index].mode_out = modes_confluence (avout);
+	      info[bb->index].mode_in = modes_confluence (avin);
+	    }
 
 	  for (i = 0; i < no_mode; i++)
 	    if (mode_bit_p (del[bb->index], j, i))
diff --git a/gcc/target.def b/gcc/target.def
index 9b14c037d3f..b08ede692f1 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7053,6 +7053,23 @@ the number of modes if it does not know what mode @var{entity} has after\n\
 Not defining the hook is equivalent to returning @var{mode}.",
  int, (int entity, int mode, rtx_insn *insn, HARD_REG_SET regs_live), NULL)
 
+DEFHOOK
+(confluence,
+ "By default, the mode-switching pass assumes that a given entity's modes\n\
+are mutually exclusive.  This means that the pass can only tell\n\
+@code{TARGET_MODE_EMIT} about an entity's previous mode if all\n\
+incoming paths of execution leave the entity in the same state.\n\
+\n\
+However, some entities might have overlapping, non-exclusive modes,\n\
+so that it is sometimes possible to represent ``mode @var{mode1} or mode\n\
+@var{mode2}'' with something more specific than ``mode not known''.\n\
+If this is true for at least one entity, you should define this hook\n\
+and make it return a mode that includes @var{mode1} and @var{mode2}\n\
+as possibilities.  (The mode can include other possibilities too.)\n\
+The hook should return the number of modes if no suitable mode exists\n\
+for the given arguments.",
+ int, (int entity, int mode1, int mode2), NULL)
+
 DEFHOOK
 (entry,
  "If this hook is defined, it is evaluated for every @var{entity} that\n\
-- 
2.25.1

[PATCH 12/12] mode-switching: Add a backprop hook

[Richard Sandiford]

This patch adds a way for targets to ask that selected mode changes
be brought forward, through a combination of:

(1) requiring a mode in blocks where the entity was previously
    transparent

(2) pushing the transition at the head of a block onto incomging edges

SME has two uses for this:

- A "one-shot" entity that, for any given path of execution,
  either stays off or makes exactly one transition from off to on.
  This relies only on (1) above; see the hook description for more info.

  The main purpose of using mode-switching for this entity is to
  shrink-wrap the code that requires it.

- A second entity for which all transitions must be from known
  modes, which is enforced using a combination of (1) and (2).
  More specifically, (1) looks for edges B1->B2 for which:

  - B2 requires a specific mode and
  - B1 does not guarantee a specific starting mode

  In this system, such an edge is only possible if the entity is
  transparent in B1.  (1) then forces B1 to require some safe common
  mode.  Applying this inductively means that all incoming edges are
  from known modes.  If different edges give different starting modes,
  (2) pushes the transitions onto the edges themselves; this only
  happens if the entity is not transparent in some predecessor block.

The patch also uses the back-propagation as an excuse to do a simple
on-the-fly optimisation.

Hopefully the comments in the patch explain things a bit better.

gcc/
	* target.def (mode_switching.backprop): New hook.
	* doc/tm.texi.in (TARGET_MODE_BACKPROP): New @hook.
	* doc/tm.texi: Regenerate.
	* mode-switching.cc (struct bb_info): Add single_succ.
	(confluence_info): Add transp field.
	(single_succ_confluence_n, single_succ_transfer): New functions.
	(backprop_confluence_n, backprop_transfer): Likewise.
	(optimize_mode_switching): Use them.  Push mode transitions onto
	a block's incoming edges, if the backprop hook requires it.
---
 gcc/doc/tm.texi       |  28 +++++
 gcc/doc/tm.texi.in    |   2 +
 gcc/mode-switching.cc | 272 ++++++++++++++++++++++++++++++++++++++++++
 gcc/target.def        |  29 +++++
 4 files changed, 331 insertions(+)

diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index cd346538fe2..d83ca73b1af 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10456,6 +10456,34 @@ The hook should return the number of modes if no suitable mode exists
 for the given arguments.
 @end deftypefn
 
+@deftypefn {Target Hook} int TARGET_MODE_BACKPROP (int @var{entity}, int @var{mode1}, int @var{mode2})
+If defined, the mode-switching pass uses this hook to back-propagate mode
+requirements through blocks that have no mode requirements of their own.
+Specifically, @var{mode1} is the mode that @var{entity} has on exit
+from a block B1 (say) and @var{mode2} is the mode that the next block
+requires @var{entity} to have.  B1 does not have any mode requirements
+of its own.
+
+The hook should return the mode that it prefers or requires @var{entity}
+to have in B1, or the number of modes if there is no such requirement.
+If the hook returns a required mode for more than one of B1's outgoing
+edges, those modes are combined as for @code{TARGET_MODE_CONFLUENCE}.
+
+For example, suppose there is a ``one-shot'' entity that,
+for a given execution of a function, either stays off or makes exactly
+one transition from off to on.  It is safe to make the transition at any
+time, but it is better not to do so unnecessarily.  This hook allows the
+function to manage such an entity without having to track its state at
+runtime.  Specifically. the entity would have two modes, 0 for off and
+1 for on, with 2 representing ``don't know''.  The system is forbidden from
+transitioning from 2 to 1, since 2 represents the possibility that the
+entity is already on (and the aim is to avoid having to emit code to
+check for that case).  This hook would therefore return 1 when @var{mode1}
+is 2 and @var{mode2} is 1, which would force the entity to be on in the
+source block.  Applying this inductively would remove all transitions
+in which the previous state is unknown.
+@end deftypefn
+
 @deftypefn {Target Hook} int TARGET_MODE_ENTRY (int @var{entity})
 If this hook is defined, it is evaluated for every @var{entity} that
 needs mode switching.  It should return the mode that @var{entity} is
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index ae23241ea1c..3d3ae12cc2f 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -6977,6 +6977,8 @@ mode or ``no mode'', depending on context.
 
 @hook TARGET_MODE_CONFLUENCE
 
+@hook TARGET_MODE_BACKPROP
+
 @hook TARGET_MODE_ENTRY
 
 @hook TARGET_MODE_EXIT
diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 87b23d2c050..720c30df72d 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -81,6 +81,7 @@ struct bb_info
   int computing;
   int mode_out;
   int mode_in;
+  int single_succ;
 };
 
 /* Clear ode I from entity J in bitmap B.  */
@@ -509,6 +510,9 @@ struct
   /* Information about each basic block, indexed by block id.  */
   struct bb_info *bb_info;
 
+  /* A bitmap of blocks for which the current entity is transparent.  */
+  sbitmap transp;
+
   /* The entity that we're processing.  */
   int entity;
 
@@ -580,6 +584,210 @@ forward_transfer (int bb_index)
   return true;
 }
 
+/* A backwards confluence function.  Update the the bb_info single_succ
+   field for E's source block, based on changes to E's destination block.
+   At the end of the dataflow problem, single_succ is the single mode
+   that all successors require (directly or indirectly), or no_mode
+   if there are conflicting requirements.
+
+   Initially, a value of no_mode + 1 means "don't know".  */
+
+static bool
+single_succ_confluence_n (edge e)
+{
+  /* The entry block has no associated mode information.  */
+  if (e->src->index == ENTRY_BLOCK)
+    return false;
+
+  /* We don't control mode changes across abnormal edges.  */
+  if (e->flags & EDGE_ABNORMAL)
+    return false;
+
+  /* Do nothing if we've already found a conflict.  */
+  struct bb_info *bb_info = confluence_info.bb_info;
+  int no_mode = confluence_info.no_mode;
+  int src_mode = bb_info[e->src->index].single_succ;
+  if (src_mode == no_mode)
+    return false;
+
+  /* Work out what mode the destination block (or its successors) require.  */
+  int dest_mode;
+  if (e->dest->index == EXIT_BLOCK)
+    dest_mode = no_mode;
+  else if (bitmap_bit_p (confluence_info.transp, e->dest->index))
+    dest_mode = bb_info[e->dest->index].single_succ;
+  else
+    dest_mode = bb_info[e->dest->index].seginfo->mode;
+
+  /* Do nothing if the destination block has no new information.  */
+  if (dest_mode == no_mode + 1 || dest_mode == src_mode)
+    return false;
+
+  /* Detect conflicting modes.  */
+  if (src_mode != no_mode + 1)
+    dest_mode = no_mode;
+
+  bb_info[e->src->index].single_succ = dest_mode;
+  return true;
+}
+
+/* A backward transfer function for computing the bb_info single_succ
+   fields, as described above single_succ_confluence.  */
+
+static bool
+single_succ_transfer (int bb_index)
+{
+  /* We don't have any field to transfer to.  Assume that, after the
+     first iteration, we are only called if single_succ has changed.
+     We should then process incoming edges if the entity is transparent.  */
+  return bitmap_bit_p (confluence_info.transp, bb_index);
+}
+
+/* Check whether the target wants to back-propagate a mode change across
+   edge E, and update the source block's computed mode if so.  Return true
+   if something changed.  */
+
+static bool
+backprop_confluence_n (edge e)
+{
+  /* The entry and exit blocks have no useful mode information.  */
+  if (e->src->index == ENTRY_BLOCK || e->dest->index == EXIT_BLOCK)
+    return false;
+
+  /* We don't control mode changes across abnormal edges.  */
+  if (e->flags & EDGE_ABNORMAL)
+    return false;
+
+  /* We can only require a new mode in the source block if the entity
+     was originally transparent there.  */
+  if (!bitmap_bit_p (confluence_info.transp, e->src->index))
+    return false;
+
+  /* Exit now if there is no required mode, or if all paths into the
+     source block leave the entity in the required mode.  */
+  struct bb_info *bb_info = confluence_info.bb_info;
+  int no_mode = confluence_info.no_mode;
+  int src_mode = bb_info[e->src->index].mode_out;
+  int dest_mode = bb_info[e->dest->index].mode_in;
+  if (dest_mode == no_mode || src_mode == dest_mode)
+    return false;
+
+  /* See what the target thinks about this transition.  */
+  int entity = confluence_info.entity;
+  int new_mode = targetm.mode_switching.backprop (entity, src_mode,
+						  dest_mode);
+  if (new_mode == no_mode)
+    return false;
+
+  /* The target doesn't like the current transition, but would be happy
+     with a transition from NEW_MODE.
+
+     If we force the source block to use NEW_MODE, we might introduce a
+     double transition on at least one path through the function (one to
+     NEW_MODE and then one to DEST_MODE).  Therefore, if all destination
+     blocks require the same mode, it is usually better to bring that
+     mode requirement forward.
+
+     If that isn't possible, merge the preference for this edge with
+     the preferences for other edges.  no_mode + 1 indicates that there
+     was no previous preference.  */
+  int old_mode = bb_info[e->src->index].computing;
+  if (bb_info[e->src->index].single_succ != no_mode)
+    new_mode = bb_info[e->src->index].single_succ;
+  else if (old_mode != no_mode + 1)
+    new_mode = mode_confluence (entity, old_mode, new_mode, no_mode);
+
+  if (old_mode == new_mode)
+    return false;
+
+  bb_info[e->src->index].computing = new_mode;
+  return true;
+}
+
+/* If the current entity was originally transparent in block BB_INDEX,
+   update the incoming mode to match the outgoing mode.  Register a mode
+   change if the entity is no longer transparent.
+
+   Also, as an on-the-fly optimization, check whether the entity was
+   originally transparent in BB_INDEX and if all successor blocks require
+   the same mode.  If so, anticipate the mode change in BB_INDEX if
+   doing it on the incoming edges would require no more mode changes than
+   doing it on the outgoing edges.  The aim is to reduce the total number
+   of mode changes emitted for the function (and thus reduce code size and
+   cfg complexity) without increasing the number of mode changes on any
+   given path through the function.  A typical case where it helps is:
+
+	  T
+	 / \
+	T   M
+	 \ /
+	  M
+
+   where the entity is transparent in the T blocks and is required to have
+   mode M in the M blocks.  If there are no redundancies leading up to this,
+   there will be two mutually-exclusive changes to mode M, one on each of
+   the T->M edges.  The optimization instead converts it to:
+
+	  T            T            M
+	 / \          / \          / \
+	T   M   ->   M   M   ->   M   M
+	 \ /          \ /          \ /
+	  M            M            M
+
+   which creates a single transition to M for both paths through the diamond.
+
+   Return true if something changed.  */
+
+static bool
+backprop_transfer (int bb_index)
+{
+  /* The entry and exit blocks have no useful mode information.  */
+  if (bb_index == ENTRY_BLOCK || bb_index == EXIT_BLOCK)
+    return false;
+
+  /* We can only require a new mode if the entity was previously
+     transparent.  */
+  if (!bitmap_bit_p (confluence_info.transp, bb_index))
+    return false;
+
+  struct bb_info *bb_info = confluence_info.bb_info;
+  basic_block bb = BASIC_BLOCK_FOR_FN (cfun, bb_index);
+  int no_mode = confluence_info.no_mode;
+  int mode_in = bb_info[bb_index].mode_in;
+  int mode_out = bb_info[bb_index].computing;
+  if (mode_out == no_mode + 1)
+    {
+      /* The entity is still transparent for this block.  See whether
+	 all successor blocks need the same mode, either directly or
+	 indirectly.  */
+      mode_out = bb_info[bb_index].single_succ;
+      if (mode_out == no_mode)
+	return false;
+
+      /* Get a minimum bound on the number of transitions that would be
+	 removed if BB itself required MODE_OUT.  */
+      unsigned int moved = 0;
+      for (edge e : bb->succs)
+	if (e->dest->index != EXIT_BLOCK
+	    && mode_out == bb_info[e->dest->index].seginfo->mode)
+	  moved += 1;
+
+      /* See whether making the mode change on all incoming edges would
+	 be no worse than making it on MOVED outgoing edges.  */
+      if (moved < EDGE_COUNT (bb->preds))
+	return false;
+
+      bb_info[bb_index].mode_out = mode_out;
+      bb_info[bb_index].computing = mode_out;
+    }
+  else if (mode_out == mode_in)
+    return false;
+
+  bb_info[bb_index].mode_in = mode_out;
+  bb_info[bb_index].seginfo->mode = mode_out;
+  return true;
+}
+
 /* Find all insns that need a particular mode setting, and insert the
    necessary mode switches.  Return true if we did work.  */
 
@@ -685,6 +893,7 @@ optimize_mode_switching (void)
 	}
 
       confluence_info.bb_info = info;
+      confluence_info.transp = nullptr;
       confluence_info.entity = entity;
       confluence_info.no_mode = no_mode;
 
@@ -818,6 +1027,53 @@ optimize_mode_switching (void)
 	    }
 	}
 
+      /* If the target requests it, back-propagate selected mode requirements
+	 through transparent blocks.  */
+      if (targetm.mode_switching.backprop)
+	{
+	  /* First work out the mode on entry to and exit from each block.  */
+	  forwprop_mode_info (info, e, no_mode);
+
+	  /* Compute the single_succ fields, as described above
+	     single_succ_confluence.  */
+	  FOR_EACH_BB_FN (bb, cfun)
+	    info[bb->index].single_succ = no_mode + 1;
+
+	  confluence_info.transp = transp_all;
+	  bitmap_set_range (blocks, 0, last_basic_block_for_fn (cfun));
+	  df_simple_dataflow (DF_BACKWARD, NULL, NULL,
+			      single_succ_confluence_n,
+			      single_succ_transfer, blocks,
+			      df_get_postorder (DF_BACKWARD),
+			      df_get_n_blocks (DF_BACKWARD));
+
+	  FOR_EACH_BB_FN (bb, cfun)
+	    {
+	      /* Repurpose mode_in as the first mode required by the block,
+		 or the output mode if none.  */
+	      if (info[bb->index].seginfo->mode != no_mode)
+		info[bb->index].mode_in = info[bb->index].seginfo->mode;
+
+	      /* In transparent blocks, use computing == no_mode + 1
+		 to indicate that no propagation has taken place.  */
+	      if (info[bb->index].computing == no_mode)
+		info[bb->index].computing = no_mode + 1;
+	    }
+
+	  bitmap_set_range (blocks, 0, last_basic_block_for_fn (cfun));
+	  df_simple_dataflow (DF_BACKWARD, NULL, NULL, backprop_confluence_n,
+			      backprop_transfer, blocks,
+			      df_get_postorder (DF_BACKWARD),
+			      df_get_n_blocks (DF_BACKWARD));
+
+	  /* Any block that now computes a mode is no longer transparent.  */
+	  FOR_EACH_BB_FN (bb, cfun)
+	    if (info[bb->index].computing == no_mode + 1)
+	      info[bb->index].computing = no_mode;
+	    else if (info[bb->index].computing != no_mode)
+	      bitmap_clear_bit (transp_all, bb->index);
+	}
+
       /* Set the anticipatable and computing arrays.  */
       for (i = 0; i < no_mode; i++)
 	{
@@ -899,6 +1155,22 @@ optimize_mode_switching (void)
 	  for (i = 0; i < no_mode; i++)
 	    if (mode_bit_p (del[bb->index], j, i))
 	      info[bb->index].seginfo->mode = no_mode;
+
+	  /* See whether the target can perform the first transition.
+	     If not, push it onto the incoming edges.  The earlier backprop
+	     pass should ensure that the resulting transitions are valid.  */
+	  if (targetm.mode_switching.backprop)
+	    {
+	      int from_mode = info[bb->index].mode_in;
+	      int to_mode = info[bb->index].seginfo->mode;
+	      if (targetm.mode_switching.backprop (entity_map[j], from_mode,
+						   to_mode) != no_mode)
+		{
+		  for (edge e : bb->preds)
+		    e->aux = (void *) (intptr_t) (to_mode + 1);
+		  info[bb->index].mode_in = to_mode;
+		}
+	    }
 	}
 
       /* Now output the remaining mode sets in all the segments.  */
diff --git a/gcc/target.def b/gcc/target.def
index b08ede692f1..0996da0f71a 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7070,6 +7070,35 @@ The hook should return the number of modes if no suitable mode exists\n\
 for the given arguments.",
  int, (int entity, int mode1, int mode2), NULL)
 
+DEFHOOK
+(backprop,
+ "If defined, the mode-switching pass uses this hook to back-propagate mode\n\
+requirements through blocks that have no mode requirements of their own.\n\
+Specifically, @var{mode1} is the mode that @var{entity} has on exit\n\
+from a block B1 (say) and @var{mode2} is the mode that the next block\n\
+requires @var{entity} to have.  B1 does not have any mode requirements\n\
+of its own.\n\
+\n\
+The hook should return the mode that it prefers or requires @var{entity}\n\
+to have in B1, or the number of modes if there is no such requirement.\n\
+If the hook returns a required mode for more than one of B1's outgoing\n\
+edges, those modes are combined as for @code{TARGET_MODE_CONFLUENCE}.\n\
+\n\
+For example, suppose there is a ``one-shot'' entity that,\n\
+for a given execution of a function, either stays off or makes exactly\n\
+one transition from off to on.  It is safe to make the transition at any\n\
+time, but it is better not to do so unnecessarily.  This hook allows the\n\
+function to manage such an entity without having to track its state at\n\
+runtime.  Specifically. the entity would have two modes, 0 for off and\n\
+1 for on, with 2 representing ``don't know''.  The system is forbidden from\n\
+transitioning from 2 to 1, since 2 represents the possibility that the\n\
+entity is already on (and the aim is to avoid having to emit code to\n\
+check for that case).  This hook would therefore return 1 when @var{mode1}\n\
+is 2 and @var{mode2} is 1, which would force the entity to be on in the\n\
+source block.  Applying this inductively would remove all transitions\n\
+in which the previous state is unknown.",
+ int, (int entity, int mode1, int mode2), NULL)
+
 DEFHOOK
 (entry,
  "If this hook is defined, it is evaluated for every @var{entity} that\n\
-- 
2.25.1

Re: [PATCH 01/12] mode-switching: Tweak the macro/hook documentation

[Jeff Law]

On 11/5/23 11:46, Richard Sandiford wrote:
> I found the documentation for the mode-switching macros/hooks
> a bit hard to follow at first.  This patch tries to add the
> information that I think would have made it easier to understand.
> 
> Of course, documentation preferences are personal, and so I could
> be changing something that others understood to something that
> seems impenetrable.
> 
> Some notes on specific changes:
> 
> - "in an optimizing compilation" didn't seem accurate; the pass
>    is run even at -O0, and often needs to be for correctness.
> 
> - "at run time" meant when the compiler was run, rather than when
>    the compiled code was run.
> 
> - Removing the list of optional macros isn't a clarification,
>    but it means that upcoming patches don't create an absurdly
>    long list.
> 
> - I don't really understand the purpose of TARGET_MODE_PRIORITY,
>    so I mostly left that alone.
> 
> gcc/
> 	* target.def: Tweak documentation of mode-switching hooks.
> 	* doc/tm.texi.in (OPTIMIZE_MODE_SWITCHING): Tweak documentation.
> 	(NUM_MODES_FOR_MODE_SWITCHING): Likewise.
> 	* doc/tm.texi: Regenerate.
OK.  I doubt very many people have looked at this stuff.  I think the 
only targets implementing mode switching are x86, sh and riscv, and soon 
aarch64 and for the most part I think most folks try to ignore it ;-)

jeff

Re: [PATCH 02/12] mode-switching: Add note problem

[Jeff Law]

On 11/5/23 11:46, Richard Sandiford wrote:
> optimize_mode_switching uses REG_DEAD notes to track register
> liveness, but it failed to tell DF to calculate up-to-date notes.
> 
> Noticed by inspection.  I don't have a testcase that fails
> because of this.
> 
> gcc/
> 	* mode-switching.cc (optimize_mode_switching): Call
> 	df_note_add_problem.
OK.  Triggering this would likely be fairly hard, so I'm not going to 
stress about a testcase.

jeff

Re: [PATCH 03/12] mode-switching: Avoid quadractic list operation

[Jeff Law]

On 11/5/23 11:47, Richard Sandiford wrote:
> add_seginfo chained insn information to the end of a list
> by starting at the head of the list.  This patch avoids the
> quadraticness by keeping track of the tail pointer.
> 
> gcc/
> 	* mode-switching.cc (add_seginfo): Replace head pointer with
> 	a pointer to the tail pointer.
> 	(optimize_mode_switching): Update calls accordingly.
OK
jeff

Re: [PATCH 04/12] mode-switching: Fix the mode passed to the emit hook

[Jeff Law]

On 11/5/23 11:47, Richard Sandiford wrote:
> optimize_mode_switching passes an entity's current mode (if known)
> to the emit hook.  However, the mode that it passed ignored the
> effect of the after hook.  Instead, the mode for the first emit
> call in a block was taken from the incoming mode, whereas the
> mode for each subsequent emit call was taken from the result
> of the previous call.
> 
> The previous pass through the insns already calculated the
> correct mode, so this patch records it in the seginfo structure.
> (There was a 32-bit hole on 64-bit hosts, so this doesn't increase
> the size of the structure for them.)
> 
> gcc/
> 	* mode-switching.cc (seginfo): Add a prev_mode field.
> 	(new_seginfo): Take and initialize the prev_mode.
> 	(optimize_mode_switching): Update calls accordingly.
> 	Use the recorded modes during the emit phase, rather than
> 	computing one on the fly.
OK
jeff

Re: [PATCH 05/12] mode-switching: Simplify recording of transparency

[Jeff Law]

On 11/5/23 11:47, Richard Sandiford wrote:
> For a given block, an entity is either transparent for
> all modes or for none.  Each update to the transparency set
> therefore used a loop like:
> 
> 		for (i = 0; i < no_mode; i++)
> 		  clear_mode_bit (transp[bb->index], j, i);
> 
> This patch instead starts out with a bit-per-block bitmap
> and updates the main bitmap at the end.
> 
> This isn't much of a simplification on its own.  The main
> purpose is to simplify later patches.
> 
> gcc/
> 	* mode-switching.cc (optimize_mode_switching): Initially
> 	compute transparency in a bit-per-block bitmap.
OK
jeff

Re: [PATCH 06/12] mode-switching: Tweak entry/exit handling

[Jeff Law]

On 11/5/23 11:48, Richard Sandiford wrote:
> An entity isn't transparent in a block that requires a specific mode.
> optimize_mode_switching took that into account for normal insns,
> but didn't for the exit block.  Later patches misbehaved because
> of this.
> 
> In contrast, an entity was correctly marked as non-transparent
> in the entry block, but the reasoning seemed a bit convoluted.
> It also referred to a function that no longer exists.
> Since KILL = ~TRANSP, the entity is by definition not transparent
> in a block that defines the entity, so I think we can make it so
> without comment.
> 
> Finally, the exit handling was nested in the entry handling,
> but that doesn't seem necessary.  A target could say that an
> entity is undefined on entry but must be defined on return,
> on a "be liberal in what you accept, be conservative in what
> you do" principle.
> 
> gcc/
> 	* mode-switching.cc (optimize_mode_switching): Mark the exit
> 	block as nontransparent if it requires a specific mode.
> 	Handle the entry and exit mode as sibling rather than nested
> 	concepts.  Remove outdated comment.
OK
jeff

Re: [PATCH 07/12] mode-switching: Allow targets to set the mode for EH handlers

[Jeff Law]

On 11/5/23 11:48, Richard Sandiford wrote:
> The mode-switching pass already had hooks to say what mode
> an entity is in on entry to a function and what mode it must
> be in on return.  For SME, we also want to say what mode an
> entity is guaranteed to be in on entry to an exception handler.
> 
> gcc/
> 	* target.def (mode_switching.eh_handler): New hook.
> 	* doc/tm.texi.in (TARGET_MODE_EH_HANDLER): New @hook.
> 	* doc/tm.texi: Regenerate.
> 	* mode-switching.cc (optimize_mode_switching): Use eh_handler
> 	to get the mode on entry to an exception handler.
Can we ever have a path to the exception handler triggered by a normal 
control flow and if so, presumably we want this to apply in that case too?

It looks like that's the semantics of the implementation by way to using 
bb_has_eh_pred.  Just want to make sure that's the semantics you want in 
that oddball case.

Assuming it is the semantics you want, it's OK for the trunk, though you 
might want to twiddle the docs slightly to mention that case.

jeff

Re: [PATCH 08/12] mode-switching: Pass set of live registers to the needed hook

[Jeff Law]

On 11/5/23 11:48, Richard Sandiford wrote:
> The emit hook already takes the set of live hard registers as input.
> This patch passes it to the needed hook too.  SME uses this to
> optimise the mode choice based on whether state is live or dead.
> 
> The main caller already had access to the required info, but the
> special handling of return values did not.
> 
> gcc/
> 	* target.def (mode_switching.needed): Add a regs_live parameter.
> 	* doc/tm.texi: Regenerate.
> 	* config/epiphany/epiphany-protos.h (epiphany_mode_needed): Update
> 	accordingly.
> 	* config/epiphany/epiphany.cc (epiphany_mode_needed): Likewise.
> 	* config/epiphany/mode-switch-use.cc (insert_uses): Likewise.
> 	* config/i386/i386.cc (ix86_mode_needed): Likewise.
> 	* config/riscv/riscv.cc (riscv_mode_needed): Likewise.
> 	* config/sh/sh.cc (sh_mode_needed): Likewise.
> 	* mode-switching.cc (optimize_mode_switching): Likewise.
> 	(create_pre_exit): Likewise, using the DF simulate functions
> 	to calculate the required information.
We should probably deprecate the epiphany port.  It's been too flakey to 
test for years at this point.  But anyway....

OK for the trunk.

jeff

Re: [PATCH 09/12] mode-switching: Pass the set of live registers to the after hook

[Jeff Law]

On 11/5/23 11:49, Richard Sandiford wrote:
> This patch passes the set of live hard registers to the after hook,
> like the previous one did for the needed hook.
> 
> gcc/
> 	* target.def (mode_switching.after): Add a regs_live parameter.
> 	* doc/tm.texi: Regenerate.
> 	* config/epiphany/epiphany-protos.h (epiphany_mode_after): Update
> 	accordingly.
> 	* config/epiphany/epiphany.cc (epiphany_mode_needed): Likewise.
> 	(epiphany_mode_after): Likewise.
> 	* config/i386/i386.cc (ix86_mode_after): Likewise.
> 	* config/riscv/riscv.cc (riscv_mode_after): Likewise.
> 	* config/sh/sh.cc (sh_mode_after): Likewise.
> 	* mode-switching.cc (optimize_mode_switching): Likewise.
OK
jeff

Re: [PATCH 10/12] mode-switching: Use 1-based edge aux fields

[Jeff Law]

On 11/5/23 11:49, Richard Sandiford wrote:
> The pass used the edge aux field to record which mode change
> should happen on the edge, with -1 meaning "none".  It's more
> convenient for later patches to leave aux zero for "none",
> and use numbers based at 1 to record a change.
> 
> gcc/
> 	* mode-switching.cc (commit_mode_sets): Use 1-based edge aux values.
So my only worry here is the state of the aux field as we enter mode 
switching.  ISTM the old code would never depend on that previous state 
since it always initialized eg->aux to -1, then conditionally overwrote 
that value if there was an insertion.  Then it could clear all the aux 
fields once a particular entity was resolved.

It appears now that for the first entity we depend on the aux field 
being clear as we enter mode switching.  Or am I missing something?

jeff

Re: [PATCH 11/12] mode-switching: Add a target-configurable confluence operator

[Jeff Law]

On 11/5/23 11:50, Richard Sandiford wrote:
> The mode-switching pass assumed that all of an entity's modes
> were mutually exclusive.  However, the upcoming SME changes
> have an entity with some overlapping modes, so that there is
> sometimes a "superunion" mode that contains two given modes.
> We can use this relationship to pass something more helpful than
> "don't know" to the emit hook.
> 
> This patch adds a new hook that targets can use to specify
> a mode confluence operator.
> 
> With mutually exclusive modes, it's possible to compute a block's
> incoming and outgoing modes by looking at its availability sets.
> With the confluence operator, we instead need to solve a full
> dataflow problem.
> 
> However, when emitting a mode transition, the upcoming SME use of
> mode-switching benefits from having as much information as possible
> about the starting mode.  Calculating this information is definitely
> worth the compile time.
> 
> The dataflow problem is written to work before and after the LCM
> problem has been solved.  A later patch makes use of this.
> 
> While there (since git blame would ping me for the reindented code),
> I used a lambda to avoid the cut-&-pasted loops.
> 
> gcc/
> 	* target.def (mode_switching.confluence): New hook.
> 	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
> 	* doc/tm.texi.in: Regenerate.
> 	* mode-switching.cc (confluence_info): New variable.
> 	(mode_confluence, forward_confluence_n, forward_transfer): New
> 	functions.
> 	(optimize_mode_switching): Use them to calculate mode_in when
> 	TARGET_MODE_CONFLUENCE is defined.
OK.  There's certain similarities between this and the compatible states 
we can use to reduce vsetvl instructions in RV-V.   I wonder if Juzhe or 
Lehua could utilize this and do less custom optimization code in the RV 
backend.

jeff

Re: [PATCH 07/12] mode-switching: Allow targets to set the mode for EH handlers

[Richard Sandiford]

Thanks for the reviews.

Jeff Law <jeffreyalaw@gmail.com> writes:
> On 11/5/23 11:48, Richard Sandiford wrote:
>> The mode-switching pass already had hooks to say what mode
>> an entity is in on entry to a function and what mode it must
>> be in on return.  For SME, we also want to say what mode an
>> entity is guaranteed to be in on entry to an exception handler.
>> 
>> gcc/
>> 	* target.def (mode_switching.eh_handler): New hook.
>> 	* doc/tm.texi.in (TARGET_MODE_EH_HANDLER): New @hook.
>> 	* doc/tm.texi: Regenerate.
>> 	* mode-switching.cc (optimize_mode_switching): Use eh_handler
>> 	to get the mode on entry to an exception handler.
> Can we ever have a path to the exception handler triggered by a normal 
> control flow and if so, presumably we want this to apply in that case too?

Not directly AFAIK.  The handler has to handle the EH_DATA_REGNOs,
call __cxa_begin_catch, etc.  So even if there is fall-through at
the source level, I think there'd always be a block that is only
reached through abnormal control flow.  So...

> It looks like that's the semantics of the implementation by way to using 
> bb_has_eh_pred.  Just want to make sure that's the semantics you want in 
> that oddball case.
>
> Assuming it is the semantics you want, it's OK for the trunk, though you 
> might want to twiddle the docs slightly to mention that case.

...I think these EH blocks are pure re-entry points.  I suppose some
targets might have entities whose state is call-preserved, so that it's
not changed by EH edges.  But that might also apply to other abnormal
control flow too, so it's probably a separate issue/feature.

Thanks,
Richard

Re: [PATCH 10/12] mode-switching: Use 1-based edge aux fields

[Richard Sandiford]

Jeff Law <jeffreyalaw@gmail.com> writes:
> On 11/5/23 11:49, Richard Sandiford wrote:
>> The pass used the edge aux field to record which mode change
>> should happen on the edge, with -1 meaning "none".  It's more
>> convenient for later patches to leave aux zero for "none",
>> and use numbers based at 1 to record a change.
>> 
>> gcc/
>> 	* mode-switching.cc (commit_mode_sets): Use 1-based edge aux values.
> So my only worry here is the state of the aux field as we enter mode 
> switching.  ISTM the old code would never depend on that previous state 
> since it always initialized eg->aux to -1, then conditionally overwrote 
> that value if there was an insertion.  Then it could clear all the aux 
> fields once a particular entity was resolved.
>
> It appears now that for the first entity we depend on the aux field 
> being clear as we enter mode switching.  Or am I missing something?

Yeah, we'd rely on that.  12/12 would too for all edges.

I could have sworn that there was something that checked that passes
left edge aux fields clear, but it looks like I misremembered.  So I
probably need to stick a clear_aux_for_edges () call above the first
main loop (for 12/12) and keep the initialisation here as well.

That kind-of takes away the point of shifting to 1-based values
in the first place.  Ah well...

Thanks,
Richard

Re: [PATCH 10/12] mode-switching: Use 1-based edge aux fields

[Jeff Law]

On 11/7/23 17:35, Richard Sandiford wrote:

> I could have sworn that there was something that checked that passes
> left edge aux fields clear, but it looks like I misremembered.  So I
> probably need to stick a clear_aux_for_edges () call above the first
> main loop (for 12/12) and keep the initialisation here as well.
That does sound vaguely familiar.   Maybe it was a one-off test someone 
did.

> 
> That kind-of takes away the point of shifting to 1-based values
> in the first place.  Ah well...
Your call.  I'd tend to lean towards inserting the clear_aux call if we 
don't have something that's consistently verifying aux state. 
Alternately we can return to the -1 handling.  I doubt it's all that 
important from a compile-time standpoint.

jeff

Re: [PATCH 07/12] mode-switching: Allow targets to set the mode for EH handlers

[Jeff Law]

On 11/7/23 17:15, Richard Sandiford wrote:
> Thanks for the reviews.
> 
> Jeff Law <jeffreyalaw@gmail.com> writes:
>> On 11/5/23 11:48, Richard Sandiford wrote:
>>> The mode-switching pass already had hooks to say what mode
>>> an entity is in on entry to a function and what mode it must
>>> be in on return.  For SME, we also want to say what mode an
>>> entity is guaranteed to be in on entry to an exception handler.
>>>
>>> gcc/
>>> 	* target.def (mode_switching.eh_handler): New hook.
>>> 	* doc/tm.texi.in (TARGET_MODE_EH_HANDLER): New @hook.
>>> 	* doc/tm.texi: Regenerate.
>>> 	* mode-switching.cc (optimize_mode_switching): Use eh_handler
>>> 	to get the mode on entry to an exception handler.
>> Can we ever have a path to the exception handler triggered by a normal
>> control flow and if so, presumably we want this to apply in that case too?
> 
> Not directly AFAIK.  The handler has to handle the EH_DATA_REGNOs,
> call __cxa_begin_catch, etc.  So even if there is fall-through at
> the source level, I think there'd always be a block that is only
> reached through abnormal control flow.  So...
> 
>> It looks like that's the semantics of the implementation by way to using
>> bb_has_eh_pred.  Just want to make sure that's the semantics you want in
>> that oddball case.
>>
>> Assuming it is the semantics you want, it's OK for the trunk, though you
>> might want to twiddle the docs slightly to mention that case.
> 
> ...I think these EH blocks are pure re-entry points.  I suppose some
> targets might have entities whose state is call-preserved, so that it's
> not changed by EH edges.  But that might also apply to other abnormal
> control flow too, so it's probably a separate issue/feature.
OK.  I wouldn't be surprised if there's state that wouldn't be correct 
if we had code which jumped directly into the EH handler path.  Makes me 
wonder if the C++ language might actually prohibit such shenanigans.

Jeff

Re: [PATCH 12/12] mode-switching: Add a backprop hook

[Jeff Law]

On 11/5/23 11:50, Richard Sandiford wrote:
> This patch adds a way for targets to ask that selected mode changes
> be brought forward, through a combination of:
> 
> (1) requiring a mode in blocks where the entity was previously
>      transparent
> 
> (2) pushing the transition at the head of a block onto incomging edges
> 
> SME has two uses for this:
> 
> - A "one-shot" entity that, for any given path of execution,
>    either stays off or makes exactly one transition from off to on.
>    This relies only on (1) above; see the hook description for more info.
> 
>    The main purpose of using mode-switching for this entity is to
>    shrink-wrap the code that requires it.
> 
> - A second entity for which all transitions must be from known
>    modes, which is enforced using a combination of (1) and (2).
>    More specifically, (1) looks for edges B1->B2 for which:
> 
>    - B2 requires a specific mode and
>    - B1 does not guarantee a specific starting mode
> 
>    In this system, such an edge is only possible if the entity is
>    transparent in B1.  (1) then forces B1 to require some safe common
>    mode.  Applying this inductively means that all incoming edges are
>    from known modes.  If different edges give different starting modes,
>    (2) pushes the transitions onto the edges themselves; this only
>    happens if the entity is not transparent in some predecessor block.
> 
> The patch also uses the back-propagation as an excuse to do a simple
> on-the-fly optimisation.
> 
> Hopefully the comments in the patch explain things a bit better.
> 
> gcc/
> 	* target.def (mode_switching.backprop): New hook.
> 	* doc/tm.texi.in (TARGET_MODE_BACKPROP): New @hook.
> 	* doc/tm.texi: Regenerate.
> 	* mode-switching.cc (struct bb_info): Add single_succ.
> 	(confluence_info): Add transp field.
> 	(single_succ_confluence_n, single_succ_transfer): New functions.
> 	(backprop_confluence_n, backprop_transfer): Likewise.
> 	(optimize_mode_switching): Use them.  Push mode transitions onto
> 	a block's incoming edges, if the backprop hook requires it.
OK.  Really curious if we might be able to use this to improve the 
vsetvl bits in the RISC-V backend.

Jeff

Re: [PATCH 10/12] mode-switching: Use 1-based edge aux fields

[Richard Sandiford]

Jeff Law <jlaw@ventanamicro.com> writes:
> On 11/7/23 17:35, Richard Sandiford wrote:
>
>> I could have sworn that there was something that checked that passes
>> left edge aux fields clear, but it looks like I misremembered.  So I
>> probably need to stick a clear_aux_for_edges () call above the first
>> main loop (for 12/12) and keep the initialisation here as well.
> That does sound vaguely familiar.   Maybe it was a one-off test someone 
> did.
>
>> 
>> That kind-of takes away the point of shifting to 1-based values
>> in the first place.  Ah well...
> Your call.  I'd tend to lean towards inserting the clear_aux call if we 
> don't have something that's consistently verifying aux state. 

Agreed.  A convention that isn't tested isn't strong enough to rely on.

> Alternately we can return to the -1 handling.  I doubt it's all that 
> important from a compile-time standpoint.

I dithered about it a bit, but in the end, zero does still seem to work
out a bit more nicely.

I think you'd basically OKed the patch with this change, but I'm feeling
extra risk-averse at the moment :) so, here's an updated patch.
Tested as before.  OK to install?

Thanks for catching this.

Richard


gcc/
	* mode-switching.cc (commit_mode_sets): Use 1-based edge aux values.
---
 gcc/mode-switching.cc | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 7a5c4993d65..6b5661131e3 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -106,10 +106,10 @@ commit_mode_sets (struct edge_list *edge_list, int e, struct bb_info *info)
   for (int ed = NUM_EDGES (edge_list) - 1; ed >= 0; ed--)
     {
       edge eg = INDEX_EDGE (edge_list, ed);
-      int mode;
 
-      if ((mode = (int)(intptr_t)(eg->aux)) != -1)
+      if (eg->aux)
 	{
+	  int mode = (int) (intptr_t) eg->aux - 1;
 	  HARD_REG_SET live_at_edge;
 	  basic_block src_bb = eg->src;
 	  int cur_mode = info[src_bb->index].mode_out;
@@ -728,14 +728,14 @@ optimize_mode_switching (void)
 	{
 	  edge eg = INDEX_EDGE (edge_list, ed);
 
-	  eg->aux = (void *)(intptr_t)-1;
+	  eg->aux = (void *) (intptr_t) 0;
 
 	  for (i = 0; i < no_mode; i++)
 	    {
 	      int m = targetm.mode_switching.priority (entity_map[j], i);
 	      if (mode_bit_p (insert[ed], j, m))
 		{
-		  eg->aux = (void *)(intptr_t)m;
+		  eg->aux = (void *) (intptr_t) (m + 1);
 		  break;
 		}
 	    }
-- 
2.25.1

Re: [PATCH 11/12] mode-switching: Add a target-configurable confluence operator

[Richard Sandiford]

Jeff Law <jeffreyalaw@gmail.com> writes:
> On 11/5/23 11:50, Richard Sandiford wrote:
>> The mode-switching pass assumed that all of an entity's modes
>> were mutually exclusive.  However, the upcoming SME changes
>> have an entity with some overlapping modes, so that there is
>> sometimes a "superunion" mode that contains two given modes.
>> We can use this relationship to pass something more helpful than
>> "don't know" to the emit hook.
>> 
>> This patch adds a new hook that targets can use to specify
>> a mode confluence operator.
>> 
>> With mutually exclusive modes, it's possible to compute a block's
>> incoming and outgoing modes by looking at its availability sets.
>> With the confluence operator, we instead need to solve a full
>> dataflow problem.
>> 
>> However, when emitting a mode transition, the upcoming SME use of
>> mode-switching benefits from having as much information as possible
>> about the starting mode.  Calculating this information is definitely
>> worth the compile time.
>> 
>> The dataflow problem is written to work before and after the LCM
>> problem has been solved.  A later patch makes use of this.
>> 
>> While there (since git blame would ping me for the reindented code),
>> I used a lambda to avoid the cut-&-pasted loops.
>> 
>> gcc/
>> 	* target.def (mode_switching.confluence): New hook.
>> 	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
>> 	* doc/tm.texi.in: Regenerate.
>> 	* mode-switching.cc (confluence_info): New variable.
>> 	(mode_confluence, forward_confluence_n, forward_transfer): New
>> 	functions.
>> 	(optimize_mode_switching): Use them to calculate mode_in when
>> 	TARGET_MODE_CONFLUENCE is defined.
> OK.  There's certain similarities between this and the compatible states 
> we can use to reduce vsetvl instructions in RV-V.   I wonder if Juzhe or 
> Lehua could utilize this and do less custom optimization code in the RV 
> backend.

Here's an update based on what you pointed out in 10/12.  The change
from last time is to add:

  if (targetm.mode_switching.backprop)
    clear_aux_for_edges ();

before the main loop.  Tested as before.

Thanks,
Richard


gcc/
	* target.def (mode_switching.confluence): New hook.
	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
	* doc/tm.texi.in: Regenerate.
	* mode-switching.cc (confluence_info): New variable.
	(mode_confluence, forward_confluence_n, forward_transfer): New
	functions.
	(optimize_mode_switching): Use them to calculate mode_in when
	TARGET_MODE_CONFLUENCE is defined.
---
 gcc/doc/tm.texi       |  16 ++++
 gcc/doc/tm.texi.in    |   2 +
 gcc/mode-switching.cc | 182 +++++++++++++++++++++++++++++++++++-------
 gcc/target.def        |  17 ++++
 4 files changed, 189 insertions(+), 28 deletions(-)

diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index b730b5bf658..cd346538fe2 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -10440,6 +10440,22 @@ the number of modes if it does not know what mode @var{entity} has after
 Not defining the hook is equivalent to returning @var{mode}.
 @end deftypefn
 
+@deftypefn {Target Hook} int TARGET_MODE_CONFLUENCE (int @var{entity}, int @var{mode1}, int @var{mode2})
+By default, the mode-switching pass assumes that a given entity's modes
+are mutually exclusive.  This means that the pass can only tell
+@code{TARGET_MODE_EMIT} about an entity's previous mode if all
+incoming paths of execution leave the entity in the same state.
+
+However, some entities might have overlapping, non-exclusive modes,
+so that it is sometimes possible to represent ``mode @var{mode1} or mode
+@var{mode2}'' with something more specific than ``mode not known''.
+If this is true for at least one entity, you should define this hook
+and make it return a mode that includes @var{mode1} and @var{mode2}
+as possibilities.  (The mode can include other possibilities too.)
+The hook should return the number of modes if no suitable mode exists
+for the given arguments.
+@end deftypefn
+
 @deftypefn {Target Hook} int TARGET_MODE_ENTRY (int @var{entity})
 If this hook is defined, it is evaluated for every @var{entity} that
 needs mode switching.  It should return the mode that @var{entity} is
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index 5360c1bb2d8..ae23241ea1c 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -6975,6 +6975,8 @@ mode or ``no mode'', depending on context.
 
 @hook TARGET_MODE_AFTER
 
+@hook TARGET_MODE_CONFLUENCE
+
 @hook TARGET_MODE_ENTRY
 
 @hook TARGET_MODE_EXIT
diff --git a/gcc/mode-switching.cc b/gcc/mode-switching.cc
index 6b5661131e3..58bc1934e81 100644
--- a/gcc/mode-switching.cc
+++ b/gcc/mode-switching.cc
@@ -485,6 +485,101 @@ create_pre_exit (int n_entities, int *entity_map, const int *num_modes)
   return pre_exit;
 }
 
+/* Return the confluence of modes MODE1 and MODE2 for entity ENTITY,
+   using NO_MODE to represent an unknown mode if nothing more precise
+   is available.  */
+
+int
+mode_confluence (int entity, int mode1, int mode2, int no_mode)
+{
+  if (mode1 == mode2)
+    return mode1;
+
+  if (mode1 != no_mode
+      && mode2 != no_mode
+      && targetm.mode_switching.confluence)
+    return targetm.mode_switching.confluence (entity, mode1, mode2);
+
+  return no_mode;
+}
+
+/* Information for the dataflow problems below.  */
+struct
+{
+  /* Information about each basic block, indexed by block id.  */
+  struct bb_info *bb_info;
+
+  /* The entity that we're processing.  */
+  int entity;
+
+  /* The number of modes defined for the entity, and thus the identifier
+     of the "don't know" mode.  */
+  int no_mode;
+} confluence_info;
+
+/* Propagate information about any mode change on edge E to the
+   destination block's mode_in.  Return true if something changed.
+
+   The mode_in and mode_out fields use no_mode + 1 to mean "not yet set".  */
+
+static bool
+forward_confluence_n (edge e)
+{
+  /* The entry and exit blocks have no useful mode information.  */
+  if (e->src->index == ENTRY_BLOCK || e->dest->index == EXIT_BLOCK)
+    return false;
+
+  /* We don't control mode changes across abnormal edges.  */
+  if (e->flags & EDGE_ABNORMAL)
+    return false;
+
+  /* E->aux is nonzero if we have computed the LCM problem and scheduled
+     E to change the mode to E->aux - 1.  Otherwise model the change
+     from the source to the destination.  */
+  struct bb_info *bb_info = confluence_info.bb_info;
+  int no_mode = confluence_info.no_mode;
+  int src_mode = bb_info[e->src->index].mode_out;
+  if (e->aux)
+    src_mode = (int) (intptr_t) e->aux - 1;
+  if (src_mode == no_mode + 1)
+    return false;
+
+  int dest_mode = bb_info[e->dest->index].mode_in;
+  if (dest_mode == no_mode + 1)
+    {
+      bb_info[e->dest->index].mode_in = src_mode;
+      return true;
+    }
+
+  int entity = confluence_info.entity;
+  int new_mode = mode_confluence (entity, src_mode, dest_mode, no_mode);
+  if (dest_mode == new_mode)
+    return false;
+
+  bb_info[e->dest->index].mode_in = new_mode;
+  return true;
+}
+
+/* Update block BB_INDEX's mode_out based on its mode_in.  Return true if
+   something changed.  */
+
+static bool
+forward_transfer (int bb_index)
+{
+  /* The entry and exit blocks have no useful mode information.  */
+  if (bb_index == ENTRY_BLOCK || bb_index == EXIT_BLOCK)
+    return false;
+
+  /* Only propagate through a block if the entity is transparent.  */
+  struct bb_info *bb_info = confluence_info.bb_info;
+  if (bb_info[bb_index].computing != confluence_info.no_mode
+      || bb_info[bb_index].mode_out == bb_info[bb_index].mode_in)
+    return false;
+
+  bb_info[bb_index].mode_out = bb_info[bb_index].mode_in;
+  return true;
+}
+
 /* Find all insns that need a particular mode setting, and insert the
    necessary mode switches.  Return true if we did work.  */
 
@@ -568,6 +663,42 @@ optimize_mode_switching (void)
 
   auto_sbitmap transp_all (last_basic_block_for_fn (cfun));
 
+  auto_bitmap blocks;
+
+  /* Forward-propagate mode information through blocks where the entity
+     is transparent, so that mode_in describes the mode on entry to each
+     block and mode_out describes the mode on exit from each block.  */
+  auto forwprop_mode_info = [&](struct bb_info *info,
+				int entity, int no_mode)
+    {
+      /* Use no_mode + 1 to mean "not yet set".  */
+      FOR_EACH_BB_FN (bb, cfun)
+	{
+	  if (bb_has_abnormal_pred (bb))
+	    info[bb->index].mode_in = info[bb->index].seginfo->mode;
+	  else
+	    info[bb->index].mode_in = no_mode + 1;
+	  if (info[bb->index].computing != no_mode)
+	    info[bb->index].mode_out = info[bb->index].computing;
+	  else
+	    info[bb->index].mode_out = no_mode + 1;
+	}
+
+      confluence_info.bb_info = info;
+      confluence_info.entity = entity;
+      confluence_info.no_mode = no_mode;
+
+      bitmap_set_range (blocks, 0, last_basic_block_for_fn (cfun));
+      df_simple_dataflow (DF_FORWARD, NULL, NULL, forward_confluence_n,
+			  forward_transfer, blocks,
+			  df_get_postorder (DF_FORWARD),
+			  df_get_n_blocks (DF_FORWARD));
+
+    };
+
+  if (targetm.mode_switching.backprop)
+    clear_aux_for_edges ();
+
   for (j = n_entities - 1; j >= 0; j--)
     {
       int e = entity_map[j];
@@ -721,6 +852,7 @@ optimize_mode_switching (void)
   for (j = n_entities - 1; j >= 0; j--)
     {
       int no_mode = num_modes[entity_map[j]];
+      struct bb_info *info = bb_info[j];
 
       /* Insert all mode sets that have been inserted by lcm.  */
 
@@ -741,39 +873,33 @@ optimize_mode_switching (void)
 	    }
 	}
 
+      /* mode_in and mode_out can be calculated directly from avin and
+	 avout if all the modes are mutually exclusive.  Use the target-
+	 provided confluence function otherwise.  */
+      if (targetm.mode_switching.confluence)
+	forwprop_mode_info (info, entity_map[j], no_mode);
+
       FOR_EACH_BB_FN (bb, cfun)
 	{
-	  struct bb_info *info = bb_info[j];
-	  int last_mode = no_mode;
-
-	  /* intialize mode in availability for bb.  */
-	  for (i = 0; i < no_mode; i++)
-	    if (mode_bit_p (avout[bb->index], j, i))
-	      {
-		if (last_mode == no_mode)
-		  last_mode = i;
-		if (last_mode != i)
+	  auto modes_confluence = [&](sbitmap *av)
+	    {
+	      for (int i = 0; i < no_mode; ++i)
+		if (mode_bit_p (av[bb->index], j, i))
 		  {
-		    last_mode = no_mode;
-		    break;
+		    for (int i2 = i + 1; i2 < no_mode; ++i2)
+		      if (mode_bit_p (av[bb->index], j, i2))
+			return no_mode;
+		    return i;
 		  }
-	      }
-	  info[bb->index].mode_out = last_mode;
+	      return no_mode;
+	    };
 
-	  /* intialize mode out availability for bb.  */
-	  last_mode = no_mode;
-	  for (i = 0; i < no_mode; i++)
-	    if (mode_bit_p (avin[bb->index], j, i))
-	      {
-		if (last_mode == no_mode)
-		  last_mode = i;
-		if (last_mode != i)
-		  {
-		    last_mode = no_mode;
-		    break;
-		  }
-	      }
-	  info[bb->index].mode_in = last_mode;
+	  /* intialize mode in/out availability for bb.  */
+	  if (!targetm.mode_switching.confluence)
+	    {
+	      info[bb->index].mode_out = modes_confluence (avout);
+	      info[bb->index].mode_in = modes_confluence (avin);
+	    }
 
 	  for (i = 0; i < no_mode; i++)
 	    if (mode_bit_p (del[bb->index], j, i))
diff --git a/gcc/target.def b/gcc/target.def
index 9b14c037d3f..b08ede692f1 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7053,6 +7053,23 @@ the number of modes if it does not know what mode @var{entity} has after\n\
 Not defining the hook is equivalent to returning @var{mode}.",
  int, (int entity, int mode, rtx_insn *insn, HARD_REG_SET regs_live), NULL)
 
+DEFHOOK
+(confluence,
+ "By default, the mode-switching pass assumes that a given entity's modes\n\
+are mutually exclusive.  This means that the pass can only tell\n\
+@code{TARGET_MODE_EMIT} about an entity's previous mode if all\n\
+incoming paths of execution leave the entity in the same state.\n\
+\n\
+However, some entities might have overlapping, non-exclusive modes,\n\
+so that it is sometimes possible to represent ``mode @var{mode1} or mode\n\
+@var{mode2}'' with something more specific than ``mode not known''.\n\
+If this is true for at least one entity, you should define this hook\n\
+and make it return a mode that includes @var{mode1} and @var{mode2}\n\
+as possibilities.  (The mode can include other possibilities too.)\n\
+The hook should return the number of modes if no suitable mode exists\n\
+for the given arguments.",
+ int, (int entity, int mode1, int mode2), NULL)
+
 DEFHOOK
 (entry,
  "If this hook is defined, it is evaluated for every @var{entity} that\n\
-- 
2.25.1

Re: [PATCH 10/12] mode-switching: Use 1-based edge aux fields

[Jeff Law]

On 11/11/23 08:51, Richard Sandiford wrote:
> Jeff Law <jlaw@ventanamicro.com> writes:
>> On 11/7/23 17:35, Richard Sandiford wrote:
>>
>>> I could have sworn that there was something that checked that passes
>>> left edge aux fields clear, but it looks like I misremembered.  So I
>>> probably need to stick a clear_aux_for_edges () call above the first
>>> main loop (for 12/12) and keep the initialisation here as well.
>> That does sound vaguely familiar.   Maybe it was a one-off test someone
>> did.
>>
>>>
>>> That kind-of takes away the point of shifting to 1-based values
>>> in the first place.  Ah well...
>> Your call.  I'd tend to lean towards inserting the clear_aux call if we
>> don't have something that's consistently verifying aux state.
> 
> Agreed.  A convention that isn't tested isn't strong enough to rely on.
> 
>> Alternately we can return to the -1 handling.  I doubt it's all that
>> important from a compile-time standpoint.
> 
> I dithered about it a bit, but in the end, zero does still seem to work
> out a bit more nicely.
> 
> I think you'd basically OKed the patch with this change, but I'm feeling
> extra risk-averse at the moment :) so, here's an updated patch.
> Tested as before.  OK to install?
> 
> Thanks for catching this.
> 
> Richard
> 
> 
> gcc/
> 	* mode-switching.cc (commit_mode_sets): Use 1-based edge aux values.
Yea, intention was to ACK whichever approach you preferred.

OK.

jeff

Re: [PATCH 11/12] mode-switching: Add a target-configurable confluence operator

[Jeff Law]

On 11/11/23 08:54, Richard Sandiford wrote:
> Jeff Law <jeffreyalaw@gmail.com> writes:
>> On 11/5/23 11:50, Richard Sandiford wrote:
>>> The mode-switching pass assumed that all of an entity's modes
>>> were mutually exclusive.  However, the upcoming SME changes
>>> have an entity with some overlapping modes, so that there is
>>> sometimes a "superunion" mode that contains two given modes.
>>> We can use this relationship to pass something more helpful than
>>> "don't know" to the emit hook.
>>>
>>> This patch adds a new hook that targets can use to specify
>>> a mode confluence operator.
>>>
>>> With mutually exclusive modes, it's possible to compute a block's
>>> incoming and outgoing modes by looking at its availability sets.
>>> With the confluence operator, we instead need to solve a full
>>> dataflow problem.
>>>
>>> However, when emitting a mode transition, the upcoming SME use of
>>> mode-switching benefits from having as much information as possible
>>> about the starting mode.  Calculating this information is definitely
>>> worth the compile time.
>>>
>>> The dataflow problem is written to work before and after the LCM
>>> problem has been solved.  A later patch makes use of this.
>>>
>>> While there (since git blame would ping me for the reindented code),
>>> I used a lambda to avoid the cut-&-pasted loops.
>>>
>>> gcc/
>>> 	* target.def (mode_switching.confluence): New hook.
>>> 	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
>>> 	* doc/tm.texi.in: Regenerate.
>>> 	* mode-switching.cc (confluence_info): New variable.
>>> 	(mode_confluence, forward_confluence_n, forward_transfer): New
>>> 	functions.
>>> 	(optimize_mode_switching): Use them to calculate mode_in when
>>> 	TARGET_MODE_CONFLUENCE is defined.
>> OK.  There's certain similarities between this and the compatible states
>> we can use to reduce vsetvl instructions in RV-V.   I wonder if Juzhe or
>> Lehua could utilize this and do less custom optimization code in the RV
>> backend.
> 
> Here's an update based on what you pointed out in 10/12.  The change
> from last time is to add:
> 
>    if (targetm.mode_switching.backprop)
>      clear_aux_for_edges ();
> 
> before the main loop.  Tested as before.
> 
> Thanks,
> Richard
> 
> 
> gcc/
> 	* target.def (mode_switching.confluence): New hook.
> 	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
> 	* doc/tm.texi.in: Regenerate.
> 	* mode-switching.cc (confluence_info): New variable.
> 	(mode_confluence, forward_confluence_n, forward_transfer): New
> 	functions.
> 	(optimize_mode_switching): Use them to calculate mode_in when
> 	TARGET_MODE_CONFLUENCE is defined.
OK.  That's the whole set, right?

jeff

Re: [PATCH 11/12] mode-switching: Add a target-configurable confluence operator

[Richard Sandiford]

Jeff Law <jeffreyalaw@gmail.com> writes:
> On 11/11/23 08:54, Richard Sandiford wrote:
>> Jeff Law <jeffreyalaw@gmail.com> writes:
>>> On 11/5/23 11:50, Richard Sandiford wrote:
>>>> The mode-switching pass assumed that all of an entity's modes
>>>> were mutually exclusive.  However, the upcoming SME changes
>>>> have an entity with some overlapping modes, so that there is
>>>> sometimes a "superunion" mode that contains two given modes.
>>>> We can use this relationship to pass something more helpful than
>>>> "don't know" to the emit hook.
>>>>
>>>> This patch adds a new hook that targets can use to specify
>>>> a mode confluence operator.
>>>>
>>>> With mutually exclusive modes, it's possible to compute a block's
>>>> incoming and outgoing modes by looking at its availability sets.
>>>> With the confluence operator, we instead need to solve a full
>>>> dataflow problem.
>>>>
>>>> However, when emitting a mode transition, the upcoming SME use of
>>>> mode-switching benefits from having as much information as possible
>>>> about the starting mode.  Calculating this information is definitely
>>>> worth the compile time.
>>>>
>>>> The dataflow problem is written to work before and after the LCM
>>>> problem has been solved.  A later patch makes use of this.
>>>>
>>>> While there (since git blame would ping me for the reindented code),
>>>> I used a lambda to avoid the cut-&-pasted loops.
>>>>
>>>> gcc/
>>>> 	* target.def (mode_switching.confluence): New hook.
>>>> 	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
>>>> 	* doc/tm.texi.in: Regenerate.
>>>> 	* mode-switching.cc (confluence_info): New variable.
>>>> 	(mode_confluence, forward_confluence_n, forward_transfer): New
>>>> 	functions.
>>>> 	(optimize_mode_switching): Use them to calculate mode_in when
>>>> 	TARGET_MODE_CONFLUENCE is defined.
>>> OK.  There's certain similarities between this and the compatible states
>>> we can use to reduce vsetvl instructions in RV-V.   I wonder if Juzhe or
>>> Lehua could utilize this and do less custom optimization code in the RV
>>> backend.
>> 
>> Here's an update based on what you pointed out in 10/12.  The change
>> from last time is to add:
>> 
>>    if (targetm.mode_switching.backprop)
>>      clear_aux_for_edges ();
>> 
>> before the main loop.  Tested as before.
>> 
>> Thanks,
>> Richard
>> 
>> 
>> gcc/
>> 	* target.def (mode_switching.confluence): New hook.
>> 	* doc/tm.texi (TARGET_MODE_CONFLUENCE): New @hook.
>> 	* doc/tm.texi.in: Regenerate.
>> 	* mode-switching.cc (confluence_info): New variable.
>> 	(mode_confluence, forward_confluence_n, forward_transfer): New
>> 	functions.
>> 	(optimize_mode_switching): Use them to calculate mode_in when
>> 	TARGET_MODE_CONFLUENCE is defined.
> OK.  That's the whole set, right?

Yeah, that's everything, thanks.  I've now pushed the series.

Richard