public inbox for gcc-patches@gcc.gnu.org
 help / color / mirror / Atom feed
* [PATCH 03/11] Handwritten part of conversion of passes to C++ classes
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (6 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-07-28  9:12   ` Basile Starynkevitch
                     ` (2 more replies)
  2013-07-26 15:05 ` [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed David Malcolm
                   ` (4 subsequent siblings)
  12 siblings, 3 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This patch is the hand-written part of the conversion of passes from
C structs to C++ classes.  It does not work without the subsequent
autogenerated part, which is huge.

Given that the autogenerated part of the conversion is very large
(500k), for the sake of human comprehension I have kept the change as
two separate patches to keep the hand-written changes separate from the
automatically-generated ones.  I would commit these as two separate
changes to SVN in order to keep this readability for posterity in the
logs as well as at review-time.

This pair of patches eliminates the mutable global variables
representing the passes, allowing for multiple compilation contexts in
one process, potentially with different combinations of passes, and with
pass instance owning additional data.

It converts the hierarchy of opt_pass types into an actual C++ class
hierarchy, where each of:

  * gimple_opt_pass
  * rtl_opt_pass
  * ipa_opt_pass_d
  * simple_ipa_opt_pass

all become subclasses of opt_pass.

The gate and execute function pointers become virtual functions.  This
gives us type-safety, allowing the possibility of adding custom fields
to opt_pass subclasses accessible from the execute vfunc without a cast.
There is an additional "clone" virtual function, which exists so that:

  (a) passes can wire themselves up within a context, so that all sister
      passes within a context can share state with each other, without
      sharing state with their cousins in another compilation context:

  (b) pass subclasses can be encapsulated, and the hardcoded sizeof()
      logic in passes.c:make_pass_instance can be removed, allowing
      pass subclasses to have additional fields.

Most of the metadata about passes is moved to a new struct pass_data,
keeping just the "sub", "next" and "static_pass_number" fields in the
pass instances.  Passes also gain a:

   gcc::context * ctxt_;

field, so that a pass instance always "knows" which compilation context
it is associated with, giving an easy way to avoid thread-local lookups
in a future gcc-as-a-shared-library build.

The pass_data has an extra pair of fields: "has_gate" and "has_execute".
These exist to deal with existing code that checks to see if a pass
has a non-NULL gate or execute function ptr.  Given that we can't
portably do that on a vfunc, it's simplest to add a couple of extra
bools to the pass_data struct, preserving the exact existing behaviors.

In this patch opt_pass has an "is a" relationship to pass_data, rather
than a "references" relationship, i.e.:

   class opt_pass : public pass_data

Doing this minimizes the size of the patch and avoids an extra
dereference when looking up patch metadata, but is perhaps something
of a wart.

gcc/
	* Makefile.in (PIPELINE_H): Add dep on pass-instances.def.
	(toplev.o): Add dep on PIPELINE_H.
	* cgraphunit.c (cgraph_process_new_functions): Rework invocation
	of early local pases to reflect this moving from a global to a
	member of gcc::pipeline.
	(cgraph_add_new_function): Likewise.
	* lto-cgraph.c (lto_output_node): Update for conversion of
	struct ipa_opt_pass_d to a C++ subclass of opt_pass.
	* passes.c (opt_pass::clone): New.
	(opt_pass::gate): New.
	(opt_pass::execute): New.
	(opt_pass::opt_pass): New.
	(pipeline::execute_early_local_passes): New.
	(pipeline::execute_pass_mode_switching): new.
	(finish_optimization_passes): Convert to...
	(pipeline::finish_optimization_passes): ...this.
	(finish_optimization_passes): Update for conversion of passes to
	C++ classes.
	(register_dump_files_1): Use has_gate since we cannot portably
	check a vtable entry against NULL.
	(dump_one_pass): Likewise.
	(ipa_write_summaries_2): Likewise.
	(ipa_write_optimization_summaries_1): Likewise.
	(ipa_read_summaries_1): Likewise.
	(ipa_read_optimization_summaries_1): Likewise.
	(execute_ipa_stmt_fixups): Likewise.
	(pipeline::pipeline): Rewrite pass-creation, invoking
	pass-creation functions rather than wiring up globals, and
	storing the results in fields of pipeline generated using
	pass-instances.def.
	(pipeline::dump_profile_report): Update for conversion of passes
	to C++ classes.
	(pipeline::execute_ipa_summary_passes): Likewise.
	(execute_one_ipa_transform_pass): Likewise.
	(execute_one_pass): Use has_gate and has_execute since we cannot
	portably check a vtable entry against NULL.
	* pipeline.h (pipeline::finish_optimization_passes): New.
	(pipeline): Use pass-instances.def to add fields for the
	various pass instances.
	* toplev.c (finalize): Update for move of
	finish_optimization_passes to a method of gcc::pipeline.
	* toplev.h (finish_optimization_passes): Move to method of class
	pipeline.
	* tree-pass.h (struct pass_data): New.
	(opt_pass): Convert to C++ class, make it a subclass of pass_data.
	(opt_pass::gate): Convert to virtual function.
	(opt_pass::~opt_pass): New.
	(opt_pass::clone): New.
	(opt_pass::execute): Convert to virtual function.
	(opt_pass::opt_pass): New.
	(opt_pass::ctxt_): new.
	(gimple_opt_pass): Convert to subclass of opt_pass.
	(gimple_opt_pass::gimple_opt_pass): New.
	(rtl_opt_pass): Convert to subclass of opt_pass.
	(rtl_opt_pass::rtl_opt_pass): New.
	(ipa_opt_pass_d): Convert to subclass of opt_pass.
	(ipa_opt_pass_d::ipa_opt_pass_d): New.
	(simple_ipa_opt_pass): Convert to subclass of opt_pass.
	(simple_ipa_opt_pass::simple_ipa_opt_pass): New.

gcc/config/
	* i386/i386.c (rest_of_handle_insert_vzeroupper): Rework
	invocation of pass_mode_switching to reflect this moving from a
	global to a member of gcc::pipeline.
	(ix86_option_override): Rework how pass_insert_vzeroupper is
	added to the pipeline to reflect autogenerated changes.
	* i386/t-i386 (i386.o) Add deps on CONTEXT_H and PIPELINE_H.

gcc/testsuite/
	* g++.dg/plugin/dumb_plugin.c (plugin_init): Rework how the pass
	is created and added to the pipeline to reflect autogenerated
	changes.
	* g++.dg/plugin/selfassign.c (plugin_init): Likewise.
	* gcc.dg/plugin/one_time_plugin.c (plugin_init): Likewise.
	* gcc.dg/plugin/selfassign.c (plugin_init): Likewise.
---
 gcc/Makefile.in                               |   4 +-
 gcc/cgraphunit.c                              |   6 +-
 gcc/config/i386/i386.c                        |   7 +-
 gcc/config/i386/t-i386                        |   2 +-
 gcc/lto-cgraph.c                              |   2 +-
 gcc/passes.c                                  | 102 +++++++++++++++-----
 gcc/pipeline.h                                |  37 ++++++++
 gcc/testsuite/g++.dg/plugin/dumb_plugin.c     |   3 +-
 gcc/testsuite/g++.dg/plugin/selfassign.c      |   3 +-
 gcc/testsuite/gcc.dg/plugin/one_time_plugin.c |   3 +-
 gcc/testsuite/gcc.dg/plugin/selfassign.c      |   3 +-
 gcc/toplev.c                                  |   3 +-
 gcc/toplev.h                                  |   1 -
 gcc/tree-pass.h                               | 129 ++++++++++++++++++++------
 14 files changed, 241 insertions(+), 64 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 6d1f32c..c1fdb8a 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -987,7 +987,7 @@ PLUGIN_VERSION_H = plugin-version.h configargs.h
 LIBFUNCS_H = libfuncs.h $(HASHTAB_H)
 GRAPHITE_HTAB_H = graphite-htab.h graphite-clast-to-gimple.h $(HASH_TABLE_H)
 CONTEXT_H = context.h
-PIPELINE_H = pipeline.h
+PIPELINE_H = pipeline.h pass-instances.def
 
 #\f
 # Now figure out from those variables how to compile and link.
@@ -2733,7 +2733,7 @@ toplev.o : toplev.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    $(OPTS_H) params.def tree-mudflap.h $(TREE_PASS_H) $(GIMPLE_H) \
    tree-ssa-alias.h $(PLUGIN_H) realmpfr.h tree-diagnostic.h \
    $(TREE_PRETTY_PRINT_H) opts-diagnostic.h $(COMMON_TARGET_H) \
-   tsan.h diagnostic-color.h $(CONTEXT_H)
+   tsan.h diagnostic-color.h $(CONTEXT_H) $(PIPELINE_H)
 
 hwint.o : hwint.c $(CONFIG_H) $(SYSTEM_H) $(DIAGNOSTIC_CORE_H)
 
diff --git a/gcc/cgraphunit.c b/gcc/cgraphunit.c
index dc489fb..0bc7d02 100644
--- a/gcc/cgraphunit.c
+++ b/gcc/cgraphunit.c
@@ -328,7 +328,7 @@ cgraph_process_new_functions (void)
 	      /* When not optimizing, be sure we run early local passes anyway
 		 to expand OMP.  */
 	      || !optimize)
-	    execute_pass_list (pass_early_local_passes.pass.sub);
+	    g->get_passes ().execute_early_local_passes ();
 	  else
 	    compute_inline_parameters (node, true);
 	  free_dominance_info (CDI_POST_DOMINATORS);
@@ -512,7 +512,7 @@ cgraph_add_new_function (tree fndecl, bool lowered)
 	    gimple_register_cfg_hooks ();
 	    bitmap_obstack_initialize (NULL);
 	    execute_pass_list (passes.all_lowering_passes);
-	    execute_pass_list (pass_early_local_passes.pass.sub);
+	    passes.execute_early_local_passes ();
 	    bitmap_obstack_release (NULL);
 	    pop_cfun ();
 
@@ -537,7 +537,7 @@ cgraph_add_new_function (tree fndecl, bool lowered)
 	gimple_register_cfg_hooks ();
 	bitmap_obstack_initialize (NULL);
 	if (!gimple_in_ssa_p (DECL_STRUCT_FUNCTION (fndecl)))
-	  execute_pass_list (pass_early_local_passes.pass.sub);
+	  g->get_passes ().execute_early_local_passes ();
 	bitmap_obstack_release (NULL);
 	pop_cfun ();
 	expand_function (node);
diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c
index 0c546af..88ba218 100644
--- a/gcc/config/i386/i386.c
+++ b/gcc/config/i386/i386.c
@@ -62,6 +62,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "dumpfile.h"
 #include "tree-pass.h"
 #include "tree-flow.h"
+#include "context.h"
+#include "pipeline.h"
 
 static rtx legitimize_dllimport_symbol (rtx, bool);
 static rtx legitimize_pe_coff_extern_decl (rtx, bool);
@@ -2596,7 +2598,7 @@ rest_of_handle_insert_vzeroupper (void)
   ix86_optimize_mode_switching[AVX_U128] = 1;
 
   /* Call optimize_mode_switching.  */
-  pass_mode_switching.pass.execute ();
+  g->get_passes ().execute_pass_mode_switching ();
   return 0;
 }
 
@@ -4028,8 +4030,9 @@ ix86_option_override_internal (bool main_args_p)
 static void
 ix86_option_override (void)
 {
+  opt_pass *pass_insert_vzeroupper = make_pass_insert_vzeroupper (g);
   static struct register_pass_info insert_vzeroupper_info
-    = { &pass_insert_vzeroupper.pass, "reload",
+    = { pass_insert_vzeroupper, "reload",
 	1, PASS_POS_INSERT_AFTER
       };
 
diff --git a/gcc/config/i386/t-i386 b/gcc/config/i386/t-i386
index 3a77e14..f10d570 100644
--- a/gcc/config/i386/t-i386
+++ b/gcc/config/i386/t-i386
@@ -24,7 +24,7 @@ i386.o: $(CONFIG_H) $(SYSTEM_H) coretypes.h dumpfile.h $(TM_H) \
   $(GGC_H) $(TARGET_H) $(TARGET_DEF_H) langhooks.h $(CGRAPH_H) \
   $(TREE_GIMPLE_H) $(DWARF2_H) $(DF_H) tm-constrs.h $(PARAMS_H) \
   i386-builtin-types.inc debug.h dwarf2out.h sbitmap.h $(FIBHEAP_H) \
-  $(OPTS_H) $(DIAGNOSTIC_H) $(COMMON_TARGET_H)
+  $(OPTS_H) $(DIAGNOSTIC_H) $(COMMON_TARGET_H) $(CONTEXT_H) $(PIPELINE_H)
 
 i386-c.o: $(srcdir)/config/i386/i386-c.c \
   $(srcdir)/config/i386/i386-protos.h $(CONFIG_H) $(SYSTEM_H) coretypes.h \
diff --git a/gcc/lto-cgraph.c b/gcc/lto-cgraph.c
index 90e3a2b..c95b128 100644
--- a/gcc/lto-cgraph.c
+++ b/gcc/lto-cgraph.c
@@ -438,7 +438,7 @@ lto_output_node (struct lto_simple_output_block *ob, struct cgraph_node *node,
   streamer_write_hwi_stream (ob->main_stream,
 			     node->ipa_transforms_to_apply.length ());
   FOR_EACH_VEC_ELT (node->ipa_transforms_to_apply, i, pass)
-    streamer_write_hwi_stream (ob->main_stream, pass->pass.static_pass_number);
+    streamer_write_hwi_stream (ob->main_stream, pass->static_pass_number);
 
   if (tag == LTO_symtab_analyzed_node)
     {
diff --git a/gcc/passes.c b/gcc/passes.c
index 1bca68e..0ea0f20 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -82,6 +82,54 @@ struct opt_pass *current_pass;
 
 static void register_pass_name (struct opt_pass *, const char *);
 
+/* Most passes are single-instance (within their context) and thus don't
+   need to implement cloning, but passes that support multiple instances
+   *must* provide their own implementation of the clone method.
+
+   Handle this by providing a default implemenation, but make it a fatal
+   error to call it.  */
+
+opt_pass *
+opt_pass::clone ()
+{
+  internal_error ("pass %s does not support cloning", name);
+}
+
+bool
+opt_pass::gate ()
+{
+  return true;
+}
+
+unsigned int
+opt_pass::execute ()
+{
+  return 0;
+}
+
+opt_pass::opt_pass(const pass_data &data, context *ctxt)
+  : pass_data(data),
+    sub(NULL),
+    next(NULL),
+    static_pass_number(0),
+    ctxt_(ctxt)
+{
+}
+
+
+void
+pipeline::execute_early_local_passes ()
+{
+  execute_pass_list (pass_early_local_passes_1->sub);
+}
+
+unsigned int
+pipeline::execute_pass_mode_switching ()
+{
+  return pass_mode_switching_1->execute ();
+}
+
+
 /* Call from anywhere to find out what pass this is.  Useful for
    printing out debugging information deep inside an service
    routine.  */
@@ -224,6 +272,7 @@ rest_of_type_compilation (tree type, int toplev)
 \f
 
 void
+pipeline::
 finish_optimization_passes (void)
 {
   int i;
@@ -233,16 +282,16 @@ finish_optimization_passes (void)
   timevar_push (TV_DUMP);
   if (profile_arc_flag || flag_test_coverage || flag_branch_probabilities)
     {
-      dump_start (pass_profile.pass.static_pass_number, NULL);
+      dump_start (pass_profile_1->static_pass_number, NULL);
       end_branch_prob ();
-      dump_finish (pass_profile.pass.static_pass_number);
+      dump_finish (pass_profile_1->static_pass_number);
     }
 
   if (optimize > 0)
     {
-      dump_start (pass_profile.pass.static_pass_number, NULL);
+      dump_start (pass_profile_1->static_pass_number, NULL);
       print_combine_total_stats ();
-      dump_finish (pass_profile.pass.static_pass_number);
+      dump_finish (pass_profile_1->static_pass_number);
     }
 
   /* Do whatever is necessary to finish printing the graphs.  */
@@ -550,7 +599,7 @@ register_dump_files_1 (struct opt_pass *pass, int properties)
 
       /* If we have a gate, combine the properties that we could have with
          and without the pass being examined.  */
-      if (pass->gate)
+      if (pass->has_gate)
         properties &= new_properties;
       else
         properties = new_properties;
@@ -679,7 +728,7 @@ dump_one_pass (struct opt_pass *pass, int pass_indent)
   const char *pn;
   bool is_on, is_really_on;
 
-  is_on = (pass->gate == NULL) ? true : pass->gate();
+  is_on = pass->has_gate ? pass->gate() : true;
   is_really_on = override_gate_status (pass, current_function_decl, is_on);
 
   if (pass->static_pass_number <= 0)
@@ -1310,12 +1359,23 @@ pipeline::pipeline (context *ctxt)
 
 #define PUSH_INSERT_PASSES_WITHIN(PASS) \
   { \
-    struct opt_pass **p = &(PASS).pass.sub;
+    struct opt_pass **p = &(PASS ## _1)->sub;
 
 #define POP_INSERT_PASSES() \
   }
 
-#define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
+#define NEXT_PASS(PASS, NUM) \
+  do { \
+    gcc_assert (NULL == PASS ## _ ## NUM); \
+    if ((NUM) == 1)                              \
+      PASS ## _1 = make_##PASS (ctxt_);          \
+    else                                         \
+      {                                          \
+        gcc_assert (PASS ## _1);                 \
+        PASS ## _ ## NUM = PASS ## _1->clone (); \
+      }                                          \
+    p = next_pass_1 (p, PASS ## _ ## NUM);  \
+  } while (0)
 
 #define TERMINATE_PASS_LIST() \
   *p = NULL;
@@ -1541,7 +1601,7 @@ pipeline::dump_profile_report () const
 		fprintf (stderr, "      ");
 
 	      /* Size/time units change across gimple and RTL.  */
-	      if (i == pass_expand.pass.static_pass_number)
+	      if (i == pass_expand_1->static_pass_number)
 		fprintf (stderr, "|----------");
 	      else
 		{
@@ -1778,11 +1838,11 @@ execute_ipa_summary_passes (struct ipa_opt_pass_d *ipa_pass)
 {
   while (ipa_pass)
     {
-      struct opt_pass *pass = &ipa_pass->pass;
+      struct opt_pass *pass = ipa_pass;
 
       /* Execute all of the IPA_PASSes in the list.  */
-      if (ipa_pass->pass.type == IPA_PASS
-	  && (!pass->gate || pass->gate ())
+      if (ipa_pass->type == IPA_PASS
+	  && ((!pass->has_gate) || pass->gate ())
 	  && ipa_pass->generate_summary)
 	{
 	  pass_init_dump_file (pass);
@@ -1799,7 +1859,7 @@ execute_ipa_summary_passes (struct ipa_opt_pass_d *ipa_pass)
 
 	  pass_fini_dump_file (pass);
 	}
-      ipa_pass = (struct ipa_opt_pass_d *)ipa_pass->pass.next;
+      ipa_pass = (struct ipa_opt_pass_d *)ipa_pass->next;
     }
 }
 
@@ -1809,7 +1869,7 @@ static void
 execute_one_ipa_transform_pass (struct cgraph_node *node,
 				struct ipa_opt_pass_d *ipa_pass)
 {
-  struct opt_pass *pass = &ipa_pass->pass;
+  struct opt_pass *pass = ipa_pass;
   unsigned int todo_after = 0;
 
   current_pass = pass;
@@ -1933,7 +1993,7 @@ execute_one_pass (struct opt_pass *pass)
 
   /* Check whether gate check should be avoided.
      User controls the value of the gate through the parameter "gate_status". */
-  gate_status = (pass->gate == NULL) ? true : pass->gate();
+  gate_status = pass->has_gate ? pass->gate() : true;
   gate_status = override_gate_status (pass, current_function_decl, gate_status);
 
   /* Override gate with plugin.  */
@@ -1990,7 +2050,7 @@ execute_one_pass (struct opt_pass *pass)
     timevar_push (pass->tv_id);
 
   /* Do it!  */
-  if (pass->execute)
+  if (pass->has_execute)
     {
       todo_after = pass->execute ();
       do_per_function (clear_last_verified, NULL);
@@ -2066,7 +2126,7 @@ ipa_write_summaries_2 (struct opt_pass *pass, struct lto_out_decl_state *state)
       gcc_assert (pass->type == SIMPLE_IPA_PASS || pass->type == IPA_PASS);
       if (pass->type == IPA_PASS
 	  && ipa_pass->write_summary
-	  && (!pass->gate || pass->gate ()))
+	  && ((!pass->has_gate) || pass->gate ()))
 	{
 	  /* If a timevar is present, start it.  */
 	  if (pass->tv_id)
@@ -2182,7 +2242,7 @@ ipa_write_optimization_summaries_1 (struct opt_pass *pass, struct lto_out_decl_s
       gcc_assert (pass->type == SIMPLE_IPA_PASS || pass->type == IPA_PASS);
       if (pass->type == IPA_PASS
 	  && ipa_pass->write_optimization_summary
-	  && (!pass->gate || pass->gate ()))
+	  && ((!pass->has_gate) || pass->gate ()))
 	{
 	  /* If a timevar is present, start it.  */
 	  if (pass->tv_id)
@@ -2259,7 +2319,7 @@ ipa_read_summaries_1 (struct opt_pass *pass)
       gcc_assert (!cfun);
       gcc_assert (pass->type == SIMPLE_IPA_PASS || pass->type == IPA_PASS);
 
-      if (pass->gate == NULL || pass->gate ())
+      if ((!pass->has_gate) || pass->gate ())
 	{
 	  if (pass->type == IPA_PASS && ipa_pass->read_summary)
 	    {
@@ -2310,7 +2370,7 @@ ipa_read_optimization_summaries_1 (struct opt_pass *pass)
       gcc_assert (!cfun);
       gcc_assert (pass->type == SIMPLE_IPA_PASS || pass->type == IPA_PASS);
 
-      if (pass->gate == NULL || pass->gate ())
+      if ((!pass->has_gate) || pass->gate ())
 	{
 	  if (pass->type == IPA_PASS && ipa_pass->read_optimization_summary)
 	    {
@@ -2388,7 +2448,7 @@ execute_ipa_stmt_fixups (struct opt_pass *pass,
     {
       /* Execute all of the IPA_PASSes in the list.  */
       if (pass->type == IPA_PASS
-	  && (!pass->gate || pass->gate ()))
+	  && ((!pass->has_gate) || pass->gate ()))
 	{
 	  struct ipa_opt_pass_d *ipa_pass = (struct ipa_opt_pass_d *) pass;
 
diff --git a/gcc/pipeline.h b/gcc/pipeline.h
index 37c90d7..60bde5c 100644
--- a/gcc/pipeline.h
+++ b/gcc/pipeline.h
@@ -58,6 +58,12 @@ public:
 
   void dump_profile_report () const;
 
+  void finish_optimization_passes ();
+
+  /* Access to specific passes, so that the majority can be private.  */
+  void execute_early_local_passes ();
+  unsigned int execute_pass_mode_switching ();
+
 public:
   /* The root of the compilation pass tree, once constructed.  */
   opt_pass *all_passes;
@@ -81,6 +87,37 @@ private:
 private:
   context *ctxt_;
 
+  /* References to all of the individual passes.
+     These fields are generated via macro expansion.
+
+     For example:
+         NEXT_PASS (pass_build_cfg, 1);
+     within pass-instances.def means that there is a field:
+         opt_pass *pass_build_cfg_1;
+
+     Similarly, the various:
+        NEXT_PASS (pass_copy_prop, 1);
+        ...
+        NEXT_PASS (pass_copy_prop, 8);
+     in pass-instances.def lead to fields:
+        opt_pass *pass_copy_prop_1;
+        ...
+        opt_pass *pass_copy_prop_8;  */
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) opt_pass *PASS ## _ ## NUM
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
 }; // class pipeline
 
 } // namespace gcc
diff --git a/gcc/testsuite/g++.dg/plugin/dumb_plugin.c b/gcc/testsuite/g++.dg/plugin/dumb_plugin.c
index 0f15140..ab69c14 100644
--- a/gcc/testsuite/g++.dg/plugin/dumb_plugin.c
+++ b/gcc/testsuite/g++.dg/plugin/dumb_plugin.c
@@ -11,6 +11,7 @@
 #include "intl.h"
 #include "toplev.h"
 #include "diagnostic.h"
+#include "context.h"
 
 int plugin_is_GPL_compatible;
 
@@ -124,7 +125,7 @@ plugin_init (struct plugin_name_args *plugin_info,
       return 1;
     }
 
-  pass_info.pass = &pass_dumb_plugin_example.pass;
+  pass_info.pass = make_pass_dumb_plugin_example (g);
   pass_info.reference_pass_name = ref_pass_name;
   pass_info.ref_pass_instance_number = ref_instance_number;
   pass_info.pos_op = PASS_POS_INSERT_AFTER;
diff --git a/gcc/testsuite/g++.dg/plugin/selfassign.c b/gcc/testsuite/g++.dg/plugin/selfassign.c
index 37a0a97..3e0b38e 100644
--- a/gcc/testsuite/g++.dg/plugin/selfassign.c
+++ b/gcc/testsuite/g++.dg/plugin/selfassign.c
@@ -15,6 +15,7 @@
 #include "intl.h"
 #include "plugin-version.h"
 #include "diagnostic.h"
+#include "context.h"
 
 int plugin_is_GPL_compatible;
 
@@ -309,7 +310,7 @@ plugin_init (struct plugin_name_args *plugin_info,
     return 1;
 
   /* Self-assign detection should happen after SSA is constructed.  */
-  pass_info.pass = &pass_warn_self_assign.pass;
+  pass_info.pass = make_pass_warn_self_assign (g);
   pass_info.reference_pass_name = "ssa";
   pass_info.ref_pass_instance_number = 1;
   pass_info.pos_op = PASS_POS_INSERT_AFTER;
diff --git a/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c b/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
index 45e6257..31dfe69 100644
--- a/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
+++ b/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
@@ -8,6 +8,7 @@
 #include "gimple.h"
 #include "tree-pass.h"
 #include "intl.h"
+#include "context.h"
 
 int plugin_is_GPL_compatible;
 
@@ -53,7 +54,7 @@ int plugin_init (struct plugin_name_args *plugin_info,
 {
   struct register_pass_info p;
 
-  p.pass = &one_pass.pass;
+  p.pass = make_one_pass (g);
   p.reference_pass_name = "cfg";
   p.ref_pass_instance_number = 1;
   p.pos_op = PASS_POS_INSERT_AFTER;
diff --git a/gcc/testsuite/gcc.dg/plugin/selfassign.c b/gcc/testsuite/gcc.dg/plugin/selfassign.c
index 37a0a97..3e0b38e 100644
--- a/gcc/testsuite/gcc.dg/plugin/selfassign.c
+++ b/gcc/testsuite/gcc.dg/plugin/selfassign.c
@@ -15,6 +15,7 @@
 #include "intl.h"
 #include "plugin-version.h"
 #include "diagnostic.h"
+#include "context.h"
 
 int plugin_is_GPL_compatible;
 
@@ -309,7 +310,7 @@ plugin_init (struct plugin_name_args *plugin_info,
     return 1;
 
   /* Self-assign detection should happen after SSA is constructed.  */
-  pass_info.pass = &pass_warn_self_assign.pass;
+  pass_info.pass = make_pass_warn_self_assign (g);
   pass_info.reference_pass_name = "ssa";
   pass_info.ref_pass_instance_number = 1;
   pass_info.pos_op = PASS_POS_INSERT_AFTER;
diff --git a/gcc/toplev.c b/gcc/toplev.c
index 1cf98dc..6097290 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -76,6 +76,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "plugin.h"
 #include "diagnostic-color.h"
 #include "context.h"
+#include "pipeline.h"
 
 #if defined(DBX_DEBUGGING_INFO) || defined(XCOFF_DEBUGGING_INFO)
 #include "dbxout.h"
@@ -1818,7 +1819,7 @@ finalize (bool no_backend)
     {
       statistics_fini ();
 
-      finish_optimization_passes ();
+      g->get_passes ().finish_optimization_passes ();
 
       ira_finish_once ();
     }
diff --git a/gcc/toplev.h b/gcc/toplev.h
index fff452c..84ffdb0 100644
--- a/gcc/toplev.h
+++ b/gcc/toplev.h
@@ -28,7 +28,6 @@ extern int toplev_main (int, char **);
 extern void rest_of_decl_compilation (tree, int, int);
 extern void rest_of_type_compilation (tree, int);
 extern void init_optimization_passes (void);
-extern void finish_optimization_passes (void);
 extern bool enable_rtl_dump_file (void);
 
 /* In except.c.  Initialize exception handling.  This is used by the Ada
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 16442ed..41f7d17 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -34,9 +34,8 @@ enum opt_pass_type
   IPA_PASS
 };
 
-/* Describe one pass; this is the common part shared across different pass
-   types.  */
-struct opt_pass
+/* Metadata for a pass, non-varying across all instances of a pass.  */
+struct pass_data
 {
   /* Optimization pass type.  */
   enum opt_pass_type type;
@@ -48,23 +47,13 @@ struct opt_pass
   /* The -fopt-info optimization group flags as defined in dumpfile.h. */
   unsigned int optinfo_flags;
 
-  /* If non-null, this pass and all sub-passes are executed only if
-     the function returns true.  */
-  bool (*gate) (void);
+  /* If true, this pass has its own implementation of the opt_pass::gate
+     method.  */
+  bool has_gate;
 
-  /* This is the code to run.  If null, then there should be sub-passes
-     otherwise this pass does nothing.  The return value contains
-     TODOs to execute in addition to those in TODO_flags_finish.   */
-  unsigned int (*execute) (void);
-
-  /* A list of sub-passes to run, dependent on gate predicate.  */
-  struct opt_pass *sub;
-
-  /* Next in the list of passes to run, independent of gate predicate.  */
-  struct opt_pass *next;
-
-  /* Static pass number, used as a fragment of the dump file name.  */
-  int static_pass_number;
+  /* If true, this pass has its own implementation of the opt_pass::execute
+     method.  */
+  bool has_execute;
 
   /* The timevar id associated with this pass.  */
   /* ??? Ideally would be dynamically assigned.  */
@@ -80,16 +69,72 @@ struct opt_pass
   unsigned int todo_flags_finish;
 };
 
+namespace gcc
+{
+  class context;
+} // namespace gcc
+
+/* An instance of a pass.  This is also "pass_data" to minimize the
+   changes in existing code.  */
+class opt_pass : public pass_data
+{
+public:
+  virtual ~opt_pass () { }
+
+  /* Create a copy of this pass.
+
+     Passes that can have multiple instances must provide their own
+     implementation of this, to ensure that any sharing of state between
+     this instance and the copy is "wired up" correctly.
+
+     The default implementation prints an error message and aborts.  */
+  virtual opt_pass *clone ();
+
+  /* If has_gate is set, this pass and all sub-passes are executed only if
+     the function returns true.  */
+  virtual bool gate ();
+
+  /* This is the code to run.  If has_execute is false, then there should
+     be sub-passes otherwise this pass does nothing.
+     The return value contains TODOs to execute in addition to those in
+     TODO_flags_finish.   */
+  virtual unsigned int execute ();
+
+protected:
+  opt_pass(const pass_data&, gcc::context *);
+
+public:
+  /* A list of sub-passes to run, dependent on gate predicate.  */
+  struct opt_pass *sub;
+
+  /* Next in the list of passes to run, independent of gate predicate.  */
+  struct opt_pass *next;
+
+  /* Static pass number, used as a fragment of the dump file name.  */
+  int static_pass_number;
+
+protected:
+  gcc::context *ctxt_;
+};
+
 /* Description of GIMPLE pass.  */
-struct gimple_opt_pass
+class gimple_opt_pass : public opt_pass
 {
-  struct opt_pass pass;
+protected:
+  gimple_opt_pass(const pass_data& data, gcc::context *ctxt)
+    : opt_pass(data, ctxt)
+  {
+  }
 };
 
 /* Description of RTL pass.  */
-struct rtl_opt_pass
+class rtl_opt_pass : public opt_pass
 {
-  struct opt_pass pass;
+protected:
+  rtl_opt_pass(const pass_data& data, gcc::context *ctxt)
+    : opt_pass(data, ctxt)
+  {
+  }
 };
 
 struct varpool_node;
@@ -98,10 +143,9 @@ struct lto_symtab_encoder_d;
 
 /* Description of IPA pass with generate summary, write, execute, read and
    transform stages.  */
-struct ipa_opt_pass_d
+class ipa_opt_pass_d : public opt_pass
 {
-  struct opt_pass pass;
-
+public:
   /* IPA passes can analyze function body and variable initializers
       using this hook and produce summary.  */
   void (*generate_summary) (void);
@@ -127,13 +171,42 @@ struct ipa_opt_pass_d
   unsigned int function_transform_todo_flags_start;
   unsigned int (*function_transform) (struct cgraph_node *);
   void (*variable_transform) (struct varpool_node *);
+
+protected:
+  ipa_opt_pass_d(const pass_data& data, gcc::context *ctxt,
+                 void (*generate_summary) (void),
+                 void (*write_summary) (void),
+                 void (*read_summary) (void),
+                 void (*write_optimization_summary) (void),
+                 void (*read_optimization_summary) (void),
+                 void (*stmt_fixup) (struct cgraph_node *, gimple *),
+                 unsigned int function_transform_todo_flags_start,
+                 unsigned int (*function_transform) (struct cgraph_node *),
+                 void (*variable_transform) (struct varpool_node *))
+    : opt_pass(data, ctxt),
+	       generate_summary(generate_summary),
+	       write_summary(write_summary),
+	       read_summary(read_summary),
+	       write_optimization_summary(write_optimization_summary),
+	       read_optimization_summary(read_optimization_summary),
+	       stmt_fixup(stmt_fixup),
+	       function_transform_todo_flags_start(
+	         function_transform_todo_flags_start),
+	       function_transform(function_transform),
+	       variable_transform(variable_transform)
+  {
+  }
 };
 
 /* Description of simple IPA pass.  Simple IPA passes have just one execute
    hook.  */
-struct simple_ipa_opt_pass
+class simple_ipa_opt_pass : public opt_pass
 {
-  struct opt_pass pass;
+protected:
+  simple_ipa_opt_pass(const pass_data& data, gcc::context *ctxt)
+    : opt_pass(data, ctxt)
+  {
+  }
 };
 
 /* Pass properties.  */
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (7 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 03/11] Handwritten part of conversion of passes to C++ classes David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-07-27 18:45   ` Bernhard Reutner-Fischer
  2013-08-01 21:45   ` Richard Henderson
  2013-07-26 15:29 ` [PATCH 09/11] Support "gcc" namespace in gengtype David Malcolm
                   ` (3 subsequent siblings)
  12 siblings, 2 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This patch makes gcc::pipeline and opt_pass instances be allocated
within the GC-heap, and adds traversal hooks for GC/PCH, so that passes
can own refs to other GC-allocated objects.

gcc/
	Make opt_pass and gcc::pipeline be GC-managed, so that pass
	instances can own GC refs.

	* Makefile.in (GTFILES): Add pipeline.h and tree-pass.h.
	* context.c (gcc::context::gt_ggc_mx): Traverse passes_.
	(gcc::context::gt_pch_nx): Likewise.
	(gcc::context::gt_pch_nx):  Likewise.
	* passes.c (opt_pass::operator new): New.
	(opt_pass::gt_ggc_mx): New.
	(opt_pass::gt_pch_nx): New.
	(opt_pass::gt_pch_nx_with_op): New.
	(gt_ggc_mx (opt_pass *)): New.
	(gt_pch_nx (opt_pass *)): New.
	(gt_pch_nx_opt_pass): New.
	(pipeline::operator new): New.
	(pipeline::gt_ggc_mx): New.
	(pipeline::gt_pch_nx): New.
	(pipeline::gt_pch_nx_with_op): New.
	(gt_ggc_mx (pipeline *)): New.
	(gt_pch_nx (pipeline *)): New.
	(gt_pch_nx_pipeline): New.
	* pipeline.h (class pipeline): Add GTY((user)) marking.
	(pipeline::operator new): New.
	(pipeline::gt_ggc_mx): New.
	(pipeline::gt_pch_nx): New.
	(pipeline::gt_pch_nx_with_op): New.
	(gt_ggc_mx (pipeline *)): New.
	(gt_pch_nx (pipeline *)): New.
	(gt_pch_nx_pipeline): New.
	* tree-pass.h (class opt_pass): Add GTY((user)) marking.
	(opt_pass::operator new): New.
	(opt_pass::gt_ggc_mx): New.
	(opt_pass::gt_pch_nx): New.
	(opt_pass::gt_pch_nx_with_op): New.
	(gt_ggc_mx (opt_pass *)): New.
	(gt_pch_nx (opt_pass *)): New.
	(gt_pch_nx_opt_pass): New.
---
 gcc/Makefile.in |   2 +
 gcc/context.c   |   9 ++--
 gcc/passes.c    | 160 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 gcc/pipeline.h  |  15 +++++-
 gcc/tree-pass.h |  19 ++++++-
 5 files changed, 198 insertions(+), 7 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index e9d6247..ec42f7b 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3820,6 +3820,8 @@ GTFILES = $(CPP_ID_DATA_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
   $(srcdir)/asan.c \
   $(srcdir)/tsan.c \
   $(srcdir)/context.h \
+  $(srcdir)/pipeline.h \
+  $(srcdir)/tree-pass.h \
   @all_gtfiles@
 
 # Compute the list of GT header files from the corresponding C sources,
diff --git a/gcc/context.c b/gcc/context.c
index 72135ed..4d9f0c6 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -42,20 +42,19 @@ gcc::context::context()
 void
 gcc::context::gt_ggc_mx ()
 {
-  /* Currently a no-op.  */
+  ::gt_ggc_mx (passes_);
 }
 
 void
 gcc::context::gt_pch_nx ()
 {
-  /* Currently a no-op.  */
+  ::gt_pch_nx (passes_);
 }
 
 void
-gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
-			 void *cookie ATTRIBUTE_UNUSED)
+gcc::context::gt_pch_nx (gt_pointer_operator op, void *cookie)
 {
-  /* Currently a no-op.  */
+  op (&passes_, cookie);
 }
 
 void gt_ggc_mx (gcc::context *ctxt)
diff --git a/gcc/passes.c b/gcc/passes.c
index ce5cdeb..dd1b0ba 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -82,6 +82,33 @@ struct opt_pass *current_pass;
 
 static void register_pass_name (struct opt_pass *, const char *);
 
+void*
+opt_pass::operator new (size_t sz)
+{
+  return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
+}
+
+void opt_pass::gt_ggc_mx ()
+{
+  ::gt_ggc_mx (ctxt_);
+  ::gt_ggc_mx (sub);
+  ::gt_ggc_mx (next);
+}
+
+void opt_pass::gt_pch_nx ()
+{
+  ::gt_pch_nx (ctxt_);
+  ::gt_pch_nx (sub);
+  ::gt_pch_nx (next);
+}
+
+void opt_pass::gt_pch_nx_with_op (gt_pointer_operator op, void *cookie)
+{
+  op (&(ctxt_), cookie);
+  op (&(sub), cookie);
+  op (&(next), cookie);
+}
+
 /* Most passes are single-instance (within their context) and thus don't
    need to implement cloning, but passes that support multiple instances
    *must* provide their own implementation of the clone method.
@@ -116,6 +143,118 @@ opt_pass::opt_pass(const pass_data &data, context *ctxt)
 {
 }
 
+void gt_ggc_mx (opt_pass *p)
+{
+  if (ggc_test_and_set_mark (p))
+    p->gt_ggc_mx ();
+}
+
+void gt_pch_nx (opt_pass *p)
+{
+  if (gt_pch_note_object (p, p, ::gt_pch_nx_opt_pass))
+    p->gt_pch_nx ();
+}
+
+void gt_pch_nx_opt_pass (void *this_obj, void *p,
+			 gt_pointer_operator op, void *cookie)
+{
+  opt_pass *pass = (opt_pass*)p;
+  if (p == this_obj)
+    pass->gt_pch_nx_with_op (op, cookie);
+}
+
+void*
+pipeline::operator new (size_t sz)
+{
+  return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
+}
+
+void
+pipeline::gt_ggc_mx ()
+{
+  ::gt_ggc_mx (all_passes);
+  ::gt_ggc_mx (all_small_ipa_passes);
+  ::gt_ggc_mx (all_lowering_passes);
+  ::gt_ggc_mx (all_regular_ipa_passes);
+  ::gt_ggc_mx (all_lto_gen_passes);
+  ::gt_ggc_mx (all_late_ipa_passes);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    ::gt_ggc_mx (passes_by_id[i]);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
+
+void
+pipeline::gt_pch_nx ()
+{
+  ::gt_pch_nx (all_passes);
+  ::gt_pch_nx (all_small_ipa_passes);
+  ::gt_pch_nx (all_lowering_passes);
+  ::gt_pch_nx (all_regular_ipa_passes);
+  ::gt_pch_nx (all_lto_gen_passes);
+  ::gt_pch_nx (all_late_ipa_passes);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    ::gt_pch_nx (passes_by_id[i]);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) ::gt_pch_nx (PASS ## _ ## NUM);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
+
+void
+pipeline::gt_pch_nx_with_op (gt_pointer_operator op, void *cookie)
+{
+  op (&(all_passes), cookie);
+  op (&(all_small_ipa_passes), cookie);
+  op (&(all_lowering_passes), cookie);
+  op (&(all_regular_ipa_passes), cookie);
+  op (&(all_lto_gen_passes), cookie);
+  op (&(all_late_ipa_passes), cookie);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    op (&(passes_by_id[i]), cookie);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) op (&(PASS ## _ ## NUM), cookie);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
 
 void
 pipeline::execute_early_local_passes ()
@@ -129,6 +268,27 @@ pipeline::execute_pass_mode_switching ()
   return pass_mode_switching_1->execute ();
 }
 
+void gt_ggc_mx (pipeline *p)
+{
+  if (ggc_test_and_set_mark (p))
+    p->gt_ggc_mx ();
+}
+
+void gt_pch_nx (pipeline *p)
+{
+  if (gt_pch_note_object (p, p, ::gt_pch_nx_pipeline))
+    p->gt_pch_nx ();
+
+}
+
+void gt_pch_nx_pipeline (void *this_obj, void *p,
+			 gt_pointer_operator op, void *cookie)
+{
+  pipeline *passes = (pipeline *)p;
+  if (p == this_obj)
+    passes->gt_pch_nx_with_op (op, cookie);
+}
+
 
 /* Call from anywhere to find out what pass this is.  Useful for
    printing out debugging information deep inside an service
diff --git a/gcc/pipeline.h b/gcc/pipeline.h
index 60bde5c..66b8212 100644
--- a/gcc/pipeline.h
+++ b/gcc/pipeline.h
@@ -44,11 +44,19 @@ namespace gcc {
 
 class context;
 
-class pipeline
+class GTY((user)) pipeline
 {
 public:
+  /* Ensure that instances are allocated in the GC-managed heap.  */
+  void *operator new (size_t sz);
+
   pipeline(context *ctxt);
 
+  /* GTY((user)) methods.  */
+  void gt_ggc_mx ();
+  void gt_pch_nx ();
+  void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
   void register_pass (struct register_pass_info *pass_info);
   void register_one_dump_file (struct opt_pass *pass);
 
@@ -122,5 +130,10 @@ private:
 
 } // namespace gcc
 
+extern void gt_ggc_mx (gcc::pipeline *passes);
+extern void gt_pch_nx (gcc::pipeline *passes);
+extern void gt_pch_nx_pipeline (void *this_obj, void *p,
+				gt_pointer_operator op, void *cookie);
+
 #endif /* ! GCC_PIPELINE_H */
 
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 41d5d92..c3e89d4 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -76,11 +76,22 @@ namespace gcc
 
 /* An instance of a pass.  This is also "pass_data" to minimize the
    changes in existing code.  */
-class opt_pass : public pass_data
+class GTY((user)) opt_pass : public pass_data
 {
 public:
+  /* Ensure that instances are allocated in the GC-managed heap.  */
+  void *operator new (size_t sz);
+
   virtual ~opt_pass () { }
 
+  /* GTY((user)) methods, to be called once per traversal.
+     opt_pass subclasses with additional GC-managed data should overide
+     these, chain up to the base class implementation, then walk their
+     extra fields.  */
+  virtual void gt_ggc_mx ();
+  virtual void gt_pch_nx ();
+  virtual void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
   /* Create a copy of this pass.
 
      Passes that can have multiple instances must provide their own
@@ -117,6 +128,12 @@ protected:
   gcc::context *ctxt_;
 };
 
+/* GTY((user)) methods.  */
+extern void gt_ggc_mx (opt_pass *p);
+extern void gt_pch_nx (opt_pass *p);
+extern void gt_pch_nx_opt_pass (void *this_obj, void *p,
+				gt_pointer_operator op, void *cookie);
+
 /* Description of GIMPLE pass.  */
 class gimple_opt_pass : public opt_pass
 {
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 06/11] Rewrite how instances of passes are cloned
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (4 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 05/11] Add -fno-rtti when building plugins David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-08-01 17:55   ` David Malcolm
  2013-07-26 15:05 ` [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c David Malcolm
                   ` (6 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

gcc/

	Rewrite how instances of passes are cloned to remove assumptions
	about their sizes (thus allowing pass subclasses to have
	additional data fields, albeit non-GC-managed ones at this point).

	* passes.c (make_pass_instance): Now that passes have clone
	methods, rewrite this function to eliminate XNEW and memcpy
	calls that used hardcoded sizes.  Since this function no longer
	creates pass instances, rename it to...
	(add_pass_instance): ...this.  Document the old way that passes
	were numbered and flagged, and rework this function to continue
	using it.
	(next_pass_1): Add an initial_pass argument for use by
	add_pass_instance.
	(position_pass): When adding multiple instances of a pass, use
	the pass's clone method, rather than relying on the XNEW/memcpy
	within the former make_pass_instance (now add_pass_instance).
	(pipeline::pipeline): When invoking next_pass_1, also supply the
	initial instance of the current pass within the pipeline.
---
 gcc/passes.c | 92 ++++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 55 insertions(+), 37 deletions(-)

diff --git a/gcc/passes.c b/gcc/passes.c
index ead41a8..ce5cdeb 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1167,68 +1167,77 @@ is_pass_explicitly_enabled_or_disabled (struct opt_pass *pass,
   return false;
 }
 
-/* Look at the static_pass_number and duplicate the pass
-   if it is already added to a list. */
 
-static struct opt_pass *
-make_pass_instance (struct opt_pass *pass, bool track_duplicates)
-{
-  /* A nonzero static_pass_number indicates that the
-     pass is already in the list.  */
-  if (pass->static_pass_number)
-    {
-      struct opt_pass *new_pass;
+/* Update static_pass_number for passes (and the flag
+   TODO_mark_first_instance).
 
-      if (pass->type == GIMPLE_PASS
-          || pass->type == RTL_PASS
-          || pass->type == SIMPLE_IPA_PASS)
-        {
-          new_pass = XNEW (struct opt_pass);
-          memcpy (new_pass, pass, sizeof (struct opt_pass));
-        }
-      else if (pass->type == IPA_PASS)
-        {
-          new_pass = (struct opt_pass *)XNEW (struct ipa_opt_pass_d);
-          memcpy (new_pass, pass, sizeof (struct ipa_opt_pass_d));
-        }
-      else
-        gcc_unreachable ();
+   Passes are constructed with static_pass_number preinitialized to 0
+
+   This field is used in two different ways: initially as instance numbers
+   of their kind, and then as ids within the entire pipeline.
+
+   Within pipeline::pipeline:
+
+   * In add_pass_instance(), as called by next_pass_1 in
+     NEXT_PASS in init_optimization_passes
 
-      new_pass->next = NULL;
+   * When the initial instance of a pass within a pipeline is seen,
+     it is flagged, and its static_pass_number is set to -1
 
+   * On subsequent times that it is seen, the static pass number
+     is decremented each time, so that if there are e.g. 4 dups,
+     they have static_pass_number -4, 2, 3, 4 respectively (note
+     how the initial one is negative and gives the count); these
+     can be thought of as instance numbers of the specific pass
+
+   * Within the register_dump_files () traversal, set_pass_for_id()
+     is called on each pass, using these instance numbers to create
+     dumpfile switches, and then overwriting them with a pass id,
+     which are global to the whole pass pipeline (based on
+     (TDI_end + current value of extra_dump_files_in_use) )  */
+
+static void
+add_pass_instance (struct opt_pass *new_pass, bool track_duplicates,
+		   opt_pass *initial_pass)
+{
+  /* Are we dealing with the first pass of its kind, or a clone?  */
+  if (new_pass != initial_pass)
+    {
+      /* We're dealing with a clone.  */
       new_pass->todo_flags_start &= ~TODO_mark_first_instance;
 
       /* Indicate to register_dump_files that this pass has duplicates,
          and so it should rename the dump file.  The first instance will
          be -1, and be number of duplicates = -static_pass_number - 1.
          Subsequent instances will be > 0 and just the duplicate number.  */
-      if ((pass->name && pass->name[0] != '*') || track_duplicates)
+      if ((new_pass->name && new_pass->name[0] != '*') || track_duplicates)
         {
-          pass->static_pass_number -= 1;
-          new_pass->static_pass_number = -pass->static_pass_number;
+          initial_pass->static_pass_number -= 1;
+          new_pass->static_pass_number = -initial_pass->static_pass_number;
 	}
-      return new_pass;
     }
   else
     {
-      pass->todo_flags_start |= TODO_mark_first_instance;
-      pass->static_pass_number = -1;
+      /* We're dealing with the first pass of its kind.  */
+      new_pass->todo_flags_start |= TODO_mark_first_instance;
+      new_pass->static_pass_number = -1;
 
-      invoke_plugin_callbacks (PLUGIN_NEW_PASS, pass);
+      invoke_plugin_callbacks (PLUGIN_NEW_PASS, new_pass);
     }
-  return pass;
 }
 
 /* Add a pass to the pass list. Duplicate the pass if it's already
    in the list.  */
 
 static struct opt_pass **
-next_pass_1 (struct opt_pass **list, struct opt_pass *pass)
+next_pass_1 (struct opt_pass **list, struct opt_pass *pass,
+	     struct opt_pass *initial_pass)
 {
   /* Every pass should have a name so that plugins can refer to them.  */
   gcc_assert (pass->name != NULL);
 
-  *list = make_pass_instance (pass, false);
+  add_pass_instance (pass, false, initial_pass);
+  *list = pass;
 
   return &(*list)->next;
 }
@@ -1278,7 +1287,16 @@ position_pass (struct register_pass_info *new_pass_info,
           struct opt_pass *new_pass;
           struct pass_list_node *new_pass_node;
 
-	  new_pass = make_pass_instance (new_pass_info->pass, true);
+	  if (new_pass_info->ref_pass_instance_number == 0)
+	    {
+	      new_pass = new_pass_info->pass->clone ();
+	      add_pass_instance (new_pass, true, new_pass_info->pass);
+	    }
+	  else
+	    {
+	      new_pass = new_pass_info->pass;
+	      add_pass_instance (new_pass, true, new_pass);
+	    }
 
           /* Insert the new pass instance based on the positioning op.  */
           switch (new_pass_info->pos_op)
@@ -1477,7 +1495,7 @@ pipeline::pipeline (context *ctxt)
         gcc_assert (PASS ## _1);                 \
         PASS ## _ ## NUM = PASS ## _1->clone (); \
       }                                          \
-    p = next_pass_1 (p, PASS ## _ ## NUM);  \
+    p = next_pass_1 (p, PASS ## _ ## NUM, PASS ## _1);  \
   } while (0)
 
 #define TERMINATE_PASS_LIST() \
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 00/11] Rewrite of pass management
@ 2013-07-26 15:05 David Malcolm
  2013-07-26 15:05 ` [PATCH 01/11] Introduce beginnings of a pipeline class David Malcolm
                   ` (12 more replies)
  0 siblings, 13 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

The following patch series eliminates the mutable global variables
representing GCC's passes, allowing for multiple compilation contexts in
one process, potentially with different combinations of passes
(e.g. JIT-compilation of JavaScript in one thread, JIT-compilation
of OpenGL shader programs in another) and with pass instances owning
additional data, including GC references.

The opt_pass hierarchy becomes a true C++ class hierarchy.

Patch 1 introduces a gcc::pipeline class and moves various non-GTY
globals relating to pass management into it.  The gcc::context gains its
first field: a pointer to the gcc::pipeline instance.

Patch 2 adds a new autogenerated file: pass-instances.def, built from
passes.def.  We need this in patch 3 and patch 11 in order to be able
to generate fields of class pipeline via macros (so that passes with
multiple instances can have unique field names).

Patches 3, 4 and 5 convert passes from C structs to C++ classes.

Patch 6 replaces XNEW/memcpy calls using fixed sizes with invocations
of opt_pass::clone, thus avoiding hard-coding the sizes of pass
subclasses, allowing opt_pass subclasses to have additional fields.

Patches 7 and 8 show a simple example of moving non-GTY state from being
global to being pass instance data.

Patch 9 adds enough special-casing to gengtype to allow it to cope
with types that are within the gcc namespace.

Patches 10 and 11 integrate the new classes with GTY (i.e. gcc::context,
the opt_pass class hierarchy and gcc::pipeline), so that pass instances
can own refs to GC-managed data.

I've successfully bootstrapped the *end result* of the patch series on
x86_64-unknown-linux-gnu: all testcases show the same results as an
unpatched build (relative to r201251).

OK for trunk? (I will bootstrap each of the above logical groups of
patches before committing).

Thanks
Dave


David Malcolm (11):
  Introduce beginnings of a pipeline class.
  Generate pass-instances.def
  Handwritten part of conversion of passes to C++ classes
  Automated conversion of passes to C++ classes
  Add -fno-rtti when building plugins.
  Rewrite how instances of passes are cloned
  Introduce virtual functions in
    testsuite/gcc.dg/plugin/one_time_plugin.c
  Example of converting global state to per-pass state
  Support "gcc" namespace in gengtype
  Make gcc::context be GC-managed
  Make opt_pass and gcc::pipeline be GC-managed

 gcc/Makefile.in                                    |  24 +-
 gcc/asan.c                                         | 109 ++-
 gcc/auto-inc-dec.c                                 |  52 +-
 gcc/bb-reorder.c                                   | 156 ++--
 gcc/bt-load.c                                      | 108 ++-
 gcc/cfgcleanup.c                                   | 102 ++-
 gcc/cfgexpand.c                                    |  55 +-
 gcc/cfgrtl.c                                       | 153 ++--
 gcc/cgraphbuild.c                                  | 157 ++--
 gcc/cgraphunit.c                                   |  28 +-
 gcc/combine-stack-adj.c                            |  52 +-
 gcc/combine.c                                      |  52 +-
 gcc/compare-elim.c                                 |  55 +-
 gcc/config/epiphany/epiphany.h                     |   4 +-
 gcc/config/epiphany/mode-switch-use.c              |  51 +-
 gcc/config/epiphany/resolve-sw-modes.c             |  53 +-
 gcc/config/i386/i386.c                             |  60 +-
 gcc/config/i386/t-i386                             |   2 +-
 gcc/config/mips/mips.c                             |  53 +-
 gcc/config/sparc/sparc.c                           |  54 +-
 gcc/context.c                                      |  46 +
 gcc/context.h                                      |  57 +-
 gcc/cprop.c                                        |  55 +-
 gcc/cse.c                                          | 164 ++--
 gcc/dce.c                                          | 104 ++-
 gcc/df-core.c                                      | 161 ++--
 gcc/dse.c                                          | 108 ++-
 gcc/dwarf2cfi.c                                    |  54 +-
 gcc/except.c                                       | 107 ++-
 gcc/final.c                                        | 204 +++--
 gcc/function.c                                     | 216 +++--
 gcc/fwprop.c                                       | 107 ++-
 gcc/gcse.c                                         | 108 ++-
 gcc/gen-pass-instances.awk                         |  66 ++
 gcc/gengtype.c                                     |  19 +-
 gcc/gimple-low.c                                   |  51 +-
 gcc/gimple-ssa-strength-reduction.c                |  54 +-
 gcc/ifcvt.c                                        | 157 ++--
 gcc/init-regs.c                                    |  52 +-
 gcc/ipa-cp.c                                       |  75 +-
 gcc/ipa-inline-analysis.c                          |  56 +-
 gcc/ipa-inline.c                                   | 122 ++-
 gcc/ipa-pure-const.c                               | 123 ++-
 gcc/ipa-reference.c                                |  72 +-
 gcc/ipa-split.c                                    | 104 ++-
 gcc/ipa.c                                          | 318 ++++---
 gcc/ira.c                                          | 102 ++-
 gcc/jump.c                                         |  51 +-
 gcc/loop-init.c                                    | 378 +++++---
 gcc/lower-subreg.c                                 | 106 ++-
 gcc/lto-cgraph.c                                   |   9 +-
 gcc/lto-streamer-out.c                             | 138 +--
 gcc/lto/Make-lang.in                               |   3 +-
 gcc/lto/lto.c                                      |   4 +-
 gcc/mode-switching.c                               |  53 +-
 gcc/modulo-sched.c                                 |  57 +-
 gcc/omp-low.c                                      | 163 ++--
 gcc/passes.c                                       | 764 ++++++++++++----
 gcc/pipeline.h                                     | 139 +++
 gcc/postreload-gcse.c                              |  53 +-
 gcc/postreload.c                                   |  53 +-
 gcc/predict.c                                      | 108 ++-
 gcc/recog.c                                        | 325 ++++---
 gcc/ree.c                                          |  53 +-
 gcc/reg-stack.c                                    | 102 ++-
 gcc/regcprop.c                                     |  53 +-
 gcc/reginfo.c                                      |  51 +-
 gcc/regmove.c                                      |  52 +-
 gcc/regrename.c                                    |  53 +-
 gcc/reorg.c                                        | 104 ++-
 gcc/sched-rgn.c                                    | 112 ++-
 gcc/stack-ptr-mod.c                                |  51 +-
 gcc/statistics.c                                   |   7 +-
 gcc/store-motion.c                                 |  54 +-
 gcc/testsuite/c-c++-common/vector-init-2.c         |   2 +-
 gcc/testsuite/g++.dg/plugin/dumb_plugin.c          |  55 +-
 gcc/testsuite/g++.dg/plugin/selfassign.c           |  55 +-
 gcc/testsuite/gcc.dg/cpp/mac-eol-at-eof.c          |   2 +-
 gcc/testsuite/gcc.dg/plugin/one_time_plugin.c      |  66 +-
 gcc/testsuite/gcc.dg/plugin/selfassign.c           |  55 +-
 gcc/testsuite/gcc.target/mips/mulsize-3.c          |   2 +-
 gcc/testsuite/gcc.target/mips/mulsize-4.c          |   2 +-
 .../gcc.target/x86_64/abi/callabi/callabi.h        |   2 +-
 gcc/testsuite/lib/plugin-support.exp               |   2 +-
 gcc/toplev.c                                       |   7 +-
 gcc/toplev.h                                       |   1 -
 gcc/tracer.c                                       |  53 +-
 gcc/trans-mem.c                                    | 377 +++++---
 gcc/tree-call-cdce.c                               |  52 +-
 gcc/tree-cfg.c                                     | 267 ++++--
 gcc/tree-cfgcleanup.c                              |  53 +-
 gcc/tree-complex.c                                 | 106 ++-
 gcc/tree-eh.c                                      | 261 ++++--
 gcc/tree-emutls.c                                  |  52 +-
 gcc/tree-if-conv.c                                 |  53 +-
 gcc/tree-into-ssa.c                                |  54 +-
 gcc/tree-loop-distribution.c                       |  52 +-
 gcc/tree-mudflap.c                                 | 106 ++-
 gcc/tree-nomudflap.c                               | 102 ++-
 gcc/tree-nrv.c                                     | 103 ++-
 gcc/tree-object-size.c                             |  52 +-
 gcc/tree-optimize.c                                | 105 ++-
 gcc/tree-pass.h                                    | 641 ++++++++------
 gcc/tree-profile.c                                 |  52 +-
 gcc/tree-sra.c                                     | 158 ++--
 gcc/tree-ssa-ccp.c                                 | 109 ++-
 gcc/tree-ssa-copy.c                                |  56 +-
 gcc/tree-ssa-copyrename.c                          |  53 +-
 gcc/tree-ssa-dce.c                                 | 160 ++--
 gcc/tree-ssa-dom.c                                 | 116 ++-
 gcc/tree-ssa-dse.c                                 |  53 +-
 gcc/tree-ssa-forwprop.c                            |  54 +-
 gcc/tree-ssa-ifcombine.c                           |  53 +-
 gcc/tree-ssa-loop-ch.c                             |  55 +-
 gcc/tree-ssa-loop.c                                | 973 ++++++++++++++-------
 gcc/tree-ssa-math-opts.c                           | 215 +++--
 gcc/tree-ssa-phiopt.c                              | 111 ++-
 gcc/tree-ssa-phiprop.c                             |  53 +-
 gcc/tree-ssa-pre.c                                 | 106 ++-
 gcc/tree-ssa-reassoc.c                             |  57 +-
 gcc/tree-ssa-sink.c                                |  56 +-
 gcc/tree-ssa-strlen.c                              |  52 +-
 gcc/tree-ssa-structalias.c                         | 160 ++--
 gcc/tree-ssa-uncprop.c                             |  53 +-
 gcc/tree-ssa-uninit.c                              |  53 +-
 gcc/tree-ssa.c                                     | 153 ++--
 gcc/tree-ssanames.c                                |  51 +-
 gcc/tree-stdarg.c                                  |  52 +-
 gcc/tree-switch-conversion.c                       |  57 +-
 gcc/tree-tailcall.c                                | 105 ++-
 gcc/tree-vect-generic.c                            | 116 ++-
 gcc/tree-vectorizer.c                              | 109 ++-
 gcc/tree-vrp.c                                     |  58 +-
 gcc/tree.c                                         |  53 +-
 gcc/tsan.c                                         | 105 ++-
 gcc/var-tracking.c                                 |  55 +-
 gcc/web.c                                          |  52 +-
 137 files changed, 9295 insertions(+), 4557 deletions(-)
 create mode 100644 gcc/gen-pass-instances.awk
 create mode 100644 gcc/pipeline.h

-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 01/11] Introduce beginnings of a pipeline class.
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-07-29 20:00   ` Jeff Law
  2013-07-26 15:05 ` [PATCH 04/11] Automated conversion of passes to C++ classes David Malcolm
                   ` (11 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This patch introduces a gcc::pipeline class and moves various non-GTY
globals relating to pass management into it.  The gcc::context gains its
first field: a pointer to the gcc::pipeline instance.

It was previously sent as:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01090.html
as part of:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01088.html
and Martin Jambor raised some stylistic concerns about it:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01155.html
I'm reposting it here so that people can see how it fits into the later
patches.

gcc/
	* Makefile.in (PIPELINE_H): New.
	(lto-cgraph.o): Depend on CONTEXT_H and PIPELINE_H.
	(passes.o): Likewise.
	(statistics.o): Likewise.
	(cgraphunit.o): Likewise.
	(context.o): Depend on PIPELINE_H.

	* pipeline.h: New.

	* cgraphunit.c (cgraph_add_new_function): Update for moves
	of globals to fields of pipeline.
	(analyze_function): Likewise.
	(expand_function): Likewise.
	(ipa_passes): Likewise.
	(compile): Likewise.

	* context.c (context::context): New.
	* context.h  (context::context): New.
	(context::get_passes): New.
	(context::passes_): New.

	* lto-cgraph.c (input_node): Update for moves of globals to
	fields of pipeline.

	* passes.c (all_passes): Remove, in favor of a field of the
	same name within the new class pipeline.
	(all_small_ipa_passes): Likewise.
	(all_lowering_passes): Likewise.
	(all_regular_ipa_passes): Likewise.
	(all_late_ipa_passes): Likewise.
	(all_lto_gen_passes): Likewise.
	(passes_by_id): Likewise.
	(passes_by_id_size): Likewise.
	(gcc_pass_lists): Remove, in favor of "pass_lists" field within
	the new class pipeline.
	(set_pass_for_id): Convert to...
	(pipeline::set_pass_for_id): ...method.
	(get_pass_for_id): Convert to...
	(pipeline::get_pass_for_id): ...method.
	(register_one_dump_file): Move body of implementation into...
	(pipeline::register_one_dump_file): ...here.
	(register_dump_files_1): Convert to...
	(pipeline::register_dump_files_1): ...method.
	(register_dump_files): Convert to...
	(pipeline::register_dump_files): ...method.
	(create_pass_tab): Update for moves of globals to fields of
	pipeline.
	(dump_passes): Move body of implementation into...
	(pipeline::dump_passes): ...here.
	(register_pass): Move body of implementation into...
	(pipeline::register_pass): ...here.
	(init_optimization_passes): Convert into...
	(pipeline::pipeline): ...constructor for new pipeline class, and
	initialize the pass_lists array.
	(check_profile_consistency): Update for moves of globals to
	fields of pipeline.
	(dump_profile_report): Move body of implementation into...
	(pipeline::dump_profile_report): ...here.
	(ipa_write_summaries_1): Update for moves of pass lists from
	being globals to fields of pipeline.
	(ipa_write_optimization_summaries): Likewise.
	(ipa_read_summaries):  Likewise.
	(ipa_read_optimization_summaries): Likewise.
	(execute_all_ipa_stmt_fixups): Likewise.

	* statistics.c (statistics_fini): Update for moves of globals to
	fields of pipeline.

	* toplev.c (general_init): Replace call to
	init_optimization_passes with construction of the pipeline
	instance.

	* tree-pass.h (all_passes): Remove, in favor of a field of the
	same name within the new class pipeline.
	(all_small_ipa_passes): Likewise.
	(all_lowering_passes): Likewise.
	(all_regular_ipa_passes): Likewise.
	(all_lto_gen_passes): Likewise.
	(all_late_ipa_passes): Likewise.
	(passes_by_id): Likewise.
	(passes_by_id_size): Likewise.
	(gcc_pass_lists): Remove, in favor of "pass_lists" field within
	the new class pipeline.
	(get_pass_for_id): Remove.

gcc/lto/

	* Make-lang.in (lto/lto.o:): Depend on CONTEXT_H and
	PIPELINE_H.

	* lto.c (do_whole_program_analysis): Update for move of
	all_regular_ipa_passes from a global to a field of class
	pipeline.
---
 gcc/Makefile.in      | 15 +++++---
 gcc/cgraphunit.c     | 22 +++++++-----
 gcc/context.c        |  6 ++++
 gcc/context.h        | 11 +++++-
 gcc/lto-cgraph.c     |  7 ++--
 gcc/lto/Make-lang.in |  3 +-
 gcc/lto/lto.c        |  4 ++-
 gcc/passes.c         | 97 +++++++++++++++++++++++++++++++++++-----------------
 gcc/pipeline.h       | 89 +++++++++++++++++++++++++++++++++++++++++++++++
 gcc/statistics.c     |  7 ++--
 gcc/toplev.c         |  4 +--
 gcc/tree-pass.h      | 29 ----------------
 12 files changed, 212 insertions(+), 82 deletions(-)
 create mode 100644 gcc/pipeline.h

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index fb0cb4b..0b28c2d 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -987,6 +987,7 @@ PLUGIN_VERSION_H = plugin-version.h configargs.h
 LIBFUNCS_H = libfuncs.h $(HASHTAB_H)
 GRAPHITE_HTAB_H = graphite-htab.h graphite-clast-to-gimple.h $(HASH_TABLE_H)
 CONTEXT_H = context.h
+PIPELINE_H = pipeline.h
 
 #\f
 # Now figure out from those variables how to compile and link.
@@ -2183,7 +2184,8 @@ lto-cgraph.o: lto-cgraph.c $(CONFIG_H) $(SYSTEM_H) coretypes.h   \
    $(HASHTAB_H) langhooks.h $(BASIC_BLOCK_H) \
    $(TREE_FLOW_H) $(CGRAPH_H) $(FUNCTION_H) $(GGC_H) $(DIAGNOSTIC_CORE_H) \
    $(EXCEPT_H) $(TIMEVAR_H) pointer-set.h $(LTO_STREAMER_H) \
-   $(GCOV_IO_H) $(DATA_STREAMER_H) $(TREE_STREAMER_H) $(TREE_PASS_H) profile.h
+   $(GCOV_IO_H) $(DATA_STREAMER_H) $(TREE_STREAMER_H) $(TREE_PASS_H) \
+   profile.h $(CONTEXT_H) $(PIPELINE_H)
 lto-streamer-in.o: lto-streamer-in.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
    $(TM_H) toplev.h $(DIAGNOSTIC_CORE_H) $(EXPR_H) $(FLAGS_H) $(PARAMS_H) \
    input.h $(HASHTAB_H) $(BASIC_BLOCK_H) $(TREE_FLOW_H) $(TREE_PASS_H) \
@@ -2745,7 +2747,8 @@ passes.o : passes.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    hosthooks.h $(CGRAPH_H) $(COVERAGE_H) $(TREE_PASS_H) $(TREE_DUMP_H) \
    $(GGC_H) $(OPTS_H) $(TREE_FLOW_H) $(TREE_INLINE_H) \
    gt-passes.h $(DF_H) $(PREDICT_H) $(LTO_STREAMER_H) \
-   $(PLUGIN_H) $(IPA_UTILS_H) passes.def
+   $(PLUGIN_H) $(IPA_UTILS_H) passes.def \
+   $(CONTEXT_H) $(PIPELINE_H)
 
 plugin.o : plugin.c $(PLUGIN_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
    $(HASH_TABLE_H) $(DIAGNOSTIC_CORE_H) $(TREE_H) $(TREE_PASS_H) \
@@ -2786,7 +2789,8 @@ function.o : function.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_ERROR_
    $(TREE_PASS_H) $(DF_H) $(PARAMS_H) bb-reorder.h \
    $(COMMON_TARGET_H)
 statistics.o : statistics.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
-   $(TREE_PASS_H) $(TREE_DUMP_H) $(HASH_TABLE_H) statistics.h $(FUNCTION_H)
+   $(TREE_PASS_H) $(TREE_DUMP_H) $(HASH_TABLE_H) statistics.h \
+   $(FUNCTION_H) $(CONTEXT_H) $(PIPELINE_H)
 stmt.o : stmt.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(DUMPFILE_H) $(TM_H) \
    $(RTL_H) \
    $(TREE_H) $(FLAGS_H) $(FUNCTION_H) insn-config.h hard-reg-set.h $(EXPR_H) \
@@ -2908,7 +2912,8 @@ cgraphunit.o : cgraphunit.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    $(FIBHEAP_H) output.h $(PARAMS_H) $(RTL_H) $(IPA_PROP_H) \
    gt-cgraphunit.h tree-iterator.h $(COVERAGE_H) $(TREE_DUMP_H) \
    $(GIMPLE_PRETTY_PRINT_H) $(IPA_INLINE_H) $(IPA_UTILS_H) $(CFGLOOP_H) \
-   $(LTO_STREAMER_H) output.h $(REGSET_H) $(EXCEPT_H) $(GCC_PLUGIN_H) plugin.h
+   $(LTO_STREAMER_H) output.h $(REGSET_H) $(EXCEPT_H) $(GCC_PLUGIN_H) \
+   plugin.h $(CONTEXT_H) $(PIPELINE_H)
 cgraphclones.o : cgraphclones.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    $(TREE_H) langhooks.h $(TREE_INLINE_H) toplev.h $(DIAGNOSTIC_CORE_H) $(FLAGS_H) $(GGC_H) \
    $(TARGET_H) $(CGRAPH_H) intl.h pointer-set.h $(FUNCTION_H) $(GIMPLE_H) \
@@ -3490,7 +3495,7 @@ $(out_object_file): $(out_file) $(CONFIG_H) coretypes.h $(TM_H) $(TREE_H) \
 	$(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) \
 		$(out_file) $(OUTPUT_OPTION)
 context.o: context.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(GGC_H) \
-   $(CONTEXT_H)
+   $(CONTEXT_H) $(PIPELINE_H)
 
 $(common_out_object_file): $(common_out_file) $(CONFIG_H) $(SYSTEM_H) \
     coretypes.h $(COMMON_TARGET_H) $(COMMON_TARGET_DEF_H) $(PARAMS_H) \
diff --git a/gcc/cgraphunit.c b/gcc/cgraphunit.c
index b82c2e0..dc489fb 100644
--- a/gcc/cgraphunit.c
+++ b/gcc/cgraphunit.c
@@ -194,6 +194,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "except.h"
 #include "cfgloop.h"
 #include "regset.h"     /* FIXME: For reg_obstack.  */
+#include "context.h"
+#include "pipeline.h"
 
 /* Queue of cgraph nodes scheduled to be added into cgraph.  This is a
    secondary queue used during optimization to accommodate passes that
@@ -478,6 +480,7 @@ cgraph_finalize_function (tree decl, bool nested)
 void
 cgraph_add_new_function (tree fndecl, bool lowered)
 {
+  gcc::pipeline &passes = g->get_passes ();
   struct cgraph_node *node;
   switch (cgraph_state)
     {
@@ -508,7 +511,7 @@ cgraph_add_new_function (tree fndecl, bool lowered)
 	    push_cfun (DECL_STRUCT_FUNCTION (fndecl));
 	    gimple_register_cfg_hooks ();
 	    bitmap_obstack_initialize (NULL);
-	    execute_pass_list (all_lowering_passes);
+	    execute_pass_list (passes.all_lowering_passes);
 	    execute_pass_list (pass_early_local_passes.pass.sub);
 	    bitmap_obstack_release (NULL);
 	    pop_cfun ();
@@ -640,7 +643,7 @@ analyze_function (struct cgraph_node *node)
 
 	  gimple_register_cfg_hooks ();
 	  bitmap_obstack_initialize (NULL);
-	  execute_pass_list (all_lowering_passes);
+	  execute_pass_list (g->get_passes ().all_lowering_passes);
 	  free_dominance_info (CDI_POST_DOMINATORS);
 	  free_dominance_info (CDI_DOMINATORS);
 	  compact_blocks ();
@@ -1588,7 +1591,7 @@ expand_function (struct cgraph_node *node)
   /* Signal the start of passes.  */
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_START, NULL);
 
-  execute_pass_list (all_passes);
+  execute_pass_list (g->get_passes ().all_passes);
 
   /* Signal the end of passes.  */
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_END, NULL);
@@ -1807,6 +1810,8 @@ output_in_order (void)
 static void
 ipa_passes (void)
 {
+  gcc::pipeline &passes = g->get_passes ();
+
   set_cfun (NULL);
   current_function_decl = NULL;
   gimple_register_cfg_hooks ();
@@ -1816,7 +1821,7 @@ ipa_passes (void)
 
   if (!in_lto_p)
     {
-      execute_ipa_pass_list (all_small_ipa_passes);
+      execute_ipa_pass_list (passes.all_small_ipa_passes);
       if (seen_error ())
 	return;
     }
@@ -1843,14 +1848,15 @@ ipa_passes (void)
       cgraph_process_new_functions ();
 
       execute_ipa_summary_passes
-	((struct ipa_opt_pass_d *) all_regular_ipa_passes);
+	((struct ipa_opt_pass_d *) passes.all_regular_ipa_passes);
     }
 
   /* Some targets need to handle LTO assembler output specially.  */
   if (flag_generate_lto)
     targetm.asm_out.lto_start ();
 
-  execute_ipa_summary_passes ((struct ipa_opt_pass_d *) all_lto_gen_passes);
+  execute_ipa_summary_passes ((struct ipa_opt_pass_d *)
+			      passes.all_lto_gen_passes);
 
   if (!in_lto_p)
     ipa_write_summaries ();
@@ -1859,7 +1865,7 @@ ipa_passes (void)
     targetm.asm_out.lto_end ();
 
   if (!flag_ltrans && (in_lto_p || !flag_lto || flag_fat_lto_objects))
-    execute_ipa_pass_list (all_regular_ipa_passes);
+    execute_ipa_pass_list (passes.all_regular_ipa_passes);
   invoke_plugin_callbacks (PLUGIN_ALL_IPA_PASSES_END, NULL);
 
   bitmap_obstack_release (NULL);
@@ -1985,7 +1991,7 @@ compile (void)
 
   cgraph_materialize_all_clones ();
   bitmap_obstack_initialize (NULL);
-  execute_ipa_pass_list (all_late_ipa_passes);
+  execute_ipa_pass_list (g->get_passes ().all_late_ipa_passes);
   symtab_remove_unreachable_nodes (true, dump_file);
 #ifdef ENABLE_CHECKING
   verify_symtab ();
diff --git a/gcc/context.c b/gcc/context.c
index 76e0dde..8ec2e60 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -22,6 +22,12 @@ along with GCC; see the file COPYING3.  If not see
 #include "coretypes.h"
 #include "ggc.h"
 #include "context.h"
+#include "pipeline.h"
 
 /* The singleton holder of global state: */
 gcc::context *g;
+
+gcc::context::context()
+{
+  passes_ = new gcc::pipeline(this);
+}
diff --git a/gcc/context.h b/gcc/context.h
index 3caf02f..a83f7b2 100644
--- a/gcc/context.h
+++ b/gcc/context.h
@@ -22,14 +22,23 @@ along with GCC; see the file COPYING3.  If not see
 
 namespace gcc {
 
+class pipeline;
+
 /* GCC's internal state can be divided into zero or more
    "parallel universe" of state; an instance of this class is one such
    context of state.  */
 class context
 {
 public:
+  context();
+
+  /* Pass-management.  */
+
+  pipeline &get_passes () { gcc_assert (passes_); return *passes_; }
 
-  /* Currently empty.  */
+private:
+  /* Pass-management.  */
+  pipeline *passes_;
 
 }; // class context
 
diff --git a/gcc/lto-cgraph.c b/gcc/lto-cgraph.c
index d60213a..90e3a2b 100644
--- a/gcc/lto-cgraph.c
+++ b/gcc/lto-cgraph.c
@@ -47,6 +47,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "gcov-io.h"
 #include "tree-pass.h"
 #include "profile.h"
+#include "context.h"
+#include "pipeline.h"
 
 static void output_cgraph_opt_summary (void);
 static void input_cgraph_opt_summary (vec<symtab_node>  nodes);
@@ -936,6 +938,7 @@ input_node (struct lto_file_decl_data *file_data,
 	    enum LTO_symtab_tags tag,
 	    vec<symtab_node> nodes)
 {
+  gcc::pipeline &passes = g->get_passes ();
   tree fn_decl;
   struct cgraph_node *node;
   struct bitpack_d bp;
@@ -981,8 +984,8 @@ input_node (struct lto_file_decl_data *file_data,
       struct opt_pass *pass;
       int pid = streamer_read_hwi (ib);
 
-      gcc_assert (pid < passes_by_id_size);
-      pass = passes_by_id[pid];
+      gcc_assert (pid < passes.passes_by_id_size);
+      pass = passes.passes_by_id[pid];
       node->ipa_transforms_to_apply.safe_push ((struct ipa_opt_pass_d *) pass);
     }
 
diff --git a/gcc/lto/Make-lang.in b/gcc/lto/Make-lang.in
index 5f2f475..1acd176 100644
--- a/gcc/lto/Make-lang.in
+++ b/gcc/lto/Make-lang.in
@@ -85,7 +85,8 @@ lto/lto.o: lto/lto.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(OPTS_H) \
 	langhooks.h $(VEC_H) $(BITMAP_H) pointer-set.h $(IPA_PROP_H) \
 	$(COMMON_H) debug.h $(GIMPLE_H) $(LTO_H) $(LTO_TREE_H) \
 	$(LTO_TAGS_H) $(LTO_STREAMER_H) $(SPLAY_TREE_H) gt-lto-lto.h \
-	$(TREE_STREAMER_H) $(DATA_STREAMER_H) lto/lto-partition.h
+	$(TREE_STREAMER_H) $(DATA_STREAMER_H) lto/lto-partition.h \
+	$(CONTEXT_H) $(PIPELINE_H)
 lto/lto-partition.o: lto/lto-partition.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
 	toplev.h $(TREE_H) $(TM_H) \
 	$(CGRAPH_H) $(TIMEVAR_H) \
diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
index c0f9328..e7ca937 100644
--- a/gcc/lto/lto.c
+++ b/gcc/lto/lto.c
@@ -46,6 +46,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "splay-tree.h"
 #include "lto-partition.h"
 #include "data-streamer.h"
+#include "context.h"
+#include "pipeline.h"
 
 static GTY(()) tree first_personality_decl;
 
@@ -3694,7 +3696,7 @@ do_whole_program_analysis (void)
   bitmap_obstack_initialize (NULL);
   cgraph_state = CGRAPH_STATE_IPA_SSA;
 
-  execute_ipa_pass_list (all_regular_ipa_passes);
+  execute_ipa_pass_list (g->get_passes ().all_regular_ipa_passes);
   symtab_remove_unreachable_nodes (false, dump_file);
 
   if (cgraph_dump_file)
diff --git a/gcc/passes.c b/gcc/passes.c
index 94fb586..e625bf2 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -70,6 +70,10 @@ along with GCC; see the file COPYING3.  If not see
 #include "plugin.h"
 #include "ipa-utils.h"
 #include "tree-pretty-print.h" /* for dump_function_header */
+#include "context.h"
+#include "pipeline.h"
+
+using namespace gcc;
 
 /* This is used for debugging.  It allows the current pass to printed
    from anywhere in compilation.
@@ -439,23 +443,11 @@ static struct rtl_opt_pass pass_postreload =
 
 
 
-/* The root of the compilation pass tree, once constructed.  */
-struct opt_pass *all_passes, *all_small_ipa_passes, *all_lowering_passes,
-  *all_regular_ipa_passes, *all_late_ipa_passes, *all_lto_gen_passes;
-
-/* This is used by plugins, and should also be used in register_pass.  */
-#define DEF_PASS_LIST(LIST) &LIST,
-struct opt_pass **gcc_pass_lists[] = { GCC_PASS_LISTS NULL };
-#undef DEF_PASS_LIST
-
-/* A map from static pass id to optimization pass.  */
-struct opt_pass **passes_by_id;
-int passes_by_id_size;
-
 /* Set the static pass number of pass PASS to ID and record that
    in the mapping from static pass number to pass.  */
 
-static void
+void
+pipeline::
 set_pass_for_id (int id, struct opt_pass *pass)
 {
   pass->static_pass_number = id;
@@ -472,7 +464,7 @@ set_pass_for_id (int id, struct opt_pass *pass)
 /* Return the pass with the static pass number ID.  */
 
 struct opt_pass *
-get_pass_for_id (int id)
+pipeline::get_pass_for_id (int id) const
 {
   if (id >= passes_by_id_size)
     return NULL;
@@ -486,6 +478,12 @@ get_pass_for_id (int id)
 void
 register_one_dump_file (struct opt_pass *pass)
 {
+  g->get_passes ().register_one_dump_file (pass);
+}
+
+void
+pipeline::register_one_dump_file (struct opt_pass *pass)
+{
   char *dot_name, *flag_name, *glob_name;
   const char *name, *full_name, *prefix;
   char num[10];
@@ -535,7 +533,8 @@ register_one_dump_file (struct opt_pass *pass)
 
 /* Recursive worker function for register_dump_files.  */
 
-static int
+int
+pipeline::
 register_dump_files_1 (struct opt_pass *pass, int properties)
 {
   do
@@ -567,7 +566,8 @@ register_dump_files_1 (struct opt_pass *pass, int properties)
    PROPERTIES reflects the properties that are guaranteed to be available at
    the beginning of the pipeline.  */
 
-static void
+void
+pipeline::
 register_dump_files (struct opt_pass *pass,int properties)
 {
   pass->properties_required |= properties;
@@ -663,7 +663,7 @@ create_pass_tab (void)
   if (!flag_dump_passes)
     return;
 
-  pass_tab.safe_grow_cleared (passes_by_id_size + 1);
+  pass_tab.safe_grow_cleared (g->get_passes ().passes_by_id_size + 1);
   name_to_pass_map.traverse <void *, passes_pass_traverse> (NULL);
 }
 
@@ -714,6 +714,12 @@ dump_pass_list (struct opt_pass *pass, int indent)
 void
 dump_passes (void)
 {
+  g->get_passes ().dump_passes ();
+}
+
+void
+pipeline::dump_passes () const
+{
   struct cgraph_node *n, *node = NULL;
 
   create_pass_tab();
@@ -1188,6 +1194,13 @@ position_pass (struct register_pass_info *new_pass_info,
 void
 register_pass (struct register_pass_info *pass_info)
 {
+  g->get_passes ().register_pass (pass_info);
+
+}
+
+void
+pipeline::register_pass (struct register_pass_info *pass_info)
+{
   bool all_instances, success;
 
   /* The checks below could fail in buggy plugins.  Existing GCC
@@ -1277,11 +1290,21 @@ register_pass (struct register_pass_info *pass_info)
 				        -> all_passes
 */
 
-void
-init_optimization_passes (void)
+pipeline::pipeline (context *ctxt)
+: all_passes(NULL), all_small_ipa_passes(NULL), all_lowering_passes(NULL),
+  all_regular_ipa_passes(NULL), all_lto_gen_passes(NULL),
+  all_late_ipa_passes(NULL), passes_by_id(NULL), passes_by_id_size(0),
+  ctxt_(ctxt)
 {
   struct opt_pass **p;
 
+  /* Initialize the pass_lists array.  */
+#define DEF_PASS_LIST(LIST) pass_lists[PASS_LIST_NO_##LIST] = &LIST;
+  GCC_PASS_LISTS
+#undef DEF_PASS_LIST
+
+  /* Build the tree of passes.  */
+
 #define INSERT_PASSES_AFTER(PASS) \
   p = &(PASS);
 
@@ -1432,12 +1455,13 @@ static struct profile_record *profile_record;
 static void
 check_profile_consistency (int index, int subpass, bool run)
 {
+  pipeline &passes = g->get_passes ();
   if (index == -1)
     return;
   if (!profile_record)
     profile_record = XCNEWVEC (struct profile_record,
-			       passes_by_id_size);
-  gcc_assert (index < passes_by_id_size && index >= 0);
+			       passes.passes_by_id_size);
+  gcc_assert (index < passes.passes_by_id_size && index >= 0);
   gcc_assert (subpass < 2);
   profile_record[index].run |= run;
   account_profile_record (&profile_record[index], subpass);
@@ -1448,6 +1472,12 @@ check_profile_consistency (int index, int subpass, bool run)
 void
 dump_profile_report (void)
 {
+  g->get_passes ().dump_profile_report ();
+}
+
+void
+pipeline::dump_profile_report () const
+{
   int i, j;
   int last_freq_in = 0, last_count_in = 0, last_freq_out = 0, last_count_out = 0;
   gcov_type last_time = 0, last_size = 0;
@@ -2067,14 +2097,15 @@ ipa_write_summaries_2 (struct opt_pass *pass, struct lto_out_decl_state *state)
 static void
 ipa_write_summaries_1 (lto_symtab_encoder_t encoder)
 {
+  pipeline &passes = g->get_passes ();
   struct lto_out_decl_state *state = lto_new_out_decl_state ();
   state->symtab_node_encoder = encoder;
 
   lto_push_out_decl_state (state);
 
   gcc_assert (!flag_wpa);
-  ipa_write_summaries_2 (all_regular_ipa_passes, state);
-  ipa_write_summaries_2 (all_lto_gen_passes, state);
+  ipa_write_summaries_2 (passes.all_regular_ipa_passes, state);
+  ipa_write_summaries_2 (passes.all_lto_gen_passes, state);
 
   gcc_assert (lto_get_out_decl_state () == state);
   lto_pop_out_decl_state ();
@@ -2205,8 +2236,9 @@ ipa_write_optimization_summaries (lto_symtab_encoder_t encoder)
     }
 
   gcc_assert (flag_wpa);
-  ipa_write_optimization_summaries_1 (all_regular_ipa_passes, state);
-  ipa_write_optimization_summaries_1 (all_lto_gen_passes, state);
+  pipeline &passes = g->get_passes ();
+  ipa_write_optimization_summaries_1 (passes.all_regular_ipa_passes, state);
+  ipa_write_optimization_summaries_1 (passes.all_lto_gen_passes, state);
 
   gcc_assert (lto_get_out_decl_state () == state);
   lto_pop_out_decl_state ();
@@ -2259,8 +2291,9 @@ ipa_read_summaries_1 (struct opt_pass *pass)
 void
 ipa_read_summaries (void)
 {
-  ipa_read_summaries_1 (all_regular_ipa_passes);
-  ipa_read_summaries_1 (all_lto_gen_passes);
+  pipeline &passes = g->get_passes ();
+  ipa_read_summaries_1 (passes.all_regular_ipa_passes);
+  ipa_read_summaries_1 (passes.all_lto_gen_passes);
 }
 
 /* Same as execute_pass_list but assume that subpasses of IPA passes
@@ -2308,8 +2341,9 @@ ipa_read_optimization_summaries_1 (struct opt_pass *pass)
 void
 ipa_read_optimization_summaries (void)
 {
-  ipa_read_optimization_summaries_1 (all_regular_ipa_passes);
-  ipa_read_optimization_summaries_1 (all_lto_gen_passes);
+  pipeline &passes = g->get_passes ();
+  ipa_read_optimization_summaries_1 (passes.all_regular_ipa_passes);
+  ipa_read_optimization_summaries_1 (passes.all_lto_gen_passes);
 }
 
 /* Same as execute_pass_list but assume that subpasses of IPA passes
@@ -2384,7 +2418,8 @@ execute_ipa_stmt_fixups (struct opt_pass *pass,
 void
 execute_all_ipa_stmt_fixups (struct cgraph_node *node, gimple *stmts)
 {
-  execute_ipa_stmt_fixups (all_regular_ipa_passes, node, stmts);
+  pipeline &passes = g->get_passes ();
+  execute_ipa_stmt_fixups (passes.all_regular_ipa_passes, node, stmts);
 }
 
 
diff --git a/gcc/pipeline.h b/gcc/pipeline.h
new file mode 100644
index 0000000..37c90d7
--- /dev/null
+++ b/gcc/pipeline.h
@@ -0,0 +1,89 @@
+/* pipeline.h - The pipeline of optimization passes
+   Copyright (C) 2013 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#ifndef GCC_PIPELINE_H
+#define GCC_PIPELINE_H
+
+class opt_pass;
+struct register_pass_info;
+
+/* Define a list of pass lists so that both passes.c and plugins can easily
+   find all the pass lists.  */
+#define GCC_PASS_LISTS \
+  DEF_PASS_LIST (all_lowering_passes) \
+  DEF_PASS_LIST (all_small_ipa_passes) \
+  DEF_PASS_LIST (all_regular_ipa_passes) \
+  DEF_PASS_LIST (all_lto_gen_passes) \
+  DEF_PASS_LIST (all_passes)
+
+#define DEF_PASS_LIST(LIST) PASS_LIST_NO_##LIST,
+enum pass_list
+{
+  GCC_PASS_LISTS
+  PASS_LIST_NUM
+};
+#undef DEF_PASS_LIST
+
+namespace gcc {
+
+class context;
+
+class pipeline
+{
+public:
+  pipeline(context *ctxt);
+
+  void register_pass (struct register_pass_info *pass_info);
+  void register_one_dump_file (struct opt_pass *pass);
+
+  opt_pass *get_pass_for_id (int id) const;
+
+  void dump_passes () const;
+
+  void dump_profile_report () const;
+
+public:
+  /* The root of the compilation pass tree, once constructed.  */
+  opt_pass *all_passes;
+  opt_pass *all_small_ipa_passes;
+  opt_pass *all_lowering_passes;
+  opt_pass *all_regular_ipa_passes;
+  opt_pass *all_lto_gen_passes;
+  opt_pass *all_late_ipa_passes;
+
+  /* A map from static pass id to optimization pass.  */
+  opt_pass **passes_by_id;
+  int passes_by_id_size;
+
+  opt_pass **pass_lists[PASS_LIST_NUM];
+
+private:
+  void set_pass_for_id (int id, opt_pass *pass);
+  int register_dump_files_1 (struct opt_pass *pass, int properties);
+  void register_dump_files (struct opt_pass *pass, int properties);
+
+private:
+  context *ctxt_;
+
+}; // class pipeline
+
+} // namespace gcc
+
+#endif /* ! GCC_PIPELINE_H */
+
diff --git a/gcc/statistics.c b/gcc/statistics.c
index 3077cc0..c7e6abd 100644
--- a/gcc/statistics.c
+++ b/gcc/statistics.c
@@ -26,6 +26,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "statistics.h"
 #include "hash-table.h"
 #include "function.h"
+#include "context.h"
+#include "pipeline.h"
 
 static int statistics_dump_nr;
 static int statistics_dump_flags;
@@ -235,6 +237,7 @@ statistics_fini_1 (statistics_counter_t **slot, opt_pass *pass)
 void
 statistics_fini (void)
 {
+  gcc::pipeline &passes = g->get_passes ();
   if (!statistics_dump_file)
     return;
 
@@ -243,10 +246,10 @@ statistics_fini (void)
       unsigned i;
       for (i = 0; i < nr_statistics_hashes; ++i)
 	if (statistics_hashes[i].is_created ()
-	    && get_pass_for_id (i) != NULL)
+	    && passes.get_pass_for_id (i) != NULL)
 	  statistics_hashes[i]
 	    .traverse_noresize <opt_pass *, statistics_fini_1>
-	    (get_pass_for_id (i));
+	    (passes.get_pass_for_id (i));
     }
 
   dump_end (statistics_dump_nr, statistics_dump_file);
diff --git a/gcc/toplev.c b/gcc/toplev.c
index de28a2d..1cf98dc 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -1158,10 +1158,10 @@ general_init (const char *argv0)
      processing.  */
   init_ggc_heuristics();
 
-  /* Create the singleton holder for global state.  */
+  /* Create the singleton holder for global state.
+     Doing so also creates the pipeline of passes.  */
   g = new gcc::context();
 
-  init_optimization_passes ();
   statistics_early_init ();
   finish_params ();
 }
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 547f355..16442ed 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -489,35 +489,9 @@ extern struct gimple_opt_pass pass_inline_parameters;
 extern struct gimple_opt_pass pass_update_address_taken;
 extern struct gimple_opt_pass pass_convert_switch;
 
-/* The root of the compilation pass tree, once constructed.  */
-extern struct opt_pass *all_passes, *all_small_ipa_passes, *all_lowering_passes,
-                       *all_regular_ipa_passes, *all_lto_gen_passes, *all_late_ipa_passes;
-
-/* Define a list of pass lists so that both passes.c and plugins can easily
-   find all the pass lists.  */
-#define GCC_PASS_LISTS \
-  DEF_PASS_LIST (all_lowering_passes) \
-  DEF_PASS_LIST (all_small_ipa_passes) \
-  DEF_PASS_LIST (all_regular_ipa_passes) \
-  DEF_PASS_LIST (all_lto_gen_passes) \
-  DEF_PASS_LIST (all_passes)
-
-#define DEF_PASS_LIST(LIST) PASS_LIST_NO_##LIST,
-enum
-{
-  GCC_PASS_LISTS
-  PASS_LIST_NUM
-};
-#undef DEF_PASS_LIST
-
-/* This is used by plugins, and should also be used in
-   passes.c:register_pass.  */
-extern struct opt_pass **gcc_pass_lists[];
-
 /* Current optimization pass.  */
 extern struct opt_pass *current_pass;
 
-extern struct opt_pass * get_pass_for_id (int);
 extern bool execute_one_pass (struct opt_pass *);
 extern void execute_pass_list (struct opt_pass *);
 extern void execute_ipa_pass_list (struct opt_pass *);
@@ -547,9 +521,6 @@ extern void register_pass (struct register_pass_info *);
    directly in jump threading, and avoid peeling them next time.  */
 extern bool first_pass_instance;
 
-extern struct opt_pass **passes_by_id;
-extern int passes_by_id_size;
-
 /* Declare for plugins.  */
 extern void do_per_function_toporder (void (*) (void *), void *);
 
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 08/11] Example of converting global state to per-pass state
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (2 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 10/11] Make gcc::context be GC-managed David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-08-01 20:58   ` Richard Henderson
  2013-07-26 15:05 ` [PATCH 05/11] Add -fno-rtti when building plugins David Malcolm
                   ` (8 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

gcc/testsuite/

	Example of converting global state to per-pass state.

	* gcc.dg/plugin/one_time_plugin.c (one_pass::execute): Convert
	global state "static int counter" to...
	(one_pass::counter): ...this instance data.
---
 gcc/testsuite/gcc.dg/plugin/one_time_plugin.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c b/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
index 7e93e65..c4dace5 100644
--- a/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
+++ b/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
@@ -33,13 +33,16 @@ class one_pass : public gimple_opt_pass
 {
 public:
   one_pass(gcc::context *ctxt)
-    : gimple_opt_pass(pass_data_one_pass, ctxt)
+    : gimple_opt_pass(pass_data_one_pass, ctxt),
+      counter(0)
   {}
 
   /* opt_pass methods: */
   bool gate ();
   unsigned int execute ();
 
+private:
+  int counter;
 }; // class one_pass
 
 } // anon namespace
@@ -51,8 +54,6 @@ bool one_pass::gate (void)
 
 unsigned int one_pass::execute ()
 {
-  static int counter = 0;
-
   if (counter > 0) {
     printf ("Executed more than once \n");
  }
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (5 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 06/11] Rewrite how instances of passes are cloned David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-08-01 20:48   ` Richard Henderson
  2013-07-26 15:05 ` [PATCH 03/11] Handwritten part of conversion of passes to C++ classes David Malcolm
                   ` (5 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This is an example of converting the "gate" and "execute" functions of
a pass into C++ virtual functions, so that in the next patch we can move
a variable into member data of the opt_pass subclass.

gcc/testsuite/

	* gcc.dg/plugin/one_time_plugin.c: (one_pass_gate): convert
	to member function...
	(one_pass::gate): ...this
	(one_pass_exec): convert to member function...
	(one_pass::impl_execute): ...this
---
 gcc/testsuite/gcc.dg/plugin/one_time_plugin.c | 36 +++++++++++++--------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c b/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
index 2d996da..7e93e65 100644
--- a/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
+++ b/gcc/testsuite/gcc.dg/plugin/one_time_plugin.c
@@ -12,22 +12,6 @@
 
 int plugin_is_GPL_compatible;
 
-static bool one_pass_gate (void)
-{
-  return true;
-}
-
-static unsigned int one_pass_exec (void)
-{
-  static int counter = 0;
-
-  if (counter > 0) {
-    printf ("Executed more than once \n");
- }
- counter++;
- return 0;
-}
-
 namespace {
 
 const pass_data pass_data_one_pass =
@@ -53,13 +37,29 @@ public:
   {}
 
   /* opt_pass methods: */
-  bool gate () { return one_pass_gate (); }
-  unsigned int execute () { return one_pass_exec (); }
+  bool gate ();
+  unsigned int execute ();
 
 }; // class one_pass
 
 } // anon namespace
 
+bool one_pass::gate (void)
+{
+  return true;
+}
+
+unsigned int one_pass::execute ()
+{
+  static int counter = 0;
+
+  if (counter > 0) {
+    printf ("Executed more than once \n");
+ }
+ counter++;
+ return 0;
+}
+
 gimple_opt_pass *
 make_one_pass (gcc::context *ctxt)
 {
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 04/11] Automated conversion of passes to C++ classes
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
  2013-07-26 15:05 ` [PATCH 01/11] Introduce beginnings of a pipeline class David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-07-26 15:05 ` [PATCH 10/11] Make gcc::context be GC-managed David Malcolm
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This is the automated part of the conversion of passes from C structs to
C++ classes.

It is generated by the refactor_passes.py script in
https://github.com/davidmalcolm/gcc-refactoring-scripts

The script has its own test suite: test_refactor_passes.py, and you can
get an idea of the behavior of the script by reviewing the test suite.

This patch converts all existing pass structs for say RTL pass_foo into
the following:

  namespace {

  const pass_data pass_data_foo =
  {
     /* initialization data */
  };

  class pass_foo : public rtl_opt_pass
  {
  public:
    pass_web(gcc::context *ctxt)
      : rtl_opt_pass(pass_data_web, ctxt)
    {}

   /* opt_pass methods: */
   bool gate () { return gate_foo (); }
   unsigned int execute () { return execute_foo (); }

  }; // class pass_foo

  } // anon namespace

  rtl_opt_pass *
  make_pass_foo (gcc::context *ctxt)
  {
    return new pass_foo (ctxt);
  }

so that the only exposed API to the pass is the factory function (i.e.
"make_pass_foo").

It also updates tree-pass.h to replace all of the struct declarations:

  extern struct rtl_opt_pass pass_foo;

with those of the new factory functions:

  extern rtl_opt_pass *make_pass_foo (gcc::context *ctxt);

The full patch is 500k in size; I have posted it to:
  http://dmalcolm.fedorapeople.org/gcc/large-patches/15e12880a4c291deb984fb628e3142429019b263-0004-Automated-conversion-of-passes-to-C-classes.patch
to avoid exceeding mailing list size limits (that version of the patch
includes the ChangeLog entries).

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 05/11] Add -fno-rtti when building plugins.
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (3 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 08/11] Example of converting global state to per-pass state David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-07-29 20:24   ` Jeff Law
  2013-07-26 15:05 ` [PATCH 06/11] Rewrite how instances of passes are cloned David Malcolm
                   ` (7 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

With the conversion of passes to C++ classes, plugins that add custom
passes must create them by creating their own derived classes of the
relevant subclass of opt_pass.  gcc itself is built with -fno-rtti,
hence there is no RTTI available for the opt_pass class hierarchy.

Hence plugins that create passes will need to be built with RTTI
disabled in order to link against gcc, or they will fail to load, with
an error like:
      cc1: error: cannot load plugin ./selfassign.so
      ./selfassign.so: undefined symbol: _ZTI8opt_pass
(aka "typeinfo for opt_pass").

gcc/testsuite

	* lib/plugin-support.exp (plugin-test-execute): Add -fno-rtti
	to optstr when building plugins.
---
 gcc/testsuite/lib/plugin-support.exp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/testsuite/lib/plugin-support.exp b/gcc/testsuite/lib/plugin-support.exp
index 88033b3..017f3fd 100644
--- a/gcc/testsuite/lib/plugin-support.exp
+++ b/gcc/testsuite/lib/plugin-support.exp
@@ -104,7 +104,7 @@ proc plugin-test-execute { plugin_src plugin_tests } {
 	set optstr [concat $optstr "-DIN_GCC -fPIC -shared -undefined dynamic_lookup"]
     } else {
 	set plug_cflags $PLUGINCFLAGS 
-	set optstr "$includes $extra_flags -DIN_GCC -fPIC -shared"
+	set optstr "$includes $extra_flags -DIN_GCC -fPIC -shared -fno-rtti"
     }
 
     # Temporarily switch to the environment for the plugin compiler.
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 10/11] Make gcc::context be GC-managed
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
  2013-07-26 15:05 ` [PATCH 01/11] Introduce beginnings of a pipeline class David Malcolm
  2013-07-26 15:05 ` [PATCH 04/11] Automated conversion of passes to C++ classes David Malcolm
@ 2013-07-26 15:05 ` David Malcolm
  2013-08-01 21:28   ` Richard Henderson
  2013-07-26 15:05 ` [PATCH 08/11] Example of converting global state to per-pass state David Malcolm
                   ` (9 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:05 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This patch makes gcc::context instances be allocated within the GC-heap,
and adds traversal hooks for GC/PCH so that a gcc::context can own refs
to other GC-allocated objects.

gcc/
	* Makefile.in (GTFILES): Add context.h.
	* context.c (gcc::context::operator new): New.
	(gcc::context::gt_ggc_mx): New.
	(gcc::context::gt_pch_nx): New.
	(gcc::context::gt_pch_nx): New.
	(gt_ggc_mx (gcc::context *)): New.
	(gt_pch_nx (gcc::context *)): New.
	(gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op,
	void *cookie)): New.
	* context.h (gcc::context): Add GTY((user)) marking.
	(gcc::context::operator new): New.
	(gcc::context::gt_ggc_mx): New.
	(gcc::context::gt_pch_nx): New.
	(gcc::context::gt_pch_nx): New.
	(g): Add GTY marking.
	(gt_ggc_mx (gcc::context *)): New.
	(gt_pch_nx (gcc::context *)): New.
	(gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op,
	void *cookie)): New.
	* gengtype.c (open_base_files) <ifiles>: Add context.h.
---
 gcc/Makefile.in |  1 +
 gcc/context.c   | 41 +++++++++++++++++++++++++++++++++++++++++
 gcc/context.h   | 46 ++++++++++++++++++++++++++++++++++++++++++++--
 gcc/gengtype.c  |  2 +-
 4 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index c1fdb8a..e9d6247 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3819,6 +3819,7 @@ GTFILES = $(CPP_ID_DATA_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
   $(srcdir)/ipa-inline.h \
   $(srcdir)/asan.c \
   $(srcdir)/tsan.c \
+  $(srcdir)/context.h \
   @all_gtfiles@
 
 # Compute the list of GT header files from the corresponding C sources,
diff --git a/gcc/context.c b/gcc/context.c
index 8ec2e60..72135ed 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -27,7 +27,48 @@ along with GCC; see the file COPYING3.  If not see
 /* The singleton holder of global state: */
 gcc::context *g;
 
+void *
+gcc::context::operator new (std::size_t size)
+{
+  return ggc_internal_cleared_alloc_stat (size MEM_STAT_INFO);
+}
+
 gcc::context::context()
 {
   passes_ = new gcc::pipeline(this);
 }
+
+/* Functions relating to the garbage collector.  */
+void
+gcc::context::gt_ggc_mx ()
+{
+  /* Currently a no-op.  */
+}
+
+void
+gcc::context::gt_pch_nx ()
+{
+  /* Currently a no-op.  */
+}
+
+void
+gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
+			 void *cookie ATTRIBUTE_UNUSED)
+{
+  /* Currently a no-op.  */
+}
+
+void gt_ggc_mx (gcc::context *ctxt)
+{
+  ctxt->gt_ggc_mx ();
+}
+
+void gt_pch_nx (gcc::context *ctxt)
+{
+  ctxt->gt_pch_nx ();
+}
+
+void gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op, void *cookie)
+{
+  ctxt->gt_pch_nx (op, cookie);
+}
diff --git a/gcc/context.h b/gcc/context.h
index a83f7b2..a65e62b 100644
--- a/gcc/context.h
+++ b/gcc/context.h
@@ -27,11 +27,30 @@ class pipeline;
 /* GCC's internal state can be divided into zero or more
    "parallel universe" of state; an instance of this class is one such
    context of state.  */
-class context
+class GTY((user)) context
 {
 public:
+  /* Ensure that instances are allocated within the GC-heap.  */
+  void *operator new (std::size_t size);
+
   context();
 
+  /* Garbage-collector integration.
+
+     Each context assumes it has full control of the GC-heap that it
+     is associated with.  It acts as a root for that GC-heap, owning
+     references to within it.
+
+     Note that context instances are allocated within their own GC
+     heap.
+
+     The methods are called the *first time* that the context is reached
+     during a ggc/pch traversal, rather than every time.  */
+
+  void gt_ggc_mx ();
+  void gt_pch_nx ();
+  void gt_pch_nx (gt_pointer_operator op, void *cookie);
+
   /* Pass-management.  */
 
   pipeline &get_passes () { gcc_assert (passes_); return *passes_; }
@@ -46,6 +65,29 @@ private:
 
 /* The global singleton context aka "g".
    (the name is chosen to be easy to type in a debugger).  */
-extern gcc::context *g;
+extern GTY(()) gcc::context *g;
+
+/* Global hooks for ggc/pch.
+
+   The gcc::context class is marked with GTY((user)), which leads to
+   gengtype creating autogenerated functions for handling context within
+   gtype-desc.c:
+
+     void gt_ggc_mx_context (void *x_p);
+     void gt_pch_nx_context (void *x_p)
+     void gt_pch_p_7context (void *this_obj,
+			     void *x_p,
+			     gt_pointer_operator op,
+			     void *cookie);
+
+   Those functions call the following global functions the first time
+   that the context is reached during a traversal, and the following
+   global functions in turn simply call the corresponding  methods of the
+   context (so that they can access private fields of the context).  */
+
+void gt_ggc_mx (gcc::context *ctxt);
+void gt_pch_nx (gcc::context *ctxt);
+void gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op, void *cookie);
+
 
 #endif /* ! GCC_CONTEXT_H */
diff --git a/gcc/gengtype.c b/gcc/gengtype.c
index 50efa9b..941c6e1 100644
--- a/gcc/gengtype.c
+++ b/gcc/gengtype.c
@@ -1733,7 +1733,7 @@ open_base_files (void)
       "tree-flow.h", "reload.h", "cpp-id-data.h", "tree-chrec.h",
       "except.h", "output.h", "gimple.h", "cfgloop.h",
       "target.h", "ipa-prop.h", "lto-streamer.h", "target-globals.h",
-      "ipa-inline.h", "dwarf2out.h", NULL
+      "ipa-inline.h", "dwarf2out.h", "context.h", NULL
     };
     const char *const *ifp;
     outf_p gtype_desc_c;
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 09/11] Support "gcc" namespace in gengtype
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (8 preceding siblings ...)
  2013-07-26 15:05 ` [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed David Malcolm
@ 2013-07-26 15:29 ` David Malcolm
  2013-08-01 21:01   ` Richard Henderson
  2013-07-26 15:33 ` [PATCH 02/11] Generate pass-instances.def David Malcolm
                   ` (2 subsequent siblings)
  12 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:29 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

This patch adds enough special-casing to gengtype to allow it to cope
with types that are within the gcc namespace.

gcc/

	* gengtype.c (type_for_name): Add special-case support for
	locating types within the "gcc::" namespace.
	(open_base_files): Emit a "using namespace gcc" directive.
---
 gcc/gengtype.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/gcc/gengtype.c b/gcc/gengtype.c
index 2085496..50efa9b 100644
--- a/gcc/gengtype.c
+++ b/gcc/gengtype.c
@@ -604,6 +604,16 @@ static type_p
 type_for_name (const char *s)
 {
   pair_p p;
+
+  /* Special-case support for types within a "gcc::" namespace.  Rather
+     than fully-supporting namespaces, simply strip off the "gcc::" prefix
+     where present.  This allows us to have GTY roots of this form:
+         extern GTY(()) gcc::some_type *some_ptr;
+     where the autogenerated functions will refer to simply "some_type",
+     where they can be resolved into their namespace.  */
+  if (0 == strncmp(s, "gcc::", 5))
+    s += 5;
+
   for (p = typedefs; p != NULL; p = p->next)
     if (strcmp (p->name, s) == 0)
       return p->type;
@@ -1735,6 +1745,13 @@ open_base_files (void)
     /* Make sure we handle "cfun" specially.  */
     oprintf (gtype_desc_c, "\n/* See definition in function.h.  */\n");
     oprintf (gtype_desc_c, "#undef cfun\n");
+
+    oprintf (gtype_desc_c,
+	     "\n"
+	     "/* Types with a \"gcc::\" prefix have the prefix stripped\n"
+	     "   during gengtype parsing.  Provide a \"using\" directive\n"
+	     "   to ensure that the fully-qualified types are found.  */\n"
+	     "using namespace gcc;\n");
   }
 }
 
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 02/11] Generate pass-instances.def
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (9 preceding siblings ...)
  2013-07-26 15:29 ` [PATCH 09/11] Support "gcc" namespace in gengtype David Malcolm
@ 2013-07-26 15:33 ` David Malcolm
  2013-07-26 15:46 ` David Malcolm
  2013-07-29 11:10 ` [PATCH 00/11] Rewrite of pass management Richard Earnshaw
  12 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:33 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

Introduce a new gen-pass-instances.awk script, and use it at build time
to make a pass-instances.def from passes.def.

An example of the result can be seen at:

  http://dmalcolm.fedorapeople.org/gcc/2013-07-25/pass-instances.def

The generated pass-instances.def contains similar content to passes.def,
but the pass instances within it are explicitly numbered, so that e.g.
the third instance of:

  NEXT_PASS (pass_copy_prop)

becomes:

  NEXT_PASS (pass_copy_prop, 3)

This is needed by a subsequent patch so that we can create fields within
the pipeline class for each pass instance, where we need unique field
names to avoid a syntax error.  For example, all 8 instances of
pass_copy_prop will need different names. e.g.

   opt_pass *pass_copy_prop_1;
   ...
   opt_pass *pass_copy_prop_8;

I have successfully tested the script with gawk, with gawk using the
"-c" compatibility option to turn off gawk extensions, and with busybox
awk (versions tested were gawk-4.0.1 and busybox-1.19.4).

This patch replaces a previous attempt at this:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg00686.html
which converted multi-instance passes to using a new NEXT_PASS_NUM
macro, requiring the instance numbering within passes.def to be
maintained by hand.

In the new approach, the instance numbers are generated automatically,
and are visible at build time, giving the uniqueness needed by later
patches, whilst avoiding manual maintenance work, and also making it
easy to see the instance numbering (by inspecting the generated
pass-instances.def).

gcc/

	* Makefile.in (pass-instances.def): New.
	(passes.o): Replace dependency on passes.def with one on
	pass-instances.def

	* gen-pass-instances.awk: New.

	* passes.c (pipeline::pipeline): Use pass-instances.def rather
	than passes.def, updating local definition of NEXT_PASS macro
	to add an extra NUM parameter (currently unused).
---
 gcc/Makefile.in            |  6 ++++-
 gcc/gen-pass-instances.awk | 66 ++++++++++++++++++++++++++++++++++++++++++++++
 gcc/passes.c               |  4 +--
 3 files changed, 73 insertions(+), 3 deletions(-)
 create mode 100644 gcc/gen-pass-instances.awk

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 0b28c2d..6d1f32c 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -2737,6 +2737,10 @@ toplev.o : toplev.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
 
 hwint.o : hwint.c $(CONFIG_H) $(SYSTEM_H) $(DIAGNOSTIC_CORE_H)
 
+pass-instances.def: $(srcdir)/passes.def $(srcdir)/gen-pass-instances.awk
+	$(AWK) -f $(srcdir)/gen-pass-instances.awk \
+	  $(srcdir)/passes.def > pass-instances.def
+
 passes.o : passes.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    $(RTL_H) $(FUNCTION_H) $(FLAGS_H) $(INPUT_H) $(INSN_ATTR_H) output.h \
    $(DIAGNOSTIC_CORE_H) debug.h insn-config.h intl.h $(RECOG_H) toplev.h \
@@ -2747,7 +2751,7 @@ passes.o : passes.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    hosthooks.h $(CGRAPH_H) $(COVERAGE_H) $(TREE_PASS_H) $(TREE_DUMP_H) \
    $(GGC_H) $(OPTS_H) $(TREE_FLOW_H) $(TREE_INLINE_H) \
    gt-passes.h $(DF_H) $(PREDICT_H) $(LTO_STREAMER_H) \
-   $(PLUGIN_H) $(IPA_UTILS_H) passes.def \
+   $(PLUGIN_H) $(IPA_UTILS_H) pass-instances.def \
    $(CONTEXT_H) $(PIPELINE_H)
 
 plugin.o : plugin.c $(PLUGIN_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
diff --git a/gcc/gen-pass-instances.awk b/gcc/gen-pass-instances.awk
new file mode 100644
index 0000000..41b5e75
--- /dev/null
+++ b/gcc/gen-pass-instances.awk
@@ -0,0 +1,66 @@
+#  Copyright (C) 2013 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 3, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+# This Awk script takes passes.def and writes pass-instances.def,
+# counting the instances of each kind of pass, adding an instance number
+# to everywhere that NEXT_PASS is used.
+#
+# For example, the single-instanced pass:
+#     NEXT_PASS (pass_warn_unused_result);
+# becomes this in the output:
+#     NEXT_PASS (pass_warn_unused_result, 1);
+#
+# The various instances of
+#   NEXT_PASS (pass_copy_prop);
+# become:
+#   NEXT_PASS (pass_copy_prop, 1);
+# through:
+#   NEXT_PASS (pass_copy_prop, 8);
+# (currently there are 8 instances of that pass)
+
+# Usage: awk -f gen-pass-instances.awk passes.def
+
+BEGIN {
+	print "/* This file is auto-generated by gen-pass-instances.awk";
+	print "   from passes.def.  */";
+}
+
+function handle_line()
+{
+	line = $0;
+	where = match(line, /NEXT_PASS \((.+)\)/)
+	if (where != 0)
+	{
+		len_of_start = length("NEXT_PASS (")
+		len_of_end = length(")")
+		len_of_pass_name = RLENGTH - (len_of_start + len_of_end)
+		line_length = length(line)
+		pass_starts_at = where + len_of_start
+		pass_name = substr(line, pass_starts_at, len_of_pass_name)
+		if (pass_name in pass_counts)
+			pass_counts[pass_name]++;
+		else
+			pass_counts[pass_name] = 1;
+		printf "%s, %s%s\n",
+			substr(line, 0, pass_starts_at + len_of_pass_name - 1),
+			pass_counts[pass_name],
+			substr(line, pass_starts_at + len_of_pass_name);
+	} else {
+		print line;
+	}
+}
+
+{ handle_line() }
diff --git a/gcc/passes.c b/gcc/passes.c
index e625bf2..1bca68e 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1315,12 +1315,12 @@ pipeline::pipeline (context *ctxt)
 #define POP_INSERT_PASSES() \
   }
 
-#define NEXT_PASS(PASS)  (p = next_pass_1 (p, &((PASS).pass)))
+#define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
 
 #define TERMINATE_PASS_LIST() \
   *p = NULL;
 
-#include "passes.def"
+#include "pass-instances.def"
 
 #undef INSERT_PASSES_AFTER
 #undef PUSH_INSERT_PASSES_WITHIN
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 02/11] Generate pass-instances.def
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (10 preceding siblings ...)
  2013-07-26 15:33 ` [PATCH 02/11] Generate pass-instances.def David Malcolm
@ 2013-07-26 15:46 ` David Malcolm
  2013-07-29 20:03   ` Jeff Law
  2024-06-28 12:13   ` Rewrite usage comment at the top of 'gcc/passes.def' " Thomas Schwinge
  2013-07-29 11:10 ` [PATCH 00/11] Rewrite of pass management Richard Earnshaw
  12 siblings, 2 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-26 15:46 UTC (permalink / raw)
  To: gcc-patches; +Cc: David Malcolm

Introduce a new gen-pass-instances.awk script, and use it at build time
to make a pass-instances.def from passes.def.

An example of the result can be seen at:

  http://dmalcolm.fedorapeople.org/gcc/2013-07-25/pass-instances.def

The generated pass-instances.def contains similar content to passes.def,
but the pass instances within it are explicitly numbered, so that e.g.
the third instance of:

  NEXT_PASS (pass_copy_prop)

becomes:

  NEXT_PASS (pass_copy_prop, 3)

This is needed by a subsequent patch so that we can create fields within
the pipeline class for each pass instance, where we need unique field
names to avoid a syntax error.  For example, all 8 instances of
pass_copy_prop will need different names. e.g.

   opt_pass *pass_copy_prop_1;
   ...
   opt_pass *pass_copy_prop_8;

I have successfully tested the script with gawk, with gawk using the
"-c" compatibility option to turn off gawk extensions, and with busybox
awk (versions tested were gawk-4.0.1 and busybox-1.19.4).

This patch replaces a previous attempt at this:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg00686.html
which converted multi-instance passes to using a new NEXT_PASS_NUM
macro, requiring the instance numbering within passes.def to be
maintained by hand.

In the new approach, the instance numbers are generated automatically,
and are visible at build time, giving the uniqueness needed by later
patches, whilst avoiding manual maintenance work, and also making it
easy to see the instance numbering (by inspecting the generated
pass-instances.def).

gcc/

	* Makefile.in (pass-instances.def): New.
	(passes.o): Replace dependency on passes.def with one on
	pass-instances.def

	* gen-pass-instances.awk: New.

	* passes.c (pipeline::pipeline): Use pass-instances.def rather
	than passes.def, updating local definition of NEXT_PASS macro
	to add an extra NUM parameter (currently unused).
---
 gcc/Makefile.in            |  6 ++++-
 gcc/gen-pass-instances.awk | 66 ++++++++++++++++++++++++++++++++++++++++++++++
 gcc/passes.c               |  4 +--
 3 files changed, 73 insertions(+), 3 deletions(-)
 create mode 100644 gcc/gen-pass-instances.awk

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 0b28c2d..6d1f32c 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -2737,6 +2737,10 @@ toplev.o : toplev.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
 
 hwint.o : hwint.c $(CONFIG_H) $(SYSTEM_H) $(DIAGNOSTIC_CORE_H)
 
+pass-instances.def: $(srcdir)/passes.def $(srcdir)/gen-pass-instances.awk
+	$(AWK) -f $(srcdir)/gen-pass-instances.awk \
+	  $(srcdir)/passes.def > pass-instances.def
+
 passes.o : passes.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    $(RTL_H) $(FUNCTION_H) $(FLAGS_H) $(INPUT_H) $(INSN_ATTR_H) output.h \
    $(DIAGNOSTIC_CORE_H) debug.h insn-config.h intl.h $(RECOG_H) toplev.h \
@@ -2747,7 +2751,7 @@ passes.o : passes.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    hosthooks.h $(CGRAPH_H) $(COVERAGE_H) $(TREE_PASS_H) $(TREE_DUMP_H) \
    $(GGC_H) $(OPTS_H) $(TREE_FLOW_H) $(TREE_INLINE_H) \
    gt-passes.h $(DF_H) $(PREDICT_H) $(LTO_STREAMER_H) \
-   $(PLUGIN_H) $(IPA_UTILS_H) passes.def \
+   $(PLUGIN_H) $(IPA_UTILS_H) pass-instances.def \
    $(CONTEXT_H) $(PIPELINE_H)
 
 plugin.o : plugin.c $(PLUGIN_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
diff --git a/gcc/gen-pass-instances.awk b/gcc/gen-pass-instances.awk
new file mode 100644
index 0000000..41b5e75
--- /dev/null
+++ b/gcc/gen-pass-instances.awk
@@ -0,0 +1,66 @@
+#  Copyright (C) 2013 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 3, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+# This Awk script takes passes.def and writes pass-instances.def,
+# counting the instances of each kind of pass, adding an instance number
+# to everywhere that NEXT_PASS is used.
+#
+# For example, the single-instanced pass:
+#     NEXT_PASS (pass_warn_unused_result);
+# becomes this in the output:
+#     NEXT_PASS (pass_warn_unused_result, 1);
+#
+# The various instances of
+#   NEXT_PASS (pass_copy_prop);
+# become:
+#   NEXT_PASS (pass_copy_prop, 1);
+# through:
+#   NEXT_PASS (pass_copy_prop, 8);
+# (currently there are 8 instances of that pass)
+
+# Usage: awk -f gen-pass-instances.awk passes.def
+
+BEGIN {
+	print "/* This file is auto-generated by gen-pass-instances.awk";
+	print "   from passes.def.  */";
+}
+
+function handle_line()
+{
+	line = $0;
+	where = match(line, /NEXT_PASS \((.+)\)/)
+	if (where != 0)
+	{
+		len_of_start = length("NEXT_PASS (")
+		len_of_end = length(")")
+		len_of_pass_name = RLENGTH - (len_of_start + len_of_end)
+		line_length = length(line)
+		pass_starts_at = where + len_of_start
+		pass_name = substr(line, pass_starts_at, len_of_pass_name)
+		if (pass_name in pass_counts)
+			pass_counts[pass_name]++;
+		else
+			pass_counts[pass_name] = 1;
+		printf "%s, %s%s\n",
+			substr(line, 0, pass_starts_at + len_of_pass_name - 1),
+			pass_counts[pass_name],
+			substr(line, pass_starts_at + len_of_pass_name);
+	} else {
+		print line;
+	}
+}
+
+{ handle_line() }
diff --git a/gcc/passes.c b/gcc/passes.c
index e625bf2..1bca68e 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1315,12 +1315,12 @@ pipeline::pipeline (context *ctxt)
 #define POP_INSERT_PASSES() \
   }
 
-#define NEXT_PASS(PASS)  (p = next_pass_1 (p, &((PASS).pass)))
+#define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
 
 #define TERMINATE_PASS_LIST() \
   *p = NULL;
 
-#include "passes.def"
+#include "pass-instances.def"
 
 #undef INSERT_PASSES_AFTER
 #undef PUSH_INSERT_PASSES_WITHIN
-- 
1.7.11.7

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-07-26 15:05 ` [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed David Malcolm
@ 2013-07-27 18:45   ` Bernhard Reutner-Fischer
  2013-08-01 21:45   ` Richard Henderson
  1 sibling, 0 replies; 74+ messages in thread
From: Bernhard Reutner-Fischer @ 2013-07-27 18:45 UTC (permalink / raw)
  To: David Malcolm, gcc-patches

On 26 July 2013 17:04:41 David Malcolm <dmalcolm@redhat.com> wrote:


> diff --git a/gcc/passes.c b/gcc/passes.c
> index ce5cdeb..dd1b0ba 100644
> --- a/gcc/passes.c
> +++ b/gcc/passes.c

> +
> +void gt_pch_nx_pipeline (void *this_obj, void *p,
> +			 gt_pointer_operator op, void *cookie)
> +{
> +  pipeline *passes = (pipeline *)p;
> +  if (p == this_obj)
> +    passes->gt_pch_nx_with_op (op, cookie);
> +}
> +
>
>  /* Call from anywhere to find out what pass this is.  Useful for
>     printing out debugging information deep inside an service

s/an service/a service/;# separate patch

>
> diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
> index 41d5d92..c3e89d4 100644
> --- a/gcc/tree-pass.h
> +++ b/gcc/tree-pass.h
> @@ -76,11 +76,22 @@ namespace gcc
>
>  /* An instance of a pass.  This is also "pass_data" to minimize the
>     changes in existing code.  */
> -class opt_pass : public pass_data
> +class GTY((user)) opt_pass : public pass_data
>  {
>  public:
> +  /* Ensure that instances are allocated in the GC-managed heap.  */
> +  void *operator new (size_t sz);
> +
>    virtual ~opt_pass () { }
>
> +  /* GTY((user)) methods, to be called once per traversal.
> +     opt_pass subclasses with additional GC-managed data should overide

s/ overide/ override/

Not a review but sounds goodish.


Sent with AquaMail for Android
http://www.aqua-mail.com


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes
  2013-07-26 15:05 ` [PATCH 03/11] Handwritten part of conversion of passes to C++ classes David Malcolm
@ 2013-07-28  9:12   ` Basile Starynkevitch
  2013-07-29 15:43     ` David Malcolm
  2013-07-29 22:39   ` Jeff Law
  2024-06-28 13:06   ` Handle 'NUM' in 'PUSH_INSERT_PASSES_WITHIN' (was: " Thomas Schwinge
  2 siblings, 1 reply; 74+ messages in thread
From: Basile Starynkevitch @ 2013-07-28  9:12 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On Fri, 2013-07-26 at 11:04 -0400, David Malcolm wrote:
> This patch is the hand-written part of the conversion of passes from
> C structs to C++ classes.  It does not work without the subsequent
> autogenerated part, which is huge.
> 
> Given that the autogenerated part of the conversion is very large
> (500k), for the sake of human comprehension I have kept the change as
> two separate patches to keep the hand-written changes separate from the
> automatically-generated ones.  I would commit these as two separate
> changes to SVN in order to keep this readability for posterity in the
> logs as well as at review-time.
> 
> This pair of patches eliminates the mutable global variables
> representing the passes, allowing for multiple compilation contexts in
> one process, potentially with different combinations of passes, and with
> pass instance owning additional data.
> 
> It converts the hierarchy of opt_pass types into an actual C++ class
> hierarchy, where each of:
> 
>   * gimple_opt_pass
>   * rtl_opt_pass
>   * ipa_opt_pass_d
>   * simple_ipa_opt_pass
> 
> all become subclasses of opt_pass.
[...]

This looks good to me. I suggest adding into the `opt_pass` class two
constant fields for the approximate source location of the pass, e.g. a
field const char* _file; and another const unsigned _lineno; initialized
with __FILE__ and __LINE__ respectively.

This won't cost much (we don't have zillions of instances of
opt_pass....) and would help a lot finding where (in which source file)
an actual pass is.

This is particularly useful for newbies writing plugins (which are
trying to add new passes). It takes a lot of time to them to find which
actual source file inside the compiler is implementing a given
(existing) pass.

Cheers

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 00/11] Rewrite of pass management
  2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
                   ` (11 preceding siblings ...)
  2013-07-26 15:46 ` David Malcolm
@ 2013-07-29 11:10 ` Richard Earnshaw
  2013-07-29 15:32   ` David Malcolm
  12 siblings, 1 reply; 74+ messages in thread
From: Richard Earnshaw @ 2013-07-29 11:10 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 26/07/13 16:04, David Malcolm wrote:
> The following patch series eliminates the mutable global variables
> representing GCC's passes, allowing for multiple compilation contexts in
> one process, potentially with different combinations of passes
> (e.g. JIT-compilation of JavaScript in one thread, JIT-compilation
> of OpenGL shader programs in another) and with pass instances owning
> additional data, including GC references.
>
> The opt_pass hierarchy becomes a true C++ class hierarchy.
>
> Patch 1 introduces a gcc::pipeline class and moves various non-GTY
> globals relating to pass management into it.  The gcc::context gains its
> first field: a pointer to the gcc::pipeline instance.
>

Why 'pipeline'?  Given that we already use the term for hardware 
scheduling, it seems particularly confusing to use that term here for 
something that seems to be completely unrelated.

R.


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 00/11] Rewrite of pass management
  2013-07-29 11:10 ` [PATCH 00/11] Rewrite of pass management Richard Earnshaw
@ 2013-07-29 15:32   ` David Malcolm
  2013-07-29 16:33     ` Mike Stump
  2013-07-29 16:37     ` Diego Novillo
  0 siblings, 2 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-29 15:32 UTC (permalink / raw)
  To: Richard Earnshaw; +Cc: gcc-patches

On Mon, 2013-07-29 at 11:19 +0100, Richard Earnshaw wrote:
> On 26/07/13 16:04, David Malcolm wrote:
> > The following patch series eliminates the mutable global variables
> > representing GCC's passes, allowing for multiple compilation contexts in
> > one process, potentially with different combinations of passes
> > (e.g. JIT-compilation of JavaScript in one thread, JIT-compilation
> > of OpenGL shader programs in another) and with pass instances owning
> > additional data, including GC references.
> >
> > The opt_pass hierarchy becomes a true C++ class hierarchy.
> >
> > Patch 1 introduces a gcc::pipeline class and moves various non-GTY
> > globals relating to pass management into it.  The gcc::context gains its
> > first field: a pointer to the gcc::pipeline instance.
> >
> 
> Why 'pipeline'?  Given that we already use the term for hardware 
> scheduling, it seems particularly confusing to use that term here for 
> something that seems to be completely unrelated.

How about "gcc::pass_manager"?  (retaining "passes" as the typical name
for instances, for brevity), so we'd have this method within
gcc::context:

  pass_manager& get_passes () { gcc_assert (passes_); return *passes_; }

and pipeline.h would become pass_manager.h

(and probably with a "using gcc::pass_manager;" to avoid spelling out
the namespace everywhere).

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes
  2013-07-28  9:12   ` Basile Starynkevitch
@ 2013-07-29 15:43     ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-29 15:43 UTC (permalink / raw)
  To: Basile Starynkevitch; +Cc: gcc-patches

On Sun, 2013-07-28 at 10:37 +0200, Basile Starynkevitch wrote:
> On Fri, 2013-07-26 at 11:04 -0400, David Malcolm wrote:
> > This patch is the hand-written part of the conversion of passes from
> > C structs to C++ classes.  It does not work without the subsequent
> > autogenerated part, which is huge.
> > 
> > Given that the autogenerated part of the conversion is very large
> > (500k), for the sake of human comprehension I have kept the change as
> > two separate patches to keep the hand-written changes separate from the
> > automatically-generated ones.  I would commit these as two separate
> > changes to SVN in order to keep this readability for posterity in the
> > logs as well as at review-time.
> > 
> > This pair of patches eliminates the mutable global variables
> > representing the passes, allowing for multiple compilation contexts in
> > one process, potentially with different combinations of passes, and with
> > pass instance owning additional data.
> > 
> > It converts the hierarchy of opt_pass types into an actual C++ class
> > hierarchy, where each of:
> > 
> >   * gimple_opt_pass
> >   * rtl_opt_pass
> >   * ipa_opt_pass_d
> >   * simple_ipa_opt_pass
> > 
> > all become subclasses of opt_pass.
> [...]
> 
> This looks good to me. I suggest adding into the `opt_pass` class two
> constant fields for the approximate source location of the pass, e.g. a
> field const char* _file; and another const unsigned _lineno; initialized
> with __FILE__ and __LINE__ respectively.
> 
> This won't cost much (we don't have zillions of instances of
> opt_pass....) and would help a lot finding where (in which source file)
> an actual pass is.
> 
> This is particularly useful for newbies writing plugins (which are
> trying to add new passes). It takes a lot of time to them to find which
> actual source file inside the compiler is implementing a given
> (existing) pass.

Thanks - I like this idea.  As a relative newbie myself, I've often
found myself hunting down the implementation of a specific pass, and it
seems like your idea is something we'd want to expose in plugins such as
MELT and the Python plugin - I wrote a script to draw a "subway map" of
GCC's passes:
https://gcc-python-plugin.readthedocs.org/en/latest/tables-of-passes.html
and it would be great to be able to add hyperlinks to the relevant
passes.

So this would be an extra (const char*) and int per pass, or per pass
instance, giving about 2 kilobytes of extra memory usage, which sounds
acceptable to me.  We could either put it in the pass_data instance, so
that this could look like this (and be purely const):

  const pass_data pass_data_vrp =
  {
    GIMPLE_PASS, /* type */
    "vrp", /* name */
    OPTGROUP_NONE, /* optinfo_flags */
    true, /* has_gate */
    true, /* has_execute */
    TV_TREE_VRP, /* tv_id */
    PROP_ssa, /* properties_required */
    0, /* properties_provided */
    0, /* properties_destroyed */
    0, /* todo_flags_start */
    ( TODO_cleanup_cfg | TODO_update_ssa
      | TODO_verify_ssa
      | TODO_verify_flow ), /* todo_flags_finish */
    __FILE__, __LINE__,
*   ^^^^^^^^^^^^^^^^^^^ New stuff here
  };

or in the opt_pass itself, looking like:

  class pass_vrp : public gimple_opt_pass
  {
  public:
    pass_vrp(gcc::context *ctxt)
      : gimple_opt_pass(pass_data_vrp, ctxt, __FILE__, __LINE__)
*                                            ^^^^^^^^^^^^^^^^^^
*                                            ^ New stuff here
    {}

A related idea occurred to me: adding links to the *documentation* of
the specific passes: right now, if you want to go from a pass to the
documentation of said pass, there doesn't seem to be a consistent URL
pattern.  For example, tree-vrp.c defines "pass_vrp", with name "vrp",
but the documentation is in 
  @item Value range propagation
within passes.texi
In the HTML build of the documentation, this is documented on this page:
http://gcc.gnu.org/onlinedocs/gccint/Tree-SSA-passes.html#Tree-SSA-passes
though the generated HTML appears to have no per-pass anchors; all I see
is:
  <li>Value range propagation
followed by a <p> tag (it's also not-well-formed as XHTML, but that's a
whole other issue).
So it would be great if 
(a) the pass docs had per-pass anchors and
(b) the runtime pass metadata contained enough information so that we
could generate URLs to the docs, so that e.g. you could ask for docs at
the command-line and get sane URLs.

It may be possible to achieve (b) by making the anchors in (a) be the
pass names.


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 00/11] Rewrite of pass management
  2013-07-29 15:32   ` David Malcolm
@ 2013-07-29 16:33     ` Mike Stump
  2013-07-29 16:37     ` Diego Novillo
  1 sibling, 0 replies; 74+ messages in thread
From: Mike Stump @ 2013-07-29 16:33 UTC (permalink / raw)
  To: David Malcolm; +Cc: Richard Earnshaw, gcc-patches

On Jul 29, 2013, at 8:23 AM, David Malcolm <dmalcolm@redhat.com> wrote:
> How about "gcc::pass_manager"?

I like this name a whole lot better.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 00/11] Rewrite of pass management
  2013-07-29 15:32   ` David Malcolm
  2013-07-29 16:33     ` Mike Stump
@ 2013-07-29 16:37     ` Diego Novillo
  1 sibling, 0 replies; 74+ messages in thread
From: Diego Novillo @ 2013-07-29 16:37 UTC (permalink / raw)
  To: David Malcolm; +Cc: Richard Earnshaw, gcc-patches

On Mon, Jul 29, 2013 at 11:23 AM, David Malcolm <dmalcolm@redhat.com> wrote:

> How about "gcc::pass_manager"?

Yes, please.


Diego.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 01/11] Introduce beginnings of a pipeline class.
  2013-07-26 15:05 ` [PATCH 01/11] Introduce beginnings of a pipeline class David Malcolm
@ 2013-07-29 20:00   ` Jeff Law
  2013-07-30 19:02     ` [Committed] Introduce beginnings of gcc::pass_manager class (was Re: [PATCH 01/11] Introduce beginnings of a pipeline class.) David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Jeff Law @ 2013-07-29 20:00 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 09:04 AM, David Malcolm wrote:
> gcc/
> 	* Makefile.in (PIPELINE_H): New.
> 	(lto-cgraph.o): Depend on CONTEXT_H and PIPELINE_H.
> 	(passes.o): Likewise.
> 	(statistics.o): Likewise.
> 	(cgraphunit.o): Likewise.
> 	(context.o): Depend on PIPELINE_H.
>
> 	* pipeline.h: New.
>
> 	* cgraphunit.c (cgraph_add_new_function): Update for moves
> 	of globals to fields of pipeline.
> 	(analyze_function): Likewise.
> 	(expand_function): Likewise.
> 	(ipa_passes): Likewise.
> 	(compile): Likewise.
>
> 	* context.c (context::context): New.
> 	* context.h  (context::context): New.
> 	(context::get_passes): New.
> 	(context::passes_): New.
>
> 	* lto-cgraph.c (input_node): Update for moves of globals to
> 	fields of pipeline.
>
> 	* passes.c (all_passes): Remove, in favor of a field of the
> 	same name within the new class pipeline.
> 	(all_small_ipa_passes): Likewise.
> 	(all_lowering_passes): Likewise.
> 	(all_regular_ipa_passes): Likewise.
> 	(all_late_ipa_passes): Likewise.
> 	(all_lto_gen_passes): Likewise.
> 	(passes_by_id): Likewise.
> 	(passes_by_id_size): Likewise.
> 	(gcc_pass_lists): Remove, in favor of "pass_lists" field within
> 	the new class pipeline.
> 	(set_pass_for_id): Convert to...
> 	(pipeline::set_pass_for_id): ...method.
> 	(get_pass_for_id): Convert to...
> 	(pipeline::get_pass_for_id): ...method.
> 	(register_one_dump_file): Move body of implementation into...
> 	(pipeline::register_one_dump_file): ...here.
> 	(register_dump_files_1): Convert to...
> 	(pipeline::register_dump_files_1): ...method.
> 	(register_dump_files): Convert to...
> 	(pipeline::register_dump_files): ...method.
> 	(create_pass_tab): Update for moves of globals to fields of
> 	pipeline.
> 	(dump_passes): Move body of implementation into...
> 	(pipeline::dump_passes): ...here.
> 	(register_pass): Move body of implementation into...
> 	(pipeline::register_pass): ...here.
> 	(init_optimization_passes): Convert into...
> 	(pipeline::pipeline): ...constructor for new pipeline class, and
> 	initialize the pass_lists array.
> 	(check_profile_consistency): Update for moves of globals to
> 	fields of pipeline.
> 	(dump_profile_report): Move body of implementation into...
> 	(pipeline::dump_profile_report): ...here.
> 	(ipa_write_summaries_1): Update for moves of pass lists from
> 	being globals to fields of pipeline.
> 	(ipa_write_optimization_summaries): Likewise.
> 	(ipa_read_summaries):  Likewise.
> 	(ipa_read_optimization_summaries): Likewise.
> 	(execute_all_ipa_stmt_fixups): Likewise.
>
> 	* statistics.c (statistics_fini): Update for moves of globals to
> 	fields of pipeline.
>
> 	* toplev.c (general_init): Replace call to
> 	init_optimization_passes with construction of the pipeline
> 	instance.
>
> 	* tree-pass.h (all_passes): Remove, in favor of a field of the
> 	same name within the new class pipeline.
> 	(all_small_ipa_passes): Likewise.
> 	(all_lowering_passes): Likewise.
> 	(all_regular_ipa_passes): Likewise.
> 	(all_lto_gen_passes): Likewise.
> 	(all_late_ipa_passes): Likewise.
> 	(passes_by_id): Likewise.
> 	(passes_by_id_size): Likewise.
> 	(gcc_pass_lists): Remove, in favor of "pass_lists" field within
> 	the new class pipeline.
> 	(get_pass_for_id): Remove.
>
> gcc/lto/
>
> 	* Make-lang.in (lto/lto.o:): Depend on CONTEXT_H and
> 	PIPELINE_H.
>
> 	* lto.c (do_whole_program_analysis): Update for move of
> 	all_regular_ipa_passes from a global to a field of class
> 	pipeline.
So as has been discussed elsewhere I'd like to see pipeline changed to 
pass_manager.

WRT references.  Not being a C++ guy, I'd always mentally equated 
references with pointers.   I've done a little reading and it seems that 
references actually convey more information, which I'm generally a fan 
of -- with the caveat that they're pointers that act more like values, 
which might get confusing.

At least in the immediate term, I think we should stick with pointers 
until we have a clearer sense of whether or not we want to be using 
references in this way.  I doubt it's terribly important, but the 
non-nullness ought to be expressable via an attribute.

With the pipeline->pass_manager change and using pointers instead of 
references on the return type, this patch is fine.  Pre-approved with 
those changes.

Jeff

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 02/11] Generate pass-instances.def
  2013-07-26 15:46 ` David Malcolm
@ 2013-07-29 20:03   ` Jeff Law
  2013-07-31  3:24     ` David Malcolm
  2024-06-28 12:13   ` Rewrite usage comment at the top of 'gcc/passes.def' " Thomas Schwinge
  1 sibling, 1 reply; 74+ messages in thread
From: Jeff Law @ 2013-07-29 20:03 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 09:04 AM, David Malcolm wrote:
> Introduce a new gen-pass-instances.awk script, and use it at build time
> to make a pass-instances.def from passes.def.
>
> An example of the result can be seen at:
>
>    http://dmalcolm.fedorapeople.org/gcc/2013-07-25/pass-instances.def
>
> The generated pass-instances.def contains similar content to passes.def,
> but the pass instances within it are explicitly numbered, so that e.g.
> the third instance of:
>
>    NEXT_PASS (pass_copy_prop)
>
> becomes:
>
>    NEXT_PASS (pass_copy_prop, 3)
>
> This is needed by a subsequent patch so that we can create fields within
> the pipeline class for each pass instance, where we need unique field
> names to avoid a syntax error.  For example, all 8 instances of
> pass_copy_prop will need different names. e.g.
>
>     opt_pass *pass_copy_prop_1;
>     ...
>     opt_pass *pass_copy_prop_8;
>
> I have successfully tested the script with gawk, with gawk using the
> "-c" compatibility option to turn off gawk extensions, and with busybox
> awk (versions tested were gawk-4.0.1 and busybox-1.19.4).
>
> This patch replaces a previous attempt at this:
>    http://gcc.gnu.org/ml/gcc-patches/2013-07/msg00686.html
> which converted multi-instance passes to using a new NEXT_PASS_NUM
> macro, requiring the instance numbering within passes.def to be
> maintained by hand.
>
> In the new approach, the instance numbers are generated automatically,
> and are visible at build time, giving the uniqueness needed by later
> patches, whilst avoiding manual maintenance work, and also making it
> easy to see the instance numbering (by inspecting the generated
> pass-instances.def).
>
> gcc/
>
> 	* Makefile.in (pass-instances.def): New.
> 	(passes.o): Replace dependency on passes.def with one on
> 	pass-instances.def
>
> 	* gen-pass-instances.awk: New.
>
> 	* passes.c (pipeline::pipeline): Use pass-instances.def rather
> 	than passes.def, updating local definition of NEXT_PASS macro
> 	to add an extra NUM parameter (currently unused).
My awk-fu isn't all that great.  I'm going to assume this works and that 
if it breaks, you own it :-)

Fine for the trunk.

jeff

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 05/11] Add -fno-rtti when building plugins.
  2013-07-26 15:05 ` [PATCH 05/11] Add -fno-rtti when building plugins David Malcolm
@ 2013-07-29 20:24   ` Jeff Law
  0 siblings, 0 replies; 74+ messages in thread
From: Jeff Law @ 2013-07-29 20:24 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 09:04 AM, David Malcolm wrote:
> With the conversion of passes to C++ classes, plugins that add custom
> passes must create them by creating their own derived classes of the
> relevant subclass of opt_pass.  gcc itself is built with -fno-rtti,
> hence there is no RTTI available for the opt_pass class hierarchy.
>
> Hence plugins that create passes will need to be built with RTTI
> disabled in order to link against gcc, or they will fail to load, with
> an error like:
>        cc1: error: cannot load plugin ./selfassign.so
>        ./selfassign.so: undefined symbol: _ZTI8opt_pass
> (aka "typeinfo for opt_pass").
>
> gcc/testsuite
>
> 	* lib/plugin-support.exp (plugin-test-execute): Add -fno-rtti
> 	to optstr when building plugins.
Approved for the trunk.

jeff

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes
  2013-07-26 15:05 ` [PATCH 03/11] Handwritten part of conversion of passes to C++ classes David Malcolm
  2013-07-28  9:12   ` Basile Starynkevitch
@ 2013-07-29 22:39   ` Jeff Law
  2013-08-01 17:13     ` [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) David Malcolm
  2013-08-05 21:03     ` Passes are now C++ classes " David Malcolm
  2024-06-28 13:06   ` Handle 'NUM' in 'PUSH_INSERT_PASSES_WITHIN' (was: " Thomas Schwinge
  2 siblings, 2 replies; 74+ messages in thread
From: Jeff Law @ 2013-07-29 22:39 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 09:04 AM, David Malcolm wrote:
> This patch is the hand-written part of the conversion of passes from
> C structs to C++ classes.  It does not work without the subsequent
> autogenerated part, which is huge.
[ ... ]
With the changes from pipeline -> pass_manager this is fine.  As is the 
follow-up autogenerated patch.

Jeff

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [Committed] Introduce beginnings of gcc::pass_manager class (was Re: [PATCH 01/11] Introduce beginnings of a pipeline class.)
  2013-07-29 20:00   ` Jeff Law
@ 2013-07-30 19:02     ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-07-30 19:02 UTC (permalink / raw)
  To: Jeff Law; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 1646 bytes --]

On Mon, 2013-07-29 at 13:56 -0600, Jeff Law wrote:
> On 07/26/2013 09:04 AM, David Malcolm wrote:
[...snip quote of ChangeLog...]
> So as has been discussed elsewhere I'd like to see pipeline changed to 
> pass_manager.
> 
> WRT references.  Not being a C++ guy, I'd always mentally equated 
> references with pointers.   I've done a little reading and it seems that 
> references actually convey more information, which I'm generally a fan 
> of -- with the caveat that they're pointers that act more like values, 
> which might get confusing.
> 
> At least in the immediate term, I think we should stick with pointers 
> until we have a clearer sense of whether or not we want to be using 
> references in this way.  I doubt it's terribly important, but the 
> non-nullness ought to be expressable via an attribute.
> 
> With the pipeline->pass_manager change and using pointers instead of 
> references on the return type, this patch is fine.  Pre-approved with 
> those changes.

Thanks.  I've committed the attached to trunk as r201351, having done
the following:
* rebased against trunk
* renamed class pipeline and file pipeline.h to class pass_manager and
file pass_manager.h
* replaced uses of references with pointers
* context.c (context::context): Fixed missing space in pass_manager
constructor invocation noted by Martin Jambor in
http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01155.html
* successfully bootstrapped the new patch on x86_64-unknown-linux-gnu
and reran the testsuite: all testcases showed the same results as an
unpatched build (relative to r201317).
* verified that the patch builds on an updated svn checkout.

Dave

[-- Attachment #2: 0001-Introduce-beginnings-of-a-pass_manager-class.patch --]
[-- Type: text/x-patch, Size: 29025 bytes --]

From d396b8c8d4382fb0337ad7813fca03dc563b96d2 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Mon, 22 Jul 2013 14:06:26 -0400
Subject: [PATCH 01/14] Introduce beginnings of a pass_manager class.

This patch introduces a gcc::pass_manager class and moves various non-GTY
globals relating to pass management into it.  The gcc::context gains its
first field: a pointer to the gcc::pass_manager instance.

It was previously sent as:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01090.html
as part of:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01088.html
and Martin Jambor raised some stylistic concerns about it:
  http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01155.html
I'm reposting it here so that people can see how it fits into the later
patches.

gcc/
	* Makefile.in (PASS_MANAGER_H): New.
	(lto-cgraph.o): Depend on CONTEXT_H and PASS_MANAGER_H.
	(passes.o): Likewise.
	(statistics.o): Likewise.
	(cgraphunit.o): Likewise.
	(context.o): Depend on PASS_MANAGER_H.

	* pass_manager.h: New.

	* cgraphunit.c (cgraph_add_new_function): Update for moves
	of globals to fields of pass_manager.
	(analyze_function): Likewise.
	(expand_function): Likewise.
	(ipa_passes): Likewise.
	(compile): Likewise.

	* context.c (context::context): New.
	* context.h  (context::context): New.
	(context::get_passes): New.
	(context::passes_): New.

	* lto-cgraph.c (input_node): Update for moves of globals to
	fields of pass_manager.

	* passes.c (all_passes): Remove, in favor of a field of the
	same name within the new class pass_manager.
	(all_small_ipa_passes): Likewise.
	(all_lowering_passes): Likewise.
	(all_regular_ipa_passes): Likewise.
	(all_late_ipa_passes): Likewise.
	(all_lto_gen_passes): Likewise.
	(passes_by_id): Likewise.
	(passes_by_id_size): Likewise.
	(gcc_pass_lists): Remove, in favor of "pass_lists" field within
	the new class pass_manager.
	(set_pass_for_id): Convert to...
	(pass_manager::set_pass_for_id): ...method.
	(get_pass_for_id): Convert to...
	(pass_manager::get_pass_for_id): ...method.
	(register_one_dump_file): Move body of implementation into...
	(pass_manager::register_one_dump_file): ...here.
	(register_dump_files_1): Convert to...
	(pass_manager::register_dump_files_1): ...method.
	(register_dump_files): Convert to...
	(pass_manager::register_dump_files): ...method.
	(create_pass_tab): Update for moves of globals to fields of
	pass_manager.
	(dump_passes): Move body of implementation into...
	(pass_manager::dump_passes): ...here.
	(register_pass): Move body of implementation into...
	(pass_manager::register_pass): ...here.
	(init_optimization_passes): Convert into...
	(pass_manager::pass_manager): ...constructor for new
	pass_manager class, and initialize the pass_lists array.
	(check_profile_consistency): Update for moves of globals to
	fields of pass_manager.
	(dump_profile_report): Move body of implementation into...
	(pass_manager::dump_profile_report): ...here.
	(ipa_write_summaries_1): Update for moves of pass lists from
	being globals to fields of pass_manager.
	(ipa_write_optimization_summaries): Likewise.
	(ipa_read_summaries):  Likewise.
	(ipa_read_optimization_summaries): Likewise.
	(execute_all_ipa_stmt_fixups): Likewise.

	* statistics.c (statistics_fini): Update for moves of globals to
	fields of pass_manager.

	* toplev.c (general_init): Replace call to
	init_optimization_passes with construction of the pass_manager
	instance.

	* tree-pass.h (all_passes): Remove, in favor of a field of the
	same name within the new class pass_manager.
	(all_small_ipa_passes): Likewise.
	(all_lowering_passes): Likewise.
	(all_regular_ipa_passes): Likewise.
	(all_lto_gen_passes): Likewise.
	(all_late_ipa_passes): Likewise.
	(passes_by_id): Likewise.
	(passes_by_id_size): Likewise.
	(gcc_pass_lists): Remove, in favor of "pass_lists" field within
	the new class pass_manager.
	(get_pass_for_id): Remove.

gcc/lto/

	* Make-lang.in (lto/lto.o:): Depend on CONTEXT_H and
	PASS_MANAGER_H.

	* lto.c (do_whole_program_analysis): Update for move of
	all_regular_ipa_passes from a global to a field of class
	pass_manager.
---
 gcc/Makefile.in      | 15 +++++---
 gcc/cgraphunit.c     | 22 +++++++-----
 gcc/context.c        |  6 ++++
 gcc/context.h        | 11 +++++-
 gcc/lto-cgraph.c     |  7 ++--
 gcc/lto/Make-lang.in |  3 +-
 gcc/lto/lto.c        |  4 ++-
 gcc/pass_manager.h   | 89 ++++++++++++++++++++++++++++++++++++++++++++++
 gcc/passes.c         | 99 +++++++++++++++++++++++++++++++++++-----------------
 gcc/statistics.c     |  7 ++--
 gcc/toplev.c         |  4 +--
 gcc/tree-pass.h      | 29 ---------------
 12 files changed, 213 insertions(+), 83 deletions(-)
 create mode 100644 gcc/pass_manager.h

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index fb0cb4b..3f8bd70 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -987,6 +987,7 @@ PLUGIN_VERSION_H = plugin-version.h configargs.h
 LIBFUNCS_H = libfuncs.h $(HASHTAB_H)
 GRAPHITE_HTAB_H = graphite-htab.h graphite-clast-to-gimple.h $(HASH_TABLE_H)
 CONTEXT_H = context.h
+PASS_MANAGER_H = pass_manager.h
 
 #\f
 # Now figure out from those variables how to compile and link.
@@ -2183,7 +2184,8 @@ lto-cgraph.o: lto-cgraph.c $(CONFIG_H) $(SYSTEM_H) coretypes.h   \
    $(HASHTAB_H) langhooks.h $(BASIC_BLOCK_H) \
    $(TREE_FLOW_H) $(CGRAPH_H) $(FUNCTION_H) $(GGC_H) $(DIAGNOSTIC_CORE_H) \
    $(EXCEPT_H) $(TIMEVAR_H) pointer-set.h $(LTO_STREAMER_H) \
-   $(GCOV_IO_H) $(DATA_STREAMER_H) $(TREE_STREAMER_H) $(TREE_PASS_H) profile.h
+   $(GCOV_IO_H) $(DATA_STREAMER_H) $(TREE_STREAMER_H) $(TREE_PASS_H) \
+   profile.h $(CONTEXT_H) $(PASS_MANAGER_H)
 lto-streamer-in.o: lto-streamer-in.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
    $(TM_H) toplev.h $(DIAGNOSTIC_CORE_H) $(EXPR_H) $(FLAGS_H) $(PARAMS_H) \
    input.h $(HASHTAB_H) $(BASIC_BLOCK_H) $(TREE_FLOW_H) $(TREE_PASS_H) \
@@ -2745,7 +2747,8 @@ passes.o : passes.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
    hosthooks.h $(CGRAPH_H) $(COVERAGE_H) $(TREE_PASS_H) $(TREE_DUMP_H) \
    $(GGC_H) $(OPTS_H) $(TREE_FLOW_H) $(TREE_INLINE_H) \
    gt-passes.h $(DF_H) $(PREDICT_H) $(LTO_STREAMER_H) \
-   $(PLUGIN_H) $(IPA_UTILS_H) passes.def
+   $(PLUGIN_H) $(IPA_UTILS_H) passes.def \
+   $(CONTEXT_H) $(PASS_MANAGER_H)
 
 plugin.o : plugin.c $(PLUGIN_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
    $(HASH_TABLE_H) $(DIAGNOSTIC_CORE_H) $(TREE_H) $(TREE_PASS_H) \
@@ -2786,7 +2789,8 @@ function.o : function.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_ERROR_
    $(TREE_PASS_H) $(DF_H) $(PARAMS_H) bb-reorder.h \
    $(COMMON_TARGET_H)
 statistics.o : statistics.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
-   $(TREE_PASS_H) $(TREE_DUMP_H) $(HASH_TABLE_H) statistics.h $(FUNCTION_H)
+   $(TREE_PASS_H) $(TREE_DUMP_H) $(HASH_TABLE_H) statistics.h \
+   $(FUNCTION_H) $(CONTEXT_H) $(PASS_MANAGER_H)
 stmt.o : stmt.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(DUMPFILE_H) $(TM_H) \
    $(RTL_H) \
    $(TREE_H) $(FLAGS_H) $(FUNCTION_H) insn-config.h hard-reg-set.h $(EXPR_H) \
@@ -2908,7 +2912,8 @@ cgraphunit.o : cgraphunit.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    $(FIBHEAP_H) output.h $(PARAMS_H) $(RTL_H) $(IPA_PROP_H) \
    gt-cgraphunit.h tree-iterator.h $(COVERAGE_H) $(TREE_DUMP_H) \
    $(GIMPLE_PRETTY_PRINT_H) $(IPA_INLINE_H) $(IPA_UTILS_H) $(CFGLOOP_H) \
-   $(LTO_STREAMER_H) output.h $(REGSET_H) $(EXCEPT_H) $(GCC_PLUGIN_H) plugin.h
+   $(LTO_STREAMER_H) output.h $(REGSET_H) $(EXCEPT_H) $(GCC_PLUGIN_H) \
+   plugin.h $(CONTEXT_H) $(PASS_MANAGER_H)
 cgraphclones.o : cgraphclones.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    $(TREE_H) langhooks.h $(TREE_INLINE_H) toplev.h $(DIAGNOSTIC_CORE_H) $(FLAGS_H) $(GGC_H) \
    $(TARGET_H) $(CGRAPH_H) intl.h pointer-set.h $(FUNCTION_H) $(GIMPLE_H) \
@@ -3490,7 +3495,7 @@ $(out_object_file): $(out_file) $(CONFIG_H) coretypes.h $(TM_H) $(TREE_H) \
 	$(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) \
 		$(out_file) $(OUTPUT_OPTION)
 context.o: context.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(GGC_H) \
-   $(CONTEXT_H)
+   $(CONTEXT_H) $(PASS_MANAGER_H)
 
 $(common_out_object_file): $(common_out_file) $(CONFIG_H) $(SYSTEM_H) \
     coretypes.h $(COMMON_TARGET_H) $(COMMON_TARGET_DEF_H) $(PARAMS_H) \
diff --git a/gcc/cgraphunit.c b/gcc/cgraphunit.c
index b82c2e0..ca36937 100644
--- a/gcc/cgraphunit.c
+++ b/gcc/cgraphunit.c
@@ -194,6 +194,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "except.h"
 #include "cfgloop.h"
 #include "regset.h"     /* FIXME: For reg_obstack.  */
+#include "context.h"
+#include "pass_manager.h"
 
 /* Queue of cgraph nodes scheduled to be added into cgraph.  This is a
    secondary queue used during optimization to accommodate passes that
@@ -478,6 +480,7 @@ cgraph_finalize_function (tree decl, bool nested)
 void
 cgraph_add_new_function (tree fndecl, bool lowered)
 {
+  gcc::pass_manager *passes = g->get_passes ();
   struct cgraph_node *node;
   switch (cgraph_state)
     {
@@ -508,7 +511,7 @@ cgraph_add_new_function (tree fndecl, bool lowered)
 	    push_cfun (DECL_STRUCT_FUNCTION (fndecl));
 	    gimple_register_cfg_hooks ();
 	    bitmap_obstack_initialize (NULL);
-	    execute_pass_list (all_lowering_passes);
+	    execute_pass_list (passes->all_lowering_passes);
 	    execute_pass_list (pass_early_local_passes.pass.sub);
 	    bitmap_obstack_release (NULL);
 	    pop_cfun ();
@@ -640,7 +643,7 @@ analyze_function (struct cgraph_node *node)
 
 	  gimple_register_cfg_hooks ();
 	  bitmap_obstack_initialize (NULL);
-	  execute_pass_list (all_lowering_passes);
+	  execute_pass_list (g->get_passes ()->all_lowering_passes);
 	  free_dominance_info (CDI_POST_DOMINATORS);
 	  free_dominance_info (CDI_DOMINATORS);
 	  compact_blocks ();
@@ -1588,7 +1591,7 @@ expand_function (struct cgraph_node *node)
   /* Signal the start of passes.  */
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_START, NULL);
 
-  execute_pass_list (all_passes);
+  execute_pass_list (g->get_passes ()->all_passes);
 
   /* Signal the end of passes.  */
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_END, NULL);
@@ -1807,6 +1810,8 @@ output_in_order (void)
 static void
 ipa_passes (void)
 {
+  gcc::pass_manager *passes = g->get_passes ();
+
   set_cfun (NULL);
   current_function_decl = NULL;
   gimple_register_cfg_hooks ();
@@ -1816,7 +1821,7 @@ ipa_passes (void)
 
   if (!in_lto_p)
     {
-      execute_ipa_pass_list (all_small_ipa_passes);
+      execute_ipa_pass_list (passes->all_small_ipa_passes);
       if (seen_error ())
 	return;
     }
@@ -1843,14 +1848,15 @@ ipa_passes (void)
       cgraph_process_new_functions ();
 
       execute_ipa_summary_passes
-	((struct ipa_opt_pass_d *) all_regular_ipa_passes);
+	((struct ipa_opt_pass_d *) passes->all_regular_ipa_passes);
     }
 
   /* Some targets need to handle LTO assembler output specially.  */
   if (flag_generate_lto)
     targetm.asm_out.lto_start ();
 
-  execute_ipa_summary_passes ((struct ipa_opt_pass_d *) all_lto_gen_passes);
+  execute_ipa_summary_passes ((struct ipa_opt_pass_d *)
+			      passes->all_lto_gen_passes);
 
   if (!in_lto_p)
     ipa_write_summaries ();
@@ -1859,7 +1865,7 @@ ipa_passes (void)
     targetm.asm_out.lto_end ();
 
   if (!flag_ltrans && (in_lto_p || !flag_lto || flag_fat_lto_objects))
-    execute_ipa_pass_list (all_regular_ipa_passes);
+    execute_ipa_pass_list (passes->all_regular_ipa_passes);
   invoke_plugin_callbacks (PLUGIN_ALL_IPA_PASSES_END, NULL);
 
   bitmap_obstack_release (NULL);
@@ -1985,7 +1991,7 @@ compile (void)
 
   cgraph_materialize_all_clones ();
   bitmap_obstack_initialize (NULL);
-  execute_ipa_pass_list (all_late_ipa_passes);
+  execute_ipa_pass_list (g->get_passes ()->all_late_ipa_passes);
   symtab_remove_unreachable_nodes (true, dump_file);
 #ifdef ENABLE_CHECKING
   verify_symtab ();
diff --git a/gcc/context.c b/gcc/context.c
index 76e0dde..b515241 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -22,6 +22,12 @@ along with GCC; see the file COPYING3.  If not see
 #include "coretypes.h"
 #include "ggc.h"
 #include "context.h"
+#include "pass_manager.h"
 
 /* The singleton holder of global state: */
 gcc::context *g;
+
+gcc::context::context()
+{
+  passes_ = new gcc::pass_manager (this);
+}
diff --git a/gcc/context.h b/gcc/context.h
index 3caf02f..66260cd 100644
--- a/gcc/context.h
+++ b/gcc/context.h
@@ -22,14 +22,23 @@ along with GCC; see the file COPYING3.  If not see
 
 namespace gcc {
 
+class pass_manager;
+
 /* GCC's internal state can be divided into zero or more
    "parallel universe" of state; an instance of this class is one such
    context of state.  */
 class context
 {
 public:
+  context();
+
+  /* Pass-management.  */
+
+  pass_manager *get_passes () { gcc_assert (passes_); return passes_; }
 
-  /* Currently empty.  */
+private:
+  /* Pass-management.  */
+  pass_manager *passes_;
 
 }; // class context
 
diff --git a/gcc/lto-cgraph.c b/gcc/lto-cgraph.c
index d60213a..19a5de8 100644
--- a/gcc/lto-cgraph.c
+++ b/gcc/lto-cgraph.c
@@ -47,6 +47,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "gcov-io.h"
 #include "tree-pass.h"
 #include "profile.h"
+#include "context.h"
+#include "pass_manager.h"
 
 static void output_cgraph_opt_summary (void);
 static void input_cgraph_opt_summary (vec<symtab_node>  nodes);
@@ -936,6 +938,7 @@ input_node (struct lto_file_decl_data *file_data,
 	    enum LTO_symtab_tags tag,
 	    vec<symtab_node> nodes)
 {
+  gcc::pass_manager *passes = g->get_passes ();
   tree fn_decl;
   struct cgraph_node *node;
   struct bitpack_d bp;
@@ -981,8 +984,8 @@ input_node (struct lto_file_decl_data *file_data,
       struct opt_pass *pass;
       int pid = streamer_read_hwi (ib);
 
-      gcc_assert (pid < passes_by_id_size);
-      pass = passes_by_id[pid];
+      gcc_assert (pid < passes->passes_by_id_size);
+      pass = passes->passes_by_id[pid];
       node->ipa_transforms_to_apply.safe_push ((struct ipa_opt_pass_d *) pass);
     }
 
diff --git a/gcc/lto/Make-lang.in b/gcc/lto/Make-lang.in
index 5f2f475..1acd176 100644
--- a/gcc/lto/Make-lang.in
+++ b/gcc/lto/Make-lang.in
@@ -85,7 +85,8 @@ lto/lto.o: lto/lto.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(OPTS_H) \
 	langhooks.h $(VEC_H) $(BITMAP_H) pointer-set.h $(IPA_PROP_H) \
 	$(COMMON_H) debug.h $(GIMPLE_H) $(LTO_H) $(LTO_TREE_H) \
 	$(LTO_TAGS_H) $(LTO_STREAMER_H) $(SPLAY_TREE_H) gt-lto-lto.h \
-	$(TREE_STREAMER_H) $(DATA_STREAMER_H) lto/lto-partition.h
+	$(TREE_STREAMER_H) $(DATA_STREAMER_H) lto/lto-partition.h \
+	$(CONTEXT_H) $(PIPELINE_H)
 lto/lto-partition.o: lto/lto-partition.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
 	toplev.h $(TREE_H) $(TM_H) \
 	$(CGRAPH_H) $(TIMEVAR_H) \
diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
index c0f9328..32f8326 100644
--- a/gcc/lto/lto.c
+++ b/gcc/lto/lto.c
@@ -46,6 +46,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "splay-tree.h"
 #include "lto-partition.h"
 #include "data-streamer.h"
+#include "context.h"
+#include "pass_manager.h"
 
 static GTY(()) tree first_personality_decl;
 
@@ -3694,7 +3696,7 @@ do_whole_program_analysis (void)
   bitmap_obstack_initialize (NULL);
   cgraph_state = CGRAPH_STATE_IPA_SSA;
 
-  execute_ipa_pass_list (all_regular_ipa_passes);
+  execute_ipa_pass_list (g->get_passes ()->all_regular_ipa_passes);
   symtab_remove_unreachable_nodes (false, dump_file);
 
   if (cgraph_dump_file)
diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
new file mode 100644
index 0000000..f66cd80
--- /dev/null
+++ b/gcc/pass_manager.h
@@ -0,0 +1,89 @@
+/* pass_manager.h - The pipeline of optimization passes
+   Copyright (C) 2013 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#ifndef GCC_PASS_MANAGER_H
+#define GCC_PASS_MANAGER_H
+
+class opt_pass;
+struct register_pass_info;
+
+/* Define a list of pass lists so that both passes.c and plugins can easily
+   find all the pass lists.  */
+#define GCC_PASS_LISTS \
+  DEF_PASS_LIST (all_lowering_passes) \
+  DEF_PASS_LIST (all_small_ipa_passes) \
+  DEF_PASS_LIST (all_regular_ipa_passes) \
+  DEF_PASS_LIST (all_lto_gen_passes) \
+  DEF_PASS_LIST (all_passes)
+
+#define DEF_PASS_LIST(LIST) PASS_LIST_NO_##LIST,
+enum pass_list
+{
+  GCC_PASS_LISTS
+  PASS_LIST_NUM
+};
+#undef DEF_PASS_LIST
+
+namespace gcc {
+
+class context;
+
+class pass_manager
+{
+public:
+  pass_manager(context *ctxt);
+
+  void register_pass (struct register_pass_info *pass_info);
+  void register_one_dump_file (struct opt_pass *pass);
+
+  opt_pass *get_pass_for_id (int id) const;
+
+  void dump_passes () const;
+
+  void dump_profile_report () const;
+
+public:
+  /* The root of the compilation pass tree, once constructed.  */
+  opt_pass *all_passes;
+  opt_pass *all_small_ipa_passes;
+  opt_pass *all_lowering_passes;
+  opt_pass *all_regular_ipa_passes;
+  opt_pass *all_lto_gen_passes;
+  opt_pass *all_late_ipa_passes;
+
+  /* A map from static pass id to optimization pass.  */
+  opt_pass **passes_by_id;
+  int passes_by_id_size;
+
+  opt_pass **pass_lists[PASS_LIST_NUM];
+
+private:
+  void set_pass_for_id (int id, opt_pass *pass);
+  int register_dump_files_1 (struct opt_pass *pass, int properties);
+  void register_dump_files (struct opt_pass *pass, int properties);
+
+private:
+  context *ctxt_;
+
+}; // class pass_manager
+
+} // namespace gcc
+
+#endif /* ! GCC_PASS_MANAGER_H */
+
diff --git a/gcc/passes.c b/gcc/passes.c
index 94fb586..b8ab1e8 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -70,6 +70,10 @@ along with GCC; see the file COPYING3.  If not see
 #include "plugin.h"
 #include "ipa-utils.h"
 #include "tree-pretty-print.h" /* for dump_function_header */
+#include "context.h"
+#include "pass_manager.h"
+
+using namespace gcc;
 
 /* This is used for debugging.  It allows the current pass to printed
    from anywhere in compilation.
@@ -439,23 +443,11 @@ static struct rtl_opt_pass pass_postreload =
 
 
 
-/* The root of the compilation pass tree, once constructed.  */
-struct opt_pass *all_passes, *all_small_ipa_passes, *all_lowering_passes,
-  *all_regular_ipa_passes, *all_late_ipa_passes, *all_lto_gen_passes;
-
-/* This is used by plugins, and should also be used in register_pass.  */
-#define DEF_PASS_LIST(LIST) &LIST,
-struct opt_pass **gcc_pass_lists[] = { GCC_PASS_LISTS NULL };
-#undef DEF_PASS_LIST
-
-/* A map from static pass id to optimization pass.  */
-struct opt_pass **passes_by_id;
-int passes_by_id_size;
-
 /* Set the static pass number of pass PASS to ID and record that
    in the mapping from static pass number to pass.  */
 
-static void
+void
+pass_manager::
 set_pass_for_id (int id, struct opt_pass *pass)
 {
   pass->static_pass_number = id;
@@ -472,7 +464,7 @@ set_pass_for_id (int id, struct opt_pass *pass)
 /* Return the pass with the static pass number ID.  */
 
 struct opt_pass *
-get_pass_for_id (int id)
+pass_manager::get_pass_for_id (int id) const
 {
   if (id >= passes_by_id_size)
     return NULL;
@@ -486,6 +478,12 @@ get_pass_for_id (int id)
 void
 register_one_dump_file (struct opt_pass *pass)
 {
+  g->get_passes ()->register_one_dump_file (pass);
+}
+
+void
+pass_manager::register_one_dump_file (struct opt_pass *pass)
+{
   char *dot_name, *flag_name, *glob_name;
   const char *name, *full_name, *prefix;
   char num[10];
@@ -535,7 +533,8 @@ register_one_dump_file (struct opt_pass *pass)
 
 /* Recursive worker function for register_dump_files.  */
 
-static int
+int
+pass_manager::
 register_dump_files_1 (struct opt_pass *pass, int properties)
 {
   do
@@ -563,11 +562,12 @@ register_dump_files_1 (struct opt_pass *pass, int properties)
   return properties;
 }
 
-/* Register the dump files for the pipeline starting at PASS.
+/* Register the dump files for the pass_manager starting at PASS.
    PROPERTIES reflects the properties that are guaranteed to be available at
    the beginning of the pipeline.  */
 
-static void
+void
+pass_manager::
 register_dump_files (struct opt_pass *pass,int properties)
 {
   pass->properties_required |= properties;
@@ -663,7 +663,7 @@ create_pass_tab (void)
   if (!flag_dump_passes)
     return;
 
-  pass_tab.safe_grow_cleared (passes_by_id_size + 1);
+  pass_tab.safe_grow_cleared (g->get_passes ()->passes_by_id_size + 1);
   name_to_pass_map.traverse <void *, passes_pass_traverse> (NULL);
 }
 
@@ -714,6 +714,12 @@ dump_pass_list (struct opt_pass *pass, int indent)
 void
 dump_passes (void)
 {
+  g->get_passes ()->dump_passes ();
+}
+
+void
+pass_manager::dump_passes () const
+{
   struct cgraph_node *n, *node = NULL;
 
   create_pass_tab();
@@ -1188,6 +1194,13 @@ position_pass (struct register_pass_info *new_pass_info,
 void
 register_pass (struct register_pass_info *pass_info)
 {
+  g->get_passes ()->register_pass (pass_info);
+
+}
+
+void
+pass_manager::register_pass (struct register_pass_info *pass_info)
+{
   bool all_instances, success;
 
   /* The checks below could fail in buggy plugins.  Existing GCC
@@ -1277,11 +1290,21 @@ register_pass (struct register_pass_info *pass_info)
 				        -> all_passes
 */
 
-void
-init_optimization_passes (void)
+pass_manager::pass_manager (context *ctxt)
+: all_passes(NULL), all_small_ipa_passes(NULL), all_lowering_passes(NULL),
+  all_regular_ipa_passes(NULL), all_lto_gen_passes(NULL),
+  all_late_ipa_passes(NULL), passes_by_id(NULL), passes_by_id_size(0),
+  ctxt_(ctxt)
 {
   struct opt_pass **p;
 
+  /* Initialize the pass_lists array.  */
+#define DEF_PASS_LIST(LIST) pass_lists[PASS_LIST_NO_##LIST] = &LIST;
+  GCC_PASS_LISTS
+#undef DEF_PASS_LIST
+
+  /* Build the tree of passes.  */
+
 #define INSERT_PASSES_AFTER(PASS) \
   p = &(PASS);
 
@@ -1432,12 +1455,13 @@ static struct profile_record *profile_record;
 static void
 check_profile_consistency (int index, int subpass, bool run)
 {
+  pass_manager *passes = g->get_passes ();
   if (index == -1)
     return;
   if (!profile_record)
     profile_record = XCNEWVEC (struct profile_record,
-			       passes_by_id_size);
-  gcc_assert (index < passes_by_id_size && index >= 0);
+			       passes->passes_by_id_size);
+  gcc_assert (index < passes->passes_by_id_size && index >= 0);
   gcc_assert (subpass < 2);
   profile_record[index].run |= run;
   account_profile_record (&profile_record[index], subpass);
@@ -1448,6 +1472,12 @@ check_profile_consistency (int index, int subpass, bool run)
 void
 dump_profile_report (void)
 {
+  g->get_passes ()->dump_profile_report ();
+}
+
+void
+pass_manager::dump_profile_report () const
+{
   int i, j;
   int last_freq_in = 0, last_count_in = 0, last_freq_out = 0, last_count_out = 0;
   gcov_type last_time = 0, last_size = 0;
@@ -2067,14 +2097,15 @@ ipa_write_summaries_2 (struct opt_pass *pass, struct lto_out_decl_state *state)
 static void
 ipa_write_summaries_1 (lto_symtab_encoder_t encoder)
 {
+  pass_manager *passes = g->get_passes ();
   struct lto_out_decl_state *state = lto_new_out_decl_state ();
   state->symtab_node_encoder = encoder;
 
   lto_push_out_decl_state (state);
 
   gcc_assert (!flag_wpa);
-  ipa_write_summaries_2 (all_regular_ipa_passes, state);
-  ipa_write_summaries_2 (all_lto_gen_passes, state);
+  ipa_write_summaries_2 (passes->all_regular_ipa_passes, state);
+  ipa_write_summaries_2 (passes->all_lto_gen_passes, state);
 
   gcc_assert (lto_get_out_decl_state () == state);
   lto_pop_out_decl_state ();
@@ -2205,8 +2236,9 @@ ipa_write_optimization_summaries (lto_symtab_encoder_t encoder)
     }
 
   gcc_assert (flag_wpa);
-  ipa_write_optimization_summaries_1 (all_regular_ipa_passes, state);
-  ipa_write_optimization_summaries_1 (all_lto_gen_passes, state);
+  pass_manager *passes = g->get_passes ();
+  ipa_write_optimization_summaries_1 (passes->all_regular_ipa_passes, state);
+  ipa_write_optimization_summaries_1 (passes->all_lto_gen_passes, state);
 
   gcc_assert (lto_get_out_decl_state () == state);
   lto_pop_out_decl_state ();
@@ -2259,8 +2291,9 @@ ipa_read_summaries_1 (struct opt_pass *pass)
 void
 ipa_read_summaries (void)
 {
-  ipa_read_summaries_1 (all_regular_ipa_passes);
-  ipa_read_summaries_1 (all_lto_gen_passes);
+  pass_manager *passes = g->get_passes ();
+  ipa_read_summaries_1 (passes->all_regular_ipa_passes);
+  ipa_read_summaries_1 (passes->all_lto_gen_passes);
 }
 
 /* Same as execute_pass_list but assume that subpasses of IPA passes
@@ -2308,8 +2341,9 @@ ipa_read_optimization_summaries_1 (struct opt_pass *pass)
 void
 ipa_read_optimization_summaries (void)
 {
-  ipa_read_optimization_summaries_1 (all_regular_ipa_passes);
-  ipa_read_optimization_summaries_1 (all_lto_gen_passes);
+  pass_manager *passes = g->get_passes ();
+  ipa_read_optimization_summaries_1 (passes->all_regular_ipa_passes);
+  ipa_read_optimization_summaries_1 (passes->all_lto_gen_passes);
 }
 
 /* Same as execute_pass_list but assume that subpasses of IPA passes
@@ -2384,7 +2418,8 @@ execute_ipa_stmt_fixups (struct opt_pass *pass,
 void
 execute_all_ipa_stmt_fixups (struct cgraph_node *node, gimple *stmts)
 {
-  execute_ipa_stmt_fixups (all_regular_ipa_passes, node, stmts);
+  pass_manager *passes = g->get_passes ();
+  execute_ipa_stmt_fixups (passes->all_regular_ipa_passes, node, stmts);
 }
 
 
diff --git a/gcc/statistics.c b/gcc/statistics.c
index 3077cc0..b198b34 100644
--- a/gcc/statistics.c
+++ b/gcc/statistics.c
@@ -26,6 +26,8 @@ along with GCC; see the file COPYING3.  If not see
 #include "statistics.h"
 #include "hash-table.h"
 #include "function.h"
+#include "context.h"
+#include "pass_manager.h"
 
 static int statistics_dump_nr;
 static int statistics_dump_flags;
@@ -235,6 +237,7 @@ statistics_fini_1 (statistics_counter_t **slot, opt_pass *pass)
 void
 statistics_fini (void)
 {
+  gcc::pass_manager *passes = g->get_passes ();
   if (!statistics_dump_file)
     return;
 
@@ -243,10 +246,10 @@ statistics_fini (void)
       unsigned i;
       for (i = 0; i < nr_statistics_hashes; ++i)
 	if (statistics_hashes[i].is_created ()
-	    && get_pass_for_id (i) != NULL)
+	    && passes->get_pass_for_id (i) != NULL)
 	  statistics_hashes[i]
 	    .traverse_noresize <opt_pass *, statistics_fini_1>
-	    (get_pass_for_id (i));
+	    (passes->get_pass_for_id (i));
     }
 
   dump_end (statistics_dump_nr, statistics_dump_file);
diff --git a/gcc/toplev.c b/gcc/toplev.c
index de28a2d..9187529 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -1158,10 +1158,10 @@ general_init (const char *argv0)
      processing.  */
   init_ggc_heuristics();
 
-  /* Create the singleton holder for global state.  */
+  /* Create the singleton holder for global state.
+     Doing so also creates the pass manager and with it the passes.  */
   g = new gcc::context();
 
-  init_optimization_passes ();
   statistics_early_init ();
   finish_params ();
 }
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 547f355..16442ed 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -489,35 +489,9 @@ extern struct gimple_opt_pass pass_inline_parameters;
 extern struct gimple_opt_pass pass_update_address_taken;
 extern struct gimple_opt_pass pass_convert_switch;
 
-/* The root of the compilation pass tree, once constructed.  */
-extern struct opt_pass *all_passes, *all_small_ipa_passes, *all_lowering_passes,
-                       *all_regular_ipa_passes, *all_lto_gen_passes, *all_late_ipa_passes;
-
-/* Define a list of pass lists so that both passes.c and plugins can easily
-   find all the pass lists.  */
-#define GCC_PASS_LISTS \
-  DEF_PASS_LIST (all_lowering_passes) \
-  DEF_PASS_LIST (all_small_ipa_passes) \
-  DEF_PASS_LIST (all_regular_ipa_passes) \
-  DEF_PASS_LIST (all_lto_gen_passes) \
-  DEF_PASS_LIST (all_passes)
-
-#define DEF_PASS_LIST(LIST) PASS_LIST_NO_##LIST,
-enum
-{
-  GCC_PASS_LISTS
-  PASS_LIST_NUM
-};
-#undef DEF_PASS_LIST
-
-/* This is used by plugins, and should also be used in
-   passes.c:register_pass.  */
-extern struct opt_pass **gcc_pass_lists[];
-
 /* Current optimization pass.  */
 extern struct opt_pass *current_pass;
 
-extern struct opt_pass * get_pass_for_id (int);
 extern bool execute_one_pass (struct opt_pass *);
 extern void execute_pass_list (struct opt_pass *);
 extern void execute_ipa_pass_list (struct opt_pass *);
@@ -547,9 +521,6 @@ extern void register_pass (struct register_pass_info *);
    directly in jump threading, and avoid peeling them next time.  */
 extern bool first_pass_instance;
 
-extern struct opt_pass **passes_by_id;
-extern int passes_by_id_size;
-
 /* Declare for plugins.  */
 extern void do_per_function_toporder (void (*) (void *), void *);
 
-- 
1.7.11.7


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 02/11] Generate pass-instances.def
  2013-07-29 20:03   ` Jeff Law
@ 2013-07-31  3:24     ` David Malcolm
  2013-07-31  8:53       ` Build broken (was: [PATCH 02/11] Generate pass-instances.def) Jan-Benedict Glaw
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-31  3:24 UTC (permalink / raw)
  To: Jeff Law; +Cc: gcc-patches

On Mon, 2013-07-29 at 14:01 -0600, Jeff Law wrote:
> On 07/26/2013 09:04 AM, David Malcolm wrote:
> > Introduce a new gen-pass-instances.awk script, and use it at build time
> > to make a pass-instances.def from passes.def.
> >
> > An example of the result can be seen at:
> >
> >    http://dmalcolm.fedorapeople.org/gcc/2013-07-25/pass-instances.def
> >
> > The generated pass-instances.def contains similar content to passes.def,
> > but the pass instances within it are explicitly numbered, so that e.g.
> > the third instance of:
> >
> >    NEXT_PASS (pass_copy_prop)
> >
> > becomes:
> >
> >    NEXT_PASS (pass_copy_prop, 3)
> >
> > This is needed by a subsequent patch so that we can create fields within
> > the pipeline class for each pass instance, where we need unique field
> > names to avoid a syntax error.  For example, all 8 instances of
> > pass_copy_prop will need different names. e.g.
> >
> >     opt_pass *pass_copy_prop_1;
> >     ...
> >     opt_pass *pass_copy_prop_8;
> >
> > I have successfully tested the script with gawk, with gawk using the
> > "-c" compatibility option to turn off gawk extensions, and with busybox
> > awk (versions tested were gawk-4.0.1 and busybox-1.19.4).
> >
> > This patch replaces a previous attempt at this:
> >    http://gcc.gnu.org/ml/gcc-patches/2013-07/msg00686.html
> > which converted multi-instance passes to using a new NEXT_PASS_NUM
> > macro, requiring the instance numbering within passes.def to be
> > maintained by hand.
> >
> > In the new approach, the instance numbers are generated automatically,
> > and are visible at build time, giving the uniqueness needed by later
> > patches, whilst avoiding manual maintenance work, and also making it
> > easy to see the instance numbering (by inspecting the generated
> > pass-instances.def).
> >
> > gcc/
> >
> > 	* Makefile.in (pass-instances.def): New.
> > 	(passes.o): Replace dependency on passes.def with one on
> > 	pass-instances.def
> >
> > 	* gen-pass-instances.awk: New.
> >
> > 	* passes.c (pipeline::pipeline): Use pass-instances.def rather
> > 	than passes.def, updating local definition of NEXT_PASS macro
> > 	to add an extra NUM parameter (currently unused).
> My awk-fu isn't all that great.  I'm going to assume this works and that 
> if it breaks, you own it :-)
> 
> Fine for the trunk.

Thanks.

Committed to trunk as r201359, having double-checked that it
bootstrapped by itself on top of what had gone before, and that the
testsuite results were unaffected by it (on x86_64-unknown-linux-gnu).


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31  3:24     ` David Malcolm
@ 2013-07-31  8:53       ` Jan-Benedict Glaw
  2013-07-31  9:37         ` Jan-Benedict Glaw
  2013-07-31 10:06         ` Jan-Benedict Glaw
  0 siblings, 2 replies; 74+ messages in thread
From: Jan-Benedict Glaw @ 2013-07-31  8:53 UTC (permalink / raw)
  To: David Malcolm; +Cc: Jeff Law, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 3328 bytes --]

On Tue, 2013-07-30 20:50:27 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
[...]
> Committed to trunk as r201359, having double-checked that it
> bootstrapped by itself on top of what had gone before, and that the
> testsuite results were unaffected by it (on x86_64-unknown-linux-gnu).

My build robot chokes on it. The list of tested targets, that broke,
is:

	cris
	fr30-elf
	iq2000-elf
	m32r-elf
	microblazeel-linux
	mmix
	ppc-linux
	rl78-elf
	sh64-linux
	spu-elf
	tilegx-linux
	tilepro-linux

It's breaking with:

[...]
mawk -f ../../../../gcc/gcc/gen-pass-instances.awk \
          ../../../../gcc/gcc/passes.def > pass-instances.def
g++ -c   -g -O2 -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE  -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -I. -I. -I../../../../gcc/gcc -I../../../../gcc/gcc/. -I../../../../gcc/gcc/../include -I../../../../gcc/gcc/../libcpp/include  -I../../../../gcc/gcc/../libdecnumber -I../../../../gcc/gcc/../libdecnumber/dpd -I../libdecnumber -I../../../../gcc/gcc/../libbacktrace    ../../../../gcc/gcc/passes.c -o passes.o
./pass-instances.def: In constructor ‘gcc::pass_manager::pass_manager(gcc::context*)’:
./pass-instances.def:36:14: error: ‘pass_warn_unused_resul’ was not declared in this scope
   NEXT_PASS (pass_warn_unused_resul, 1);
              ^
../../../../gcc/gcc/passes.c:1318:55: note: in definition of macro ‘NEXT_PASS’
 #define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
                                                       ^
./pass-instances.def:37:14: error: ‘pass_diagnose_omp_block’ was not declared in this scope
   NEXT_PASS (pass_diagnose_omp_block, 1);
              ^
../../../../gcc/gcc/passes.c:1318:55: note: in definition of macro ‘NEXT_PASS’
 #define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
                                                       ^
./pass-instances.def:38:14: error: ‘pass_diagnose_tm_block’ was not declared in this scope
   NEXT_PASS (pass_diagnose_tm_block, 1);
              ^
../../../../gcc/gcc/passes.c:1318:55: note: in definition of macro ‘NEXT_PASS’
 #define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
                                                       ^
./pass-instances.def:39:14: error: ‘pass_mudflap_’ was not declared in this scope
   NEXT_PASS (pass_mudflap_, 1);
              ^
../../../../gcc/gcc/passes.c:1318:55: note: in definition of macro ‘NEXT_PASS’
 #define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
                                                       ^
./pass-instances.def:40:14: error: ‘pass_lower_om’ was not declared in this scope
   NEXT_PASS (pass_lower_om, 1);
              ^
[...]

It seems this does only happen on one of the three running build
clients. That one is using `mawk' instead of `gawk', what the two
other builders (which are not affected) use.

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
 Signature of:                    Don't believe in miracles: Rely on them!
 the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31  8:53       ` Build broken (was: [PATCH 02/11] Generate pass-instances.def) Jan-Benedict Glaw
@ 2013-07-31  9:37         ` Jan-Benedict Glaw
  2013-07-31 10:06         ` Jan-Benedict Glaw
  1 sibling, 0 replies; 74+ messages in thread
From: Jan-Benedict Glaw @ 2013-07-31  9:37 UTC (permalink / raw)
  To: David Malcolm; +Cc: Jeff Law, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> On Tue, 2013-07-30 20:50:27 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
> [...]
> > Committed to trunk as r201359, having double-checked that it
> > bootstrapped by itself on top of what had gone before, and that the
> > testsuite results were unaffected by it (on x86_64-unknown-linux-gnu).
> 
> [...]
> mawk -f ../../../../gcc/gcc/gen-pass-instances.awk \
>           ../../../../gcc/gcc/passes.def > pass-instances.def
> g++ -c   -g -O2 -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE  -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -I. -I. -I../../../../gcc/gcc -I../../../../gcc/gcc/. -I../../../../gcc/gcc/../include -I../../../../gcc/gcc/../libcpp/include  -I../../../../gcc/gcc/../libdecnumber -I../../../../gcc/gcc/../libdecnumber/dpd -I../libdecnumber -I../../../../gcc/gcc/../libbacktrace    ../../../../gcc/gcc/passes.c -o passes.o
> ./pass-instances.def: In constructor ‘gcc::pass_manager::pass_manager(gcc::context*)’:
> ./pass-instances.def:36:14: error: ‘pass_warn_unused_resul’ was not declared in this scope
>    NEXT_PASS (pass_warn_unused_resul, 1);
>               ^
[...]
> It seems this does only happen on one of the three running build
> clients. That one is using `mawk' instead of `gawk', what the two
> other builders (which are not affected) use.

Jup, that's it. With `mawk', it seems it's always missing the last
byte of the pass name. This is (one small fragment of) the diff
between gawk and mawk output:

-         NEXT_PASS (pass_forwprop, 1);
+         NEXT_PASS (pass_forwpro, 1);

All other content is wrong in the same way. Maybe they're counting the
line terminating '\n' differently?

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
  Signature of:                        Lauf nicht vor Deinem Glück davon:
  the second  :                             Es könnte hinter Dir stehen!

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31  8:53       ` Build broken (was: [PATCH 02/11] Generate pass-instances.def) Jan-Benedict Glaw
  2013-07-31  9:37         ` Jan-Benedict Glaw
@ 2013-07-31 10:06         ` Jan-Benedict Glaw
  2013-07-31 15:58           ` David Malcolm
  1 sibling, 1 reply; 74+ messages in thread
From: Jan-Benedict Glaw @ 2013-07-31 10:06 UTC (permalink / raw)
  To: David Malcolm; +Cc: Jeff Law, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 1704 bytes --]

On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> On Tue, 2013-07-30 20:50:27 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
> [...]
> > Committed to trunk as r201359, having double-checked that it
> > bootstrapped by itself on top of what had gone before, and that the
> > testsuite results were unaffected by it (on x86_64-unknown-linux-gnu).
> 
> [...]
> mawk -f ../../../../gcc/gcc/gen-pass-instances.awk \
>           ../../../../gcc/gcc/passes.def > pass-instances.def
[...]

> It seems this does only happen on one of the three running build
> clients. That one is using `mawk' instead of `gawk', what the two
> other builders (which are not affected) use.

The substr() was wrong, awk starts all its indices with 1, while the
script used 0. gawk ignores this, mawk starts the substring at 1, but
counts from 0 upwards.

Fixed as trivial:

2013-07-31  Jan-Benedict Glaw  <jbglaw@owl.de>

	* gen-pass-instances.awk: Fix offset of substr().


Index: gcc/gen-pass-instances.awk
===================================================================
--- gcc/gen-pass-instances.awk	(revision 201363)
+++ gcc/gen-pass-instances.awk	(working copy)
@@ -55,7 +55,7 @@
 		else
 			pass_counts[pass_name] = 1;
 		printf "%s, %s%s\n",
-			substr(line, 0, pass_starts_at + len_of_pass_name - 1),
+			substr(line, 1, pass_starts_at + len_of_pass_name - 1),
 			pass_counts[pass_name],
 			substr(line, pass_starts_at + len_of_pass_name);
 	} else {


MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
Signature of:                http://catb.org/~esr/faqs/smart-questions.html
the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31 10:06         ` Jan-Benedict Glaw
@ 2013-07-31 15:58           ` David Malcolm
  2013-07-31 16:18             ` Jan-Benedict Glaw
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-31 15:58 UTC (permalink / raw)
  To: Jan-Benedict Glaw; +Cc: Jeff Law, gcc-patches

On Wed, 2013-07-31 at 11:36 +0200, Jan-Benedict Glaw wrote:
> On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> > On Tue, 2013-07-30 20:50:27 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
> > [...]
> > > Committed to trunk as r201359, having double-checked that it
> > > bootstrapped by itself on top of what had gone before, and that the
> > > testsuite results were unaffected by it (on x86_64-unknown-linux-gnu).
> > 
> > [...]
> > mawk -f ../../../../gcc/gcc/gen-pass-instances.awk \
> >           ../../../../gcc/gcc/passes.def > pass-instances.def
> [...]
> 
> > It seems this does only happen on one of the three running build
> > clients. That one is using `mawk' instead of `gawk', what the two
> > other builders (which are not affected) use.
> 
> The substr() was wrong, awk starts all its indices with 1, while the
> script used 0. gawk ignores this, mawk starts the substring at 1, but
> counts from 0 upwards.
> 
> Fixed as trivial:
> 
> 2013-07-31  Jan-Benedict Glaw  <jbglaw@owl.de>
> 
> 	* gen-pass-instances.awk: Fix offset of substr().

A thousand apologies, and thanks for fixing this - I guess I owe you a
$FAVORITE_BEVERAGE.  I had only tested with gawk (with and without -c)
and with busybox awk.  I've now installed mawk and nawk on my dev box.

Are that any other awks I should be testing with - and is this
information captured somewhere for reference?

I've verified that gawk, gawk -c, mawk, nawk and busybox awk all
generate the same output on your version (r201364)  thusly in bash:

$ for AWK in gawk "gawk -c" mawk nawk "busybox awk" ; \
    do \
      $AWK -f gen-pass-instances.awk.new passes.def \
        > "pass-instances-new-$AWK.def" ; \
    done

$ md5sum pass-instances-new-*
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-new-busybox awk.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-new-gawk -c.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-new-gawk.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-new-mawk.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-new-nawk.def

However, having said that, with my original version (as of r201359), I
get the same results as with your version:

$ md5sum gen-pass-instances.awk.*
6dcc4e2de8241f1811435013da44b757  gen-pass-instances.awk.new
7e5ad85f1f919dea9862b420ddbb0fdd  gen-pass-instances.awk.old

$ diff gen-pass-instances.awk.old gen-pass-instances.awk.new
58c58
< 			substr(line, 0, pass_starts_at + len_of_pass_name - 1),
---
> 			substr(line, 1, pass_starts_at + len_of_pass_name - 1),

$ for AWK in gawk "gawk -c" mawk nawk "busybox awk" ; \
    do \
      $AWK -f gen-pass-instances.awk.old passes.def \
        > "pass-instances-old-$AWK.def" ; \
    done

$ md5sum pass-instances-old-*
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-busybox awk.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-gawk -c.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-gawk.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-mawk.def
e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-nawk.def

so it seems they're all generating the same output, with both my old
version and your new version.

Have I messed up my testing above, or is something else going on?  What
version of mawk are you using?

I'm using versions packaged on a Fedora 17 box:

$ rpm -q gawk mawk nawk busybox
gawk-4.0.1-1.fc17.x86_64
mawk-1.3.4-1.20130219.fc17.x86_64
nawk-20110810-3.fc17.x86_64
busybox-1.19.4-4.fc17.x86_64

i.e.:
* GNU Awk 4.0.1
* mawk 1.3.4
* nawk --version gives "awk version 20110810"
* BusyBox v1.19.4 (2012-04-18 15:11:20 UTC) multi-call binary.


Thanks again
Dave

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31 15:58           ` David Malcolm
@ 2013-07-31 16:18             ` Jan-Benedict Glaw
  2013-07-31 16:26               ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Jan-Benedict Glaw @ 2013-07-31 16:18 UTC (permalink / raw)
  To: David Malcolm; +Cc: Jeff Law, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 3179 bytes --]

On Wed, 2013-07-31 11:31:35 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
> On Wed, 2013-07-31 at 11:36 +0200, Jan-Benedict Glaw wrote:
> > On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
[breakage with mawk]
> > > It seems this does only happen on one of the three running build
> > > clients. That one is using `mawk' instead of `gawk', what the two
> > > other builders (which are not affected) use.
> > 
> > The substr() was wrong, awk starts all its indices with 1, while the
> > script used 0. gawk ignores this, mawk starts the substring at 1, but
> > counts from 0 upwards.
> 
> A thousand apologies, and thanks for fixing this - I guess I owe you a
> $FAVORITE_BEVERAGE.  I had only tested with gawk (with and without -c)
> and with busybox awk.  I've now installed mawk and nawk on my dev box.

That's why I'm running the build robot :)  Once I've got some
time[tm], I'll spend it a small web frontend to look into the basic
information (which builds worked/failed, show build logfile, show git
log between working/non-working version.)

> Are that any other awks I should be testing with - and is this
> information captured somewhere for reference?

I don't know of any--I'm actually not a regular awk user at all.

And actually testing specific awk features isn't codified IMHO. And
while indices start at 1 (so the code was buggy in that way), it might
also be true that the implementation-defined behavior of virtually all
awk variants is to silently s/0/1/. Then, this would be an additional
glitch (not bug) in my mawk.

> However, having said that, with my original version (as of r201359), I
> get the same results as with your version:
> 
> $ md5sum gen-pass-instances.awk.*
> 6dcc4e2de8241f1811435013da44b757  gen-pass-instances.awk.new
> 7e5ad85f1f919dea9862b420ddbb0fdd  gen-pass-instances.awk.old
> 
> $ diff gen-pass-instances.awk.old gen-pass-instances.awk.new
> 58c58
> < 			substr(line, 0, pass_starts_at + len_of_pass_name - 1),
> ---
> > 			substr(line, 1, pass_starts_at + len_of_pass_name - 1),
> 
> $ for AWK in gawk "gawk -c" mawk nawk "busybox awk" ; \
>     do \
>       $AWK -f gen-pass-instances.awk.old passes.def \
>         > "pass-instances-old-$AWK.def" ; \
>     done
> 
> $ md5sum pass-instances-old-*
> e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-busybox awk.def
> e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-gawk -c.def
> e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-gawk.def
> e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-mawk.def
> e7d0f2c2ab4f2e83fbd032f222d66530  pass-instances-old-nawk.def
> 
> so it seems they're all generating the same output, with both my old
> version and your new version.
> 
> Have I messed up my testing above, or is something else going on?  What
> version of mawk are you using?

Your testing looks fine to me. My Debian "unstable" mawk is slightly
older than yours: 1.3.3-17

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
  Signature of:                          Zensur im Internet? Nein danke!
  the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31 16:18             ` Jan-Benedict Glaw
@ 2013-07-31 16:26               ` David Malcolm
  2013-07-31 16:50                 ` Jan-Benedict Glaw
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-07-31 16:26 UTC (permalink / raw)
  To: Jan-Benedict Glaw; +Cc: Jeff Law, gcc-patches

On Wed, 2013-07-31 at 18:00 +0200, Jan-Benedict Glaw wrote:
> On Wed, 2013-07-31 11:31:35 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
> > On Wed, 2013-07-31 at 11:36 +0200, Jan-Benedict Glaw wrote:
> > > On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> [breakage with mawk]
> > > > It seems this does only happen on one of the three running build
> > > > clients. That one is using `mawk' instead of `gawk', what the two
> > > > other builders (which are not affected) use.
> > > 
> > > The substr() was wrong, awk starts all its indices with 1, while the
> > > script used 0. gawk ignores this, mawk starts the substring at 1, but
> > > counts from 0 upwards.
> > 
> > A thousand apologies, and thanks for fixing this - I guess I owe you a
> > $FAVORITE_BEVERAGE.  I had only tested with gawk (with and without -c)
> > and with busybox awk.  I've now installed mawk and nawk on my dev box.
> 
> That's why I'm running the build robot :)  Once I've got some
> time[tm], I'll spend it a small web frontend to look into the basic
> information (which builds worked/failed, show build logfile, show git
> log between working/non-working version.)

BTW, have you seen buildbot?  i.e. http://buildbot.net/

MIT-licensed and Python-based.

It's in Fedora, and I see that it's in Debian:
http://packages.debian.org/unstable/buildbot

(With my Python hat on, we use that for verifying every commit of
CPython builds and passes the testsuite, across multiple build
environments, and it's fairly easy for 3rd-parties to hook in their own
build slaves into a build farm, for the less common envs).

> > Are that any other awks I should be testing with - and is this
> > information captured somewhere for reference?
> 
> I don't know of any--I'm actually not a regular awk user at all.
> And actually testing specific awk features isn't codified IMHO. And
> while indices start at 1 (so the code was buggy in that way), it might
> also be true that the implementation-defined behavior of virtually all
> awk variants is to silently s/0/1/. Then, this would be an additional
> glitch (not bug) in my mawk.

It sounds like your awk expertise is greater than mine, though :)

[...snip my testing...]

> > Have I messed up my testing above, or is something else going on?  What
> > version of mawk are you using?
> 
> Your testing looks fine to me. My Debian "unstable" mawk is slightly
> older than yours: 1.3.3-17

Looking at mawk's CHANGES file I see this entry:

20090726
[...snip...]
	+ modify workaround for (incorrect) scripts which use a zero-parameter
	  for substr to ensure the overall length of the result stays the same.
	  For example, from makewhatis:
		filename_no_gz = substr(filename, 0, RSTART - 1);

so perhaps that's it.

Thanks again.
Dave

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Build broken (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-31 16:26               ` David Malcolm
@ 2013-07-31 16:50                 ` Jan-Benedict Glaw
  0 siblings, 0 replies; 74+ messages in thread
From: Jan-Benedict Glaw @ 2013-07-31 16:50 UTC (permalink / raw)
  To: David Malcolm; +Cc: Jeff Law, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 2741 bytes --]

On Wed, 2013-07-31 12:18:42 -0400, David Malcolm <dmalcolm@redhat.com> wrote:
> On Wed, 2013-07-31 at 18:00 +0200, Jan-Benedict Glaw wrote:
> > That's why I'm running the build robot :)  Once I've got some
> > time[tm], I'll spend it a small web frontend to look into the basic
> > information (which builds worked/failed, show build logfile, show git
> > log between working/non-working version.)
> 
> BTW, have you seen buildbot?  i.e. http://buildbot.net/
> 
> MIT-licensed and Python-based.

Yes, I have!  Though I'm not yet enough of a Python wizard to build me
a working configuration the way I wanted to have it.  It's like 400
LoC shell script right now (the build master), plus a shell script
doing the actual building on the clients (160 LoC).

It basically builds as many crosscompilers on all nodes as possible,
until a new commit shows up. And once a new commit shows up, new build
jobs are only triggered if they're expected to finish before the last
of all currently jobs is expected to be done. Then, all builders are
updated and start building again.

> It's in Fedora, and I see that it's in Debian:
> http://packages.debian.org/unstable/buildbot
> 
> (With my Python hat on, we use that for verifying every commit of
> CPython builds and passes the testsuite, across multiple build
> environments, and it's fairly easy for 3rd-parties to hook in their own
> build slaves into a build farm, for the less common envs).

If you have a few hours, you'd help substituting my scripts with
something that's possibly better cared for upstream :)

> > I don't know of any--I'm actually not a regular awk user at all.
> > And actually testing specific awk features isn't codified IMHO. And
> > while indices start at 1 (so the code was buggy in that way), it might
> > also be true that the implementation-defined behavior of virtually all
> > awk variants is to silently s/0/1/. Then, this would be an additional
> > glitch (not bug) in my mawk.
> 
> It sounds like your awk expertise is greater than mine, though :)

I just read the docs after verifying that the helper variables were
all common between gawk and mawk.

> 20090726
> [...snip...]
> 	+ modify workaround for (incorrect) scripts which use a zero-parameter
> 	  for substr to ensure the overall length of the result stays the same.
> 	  For example, from makewhatis:
> 		filename_no_gz = substr(filename, 0, RSTART - 1);
> 
> so perhaps that's it.

Probably it is.

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
  Signature of:                        Lauf nicht vor Deinem Glück davon:
  the second  :                             Es könnte hinter Dir stehen!

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-07-29 22:39   ` Jeff Law
@ 2013-08-01 17:13     ` David Malcolm
  2013-08-03  0:39       ` David Malcolm
  2013-08-05 21:03     ` Passes are now C++ classes " David Malcolm
  1 sibling, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-01 17:13 UTC (permalink / raw)
  To: Jeff Law; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 3044 bytes --]

On Mon, 2013-07-29 at 15:41 -0600, Jeff Law wrote:
> On 07/26/2013 09:04 AM, David Malcolm wrote:
> > This patch is the hand-written part of the conversion of passes from
> > C structs to C++ classes.  It does not work without the subsequent
> > autogenerated part, which is huge.
> [ ... ]
> With the changes from pipeline -> pass_manager this is fine.  As is the 
> follow-up autogenerated patch.

Thanks.

The current status is that Jeff has approved patches 1-5 for the trunk,
and patches 6-11 haven't yet been reviewed by a global reviewer.  I have
committed patches 1 and 2, having bootstrapped+tested each in turn, but
I can't commit any more of these without further review - details
follow.

I had only bootstrapped and tested the combination of all 11 patches
together, so I've been attempting to test the approved patches.  For
reference:
  * Patch 3 is the handwritten part of the conversion of passes to C++
    classes
  * Patch 4 is the autogenerated part
  * Patch 5 adds -fno-rtti to the testsuite when building plugins
  * Patch 6 is the not-yet-approved patch currently named "Rewrite how
instances of passes are cloned" (but that's not all it does, see below).

I had thought that the combination of patch 3 + 4 + 5 would work as a
unit, and hoped to commit each of these after testing the combination of
(3, 4, 5), but upon attempting a bootstrap I found two bugs:

  (a) patch 3 adds an gcc_assert to the pass_manager constructor to
ensure that pass instance fields are NULL when created, to ensure that
the macro-driven logic is correct.   However, the instance fields
haven't been explicitly initialized at that point, leading to sporadic
assertion failures.  This wasn't an issue for the full patch series
since patch 11 adds an operator new for pass_manager, allocating it from
the GC heap, using the *cleared* allocator:
   void*
   pass_manager::operator new (size_t sz)
   {
     return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
   }
hence ensuring that the fields are zeroed.  So patch 3 works with patch
11, but not without.  In the meantime, I'm attaching a new patch "3.1",
which explicitly zeroes these fields early on in the pass_manager ctor.

  (b) with just these patches, although static_pass_number for every
pass instance is correct, the dumpfile *switch* numbering is wrong,
leading to numerous testsuite failures (e.g. "unrecognized command line
option '-fdump-tree-dce2'") .  Patch 6 is needed: it does the necessary
fixups at the right time to ensure that the per-pass-instance dump files
get the correct switch names (I'll add some more notes on this to that
email).

With the attached patch, the combination of patches (03, 03.1, 04, 05
*and* 06) successfully bootstraps on x86_64-unknown-linux-gnu, and all
testcases show the same results as an unpatched build (relative to
r201397).

So I'm looking for review of the attached patch, and of at least patch 6
before I can proceed with this.  AIUI, Jeff is away at the moment, so
another global reviewer would need to do it.

Thanks
Dave

[-- Attachment #2: 0003.1-Explicitly-initialize-the-macro-generated-pass-field.patch --]
[-- Type: text/x-patch, Size: 1396 bytes --]

From 49d7df3c645459a0f90179dfb1a24cef459b186e Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Wed, 31 Jul 2013 14:20:07 -0400
Subject: [PATCH 03.1/15] Explicitly initialize the macro-generated pass fields
 to NULL

gcc/

	* passes.c (pass_manager::pass_manager): Explicitly initialize
	the macro-generated pass fields to NULL.
---
 gcc/passes.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gcc/passes.c b/gcc/passes.c
index fcbd630..c4f4f39 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1352,6 +1352,28 @@ pass_manager::pass_manager (context *ctxt)
   GCC_PASS_LISTS
 #undef DEF_PASS_LIST
 
+  /* For safety, NULL-initialize the various fields for individual pass
+     instances (mainly so that the:
+	gcc_assert (NULL == PASS ## _ ## NUM);
+     sees an initial NULL value).
+
+     We cannot use member initializers for this since pass-instances.def
+     contains semicolons.  */
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) PASS ## _ ## NUM = NULL;
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
   /* Build the tree of passes.  */
 
 #define INSERT_PASSES_AFTER(PASS) \
-- 
1.7.11.7


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 06/11] Rewrite how instances of passes are cloned
  2013-07-26 15:05 ` [PATCH 06/11] Rewrite how instances of passes are cloned David Malcolm
@ 2013-08-01 17:55   ` David Malcolm
  2013-08-01 18:11     ` David Malcolm
  2013-08-01 20:56     ` Richard Henderson
  0 siblings, 2 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-01 17:55 UTC (permalink / raw)
  To: gcc-patches

This patch does more than just remove the hardcoded assumptions about
pass sizes - as noted in
http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00041.html
it also is needed to ensure that per-pass dumpfiles get the correct
switch names.

So the short version is that patch 6 is needed for patches 3-5 to work,
ensuring that the command-line switches for the per-pass dumpfiles don't
change.

Here's the long version:

AIUI, the "static_pass_number" field is used in two different ways:
during pass creation for tracking instance counts within a pass family,
and after that, for tracking the index within the entire pipeline of
passes (the pass id).

In the current implementation, during pass creation, the first time a
pass is seen, the static_pass_number is set to -1.  On subsequent
instances of the pass, the pass is copied, and that number decremented,
so that if you have a pass with 4 instances, their static_pass_number
fields will be -4, 2, 3, 4 (note how the initial one is negative and
gives the count).

There is this logic in register_one_dump_file to determine the suffix
used for per-pass-instance dumpfiles:

  if (pass->static_pass_number != -1)
    sprintf (num, "%d", ((int) pass->static_pass_number < 0
			 ? 1 : pass->static_pass_number));

This is then used by dump_register, and affects the "swtch" field of the
relevant entry within extra_dump_files, so that unique passes have no
num in their dumpfile switch, and multiinstance passes have dumpfile
switches with numeric suffices.

This works because the code is always referring to the same pass struct,
and thus the changes to say pass_copy_prop.static_pass_number are seen
and affect later uses.

(Later on in pass initialization, the static_pass_number is overridden
and becomes an id for the pass within the entire pipeline, rather than
just within its category - see the notes in the comment in the patch
below).

In my patch series, with just patches 3 through 5, we're instead making
separate calls to e.g. make_pass_copy_prop, and each instance gets an
initial static_pass_number of 0, which is the changed to -1 by
make_pass_instance (as if it were always the first instance) and hence
the logic above in register_one_dump_file fails, and thus the dump file
of each pass erroneously has no numeric suffix in its dumpfile switch,
as if every pass were a unique instance of its kind.

With patch 6, the new function add_pass_instance sets up
static_pass_number on the new pass instances, mimicking the behavior of
the old code, and so the logic above in register_one_dump_file gets the
dump file switch names correct.

I reviewed this in gdb using this Python one-liner to see the ->swtch
field of each element of the extra_dump_files array:
(gdb) python print \
[gdb.parse_and_eval('extra_dump_files')[i]['swtch'].string() for i in
range(gdb.parse_and_eval('extra_dump_files_in_use'))]

and without patch 6, all of the extra dumpfiles for the 8 copy_prop
instances erroneously share the switch "tree-copyprop".

With patch 6, they correctly have switches "tree-copyprop1" through
"tree-copyprop8".

Much of this is covered by the comment that the patch adds, but I should
have spelled things out more in the initial posting of the patch; sorry.

OK for trunk?  (on top of the other patches, of course; see notes in 
http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00041.html in how I've
tested this).

On Fri, 2013-07-26 at 11:04 -0400, David Malcolm wrote:
> gcc/
> 
> 	Rewrite how instances of passes are cloned to remove assumptions
> 	about their sizes (thus allowing pass subclasses to have
> 	additional data fields, albeit non-GC-managed ones at this point).
> 
> 	* passes.c (make_pass_instance): Now that passes have clone
> 	methods, rewrite this function to eliminate XNEW and memcpy
> 	calls that used hardcoded sizes.  Since this function no longer
> 	creates pass instances, rename it to...
> 	(add_pass_instance): ...this.  Document the old way that passes
> 	were numbered and flagged, and rework this function to continue
> 	using it.
> 	(next_pass_1): Add an initial_pass argument for use by
> 	add_pass_instance.
> 	(position_pass): When adding multiple instances of a pass, use
> 	the pass's clone method, rather than relying on the XNEW/memcpy
> 	within the former make_pass_instance (now add_pass_instance).
> 	(pipeline::pipeline): When invoking next_pass_1, also supply the
> 	initial instance of the current pass within the pipeline.
> ---
>  gcc/passes.c | 92 ++++++++++++++++++++++++++++++++++++------------------------
>  1 file changed, 55 insertions(+), 37 deletions(-)
> 
> diff --git a/gcc/passes.c b/gcc/passes.c
> index ead41a8..ce5cdeb 100644
> --- a/gcc/passes.c
> +++ b/gcc/passes.c
> @@ -1167,68 +1167,77 @@ is_pass_explicitly_enabled_or_disabled (struct opt_pass *pass,
>    return false;
>  }
>  
> -/* Look at the static_pass_number and duplicate the pass
> -   if it is already added to a list. */
>  
> -static struct opt_pass *
> -make_pass_instance (struct opt_pass *pass, bool track_duplicates)
> -{
> -  /* A nonzero static_pass_number indicates that the
> -     pass is already in the list.  */
> -  if (pass->static_pass_number)
> -    {
> -      struct opt_pass *new_pass;
> +/* Update static_pass_number for passes (and the flag
> +   TODO_mark_first_instance).
>  
> -      if (pass->type == GIMPLE_PASS
> -          || pass->type == RTL_PASS
> -          || pass->type == SIMPLE_IPA_PASS)
> -        {
> -          new_pass = XNEW (struct opt_pass);
> -          memcpy (new_pass, pass, sizeof (struct opt_pass));
> -        }
> -      else if (pass->type == IPA_PASS)
> -        {
> -          new_pass = (struct opt_pass *)XNEW (struct ipa_opt_pass_d);
> -          memcpy (new_pass, pass, sizeof (struct ipa_opt_pass_d));
> -        }
> -      else
> -        gcc_unreachable ();
> +   Passes are constructed with static_pass_number preinitialized to 0
> +
> +   This field is used in two different ways: initially as instance numbers
> +   of their kind, and then as ids within the entire pipeline.
> +
> +   Within pipeline::pipeline:
> +
> +   * In add_pass_instance(), as called by next_pass_1 in
> +     NEXT_PASS in init_optimization_passes
>  
> -      new_pass->next = NULL;
> +   * When the initial instance of a pass within a pipeline is seen,
> +     it is flagged, and its static_pass_number is set to -1
>  
> +   * On subsequent times that it is seen, the static pass number
> +     is decremented each time, so that if there are e.g. 4 dups,
> +     they have static_pass_number -4, 2, 3, 4 respectively (note
> +     how the initial one is negative and gives the count); these
> +     can be thought of as instance numbers of the specific pass
> +
> +   * Within the register_dump_files () traversal, set_pass_for_id()
> +     is called on each pass, using these instance numbers to create
> +     dumpfile switches, and then overwriting them with a pass id,
> +     which are global to the whole pass pipeline (based on
> +     (TDI_end + current value of extra_dump_files_in_use) )  */
> +
> +static void
> +add_pass_instance (struct opt_pass *new_pass, bool track_duplicates,
> +		   opt_pass *initial_pass)
> +{
> +  /* Are we dealing with the first pass of its kind, or a clone?  */
> +  if (new_pass != initial_pass)
> +    {
> +      /* We're dealing with a clone.  */
>        new_pass->todo_flags_start &= ~TODO_mark_first_instance;
>  
>        /* Indicate to register_dump_files that this pass has duplicates,
>           and so it should rename the dump file.  The first instance will
>           be -1, and be number of duplicates = -static_pass_number - 1.
>           Subsequent instances will be > 0 and just the duplicate number.  */
> -      if ((pass->name && pass->name[0] != '*') || track_duplicates)
> +      if ((new_pass->name && new_pass->name[0] != '*') || track_duplicates)
>          {
> -          pass->static_pass_number -= 1;
> -          new_pass->static_pass_number = -pass->static_pass_number;
> +          initial_pass->static_pass_number -= 1;
> +          new_pass->static_pass_number = -initial_pass->static_pass_number;
>  	}
> -      return new_pass;
>      }
>    else
>      {
> -      pass->todo_flags_start |= TODO_mark_first_instance;
> -      pass->static_pass_number = -1;
> +      /* We're dealing with the first pass of its kind.  */
> +      new_pass->todo_flags_start |= TODO_mark_first_instance;
> +      new_pass->static_pass_number = -1;
>  
> -      invoke_plugin_callbacks (PLUGIN_NEW_PASS, pass);
> +      invoke_plugin_callbacks (PLUGIN_NEW_PASS, new_pass);
>      }
> -  return pass;
>  }
>  
>  /* Add a pass to the pass list. Duplicate the pass if it's already
>     in the list.  */
>  
>  static struct opt_pass **
> -next_pass_1 (struct opt_pass **list, struct opt_pass *pass)
> +next_pass_1 (struct opt_pass **list, struct opt_pass *pass,
> +	     struct opt_pass *initial_pass)
>  {
>    /* Every pass should have a name so that plugins can refer to them.  */
>    gcc_assert (pass->name != NULL);
>  
> -  *list = make_pass_instance (pass, false);
> +  add_pass_instance (pass, false, initial_pass);
> +  *list = pass;
>  
>    return &(*list)->next;
>  }
> @@ -1278,7 +1287,16 @@ position_pass (struct register_pass_info *new_pass_info,
>            struct opt_pass *new_pass;
>            struct pass_list_node *new_pass_node;
>  
> -	  new_pass = make_pass_instance (new_pass_info->pass, true);
> +	  if (new_pass_info->ref_pass_instance_number == 0)
> +	    {
> +	      new_pass = new_pass_info->pass->clone ();
> +	      add_pass_instance (new_pass, true, new_pass_info->pass);
> +	    }
> +	  else
> +	    {
> +	      new_pass = new_pass_info->pass;
> +	      add_pass_instance (new_pass, true, new_pass);
> +	    }
>  
>            /* Insert the new pass instance based on the positioning op.  */
>            switch (new_pass_info->pos_op)
> @@ -1477,7 +1495,7 @@ pipeline::pipeline (context *ctxt)
>          gcc_assert (PASS ## _1);                 \
>          PASS ## _ ## NUM = PASS ## _1->clone (); \
>        }                                          \
> -    p = next_pass_1 (p, PASS ## _ ## NUM);  \
> +    p = next_pass_1 (p, PASS ## _ ## NUM, PASS ## _1);  \
>    } while (0)
>  
>  #define TERMINATE_PASS_LIST() \


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 06/11] Rewrite how instances of passes are cloned
  2013-08-01 17:55   ` David Malcolm
@ 2013-08-01 18:11     ` David Malcolm
  2013-08-01 20:56     ` Richard Henderson
  1 sibling, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-01 18:11 UTC (permalink / raw)
  To: gcc-patches

On Thu, 2013-08-01 at 13:55 -0400, David Malcolm wrote:
[...snip...]
> OK for trunk?  (on top of the other patches, of course; see notes in 
> http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00041.html in how I've
> tested this).

...with "pipeline" renamed to "pass_manager", of course.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c
  2013-07-26 15:05 ` [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c David Malcolm
@ 2013-08-01 20:48   ` Richard Henderson
  2013-08-13  0:42     ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-01 20:48 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 05:04 AM, David Malcolm wrote:
> This is an example of converting the "gate" and "execute" functions of
> a pass into C++ virtual functions, so that in the next patch we can move
> a variable into member data of the opt_pass subclass.
> 
> gcc/testsuite/
> 
> 	* gcc.dg/plugin/one_time_plugin.c: (one_pass_gate): convert
> 	to member function...
> 	(one_pass::gate): ...this
> 	(one_pass_exec): convert to member function...
> 	(one_pass::impl_execute): ...this

Ok.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 06/11] Rewrite how instances of passes are cloned
  2013-08-01 17:55   ` David Malcolm
  2013-08-01 18:11     ` David Malcolm
@ 2013-08-01 20:56     ` Richard Henderson
  1 sibling, 0 replies; 74+ messages in thread
From: Richard Henderson @ 2013-08-01 20:56 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/01/2013 07:55 AM, David Malcolm wrote:
> On Fri, 2013-07-26 at 11:04 -0400, David Malcolm wrote:
>> > gcc/
>> > 
>> > 	Rewrite how instances of passes are cloned to remove assumptions
>> > 	about their sizes (thus allowing pass subclasses to have
>> > 	additional data fields, albeit non-GC-managed ones at this point).
>> > 
>> > 	* passes.c (make_pass_instance): Now that passes have clone
>> > 	methods, rewrite this function to eliminate XNEW and memcpy
>> > 	calls that used hardcoded sizes.  Since this function no longer
>> > 	creates pass instances, rename it to...
>> > 	(add_pass_instance): ...this.  Document the old way that passes
>> > 	were numbered and flagged, and rework this function to continue
>> > 	using it.
>> > 	(next_pass_1): Add an initial_pass argument for use by
>> > 	add_pass_instance.
>> > 	(position_pass): When adding multiple instances of a pass, use
>> > 	the pass's clone method, rather than relying on the XNEW/memcpy
>> > 	within the former make_pass_instance (now add_pass_instance).
>> > 	(pipeline::pipeline): When invoking next_pass_1, also supply the
>> > 	initial instance of the current pass within the pipeline.

Ok (with the pass_manager change).


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 08/11] Example of converting global state to per-pass state
  2013-07-26 15:05 ` [PATCH 08/11] Example of converting global state to per-pass state David Malcolm
@ 2013-08-01 20:58   ` Richard Henderson
  2013-08-13  0:46     ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-01 20:58 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 05:04 AM, David Malcolm wrote:
> gcc/testsuite/
> 
> 	Example of converting global state to per-pass state.
> 
> 	* gcc.dg/plugin/one_time_plugin.c (one_pass::execute): Convert
> 	global state "static int counter" to...
> 	(one_pass::counter): ...this instance data.

Ok.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 09/11] Support "gcc" namespace in gengtype
  2013-07-26 15:29 ` [PATCH 09/11] Support "gcc" namespace in gengtype David Malcolm
@ 2013-08-01 21:01   ` Richard Henderson
  2013-08-16 19:53     ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-01 21:01 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 05:04 AM, David Malcolm wrote:
> +	     "/* Types with a \"gcc::\" prefix have the prefix stripped\n"
> +	     "   during gengtype parsing.  Provide a \"using\" directive\n"
> +	     "   to ensure that the fully-qualified types are found.  */\n"

I'd rather not use the word "prefix"; the term "namespace" is both more exact
and less confusing in this context.

Otherwise ok.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 10/11] Make gcc::context be GC-managed
  2013-07-26 15:05 ` [PATCH 10/11] Make gcc::context be GC-managed David Malcolm
@ 2013-08-01 21:28   ` Richard Henderson
  2013-08-02 18:31     ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-01 21:28 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 05:04 AM, David Malcolm wrote:
> +/* Functions relating to the garbage collector.  */
> +void
> +gcc::context::gt_ggc_mx ()
> +{
> +  /* Currently a no-op.  */
> +}
> +
> +void
> +gcc::context::gt_pch_nx ()
> +{
> +  /* Currently a no-op.  */
> +}
> +
> +void
> +gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
> +			 void *cookie ATTRIBUTE_UNUSED)
> +{
> +  /* Currently a no-op.  */
> +}

I suppose these are members because you'll want to access private members
later, and that's easier than playing with "friend"?

> +void gt_ggc_mx (gcc::context *ctxt)
> +{
> +  ctxt->gt_ggc_mx ();
> +}
> +
> +void gt_pch_nx (gcc::context *ctxt)
> +{
> +  ctxt->gt_pch_nx ();
> +}
> +
> +void gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op, void *cookie)
> +{
> +  ctxt->gt_pch_nx (op, cookie);
> +}

Should these be inline functions in context.h, so that the call does direct?
Or do we take their address, making that sorta pointless?

It seems like there's one level of indirection here that is avoidable...


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-07-26 15:05 ` [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed David Malcolm
  2013-07-27 18:45   ` Bernhard Reutner-Fischer
@ 2013-08-01 21:45   ` Richard Henderson
  2013-08-02 19:08     ` David Malcolm
  1 sibling, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-01 21:45 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 07/26/2013 05:04 AM, David Malcolm wrote:
> 	(opt_pass::gt_ggc_mx): New.
> 	(opt_pass::gt_pch_nx): New.
> 	(opt_pass::gt_pch_nx_with_op): New.
> 	(gt_ggc_mx (opt_pass *)): New.
> 	(gt_pch_nx (opt_pass *)): New.
> 	(gt_pch_nx_opt_pass): New.
> 	(pipeline::operator new): New.
> 	(pipeline::gt_ggc_mx): New.
> 	(pipeline::gt_pch_nx): New.
> 	(pipeline::gt_pch_nx_with_op): New.
> 	(gt_ggc_mx (pipeline *)): New.
> 	(gt_pch_nx (pipeline *)): New.
> 	(gt_pch_nx_pipeline): New.

I guess my previous comments about ::gt_ggc_mx vs class::gt_ggc_mx wrt
the context structure apply as well to this patch.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 10/11] Make gcc::context be GC-managed
  2013-08-01 21:28   ` Richard Henderson
@ 2013-08-02 18:31     ` David Malcolm
  2013-08-03  0:43       ` Updated patch 10 (was Re: [PATCH 10/11] Make gcc::context be GC-managed) David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-02 18:31 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Thu, 2013-08-01 at 11:28 -1000, Richard Henderson wrote:
> On 07/26/2013 05:04 AM, David Malcolm wrote:
> > +/* Functions relating to the garbage collector.  */
> > +void
> > +gcc::context::gt_ggc_mx ()
> > +{
> > +  /* Currently a no-op.  */
> > +}
> > +
> > +void
> > +gcc::context::gt_pch_nx ()
> > +{
> > +  /* Currently a no-op.  */
> > +}
> > +
> > +void
> > +gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
> > +			 void *cookie ATTRIBUTE_UNUSED)
> > +{
> > +  /* Currently a no-op.  */
> > +}
> 
> I suppose these are members because you'll want to access private members
> later, and that's easier than playing with "friend"?

Exactly.  The patch adds a comment about this to gcc/context.h.

> > +void gt_ggc_mx (gcc::context *ctxt)
> > +{
> > +  ctxt->gt_ggc_mx ();
> > +}
> > +
> > +void gt_pch_nx (gcc::context *ctxt)
> > +{
> > +  ctxt->gt_pch_nx ();
> > +}
> > +
> > +void gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op, void *cookie)
> > +{
> > +  ctxt->gt_pch_nx (op, cookie);
> > +}
> 
> Should these be inline functions in context.h, so that the call does direct?
> Or do we take their address, making that sorta pointless?
> 
> It seems like there's one level of indirection here that is avoidable...

It looks like they can be inlined - their addresses are not taken, and
they're only used by gtype-desc.c, inside the autogenerated functions
gt_ggc_mx_context, gt_pch_nx_context, and gt_pch_p_7context.

I'll make them inline in context.h

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-08-01 21:45   ` Richard Henderson
@ 2013-08-02 19:08     ` David Malcolm
  2013-08-02 20:01       ` Richard Henderson
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-02 19:08 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Thu, 2013-08-01 at 11:45 -1000, Richard Henderson wrote:
> On 07/26/2013 05:04 AM, David Malcolm wrote:
> > 	(opt_pass::gt_ggc_mx): New.
> > 	(opt_pass::gt_pch_nx): New.
> > 	(opt_pass::gt_pch_nx_with_op): New.
> > 	(gt_ggc_mx (opt_pass *)): New.
> > 	(gt_pch_nx (opt_pass *)): New.
> > 	(gt_pch_nx_opt_pass): New.
> > 	(pipeline::operator new): New.
> > 	(pipeline::gt_ggc_mx): New.
> > 	(pipeline::gt_pch_nx): New.
> > 	(pipeline::gt_pch_nx_with_op): New.
> > 	(gt_ggc_mx (pipeline *)): New.
> > 	(gt_pch_nx (pipeline *)): New.
> > 	(gt_pch_nx_pipeline): New.
> 
> I guess my previous comments about ::gt_ggc_mx vs class::gt_ggc_mx wrt
> the context structure apply as well to this patch.

For context.h, the global functions are used by autogenerated code in
gtype-desc.c, which calls them the first time the context is visited in
a gc or pch traversal; they merely call into the class, to avoid needing
friend decls.

For opt_pass and pass_manager, something different is going on.

For some reason gengtype doesn't generate the triad of gt_ggc_mx_FOO,
gt_pch_nx_FOO, gt_pch_p_NFOO functions in gtype-desc.c, for types
FOO=opt_pass and pass_manager.  Presumably this is because the types are
only visited by code in context.c

So the global functions for opt_pass and pass_manager are a hand-written
implementation of what gengtype would write; they are called *each time*
the entity is reached during a traversal.  The member functions are
called only the *first time* the entity is visited.

Does this need a descriptive comment in the source code?

Thanks
Dave

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-08-02 19:08     ` David Malcolm
@ 2013-08-02 20:01       ` Richard Henderson
  2013-08-02 21:53         ` David Malcolm
  2013-08-03  0:48         ` Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed) David Malcolm
  0 siblings, 2 replies; 74+ messages in thread
From: Richard Henderson @ 2013-08-02 20:01 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/02/2013 09:08 AM, David Malcolm wrote:
> For opt_pass and pass_manager, something different is going on.

I'd wondered about that.

> For some reason gengtype doesn't generate the triad of gt_ggc_mx_FOO,
> gt_pch_nx_FOO, gt_pch_p_NFOO functions in gtype-desc.c, for types
> FOO=opt_pass and pass_manager.  Presumably this is because the types are
> only visited by code in context.c
> 
> So the global functions for opt_pass and pass_manager are a hand-written
> implementation of what gengtype would write; they are called *each time*
> the entity is reached during a traversal.  The member functions are
> called only the *first time* the entity is visited.

I wonder if we can reduce the amount of boiler-plate for this.  Perhaps,

-------------------------------------------------------------------
//
// These templates assume the presence of several member functions.
//

template<class T>
inline void gt_ggc_mx (T *p)
{
  if (ggc_test_and_set_mark (p))
    p->gt_ggc_mx ();
}

template<class T>
void gt_pch_nx_with_obj(void *this_obj, void *p,
                        gt_pointer_operator op, void *cookie)
{
  if (p == this_obj)
    {
      T *t = static_cast<T *>(p);
      t->gt_pch_nx_with_obj (op, cookie);
    }
}

template<class T>
inline void gt_pch_nx (T *p)
{
  if (gt_pch_note_object (p, p, gt_pch_nx_with_obj<T>))
    p->gt_pch_nx ();
}

---------------------------------------------------------------------

I had thought about an abstract base class instead of the templates, but
that would unnecessarily force the use of vtables in places that don't
need them.  In some cases this would be relatively harmless (e.g. the
very few pass_manager objects), but in others it might be a no-go.

The use of the template obviates that.  It ought only be instantiated when
necessary for gty user objects that don't have their own specialization.

Thoughts?


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-08-02 20:01       ` Richard Henderson
@ 2013-08-02 21:53         ` David Malcolm
  2013-08-03  0:21           ` Richard Henderson
  2013-08-03  0:48         ` Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed) David Malcolm
  1 sibling, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-02 21:53 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Fri, 2013-08-02 at 10:01 -1000, Richard Henderson wrote:
> On 08/02/2013 09:08 AM, David Malcolm wrote:
> > For opt_pass and pass_manager, something different is going on.
> 
> I'd wondered about that.
> 
> > For some reason gengtype doesn't generate the triad of gt_ggc_mx_FOO,
> > gt_pch_nx_FOO, gt_pch_p_NFOO functions in gtype-desc.c, for types
> > FOO=opt_pass and pass_manager.  Presumably this is because the types are
> > only visited by code in context.c
> > 
> > So the global functions for opt_pass and pass_manager are a hand-written
> > implementation of what gengtype would write; they are called *each time*
> > the entity is reached during a traversal.  The member functions are
> > called only the *first time* the entity is visited.
> 
> I wonder if we can reduce the amount of boiler-plate for this.  Perhaps,
> 
> -------------------------------------------------------------------
> //
> // These templates assume the presence of several member functions.
> //
> 
> template<class T>
> inline void gt_ggc_mx (T *p)
> {
>   if (ggc_test_and_set_mark (p))
>     p->gt_ggc_mx ();
> }
> 
> template<class T>
> void gt_pch_nx_with_obj(void *this_obj, void *p,
>                         gt_pointer_operator op, void *cookie)
> {
>   if (p == this_obj)
>     {
>       T *t = static_cast<T *>(p);
>       t->gt_pch_nx_with_obj (op, cookie);
>     }
> }
> 
> template<class T>
> inline void gt_pch_nx (T *p)
> {
>   if (gt_pch_note_object (p, p, gt_pch_nx_with_obj<T>))
>     p->gt_pch_nx ();
> }
> 
> ---------------------------------------------------------------------
> 
> I had thought about an abstract base class instead of the templates, but
> that would unnecessarily force the use of vtables in places that don't
> need them.  In some cases this would be relatively harmless (e.g. the
> very few pass_manager objects), but in others it might be a no-go.
> 
> The use of the template obviates that.  It ought only be instantiated when
> necessary for gty user objects that don't have their own specialization.

I like this approach, and I'm trying a bootstrap now with the templates.
I changed the "with_obj" to "with_op", on the grounds that you always
have an obj, but you don't always have an op.

FWIW I had a go at avoiding templates by attempting to tell gengtype to
write out functions for all GTY((user)) types, regardless of whether it
thinks they're referenced, with this:
@@ -3697,7 +3697,8 @@ write_types (outf_p output_header, type_p structures, type_p param_structs,
   /* At last we emit the functions code.  */
   oprintf (output_header, "\n/* functions code */\n");
   for (s = structures; s; s = s->next)
-    if (s->gc_used == GC_POINTED_TO || s->gc_used == GC_MAYBE_POINTED_TO)
+    if (s->gc_used == GC_POINTED_TO || s->gc_used == GC_MAYBE_POINTED_TO
+        || s->kind == TYPE_USER_STRUCT)
       {
        options_p opt;
 
but I ran into various "incomplete structure" errors due to gengtype's
lack of a full C/C++ parser.

Hence templates seem the sanest option.


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-08-02 21:53         ` David Malcolm
@ 2013-08-03  0:21           ` Richard Henderson
  2013-08-27 10:07             ` Richard Biener
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-03  0:21 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/02/2013 11:53 AM, David Malcolm wrote:
> FWIW I had a go at avoiding templates by attempting to tell gengtype to
> write out functions for all GTY((user)) types, regardless of whether it
> thinks they're referenced, with this:
> @@ -3697,7 +3697,8 @@ write_types (outf_p output_header, type_p structures, type_p param_structs,
>    /* At last we emit the functions code.  */
>    oprintf (output_header, "\n/* functions code */\n");
>    for (s = structures; s; s = s->next)
> -    if (s->gc_used == GC_POINTED_TO || s->gc_used == GC_MAYBE_POINTED_TO)
> +    if (s->gc_used == GC_POINTED_TO || s->gc_used == GC_MAYBE_POINTED_TO
> +        || s->kind == TYPE_USER_STRUCT)
>        {
>         options_p opt;
>  
> but I ran into various "incomplete structure" errors due to gengtype's
> lack of a full C/C++ parser.
> 
> Hence templates seem the sanest option.

I was actually contemplating going the other direction -- have gengtype
write out _less_, relying on the templates more.  I hadn't gone very far
down that road yet...


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-08-01 17:13     ` [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) David Malcolm
@ 2013-08-03  0:39       ` David Malcolm
  2013-08-03 17:59         ` Richard Henderson
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-03  0:39 UTC (permalink / raw)
  To: Jeff Law, rth; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 3847 bytes --]

On Thu, 2013-08-01 at 13:13 -0400, David Malcolm wrote:
> On Mon, 2013-07-29 at 15:41 -0600, Jeff Law wrote:
> > On 07/26/2013 09:04 AM, David Malcolm wrote:
> > > This patch is the hand-written part of the conversion of passes from
> > > C structs to C++ classes.  It does not work without the subsequent
> > > autogenerated part, which is huge.
> > [ ... ]
> > With the changes from pipeline -> pass_manager this is fine.  As is the 
> > follow-up autogenerated patch.
> 
> Thanks.
> 
> The current status is that Jeff has approved patches 1-5 for the trunk,
> and patches 6-11 haven't yet been reviewed by a global reviewer.  I have
> committed patches 1 and 2, having bootstrapped+tested each in turn, but
> I can't commit any more of these without further review - details
> follow.
> 
> I had only bootstrapped and tested the combination of all 11 patches
> together, so I've been attempting to test the approved patches.  For
> reference:
>   * Patch 3 is the handwritten part of the conversion of passes to C++
>     classes
>   * Patch 4 is the autogenerated part
>   * Patch 5 adds -fno-rtti to the testsuite when building plugins
>   * Patch 6 is the not-yet-approved patch currently named "Rewrite how
> instances of passes are cloned" (but that's not all it does, see below).
> 
> I had thought that the combination of patch 3 + 4 + 5 would work as a
> unit, and hoped to commit each of these after testing the combination of
> (3, 4, 5), but upon attempting a bootstrap I found two bugs:
> 
>   (a) patch 3 adds an gcc_assert to the pass_manager constructor to
> ensure that pass instance fields are NULL when created, to ensure that
> the macro-driven logic is correct.   However, the instance fields
> haven't been explicitly initialized at that point, leading to sporadic
> assertion failures.  This wasn't an issue for the full patch series
> since patch 11 adds an operator new for pass_manager, allocating it from
> the GC heap, using the *cleared* allocator:
>    void*
>    pass_manager::operator new (size_t sz)
>    {
>      return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
>    }
> hence ensuring that the fields are zeroed.  So patch 3 works with patch
> 11, but not without.  In the meantime, I'm attaching a new patch "3.1",
> which explicitly zeroes these fields early on in the pass_manager ctor.
> 
>   (b) with just these patches, although static_pass_number for every
> pass instance is correct, the dumpfile *switch* numbering is wrong,
> leading to numerous testsuite failures (e.g. "unrecognized command line
> option '-fdump-tree-dce2'") .  Patch 6 is needed: it does the necessary
> fixups at the right time to ensure that the per-pass-instance dump files
> get the correct switch names (I'll add some more notes on this to that
> email).
> 
> With the attached patch, the combination of patches (03, 03.1, 04, 05
> *and* 06) successfully bootstraps on x86_64-unknown-linux-gnu, and all
> testcases show the same results as an unpatched build (relative to
> r201397).
> 
> So I'm looking for review of the attached patch, and of at least patch 6
> before I can proceed with this.  AIUI, Jeff is away at the moment, so
> another global reviewer would need to do it.
Here's a better version of this patch: instead of explicitly
initializing the fields (which would become redundant when patch 11 goes
in), instead this patch adds an operator new, using xcalloc.  This
operator new gets reimplemented in a later version of patch 11 to use
ggc_internal_cleared_alloc_stat when the class becomes GC-managed, hence
at each stage of committing, the initialization is sane.

I've successfully bootstrapped the *end result* of the revised patch
series 3-11 on x86_64-unknown-linux-gnu: all testcases show the same
results as an unpatched build (relative to r201397).

OK for trunk? (as part of patches 3-6)


[-- Attachment #2: 0004-Zero-initialize-pass_manager.patch --]
[-- Type: text/x-patch, Size: 1390 bytes --]

From 3198835b85b9a35906bf93ba8f510762e8e017b9 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Wed, 31 Jul 2013 14:20:07 -0400
Subject: [PATCH 04/15] Zero-initialize pass_manager

Ensure that pass_manager instances are fully zero-initialized, by
providing an operator new, implemented using xcalloc.

gcc/

	* passes.c (pass_manager::operator new): New.
---
 gcc/pass_manager.h | 2 ++
 gcc/passes.c       | 7 +++++++
 2 files changed, 9 insertions(+)

diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
index ea078a5..00f0b1c 100644
--- a/gcc/pass_manager.h
+++ b/gcc/pass_manager.h
@@ -47,6 +47,8 @@ class context;
 class pass_manager
 {
 public:
+  void *operator new (size_t sz);
+
   pass_manager(context *ctxt);
 
   void register_pass (struct register_pass_info *pass_info);
diff --git a/gcc/passes.c b/gcc/passes.c
index fcbd630..8efce30 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1339,6 +1339,13 @@ pass_manager::register_pass (struct register_pass_info *pass_info)
 				        -> all_passes
 */
 
+void *
+pass_manager::operator new (size_t sz)
+{
+  /* Ensure that all fields of the pass manager are zero-initialized.  */
+  return xcalloc (1, sz);
+}
+
 pass_manager::pass_manager (context *ctxt)
 : all_passes(NULL), all_small_ipa_passes(NULL), all_lowering_passes(NULL),
   all_regular_ipa_passes(NULL), all_lto_gen_passes(NULL),
-- 
1.7.11.7


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Updated patch 10 (was Re: [PATCH 10/11] Make gcc::context be GC-managed)
  2013-08-02 18:31     ` David Malcolm
@ 2013-08-03  0:43       ` David Malcolm
  2013-08-03 18:19         ` Richard Henderson
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-03  0:43 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 2012 bytes --]

On Fri, 2013-08-02 at 14:31 -0400, David Malcolm wrote:
> On Thu, 2013-08-01 at 11:28 -1000, Richard Henderson wrote:
> > On 07/26/2013 05:04 AM, David Malcolm wrote:
> > > +/* Functions relating to the garbage collector.  */
> > > +void
> > > +gcc::context::gt_ggc_mx ()
> > > +{
> > > +  /* Currently a no-op.  */
> > > +}
> > > +
> > > +void
> > > +gcc::context::gt_pch_nx ()
> > > +{
> > > +  /* Currently a no-op.  */
> > > +}
> > > +
> > > +void
> > > +gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
> > > +			 void *cookie ATTRIBUTE_UNUSED)
> > > +{
> > > +  /* Currently a no-op.  */
> > > +}
> > 
> > I suppose these are members because you'll want to access private members
> > later, and that's easier than playing with "friend"?
> 
> Exactly.  The patch adds a comment about this to gcc/context.h.
> 
> > > +void gt_ggc_mx (gcc::context *ctxt)
> > > +{
> > > +  ctxt->gt_ggc_mx ();
> > > +}
> > > +
> > > +void gt_pch_nx (gcc::context *ctxt)
> > > +{
> > > +  ctxt->gt_pch_nx ();
> > > +}
> > > +
> > > +void gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op, void *cookie)
> > > +{
> > > +  ctxt->gt_pch_nx (op, cookie);
> > > +}
> > 
> > Should these be inline functions in context.h, so that the call does direct?
> > Or do we take their address, making that sorta pointless?
> > 
> > It seems like there's one level of indirection here that is avoidable...
> 
> It looks like they can be inlined - their addresses are not taken, and
> they're only used by gtype-desc.c, inside the autogenerated functions
> gt_ggc_mx_context, gt_pch_nx_context, and gt_pch_p_7context.
> 
> I'll make them inline in context.h

Here's an updated version of the patch which makes them inline.

I've successfully bootstrapped the *end result* of the revised patch
series 3-11 (plus the patch "3.1" from [1]) on x86_64-unknown-linux-gnu:
all testcases show the same results as an unpatched build (relative to
r201397).

OK for trunk?

[1] http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00118.html

[-- Attachment #2: 0011-Make-gcc-context-be-GC-managed.patch --]
[-- Type: text/x-patch, Size: 5276 bytes --]

From f5fba88b187c68f92e3c797b83a3ff4b6e6c2e48 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Tue, 30 Jul 2013 12:30:15 -0400
Subject: [PATCH 11/15] Make gcc::context be GC-managed

This patch makes gcc::context instances be allocated within the GC-heap,
and adds traversal hooks for GC/PCH so that a gcc::context can own refs
to other GC-allocated objects.

gcc/
	* Makefile.in (GTFILES): Add context.h.
	* context.c (gcc::context::operator new): New.
	(gcc::context::gt_ggc_mx): New.
	(gcc::context::gt_pch_nx): New.
	(gcc::context::gt_pch_nx): New.
	* context.h (gcc::context): Add GTY((user)) marking.
	(gcc::context::operator new): New.
	(gcc::context::gt_ggc_mx): New.
	(gcc::context::gt_pch_nx): New.
	(gcc::context::gt_pch_nx): New.
	(g): Add GTY marking.
	(gt_ggc_mx (gcc::context *)): New.
	(gt_pch_nx (gcc::context *)): New.
	(gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op,
	void *cookie)): New.
	* gengtype.c (open_base_files) <ifiles>: Add context.h.
---
 gcc/Makefile.in |  1 +
 gcc/context.c   | 26 +++++++++++++++++++++++++
 gcc/context.h   | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 gcc/gengtype.c  |  2 +-
 4 files changed, 85 insertions(+), 3 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index afce540..61a4d7c 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3819,6 +3819,7 @@ GTFILES = $(CPP_ID_DATA_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
   $(srcdir)/ipa-inline.h \
   $(srcdir)/asan.c \
   $(srcdir)/tsan.c \
+  $(srcdir)/context.h \
   @all_gtfiles@
 
 # Compute the list of GT header files from the corresponding C sources,
diff --git a/gcc/context.c b/gcc/context.c
index b515241..ba6f335 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -27,7 +27,33 @@ along with GCC; see the file COPYING3.  If not see
 /* The singleton holder of global state: */
 gcc::context *g;
 
+void *
+gcc::context::operator new (std::size_t size)
+{
+  return ggc_internal_cleared_alloc_stat (size MEM_STAT_INFO);
+}
+
 gcc::context::context()
 {
   passes_ = new gcc::pass_manager (this);
 }
+
+/* Functions relating to the garbage collector.  */
+void
+gcc::context::gt_ggc_mx ()
+{
+  /* Currently a no-op.  */
+}
+
+void
+gcc::context::gt_pch_nx ()
+{
+  /* Currently a no-op.  */
+}
+
+void
+gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
+			 void *cookie ATTRIBUTE_UNUSED)
+{
+  /* Currently a no-op.  */
+}
diff --git a/gcc/context.h b/gcc/context.h
index 66260cd..2211dc4 100644
--- a/gcc/context.h
+++ b/gcc/context.h
@@ -27,11 +27,30 @@ class pass_manager;
 /* GCC's internal state can be divided into zero or more
    "parallel universe" of state; an instance of this class is one such
    context of state.  */
-class context
+class GTY((user)) context
 {
 public:
+  /* Ensure that instances are allocated within the GC-heap.  */
+  void *operator new (std::size_t size);
+
   context();
 
+  /* Garbage-collector integration.
+
+     Each context assumes it has full control of the GC-heap that it
+     is associated with.  It acts as a root for that GC-heap, owning
+     references to within it.
+
+     Note that context instances are allocated within their own GC
+     heap.
+
+     The methods are called the *first time* that the context is reached
+     during a ggc/pch traversal, rather than every time.  */
+
+  void gt_ggc_mx ();
+  void gt_pch_nx ();
+  void gt_pch_nx (gt_pointer_operator op, void *cookie);
+
   /* Pass-management.  */
 
   pass_manager *get_passes () { gcc_assert (passes_); return passes_; }
@@ -46,6 +65,42 @@ private:
 
 /* The global singleton context aka "g".
    (the name is chosen to be easy to type in a debugger).  */
-extern gcc::context *g;
+extern GTY(()) gcc::context *g;
+
+/* Global hooks for ggc/pch.
+
+   The gcc::context class is marked with GTY((user)), which leads to
+   gengtype creating autogenerated functions for handling context within
+   gtype-desc.c:
+
+     void gt_ggc_mx_context (void *x_p);
+     void gt_pch_nx_context (void *x_p)
+     void gt_pch_p_7context (void *this_obj,
+			     void *x_p,
+			     gt_pointer_operator op,
+			     void *cookie);
+
+   Those functions call the following global functions the first time
+   that the context is reached during a traversal, and the following
+   global functions in turn simply call the corresponding  methods of the
+   context (so that they can access private fields of the context).  */
+
+inline void
+gt_ggc_mx (gcc::context *ctxt)
+{
+  ctxt->gt_ggc_mx ();
+}
+
+inline void
+gt_pch_nx (gcc::context *ctxt)
+{
+  ctxt->gt_pch_nx ();
+}
+
+inline void
+gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op, void *cookie)
+{
+  ctxt->gt_pch_nx (op, cookie);
+}
 
 #endif /* ! GCC_CONTEXT_H */
diff --git a/gcc/gengtype.c b/gcc/gengtype.c
index 50efa9b..941c6e1 100644
--- a/gcc/gengtype.c
+++ b/gcc/gengtype.c
@@ -1733,7 +1733,7 @@ open_base_files (void)
       "tree-flow.h", "reload.h", "cpp-id-data.h", "tree-chrec.h",
       "except.h", "output.h", "gimple.h", "cfgloop.h",
       "target.h", "ipa-prop.h", "lto-streamer.h", "target-globals.h",
-      "ipa-inline.h", "dwarf2out.h", NULL
+      "ipa-inline.h", "dwarf2out.h", "context.h", NULL
     };
     const char *const *ifp;
     outf_p gtype_desc_c;
-- 
1.7.11.7


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-02 20:01       ` Richard Henderson
  2013-08-02 21:53         ` David Malcolm
@ 2013-08-03  0:48         ` David Malcolm
  2013-08-03 18:39           ` Richard Henderson
  1 sibling, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-03  0:48 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]

On Fri, 2013-08-02 at 10:01 -1000, Richard Henderson wrote:
> On 08/02/2013 09:08 AM, David Malcolm wrote:
> > For opt_pass and pass_manager, something different is going on.
> 
> I'd wondered about that.
> 
> > For some reason gengtype doesn't generate the triad of gt_ggc_mx_FOO,
> > gt_pch_nx_FOO, gt_pch_p_NFOO functions in gtype-desc.c, for types
> > FOO=opt_pass and pass_manager.  Presumably this is because the types are
> > only visited by code in context.c
> > 
> > So the global functions for opt_pass and pass_manager are a hand-written
> > implementation of what gengtype would write; they are called *each time*
> > the entity is reached during a traversal.  The member functions are
> > called only the *first time* the entity is visited.
> 
> I wonder if we can reduce the amount of boiler-plate for this.  Perhaps,
> 
> -------------------------------------------------------------------
> //
> // These templates assume the presence of several member functions.
> //
> 
> template<class T>
> inline void gt_ggc_mx (T *p)
> {
>   if (ggc_test_and_set_mark (p))
>     p->gt_ggc_mx ();
> }
> 
> template<class T>
> void gt_pch_nx_with_obj(void *this_obj, void *p,
>                         gt_pointer_operator op, void *cookie)
> {
>   if (p == this_obj)
>     {
>       T *t = static_cast<T *>(p);
>       t->gt_pch_nx_with_obj (op, cookie);
>     }
> }
> 
> template<class T>
> inline void gt_pch_nx (T *p)
> {
>   if (gt_pch_note_object (p, p, gt_pch_nx_with_obj<T>))
>     p->gt_pch_nx ();
> }
> 
> ---------------------------------------------------------------------
> 
> I had thought about an abstract base class instead of the templates, but
> that would unnecessarily force the use of vtables in places that don't
> need them.  In some cases this would be relatively harmless (e.g. the
> very few pass_manager objects), but in others it might be a no-go.
> 
> The use of the template obviates that.  It ought only be instantiated when
> necessary for gty user objects that don't have their own specialization.

Thanks.

Here's an updated version of the patch which uses your template idea,
removing the handcoded functions.

I put the templates in ggc.h, with a big comment.  I changed the
"with_obj" to "with_op", on the grounds that you always have an obj, but
you don't always have an op. 

I stepped through a collection, and observed gcc::context::gt_ggc_mx
calling gt_ggc_mx<gcc::pass_manager>, which then called
gt_ggc_mx<opt_pass>.

I've successfully bootstrapped the *end result* of the revised patch
series 3-11 (plus the patch "3.1" from [1], which upsets the numbering)
on x86_64-unknown-linux-gnu, and ran the testsuite: all testcases show
the same results as an unpatched build (relative to r201397).

OK for trunk?

[1] http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00118.html

[-- Attachment #2: 0012-Make-opt_pass-and-gcc-pass_manager-be-GC-managed.patch --]
[-- Type: text/x-patch, Size: 9964 bytes --]

From 2e3002e597eb7cfdbfb274ede7c02c0510e35e70 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Fri, 2 Aug 2013 15:58:46 -0400
Subject: [PATCH 12/15] Make opt_pass and gcc::pass_manager be GC-managed

This patch makes gcc::pass_manager and opt_pass instances be allocated
within the GC-heap, and adds traversal hooks for GC/PCH, so that passes
can own refs to other GC-allocated objects.

It also adds templates to ggc.h, to create boilerplate for GTY((user))
types that gengtype can't handle.

gcc/
	Make opt_pass and gcc::pass_manager be GC-managed, so that pass
	instances can own GC refs.

	* Makefile.in (GTFILES): Add pass_manager.h and tree-pass.h.
	* context.c (gcc::context::gt_ggc_mx): Traverse passes_.
	(gcc::context::gt_pch_nx): Likewise.
	(gcc::context::gt_pch_nx):  Likewise.
	* ggc.h (gt_ggc_mx <T>): New.
	(gt_pch_nx_with_op <T>): New.
	(gt_pch_nx <T>): New.
	* passes.c (opt_pass::gt_ggc_mx): New.
	(opt_pass::gt_pch_nx): New.
	(opt_pass::gt_pch_nx_with_op): New.
	(pass_manager::gt_ggc_mx): New.
	(pass_manager::gt_pch_nx): New.
	(pass_manager::gt_pch_nx_with_op): New.
	(pass_manager::operator new): Use
	ggc_internal_cleared_alloc_stat rather than xcalloc.
	* pass_manager.h (class pass_manager): Add GTY((user)) marking.
	(pass_manager::gt_ggc_mx): New.
	(pass_manager::gt_pch_nx): New.
	(pass_manager::gt_pch_nx_with_op): New.
	* tree-pass.h (class opt_pass): Add GTY((user)) marking.
	(opt_pass::operator new): New.
	(opt_pass::gt_ggc_mx): New.
	(opt_pass::gt_pch_nx): New.
	(opt_pass::gt_pch_nx_with_op): New.
---
 gcc/Makefile.in    |   2 +
 gcc/context.c      |   6 +--
 gcc/ggc.h          |  46 +++++++++++++++++++++
 gcc/pass_manager.h |   9 ++++-
 gcc/passes.c       | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 gcc/tree-pass.h    |  13 +++++-
 6 files changed, 184 insertions(+), 8 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 61a4d7c..df24bdc 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3820,6 +3820,8 @@ GTFILES = $(CPP_ID_DATA_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
   $(srcdir)/asan.c \
   $(srcdir)/tsan.c \
   $(srcdir)/context.h \
+  $(srcdir)/pass_manager.h \
+  $(srcdir)/tree-pass.h \
   @all_gtfiles@
 
 # Compute the list of GT header files from the corresponding C sources,
diff --git a/gcc/context.c b/gcc/context.c
index ba6f335..698cc57 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -42,18 +42,18 @@ gcc::context::context()
 void
 gcc::context::gt_ggc_mx ()
 {
-  /* Currently a no-op.  */
+  ::gt_ggc_mx (passes_);
 }
 
 void
 gcc::context::gt_pch_nx ()
 {
-  /* Currently a no-op.  */
+  ::gt_pch_nx (passes_);
 }
 
 void
 gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
 			 void *cookie ATTRIBUTE_UNUSED)
 {
-  /* Currently a no-op.  */
+  op (&passes_, cookie);
 }
diff --git a/gcc/ggc.h b/gcc/ggc.h
index b31bc80..e2a1aaf 100644
--- a/gcc/ggc.h
+++ b/gcc/ggc.h
@@ -276,4 +276,50 @@ ggc_alloc_cleared_gimple_statement_d_stat (size_t s MEM_STAT_DECL)
     ggc_internal_cleared_alloc_stat (s PASS_MEM_STAT);
 }
 
+/* gengtype will autogenerate traversal functions (in gtype-desc.c) for
+   all GTY-marked types that it sees are referenced by a GTY marker.
+
+   Unfortunately, it will not generate traveral functions for types that
+   are only referenced by GTY((user)) types.
+
+   The following templates are a substitute, providing equivalent
+   traversal functions for such types.  They are instantiated for
+   types whose objects that are traversed during GC/PCH, and are
+   called *every time* that an instance of type T is traversed during
+   GC/PCH.
+
+   They require the presence of the following member functions
+
+     void gt_ggc_mx ();
+     void gt_pch_nx ();
+     void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
+   within class T, which are called *once* per object - the first
+   time the object is visited during the traversal.  */
+
+template<class T>
+inline void gt_ggc_mx (T *p)
+{
+  if (ggc_test_and_set_mark (p))
+    p->gt_ggc_mx ();
+}
+
+template<class T>
+void gt_pch_nx_with_op (void *this_obj, void *p,
+			gt_pointer_operator op, void *cookie)
+{
+  if (p == this_obj)
+    {
+      T *t = static_cast<T *>(p);
+      t->gt_pch_nx_with_op (op, cookie);
+    }
+}
+
+template<class T>
+inline void gt_pch_nx (T *p)
+{
+  if (gt_pch_note_object (p, p, gt_pch_nx_with_op<T>))
+    p->gt_pch_nx ();
+}
+
 #endif
diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
index 00f0b1c..c861893 100644
--- a/gcc/pass_manager.h
+++ b/gcc/pass_manager.h
@@ -44,13 +44,19 @@ namespace gcc {
 
 class context;
 
-class pass_manager
+class GTY((user)) pass_manager
 {
 public:
+  /* Ensure that instances are allocated in the GC-managed heap.  */
   void *operator new (size_t sz);
 
   pass_manager(context *ctxt);
 
+  /* GTY((user)) methods.  */
+  void gt_ggc_mx ();
+  void gt_pch_nx ();
+  void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
   void register_pass (struct register_pass_info *pass_info);
   void register_one_dump_file (struct opt_pass *pass);
 
@@ -125,4 +131,3 @@ private:
 } // namespace gcc
 
 #endif /* ! GCC_PASS_MANAGER_H */
-
diff --git a/gcc/passes.c b/gcc/passes.c
index aa273fb..db6fc3b 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -82,6 +82,33 @@ struct opt_pass *current_pass;
 
 static void register_pass_name (struct opt_pass *, const char *);
 
+void*
+opt_pass::operator new (size_t sz)
+{
+  return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
+}
+
+void opt_pass::gt_ggc_mx ()
+{
+  ::gt_ggc_mx (ctxt_);
+  ::gt_ggc_mx (sub);
+  ::gt_ggc_mx (next);
+}
+
+void opt_pass::gt_pch_nx ()
+{
+  ::gt_pch_nx (ctxt_);
+  ::gt_pch_nx (sub);
+  ::gt_pch_nx (next);
+}
+
+void opt_pass::gt_pch_nx_with_op (gt_pointer_operator op, void *cookie)
+{
+  op (&(ctxt_), cookie);
+  op (&(sub), cookie);
+  op (&(next), cookie);
+}
+
 /* Most passes are single-instance (within their context) and thus don't
    need to implement cloning, but passes that support multiple instances
    *must* provide their own implementation of the clone method.
@@ -116,6 +143,92 @@ opt_pass::opt_pass(const pass_data &data, context *ctxt)
 {
 }
 
+void
+pass_manager::gt_ggc_mx ()
+{
+  ::gt_ggc_mx (all_passes);
+  ::gt_ggc_mx (all_small_ipa_passes);
+  ::gt_ggc_mx (all_lowering_passes);
+  ::gt_ggc_mx (all_regular_ipa_passes);
+  ::gt_ggc_mx (all_lto_gen_passes);
+  ::gt_ggc_mx (all_late_ipa_passes);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    ::gt_ggc_mx (passes_by_id[i]);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
+
+void
+pass_manager::gt_pch_nx ()
+{
+  ::gt_pch_nx (all_passes);
+  ::gt_pch_nx (all_small_ipa_passes);
+  ::gt_pch_nx (all_lowering_passes);
+  ::gt_pch_nx (all_regular_ipa_passes);
+  ::gt_pch_nx (all_lto_gen_passes);
+  ::gt_pch_nx (all_late_ipa_passes);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    ::gt_pch_nx (passes_by_id[i]);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) ::gt_pch_nx (PASS ## _ ## NUM);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
+
+void
+pass_manager::gt_pch_nx_with_op (gt_pointer_operator op, void *cookie)
+{
+  op (&(all_passes), cookie);
+  op (&(all_small_ipa_passes), cookie);
+  op (&(all_lowering_passes), cookie);
+  op (&(all_regular_ipa_passes), cookie);
+  op (&(all_lto_gen_passes), cookie);
+  op (&(all_late_ipa_passes), cookie);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    op (&(passes_by_id[i]), cookie);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) op (&(PASS ## _ ## NUM), cookie);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
 
 void
 pass_manager::execute_early_local_passes ()
@@ -129,7 +242,6 @@ pass_manager::execute_pass_mode_switching ()
   return pass_mode_switching_1->execute ();
 }
 
-
 /* Call from anywhere to find out what pass this is.  Useful for
    printing out debugging information deep inside an service
    routine.  */
@@ -1464,7 +1576,7 @@ void *
 pass_manager::operator new (size_t sz)
 {
   /* Ensure that all fields of the pass manager are zero-initialized.  */
-  return xcalloc (1, sz);
+  return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
 }
 
 pass_manager::pass_manager (context *ctxt)
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 41d5d92..063b166 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -76,11 +76,22 @@ namespace gcc
 
 /* An instance of a pass.  This is also "pass_data" to minimize the
    changes in existing code.  */
-class opt_pass : public pass_data
+class GTY((user)) opt_pass : public pass_data
 {
 public:
+  /* Ensure that instances are allocated in the GC-managed heap.  */
+  void *operator new (size_t sz);
+
   virtual ~opt_pass () { }
 
+  /* GTY((user)) methods, to be called once per traversal.
+     opt_pass subclasses with additional GC-managed data should overide
+     these, chain up to the base class implementation, then walk their
+     extra fields.  */
+  virtual void gt_ggc_mx ();
+  virtual void gt_pch_nx ();
+  virtual void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
   /* Create a copy of this pass.
 
      Passes that can have multiple instances must provide their own
-- 
1.7.11.7


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-08-03  0:39       ` David Malcolm
@ 2013-08-03 17:59         ` Richard Henderson
  0 siblings, 0 replies; 74+ messages in thread
From: Richard Henderson @ 2013-08-03 17:59 UTC (permalink / raw)
  To: David Malcolm; +Cc: Jeff Law, gcc-patches

On 08/02/2013 02:38 PM, David Malcolm wrote:
> OK for trunk? (as part of patches 3-6)

Ok.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Updated patch 10 (was Re: [PATCH 10/11] Make gcc::context be GC-managed)
  2013-08-03  0:43       ` Updated patch 10 (was Re: [PATCH 10/11] Make gcc::context be GC-managed) David Malcolm
@ 2013-08-03 18:19         ` Richard Henderson
  2013-08-20  0:55           ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-03 18:19 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/02/2013 02:43 PM, David Malcolm wrote:
> gcc/
> 	* Makefile.in (GTFILES): Add context.h.
> 	* context.c (gcc::context::operator new): New.
> 	(gcc::context::gt_ggc_mx): New.
> 	(gcc::context::gt_pch_nx): New.
> 	(gcc::context::gt_pch_nx): New.
> 	* context.h (gcc::context): Add GTY((user)) marking.
> 	(gcc::context::operator new): New.
> 	(gcc::context::gt_ggc_mx): New.
> 	(gcc::context::gt_pch_nx): New.
> 	(gcc::context::gt_pch_nx): New.
> 	(g): Add GTY marking.
> 	(gt_ggc_mx (gcc::context *)): New.
> 	(gt_pch_nx (gcc::context *)): New.
> 	(gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op,
> 	void *cookie)): New.
> 	* gengtype.c (open_base_files) <ifiles>: Add context.h.

Ok.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-03  0:48         ` Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed) David Malcolm
@ 2013-08-03 18:39           ` Richard Henderson
  2013-08-05 15:18             ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-03 18:39 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/02/2013 02:48 PM, David Malcolm wrote:
> +pass_manager::gt_ggc_mx ()
> +{
> +  ::gt_ggc_mx (all_passes);
> +  ::gt_ggc_mx (all_small_ipa_passes);
> +  ::gt_ggc_mx (all_lowering_passes);
> +  ::gt_ggc_mx (all_regular_ipa_passes);
> +  ::gt_ggc_mx (all_lto_gen_passes);
> +  ::gt_ggc_mx (all_late_ipa_passes);
> +
> +  for (int i = 0; i < passes_by_id_size; i++)
> +    ::gt_ggc_mx (passes_by_id[i]);
> +
> +#define INSERT_PASSES_AFTER(PASS)
> +#define PUSH_INSERT_PASSES_WITHIN(PASS)
> +#define POP_INSERT_PASSES()
> +#define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
> +#define TERMINATE_PASS_LIST()
> +
> +#include "pass-instances.def"
> +
> +#undef INSERT_PASSES_AFTER
> +#undef PUSH_INSERT_PASSES_WITHIN
> +#undef POP_INSERT_PASSES
> +#undef NEXT_PASS
> +#undef TERMINATE_PASS_LIST
> +
> +}

You're marking all passes, and also walking through the chain of sub/next
within the passes themselves?  That seems unnecessary.  Either the passes
are reachable via sub/next or they aren't.

Alternately, don't walk the sub/next fields and instead only walk all of
the passes explicitly, here.  That avoids some of the recursion in the
opt_pass marking, and keeps the call chain flatter.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-03 18:39           ` Richard Henderson
@ 2013-08-05 15:18             ` David Malcolm
  2013-08-05 16:59               ` Richard Henderson
  0 siblings, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-05 15:18 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Sat, 2013-08-03 at 08:39 -1000, Richard Henderson wrote:
> On 08/02/2013 02:48 PM, David Malcolm wrote:
> > +pass_manager::gt_ggc_mx ()
> > +{
> > +  ::gt_ggc_mx (all_passes);
> > +  ::gt_ggc_mx (all_small_ipa_passes);
> > +  ::gt_ggc_mx (all_lowering_passes);
> > +  ::gt_ggc_mx (all_regular_ipa_passes);
> > +  ::gt_ggc_mx (all_lto_gen_passes);
> > +  ::gt_ggc_mx (all_late_ipa_passes);
> > +
> > +  for (int i = 0; i < passes_by_id_size; i++)
> > +    ::gt_ggc_mx (passes_by_id[i]);
> > +
> > +#define INSERT_PASSES_AFTER(PASS)
> > +#define PUSH_INSERT_PASSES_WITHIN(PASS)
> > +#define POP_INSERT_PASSES()
> > +#define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
> > +#define TERMINATE_PASS_LIST()
> > +
> > +#include "pass-instances.def"
> > +
> > +#undef INSERT_PASSES_AFTER
> > +#undef PUSH_INSERT_PASSES_WITHIN
> > +#undef POP_INSERT_PASSES
> > +#undef NEXT_PASS
> > +#undef TERMINATE_PASS_LIST
> > +
> > +}
> 
> You're marking all passes, and also walking through the chain of sub/next
> within the passes themselves?  That seems unnecessary.  Either the passes
> are reachable via sub/next or they aren't.
> 
> Alternately, don't walk the sub/next fields and instead only walk all of
> the passes explicitly, here.  That avoids some of the recursion in the
> opt_pass marking, and keeps the call chain flatter.

Each _mx call is implemented like this:
  if (ggc_test_and_set_mark (p))
    p->gt_ggc_mx ();
i.e. each pass only recurses the first time it is seen.

So if the goal is to avoid a deep traversal of the call chain, then
walking the passes_by_id array *backwards* ought to walk the pass tree
from the leaf passes first, eventually reaching the trunk passes, and
hence none of the calls should recurse, given that at each point in the
iteration pass->sub and pass->next will already been marked.

So I *think* the most efficient traversal is to do this first (with a
suitable comment):

 for (int i = passes_by_id_size ; i > 0; )
   ::gt_ggc_mx (passes_by_id[--i]);

That ought to visit all of the passes without triggering recursion
(unless someone does something weird to the pass structure in a plugin).

I'm nervous about omitting the traversal for other pointers the class
has (though I *think* the passes by id array captures it all) so for
completeness I'd prefer to then to also do it for all of:

 ::gt_ggc_mx (all_passes);
 ::gt_ggc_mx (all_small_ipa_passes);
 ::gt_ggc_mx (all_lowering_passes);
 ::gt_ggc_mx (all_regular_ipa_passes);
 ::gt_ggc_mx (all_lto_gen_passes);
 ::gt_ggc_mx (all_late_ipa_passes);

#define INSERT_PASSES_AFTER(PASS)
#define PUSH_INSERT_PASSES_WITHIN(PASS)
#define POP_INSERT_PASSES()
#define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
#define TERMINATE_PASS_LIST()

#include "pass-instances.def"

#undef INSERT_PASSES_AFTER
#undef PUSH_INSERT_PASSES_WITHIN
#undef POP_INSERT_PASSES
#undef NEXT_PASS
#undef TERMINATE_PASS_LIST

Is it standard in handwritten GC code to omit traversals based on this
higher-level knowledge?  Presumably it warrants a source comment?
(having spent too much time lately debugging PCH issues I'm nervous
about trying to optimize this).

AIUI we could do similar optimizations in the PCH object-noting hook,
but can't do similar optimizations in the PCH *op* hook, since every
field needs to reconstructed when reading the data back from disk.

Thanks
Dave


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-05 15:18             ` David Malcolm
@ 2013-08-05 16:59               ` Richard Henderson
  2013-08-05 17:28                 ` David Malcolm
  2013-08-16 15:33                 ` v3 of " David Malcolm
  0 siblings, 2 replies; 74+ messages in thread
From: Richard Henderson @ 2013-08-05 16:59 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/05/2013 05:18 AM, David Malcolm wrote:
> So I *think* the most efficient traversal is to do this first (with a
> suitable comment):
> 
>  for (int i = passes_by_id_size ; i > 0; )
>    ::gt_ggc_mx (passes_by_id[--i]);
> 
> That ought to visit all of the passes without triggering recursion
> (unless someone does something weird to the pass structure in a plugin).

Reasonable.

> I'm nervous about omitting the traversal for other pointers the class
> has (though I *think* the passes by id array captures it all) so for
> completeness I'd prefer to then to also do it for all of:
> 
>  ::gt_ggc_mx (all_passes);
>  ::gt_ggc_mx (all_small_ipa_passes);
>  ::gt_ggc_mx (all_lowering_passes);
>  ::gt_ggc_mx (all_regular_ipa_passes);
>  ::gt_ggc_mx (all_lto_gen_passes);
>  ::gt_ggc_mx (all_late_ipa_passes);
> 
> #define INSERT_PASSES_AFTER(PASS)
> #define PUSH_INSERT_PASSES_WITHIN(PASS)
> #define POP_INSERT_PASSES()
> #define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
> #define TERMINATE_PASS_LIST()
> 
> #include "pass-instances.def"
> 
> #undef INSERT_PASSES_AFTER
> #undef PUSH_INSERT_PASSES_WITHIN
> #undef POP_INSERT_PASSES
> #undef NEXT_PASS
> #undef TERMINATE_PASS_LIST
> 
> Is it standard in handwritten GC code to omit traversals based on this
> higher-level knowledge?  Presumably it warrants a source comment?
> (having spent too much time lately debugging PCH issues I'm nervous
> about trying to optimize this).

It's something that we should think about when doing any kind of GC.
The deep recursion has bitten us before, and lead to the chain_next
and chain_prev annotations for GTY.

> AIUI we could do similar optimizations in the PCH object-noting hook,
> but can't do similar optimizations in the PCH *op* hook, since every
> field needs to reconstructed when reading the data back from disk.

Correct.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-05 16:59               ` Richard Henderson
@ 2013-08-05 17:28                 ` David Malcolm
  2013-08-16 15:33                 ` v3 of " David Malcolm
  1 sibling, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-05 17:28 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Mon, 2013-08-05 at 06:59 -1000, Richard Henderson wrote:
> On 08/05/2013 05:18 AM, David Malcolm wrote:
> > So I *think* the most efficient traversal is to do this first (with a
> > suitable comment):
> > 
> >  for (int i = passes_by_id_size ; i > 0; )
> >    ::gt_ggc_mx (passes_by_id[--i]);
> > 
> > That ought to visit all of the passes without triggering recursion
> > (unless someone does something weird to the pass structure in a plugin).
> 
> Reasonable.
> 
> > I'm nervous about omitting the traversal for other pointers the class
> > has (though I *think* the passes by id array captures it all) so for
> > completeness I'd prefer to then to also do it for all of:
> > 
> >  ::gt_ggc_mx (all_passes);
> >  ::gt_ggc_mx (all_small_ipa_passes);
> >  ::gt_ggc_mx (all_lowering_passes);
> >  ::gt_ggc_mx (all_regular_ipa_passes);
> >  ::gt_ggc_mx (all_lto_gen_passes);
> >  ::gt_ggc_mx (all_late_ipa_passes);
> > 
> > #define INSERT_PASSES_AFTER(PASS)
> > #define PUSH_INSERT_PASSES_WITHIN(PASS)
> > #define POP_INSERT_PASSES()
> > #define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
> > #define TERMINATE_PASS_LIST()
> > 
> > #include "pass-instances.def"
> > 
> > #undef INSERT_PASSES_AFTER
> > #undef PUSH_INSERT_PASSES_WITHIN
> > #undef POP_INSERT_PASSES
> > #undef NEXT_PASS
> > #undef TERMINATE_PASS_LIST
> > 
> > Is it standard in handwritten GC code to omit traversals based on this
> > higher-level knowledge?  Presumably it warrants a source comment?
> > (having spent too much time lately debugging PCH issues I'm nervous
> > about trying to optimize this).
> 
> It's something that we should think about when doing any kind of GC.
> The deep recursion has bitten us before, and lead to the chain_next
> and chain_prev annotations for GTY.

Note to self: the chain_next and chain_prev options are documented at:
http://gcc.gnu.org/onlinedocs/gccint/GTY-Options.html

BTW, this issue seems very reminiscent to me of an implementation detail
within the CPython interpreter called the "trashcan".  CPython uses
reference-counting, and if you have a deep chain of references e.g.:
  # make a very deep chain of list-of-lists:
  foo = []
  for i in range(depth):
     foo = [foo]
  # at this point we have
  # foo = [[[[[[[[[[[[[[[[[[[[[[[[[[[[ \
  #       ]]]]]]]]]]]]]]]]]]]]]]]]]]]] for some depth

  # try to overflow the stack during deallocation:
  del foo

...you would have a deeply nested chain of decrefs, each triggering
deleting of the object, and hence a potential stack overflow.

The way CPython avoids such deep reference chains from killing the
process with a stack overflow is a pair of macros used in the
deallocator for container types, which track the depth of the traversal,
and when it reaches a certain threshold, start appending objects to a
singly-linked list of deferred deallocations (using a spare pointer in
the objects that's safe to use during finalization).  Hence the
recursion becomes an iteration when a certain stack depth is reached,
and diabolical reference graphs can't blow the stack (modulo bugs in
3rd-party container code, of course).

The equivalent for GC would be for mx routines to change from:

  if (ggc_test_and_set_mark (p))
    gt_ggc_mx (p);

to:

  if (ggc_test_and_set_mark (p))
    add_to_worklist (p, marking_cb);

I suspect that this technique can't be used in GGC since it would
require (a) having a spare pointer per-object and (b) tracking the type
of the marking callback, which would be a jump through a function ptr
where there wasn't one before, and thus unacceptable in the general
case.

> > AIUI we could do similar optimizations in the PCH object-noting hook,
> > but can't do similar optimizations in the PCH *op* hook, since every
> > field needs to reconstructed when reading the data back from disk.
> 
> Correct.

I'll update the patch with the optimizations you suggest (and the
reverse-order marking I mentioned above).

Thanks
Dave

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-07-29 22:39   ` Jeff Law
  2013-08-01 17:13     ` [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) David Malcolm
@ 2013-08-05 21:03     ` David Malcolm
  2013-08-06 15:16       ` Steve Ellcey
  2013-08-06 19:12       ` Basile Starynkevitch
  1 sibling, 2 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-05 21:03 UTC (permalink / raw)
  To: Jeff Law; +Cc: gcc-patches

On Mon, 2013-07-29 at 15:41 -0600, Jeff Law wrote:
> On 07/26/2013 09:04 AM, David Malcolm wrote:
> > This patch is the hand-written part of the conversion of passes from
> > C structs to C++ classes.  It does not work without the subsequent
> > autogenerated part, which is huge.
> [ ... ]
> With the changes from pipeline -> pass_manager this is fine.  As is the 
> follow-up autogenerated patch.

I've committed this and the other patches that needed to go with it to
trunk, with the name fix, having successfully bootstrapped and tested
them on x86_64-unknown-linux-gnu - so opt_pass is now a C++ class
hierarchy, allowing for pass instances to hold pass-specific data
(albeit not GTY refs yet), and the future possibility of multiple pass
managers and pass pipelines (e.g. for embedding GCC for
JIT-compilation).

Specifically, I fixed the names and rebased against master and tested
repeatedly (for the later GC patches, which are still being reviewed).
I refreshed the "03/11" patch (which became r201505) to resolve a
trivial conflict in cgraphunit.c, and then bootstrapped and tested the
series of 5 patches against r201494 on x86_64-unknown-linux-gnu (RHEL
6.4 in fact).

It successfully built all 3 stages, and gave the same testing results as
an unpatched build of r201494.

Given that, I've committed the following to trunk:
  * r201505: this patch (aka 03/11; handwritten part of conversion,
approved above by Jeff)
  * r201506: the zero-initialization of pass_manager (aka 3.1/11;
approved by rth in
http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00147.html )
  * r201508: the big autogenerated patch (aka 04/11; approved above by
Jeff)
  * r201509: adding -fno-rtti when building plugins in the testsuite
(aka 05/11; approved by Jeff in
http://gcc.gnu.org/ml/gcc-patches/2013-07/msg01433.html )
  * r201511: aka 06/11; using pass->clone and fixing pass switch
numbering (approved by rth in
http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00054.html )

Note that r201505 and r201506 don't compile by themselves; they need
r201508 to update all of the passes to the new internal API, and r201509
and r201511 are then needed otherwise you'll see regressions in the
testsuite.

Given all of the above testing I'm reasonably confident that this works.
However this is such a large change [1] that there's a non-zero chance
of at least one glitch - let me know if you see any breakages.  One
glitch that I ran into when applying the patches to svn was that I saw
some end-of-file whitespace changes in some of the gcc/testsuite files:
I'm not sure how those crept in, but I used "svn revert" by hand on them
before committing to ensure that only the files I was expecting to
change were changed.

Some of followup patches in the series are now approved, some aren't, so
I'm next going to bootstrap/test/commit the approved ones as
appropriate.

Dave

[1] 127 files changed, 10227 insertions(+), 4465 deletions(-)

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-08-05 21:03     ` Passes are now C++ classes " David Malcolm
@ 2013-08-06 15:16       ` Steve Ellcey
  2013-08-06 17:55         ` [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)) David Malcolm
  2013-08-07 13:10         ` Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) Eric Botcazou
  2013-08-06 19:12       ` Basile Starynkevitch
  1 sibling, 2 replies; 74+ messages in thread
From: Steve Ellcey @ 2013-08-06 15:16 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On Mon, 2013-08-05 at 17:03 -0400, David Malcolm wrote:

> Given all of the above testing I'm reasonably confident that this works.
> However this is such a large change [1] that there's a non-zero chance
> of at least one glitch - let me know if you see any breakages.

The mips*-*-* targets are not building.  It looks like the mips reorg
pass (pass_mips_machine_reorg2) in config/mips/mips.c was not converted
and/or was not converted correctly.

Steve Ellcey
sellcey@mips.com


^ permalink raw reply	[flat|nested] 74+ messages in thread

* [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes))
  2013-08-06 15:16       ` Steve Ellcey
@ 2013-08-06 17:55         ` David Malcolm
  2013-08-06 18:12           ` Richard Sandiford
  2013-08-07 13:10         ` Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) Eric Botcazou
  1 sibling, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-06 17:55 UTC (permalink / raw)
  To: Steve Ellcey; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]

On Tue, 2013-08-06 at 08:16 -0700, Steve Ellcey wrote:
> On Mon, 2013-08-05 at 17:03 -0400, David Malcolm wrote:
> 
> > Given all of the above testing I'm reasonably confident that this works.
> > However this is such a large change [1] that there's a non-zero chance
> > of at least one glitch - let me know if you see any breakages.
> 
> The mips*-*-* targets are not building.  It looks like the mips reorg
> pass (pass_mips_machine_reorg2) in config/mips/mips.c was not converted
> and/or was not converted correctly.

Sorry about this.

I was able to reproduce this build error with configure
--target=mips-linux-elf:
../../src/gcc/config/mips/mips.c:16379:28: error: expected
primary-expression before ‘.’ token

I'm attaching a patch which fixes that issue for me.  Only lightly
tested (build&host=x86_64, target as above) - I'm able to build stage1,
and cc1 appears to generate assembler on a simple test case.  I stepped
through the changed code in the debugger and it appears to do the right
thing.  However I'm not familiar with the internals of the pass in
question.


[-- Attachment #2: port-mips-to-new-api.patch --]
[-- Type: text/x-patch, Size: 1856 bytes --]

commit 11d46884e8bd9802b0f528a16b3970b4076fe8a9
Author: David Malcolm <dmalcolm@redhat.com>
Date:   Tue Aug 6 13:48:59 2013 -0400

    gcc/
    	* config/mips/mips.c (insert_pass_mips_machine_reorg2): Move
    	into...
    	(mips_option_override): ...here, porting to new C++ API for
    	passes.

diff --git a/gcc/config/mips/mips.c b/gcc/config/mips/mips.c
index 05ba003..4da80f4 100644
--- a/gcc/config/mips/mips.c
+++ b/gcc/config/mips/mips.c
@@ -56,6 +56,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "target-globals.h"
 #include "opts.h"
 #include "tree-pass.h"
+#include "context.h"
 
 /* True if X is an UNSPEC wrapper around a SYMBOL_REF or LABEL_REF.  */
 #define UNSPEC_ADDRESS_P(X)					\
@@ -16374,13 +16375,6 @@ make_pass_mips_machine_reorg2 (gcc::context *ctxt)
   return new pass_mips_machine_reorg2 (ctxt);
 }
 
-struct register_pass_info insert_pass_mips_machine_reorg2 =
-{
-  &pass_mips_machine_reorg2.pass,	/* pass */
-  "dbr",				/* reference_pass_name */
-  1,					/* ref_pass_instance_number */
-  PASS_POS_INSERT_AFTER			/* po_op */
-};
 \f
 /* Implement TARGET_ASM_OUTPUT_MI_THUNK.  Generate rtl rather than asm text
    in order to avoid duplicating too much logic from elsewhere.  */
@@ -17174,6 +17168,14 @@ mips_option_override (void)
   /* We register a second machine specific reorg pass after delay slot
      filling.  Registering the pass must be done at start up.  It's
      convenient to do it here.  */
+  opt_pass *new_pass = make_pass_mips_machine_reorg2 (g);
+  struct register_pass_info insert_pass_mips_machine_reorg2 =
+    {
+      new_pass,		/* pass */
+      "dbr",			/* reference_pass_name */
+      1,			/* ref_pass_instance_number */
+      PASS_POS_INSERT_AFTER	/* po_op */
+    };
   register_pass (&insert_pass_mips_machine_reorg2);
 
   if (TARGET_HARD_FLOAT_ABI && TARGET_MIPS5900)

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes))
  2013-08-06 17:55         ` [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)) David Malcolm
@ 2013-08-06 18:12           ` Richard Sandiford
  2013-08-06 18:52             ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Sandiford @ 2013-08-06 18:12 UTC (permalink / raw)
  To: David Malcolm; +Cc: Steve Ellcey, gcc-patches

David Malcolm <dmalcolm@redhat.com> writes:
> commit 11d46884e8bd9802b0f528a16b3970b4076fe8a9
> Author: David Malcolm <dmalcolm@redhat.com>
> Date:   Tue Aug 6 13:48:59 2013 -0400
>
>     gcc/
>     	* config/mips/mips.c (insert_pass_mips_machine_reorg2): Move
>     	into...
>     	(mips_option_override): ...here, porting to new C++ API for
>     	passes.

OK.  Thanks for the quick fix!

Richard

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes))
  2013-08-06 18:12           ` Richard Sandiford
@ 2013-08-06 18:52             ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-06 18:52 UTC (permalink / raw)
  To: Richard Sandiford; +Cc: Steve Ellcey, gcc-patches

On Tue, 2013-08-06 at 19:11 +0100, Richard Sandiford wrote:
> David Malcolm <dmalcolm@redhat.com> writes:
> > commit 11d46884e8bd9802b0f528a16b3970b4076fe8a9
> > Author: David Malcolm <dmalcolm@redhat.com>
> > Date:   Tue Aug 6 13:48:59 2013 -0400
> >
> >     gcc/
> >     	* config/mips/mips.c (insert_pass_mips_machine_reorg2): Move
> >     	into...
> >     	(mips_option_override): ...here, porting to new C++ API for
> >     	passes.
> 
> OK.  Thanks for the quick fix!

Thanks; committed to trunk as r201542.


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-08-05 21:03     ` Passes are now C++ classes " David Malcolm
  2013-08-06 15:16       ` Steve Ellcey
@ 2013-08-06 19:12       ` Basile Starynkevitch
  1 sibling, 0 replies; 74+ messages in thread
From: Basile Starynkevitch @ 2013-08-06 19:12 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On Mon, 2013-08-05 at 17:03 -0400, David Malcolm wrote:
> On Mon, 2013-07-29 at 15:41 -0600, Jeff Law wrote:
> > On 07/26/2013 09:04 AM, David Malcolm wrote:
> > > This patch is the hand-written part of the conversion of passes from
> > > C structs to C++ classes.  It does not work without the subsequent
> > > autogenerated part, which is huge.
> > [ ... ]
> > With the changes from pipeline -> pass_manager this is fine.  As is the 
> > follow-up autogenerated patch.
> 
> I've committed this and the other patches that needed to go with it to
> trunk, with the name fix, having successfully bootstrapped and tested
> them on x86_64-unknown-linux-gnu - so opt_pass is now a C++ class
> hierarchy, allowing for pass instances to hold pass-specific data
> (albeit not GTY refs yet), 

Did you follow my suggestion of putting __FILE__ and __LINE__ in
instances of opt_pass? I really hope that will go into 4.9!

Cheers


-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-08-06 15:16       ` Steve Ellcey
  2013-08-06 17:55         ` [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)) David Malcolm
@ 2013-08-07 13:10         ` Eric Botcazou
  1 sibling, 0 replies; 74+ messages in thread
From: Eric Botcazou @ 2013-08-07 13:10 UTC (permalink / raw)
  To: gcc-patches; +Cc: Steve Ellcey, David Malcolm

> The mips*-*-* targets are not building.  It looks like the mips reorg
> pass (pass_mips_machine_reorg2) in config/mips/mips.c was not converted
> and/or was not converted correctly.

Likewise for the SPARC targets, because of the same issue.  David, could you 
take a quick look?  Thanks in advance.

-- 
Eric Botcazou

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c
  2013-08-01 20:48   ` Richard Henderson
@ 2013-08-13  0:42     ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-13  0:42 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Thu, 2013-08-01 at 10:48 -1000, Richard Henderson wrote:
[...]
> Ok.

Thanks; committed to trunk as r201680 (having rebootstrapped and tested)


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 08/11] Example of converting global state to per-pass state
  2013-08-01 20:58   ` Richard Henderson
@ 2013-08-13  0:46     ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-13  0:46 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Thu, 2013-08-01 at 10:58 -1000, Richard Henderson wrote:
[...]
> Ok.
Thanks; committed to svn trunk as r201681 (having rebootstrap and
tested).

^ permalink raw reply	[flat|nested] 74+ messages in thread

* v3 of patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-05 16:59               ` Richard Henderson
  2013-08-05 17:28                 ` David Malcolm
@ 2013-08-16 15:33                 ` David Malcolm
  2013-08-19 17:57                   ` Richard Henderson
  1 sibling, 1 reply; 74+ messages in thread
From: David Malcolm @ 2013-08-16 15:33 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

[-- Attachment #1: Type: text/plain, Size: 3174 bytes --]

On Mon, 2013-08-05 at 06:59 -1000, Richard Henderson wrote:
> On 08/05/2013 05:18 AM, David Malcolm wrote:
> > So I *think* the most efficient traversal is to do this first (with a
> > suitable comment):
> > 
> >  for (int i = passes_by_id_size ; i > 0; )
> >    ::gt_ggc_mx (passes_by_id[--i]);
> > 
> > That ought to visit all of the passes without triggering recursion
> > (unless someone does something weird to the pass structure in a plugin).
> 
> Reasonable.
> 
> > I'm nervous about omitting the traversal for other pointers the class
> > has (though I *think* the passes by id array captures it all) so for
> > completeness I'd prefer to then to also do it for all of:
> > 
> >  ::gt_ggc_mx (all_passes);
> >  ::gt_ggc_mx (all_small_ipa_passes);
> >  ::gt_ggc_mx (all_lowering_passes);
> >  ::gt_ggc_mx (all_regular_ipa_passes);
> >  ::gt_ggc_mx (all_lto_gen_passes);
> >  ::gt_ggc_mx (all_late_ipa_passes);
> > 
> > #define INSERT_PASSES_AFTER(PASS)
> > #define PUSH_INSERT_PASSES_WITHIN(PASS)
> > #define POP_INSERT_PASSES()
> > #define NEXT_PASS(PASS, NUM) ::gt_ggc_mx (PASS ## _ ## NUM);
> > #define TERMINATE_PASS_LIST()
> > 
> > #include "pass-instances.def"
> > 
> > #undef INSERT_PASSES_AFTER
> > #undef PUSH_INSERT_PASSES_WITHIN
> > #undef POP_INSERT_PASSES
> > #undef NEXT_PASS
> > #undef TERMINATE_PASS_LIST
> > 
> > Is it standard in handwritten GC code to omit traversals based on this
> > higher-level knowledge?  Presumably it warrants a source comment?
> > (having spent too much time lately debugging PCH issues I'm nervous
> > about trying to optimize this).
> 
> It's something that we should think about when doing any kind of GC.
> The deep recursion has bitten us before, and lead to the chain_next
> and chain_prev annotations for GTY.
> 
> > AIUI we could do similar optimizations in the PCH object-noting hook,
> > but can't do similar optimizations in the PCH *op* hook, since every
> > field needs to reconstructed when reading the data back from disk.
> 
> Correct.

Sorry about the delay in following up on this.

I'm attaching an implementation of patch which incorporates the ideas
for optimizing the GC traversal.

It turned out that the passes_by_id array only contains *most* of the
passes, not all of them:  it only contains passes that had
register_one_dump_file called on them, thus omitting those that have a
"*" at the front of their names (or a NULL name, but I don't think there
are any of these).  So the pass_manager traversal hooks need to do some
extra work after the backwards traversal of the passes_by_id array to
ensure that we visit everything.

I also tweaked the traversal hooks for opt_pass to emulate "chain_next",
since this is where the really deep callchains could otherwise occur.

See the patch for details (given the subtleties I opted to put big
comments in the relevant routines).

(I also fixed the typo "overide" to "override" as noted by Bernhard)

Successfully bootstrapped and tested on x86_64-unknown-linux-gnu, on top
of the other patches (Note that the gengtype fix I committed just now as
r201791 is also needed [1]).

OK for trunk?

[1] http://gcc.gnu.org/ml/gcc-patches/2013-08/msg00882.html

[-- Attachment #2: passes-and-gty-v3.patch --]
[-- Type: text/x-patch, Size: 11307 bytes --]

commit 5deba6060b8d9991b6298af2d1122310e0415043
Author: David Malcolm <dmalcolm@redhat.com>
Date:   Fri Aug 2 15:58:46 2013 -0400

    Make opt_pass and gcc::pass_manager be GC-managed
    
    This patch makes gcc::pass_manager and opt_pass instances be allocated
    within the GC-heap, and adds traversal hooks for GC/PCH, so that passes
    can own refs to other GC-allocated objects.
    
    It also adds templates to ggc.h, to create boilerplate for GTY((user))
    types that gengtype can't handle.
    
    gcc/
    	Make opt_pass and gcc::pass_manager be GC-managed, so that pass
    	instances can own GC refs.
    
    	* Makefile.in (GTFILES): Add pass_manager.h and tree-pass.h.
    	* context.c (gcc::context::gt_ggc_mx): Traverse passes_.
    	(gcc::context::gt_pch_nx): Likewise.
    	(gcc::context::gt_pch_nx):  Likewise.
    	* ggc.h (gt_ggc_mx <T>): New.
    	(gt_pch_nx_with_op <T>): New.
    	(gt_pch_nx <T>): New.
    	* passes.c (opt_pass::gt_ggc_mx): New.
    	(opt_pass::gt_pch_nx): New.
    	(opt_pass::gt_pch_nx_with_op): New.
    	(pass_manager::gt_ggc_mx): New.
    	(pass_manager::gt_pch_nx): New.
    	(pass_manager::gt_pch_nx_with_op): New.
    	(pass_manager::operator new): Use
    	ggc_internal_cleared_alloc_stat rather than xcalloc.
    	* pass_manager.h (class pass_manager): Add GTY((user)) marking.
    	(pass_manager::gt_ggc_mx): New.
    	(pass_manager::gt_pch_nx): New.
    	(pass_manager::gt_pch_nx_with_op): New.
    	* tree-pass.h (class opt_pass): Add GTY((user)) marking.
    	(opt_pass::operator new): New.
    	(opt_pass::gt_ggc_mx): New.
    	(opt_pass::gt_pch_nx): New.
    	(opt_pass::gt_pch_nx_with_op): New.

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index b874a6b..9917aec 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -3828,6 +3828,8 @@ GTFILES = $(CPP_ID_DATA_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
   $(srcdir)/asan.c \
   $(srcdir)/tsan.c \
   $(srcdir)/context.h \
+  $(srcdir)/pass_manager.h \
+  $(srcdir)/tree-pass.h \
   @all_gtfiles@
 
 # Compute the list of GT header files from the corresponding C sources,
diff --git a/gcc/context.c b/gcc/context.c
index ba6f335..698cc57 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -42,18 +42,18 @@ gcc::context::context()
 void
 gcc::context::gt_ggc_mx ()
 {
-  /* Currently a no-op.  */
+  ::gt_ggc_mx (passes_);
 }
 
 void
 gcc::context::gt_pch_nx ()
 {
-  /* Currently a no-op.  */
+  ::gt_pch_nx (passes_);
 }
 
 void
 gcc::context::gt_pch_nx (gt_pointer_operator op ATTRIBUTE_UNUSED,
 			 void *cookie ATTRIBUTE_UNUSED)
 {
-  /* Currently a no-op.  */
+  op (&passes_, cookie);
 }
diff --git a/gcc/ggc.h b/gcc/ggc.h
index b31bc80..e2a1aaf 100644
--- a/gcc/ggc.h
+++ b/gcc/ggc.h
@@ -276,4 +276,50 @@ ggc_alloc_cleared_gimple_statement_d_stat (size_t s MEM_STAT_DECL)
     ggc_internal_cleared_alloc_stat (s PASS_MEM_STAT);
 }
 
+/* gengtype will autogenerate traversal functions (in gtype-desc.c) for
+   all GTY-marked types that it sees are referenced by a GTY marker.
+
+   Unfortunately, it will not generate traveral functions for types that
+   are only referenced by GTY((user)) types.
+
+   The following templates are a substitute, providing equivalent
+   traversal functions for such types.  They are instantiated for
+   types whose objects that are traversed during GC/PCH, and are
+   called *every time* that an instance of type T is traversed during
+   GC/PCH.
+
+   They require the presence of the following member functions
+
+     void gt_ggc_mx ();
+     void gt_pch_nx ();
+     void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
+   within class T, which are called *once* per object - the first
+   time the object is visited during the traversal.  */
+
+template<class T>
+inline void gt_ggc_mx (T *p)
+{
+  if (ggc_test_and_set_mark (p))
+    p->gt_ggc_mx ();
+}
+
+template<class T>
+void gt_pch_nx_with_op (void *this_obj, void *p,
+			gt_pointer_operator op, void *cookie)
+{
+  if (p == this_obj)
+    {
+      T *t = static_cast<T *>(p);
+      t->gt_pch_nx_with_op (op, cookie);
+    }
+}
+
+template<class T>
+inline void gt_pch_nx (T *p)
+{
+  if (gt_pch_note_object (p, p, gt_pch_nx_with_op<T>))
+    p->gt_pch_nx ();
+}
+
 #endif
diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
index 41d2c76..a442bf1 100644
--- a/gcc/pass_manager.h
+++ b/gcc/pass_manager.h
@@ -44,13 +44,19 @@ namespace gcc {
 
 class context;
 
-class pass_manager
+class GTY((user)) pass_manager
 {
 public:
+  /* Ensure that instances are allocated in the GC-managed heap.  */
   void *operator new (size_t sz);
 
   pass_manager(context *ctxt);
 
+  /* GTY((user)) methods.  */
+  void gt_ggc_mx ();
+  void gt_pch_nx ();
+  void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
   void register_pass (struct register_pass_info *pass_info);
   void register_one_dump_file (struct opt_pass *pass);
 
@@ -134,4 +140,3 @@ private:
 } // namespace gcc
 
 #endif /* ! GCC_PASS_MANAGER_H */
-
diff --git a/gcc/passes.c b/gcc/passes.c
index e3a7212..3fa4393 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -82,6 +82,58 @@ struct opt_pass *current_pass;
 
 static void register_pass_name (struct opt_pass *, const char *);
 
+void*
+opt_pass::operator new (size_t sz)
+{
+  return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
+}
+
+void opt_pass::gt_ggc_mx ()
+{
+  ::gt_ggc_mx (ctxt_);
+  ::gt_ggc_mx (sub);
+
+  /* Avoid deep stack usage by iteratively walking the chain of "next"
+     passes, rather than recursing (analogous to the chain_next/chain_prev
+     GTY options).  */
+
+  /* "this" has already been marked.
+     Mark a chain of as-yet-unmarked passes.  */
+  opt_pass *limit = this->next;
+  while (ggc_test_and_set_mark (limit))
+    limit = limit->next;
+
+  /* "limit" is the first in the chain that wasn't just marked, either
+     because it is NULL, or because it was already marked.
+     Hence all of the passes in the half-open interval:
+	[this->next...limit)
+     have just been marked: visit them.  */
+  for (opt_pass *iter = this->next; iter != limit; iter = iter->next)
+    iter->gt_ggc_mx ();
+}
+
+void opt_pass::gt_pch_nx ()
+{
+  ::gt_pch_nx (ctxt_);
+  ::gt_pch_nx (sub);
+
+  /* Analogous to opt_pass::gt_ggc_mx.  */
+  opt_pass *limit = this->next;
+  while (gt_pch_note_object (limit, limit, ::gt_pch_nx_with_op<opt_pass>))
+    limit = limit->next;
+
+ for (opt_pass *iter = this->next; iter != limit; iter = iter->next)
+    iter->gt_pch_nx ();
+
+}
+
+void opt_pass::gt_pch_nx_with_op (gt_pointer_operator op, void *cookie)
+{
+  op (&(ctxt_), cookie);
+  op (&(sub), cookie);
+  op (&(next), cookie);
+}
+
 /* Most passes are single-instance (within their context) and thus don't
    need to implement cloning, but passes that support multiple instances
    *must* provide their own implementation of the clone method.
@@ -116,6 +168,92 @@ opt_pass::opt_pass(const pass_data &data, context *ctxt)
 {
 }
 
+void
+pass_manager::gt_ggc_mx ()
+{
+  /* We want to efficiently visit all pass objects without incurring deep
+     call chains that could blow the stack.
+
+     Although there are multiple fields referencing passes within the
+     pass_manager, *almost* all of the underlying pass instances are
+     referenced by the passes_by_id array.
+
+     Specifically, passes are in the passes_by_id array if they have
+     register_one_dump_file called on them, which requires them to have
+     a name, and for that name to *not* begin with "*".  Currently
+     there are 25 passes with a "*" prefix and thus not in the array.
+
+     Each pass holds references to its sub and next, so visiting a pass
+     will potentially trigger a recursive traversal through these
+     neighbours - if these passes haven't been visited yet.
+
+     By walking the passes_by_id array *backwards*, in the common case
+     this leads to us walking the pass tree from the leaf passes first,
+     eventually reaching the trunk passes, and hence none of the calls
+     should recurse, given that at each point in the iteration pass->sub
+     and pass->next will already have been marked.
+
+     Having walked the array, we then walk the higher-level fields,
+     again in bottom-up order, which will ensure that we visit all
+     remaining passes.  Most of the passes will have already been
+     visited, which should minimize further recursion.  */
+  for (int i = passes_by_id_size ; i > 0; )
+    ::gt_ggc_mx (passes_by_id[--i]);
+
+  ::gt_ggc_mx (all_late_ipa_passes);
+  ::gt_ggc_mx (all_lto_gen_passes);
+  ::gt_ggc_mx (all_regular_ipa_passes);
+  ::gt_ggc_mx (all_lowering_passes);
+  ::gt_ggc_mx (all_small_ipa_passes);
+  ::gt_ggc_mx (all_passes);
+}
+
+void
+pass_manager::gt_pch_nx ()
+{
+  /* Analogous to pass_manager::gt_ggc_mx */
+  for (int i = passes_by_id_size ; i > 0; )
+    ::gt_pch_nx (passes_by_id[--i]);
+
+  ::gt_pch_nx (all_late_ipa_passes);
+  ::gt_pch_nx (all_lto_gen_passes);
+  ::gt_pch_nx (all_regular_ipa_passes);
+  ::gt_pch_nx (all_lowering_passes);
+  ::gt_pch_nx (all_small_ipa_passes);
+  ::gt_pch_nx (all_passes);
+}
+
+void
+pass_manager::gt_pch_nx_with_op (gt_pointer_operator op, void *cookie)
+{
+  /* Unlike the _mx and _nx hooks, we must visit *every* field, since
+     they must each be reconstructed when reading the data back from
+     disk.  */
+  op (&(all_passes), cookie);
+  op (&(all_small_ipa_passes), cookie);
+  op (&(all_lowering_passes), cookie);
+  op (&(all_regular_ipa_passes), cookie);
+  op (&(all_lto_gen_passes), cookie);
+  op (&(all_late_ipa_passes), cookie);
+
+  for (int i = 0; i < passes_by_id_size; i++)
+    op (&(passes_by_id[i]), cookie);
+
+#define INSERT_PASSES_AFTER(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define POP_INSERT_PASSES()
+#define NEXT_PASS(PASS, NUM) op (&(PASS ## _ ## NUM), cookie);
+#define TERMINATE_PASS_LIST()
+
+#include "pass-instances.def"
+
+#undef INSERT_PASSES_AFTER
+#undef PUSH_INSERT_PASSES_WITHIN
+#undef POP_INSERT_PASSES
+#undef NEXT_PASS
+#undef TERMINATE_PASS_LIST
+
+}
 
 void
 pass_manager::execute_early_local_passes ()
@@ -1464,7 +1602,7 @@ void *
 pass_manager::operator new (size_t sz)
 {
   /* Ensure that all fields of the pass manager are zero-initialized.  */
-  return xcalloc (1, sz);
+  return ggc_internal_cleared_alloc_stat (sz MEM_STAT_INFO);
 }
 
 pass_manager::pass_manager (context *ctxt)
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 787a49b..b2182c5 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -76,11 +76,22 @@ namespace gcc
 
 /* An instance of a pass.  This is also "pass_data" to minimize the
    changes in existing code.  */
-class opt_pass : public pass_data
+class GTY((user)) opt_pass : public pass_data
 {
 public:
+  /* Ensure that instances are allocated in the GC-managed heap.  */
+  void *operator new (size_t sz);
+
   virtual ~opt_pass () { }
 
+  /* GTY((user)) methods, to be called once per traversal.
+     opt_pass subclasses with additional GC-managed data should override
+     these, chain up to the base class implementation, then walk their
+     extra fields.  */
+  virtual void gt_ggc_mx ();
+  virtual void gt_pch_nx ();
+  virtual void gt_pch_nx_with_op (gt_pointer_operator op, void *cookie);
+
   /* Create a copy of this pass.
 
      Passes that can have multiple instances must provide their own

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 09/11] Support "gcc" namespace in gengtype
  2013-08-01 21:01   ` Richard Henderson
@ 2013-08-16 19:53     ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-16 19:53 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Thu, 2013-08-01 at 11:01 -1000, Richard Henderson wrote:
> On 07/26/2013 05:04 AM, David Malcolm wrote:
> > +	     "/* Types with a \"gcc::\" prefix have the prefix stripped\n"
> > +	     "   during gengtype parsing.  Provide a \"using\" directive\n"
> > +	     "   to ensure that the fully-qualified types are found.  */\n"
> 
> I'd rather not use the word "prefix"; the term "namespace" is both more exact
> and less confusing in this context.
> 
> Otherwise ok.

Thanks; committed to trunk as r201800, having fixed up the wording, and
bootstrapped and tested.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: v3 of patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-16 15:33                 ` v3 of " David Malcolm
@ 2013-08-19 17:57                   ` Richard Henderson
  2013-08-20  0:59                     ` David Malcolm
  0 siblings, 1 reply; 74+ messages in thread
From: Richard Henderson @ 2013-08-19 17:57 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc-patches

On 08/16/2013 08:33 AM, David Malcolm wrote:
> I also tweaked the traversal hooks for opt_pass to emulate "chain_next",
> since this is where the really deep callchains could otherwise occur.
> 
> See the patch for details (given the subtleties I opted to put big
> comments in the relevant routines).
> 
> (I also fixed the typo "overide" to "override" as noted by Bernhard)
> 
> Successfully bootstrapped and tested on x86_64-unknown-linux-gnu, on top
> of the other patches (Note that the gengtype fix I committed just now as
> r201791 is also needed [1]).
> 
> OK for trunk?

Ok.


r~

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Updated patch 10 (was Re: [PATCH 10/11] Make gcc::context be GC-managed)
  2013-08-03 18:19         ` Richard Henderson
@ 2013-08-20  0:55           ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-20  0:55 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Sat, 2013-08-03 at 08:19 -1000, Richard Henderson wrote:
> On 08/02/2013 02:43 PM, David Malcolm wrote:
> > gcc/
> > 	* Makefile.in (GTFILES): Add context.h.
> > 	* context.c (gcc::context::operator new): New.
> > 	(gcc::context::gt_ggc_mx): New.
> > 	(gcc::context::gt_pch_nx): New.
> > 	(gcc::context::gt_pch_nx): New.
> > 	* context.h (gcc::context): Add GTY((user)) marking.
> > 	(gcc::context::operator new): New.
> > 	(gcc::context::gt_ggc_mx): New.
> > 	(gcc::context::gt_pch_nx): New.
> > 	(gcc::context::gt_pch_nx): New.
> > 	(g): Add GTY marking.
> > 	(gt_ggc_mx (gcc::context *)): New.
> > 	(gt_pch_nx (gcc::context *)): New.
> > 	(gt_pch_nx (gcc::context *ctxt, gt_pointer_operator op,
> > 	void *cookie)): New.
> > 	* gengtype.c (open_base_files) <ifiles>: Add context.h.
> 
> Ok.

Thanks; committed to trunk as r201864.


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: v3 of patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed)
  2013-08-19 17:57                   ` Richard Henderson
@ 2013-08-20  0:59                     ` David Malcolm
  0 siblings, 0 replies; 74+ messages in thread
From: David Malcolm @ 2013-08-20  0:59 UTC (permalink / raw)
  To: Richard Henderson; +Cc: gcc-patches

On Mon, 2013-08-19 at 10:52 -0700, Richard Henderson wrote:
> On 08/16/2013 08:33 AM, David Malcolm wrote:
> > I also tweaked the traversal hooks for opt_pass to emulate "chain_next",
> > since this is where the really deep callchains could otherwise occur.
> > 
> > See the patch for details (given the subtleties I opted to put big
> > comments in the relevant routines).
> > 
> > (I also fixed the typo "overide" to "override" as noted by Bernhard)
> > 
> > Successfully bootstrapped and tested on x86_64-unknown-linux-gnu, on top
> > of the other patches (Note that the gengtype fix I committed just now as
> > r201791 is also needed [1]).
> > 
> > OK for trunk?
> 
> Ok.

Thanks; committed to trunk as r201865.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed
  2013-08-03  0:21           ` Richard Henderson
@ 2013-08-27 10:07             ` Richard Biener
  0 siblings, 0 replies; 74+ messages in thread
From: Richard Biener @ 2013-08-27 10:07 UTC (permalink / raw)
  To: Richard Henderson; +Cc: David Malcolm, GCC Patches

On Sat, Aug 3, 2013 at 2:21 AM, Richard Henderson <rth@redhat.com> wrote:
> On 08/02/2013 11:53 AM, David Malcolm wrote:
>> FWIW I had a go at avoiding templates by attempting to tell gengtype to
>> write out functions for all GTY((user)) types, regardless of whether it
>> thinks they're referenced, with this:
>> @@ -3697,7 +3697,8 @@ write_types (outf_p output_header, type_p structures, type_p param_structs,
>>    /* At last we emit the functions code.  */
>>    oprintf (output_header, "\n/* functions code */\n");
>>    for (s = structures; s; s = s->next)
>> -    if (s->gc_used == GC_POINTED_TO || s->gc_used == GC_MAYBE_POINTED_TO)
>> +    if (s->gc_used == GC_POINTED_TO || s->gc_used == GC_MAYBE_POINTED_TO
>> +        || s->kind == TYPE_USER_STRUCT)
>>        {
>>         options_p opt;
>>
>> but I ran into various "incomplete structure" errors due to gengtype's
>> lack of a full C/C++ parser.
>>
>> Hence templates seem the sanest option.
>
> I was actually contemplating going the other direction -- have gengtype
> write out _less_, relying on the templates more.  I hadn't gone very far
> down that road yet...

That was the original idea, btw (well, the original idea was to get rid of
gengtype completely of course).

Btw, I don't like moving the pass pipeline and pass objects into GC space.
It doesn't make much sense as pass execution is quite constrained so
passes should not allocate objects that extend the passes lifetime.

Richard.

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Rewrite usage comment at the top of 'gcc/passes.def' (was: [PATCH 02/11] Generate pass-instances.def)
  2013-07-26 15:46 ` David Malcolm
  2013-07-29 20:03   ` Jeff Law
@ 2024-06-28 12:13   ` Thomas Schwinge
  2024-06-28 12:57     ` Richard Biener
  1 sibling, 1 reply; 74+ messages in thread
From: Thomas Schwinge @ 2024-06-28 12:13 UTC (permalink / raw)
  To: David Malcolm, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

Hi!

On 2013-07-26T11:04:32-0400, David Malcolm <dmalcolm@redhat.com> wrote:
> Introduce a new gen-pass-instances.awk script, and use it at build time
> to make a pass-instances.def from passes.def.

(The script has later been rewritten and extended, but the issue I'm
discussing is relevant already in its original version.)

> The generated pass-instances.def contains similar content to passes.def,
> but the pass instances within it are explicitly numbered, so that e.g.
> the third instance of:
>
>   NEXT_PASS (pass_copy_prop)
>
> becomes:
>
>   NEXT_PASS (pass_copy_prop, 3)

> --- a/gcc/passes.c
> +++ b/gcc/passes.c
> @@ -1315,12 +1315,12 @@ pipeline::pipeline (context *ctxt)
>  #define POP_INSERT_PASSES() \
>    }
>  
> -#define NEXT_PASS(PASS)  (p = next_pass_1 (p, &((PASS).pass)))
> +#define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
>  
>  #define TERMINATE_PASS_LIST() \
>    *p = NULL;
>  
> -#include "passes.def"
> +#include "pass-instances.def"

Given this, the usage comment at the top of 'gcc/passes.def' (see below)
no longer is accurate (even if that latter file does continue to use the
'NEXT_PASS' form without 'NUM') -- and, worse, the 'NEXT_PASS' etc. in
that usage comment are processed by the 'gcc/gen-pass-instances.awk'
script:

    --- source-gcc/gcc/passes.def   2024-06-24 18:55:15.132561641 +0200
    +++ build-gcc/gcc/pass-instances.def    2024-06-24 18:55:27.768562714 +0200
    [...]
    @@ -20,546 +22,578 @@
     /*
      Macros that should be defined when using this file:
        INSERT_PASSES_AFTER (PASS)
        PUSH_INSERT_PASSES_WITHIN (PASS)
        POP_INSERT_PASSES ()
    -   NEXT_PASS (PASS)
    +   NEXT_PASS (PASS, 1)
        TERMINATE_PASS_LIST (PASS)
      */
    [...]

(That is, this is 'NEXT_PASS' for the first instance of pass 'PASS'.)
That's benign so far, but with another thing that I'll be extending, I'd
then run into an error while the script handles this comment block.  ;-\

OK to push "Rewrite usage comment at the top of 'gcc/passes.def'", see
attached?


Grüße
 Thomas



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Rewrite-usage-comment-at-the-top-of-gcc-passes.def.patch --]
[-- Type: text/x-diff, Size: 2343 bytes --]

From 072cdf7d9cf86fb2b0553b93365648e153b4376b Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <tschwinge@baylibre.com>
Date: Fri, 28 Jun 2024 14:05:04 +0200
Subject: [PATCH] Rewrite usage comment at the top of 'gcc/passes.def'

Since Subversion r201359 (Git commit a167b052dfe9a8509bb23c374ffaeee953df0917)
"Introduce gen-pass-instances.awk and pass-instances.def", the usage comment at
the top of 'gcc/passes.def' no longer is accurate (even if that latter file
does continue to use the 'NEXT_PASS' form without 'NUM') -- and, worse, the
'NEXT_PASS' etc. in that usage comment are processed by the
'gcc/gen-pass-instances.awk' script:

    --- source-gcc/gcc/passes.def   2024-06-24 18:55:15.132561641 +0200
    +++ build-gcc/gcc/pass-instances.def    2024-06-24 18:55:27.768562714 +0200
    [...]
    @@ -20,546 +22,578 @@
     /*
      Macros that should be defined when using this file:
        INSERT_PASSES_AFTER (PASS)
        PUSH_INSERT_PASSES_WITHIN (PASS)
        POP_INSERT_PASSES ()
    -   NEXT_PASS (PASS)
    +   NEXT_PASS (PASS, 1)
        TERMINATE_PASS_LIST (PASS)
      */
    [...]

(That is, this is 'NEXT_PASS' for the first instance of pass 'PASS'.)
That's benign so far, but with another thing that I'll be extending, I'd
then run into an error while the script handles this comment block.  ;-\

	gcc/
	* passes.def: Rewrite usage comment at the top.
---
 gcc/passes.def | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/gcc/passes.def b/gcc/passes.def
index 1f222729d39..3f65fcf71d6 100644
--- a/gcc/passes.def
+++ b/gcc/passes.def
@@ -17,14 +17,11 @@ You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */
 
-/*
- Macros that should be defined when using this file:
-   INSERT_PASSES_AFTER (PASS)
-   PUSH_INSERT_PASSES_WITHIN (PASS)
-   POP_INSERT_PASSES ()
-   NEXT_PASS (PASS)
-   TERMINATE_PASS_LIST (PASS)
- */
+/* Note that this file is processed by a simple parser:
+   'gen-pass-instances.awk', so carefully verify the generated
+   'pass-instances.def' if you deviate from the syntax otherwise used in
+   here.  */
+
 
  /* All passes needed to lower the function into shape optimizers can
     operate on.  These passes are always run first on the function, but
-- 
2.34.1


^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: Rewrite usage comment at the top of 'gcc/passes.def' (was: [PATCH 02/11] Generate pass-instances.def)
  2024-06-28 12:13   ` Rewrite usage comment at the top of 'gcc/passes.def' " Thomas Schwinge
@ 2024-06-28 12:57     ` Richard Biener
  0 siblings, 0 replies; 74+ messages in thread
From: Richard Biener @ 2024-06-28 12:57 UTC (permalink / raw)
  To: Thomas Schwinge; +Cc: David Malcolm, gcc-patches

On Fri, Jun 28, 2024 at 2:14 PM Thomas Schwinge <tschwinge@baylibre.com> wrote:
>
> Hi!
>
> On 2013-07-26T11:04:32-0400, David Malcolm <dmalcolm@redhat.com> wrote:
> > Introduce a new gen-pass-instances.awk script, and use it at build time
> > to make a pass-instances.def from passes.def.
>
> (The script has later been rewritten and extended, but the issue I'm
> discussing is relevant already in its original version.)
>
> > The generated pass-instances.def contains similar content to passes.def,
> > but the pass instances within it are explicitly numbered, so that e.g.
> > the third instance of:
> >
> >   NEXT_PASS (pass_copy_prop)
> >
> > becomes:
> >
> >   NEXT_PASS (pass_copy_prop, 3)
>
> > --- a/gcc/passes.c
> > +++ b/gcc/passes.c
> > @@ -1315,12 +1315,12 @@ pipeline::pipeline (context *ctxt)
> >  #define POP_INSERT_PASSES() \
> >    }
> >
> > -#define NEXT_PASS(PASS)  (p = next_pass_1 (p, &((PASS).pass)))
> > +#define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
> >
> >  #define TERMINATE_PASS_LIST() \
> >    *p = NULL;
> >
> > -#include "passes.def"
> > +#include "pass-instances.def"
>
> Given this, the usage comment at the top of 'gcc/passes.def' (see below)
> no longer is accurate (even if that latter file does continue to use the
> 'NEXT_PASS' form without 'NUM') -- and, worse, the 'NEXT_PASS' etc. in
> that usage comment are processed by the 'gcc/gen-pass-instances.awk'
> script:
>
>     --- source-gcc/gcc/passes.def   2024-06-24 18:55:15.132561641 +0200
>     +++ build-gcc/gcc/pass-instances.def    2024-06-24 18:55:27.768562714 +0200
>     [...]
>     @@ -20,546 +22,578 @@
>      /*
>       Macros that should be defined when using this file:
>         INSERT_PASSES_AFTER (PASS)
>         PUSH_INSERT_PASSES_WITHIN (PASS)
>         POP_INSERT_PASSES ()
>     -   NEXT_PASS (PASS)
>     +   NEXT_PASS (PASS, 1)
>         TERMINATE_PASS_LIST (PASS)
>       */
>     [...]
>
> (That is, this is 'NEXT_PASS' for the first instance of pass 'PASS'.)
> That's benign so far, but with another thing that I'll be extending, I'd
> then run into an error while the script handles this comment block.  ;-\
>
> OK to push "Rewrite usage comment at the top of 'gcc/passes.def'", see
> attached?

OK

>
> Grüße
>  Thomas
>
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Handle 'NUM' in 'PUSH_INSERT_PASSES_WITHIN' (was: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)
  2013-07-26 15:05 ` [PATCH 03/11] Handwritten part of conversion of passes to C++ classes David Malcolm
  2013-07-28  9:12   ` Basile Starynkevitch
  2013-07-29 22:39   ` Jeff Law
@ 2024-06-28 13:06   ` Thomas Schwinge
  2 siblings, 0 replies; 74+ messages in thread
From: Thomas Schwinge @ 2024-06-28 13:06 UTC (permalink / raw)
  To: David Malcolm, gcc-patches

[-- Attachment #1: Type: text/plain, Size: 3401 bytes --]

Hi!

As part of this:

On 2013-07-26T11:04:33-0400, David Malcolm <dmalcolm@redhat.com> wrote:
> This patch is the hand-written part of the conversion of passes from
> C structs to C++ classes.

> --- a/gcc/passes.c
> +++ b/gcc/passes.c

..., we did hard-code 'PUSH_INSERT_PASSES_WITHIN(PASS)' to always refer
to the first instance of 'PASS':

>  #define PUSH_INSERT_PASSES_WITHIN(PASS) \
>    { \
> -    struct opt_pass **p = &(PASS).pass.sub;
> +    struct opt_pass **p = &(PASS ## _1)->sub;

..., however we did change 'NEXT_PASS(PASS, NUM)' to actually use 'NUM':

> -#define NEXT_PASS(PASS, NUM)  (p = next_pass_1 (p, &((PASS).pass)))
> +#define NEXT_PASS(PASS, NUM) \
> +  do { \
> +    gcc_assert (NULL == PASS ## _ ## NUM); \
> +    if ((NUM) == 1)                              \
> +      PASS ## _1 = make_##PASS (ctxt_);          \
> +    else                                         \
> +      {                                          \
> +        gcc_assert (PASS ## _1);                 \
> +        PASS ## _ ## NUM = PASS ## _1->clone (); \
> +      }                                          \
> +    p = next_pass_1 (p, PASS ## _ ## NUM);  \
> +  } while (0)

This was never re-synchronized later on, and is problematic if you try to
do something like this; change:

    [...]
    NEXT_PASS (pass_postreload);
    PUSH_INSERT_PASSES_WITHIN (pass_postreload)
        NEXT_PASS (pass_postreload_cse);
        [...]
        NEXT_PASS (pass_cprop_hardreg);
        NEXT_PASS (pass_fast_rtl_dce);
        NEXT_PASS (pass_reorder_blocks);
        [...]
    POP_INSERT_PASSES ()
    [...]

... into:

    [...]
    NEXT_PASS (pass_postreload);
    PUSH_INSERT_PASSES_WITHIN (pass_postreload)
        NEXT_PASS (pass_postreload_cse);
        [...]
        NEXT_PASS (pass_cprop_hardreg);
    POP_INSERT_PASSES ()
    NEXT_PASS (pass_fast_rtl_dce);
    NEXT_PASS (pass_postreload);
    PUSH_INSERT_PASSES_WITHIN (pass_postreload)
        NEXT_PASS (pass_reorder_blocks);
        [...]
    POP_INSERT_PASSES ()
    [...]

That is, interrupt the pass pipeline within 'pass_postreload', in order
to unconditionally run 'pass_fast_rtl_dce' even if not running
'pass_postreload'.  What happens is that the second
'PUSH_INSERT_PASSES_WITHIN (pass_postreload)' overwrites the first
'PUSH_INSERT_PASSES_WITHIN (pass_postreload)' instead of applying to the
second (preceding) 'NEXT_PASS (pass_postreload);'.

(I ran into this in context of what I tried in
<https://inbox.sourceware.org/87ed8i2ekt.fsf@euler.schwinge.ddns.net>
"nvptx vs. [PATCH] Add a late-combine pass [PR106594]"; discuss that
specific use case over there, not here.)

OK to address this with the attached
"Handle 'NUM' in 'PUSH_INSERT_PASSES_WITHIN'"?

This depends on
<https://inbox.sourceware.org/87jzi9tgcw.fsf@euler.schwinge.ddns.net>
"Rewrite usage comment at the top of 'gcc/passes.def'" to avoid running
into the 'ERROR: Can't locate [...]' that I'm adding, while processing
the 'PUSH_INSERT_PASSES_WITHIN (PASS)' in the usage comment at the top of
'gcc/passes.def', where 'NEXT_PASS (PASS)' only appears later.  ;-)

I've verified this does the expected thing for the main 'gcc/passes.def',
and that 'PUSH_INSERT_PASSES_WITHIN' is not used/not applicable for
'PASSES_EXTRA' ('gcc/config/*/*-passes.def').


Grüße
 Thomas



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Handle-NUM-in-PUSH_INSERT_PASSES_WITHIN.patch --]
[-- Type: text/x-diff, Size: 3302 bytes --]

From e368ccba93f5bbaee882076c80849adb55a68fa0 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <tschwinge@baylibre.com>
Date: Fri, 28 Jun 2024 12:10:12 +0200
Subject: [PATCH] Handle 'NUM' in 'PUSH_INSERT_PASSES_WITHIN'

..., such that also for repeated 'NEXT_PASS', 'PUSH_INSERT_PASSES_WITHIN' for a
given 'PASS', the 'PUSH_INSERT_PASSES_WITHIN' applies to the preceeding
'NEXT_PASS', and not unconditionally applies to the first 'NEXT_PASS'.

	gcc/
	* gen-pass-instances.awk: Handle 'PUSH_INSERT_PASSES_WITHIN'.
	* pass_manager.h (PUSH_INSERT_PASSES_WITHIN): Adjust.
	* passes.cc (PUSH_INSERT_PASSES_WITHIN): Likewise.
---
 gcc/gen-pass-instances.awk | 28 +++++++++++++++++++++++++---
 gcc/pass_manager.h         |  2 +-
 gcc/passes.cc              |  6 +++---
 3 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/gcc/gen-pass-instances.awk b/gcc/gen-pass-instances.awk
index 449889663f7..871ac0cdb52 100644
--- a/gcc/gen-pass-instances.awk
+++ b/gcc/gen-pass-instances.awk
@@ -16,7 +16,7 @@
 
 # This Awk script takes passes.def and writes pass-instances.def,
 # counting the instances of each kind of pass, adding an instance number
-# to everywhere that NEXT_PASS is used.
+# to everywhere that NEXT_PASS or PUSH_INSERT_PASSES_WITHIN are used.
 # Also handle INSERT_PASS_AFTER, INSERT_PASS_BEFORE and REPLACE_PASS
 # directives.
 #
@@ -222,9 +222,31 @@ END {
 	  if (with_arg)
 	    printf ",%s", with_arg;
 	  printf ")%s\n", postfix;
+
+	  continue;
 	}
-      else
-	print lines[i];
+
+      ret = parse_line(lines[i], "PUSH_INSERT_PASSES_WITHIN");
+      if (ret)
+	{
+	  pass_name = args[1];
+
+	  pass_num = pass_final_counts[pass_name];
+	  if (!pass_num)
+	    {
+	      print "ERROR: Can't locate instance of the pass mentioned in " pass_name;
+	      exit 1;
+	    }
+
+	  printf "%s", prefix;
+	  printf "PUSH_INSERT_PASSES_WITHIN";
+	  printf " (%s, %s", pass_name, pass_num;
+	  printf ")%s\n", postfix;
+
+	  continue;
+	}
+
+      print lines[i];
     }
 }
 
diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
index be324d5dff7..edd775e9a9c 100644
--- a/gcc/pass_manager.h
+++ b/gcc/pass_manager.h
@@ -126,7 +126,7 @@ private:
         opt_pass *pass_copy_prop_8;  */
 
 #define INSERT_PASSES_AFTER(PASS)
-#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS, NUM)
 #define POP_INSERT_PASSES()
 #define NEXT_PASS(PASS, NUM) opt_pass *PASS ## _ ## NUM
 #define NEXT_PASS_WITH_ARG(PASS, NUM, ARG) NEXT_PASS (PASS, NUM)
diff --git a/gcc/passes.cc b/gcc/passes.cc
index b01a79ef96c..023631046a6 100644
--- a/gcc/passes.cc
+++ b/gcc/passes.cc
@@ -1574,7 +1574,7 @@ pass_manager::pass_manager (context *ctxt)
 
   /* Zero-initialize pass members.  */
 #define INSERT_PASSES_AFTER(PASS)
-#define PUSH_INSERT_PASSES_WITHIN(PASS)
+#define PUSH_INSERT_PASSES_WITHIN(PASS, NUM)
 #define POP_INSERT_PASSES()
 #define NEXT_PASS(PASS, NUM) PASS ## _ ## NUM = NULL
 #define NEXT_PASS_WITH_ARG(PASS, NUM, ARG) NEXT_PASS (PASS, NUM)
@@ -1604,9 +1604,9 @@ pass_manager::pass_manager (context *ctxt)
     *p = NULL;					\
   }
 
-#define PUSH_INSERT_PASSES_WITHIN(PASS) \
+#define PUSH_INSERT_PASSES_WITHIN(PASS, NUM) \
   { \
-    opt_pass **p = &(PASS ## _1)->sub;
+    opt_pass **p = &(PASS ## _ ## NUM)->sub;
 
 #define POP_INSERT_PASSES() \
   }
-- 
2.34.1


^ permalink raw reply	[flat|nested] 74+ messages in thread

end of thread, other threads:[~2024-06-28 13:06 UTC | newest]

Thread overview: 74+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-26 15:05 [PATCH 00/11] Rewrite of pass management David Malcolm
2013-07-26 15:05 ` [PATCH 01/11] Introduce beginnings of a pipeline class David Malcolm
2013-07-29 20:00   ` Jeff Law
2013-07-30 19:02     ` [Committed] Introduce beginnings of gcc::pass_manager class (was Re: [PATCH 01/11] Introduce beginnings of a pipeline class.) David Malcolm
2013-07-26 15:05 ` [PATCH 04/11] Automated conversion of passes to C++ classes David Malcolm
2013-07-26 15:05 ` [PATCH 10/11] Make gcc::context be GC-managed David Malcolm
2013-08-01 21:28   ` Richard Henderson
2013-08-02 18:31     ` David Malcolm
2013-08-03  0:43       ` Updated patch 10 (was Re: [PATCH 10/11] Make gcc::context be GC-managed) David Malcolm
2013-08-03 18:19         ` Richard Henderson
2013-08-20  0:55           ` David Malcolm
2013-07-26 15:05 ` [PATCH 08/11] Example of converting global state to per-pass state David Malcolm
2013-08-01 20:58   ` Richard Henderson
2013-08-13  0:46     ` David Malcolm
2013-07-26 15:05 ` [PATCH 05/11] Add -fno-rtti when building plugins David Malcolm
2013-07-29 20:24   ` Jeff Law
2013-07-26 15:05 ` [PATCH 06/11] Rewrite how instances of passes are cloned David Malcolm
2013-08-01 17:55   ` David Malcolm
2013-08-01 18:11     ` David Malcolm
2013-08-01 20:56     ` Richard Henderson
2013-07-26 15:05 ` [PATCH 07/11] Introduce virtual functions in testsuite/gcc.dg/plugin/one_time_plugin.c David Malcolm
2013-08-01 20:48   ` Richard Henderson
2013-08-13  0:42     ` David Malcolm
2013-07-26 15:05 ` [PATCH 03/11] Handwritten part of conversion of passes to C++ classes David Malcolm
2013-07-28  9:12   ` Basile Starynkevitch
2013-07-29 15:43     ` David Malcolm
2013-07-29 22:39   ` Jeff Law
2013-08-01 17:13     ` [PATCH 3.1/11] Explicitly initialize the macro-generated pass fields (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) David Malcolm
2013-08-03  0:39       ` David Malcolm
2013-08-03 17:59         ` Richard Henderson
2013-08-05 21:03     ` Passes are now C++ classes " David Malcolm
2013-08-06 15:16       ` Steve Ellcey
2013-08-06 17:55         ` [Patch] Unbreak build on mips (was Re: Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes)) David Malcolm
2013-08-06 18:12           ` Richard Sandiford
2013-08-06 18:52             ` David Malcolm
2013-08-07 13:10         ` Passes are now C++ classes (was Re: [PATCH 03/11] Handwritten part of conversion of passes to C++ classes) Eric Botcazou
2013-08-06 19:12       ` Basile Starynkevitch
2024-06-28 13:06   ` Handle 'NUM' in 'PUSH_INSERT_PASSES_WITHIN' (was: " Thomas Schwinge
2013-07-26 15:05 ` [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed David Malcolm
2013-07-27 18:45   ` Bernhard Reutner-Fischer
2013-08-01 21:45   ` Richard Henderson
2013-08-02 19:08     ` David Malcolm
2013-08-02 20:01       ` Richard Henderson
2013-08-02 21:53         ` David Malcolm
2013-08-03  0:21           ` Richard Henderson
2013-08-27 10:07             ` Richard Biener
2013-08-03  0:48         ` Updated patch (was Re: [PATCH 11/11] Make opt_pass and gcc::pipeline be GC-managed) David Malcolm
2013-08-03 18:39           ` Richard Henderson
2013-08-05 15:18             ` David Malcolm
2013-08-05 16:59               ` Richard Henderson
2013-08-05 17:28                 ` David Malcolm
2013-08-16 15:33                 ` v3 of " David Malcolm
2013-08-19 17:57                   ` Richard Henderson
2013-08-20  0:59                     ` David Malcolm
2013-07-26 15:29 ` [PATCH 09/11] Support "gcc" namespace in gengtype David Malcolm
2013-08-01 21:01   ` Richard Henderson
2013-08-16 19:53     ` David Malcolm
2013-07-26 15:33 ` [PATCH 02/11] Generate pass-instances.def David Malcolm
2013-07-26 15:46 ` David Malcolm
2013-07-29 20:03   ` Jeff Law
2013-07-31  3:24     ` David Malcolm
2013-07-31  8:53       ` Build broken (was: [PATCH 02/11] Generate pass-instances.def) Jan-Benedict Glaw
2013-07-31  9:37         ` Jan-Benedict Glaw
2013-07-31 10:06         ` Jan-Benedict Glaw
2013-07-31 15:58           ` David Malcolm
2013-07-31 16:18             ` Jan-Benedict Glaw
2013-07-31 16:26               ` David Malcolm
2013-07-31 16:50                 ` Jan-Benedict Glaw
2024-06-28 12:13   ` Rewrite usage comment at the top of 'gcc/passes.def' " Thomas Schwinge
2024-06-28 12:57     ` Richard Biener
2013-07-29 11:10 ` [PATCH 00/11] Rewrite of pass management Richard Earnshaw
2013-07-29 15:32   ` David Malcolm
2013-07-29 16:33     ` Mike Stump
2013-07-29 16:37     ` Diego Novillo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).