[PATCH][LTO] Fix PR41669, avoid recursing in get_alias_set

[PATCH][LTO] Fix PR41669, avoid recursing in get_alias_set

[Richard Guenther]

This avoids the endless recursions we see from time to time between
get_alias_set and the gimple langhook implementation.  It happens
when TYPE_CANONICAL and TYPE_MAIN_VARIANT together with
TYPE_POINTER_TO/TYPE_REFERENCE_TO build a cycle.  While this is
surely a problem with type-merging it's exposed easily by incrementally
changing that code for good.

This patch papers over the problem - I will keep one bug open to
track this.

Bootstrap and regtest running - ok if that succeeds?

Thanks,
Richard.

2009-10-15  Richard Guenther  <rguenther@suse.de>

	PR lto/41669
	* gimple.c (gimple_get_alias_set): Avoid recursing on
	invalid type topology.

	* gcc.dg/lto/20091015-1_0.c: New testcase.
	* gcc.dg/lto/20091015-1_1.c: Likewise.
	* gcc.dg/lto/20091015-1_2.c: Likewise.
	* gcc.dg/lto/20091015-1_a.h: Likewise.
	* gcc.dg/lto/20091015-1_b.h: Likewise.

Index: gcc/testsuite/gcc.dg/lto/20091015-1_0.c
===================================================================
*** gcc/testsuite/gcc.dg/lto/20091015-1_0.c	(revision 0)
--- gcc/testsuite/gcc.dg/lto/20091015-1_0.c	(revision 0)
***************
*** 0 ****
--- 1,5 ----
+ /* { dg-lto-do link } */
+ /* { dg-lto-options {{-fPIC -shared -O2 -flto} {-fPIC -shared -O2 -fwhopr}} } */
+ 
+ #include "20091015-1_b.h"
+ void diagnostic_initialize (FILE **stream) { *stream = stderr; }
Index: gcc/testsuite/gcc.dg/lto/20091015-1_1.c
===================================================================
*** gcc/testsuite/gcc.dg/lto/20091015-1_1.c	(revision 0)
--- gcc/testsuite/gcc.dg/lto/20091015-1_1.c	(revision 0)
***************
*** 0 ****
--- 1,4 ----
+ #include "20091015-1_a.h"
+ #include "20091015-1_b.h"
+ void ggc_print_common_statistics (FILE *stream) {
+ }
Index: gcc/testsuite/gcc.dg/lto/20091015-1_2.c
===================================================================
*** gcc/testsuite/gcc.dg/lto/20091015-1_2.c	(revision 0)
--- gcc/testsuite/gcc.dg/lto/20091015-1_2.c	(revision 0)
***************
*** 0 ****
--- 1,5 ----
+ #include "20091015-1_a.h"
+ #include "20091015-1_b.h"
+ void debug_optab_libfuncs (void) {
+ foo (stderr,       4         );
+ }
Index: gcc/testsuite/gcc.dg/lto/20091015-1_a.h
===================================================================
*** gcc/testsuite/gcc.dg/lto/20091015-1_a.h	(revision 0)
--- gcc/testsuite/gcc.dg/lto/20091015-1_a.h	(revision 0)
***************
*** 0 ****
--- 1,2 ----
+ struct _IO_FILE { int _flags;
+ };
Index: gcc/testsuite/gcc.dg/lto/20091015-1_b.h
===================================================================
*** gcc/testsuite/gcc.dg/lto/20091015-1_b.h	(revision 0)
--- gcc/testsuite/gcc.dg/lto/20091015-1_b.h	(revision 0)
***************
*** 0 ****
--- 1,2 ----
+ typedef struct _IO_FILE FILE;
+ extern struct _IO_FILE *stderr;
Index: gcc/gimple.c
===================================================================
--- gcc/gimple.c.orig	2009-10-15 15:24:59.000000000 +0200
+++ gcc/gimple.c	2009-10-15 15:19:11.000000000 +0200
@@ -4121,6 +4121,7 @@ gimple_signed_type (tree type)
 alias_set_type
 gimple_get_alias_set (tree t)
 {
+  static bool recursing_p;
   tree u;
 
   /* Permit type-punning when accessing a union, provided the access
@@ -4162,6 +4163,12 @@ gimple_get_alias_set (tree t)
     {
       tree t1;
 
+      /* ???  We can end up creating cycles with TYPE_MAIN_VARIANT
+	 and TYPE_CANONICAL.  Avoid recursing endlessly between
+	 this langhook and get_alias_set.  */
+      if (recursing_p)
+	return -1;
+
       /* Unfortunately, there is no canonical form of a pointer type.
 	 In particular, if we have `typedef int I', then `int *', and
 	 `I *' are different types.  So, we have to pick a canonical
@@ -4186,7 +4193,13 @@ gimple_get_alias_set (tree t)
 	 C++ committee.  */
       t1 = build_type_no_quals (t);
       if (t1 != t)
-	return get_alias_set (t1);
+	{
+	  alias_set_type set;
+	  recursing_p = true;
+	  set = get_alias_set (t1);
+	  recursing_p = false;
+	  return set;
+	}
     }
 
   return -1;

Re: [PATCH][LTO] Fix PR41669, avoid recursing in get_alias_set

[Diego Novillo]

On Thu, Oct 15, 2009 at 09:29, Richard Guenther <rguenther@suse.de> wrote:

> 2009-10-15  Richard Guenther  <rguenther@suse.de>
>
>        PR lto/41669
>        * gimple.c (gimple_get_alias_set): Avoid recursing on
>        invalid type topology.
>
>        * gcc.dg/lto/20091015-1_0.c: New testcase.
>        * gcc.dg/lto/20091015-1_1.c: Likewise.
>        * gcc.dg/lto/20091015-1_2.c: Likewise.
>        * gcc.dg/lto/20091015-1_a.h: Likewise.
>        * gcc.dg/lto/20091015-1_b.h: Likewise.

Ugh, gross.  But OK.


Diego.