[Bug c++/12252] New: ICE on very large source file : GC problem?

[Bug c++/12252] New: ICE on very large source file : GC problem?

[warren_baird at cimmetry dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252

           Summary: ICE on very large source file : GC problem?
           Product: gcc
           Version: 3.3.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: c++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: warren_baird at cimmetry dot com
                CC: gcc-bugs at gcc dot gnu dot org

I've come across an internal compiler error in gcc-3.3.1.  We upgraded
to gcc-3.3.1 a few weeks ago, and I'm getting internal compiler errors
in a number of places.  I've done some experiments, and the only thing
I can determine is that the ICE seems to be related in some way to the
length of the preprocessed file.  I can make the ICE go away by
removing about 15 lines of code from the end of a huge file, and then
make it appear again by adding 15 completely different lines of code
back into the file.

I haven't been able to create a file that exhibits this behaviour and
doesn't also contain a lot of proprietary information, but I've
collected enough information that I'm going to open a bug report
anyways.  

More details follow:

I started with a preprocessed version of the file generating the ICE,
and started trying to chop things out to find a minimal file that
repros the problem.  I soon had 3 variants of the file, called tmp1.ii,
tmp2.ii and tmp3.ii.

  - tmp1.ii was the smallest file I found that generated the ICE (It's
  about 110,000 lines long)

  - tmp2.ii was a copy of tmp1.ii with the last struct in the file
  removed --- it was a 15 line long struct from freetype.  tmp2.ii
  compiles without any errors

  - tmp3.ii was a copy of tmp2.ii with 5 copies of a simple 1 line
  function added where the freetype struct was.  It also generates an
  ICE.

Here are the diffs between the files:

   diff -u tmp1.ii tmp2.ii
--- tmp1.ii     Thu Sep  4 14:29:41 2003
+++ tmp2.ii     Thu Sep  4 14:34:04 2003
@@ -110740,21 +110740,6 @@
 # 1 "src/support/freetype/include/freetype/t1tables.h" 1
 # 28 "src/support/freetype/include/freetype/t1tables.h"
 extern "C" {
-# 63 "src/support/freetype/include/freetype/t1tables.h"
-  typedef struct PS_FontInfoRec
-  {
-    FT_String* version;
-    FT_String* notice;
-    FT_String* full_name;
-    FT_String* family_name;
-    FT_String* weight;
-    FT_Long italic_angle;
-    FT_Bool is_fixed_pitch;
-    FT_Short underline_position;
-    FT_UShort underline_thickness;
-
-  } PS_FontInfoRec, *PS_FontInfo;
-# 88 "src/support/freetype/include/freetype/t1tables.h"
 
--- tmp2.ii     Thu Sep  4 14:34:04 2003
+++ tmp3.ii     Thu Sep  4 14:36:01 2003
@@ -110741,6 +110741,24 @@
 # 28 "src/support/freetype/include/freetype/t1tables.h"
 extern "C" {
 
+       void myFunc1(int a) {
+          float b = 5;
+}
 
+       void myFunc2(int a) {
+          float b = 5;
+}
 
+       void myFunc3(int a) {
+          float b = 5;
+}
+
+       void myFunc4(int a) {
+          float b = 5;
+}
+
+
+       void myFunc5(int a) {
+          float b = 5;
+}
 }

Other bits of info: I'm doing this on a solaris 8 machine: "gcc -c -v
-g -O2 tmp1.ii" says:

Reading specs from /usr/local/gcc-3.3.1/lib/gcc-lib/sparc-sun-solaris2.8/3.3.1/specs
Configured with: ../gcc-3.3.1/configure --prefix=/usr/local/gcc-3.3.1
--with-gnu-as --with-gnu-ld --with-gnu-nm
--with-as=/usr/local/binutils-2.13.2.1/bin/as
--with-ld=/usr/local/binutils-2.13.2.1/bin/ld
--with-nm=/usr/local/binutils-2.13.2.1/bin/nm
Thread model: posix
gcc version 3.3.1
 /usr/local/gcc-3.3.1/lib/gcc-lib/sparc-sun-solaris2.8/3.3.1/cc1plus
-fpreprocessed tmp1.ii -quiet -dumpbase tmp1.ii -auxbase tmp1 -g -O2 -version -o
/var/tmp//ccTZyWKm.s
GNU C++ version 3.3.1 (sparc-sun-solaris2.8)
        compiled by GNU C version 3.3.1.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
[ various compiler warnings removed ] 
/usr/local/gcc-3.3.1/include/c++/3.3.1/bits/basic_string.tcc:288: internal
compiler error: Segmentation
   Fault
Please submit a full bug report,
with preprocessed source if appropriate.
See <URL:http://gcc.gnu.org/bugs.html> for instructions.

If I just use -g, or I just use -O2, it compiles fine --- it only
fails if I use -g -O2.  If I compile the same code with gcc-3.2.2, it
works fine.  Also, the original code compiles fine on linux with
gcc-3.3.1.

The problem doesn't seem to be related to stack size --- I tried both
setting the stacksize limit much larger (128MB instead of the default
8MB), and unlimiting it, and it didn't make any difference...

On Eric Botcazou's advice, I tried increasing ggc-min-expand to 1000
nd ggc-min-heapsize to 1310720 and the code compiled fine, so it seems
to be GC related...

I've done some experimentation with gdb, partially guided by Eric ---
Here's a partial transcript of the gdb session.  I have a full
traceback, but it's over 2000 lines long, so I'll attach it rather
than include it here...

gdb /usr/local/gcc-3.3.1/lib/gcc-lib/sparc-sun-solaris2.8/3.3.1/cc1plus
GNU gdb 5.3.0.90_20030306
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "sparc-sun-solaris2.8"...
(gdb)  run  -fpreprocessed tmp1.ii -quiet -dumpbase tmp1.ii -auxbase tmp1 -g -O2
-version -o /var/tmp//ccnRR2Dg.s
Starting program:
/home/local/gcc-3.3.1/lib/gcc-lib/sparc-sun-solaris2.8/3.3.1/cc1plus
-fpreprocessed tmp1.ii -quiet -dumpbase tmp1.ii -auxbase tmp1 -g -O2 -version -o
/var/tmp//ccnRR2Dg.s
GNU C++ version 3.3.1 (sparc-sun-solaris2.8)
	compiled by GNU C version 3.3.1.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
[ various warnings removed ...]
Program received signal SIGSEGV, Segmentation fault.
0x00254f58 in ggc_set_mark (p=0xea7b70f8) at ../../gcc-3.3.1/gcc/ggc-page.c:525
525	  L2 = LOOKUP_L2 (p);

(gdb) p base
$7 = (page_entry ***) 0x0
(gdb) p G
$9 = {pages = {0x0, 0x0, 0x0, 0x98b820, 0x98ac88, 0xbc7060, 0x98ae48, 0x0, 
    0xb25e18, 0xb65040, 0xb659a8, 0xb26c48, 0xb63fa0, 0xb650e0, 0xb28e20, 
    0xb2bcc0, 0xb64fa0, 0x7a5e18, 0x4adb58, 0x0 <repeats 13 times>, 0x98ae10, 
    0x0, 0x98afd0, 0x98add8}, page_tails = {0x0, 0x0, 0x0, 0x3dca00, 0x3defb0, 
    0x3deec0, 0x3dee70, 0x0, 0x3c05e8, 0x3c09d0, 0x3c05c0, 0x3c0890, 0x3c08b8, 
    0x3c0908, 0x3c08e0, 0x3c09f8, 0x3c0ac0, 0x4fbae8, 0x4adb58, 
    0x0 <repeats 13 times>, 0x3dece0, 0x0, 0x3def60, 0x3dee20}, lookup = {
    0x0 <repeats 223 times>, 0xb3ace8, 0xa44790, 0xa21648, 0x9fd560, 0x9b3668, 
    0x994008, 0x96c2e8, 0x5e9c00, 0x8b1260, 0x8725e0, 0x7d3c50, 0x79c8a0, 
    0x7442d0, 0x717f18, 0x4a7138, 0x69f460, 0x678e08, 0x5090f8, 0x5ac4a8, 
    0x495a88, 0x5286f8, 0x4e28c0, 0x4bfd10, 0x430940, 0x3dcaa8, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, pagesize = 8192, lg_pagesize = 13, 
  allocated = 0, allocated_last_gc = 0, bytes_mapped = 268574720, 
  context_depth_allocations = 1, context_depth_collections = 1, 
  context_depth = 0, free_pages = 0x0, debug_file = 0x3604c8, 
  depth_in_use = 1, depth_max = 10, depth = 0x3c2cc0, by_depth_in_use = 32630, 
  by_depth_max = 32768, by_depth = 0x75c890, save_in_use = 0x77c898}


Also, I took a look at the isns gdb showed me:  The crash is happening at
0x254f58 --- Here's a few lines of isns on either side:

    0x254f44 <ggc_set_mark+40>:	sub  %g1, %o4, %g1
    0x254f48 <ggc_set_mark+44>:	sll  %l2, %g1, %g1
    0x254f4c <ggc_set_mark+48>:	add  %g1, -1, %g1
    0x254f50 <ggc_set_mark+52>:	and  %o5, %g1, %o5
    0x254f54 <ggc_set_mark+56>:	sll  %o5, 2, %o5
*** 0x254f58 <ggc_set_mark+60>:	ld  [ %o3 + %o5 ], %l3
    0x254f5c <ggc_set_mark+64>:	sethi  %hi(0x388800), %l1
    0x254f60 <ggc_set_mark+68>:	ldub  [ %l3 + 0x16 ], %l0
    0x254f64 <ggc_set_mark+72>:	ld  [ %l3 + 8 ], %g1
    0x254f68 <ggc_set_mark+76>:	or  %l1, 0x380, %l1
    0x254f6c <ggc_set_mark+80>:	sll  %l0, 3, %l0

I also set a breakpoint at this part of the code --- when it doesn't crash, I
saw some of the following values for the 3 registers used in the isn:

$o3 : 0x3e67a8, 0x98b690, 0x5bf950
$o5 :   0x146c,    0xe94,   0x17b4  
$l3 : 0x3dcaa8, 0xb3ace8, 0x8b1260

but when it crashed, gdb shows all 3 registers as containing 0's...

[Bug c++/12252] ICE on very large source file : GC problem?

[warren_baird at cimmetry dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252



------- Additional Comments From warren_baird at cimmetry dot com  2003-09-11 20:37 -------
Created an attachment (id=4740)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=4740&action=view)
Backtrace from the ICE

[Bug c++/12252] ICE on very large source file : GC problem?

[mec at shout dot net]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252



------- Additional Comments From mec at shout dot net  2003-09-13 20:27 -------
I'm not an expert on this at all, but I have a suggestion.

Put a breakpoint inside ggc_collect about 10 lines in, after this expression:

  if (G.allocated < allocated_last_gc + min_expand)
    return;

  timevar_push (TV_GC);  /* BREAKPOINT HERE */

This breakpoint will get hit when the garbage collector is going to do some
actual work.  (You can't just break on ggc_collect because most of the calls
take the early return without doing work).

Each time that breakpoint is hit, take a full stack trace.  Then report those
stack traces.  Attach all of them, not just the last one.

If this is a garbage collection bug, then one of the stack traces is likely show
some function that thinks it is safe from gc but is actually getting gc called.
 I can't tell which function it will be -- that is for a real gcc hacker.

Also, someone let me know if this is actually helpful for garbage collection
bugs.  My understanding is that it's useful but if I am mistaken I welcome a
correction.

Hope this helps,

Michael C

[Bug c++/12252] ICE on very large source file : GC problem?

[ebotcazou at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252



------- Additional Comments From ebotcazou at gcc dot gnu dot org  2003-09-14 07:07 -------
Here's a tip from our resident GC expert:

> The best way to start investigating a GC problem is to use
> --param ggc-min-heapsize=0 --param ggc-min-expand=0
> which will force garbage collection to happen every time it can.
> This should make the problem reproducible and make it easier to produce
> a cut-down testcase.

He also said that GC problems are safely fixable on a release branch, so if you
can trim down your testcase and post it here, I'll investigate.

[Bug c++/12252] ICE on very large source file : GC problem?

[warren_baird at cimmetry dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252



------- Additional Comments From warren_baird at cimmetry dot com  2003-09-15 22:07 -------
Thanks for the suggestions, Michael and Eric...  I haven't Tried Michael's yet
---  But I probably will tomorrow...   I tried Eric's suggestion of setting the
GC heap parameters to 0, and gcc is now taking slightly over an hour before it
hits the point when it crashes (on a blade 1000 with 900Mhz Ultrasparc III and
2.5 bg of ram), so it has been a bit slow...   so far I've determined that the
original file that succeeded (tmp2.ii in the above description) now also seg
faults, but I've tried trimming it down to 40k and 65k lines, and both of the
shorter versions compiled..  I'm trying 85k lines now...  I'll post more details
here as I have them...

[Bug c++/12252] ICE on very large source file : GC problem?

[ebotcazou at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252



------- Additional Comments From ebotcazou at gcc dot gnu dot org  2003-09-16 05:55 -------
I should have said that you can try to gradually decrease the GC parameters
before going to 0, the theory being that this could let you gradually decrease
the size of the testcase. However, if the original testcase doesn't crash at 0,
we're stuck.

[Bug c++/12252] ICE on very large source file : GC problem?

[warren_baird at cimmetry dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252



------- Additional Comments From warren_baird at cimmetry dot com  2003-09-16 13:43 -------
I guess I wasn't clear enough in my post yesterday --- sorry...  Originally I
had 2 files - tmp1.ii and tmp2.ii - both over 110k lines long --- tmp1.ii
segfaults with the default parameters, and with the parameters set to 0. 
tmp2.ii doesn't segfault with the default params, but does with them set to 0...
 However, I've tried chopping tmp2.ii down to get a smaller test case and
haven't had any luck ---  I cut out the last 30k lines, and it took over two
hours, but compiled successfully.  I might eventually be able to cut it down to
a workable test case, but it's going to take a *long* time...

I also tried Michael's suggestion - the results were more interesting...  We
only hit the 'timevar_push' line in ggc_collect once before things crash.  
Here's the traceback:


Breakpoint 1, ggc_collect () at ../../gcc-3.3.1/gcc/ggc-page.c:1711
(gdb) where
#0  ggc_collect () at ../../gcc-3.3.1/gcc/ggc-page.c:1711
#1  0x00236820 in rest_of_compilation (decl=0xf8013700) at
../../gcc-3.3.1/gcc/toplev.c:2553
#2  0x00093d90 in genrtl_finish_function (fn=0xf8013700) at
../../gcc-3.3.1/gcc/cp/semantics.c:2589
#3  0x00093a2c in expand_body (fn=0xf8013700) at
../../gcc-3.3.1/gcc/cp/semantics.c:2412
#4  0x0004860c in instantiate_decl (d=0xf8013700, defer_ok=0) at
../../gcc-3.3.1/gcc/cp/pt.c:10492
#5  0x000489b0 in instantiate_pending_templates () at
../../gcc-3.3.1/gcc/cp/pt.c:10569
#6  0x0005bd1c in finish_file () at ../../gcc-3.3.1/gcc/cp/decl2.c:2803
#7  0x00069000 in yyparse () at parse.y:489
#8  0x000b0cc4 in c_common_parse_file (set_yydebug=0) at
../../gcc-3.3.1/gcc/c-lex.c:159
#9  0x0023600c in compile_file () at ../../gcc-3.3.1/gcc/toplev.c:2130
#10 0x0023b44c in do_compile () at ../../gcc-3.3.1/gcc/toplev.c:5383
#11 0x0023b50c in toplev_main (argc=13, argv=0xffbef7a4) at
../../gcc-3.3.1/gcc/toplev.c:5414

So I assume this means that cc1 is crashing on the very first attempt to garbage
collect?

[Bug c++/12252] ICE on very large source file : GC problem?

[ebotcazou at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252


ebotcazou at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ebotcazou at gcc dot gnu dot
                   |                            |org

[Bug c++/12252] ICE on very large source file : GC problem?

[ebotcazou at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252


ebotcazou at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  GCC build triplet|                            |sparc-sun-solaris2.8
   GCC host triplet|                            |sparc-sun-solaris2.8
 GCC target triplet|                            |sparc-sun-solaris2.8

[Bug c++/12252] ICE on very large source file : GC problem?

[reichelt at gcc dot gnu dot org]

------- Additional Comments From reichelt at gcc dot gnu dot org  2003-11-26 12:41 -------
Maybe you shouldn't try --param ggc-min-heapsize=0, but small values like 2-10.
This compiles must faster, but still might trigger GC often enough.
Sometimes you might have to change the value while reducing the testcase.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |reichelt at gcc dot gnu dot
                   |                            |org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252

[Bug c++/12252] ICE on very large source file : GC problem?

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-01-25 21:53 -------
Can you try 3.3.2 or a snapshot of 3.3.3 or a snapshot 3.4?

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252

[Bug c++/12252] ICE on very large source file : GC problem?

[ebotcazou at gcc dot gnu dot org]

------- Additional Comments From ebotcazou at gcc dot gnu dot org  2004-03-29 13:32 -------
No feedback for more than 3 months.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |INVALID


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12252