[Bug lto/62034] New: ICE for big statically initialized arrays compiled with LTO

[Bug lto/62034] New: ICE for big statically initialized arrays compiled with LTO

[enkovich.gnu at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62034

            Bug ID: 62034
           Summary: ICE for big statically initialized arrays compiled
                    with LTO
           Product: gcc
           Version: 4.10.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
          Assignee: unassigned at gcc dot gnu.org
          Reporter: enkovich.gnu at gmail dot com

Created attachment 33259
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=33259&action=edit
Reproducer

I get ICE when try to compile tests with big amount of statically initialized
data.

gcc --version
gcc (GCC) 4.10.0 20140806 (experimental)
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

gcc -flto test.c
gcc: internal compiler error: Segmentation fault (program lto1)
0x405c80 execute
        ../../gcc-ref/gcc/gcc.c:2900
0x409fe9 do_spec_1
        ../../gcc-ref/gcc/gcc.c:4704
0x40d475 process_brace_body
        ../../gcc-ref/gcc/gcc.c:5987
0x40d2b1 handle_braces
        ../../gcc-ref/gcc/gcc.c:5901
0x40bf9d do_spec_1
        ../../gcc-ref/gcc/gcc.c:5358
0x40d475 process_brace_body
        ../../gcc-ref/gcc/gcc.c:5987
0x40d2b1 handle_braces
        ../../gcc-ref/gcc/gcc.c:5901
0x40bf9d do_spec_1
        ../../gcc-ref/gcc/gcc.c:5358
0x40c38c do_spec_1
        ../../gcc-ref/gcc/gcc.c:5473
0x40d475 process_brace_body
        ../../gcc-ref/gcc/gcc.c:5987
0x40d2b1 handle_braces
        ../../gcc-ref/gcc/gcc.c:5901
0x40bf9d do_spec_1
        ../../gcc-ref/gcc/gcc.c:5358
0x409664 do_spec_2
        ../../gcc-ref/gcc/gcc.c:4405
0x409582 do_spec(char const*)
        ../../gcc-ref/gcc/gcc.c:4372
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: gcc-ref-build/bin/gcc returned 4 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status

Debugger shows that problem appears when lto_input_tree tries to dig through a
bunch of SCC entries in input stream.  Each SCC entry cause two new functions
(lto_input_tree and lto_input_tree_1) in the call stack.  With many consequent
SCC entries stack may grow too much (in my case compiler segfaulted with ~600
000 entries in the call stack).

Attached test has a statically initialized array with a million elements. 
Bigger data set may be required to break the compiler if you use increased
stack size.

Problem appeared after this commit:
https://gcc.gnu.org/ml/gcc-cvs/2014-07/msg00291.html

Following patch removing recursion helps me to compile my tests:

diff --git a/gcc/lto-streamer-in.c b/gcc/lto-streamer-in.c
index 698f926..25657da 100644
--- a/gcc/lto-streamer-in.c
+++ b/gcc/lto-streamer-in.c
@@ -1345,7 +1345,16 @@ lto_input_tree_1 (struct lto_input_block *ib, struct
data_in *data_in,
 tree
 lto_input_tree (struct lto_input_block *ib, struct data_in *data_in)
 {
-  return lto_input_tree_1 (ib, data_in, streamer_read_record_start (ib), 0);
+  enum LTO_tags tag;
+
+  /* Skip SCC entries.  */
+  while ((tag = streamer_read_record_start (ib)) == LTO_tree_scc)
+    {
+      unsigned len, entry_len;
+      lto_input_scc (ib, data_in, &len, &entry_len);
+    }
+
+  return lto_input_tree_1 (ib, data_in, tag, 0);
 }

Did not fully test this patch yet.

[Bug lto/62034] ICE for big statically initialized arrays compiled with LTO

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62034

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2014-08-06
     Ever confirmed|0                           |1

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
Ah, the issue is that the "tail-recursion" doesn't work.  Mine.

[Bug lto/62034] ICE for big statically initialized arrays compiled with LTO

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62034

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Better patch:

Index: gcc/lto-streamer-in.c
===================================================================
--- gcc/lto-streamer-in.c       (revision 213660)
+++ gcc/lto-streamer-in.c       (working copy)
@@ -1325,13 +1325,17 @@ lto_input_tree_1 (struct lto_input_block
     }
   else if (tag == LTO_tree_scc)
     {
-      unsigned len, entry_len;
-
-      /* Input and skip the SCC.  */
-      lto_input_scc (ib, data_in, &len, &entry_len);
+      /* Input and skip SCCs.  */
+      do
+       {
+         unsigned len, entry_len;
+         lto_input_scc (ib, data_in, &len, &entry_len);
+         tag = streamer_read_record_start (ib);
+       }
+      while (tag == LTO_tree_scc);

       /* Recurse.  */
-      return lto_input_tree (ib, data_in);
+      return lto_input_tree_1 (ib, data_in, tag, 0);
     }
   else
     {

[Bug lto/62034] ICE for big statically initialized arrays compiled with LTO

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62034

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
Or actually

Index: gcc/lto-streamer-in.c
===================================================================
--- gcc/lto-streamer-in.c       (revision 213660)
+++ gcc/lto-streamer-in.c       (working copy)
@@ -1324,15 +1324,7 @@ lto_input_tree_1 (struct lto_input_block
       streamer_tree_cache_append (data_in->reader_cache, result, hash);
     }
   else if (tag == LTO_tree_scc)
-    {
-      unsigned len, entry_len;
-
-      /* Input and skip the SCC.  */
-      lto_input_scc (ib, data_in, &len, &entry_len);
-
-      /* Recurse.  */
-      return lto_input_tree (ib, data_in);
-    }
+    gcc_unreachable ();
   else
     {
       /* Otherwise, materialize a new node from IB.  */
@@ -1345,7 +1337,15 @@ lto_input_tree_1 (struct lto_input_block
 tree
 lto_input_tree (struct lto_input_block *ib, struct data_in *data_in)
 {
-  return lto_input_tree_1 (ib, data_in, streamer_read_record_start (ib), 0);
+  enum LTO_tags tag;
+
+  /* Input and skip SCCs.  */
+  while ((tag = streamer_read_record_start (ib)) == LTO_tree_scc)
+    {
+      unsigned len, entry_len;
+      lto_input_scc (ib, data_in, &len, &entry_len);
+    }
+  return lto_input_tree_1 (ib, data_in, tag, 0);
 }

[Bug lto/62034] ICE for big statically initialized arrays compiled with LTO

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62034

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
Author: rguenth
Date: Wed Aug  6 13:53:09 2014
New Revision: 213664

URL: https://gcc.gnu.org/viewcvs?rev=213664&root=gcc&view=rev
Log:
2014-08-06  Richard Biener  <rguenther@suse.de>

    PR lto/62034
    * lto-streamer-in.c (lto_input_tree_1): Assert we do not read
    SCCs here.
    (lto_input_tree): Pop SCCs here.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/lto-streamer-in.c

[Bug lto/62034] ICE for big statically initialized arrays compiled with LTO

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62034

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed.