[Bug lto/44966] New: weak LTO with gold causes ICE in lto_symtab_merge_decls_1

[Bug lto/44966] New: weak LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

With the attached test case which tests WHOPR with weak aliases 
I get a segfault in lto_symtab_merge_decls_1 line 758

This happens in

          if (prevailing->node->same_body_alias)
            prevailing->node->same_body->local.used_from_object_file = true;
          else
            prevailing->node->local.used_from_object_file = true;

because node is NULL

Interestingly this only happens with gold, if I drop -fuse-linker-plugin
there is no segfault

This is an extract of a failure that occurred while buidling a much larger
project.


-- 
           Summary: weak LTO with gold causes ICE in
                    lto_symtab_merge_decls_1
           Product: gcc
           Version: 4.6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: andi-gcc at firstfloor dot org
 GCC build triplet: x86_64-linux
  GCC host triplet: x86_64-linux
GCC target triplet: x86_64-linux


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #1 from andi-gcc at firstfloor dot org  2010-07-16 23:23 -------
Created an attachment (id=21228)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=21228&action=view)
test case


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at gcc dot gnu dot org]

------- Comment #2 from hubicka at gcc dot gnu dot org  2010-07-19 17:50 -------
mine


-- 

hubicka at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |hubicka at gcc dot gnu dot
                   |dot org                     |org
             Status|UNCONFIRMED                 |ASSIGNED
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2010-07-19 17:50:52
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at gcc dot gnu dot org]

------- Comment #3 from hubicka at gcc dot gnu dot org  2010-07-19 17:52 -------
Created an attachment (id=21251)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=21251&action=view)
fix

Hi,
the problem is that alias is defined in one module, but aliases don't get
cgraph nodes unless they are used.  It is used in other modules and while
merging we clash.

It seems OK to just create new node in this case though it is bit ugly (at
least ENABLE_CHECKING should verify that decl is indeed alias) or perhaps we
can always create node.  I will look into it more, but this patch at least
solves ICE on the testcase.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #4 from andi-gcc at firstfloor dot org  2010-07-19 19:13 -------
The test case works, but my big project now crashes with

758                 prevailing->node->local.used_from_object_file = false;

(gdb) p prevailing->vnode
$1 = (struct varpool_node *) 0x0

I think it's a different variable now. I assume you'll need a test case for
this
too?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at ucw dot cz]

------- Comment #5 from hubicka at ucw dot cz  2010-07-19 22:13 -------
Subject: Re:  weak aliases LTO with gold causes ICE in
        lto_symtab_merge_decls_1

Hmm, I guess we can do the same trick for variables. Will make patch.

Honza


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at ucw dot cz]

------- Comment #6 from hubicka at ucw dot cz  2010-07-19 23:50 -------
Subject: Re:  weak aliases LTO with gold causes ICE in
        lto_symtab_merge_decls_1

Hi,
try this one

Index: lto-symtab.c
===================================================================
--- lto-symtab.c        (revision 162310)
+++ lto-symtab.c        (working copy)
@@ -783,9 +783,25 @@ lto_symtab_merge_cgraph_nodes_1 (void **
   for (e = prevailing->next; e; e = e->next)
     {
       if (e->node != NULL)
-       lto_cgraph_replace_node (e->node, prevailing->node);
+       {
+         if (!prevailing->node)
+           {
+             gcc_assert (!e->node->analyzed);
+             prevailing->node = cgraph_node (prevailing->decl);
+             prevailing->node->alias = true;
+           }
+         lto_cgraph_replace_node (e->node, prevailing->node);
+       }
       if (e->vnode != NULL)
-       lto_varpool_replace_node (e->vnode, prevailing->vnode);
+       {
+         if (!prevailing->vnode)
+           {
+             gcc_assert (!e->vnode->finalized);
+             prevailing->vnode = varpool_node (prevailing->decl);
+             prevailing->vnode->alias = true;
+           }
+         lto_varpool_replace_node (e->vnode, prevailing->vnode);
+       }
     }

   /* Drop all but the prevailing decl from the symtab.  */


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #7 from andi-gcc at firstfloor dot org  2010-07-20 00:23 -------
Thanks. Unfortunately with that patch I still get the same ICE:

758                 prevailing->node->local.used_from_object_file = false;


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at ucw dot cz]

------- Comment #8 from hubicka at ucw dot cz  2010-07-20 00:29 -------
Subject: Re:  weak aliases LTO with gold causes ICE in
        lto_symtab_merge_decls_1

> Thanks. Unfortunately with that patch I still get the same ICE:
> 
> 758                 prevailing->node->local.used_from_object_file = false;
You might get around by just checking prevailing->node != NULL (and same for
vnode).
For aliases where we don't have nodes/vnoes we don't need this flag.
I can try to send you updated patch tomorrow (now really need some sleep)

Honza


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #9 from andi-gcc at firstfloor dot org  2010-07-20 10:15 -------
I ignored all nodes with NULL node/vnode now as 
you suggested and get much further.

It takes a lot of memory in multiple lto1 runs (>2GB in some cases)

Then after a long time I have a new ICE in the garbage collector

Any ideas how to debug this?

600       while (table->high_bits != high_bits)
(gdb) p table
$1 = (struct page_table_chain *) 0x0

Apparently there's a non GC object in the garbage collector?


#8  lookup_page_table_entry (p=0xffa2190) at ../../gcc/gcc/ggc-page.c:600
#9  ggc_set_mark (p=0xffa2190) at ../../gcc/gcc/ggc-page.c:1339
#10 0x00000000005cd1b0 in gt_ggc_mx_cgraph_node (x_p=<value optimized out>) at
gtype-desc.c:552
#11 0x00000000005cd26e in gt_ggc_m_P11cgraph_node4htab (x_p=0x2b74ddb9c1c0) at
gtype-desc.c:2448
#12 0x00000000005a9406 in ggc_mark_roots () at ../../gcc/gcc/ggc-common.c:169
#13 0x00000000004b04fd in ggc_collect () at ../../gcc/gcc/ggc-page.c:1950
#14 0x00000000004acb9b in read_cgraph_and_symbols (debug_p=<value optimized
out>) at ../../gcc/gcc/lto/lto.c:1783
#15 lto_main (debug_p=<value optimized out>) at ../../gcc/gcc/lto/lto.c:2023
#16 0x00000000006c5cde in compile_file (argc=75, argv=0x105d4b0) at
../../gcc/gcc/toplev.c:978
#17 do_compile (argc=75, argv=0x105d4b0) at ../../gcc/gcc/toplev.c:2333
#18 toplev_main (argc=75, argv=0x105d4b0) at ../../gcc/gcc/toplev.c:2374


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #10 from andi-gcc at firstfloor dot org  2010-07-20 12:36 -------
I hacked around the GC problem by ignoring the NULL pointer here.
It works now when I disable vma randomization, if i keep it enabled it 
crashes occasionally.

Main issue is that it uses a lot of memory now, drove my 4GB workstation
several times into out of memory.

Now I get a full link, but a lot of new problems that I'm looking into.
It seems like a lot of functions, both in assembler and in C, disappear.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #11 from andi-gcc at firstfloor dot org  2010-07-20 20:39 -------
Some more debugging on the gc crash, suggested by honza:

The corruption seems to happen in cgraph_hash

(gdb) p *x
$2 = {hash_f = 0x827750 <hash_node>, eq_f = 0x827760 <eq_node>, del_f = 0,
entries = 0x7f67b734c000, size = 262139, n_elements = 98498, 
  n_deleted = 0, searches = 159361, collisions = 161480, alloc_f = 0x5a9800
<ggc_cleared_alloc_ptr_array_two_args>, 
  free_f = 0x4b0080 <ggc_free>, alloc_arg = 0x0, alloc_with_arg_f = 0,
free_with_arg_f = 0, size_prime_index = 15}
(gdb) p cgraph_hash
$3 = (htab_t) 0x7f67bb3a8000
(gdb) p x
$4 = (struct htab * const) 0x7f67bb3a8000
(gdb) 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at ucw dot cz]

------- Comment #12 from hubicka at ucw dot cz  2010-07-21 11:05 -------
Subject: Re:  weak aliases LTO with gold causes ICE in
        lto_symtab_merge_decls_1

If you link .c and .s produced objects into single .o file via incremental
linking, you probably get all the assembly
files lost, since LTO plugin claims the whole .o file and thus linker will
never look into the assembly part of
the object.

I looked into the section you sent me.  There seems to be nothing obvious.  The
decl sectionis gigantic, but in dump it seems it is all trees, so we need to
figure out what they are that is probably best done by adding dump for
streamer.

Honza


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #13 from andi-gcc at firstfloor dot org  2010-07-21 15:21 -------
I know I lost some assembler files, but I think I didn't lose all of them.
Some assembler code is still there. Any suggestion how to fix this?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at gcc dot gnu dot org]

------- Comment #14 from hubicka at gcc dot gnu dot org  2010-07-26 21:09 -------
Hmm, hard to tell.  The assembler .o files are not LTO and they will be
incrementally linked into non-LTO part of GCC produced fat .o files during
incremental linking and I guess it is hard to undo?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[andi-gcc at firstfloor dot org]

------- Comment #15 from andi-gcc at firstfloor dot org  2010-07-27 06:56 -------
I'm working on a solution for the lost code problem. It will require
-ffunction/data-sections so that it can be undone section by section.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

[Bug lto/44966] weak aliases LTO with gold causes ICE in lto_symtab_merge_decls_1

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44966

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #16 from Jan Hubicka <hubicka at gcc dot gnu.org> 2010-12-14 23:34:47 UTC ---
Fixed. Sorry for the delay.  The patch turned out to be almost correct except
for not seeting the alias flag.