[Bug lto/55466] New: [4.8 Regression] Revision 191466 destroyed DWARF debug info

[Bug lto/55466] New: [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

             Bug #: 55466
           Summary: [4.8 Regression] Revision 191466 destroyed DWARF debug
                    info
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: hjl.tools@gmail.com
                CC: hubicka@gcc.gnu.org


On Linux/x86-64, after revision 191466:

http://gcc.gnu.org/ml/gcc-cvs/2012-09/msg00623.html

GCC configured with

--enable-bootstrap --with-build-config=bootstrap-lto

lost debug info on data:

gnu-mic-2:pts/4[441]> gdb gcc/cc1
GNU gdb (GDB) 7.5.50.20121119-cvs
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/export/project/git/gcc-regression-bootstrap/master/191466/bld/gcc/cc1...done.
(gdb) whatis global_options
type = <data variable, no debug info>
(gdb) whatis cl_options
type = <data variable, no debug info>
(gdb) whatis cl_enums
type = <data variable, no debug info>
(gdb)  whatis cl_enums_count
type = <data variable, no debug info>
(gdb) whatis global_options_set
type = <data variable, no debug info>
(gdb) whatis sched_max_luid
type = <data variable, no debug info>
(gdb)

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hubicka at ucw dot cz]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #1 from Jan Hubicka <hubicka at ucw dot cz> 2012-11-26 12:45:23 UTC ---
> /export/project/git/gcc-regression-bootstrap/master/191466/bld/gcc/cc1...done.
> (gdb) whatis global_options
> type = <data variable, no debug info>
> (gdb) whatis cl_options
> type = <data variable, no debug info>
> (gdb) whatis cl_enums
> type = <data variable, no debug info>
> (gdb)  whatis cl_enums_count
> type = <data variable, no debug info>
> (gdb) whatis global_options_set
> type = <data variable, no debug info>
> (gdb) whatis sched_max_luid
> type = <data variable, no debug info>
> (gdb)

Hmm, I suppose this is because we no longer merge symbols that are not part of
symtab, but
used only for debugging....

Honza

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |4.8.0

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> 2012-11-26 15:57:37 UTC ---
(In reply to comment #1)
> > /export/project/git/gcc-regression-bootstrap/master/191466/bld/gcc/cc1...done.
> > (gdb) whatis global_options
> > type = <data variable, no debug info>
> > (gdb) whatis cl_options
> > type = <data variable, no debug info>
> > (gdb) whatis cl_enums
> > type = <data variable, no debug info>
> > (gdb)  whatis cl_enums_count
> > type = <data variable, no debug info>
> > (gdb) whatis global_options_set
> > type = <data variable, no debug info>
> > (gdb) whatis sched_max_luid
> > type = <data variable, no debug info>
> > (gdb)
> 
> Hmm, I suppose this is because we no longer merge symbols that are not part of
> symtab, but
> used only for debugging....
> 
> Honza

global_options is certainly used.  Aggressively removing unused vars is ok
even if that drops debug info for them.

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> 2012-11-26 16:36:10 UTC ---
(In reply to comment #2)
>
> > 
> > Hmm, I suppose this is because we no longer merge symbols that are not part of
> > symtab, but
> > used only for debugging....
> > 
> > Honza
> 
> global_options is certainly used.  Aggressively removing unused vars is ok
> even if that drops debug info for them.

We lost debug info on most, if not all, used data variables. It
seems that their type infos are also gone. We can't tell

extern int *foo;

from

extern int foo[];

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2012-11-28
     Ever Confirmed|0                           |1

--- Comment #4 from H.J. Lu <hjl.tools at gmail dot com> 2012-11-28 21:26:30 UTC ---
The problem is in

static void 
lto_write_globals (void)
{
  tree *vec = lto_global_var_decls->address ();
  int len = lto_global_var_decls->length ();
  wrapup_global_declarations (vec, len);
  emit_debug_global_declarations (vec, len);
  vec_free (lto_global_var_decls);
}

LTO no longer maintains its own symbol table and it uses varpool for
variables instead. lto_global_var_decls is almost empty.  That
is why debug info on variables is lost.

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #5 from Jan Hubicka <hubicka at gcc dot gnu.org> 2012-11-29 18:52:27 UTC ---
Created attachment 28829
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28829
Proposed fix

I suppose something across these lines should do the trick.  I am not sure what
exactly is the codition for var to be needed to go through wrapup_global_decl. 
Also I think types are usually going in there, too.

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #6 from H.J. Lu <hjl.tools at gmail dot com> 2012-11-29 19:07:23 UTC ---
(In reply to comment #5)
> Created attachment 28829 [details]
> Proposed fix
> 
> I suppose something across these lines should do the trick.  I am not sure what
> exactly is the codition for var to be needed to go through wrapup_global_decl. 
> Also I think types are usually going in there, too.

It doesn't work:

lto1: internal compiler error: Segmentation fault
0x994661 crash_signal
        /export/gnu/import/git/gcc/gcc/toplev.c:334
0x5206c8 gt_ggc_mx_lang_tree_node(void*)
        ./gtype-lto.h:37
0x522623 gt_ggc_mx_lang_tree_node(void*)
        ./gtype-lto.h:389
0x522539 gt_ggc_mx_lang_tree_node(void*)
        ./gtype-lto.h:380
0x5399ab gt_ggc_ma_gimple_type_leader
        ./gt-lto-lto.h:154
0x7419c4 ggc_mark_root_tab
        /export/gnu/import/git/gcc/gcc/ggc-common.c:138
0x741a74 ggc_mark_roots()
        /export/gnu/import/git/gcc/gcc/ggc-common.c:157
0x5433e5 ggc_collect()
        /export/gnu/import/git/gcc/gcc/ggc-page.c:2087
0x5388b4 read_cgraph_and_symbols
        /export/gnu/import/git/gcc/gcc/lto/lto.c:2993
0x5393c9 lto_main()
        /export/gnu/import/git/gcc/gcc/lto/lto.c:3383
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #7 from H.J. Lu <hjl.tools at gmail dot com> 2012-11-29 19:08:28 UTC ---
This patch:

diff --git a/gcc/lto-symtab.c b/gcc/lto-symtab.c
index 0b0cdac..295fd37 100644
--- a/gcc/lto-symtab.c
+++ b/gcc/lto-symtab.c
@@ -443,10 +443,6 @@ lto_symtab_merge_decls_1 (symtab_node first)

   symtab_prevail_in_asm_name_hash (prevailing);

-  /* Record the prevailing variable.  */
-  if (TREE_CODE (prevailing->symbol.decl) == VAR_DECL)
-    vec_safe_push (lto_global_var_decls, prevailing->symbol.decl);
-
   /* Diagnose mismatched objects.  */
   for (e = prevailing->symbol.next_sharing_asm_name;
        e; e = e->symbol.next_sharing_asm_name)
diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
index 376af85..177fbfc 100644
--- a/gcc/lto/lto.c
+++ b/gcc/lto/lto.c
@@ -2910,6 +2910,7 @@ read_cgraph_and_symbols (unsigned nfiles, const char
**fnames)
   struct cgraph_node *node;
   int count = 0;
   struct lto_file_decl_data **decl_data;
+  struct varpool_node *vnode;

   init_cgraph ();

@@ -3088,6 +3089,10 @@ read_cgraph_and_symbols (unsigned nfiles, const char
**fnames)

   timevar_pop (TV_IPA_LTO_CGRAPH_MERGE);

+  /* Record the global variables.  */
+  FOR_EACH_DEFINED_VARIABLE (vnode)
+    vec_safe_push (lto_global_var_decls, vnode->symbol.decl);
+
   timevar_push (TV_IPA_LTO_DECL_INIT_IO);

   /* Indicate that the cgraph is built and ready.  */

or

diff --git a/gcc/lto-symtab.c b/gcc/lto-symtab.c
index 0b0cdac..295fd37 100644
--- a/gcc/lto-symtab.c
+++ b/gcc/lto-symtab.c
@@ -443,10 +443,6 @@ lto_symtab_merge_decls_1 (symtab_node first)

   symtab_prevail_in_asm_name_hash (prevailing);

-  /* Record the prevailing variable.  */
-  if (TREE_CODE (prevailing->symbol.decl) == VAR_DECL)
-    vec_safe_push (lto_global_var_decls, prevailing->symbol.decl);
-
   /* Diagnose mismatched objects.  */
   for (e = prevailing->symbol.next_sharing_asm_name;
        e; e = e->symbol.next_sharing_asm_name)
diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
index 376af85..c7e1100 100644
--- a/gcc/lto/lto.c
+++ b/gcc/lto/lto.c
@@ -3373,6 +3373,8 @@ lto_main (void)

   if (!seen_error ())
     {
+      struct varpool_node *vnode;
+
       /* If WPA is enabled analyze the whole call graph and create an
      optimization plan.  Otherwise, read in all the function
      bodies and continue with optimization.  */
@@ -3398,6 +3400,10 @@ lto_main (void)
       if (flag_lto_report)
         print_lto_report_1 ();
     }
+
+      /* Record the global variables.  */
+      FOR_EACH_DEFINED_VARIABLE (vnode)
+    vec_safe_push (lto_global_var_decls, vnode->symbol.decl);
     }

   /* Here we make LTO pretend to be a parser.  */

seem to work.

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P1

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> 2012-12-07 10:42:42 UTC ---
Only LTRANS should be affected - WPA definitely does not need any of this
(rest_of_decl_compilation).  LTRANS OTOH does not need any type / symtab
merging.  Merged WPA/LTRANS phase of course complicates things here.

What I can't see is why this all should make a difference at LTRANS?
Do we garbage-collect unused globals there before wrapping up global decls?
Do we somehow not ship them to the correct LTRANS units?

We'd wanted to get rid of lto_global_var_decls orginally (replace it by
symtab).  At the moment it's _only_ used for calling rest_of_decl_compilation
on globals (and for static we call that twice ...).  So, HJ, your patch does
look reasonable for 4.8 - can you give it proper testing and submit it?

OTOH the LTO path should not call rest_of_decl_compilation but sth more
specific ... (most of it is disabled for the LTO path anyway).

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #9 from H.J. Lu <hjl.tools at gmail dot com> 2012-12-07 13:02:12 UTC ---
I am testing this patch:

---
diff --git a/gcc/lto-symtab.c b/gcc/lto-symtab.c
index 0b0cdac..295fd37 100644
--- a/gcc/lto-symtab.c
+++ b/gcc/lto-symtab.c
@@ -443,10 +443,6 @@ lto_symtab_merge_decls_1 (symtab_node first)

   symtab_prevail_in_asm_name_hash (prevailing);

-  /* Record the prevailing variable.  */
-  if (TREE_CODE (prevailing->symbol.decl) == VAR_DECL)
-    vec_safe_push (lto_global_var_decls, prevailing->symbol.decl);
-
   /* Diagnose mismatched objects.  */
   for (e = prevailing->symbol.next_sharing_asm_name;
        e; e = e->symbol.next_sharing_asm_name)
diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
index 376af85..c637ec8 100644
--- a/gcc/lto/lto.c
+++ b/gcc/lto/lto.c
@@ -3083,8 +3083,16 @@ read_cgraph_and_symbols (unsigned nfiles, const char
**fnames)
      supports inlining, so we can push it here by hand.  In future we need to
stream
      this field into ltrans compilation.  */
   if (flag_ltrans)
-    FOR_EACH_DEFINED_FUNCTION (node)
-      node->ipa_transforms_to_apply.safe_push
((ipa_opt_pass)&pass_ipa_inline);
+    {
+      struct varpool_node *vnode;
+
+      FOR_EACH_DEFINED_FUNCTION (node)
+    node->ipa_transforms_to_apply.safe_push ((ipa_opt_pass)&pass_ipa_inline);
+
+      /* Record the global variables.  */
+      FOR_EACH_DEFINED_VARIABLE (vnode)
+    vec_safe_push (lto_global_var_decls, vnode->symbol.decl);
+    }

   timevar_pop (TV_IPA_LTO_CGRAPH_MERGE);

---

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #10 from H.J. Lu <hjl.tools at gmail dot com> 2012-12-07 18:49:10 UTC ---
(In reply to comment #7)
> This patch:
> 
> diff --git a/gcc/lto-symtab.c b/gcc/lto-symtab.c
> index 0b0cdac..295fd37 100644
> --- a/gcc/lto-symtab.c
> +++ b/gcc/lto-symtab.c
> @@ -443,10 +443,6 @@ lto_symtab_merge_decls_1 (symtab_node first)
> 
>    symtab_prevail_in_asm_name_hash (prevailing);
> 
> -  /* Record the prevailing variable.  */
> -  if (TREE_CODE (prevailing->symbol.decl) == VAR_DECL)
> -    vec_safe_push (lto_global_var_decls, prevailing->symbol.decl);
> -
>    /* Diagnose mismatched objects.  */
>    for (e = prevailing->symbol.next_sharing_asm_name;
>         e; e = e->symbol.next_sharing_asm_name)
> diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
> index 376af85..177fbfc 100644
> --- a/gcc/lto/lto.c
> +++ b/gcc/lto/lto.c
> @@ -2910,6 +2910,7 @@ read_cgraph_and_symbols (unsigned nfiles, const char
> **fnames)
>    struct cgraph_node *node;
>    int count = 0;
>    struct lto_file_decl_data **decl_data;
> +  struct varpool_node *vnode;
> 
>    init_cgraph ();
> 
> @@ -3088,6 +3089,10 @@ read_cgraph_and_symbols (unsigned nfiles, const char
> **fnames)
> 
>    timevar_pop (TV_IPA_LTO_CGRAPH_MERGE);
> 
> +  /* Record the global variables.  */
> +  FOR_EACH_DEFINED_VARIABLE (vnode)
> +    vec_safe_push (lto_global_var_decls, vnode->symbol.decl);
> +
>    timevar_push (TV_IPA_LTO_DECL_INIT_IO);
> 
>    /* Indicate that the cgraph is built and ready.  */
> 
> or
> 
> diff --git a/gcc/lto-symtab.c b/gcc/lto-symtab.c
> index 0b0cdac..295fd37 100644
> --- a/gcc/lto-symtab.c
> +++ b/gcc/lto-symtab.c
> @@ -443,10 +443,6 @@ lto_symtab_merge_decls_1 (symtab_node first)
> 
>    symtab_prevail_in_asm_name_hash (prevailing);
> 
> -  /* Record the prevailing variable.  */
> -  if (TREE_CODE (prevailing->symbol.decl) == VAR_DECL)
> -    vec_safe_push (lto_global_var_decls, prevailing->symbol.decl);
> -
>    /* Diagnose mismatched objects.  */
>    for (e = prevailing->symbol.next_sharing_asm_name;
>         e; e = e->symbol.next_sharing_asm_name)
> diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
> index 376af85..c7e1100 100644
> --- a/gcc/lto/lto.c
> +++ b/gcc/lto/lto.c
> @@ -3373,6 +3373,8 @@ lto_main (void)
> 
>    if (!seen_error ())
>      {
> +      struct varpool_node *vnode;
> +
>        /* If WPA is enabled analyze the whole call graph and create an
>       optimization plan.  Otherwise, read in all the function
>       bodies and continue with optimization.  */
> @@ -3398,6 +3400,10 @@ lto_main (void)
>        if (flag_lto_report)
>          print_lto_report_1 ();
>      }
> +
> +      /* Record the global variables.  */
> +      FOR_EACH_DEFINED_VARIABLE (vnode)
> +    vec_safe_push (lto_global_var_decls, vnode->symbol.decl);
>      }
> 
>    /* Here we make LTO pretend to be a parser.  */
> 
> seem to work.

It doesn't work.  When gcc is configured with

--enable-languages=c,c++,fortran,java,lto,objc,obj-c++,go, I got

lto1: internal compiler error: in add_AT_specification, at dwarf2out.c:3985
0x629ad9 add_AT_specification
        /export/gnu/import/git/gcc/gcc/dwarf2out.c:3985
0x64dabb gen_variable_die
        /export/gnu/import/git/gcc/gcc/dwarf2out.c:18327
0x65336b gen_decl_die
        /export/gnu/import/git/gcc/gcc/dwarf2out.c:20033
0x653fcd dwarf2out_decl(tree_node*)
        /export/gnu/import/git/gcc/gcc/dwarf2out.c:20348
0x6535f9 dwarf2out_global_decl
        /export/gnu/import/git/gcc/gcc/dwarf2out.c:20083
0x997703 emit_debug_global_declarations(tree_node**, int)
        /export/gnu/import/git/gcc/gcc/toplev.c:530
0x521aaa lto_write_globals
        /export/gnu/import/git/gcc/gcc/lto/lto-lang.c:1067
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[6]: *** [/tmp/cc3yYR7d.ltrans4.ltrans.o] Error 1
lto-wrapper: make returned 2 exit status
lto-wrapper: make returned 2 exit status
/usr/local/x86_64-unknown-linux-gnu/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[5]: *** [go1] Error 1

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #11 from hjl at gcc dot gnu.org <hjl at gcc dot gnu.org> 2012-12-10 14:44:00 UTC ---
Author: hjl
Date: Mon Dec 10 14:43:54 2012
New Revision: 194359

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=194359
Log:
Record the global variables if WPA isn't enabled

    PR lto/55466
    * lto-symtab.c (lto_symtab_merge_decls_1): Don't record the
    prevailing variable.
    * lto.c (lto_register_var_decl_in_symtab): Don't record static
    variables.
    (lto_main): Record the global variables if WPA isn't enabled.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/lto-symtab.c
    trunk/gcc/lto/lto.c

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #12 from H.J. Lu <hjl.tools at gmail dot com> 2012-12-10 16:39:40 UTC ---
Fixed.

[Bug lto/55466] [4.8 Regression] Revision 191466 destroyed DWARF debug info

[hjl at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55466

--- Comment #13 from hjl at gcc dot gnu.org <hjl at gcc dot gnu.org> 2013-01-03 16:54:07 UTC ---
Author: hjl
Date: Thu Jan  3 16:53:54 2013
New Revision: 194861

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=194861
Log:
Fix ChangeLog entry for PR lto/55466

Modified:
    trunk/gcc/ChangeLog-2012
    trunk/gcc/lto/ChangeLog