[Bug debug/53453] New: darwin linker expects both AT_name and AT_comp_dir debug notes

[Bug debug/53453] New: darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

             Bug #: 53453
           Summary: darwin linker expects both AT_name and AT_comp_dir
                    debug notes
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: debug
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: howarth@nitro.med.uc.edu


Future versions of dsymutil on darwin will emit warnings of the form...

warning: no debug symbols in executable (-arch x86_64)

warning: no debug symbols in executable (-arch i386)

when the debug note for AT_comp_dir is missing from linked object files.
Upstream had the following comments...

------------------------------------------------------------------------
The issue is not directly dsymutil.  The way it works on darwin, is that 
the linker does not copy dwarf debug info.  Instead, the linker leaves 
the debug info in the .o files and then puts "debug notes" into the final 
linked image. If you run the program under gdb, the debugger sees the 
notes and opens the .of files and reads the dwarf. If instead you run 
dysmtuil, it reads the debug notes which tells it where to find the 
.o files from which to read the dwarf and merge all the dwarf into 
the .dSYM.

The dsymtuil warning is that no dwarf was found.  In this case that 
is because there were no debug notes recorded by the linker which 
was because of the lack of AT_comp_dir.   In some of the cases you've 
set up, there was no warning because there was some dwarf found but 
not all dwarf was found, so the .dSYM is less than ideal.

The format of the debug notes requires a pair of dir/file.  Because 
it always just worked before, the linker just copied the dir from 
AT_comp_dir and the file from AT_name. It does not look like the 
dwarf spec requires the AT_comp_dir, but it has always been there 
before and the linker has come to depend on it.
-------------------------------------------------------------------------

This issue can be demonstrated with Xcode 4.2 under 10.7.4 using gcc trunk...


howarth% gcc-fsf-4.8
/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120520/gcc/testsuite/gcc.c-torture/execute/builtins/20010124-1.c
/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120520/gcc/testsuite/gcc.c-torture/execute/builtins/20010124-1-lib.c
/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120520/gcc/testsuite/gcc.c-torture/execute/builtins/lib/main.c
-fno-diagnostics-show-caret --save-temps -w -O3 -g -lm -m64 -o 20010124-1.x4
howarth% dwarfdump --debug-info main.o | grep -A3 AT_language
             AT_language( DW_LANG_C89 )
             AT_name(
"/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120520/gcc/testsuite/gcc.c-torture/execute/builtins/lib/main.c"
)
             AT_stmt_list( 0x00000000 )

whereas if you copy or symlink the source files into a local directory and
compile with...

howarth% gcc-fsf-4.8 20010124-1.c 20010124-1-lib.c main.c
-fno-diagnostics-show-caret --save-temps -w -O3 -g -lm -m64 -o 20010124-1.x4

...both the AT_name and AT_comp_dir debug notes are present...

howarth% dwarfdump --debug-info main.o | grep -A3 AT_language                  
                               AT_language( DW_LANG_C89 )
             AT_name( "main.c" )
             AT_comp_dir( "/Users/howarth/xcode44_bugv2/good_binary" )
             AT_stmt_list( 0x00000000 )

and future dsymutil releases won't emit the warning.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #1 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-22 13:33:01 UTC ---
It looks like the lines...

      if (!IS_ABSOLUTE_PATH (filename) && filename[0] != '<')
        add_comp_dir_attribute (die);

in gen_compile_unit_die() of dwarf2out.c need adjusted for darwin to
insure that add_comp_dir_attribute (die) is always called.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #2 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-22 13:58:13 UTC ---
(In reply to comment #1)
> It looks like the lines...
> 
>       if (!IS_ABSOLUTE_PATH (filename) && filename[0] != '<')
>         add_comp_dir_attribute (die);
> 
> in gen_compile_unit_die() of dwarf2out.c need adjusted for darwin to
> insure that add_comp_dir_attribute (die) is always called.

I notice later in dwarf2out_finish() of dwarf2out.c that we have...


 if (!IS_ABSOLUTE_PATH (filename))
    add_comp_dir_attribute (comp_unit_die ());
  else if (get_AT (comp_unit_die (), DW_AT_comp_dir) == NULL)
    {
      bool p = false;
      htab_traverse (file_table, file_table_relative_p, &p);
      if (p)
        add_comp_dir_attribute (comp_unit_die ());
    }

perhaps gen_compile_unit_die()  needs a similar else statement?

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-05-22 14:39:29 UTC ---
There is no point to emit DW_AT_comp_dir if all filenames in the file/directory
table referenced by DW_AT_stmt_list are absolute.  Seems to be a request to
work around broken vendor tools again.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #4 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-22 23:10:03 UTC ---
(In reply to comment #3)
> There is no point to emit DW_AT_comp_dir if all filenames in the file/directory
> table referenced by DW_AT_stmt_list are absolute.  Seems to be a request to
> work around broken vendor tools again.

Below is the explanation from the darwin linker developer on the current
requirement of the DW_AT_comp_dir for each object file...

If you run:
   nm -ap
on the final executable, you'll see the "debug  notes" which are stabs:

[/tmp/xcode44_bugv2/good_binary]> nm -map 20010124-1.x4

0000000000000000 - 00 0000    SO /Users/howarth/xcode44_bug/good_binary/
0000000000000000 - 00 0000    SO 20010124-1.c
000000004fb8299f - 03 0001   OSO
/private/tmp/xcode44_bugv2/good_binary/20010124-1.o

For some historical reason, stabs like to have two SO entries, one for the dir
and one for the leaf file.  That happened to line up exactly with how dwarf
compilation info was emitted.  So the linker just transformed the AT_name into
one SO and the AT_comp_dir into the other, but only if both attributes existed.

Now gcc has changed the dwarf emitted and the fallback for the linker is to not
emit any debug notes for that object file.

Yes, we could enhance a future linker to be more robust and if there is not
AT_comp_dir to split up the AT_name in to it directory and name.  But existing
darwin linkers will create binaries without debug information.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-05-22 23:32:34 UTC ---
gcc hasn't changed the emitted DWARF in this case for more than 12 years.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #6 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-24 13:53:34 UTC ---
(In reply to comment #5)
> gcc hasn't changed the emitted DWARF in this case for more than 12 years.

Actually Apple gcc-4.2.1 doesn't seem to emit AT_comp_dir either but both
llvm-gcc-4.2 and clang do. Since Apple has deprecated support for the legacy
non-llvm based compilers, I believe they are expected the same dwarf output as
from llvm based compilers for the linker now. This means that both AT_name and
AT_comp_dir have to be present in the TAG_compile_unit section of each object
file.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #7 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-24 19:35:09 UTC ---
Are we really sure that gen_compile_unit_die() shouldn't being emitting calling
add_comp_dir_attribute  in this case. It seems that the origin of test...

      /* Don't add cwd for <built-in>.  */
      if (!IS_ABSOLUTE_PATH (filename) && filename[0] != '<')
        add_comp_dir_attribute (die);

Dates back to...

Author: amylaar
Date:   Wed Jun 4 17:19:36 2003 UTC (8 years, 11 months ago)
Changed paths:  3
Log Message:
        * c-decl.c (c_init_decl_processing): Clear input_file_name
        while building common nodes.
        * dwarf2out.c (gen_compile_unit_die, dwarf2out_finish):
        Don't add working directory for strings like <built-in> .

The reasoning for that change is described in the patch proposal at
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00149.html
which says...

I have found that pch/system-1.c fails on sh-elf because
the function that is called by BUILD_VA_LIST_TYPE generates
some declarations which end up in the debugging output for
this testcase.  toplev.c has previously set the input filename
to that of the main file - which is different when you compile
a header than when you do an ordinary source compilation.
This causes an anadorned "system-1.h" to be considered among
the source files in the pch case (even though it does nothing
but include other header files).  When dwarf2out sees an input
file without a dir separator, it puts the current working directory
in the list of directories to search.

The patch was revised a number of times...

http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00289.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00151.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00279.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00284.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00289.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00301.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00303.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00308.html
http://gcc.gnu.org/ml/gcc-patches/2003-06/msg00337.html

Reading through reviews doesn't give much confidence that this code is test is
bullet-proof.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #8 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-24 19:52:04 UTC ---
Are we sure this test for...

      /* Don't add cwd for <built-in>.  */
      if (!IS_ABSOLUTE_PATH (filename) && filename[0] != '<')
        add_comp_dir_attribute (die);

is still required? The original description says "When dwarf2out sees an input
file without a dir separator, it puts the current working directory
in the list of directories to search.". However this change predates....

r70189
Author: aoliva
Date:   Tue Aug 5 21:15:57 2003 UTC (8 years, 9 months ago)
Changed paths:  17
Log Message:
* c.opt: Introduce -fworking-directory.
* doc/cpp.texi, doc/invoke.texi, doc/cppopts.texi: Document it.
* c-common.h (flag_working_directory): Declare.
* c-common.c (flag_working_directory): Define.
* c-opts.c (c_common_handle_options): Set it.
(sanitize_cpp_opts): Set...
* cpplib.h (struct cpp_options): ... working_directory option.
(struct cpp_callbacks): Add dir_change.
* cppinit.c (read_original_filename): Call...
(read_original_directory): New.  Look for # 1 "directory//"
and process it.
(cpp_read_main_file): Call dir_change callback if working_directory
option is set.
* gcc.c (cpp_unique_options): Pass -g*.
* c-lex.c (cb_dir_change): New.
(init_c_lex): Set dir_change callback.
* toplev.c (src_pwd): New static variable.
(set_src_pwd, get_src_pwd): New functions.
* toplev.h (get_src_pwd, set_src_pwd): Declare.
* dbxout.c (dbxout_init): Call get_src_pwd() instead of getpwd().
* dwarf2out.c (gen_compile_unit_die): Likewise.
* dwarfout.c (output_compile_unit_die, dwarfout_init): Likewise.      

which changed...

--- trunk/gcc/dwarf2out.c       2003/08/01 21:51:13     70072
+++ trunk/gcc/dwarf2out.c       2003/08/05 21:15:57     70189
@@ -9506,7 +9506,7 @@
 static void
 add_comp_dir_attribute (dw_die_ref die)
 {
-  const char *wd = getpwd ();
+  const char *wd = get_src_pwd (); 
   if (wd != NULL)
     add_AT_string (die, DW_AT_comp_dir, wd);
 }

so that DW_AT_comp_dir pointed at the source directory rather than at the
working directory. Might that not eliminate dwarf2out putting the current
working directory
in the list of directories to search? Perhaps the hack...

      /* Don't add cwd for <built-in>.  */
      if (!IS_ABSOLUTE_PATH (filename) && filename[0] != '<')

is no longer required post revision 70189.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-05-24 20:12:52 UTC ---
It is correct and desirable.  If you really need and we want to continue the
endless stream of workarounds for Darwin toolchain bugs, then you'd add some
target macro TARGET_FORCE_AT_COMP_DIR or similar, define it in darwin.h and if
non-zero, force addition of the attribute even when the filename is absolute.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #10 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-24 23:22:21 UTC ---
Created attachment 27494
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=27494
proposed patch to emit AT_comp_dir on darwin using target hook

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #11 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-24 23:26:22 UTC ---
Proposed patch bootstraps on x86_64-apple-darwin12 under Xcode 4.4 and
eliminates the dsymutil warnings for...

gcc-fsf-4.8 
/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120524/gcc/testsuite/gcc.c-torture/execute/builtins/20010124-1.c
/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120524/gcc/testsuite/gcc.c-torture/execute/builtins/20010124-1-lib.c
/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120524/gcc/testsuite/gcc.c-torture/execute/builtins/lib/main.c
-fno-diagnostics-show-caret --save-temps -v -w -O3 -g -lm -m64 -o 20010124-1.x4

with the missing AT_comp_dir attribute now being emitted on darwin...


dwarfdump --debug-info main.o
---------------------------------------------------------------------- File:
main.o
(x86_64)----------------------------------------------------------------------
.debug_info contents:

0x00000000: Compile Unit: length = 0x00000178  version = 0x0002  abbr_offset =
0x00000000  addr_size = 0x08  (next CU at 0x0000017c)

0x0000000b: TAG_compile_unit [1] *
             AT_producer( "GNU C 4.8.0 20120524 (experimental) -fpreprocessed
-fPIC -feliminate-unused-debug-symbols -mmacosx-version-min=10.8.0 -m64
-mtune=core2 -g -O3" )
             AT_language( DW_LANG_C89 )
             AT_name(
"/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120524/gcc/testsuite/gcc.c-torture/execute/builtins/lib/main.c"
)
             AT_comp_dir( "/Users/howarth/xcode44_bugv2/bad_binary" )
             AT_stmt_list( 0x00000000 )

0x00000135:     TAG_subprogram [2]  
                 AT_external( 0x01 )
                 AT_name( "main" )
                 AT_decl_file(
"/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120524/gcc/testsuite/gcc.c-torture/execute/builtins/lib/main.c"
)
                 AT_decl_line( 7 )
                 AT_type( {0x00000156} ( int ) )
                 AT_low_pc( 0x0000000000000000 )
                 AT_high_pc( 0x000000000000001d )
                 AT_frame_base( 0x00000000
                    0x0000000000000000 - 0x0000000000000001: rsp+8
                    0x0000000000000001 - 0x000000000000001c: rsp+16
                    0x000000000000001c - 0x000000000000001d: rsp+8 )

0x00000156:     TAG_base_type [3]  
                 AT_byte_size( 0x04 )
                 AT_encoding( DW_ATE_signed )
                 AT_name( "int" )

0x0000015d:     TAG_variable [4]  
                 AT_name( "inside_main" )
                 AT_decl_file(
"/sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20120524/gcc/testsuite/gcc.c-torture/execute/builtins/lib/main.c"
)
                 AT_decl_line( 4 )
                 AT_type( {0x00000156} ( int ) )
                 AT_external( 0x01 )
                 AT_location( [0x0000000000000000] )

0x0000017b:     NULL

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[mrs at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

mrs@gcc.gnu.org <mrs at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mrs at gcc dot gnu.org

--- Comment #12 from mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> 2012-05-25 05:53:19 UTC ---
Darwin bits are Ok.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #13 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-25 12:53:17 UTC ---
Regression test results on x86_64-apple-darwin12 are at
http://gcc.gnu.org/ml/gcc-testresults/2012-05/msg02331.html. Note that the
remaining cfstring failures are PR53283.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #14 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-05-25 13:38:12 UTC ---
Revised patch with Jakub's documentation correction posted at
http://gcc.gnu.org/ml/gcc-patches/2012-05/msg01710.html.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[mrs at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #15 from mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> 2012-05-29 23:44:13 UTC ---
Author: mrs
Date: Tue May 29 23:44:09 2012
New Revision: 187994

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=187994
Log:
2012-05-29  Jack Howarth  <howarth@bromo.med.uc.edu>

    PR debug/53453
    * doc/tm.texi: Update.
    * doc/tm.texi.in (SDB and DWARF) <TARGET_FORCE_AT_COMP_DIR>: Add @hook.
    * target.def (force_at_comp_dir): New hook.
    * config/darwin.h (TARGET_FORCE_AT_COMP_DIR): Define.
    * dwarf2out.c (dwarf2out_finish): Check targetm.force_at_comp_dir.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/config/darwin.h
    trunk/gcc/doc/tm.texi
    trunk/gcc/doc/tm.texi.in
    trunk/gcc/dwarf2out.c
    trunk/gcc/target.def

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[mrs at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

mrs@gcc.gnu.org <mrs at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
      Known to work|                            |4.8.0
         Resolution|                            |FIXED
      Known to fail|                            |4.7.0

--- Comment #16 from mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> 2012-05-30 00:25:36 UTC ---
Fixed.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[ebotcazou at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ebotcazou at gcc dot
                   |                            |gnu.org

--- Comment #17 from Eric Botcazou <ebotcazou at gcc dot gnu.org> 2013-04-17 07:17:18 UTC ---
The patch was silently backported yesterday, but the wrong ChangeLog has been
modified.  Please post a message on gcc-patches@ and fix the ChangeLog.  TIA.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[mrs at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

mrs@gcc.gnu.org <mrs at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |4.7.4

--- Comment #18 from mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> 2013-04-17 15:55:25 UTC ---
Fixed the ChangeLog, thanks for spotting it.

[Bug debug/53453] darwin linker expects both AT_name and AT_comp_dir debug notes

[mrs at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53453

--- Comment #19 from mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> 2013-04-17 16:21:39 UTC ---
I've sent a message to the gcc-patches list with a pointer to the gcc-patches
list for the work.