[Bug other/32998] New: -frecord-gcc-switches issues

[Bug other/32998] New: -frecord-gcc-switches issues

[jakub at gcc dot gnu dot org]

./xgcc -B ./ -frecord-gcc-switches -D_GNU_SOURCE -O2 a.c -S
gives me:
        .section        .GCC.command.line,"MS",@progbits,1
        .ascii  "-iprefix /usr/src/gcc/obj/gcc/../lib/gcc/x86_64-unknown-linu"
        .ascii  "x-gnu/4.3.0/"
        .zero   1
        .ascii  "-isystem ./include"
        .zero   1
        .ascii  "-isystem ./include-fixed"
        .zero   1
        .ascii  "-D_GNU_SOURCE a.c"
        .zero   1
        .ascii  "-mtune=generic"
        .zero   1
        .ascii  "-O2"
        .zero   1
        .ascii  "-frecord-gcc-switches"

The main issue I have with this is the separation of options into separate
strings (though, look above at "-D_GNU_SOURCE a.c" string to see it is not
done always anyway).  This means smaller .GNU.command.line section, sure, but
makes the section far less useful.  Because if you link shared library
or binary with many -frecord-gcc-switches compiled objects, all you can query
is e.g. was at least one of the input objects compiled with -O2, or whatever
other option.  On the other side, if you have one big string with all command
line options (but one that should not reference the source file name and IMNSHO
not even any of the -I/-iprefix/-isystem options, that's what .debug_line is
for and in this section where you no longer know which filename was it and
which directory was it in I can't see what it would be useful for), then you
can query whether each source was compiled with -fstack-protector or not, etc.
With the long strings you still can have the longer option strings from all
objects merged into just one string (if all sources were compiled with the same
options).
-frecord-gcc-switches among the saved options doesn't make sense either (if it
was not used, they wouldn't be recorded).  Not sure if we shouldn't try to do
any kind of sorting among unrelated non-positional options, it would certainly
be harded (but guess in *.opt we know everything we need), but perhaps would
result in better merging.  Say if you compile some sources with -O2
-fstack-protector and some with -fstack-protector -O2, if we do sorting it
could be merged, otherwise it can't.  E.g. -fstrict-aliasing
-fno-strict-aliasing can't be reordered.


-- 
           Summary: -frecord-gcc-switches issues
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: other
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: jakub at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[nickc at redhat dot com]

------- Comment #1 from nickc at redhat dot com  2007-08-06 08:12 -------
Subject: Re:   New: -frecord-gcc-switches issues

Hi Jakub,

>         .ascii  "-isystem ./include-fixed"
>         .zero   1
>         .ascii  "-D_GNU_SOURCE a.c"
>         .zero   1

> The main issue I have with this is the separation of options into separate
> strings (though, look above at "-D_GNU_SOURCE a.c" string to see it is not
> done always anyway).

   Right - I could not find 100% effective heuristic to work out when a "word" 
on the command line was the argument to a previous command line option or a 
separate option in its own right.  The current heuristic assumes that if the 
word does not start with a dash, then it is an argument to previous word. Hence 
"./include-fixed" is correctly taken as an argument to "-isystem" but "a.c" is 
mistakenly taken as an argument to "-D_GNU_SOURCE".

   I think that in order to fix this the .GCC.command.line section creation 
code will have to be made more complex and have access to the entire command 
line options table.  When I submitted the original patch to create this feature 
I wanted to keep things simple, so I did not try to do this.


> On the other side, if you have one big string with all command
> line options (but one that should not reference the source file name
[snip]
 > ) then you can query whether each source was compiled with -fstack-protector
> or not, etc.

Except that you will not have the source file names in the strings.  So if one 
or more of the component objects were not compiled with -fstack-protector you 
then have to go through several more hoops in order to find out which ones they 
are.

> (but one that should not reference the source file name and IMNSHO
> not even any of the -I/-iprefix/-isystem options, that's what .debug_line is
> for and in this section where you no longer know which filename was it and
> which directory was it in I can't see what it would be useful for)

Personally I think that the pathname information (source files and include 
directories) should be left in, since it is pertinent to how the object file 
was created.  (Yes you can track this information down from the .debug_line 
section, but that is more hassle if you are already looking at the 
.GCC.command.line section for the other options used to make the object file).

I agree that relative pathnames in this information may not be very useful, but 
it should be possible to convert all relative paths into absolute paths as the 
.GCC.command.line section is generated.


> -frecord-gcc-switches among the saved options doesn't make sense either (if it
> was not used, they wouldn't be recorded).

True.  But I was working on the keep-it-simple principle when I wrote the code. 
  Also it seems to me that not displaying it would be opening the door to 
censoring the command line.  If we do not record the -frecord-gcc-switches 
option then we have established the principle that some options are not 
recorded.  So maybe the -I options should be skipped as well.  Then maybe the 
-O switches can be ignored (everyone uses optimization, right ?)  I am 
exaggerating of course, but my point is, once we start censoring the command 
line, the consumers of .GCC.command.line can no longer trust it to contain a 
full and accurate record of how the object file was compiled.

As a theoretical example of why this might be important, suppose that there was 
a bug whereby cc1 would ignore any command line options beyond argv[511], 
because of some system limitation.  If we did not record the full command line 
in the .GCC.command.line section then someone using the information in the 
section to try to reproduce the bug might not be able to do so, because their 
command line would be shorter.


Having said all that I am happy to see this feature improved
so please do submit patches for it.

Cheers
   Nick


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[bkoz at gcc dot gnu dot org]

------- Comment #2 from bkoz at gcc dot gnu dot org  2007-08-06 16:16 -------

thanks for adding this bug report here and ccing me.

Is there an easy way to separate out the include and link (-I, -L) bits from
the macro defines and compiler option flags? Could the just the include bits be
put into one string?

When doing this does it make sense to define the base_dir and then use it as a
substitution instead of putting in absolute addresses everywhere? This might
cut down on size.

>  I think that in order to fix this the .GCC.command.line section creation 
> code will have to be made more complex and have access to the entire command 
> line options table.  When I submitted the original patch to create this 
> feature I wanted to keep things simple, so I did not try to do this.

This seems like a pretty smart strategy for getting this in.

However.... can you expand on your comment above? What do you mean by "have
access to the entire command line options table?" Would you dump the entire
table? 

-benjamin


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[roland at redhat dot com]

------- Comment #3 from roland at redhat dot com  2007-08-06 19:19 -------
Absolute file names are a very bad idea.  That makes for gratuitous differences
in builds due to the build or source directory name, i.e. unrepeatable builds. 
The names in .debug_line and .debug_info are already expected by
post-processing tools and taken care of.  Do not add another location in the
object that might contain absolute file names.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[nickc at redhat dot com]

------- Comment #4 from nickc at redhat dot com  2007-08-07 08:11 -------
Subject: Re:  -frecord-gcc-switches issues

Hi Roland,

> Absolute file names are a very bad idea.  That makes for gratuitous differences
> in builds due to the build or source directory name, i.e. unrepeatable builds. 
> The names in .debug_line and .debug_info are already expected by
> post-processing tools and taken care of.  Do not add another location in the
> object that might contain absolute file names.

Ok. :-}  I think at the moment we are just discussing how the 
-frecord-gcc-switches feature can be improved.  Nothing is set in stone.

Cheers
   Nick


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[nickc at redhat dot com]

------- Comment #5 from nickc at redhat dot com  2007-08-07 08:25 -------
Subject: Re:  -frecord-gcc-switches issues

Hi Ben,

> Is there an easy way to separate out the include and link (-I, -L) bits from
> the macro defines and compiler option flags? Could the just the include bits be
> put into one string?

Sure.  We could add some intelligence to the recording function which looks for 
these switches and groups them all together.  There are quite a lot of them 
though: -B, -I, -J, -L, -iquote, -isystem, -include, -imacros, -idirafter, 
-iwithprefixbefore, -imultilib, -isysroot.

> When doing this does it make sense to define the base_dir and then use it as a
> substitution instead of putting in absolute addresses everywhere? This might
> cut down on size.

Well that sounds like a good idea to me.  But Roland was objecting to recording 
absolute paths of any kind, so he might not like this.

>>  I think that in order to fix this the .GCC.command.line section creation 
>> code will have to be made more complex and have access to the entire command 
>> line options table.

> However.... can you expand on your comment above? What do you mean by "have
> access to the entire command line options table?" Would you dump the entire
> table? 

Oh no.  What I meant was that in order for the recording code to be more 
intelligent it would need access to gcc's command line options structure, so 
that, for example, it can tell when a switch takes an argument (and hence 
correctly deduce whether an entry in the argv[] array should be included in the 
same string as the previous entry or if it is a new command line switch in its 
own right).  Access to this table would also make it possible to do things
like:

   * group all the optimization switches together in one part of the 
.GCC.command.line section.

   * locate switches which negate the effect of previous switches and then skip 
recording those previous switches.

My concern however is that adding this sort of thing complicates the code, and 
hence is more likely to introduce bugs.  Still if it makes the feature useful, 
then it is worth considering.

Cheers
   Nick


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[jsm28 at gcc dot gnu dot org]

------- Comment #6 from jsm28 at gcc dot gnu dot org  2010-06-20 21:03 -------
Subject: Bug 32998

Author: jsm28
Date: Sun Jun 20 21:02:46 2010
New Revision: 161053

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=161053
Log:
        PR other/32998
        * opth-gen.awk: Generate definitions of OPT_SPECIAL_unknown,
        OPT_SPECIAL_program_name and OPT_SPECIAL_input_file.
        * opts-common.c (find_opt): Return OPT_SPECIAL_unknown on failure.
        (decode_cmdline_option): Update for this return value.  Set
        orig_option_with_args_text field.  Set arg field for unknown
        options.  Make static.
        (decode_cmdline_options_to_array): New.
        (prune_options): Update handling of find_opt return value.
        * opts.c (read_cmdline_option): Take decoded option.  Return void.
        (read_cmdline_options): Take decoded options.
        (decode_options): Add parameters for decoded options.  Use
        decode_cmdline_options_to_array.  Use decoded options for -O
        scan.  Use integral_argument for -O parameters.  Update call to
        read_cmdline_options.
        (enable_warning_as_error): Update handling of find_opt return
        value.
        * opts.h: Update comment on unknown options.
        (struct cl_decoded_option): Update comments on opt_index and arg.
        Add orig_option_with_args_text.
        (decode_cmdline_option): Remove.
        (decode_cmdline_options_to_array): Declare.
        (decode_options): Update prototype.
        * toplev.c (save_argv): Remove.
        (save_decoded_options, save_decoded_options_count): New.
        (read_integral_parameter): Remove.
        (print_switch_values): Use decoded options.
        (toplev_main): Don't set save_argv.  Update call to
        decode_options.
        * toplev.h (read_integral_parameter): Remove.
        * varasm.c (elf_record_gcc_switches): Don't handle holding back
        names.

c-family:
        * c-common.c (parse_optimize_options): Update call to
        decode_options.

fortran:
        * options.c (gfc_handle_option): Don't handle N_OPTS.

testsuite:
        * gcc.dg/opts-2.c: New test.

Added:
    trunk/gcc/testsuite/gcc.dg/opts-2.c
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/c-family/ChangeLog
    trunk/gcc/c-family/c-common.c
    trunk/gcc/fortran/ChangeLog
    trunk/gcc/fortran/options.c
    trunk/gcc/opth-gen.awk
    trunk/gcc/opts-common.c
    trunk/gcc/opts.c
    trunk/gcc/opts.h
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/toplev.c
    trunk/gcc/toplev.h
    trunk/gcc/varasm.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[jsm28 at gcc dot gnu dot org]

------- Comment #7 from jsm28 at gcc dot gnu dot org  2010-06-20 21:05 -------
The "-D_GNU_SOURCE a.c" issue is now fixed.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[manu at gcc dot gnu dot org]

------- Comment #8 from manu at gcc dot gnu dot org  2010-06-20 21:20 -------
I think this is pretty much confirmed.


-- 

manu at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2010-06-20 21:20:44
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

[Bug other/32998] -frecord-gcc-switches issues

[jan.kratochvil at redhat dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #9 from Jan Kratochvil <jan.kratochvil at redhat dot com> 2010-10-02 16:06:39 UTC ---
Wouldn't be appropriate to append these flags also/instead to DW_AT_producer?
This way they get easily associated with the specific CU.

[Bug other/32998] -frecord-gcc-switches issues

[roland at redhat dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #10 from Roland McGrath <roland at redhat dot com> 2010-10-05 00:41:45 UTC ---
(In reply to comment #9)
> Wouldn't be appropriate to append these flags also/instead to DW_AT_producer?
> This way they get easily associated with the specific CU.

That makes some sense.  It also should get the strings nicely merged by the
linker in the .debug_str section.

[Bug other/32998] -frecord-gcc-switches issues

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #11 from Jakub Jelinek <jakub at gcc dot gnu.org> 2010-10-05 09:48:32 UTC ---
I agree, it would be a good idea to append all preprocessing and compilation
options, except for -I/-i* paths (and leave out driver/link options like -B,
-V, -L, -Wl, too), to DW_AT_producer; it would solve the issue that -g and -g0
compiled objects wouldn't result in different object files.

[Bug other/32998] -frecord-gcc-switches issues

[roland at redhat dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #12 from Roland McGrath <roland at redhat dot com> 2010-10-05 19:24:56 UTC ---
Preprocessing stuff is probably best left to the -g3 info instead.  cc1*
options make sense for DW_AT_producer.

[Bug other/32998] -frecord-gcc-switches issues

[jan.kratochvil at redhat dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #13 from Jan Kratochvil <jan.kratochvil at redhat dot com> 2011-04-22 18:49:06 UTC ---
CU specific -O flag indication is needed for proper prologue skipping:
  http://sourceware.org/bugzilla/show_bug.cgi?id=12573
  [rfc, 7.3?] -O2 -g breakpoints internal error + prologue skipping
  http://sourceware.org/ml/gdb-patches/2011-04/msg00229.html

Currently a loclist reference heuristic is implemented which occasionally
fails.

[Bug other/32998] -frecord-gcc-switches issues

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #14 from Jakub Jelinek <jakub at gcc dot gnu.org> 2011-07-12 16:26:10 UTC ---
Created attachment 24746
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=24746
gcc47-pr32998.patch

Untested patch to implement a new switch, -grecord-gcc-switches, which records
passed options as a string appended to DW_AT_producer attribute in .debug_info.
Apparently we aren't the first compiler to do that based on the dwarf2out.c
comments...
The main intent is to have the code generation options in there, as Roland
said, -D* etc. options should be investigable through -g3 .debug_macinfo
(especially if we manage to cut it down considerably), similarly -I/-i* etc.
options related to preprocessing are either in .debug_macinfo and/or in
.debug_line).  If possible I'd like not to see filenames and paths in the
string.

I'm not entirely happy with it this way, as if passed command line options
override each other, this approach will print all of them in the order
specified
(say -O2 -O3 -O0 inserts -O2 -O3 -O0 instead of just -O0) and due to the
overriding the options can't be sorted, which would increase the likelyhood
that the DW_AT_producer strings are mergeable between many object files.

Joseph, is there a way to instead iterate over the explicitly passed options
after override processing?  I see global_options_set, but not sure what exactly
to expect from it.

[Bug other/32998] -frecord-gcc-switches issues

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

--- Comment #15 from Jakub Jelinek <jakub at gcc dot gnu.org> 2011-07-22 20:03:35 UTC ---
Author: jakub
Date: Fri Jul 22 20:03:33 2011
New Revision: 176652

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=176652
Log:
    PR other/32998
    * common.opt (grecord-gcc-switches, gno-record-gcc-switches): New
    options.
    * dwarf2out.c: Include opts.h.
    (dchar_p): New typedef.  Define heap VEC for it.
    (producer_string): New variable.
    (gen_producer_string): New function.
    (gen_compile_unit_die): Use it.
    (dwarf2out_finish): Fix up comp_unit_die () DW_AT_producer
    if needed.
    * Makefile.in (dwarf2out.o): Depend on $(OPTS_H).
    * doc/invoke.texi: Document -grecord-gcc-switches and
    -gno-record-gcc-switches, add a -grecord-gcc-switches reference
    to -frecord-gcc-switches description.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/Makefile.in
    trunk/gcc/common.opt
    trunk/gcc/doc/invoke.texi
    trunk/gcc/dwarf2out.c

[Bug other/32998] -frecord-gcc-switches issues

[philomath868 at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32998

philomath <philomath868 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |philomath868 at gmail dot
                   |                            |com

--- Comment #16 from philomath <philomath868 at gmail dot com> 2012-06-14 09:11:25 UTC ---
The wording in the manual is incorrect.  this option does not record "the
command line that was used to invoke the compiler", but the options that where
in affect while compiling.  two simple examples:

~ $ gcc -fno-omit-frame-pointer -o tst tst.c -frecord-gcc-switches
-fomit-frame-pointer

~ $ objdump -s --section=.GCC.command.line tst

tst:     file format elf64-x86-64

Contents of section .GCC.command.line:
 0000 7473742e 63002d6d 74756e65 3d67656e  tst.c.-mtune=gen
 0010 65726963 002d6d61 7263683d 7838362d  eric.-march=x86-
 0020 3634002d 7374643d 63393900 2d667265  64.-std=c99.-fre
 0030 636f7264 2d676363 2d737769 74636865  cord-gcc-switche
 0040 73002d66 6f6d6974 2d667261 6d652d70  s.-fomit-frame-p
 0050 6f696e74 657200                      ointer.

~ $ gcc -o tst tst.c -march=native -frecord-gcc-switches

~ $ objdump -s --section=.GCC.command.line tst

tst:     file format elf64-x86-64

Contents of section .GCC.command.line:
 0000 7473742e 63002d6d 61726368 3d636f72  tst.c.-march=cor
 0010 65693700 2d6d6378 3136002d 6d736

--- Comment #17 from philomath <philomath868 at gmail dot com> 2012-06-14 09:20:56 UTC ---
Sorry, the previous comment was interrupted while posting. here is it again.

The wording in the manual is incorrect.  this option does not record "the
command line that was used to invoke the compiler", but the options that where
in affect while compiling.  two simple examples:

~ $ gcc -fno-omit-frame-pointer -o tst tst.c -frecord-gcc-switches
-fomit-frame-pointer

~ $ objdump -s --section=.GCC.command.line tst

tst:     file format elf64-x86-64

Contents of section .GCC.command.line:
 0000 7473742e 63002d6d 74756e65 3d67656e  tst.c.-mtune=gen
 0010 65726963 002d6d61 7263683d 7838362d  eric.-march=x86-
 0020 3634002d 7374643d 63393900 2d667265  64.-std=c99.-fre
 0030 636f7264 2d676363 2d737769 74636865  cord-gcc-switche
 0040 73002d66 6f6d6974 2d667261 6d652d70  s.-fomit-frame-p
 0050 6f696e74 657200                      ointer.

~ $ gcc -o tst tst.c -march=native -frecord-gcc-switches

~ $ objdump -s --section=.GCC.command.line tst

tst:     file format elf64-x86-64

Contents of section .GCC.command.line:
 0000 7473742e 63002d6d 61726368 3d636f72  tst.c.-march=cor
 0010 65693700 2d6d6378 3136002d 6d736168  ei7.-mcx16.-msah
 0020 66002d6d 6e6f2d6d 6f766265 002d6d6e  f.-mno-movbe.-mn
 0030 6f2d6165 73002d6d 6e6f2d70 636c6d75  o-aes.-mno-pclmu
 0040 6c002d6d 706f7063 6e74002d 6d6e6f2d  l.-mpopcnt.-mno-
 0050 61626d00 2d6d6e6f 2d6c7770 002d6d6e  abm.-mno-lwp.-mn
 0060 6f2d666d 61002d6d 6e6f2d66 6d613400  o-fma.-mno-fma4.
 0070 2d6d6e6f 2d786f70 002d6d6e 6f2d626d  -mno-xop.-mno-bm
 0080 69002d6d 6e6f2d62 6d693200 2d6d6e6f  i.-mno-bmi2.-mno
 0090 2d74626d 002d6d6e 6f2d6176 78002d6d  -tbm.-mno-avx.-m
 00a0 6e6f2d61 76783200 2d6d7373 65342e32  no-avx2.-msse4.2
 00b0 002d6d73 7365342e 31002d6d 6e6f2d6c  .-msse4.1.-mno-l
 00c0 7a636e74 002d2d70 6172616d 206c312d  zcnt.--param l1-
 00d0 63616368 652d7369 7a653d33 32002d2d  cache-size=32.--
 00e0 70617261 6d206c31 2d636163 68652d6c  param l1-cache-l
 00f0 696e652d 73697a65 3d363400 2d2d7061  ine-size=64.--pa
 0100 72616d20 6c322d63 61636865 2d73697a  ram l2-cache-siz
 0110 653d3330 3732002d 6d74756e 653d636f  e=3072.-mtune=co
 0120 72656937 002d7374 643d6339 39002d66  rei7.-std=c99.-f
 0130 7265636f 72642d67 63632d73 77697463  record-gcc-switc
 0140 68657300                             hes.