how to "parse" gcc -v output

how to "parse" gcc -v output

[Ralf Wildenhues]

Hello gcc and libtool lists,

Both Autoconf and Libtool have macros which "parse" output of 'gcc -v'
for some link commands.  Yes, I know, almost by definition this is not
"parseable" text, but GCC does not provide a stable API that allows to
find out
- which libraries it needs to link shared modules against, or
- which libraries are needed to let g++ link mixed C++/Fortran code,
so in practice there is no other choice (unless the user provides this
information).

In some situations, esp. when -flto is used, -v output consists of many
parts.  Libtool needs to know which parts it can drop, or needs to look
at.  Can we be fairly confident that, iff there is a line matching
  '^ [^ ].*collect'

it will pick the one line that contains the verbose driver invocation
that lets us pick up init/fini objects as well as compiler libs to link
against?  I.e., can we assume the leading space to be stable, and the
'collect2' name to be fairly stable across GCC versions and systems?
It seems to work with all versions starting from 2.95 on those systems
where collect2 is used (the 2.7.2.3 I can still find here matches
'^ [^ ].*ld ').  If we don't find such a line, then picking the first
line with a single leading space seems prudent.  That would e.g. match
linking with ld on FreeBSD.

If not, can you suggest a different approach?  Should we be using -###
instead?  Looking for the first line with leading space and containing
' -L' (I think Alexandre Oliva suggested that a while ago)?

I'm willing to add API to a future GCC if that's the only viable way
(that way at least we have a fixed set of past and present versions to
match).

Thanks,
Ralf

Re: how to "parse" gcc -v output

[Joseph S. Myers]

On Sun, 4 Apr 2010, Ralf Wildenhues wrote:

> - which libraries it needs to link shared modules against, or
> - which libraries are needed to let g++ link mixed C++/Fortran code,
> so in practice there is no other choice (unless the user provides this
> information).
[...]
> I'm willing to add API to a future GCC if that's the only viable way
> (that way at least we have a fixed set of past and present versions to
> match).

For both of the above, at least on ELF systems, the correct thing to do is 
to make GCC (used by any of the drivers "gcc", "g++", etc.) do the right 
thing by default: link with the libraries required by the objects being 
linked in, using those appropriate for shared libraries if -shared is 
passed, and respecting options such as -static-libstdc++.  That is, make 
the driver/collect2 combination smart enough that libtool does not need to 
query this information.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: how to "parse" gcc -v output

[Ralf Wildenhues]

* Joseph S. Myers wrote on Sun, Apr 04, 2010 at 11:40:19AM CEST:
> On Sun, 4 Apr 2010, Ralf Wildenhues wrote:
> 
> > - which libraries it needs to link shared modules against, or
> > - which libraries are needed to let g++ link mixed C++/Fortran code,
> > so in practice there is no other choice (unless the user provides this
> > information).
> [...]
> > I'm willing to add API to a future GCC if that's the only viable way
> > (that way at least we have a fixed set of past and present versions to
> > match).
> 
> For both of the above, at least on ELF systems, the correct thing to do is 
> to make GCC (used by any of the drivers "gcc", "g++", etc.) do the right 
> thing by default: link with the libraries required by the objects being 
> linked in, using those appropriate for shared libraries if -shared is 
> passed, and respecting options such as -static-libstdc++.  That is, make 
> the driver/collect2 combination smart enough that libtool does not need to 
> query this information.

How would g++ know that part of the objects were from Fortran sources?
Maybe from a different (non-GCC) Fortran compiler, that needs a
different set of libraries at link time?  I don't see how this
information can fully be embedded in GCC.

Also, in order to actually be able to run programs, libtool needs to
also be able to ask for paths missing in the run paths, so they can be
added to the program *if the user so desires* and if they are not in the
default run path.

Until these are available, the other information are still useful to,
and needed for, Libtool and Autoconf.

Thanks,
Ralf

Re: how to "parse" gcc -v output

[Joseph S. Myers]

On Sun, 4 Apr 2010, Ralf Wildenhues wrote:

> How would g++ know that part of the objects were from Fortran sources?

By looking for undefined symbols matching patterns for symbols from 
libgfortran.

> Maybe from a different (non-GCC) Fortran compiler, that needs a
> different set of libraries at link time?  I don't see how this

I don't see how querying GCC is ever going to help with this case.

> Also, in order to actually be able to run programs, libtool needs to
> also be able to ask for paths missing in the run paths, so they can be
> added to the program *if the user so desires* and if they are not in the
> default run path.

(That is not a case mentioned in your original message so was not covered 
by my previous reply.)

The interface -print-search-dirs already exists; you can use it along with 
the multilib options to the compiler to get the list of directories it 
searches for the selected multilib.  That's better than parsing the 
commands by which the compiler runs the linker.  Both will involve a long 
list of paths, most of which are not relevant and involve ../ parts you'll 
want to resolve before embedding.

Embedding runpaths seems like a case for an ld option or options to 
specify which directories (in which shared libraries are found) do or do 
not get paths embedded and how those paths are remapped if they are 
embedded.

I think extracting compiler/linker *internal commands* and trying to 
process or adapt them is inherently fragile and liable to break whenever 
new compiler/linker options (internal or otherwise) are added.  If 
possible the aim should be to work out user-friendly interfaces for direct 
GCC users and have libtool use the same interfaces while expecting how 
they are implemented to change over time.  Interfaces by which GCC does 
things (e.g. link a shared library for the multilib implied by the given 
options) seem safer than interfaces where it gives information (if you ask 
it for directories and lists of libraries, you might then find that 
interface inadequate for handling per-library choice of static or shared 
libraries, for example).  Linking is a complicated iterative process that 
can involve collect2 rerunning the compiler, making it hard for the driver 
ever to give an answer about how something would be linked and be sure 
that it would still be valid when libtool makes some unspecified change to 
the link process (and if it isn't making such a change, it could just let 
the driver do the linking instead of asking it questions).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: how to "parse" gcc -v output

[Dave Korn]

On 04/04/2010 20:08, Joseph S. Myers wrote:

> I think extracting compiler/linker *internal commands* and trying to 
> process or adapt them is inherently fragile and liable to break whenever 
> new compiler/linker options (internal or otherwise) are added.  If 
> possible the aim should be to work out user-friendly interfaces for direct 
> GCC users and have libtool use the same interfaces while expecting how 
> they are implemented to change over time.  Interfaces by which GCC does 
> things (e.g. link a shared library for the multilib implied by the given 
> options) seem safer than interfaces where it gives information (if you ask 
> it for directories and lists of libraries, you might then find that 
> interface inadequate for handling per-library choice of static or shared 
> libraries, for example).  

  Essentially, libtool needs to know about gcc's specs, and what they do to a
command-line.  ISTM that using "-###" and the appropriate language-dependent
driver should do most things that libtool needs, but maybe we should add an
option to the driver that turns it into a command-line driven arbitrary specs
processor of some kind.  Ralf, might that help the situation, if you could
pass arbitrary command-lines to the driver and have it report back the results
of spec processing in some controlled and parseable fashion?

    cheers,
      DaveK