Re: Fwd: LLVM collaboration?

[Jan Hubicka <hubicka@ucw.cz>]

> >> Since both toolchains do the magic, binutils has no incentive to
> >> create any automatic detection of objects.
> 
> It is mostly a historical decision. At the time the design was for the
> plugin to be matched to the compiler, and so the compiler could pass
> that information down to the linker.
> 
> > The trouble however is that one needs to pass explicit --plugin argument
> > specifying the particular plugin to load and so GCC ships with its own wrappers
> > (gcc-nm/gcc-ld/gcc-ar and the gcc driver itself) while LLVM does similar thing.
> 
> These wrappers should not be necessary. While the linker currently
> requires a command line option, bfd has support for searching for a
> plugin. It will search <inst>/lib/bfd-plugin. See for example the
> instructions at http://llvm.org/docs/GoldPlugin.html.

My reading of bfd/plugin.c is that it basically walks the directory and looks
for first plugin that returns OK for onload. (that is always the case for
GCC/LLVM plugins).  So if I instlal GCC and llvm plugin there it will
depend who will end up being first and only that plugin will be used.

We need multiple plugin support as suggested by the directory name ;)

Also it sems that currently plugin is not used if file is ELF for ar/nm/ranlib
(as mentioned by Markus) and also GNU-ld seems to choke on LLVM object files
even if it has plugin.

This probably needs ot be sanitized.

> 
> This was done because ar and nm are not normally bound to any
> compiler. Had we realized this issue earlier we would probably have
> supported searching for plugins in the linker too.
> 
> So it seems that what you want could be done by
> 
> * having bfd-ld and gold search bfd-plugins (maybe rename the directory?)
> * support loading multiple plugins, and asking each to see if it
> supports a given file. That ways we could LTO when having a part GCC
> and part LLVM build.

Yes, that is what I have in mind.

Plus perhaps additional configuration file to avoid loading everything.  Say
user instealls 3 versions of LLVM, open64 and ICC. If all of them loads as a
shared library, like LLVM does, it will probably slow down the tools
measurably.

> * maybe be smart about version and load new ones first? (libLLVM-3.4
> before libLLVM-3.3 for example). Probably the first one should always
> be the one given in the command line.

Yes, i think we may want to prioritize the list.  So user can prevail
his own version of GCC over the system one, for example.
> 
> For OS X the situation is a bit different. There instead of a plugin
> the linker loads a library: libLTO.dylib. When doing LTO with a newer
> llvm, one needs to set DYLD_LIBRARY_PATH. I think I proposed setting
> that from clang some time ago, but I don't remember the outcome.
> 
> In theory GCC could implement a libLTO.dylib and set
> DYLD_LIBRARY_PATH. The gold/bfd plugin that LLVM uses is basically a
> API mapping the other way, so the job would be inverting it. The LTO
> model ld64 is a bit more strict about knowing all symbol definitions
> and uses (including inline asm), so there would be work to be done to
> cover that, but the simple cases shouldn't be too hard.

I would not care that much about symbols in asm definitions to start with.
Even if we will force users to non-LTO those object files, it would be an
improvement over what we have now.

One problem is that we need a volunteer to implement the reverse glue
(libLTO->plugin API), since I do not have an OS X box (well, have an old G5,
but even that is quite far from me right now)

Why complete symbol tables are required? Can't ld64 be changed to ignore
unresolved symbols in the first stage just like gold/gnu-ld does?

Honza
> 
> Cheers,
> Rafael