Re: Fwd: LLVM collaboration?

[Uday Khedker <uday@cse.iitb.ac.in>]

On Tuesday 11 February 2014 03:25 PM, Renato Golin wrote:
> Hi Jan,
>
> I think this is a very good example where we could all collaborate
> (including binutils).
>
> I'll leave your reply intact, so that Chandler (CC'd) can get a bit
> more context. I'm copying him because he (and I believe Diego) had
> more contact with LTO than I had.
>
> If I got it right, LTO today:
>
> - needs the drivers to explicitly declare the plugin
> - needs the library available somewhere
> - may have to change the library loading semantics (via LD_PRELOAD)

There is another need that I have felt in LTO for quite some time. 
Currently, it has a non-partitioned mode or a partitioned mode but this 
decision is taken before the compilation begins. It would be nice to 
have a mode that allows dynamic loading of function bodies so that a 
flow and context sensitive IPA could load functions bodies on demand, 
and unload them when they are not needed.

Uday.

>
> Since both toolchains do the magic, binutils has no incentive to
> create any automatic detection of objects.
>
> The part that I didn't get is when you say about backward
> compatibility. Would LTO work on a newer binutils with the liblto but
> on an older compiler that knew nothing about LTO?
>
> Your proposal is, then, to get binutils:
>
> - recognizing LTO logic in the objects
> - automatically loading liblto if recognized
> - warning if not
>
> I'm assuming the extra symbols would be discarded if no library is
> found, together with the warning, right? Maybe an error if -Wall or
> whatever.
>
> Can we get someone from the binutils community to opine on that?
>
> cheers,
> --renato
>
> On 11 February 2014 02:29, Jan Hubicka <hubicka@ucw.cz> wrote:
>> One practical experience I have with LLVM developers is sharing experiences
>> about getting Firefox to work with LTO with Rafael Espindola and I think it was
>> useful for both of us. I am definitly open to more discussion.
>>
>> Lets try a specific topic that is on my TODO list for some time.
>>
>> I would like to make it possible for mutliple compilers to be used to LTO a
>> single binary. As we are all making LTO more useful, I think it is matter of
>> time until people will start shipping LTO object files by default and users
>> will end up feeding them into different compilers or incompatible version of
>> the same compiler. We probably want to make this work, even thought the
>> cross-module optimization will not happen in this case.
>>
>> The plugin interface in binutils seems to do its job well both for GCC and LLVM
>> and I hope that open64 and ICC will eventually join, too.
>>
>> The trouble however is that one needs to pass explicit --plugin argument
>> specifying the particular plugin to load and so GCC ships with its own wrappers
>> (gcc-nm/gcc-ld/gcc-ar and the gcc driver itself) while LLVM does similar thing.
>>
>> It may be smoother if binutils was able to load multiple plugins at once and
>> grab plugins from system and user installed compilers without explicit --plugin
>> argument.
>>
>> Binutils probably should also have a way to detect LTO object files and produce
>> more useful diagnostic than they do now, when there is no plugin claiming them.
>>
>> There are some PRs filled on the topic
>> http://cygwin.com/frysk/bugzilla/show_bug.cgi?id=15300
>> http://cygwin.com/frysk/bugzilla/show_bug.cgi?id=13227
>> but not much progress on them.
>>
>> I wonder if we can get this designed and implemented.
>>
>> On the other hand, GCC current maintains non-plugin path for LTO that is now
>> only used by darwin port due to lack of plugin enabled LD there.  It seems
>> that liblto used by darwin is losely compatible with the plugin API, but it makes
>> it harder to have different compilers share it (one has to LD_PRELOAD liblto
>> to different one prior executing the linker?)
>>
>> I wonder, is there chance to implement linker plugin API to libLTO glue or add
>> plugin support to native Darwin tools?
>>
>> Honza