Re: LLVM collaboration?

[Hal Finkel <hfinkel@anl.gov>]

[-- Attachment #1: Type: text/plain, Size: 3079 bytes --]

----- Original Message -----
> From: "Rafael Espíndola" <rafael.espindola@gmail.com>
> To: "Jan Hubicka" <hubicka@ucw.cz>
> Cc: "Renato Golin" <renato.golin@linaro.org>, "gcc" <gcc@gcc.gnu.org>, "Hal Finkel" <hfinkel@anl.gov>
> Sent: Tuesday, February 11, 2014 3:38:40 PM
> Subject: Re: Fwd: LLVM collaboration?
> 
> > My reading of bfd/plugin.c is that it basically walks the directory
> > and looks
> > for first plugin that returns OK for onload. (that is always the
> > case for
> > GCC/LLVM plugins).  So if I instlal GCC and llvm plugin there it
> > will
> > depend who will end up being first and only that plugin will be
> > used.
> >
> > We need multiple plugin support as suggested by the directory name
> > ;)
> >
> > Also it sems that currently plugin is not used if file is ELF for
> > ar/nm/ranlib
> > (as mentioned by Markus) and also GNU-ld seems to choke on LLVM
> > object files
> > even if it has plugin.
> >
> > This probably needs ot be sanitized.
> 
> CCing Hal Finkel. He got this to work some time ago. Not sure if he
> ever ported the patches to bfd trunk.

I have a patch for binutils 2.24 (attached -- I think this works, I hand isolated it from my BG/Q patchset). I would not consider it to be of upstream quality, but I'd obviously appreciate any assistance on making everything clean and proper ;)

 -Hal

> 
> >> For OS X the situation is a bit different. There instead of a
> >> plugin
> >> the linker loads a library: libLTO.dylib. When doing LTO with a
> >> newer
> >> llvm, one needs to set DYLD_LIBRARY_PATH. I think I proposed
> >> setting
> >> that from clang some time ago, but I don't remember the outcome.
> >>
> >> In theory GCC could implement a libLTO.dylib and set
> >> DYLD_LIBRARY_PATH. The gold/bfd plugin that LLVM uses is basically
> >> a
> >> API mapping the other way, so the job would be inverting it. The
> >> LTO
> >> model ld64 is a bit more strict about knowing all symbol
> >> definitions
> >> and uses (including inline asm), so there would be work to be done
> >> to
> >> cover that, but the simple cases shouldn't be too hard.
> >
> > I would not care that much about symbols in asm definitions to
> > start with.
> > Even if we will force users to non-LTO those object files, it would
> > be an
> > improvement over what we have now.
> >
> > One problem is that we need a volunteer to implement the reverse
> > glue
> > (libLTO->plugin API), since I do not have an OS X box (well, have
> > an old G5,
> > but even that is quite far from me right now)
> >
> > Why complete symbol tables are required? Can't ld64 be changed to
> > ignore
> > unresolved symbols in the first stage just like gold/gnu-ld does?
> 
> I am not sure about this. My *guess* is that it does dead stripping
> computation before asking libLTO for the object file. I noticed the
> issue while trying to LTO firefox some time ago.
> 
> Cheers,
> Rafael
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: binutils-2.24-ld-plugin.patch --]
[-- Type: text/x-patch; name="binutils-2.24-ld-plugin.patch", Size: 3087 bytes --]

diff --git a/bfd/elflink.c b/bfd/elflink.c
index 99b7ca1..c2bf9c3 100644
--- a/bfd/elflink.c
+++ b/bfd/elflink.c
@@ -5054,7 +5054,9 @@ elf_link_add_archive_symbols (bfd *abfd, struct bfd_link_info *info)
 	    goto error_return;
 
 	  if (! bfd_check_format (element, bfd_object))
-	    goto error_return;
+	    /* goto error_return; */
+            /* this might be an object understood only by an LTO plugin */
+            bfd_elf_make_object (element);
 
 	  /* Doublecheck that we have not included this object
 	     already--it should be impossible, but there may be
diff --git a/ld/ldfile.c b/ld/ldfile.c
index 16baef8..159a60c 100644
--- a/ld/ldfile.c
+++ b/ld/ldfile.c
@@ -38,6 +38,7 @@
 #ifdef ENABLE_PLUGINS
 #include "plugin-api.h"
 #include "plugin.h"
+#include "elf-bfd.h"
 #endif /* ENABLE_PLUGINS */
 
 bfd_boolean  ldfile_assumed_script = FALSE;
@@ -124,6 +125,7 @@ bfd_boolean
 ldfile_try_open_bfd (const char *attempt,
 		     lang_input_statement_type *entry)
 {
+  int is_obj = 0;
   entry->the_bfd = bfd_openr (attempt, entry->target);
 
   if (verbose)
@@ -168,6 +170,34 @@ ldfile_try_open_bfd (const char *attempt,
 	{
 	  if (! bfd_check_format (check, bfd_object))
 	    {
+#ifdef ENABLE_PLUGINS
+	      if (check == entry->the_bfd
+		  && bfd_get_error () == bfd_error_file_not_recognized
+		  && ! ldemul_unrecognized_file (entry))
+		{
+                  if (plugin_active_plugins_p ()
+                      && !no_more_claiming)
+                    {
+                      int fd = open (attempt, O_RDONLY | O_BINARY);
+                      if (fd >= 0)
+                        {
+                          struct ld_plugin_input_file file;
+
+                          bfd_elf_make_object (entry->the_bfd);
+ 
+                          file.name = attempt;
+                          file.offset = 0;
+                          file.filesize = lseek (fd, 0, SEEK_END);
+                          file.fd = fd;
+                          plugin_maybe_claim (&file, entry);
+
+                          if (entry->flags.claimed)
+                            return TRUE;
+                        }
+                    }
+                }
+#endif /* ENABLE_PLUGINS */
+
 	      if (check == entry->the_bfd
 		  && entry->flags.search_dirs
 		  && bfd_get_error () == bfd_error_file_not_recognized
@@ -303,7 +333,9 @@ success:
      bfd_object that it sets the bfd's arch and mach, which
      will be needed when and if we want to bfd_create a new
      one using this one as a template.  */
-  if (bfd_check_format (entry->the_bfd, bfd_object)
+  if (((is_obj = bfd_check_format (entry->the_bfd, bfd_object))
+       || (bfd_get_format(entry->the_bfd) == bfd_unknown
+           && bfd_get_error () == bfd_error_file_not_recognized))
       && plugin_active_plugins_p ()
       && !no_more_claiming)
     {
@@ -312,6 +344,9 @@ success:
 	{
 	  struct ld_plugin_input_file file;
 
+          if (!is_obj)
+            bfd_elf_make_object (entry->the_bfd);
+
 	  file.name = attempt;
 	  file.offset = 0;
 	  file.filesize = lseek (fd, 0, SEEK_END);