Updated slim-lto patchkit

Updated slim-lto patchkit

[Andi Kleen]

Updated version of the slim lto patchkit.

I addressed all the earlier review feedback from v1, 
except for the request to add plugin support.

Passes full bootstrap and testing on x86_64-linux.

Ok to commit?

-Andi

[PATCH 1/3] Add native ELF and LTO support in collect2

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Change collect2 to read the symbol table directly on ELF systems
using libelf. Also add support for the LTO symbol table.
This way collect2 can resolve symbols in a object file
that only has LTO information.

The LTO parser is closely patterned after the code
in the lto-plugin.

v2: Addressed review feedback. Enabled on all ELF systems now.

gcc/

2010-10-07  Andi Kleen  <ak@linux.intel.com>

	* collect2.c: Add ifdefs for OBJECT_FORMAT_ELF.
	(main): Move use_plugin to top level.
	(scan_prog_file): Move switch statement to ..
	(handle_pass): Separate function here.
	(LTO_SYMTAB_NAME, scan_lto_symtab, scan_elf_symtab, is_ar,
	 scan_prog_file): Add.
	* config.in: Regenerate.
	* config/i386/linux.h (OBJECT_FORMAT_ELF): Define.
	* configure: Regenerate.
	* configure.ac: Check for libelf.h and gelf.h. Adjust
	libelf test.
---
 gcc/collect2.c   |  309 +++++++++++++++++++++++++++++++++++++++++++++---------
 gcc/config.in    |   12 ++
 gcc/configure    |   30 +++++-
 gcc/configure.ac |    8 +-
 4 files changed, 305 insertions(+), 54 deletions(-)

diff --git a/gcc/collect2.c b/gcc/collect2.c
index a8cd232..b3a61ac 100644
--- a/gcc/collect2.c
+++ b/gcc/collect2.c
@@ -68,12 +68,19 @@ along with GCC; see the file COPYING3.  If not see
 #undef REAL_STRIP_FILE_NAME
 #endif
 
+#if  !defined (HAVE_LIBELF_H) || !defined (HAVE_GELF_H)  \
+  || !defined (HAVE_UNISTD_H) || !defined (HAVE_FCNTL_H)
+#undef OBJECT_FORMAT_ELF
+#else
+#undef REAL_NM_FILE_NAME
+#endif
+
 /* If we cannot use a special method, use the ordinary one:
    run nm to find what symbols are present.
    In a cross-compiler, this means you need a cross nm,
    but that is not quite as unpleasant as special headers.  */
 
-#if !defined (OBJECT_FORMAT_COFF)
+#if !defined (OBJECT_FORMAT_COFF) && !defined(OBJECT_FORMAT_ELF)
 #define OBJECT_FORMAT_NONE
 #endif
 
@@ -869,7 +876,7 @@ prefix_from_string (const char *p, struct path_prefix *pprefix)
   free (nstore);
 }
 
-#ifdef OBJECT_FORMAT_NONE
+#if defined(OBJECT_FORMAT_NONE) || defined(OBJECT_FORMAT_ELF)
 
 /* Add an entry for the object file NAME to object file list LIST.
    New entries are added at the end of the list. The original pointer
@@ -889,7 +896,7 @@ add_lto_object (struct lto_object_list *list, const char *name)
 
   list->last = n;
 }
-#endif /* OBJECT_FORMAT_NONE */
+#endif /* OBJECT_FORMAT_NONE || OBJECT_FORMAT_ELF */
 
 
 /* Perform a link-time recompilation and relink if any of the object
@@ -1070,6 +1077,8 @@ maybe_run_lto_and_relink (char **lto_ld_argv, char **object_lst,
     }
 }
 \f
+bool use_plugin = false;
+
 /* Main program.  */
 
 int
@@ -1132,7 +1141,6 @@ main (int argc, char **argv)
   const char **c_ptr;
   char **ld1_argv;
   const char **ld1;
-  bool use_plugin = false;
 
   /* The kinds of symbols we will have to consider when scanning the
      outcome of a first pass link.  This is ALL to start with, then might
@@ -2521,6 +2529,63 @@ write_aix_file (FILE *stream, struct id *list)
 }
 #endif
 \f
+#if defined (OBJECT_FORMAT_NONE) || defined (OBJECT_FORMAT_ELF)
+
+/* Handle a defined symbol */
+
+static void
+handle_pass (const char *name, scanpass which_pass, scanfilter filter,
+	     const char *prog_name)
+{
+  switch (is_ctor_dtor (name))
+    {
+    case SYM_CTOR:
+      if (! (filter & SCAN_CTOR))
+	break;
+      if (which_pass != PASS_LIB)
+	add_to_list (&constructors, name);
+      break;
+      
+    case SYM_DTOR:
+      if (! (filter & SCAN_DTOR))
+	break;
+      if (which_pass != PASS_LIB)
+	add_to_list (&destructors, name);
+      break;
+      
+    case SYM_INIT:
+      if (! (filter & SCAN_INIT))
+	break;
+      if (which_pass != PASS_LIB)
+	fatal ("init function found in object %s", prog_name);
+#ifndef LD_INIT_SWITCH
+      add_to_list (&constructors, name);
+#endif
+      break;
+      
+    case SYM_FINI:
+      if (! (filter & SCAN_FINI))
+	break;
+      if (which_pass != PASS_LIB)
+	fatal ("fini function found in object %s", prog_name);
+#ifndef LD_FINI_SWITCH
+      add_to_list (&destructors, name);
+#endif
+      break;
+      
+    case SYM_DWEH:
+      if (! (filter & SCAN_DWEH))
+	break;
+      if (which_pass != PASS_LIB)
+	add_to_list (&frame_tables, name);
+      break;
+
+    case SYM_REGULAR:
+      break;
+    }
+}
+#endif
+
 #ifdef OBJECT_FORMAT_NONE
 
 /* Check to make sure the file is an LTO object file.  */
@@ -2703,52 +2768,7 @@ scan_prog_file (const char *prog_name, scanpass which_pass,
 
 
       *end = '\0';
-      switch (is_ctor_dtor (name))
-	{
-	case SYM_CTOR:
-	  if (! (filter & SCAN_CTOR))
-	    break;
-	  if (which_pass != PASS_LIB)
-	    add_to_list (&constructors, name);
-	  break;
-
-	case SYM_DTOR:
-	  if (! (filter & SCAN_DTOR))
-	    break;
-	  if (which_pass != PASS_LIB)
-	    add_to_list (&destructors, name);
-	  break;
-
-	case SYM_INIT:
-	  if (! (filter & SCAN_INIT))
-	    break;
-	  if (which_pass != PASS_LIB)
-	    fatal ("init function found in object %s", prog_name);
-#ifndef LD_INIT_SWITCH
-	  add_to_list (&constructors, name);
-#endif
-	  break;
-
-	case SYM_FINI:
-	  if (! (filter & SCAN_FINI))
-	    break;
-	  if (which_pass != PASS_LIB)
-	    fatal ("fini function found in object %s", prog_name);
-#ifndef LD_FINI_SWITCH
-	  add_to_list (&destructors, name);
-#endif
-	  break;
-
-	case SYM_DWEH:
-	  if (! (filter & SCAN_DWEH))
-	    break;
-	  if (which_pass != PASS_LIB)
-	    add_to_list (&frame_tables, name);
-	  break;
-
-	default:		/* not a constructor or destructor */
-	  continue;
-	}
+      handle_pass (name, which_pass, filter, prog_name);
     }
 
   if (debug)
@@ -3218,3 +3238,192 @@ resolve_lib_name (const char *name)
   return (NULL);
 }
 #endif /* COLLECT_EXPORT_LIST */
+
+#ifdef OBJECT_FORMAT_ELF
+#include <libelf.h>
+#include <gelf.h>
+#include <plugin-api.h>
+
+#include <sys/fcntl.h>
+#include <unistd.h>
+
+#define LTO_SYMTAB_NAME ".gnu.lto_.symtab"
+
+/* Scan a LTO symbol table section. */
+
+static unsigned
+scan_lto_symtab (Elf_Data *tab, scanpass which_pass, 
+		 scanfilter filter, const char *prog_name)
+{
+  unsigned nsyms = 0;
+  char *p;
+
+  for (p = (char *)tab->d_buf; p < (char *)tab->d_buf + tab->d_size; ) 
+    {
+      const char *name;
+      int skip = 0;
+
+      /* Done in the same way as the lto-plugin. */
+
+      /* name */
+      name = p;
+      while (*p++)
+	;
+      /* comdat */
+      while (*p++)
+	;
+      /* translate */
+      if (*p == LDPK_UNDEF || *p == LDPK_WEAKUNDEF)
+	skip = 1;
+      p++;
+      /* visibility */
+      p++;
+      /* size */
+      p += 8;
+      /* slot */
+      p += 4;
+
+      if (!skip)
+	handle_pass (name, which_pass, filter, prog_name);
+      nsyms++;
+    }
+  return nsyms;
+}
+
+/* Scan a ELF symbol table */
+
+static unsigned
+scan_elf_symtab (Elf *elf, Elf_Data *data, GElf_Shdr *shdr,
+		 scanpass which_pass, scanfilter filter, const char *prog_name)
+{
+  unsigned i;
+  unsigned nsyms = shdr->sh_size / shdr->sh_entsize;
+  unsigned proc = 0;
+
+  for (i = 0; i < nsyms; i++) 
+    {
+      GElf_Sym sym;
+      const char *name;
+
+      gelf_getsym (data, i, &sym);
+
+      if (ELF32_ST_TYPE (sym.st_info) >= STT_SECTION)
+	continue;
+      if (sym.st_shndx == SHN_UNDEF || sym.st_shndx >= SHN_LORESERVE)
+	continue;
+      name = elf_strptr (elf, shdr->sh_link, sym.st_name);
+      handle_pass (name, which_pass, filter, prog_name);
+      proc++;
+    }
+
+  return nsyms;
+}
+
+/* Is FD an ar file? */
+
+static int
+is_ar (int fd)
+{
+  char buf[8];
+  bool isar = false;
+ 
+  if (read (fd, buf, 8) == 8) 
+    isar = !memcmp (buf, "!<arch>\r", 8);
+  lseek (fd, 0, SEEK_SET);
+  return isar;
+}
+
+/* ELF version to scan the name list of the loaded program for
+   the symbols g++ uses for static constructors and destructors.
+   This also supports LTO symbol tables. */
+
+static void
+scan_prog_file (const char *prog_name, scanpass which_pass,
+		scanfilter filter)
+{
+  int fd;
+  GElf_Ehdr header;
+  Elf *elf;
+  Elf_Scn *section;
+  unsigned syms = 0;
+  unsigned sections = 0;
+  unsigned lto = 0;
+
+  if (which_pass == PASS_SECOND)
+    return;
+
+  if (debug)
+    fprintf (stderr, "Scanning file '%s'\n", prog_name);
+
+  elf_version (EV_CURRENT);
+  fd = open (prog_name, O_RDONLY);
+  if (fd < 0)
+    {
+      fprintf (stderr, "Cannot open %s\n", prog_name);
+      return;
+    }
+
+  /* It is difficult to figure out if an ar file needs LTO or not.
+     If we use the linker plugin and it is an ar file just handle
+     it like a LTO file unconditionally.  */
+  if (which_pass == PASS_LTOINFO && use_plugin && is_ar (fd)) 
+    {
+      if (debug)
+        fprintf (stderr, "Handling ar file %s as LTO\n", prog_name);
+      add_lto_object (&lto_objects, prog_name);
+      return;
+    }
+
+  elf = elf_begin (fd, ELF_C_READ, NULL);
+  if (elf == NULL)
+    {
+      fprintf (stderr, "Cannot run elf_begin on %s: %s\n", prog_name,
+                       elf_errmsg (0));
+      close (fd);
+      return;
+    }
+  if (!gelf_getehdr (elf, &header))
+    fatal ("Cannot find EHDR in %s: %s", prog_name, elf_errmsg (0));
+
+  section = NULL;
+  while ((section = elf_nextscn (elf, section)) != 0)
+    {
+      GElf_Shdr shdr_mem;
+      GElf_Shdr *shdr = gelf_getshdr (section, &shdr_mem);
+      char *name;
+      int islto;
+
+      sections++;
+
+      if (!shdr)
+        fatal("Cannot read SHDR for section %d: %s", sections, elf_errmsg (0));
+
+      name = elf_strptr (elf, header.e_shstrndx, shdr->sh_name);
+      islto = !strncmp (name, LTO_SYMTAB_NAME, strlen (LTO_SYMTAB_NAME));
+      lto += islto;
+
+      if (which_pass == PASS_LTOINFO)
+	{
+	  if (!islto)
+	    continue;
+	  add_lto_object (&lto_objects, prog_name);
+	  break;
+	}
+
+      if (islto)
+	syms += scan_lto_symtab (elf_getdata (section, NULL), which_pass, 
+				 filter, prog_name);
+      else if (shdr->sh_type == SHT_SYMTAB && shdr->sh_entsize > 0)
+	syms += scan_elf_symtab (elf, elf_getdata (section, NULL), shdr,
+				 which_pass, filter, prog_name);
+    }
+
+  if (debug)
+    fprintf (stderr, "Scanned %u symbols, %u sections, %u LTO\n", syms, 
+                     sections, lto);
+
+  elf_end (elf);
+  close (fd);
+}
+
+#endif
diff --git a/gcc/config.in b/gcc/config.in
index 4576de0..d141e6a 100644
--- a/gcc/config.in
+++ b/gcc/config.in
@@ -1017,6 +1017,12 @@
 #endif
 
 
+/* Define to 1 if you have the <gelf.h> header file. */
+#ifndef USED_FOR_TARGET
+#undef HAVE_GELF_H
+#endif
+
+
 /* Define to 1 if you have the `getchar_unlocked' function. */
 #ifndef USED_FOR_TARGET
 #undef HAVE_GETCHAR_UNLOCKED
@@ -1215,6 +1221,12 @@
 #endif
 
 
+/* Define to 1 if you have the <libelf.h> header file. */
+#ifndef USED_FOR_TARGET
+#undef HAVE_LIBELF_H
+#endif
+
+
 /* Define to 1 if you have the <limits.h> header file. */
 #ifndef USED_FOR_TARGET
 #undef HAVE_LIMITS_H
diff --git a/gcc/configure b/gcc/configure
index bceffd6..4b30df4 100755
--- a/gcc/configure
+++ b/gcc/configure
@@ -7870,7 +7870,8 @@ fi
 for ac_header in limits.h stddef.h string.h strings.h stdlib.h time.h iconv.h \
 		 fcntl.h unistd.h sys/file.h sys/time.h sys/mman.h \
 		 sys/resource.h sys/param.h sys/times.h sys/stat.h \
-		 direct.h malloc.h langinfo.h ldfcn.h locale.h wchar.h
+		 direct.h malloc.h langinfo.h ldfcn.h locale.h wchar.h \
+                 libelf.h gelf.h
 do :
   as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
 ac_fn_c_check_header_preproc "$LINENO" "$ac_header" "$as_ac_Header"
@@ -8209,7 +8210,7 @@ if test "${gcc_cv_collect2_libs+set}" = set; then :
   $as_echo_n "(cached) " >&6
 else
   save_LIBS="$LIBS"
-for libs in '' -lld -lmld \
+for libs in '' -lelf -lld -lmld \
 		'-L/usr/lib/cmplrs/cc2.11 -lmld' \
 		'-L/usr/lib/cmplrs/cc3.11 -lmld'
 do
@@ -8238,6 +8239,31 @@ fi
 rm -f core conftest.err conftest.$ac_objext \
     conftest$ac_exeext conftest.$ac_ext
 done
+LIBS=-lelf
+test -z "$gcc_cv_collect2_libs" &&
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char elf_version ();
+int
+main ()
+{
+return elf_version ();
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  gcc_cv_collect2_libs=-lelf
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
 LIBS="$save_LIBS"
 test -z "$gcc_cv_collect2_libs" && gcc_cv_collect2_libs='none required'
 fi
diff --git a/gcc/configure.ac b/gcc/configure.ac
index 1300f82..a567619 100644
--- a/gcc/configure.ac
+++ b/gcc/configure.ac
@@ -893,7 +893,8 @@ AC_HEADER_SYS_WAIT
 AC_CHECK_HEADERS(limits.h stddef.h string.h strings.h stdlib.h time.h iconv.h \
 		 fcntl.h unistd.h sys/file.h sys/time.h sys/mman.h \
 		 sys/resource.h sys/param.h sys/times.h sys/stat.h \
-		 direct.h malloc.h langinfo.h ldfcn.h locale.h wchar.h)
+		 direct.h malloc.h langinfo.h ldfcn.h locale.h wchar.h \
+                 libelf.h gelf.h)
 
 # Check for thread headers.
 AC_CHECK_HEADER(thread.h, [have_thread_h=yes], [have_thread_h=])
@@ -912,7 +913,7 @@ AC_C_BIGENDIAN
 # We may need a special search path to get them linked.
 AC_CACHE_CHECK(for collect2 libraries, gcc_cv_collect2_libs,
 [save_LIBS="$LIBS"
-for libs in '' -lld -lmld \
+for libs in '' -lelf -lld -lmld \
 		'-L/usr/lib/cmplrs/cc2.11 -lmld' \
 		'-L/usr/lib/cmplrs/cc3.11 -lmld'
 do
@@ -920,6 +921,9 @@ do
 	AC_TRY_LINK_FUNC(ldopen,
 		[gcc_cv_collect2_libs="$libs"; break])
 done
+LIBS=-lelf
+test -z "$gcc_cv_collect2_libs" && 
+AC_TRY_LINK_FUNC(elf_version, [gcc_cv_collect2_libs=-lelf])
 LIBS="$save_LIBS"
 test -z "$gcc_cv_collect2_libs" && gcc_cv_collect2_libs='none required'])
 case $gcc_cv_collect2_libs in
-- 
1.7.1

[PATCH 2/3] Implement -flto-slim

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

This adds a new LTO mode "lto slim" that only puts LTO information
into the object file, no "fallback code". This improves compilation
performance because the code has to be only generated once. It also
allows some future extensions I am planning to do.

The disadvantage is that a slim build requires more tool chain support
because there is no "safety net" of fallback code in the object file.

For example ar and ranlib need use the gcc lto plugin (and currently
need to be binutils mainline) to be able to create the symbol indexes.
Also libtool needs to be updated. And it will only work with
the linker plugin.

Because of these complications slim LTO is not default, but a separate
-flto-slim option. The option works with whopr and with LTO.

Spreading the checking of the flag into the various frontends is ugly,
but I didn't find a simple nicer way to do this.

Thanks to Jan Hubicka for some fixes to my initial version.

Requires earlier patches to fix up collect2.

This passes normal bootstrap and testing.

gcc/ada/

2010-10-11  Andi Kleen  <ak@linux.intel.com>

	* utils.c (gnat_write_global_declarations): Bail out early
	when flag_lto_slim is set.

gcc/

2010-10-11  Andi Kleen  <ak@linux.intel.com>
	    Jan Hubicka <jh@suse.cz>

	* cgrapunit.c (ipa_passes): Check for flag_lto_slim.
	(cgraph_optimize): Bail out early if flag_lto_slim is set.
	* common.opt (flto-slim): Add.
	* doc/invoke.texi (-flto-slim): Document.
	* langhooks.c (write_global_declarations): Check for
	flag_lto_slim.
	* toplev.c (compile_file): Check for flag_lto_slim.

gcc/cp

2010-10-11  Andi Kleen  <ak@linux.intel.com>

	* decl2.c (cp_write_global_declarations): Check for
	flag_lto_slim.

gcc/lto

2010-10-11  Andi Kleen  <ak@linux.intel.com>

	* lto.c (lto_main): Set flag_lto_slim to zero.
---
 gcc/ada/gcc-interface/utils.c |    3 +++
 gcc/cgraphunit.c              |   13 ++++++++++++-
 gcc/common.opt                |    5 +++++
 gcc/cp/decl2.c                |    8 +++++---
 gcc/doc/invoke.texi           |   10 +++++++++-
 gcc/langhooks.c               |    7 +++++--
 gcc/lto/lto.c                 |    3 +++
 gcc/toplev.c                  |    7 +++++--
 8 files changed, 47 insertions(+), 9 deletions(-)

diff --git a/gcc/ada/gcc-interface/utils.c b/gcc/ada/gcc-interface/utils.c
index 876556c..e339d88 100644
--- a/gcc/ada/gcc-interface/utils.c
+++ b/gcc/ada/gcc-interface/utils.c
@@ -4646,6 +4646,9 @@ gnat_write_global_declarations (void)
      FIXME: shouldn't be the front end's responsibility to call this.  */
   cgraph_finalize_compilation_unit ();
 
+  if (flag_lto_slim)
+    return;
+
   /* Emit debug info for all global declarations.  */
   emit_debug_global_declarations (VEC_address (tree, global_decls),
 				  VEC_length (tree, global_decls));
diff --git a/gcc/cgraphunit.c b/gcc/cgraphunit.c
index 2abcb67..ea5019a 100644
--- a/gcc/cgraphunit.c
+++ b/gcc/cgraphunit.c
@@ -1726,7 +1726,7 @@ ipa_passes (void)
   if (flag_generate_lto)
     targetm.asm_out.lto_end ();
 
-  if (!flag_ltrans)
+  if (!flag_ltrans && !flag_lto_slim)
     execute_ipa_pass_list (all_regular_ipa_passes);
   invoke_plugin_callbacks (PLUGIN_ALL_IPA_PASSES_END, NULL);
 
@@ -1771,6 +1771,17 @@ cgraph_optimize (void)
       return;
     }
 
+  if (flag_lto_slim)
+    {
+      if (!quiet_flag)
+	fprintf (stderr, "In slim LTO mode. Stopping output.\n");
+      timevar_pop (TV_CGRAPHOPT);
+      /* Release the trees here? Right now it doesn't matter because
+	 we exit soon. */
+      return;
+    }
+
+
   /* This pass remove bodies of extern inline functions we never inlined.
      Do this later so other IPA passes see what is really going on.  */
   cgraph_remove_unreachable_nodes (false, dump_file);
diff --git a/gcc/common.opt b/gcc/common.opt
index 8fe796f..09a546c 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -1092,6 +1092,11 @@ flto-report
 Common Report Var(flag_lto_report) Init(0) Optimization
 Report various link-time optimization statistics
 
+flto-slim
+Common Report Var(flag_lto_slim) Init(0) Optimization
+Only generate LTO output, no assembler. This flag is ignored for the LTO
+frontend. Will only work with -fuse-linker-plugin.
+
 fmath-errno
 Common Report Var(flag_errno_math) Init(1) Optimization
 Set errno after built-in math functions
diff --git a/gcc/cp/decl2.c b/gcc/cp/decl2.c
index fcc83fb..ebac286 100644
--- a/gcc/cp/decl2.c
+++ b/gcc/cp/decl2.c
@@ -3934,8 +3934,9 @@ cp_write_global_declarations (void)
     {
       check_global_declarations (VEC_address (tree, pending_statics),
 				 VEC_length (tree, pending_statics));
-      emit_debug_global_declarations (VEC_address (tree, pending_statics),
-				      VEC_length (tree, pending_statics));
+      if (!flag_lto_slim)
+        emit_debug_global_declarations (VEC_address (tree, pending_statics),
+	  			        VEC_length (tree, pending_statics));
     }
 
   perform_deferred_noexcept_checks ();
@@ -3943,7 +3944,8 @@ cp_write_global_declarations (void)
   /* Generate hidden aliases for Java.  */
   if (candidates)
     {
-      build_java_method_aliases (candidates);
+      if (!flag_lto_slim)
+        build_java_method_aliases (candidates);
       pointer_set_destroy (candidates);
     }
 
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index ee68454..28ac99d 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -355,7 +355,7 @@ Objective-C and Objective-C++ Dialects}.
 -fivopts -fkeep-inline-functions -fkeep-static-consts @gol
 -floop-block -floop-flatten -floop-interchange -floop-strip-mine @gol
 -floop-parallelize-all -flto -flto-compression-level -flto-partition=@var{alg} @gol
--flto-report -fltrans -fltrans-output-list -fmerge-all-constants @gol
+-flto-report -flto-slim -fltrans -fltrans-output-list -fmerge-all-constants @gol
 -fmerge-constants -fmodulo-sched -fmodulo-sched-allow-regmoves @gol
 -fmove-loop-invariants fmudflap -fmudflapir -fmudflapth -fno-branch-count-reg @gol
 -fno-default-inline @gol
@@ -7655,6 +7655,14 @@ files in LTO mode (via @option{-fwhopr} or @option{-flto}).
 
 Disabled by default.
 
+@item -flto-slim
+Only generate LTO output, no assembler. The result is that the assembler
+code is only generated once for a LTO build. This flag is ignored for the LTO
+frontend. Will only work with the @option{-fuse-linker-plugin} and
+with the @command{gold} linker.
+
+Disabled by default
+
 @item -fuse-linker-plugin
 Enables the extraction of objects with GIMPLE bytecode information
 from library archives.  This option relies on features available only
diff --git a/gcc/langhooks.c b/gcc/langhooks.c
index 76c066b..b3f7498 100644
--- a/gcc/langhooks.c
+++ b/gcc/langhooks.c
@@ -323,9 +323,12 @@ write_global_declarations (void)
   for (i = 0, decl = globals; i < len; i++, decl = DECL_CHAIN (decl))
     vec[len - i - 1] = decl;
 
-  wrapup_global_declarations (vec, len);
+  if (!flag_lto_slim)
+    wrapup_global_declarations (vec, len);
   check_global_declarations (vec, len);
-  emit_debug_global_declarations (vec, len);
+
+  if (!flag_lto_slim)
+    emit_debug_global_declarations (vec, len);
 
   /* Clean up.  */
   free (vec);
diff --git a/gcc/lto/lto.c b/gcc/lto/lto.c
index d1a2206..a69db40 100644
--- a/gcc/lto/lto.c
+++ b/gcc/lto/lto.c
@@ -2423,6 +2423,9 @@ lto_main (int debug_p ATTRIBUTE_UNUSED)
 
   lto_init_reader ();
 
+  /* In LTO mode don't ever be slim */
+  flag_lto_slim = 0;
+
   /* Read all the symbols and call graph from all the files in the
      command line.  */
   read_cgraph_and_symbols (num_in_fnames, in_fnames);
diff --git a/gcc/toplev.c b/gcc/toplev.c
index 808248b..a24d6b9 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -930,8 +930,11 @@ compile_file (void)
   /* This must also call cgraph_finalize_compilation_unit.  */
   lang_hooks.decls.final_write_globals ();
 
-  if (seen_error ())
-    return;
+  if (seen_error () || flag_lto_slim)
+    {
+      invoke_plugin_callbacks (PLUGIN_FINISH_UNIT, NULL);
+      return;
+    }
 
   varpool_assemble_pending_decls ();
   finish_aliases_2 ();
-- 
1.7.1

[PATCH 3/3] Add bootstrap-lto-slim build config

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

/

2010-10-03  Andi Kleen <ak@linux.intel.com>

	* config/bootstrap-lto-slim.mk: Add.

gcc/

2010-10-03  Andi Kleen <ak@linux.intel.com>

	* doc/install.texi (bootstrap-lto-slim): Document.
---
 config/bootstrap-lto-slim.mk |    8 ++++++++
 gcc/doc/install.texi         |    3 +++
 2 files changed, 11 insertions(+), 0 deletions(-)
 create mode 100644 config/bootstrap-lto-slim.mk

diff --git a/config/bootstrap-lto-slim.mk b/config/bootstrap-lto-slim.mk
new file mode 100644
index 0000000..af08db4
--- /dev/null
+++ b/config/bootstrap-lto-slim.mk
@@ -0,0 +1,8 @@
+# This option enables LTO slim for stage2 and stage3.  It requires lto to
+# be enabled for stage1 with --enable-stage1-languages.
+
+STAGE2_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1 -flto-slim
+STAGE3_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1 -flto-slim
+
+# Ada fails to build with LTO, turn it off for now.
+BOOT_ADAFLAGS += -fno-lto
diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi
index ce6b5cf..0ef8e60 100644
--- a/gcc/doc/install.texi
+++ b/gcc/doc/install.texi
@@ -2197,6 +2197,9 @@ Enables Link-Time Optimization for host tools during bootstrapping.
 @samp{BUILD_CONFIG=bootstrap-lto} is equivalent to adding
 @option{-flto} to @samp{BOOT_CFLAGS}.
 
+@item @samp{bootstrap-lto-slim}
+Analogous to @code{bootstrap-lto}.
+
 @item @samp{bootstrap-debug}
 Verifies that the compiler generates the same executable code, whether
 or not it is asked to emit debug information.  To this end, this
-- 
1.7.1

Re: [PATCH 2/3] Implement -flto-slim

[Diego Novillo]

On 10-10-16 11:09 , Andi Kleen wrote:

> +@item -flto-slim
> +Only generate LTO output, no assembler. The result is that the assembler
> +code is only generated once for a LTO build. This flag is ignored for the LTO
> +frontend. Will only work with the @option{-fuse-linker-plugin} and
> +with the @command{gold} linker.

s/Will only/This option will only/

The patch looks fine, but the option is currently not tested.  Could you 
add it to LTO_OPTIONS in testsuite/lib/lto.exp:lto_init ?


Diego.

Re: [PATCH 2/3] Implement -flto-slim

[Richard Guenther]

On Tue, Nov 2, 2010 at 12:01 PM, Diego Novillo <dnovillo@google.com> wrote:
> On 10-10-16 11:09 , Andi Kleen wrote:
>
>> +@item -flto-slim
>> +Only generate LTO output, no assembler. The result is that the assembler
>> +code is only generated once for a LTO build. This flag is ignored for the
>> LTO
>> +frontend. Will only work with the @option{-fuse-linker-plugin} and
>> +with the @command{gold} linker.
>
> s/Will only/This option will only/
>
> The patch looks fine, but the option is currently not tested.  Could you add
> it to LTO_OPTIONS in testsuite/lib/lto.exp:lto_init ?

Hm, it won't work without the linker plugin, will it?  So it's not that easy.

Richard.

>
> Diego.
>

Re: [PATCH 2/3] Implement -flto-slim

[Diego Novillo]

On Tue, Nov 2, 2010 at 07:05, Richard Guenther
<richard.guenther@gmail.com> wrote:
> On Tue, Nov 2, 2010 at 12:01 PM, Diego Novillo <dnovillo@google.com> wrote:
>> On 10-10-16 11:09 , Andi Kleen wrote:
>>
>>> +@item -flto-slim
>>> +Only generate LTO output, no assembler. The result is that the assembler
>>> +code is only generated once for a LTO build. This flag is ignored for the
>>> LTO
>>> +frontend. Will only work with the @option{-fuse-linker-plugin} and
>>> +with the @command{gold} linker.
>>
>> s/Will only/This option will only/
>>
>> The patch looks fine, but the option is currently not tested.  Could you add
>> it to LTO_OPTIONS in testsuite/lib/lto.exp:lto_init ?
>
> Hm, it won't work without the linker plugin, will it?  So it's not that easy.

Oh, yeah.  We'd need to wait for the linker plugin to be built unconditionally.


Diego.

Re: [PATCH 2/3] Implement -flto-slim

[Andi Kleen]

> Oh, yeah.  We'd need to wait for the linker plugin to be built unconditionally.

There's that.

FWIW -- i currently use a private LTO test suite because I found it difficult
to express many of the multi file test cases I have in dejagnu.   It simply
uses Makefiles.

I realize it's not a very satisfying situation.

-Andi

Re: [PATCH 2/3] Implement -flto-slim

[Richard Guenther]

On Tue, Nov 2, 2010 at 12:42 PM, Andi Kleen <andi@firstfloor.org> wrote:
>> Oh, yeah.  We'd need to wait for the linker plugin to be built unconditionally.
>
> There's that.
>
> FWIW -- i currently use a private LTO test suite because I found it difficult
> to express many of the multi file test cases I have in dejagnu.   It simply
> uses Makefiles.
>
> I realize it's not a very satisfying situation.

;)

I guess it wouldn't be too hard to detect a $testcase.mk file and
invoke that with 'build' 'run' and 'clean' rules from dejagnu.  At least
it wasn't too hard to add linker-plugin support or suppor for multi-language
tests.  For a non-TCL speaker, that is ;)

Richard.

> -Andi
>

Re: [PATCH 1/3] Add native ELF and LTO support in collect2

[Diego Novillo]

On 10-10-16 11:09 , Andi Kleen wrote:

>
> 	* collect2.c: Add ifdefs for OBJECT_FORMAT_ELF.
> 	(main): Move use_plugin to top level.
> 	(scan_prog_file): Move switch statement to ..
> 	(handle_pass): Separate function here.
> 	(LTO_SYMTAB_NAME, scan_lto_symtab, scan_elf_symtab, is_ar,
> 	 scan_prog_file): Add.
> 	* config.in: Regenerate.
> 	* config/i386/linux.h (OBJECT_FORMAT_ELF): Define.
> 	* configure: Regenerate.
> 	* configure.ac: Check for libelf.h and gelf.h. Adjust
> 	libelf test.

Hm, why not just force the use of a linker that has plugin support?


Diego.

Re: [PATCH 1/3] Add native ELF and LTO support in collect2

[Richard Guenther]

On Tue, Nov 2, 2010 at 12:04 PM, Diego Novillo <dnovillo@google.com> wrote:
> On 10-10-16 11:09 , Andi Kleen wrote:
>
>>
>>        * collect2.c: Add ifdefs for OBJECT_FORMAT_ELF.
>>        (main): Move use_plugin to top level.
>>        (scan_prog_file): Move switch statement to ..
>>        (handle_pass): Separate function here.
>>        (LTO_SYMTAB_NAME, scan_lto_symtab, scan_elf_symtab, is_ar,
>>         scan_prog_file): Add.
>>        * config.in: Regenerate.
>>        * config/i386/linux.h (OBJECT_FORMAT_ELF): Define.
>>        * configure: Regenerate.
>>        * configure.ac: Check for libelf.h and gelf.h. Adjust
>>        libelf test.
>
> Hm, why not just force the use of a linker that has plugin support?

Or instead use the new facility Ian added to libiberty (or use it once
that is committed).

Richard.

>
> Diego.
>

Re: [PATCH 1/3] Add native ELF and LTO support in collect2

[Andi Kleen]

On Tue, Nov 02, 2010 at 12:06:37PM +0100, Richard Guenther wrote:
> On Tue, Nov 2, 2010 at 12:04 PM, Diego Novillo <dnovillo@google.com> wrote:
> > On 10-10-16 11:09 , Andi Kleen wrote:
> >
> >>
> >>        * collect2.c: Add ifdefs for OBJECT_FORMAT_ELF.
> >>        (main): Move use_plugin to top level.
> >>        (scan_prog_file): Move switch statement to ..
> >>        (handle_pass): Separate function here.
> >>        (LTO_SYMTAB_NAME, scan_lto_symtab, scan_elf_symtab, is_ar,
> >>         scan_prog_file): Add.
> >>        * config.in: Regenerate.
> >>        * config/i386/linux.h (OBJECT_FORMAT_ELF): Define.
> >>        * configure: Regenerate.
> >>        * configure.ac: Check for libelf.h and gelf.h. Adjust
> >>        libelf test.
> >
> > Hm, why not just force the use of a linker that has plugin support?
> 
> Or instead use the new facility Ian added to libiberty (or use it once
> that is committed).

I can do that, shouldn't be too hard to convert it.
Should I wait for that?

-andi

Re: [PATCH 1/3] Add native ELF and LTO support in collect2

[Dave Korn]

On 02/11/2010 11:27, Andi Kleen wrote:
> On Tue, Nov 02, 2010 at 12:06:37PM +0100, Richard Guenther wrote:
>> On Tue, Nov 2, 2010 at 12:04 PM, Diego Novillo <dnovillo@google.com> wrote:
>>> On 10-10-16 11:09 , Andi Kleen wrote:
>>>
>>>>        * collect2.c: Add ifdefs for OBJECT_FORMAT_ELF.
>>>>        (main): Move use_plugin to top level.
>>>>        (scan_prog_file): Move switch statement to ..
>>>>        (handle_pass): Separate function here.
>>>>        (LTO_SYMTAB_NAME, scan_lto_symtab, scan_elf_symtab, is_ar,
>>>>         scan_prog_file): Add.
>>>>        * config.in: Regenerate.
>>>>        * config/i386/linux.h (OBJECT_FORMAT_ELF): Define.
>>>>        * configure: Regenerate.
>>>>        * configure.ac: Check for libelf.h and gelf.h. Adjust
>>>>        libelf test.
>>> Hm, why not just force the use of a linker that has plugin support?
>> Or instead use the new facility Ian added to libiberty (or use it once
>> that is committed).
> 
> I can do that, shouldn't be too hard to convert it.
> Should I wait for that?

  It's gone in now.  If you want an example of how it can be used, take a look
at the patch I wrote to convert the lto-plugin.

http://gcc.gnu.org/ml/gcc/2010-10/msg00483.html

N.B. that there's a bug in that version: "free (data);" in two locations in
process_symtab() should in fact read "free (secdata);".  (Also it was written
before the objfile functions all got renamed, so I need to update it there too.)

    cheers,
      DaveK

Re: [PATCH 1/3] Add native ELF and LTO support in collect2

[Andi Kleen]

> Hm, why not just force the use of a linker that has plugin support?

I mainly did it so that the constructors etc. would be still
generated by collect2. But maybe that could be fully done
by the linker now. I must admit I don't understand all 
the implications of such a change, so I preferred to keep
the old mode and just make it work with LTO symbol tables.

There's also two additional uses of this infrastructure
(which are not in this patchkit yet):

- Detect leftover non LTO code from earlier ld -r and use objcopy to copy
it to a new object file and include it. This can be easily done
with slim LTO.
The main use of that is handling .S files LTO build that uses ld -r.
Without it that code would disappear during LTO.
I need that for a project I'm interested in. This was the main
reason I implemented slim LTO in the first place.

- Automatic detection of LTO so that if a Makefile forgets 
to add the lto options for the final link it would still work.
This is actually needed for a full gcc lto slim bootstrap right
now because libiberty doesn't set correct stage2/3 link flags.

Perhaps that's obsolete, if ld plugin support is universal
the plugin could be just enabled unconditionally. I was a bit
wary of this before because it would mean unconditional gold
and gold still seems to have trouble with some code.

-Andi

*PING* Re: Updated slim-lto patchkit

[Andi Kleen]

Andi Kleen <andi@firstfloor.org> writes:

> Updated version of the slim lto patchkit.
>
> I addressed all the earlier review feedback from v1, 
> except for the request to add plugin support.
>
> Passes full bootstrap and testing on x86_64-linux.

Could someone please review these three patches?

Thanks,
-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: *PING* Re: Updated slim-lto patchkit

[Diego Novillo]

On Wed, Oct 20, 2010 at 17:17, Andi Kleen <andi@firstfloor.org> wrote:
> Andi Kleen <andi@firstfloor.org> writes:
>
>> Updated version of the slim lto patchkit.
>>
>> I addressed all the earlier review feedback from v1,
>> except for the request to add plugin support.
>>
>> Passes full bootstrap and testing on x86_64-linux.
>
> Could someone please review these three patches?

They're in my queue.  Not sure if I'll have enough time to go over
them before next week, though.  Are you going to the summit by any
chance?


Diego.

Re: *PING* Re: Updated slim-lto patchkit

[Andi Kleen]

On Wed, Oct 20, 2010 at 05:22:14PM -0400, Diego Novillo wrote:
> On Wed, Oct 20, 2010 at 17:17, Andi Kleen <andi@firstfloor.org> wrote:
> > Andi Kleen <andi@firstfloor.org> writes:
> >
> >> Updated version of the slim lto patchkit.
> >>
> >> I addressed all the earlier review feedback from v1,
> >> except for the request to add plugin support.
> >>
> >> Passes full bootstrap and testing on x86_64-linux.
> >
> > Could someone please review these three patches?
> 
> They're in my queue.  Not sure if I'll have enough time to go over

Thanks.  That's fine. FWIW I plan to submit one or two followup
patches who built on top of this.

> them before next week, though.  Are you going to the summit by any
> chance?

Nope sorry.

-Andi