Re: [PATCH] Add gcc-ar/nm/ranlib wrappers for slim LTO

[Richard Guenther <richard.guenther@gmail.com>]

On Thu, Oct 20, 2011 at 9:24 AM, Andi Kleen <andi@firstfloor.org> wrote:
> From: Andi Kleen <ak@linux.intel.com>
>
> Slim LTO requires running ar/nm/ranlib with the LTO plugin. The most
> convenient way to get this into existing Makefiles is using small
> wrappers that pass the plugin. This matches how other compilers
> (LLVM, icc) do this too.
>
> My previous attempt at using shell scripts for this
> http://gcc.gnu.org/ml/gcc-patches/2010-10/msg02471.html
> was not approved. Here's another attempt using wrappers written
> in C. It's only a single wrapper which just adds a --plugin
> argument before calling the respective binutils utilities.

Thanks for doing this.  How do they end up being used?  I suppose
Makefiles will need to call gcc-ar then instead of ar?  In which case
I wonder if ...

> The logic gcc.c uses to find the files is very complicated. I didn't
> try to replicate it 100% and left out some magic. I would be interested
> if this simple method works for everyone or if more code needs
> to be added. This only needs to support LTO supporting hosts of course.

;)

... using something like gcc --ar would be more convenient (as you
can then trivially share the find-the-files logic)?  Did you consider
factoring out the find-the-file logic to a shared file that you can re-use?

Thanks,
Richard.

> I didn't add any documentation because the syntax is exactly the same as
> the native ar/ranlib/nm.
>
> Passed bootstrap and test suite on x86_64-linux.
>
> gcc/:
> 2011-10-19  Andi Kleen  <ak@linux.intel.com>
>
>        * Makefile.in (MOSTLYCLEANFILES): Add gcc-ar/nm/ranlib.
>        (native): Add gcc-ar.
>        (AR_OBJS, AR_LIBS, gcc-ar, gcc-ar.o): Add.
>        (install): Depend on install-gcc-ar.
>        (install-gcc-ar): Add.
>        (uninstall): Uninstall gcc-ar/nm/ranlib.
>        * gcc-ar.c: Add new file.
> ---
>  gcc/Makefile.in |   28 +++++++++++++--
>  gcc/gcc-ar.c    |  109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 134 insertions(+), 3 deletions(-)
>  create mode 100644 gcc/gcc-ar.c
>
> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
> index 6b28ef5..7816243 100644
> --- a/gcc/Makefile.in
> +++ b/gcc/Makefile.in
> @@ -1545,7 +1545,8 @@ MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
>  genrtl.h gt-*.h gtype-*.h gtype-desc.c gtyp-input.list \
>  xgcc$(exeext) cpp$(exeext) cc1$(exeext) $(EXTRA_PASSES) \
>  $(EXTRA_PARTS) $(EXTRA_PROGRAMS) gcc-cross$(exeext) \
> - $(SPECS) collect2$(exeext) lto-wrapper$(exeext) \
> + $(SPECS) collect2$(exeext) gcc-ar$(exeext) gcc-nm$(exeext) \
> + gcc-ranlib$(exeext) \
>  gcov-iov$(build_exeext) gcov$(exeext) gcov-dump$(exeext) \
>  gengtype$(exeext) *.[0-9][0-9].* *.[si] *-checksum.c libbackend.a \
>  libcommon-target.a libcommon.a libgcc.mk
> @@ -1791,7 +1792,8 @@ rest.encap: lang.rest.encap
>  # This is what is made with the host's compiler
>  # whether making a cross compiler or not.
>  native: config.status auto-host.h build-@POSUB@ $(LANGUAGES) \
> -       $(EXTRA_PASSES) $(EXTRA_PROGRAMS) $(COLLECT2) lto-wrapper$(exeext)
> +       $(EXTRA_PASSES) $(EXTRA_PROGRAMS) $(COLLECT2) lto-wrapper$(exeext) \
> +       gcc-ar$(exeext)
>
>  ifeq ($(enable_plugin),yes)
>  native: gengtype$(exeext)
> @@ -2049,6 +2051,17 @@ sbitmap.o: sbitmap.c sbitmap.h $(CONFIG_H) $(SYSTEM_H) coretypes.h $(BASIC_BLOCK
>  ebitmap.o: ebitmap.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(EBITMAP_H)
>  sparseset.o: sparseset.c $(SYSTEM_H) sparseset.h $(CONFIG_H)
>
> +AR_OBJS = gcc-ar.o
> +AR_LIBS = @COLLECT2_LIBS@
> +gcc-ar$(exeext): $(AR_OBJS) $(LIBDEPS)
> +       +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> +               $(AR_OBJS) $(LIBS) $(AR_LIBS)
> +
> +gcc-ar.o: gcc-ar.c $(CONFIG_H) $(SYSTEM_H) $(LIBIBERTY_H)
> +       $(COMPILER) $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(DRIVER_DEFINES) \
> +       -DTARGET_MACHINE=\"$(target_noncanonical)\" \
> +       -c $(srcdir)/gcc-ar.c $(OUTPUT_OPTION) @TARGET_SYSTEM_ROOT_DEFINE@
> +
>  COLLECT2_OBJS = collect2.o collect2-aix.o tlink.o
>  COLLECT2_LIBS = @COLLECT2_LIBS@
>  collect2$(exeext): $(COLLECT2_OBJS) $(LIBDEPS)
> @@ -4576,7 +4589,7 @@ maintainer-clean:
>  # broken is small.
>  install: install-common $(INSTALL_HEADERS) \
>     install-cpp install-man install-info install-@POSUB@ \
> -    install-driver install-lto-wrapper
> +    install-driver install-lto-wrapper install-gcc-ar
>
>  ifeq ($(enable_plugin),yes)
>  install: install-plugin
> @@ -4901,6 +4914,12 @@ install-collect2: collect2 installdirs
>  install-lto-wrapper: lto-wrapper$(exeext)
>        $(INSTALL_PROGRAM) lto-wrapper$(exeext) $(DESTDIR)$(libexecsubdir)/lto-wrapper$(exeext)
>
> +# XXX hardlink if system supports it
> +install-gcc-ar:
> +       $(INSTALL_PROGRAM) gcc-ar$(exeext) $(DESTDIR)$(bindir)/gcc-ar$(exeext)
> +       $(INSTALL_PROGRAM) gcc-ar$(exeext) $(DESTDIR)$(bindir)/gcc-nm$(exeext)
> +       $(INSTALL_PROGRAM) gcc-ar$(exeext) $(DESTDIR)$(bindir)/gcc-ranlib$(exeext)
> +
>  # Cancel installation by deleting the installed files.
>  uninstall: lang.uninstall
>        -rm -rf $(DESTDIR)$(libsubdir)
> @@ -4915,6 +4934,9 @@ uninstall: lang.uninstall
>        -rm -rf $(DESTDIR)$(man1dir)/cpp$(man1ext)
>        -rm -f $(DESTDIR)$(infodir)/cpp.info* $(DESTDIR)$(infodir)/gcc.info*
>        -rm -f $(DESTDIR)$(infodir)/cppinternals.info* $(DESTDIR)$(infodir)/gccint.info*
> +       -rm -f $(DESTDIR)$(bindir)/gcc-ar$(exeext)
> +       -rm -f $(DESTDIR)$(bindir)/gcc-nm$(exeext)
> +       -rm -f $(DESTDIR)$(bindir)/gcc-ranlib$(exeext)
>  #
>  # These targets are for the dejagnu testsuites. The file site.exp
>  # contains global variables that all the testsuites will use.
> diff --git a/gcc/gcc-ar.c b/gcc/gcc-ar.c
> new file mode 100644
> index 0000000..2c51e9a
> --- /dev/null
> +++ b/gcc/gcc-ar.c
> @@ -0,0 +1,109 @@
> +/* Wrapper for ar/ranlib/nm to pass the LTO plugin. The same executable
> +   handles all.
> +   Copyright (C) 2011 Free Software Foundation, Inc.
> +   Contributed by Andi Kleen.
> +
> +This file is part of GCC.
> +
> +GCC is free software; you can redistribute it and/or modify it under
> +the terms of the GNU General Public License as published by the Free
> +Software Foundation; either version 3, or (at your option) any later
> +version.
> +
> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> +for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GCC; see the file COPYING3.  If not see
> +<http://www.gnu.org/licenses/>.  */
> +
> +#include <stdio.h>
> +#include "config.h"
> +#include "system.h"
> +#include "libiberty.h"
> +
> +static const char *personas[] =
> +  {
> +    "ar",
> +    "ranlib",
> +    "nm",
> +    NULL
> +  };
> +
> +static const char standard_libexec_prefix[] = STANDARD_LIBEXEC_PREFIX;
> +
> +static const char dir_separator[] = { DIR_SEPARATOR, 0 };
> +
> +int
> +main(int ac, char **av)
> +{
> +  char *nprefix;
> +  const char *exe_name;
> +  char *plugin;
> +  int len, k, status, err;
> +  const char *err_msg;
> +  const char **nargv;
> +  bool is_ar = false;
> +
> +  /* Determine which utility */
> +  len = strlen (av[0]);
> +  for (k = 0; personas[k]; k++)
> +    {
> +      int w = strlen (personas[k]);
> +      if (len >= w && !strcmp (av[0] + len - w, personas[k]))
> +       break;
> +    }
> +  if (personas[k] == NULL)
> +    {
> +      fprintf (stderr, "Unknown executable name %s\n", av[0]);
> +      exit (1);
> +    }
> +  is_ar = !strcmp (personas[k], "ar");
> +
> +  exe_name = personas[k];
> +#ifdef CROSS_DIRECTORY_STRUCTURE
> +  exe_name = concat (target_machine, "-", exe_name, NULL);
> +#endif
> +
> +  /* Find plugin */
> +  /* XXX implement more magic from gcc.c? */
> +  nprefix = getenv ("GCC_EXEC_PREFIX");
> +  plugin = concat (nprefix ? nprefix : standard_libexec_prefix,
> +                  dir_separator,
> +                   DEFAULT_TARGET_MACHINE,
> +                  dir_separator,
> +                  DEFAULT_TARGET_VERSION,
> +                  dir_separator,
> +                  LTOPLUGINSONAME,
> +                  NULL);
> +  if (access (plugin, X_OK))
> +    {
> +      fprintf (stderr, "%s: Cannot find plugin %s\n", av[0], plugin);
> +      exit (1);
> +    }
> +
> +  /* Create new command line with plugin */
> +  nargv = XCNEWVEC (const char *, ac + 4);
> +  nargv[0] = exe_name;
> +  nargv[1] = "--plugin";
> +  nargv[2] = plugin;
> +  if (is_ar && av[1] && av[1][0] != '-')
> +    av[1] = concat("-", av[1], NULL);
> +  for (k = 1; k < ac; k++)
> +    nargv[2 + k] = av[k];
> +  nargv[2 + k] = NULL;
> +
> +  /* Run utility */
> +  /* ??? the const is misplaced in pex_one's argv? */
> +  err_msg = pex_one (PEX_LAST|PEX_SEARCH,
> +                    exe_name,
> +                    CONST_CAST2 (char * const *, const char **, nargv),
> +                    concat("gcc-", exe_name, NULL),
> +                    NULL,NULL,  &status, &err);
> +  if (err_msg)
> +    fprintf(stderr, "Error running %s: %s\n", exe_name, err_msg);
> +
> +  return err;
> +}
> --
> 1.7.5.4
>
>