[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

Dave Korn <davek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2010.11.10 03:59:21
            Version|4.5.0                       |4.6.0
     Ever Confirmed|0                           |1

--- Comment #16 from Dave Korn <davek at gcc dot gnu.org> 2010-11-10 03:59:21 UTC ---
Confirmed that this still happens on HEAD.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

Dave Korn <davek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
         AssignedTo|unassigned at gcc dot       |davek at gcc dot gnu.org
                   |gnu.org                     |

--- Comment #18 from Dave Korn <davek at gcc dot gnu.org> 2010-11-16 15:03:21 UTC ---
Testing went well, submitting the patch shortly.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

Dave Korn <davek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch
                URL|                            |http://gcc.gnu.org/ml/gcc-p
                   |                            |atches/2010-11/msg01688.htm
                   |                            |l

--- Comment #19 from Dave Korn <davek at gcc dot gnu.org> 2010-11-16 16:19:39 UTC ---
Posted.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hjl.tools at gmail dot com

--- Comment #20 from H.J. Lu <hjl.tools at gmail dot com> 2010-11-20 20:49:56 UTC ---
If we have to pass libgcc archive in GCC driver, what happens to
random archives on command line? Will LTO ignore them? It sounds
like a bad idea. I think the fix should be in linker, not in GCC
driver.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #21 from Dave Korn <davek at gcc dot gnu.org> 2010-11-20 20:52:38 UTC ---
(In reply to comment #20)
> If we have to pass libgcc archive in GCC driver, what happens to
> random archives on command line? Will LTO ignore them? It sounds
> like a bad idea. I think the fix should be in linker, not in GCC
> driver.

  Link-time optimisations can only generate new undefined references to
functions that GCC knows how to optimise as builtins, so I think this can only
happen with libgcc and libc references.  In the case of user code and
libraries, there should always be undefined references in the LTO symtabs to
the required functions.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hubicka at gcc dot gnu.org

--- Comment #22 from Jan Hubicka <hubicka at gcc dot gnu.org> 2010-11-20 21:04:06 UTC ---
GCC will also auto-generate calls to libgcov at link time. This is one of
reason why profiledbootstrap does not work with LTO (yet)

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #23 from Jan Hubicka <hubicka at gcc dot gnu.org> 2010-11-20 21:06:50 UTC ---
BTW I really think both gold and GNU ld will need to do two stage linking (i.e.
fully re-link after LTO step).  Other problems involved in this include the
fact that we can optimize out some library calls and currently I think the
library will be still linked in and the issue with COMDATs.
(i.e. when GCC announce COMDAT function in symbol table and later inline it, it
still need to produce out-of-line function body because linker handles is as
PREVAIL). Finally -fwhole-program makes lto symtab entries to disappear.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #24 from H.J. Lu <hjl.tools at gmail dot com> 2010-11-20 21:14:01 UTC ---
Can we do

1. GCC calls ld with -r to generate a .o file first.
2. GCC calls ld with the single .o input?

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #25 from H.J. Lu <hjl.tools at gmail dot com> 2010-11-22 18:02:47 UTC ---
The current linker plugin scheme may be flawed.  The order of
linking libraries (archive and DSO) is very important. They
have to be placed between crti.o and crtn.o. We may not link
an archive after crtn.o.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #26 from H.J. Lu <hjl.tools at gmail dot com> 2010-11-22 18:27:58 UTC ---
One approach for linker plugin support may be:

1. GCC driver marks crtbegin.o and crtend.o.
2. Linker processes everything between crtbegin.o and crtend.o with GCC
LTO plugin.
3. Linker generates executable with the proper command line and GCC LTO
plugin outputs.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #27 from Dave Korn <davek at gcc dot gnu.org> 2010-11-23 19:08:45 UTC ---
(In reply to comment #25)
> The current linker plugin scheme may be flawed.  The order of
> linking libraries (archive and DSO) is very important. They
> have to be placed between crti.o and crtn.o. We may not link
> an archive after crtn.o.

Sounds like crtn.o should be sent to the link via a further -pass-through
option (positioned after the ones that re-scan the libraries) rather than
directly on the command-line when the plugin is in use.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #28 from Dave Korn <davek at gcc dot gnu.org> 2010-11-23 19:18:46 UTC ---
Author: davek
Date: Tue Nov 23 19:18:39 2010
New Revision: 167091

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=167091
Log:
    PR driver/42690
    * gcc.c (LINK_COMMAND_SPEC): Remove hard-coded pass-through plugin
    options, replace by call of pass-through-libs spec function to process
    link_gcc_c_sequence spec.
    (lto_libgcc_spec): Delete variable.
    (static_specs[]): Remove related entry.
    (static_spec_functions[]): Add new entry for pass-through-libs.
    (main): Don't generate deleted lto_libgcc_spec.
    (pass_through_libs_spec_func): New function to implement the new
    pass-through-libs spec function.
    * doc/invoke.texi (pass-through-libs): Document new spec function.


Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/doc/invoke.texi
    trunk/gcc/gcc.c

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #29 from H.J. Lu <hjl.tools at gmail dot com> 2010-11-23 23:43:55 UTC ---
Please note that -pass-through is a hack, not a real solution.
See

http://www.sourceware.org/bugzilla/show_bug.cgi?id=12248

for a testcase to show why it doesn't work correctly.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #30 from H.J. Lu <hjl.tools at gmail dot com> 2010-11-24 15:07:17 UTC ---
Goal:  We should preserve the same linker command line order as if there are no
IR.
Problem:
    a. LTO may generate extra symbol references which aren't in IR.
    b. It was worked around with -pass-through hack.  But it doesn't preserve
the link command line order.

Proposal:
    a. Remove -pass-through hack.
    b. GCC always passes the same command line to linker, with or without LTO. 
The only additions for LTO are LTO options.
    c. For linker:
        i. For a .o or DSO file on command line, don't process it.
            1) Present it to LTO:
                a) If it isn't claimed by LTO, send it to LTO as pass-through.
        ii. For an archive or linker script on command line:
            1) Present it to LTO recursively
            2) Send it to LTO as pass-through after it has been processed
recursively. 
    d. For LTO:
        i. Send translated files as well as pass-through files to linker in the
same order in which they are presented from linker.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dann at ics dot uci.edu

--- Comment #31 from Richard Guenther <rguenth at gcc dot gnu.org> 2010-11-25 10:50:34 UTC ---
*** Bug 46652 has been marked as a duplicate of this bug. ***

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #32 from H.J. Lu <hjl.tools at gmail dot com> 2010-12-02 17:15:52 UTC ---
Another testcase:

[hjl@gnu-6 pr12245-6]$ cat y.c
#include <stdio.h>
#include <stdlib.h>
#include <math.h>

int
main (int argc, char **argv)
{
  int d = atoi (argv[1]);
  printf ("%f\n", sin (d));
  return 0;
}
[hjl@gnu-6 pr12245-6]$ make
/usr/gcc-4.6/bin/gcc -O2 -fwhole-program -flto=jobserver -fuse-linker-plugin  
-c -o y.o y.c
/usr/gcc-4.6/bin/gcc -static -o foo -O2 -fwhole-program -flto=jobserver
-fuse-linker-plugin y.o -lm
/tmp/ccrHdm25.ltrans0.ltrans.o: In function `main':
ccrHdm25.ltrans0.o:(.text.startup+0x19): undefined reference to `sin'
collect2: ld returned 1 exit status
make: *** [foo] Error 1
[hjl@gnu-6 pr12245-6]$

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #33 from H.J. Lu <hjl.tools at gmail dot com> 2010-12-04 17:37:39 UTC ---
This bug should be fixed with the 2 stage BFD linker patch:

http://sourceware.org/ml/binutils/2010-12/msg00182.html

No changes to GCC are needed.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d.g.gorbachev at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #34 from Dmitry Gorbachev <d.g.gorbachev at gmail dot com> 2010-12-04 22:29:59 UTC ---
Created attachment 22633
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=22633
Testcase for ld.bfd with Hongjiu Lu's patch

(In reply to comment #33)

Received SIGSEGV on this testcase.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d.g.gorbachev at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #35 from Dmitry Gorbachev <d.g.gorbachev at gmail dot com> 2010-12-04 22:31:01 UTC ---
Created attachment 22634
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=22634
Backtrace from ld.bfd with Hongjiu Lu's patch

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #36 from H.J. Lu <hjl.tools at gmail dot com> 2010-12-05 00:44:37 UTC ---
(In reply to comment #34)
> Created attachment 22633 [details]
> Testcase for ld.bfd with Hongjiu Lu's patch
> 
> (In reply to comment #33)
> 
> Received SIGSEGV on this testcase.

Please try:

http://sourceware.org/ml/binutils/2010-12/msg00191.html

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d.g.gorbachev at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #37 from Dmitry Gorbachev <d.g.gorbachev at gmail dot com> 2010-12-05 12:37:26 UTC ---
(In reply to comment #36)

It seems to work. Thanks.

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d.g.gorbachev at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #38 from Dmitry Gorbachev <d.g.gorbachev at gmail dot com> 2010-12-05 16:48:38 UTC ---
(In reply to comment #36)

$ touch empty.c
$ gcc -nodefaultlibs -r -flto -fuse-linker-plugin empty.c
/usr/local/bin/ld: BFD (GNU Binutils) 2.21.51.20101204 assertion fail
../../binutils-2.21.51/bfd/elflink.c:9661
/usr/local/bin/ld: BFD (GNU Binutils) 2.21.51.20101204 assertion fail
../../binutils-2.21.51/bfd/elflink.c:7972
[...]

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

--- Comment #39 from H.J. Lu <hjl.tools at gmail dot com> 2010-12-05 18:23:55 UTC ---
(In reply to comment #38)
> (In reply to comment #36)
> 
> $ touch empty.c
> $ gcc -nodefaultlibs -r -flto -fuse-linker-plugin empty.c
> /usr/local/bin/ld: BFD (GNU Binutils) 2.21.51.20101204 assertion fail
> ../../binutils-2.21.51/bfd/elflink.c:9661
> /usr/local/bin/ld: BFD (GNU Binutils) 2.21.51.20101204 assertion fail
> ../../binutils-2.21.51/bfd/elflink.c:7972
> [...]

Please try:

http://sourceware.org/ml/binutils/2010-12/msg00197.html

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

Dave Korn <davek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
           See Also|                            |http://www.sourceware.org/b
                   |                            |ugzilla/show_bug.cgi?id=122
                   |                            |48
         Resolution|                            |FIXED

--- Comment #40 from Dave Korn <davek at gcc dot gnu.org> 2010-12-06 17:54:12 UTC ---
The implicit -static problem is fixed at the gcc end, and the remaining aspects
of this problem will be resolved by HJ's patch to binutils (see 'See also'
URL).

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d dot g dot gorbachev at gmail dot com]

------- Comment #15 from d dot g dot gorbachev at gmail dot com  2010-01-18 18:48 -------
Created an attachment (id=19649)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19649&action=view)
Simple patch

It leaves -plugin-opt in LINK_COMMAND_SPEC, but I think it is not quite right,
as LIBGCC_SPEC and LIB_SPEC are redefined by many targets (and, for example,
choose libc or libc_p depending on -p / -pg / -profile).

GCC r155915 bootstrapped with this patch on i686-pc-linux-gnu with and without
--disable-shared option. It seems to work.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[espindola at gcc dot gnu dot org]

------- Comment #14 from espindola at gcc dot gnu dot org  2010-01-17 05:47 -------
I don't think that in general we can support things like -nodefaultlibs -lc
-lgcc. Doing that with static libraries and lto is such an uncommon case that
it is probably OK to ask the user to also pass -Wl,-pass-through...

I am not familiar with the driver code, but if we already have SPEC stings that
are used only when linking libgcc or libc, that is the perfect place to add
%{fuse-linker-plugin: -plugin-opt=-pass-through=...}


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d dot g dot gorbachev at gmail dot com]

------- Comment #13 from d dot g dot gorbachev at gmail dot com  2010-01-16 16:55 -------
(In reply to comment #12)

> What options other than -nostdlib prevent gcc from linking libgcc and
> libc in the produced binary?

I know of -nodefaultlibs.

> To implement this we should always pass -plugin-opt=-pass-through=
> %(lto_libgcc) and -plugin-opt=-pass-through=-lc to the linker unless
> something like -nostdlib is used.

Wouldn't it be better to add %{fuse-linker-plugin:
-plugin-opt=-pass-through=...} to LIBGCC_SPEC and to LIB_SPEC?

However, the driver does not really know, whether we link with libgcc and/or
libc. Someone can use -nodefaultlibs together with -lc -lgcc and the linker
will still fail.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[espindola at gcc dot gnu dot org]

------- Comment #12 from espindola at gcc dot gnu dot org  2010-01-15 15:28 -------
(In reply to comment #11)
> GCC was configured with --disable-shared option. It seems that the driver does
> not understand that --disable-shared implies static libgcc
> 
That is correct. Currently we have

   %{static|static-libgcc:-plugin-opt=-pass-through=%(lto_libgcc)}     \
    %{static:-plugin-opt=-pass-through=-lc}     \

That is, --pass-through is only used for libgcc if -static or -static-libgcc is
given. For libgcc we could probably find out if we are using a static one or
not in a more accurate way.

For libc it is harder. The driver doesn't know that libc is static. It just
passes -lc to ld.

It might be possible to change what we test for. I think it would be OK to
always ask the plugin to pass libgcc and libc to the linker a second time *if
they are being used at all*. My understanding is that an unnecessary static
library will be ignored and a shared library will have no effect since it was
already used anyway. 

To implement this we should always pass -plugin-opt=-pass-through=%(lto_libgcc)
and -plugin-opt=-pass-through=-lc to the linker unless something like -nostdlib
is used.

What options other than -nostdlib prevent gcc from linking libgcc and libc in
the produced binary?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690

[Bug driver/42690] Undefined reference errors with -flto -fuse-linker-plugin

[d dot g dot gorbachev at gmail dot com]

------- Comment #11 from d dot g dot gorbachev at gmail dot com  2010-01-15 11:03 -------
GCC was configured with --disable-shared option. It seems that the driver does
not understand that --disable-shared implies static libgcc


-- 

d dot g dot gorbachev at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
          Component|lto                         |driver
         Resolution|WORKSFORME                  |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690