[Bug lto/47497] New: [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[Bug lto/47497] New: [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

           Summary: [4.6 Regression] SPEC CPU 2006 failed to link with LTO
           Product: gcc
           Version: 4.6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: hjl.tools@gmail.com


On Linux/x86-64, I got

 cat
spec/2006/x86_64/spec/benchspec/CPU2006/450.soplex/build/build_peak_lto.0000/make.err
make[4]: warning: jobserver unavailable: using -j1.  Add `+' to parent make
rule.
/tmp/cc4HXntx.ltrans9.ltrans.o: In function `non-virtual thunk to
soplex::SoPlex::~SoPlex()':
cc4HXntx.ltrans9.o:(.text._ZN6soplex6SoPlexD1Ev+0x8): undefined reference to
`.LTHUNK0.11251'
collect2: ld returned 1 exit status
specmake[3]: *** [soplex] Error 1

Revision 169192 is OK.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #1 from H.J. Lu <hjl.tools at gmail dot com> 2011-01-27 20:49:51 UTC ---
Revision 169303 is bad.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rguenth at gcc dot gnu.org
   Target Milestone|---                         |4.6.0

--- Comment #2 from H.J. Lu <hjl.tools at gmail dot com> 2011-01-27 22:42:13 UTC ---
It is caused by revision 169284:

http://gcc.gnu.org/ml/gcc-cvs/2011-01/msg00908.html

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #3 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-01-28 11:42:11 UTC ---
Please specify the linker you are using and compile-flags you are using to
build SPEC - you should know the information you gave is not enough to
reproduce the problem.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #4 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-01-28 12:01:13 UTC ---
Works for me with -O3 -ffast-math -funroll-loops and

> rpm -q binutils
binutils-2.20.0-0.7.9

using gold as plugin-ld.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #5 from H.J. Lu <hjl.tools at gmail dot com> 2011-01-28 13:46:06 UTC ---
(In reply to comment #3)
> Please specify the linker you are using and compile-flags you are using to
> build SPEC - you should know the information you gave is not enough to
> reproduce the problem.

I used

-DSPEC_CPU -DNDEBUG   -O2 -ffast-math -fwhole-program -flto=jobserver
-fuse-linker-plugin   -DSPEC_CPU_LP64  -fno-strict-aliasing

I got the same error with gold 1.11 from CVS:

# /export/regression/rrs/169284/usr/bin/g++ -B./ -O2 -ffast-math
-fwhole-program -flto=jobserver -fuse-linker-plugin  -DSPEC_CPU_LP64       
changesoplex.o didxset.o dsvector.o dvector.o enter.o example.o factor.o
forest.o idxset.o leave.o lpcolset.o lprow.o lprowset.o message.o mpsinput.o
nameset.o slufactor.o solve.o soplex.o spxaggregatesm.o spxbasis.o spxbounds.o
spxchangebasis.o spxdefaultpr.o spxdefaultrt.o spxdefines.o spxdesc.o
spxdevexpr.o spxequilisc.o spxfastrt.o spxgeneralsm.o spxharrisrt.o
spxhybridpr.o spxid.o spxio.o spxlp.o spxlpfread.o spxmpsread.o spxmpswrite.o
spxparmultpr.o spxquality.o spxredundantsm.o spxrem1sm.o spxscaler.o spxshift.o
spxsolve.o spxsolver.o spxstarter.o spxsteeppr.o spxsumst.o spxvecs.o
spxvectorst.o spxweightpr.o spxweightst.o ssvector.o svector.o svset.o timer.o
unitvector.o update.o updatevector.o vector.o vsolve.o                     -o
soplex
/usr/local/bin/ld: /tmp/ccZ4RILg.ltrans8.ltrans.o: in function non-virtual
thunk to
soplex::SoPlex::~SoPlex():ccZ4RILg.ltrans8.o(.text._ZN6soplex6SoPlexD1Ev+0x8):
error: undefined reference to '.LTHUNK0.11251'
collect2: ld returned 1 exit status

GCC is configured to use ppl-0.11 and cloog-0.16.1 with

 --enable-clocale=gnu --with-system-zlib --with-demangler-in-ld --enable-shared
--enable-threads=posix --enable-haifa --enable-checking
--enable-languages=c,c++,fortran --enable-cloog-backend=isl
--with-ppl-include=/opt/gnu/include --with-ppl-lib=/opt/gnu/lib64
--with-cloog-include=/opt/gnu/include --with-cloog-lib=/opt/gnu/lib64

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2011.01.28 14:07:32
                 CC|                            |hubicka at gcc dot gnu.org
     Ever Confirmed|0                           |1

--- Comment #6 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-01-28 14:07:32 UTC ---
I can reproduce it.  It seems to be an interaction of -fuse-linker-plugin
and -fwhole-program.  The combinations "-fuse-linker-plugin"
and "-fwhole-program -fno-use-linker-plugin" work ok.  I think mixing
linker-plugin and whole-program does not make much sense (linker-plugin
provides a more precise whole-program idea).

Honza?  Why is -fwhole-program not ignored, or why should it not be?
Why does it make a difference at all here?

I can't see how the patch should cause this btw, eventually it leads to
different linker resolutions(?) or to different partitioning decisions.
The issue also reproduces with -flto-partition=1to1 (but not with =none,
though that doesn't go through the WPA path).

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[hubicka at ucw dot cz]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #7 from Jan Hubicka <hubicka at ucw dot cz> 2011-01-30 17:36:01 UTC ---
> Honza?  Why is -fwhole-program not ignored, or why should it not be?
> Why does it make a difference at all here?
Well, it is effectively ignored - i.e. on symbols with resolution info attached
it should do nothing.  So it seems we fail to get resolution info from plugin
correctly.  I will check.

Honza

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #8 from H.J. Lu <hjl.tools at gmail dot com> 2011-02-06 16:57:48 UTC ---
(In reply to comment #7)
> > Honza?  Why is -fwhole-program not ignored, or why should it not be?
> > Why does it make a difference at all here?
> Well, it is effectively ignored - i.e. on symbols with resolution info attached
> it should do nothing.  So it seems we fail to get resolution info from plugin
> correctly.  I will check.
> 
> Honza

Any updates on this?

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P1
            Summary|[4.6 Regression] SPEC CPU   |[4.6 Regression] SPEC CPU
                   |2006 failed to link with    |2006 failed to link with
                   |LTO                         |LTO -fuse-linker-plugin
                   |                            |-fwhole-program

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
         AssignedTo|unassigned at gcc dot       |hubicka at gcc dot gnu.org
                   |gnu.org                     |

--- Comment #9 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-02-12 13:03:25 UTC ---
Hmm, I guess the problem is that we ignore resolution info of aliases when
deciding on privatizing the symbol and then we privatize everything including
the aliases.  This should be easy to fix.

I am travelling till 15th, so will look more into this then. It would be great
to have a testcase ;)

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[steven at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Steven Bosscher <steven at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |WAITING
                 CC|                            |steven at gcc dot gnu.org

--- Comment #10 from Steven Bosscher <steven at gcc dot gnu.org> 2011-02-25 23:04:07 UTC ---
No test case available. Please, someone, provide a test case so that those
unlucky souls without access to SPEC2k6 can try to help debug this problem.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at ucw dot cz]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #11 from Jan Hubicka <hubicka at ucw dot cz> 2011-02-26 09:36:00 UTC ---
testcases for this kind of -fwhole-program issues tends to be tricky.  I had
quite busy week and now I am travelling till Monday night. So far I only
reviewed
the visibility code and it does not seem to miss aliases. Unless someone
beats me, I will fix this at Tuesday since this is last (one of latest?) P1.

Honza

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #12 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-02-26 10:50:30 UTC ---
Hmm, this is actually same body alias internal thunk.  Will need to look more.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #13 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-02-26 13:52:31 UTC ---
The think in question (.LTHUNK0.10948) appears in:

__base_dtor /613(-1) @0x7f43c6b49000 (asm: _ZN6soplex6SoPlexD2Ev.local.405)
availability:available analyzed 194 time, 22 benefit (419 after inlining) 138
size, 22 benefit (239 after inlining) address_taken prevailing_def_ironly
finalized inlinable                                                             
  called by: __base_dtor /3839 __base_dtor /3679 __base_dtor /5838
_ZN6soplex9SPxSolverC2ENS_6SoPlex4TypeENS1_14RepresentationE.constprop.576/552
__base_dtor /641
  calls: __base_dtor /80 __base_dtor /6133 operator delete []/5260 __base_dtor
/292 __base_dtor /611 __base_dtor /292 __base_dtor /611 __base_dtor /611
__base_dtor /292 __base_dtor /292 __base_dtor /292 __base_dtor /292 __base_dtor
/292 __base_dtor /292 __base_dtor /292 __base_dtor /4379 (inlined) (1.00 per
call) (can throw external) __base_dtor /6460 (inlined) (1.00 per call) operator
delete []/5260 (0.51 per call) __base_dtor /292 (1.00 per call) __base_dtor
/6087 (inlined) (1.00 per call) __base_dtor /292 (1.00 per call) __base_dtor
/6053 (inlined) (1.00 per call) __base_dtor /6079 (inlined) (1.00 per call)
__base_dtor /292 (1.00 per call) __base_dtor /292 (1.00 per call) __base_dtor
/292 (1.00 per call) __base_dtor /292 (1.00 per call) __base_dtor /292 (1.00
per call) __base_dtor /292 (1.00 per call) __base_dtor /292 (1.00 per call)
__base_dtor /292 (1.00 per call)                   
  References:  var:_ZTVN6soplex6SoPlexE (addr) var:_ZTVN6soplex6SoPlexE (addr)
fn:__gxx_personality_v0/6126 (addr)

In ltrans9 it however appears as
__base_dtor /69(-1) @0x7feac53e5840 (asm: _ZN6soplex6SoPlexD2Ev.local.405)
availability:available analyzed 194 time, 22 benefit 138 size, 22 benefit
needed address_taken prevailing_def_ironly finalized inlinable asm_written
  called by:
  calls:
  References:  var:_ZTVN6soplex6SoPlexE (addr) var:_ZTVN6soplex6SoPlexE (addr)
fn:__gxx_personality_v0/76 (addr)
  Refering this function:  var:_ZTVN6soplex6SoPlexE (addr)
var:_ZTVN6soplex6SoPlexE (addr) var:_ZTVN6soplex6SoPlexE (addr)
var:_ZTVN6soplex6SoPlexE (addr) var:_ZTVN6soplex6SoPlexE (addr)
var:_ZTVN6soplex6SoPlexE (addr)
  aliases & thunks: *.LTHUNK1/75 (asm: *.LTHUNK1.25152) *.LTHUNK0/74 (asm:
*.LTHUNK0.74174) __comp_dtor /73 (asm: _ZN6soplex6SoPlexD1Ev.local.406)
_ZThn352_N6soplex6SoPlexD1Ev/72 (thunk of *.LTHUNK0 fixed offset -352 virtual
value 0 has virtual offset 0) (asm: _ZThn352_N6soplex6SoPlexD1Ev)

So apparently the problem is that THUNK0 decl gets two different manglings. (it
gets output as *.LTHUNK0.74174)

I am puzzled however how this can happen - even if we don't merge alias decls
from thunk info, I don't see how local decl can possibly get duplicated.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #14 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-02-26 14:32:42 UTC ---
The problem is in the alias merging code in lto-symtab. It does:
alias->thunk.alias = prevailing_node->decl;
that is wrong for thunks, as for thunks pointing to thunk (like this one) the
alias is not the decl of function thunk is associated for but decl of thunk.

Fixing this problem however leads to futher problems (ICE at ltrans streaming
in time, because we merge aliases incorrectly...)

I don't think the aliases should be merged actually, just the aliases
associated with prevailed decl removed. thunk.alias should always point to the
prevailing decl, but I guess if we just drop aliases from other definitions
this should just work.

Jakub, what was logic for the current alias merging code?

The problem in soplex is that function in question is comdat and it has two
actual definitions. With decl merging we mix together local label for thunk
from the first definition with local label for the thunk from second
definition.
It bit surprises me how this happens, given that the aliases should match and
we should consistently choose one or another.

Honza

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #15 from Jakub Jelinek <jakub at gcc dot gnu.org> 2011-02-28 08:55:33 UTC ---
(In reply to comment #14)
> Jakub, what was logic for the current alias merging code?

Is that my code?  That would surprise me, I had to do something primarily with
same body ctors/dtors, enhancing it for thunks was something you've done.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #17 from Jakub Jelinek <jakub at gcc dot gnu.org> 2011-02-28 09:14:18 UTC ---
For more on LTHUNK see http://gcc.gnu.org/ml/gcc-patches/2003-06/msg02603.html
When merging you either should take all thunks and real definitions from one
object, or another one, not mix them, so I don't currently understand how you
could mix LTHUNK from different objects.  But you know, I'm not familiar with
LTO too much.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at ucw dot cz]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #16 from Jan Hubicka <hubicka at ucw dot cz> 2011-02-28 08:58:39 UTC ---
> > Jakub, what was logic for the current alias merging code?
> 
> Is that my code?  That would surprise me, I had to do something primarily with
> same body ctors/dtors, enhancing it for thunks was something you've done.

I think that bit was inherited from same body code (LTHUNK probably wants to be
same
body alias anyway).  I will investigate it more tonight.
Obviously merging it all together is not what happens in linker and we should
immidate
linker behaviour here.

Honza

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #18 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-03-01 19:07:36 UTC ---
Created attachment 23507
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=23507
patch I am testing

Hi,
the problem is with thunks referring thunks or aliases.

lto-symtab is wrong here and when moving thunks&aliases associated with one
node to the other node, it overwrites thunk.alias by the destination node. It
consequently turns thunk referring another thunk into thunk referring the
functoin itself.

I fixed this by adding extra loop setting alias decl to prevailing decl. Richi,
perhaps there is better place to put this?  I don't like it being in the loop
redirecting thunks&aliases from one node to another because

 1) I think that loop is not quite correct.  When one function is prevailed by
another, local static thunk from the first function should not be redirected to
another.  That happens correctly and is harmless (we end up with dead thunk)
 2) It may happen that thunks get prevailed other way than nodes they are
aliased with.  We use comdat groups that prevents this from happening, but I
would rather not 100% rely on this on all targets since not all targets
implements comdat groups.  So I think it is more robust to simply merge the
decls in alias field like we merge other decls.

Fixing this problem cause different problem with streaming.  When we have alias
A referring function F and alias B referring alias A and we are unlucky with
prevailing and other things, we might end up streaming alias B before alias A. 
This leads us to call cgraph_same_body_alias on decl of A before A is added to
cgraph as an alias.  Consequently cgraph_same_body_alias does nothing later
when we try to create alias A itself, because the node already exists.

This patch fixes it by adding node pointer into the cgraph_same_body_alias and
cgraph_add_thunk so the thunks&aliases can be added in random order w/o
problems as long as the function they are associated with is already in cgraph.

I am testing patch now.

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |ASSIGNED

--- Comment #19 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-03-01 19:09:36 UTC ---
mine, obviously ;)

Still have no simple testcase...

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

--- Comment #20 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-03-04 18:49:27 UTC ---
Author: hubicka
Date: Fri Mar  4 18:49:23 2011
New Revision: 170682

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=170682
Log:
    PR lto/47497
    * lto-symtab.c (lto_cgraph_replace_node): Do not set thunk.alias.
    (lto_symtab_merge_cgraph_nodes_1): Update thunk.alias pointers here.
    * cgraph.h (cgraph_same_body_alias, cgraph_add_thunk): Add node pointers.
    * cgraph.c (cgraph_same_body_alias_1, cgraph_same_body_alias,
    cgraph_add_thunk): Add node pointers.
    * lto-cgraph.c (lto_output_node): Verify that thunks&aliases are
    associated to right node.
    (input_node): Update use of cgraph_same_body_alias
    and cgraph_add_thunk.

    * optimize.c (maybe_clone_body): Update call of cgraph_same_body_alias
    and cgraph_add_thunk.
    * method.c (make_alias_for_thunk, use_thunk): Likewise.
    * mangle.c (mangle_decl): Likewise.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/cgraph.c
    trunk/gcc/cgraph.h
    trunk/gcc/cp/ChangeLog
    trunk/gcc/cp/mangle.c
    trunk/gcc/cp/method.c
    trunk/gcc/cp/optimize.c
    trunk/gcc/lto-cgraph.c
    trunk/gcc/lto-symtab.c

[Bug lto/47497] [4.6 Regression] SPEC CPU 2006 failed to link with LTO -fuse-linker-plugin -fwhole-program

[hubicka at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47497

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #21 from Jan Hubicka <hubicka at gcc dot gnu.org> 2011-03-04 18:51:26 UTC ---
Fixed.