[Bug lto/48200] New: linking shared library with LTO results in different exported symbols

[Bug lto/48200] New: linking shared library with LTO results in different exported symbols

[zeev.tarantov at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

           Summary: linking shared library with LTO results in different
                    exported symbols
           Product: gcc
           Version: 4.6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: zeev.tarantov@gmail.com


The library is libpci.so.3.1.7, released 2010-01-31, part of every linux
distro.
Homepage is: http://mj.ucw.cz/pciutils.html, source repository is:
http://www.kernel.org/git/?p=utils/pciutils/pciutils.git;a=shortlog

Object files compiled with CFLAGS: "-O2 -fPIC -Wall -Wno-parentheses
-Wstrict-prototypes -Wmissing-prototypes -flto".

If linked with: gcc -shared -Wl,-O1 -Wl,--as-needed -Wl,-soname,libpci.so.3
-Wl,--version-script=libpci.ver -o libpci.so.3.1.7 init.o access.o generic.o
dump.o names.o filter.o names-hash.o names-parse.o names-net.o names-cache.o
params.o caps.o sysfs.o proc.o i386-ports.o  -lz -lresolv

Then (as expected): objdump -T libpci.so.3.1.7 | grep pci_fill_info
00000000000032f0 g    DF .text  000000000000006c (LIBPCI_3.0) pci_fill_info
00000000000032f0 g    DF .text  000000000000006c  LIBPCI_3.1  pci_fill_info

But if linked with LTO: gcc -shared -Wl,-O1 -Wl,--as-needed -Wl,-flto
-Wl,-soname,libpci.so.3 -Wl,--version-script=libpci.ver -o libpci.so.3.1.7
init.o access.o generic.o dump.o names.o filter.o names-hash.o names-parse.o
names-net.o names-cache.o params.o caps.o sysfs.o proc.o i386-ports.o  -lz
-lresolv

Then: objdump -T libpci.so.3.1.7 | grep pci_fill_info

No such symbol.

gcc (Gentoo SVN) 4.6.0-pre9999 20110318 (prerelease) rev. 171169
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

GNU ld (Linux/GNU Binutils) 2.21.51.0.7.20110306
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

I don't know enough to tell whether the problem is in pciutils, binutils or
gcc. If this is invalid and I'm just spamming, I apologize. Please kindly point
me in the right direction to fix this issue.

Preproccessed source is (relevant parts):

int pci_fill_info(struct pci_dev *, int flags)
__attribute__((visibility("default")));
# 170 "pci.h"

int pci_fill_info_v30(struct pci_dev *, int flags)
__attribute__((visibility("default")));
int pci_fill_info_v31(struct pci_dev *, int flags)
__attribute__((visibility("default")));


int
pci_fill_info_v31(struct pci_dev *d, int flags)
{
  if (flags & 0x10000)
    {
      flags &= ~0x10000;
      d->known_fields = 0;
      pci_free_caps(d);
    }
  if (flags & ~d->known_fields)
    d->known_fields |= d->methods->fill_info(d, flags & ~d->known_fields);
  return d->known_fields;
}


;
extern int pci_fill_info_v30(struct pci_dev *d, int flags)
__attribute__((alias("pci_fill_info_v31")));
asm(".symver " "pci_fill_info_v30" "," "pci_fill_info@LIBPCI_3.0");
asm(".symver " "pci_fill_info_v31" "," "pci_fill_info@@LIBPCI_3.1");

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2011.03.21 12:58:42
     Ever Confirmed|0                           |1

--- Comment #1 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-03-21 12:58:42 UTC ---
This is not enough to reproduce the issue, even making the source compile
ends up failing with

/usr/bin/ld: t.o: version node not found for symbol pci_fill_info@LIBPCI_3.0
/usr/bin/ld: failed to set dynamic section sizes: Bad value
collect2: ld returned 1 exit status

please provide a complete compilable testcase.  Thanks.

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[zeev.tarantov at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #2 from Zeev Tarantov <zeev.tarantov at gmail dot com> 2011-03-21 18:18:26 UTC ---
Created attachment 23740
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=23740
sources, object files and two shared objects: one good, one bad (resulted from
linking with LTO)

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[zeev.tarantov at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #3 from Zeev Tarantov <zeev.tarantov at gmail dot com> 2011-03-21 18:26:46 UTC ---
I am sorry but I don't know which parts are relevant. The source is GPL2, I
just ran "make SHARED=yes CFLAGS="-fPIC -O2 -flto" lib/libpci.so.3.1.7" and
then linked the SO a second time with -Wl,-flto (as if it was in the LDFLAGS)
and attached a tarball with the result. You can compare the two (renamed)
shared objects, built from the same object files and version script.

I'll try to construct a minimal (single file, single symbol) test case.

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[zeev.tarantov at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #4 from Zeev Tarantov <zeev.tarantov at gmail dot com> 2011-03-21 22:40:12 UTC ---
I've constructed a trivial example, the bug did not reproduce in it.
I've tried -save-temps and -Wl,--verbose but those did not reveal anything.
I do not have a nicer testcase than comment #2.

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW
                 CC|                            |hubicka at gcc dot gnu.org

--- Comment #5 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-03-22 10:02:24 UTC ---
Confirmed.

I think what happens is that the symver global asms get partitioned away
from the function definitions.  With -flto-partition=none it works for me.

> grep pci_fill_info libpci.so.3.1.7.ltrans?.s
libpci.so.3.1.7.ltrans0.s:      .symver
pci_fill_info_v30,pci_fill_info@LIBPCI_3.0
libpci.so.3.1.7.ltrans0.s:      .symver
pci_fill_info_v31,pci_fill_info@@LIBPCI_3.1
libpci.so.3.1.7.ltrans1.s:      call    pci_fill_info_v31@PLT
libpci.so.3.1.7.ltrans1.s:      .globl  pci_fill_info_v31
libpci.so.3.1.7.ltrans1.s:      .type   pci_fill_info_v31, @function
libpci.so.3.1.7.ltrans1.s:pci_fill_info_v31:
libpci.so.3.1.7.ltrans1.s:      .size   pci_fill_info_v31, .-pci_fill_info_v31
libpci.so.3.1.7.ltrans1.s:      .globl  pci_fill_info_v30
libpci.so.3.1.7.ltrans1.s:      .set    pci_fill_info_v30,pci_fill_info_v31
libpci.so.3.1.7.ltrans2.s:      call    pci_fill_info_v31@PLT

so indeed that is what happens (-save-temps appended to the link command
produces those intermediate files).

We can't really know better (we do not parse asm strings), -flto-partition=none
is a workaround.  A fix would be to not use toplevel asms, but I'm not sure
a different way for the symvers exists (maybe it's possible to do entirely
in the linker script ...).

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #6 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-03-22 10:13:49 UTC ---
Btw, the minimal partition size is quite small (1000), 10 times smaller
than our large-unit-insns inline limit and 2.7 times smaller than our
large function inline limit.  Upping that limit by a factor of 10 magically
solves this problem by only using a single partition automagically
(it looks to me that it should be similar to large-unit-insns anyway).

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[hubicka at ucw dot cz]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #7 from Jan Hubicka <hubicka at ucw dot cz> 2011-03-22 23:04:08 UTC ---
Well, upping it only workarounds the problem and you loose parallelizm linking
small files.  I did bit of tunning on this compiling one of spec2k benchmarks
(twolf?) to get smallest time with the cpus I had (4 if I remember)

Honza

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |lto

--- Comment #8 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-03-23 10:06:17 UTC ---
(In reply to comment #7)
> Well, upping it only workarounds the problem and you loose parallelizm linking
> small files.  I did bit of tunning on this compiling one of spec2k benchmarks
> (twolf?) to get smallest time with the cpus I had (4 if I remember)

Well ... I suppose this also heavily depends on your I/O speed.  Nevetheless
the number _looks_ awfully low compared to those others.

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[hubicka at ucw dot cz]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #9 from Jan Hubicka <hubicka at ucw dot cz> 2011-03-23 13:12:37 UTC ---
> Well ... I suppose this also heavily depends on your I/O speed.  Nevetheless
> the number _looks_ awfully low compared to those others.
Well, totally minimizing compilation of small unit is not very worthwhile.  It
was from time I was
trying to stress whopr to get issues like this one soon.

It does not matter if we build application in 1/100th of second or 1/200th of
second and too much parallelizm could slow down build of many small binaries.
I guess we could shoot for resonable user response. UI guys thinks it is about
1/3rd of second, so I would not mind tuning number up until compilation on not
terribly high end system approximately gets to this time.

I don't think this is however terribly related to large unit sizes and inliner
stuff.

Honza

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[khimov at altell dot ru]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Roman Khimov <khimov at altell dot ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |khimov at altell dot ru

--- Comment #10 from Roman Khimov <khimov at altell dot ru> ---
I can confirm the same bug with pciutils 3.2.1 and GCC 4.9.0, although now it's
pci_fill_info_v32().

[Bug lto/48200] linking shared library with LTO results in different exported symbols

[amodra at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Alan Modra <amodra at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |amodra at gmail dot com

--- Comment #11 from Alan Modra <amodra at gmail dot com> ---
There's a similar issue when compiling libgcc for powerpc64.
libgcc/config/rs6000/ibm-ldouble.c has these aliases:
__asm__ (".symver __gcc_qadd,_xlqadd@GCC_3.4\n\t"
     ".symver __gcc_qsub,_xlqsub@GCC_3.4\n\t"
     ".symver __gcc_qmul,_xlqmul@GCC_3.4\n\t"
     ".symver __gcc_qdiv,_xlqdiv@GCC_3.4\n\t"
     ".symver .__gcc_qadd,._xlqadd@GCC_3.4\n\t"
     ".symver .__gcc_qsub,._xlqsub@GCC_3.4\n\t"
     ".symver .__gcc_qmul,._xlqmul@GCC_3.4\n\t"
     ".symver .__gcc_qdiv,._xlqdiv@GCC_3.4");

Which seem to get dropped if you use -flto.

[Bug lto/48200] Implement function attribute for symbol versioning (.symver)

[mark at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #43 from Mark Wielaard <mark at gcc dot gnu.org> ---
It looks there is now some support for a symver function attribute. But it only
accepts the single and double @ forms. This makes things a little awkward when
using a symbol foo itself for foo@@DEFAULT_VERSION. It causes the
(non-versioned) foo symbol to still appear in the object, which doesn't work
very nicely when combined with linker version scripts.

For elfutils I had to workaround that by adding foo also to a non-exported
version section, so that the assembler and linker didn't both try to create a
default version for foo:
https://sourceware.org/pipermail/elfutils-devel/2020q2/002609.html

It works, but feels like a hack. Please also support the @@@ symver variant.

[Bug lto/48200] Implement function attribute for symbol versioning (.symver)

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #44 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Thanks, I am happy we now have real-world use of symver attribute.  I have WIP
patch for better control over the symbol visibility, but I have run into
problems with gas limitations which was fixed by HJ about two weeks ago.
I will try to update the patch and aim for backporting to gcc 10.2.

[Bug lto/48200] Implement function attribute for symbol versioning (.symver)

[mark at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

--- Comment #45 from Mark Wielaard <mark at gcc dot gnu.org> ---
Note that the hack in comment 43 doesn't really work with elfutils since the
.symver attribute doesn't work exactly like the assembly construct with @@@.
The @@@ symver variant see https://sourceware.org/binutils/docs/as/Symver.html

The third usage of the .symver directive is:

.symver name, name2@@@nodename

When name is not defined within the file being assembled, it is treated as
name2@nodename. When name is defined within the file being assembled, the
symbol name, name, will be changed to name2@@nodename. 

Which means it is renamed, so that it doesn't clash when used in a version map
file.

[Bug lto/48200] Implement function attribute for symbol versioning (.symver)

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |10.0
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #46 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
LTO was fixed at r10-5568.

I think we can close this as fixed for GCC 10.

[Bug lto/48200] Implement function attribute for symbol versioning (.symver)

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |ASSIGNED
         Resolution|FIXED                       |---

--- Comment #47 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Mark Wielaard from comment #45)
> Note that the hack in comment 43 doesn't really work with elfutils since the
> .symver attribute doesn't work exactly like the assembly construct with @@@.
> The @@@ symver variant see
> https://sourceware.org/binutils/docs/as/Symver.html
> 
> The third usage of the .symver directive is:
> 
> .symver name, name2@@@nodename
> 
> When name is not defined within the file being assembled, it is treated as
> name2@nodename. When name is defined within the file being assembled, the
> symbol name, name, will be changed to name2@@nodename. 
> 
> Which means it is renamed, so that it doesn't clash when used in a version
> map file.

Reopen for this part of the discussion since I misunderstood it at first.

[Bug lto/48200] Implement function attribute for symbol versioning (.symver)

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|10.0                        |---