visibility and -Wl,--version-script

visibility and -Wl,--version-script

[Mathieu Lacage]

hi,

Here is below some test code I wrote to attempt to understand how I
could use -Wl,--version-script. Obviously, it does not work (just
download, untar and make).
http://www-sop.inria.fr/members/Mathieu.Lacage/test-visib.tar.gz

For those who don't want to download this tarball, what I am doing is
first build a shared library with this header:
void A (void);
class Test
{
public:
  void Do (void);
};

Then, I build a main program:
#include "libtest.h"

int main (int argc, char *argv[])
{
  Test a;
  a.Do ();
  A ();
}

The above is pretty straightforward but what I am doing is this:
mmathieu@mathieu-boulot:~/code/tmp/test-visib$ make
g++ -c -fpic -fvisibility=hidden -o libtest.o libtest.cc
g++ -shared -Wl,--version-script=libtest.so.map -o libtest.so libtest.o
g++ -c -o maintest.o maintest.cc
g++ -L. -ltest -o maintest maintest.o
maintest.o: In function `main':
maintest.cc:(.text+0x18): undefined reference to `Test::Do()'
maintest.cc:(.text+0x1d): undefined reference to `A()'
collect2: ld returned 1 exit status
make: *** [maintest] Error 1
mathieu@mathieu-boulot:~/code/tmp/test-visib$

The intent of the above build commands is to build the libtest.so
library in a way which avoids any indirections through the GOT and PLT
to call functions internally and to specify the set of public
functions I want to export to user programs with libtest.so.map
defined as:
{
global:
  extern "C++" {
    Test*;
    A*;
  };
local: *;
};

So, what I expected the --version-script option to do is to generate a
set of of GOT/PLT entries which give access to the underlying hidden
symbols. It seems, however, that what this option does is merely
filter out from the set of existing GOT/PLT entries the ones which
should appear in the final symbol table.

The question, then, is whether or not what I am trying to do is
possible without having to define a large set of symbol aliases (I am
not even sure how to define a set of aliases for a class) which define
different visibility attributes for the original symbol and for the
alias symbol. The crux of the issue here is that what I am trying to
do is to make my library not use the GOT/PLT for any internal function
calls but to allow client programs to access all of the libraries'
public functions through the GOT/PLT. I understand that what I want to
do is likely to break some of the fine print of the C++ standard but,
I think I can live with that so, is there a way to do this ?

any help would be welcome,
Mathieu
-- 
Mathieu Lacage <mathieu.lacage@gmail.com>

Re: visibility and -Wl,--version-script

[Brian Dessent]

Mathieu Lacage wrote:

> g++ -L. -ltest -o maintest maintest.o

It's not related to your issue, but this is wrong.  The order that you
specify things on the link command line matters, and -ltest needs to
come after objects that reference symbols in libtest.so.  In your
specific situation you can get away with it, but not in other situations
such as if -ltest was a static archive, or with shared libraries on
non-Linux systems.

> The question, then, is whether or not what I am trying to do is
> possible without having to define a large set of symbol aliases (I am
> not even sure how to define a set of aliases for a class) which define
> different visibility attributes for the original symbol and for the
> alias symbol. The crux of the issue here is that what I am trying to
> do is to make my library not use the GOT/PLT for any internal function
> calls but to allow client programs to access all of the libraries'
> public functions through the GOT/PLT. I understand that what I want to
> do is likely to break some of the fine print of the C++ standard but,
> I think I can live with that so, is there a way to do this ?

You've read <http://people.redhat.com/drepper/dsohowto.pdf>, correct?

You're confusing two things.  The version script can cause the linker to
hide or expose symbols from the symbol table, but it does not alter
their visibility.  Your Test::Do() and A() are still hidden and thus
can't be exported as dynamic symbols.

If you want to use -fvisibility=hidden you need to arrange for default
visibility for those symbols that you do want to export with
__attribute__ ((visibility("default"))) or using #pragma GCC visibility
push/pop.

But if you simply want to avoid PLT/GOT overhead for symbols not
exported, why not just make those functions/variables static?  It has
the same effect, and works with any compiler including gcc < 4.x which
didn't have -fvisibility.  Of course this doesn't apply if you need to
access the internal symbols from another object in the same library.

Brian

Re: visibility and -Wl,--version-script

[Mathieu Lacage]

On Fri, Jul 18, 2008 at 7:42 PM, Brian Dessent <brian@dessent.net> wrote:

>> The question, then, is whether or not what I am trying to do is
>> possible without having to define a large set of symbol aliases (I am
>> not even sure how to define a set of aliases for a class) which define
>> different visibility attributes for the original symbol and for the
>> alias symbol. The crux of the issue here is that what I am trying to
>> do is to make my library not use the GOT/PLT for any internal function
>> calls but to allow client programs to access all of the libraries'
>> public functions through the GOT/PLT. I understand that what I want to
>> do is likely to break some of the fine print of the C++ standard but,
>> I think I can live with that so, is there a way to do this ?
>
> You've read <http://people.redhat.com/drepper/dsohowto.pdf>, correct?

yes.

>
> You're confusing two things.  The version script can cause the linker to
> hide or expose symbols from the symbol table, but it does not alter
> their visibility.  Your Test::Do() and A() are still hidden and thus
> can't be exported as dynamic symbols.

yes, the output of my test program clearly shows that my expectations
were indeed invalid.

> If you want to use -fvisibility=hidden you need to arrange for default
> visibility for those symbols that you do want to export with
> __attribute__ ((visibility("default"))) or using #pragma GCC visibility
> push/pop.

Right, but if I do that, gcc will use GOT/PLT lookups to access these
symbols even when doing accesses from within the library and that is
precisely what I would like to avoid.

> But if you simply want to avoid PLT/GOT overhead for symbols not
> exported, why not just make those functions/variables static?  It has
> the same effect, and works with any compiler including gcc < 4.x which
> didn't have -fvisibility.  Of course this doesn't apply if you need to
> access the internal symbols from another object in the same library.

Yes.

Mathieu
-- 
Mathieu Lacage <mathieu.lacage@gmail.com>

Re: visibility and -Wl,--version-script

[Brian Dessent]

Mathieu Lacage wrote:

> Right, but if I do that, gcc will use GOT/PLT lookups to access these
> symbols even when doing accesses from within the library and that is
> precisely what I would like to avoid.

In that case you need to create a second alias of the symbol with hidden
visibility for internal use within the library.  The Drepper paper
explains how to do this in section 2.2.7.

The problem with doing this in C++ code is that the alias declaration
needs the assembler name of the function which includes the mangling,
and there's no way to automatically generate that short of compiling a
testcase and looking at the assembly output.  You could arrange for this
to be done automatically with some kind of multi-pass compilation scheme
with tons of macro magic: first run g++ -S over the source, then use a
perl/awk/whatever script to extract the mangled names of exported
functions and substitute them back into a second compilation of the same
file with the alias declarations added.  To me that sounds like a
fragile and complicated way of achieving a modest optimization.  The
wrapper function approach mentioned in that same section is probably the
best way to go in the face of C++.

Brian

Re: visibility and -Wl,--version-script

[Mathieu Lacage]

On Fri, Jul 18, 2008 at 8:43 PM, Brian Dessent <brian@dessent.net> wrote:
> Mathieu Lacage wrote:
>
>> Right, but if I do that, gcc will use GOT/PLT lookups to access these
>> symbols even when doing accesses from within the library and that is
>> precisely what I would like to avoid.
>
> In that case you need to create a second alias of the symbol with hidden
> visibility for internal use within the library.  The Drepper paper
> explains how to do this in section 2.2.7.
>
> The problem with doing this in C++ code is that the alias declaration
> needs the assembler name of the function which includes the mangling,
> and there's no way to automatically generate that short of compiling a
> testcase and looking at the assembly output.  You could arrange for this
> to be done automatically with some kind of multi-pass compilation scheme
> with tons of macro magic: first run g++ -S over the source, then use a
> perl/awk/whatever script to extract the mangled names of exported
> functions and substitute them back into a second compilation of the same
> file with the alias declarations added.  To me that sounds like a
> fragile and complicated way of achieving a modest optimization.  The
> wrapper function approach mentioned in that same section is probably the
> best way to go in the face of C++.

Both options require different names to be used when calling a
function internally and when calling a function through the GOT/PLT.
That is clearly not something I will be able to retrofit in our
codebase. So, it looks like all bets are off and that I should just
build everything statically without -fpic if  I want to avoid the
GOT/PLT lookups.

Mathieu
-- 
Mathieu Lacage <mathieu.lacage@gmail.com>