Re: gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith <psmith@gnu.org>]

On Thu, 2018-02-08 at 01:17 +0100, Marc Glisse wrote:
> On Wed, 7 Feb 2018, Paul Smith wrote:
> > > > My question is, what do I need to do to ensure this behavior
> > > > persists if I create a global operator new/delete?
> > > > 
> > > > Is it sufficient to ensure that the symbol for our shared
> > > > library global new/delete symbols are hidden and not global,
> > > > using a linker map or -fvisibility=hidden?
> > > 
> > > I think so (hidden implies not-interposable, so locally bound),
> > > but I don't have much experience there.
> > 
> > OK I'll pursue this for now.
> 
> I answered too fast. It isn't just new/delete that need to be hidden.
> It is also anything that uses them and might be used in both
> contexts.  For instance, std::allocator<char>::allocate is an inline
> function that calls operator new. You get one version that calls
> new1, and one version that calls new2. If you don't do anything
> special, the linker keeps only one (more or less arbitrarily). So I
> believe you need -fvisibility=hidden to hide everything but a few
> carefully chosen interfaces.

At the moment I'm compiling all my code with -fvisibility=hidden, and
marking out specific symbols to be public in the source code, with
__attribute__((visibility("default"))).

This is nice andeasy, but I've grown frustrated with it.  In order to
run our unit tests we must statically link them with the code; we can't
create a shared library because all the internal symbols that the unit
tests want to refer to are hidden.  Since we have about 150 individual
unit test programs and each one statically links all the code it uses
(although we're using GTest framework and I call these "unit tests",
they're not really unit tests in the classical sense that they mock out
a single class for test), that uses a lot of disk space and takes a lot
of link time during the builds... it's already aggravating and we're
only adding more unit tests over time.

So in the near future I intend to reset this and compile all the code
without -fvisibility=hidden, then when I create our shared library I'll
generate a linker map to make only specific symbols visible and hide
everything else.

What would be ideal is if I could continue to use the visibility
attributes in the source code to mark out symbols I wanted to publish. 
If that information were preserved in the ELF output in a way I could
extract with a script running objdump or readelf on the object files,
for example, then I could automate the generation of the proper linker
map for my shared library.

However the last time I checked this, visibility("default") didn't
leave a trace in the object file unless -fvisibility=hidden was used to
make the default visibility hidden.

Too bad.