Re: gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith <psmith@gnu.org>]

On Wed, 2018-02-07 at 16:38 -0700, Martin Sebor wrote:
> I'm not sure I see how defining operator new inline can work
> unless you recompile the world (i.e., all the DSOs used by
> the program, including libstdc++). As Marc already hinted,
> if libstdc++ dynamically allocates an object using the default
> operator new and returns that object to the program to destroy,
> it will end up causing a mismatch.  The same can happen in
> the opposite direction.  To avoid such mismatches the entire
> program needs to use a single operator new (each of
> the required forms), and the only safe way to do that
> is to define each out-of-line.

I'm aware of these issues, and I know it's a dangerous game.  As I
mentioned I wasn't surprised, really, that eventually it caught us out.

However, it did work.  We don't use huge amounts of the STL but we use
basics like vector, string, a bit of unordered_map, etc., and those
worked fine.  All that's required for it to work is that either both
the new and delete were both performed inside libstdc++, or both the
new and delete were performed in STL header file implementations.  In
the former case that memory is coming from the system alloc, not
jemalloc, but the amount is small enough that it's not worrisome.  In
the latter case it will use jemalloc via the inline.

The problem comes in where you do the new in an STL header and the
delete in the compiled libstdc++, or vice versa... that's what I ran
into in GCC 7.3.

On GNU/Linux you can just replace malloc/free using non-global symbols
in the shared library, to ensure even libstdc++.a implementations use
jemalloc.  Unfortunately this is not possible in Windows.


On Wed, 7 Feb 2018 Jonathan Wakely <jwakely.gcc@gmail.com> writes:
> The code to read a string from an istream is instantiated in the
> library, so the string gets resized (allocating using malloc) by code
> inside libstdc++.so.6 and then the destructor is run (deallocating
> using your inline operator delete) in main.

We don't use C++ iostream in our code.  Although, I'm sure GTest does
use it in THEIR code.  Of course, for unit tests we don't really care
which allocator is used as long as there's no mismatch.  I haven't
investigated exactly how it all worked in 6.2, I can only tell you that
for our use, it did :).

Thanks for the conversation.  I'm moving forward with a real global
operator new/delete and working out the magic needed to ensure those
symbols are not global in our shared library.