Long-term plan for C++98/C++11 incompatibility

Long-term plan for C++98/C++11 incompatibility

[James Y Knight]

I just noted at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49561 (due to
std::list), that it's currently impossible to use any C++11-compiled code
in a program which also uses any C++98 code, even if the two pieces of
code never actually touch each other or share objects. After I noted that,
paolo and redi then told me it was in fact already broken long before, due
to numerous other ODR violations in libstdc++ when compiled in the
different modes. I just hadn't noticed. D'oh.

So, it's actually rather tricky to correctly use --std=c++0x. While it
*looks* like you can just compile your own code with that flag (it appears
to work), it seems that it doesn't actually work, and is expected to work
less and less as time goes on. (Presumably std::string will be made
non-refcounting soon to be C++11-conformant, breaking interoperability
much more.)

I guess to start, it would have been nice if there was a big warning on
http://gcc.gnu.org/projects/cxx0x.html telling me not to use c++0x mode
unless there are no objects compiled with c++98 linked into the same
executable. I know I'm not the only one who didn't realize how broken it
was to do that. I also suspect there's plenty of other people who still
don't know that, and will be unpleasantly surprised by mysterious
seg-faults soon.

I completely understand that the current support in GCC is still
experimental, and I should expect nothing from it. But what I'd really
like to have a handle on is what things are supposed to look like when
C++11 support is no longer experimental. What's the planned final state of
affairs?

* Will STL-using code compiled with -std=c++98 and -std=c++11 remain
incompatible forever, or only until it's no longer experimental?

* If so, will version namespaces be used to give c++98 std::* and c++11
std::* non-conflicting symbols eventually, so that at least the same
program can load two shared libs that only use the STL internally and not
in their interfaces?

* For that matter, are the c++98 and c++11 languages even link-compatible
without STL? Is the incompatibility only due to the #ifdefs in the STL
headers or is it more fundamental than that?

* How are binary linux distributions expected to support c++11 programs?
(in that they'll need to eventually provide shared libraries to link
against which a c++11-compiled binary can use).

It seems to me that it can't be done like previous ABI transitions (just
recompile the whole world with the new ABI, boom done), since C++11 is not
source compatible with c++98.

So should a distro compile two copies of all C++ libraries, one for C++11
and one for C++98? Then, I suppose people will have to change e.g.
"-lPocoNet" to "-lPocoNetCXX11" in their link lines? Or maybe ld/gcc will
be modified to automatically choose a different library search path, based
on the compilation mode?

Or will there be a giant flag day after most projects compile can compile
in both c++11 and c++98, where c++98 support is removed, and everything is
compiled in c++11 from then on? (but then, how would we get there: all
developers would have to compile their entire dependent set of libraries
from scratch themselves to test their code, I guess.)

I hope the experts here have a better idea of how this is all going to
work that I do. IMO it would be best if if the new ABI could be enabled
separately from the C++11 language, so that you *could* compile everything
with the new ABI, and have c++98 and c++11 code interoperate. But I have
no clue if that's even remotely feasible.


Thanks for any insight! This seems to be a lot more complex than I naively
thought the transition was going to be.

Re: Long-term plan for C++98/C++11 incompatibility

[Jonathan Wakely]

On 7 October 2011 23:24, James Y Knight wrote:
> I just noted at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49561 (due to
> std::list), that it's currently impossible to use any C++11-compiled code
> in a program which also uses any C++98 code, even if the two pieces of
> code never actually touch each other or share objects.

I think that's overstating the case.  There are some incompatiblities
in the standard library, but there's no problem linking e.g.

int f() { return 0; }

and

int f();
int g() { return f(); }

So it's not true that it's "impossible" to link "any" code using the two modes.

> (Presumably std::string will be made
> non-refcounting soon to be C++11-conformant, breaking interoperability
> much more.)

Yes, std::string will be made non-reference counting at some point,
but as a non-reference-counted string is valid in C++98 as well, one
option would be to switch to a non-ref-counted string for both
-std=c++98 and -std=c++11 modes.  I'm not saying that's what will
happen, but it could be.  The same is true of an O(1)
std::list::size().  For the next major ABI change we could switch
c++98 mode to use a non-ref-counted string and O(1) list::size.
Currently we aren't changing string or list in c++98 mode, because
that would break code. But we are changing it for c++0x mode, because
people want the C++11 semantics.  Breaking things for people using the
experimental c++0x mode is less catastrophic, as noone is forced to
use that mode and noone has years of existing code relying on its
semantics.

> * Will STL-using

Standard Library, not STL, please.

> code compiled with -std=c++98 and -std=c++11 remain
> incompatible forever, or only until it's no longer experimental?

I don't think it's been settled, but as I said above, there are ways
to change both modes at once. Currently we can't do that
> * If so, will version namespaces be used to give c++98 std::* and c++11
> std::* non-conflicting symbols eventually, so that at least the same
> program can load two shared libs that only use the STL internally and not
> in their interfaces?

Maybe.  Or symbol versioning.

> * For that matter, are the c++98 and c++11 languages even link-compatible
> without STL? Is the incompatibility only due to the #ifdefs in the STL
> headers or is it more fundamental than that?

The basic ABI hasn't changed, the differences are because the standard
library code is different in c++0x mode (e.g. functions take different
parameter types), not because the compiler does different things when
compiling it.

> * How are binary linux distributions expected to support c++11 programs?
> (in that they'll need to eventually provide shared libraries to link
> against which a c++11-compiled binary can use).
>
> It seems to me that it can't be done like previous ABI transitions (just
> recompile the whole world with the new ABI, boom done), since C++11 is not
> source compatible with c++98.
>
> So should a distro compile two copies of all C++ libraries, one for C++11
> and one for C++98? Then, I suppose people will have to change e.g.
> "-lPocoNet" to "-lPocoNetCXX11" in their link lines? Or maybe ld/gcc will
> be modified to automatically choose a different library search path, based
> on the compilation mode?

Having separate libs is a posibility, but it might make more sense to
have a single lib with versioned symbols.

> Or will there be a giant flag day after most projects compile can compile
> in both c++11 and c++98, where c++98 support is removed, and everything is
> compiled in c++11 from then on? (but then, how would we get there: all
> developers would have to compile their entire dependent set of libraries
> from scratch themselves to test their code, I guess.)

Not going to happen.

> I hope the experts here have a better idea of how this is all going to
> work that I do. IMO it would be best if if the new ABI could be enabled
> separately from the C++11 language, so that you *could* compile everything
> with the new ABI, and have c++98 and c++11 code interoperate. But I have
> no clue if that's even remotely feasible.

The current ABI changes are caused by C++11 requirements on the
library, so I don't see how that's possible.

Re: Long-term plan for C++98/C++11 incompatibility

[Jonathan Wakely]

P.S. we already document how to link applications using two
incompatible versions of libstdc++:
http://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html#abi.testing.multi

(whether you can actually make that work in practice is another matter!)

The situation is similar for C++98 and C++11 code, if the incompatible
standard library types aren't used in the library interfaces then they
should be able to coexist and not interfere.

If in a future release we use namespace and/or symbol versioning to
keep the c++98 and c++11 code distinct then they could coexist in the
same library and won't interfere at all.

Re: Long-term plan for C++98/C++11 incompatibility

[Gabriel Dos Reis]

On Fri, Oct 7, 2011 at 5:24 PM, James Y Knight <foom@fuhm.net> wrote:

> I guess to start, it would have been nice if there was a big warning on
> http://gcc.gnu.org/projects/cxx0x.html telling me not to use c++0x mode
> unless there are no objects compiled with c++98 linked into the same
> executable.

I was under the impression c++0x was explicitly documented as experimental.

-- Gaby

RE: Long-term plan for C++98/C++11 incompatibility

[Joe Buck]

On Fri, Oct 7, 2011 at 5:24 PM, James Y Knight <foom@fuhm.net> wrote:

> I guess to start, it would have been nice if there was a big warning on
> http://gcc.gnu.org/projects/cxx0x.html telling me not to use c++0x mode
> unless there are no objects compiled with c++98 linked into the same
> executable.

Gabriel Dos Reis [gdr@integrable-solutions.net] wrote:
> I was under the impression c++0x was explicitly documented as experimental.

Yes. But I hope that some thought is devoted to figuring out how this problem
can be dealt with when c++11 support is transitioned to a fully supported feature.
Eventually there would need to be one libstdc++ that programs link against and
run whether they use c++98 or c++11. I would expect there to be restrictions,
but it's a problem that eventually needs to be tackled.

Re: Long-term plan for C++98/C++11 incompatibility

[Gabriel Dos Reis]

On Fri, Oct 7, 2011 at 9:16 PM, Joe Buck <Joe.Buck@synopsys.com> wrote:
> On Fri, Oct 7, 2011 at 5:24 PM, James Y Knight <foom@fuhm.net> wrote:
>
>> I guess to start, it would have been nice if there was a big warning on
>> http://gcc.gnu.org/projects/cxx0x.html telling me not to use c++0x mode
>> unless there are no objects compiled with c++98 linked into the same
>> executable.
>
> Gabriel Dos Reis [gdr@integrable-solutions.net] wrote:
>> I was under the impression c++0x was explicitly documented as experimental.
>
> Yes. But I hope that some thought is devoted to figuring out how this problem
> can be dealt with when c++11 support is transitioned to a fully supported feature.

C++11 is essentially binary incompatible with C++98.

The best thing people should do is to take it seriously that that they should
not attempt to mix or play loose.

> Eventually there would need to be one libstdc++ that programs link against and
> run whether they use c++98 or c++11. I would expect there to be restrictions,
> but it's a problem that eventually needs to be tackled.

My opinion is that it would an exercise in futility, frustration, and
possibly deception
to try to make people believe that there are sets of simple rules they
can follow
to mix their C++98 binaries with fully compatible C++11 library.  They
would have to
recompile the source code.

Re: Long-term plan for C++98/C++11 incompatibility

[Miles Bader]

FWIW, I seem to have no obvious problems compiling with -std=c++0x,
and then linking with system c++ libraries that were presumably
compiled using default options.... (e.g., I use the OpenEXR library,
which is C++)

So if there are incompatibilities, they don't seem to be fatal...

-Miles

-- 
Hers, pron. His.

Re: Long-term plan for C++98/C++11 incompatibility

[Joe Buck]

On Fri, Oct 07, 2011 at 07:35:17PM -0700, Gabriel Dos Reis wrote:
> C++11 is essentially binary incompatible with C++98.

Only partially.  The layout for user-defined classes is the same, and
code sequences for calls that don't include new features like rvalue
references is the same.  Some very important classes from the standard
library are different, and that creates an incompatibility.

> The best thing people should do is to take it seriously that that they should
> not attempt to mix or play loose.

Unfortunately, distros aren't going to start shipping two versions of
every C++ library, so people are going to have to solve the problem that
you are saying should not be solved, or at least determine which cases
are problems and which aren't.  If common libraries are all c++98-only
this will slow the adoption of c++11: forcing a flag day when this is not
necessary is probably not the right thing to do.

I wrote:
> Eventually there would need to be one libstdc++ that programs link against and
> run whether they use c++98 or c++11. I would expect there to be restrictions,
> but it's a problem that eventually needs to be tackled.

Gaby writes:

> My opinion is that it would an exercise in futility, frustration, and
> possibly deception to try to make people believe that there are sets of
> simple rules they can follow to mix their C++98 binaries with fully
> compatible C++11 library.  They would have to recompile the source code.

They will need to build their code with the same compiler, yes, and it
won't be binary-compatible with older versions.  But as of today, the
templates in libstdc++ expand differently depending on language mode, so
the library is already providing both sets of interfaces.

A bump to libstdc++ major version number could be made that ensures, for
example, that std::string satisfies the requirements of both languages.
It's also possible that the code in the standard library uses c++11 features
internally even when the user has specified c++98 (to get move semantics,
for example).

Re: Long-term plan for C++98/C++11 incompatibility

[Gabriel Dos Reis]

On Mon, Oct 10, 2011 at 5:25 PM, Joe Buck <Joe.Buck@synopsys.com> wrote:
> On Fri, Oct 07, 2011 at 07:35:17PM -0700, Gabriel Dos Reis wrote:
>> C++11 is essentially binary incompatible with C++98.
>
> Only partially.  The layout for user-defined classes is the same, and

PODness has changed from C++98.


> code sequences for calls that don't include new features like rvalue
> references is the same.  Some very important classes from the standard
> library are different, and that creates an incompatibility.

It is not just class layout.  It is also function signatures.

>
>> The best thing people should do is to take it seriously that that they should
>> not attempt to mix or play loose.
>
> Unfortunately, distros aren't going to start shipping two versions of
> every C++ library, so people are going to have to solve the problem that
> you are saying should not be solved, or at least determine which cases
> are problems and which aren't.  If common libraries are all c++98-only
> this will slow the adoption of c++11: forcing a flag day when this is not
> necessary is probably not the right thing to do.

By no means am I preventing anybody from solving that problem if they
believe that is how their time is best spent.  I am offering only a realistic
assessment.  When distros deam g++ support for C++11 sufficiently
well advanced that they feel adventurous enough to officially support
the experimental support, then I am confident they will do the responsible
thing: offer two sets of libraries, as they did in the past when we
moved to GCC3.

>
> I wrote:
>> Eventually there would need to be one libstdc++ that programs link against and
>> run whether they use c++98 or c++11. I would expect there to be restrictions,
>> but it's a problem that eventually needs to be tackled.
>
> Gaby writes:
>
>> My opinion is that it would an exercise in futility, frustration, and
>> possibly deception to try to make people believe that there are sets of
>> simple rules they can follow to mix their C++98 binaries with fully
>> compatible C++11 library.  They would have to recompile the source code.
>
> They will need to build their code with the same compiler, yes, and it
> won't be binary-compatible with older versions.  But as of today, the
> templates in libstdc++ expand differently depending on language mode, so
> the library is already providing both sets of interfaces.

For the record, I am not endorsing this as a recommendation for those
interested in C++11 -- I am already stated my recommendation.

>
> A bump to libstdc++ major version number could be made that ensures, for
> example, that std::string satisfies the requirements of both languages.
> It's also possible that the code in the standard library uses c++11 features
> internally even when the user has specified c++98 (to get move semantics,
> for example).
>
>

Re: Long-term plan for C++98/C++11 incompatibility

[Jeffrey Yasskin]

On Mon, Oct 10, 2011 at 5:07 PM, Gabriel Dos Reis
<gdr@integrable-solutions.net> wrote:
> On Mon, Oct 10, 2011 at 5:25 PM, Joe Buck <Joe.Buck@synopsys.com> wrote:
>> On Fri, Oct 07, 2011 at 07:35:17PM -0700, Gabriel Dos Reis wrote:
>>> C++11 is essentially binary incompatible with C++98.
>>
>> Only partially.  The layout for user-defined classes is the same, and
>
> PODness has changed from C++98.

Sorry for digging this up months later, but this caught my eye. I've
heard several people strongly claim that the core ABI didn't change,
and here you seem to be contradicting them. Can you give a concrete
example of a user-defined class that doesn't use any standard library
features but changes layout between C++98 and C++11 when compiled with
gcc?

Thanks,
Jeffrey

Re: Long-term plan for C++98/C++11 incompatibility

[Jason Merrill]

On 10/10/2011 08:07 PM, Gabriel Dos Reis wrote:
> PODness has changed from C++98.

Class layout in the ABI still uses the C++98 definition of POD.

Jason

Re: Long-term plan for C++98/C++11 incompatibility

[Jason Merrill]

On 10/10/2011 08:07 PM, Gabriel Dos Reis wrote:
> PODness has changed from C++98.

Class layout in the ABI still uses the C++98 definition of POD.

Jason

RE: Long-term plan for C++98/C++11 incompatibility

[Joe Buck]

On 10/10/2011 08:07 PM, Gabriel Dos Reis wrote:
> PODness has changed from C++98.

Jason Merrill wrote:

> Class layout in the ABI still uses the C++98 definition of POD.

But does this actually matter?  If I understand correctly, more classes are POD under the C++11
rules than the C++98 rules, but are there any classes that are legal C++98 that require a different
layout under the new rules?  Can anyone produce an example of a real (and not a theoretical)
binary incompatibility?

Re: Long-term plan for C++98/C++11 incompatibility

[Jason Merrill]

On 01/05/2012 01:38 PM, Joe Buck wrote:
>> Class layout in the ABI still uses the C++98 definition of POD.
>
> But does this actually matter?

Yes, since PODness affects the use of tail padding.  But it isn't a 
source of ABI incompatibility since "POD for the purpose of layout" 
isn't changing.

Jason