Dealing with C++98/11 ABI incompatibilities

Dealing with C++98/11 ABI incompatibilities

[Jason Merrill]

There seem to be two areas of concern with compatibility between the 
C++98 and C++11 ABIs:

1) Changes to member function return types, as with std::complex.  For 
these, the two functions could coexist except that they are mangled the 
same way.  So I propose that we introduce a mechanism to change the 
mangled name of a member function.  Unlike with attribute ((alias)), the 
declared name would only be visible at the source level, not in the 
assembly output.

My inclination is to just have a single declaration and have an 
attribute to change its name for linkage purposes.

In the passed we have discussed using inline namespaces to do this sort 
of renaming, but that doesn't help with individual member functions 
within a class, like the situation in std::complex.

2) Object layout changes to std::list and std::basic_string.  For these 
types, there is no way to both retain backward compatibility with older 
C++98 code and conform to the C++11 standard.  The best we can hope for 
is to allow old code to coexist with new code so long as they don't try 
to touch the same string/list objects.

As above, one solution to this would be to change the linkage name of 
the new versions so that they don't clash with the old versions.  But 
that isn't enough in this case, as it wouldn't affect the linkage name 
of a class like

struct Wrap { std::string s; };

so we would need some way to cause the name decoration to propagate to 
other containing/derived classes.

It seems that ELF symbol versioning could be useful for this purpose. 
If we were to extend the visibility attribute to also handle symbol 
versions, that could handle a lot of issues.  If Wrap uses the GLIBCXX_4 
version of string, then Wrap would also be marked with the GLIBCXX_4 
version, and any functions that deal with Wrap would be marked with that 
version, and so on.

I'm not familiar enough with the intricacies of ELF versioning to be 
confident that this would work; is anyone else?  In particular I'm not 
sure how the interaction of versioned and non-versioned code will work.

Jason

Re: Dealing with C++98/11 ABI incompatibilities

[Paolo Carlini]

Hi,

On 07/03/2012 09:18 PM, Jason Merrill wrote:
> 2) Object layout changes to std::list and std::basic_string.  For 
> these types, there is no way to both retain backward compatibility 
> with older C++98 code and conform to the C++11 standard.  The best we 
> can hope for is to allow old code to coexist with new code so long as 
> they don't try to touch the same string/list objects.
Just wanted to mention / record the case of std::time_get which has an 
additional virtual function (do_get) in C++11. I don't think it affects 
in important ways the points you are making.

Paolo.

Re: Dealing with C++98/11 ABI incompatibilities

[Daniel Krügler]

2012/7/3 Paolo Carlini <paolo.carlini@oracle.com>:
> Hi,
>
> On 07/03/2012 09:18 PM, Jason Merrill wrote:
>>
>> 2) Object layout changes to std::list and std::basic_string.  For these
>> types, there is no way to both retain backward compatibility with older
>> C++98 code and conform to the C++11 standard.  The best we can hope for is
>> to allow old code to coexist with new code so long as they don't try to
>> touch the same string/list objects.
>
> Just wanted to mention / record the case of std::time_get which has an
> additional virtual function (do_get) in C++11. I don't think it affects in
> important ways the points you are making.

Isn't there a similar problem with the long long related additions of
virtual function
to IO/localization as in std::num_get and std::num_put?

- Daniel

Re: Dealing with C++98/11 ABI incompatibilities

[Paolo Carlini]

On 07/03/2012 10:10 PM, Daniel Krügler wrote:
> Isn't there a similar problem with the long long related additions of 
> virtual function to IO/localization as in std::num_get and std::num_put? 
Probably not, because, if I understand correctly what you are saying, we 
have long long overloads in C++98 mode too. Pretty old stuff.

Paolo.

Re: Dealing with C++98/11 ABI incompatibilities

[Daniel Krügler]

2012/7/3 Paolo Carlini <paolo.carlini@oracle.com>:
> On 07/03/2012 10:10 PM, Daniel Krügler wrote:
>>
>> Isn't there a similar problem with the long long related additions of
>> virtual function to IO/localization as in std::num_get and std::num_put?
>
> Probably not, because, if I understand correctly what you are saying, we
> have long long overloads in C++98 mode too. Pretty old stuff.

Yes, you understand me perfectly - thanks for clarification!

-Daniel

Re: Dealing with C++98/11 ABI incompatibilities

[Jason Merrill]

On 07/03/2012 04:02 PM, Paolo Carlini wrote:
> Just wanted to mention / record the case of std::time_get which has an
> additional virtual function (do_get) in C++11. I don't think it affects
> in important ways the points you are making.

As long as the new virtual function is at the end, it would only affect 
classes that derive from std::time_get in a multiple inheritance 
context.  But to be safe we could bump the time_get ABI as well as the 
others.

Jason

Re: Dealing with C++98/11 ABI incompatibilities

[Jason Merrill]

On 07/03/2012 03:18 PM, Jason Merrill wrote:
> It seems that ELF symbol versioning could be useful for this purpose. If
> we were to extend the visibility attribute to also handle symbol
> versions, that could handle a lot of issues. If Wrap uses the GLIBCXX_4
> version of string, then Wrap would also be marked with the GLIBCXX_4
> version, and any functions that deal with Wrap would be marked with that
> version, and so on.
>
> I'm not familiar enough with the intricacies of ELF versioning to be
> confident that this would work; is anyone else? In particular I'm not
> sure how the interaction of versioned and non-versioned code will work.

Jakub pointed out to me on IRC that there's no way to specify the 
version of an undefined symbol, which would be necessary to make this 
work.  So we're back to some change to the mangled name.

Jason

Re: Dealing with C++98/11 ABI incompatibilities

[Jeffrey Yasskin]

On Tue, Jul 3, 2012 at 1:02 PM, Paolo Carlini <paolo.carlini@oracle.com> wrote:
> Hi,
>
>
> On 07/03/2012 09:18 PM, Jason Merrill wrote:
>>
>> 2) Object layout changes to std::list and std::basic_string.  For these
>> types, there is no way to both retain backward compatibility with older
>> C++98 code and conform to the C++11 standard.  The best we can hope for is
>> to allow old code to coexist with new code so long as they don't try to
>> touch the same string/list objects.
>
> Just wanted to mention / record the case of std::time_get which has an
> additional virtual function (do_get) in C++11. I don't think it affects in
> important ways the points you are making.

It would be good to add this and anything else I missed to
http://gcc.gnu.org/wiki/Cxx11AbiCompatibility so there's one place
people can read to get the lay of the land.


At Google we found several libraries that had C-only interfaces, but
exposed C++ symbols from their implementations. Clearly it would be
better for them to just hide the symbols, but not all vendors have
enough build system expertise to do that. Jason's proposal to get the
conflicting names mangled differently for different language versions
would definitely help with the migration process.

Thanks,
Jeffrey

Re: Dealing with C++98/11 ABI incompatibilities

[Jakub Jelinek]

On Tue, Jul 03, 2012 at 05:01:29PM -0400, Jason Merrill wrote:
> On 07/03/2012 03:18 PM, Jason Merrill wrote:
> >It seems that ELF symbol versioning could be useful for this purpose. If
> >we were to extend the visibility attribute to also handle symbol
> >versions, that could handle a lot of issues. If Wrap uses the GLIBCXX_4
> >version of string, then Wrap would also be marked with the GLIBCXX_4
> >version, and any functions that deal with Wrap would be marked with that
> >version, and so on.
> >
> >I'm not familiar enough with the intricacies of ELF versioning to be
> >confident that this would work; is anyone else? In particular I'm not
> >sure how the interaction of versioned and non-versioned code will work.
> 
> Jakub pointed out to me on IRC that there's no way to specify the
> version of an undefined symbol, which would be necessary to make

Yeah, plus it might interfere with C++ libraries using symbol versioning
for the things it has been designed for.

> this work.  So we're back to some change to the mangled name.

Yeah.  I'll repeat what I said on IRC, I'd add some attribute on classes
where the compiler would propagate either a bitmask from the attribute
(using bit or) or some version level (using max) to all classes related
to that class somehow (having class with the attribute as data member, or
pointer/reference to it, etc., recursively) and all mangled names
referencing those use some mangling extension (letter + the bitmask or
version level somehow encoded) inside of the mangled names.
We could reserve a couple of bits for libstdc++, and pick one of the bits
for C++11 ABI incompatible changes, say std::list (_List_Impl in it?) would
get the attribute, similarly std::string, etc.  Anything referring to
std::list somehow, including your wrapper example, or perhaps even
struct foo { std::list<whatever> ***ptr; }; would then be mangled
differently in C++11 and could coexist with C++03 std::list/std::string.

	Jakub

Re: Dealing with C++98/11 ABI incompatibilities

[Richard Guenther]

On Wed, Jul 4, 2012 at 8:14 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Tue, Jul 03, 2012 at 05:01:29PM -0400, Jason Merrill wrote:
>> On 07/03/2012 03:18 PM, Jason Merrill wrote:
>> >It seems that ELF symbol versioning could be useful for this purpose. If
>> >we were to extend the visibility attribute to also handle symbol
>> >versions, that could handle a lot of issues. If Wrap uses the GLIBCXX_4
>> >version of string, then Wrap would also be marked with the GLIBCXX_4
>> >version, and any functions that deal with Wrap would be marked with that
>> >version, and so on.
>> >
>> >I'm not familiar enough with the intricacies of ELF versioning to be
>> >confident that this would work; is anyone else? In particular I'm not
>> >sure how the interaction of versioned and non-versioned code will work.
>>
>> Jakub pointed out to me on IRC that there's no way to specify the
>> version of an undefined symbol, which would be necessary to make
>
> Yeah, plus it might interfere with C++ libraries using symbol versioning
> for the things it has been designed for.
>
>> this work.  So we're back to some change to the mangled name.
>
> Yeah.  I'll repeat what I said on IRC, I'd add some attribute on classes
> where the compiler would propagate either a bitmask from the attribute
> (using bit or) or some version level (using max) to all classes related
> to that class somehow (having class with the attribute as data member, or
> pointer/reference to it, etc., recursively) and all mangled names
> referencing those use some mangling extension (letter + the bitmask or
> version level somehow encoded) inside of the mangled names.
> We could reserve a couple of bits for libstdc++, and pick one of the bits
> for C++11 ABI incompatible changes, say std::list (_List_Impl in it?) would
> get the attribute, similarly std::string, etc.  Anything referring to
> std::list somehow, including your wrapper example, or perhaps even
> struct foo { std::list<whatever> ***ptr; }; would then be mangled
> differently in C++11 and could coexist with C++03 std::list/std::string.

That would not address the issue of interoperability of a C++03 library
with a C++11 program or vice versa if they are using any of the changed
interfaces for interoperability.  Passing pointers to such objects still would
work and break unnoticed.

I think that it would be best to keep C++03 and C++11 interoperable
for all classes/objects as much as possible within the v6 ABI.

I understand that std::list for example would still be conforming with
C++03 with the extra member, so we can have a v7 ABI which still
allows interoperation between the two languages within the same ABI.
Not sure about the other changes.  So, what's your proposal for the
v7 ABI - drop the version manglings again and have a compatible
C++03 and C++11?

Thanks,
Richard.

Re: Dealing with C++98/11 ABI incompatibilities

[Michael Matz]

Hi,

On Wed, 4 Jul 2012, Richard Guenther wrote:

> > [... mangling change ...]
> 
> That would not address the issue of interoperability of a C++03 library 
> with a C++11 program or vice versa if they are using any of the changed 
> interfaces for interoperability.  Passing pointers to such objects still 
> would work and break unnoticed.

But it's the best we can possibly do _if_ we want to conform to c++11 in 
the v6 ABI.  IMHO we want to, so we have to at least make 98/11 code 
coexist in the same process image, which means mangling changes.  That the 
code can't interoperate is understood.

It will by the way not break unnoticed: interfaces using the problematic 
types would be mangled differently, and hence c++98 code would silently be 
resolved to a c++11 variant expecting a different layout.


Ciao,
Michael.

Re: Dealing with C++98/11 ABI incompatibilities

[Richard Guenther]

On Wed, Jul 4, 2012 at 3:22 PM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Wed, 4 Jul 2012, Richard Guenther wrote:
>
>> > [... mangling change ...]
>>
>> That would not address the issue of interoperability of a C++03 library
>> with a C++11 program or vice versa if they are using any of the changed
>> interfaces for interoperability.  Passing pointers to such objects still
>> would work and break unnoticed.
>
> But it's the best we can possibly do _if_ we want to conform to c++11 in
> the v6 ABI.  IMHO we want to, so we have to at least make 98/11 code
> coexist in the same process image, which means mangling changes.  That the
> code can't interoperate is understood.
>
> It will by the way not break unnoticed: interfaces using the problematic
> types would be mangled differently, and hence c++98 code would silently be
> resolved to a c++11 variant expecting a different layout.

Hmm, ok.  I can see people wanting to use C++11 as implementation language
but interfacing with C++98, thus have a way to use the C++98 std::list from
C++11 code.

Btw, why use a bitmask?  Isn't it enough to have a single number, viewed
as an ABI suffix?

Richard.

>
> Ciao,
> Michael.

Re: Dealing with C++98/11 ABI incompatibilities

[Michael Matz]

On Wed, 4 Jul 2012, Michael Matz wrote:

> It will by the way not break unnoticed: interfaces using the problematic 
> types would be mangled differently, and hence c++98 code would silently 

... would _not_ silently ...

> be resolved to a c++11 variant expecting a different layout.

Re: Dealing with C++98/11 ABI incompatibilities

[Jason Merrill]

On 07/04/2012 09:41 AM, Richard Guenther wrote:
> Btw, why use a bitmask?  Isn't it enough to have a single number, viewed
> as an ABI suffix?

Well, I think the question is whether we want to create a mechanism 
which is only useful for libstdc++ versioning or one which is useful for 
ABI changes in other libraries as well.  A single number which applies 
to the entire mangled name would be fine for just libstdc++, but I think 
people would appreciate having a more general facility.

I was thinking of something like defining an ABI suffix for a name, so 
that list would look like list__2 or some such in the mangled name.  And 
then a wrapper class would be wrapper__2.  If the wrapper class also 
contains a user class with an ABI suffix of "Boost2", then the wrapper 
would look like wrapper__2__Boost2 or wrapper__Boost2__2, depending on 
the order of the members.  There would be no need to repeat a suffix 
from something that is itself mangled, such as a function argument type 
or a template argument, but we would need to propagate a suffix from the 
return type of a (non-template) function or the type of a variable as 
well as from member/base types in a class.

Jason

Re: Dealing with C++98/11 ABI incompatibilities

[Jason Merrill]

On 07/03/2012 03:18 PM, Jason Merrill wrote:
> 2) Object layout changes to std::list and std::basic_string.  For these
> types, there is no way to both retain backward compatibility with older
> C++98 code and conform to the C++11 standard.  The best we can hope for
> is to allow old code to coexist with new code so long as they don't try
> to touch the same string/list objects.
>
> As above, one solution to this would be to change the linkage name of
> the new versions so that they don't clash with the old versions.  But
> that isn't enough in this case, as it wouldn't affect the linkage name
> of a class like
>
> struct Wrap { std::string s; };
>
> so we would need some way to cause the name decoration to propagate to
> other containing/derived classes.

I raised this issue on the ABI list, and someone pointed out to me that 
incomplete classes make quiet propagation impossible; any full 
coexistence solution will require changes to affected user types.  There 
seem to be two options:

1) No propagation, add a warning to suggest that users decorate their 
types appropriately.
2) Propagate to class definitions, give an error if there was an earlier 
forward declaration without the appropriate tag.

I lean toward #1; people who rebuild everything regularly won't be 
affected by it, and people who need coexistence can use the warning flag 
to help them adjust.

Jason