C++ ABI Issues

C++ ABI Issues

[Mark Mitchell]

Our testing and investigation has lead to the discovery of several places
where G++'s class layout still does not match the published ABI
specification.  Other vendors (notably HP and Intel) do match the ABI
specification.  Therefore, the ABI specification is probably not going
to change to validate G++'s behavior.

Therefore, we will have to choose whether or not we want to change G++
to match the specification.

Here are the issues:

1. Tail-padding of bit-fields

  Given:

    struct B { virtual void f(); char i1 : 28; };
    struct D : public B { int i2 : 4; };

  G++ incorrectly packs D::i2 into the four bits of tail-padding in
  B.

  This is not only an ABI divergence, but also an optimization issue.
  G++ generates a copy-constructor for B that performs the bit-masking
  operations required to copy only the bitfield at the end of B, which
  is more expensive than copying the entire word.

2. Tail-padding and virtual base classes

  Given:

    struct A { virtual void f(); char c1; };
    struct B { B(); char c2; };
    struct C : public A, public virtual B {};

  G++ does not pack B into the tail padding of A, as required by the ABI
  specification and as implemented in other compilers.

  G++'s behavior is slightly wasteful of space, but is otherwise correct
  in the abstract.

My expectation is that case 1 is relatively uncommon, but that case 2 is
relatively common.

I propose that we fix G++ to match the ABI, but that we issue warnings
about classes whose layout has changed from GCC 3.2.

If we are going to fix G++, we also have to decide how urgently to do
another release.

Thoughts?

(The flame you're about to send about the fact that we didn't catch this
before GCC 3.2 isn't helpful.  Until we have a comprehensive testsuite
(in the works, as discussed), it's very hard to find these things.  And,
even then, there may be issues.  Several free software distributors pushed
hard for a GCC 3.2 release, and everyone knew that there might be more
problems.)

--
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Re: C++ ABI Issues

[Mike Stump]

On Monday, August 26, 2002, at 03:16 PM, Mark Mitchell wrote:

Re: C++ ABI Issues

[Zack Weinberg]

On Mon, Aug 26, 2002 at 03:16:36PM -0700, Mark Mitchell wrote:
> Our testing and investigation has lead to the discovery of several places
> where G++'s class layout still does not match the published ABI
> specification.  Other vendors (notably HP and Intel) do match the ABI
> specification.  Therefore, the ABI specification is probably not going
> to change to validate G++'s behavior.
> 
> Therefore, we will have to choose whether or not we want to change G++
> to match the specification.

Independent of what we decide to do about this specific issue, I
propose that the release of 3.3 be delayed as long as necessary for
testing tools to mature to the point that we have confidence we will
not find further bugs in our implementation of the C++ ABI.

zw

Re: C++ ABI Issues

[Loren James Rittle]

Hi Mark,

> Our testing and investigation has lead to the discovery of several places
> where G++'s class layout still does not match the published ABI
> specification.  Other vendors (notably HP and Intel) do match the ABI
> specification.  Therefore, the ABI specification is probably not going
> to change to validate G++'s behavior. [...]

> I propose that we fix G++ to match the ABI, but that we issue warnings
> about classes whose layout has changed from GCC 3.2.

This sounds quite reasonable to me.  However, this will be quite
annoying in practice, if e.g. libstdc++-v3 exposes something that
triggers issue 2 (I'm fairly sure that it will not trigger issue 1).

> If we are going to fix G++, we also have to decide how urgently to do
> another release.

Seems that low-level C++ ABI issues should be fixed sooner rather than
later now that we claim to adhere to a multi-vendor standard.

Regards,
Loren

Re: C++ ABI Issues

[Joe Buck]

Mark writes:

> Our testing and investigation has lead to the discovery of several places
> where G++'s class layout still does not match the published ABI
> specification. ...
> 
> Here are the issues:
> 
> 1. Tail-padding of bit-fields
> 2. Tail-padding and virtual base classes

Sigh.

> My expectation is that case 1 is relatively uncommon, but that case 2 is
> relatively common.

Question: does anyone know if case 2 affects anything in libstdc++?
(e.g. iostream classes?)

I would be against jumping quickly to a 3.3 that fixes these; we still
won't know if we have caught all of the problems.  These things usually
follow an exponential curve (that is, the rate of finding new bugs in a
given area that you aren't actively changing decays exponentially with
time), so if you've already found two errors in only a week, we probably
aren't done yet.  My wild guess (based just on intuition) is that we'll
find a third within two months, at least.

I recommend instead that, for 3.2.1, we add code to issue a warning when
one of the above situations occur.  We might even want to make the patch
to generate such warnings more prominently available, so that if people
suspect a problem they can check for it now.  Then people who want to
write code that won't be affected by the bug can add explicit padding
as needed to eliminate the tail-padding slot that some vendors would
otherwise exploit.

Old-time GCC users may remember that for years GCC used a different
convention by default to return structs on some OSes, and people dealt
with that by adding a warning that would flag the returning of structs.
The concept was to allow programmers to avoid the areas where
incompatibilities occur.

A more controversial possibility is to add an *option* to correct the
ABI bugs in 3.2.1, but make 3.2 compatibility the default.  I'm less
sure whether this is the right approach; we should first try to determine
how pervasive this problem is.  One possibility: do the ABI-warning patch,
ask our friends at Red Hat, Debian, and FreeBSD to build their whole
distributions+ports with it and see how many warnings we get.  If it
happens a lot, we have something to think about.

> I propose that we fix G++ to match the ABI, but that we issue warnings
> about classes whose layout has changed from GCC 3.2.

I want to see the warnings ASAP, but I'm concerned about the disruption of
another ABI change when we can't be sure that it will be the last one.

Finally, there's the issue of how far we bend over to make it easier to
accomodate the proprietary competition to GCC on GNU/Linux.  That is, who
are we doing this ABI for?  Describing GCC so precisely so that it can be
easily replaced?  Mind you, I *do* want good documentation and adherence
to standards, and I'm against the introduction of any artificial barriers,
but the distributors really, really wanted a release that would re-unify
the world and we gave it to them.  Maybe the best thing in the short term
is for the competition to release patches to make their compilers
bug-compatible with GCC.

As you say, there was plenty of warnings that we might not have caught
everything.

Re: C++ ABI Issues

[David Edelsohn]

>>>>> Joe Buck writes:

Joe> Finally, there's the issue of how far we bend over to make it easier to
Joe> accomodate the proprietary competition to GCC on GNU/Linux.  That is, who
Joe> are we doing this ABI for?  Describing GCC so precisely so that it can be
Joe> easily replaced?  Mind you, I *do* want good documentation and adherence
Joe> to standards, and I'm against the introduction of any artificial barriers,
Joe> but the distributors really, really wanted a release that would re-unify
Joe> the world and we gave it to them.  Maybe the best thing in the short term
Joe> is for the competition to release patches to make their compilers
Joe> bug-compatible with GCC.

	I think it is highly arrogant if GCC says that the ABI is whatever
we implemented.  There is an external ABI specification.  G++ claims that
it follows the spec.  Other proprietary compilers implementing the ABI
were able to get these cases correct without an external conformance
testsuite.  Acting as if the entire world revolves around GCC will not
elicit much respect for GNU/Linux and GCC from the marketplace.

David

Re: C++ ABI Issues

[David S. Miller]

   From: David Edelsohn <dje@watson.ibm.com>
   Date: Mon, 26 Aug 2002 21:31:59 -0400
   
   Other proprietary compilers implementing the ABI were able to get
   these cases correct without an external conformance testsuite.

Note that it is entirely possible that GCC gets areas of
the ABI right that the proprietary compilers got wrong.

Don't make it sound like the proprietary compilers have
gotten the ABI perfectly implemented, it is unlikely that
they have.

Re: C++ ABI Issues

[Joe Buck]

I wrote:
> Finally, there's the issue of how far we bend over to make it easier to
> accomodate the proprietary competition to GCC on GNU/Linux.  That is, who
> are we doing this ABI for?  Describing GCC so precisely so that it can be
> easily replaced?  Mind you, I *do* want good documentation and adherence
> to standards, and I'm against the introduction of any artificial barriers,
> but the distributors really, really wanted a release that would re-unify
> the world and we gave it to them.  Maybe the best thing in the short term
> is for the competition to release patches to make their compilers
> bug-compatible with GCC.

While I don't really think that the last sentence is *really* what should
be done, note that this isn't unprecedented.  For years, there were a
number of deviations between Berkeley's TCP/IP stack and the RFC's, but
most people ran Berkeley's code (even Microsoft), so the world had to be
compatible with the bugs.

David Edelsohn writes:

> 	I think it is highly arrogant if GCC says that the ABI is whatever
> we implemented.  There is an external ABI specification.  G++ claims that
> it follows the spec.  Other proprietary compilers implementing the ABI
> were able to get these cases correct without an external conformance
> testsuite.

While the other vendors may have gotten these two cases correct, how do
you know that they haven't made other errors?  That is, are you certain
that no bugs will be discovered in their conformance?

>  Acting as if the entire world revolves around GCC will not
> elicit much respect for GNU/Linux and GCC from the marketplace.

The GCC 3.2 ABI is what it is.  We shipped the thing.  Saying so isn't
arrogant, it is simply a fact.  We did our best, and there are bugs.  If
you ask Intel whether they will legally warrant that their compiler
matches the spec in all respects, they will decline, as they should.  No
one can make such a promise.

Should we simply withdraw GCC 3.2 at this time, after making promises to
various distributors about it, we will do even more damage.

We will, eventually, need to fix the ABI.  But we should not rush to do
so.  Instead, I proposed to add warnings for areas where we don't match
the spec.

As for "respect from the marketplace": I know of a number of people who
have communicated internally to their organizations that 3.2 was going to
have a stable ABI that would stand up for a while (I was one of them, and
I wasn't alone).  Take that away, and you will do vastly more damage to
GCC's credibility than if you insist that we keep yanking releases and do
3.3, 3.4, 3.5, 3.6 until it works just like Intel's compiler.

Re: C++ ABI Issues

[Jamie Lokier]

Mark Mitchell wrote:
>    G++ incorrectly packs D::i2 into the four bits of tail-padding in
>    B.
> 
>    This is not only an ABI divergence, but also an optimization issue.
>    G++ generates a copy-constructor for B that performs the bit-masking
>    operations required to copy only the bitfield at the end of B, which
>    is more expensive than copying the entire word.

Is there any scenario where the bit-masking is actually necessary?

It seems to me that when B is constructed, it's ok to write to the
padding bits of the destination word.  They might get overwritten again
by later constructors in the sequence, but that is ok.

-- Jamie

Re: C++ ABI Issues

[David Edelsohn]

	All compilers have bugs.  We already know of an ABI mistake in one
of the proprietary compilers which GCC did get right.  The issue is not
whose compiler is better or who has less bugs, it is how GCC addresses ABI
conformance problems, such as the one Mark discovered.

	We need to find a balance between internal GCC stability from
release to release and conformance to external standards.

	As was discussed during the GCC 3.2 transition, my personal
opinion is that GCC needs a binary -fabi-correctly switch which defaults
to *OFF*.  We maintain the GCC 3.2 ABI until a well-defined transition
point in the future when the switch is changed to default to *ON*.  This
allows GCC stability for the time being and interoperability with
proprietary compilers outside the GNU/Linux and *BSD environments.
Propritary compilers which run on GNU/Linux will probably have a GCC
compatibility switch.

	The important points are that we make such a decision for GCC
stability and we have a plan for eventually becoming conformant.  GCC 3.2
may be a transient ABI, but it should not become a GNU/Linux
counter-proposal to the C++ABI which it was trying to implement.  Also, we
should develop a transition plan, as opposed to "we'll switch the option
in some future release when it seems convenient."

	We shouldn't make GCC arbitrarily compatible with Intel's
compiler, and we aren't.  Similarly, we should not ask Intel or HP or any
other proprietary compiler to be arbitrarily compatible with GCC.  It cuts
both ways.  GCC should avoid any appearance of making arbitrary changes to
standards and forcing others to play catch-up.

David

Re: C++ ABI Issues

[Per Bothner]

David Edelsohn wrote:

Re: C++ ABI Issues

[Geoff Keating]

Zack Weinberg <zack@codesourcery.com> writes:

> On Mon, Aug 26, 2002 at 03:16:36PM -0700, Mark Mitchell wrote:
> > Our testing and investigation has lead to the discovery of several places
> > where G++'s class layout still does not match the published ABI
> > specification.  Other vendors (notably HP and Intel) do match the ABI
> > specification.  Therefore, the ABI specification is probably not going
> > to change to validate G++'s behavior.
> > 
> > Therefore, we will have to choose whether or not we want to change G++
> > to match the specification.
> 
> Independent of what we decide to do about this specific issue, I
> propose that the release of 3.3 be delayed as long as necessary for
> testing tools to mature to the point that we have confidence we will
> not find further bugs in our implementation of the C++ ABI.

I think users would prefer a release that might have bugs to no
release at all.  I expect we'll be finding ABI-related bugs for years
to come; we were finding ABI-changing bugs in the _old_ ABI right up
until we switched to the new one.

-- 
- Geoffrey Keating <geoffk@geoffk.org> <geoffk@redhat.com>

Re: C++ ABI Issues

[Jakub Jelinek]

On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
> A more controversial possibility is to add an *option* to correct the
> ABI bugs in 3.2.1, but make 3.2 compatibility the default.  I'm less
> sure whether this is the right approach; we should first try to determine
> how pervasive this problem is.  One possibility: do the ABI-warning patch,
> ask our friends at Red Hat, Debian, and FreeBSD to build their whole
> distributions+ports with it and see how many warnings we get.  If it

This is what we at Red Hat are about to do today.

Personally, I think there is no point to make 3.3 ABI incompatible
with 3.2 without extensive ABI testsuite in place (and I don't think it
is manageable to come up with one in 3.3 timeframe unless it is delayed),
otherwise we're back where G++ used to be - every single version is ABI
incompatible with all the others. The chances some ABI problem will come
up in the next half a year is simply too high without testsuite.
So IMHO the ABI changes could wait for 3.4 together with ABI testsuite.

	Jakub

Re: C++ ABI Issues

[Mark Mitchell]

--On Tuesday, August 27, 2002 03:17:31 AM +0100 Jamie Lokier 
<egcs@tantalophile.demon.co.uk> wrote:

Mark Mitchell wrote:

Re: C++ ABI Issues

[Mark Mitchell]

--On Tuesday, August 27, 2002 08:26:58 AM +0200 Jakub Jelinek 
<jakub@redhat.com> wrote:

On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:

Re: C++ ABI Issues

[Nathan Sidwell]

Mark Mitchell wrote:

> Therefore, we will have to choose whether or not we want to change G++
> to match the specification.

> 
> If we are going to fix G++, we also have to decide how urgently to do
> another release.
We should not rush into producing another release with a slightly tweaked
abi.

My thoughts are,

Patch 3.2 with a -warn-future-abi (defaults to off), and
use that to add warnings of known discrepancies. This should be added
to HEAD.

Add -fabi-patch-level (defaults to '3.2'), to HEAD, and put
the fixes in there, gated on that switch being > 3.2. When 3.3 is due for
release, we can decide whether -fabi-patch-level should default to 3.3 or
remain at 3.2.

If we discover further discrepencies during 3.4's development, they
should be fixed and gated on the switch > 3.3.

nathan

-- 
Dr Nathan Sidwell   ::   http://www.codesourcery.com   ::   CodeSourcery LLC
         'But that's a lie.' - 'Yes it is. What's your point?'
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: C++ ABI Issues

[Allan Sandfeld Jensen]

On Tuesday 27 August 2002 14:33, Nathan Sidwell wrote:
> Mark Mitchell wrote:
> > Therefore, we will have to choose whether or not we want to change G++
> > to match the specification.
> >
> >
> > If we are going to fix G++, we also have to decide how urgently to do
> > another release.
>
> We should not rush into producing another release with a slightly tweaked
> abi.
>
> My thoughts are,
>
> Patch 3.2 with a -warn-future-abi (defaults to off), and
> use that to add warnings of known discrepancies. This should be added
> to HEAD.
Great idea, but I would also like it in the 3.2 branch to be included in 3.2.1 
and called -Wabi

> Add -fabi-patch-level (defaults to '3.2'), to HEAD, and put
> the fixes in there, gated on that switch being > 3.2. When 3.3 is due for
> release, we can decide whether -fabi-patch-level should default to 3.3 or
> remain at 3.2.
Since 3.3 has to stay compatible with 3.2, the 3.2 ABI will most likely have 
to stay default. As such I think your versioning of the ABI is unfortunate.
Instead I would like a -fabi= switch that defaults to 1 being the g++ 3.2 abi, 
and increased at every point-release if new ABI-fixes are in. If you set 
-fabi=0 or -fabi-latest it would default to the most recent patch level. 

Unfortunatly, that way we could end up with a lot of supported patch levels. 
Another solution might just be to only support two ABI levels: 3.2 and "the 
best we can do". "The best we can do" would imply that it could change 
between any two releases, even minor ones! (3.3.1 to 3.3.2) If at one point 
we decide "the best we can do" is good enough it might be stabilized and 
become the ABI for GCC 3.4 or later.

Re: C++ ABI Issues

[Nathan Sidwell]

Allan Sandfeld Jensen wrote:

> > Patch 3.2 with a -warn-future-abi (defaults to off), and
> > use that to add warnings of known discrepancies. This should be added
> > to HEAD.
> Great idea, but I would also like it in the 3.2 branch to be included in 3.2.1
yeah, I meant add it to both.

> Unfortunatly, that way we could end up with a lot of supported patch levels.
> Another solution might just be to only support two ABI levels: 3.2 and "the
> best we can do". "The best we can do" would imply that it could change
> between any two releases, even minor ones! (3.3.1 to 3.3.2) If at one point
> we decide "the best we can do" is good enough it might be stabilized and
> become the ABI for GCC 3.4 or later.
I agree that my suggested version numbering might not be the best. The problem
with a binary switch of 3.2/best-so-far, is that we end up in the same
situation down the road when we release a new compiler with an updated
abi by default, and then discover a new bug. Maybe disociating the abi
patch level from the version number would be best, and only bumping it when
we decide to release a compiler with the best-so-far enabled.

To be specific,

Add -fabi-patch-level-n, which defaults to whatever patchlevel we deem
stable enough. At the moment this will be zero. Any abi patches
are only enabled when abi-patch-level is > stable-patch-level-at-time-of-fix

Add -warn-abi-patch-level, which defaults to off. When enabled, it will
issue a warning for any discrepancy which would be fixed by increasing
-fabi-patch-level. We could make -warn-abi-patch-level indicate the level
to which you'd like to test, the default value would then be
-fabi-patch-level.

To get the latest abi conformance, -fabi-patch-level suffices.

At some random time in the distant future, we can remove support for
lower levels of -fabi-patch-level.

nathan
-- 
Dr Nathan Sidwell   ::   http://www.codesourcery.com   ::   CodeSourcery LLC
         'But that's a lie.' - 'Yes it is. What's your point?'
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: C++ ABI Issues

[Jeff Law]

In message < 20020827082658.A7917@sunsite.ms.mff.cuni.cz >, Jakub Jelinek writes:
 >On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
 >> A more controversial possibility is to add an *option* to correct the
 >> ABI bugs in 3.2.1, but make 3.2 compatibility the default.  I'm less
 >> sure whether this is the right approach; we should first try to determine
 >> how pervasive this problem is.  One possibility: do the ABI-warning patch,
 >> ask our friends at Red Hat, Debian, and FreeBSD to build their whole
 >> distributions+ports with it and see how many warnings we get.  If it
 >
 >This is what we at Red Hat are about to do today.
And I think this needs to become standard operating procedure when we
encounter ABI changes.  ie, we update both the current release branch
and the mainline sources to detect and optionally warn/error about the ABI
change.  In fact, we should track ABI changes with some kind of tracking
number (see below).

One of the big issues going forward is that we be able to run code 
through the compiler and have the compiler tell us if that code triggers
an ABI bug.

What would be even better would be ways to annotate objects with this
kind of information.  It wouldn't be perfect, but the ability to add a
note section which could tell us a few things:

  1. What version of the compiler was used to compile the object
     (which implicitly gives us the ABI, though we may want to make
     the ABI revision explicit).

  2. Record what (if any) ABI bugs this file is known to tickle.


With some work we could even have the linker and dynamic loader issue
errors/warnings when mixing/matching code from different ABIs with known
issues.

Clearly this isn't perfect as it's impossible to annotate an object with
"violates bug X" until after bug X is found.  But I still think it would
be extremely helpful going forward.  Further revisions on the scheme would
be greatly appreciated.

 >
 >Personally, I think there is no point to make 3.3 ABI incompatible
 >with 3.2 without extensive ABI testsuite in place (and I don't think it
 >is manageable to come up with one in 3.3 timeframe unless it is delayed),
 >otherwise we're back where G++ used to be - every single version is ABI
 >incompatible with all the others. The chances some ABI problem will come
 >up in the next half a year is simply too high without testsuite.
 >So IMHO the ABI changes could wait for 3.4 together with ABI testsuite.
Agreed.  Let's get the ABI/API testsuite into a state where we have a good
amount of confidence that it's testing all the things we care about before
we twiddle the ABI again.  Otherwise we've really not made any significant
progress for our end-users on the "every major release of the compiler is
incompatible with every other major release" problem.

jeff

Re: C++ ABI Issues

[Joe Buck]

David Edelsohn writes:

> 	As was discussed during the GCC 3.2 transition, my personal
> opinion is that GCC needs a binary -fabi-correctly switch which defaults
> to *OFF*.  We maintain the GCC 3.2 ABI until a well-defined transition
> point in the future when the switch is changed to default to *ON*.

I would be satisfied with this approach, though we could probably come up
with a better choice of option names.  After all, we can never be certain
that we are correct.  All we can do is have two modes: 3.2 compatibility,
and 3.2 plus any ABI corrections found to date.

Maybe the flag should be -fabi-gcc-3.2 .  If true, we are compatible with
GCC 3.2; if false, we include as many bug fixes as we have to ABI conformance
at the time.  It would be true by default until we have reasonable
confidence that we've converged, and I think we need six months to achieve
such confidence, assuming that we do some rigorous testing in that time.

>  This
> allows GCC stability for the time being and interoperability with
> proprietary compilers outside the GNU/Linux and *BSD environments.
> Propritary compilers which run on GNU/Linux will probably have a GCC
> compatibility switch.

You are pretty much saying what I said; I suspect that we would only
differ on details of timing.

> 	The important points are that we make such a decision for GCC
> stability and we have a plan for eventually becoming conformant.  GCC 3.2
> may be a transient ABI, but it should not become a GNU/Linux
> counter-proposal to the C++ABI which it was trying to implement.  Also, we
> should develop a transition plan, as opposed to "we'll switch the option
> in some future release when it seems convenient."

To have a reasonable chance of catching nearly all the ABI bugs, we're
probably going to need at least six months.  What this will mean, assuming
that Red Hat 8, Debian, FreeBSD, and SuSE all adopt 3.2 is that 3.2 will
be a de-facto standard ABI.  The effect of this can be minimized if we
provide warnings at points where there is a known error in 3.2's ABI
conformance and encourage distributors to avoid such coding in any C++
libraries (if a "hit" does occur, it's easy to make it go away by adding a
dummy field that fills in the empty space in the base class's padding).
If this can be done, then an eventual switch to a compiler that does
correct ABI will not cause compatibility problems, because code that could
trigger compatibility problems would be avoided.

By the way, I'd be eager to hear from the decision-makers for the
distributions on what their opinions are on this one.  It's important
to achieve some consensus on this.

Next question is whether the ABI errors affect KDE (the largest collection
of free C++ library code that distributors will have to deal with).  Once
we have a patch that flags ABI violations, I'd appreciate it if some
volunteer will build KDE with it and report on whether we get any hits.

> 	We shouldn't make GCC arbitrarily compatible with Intel's
> compiler, and we aren't.  Similarly, we should not ask Intel or HP or any
> other proprietary compiler to be arbitrarily compatible with GCC.  It cuts
> both ways.  GCC should avoid any appearance of making arbitrary changes to
> standards and forcing others to play catch-up.

I don't disagree, but I don't want to rush out a correction.

Once we have consensus on how to proceed, I think that we should announce
the decision and any recommendations and publicize them widely.

Re: C++ ABI Issues

[Phil Edwards]

On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
> > 2. Tail-padding and virtual base classes
[...]
> Question: does anyone know if case 2 affects anything in libstdc++?
> (e.g. iostream classes?)

No, I don't think anyone knows yet.  As soon as warnings are added to
detect this kind of thing, I'll find out.  :-)

Phil

-- 
I would therefore like to posit that computing's central challenge, viz. "How
not to make a mess of it," has /not/ been met.
                                                 - Edsger Dijkstra, 1930-2002

Re: C++ ABI Issues

[Joe Buck]

> On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
> > > 2. Tail-padding and virtual base classes
> [...]
> > Question: does anyone know if case 2 affects anything in libstdc++?
> > (e.g. iostream classes?)
> 
> No, I don't think anyone knows yet.  As soon as warnings are added to
> detect this kind of thing, I'll find out.  :-)

It seems that even if it does, we can preserve libstdc++'s binary
compatibility by adding a dummy field that exactly fills up any padding.
This will make no difference in 3.2, and will force any compiler correctly
implementing the ABI to match the 3.2 interface.

Re: C++ ABI Issues

[Jan Hubicka]

On Tue, Aug 27, 2002 at 11:32:12AM -0700, Joe Buck wrote:
> 
> > On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
> > > > 2. Tail-padding and virtual base classes
> > [...]
> > > Question: does anyone know if case 2 affects anything in libstdc++?
> > > (e.g. iostream classes?)
> > 
> > No, I don't think anyone knows yet.  As soon as warnings are added to
> > detect this kind of thing, I'll find out.  :-)
> 
> It seems that even if it does, we can preserve libstdc++'s binary
> compatibility by adding a dummy field that exactly fills up any padding.
> This will make no difference in 3.2, and will force any compiler correctly
> implementing the ABI to match the 3.2 interface.

Sorr for jumping into discussion, but :0
What exactly will this bring us?  libstdc++ is not only C++ library installed
in the system and it is one of the most easilly upgardable when reinstalling
compiler?

Honza
> 

Re: C++ ABI Issues

[Jeff Law]

In message < 200208271708.KAA26617@atrus.synopsys.com >, Joe Buck writes:
 >David Edelsohn writes:
 >
 >> 	As was discussed during the GCC 3.2 transition, my personal
 >> opinion is that GCC needs a binary -fabi-correctly switch which defaults
 >> to *OFF*.  We maintain the GCC 3.2 ABI until a well-defined transition
 >> point in the future when the switch is changed to default to *ON*.
 >
 >I would be satisfied with this approach, though we could probably come up
 >with a better choice of option names.  After all, we can never be certain
 >that we are correct.  All we can do is have two modes: 3.2 compatibility,
 >and 3.2 plus any ABI corrections found to date.
 >
 >Maybe the flag should be -fabi-gcc-3.2 .  If true, we are compatible with
 >GCC 3.2; if false, we include as many bug fixes as we have to ABI conformance
 >at the time.  It would be true by default until we have reasonable
 >confidence that we've converged, and I think we need six months to achieve
 >such confidence, assuming that we do some rigorous testing in that time.
Agreed.

 >To have a reasonable chance of catching nearly all the ABI bugs, we're
 >probably going to need at least six months.  What this will mean, assuming
 >that Red Hat 8, Debian, FreeBSD, and SuSE all adopt 3.2 is that 3.2 will
 >be a de-facto standard ABI.  The effect of this can be minimized if we
 >provide warnings at points where there is a known error in 3.2's ABI
 >conformance and encourage distributors to avoid such coding in any C++
 >libraries
I've recommended this internally and externally.  It's one of the reasons
why I believe every time we find a bug in our ABI implementation we should
add a warning to the compiler so folks can find out if their code triggers
the ABI bug.

I've already recommended internally that as these issues are found that
we should push a new rev of the compiler to our Red Hat Linux customers
ASAP to make sure compilers with the warnings are as widely available as
possible.

 > (if a "hit" does occur, it's easy to make it go away by adding a
 >dummy field that fills in the empty space in the base class's padding).
 >If this can be done, then an eventual switch to a compiler that does
 >correct ABI will not cause compatibility problems, because code that could
 >trigger compatibility problems would be avoided.
Even better!


jeff

Re: C++ ABI Issues

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| > On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
| > > > 2. Tail-padding and virtual base classes
| > [...]
| > > Question: does anyone know if case 2 affects anything in libstdc++?
| > > (e.g. iostream classes?)
| > 
| > No, I don't think anyone knows yet.  As soon as warnings are added to
| > detect this kind of thing, I'll find out.  :-)
| 
| It seems that even if it does, we can preserve libstdc++'s binary
| compatibility by adding a dummy field that exactly fills up any padding.

Isn't this the compiler should be doing (with appropriate flags if
necessary)?  It certainly does know where to fill things.  I would
certainly prefer that to clutering V3 with pad0, pad1, ...

-- Gaby

Re: C++ ABI Issues

[Gabriel Dos Reis]

Jan Hubicka <jh@suse.cz> writes:

| On Tue, Aug 27, 2002 at 11:32:12AM -0700, Joe Buck wrote:
| > 
| > > On Mon, Aug 26, 2002 at 05:10:19PM -0700, Joe Buck wrote:
| > > > > 2. Tail-padding and virtual base classes
| > > [...]
| > > > Question: does anyone know if case 2 affects anything in libstdc++?
| > > > (e.g. iostream classes?)
| > > 
| > > No, I don't think anyone knows yet.  As soon as warnings are added to
| > > detect this kind of thing, I'll find out.  :-)
| > 
| > It seems that even if it does, we can preserve libstdc++'s binary
| > compatibility by adding a dummy field that exactly fills up any padding.
| > This will make no difference in 3.2, and will force any compiler correctly
| > implementing the ABI to match the 3.2 interface.
| 
| Sorr for jumping into discussion, but :0
| What exactly will this bring us?  libstdc++ is not only C++ library installed
| in the system and it is one of the most easilly upgardable when reinstalling
| compiler?

Yes.  Furthermore, libstdc++ has an ABI versioning both in the source
code and in the built binary.  I much prefer we use those menchanisms.

-- Gaby

Re: C++ ABI Issues

[Joe Buck]

> > It seems that even if it does, we can preserve libstdc++'s binary
> > compatibility by adding a dummy field that exactly fills up any padding.
> > This will make no difference in 3.2, and will force any compiler correctly
> > implementing the ABI to match the 3.2 interface.
> 
> Sorr for jumping into discussion, but :0
> What exactly will this bring us?  libstdc++ is not only C++ library installed
> in the system and it is one of the most easilly upgardable when reinstalling
> compiler?

But we don't want to have to bump the major version number, forcing people
to have two library copies if they need to run old binaries.  We want the
new libstdc++ to work with old binaries, meaning that it supports the same
ABI.

We'd have to tell people how to do the same for all other libraries
(e.g. KDE).  That is, this would be a methodology, just like avoiding
returning structs in C was the way to go in the old days (because it
triggered ABI differences between gcc and some proprietary compilers).

Hopefully this will be needed in very few places.  If we're very lucky it
won't be needed at all.

Re: C++ ABI Issues

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| > > It seems that even if it does, we can preserve libstdc++'s binary
| > > compatibility by adding a dummy field that exactly fills up any padding.
| > > This will make no difference in 3.2, and will force any compiler correctly
| > > implementing the ABI to match the 3.2 interface.
| > 
| > Sorr for jumping into discussion, but :0
| > What exactly will this bring us?  libstdc++ is not only C++ library installed
| > in the system and it is one of the most easilly upgardable when reinstalling
| > compiler?
| 
| But we don't want to have to bump the major version number, forcing people
| to have two library copies if they need to run old binaries.  We want the
| new libstdc++ to work with old binaries, meaning that it supports the same
| ABI.

Then, a much more viable way to handle this is to have the compiler do
the job. 


-- Gaby

Re: C++ ABI Issues

[Joe Buck]

I wrote:

> | It seems that even if it does, we can preserve libstdc++'s binary
> | compatibility by adding a dummy field that exactly fills up any padding.

Gaby writes:
> Isn't this the compiler should be doing (with appropriate flags if
> necessary)?  It certainly does know where to fill things.  I would
> certainly prefer that to clutering V3 with pad0, pad1, ...

How could we do that?  We'd have to either tell the compiler somehow which
classes to compile in the old way and which classes to compile in the new
way, or else use the 3.2 ABI for everything, forever.

In practice, we may be lucky and not need to do anything at all.  Let's
reopen this one when we have some data.

Re: C++ ABI Issues

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| I wrote:
| 
| > | It seems that even if it does, we can preserve libstdc++'s binary
| > | compatibility by adding a dummy field that exactly fills up any padding.
| 
| Gaby writes:
| > Isn't this the compiler should be doing (with appropriate flags if
| > necessary)?  It certainly does know where to fill things.  I would
| > certainly prefer that to clutering V3 with pad0, pad1, ...
| 
| How could we do that?  We'd have to either tell the compiler somehow which
| classes to compile in the old way and which classes to compile in the new
| way, or else use the 3.2 ABI for everything, forever.

Well, if the compiler were to be issueing a warning about a class
being laid out differently according to ABI differences, then it
certainly should be possible to tell the nature of the difference and
how things change from one set to the other.  That means, the compiler
should be able to do the adjustment.

Now, why should we really uglify the class definition?  According to
your statement, it is because we wanted the library to accomodate the
compiler deficiencies.  But then, the library isn't the only thing to
be changed to accomodate the compiler deficiencies, programs being
linked against the library also presumably need the same uglification.
Which means that a way to solve the problem ought to be in the compiler.

-- Gaby

Re: C++ ABI Issues

[Joe Buck]

> Yes.  Furthermore, libstdc++ has an ABI versioning both in the source
> code and in the built binary.  I much prefer we use those menchanisms.

We may not need them: I believe that libstdc++ is not affected at all.  It
appears that the only uses of virtual baseclasses are for basic_istream
and basic_ostream, and the last field of their base class is a pointer,
indicating that there will be no padding space on ILP32 or LP64 systems.
Mark's patch should confirm this.  But if I'm right, libstdc++ needs no
versioning at all.

Re: C++ ABI Issues

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| But if I'm right, libstdc++ needs no versioning at all.

That would be good news.

-- Gaby

Re: C++ ABI Issues

[Joe Buck]

Gaby writes:
> Well, if the compiler were to be issueing a warning about a class
> being laid out differently according to ABI differences, then it
> certainly should be possible to tell the nature of the difference and
> how things change from one set to the other.  That means, the compiler
> should be able to do the adjustment.

The compiler can warn that gcc 3.2 and a compiler that implements the ABI
document correctly will lay out a class differently.  But given this
knowledge, how is it supposed to figure out what the user wants to happen?
It doesn't know which compiler compiled the "other half" of the code: the
library being called, or the caller of the library.

Of course the compiler can and will do the adjustment if it implements
-fabi-gcc-3.2 or whatever we call it.  But if the code being called is
compiled by Intel, it had better *not* do the adjustment.  Again: how can
it "know"?

> Now, why should we really uglify the class definition?  According to
> your statement, it is because we wanted the library to accomodate the
> compiler deficiencies.

Rather, *if* a warning appears, *and* it is important to the developer of
the code that gcc 3.2 and Intel's compiler and a future, fixed gcc will
agree on the layout of some class, then the only safe thing to do is to
design the class in such a way that the warning goes away.  Library
versioning won't suffice, and in any case is much more difficult to do
than simply adjusting the class layout.

As I said, it appears that libstdc++ is unaffected.  C++ implementations
of CORBA should be looked at, as they are likely to use multiple
inheritance.

You appear to assume that these padding objects will be needed all over
the place.  In practice, I expect the need for them to be extremely rare,
because even to use virtual base classes is rare (I despise the things
myself: they violate encapsulation severely unless the virtual base has no
data members, since the most-derived class always has to explicitly
construct the virtual base.  And if there are no members, then the ABI bug
does not arise).

Re: C++ ABI Issues

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| Gaby writes:
| > Well, if the compiler were to be issueing a warning about a class
| > being laid out differently according to ABI differences, then it
| > certainly should be possible to tell the nature of the difference and
| > how things change from one set to the other.  That means, the compiler
| > should be able to do the adjustment.
| 
| The compiler can warn that gcc 3.2 and a compiler that implements the ABI
| document correctly will lay out a class differently.  But given this
| knowledge, how is it supposed to figure out what the user wants to happen?
| It doesn't know which compiler compiled the "other half" of the code: the
| library being called, or the caller of the library.

Aren't we talking of a bug in GCC -- which other compilers known to
implement the ABI don't have?

[...]

| You appear to assume that these padding objects will be needed all over
| the place.

No.  I'm not comfortable with using warts and code uglification to try
to do the compiler's work.  We do know the bugs we're talking about.

-- Gaby

Re: C++ ABI Issues

[Joe Buck]

> | The compiler can warn that gcc 3.2 and a compiler that implements the ABI
> | document correctly will lay out a class differently.  But given this
> | knowledge, how is it supposed to figure out what the user wants to happen?
> | It doesn't know which compiler compiled the "other half" of the code: the
> | library being called, or the caller of the library.

> Aren't we talking of a bug in GCC -- which other compilers known to
> implement the ABI don't have?

It's a bug in GCC, yes.  Given the presence of that bug, what should a C++
software developer do?  He or she has no choice but to use padding.

> | You appear to assume that these padding objects will be needed all over
> | the place.
> 
> No.  I'm not comfortable with using warts and code uglification to try
> to do the compiler's work.  We do know the bugs we're talking about.

Do you intend to implement time travel and go back and fix GCC 3.2?  If
not, what advice do you want to give to users, who might need both
compilers, or gcc 3.2 plus future, fixed gcc's, to work and to talk to
each other?  You appear to be focussed only on fixing C++ libraries we
ship (libstdc++).  Let's say you're putting together the Debian "sarge"
release and you have to ship 800 or so packages containing C++ code
(remember, their package count recently passed 10,000).  Your job is to
assure that all of that code comes out the same on gcc 3.2 as on a fixed
gcc 3.4, because if you don't, you'll have another painful episode where
every binary C++-containing package has to be replaced when someone
upgrades the compiler.  How do you do it?

My answer is that you use Mark's warning patch, find all the code that
triggers warnings, and add pads until the warning goes away.  Your answer
appears to be, um, what?  Punt, saying that you're not comfortable with
code uglification?  It doesn't appear that you have a solution that will
allow for 3.2 compatibility.

In any case, I suggest postponing further debate until we encounter some
real examples of "violations" in user code and get a handle on how common
they are.  If they are extremely rare, "uglification" will be minimal.
If they are common, then you may be right and we'll need to think of
something else.

Re: C++ ABI Issues

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| > | The compiler can warn that gcc 3.2 and a compiler that implements the ABI
| > | document correctly will lay out a class differently.  But given this
| > | knowledge, how is it supposed to figure out what the user wants to happen?
| > | It doesn't know which compiler compiled the "other half" of the code: the
| > | library being called, or the caller of the library.
| 
| > Aren't we talking of a bug in GCC -- which other compilers known to
| > implement the ABI don't have?
| 
| It's a bug in GCC, yes.  Given the presence of that bug, what should a C++
| software developer do?  He or she has no choice but to use padding.

I'm not convinced.  We're not talking of abstract bugs.  We do know
what they are and how we diverge from other implementors (they got it
right).  So, we're talking of a bug compared to ABI correct
implementation, so we know what the fix should be.

| > | You appear to assume that these padding objects will be needed all over
| > | the place.
| > 
| > No.  I'm not comfortable with using warts and code uglification to try
| > to do the compiler's work.  We do know the bugs we're talking about.
| 
| Do you intend to implement time travel and go back and fix GCC 3.2?

I assume that isn't a question that asks for a sensical answer, right?

|  If
| not, what advice do you want to give to users, who might need both
| compilers, or gcc 3.2 plus future, fixed gcc's, to work and to talk to
| each other?

The future fixed GCC is where there is a possible divergence from
GCC-3.2.  That future GCC ought to know about the fix -- since it will
lay out differently compared to GCC-3.2. That future GCC-3.2 can add the
missing bits if necessary.

| You appear to be focussed only on fixing C++ libraries we
| ship (libstdc++). 

As you certainly read in my previous messages, I'm certainly worrying
about libraries and programs other than libstdc++.

[...]

| Your answer
| appears to be, um, what?  Punt, saying that you're not comfortable with
| code uglification?

That is a mischaracterization of my answer.

-- Gaby

Re: C++ ABI Issues

[Richard Earnshaw]

>  >Maybe the flag should be -fabi-gcc-3.2 .  If true, we are compatible with
>  >GCC 3.2; if false, we include as many bug fixes as we have to ABI conformance
>  >at the time.  It would be true by default until we have reasonable
>  >confidence that we've converged, and I think we need six months to achieve
>  >such confidence, assuming that we do some rigorous testing in that time.
> Agreed.

Well, I'd prefer a flag of the form -fabi=gcc-3.2 then we could easily 
extend that to any later version.  It's also then clear that you can't 
specify more than one such flag (or -fno-abi-gcc-3.2).

Finally, we could then also have -fabi=std meaning compile to the standard 
(well, more precisely, don't do anything we know to violate the ABI).

R.

Re: C++ ABI Issues

[Joe Buck]

> | Do you intend to implement time travel and go back and fix GCC 3.2?
> 
> I assume that isn't a question that asks for a sensical answer, right?

I was trying to point out that you are missing the point.  I am puzzled as
to why this is so, but I'm convinced of it.

> |  If
> | not, what advice do you want to give to users, who might need both
> | compilers, or gcc 3.2 plus future, fixed gcc's, to work and to talk to
> | each other?
> 
> The future fixed GCC is where there is a possible divergence from
> GCC-3.2.  That future GCC ought to know about the fix -- since it will
> lay out differently compared to GCC-3.2. That future GCC-3.2 can add the
> missing bits if necessary.

That will not suffice.

The divergence with GCC-3.2 already exists: Intel is shipping a compiler.
What we do in a future GCC is IRRELEVANT.  Anyone developing C++ code that
wants to ship a library on GNU/Linux, who wants people to be able to use
that library for a long time, with a choice of compilers (gcc 3.2, future
gcc's, as well as gcc competitors) has only one choice: avoid the problem.
How, pray tell, can any patch we make to the already-shipped gcc 3.2 make
it match the already-shipped Intel compiler?

Furthermore, if we do as you say and make the future GCC 3.2 "add the
missing bits", just what bits do you intend to add?  Padding?  That will
make it match the GCC 3.2 ABI.  But what if the code being linked to was
built by Intel?  What if the code being linked to will be built with
GCC 3.5?

What has not shipped yet is a production distribution based on GCC 3.2.
This means that the only thing we have an opportunity to change is the
C++ code that is to be compiled.  We cannot implement time travel and
change the compilers.

> | You appear to be focussed only on fixing C++ libraries we
> | ship (libstdc++). 
> 
> As you certainly read in my previous messages, I'm certainly worrying
> about libraries and programs other than libstdc++.

OK.  Perhaps I'm clueless and so are people like Jeff Law, who liked my
suggestion.  Please explain your "add the missing bits" proposal and how
it addresses the requirement of achieving unified and stable ABIs for C++
libraries on GNU/Linux, for users of gcc 3.2, of other compilers that
(attempt to) implement the standard ABI, and for users of future gcc's,
without forcing gcc to be bug-compatible with gcc 3.2 forever.

Re: C++ ABI Issues

[David O'Brien]

On Tue, Aug 27, 2002 at 02:57:58PM +0200, Allan Sandfeld Jensen wrote:
> Since 3.3 has to stay compatible with 3.2, the 3.2 ABI will most likely have 
> to stay default.

WHY??  3.2->3.3 is a major version release, so ABI changes are allowed.

For FreeBSD we would like to see all known C++ ABI bugs fixed in 3.3.0.

It is not too late for us to move from 3.2 -> 3.3.  In fact we've had so
many problems with our update to 3.2 from 3.1-prerelease that we may have
to go to 3.3 anyway.

The proposals for a switch to choose the C++ ABI sound excellent to
FreeBSD.  I don't care if the default ABI is the 3.2 one, or the "bug
free" one -- I can easily set what we want in config/freebsd-specs.h.

A -Wabi in 3.2.1 would also be well received by FreeBSD as we can use
that to help decided if we should move to 3.3 for our 5.0 release.

Re: C++ ABI Issues

[David O'Brien]

On Mon, Aug 26, 2002 at 03:16:36PM -0700, Mark Mitchell wrote:
> Our testing and investigation has lead to the discovery of several places
> where G++'s class layout still does not match the published ABI
> specification.  Other vendors (notably HP and Intel) do match the ABI
> specification.  Therefore, the ABI specification is probably not going
> to change to validate G++'s behavior.

Hi Mark,

We really need to know what the SC decided about this issue for 3.3?
FreeBSD would really like to see the C++ ABI bugs fixed in 3.3 (even if
selectable via a command line switch).

-- David

Re: C++ ABI Issues

[Mark Mitchell]

> We really need to know what the SC decided about this issue for 3.3?
> FreeBSD would really like to see the C++ ABI bugs fixed in 3.3 (even if
> selectable via a command line switch).

The SC didn't decide anything, nor is it even debating the issue.  We
can raise it, if you like, but the consensus on the lists seemed to be
to leave things alone for a while.

Nor has anybody yet implemented the fixes.  Until that happens, there's
little that we can do.

I plan to implement the fixes, soon, but probably not before 3.3 branches.

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Re: C++ ABI Issues

[Joe Buck]

> > We really need to know what the SC decided about this issue for 3.3?
> > FreeBSD would really like to see the C++ ABI bugs fixed in 3.3 (even if
> > selectable via a command line switch).
> 
> The SC didn't decide anything, nor is it even debating the issue.  We
> can raise it, if you like, but the consensus on the lists seemed to be
> to leave things alone for a while.

To be technical, a number of SC members discussed the issue, on this list,
so you can find the discussion in the archives.  There hasn't been any
separate discussion on the private list.

The consensus, at least as I interpreted it, was to leave 3.2 compatible
with 3.3 by default, add warning code to detect cases that would be
handled differently, and have some option to generate "correct" code for
the known bad cases.  The details were not completely fleshed out.


> Nor has anybody yet implemented the fixes.  Until that happens, there's
> little that we can do.
> 
> I plan to implement the fixes, soon, but probably not before 3.3 branches.

Re: C++ ABI Issues

[David O'Brien]

On Mon, Sep 23, 2002 at 09:51:16AM -0700, Joe Buck wrote:
> > The SC didn't decide anything, nor is it even debating the issue.  We
> > can raise it, if you like, but the consensus on the lists seemed to be
> > to leave things alone for a while.
... 

> The consensus, at least as I interpreted it, was to leave 3.2 compatible
> with 3.3 by default, add warning code to detect cases that would be
> handled differently, and have some option to generate "correct" code for
> the known bad cases.  The details were not completely fleshed out.

"Leaving things along" does not match my interpretation of the discussion
at all.  Joe's interpretation matches what I came away from the
discussion with.  That being that 3.3 and 3.2 would be able to be ABI
compatable (either by default, or with a command line switch for 3.3).
And that 3.3 would have a fixed C++ ABI that was either the default ABI,
or selectable via a command line switch.

Re: C++ ABI Issues

[David O'Brien]

On Tue, Aug 27, 2002 at 11:07:00AM -0700, Ziemowit Laski wrote:
> 
> On Tuesday, Aug 27, 2002, at 10:13 US/Pacific, Benjamin Kosnik wrote:
> 
> >
> >I'll restrain my comments to one area only.
> >
> >Please, let's use the versioning info that already exists. Currently,
> >the compiler ABI is tracked with __GXX_ABI_VERSION. For the 3.2.0
> >release, this value is 102.
> >
> >If there is to be a -fabi=xxx switch, then please use this information.
> >The idea of having a "-fabi-newest" is completely relative and useless:
> >newest from what vantage point?

I disagree.  I don't know what __GXX_ABI_VERSION I have, but I certainly
know what GCC version I have.

Re: C++ ABI Issues

[Richard Earnshaw]

> 
> I'll restrain my comments to one area only.
> 
> Please, let's use the versioning info that already exists. Currently,
> the compiler ABI is tracked with __GXX_ABI_VERSION. For the 3.2.0
> release, this value is 102.
> 
> If there is to be a -fabi=xxx switch, then please use this information.
> The idea of having a "-fabi-newest" is completely relative and useless:
> newest from what vantage point? 

I think 'newest' in this sense is 'don't do anything you know violates 
that standard'.  That is it should map on to the highest ABI (eg 103) 
number that we've defined.

R.

Re: C++ ABI Issues

[Toon Moene]

Ziemowit Laski wrote:

Re: C++ ABI Issues

[Mark Mitchell]

Re: C++ ABI Issues

[Joe Buck]

Benjamin Kosnik writes:
> I'll restrain my comments to one area only.
> 
> Please, let's use the versioning info that already exists. Currently,
> the compiler ABI is tracked with __GXX_ABI_VERSION. For the 3.2.0
> release, this value is 102.

I agree that if an -fabi flag is added, it should affect the
__GXX_ABI_VERSION symbol.  We would increment the value if a newer
ABI is used.

> If there is to be a -fabi=xxx switch, then please use this information.
> The idea of having a "-fabi-newest" is completely relative and useless:
> newest from what vantage point? 

The difficulty appears to be that if we first do a patch that corrects the
two ABI problems we know about, and this goes in 3.2.1 plus the versions
put out by Red Hat and others, and then we find a third and then we find a
fourth, the meaning of "abi-newest" could change in each minor release.

Here's a modest proposal, just for discussion:

-fcplusplus_abi=gcc3.2

Be compatible with gcc 3.2.

-fcplusplus_abi=20020915

Include all ABI corrections as of Sept. 15, 2002.  (this should be OK to
maintain if we only find one or two more bugs, and we don't use this as a
mechanism to support ABIs before 3.2, though I suppose that if someone
wants 3.1 compatibility bad enough to submit a patch it might be doable).

Folks who really want -fabi-latest could type

-fcplusplus_abi=30000101 (Jan 1, 3000).

Re: C++ ABI Issues

[Per Bothner]

Devang Patel wrote:

Re: C++ ABI Issues

[Mike Stump]

On Tuesday, August 27, 2002, at 11:27 AM, Ziemowit Laski wrote:

Re: C++ ABI Issues

[Ziemowit Laski]

On Tuesday, Aug 27, 2002, at 11:15 US/Pacific, Mike Stump wrote:

Re: C++ ABI Issues

[Mike Stump]

On Tuesday, August 27, 2002, at 11:07 AM, Ziemowit Laski wrote:

Re: C++ ABI Issues

[Ziemowit Laski]

On Tuesday, Aug 27, 2002, at 10:13 US/Pacific, Benjamin Kosnik wrote:

Re: C++ ABI Issues

[Devang Patel]

On Tuesday, August 27, 2002, at 09:49  AM, Mark Mitchell wrote:

Re: C++ ABI Issues

[Mike Stump]

On Monday, August 26, 2002, at 08:28 PM, Benjamin Kosnik wrote:

Re: C++ ABI Issues

[Benjamin Kosnik]

I'll restrain my comments to one area only.

Please, let's use the versioning info that already exists. Currently,
the compiler ABI is tracked with __GXX_ABI_VERSION. For the 3.2.0
release, this value is 102.

If there is to be a -fabi=xxx switch, then please use this information.
The idea of having a "-fabi-newest" is completely relative and useless:
newest from what vantage point? 

-benjamin

Re: C++ ABI Issues

[Mark Mitchell]

--On Monday, August 26, 2002 08:28:38 PM -0700 Benjamin Kosnik 
<bkoz@redhat.com> wrote:

Dude.

Re: C++ ABI Issues

[Janis Johnson]

On Tue, Aug 27, 2002 at 08:43:18AM -0700, Mark Mitchell wrote:
> > If we're going to be serious about ABI compatibility going forward, then
> > the next step is clearly to have a comprehensive open testsuite for the
> > ABI and the library APIs.
> 
> Because we're focusing a lot of effort on the compiler ABI issues,
> I'd suggest that others focus on the library API and testing in that
> regard.  I believe this route will deliver the maximum amount of total
> testing framework.

There are several things to be concerned about here that might require
different testing approaches:

- compiler's conformance to the C++ ABI specification

- consistency of code and externally visible symbols generated by the
  compiler, right or wrong

- interoperability between GCC and proprietary compilers, including
  platforms that use a different formal C++ ABI specification

- consistency of libstdc++, for which there is no formal ABI
  specification

When will you be able to tell us more about the C++ ABI test suite that
CodeSourcery is developing?  If it's possible that it won't be open
then we might want to duplicate some of its coverage in tests that will
be available to all GCC developers.

Janis

Re: C++ ABI Issues

[Mark Mitchell]

If we're going to be serious about ABI compatibility going forward, then
the next step is clearly to have a comprehensive open testsuite for the
ABI and the library APIs.

Re: C++ ABI Issues

[Jeff Law]

 In message < 17480000.1030431228@warlock.codesourcery.com >, Mark Mitchell 
writes:
 >Certainly, one reasonable position is to do nothing.  Another is to
 >(as several have suggested) support both modes (which seems like
 >a good idea to me).  Another is my initial suggestion (to fix the
 >problems right away.)  To me, the best argument for my suggestion is
 >that at this point there aren't too many people dependent on 3.2; the
 >longer we leave it around the harder it may to be change it later.
Believe me I understand your point.

Unfortunately, Red Hat is far enough along in it's cycle that revving the
compiler again isn't really feasible.  So for Red Hat at least, GCC 3.2 is
the compiler for the next major release of Red Hat Linux and for any
follow-on minor releases.


 >Your point, made previous by Joe, that we are likely to find more bugs,
 >is also a very good one.
Yes, which is why I think it's so important that we build our our testing
infrastructure so that next time we say "we've implemented the multi-vendor
ABI and we're going to be compatible going forward" that we actually have
some confidence in that statement.

If that means that we don't rev the ABI for GCC 3.3, but instead wait for
GCC 3.4, then, well, so be it.  That would actually be an improvement in
a lot of ways as we've never had major releases without an ABI change.

 >I recently had discussions with folks at HP about what to do when these
 >things come up.  They've got an almost-compliant compiler deployed in
 >the field; so do we.  They're very interested in being compatible with
 >GCC.  Nobody wants to break things for their users.  It's a difficult
 >situation, and the discussions didn't reach an easy solution -- although
 >everyone was very clear that the ultimate goal is to keep from messing
 >up things for users.  I'd love it if we had a process in place to deal
 >with these issues when they arise.
Not at all unlike the discussions I've had with Intel.  We're all trying to
get to the same place -- the problem is that we're not all going to get there
at the same time.  Though sharing of testcode, results and the like between
the interested parties will be a significant step in the right direction.

jeff

Re: C++ ABI Issues

[Jeff Law]

 In message < 20020826202838.4996c482.bkoz@redhat.com >, Benjamin Kosnik writes:
 >Please, let's not do another release where we have a two week period
 >to find bugs. That's insanity. Instead, let's actually have some kind
 >of public, check-in-and-GPL'd ABI testsuite g++ users can cling to and
 >say, hey! We have an ABI and here's what we are checking.
 >
 >If I were king, I'd say, finish up the codesourcery ABI tester. Fix
 >all the bugs you find. Then, do a bug-fix ABI release in a year, after
 >g++ has successfully shown to be inter-operable with another compiler
 >so it actually means something.
This all sounds good to me.  Though I'd mention that we should be using
bits from all parties that have been building ABI conformance testsuites,
including Intel, Codesourcery, Red Hat, etc.

If we're going to be serious about ABI compatibility going forward, then
the next step is clearly to have a comprehensive open testsuite for the
ABI and the library APIs.

jeff

Re: C++ ABI Issues

[Jeff Law]

In message < hoadn8to72.fsf@gee.suse.de >, Andreas Jaeger writes:
 >Mark Mitchell <mark@codesourcery.com> writes:
 >
 >> [...]
 >> Certainly, one reasonable position is to do nothing.  Another is to
 >> (as several have suggested) support both modes (which seems like
 >> a good idea to me).  Another is my initial suggestion (to fix the
 >> problems right away.)  To me, the best argument for my suggestion is
 >> that at this point there aren't too many people dependent on 3.2; the
 >> longer we leave it around the harder it may to be change it later.
 >
 >It's too late already. Some distributors, AFAIR including Mandrake and
 >Red Hat - have planned to switch to GCC 3.2 in August and announced
 >this during the discussion about 3.2.  SuSE has just done the step
 >internally and it's too late for such major changes, I don't think
 >that you can release a tested compiler with those bugfixes in a
 >timeframe that would make us happy.
Red Hat is in the same position.   It's simply too late in our process to
switch to a new compiler ABI.

It's unfortunate, but not unexpected that we found ABI issues in GCC 3.2.
Everyone's life would be better if that hadn't happened, but it's not the
end of the world.  

I do think this situation highlights how important it is that we build
testers for the ABI.  In fact, when I look into the future, I think that
having a comprehensive ABI testsuite that the compiler passes should be 
a release critera.  I don't know if we'll have all that in place for 
GCC 3.3, but I would hope that we'd have it in place by GCC 3.4.


jeff

Re: C++ ABI Issues

[Andreas Jaeger]

Mark Mitchell <mark@codesourcery.com> writes:

> [...]
> Certainly, one reasonable position is to do nothing.  Another is to
> (as several have suggested) support both modes (which seems like
> a good idea to me).  Another is my initial suggestion (to fix the
> problems right away.)  To me, the best argument for my suggestion is
> that at this point there aren't too many people dependent on 3.2; the
> longer we leave it around the harder it may to be change it later.

It's too late already. Some distributors, AFAIR including Mandrake and
Red Hat - have planned to switch to GCC 3.2 in August and announced
this during the discussion about 3.2.  SuSE has just done the step
internally and it's too late for such major changes, I don't think
that you can release a tested compiler with those bugfixes in a
timeframe that would make us happy.

>
> Your point, made previous by Joe, that we are likely to find more bugs,
> is also a very good one.
>
> I recently had discussions with folks at HP about what to do when these
> things come up.  They've got an almost-compliant compiler deployed in
> the field; so do we.  They're very interested in being compatible with
> GCC.  Nobody wants to break things for their users.  It's a difficult
> situation, and the discussions didn't reach an easy solution -- although
> everyone was very clear that the ultimate goal is to keep from messing
> up things for users.  I'd love it if we had a process in place to deal
> with these issues when they arise.
>
> I don't have a strong opinion about what we do at this point; I'm happy
> to see what the consensus is and do whatever that is.  At this point,
> it looks like people are leaning towards David's -fabi-with-fewer-bugs
> switch, which is fine by me.

Such a switch is fine with me,
Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj

Re: C++ ABI Issues

[Daniel Jacobowitz]

On Mon, Aug 26, 2002 at 11:53:48PM -0700, Mark Mitchell wrote:
> 
> 
> --On Monday, August 26, 2002 08:28:38 PM -0700 Benjamin Kosnik 
> <bkoz@redhat.com> wrote:
> 
> >
> >Dude.
> >
> >>I propose that we fix G++ to match the ABI, but that we issue warnings
> >>about classes whose layout has changed from GCC 3.2.
> >
> >>If we are going to fix G++, we also have to decide how urgently to do
> >>another release.
> >
> >You seem to be very cavalier about the downstream impacts of changing
> >the C++ ABI, and what this means for sane tool versioning.
> 
> I tried to bring these issues to the attention of the GCC community
> promptly, and to present some realistic assessment of how important
> they are, or are not.
> 
> Certainly, one reasonable position is to do nothing.  Another is to
> (as several have suggested) support both modes (which seems like
> a good idea to me).  Another is my initial suggestion (to fix the
> problems right away.)  To me, the best argument for my suggestion is
> that at this point there aren't too many people dependent on 3.2; the
> longer we leave it around the harder it may to be change it later.

It's too late already.

A large number of distributions specifically said that they were
looking to change to a new system compiler, and that the previous
schedule for 3.2 would be too late for them.  I'd imagine most of them
have started (or in Debian's case, started arguing about...) the
transition by now.  Slipping another compiler in there is not going to
happen; GCC 3.2 is committed to widespread use at this point by nature
of its schedule.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: C++ ABI Issues

[Mark Mitchell]

--On Monday, August 26, 2002 08:28:38 PM -0700 Benjamin Kosnik 
<bkoz@redhat.com> wrote:

Dude.

Re: C++ ABI Issues

[Benjamin Kosnik]

Dude. 

> I propose that we fix G++ to match the ABI, but that we issue warnings
> about classes whose layout has changed from GCC 3.2.

> If we are going to fix G++, we also have to decide how urgently to do
> another release.

You seem to be very cavalier about the downstream impacts of changing
the C++ ABI, and what this means for sane tool versioning.

Every time you change the underlying compiler ABI, libstdc++ has to
adapt and meaningfully mark the changing versions.

I do not think doing a g++ release, and bumping __GXX_ABI_VERSION to
103, and changing libstdc++.so.5 to libstdc++.so.6, and updating
GLIBCPP_3.2 to GLIBCPP_3.3, and CXXABI_1.2 to CXXABI_1.3 should happen
every three months. 

Come on. 

You found bugs, great. You fixed the bugs, even better. I bet you
fifty bucks you find more by the end of December, or two tickets to a
Giants home game, whichever is more.

Please, let's not do another release where we have a two week period
to find bugs. That's insanity. Instead, let's actually have some kind
of public, check-in-and-GPL'd ABI testsuite g++ users can cling to and
say, hey! We have an ABI and here's what we are checking.

If I were king, I'd say, finish up the codesourcery ABI tester. Fix
all the bugs you find. Then, do a bug-fix ABI release in a year, after
g++ has successfully shown to be inter-operable with another compiler
so it actually means something.

Sorry if this sounds pissy. I know you are working on the side of
angels, but please. Try not to abuse your considerable power. Let's
try to come up with a long term plan for managing C++ ABI changes, and
stop calling "FIRE!!!" every time there is a bug. Pretty soon nobody
will care.

yours in struggle,
benjamin

Re: C++ ABI issues

[Jason Merrill]

>>>>> "Benjamin" == Benjamin Kosnik <bkoz@redhat.com> writes:

> Do you know of other ABI-changing bits? I don't think I got them all.

We've tried to document them in gcc/cp/NEWS, but I forgot to mention the
mangling change.  Will fix.

The 128-bit ints patch wasn't an ABI change; previously we crashed on
affected code, now we do the right thing.

>> > Why just non-ia64 targets?
>> 
>> Because for ia64, the ABI is a cross-vendor standard.  If we bumped the
>> version, we would be incompatible with all other compilers, which would
>> subvert the entire point of the ABI.

> So, how are differences between the C++ ABI in 3.0.x and 3.1.x deduced?

What do you mean?  3.1 is more compliant with the ABI standard than 3.0
was.  The ABI changes are all bugfixes to make us more compliant.

Jason

Re: C++ ABI issues

[Benjamin Kosnik]

Do you know of other ABI-changing bits? I don't think I got them all.

> > Why just non-ia64 targets?
> 
> Because for ia64, the ABI is a cross-vendor standard.  If we bumped the
> version, we would be incompatible with all other compilers, which would
> subvert the entire point of the ABI.

So, how are differences between the C++ ABI in 3.0.x and 3.1.x deduced?

> We have a few tests in the testsuite.  I thought that CodeSourcery had more
> internally, but I haven't heard about them in quite a while.

Hmmmm. 

Re: C++ ABI issues

[Mark Mitchell]

>> I'd feel better if there was a better mechanism for testing the C++
>> ABI. Thoughts?
>
> We have a few tests in the testsuite.  I thought that CodeSourcery had
> more internally, but I haven't heard about them in quite a while.

Our customer decided not to release them, unfortunately.  We've got
other ideas about how to do more robust testing as well, and we are
talking to one customer about that a bit, but it's not yet clear
whether anything will come of it.

I think bumping the library version number ought to be enough, personally.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: C++ ABI issues

[mike stump]

> To: Benjamin Kosnik <bkoz@redhat.com>
> Cc: gcc@gcc.gnu.org
> From: Jason Merrill <jason@redhat.com>
> Date: Mon, 18 Mar 2002 21:03:45 +0000

> > Why just non-ia64 targets?

> Because for ia64, the ABI is a cross-vendor standard.

Ah, but don't we want as much from the standard for other targets, as
we want for ia64?  If we never treat it this way, can we ever achieve
our goal?  My take is that we should be more interested in all the
other targets than ia64, not less.

> If we bumped the version, we would be incompatible with all other
> compilers, which would subvert the entire point of the ABI.

Bingo, which is why I think we should not.

Re: C++ ABI issues

[Jason Merrill]

>>>>> "Benjamin" == Benjamin Kosnik <bkoz@redhat.com> writes:

>> One answer would be to bump the mangling version for non-ia64 targets,
>> so that incompatible code will fail to link.

> Why just non-ia64 targets?

Because for ia64, the ABI is a cross-vendor standard.  If we bumped the
version, we would be incompatible with all other compilers, which would
subvert the entire point of the ABI.

> Won't the major version number change from .3 to .4 in libstdc++-v3
> also prevent linkage?

For code that uses libstdc++, yes.  Perhaps that is true of most programs,
and the problem is minor.

> I'd feel better if there was a better mechanism for testing the C++
> ABI. Thoughts? 

We have a few tests in the testsuite.  I thought that CodeSourcery had more
internally, but I haven't heard about them in quite a while.

Jason

Re: C++ ABI issues

[Benjamin Kosnik]

Hey Jason. I've been wondering about this myself. One of the things I
think might be helpful, when thinking about this, would be a detailed
list of ABI changes between gcc-3.0.4 and gcc-3.1.0. I've noticed that
you have started marking ABI-changing patches explicitly: thank you
very much for this, it's very illuminating. I could come up with a
list of library changes, if that would be helpful.

I'm not as up on this stuff as yourself, but it appears as if the
following are a subset of the things that have changed in the FE:

- pointers to cv-qualified member functions
  http://gcc.gnu.org/ml/gcc-patches/2002-02/msg01363.html

- c++/3948
  run destructors for value parms in the caller, pass params by invisible ref
  http://gcc.gnu.org/ml/gcc-patches/2002-03/msg01056.html

- 128-bit ints
  http://gcc.gnu.org/ml/gcc-patches/2002-03/msg00761.html

In addition, it appears as if non-C++ FE work has ABI impacts for some targets:

- mips o32 ABI
http://gcc.gnu.org/ml/gcc-patches/2002-03/msg00706.html

> One answer would be to bump the mangling version for non-ia64 targets,
> so that incompatible code will fail to link.

Why just non-ia64 targets?

Won't the major version number change from .3 to .4 in libstdc++-v3
also prevent linkage?

I'd feel better if there was a better mechanism for testing the C++
ABI. Thoughts? 

In a related note, there doesn't seem to be a plan in place to test
the C++ library versioning. Even if it's not documented, it would be
nice to have some procedure that's documented.

best,
benjamin

C++ ABI issues

[Jason Merrill]

There are several C++ ABI incompatibilities between 3.0.x and 3.1.x.  Most
of these will not prevent code built with different compilers from linking
together, but will cause a resulting program to fail in obscure ways.  I'm
not happy with this state of affairs, but I'm not sure what to do about
it.  One answer would be to bump the mangling version for non-ia64 targets,
so that incompatible code will fail to link.

Thoughts?

Jason