Re: __register_frame_info & shared library compatibility

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   From: Jeffrey A Law (law@cygnus.com)

   Back in 1997 we tried _very_ hard to convince Richard Kenner and Richard
   Stallman to keep the old "__register_frame_info" interface in libgcc.a for
   gcc2 for precisely this problem.

I think that you mean "__register_frame" instead of
"__register_frame_info".

   Against our recommendations, the interface was removed from gcc2.8.

The "__register_frame" interface was never included in any official
FSF release of gcc before 2.8.  Back then, egcs was still
"experimental" which was probably the reason why Kenner didn't want to
include the (broken) "__register_frame" interface that was part of
egcs 1.0 in gcc 2.8.  However, compatibility for the old
"__register_frame" was included in gcc 2.8.1.  This has not been
noticed by a lot of people since, where the "__register_frame_info"
and "__register_frame" functions are part of the same object module in
egcs's libgcc.a, they are in different modules in gcc's libgcc.a.
This was exactly why 

However, this issue is irrelevant for the current
"__register_frame_info & shared library compatibility" discussion.
The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
indeed that egcs 1.1.2 now puts only a weak reference to
"__register_frame_info" in crtstuff.  This means that "du moment" one
recompiles a shared library that previously contained
"__register_frame_info" it probably does not anymore, and programs
linked with that library stop running.

Now, things get really complicated because of the egcs/Linux
``releases''.  In an attempt to fix problems the author has messed
around with the way "__register_frame_info" is used in the varied
egcs-1.1.1/Linux ``releases''.  This might mean that some people that
use these versions will experience the same problems, but report that
they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
prevent that this happens again for egcs-1.1.2, and discourage the
author to release any "fixed" version of egcs-1.1.2.

It is probably not a bad idea to revert the weak reference to
"__register_frame_info" patch in egcs-1.1.2.  It may result in larger
executables with glibc 2.0, but at least we get rid of the shared
library compatibility problem.  For glibc 2.1 things shouldn't matter
since "__register_frame_info" is exported from libc, which by default
every program and shared library is linked against.

Mark

P.S. I haven't tested this, but using

   "-Wl,-u __register_frame_info"

when rebuilding a shared library with egcs-1.1.2 (or egcs-1.1.1/Linux)
will probably avoid the problems discussed in this thread.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904071214.OAA13954@landau.wins.uva.nl >you write:
  > I think that you mean "__register_frame" instead of
  > "__register_frame_info".
Yea.  Minor details :-)


  >    Against our recommendations, the interface was removed from gcc2.8.
  > 
  > The "__register_frame" interface was never included in any official
  > FSF release of gcc before 2.8.
I'm well aware of that.  It had been the interface that had been in the gcc
tree for nearly 2 years though.  It was changed about 2 weeks before gcc-2.8
went out the door and just after egcs-1.0 had been released.


  > Back then, egcs was still
  > "experimental" which was probably the reason why Kenner didn't want to
  > include the (broken) "__register_frame" interface that was part of
  > egcs 1.0 in gcc 2.8.
There were other reasons.  Some technical, but mostly political.


  > However, compatibility for the old "__register_frame" was included in
  > gcc 2.8.1.  This has not been noticed by a lot of people since,
  > where the "__register_frame_info" and "__register_frame" functions are
  > part of the same object module in egcs's libgcc.a, they are in different
  > modules in gcc's libgcc.a. This was exactly why 
I didn't know they'd included it in gcc-2.8.1.   Good.  However keeping them
in different modules is still not necessarily right, particularly when one
considers shared libraries and dynamically loaded code.

  > However, this issue is irrelevant for the current
  > "__register_frame_info & shared library compatibility" discussion.
  > The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
  > and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
  > indeed that egcs 1.1.2 now puts only a weak reference to
  > "__register_frame_info" in crtstuff.  This means that "du moment" one
  > recompiles a shared library that previously contained
  > "__register_frame_info" it probably does not anymore, and programs
  > linked with that library stop running.
Yup.  And I consider this a horrible breakage.  I trusted a couple of folks
with more experience in this area to guide the decision to include that
patch.  They made a mistake.  We need to rectify it.

  > Now, things get really complicated because of the egcs/Linux
  > ``releases''.  In an attempt to fix problems the author has messed
  > around with the way "__register_frame_info" is used in the varied
  > egcs-1.1.1/Linux ``releases''.  This might mean that some people that
  > use these versions will experience the same problems, but report that
  > they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
  > prevent that this happens again for egcs-1.1.2, and discourage the
  > author to release any "fixed" version of egcs-1.1.2.
Or fix this (and only this) problem and make egcs-1.1.3.

  > It is probably not a bad idea to revert the weak reference to
  > "__register_frame_info" patch in egcs-1.1.2.  It may result in larger
  > executables with glibc 2.0, but at least we get rid of the shared
  > library compatibility problem.  For glibc 2.1 things shouldn't matter
  > since "__register_frame_info" is exported from libc, which by default
  > every program and shared library is linked against.
jeff

Re: __register_frame_info & shared library compatibility

[John Polstra]

In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote:

>   > The problem is indeed that egcs 1.1.2 now puts only a weak
>   > reference to "__register_frame_info" in crtstuff.  This means
>   > that "du moment" one recompiles a shared library that previously
>   > contained "__register_frame_info" it probably does not anymore,
>   > and programs linked with that library stop running.
> Yup.  And I consider this a horrible breakage.  I trusted a couple
> of folks with more experience in this area to guide the decision to
> include that patch.  They made a mistake.  We need to rectify it.

I don't understand why recompiling a shared library with the
weak-referencing version of egcs would cause programs linked with it
to stop working.  The library either uses exceptions, or it doesn't.
If it uses exceptions, it contains calls to __throw and/or other
functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
contains a strong reference to __frame_state_for. __frame_state_for
is defined in frame.o, so that will also be pulled in from libgcc.a.
Presto, the library has __register_frame_info just the same as always.

If the library doesn't use exceptions, then it won't pull in frame.o
and it won't have __register_frame_info.  That won't matter for the
library itself, because of the weak references in crtbeginS.o.  Now,
the executable and any other shared libraries that use exceptions
(if any) will each contain __register_frame_info, and the weak
references to __register_frame_info from the shared library in
question will presumably be resolved by the dynamic linker to the
first of those copies.  So the library will be calling a copy of
__register_frame_info that resides in the executable or in a different
shared library.  Is that a problem?  I.e., does each shared library
need to have its own copy of __register_frame_info and the associated
static variables?

My usual context is FreeBSD rather than Linux, so maybe I'm missing
something fundamental here.  If so, please enlighten me, as we are
working on switching to egcs-1.1.2 in the FreeBSD project as we speak.

John
-- 
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904080111.SAA76942@vashon.polstra.com >you write:
  > In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote
  > :
  > 
  > >   > The problem is indeed that egcs 1.1.2 now puts only a weak
  > >   > reference to "__register_frame_info" in crtstuff.  This means
  > >   > that "du moment" one recompiles a shared library that previously
  > >   > contained "__register_frame_info" it probably does not anymore,
  > >   > and programs linked with that library stop running.
  > > Yup.  And I consider this a horrible breakage.  I trusted a couple
  > > of folks with more experience in this area to guide the decision to
  > > include that patch.  They made a mistake.  We need to rectify it.
  > 
  > I don't understand why recompiling a shared library with the
  > weak-referencing version of egcs would cause programs linked with it
  > to stop working.  The library either uses exceptions, or it doesn't.
  > If it uses exceptions, it contains calls to __throw and/or other
  > functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
  > contains a strong reference to __frame_state_for. __frame_state_for
  > is defined in frame.o, so that will also be pulled in from libgcc.a.
  > Presto, the library has __register_frame_info just the same as always.
Previously the library would always have those routines, even if it did
not use EH.

And any program which referenced that library on the link line would
get those routines via the library intead of getting its own copies.

Now you recompile the library.  It no longer gets the EH routines.  Blam
your user program stops working.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Presumably this is all solved by using Glibc 2.1?

In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
the weak references can stay, and executables avoid pointless bloat.

Long term I would like to see the weak references long term because
anything else is a waste of space.  Short term, if there's a migration
path where the weak references patch is omitted for a while until
"everyone" has upgraded to Glibc 2.1, that would be fine.

-- Jamie

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> > Previously the library would always have those routines, even if it did
> > not use EH.
> > 
> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 

Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
ftp site with tar files.


H.J.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408185632.A21668@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Previously the library would always have those routines, even if it did
  > > not use EH.
  > > 
  > > And any program which referenced that library on the link line would
  > > get those routines via the library intead of getting its own copies.
  > > 
  > > Now you recompile the library.  It no longer gets the EH routines.  Blam
  > > your user program stops working.
  > 
  > Presumably this is all solved by using Glibc 2.1?
  > 
  > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
  > the weak references can stay, and executables avoid pointless bloat.
No.  It is not an acceptable solution.  The binary incompatability brought on
by egcs-1.1.2 is not acceptable and must be fixed.  Requiring folks to
upgrade to glibc-2.1 is not an acceptable "solution".

This is precisely why we have to think much harder about the interfaces that
we put into libgcc.  Once there, they have to remain forever.


jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VIdz-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Jeffrey A Law wrote:
  > > > Previously the library would always have those routines, even if it did
  > > > not use EH.
  > > > 
  > > > And any program which referenced that library on the link line would
  > > > get those routines via the library intead of getting its own copies.
  > > > 
  > > > Now you recompile the library.  It no longer gets the EH routines.  Bla
  > m
  > > > your user program stops working.
  > > 
  > > Presumably this is all solved by using Glibc 2.1?
  > > 
  > 
  > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
  > ftp site with tar files.
What am I not being clear about?

This kind of binary breakage is not acceptable.  It doesn't matter if you
put a new glibc somewhere.  Breaking binaries like that patch did is not
acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
>   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
>   > ftp site with tar files.
> What am I not being clear about?
> 
> This kind of binary breakage is not acceptable.  It doesn't matter if you
> put a new glibc somewhere.  Breaking binaries like that patch did is not
> acceptable.

The patch does *not break binaries*.

Installing EGCS and using it to compile new applications does
*not break binaries*.

But recompiling some shared libraries using EGCS with the patch breaks some
binaries.  Is it unreasonable to request a user upgrades to the latest
Glibc 2.0, or any Glibc 2.1 before doing this?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Is is possible to include a stub weak *definition* of
__register_frame_info in crtbeginS.o, which is nothing more than a
return instruction?

Would that solve these temporary binary compatibility problems, without
the permanent bloat?

-- Jamie

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
> >   > ftp site with tar files.
> > What am I not being clear about?
> > 
> > This kind of binary breakage is not acceptable.  It doesn't matter if you
> > put a new glibc somewhere.  Breaking binaries like that patch did is not
> > acceptable.
> 
> The patch does *not break binaries*.
> 
> Installing EGCS and using it to compile new applications does
> *not break binaries*.
> 
> But recompiling some shared libraries using EGCS with the patch breaks some
> binaries.  Is it unreasonable to request a user upgrades to the latest
> Glibc 2.0, or any Glibc 2.1 before doing this?
> 

Exactly. That is a glibc problem. not an egcs problem. I don't think
egcs should get involved with it. Maybe we should put it in the glibc
FAQ.

-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Is is possible to include a stub weak *definition* of
> __register_frame_info in crtbeginS.o, which is nothing more than a
> return instruction?
> 
> Would that solve these temporary binary compatibility problems, without
> the permanent bloat?
> 

That means we have to do more testing without much benefit. I don't
think it is wise for egcs people to spend any more time on this. It is
a glibc problem and let glibc people deal with it. If some glibc users
won't listen to the glibc people, they get what they deserved.


-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408214924.A21750@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up 
  > a
  > >   > ftp site with tar files.
  > > What am I not being clear about?
  > > 
  > > This kind of binary breakage is not acceptable.  It doesn't matter if you
  > > put a new glibc somewhere.  Breaking binaries like that patch did is not
  > > acceptable.
  > 
  > The patch does *not break binaries*.
It does if one rebuilds shared libraries after installing that patch.  Existing
binaries may no longer run.

  > Installing EGCS and using it to compile new applications does
  > *not break binaries*.
Agreed.


  > But recompiling some shared libraries using EGCS with the patch breaks some
  > binaries.  Is it unreasonable to request a user upgrades to the latest
  > Glibc 2.0, or any Glibc 2.1 before doing this?
Yes, it is unreasonable.    This kind of breakage is not acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408215451.B21750@pcep-jamie.cern.ch >you write:
  > Is is possible to include a stub weak *definition* of
  > __register_frame_info in crtbeginS.o, which is nothing more than a
  > return instruction?
No, because if the user binary actually expected to use the
__register_frame_info that was originally satisfied by the library, then it'll
get the useless stub instead of the real implementation.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKvO-000ErNC@ocean.lucon.org >you write:
  > Exactly. That is a glibc problem. not an egcs problem. I don't think
  > egcs should get involved with it. Maybe we should put it in the glibc
  > FAQ.
No, this is an egcs problem.  We can't have these kinds of problems show
up when folks rebuild their libraries with newer versions of the compiler.

Can I be any clearer?

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

[BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]

> No, this is an egcs problem.  We can't have these kinds of problems show
> up when folks rebuild their libraries with newer versions of the compiler.
> 
> Can I be any clearer?

The policy is pretty clear.

The danger is that we get locked in to including bloat in all binaries,
that can't ever be removed.  I'm interested in a solution to that long
term problem.  If we have to include the bloat in the interim to provide
binary compatibility, that is fine.

Though frankly, the bloat due to the redundant .eh_frame in C binaries
much more significant.  Is throwing an exception through C really
supposed to have defined behaviour?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKyt-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Is is possible to include a stub weak *definition* of
  > > __register_frame_info in crtbeginS.o, which is nothing more than a
  > > return instruction?
  > > 
  > > Would that solve these temporary binary compatibility problems, without
  > > the permanent bloat?
  > > 
  > 
  > That means we have to do more testing without much benefit. I don't
  > think it is wise for egcs people to spend any more time on this. It is
  > a glibc problem and let glibc people deal with it. If some glibc users
  > won't listen to the glibc people, they get what they deserved.
It's not just a glibc problem.

Consider a system where all the system libraries were compiled with gcc-2.7
(like an older RedHat distribution).

A developer later installed egcs-1.1.1 and compiled some package that
installed some new shared libraries (let's say X11 libraries).    Then he
built a bunch of apps that link against those libraries (lots of X11 clients)

Later that developer installs egcs-1.1.2 and rebuilds the shared libraries.
He should be able to reasonably assume that the clients would still work.  But
because of your incompatible change they will not.



jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408222418.C21807@pcep-jamie.cern.ch >you write:
  > The danger is that we get locked in to including bloat in all binaries,
  > that can't ever be removed.  I'm interested in a solution to that long
  > term problem.  If we have to include the bloat in the interim to provide
  > binary compatibility, that is fine.
We already know at some point we will have to break binary compatibility in
the future to get some new C++ features.  That would be the time to drop the
old interfaces.

But in general, if an interface is added with external scope in libgcc, then it
has to stay indefinitely (until the next planned binary breakage).  Thus we
have to be a *LOT* more careful about what interfaces we put in that library --
we don't want to plan many binary breakages.


  > Though frankly, the bloat due to the redundant .eh_frame in C binaries
  > much more significant.  Is throwing an exception through C really
  > supposed to have defined behaviour?
?!?  I don't think we create eh_frames for C code by default now.  We do
if one specifies -fexceptions though.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Later that developer installs egcs-1.1.2 and rebuilds the shared
> libraries.  He should be able to reasonably assume that the clients
> would still work.  But because of your incompatible change they will
> not.

The error on the part of the developer was to install egcs-1.1.2 without
upgrading glibc.

Could we not guard against that error, by failing at link time against
the older glibc?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> We already know at some point we will have to break binary compatibility in
> the future to get some new C++ features.  That would be the time to drop the
> old interfaces.

If you're prepared to break the C shared libraries at the same time,
your proposal sounds perfect.

>   > Though frankly, the bloat due to the redundant .eh_frame in C binaries
>   > much more significant.  Is throwing an exception through C really
>   > supposed to have defined behaviour?
> ?!?  I don't think we create eh_frames for C code by default now.  We do
> if one specifies -fexceptions though.

Sorry, you're right.  I have seen big ones before, and I just checked
.eh_frame and found a small one on a recent C binary.  Even a trivial
program has a tiny .eh_frame of 4 bytes.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408223252.A21853@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Later that developer installs egcs-1.1.2 and rebuilds the shared
  > > libraries.  He should be able to reasonably assume that the clients
  > > would still work.  But because of your incompatible change they will
  > > not.
  > 
  > The error on the part of the developer was to install egcs-1.1.2 without
  > upgrading glibc.
Installing egcs-1.1.2 should never require updating the system library. 
Policy, not subject for debate.

  > Could we not guard against that error, by failing at link time against
  > the older glibc?
No.  egcs-1.1.2 has to work with older libcs.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408224135.A21864@pcep-jamie.cern.ch >you write:

  > If you're prepared to break the C shared libraries at the same time,
  > your proposal sounds perfect.
I'd _much_ prefer not to, but if people are going to demand removal of those
dead procedures then the only time that will be possible is when we already
have to break binary compatibility for C++.



  > Sorry, you're right.  I have seen big ones before, and I just checked
  > .eh_frame and found a small one on a recent C binary.  Even a trivial
  > program has a tiny .eh_frame of 4 bytes.
Hmmm, I wonder where the tiny .eh_frame came from..  Probably  something in
libgcc :-)

jeff

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Thu, Apr 08, 1999 at 10:24:18PM +0200, Jamie Lokier wrote:
> [BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]
> 
> > No, this is an egcs problem.  We can't have these kinds of problems show
> > up when folks rebuild their libraries with newer versions of the compiler.
> > 
> > Can I be any clearer?
> 
> The policy is pretty clear.
> 
> The danger is that we get locked in to including bloat in all binaries,
> that can't ever be removed.  I'm interested in a solution to that long
> term problem.  If we have to include the bloat in the interim to provide
> binary compatibility, that is fine.

It's better to have bloated libraries than broken libraries. ;-) Binary
compatibility issues affect thousands of projects and millions of users.

I build binary packages for a number of projects, and maintain Linux
systems for various people. My life is easiest when the C compiler and libc
adhere to well-defined rules. When things get confusing, I make mistakes
and cause trouble for downstream users. Other people make mistakes, too,
causing me grief. (Right now, both my desktop and latest Mozilla build are
broken because somebody at RHAD labs thought it would be fun to compile
glib and gtk with egcs. I'll fix this when I have nothing better to do.)

Now, whether egcs requires updates to libc is your business, not mine.
What *is* essential, though, is that all of us downstream packagers and
users understand the rules. Otherwise, we'll screw up and make a mess.

So please keep the rules as simple as possible, and document all breakages
carefully. Legacy installations are a pain in the neck, but they're also a
fact of life.

Thank you for letting me get this off my chest. :-)

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[John Polstra]

Jeffrey A Law wrote:
>   > I don't understand why recompiling a shared library with the
>   > weak-referencing version of egcs would cause programs linked with it
>   > to stop working.
...
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Thanks for the explanation.  That clears it up.

It seems like this is an instance of a more general problem that often
crops up when you try to mix archive libraries with shared libraries.
If libgcc were shared, then this problem wouldn't exist ... right?

John
---
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < XFMail.990408145548.jdp@polstra.com >you write:
  > It seems like this is an instance of a more general problem that often
  > crops up when you try to mix archive libraries with shared libraries.
  > If libgcc were shared, then this problem wouldn't exist ... right?
Yes.  But there's a whole different can of worms if you make libgcc
a shared library :-)


jeff

Re: __register_frame_info & shared library compatibility

[Philipp Thomas]

On Thu, 08 Apr 1999 14:55:48 -0700 (PDT), you wrote:

>If libgcc were shared, then this problem wouldn't exist ... right?

Yea, but you would trade one problem with a whole bunch of new ones a
shared libgcc would introduce as the discussion some time ago showed.

Philipp Thomas

-- 
caffeine low .... brain halted

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 
> In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> the weak references can stay, and executables avoid pointless bloat.

I know I'm a little bit late on this discussion, but I would really like
to point out to most of the list members one of Henry Spencer's "Ten
Commandments for C Programmers" :

X   Thou shalt foreswear, renounce, and abjure the vile heresy which
    claimeth that ``All the world's a VAX'', and have no commerce with
    the benighted heathens who cling to this barbarous belief, that
    the days of thy program may be long even though the days of thy
    current machine be short.

        This particular heresy bids fair to be replaced by ``All the
        world's a Sun'' or ``All the world's a 386'' (this latter
..snip..


I other words:  ALL THE WORLD IS NOT LINUX!

FreeBSD has moved to using EGCS 1.1.2 as the base compiler.  And I don't
just mean for userland, it is our kernel compiler too.  We have no other
C/C++ compilers in the base system.  NetBSD has moved some of their
architectures to this point, and OpenBSD has plans to.

So Linux-centric hacks hacks that bring havoc into the world are NOT
appropriate.

-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

> > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> > the weak references can stay, and executables avoid pointless bloat.

David O'Brien wrote:
> ALL THE WORLD IS NOT LINUX!

We were discussing a problem that only appeared in Linux distributions.
The suggestion to upgrade Glibc was meant only for Linux users.

Sorry you got the wrong end of that stick.

Anyway, the discussion is over now.

-- Jamie

"all the world's not Linux"

[Joe Buck]

> > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> > the weak references can stay, and executables avoid pointless bloat.
> 
> I know I'm a little bit late on this discussion, but I would really like
> to point out to most of the list members one of Henry Spencer's "Ten
> Commandments for C Programmers" :

The discussion was about how to fix some problems with glibc version
clashes, and it's not relevant to folks on other platforms.  Remain calm.

However:

> I other words:  ALL THE WORLD IS NOT LINUX!
> 
> FreeBSD has moved to using EGCS 1.1.2 as the base compiler.

We have lots of Linux folks actively cooperating with development.  We
also have Marc Espie from OpenBSD working to get patches contributed
and merged.  But the FreeBSD folks have not been communicating with the
egcs developers, and haven't contributed their changes.  If this persists,
then you can expect that egcs/gcc will break things on FreeBSD regularly,
simply because there's no communication channel open and the developers
won't know.

If you want good FreeBSD support, you need to help.

> So Linux-centric hacks hacks that bring havoc into the world are NOT
> appropriate.

No such hack has been proposed (well, OK, HJ keeps proposing
Linux-specific hacks, but Jeff always then demands a more general
solution).

Re: "all the world's not Linux"

[David O'Brien]

> But the FreeBSD folks have not been communicating with the egcs
> developers, and haven't contributed their changes.  

I actually have tried to contribute some of our patches, and they were
rejected in a way that I have no hope they ever will.

I am reading up a set of patches for FreeBSD, but I would like some
assurances that the changes that I submit (and only affect FreeBSD) will
be committed.  FreeBSD people should be able to some influence on the
direction EGCS takes as it supports FreeBSD.

-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> We were discussing a problem that only appeared in Linux distributions.
> The suggestion to upgrade Glibc was meant only for Linux users.

Postings from John Polstra <jdp@polstra.com> *were* about problems with
__register_frame_info on FreeBSD.

I will drop it now, but I just wanted others to remember that as Jeff Law
said, interface changes to libgcc affect everyone.
 
-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: "all the world's not Linux"

[David Edelsohn]

>>>>> "David O'Brien" writes:

David> I am reading up a set of patches for FreeBSD, but I would like some
David> assurances that the changes that I submit (and only affect FreeBSD) will
David> be committed.  FreeBSD people should be able to some influence on the
David> direction EGCS takes as it supports FreeBSD.

	If the changes only affect FreeBSD *AND* do not place any
unreasonable compatibility burdens on the rest of EGCS, then I do not see
any reason that they would be rejected.

David

Re: __register_frame_info & shared library compatibility

[Joe Buck]

> > We were discussing a problem that only appeared in Linux distributions.
> > The suggestion to upgrade Glibc was meant only for Linux users.
> 
> Postings from John Polstra <jdp@polstra.com> *were* about problems with
> __register_frame_info on FreeBSD.

Whoops (removes foot from mouth :-).  I was thinking of another part of
this thread which was glibc-specific.

Yet another reason why we need closer cooperation with the BSD folks
(so we find out about these things sooner).

Re: __register_frame_info & shared library compatibility

[Marc Espie]

In article < 199904212017.NAA20578@atrus.synopsys.com > you write:
>
>Yet another reason why we need closer cooperation with the BSD folks
>(so we find out about these things sooner).

Now that OpenBSD 2.5 is out the door, I am starting the process of trying
to switch. i386 going well so far, I expect a few ICEs from m68k.

Don't worry, you'll get *very* close cooperation from us :)

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904212033.WAA27543@quatramaran.ens.fr >you write:
  > Now that OpenBSD 2.5 is out the door, I am starting the process of trying
  > to switch. i386 going well so far, I expect a few ICEs from m68k.
Yea.  However, this kind of m68k testing would be _greatly_ appreciated.  I do
builds on m68ks once in a while, and build crosses regularly to test bugs.

But it's rare for me to build any serious code other than the compiler
itself on m68ks -- too slow and I had to return my m68k hp to the UofU.

jeff

"all the world's not Linux"

[Joe Buck]

> > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> > the weak references can stay, and executables avoid pointless bloat.
> 
> I know I'm a little bit late on this discussion, but I would really like
> to point out to most of the list members one of Henry Spencer's "Ten
> Commandments for C Programmers" :

The discussion was about how to fix some problems with glibc version
clashes, and it's not relevant to folks on other platforms.  Remain calm.

However:

> I other words:  ALL THE WORLD IS NOT LINUX!
> 
> FreeBSD has moved to using EGCS 1.1.2 as the base compiler.

We have lots of Linux folks actively cooperating with development.  We
also have Marc Espie from OpenBSD working to get patches contributed
and merged.  But the FreeBSD folks have not been communicating with the
egcs developers, and haven't contributed their changes.  If this persists,
then you can expect that egcs/gcc will break things on FreeBSD regularly,
simply because there's no communication channel open and the developers
won't know.

If you want good FreeBSD support, you need to help.

> So Linux-centric hacks hacks that bring havoc into the world are NOT
> appropriate.

No such hack has been proposed (well, OK, HJ keeps proposing
Linux-specific hacks, but Jeff always then demands a more general
solution).

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408215451.B21750@pcep-jamie.cern.ch >you write:
  > Is is possible to include a stub weak *definition* of
  > __register_frame_info in crtbeginS.o, which is nothing more than a
  > return instruction?
No, because if the user binary actually expected to use the
__register_frame_info that was originally satisfied by the library, then it'll
get the useless stub instead of the real implementation.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408185632.A21668@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Previously the library would always have those routines, even if it did
  > > not use EH.
  > > 
  > > And any program which referenced that library on the link line would
  > > get those routines via the library intead of getting its own copies.
  > > 
  > > Now you recompile the library.  It no longer gets the EH routines.  Blam
  > > your user program stops working.
  > 
  > Presumably this is all solved by using Glibc 2.1?
  > 
  > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
  > the weak references can stay, and executables avoid pointless bloat.
No.  It is not an acceptable solution.  The binary incompatability brought on
by egcs-1.1.2 is not acceptable and must be fixed.  Requiring folks to
upgrade to glibc-2.1 is not an acceptable "solution".

This is precisely why we have to think much harder about the interfaces that
we put into libgcc.  Once there, they have to remain forever.


jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904071214.OAA13954@landau.wins.uva.nl >you write:
  > I think that you mean "__register_frame" instead of
  > "__register_frame_info".
Yea.  Minor details :-)


  >    Against our recommendations, the interface was removed from gcc2.8.
  > 
  > The "__register_frame" interface was never included in any official
  > FSF release of gcc before 2.8.
I'm well aware of that.  It had been the interface that had been in the gcc
tree for nearly 2 years though.  It was changed about 2 weeks before gcc-2.8
went out the door and just after egcs-1.0 had been released.


  > Back then, egcs was still
  > "experimental" which was probably the reason why Kenner didn't want to
  > include the (broken) "__register_frame" interface that was part of
  > egcs 1.0 in gcc 2.8.
There were other reasons.  Some technical, but mostly political.


  > However, compatibility for the old "__register_frame" was included in
  > gcc 2.8.1.  This has not been noticed by a lot of people since,
  > where the "__register_frame_info" and "__register_frame" functions are
  > part of the same object module in egcs's libgcc.a, they are in different
  > modules in gcc's libgcc.a. This was exactly why 
I didn't know they'd included it in gcc-2.8.1.   Good.  However keeping them
in different modules is still not necessarily right, particularly when one
considers shared libraries and dynamically loaded code.

  > However, this issue is irrelevant for the current
  > "__register_frame_info & shared library compatibility" discussion.
  > The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
  > and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
  > indeed that egcs 1.1.2 now puts only a weak reference to
  > "__register_frame_info" in crtstuff.  This means that "du moment" one
  > recompiles a shared library that previously contained
  > "__register_frame_info" it probably does not anymore, and programs
  > linked with that library stop running.
Yup.  And I consider this a horrible breakage.  I trusted a couple of folks
with more experience in this area to guide the decision to include that
patch.  They made a mistake.  We need to rectify it.

  > Now, things get really complicated because of the egcs/Linux
  > ``releases''.  In an attempt to fix problems the author has messed
  > around with the way "__register_frame_info" is used in the varied
  > egcs-1.1.1/Linux ``releases''.  This might mean that some people that
  > use these versions will experience the same problems, but report that
  > they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
  > prevent that this happens again for egcs-1.1.2, and discourage the
  > author to release any "fixed" version of egcs-1.1.2.
Or fix this (and only this) problem and make egcs-1.1.3.

  > It is probably not a bad idea to revert the weak reference to
  > "__register_frame_info" patch in egcs-1.1.2.  It may result in larger
  > executables with glibc 2.0, but at least we get rid of the shared
  > library compatibility problem.  For glibc 2.1 things shouldn't matter
  > since "__register_frame_info" is exported from libc, which by default
  > every program and shared library is linked against.
jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VIdz-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Jeffrey A Law wrote:
  > > > Previously the library would always have those routines, even if it did
  > > > not use EH.
  > > > 
  > > > And any program which referenced that library on the link line would
  > > > get those routines via the library intead of getting its own copies.
  > > > 
  > > > Now you recompile the library.  It no longer gets the EH routines.  Bla
  > m
  > > > your user program stops working.
  > > 
  > > Presumably this is all solved by using Glibc 2.1?
  > > 
  > 
  > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
  > ftp site with tar files.
What am I not being clear about?

This kind of binary breakage is not acceptable.  It doesn't matter if you
put a new glibc somewhere.  Breaking binaries like that patch did is not
acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> We already know at some point we will have to break binary compatibility in
> the future to get some new C++ features.  That would be the time to drop the
> old interfaces.

If you're prepared to break the C shared libraries at the same time,
your proposal sounds perfect.

>   > Though frankly, the bloat due to the redundant .eh_frame in C binaries
>   > much more significant.  Is throwing an exception through C really
>   > supposed to have defined behaviour?
> ?!?  I don't think we create eh_frames for C code by default now.  We do
> if one specifies -fexceptions though.

Sorry, you're right.  I have seen big ones before, and I just checked
.eh_frame and found a small one on a recent C binary.  Even a trivial
program has a tiny .eh_frame of 4 bytes.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904080111.SAA76942@vashon.polstra.com >you write:
  > In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote
  > :
  > 
  > >   > The problem is indeed that egcs 1.1.2 now puts only a weak
  > >   > reference to "__register_frame_info" in crtstuff.  This means
  > >   > that "du moment" one recompiles a shared library that previously
  > >   > contained "__register_frame_info" it probably does not anymore,
  > >   > and programs linked with that library stop running.
  > > Yup.  And I consider this a horrible breakage.  I trusted a couple
  > > of folks with more experience in this area to guide the decision to
  > > include that patch.  They made a mistake.  We need to rectify it.
  > 
  > I don't understand why recompiling a shared library with the
  > weak-referencing version of egcs would cause programs linked with it
  > to stop working.  The library either uses exceptions, or it doesn't.
  > If it uses exceptions, it contains calls to __throw and/or other
  > functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
  > contains a strong reference to __frame_state_for. __frame_state_for
  > is defined in frame.o, so that will also be pulled in from libgcc.a.
  > Presto, the library has __register_frame_info just the same as always.
Previously the library would always have those routines, even if it did
not use EH.

And any program which referenced that library on the link line would
get those routines via the library intead of getting its own copies.

Now you recompile the library.  It no longer gets the EH routines.  Blam
your user program stops working.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408224135.A21864@pcep-jamie.cern.ch >you write:

  > If you're prepared to break the C shared libraries at the same time,
  > your proposal sounds perfect.
I'd _much_ prefer not to, but if people are going to demand removal of those
dead procedures then the only time that will be possible is when we already
have to break binary compatibility for C++.



  > Sorry, you're right.  I have seen big ones before, and I just checked
  > .eh_frame and found a small one on a recent C binary.  Even a trivial
  > program has a tiny .eh_frame of 4 bytes.
Hmmm, I wonder where the tiny .eh_frame came from..  Probably  something in
libgcc :-)

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Later that developer installs egcs-1.1.2 and rebuilds the shared
> libraries.  He should be able to reasonably assume that the clients
> would still work.  But because of your incompatible change they will
> not.

The error on the part of the developer was to install egcs-1.1.2 without
upgrading glibc.

Could we not guard against that error, by failing at link time against
the older glibc?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   From: Jeffrey A Law (law@cygnus.com)

   Back in 1997 we tried _very_ hard to convince Richard Kenner and Richard
   Stallman to keep the old "__register_frame_info" interface in libgcc.a for
   gcc2 for precisely this problem.

I think that you mean "__register_frame" instead of
"__register_frame_info".

   Against our recommendations, the interface was removed from gcc2.8.

The "__register_frame" interface was never included in any official
FSF release of gcc before 2.8.  Back then, egcs was still
"experimental" which was probably the reason why Kenner didn't want to
include the (broken) "__register_frame" interface that was part of
egcs 1.0 in gcc 2.8.  However, compatibility for the old
"__register_frame" was included in gcc 2.8.1.  This has not been
noticed by a lot of people since, where the "__register_frame_info"
and "__register_frame" functions are part of the same object module in
egcs's libgcc.a, they are in different modules in gcc's libgcc.a.
This was exactly why 

However, this issue is irrelevant for the current
"__register_frame_info & shared library compatibility" discussion.
The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
indeed that egcs 1.1.2 now puts only a weak reference to
"__register_frame_info" in crtstuff.  This means that "du moment" one
recompiles a shared library that previously contained
"__register_frame_info" it probably does not anymore, and programs
linked with that library stop running.

Now, things get really complicated because of the egcs/Linux
``releases''.  In an attempt to fix problems the author has messed
around with the way "__register_frame_info" is used in the varied
egcs-1.1.1/Linux ``releases''.  This might mean that some people that
use these versions will experience the same problems, but report that
they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
prevent that this happens again for egcs-1.1.2, and discourage the
author to release any "fixed" version of egcs-1.1.2.

It is probably not a bad idea to revert the weak reference to
"__register_frame_info" patch in egcs-1.1.2.  It may result in larger
executables with glibc 2.0, but at least we get rid of the shared
library compatibility problem.  For glibc 2.1 things shouldn't matter
since "__register_frame_info" is exported from libc, which by default
every program and shared library is linked against.

Mark

P.S. I haven't tested this, but using

   "-Wl,-u __register_frame_info"

when rebuilding a shared library with egcs-1.1.2 (or egcs-1.1.1/Linux)
will probably avoid the problems discussed in this thread.

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

> > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> > the weak references can stay, and executables avoid pointless bloat.

David O'Brien wrote:
> ALL THE WORLD IS NOT LINUX!

We were discussing a problem that only appeared in Linux distributions.
The suggestion to upgrade Glibc was meant only for Linux users.

Sorry you got the wrong end of that stick.

Anyway, the discussion is over now.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Is is possible to include a stub weak *definition* of
__register_frame_info in crtbeginS.o, which is nothing more than a
return instruction?

Would that solve these temporary binary compatibility problems, without
the permanent bloat?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Thu, Apr 08, 1999 at 10:24:18PM +0200, Jamie Lokier wrote:
> [BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]
> 
> > No, this is an egcs problem.  We can't have these kinds of problems show
> > up when folks rebuild their libraries with newer versions of the compiler.
> > 
> > Can I be any clearer?
> 
> The policy is pretty clear.
> 
> The danger is that we get locked in to including bloat in all binaries,
> that can't ever be removed.  I'm interested in a solution to that long
> term problem.  If we have to include the bloat in the interim to provide
> binary compatibility, that is fine.

It's better to have bloated libraries than broken libraries. ;-) Binary
compatibility issues affect thousands of projects and millions of users.

I build binary packages for a number of projects, and maintain Linux
systems for various people. My life is easiest when the C compiler and libc
adhere to well-defined rules. When things get confusing, I make mistakes
and cause trouble for downstream users. Other people make mistakes, too,
causing me grief. (Right now, both my desktop and latest Mozilla build are
broken because somebody at RHAD labs thought it would be fun to compile
glib and gtk with egcs. I'll fix this when I have nothing better to do.)

Now, whether egcs requires updates to libc is your business, not mine.
What *is* essential, though, is that all of us downstream packagers and
users understand the rules. Otherwise, we'll screw up and make a mess.

So please keep the rules as simple as possible, and document all breakages
carefully. Legacy installations are a pain in the neck, but they're also a
fact of life.

Thank you for letting me get this off my chest. :-)

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < XFMail.990408145548.jdp@polstra.com >you write:
  > It seems like this is an instance of a more general problem that often
  > crops up when you try to mix archive libraries with shared libraries.
  > If libgcc were shared, then this problem wouldn't exist ... right?
Yes.  But there's a whole different can of worms if you make libgcc
a shared library :-)


jeff

Re: __register_frame_info & shared library compatibility

[John Polstra]

Jeffrey A Law wrote:
>   > I don't understand why recompiling a shared library with the
>   > weak-referencing version of egcs would cause programs linked with it
>   > to stop working.
...
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Thanks for the explanation.  That clears it up.

It seems like this is an instance of a more general problem that often
crops up when you try to mix archive libraries with shared libraries.
If libgcc were shared, then this problem wouldn't exist ... right?

John
---
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408222418.C21807@pcep-jamie.cern.ch >you write:
  > The danger is that we get locked in to including bloat in all binaries,
  > that can't ever be removed.  I'm interested in a solution to that long
  > term problem.  If we have to include the bloat in the interim to provide
  > binary compatibility, that is fine.
We already know at some point we will have to break binary compatibility in
the future to get some new C++ features.  That would be the time to drop the
old interfaces.

But in general, if an interface is added with external scope in libgcc, then it
has to stay indefinitely (until the next planned binary breakage).  Thus we
have to be a *LOT* more careful about what interfaces we put in that library --
we don't want to plan many binary breakages.


  > Though frankly, the bloat due to the redundant .eh_frame in C binaries
  > much more significant.  Is throwing an exception through C really
  > supposed to have defined behaviour?
?!?  I don't think we create eh_frames for C code by default now.  We do
if one specifies -fexceptions though.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKvO-000ErNC@ocean.lucon.org >you write:
  > Exactly. That is a glibc problem. not an egcs problem. I don't think
  > egcs should get involved with it. Maybe we should put it in the glibc
  > FAQ.
No, this is an egcs problem.  We can't have these kinds of problems show
up when folks rebuild their libraries with newer versions of the compiler.

Can I be any clearer?

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKyt-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Is is possible to include a stub weak *definition* of
  > > __register_frame_info in crtbeginS.o, which is nothing more than a
  > > return instruction?
  > > 
  > > Would that solve these temporary binary compatibility problems, without
  > > the permanent bloat?
  > > 
  > 
  > That means we have to do more testing without much benefit. I don't
  > think it is wise for egcs people to spend any more time on this. It is
  > a glibc problem and let glibc people deal with it. If some glibc users
  > won't listen to the glibc people, they get what they deserved.
It's not just a glibc problem.

Consider a system where all the system libraries were compiled with gcc-2.7
(like an older RedHat distribution).

A developer later installed egcs-1.1.1 and compiled some package that
installed some new shared libraries (let's say X11 libraries).    Then he
built a bunch of apps that link against those libraries (lots of X11 clients)

Later that developer installs egcs-1.1.2 and rebuilds the shared libraries.
He should be able to reasonably assume that the clients would still work.  But
because of your incompatible change they will not.



jeff

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> > Previously the library would always have those routines, even if it did
> > not use EH.
> > 
> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 

Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
ftp site with tar files.


H.J.

Re: "all the world's not Linux"

[David Edelsohn]

>>>>> "David O'Brien" writes:

David> I am reading up a set of patches for FreeBSD, but I would like some
David> assurances that the changes that I submit (and only affect FreeBSD) will
David> be committed.  FreeBSD people should be able to some influence on the
David> direction EGCS takes as it supports FreeBSD.

	If the changes only affect FreeBSD *AND* do not place any
unreasonable compatibility burdens on the rest of EGCS, then I do not see
any reason that they would be rejected.

David

Re: __register_frame_info & shared library compatibility

[Philipp Thomas]

On Thu, 08 Apr 1999 14:55:48 -0700 (PDT), you wrote:

>If libgcc were shared, then this problem wouldn't exist ... right?

Yea, but you would trade one problem with a whole bunch of new ones a
shared libgcc would introduce as the discussion some time ago showed.

Philipp Thomas

-- 
caffeine low .... brain halted

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

[BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]

> No, this is an egcs problem.  We can't have these kinds of problems show
> up when folks rebuild their libraries with newer versions of the compiler.
> 
> Can I be any clearer?

The policy is pretty clear.

The danger is that we get locked in to including bloat in all binaries,
that can't ever be removed.  I'm interested in a solution to that long
term problem.  If we have to include the bloat in the interim to provide
binary compatibility, that is fine.

Though frankly, the bloat due to the redundant .eh_frame in C binaries
much more significant.  Is throwing an exception through C really
supposed to have defined behaviour?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
>   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
>   > ftp site with tar files.
> What am I not being clear about?
> 
> This kind of binary breakage is not acceptable.  It doesn't matter if you
> put a new glibc somewhere.  Breaking binaries like that patch did is not
> acceptable.

The patch does *not break binaries*.

Installing EGCS and using it to compile new applications does
*not break binaries*.

But recompiling some shared libraries using EGCS with the patch breaks some
binaries.  Is it unreasonable to request a user upgrades to the latest
Glibc 2.0, or any Glibc 2.1 before doing this?

-- Jamie

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> We were discussing a problem that only appeared in Linux distributions.
> The suggestion to upgrade Glibc was meant only for Linux users.

Postings from John Polstra <jdp@polstra.com> *were* about problems with
__register_frame_info on FreeBSD.

I will drop it now, but I just wanted others to remember that as Jeff Law
said, interface changes to libgcc affect everyone.
 
-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904212033.WAA27543@quatramaran.ens.fr >you write:
  > Now that OpenBSD 2.5 is out the door, I am starting the process of trying
  > to switch. i386 going well so far, I expect a few ICEs from m68k.
Yea.  However, this kind of m68k testing would be _greatly_ appreciated.  I do
builds on m68ks once in a while, and build crosses regularly to test bugs.

But it's rare for me to build any serious code other than the compiler
itself on m68ks -- too slow and I had to return my m68k hp to the UofU.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Presumably this is all solved by using Glibc 2.1?

In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
the weak references can stay, and executables avoid pointless bloat.

Long term I would like to see the weak references long term because
anything else is a waste of space.  Short term, if there's a migration
path where the weak references patch is omitted for a while until
"everyone" has upgraded to Glibc 2.1, that would be fine.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408214924.A21750@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up 
  > a
  > >   > ftp site with tar files.
  > > What am I not being clear about?
  > > 
  > > This kind of binary breakage is not acceptable.  It doesn't matter if you
  > > put a new glibc somewhere.  Breaking binaries like that patch did is not
  > > acceptable.
  > 
  > The patch does *not break binaries*.
It does if one rebuilds shared libraries after installing that patch.  Existing
binaries may no longer run.

  > Installing EGCS and using it to compile new applications does
  > *not break binaries*.
Agreed.


  > But recompiling some shared libraries using EGCS with the patch breaks some
  > binaries.  Is it unreasonable to request a user upgrades to the latest
  > Glibc 2.0, or any Glibc 2.1 before doing this?
Yes, it is unreasonable.    This kind of breakage is not acceptable.

jeff

Re: "all the world's not Linux"

[David O'Brien]

> But the FreeBSD folks have not been communicating with the egcs
> developers, and haven't contributed their changes.  

I actually have tried to contribute some of our patches, and they were
rejected in a way that I have no hope they ever will.

I am reading up a set of patches for FreeBSD, but I would like some
assurances that the changes that I submit (and only affect FreeBSD) will
be committed.  FreeBSD people should be able to some influence on the
direction EGCS takes as it supports FreeBSD.

-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408223252.A21853@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Later that developer installs egcs-1.1.2 and rebuilds the shared
  > > libraries.  He should be able to reasonably assume that the clients
  > > would still work.  But because of your incompatible change they will
  > > not.
  > 
  > The error on the part of the developer was to install egcs-1.1.2 without
  > upgrading glibc.
Installing egcs-1.1.2 should never require updating the system library. 
Policy, not subject for debate.

  > Could we not guard against that error, by failing at link time against
  > the older glibc?
No.  egcs-1.1.2 has to work with older libcs.

jeff

Re: __register_frame_info & shared library compatibility

[Joe Buck]

> > We were discussing a problem that only appeared in Linux distributions.
> > The suggestion to upgrade Glibc was meant only for Linux users.
> 
> Postings from John Polstra <jdp@polstra.com> *were* about problems with
> __register_frame_info on FreeBSD.

Whoops (removes foot from mouth :-).  I was thinking of another part of
this thread which was glibc-specific.

Yet another reason why we need closer cooperation with the BSD folks
(so we find out about these things sooner).

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Is is possible to include a stub weak *definition* of
> __register_frame_info in crtbeginS.o, which is nothing more than a
> return instruction?
> 
> Would that solve these temporary binary compatibility problems, without
> the permanent bloat?
> 

That means we have to do more testing without much benefit. I don't
think it is wise for egcs people to spend any more time on this. It is
a glibc problem and let glibc people deal with it. If some glibc users
won't listen to the glibc people, they get what they deserved.


-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[John Polstra]

In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote:

>   > The problem is indeed that egcs 1.1.2 now puts only a weak
>   > reference to "__register_frame_info" in crtstuff.  This means
>   > that "du moment" one recompiles a shared library that previously
>   > contained "__register_frame_info" it probably does not anymore,
>   > and programs linked with that library stop running.
> Yup.  And I consider this a horrible breakage.  I trusted a couple
> of folks with more experience in this area to guide the decision to
> include that patch.  They made a mistake.  We need to rectify it.

I don't understand why recompiling a shared library with the
weak-referencing version of egcs would cause programs linked with it
to stop working.  The library either uses exceptions, or it doesn't.
If it uses exceptions, it contains calls to __throw and/or other
functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
contains a strong reference to __frame_state_for. __frame_state_for
is defined in frame.o, so that will also be pulled in from libgcc.a.
Presto, the library has __register_frame_info just the same as always.

If the library doesn't use exceptions, then it won't pull in frame.o
and it won't have __register_frame_info.  That won't matter for the
library itself, because of the weak references in crtbeginS.o.  Now,
the executable and any other shared libraries that use exceptions
(if any) will each contain __register_frame_info, and the weak
references to __register_frame_info from the shared library in
question will presumably be resolved by the dynamic linker to the
first of those copies.  So the library will be calling a copy of
__register_frame_info that resides in the executable or in a different
shared library.  Is that a problem?  I.e., does each shared library
need to have its own copy of __register_frame_info and the associated
static variables?

My usual context is FreeBSD rather than Linux, so maybe I'm missing
something fundamental here.  If so, please enlighten me, as we are
working on switching to egcs-1.1.2 in the FreeBSD project as we speak.

John
-- 
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Marc Espie]

In article < 199904212017.NAA20578@atrus.synopsys.com > you write:
>
>Yet another reason why we need closer cooperation with the BSD folks
>(so we find out about these things sooner).

Now that OpenBSD 2.5 is out the door, I am starting the process of trying
to switch. i386 going well so far, I expect a few ICEs from m68k.

Don't worry, you'll get *very* close cooperation from us :)

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 
> In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> the weak references can stay, and executables avoid pointless bloat.

I know I'm a little bit late on this discussion, but I would really like
to point out to most of the list members one of Henry Spencer's "Ten
Commandments for C Programmers" :

X   Thou shalt foreswear, renounce, and abjure the vile heresy which
    claimeth that ``All the world's a VAX'', and have no commerce with
    the benighted heathens who cling to this barbarous belief, that
    the days of thy program may be long even though the days of thy
    current machine be short.

        This particular heresy bids fair to be replaced by ``All the
        world's a Sun'' or ``All the world's a 386'' (this latter
..snip..


I other words:  ALL THE WORLD IS NOT LINUX!

FreeBSD has moved to using EGCS 1.1.2 as the base compiler.  And I don't
just mean for userland, it is our kernel compiler too.  We have no other
C/C++ compilers in the base system.  NetBSD has moved some of their
architectures to this point, and OpenBSD has plans to.

So Linux-centric hacks hacks that bring havoc into the world are NOT
appropriate.

-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
> >   > ftp site with tar files.
> > What am I not being clear about?
> > 
> > This kind of binary breakage is not acceptable.  It doesn't matter if you
> > put a new glibc somewhere.  Breaking binaries like that patch did is not
> > acceptable.
> 
> The patch does *not break binaries*.
> 
> Installing EGCS and using it to compile new applications does
> *not break binaries*.
> 
> But recompiling some shared libraries using EGCS with the patch breaks some
> binaries.  Is it unreasonable to request a user upgrades to the latest
> Glibc 2.0, or any Glibc 2.1 before doing this?
> 

Exactly. That is a glibc problem. not an egcs problem. I don't think
egcs should get involved with it. Maybe we should put it in the glibc
FAQ.

-- 
H.J. Lu (hjl@gnu.org)