Re: __register_frame_info & shared library compatibility

Re: __register_frame_info & shared library compatibility

[N8TM]

In a message dated 4/21/99 11:55:19 AM Pacific Daylight Time, obrien@NUXI.com 
writes:

> X   Thou shalt foreswear, renounce, and abjure the vile heresy which
>      claimeth that ``All the world's a VAX'', and have no commerce with
>      the benighted heathens who cling to this barbarous belief, that
>      the days of thy program may be long even though the days of thy
>      current machine be short.
Well, there's some truth in it.  We're still running VAX, and they've seen 
fit to support something resembling standard C and a Fortran with more g77 
compatibility.

Re: __register_frame_info & shared library compatibility

[N8TM]

In a message dated 4/21/99 11:55:19 AM Pacific Daylight Time, obrien@NUXI.com 
writes:

> X   Thou shalt foreswear, renounce, and abjure the vile heresy which
>      claimeth that ``All the world's a VAX'', and have no commerce with
>      the benighted heathens who cling to this barbarous belief, that
>      the days of thy program may be long even though the days of thy
>      current machine be short.
Well, there's some truth in it.  We're still running VAX, and they've seen 
fit to support something resembling standard C and a Fortran with more g77 
compatibility.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990406170202.J29505@moebius.dartmouth.edu >you write:
  >  Object      Soname           Compiler
  >  ------      ------           --------
  >  glibc 2.0   libc.so.6        gcc 2.7.2.3
  >  gtk+ 1.2    libgtk-1.2.so.0  gcc 2.7.2.3
  >  randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)
  > 
  > Now, as I understand it, the purpose of sonames is to manage binary
  > compatibility of shared objects. Any file providing a soname
  > (libgtk-1.2.so.0, for example) should be compatible with all other files
  > that ever provided that soname. If somebody breaks that invariant, then my
  > system supposedly dies in flames.
  > 
  > As far as I can tell, breaking sonames is (1) incorrect from a technical
  > perspective, (2) disasterous for users and (3) really unprofessional.
  > 
  > Now, let's pretend that my vendor issues some updated gtk+ packages. These
  > are a bug fix release--and don't change any APIs--so they still use the
  > soname 'libgtk-1.2.so.0'. However, my vendor has chosen to compile this
  > library with egcs:
  > 
  >  Object      Soname           Compiler
  >  ------      ------           --------
  >  glibc 2.0   libc.so.6        gcc 2.7.2.3
  >  gtk+ 1.2    libgtk-1.2.so.0  egcs-1.0.3
  >  randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)
  > 
  > In theory, randomapp should still run. After all, ld.so can still find
  > libraries with the appropriate sonames. But when I try to run randomapp,
  > things get ugly:
  > 
  >   undefined symbol: __register_frame_info
Yup.  As Zack and others have mentioned, this is a well known problem.

Back in 1997 we tried _very_ hard to convince Richard Kenner and Richard
Stallman to keep the old "__register_frame_info" interface in libgcc.a for
gcc2 for precisely this problem.

Against our recommendations, the interface was removed from gcc2.8.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408215451.B21750@pcep-jamie.cern.ch >you write:
  > Is is possible to include a stub weak *definition* of
  > __register_frame_info in crtbeginS.o, which is nothing more than a
  > return instruction?
No, because if the user binary actually expected to use the
__register_frame_info that was originally satisfied by the library, then it'll
get the useless stub instead of the real implementation.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904091149.NAA16737@landau.wins.uva.nl >you write:
  > * All shared libraries compiled with gcc 2.7 do not contain or
  >   reference __register_frame_info.
Correct.


  > * GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
  >   reference __register_frame_info [1].
Incorrect.  If you compile glibc with egcs-1.1, then the resulting library
will have __register_frame_info.  This happens because the C startup files
have a reference to __register_frame_info, which in turn cause
__register_frame_info to be brought in from libgcc::frame.o.

Furthermore, the definition in the library will have external visibility.

  > * All other shared libraries compiled with gcc 2.8 or egcs-1.1 do
  >   contain __register_frame_info, except when they are linked against
  >   an other shared library that contains __register_frame_info.  They
  >   will all reference __register_frame_info.
Correct.

  > * All shared libraries compiled with egcs-1.1.2 do not contain
  >   __register_frame_info, but have a weak reference to
  >   __register_frame_info.
Correct.


  > * All programs compiled with gcc 2.7 do not contain or reference
  >   __register_frame_info.
Correct.

  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked
  >   only against libc contain and reference __register_frame_info.
Depends on whether or not they link against any libraries which provide
register_frame_info.


  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked against an
  >   other shared library besides libc do not contain
  >   __register_frame_info but do reference __register_frame_info.
Incorrect.  It depends on what is in the libraries that the program is
linked again.


  > * All programs compiled with egcs 1.1.2 do not contain
  >   __register_frame_info but have a weak reference to
  >   __register_frame_info.
Correct.

  > Now there are a few possibilities where programs will not run:
I've already explained the cases where program will stop running.

If you have a program which is linked against a shared library and the
shared library originally contained the frame functions, then that program
will expect to get the frame functions from the library and will not have
its own copies.

If you then recompile that library with egcs-1.1.2, then that library may
not get a copy of the frame functions if the only reference to them is
the weak symbol in the crt*.o files.

Now the progrm which was originally linked against that library will fail
because there is no definition of the frame functions.

This problem can be solved by making the reference to the frame function
in the crt files a normal (non-weak) reference again.  This will not break
existing code.

  >    I don't believe there is a right answer.
The right answer is to put the normal reference back.  It will not break
programs, just make them a little larger.  That was the only reason those
symbosl were make weak to begin with -- to reduce the size of programs
which did not actually use exception handling.

jeff

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Tue, Apr 06, 1999 at 06:51:23PM -0400, Zack Weinberg wrote:
> On Tue, 6 Apr 1999 18:41:17 -0400, Eric Kidd wrote:
> >I help maintain a compiler that's hard to bootstrap. Currently, I must be
> >able to compile glibc-2.0/gcc-2.7 binaries on my system. So my upgrade
> >options are very limited. :-(
> 
> Can you give details?  Maybe I can help disentangle it.

I build glibc2 binaries and RedHat 5.2 RPMS for the Gwydion Dylan compiler:

  http://www.gwydiondylan.org/

It generates ANSI C code, which we then run through a system-dependent
compiler (normally GCC). This works better than you might expect, because
the optimizer is aggressive enough to generate reasonable C code.

The public releases of the compiler link against the C library dynamically,
but use a statically-linked Dylan runtime. So we're pretty well insulated
from many common problems. But I still need to support glibc 2.0 for all
the RedHat 5.x users out there.

> Unfortunately, 2.0 and 2.1 do not coexist well.  You would almost need a
> complete chroot environment to retain the ability to build 2.0 binaries.
> Better use two machines...

I've got diskspace to spare. Is it possible to install a chrooted RedHat or
Debian build environment, given enough patience?

> You will definitely lose if you have any references to stdin/out/err
> inside the library.  If you don't, you might survive, but you might
> not.  Best advice is `try it and see'.

OK, but existing *.a files should be safe, since they haven't been linked
yet? If so, our users are OK--none of our stdio code is in shared
libraries.
 
> > * If you compile glibc with egcs, you can often use egcs-compiled shared
> >   libraries with gcc-compiled applications.
> 
> Yes - and if it doesn't work, that's a bug.

Good. So there's hope. :-) My many thanks for helping me sort through this
problem; I'll pass your advice along to other folks.

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408185632.A21668@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Previously the library would always have those routines, even if it did
  > > not use EH.
  > > 
  > > And any program which referenced that library on the link line would
  > > get those routines via the library intead of getting its own copies.
  > > 
  > > Now you recompile the library.  It no longer gets the EH routines.  Blam
  > > your user program stops working.
  > 
  > Presumably this is all solved by using Glibc 2.1?
  > 
  > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
  > the weak references can stay, and executables avoid pointless bloat.
No.  It is not an acceptable solution.  The binary incompatability brought on
by egcs-1.1.2 is not acceptable and must be fixed.  Requiring folks to
upgrade to glibc-2.1 is not an acceptable "solution".

This is precisely why we have to think much harder about the interfaces that
we put into libgcc.  Once there, they have to remain forever.


jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904141258.OAA22629@landau.wins.uva.nl >you write:
  >    If you can find a solution that works in both worlds, then great.
  >    But I do not believe one exists.  And in the absence of a perfect
  >    solution we must continue be compatible with our own releases over
  >    compatibility with gcc-2.7.
  > 
  > What about my suggestion to use weak references to
  > __register_frame_info and __deregister_frame_info in crtbegin.o and
  > normal (non-weak) references in crtbeginS.o?
I must have glossed over it.  If you've got it handy, can you resend (or I'll
grope around the archives for it).


jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904071214.OAA13954@landau.wins.uva.nl >you write:
  > I think that you mean "__register_frame" instead of
  > "__register_frame_info".
Yea.  Minor details :-)


  >    Against our recommendations, the interface was removed from gcc2.8.
  > 
  > The "__register_frame" interface was never included in any official
  > FSF release of gcc before 2.8.
I'm well aware of that.  It had been the interface that had been in the gcc
tree for nearly 2 years though.  It was changed about 2 weeks before gcc-2.8
went out the door and just after egcs-1.0 had been released.


  > Back then, egcs was still
  > "experimental" which was probably the reason why Kenner didn't want to
  > include the (broken) "__register_frame" interface that was part of
  > egcs 1.0 in gcc 2.8.
There were other reasons.  Some technical, but mostly political.


  > However, compatibility for the old "__register_frame" was included in
  > gcc 2.8.1.  This has not been noticed by a lot of people since,
  > where the "__register_frame_info" and "__register_frame" functions are
  > part of the same object module in egcs's libgcc.a, they are in different
  > modules in gcc's libgcc.a. This was exactly why 
I didn't know they'd included it in gcc-2.8.1.   Good.  However keeping them
in different modules is still not necessarily right, particularly when one
considers shared libraries and dynamically loaded code.

  > However, this issue is irrelevant for the current
  > "__register_frame_info & shared library compatibility" discussion.
  > The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
  > and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
  > indeed that egcs 1.1.2 now puts only a weak reference to
  > "__register_frame_info" in crtstuff.  This means that "du moment" one
  > recompiles a shared library that previously contained
  > "__register_frame_info" it probably does not anymore, and programs
  > linked with that library stop running.
Yup.  And I consider this a horrible breakage.  I trusted a couple of folks
with more experience in this area to guide the decision to include that
patch.  They made a mistake.  We need to rectify it.

  > Now, things get really complicated because of the egcs/Linux
  > ``releases''.  In an attempt to fix problems the author has messed
  > around with the way "__register_frame_info" is used in the varied
  > egcs-1.1.1/Linux ``releases''.  This might mean that some people that
  > use these versions will experience the same problems, but report that
  > they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
  > prevent that this happens again for egcs-1.1.2, and discourage the
  > author to release any "fixed" version of egcs-1.1.2.
Or fix this (and only this) problem and make egcs-1.1.3.

  > It is probably not a bad idea to revert the weak reference to
  > "__register_frame_info" patch in egcs-1.1.2.  It may result in larger
  > executables with glibc 2.0, but at least we get rid of the shared
  > library compatibility problem.  For glibc 2.1 things shouldn't matter
  > since "__register_frame_info" is exported from libc, which by default
  > every program and shared library is linked against.
jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VIdz-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Jeffrey A Law wrote:
  > > > Previously the library would always have those routines, even if it did
  > > > not use EH.
  > > > 
  > > > And any program which referenced that library on the link line would
  > > > get those routines via the library intead of getting its own copies.
  > > > 
  > > > Now you recompile the library.  It no longer gets the EH routines.  Bla
  > m
  > > > your user program stops working.
  > > 
  > > Presumably this is all solved by using Glibc 2.1?
  > > 
  > 
  > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
  > ftp site with tar files.
What am I not being clear about?

This kind of binary breakage is not acceptable.  It doesn't matter if you
put a new glibc somewhere.  Breaking binaries like that patch did is not
acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> The correct binary interface for libc 2.0 is the one you get if you
> compile all purely C libraries with gcc 2.7.  Programs that expect

That is why we have a build machine for RedHat 5.2 with no changes to
libraries and compilers. All our binaries for RedHat 5.2 are compiled
on the build machine.

> something else are broken, sorry.  (There's no reason to prefer this
> rule except that that's what all the major distributions shipped.)
> 

You can use egcs 1.1.2 if you upgrade your glibc 2.0 to glibc 2.0
1998-12-11 or above via CVS. That should make all C shared libraries
compatible with the old binaries.


-- 
H.J. Lu (hjl@gnu.org)

__register_frame_info & shared library compatibility

[Eric Kidd]

First, some background. I run Red Hat Linux 5.2, and build binaries for a
couple of projects involving shared libraries. I also maintain and upgrade
a number of Linux systems with locally compiled software.

egcs is starting to cause problems for me--or more precisely, the way
people *use* egcs is starting to cause problems for me. Let's start with
the latest problem I've encountered:

On Tue, Apr 06, 1999 at 11:23:23AM -0400, Owen Taylor wrote:
> Not true. As has been explained a couple of times here before, these
> RPM's are linked against glibc2.0. However, they are compiled against
> egcs. (The version of egcs shipped with 5.2, in fact)
>
> The consequence of the second thing is that you can't use those libraries
> with applications compiled with gcc, so if you want to compile other
> GNOME applications against these libraries, you'll temporarily have to
> make egcs your default compiler.

So here's the scenario, as best I understand it. My hypothetical system
initially contains:

 Object      Soname           Compiler
 ------      ------           --------
 glibc 2.0   libc.so.6        gcc 2.7.2.3
 gtk+ 1.2    libgtk-1.2.so.0  gcc 2.7.2.3
 randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)

Now, as I understand it, the purpose of sonames is to manage binary
compatibility of shared objects. Any file providing a soname
(libgtk-1.2.so.0, for example) should be compatible with all other files
that ever provided that soname. If somebody breaks that invariant, then my
system supposedly dies in flames.

As far as I can tell, breaking sonames is (1) incorrect from a technical
perspective, (2) disasterous for users and (3) really unprofessional.

Now, let's pretend that my vendor issues some updated gtk+ packages. These
are a bug fix release--and don't change any APIs--so they still use the
soname 'libgtk-1.2.so.0'. However, my vendor has chosen to compile this
library with egcs:

 Object      Soname           Compiler
 ------      ------           --------
 glibc 2.0   libc.so.6        gcc 2.7.2.3
 gtk+ 1.2    libgtk-1.2.so.0  egcs-1.0.3
 randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)

In theory, randomapp should still run. After all, ld.so can still find
libraries with the appropriate sonames. But when I try to run randomapp,
things get ugly:

  undefined symbol: __register_frame_info

A little bit of research discovers something nasty: egcs and gcc have
incompatible runtimes, and ld.so can't deal with this gracefully. When you
compile a program with egcs, it pulls in a runtime providing
__register_frame_info. When you compile an application with gcc 2.7, it
pulls in an older runtime that lacks this symbol.

This apparently means that you can't use an egcs-compiled shared library
with a gcc-compiled application. Or am I missing something?

If my understanding is correct, however, my hypothetical system is now
broken. The bugfix upgrade to gtk+ broke binary compatibility for the
soname 'libgtk+-1.2.so.0', with potentially disasterous consequences.

Three questions for the audience:

 1) Am I badly confused? (I often am.)
 2) What happens if I recompile glibc 2.0 with ecgs and install it under
    the soname 'libc.so.6'?
 3) Will any of this cause bad problems in the future, and if so, how
    can these problems be prevented?

Thank you for your time!

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

Here is a rather long mail, but I hope that people who want a real
solution to the __register_frame_info & shared library compatibility
problem will take the time to read it.  If you think I'm wrong, please
tell me!

   Leaving the patch out causes a different set of people problems,
   namely the ones that want to compile stuff with egcs-1.1.x and have it
   port over to systems with gcc 2.7-compiled libc.

OK, here are the facts as I understand them.  I only look at C
programs (that should not explicitly be using exceptions).  Binary
compatibility for C++ should be OK if they use the right shared
libstdc++.

gcc 2.7 = all sub-versions of gcc 2.7

gcc 2.8 = all sub-versions of gcc 2.8

egcs 1.1 = egcs-1.1 or egcs-1.1.1 and probably egcs-1.0.3

* All shared libraries compiled with gcc 2.7 do not contain or
  reference __register_frame_info.

* GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
  reference __register_frame_info [1].

* All other shared libraries compiled with gcc 2.8 or egcs-1.1 do
  contain __register_frame_info, except when they are linked against
  an other shared library that contains __register_frame_info.  They
  will all reference __register_frame_info.

* All shared libraries compiled with egcs-1.1.2 do not contain
  __register_frame_info, but have a weak reference to
  __register_frame_info.

* All programs compiled with gcc 2.7 do not contain or reference
  __register_frame_info.

* All programs compiled with gcc 2.8 or egcs-1.1 and linked
  only against libc contain and reference __register_frame_info.

* All programs compiled with gcc 2.8 or egcs-1.1 and linked against an
  other shared library besides libc do not contain
  __register_frame_info but do reference __register_frame_info.

* All programs compiled with egcs 1.1.2 do not contain
  __register_frame_info but have a weak reference to
  __register_frame_info.

Now there are a few possibilities where programs will not run:

* Any program built with gcc 2.8 or egcs-1.1 linked against an other
  shared library besides libc will not run on a system where this
  shared library was compiled with gcc 2.7 or egcs-1.1.2.

* Any program built with gcc 2.8 or egcs-1.1 linked against lib A
  where lib A was linked against lib B will not run on a system where
  lib A was not linked against lib B.

The last case is a bit far-fetched and I don't think that we should
care about it.  The first case is an unacceptable problem.

How can it be solved:

* Revert the weak __register_frame_info patch.  This solves the
  problem where the shared library was compiled with egcs-1.1.3, but
  does not solve the problem for shared libraries compiled with gcc
  2.7.  Moreover programs built with egcs-1.1.3 linked against a
  library other than libc compiled with gcc 2.8 or egcs will also not
  work with shared libraries compiled with gcc 2.7.

* Keep the weak __register_frame_info patch.  Tell people to link
  their shared libraries with the -u __register_frame_info linker
  option, or put it in the specs.

* Make crtbegin.o have a weak reference to __register_frame_info, and
  crtbeginS.o have an ordinary reference to __register_frame_info.

The third solution is probably the best one, but I don't know if all
ELF targets have a separate crtbegin.o and crtbeginS.o.

The second solution would also work.  Putting the -u option in the
specs has the advantage that it does not require any user
interaction.  However this does not work if people invoke the linker
directly.  Not putting it in the specs makes it possible for people
to omit it when they are changing the shared libraries ABI anyway.  It
should be throughly documented, but since a lot of people will not
read the documentation it will lead to a lot of "bug"-reports.

The first option keeps things as they were before egcs 1.1.2.

Note that this discussion is rather specific to Linux with glibc 2.0.
Linux with glibc 2.1 has no problems with __register_frame_info,
because __register_frame_info is included in glibc's ABI.  I don't
know how things work for the old ELF libc (libc5).

Other targets that use ELF will have similar problems, although they
might not be detected because there are less people distributing
binaries for these targets.  (The various BSD variants come to mind
where one typically builds the entire tree with only one compiler).
If it is no problem for such a target to break binary compatibility,
using the weak __register_frame_info patch as in egcs-1.1.2 is the
right thing to do.  Otherwise one of the options mentioned above
should probably be used there too.  For new ELF targets the weak
__register_frame_info patch is the right thing to do, both for
programs and shared libraries.

   I don't believe there is a right answer.

It is probably not possible to make absolutely sure that nobody will
have __register_frame_info problems anymore, but I think it is
possible to avoid problems in the more common cases.

Mark

[1] I'm not sure if this is really the case, because people always
tend to find ways to break things, but a lot of effort was made to
make sure that this is the case.

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Sun, 11 Apr 1999 22:19:53 -0600, Jeffrey A Law wrote:
>
>
>  > GNU libc has its own versions of these C startup files, and I thought
>  > they had been changed to do something similar as the weak
>  > __register_frame_info patch in egcs-1.1.2, but it turns out that this
>  > is not true.  So you're right.
>Only in releases most people do not have and are not using.

Every single user of Redhat 5.2 and Debian 2.0 has those changes.  I
wouldn't call that `not most people'

>  >    This problem can be solved by making the reference to the
>  >    frame function in the crt files a normal (non-weak) reference
>  >    again.  This will not break existing code.
>  > 
>  > But it doesn't solve the problems with a program built with egcs-1.1,
>  > linked against shared libraries built with egcs-1.1, running on a
>  > system where the shared libraries were built with gcc 2.7.  Because in
>  > that case both the program and the shared library do not contain
>  > __register_frame_info but it is refernced by the program.
>True.  But we've made not attempt to make that setup work and have no
>intention of making that setup work.

You may not have, but Mark, HJ, Ulrich and I spent about a month
trying to fix exactly this setup.  The conclusion was that the patch
you don't like was the best solution.

>But it is unacceptable when a library that was originally compiled
>with egcs-1.1.1, after compiling with egcs-1.1.2 to cause libraries
>to stop working.  Plain and simple, it is unacceptable.

The library should not have been compiled with egcs 1.1.1 in the first
place.

Yes, I realize this is an unrealistic attitude.  Would you at least
consider an -mcompat-1.0 switch?

>  > We both know that the original submitter of the patch is not very
>  > good at explaining the exact technical reasons why his patches should
>  > go in, but I remember him writing that "it fixes a bug".  I would not
>  > call generating slightly bigger binaries as bug, so I am under the
>  > impression that he was trying to solve the problem illustrated in the
>  > previous paragraph.  And I know for sure
>  > that some of people that advocated the patch were trying to solve that
>  > problem. That was the case Zack was mentioning a few messages back.
>Then I would strongly recommend that HJ explain in painful detail why this
>fixed a bug.

Here's the original bug report glibc development got:

# When I compile glibc 2.0.{6,7} with egcs, glibc ends up defining the
# symbols __register_frame_info & __deregister_frame_info. I noticed
# that these are also defined in egcs' libgcc.a Should this be
# happening?
#
# The real problem happens when I upgrade to a glibc that was not
# built with egcs.  Some of the programs I compiled with egcs under a
# egcs built glibc also have a dependency on these symbols.  If it
# wasn't /bin/login, I might not have cared so much. :-/

Please note the wording: "upgrade to a glibc that was not built with
egcs".  As long as the official libc binaries shipped by the
distributions are built by gcc 2.7, egcs MUST match gcc 2.7's ABI for
C programs.  That is what the patch achieves.  And we told the
distributions to use gcc 2.7 until they switch to glibc 2.1, so this
will continue to be a problem.

The date on that, incidentally, is last June.

Here's one of several painfully long explanations of the problem which
I wrote back in December.

# The bug is a subtle interaction between the way EGCS (all versions)
# implements exception handling, and the way shared libraries are
# constructed.
# 
# EGCS' exception handling uses a bunch of internal library functions.
# They are in libgcc.a.  Those functions are referenced from every
# translation unit compiled without -fno-exceptions, whether or not it
# has exceptions in it.  gcc 2.7/2.8 does not have these functions in
# its libgcc.a, and doesn't refer to them in programs it compiles.
# 
# When libc is built, it is linked with libgcc.  We have to do that
# because libc may need libgcc functions that some executable that uses
# it doesn't need.  The 2.0 libc then turns around and re-exports all
# the libgcc functions that it pulled in.  If 2.0 libc is compiled by
# egcs, that includes the exception handling functions.  In 2.1 we can
# prevent this but we've chosen not to; see below for why.
# 
# When executables are built, gcc links first with libgcc, second with
# libc.  The executable gets static copies of the exception functions,
# and all is well.
# 
# When other shared libraries are built, they are linked first with 
# libc and second with libgcc.  That means they don't get static copies
# of the eh functions; they refer to the ones in libc.  That
# reference will only be satisfied at runtime if the libc.so in use then
# was compiled by egcs.
# 
# My patch for 2.0 changed the link ordering for extra-libs provided by
# libc, so that they would get their own static copy of the eh
# functions.  Now there is no dependency of libm (for example) on an
# egcs-compiled libc.  But this only shifts the problem, because if you
# give gcc any -l options, it links with those before it links with
# libgcc.  Therefore the executable will want the eh functions to be in
# libm.  That will only be true at runtime if the libm.so in use then
# was compiled by egcs.
# 
# In 2.1, we can ensure that all our shared libraries have version maps
# that don't export the eh functions.  That doesn't fix the general
# problem, because most shared libraries are not provided by glibc.  A
# program linked with libX11-compiled-by-egcs will have a dependency
# which is not satisfied by libX11-compiled-by-gcc-2.8.  Instead, Ulrich
# has chosen to explicitly export the eh functions from libc.  That
# means they are part of the official 2.1 ABI, and everyone is supposed
# to reference our copy and not the one in libgcc.  This works as long
# as libc 2.1 is always compiled by egcs.  If it is ever compiled by gcc
# 2.8, it will be binary incompatible with the version compiled by egcs.
# 
# The right fix to the problem is to change gcc such that libgcc is
# always linked ahead of all libraries specified on the command line -
# i.e. if I write
# 
# gcc -o prog prog.o -lfoo -lbar
# 
# the linker command line must wind up like this:
# 
# ld -o prog crt1.o ... prog.o -lgcc -lfoo -lbar -lc -lgcc ... crtn.o
# 
# and similarly when shared libraries are linked (using gcc -shared).
# Then everyone will get a private copy of the eh functions and the
# problem will not arise.
# 
# Until this is done, the right thing to do is:
# 
# never compile glibc 2.0 with egcs
# always compile glibc 2.1 with egcs.

zw

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> We already know at some point we will have to break binary compatibility in
> the future to get some new C++ features.  That would be the time to drop the
> old interfaces.

If you're prepared to break the C shared libraries at the same time,
your proposal sounds perfect.

>   > Though frankly, the bloat due to the redundant .eh_frame in C binaries
>   > much more significant.  Is throwing an exception through C really
>   > supposed to have defined behaviour?
> ?!?  I don't think we create eh_frames for C code by default now.  We do
> if one specifies -fexceptions though.

Sorry, you're right.  I have seen big ones before, and I just checked
.eh_frame and found a small one on a recent C binary.  Even a trivial
program has a tiny .eh_frame of 4 bytes.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904080111.SAA76942@vashon.polstra.com >you write:
  > In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote
  > :
  > 
  > >   > The problem is indeed that egcs 1.1.2 now puts only a weak
  > >   > reference to "__register_frame_info" in crtstuff.  This means
  > >   > that "du moment" one recompiles a shared library that previously
  > >   > contained "__register_frame_info" it probably does not anymore,
  > >   > and programs linked with that library stop running.
  > > Yup.  And I consider this a horrible breakage.  I trusted a couple
  > > of folks with more experience in this area to guide the decision to
  > > include that patch.  They made a mistake.  We need to rectify it.
  > 
  > I don't understand why recompiling a shared library with the
  > weak-referencing version of egcs would cause programs linked with it
  > to stop working.  The library either uses exceptions, or it doesn't.
  > If it uses exceptions, it contains calls to __throw and/or other
  > functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
  > contains a strong reference to __frame_state_for. __frame_state_for
  > is defined in frame.o, so that will also be pulled in from libgcc.a.
  > Presto, the library has __register_frame_info just the same as always.
Previously the library would always have those routines, even if it did
not use EH.

And any program which referenced that library on the link line would
get those routines via the library intead of getting its own copies.

Now you recompile the library.  It no longer gets the EH routines.  Blam
your user program stops working.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408224135.A21864@pcep-jamie.cern.ch >you write:

  > If you're prepared to break the C shared libraries at the same time,
  > your proposal sounds perfect.
I'd _much_ prefer not to, but if people are going to demand removal of those
dead procedures then the only time that will be possible is when we already
have to break binary compatibility for C++.



  > Sorry, you're right.  I have seen big ones before, and I just checked
  > .eh_frame and found a small one on a recent C binary.  Even a trivial
  > program has a tiny .eh_frame of 4 bytes.
Hmmm, I wonder where the tiny .eh_frame came from..  Probably  something in
libgcc :-)

jeff

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 6 Apr 1999 17:02:02 -0400, Eric Kidd wrote:
>First, some background. I run Red Hat Linux 5.2, and build binaries for a
>couple of projects involving shared libraries. I also maintain and upgrade
>a number of Linux systems with locally compiled software.
>
>egcs is starting to cause problems for me--or more precisely, the way
>people *use* egcs is starting to cause problems for me. Let's start with
>the latest problem I've encountered:
[...]
>In theory, randomapp should still run. After all, ld.so can still find
>libraries with the appropriate sonames. But when I try to run randomapp,
>things get ugly:
>
>  undefined symbol: __register_frame_info
>
>A little bit of research discovers something nasty: egcs and gcc have
>incompatible runtimes, and ld.so can't deal with this gracefully. When you
>compile a program with egcs, it pulls in a runtime providing
>__register_frame_info. When you compile an application with gcc 2.7, it
>pulls in an older runtime that lacks this symbol.

This is a well known problem to which there is, unfortunately, no
clean solution.

The best suggestion I can make is:  Get glibc 2.1 and compile it with
egcs.  When you do that, glibc absorbs and re-exports the runtime
functions that are causing the problem.  Everything else will pick
them up from there instead of libgcc.a.  You'll have to recompile any
libraries which are currently exporting __register_frame_info.  You
will not be able to run binaries built in this environment on
non-glibc-2.1 systems.

You can not do this with glibc 2.0 because then you lose binary
compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
as you describe).

If your program and libraries are purely C, you can compile
*everything* with -fno-exceptions, which prevents the offending
functions from being linked in the first place.  You must not do this
if you care about C++.  Even if a library is written in C, it has to
use these functions to be compatible with C++ programs.  [I personally
think it's a mistake to use C++ for anything, but YMMV.]

zw

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Later that developer installs egcs-1.1.2 and rebuilds the shared
> libraries.  He should be able to reasonably assume that the clients
> would still work.  But because of your incompatible change they will
> not.

The error on the part of the developer was to install egcs-1.1.2 without
upgrading glibc.

Could we not guard against that error, by failing at link time against
the older glibc?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   From: Jeffrey A Law (law@cygnus.com)

   Back in 1997 we tried _very_ hard to convince Richard Kenner and Richard
   Stallman to keep the old "__register_frame_info" interface in libgcc.a for
   gcc2 for precisely this problem.

I think that you mean "__register_frame" instead of
"__register_frame_info".

   Against our recommendations, the interface was removed from gcc2.8.

The "__register_frame" interface was never included in any official
FSF release of gcc before 2.8.  Back then, egcs was still
"experimental" which was probably the reason why Kenner didn't want to
include the (broken) "__register_frame" interface that was part of
egcs 1.0 in gcc 2.8.  However, compatibility for the old
"__register_frame" was included in gcc 2.8.1.  This has not been
noticed by a lot of people since, where the "__register_frame_info"
and "__register_frame" functions are part of the same object module in
egcs's libgcc.a, they are in different modules in gcc's libgcc.a.
This was exactly why 

However, this issue is irrelevant for the current
"__register_frame_info & shared library compatibility" discussion.
The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
indeed that egcs 1.1.2 now puts only a weak reference to
"__register_frame_info" in crtstuff.  This means that "du moment" one
recompiles a shared library that previously contained
"__register_frame_info" it probably does not anymore, and programs
linked with that library stop running.

Now, things get really complicated because of the egcs/Linux
``releases''.  In an attempt to fix problems the author has messed
around with the way "__register_frame_info" is used in the varied
egcs-1.1.1/Linux ``releases''.  This might mean that some people that
use these versions will experience the same problems, but report that
they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
prevent that this happens again for egcs-1.1.2, and discourage the
author to release any "fixed" version of egcs-1.1.2.

It is probably not a bad idea to revert the weak reference to
"__register_frame_info" patch in egcs-1.1.2.  It may result in larger
executables with glibc 2.0, but at least we get rid of the shared
library compatibility problem.  For glibc 2.1 things shouldn't matter
since "__register_frame_info" is exported from libc, which by default
every program and shared library is linked against.

Mark

P.S. I haven't tested this, but using

   "-Wl,-u __register_frame_info"

when rebuilding a shared library with egcs-1.1.2 (or egcs-1.1.1/Linux)
will probably avoid the problems discussed in this thread.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904101341.JAA27027@blastula.phys.columbia.edu >you write:
  > >This problem can be solved by making the reference to the frame function
  > >in the crt files a normal (non-weak) reference again.  This will not break
  > >existing code.
  > 
  > This is technically true, however you will break people's
  > expectations.  Libc has been going around telling everyone that they
  > need to compile their libraries with gcc 2.7 or else apply the
  > controversial patch to egcs.  Oh, and the patch is official in egcs
  > 1.1.2.  From libc's point of view, any library that was not compiled
  > as above is broken; it fails to conform to the de facto ABI.
It does not matter if you and the glibc maintainers consider this a glibc
problem.  It is NOT ACCEPTABLE for egcs to break binary compatibility in
the manner that happened with egcs-1.1.2.  Period.

jeff

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 6 Apr 1999 17:54:09 -0400, Eric Kidd wrote:
>On Tue, Apr 06, 1999 at 05:15:31PM -0400, Zack Weinberg wrote:
>> The best suggestion I can make is: Get glibc 2.1 and compile it with
>> egcs.  When you do that, glibc absorbs and re-exports the runtime
>> functions that are causing the problem.  Everything else will pick them
>> up from there instead of libgcc.a.  You'll have to recompile any
>> libraries which are currently exporting __register_frame_info.  You will
>> not be able to run binaries built in this environment on non-glibc-2.1
>> systems.
>
>OK, this is starting to make sense. If __register_frame_info gets linked
>into glibc-2.1, then any library linked against glibc will pick up the new
>runtime functions? This seems like a fairly safe workaround, assuming that
>everything using shared libraries links dynamically against glibc.

Yes.  To make sure it works, when you link shared libraries, put an
explicit -lc on the command line.  It is wise to use the same compiler
for all the libraries, but it should work to use gcc 2.7 to compile
executables.  (Modulo all the bugs in gcc 2.7...)

Make sure you have a current binutils (2.9.1.0.2x).  You may want to
upgrade to kernel 2.2; glibc 2.1 wants to use the new features (it
does fall back gracefully) and you won't have to keep gcc 2.7 around.

>Does glibc 2.1 provide the soname libc.so.6? Does your solution somehow
>rely on this fact, or are you advising me to break my system into two
>parts: glibc 2.0 with gcc, and glibc 2.1 with egcs?

glibc 2.1 has the same soname as 2.0.  It is backward compatible as
long as your program does not use internal functions such as __write.
For some reason the most common mistake people make in this area is
to go groveling through the internals of the dynamic linker.  If you
get errors about symbols named _dl_somethingorother, that's the
problem.

Any library that uses stdio internally may need to be recompiled in
order to work with binaries built with glibc 2.1.  libstdc++,
libncurses, libslang, and libglib are known to need this.

Some C++ programs built with 2.0 break mysteriously when run on a 2.1
system.  This is due to bugs in the stdio backward compatibility
code.  We think we've fixed them all in 2.1.1, but there *is* always
one more bug :)

You can't use binaries or libraries compiled against 2.1 on a system
that has libc 2.0.  Hopefully everyone will be using 2.1 soon and this
won't be a problem.

>How does this solution affect pre-existing applications that were compiled
>with egcs and linked against a glibc 2.0 that was compiled with gcc?

Programs should work.  Libraries might not.

>> You can not do this with glibc 2.0 because then you lose binary
>> compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
>> as you describe).
>
>I still don't understand something. Why should linking
>__register_frame_info into glibc 2.1 solve the problem, but linking it into
>glibc 2.0 *cause* the problem?

glibc 2.1 can't be compiled with gcc 2.7 (it miscompiles the math
library), so we make you use egcs, so __register_frame_info will
always be present in libc.so.6.  2.0 could be compiled with gcc 2.7,
and most distributions shipped it that way.

>Is there any hope that Red Hat and other distribution vendors will get this
>right, or will I have to spend a week or two cleaning up the mess?

I know Debian's got it right, I think Redhat does too.  SuSE is having
problems, but will hopefully straighten out.  I don't know about the
others.

zw

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

> > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> > the weak references can stay, and executables avoid pointless bloat.

David O'Brien wrote:
> ALL THE WORLD IS NOT LINUX!

We were discussing a problem that only appeared in Linux distributions.
The suggestion to upgrade Glibc was meant only for Linux users.

Sorry you got the wrong end of that stick.

Anyway, the discussion is over now.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Is is possible to include a stub weak *definition* of
__register_frame_info in crtbeginS.o, which is nothing more than a
return instruction?

Would that solve these temporary binary compatibility problems, without
the permanent bloat?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Sat, 10 Apr 1999 02:28:42 -0600, Jeffrey A Law wrote:
>
>  > * GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
>  >   reference __register_frame_info [1].
>Incorrect.  If you compile glibc with egcs-1.1, then the resulting library
>will have __register_frame_info.  This happens because the C startup files
>have a reference to __register_frame_info, which in turn cause
>__register_frame_info to be brought in from libgcc::frame.o.
>
>Furthermore, the definition in the library will have external visibility.

This is true or not true depending on which point revision of glibc
2.0 you have.  Patches were added somewhere in there which prevent
libc.so from exporting __register_frame_info - it uses its own private
crtbeginS.o (effectively) which has the weak symbol refs.

Unfortunately, that just means the externally visible definition of
__register_frame_info moves over to libm.so.

To make matters worse, I think this particular patch was added to libc
CVS after Ulrich stopped making official point releases.  That means
Redhat and Debian have it and no one else does.

>  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked
>  >   only against libc contain and reference __register_frame_info.
>Depends on whether or not they link against any libraries which provide
>register_frame_info.

He said `only against libc'.  We always do -lgcc -lc -lgcc, so the
above assertion is correct.

>  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked against an
>  >   other shared library besides libc do not contain
>  >   __register_frame_info but do reference __register_frame_info.
>Incorrect.  It depends on what is in the libraries that the program is
>linked again.

In practice, this assertion is correct if and only if the `other
shared library' was compiled with egcs with a version number in
[1.0, 1.1.1] or with gcc 2.8.

>  > Now there are a few possibilities where programs will not run:
>I've already explained the cases where program will stop running.
>
>If you have a program which is linked against a shared library and the
>shared library originally contained the frame functions, then that program
>will expect to get the frame functions from the library and will not have
>its own copies.
Yes.

>If you then recompile that library with egcs-1.1.2, then that library may
>not get a copy of the frame functions if the only reference to them is
>the weak symbol in the crt*.o files.
Yes.

>Now the progrm which was originally linked against that library will fail
>because there is no definition of the frame functions.
Yes.

>This problem can be solved by making the reference to the frame function
>in the crt files a normal (non-weak) reference again.  This will not break
>existing code.

This is technically true, however you will break people's
expectations.  Libc has been going around telling everyone that they
need to compile their libraries with gcc 2.7 or else apply the
controversial patch to egcs.  Oh, and the patch is official in egcs
1.1.2.  From libc's point of view, any library that was not compiled
as above is broken; it fails to conform to the de facto ABI.

zw

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Thu, Apr 08, 1999 at 10:24:18PM +0200, Jamie Lokier wrote:
> [BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]
> 
> > No, this is an egcs problem.  We can't have these kinds of problems show
> > up when folks rebuild their libraries with newer versions of the compiler.
> > 
> > Can I be any clearer?
> 
> The policy is pretty clear.
> 
> The danger is that we get locked in to including bloat in all binaries,
> that can't ever be removed.  I'm interested in a solution to that long
> term problem.  If we have to include the bloat in the interim to provide
> binary compatibility, that is fine.

It's better to have bloated libraries than broken libraries. ;-) Binary
compatibility issues affect thousands of projects and millions of users.

I build binary packages for a number of projects, and maintain Linux
systems for various people. My life is easiest when the C compiler and libc
adhere to well-defined rules. When things get confusing, I make mistakes
and cause trouble for downstream users. Other people make mistakes, too,
causing me grief. (Right now, both my desktop and latest Mozilla build are
broken because somebody at RHAD labs thought it would be fun to compile
glib and gtk with egcs. I'll fix this when I have nothing better to do.)

Now, whether egcs requires updates to libc is your business, not mine.
What *is* essential, though, is that all of us downstream packagers and
users understand the rules. Otherwise, we'll screw up and make a mess.

So please keep the rules as simple as possible, and document all breakages
carefully. Legacy installations are a pain in the neck, but they're also a
fact of life.

Thank you for letting me get this off my chest. :-)

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On 07 Apr 1999 10:33:55 +0200, Andreas Schwab wrote:
>Zack Weinberg <zack@rabi.columbia.edu> writes:
>
>|> Using an explicit -lc is always recommended, but it has nothing to do
>|> with the above - it just ensures that the dynamic linker has a full
>|> dependency graph for the shared libraries.
>
>Note that you don't have to specify -lc explicitly, the gcc driver already
>does this by default, even with -shared, since gcc 2.8.1.

Not on all platforms - possibly only Linux.

zw

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < XFMail.990408145548.jdp@polstra.com >you write:
  > It seems like this is an instance of a more general problem that often
  > crops up when you try to mix archive libraries with shared libraries.
  > If libgcc were shared, then this problem wouldn't exist ... right?
Yes.  But there's a whole different can of worms if you make libgcc
a shared library :-)


jeff

Re: __register_frame_info & shared library compatibility

[John Polstra]

Jeffrey A Law wrote:
>   > I don't understand why recompiling a shared library with the
>   > weak-referencing version of egcs would cause programs linked with it
>   > to stop working.
...
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Thanks for the explanation.  That clears it up.

It seems like this is an instance of a more general problem that often
crops up when you try to mix archive libraries with shared libraries.
If libgcc were shared, then this problem wouldn't exist ... right?

John
---
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Wed, 07 Apr 1999 02:57:47 -0600, Jeffrey A Law wrote:
>
>  In message < 199904070035.UAA17069@blastula.phys.columbia.edu >you write:
>  > That's strange.  That patch should theoretically make shared libs
>  > compiled by egcs act like shared libs compiled by gcc.
>But that's not necessarily good!  gcc2's handling of shared libraries is
>bad in a variety of ways.  Most notably the lack of backwards compatible
>EH interfaces

`act like' in the sense of `compatible ABI'.  The EH support should
still be present, just hiding.

>  > The correct binary interface for libc 2.0 is the one you get if you
>  > compile all purely C libraries with gcc 2.7.  Programs that expect
>  > something else are broken, sorry.  (There's no reason to prefer this
>  > rule except that that's what all the major distributions shipped.)
>No, the problem was soemone recompiled a library previously compiled with
>egcs-1.1.1 with egcs-1.1.2 and applications started breaking due to the
>weak handling.  At least that was my impression.  And that's horrible.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408222418.C21807@pcep-jamie.cern.ch >you write:
  > The danger is that we get locked in to including bloat in all binaries,
  > that can't ever be removed.  I'm interested in a solution to that long
  > term problem.  If we have to include the bloat in the interim to provide
  > binary compatibility, that is fine.
We already know at some point we will have to break binary compatibility in
the future to get some new C++ features.  That would be the time to drop the
old interfaces.

But in general, if an interface is added with external scope in libgcc, then it
has to stay indefinitely (until the next planned binary breakage).  Thus we
have to be a *LOT* more careful about what interfaces we put in that library --
we don't want to plan many binary breakages.


  > Though frankly, the bloat due to the redundant .eh_frame in C binaries
  > much more significant.  Is throwing an exception through C really
  > supposed to have defined behaviour?
?!?  I don't think we create eh_frames for C code by default now.  We do
if one specifies -fexceptions though.

jeff

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   Date: Sat, 10 Apr 1999 02:28:42 -0600
   From: Jeffrey A Law <law@upchuck.cygnus.com>

     > * GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
     >   reference __register_frame_info [1].
   Incorrect.  If you compile glibc with egcs-1.1, then the resulting library
   will have __register_frame_info.  This happens because the C startup files
   have a reference to __register_frame_info, which in turn cause
   __register_frame_info to be brought in from libgcc::frame.o.

GNU libc has its own versions of these C startup files, and I thought
they had been changed to do something similar as the weak
__register_frame_info patch in egcs-1.1.2, but it turns out that this
is not true.  So you're right.

   If you then recompile that library with egcs-1.1.2, then that library may
   not get a copy of the frame functions if the only reference to them is
   the weak symbol in the crt*.o files.

Therefore the crt*.o files used in building shared libraries must have
normal (non-weak) references to __register_frame_info.

   Now the progrm which was originally linked against that library will fail
   because there is no definition of the frame functions.

Except when the crt*.o files used in building the program had only
weak references to __register_frame_info.  That's why it is a good
idea to do this if the target has two sets of crt*.o files, as Linux does.

   This problem can be solved by making the reference to the frame function
   in the crt files a normal (non-weak) reference again.  This will not break
   existing code.

But it doesn't solve the problems with a program built with egcs-1.1,
linked against shared libraries built with egcs-1.1, running on a
system where the shared libraries were built with gcc 2.7.  Because in
that case both the program and the shared library do not contain
__register_frame_info but it is refernced by the program.

     >    I don't believe there is a right answer.
   The right answer is to put the normal reference back.  It will not break
   programs, just make them a little larger.  That was the only reason those
   symbosl were make weak to begin with -- to reduce the size of programs
   which did not actually use exception handling.

We both know that the original submitter of the patch is not very
good at explaining the exact technical reasons why his patches should
go in, but I remember him writing that "it fixes a bug".  I would not
call generating slightly bigger binaries as bug, so I am under the
impression that he was trying to solve the problem illustrated in the
previous paragraph.  And I know for sure
that some of people that advocated the patch were trying to solve that
problem. That was the case Zack was mentioning a few messages back.

Mark

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKvO-000ErNC@ocean.lucon.org >you write:
  > Exactly. That is a glibc problem. not an egcs problem. I don't think
  > egcs should get involved with it. Maybe we should put it in the glibc
  > FAQ.
No, this is an egcs problem.  We can't have these kinds of problems show
up when folks rebuild their libraries with newer versions of the compiler.

Can I be any clearer?

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKyt-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Is is possible to include a stub weak *definition* of
  > > __register_frame_info in crtbeginS.o, which is nothing more than a
  > > return instruction?
  > > 
  > > Would that solve these temporary binary compatibility problems, without
  > > the permanent bloat?
  > > 
  > 
  > That means we have to do more testing without much benefit. I don't
  > think it is wise for egcs people to spend any more time on this. It is
  > a glibc problem and let glibc people deal with it. If some glibc users
  > won't listen to the glibc people, they get what they deserved.
It's not just a glibc problem.

Consider a system where all the system libraries were compiled with gcc-2.7
(like an older RedHat distribution).

A developer later installed egcs-1.1.1 and compiled some package that
installed some new shared libraries (let's say X11 libraries).    Then he
built a bunch of apps that link against those libraries (lots of X11 clients)

Later that developer installs egcs-1.1.2 and rebuilds the shared libraries.
He should be able to reasonably assume that the clients would still work.  But
because of your incompatible change they will not.



jeff

__register_frame_info & shared library compatibility

[Mark Kettenis]

   From: Jeffrey A Law (law@cygnus.com)
   Date: Tue, 13 Apr 1999 21:57:56 -0600
 
     > Please note the wording: "upgrade to a glibc that was not built with
     > egcs".  As long as the official libc binaries shipped by the
     > distributions are built by gcc 2.7, egcs MUST match gcc 2.7's ABI for
     > C programs.  That is what the patch achieves.  And we told the
     > distributions to use gcc 2.7 until they switch to glibc 2.1, so this
     > will continue to be a problem.
   OK.  This kind of compatibility is not a priority and never has been.  We've
   always known this would not work.

I don't think there are many people that are aware of this.

   If you can find a solution that works in both worlds, then great.
   But I do not believe one exists.  And in the absence of a perfect
   solution we must continue be compatible with our own releases over
   compatibility with gcc-2.7.

What about my suggestion to use weak references to
__register_frame_info and __deregister_frame_info in crtbegin.o and
normal (non-weak) references in crtbeginS.o?

   At this point I'm still planning to remove the patch and make an
   egcs-1.1.3 release.

Unless you think the solution suggested above is viable, and we can
test it on short term, that's indeed the only right solution (sorry Zack).

Mark

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> > Previously the library would always have those routines, even if it did
> > not use EH.
> > 
> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 

Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
ftp site with tar files.


H.J.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904070035.UAA17069@blastula.phys.columbia.edu >you write:
  > That's strange.  That patch should theoretically make shared libs
  > compiled by egcs act like shared libs compiled by gcc.
But that's not necessarily good!  gcc2's handling of shared libraries is
bad in a variety of ways.  Most notably the lack of backwards compatible
EH interfaces

  > The correct binary interface for libc 2.0 is the one you get if you
  > compile all purely C libraries with gcc 2.7.  Programs that expect
  > something else are broken, sorry.  (There's no reason to prefer this
  > rule except that that's what all the major distributions shipped.)
No, the problem was soemone recompiled a library previously compiled with
egcs-1.1.1 with egcs-1.1.2 and applications started breaking due to the
weak handling.  At least that was my impression.  And that's horrible.

jeff

Re: __register_frame_info & shared library compatibility

[Philipp Thomas]

On Thu, 08 Apr 1999 14:55:48 -0700 (PDT), you wrote:

>If libgcc were shared, then this problem wouldn't exist ... right?

Yea, but you would trade one problem with a whole bunch of new ones a
shared libgcc would introduce as the discussion some time ago showed.

Philipp Thomas

-- 
caffeine low .... brain halted

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 6 Apr 1999 18:41:17 -0400, Eric Kidd wrote:
>On Tue, Apr 06, 1999 at 06:15:49PM -0400, Zack Weinberg wrote:
>> Make sure you have a current binutils (2.9.1.0.2x).  You may want to
>> upgrade to kernel 2.2; glibc 2.1 wants to use the new features (it does
>> fall back gracefully) and you won't have to keep gcc 2.7 around.
>
>I help maintain a compiler that's hard to bootstrap. Currently, I must be
>able to compile glibc-2.0/gcc-2.7 binaries on my system. So my upgrade
>options are very limited. :-(

Can you give details?  Maybe I can help disentangle it.

Unfortunately, 2.0 and 2.1 do not coexist well.  You would almost need
a complete chroot environment to retain the ability to build 2.0
binaries.  Better use two machines...

>> Any library that uses stdio internally may need to be recompiled in
>> order to work with binaries built with glibc 2.1.  libstdc++,
>> libncurses, libslang, and libglib are known to need this.
>
>Oooh. We've got a library which uses stdio in a big way. What qualifies as
>"using stdio internally"? Where can I find documentation on this, so I
>don't screw over our users?

You will definitely lose if you have any references to stdin/out/err
inside the library.  If you don't, you might survive, but you might
not.  Best advice is `try it and see'.

Executables are unaffected - a program that uses your library does not
need to be recompiled to migrate to 2.1, although the library may.

>> glibc 2.1 can't be compiled with gcc 2.7 (it miscompiles the math
>> library), so we make you use egcs, so __register_frame_info will
>> always be present in libc.so.6.  2.0 could be compiled with gcc 2.7,
>> and most distributions shipped it that way.
>
>Let me try to summarize the rules in a nutshell:
>
> * glibc 2.1 is always compiled with egcs

Technically you can use gcc 2.8 too.  I don't recommend it.

> * glibc 2.0 can be compiled with egcs, but is normally compiled with gcc

Correct.

> * If you compile glibc with gcc, you must use -lc when compiling shared
>   libraries with egcs. Otherwise, gcc-compiled applications will die
>   horrible, screaming deaths.

If you compile glibc 2.0 with gcc, you must also compile all other
libraries with gcc.  Otherwise, egcs-compiled applications may die
horribly.  It probably does not matter which compiler you use to
compile other libraries on a 2.1 system, but I would use egcs just to
be safe.

Using an explicit -lc is always recommended, but it has nothing to do
with the above - it just ensures that the dynamic linker has a full
dependency graph for the shared libraries.  If you have two shared
libraries and libA uses functions in libB, link libA with -lB too.

> * If you compile glibc with egcs, you can often use egcs-compiled shared
>   libraries with gcc-compiled applications.

Yes - and if it doesn't work, that's a bug.

zw

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

[BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]

> No, this is an egcs problem.  We can't have these kinds of problems show
> up when folks rebuild their libraries with newer versions of the compiler.
> 
> Can I be any clearer?

The policy is pretty clear.

The danger is that we get locked in to including bloat in all binaries,
that can't ever be removed.  I'm interested in a solution to that long
term problem.  If we have to include the bloat in the interim to provide
binary compatibility, that is fine.

Though frankly, the bloat due to the redundant .eh_frame in C binaries
much more significant.  Is throwing an exception through C really
supposed to have defined behaviour?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904121315.JAA13984@blastula.phys.columbia.edu >you write:
  > >True.  But we've made not attempt to make that setup work and have no
  > >intention of making that setup work.
  > 
  > You may not have, but Mark, HJ, Ulrich and I spent about a month
  > trying to fix exactly this setup.  The conclusion was that the patch
  > you don't like was the best solution.
The egcs project has never considered this kind of compatibility with
gcc-2.7 compiled binaries a priority.

Giving up compatibility with egcs itself to achieve compatibility with
an outdated, pre-EH compiler from the FSF is not acceptable.  Sorry.

  > 
  > >But it is unacceptable when a library that was originally compiled
  > >with egcs-1.1.1, after compiling with egcs-1.1.2 to cause libraries
  > >to stop working.  Plain and simple, it is unacceptable.
  > 
  > The library should not have been compiled with egcs 1.1.1 in the first
  > place.
  > 
  > Yes, I realize this is an unrealistic attitude.  Would you at least
  > consider an -mcompat-1.0 switch?
The default must be to be compatible with prior egcs releases.  I'd much
prefer not add a switch to enable the losing behavior because it just adds
to the long term maintenance and confusion.


  > # When I compile glibc 2.0.{6,7} with egcs, glibc ends up defining the
  > # symbols __register_frame_info & __deregister_frame_info. I noticed
  > # that these are also defined in egcs' libgcc.a Should this be
  > # happening?
  > #
  > # The real problem happens when I upgrade to a glibc that was not
  > # built with egcs.  Some of the programs I compiled with egcs under a
  > # egcs built glibc also have a dependency on these symbols.  If it
  > # wasn't /bin/login, I might not have cared so much. :-/
  > 
  > Please note the wording: "upgrade to a glibc that was not built with
  > egcs".  As long as the official libc binaries shipped by the
  > distributions are built by gcc 2.7, egcs MUST match gcc 2.7's ABI for
  > C programs.  That is what the patch achieves.  And we told the
  > distributions to use gcc 2.7 until they switch to glibc 2.1, so this
  > will continue to be a problem.
OK.  This kind of compatibility is not a priority and never has been.  We've
always known this would not work.

If you can find a solution that works in both worlds, then great.  But I do
not believe one exists.  And in the absence of a perfect solution we must
continue be compatible with our own releases over compatibility with gcc-2.7.

At this point I'm still planning to remove the patch and make an egcs-1.1.3
release.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904062115.RAA15222@blastula.phys.columbia.edu >you write:
  > The best suggestion I can make is:  Get glibc 2.1 and compile it with
  > egcs.  When you do that, glibc absorbs and re-exports the runtime
  > functions that are causing the problem.  Everything else will pick
  > them up from there instead of libgcc.a.  You'll have to recompile any
  > libraries which are currently exporting __register_frame_info.  You
  > will not be able to run binaries built in this environment on
  > non-glibc-2.1 systems.
  > 
  > You can not do this with glibc 2.0 because then you lose binary
  > compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
  > as you describe).
This points out the terrible problem with the patch to make the *_frame_info
functions weakly referenced in crtstuff.

We're already getting reports of programs that no longer run because of this
change.  We may have to revert it.



jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
>   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
>   > ftp site with tar files.
> What am I not being clear about?
> 
> This kind of binary breakage is not acceptable.  It doesn't matter if you
> put a new glibc somewhere.  Breaking binaries like that patch did is not
> acceptable.

The patch does *not break binaries*.

Installing EGCS and using it to compile new applications does
*not break binaries*.

But recompiling some shared libraries using EGCS with the patch breaks some
binaries.  Is it unreasonable to request a user upgrades to the latest
Glibc 2.0, or any Glibc 2.1 before doing this?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 06 Apr 1999 17:46:23 -0600, Jeffrey A Law wrote:
>
>  In message < 199904062115.RAA15222@blastula.phys.columbia.edu >you write:
>  > The best suggestion I can make is:  Get glibc 2.1 and compile it with
>  > egcs.  When you do that, glibc absorbs and re-exports the runtime
>  > functions that are causing the problem.  Everything else will pick
>  > them up from there instead of libgcc.a.  You'll have to recompile any
>  > libraries which are currently exporting __register_frame_info.  You
>  > will not be able to run binaries built in this environment on
>  > non-glibc-2.1 systems.
>  > 
>  > You can not do this with glibc 2.0 because then you lose binary
>  > compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
>  > as you describe).
>This points out the terrible problem with the patch to make the *_frame_info
>functions weakly referenced in crtstuff.
>
>We're already getting reports of programs that no longer run because of this
>change.  We may have to revert it.

That's strange.  That patch should theoretically make shared libs
compiled by egcs act like shared libs compiled by gcc.

The correct binary interface for libc 2.0 is the one you get if you
compile all purely C libraries with gcc 2.7.  Programs that expect
something else are broken, sorry.  (There's no reason to prefer this
rule except that that's what all the major distributions shipped.)

zw

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   Date: Wed, 14 Apr 1999 15:09:36 -0600
   From: Jeffrey A Law <law@upchuck.cygnus.com>

     In message < 199904141258.OAA22629@landau.wins.uva.nl >you write:
     >    If you can find a solution that works in both worlds, then great.
     >    But I do not believe one exists.  And in the absence of a perfect
     >    solution we must continue be compatible with our own releases over
     >    compatibility with gcc-2.7.
     > 
     > What about my suggestion to use weak references to
     > __register_frame_info and __deregister_frame_info in crtbegin.o and
     > normal (non-weak) references in crtbeginS.o?

   I must have glossed over it.  If you've got it handy, can you
   resend (or I'll grope around the archives for it).

I don't have a patch at the moment.  It is just an idea that I wanted
to check with a few people before checking out how to implement it in
a clean way.

The idea is to use the fact that for a number of ELF targets there are
two crtbegin files.  crtbegin.o is used for normal programs and
crtbeginS.o is copiled with -fPIC for use in shared libraries.  The
idea is that as long as we keep the normal (non-weak) references to
__register_frame_info and __deregister_frame_info in crtbeginS.o the
same shared libraries end up with __register_frame_info and
__deregister_frame_info as with egcs-1.1.1.  But I think we are free
to make the references in crtbegin.o weak.  This would mean that
programs that don't explicitly use exception handling would run fine
if these symbols are not available.

What I'd like to know is if it is acceptable to make crtbegin.o and
crtbeginS.o different in this respect, and if you can shoot any holes
in my little theory.

Note that this solution isn't perfect.  Not all targets seem to have
seperate crtbegin.o and crtbeginS.o files.  And it is still not
possible to arbitrarily exchange shared libraries between systems with
an egcs compiled glibc 2.0 and an gcc 2.7 compiled glibc 2.0.  But I
think it is better than the egcs 1.1.1 situation.

Mark

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> We were discussing a problem that only appeared in Linux distributions.
> The suggestion to upgrade Glibc was meant only for Linux users.

Postings from John Polstra <jdp@polstra.com> *were* about problems with
__register_frame_info on FreeBSD.

I will drop it now, but I just wanted others to remember that as Jeff Law
said, interface changes to libgcc affect everyone.
 
-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  > GNU libc has its own versions of these C startup files, and I thought
  > they had been changed to do something similar as the weak
  > __register_frame_info patch in egcs-1.1.2, but it turns out that this
  > is not true.  So you're right.
Only in releases most people do not have and are not using.

  >    This problem can be solved by making the reference to the frame function
  >    in the crt files a normal (non-weak) reference again.  This will not bre
  > ak
  >    existing code.
  > 
  > But it doesn't solve the problems with a program built with egcs-1.1,
  > linked against shared libraries built with egcs-1.1, running on a
  > system where the shared libraries were built with gcc 2.7.  Because in
  > that case both the program and the shared library do not contain
  > __register_frame_info but it is refernced by the program.
True.  But we've made not attempt to make that setup work and have no intention
of making that setup work.  But it is unacceptable when a library that was
originally compiled with egcs-1.1.1, after compiling with egcs-1.1.2 to cause
libraries to stop working.  Plain and simple, it is unacceptable.


  > We both know that the original submitter of the patch is not very
  > good at explaining the exact technical reasons why his patches should
  > go in, but I remember him writing that "it fixes a bug".  I would not
  > call generating slightly bigger binaries as bug, so I am under the
  > impression that he was trying to solve the problem illustrated in the
  > previous paragraph.  And I know for sure
  > that some of people that advocated the patch were trying to solve that
  > problem. That was the case Zack was mentioning a few messages back.
Then I would strongly recommend that HJ explain in painful detail why this
fixed a bug.  As soon as I can find a few free minutes I'm planning to revert
the busted patch (both in the mainline tree and in the egcs-1.1.x release
tree).

jeff

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 13 Apr 1999 21:57:56 -0600, Jeffrey A Law wrote:
>
>  In message < 199904121315.JAA13984@blastula.phys.columbia.edu >you write:
>  > >True.  But we've made not attempt to make that setup work and have no
>  > >intention of making that setup work.
>  > 
>  > You may not have, but Mark, HJ, Ulrich and I spent about a month
>  > trying to fix exactly this setup.  The conclusion was that the patch
>  > you don't like was the best solution.
>The egcs project has never considered this kind of compatibility with
>gcc-2.7 compiled binaries a priority.
>
>Giving up compatibility with egcs itself to achieve compatibility with
>an outdated, pre-EH compiler from the FSF is not acceptable.  Sorry.

I've been arguing entirely from the point of view of the libc
project.  And there, `compatibility with an outdated, pre-EH compiler
from the FSF' is critical, because like it or not, >90% of Linux
installations are still using gcc 2.7.  Furthermore, this is no small
bug; if you install a libc compiled by gcc 2.7 over a system compiled
by egcs 1.0/1.1.[01], you hose the entire system.

Libc is in a somewhat unique position, in that we don't have to worry
about intersystem portability unless you count Hurd, but we do have to
make it work for all combinations of compiler, kernel, and source
package - even the ones you don't care to support.

I am not saying that egcs should necessarily cater to libc, but I
think you may be underestimating the magnitude of the problem we face
if you take the patch back out.

zw

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Tue, Apr 06, 1999 at 05:15:31PM -0400, Zack Weinberg wrote:
> This is a well known problem to which there is, unfortunately, no
> clean solution.

I was afraid of this. :-( Unless the distribution vendors are very careful,
migrating from gcc to egcs will cause as many problems as the libc5 to
glibc2 upgrade. Ugh.
 
> The best suggestion I can make is: Get glibc 2.1 and compile it with
> egcs.  When you do that, glibc absorbs and re-exports the runtime
> functions that are causing the problem.  Everything else will pick them
> up from there instead of libgcc.a.  You'll have to recompile any
> libraries which are currently exporting __register_frame_info.  You will
> not be able to run binaries built in this environment on non-glibc-2.1
> systems.

OK, this is starting to make sense. If __register_frame_info gets linked
into glibc-2.1, then any library linked against glibc will pick up the new
runtime functions? This seems like a fairly safe workaround, assuming that
everything using shared libraries links dynamically against glibc.

Does glibc 2.1 provide the soname libc.so.6? Does your solution somehow
rely on this fact, or are you advising me to break my system into two
parts: glibc 2.0 with gcc, and glibc 2.1 with egcs?

How does this solution affect pre-existing applications that were compiled
with egcs and linked against a glibc 2.0 that was compiled with gcc?

> You can not do this with glibc 2.0 because then you lose binary
> compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
> as you describe).

I still don't understand something. Why should linking
__register_frame_info into glibc 2.1 solve the problem, but linking it into
glibc 2.0 *cause* the problem?

Is there any hope that Red Hat and other distribution vendors will get this
right, or will I have to spend a week or two cleaning up the mess?

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Tue, Apr 06, 1999 at 06:15:49PM -0400, Zack Weinberg wrote:
> Make sure you have a current binutils (2.9.1.0.2x).  You may want to
> upgrade to kernel 2.2; glibc 2.1 wants to use the new features (it does
> fall back gracefully) and you won't have to keep gcc 2.7 around.

I help maintain a compiler that's hard to bootstrap. Currently, I must be
able to compile glibc-2.0/gcc-2.7 binaries on my system. So my upgrade
options are very limited. :-(
 
> Any library that uses stdio internally may need to be recompiled in
> order to work with binaries built with glibc 2.1.  libstdc++,
> libncurses, libslang, and libglib are known to need this.

Oooh. We've got a library which uses stdio in a big way. What qualifies as
"using stdio internally"? Where can I find documentation on this, so I
don't screw over our users?
 
> You can't use binaries or libraries compiled against 2.1 on a system that
> has libc 2.0.  Hopefully everyone will be using 2.1 soon and this won't
> be a problem.

OK, this is a reasonably normal (and expected) constraint. Understood.

> glibc 2.1 can't be compiled with gcc 2.7 (it miscompiles the math
> library), so we make you use egcs, so __register_frame_info will
> always be present in libc.so.6.  2.0 could be compiled with gcc 2.7,
> and most distributions shipped it that way.

Let me try to summarize the rules in a nutshell:

 * glibc 2.1 is always compiled with egcs
 * glibc 2.0 can be compiled with egcs, but is normally compiled with gcc
 * If you compile glibc with gcc, you must use -lc when compiling shared
   libraries with egcs. Otherwise, gcc-compiled applications will die
   horrible, screaming deaths.
 * If you compile glibc with egcs, you can often use egcs-compiled shared
   libraries with gcc-compiled applications.

Am I getting close?

> >Is there any hope that Red Hat and other distribution vendors will get
> >this right, or will I have to spend a week or two cleaning up the mess?
> 
> I know Debian's got it right, I think Redhat does too.  SuSE is having
> problems, but will hopefully straighten out.  I don't know about the
> others.

I really, really hope that Red Hat doesn't screw this up. The most recent
Gnome RPMS out of RHAD Labs have lots of issues in this area.

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[alex.buell]

On Wed, 7 Apr 1999, Zack Weinberg wrote:

> I don't believe there is a right answer.  Mark understands the issues
> better, maybe he has one.

Jump to glibc-2.1. I think this is what saved my butt when moving from
egcs-1.0.3 through to egcs-1.1.2. I have built lots of libraries under
egcs-1.1.1 that didn't break when I moved over to egcs-1.1.2, so whoever
wrote glibc-2.1 deserves a medal. 
 
Cheers,
Alex
--
 /_/\  Legalise cannabis now! 
( o.o ) Grow some cannabis today!
 > ^ <  Peace, Love, Unity and Respect to all.

http://www.tahallah.demon.co.uk - Now back in the United Kingdom!

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904212033.WAA27543@quatramaran.ens.fr >you write:
  > Now that OpenBSD 2.5 is out the door, I am starting the process of trying
  > to switch. i386 going well so far, I expect a few ICEs from m68k.
Yea.  However, this kind of m68k testing would be _greatly_ appreciated.  I do
builds on m68ks once in a while, and build crosses regularly to test bugs.

But it's rare for me to build any serious code other than the compiler
itself on m68ks -- too slow and I had to return my m68k hp to the UofU.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Presumably this is all solved by using Glibc 2.1?

In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
the weak references can stay, and executables avoid pointless bloat.

Long term I would like to see the weak references long term because
anything else is a waste of space.  Short term, if there's a migration
path where the weak references patch is omitted for a while until
"everyone" has upgraded to Glibc 2.1, that would be fine.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408214924.A21750@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up 
  > a
  > >   > ftp site with tar files.
  > > What am I not being clear about?
  > > 
  > > This kind of binary breakage is not acceptable.  It doesn't matter if you
  > > put a new glibc somewhere.  Breaking binaries like that patch did is not
  > > acceptable.
  > 
  > The patch does *not break binaries*.
It does if one rebuilds shared libraries after installing that patch.  Existing
binaries may no longer run.

  > Installing EGCS and using it to compile new applications does
  > *not break binaries*.
Agreed.


  > But recompiling some shared libraries using EGCS with the patch breaks some
  > binaries.  Is it unreasonable to request a user upgrades to the latest
  > Glibc 2.0, or any Glibc 2.1 before doing this?
Yes, it is unreasonable.    This kind of breakage is not acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408223252.A21853@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Later that developer installs egcs-1.1.2 and rebuilds the shared
  > > libraries.  He should be able to reasonably assume that the clients
  > > would still work.  But because of your incompatible change they will
  > > not.
  > 
  > The error on the part of the developer was to install egcs-1.1.2 without
  > upgrading glibc.
Installing egcs-1.1.2 should never require updating the system library. 
Policy, not subject for debate.

  > Could we not guard against that error, by failing at link time against
  > the older glibc?
No.  egcs-1.1.2 has to work with older libcs.

jeff

Re: __register_frame_info & shared library compatibility

[Joe Buck]

> > We were discussing a problem that only appeared in Linux distributions.
> > The suggestion to upgrade Glibc was meant only for Linux users.
> 
> Postings from John Polstra <jdp@polstra.com> *were* about problems with
> __register_frame_info on FreeBSD.

Whoops (removes foot from mouth :-).  I was thinking of another part of
this thread which was glibc-specific.

Yet another reason why we need closer cooperation with the BSD folks
(so we find out about these things sooner).

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904140616.CAA07720@blastula.phys.columbia.edu >you write:
  > I've been arguing entirely from the point of view of the libc
  > project.  And there, `compatibility with an outdated, pre-EH compiler
  > from the FSF' is critical, because like it or not, >90% of Linux
  > installations are still using gcc 2.7.  Furthermore, this is no small
  > bug; if you install a libc compiled by gcc 2.7 over a system compiled
  > by egcs 1.0/1.1.[01], you hose the entire system.
I understand that.  But how often do people overwrite a library with one
built with an _older_ compiler?  I don't think we should cater to this
case.  And certainly not at the expense of breaking the ability to install
a library compiled by a newer compiler!


  > I am not saying that egcs should necessarily cater to libc, but I
  > think you may be underestimating the magnitude of the problem we face
  > if you take the patch back out.
Possibly.  In the same light, you may be underestimating the problems the
patch is causing and will cause in the future.  And it's simply absurd to
have these binary incompatibilities between egcs releases.

jeff

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Is is possible to include a stub weak *definition* of
> __register_frame_info in crtbeginS.o, which is nothing more than a
> return instruction?
> 
> Would that solve these temporary binary compatibility problems, without
> the permanent bloat?
> 

That means we have to do more testing without much benefit. I don't
think it is wise for egcs people to spend any more time on this. It is
a glibc problem and let glibc people deal with it. If some glibc users
won't listen to the glibc people, they get what they deserved.


-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[John Polstra]

In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote:

>   > The problem is indeed that egcs 1.1.2 now puts only a weak
>   > reference to "__register_frame_info" in crtstuff.  This means
>   > that "du moment" one recompiles a shared library that previously
>   > contained "__register_frame_info" it probably does not anymore,
>   > and programs linked with that library stop running.
> Yup.  And I consider this a horrible breakage.  I trusted a couple
> of folks with more experience in this area to guide the decision to
> include that patch.  They made a mistake.  We need to rectify it.

I don't understand why recompiling a shared library with the
weak-referencing version of egcs would cause programs linked with it
to stop working.  The library either uses exceptions, or it doesn't.
If it uses exceptions, it contains calls to __throw and/or other
functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
contains a strong reference to __frame_state_for. __frame_state_for
is defined in frame.o, so that will also be pulled in from libgcc.a.
Presto, the library has __register_frame_info just the same as always.

If the library doesn't use exceptions, then it won't pull in frame.o
and it won't have __register_frame_info.  That won't matter for the
library itself, because of the weak references in crtbeginS.o.  Now,
the executable and any other shared libraries that use exceptions
(if any) will each contain __register_frame_info, and the weak
references to __register_frame_info from the shared library in
question will presumably be resolved by the dynamic linker to the
first of those copies.  So the library will be calling a copy of
__register_frame_info that resides in the executable or in a different
shared library.  Is that a problem?  I.e., does each shared library
need to have its own copy of __register_frame_info and the associated
static variables?

My usual context is FreeBSD rather than Linux, so maybe I'm missing
something fundamental here.  If so, please enlighten me, as we are
working on switching to egcs-1.1.2 in the FreeBSD project as we speak.

John
-- 
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Marc Espie]

In article < 199904212017.NAA20578@atrus.synopsys.com > you write:
>
>Yet another reason why we need closer cooperation with the BSD folks
>(so we find out about these things sooner).

Now that OpenBSD 2.5 is out the door, I am starting the process of trying
to switch. i386 going well so far, I expect a few ICEs from m68k.

Don't worry, you'll get *very* close cooperation from us :)

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 
> In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> the weak references can stay, and executables avoid pointless bloat.

I know I'm a little bit late on this discussion, but I would really like
to point out to most of the list members one of Henry Spencer's "Ten
Commandments for C Programmers" :

X   Thou shalt foreswear, renounce, and abjure the vile heresy which
    claimeth that ``All the world's a VAX'', and have no commerce with
    the benighted heathens who cling to this barbarous belief, that
    the days of thy program may be long even though the days of thy
    current machine be short.

        This particular heresy bids fair to be replaced by ``All the
        world's a Sun'' or ``All the world's a 386'' (this latter
..snip..


I other words:  ALL THE WORLD IS NOT LINUX!

FreeBSD has moved to using EGCS 1.1.2 as the base compiler.  And I don't
just mean for userland, it is our kernel compiler too.  We have no other
C/C++ compilers in the base system.  NetBSD has moved some of their
architectures to this point, and OpenBSD has plans to.

So Linux-centric hacks hacks that bring havoc into the world are NOT
appropriate.

-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
> >   > ftp site with tar files.
> > What am I not being clear about?
> > 
> > This kind of binary breakage is not acceptable.  It doesn't matter if you
> > put a new glibc somewhere.  Breaking binaries like that patch did is not
> > acceptable.
> 
> The patch does *not break binaries*.
> 
> Installing EGCS and using it to compile new applications does
> *not break binaries*.
> 
> But recompiling some shared libraries using EGCS with the patch breaks some
> binaries.  Is it unreasonable to request a user upgrades to the latest
> Glibc 2.0, or any Glibc 2.1 before doing this?
> 

Exactly. That is a glibc problem. not an egcs problem. I don't think
egcs should get involved with it. Maybe we should put it in the glibc
FAQ.

-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[Andreas Schwab]

Zack Weinberg <zack@rabi.columbia.edu> writes:

|> Using an explicit -lc is always recommended, but it has nothing to do
|> with the above - it just ensures that the dynamic linker has a full
|> dependency graph for the shared libraries.

Note that you don't have to specify -lc explicitly, the gcc driver already
does this by default, even with -shared, since gcc 2.8.1.

-- 
Andreas Schwab                                      "And now for something
schwab@issan.cs.uni-dortmund.de                      completely different"
schwab@gnu.org

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904212033.WAA27543@quatramaran.ens.fr >you write:
  > Now that OpenBSD 2.5 is out the door, I am starting the process of trying
  > to switch. i386 going well so far, I expect a few ICEs from m68k.
Yea.  However, this kind of m68k testing would be _greatly_ appreciated.  I do
builds on m68ks once in a while, and build crosses regularly to test bugs.

But it's rare for me to build any serious code other than the compiler
itself on m68ks -- too slow and I had to return my m68k hp to the UofU.

jeff

Re: __register_frame_info & shared library compatibility

[Marc Espie]

In article < 199904212017.NAA20578@atrus.synopsys.com > you write:
>
>Yet another reason why we need closer cooperation with the BSD folks
>(so we find out about these things sooner).

Now that OpenBSD 2.5 is out the door, I am starting the process of trying
to switch. i386 going well so far, I expect a few ICEs from m68k.

Don't worry, you'll get *very* close cooperation from us :)

Re: __register_frame_info & shared library compatibility

[Joe Buck]

> > We were discussing a problem that only appeared in Linux distributions.
> > The suggestion to upgrade Glibc was meant only for Linux users.
> 
> Postings from John Polstra <jdp@polstra.com> *were* about problems with
> __register_frame_info on FreeBSD.

Whoops (removes foot from mouth :-).  I was thinking of another part of
this thread which was glibc-specific.

Yet another reason why we need closer cooperation with the BSD folks
(so we find out about these things sooner).

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> We were discussing a problem that only appeared in Linux distributions.
> The suggestion to upgrade Glibc was meant only for Linux users.

Postings from John Polstra <jdp@polstra.com> *were* about problems with
__register_frame_info on FreeBSD.

I will drop it now, but I just wanted others to remember that as Jeff Law
said, interface changes to libgcc affect everyone.
 
-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

> > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> > the weak references can stay, and executables avoid pointless bloat.

David O'Brien wrote:
> ALL THE WORLD IS NOT LINUX!

We were discussing a problem that only appeared in Linux distributions.
The suggestion to upgrade Glibc was meant only for Linux users.

Sorry you got the wrong end of that stick.

Anyway, the discussion is over now.

-- Jamie

Re: __register_frame_info & shared library compatibility

[David O'Brien]

> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 
> In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
> the weak references can stay, and executables avoid pointless bloat.

I know I'm a little bit late on this discussion, but I would really like
to point out to most of the list members one of Henry Spencer's "Ten
Commandments for C Programmers" :

X   Thou shalt foreswear, renounce, and abjure the vile heresy which
    claimeth that ``All the world's a VAX'', and have no commerce with
    the benighted heathens who cling to this barbarous belief, that
    the days of thy program may be long even though the days of thy
    current machine be short.

        This particular heresy bids fair to be replaced by ``All the
        world's a Sun'' or ``All the world's a 386'' (this latter
..snip..


I other words:  ALL THE WORLD IS NOT LINUX!

FreeBSD has moved to using EGCS 1.1.2 as the base compiler.  And I don't
just mean for userland, it is our kernel compiler too.  We have no other
C/C++ compilers in the base system.  NetBSD has moved some of their
architectures to this point, and OpenBSD has plans to.

So Linux-centric hacks hacks that bring havoc into the world are NOT
appropriate.

-- 
-- David    (obrien@NUXI.com  -or-  obrien@FreeBSD.org)

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   Date: Wed, 14 Apr 1999 15:09:36 -0600
   From: Jeffrey A Law <law@upchuck.cygnus.com>

     In message < 199904141258.OAA22629@landau.wins.uva.nl >you write:
     >    If you can find a solution that works in both worlds, then great.
     >    But I do not believe one exists.  And in the absence of a perfect
     >    solution we must continue be compatible with our own releases over
     >    compatibility with gcc-2.7.
     > 
     > What about my suggestion to use weak references to
     > __register_frame_info and __deregister_frame_info in crtbegin.o and
     > normal (non-weak) references in crtbeginS.o?

   I must have glossed over it.  If you've got it handy, can you
   resend (or I'll grope around the archives for it).

I don't have a patch at the moment.  It is just an idea that I wanted
to check with a few people before checking out how to implement it in
a clean way.

The idea is to use the fact that for a number of ELF targets there are
two crtbegin files.  crtbegin.o is used for normal programs and
crtbeginS.o is copiled with -fPIC for use in shared libraries.  The
idea is that as long as we keep the normal (non-weak) references to
__register_frame_info and __deregister_frame_info in crtbeginS.o the
same shared libraries end up with __register_frame_info and
__deregister_frame_info as with egcs-1.1.1.  But I think we are free
to make the references in crtbegin.o weak.  This would mean that
programs that don't explicitly use exception handling would run fine
if these symbols are not available.

What I'd like to know is if it is acceptable to make crtbegin.o and
crtbeginS.o different in this respect, and if you can shoot any holes
in my little theory.

Note that this solution isn't perfect.  Not all targets seem to have
seperate crtbegin.o and crtbeginS.o files.  And it is still not
possible to arbitrarily exchange shared libraries between systems with
an egcs compiled glibc 2.0 and an gcc 2.7 compiled glibc 2.0.  But I
think it is better than the egcs 1.1.1 situation.

Mark

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904141258.OAA22629@landau.wins.uva.nl >you write:
  >    If you can find a solution that works in both worlds, then great.
  >    But I do not believe one exists.  And in the absence of a perfect
  >    solution we must continue be compatible with our own releases over
  >    compatibility with gcc-2.7.
  > 
  > What about my suggestion to use weak references to
  > __register_frame_info and __deregister_frame_info in crtbegin.o and
  > normal (non-weak) references in crtbeginS.o?
I must have glossed over it.  If you've got it handy, can you resend (or I'll
grope around the archives for it).


jeff

__register_frame_info & shared library compatibility

[Mark Kettenis]

   From: Jeffrey A Law (law@cygnus.com)
   Date: Tue, 13 Apr 1999 21:57:56 -0600
 
     > Please note the wording: "upgrade to a glibc that was not built with
     > egcs".  As long as the official libc binaries shipped by the
     > distributions are built by gcc 2.7, egcs MUST match gcc 2.7's ABI for
     > C programs.  That is what the patch achieves.  And we told the
     > distributions to use gcc 2.7 until they switch to glibc 2.1, so this
     > will continue to be a problem.
   OK.  This kind of compatibility is not a priority and never has been.  We've
   always known this would not work.

I don't think there are many people that are aware of this.

   If you can find a solution that works in both worlds, then great.
   But I do not believe one exists.  And in the absence of a perfect
   solution we must continue be compatible with our own releases over
   compatibility with gcc-2.7.

What about my suggestion to use weak references to
__register_frame_info and __deregister_frame_info in crtbegin.o and
normal (non-weak) references in crtbeginS.o?

   At this point I'm still planning to remove the patch and make an
   egcs-1.1.3 release.

Unless you think the solution suggested above is viable, and we can
test it on short term, that's indeed the only right solution (sorry Zack).

Mark

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904140616.CAA07720@blastula.phys.columbia.edu >you write:
  > I've been arguing entirely from the point of view of the libc
  > project.  And there, `compatibility with an outdated, pre-EH compiler
  > from the FSF' is critical, because like it or not, >90% of Linux
  > installations are still using gcc 2.7.  Furthermore, this is no small
  > bug; if you install a libc compiled by gcc 2.7 over a system compiled
  > by egcs 1.0/1.1.[01], you hose the entire system.
I understand that.  But how often do people overwrite a library with one
built with an _older_ compiler?  I don't think we should cater to this
case.  And certainly not at the expense of breaking the ability to install
a library compiled by a newer compiler!


  > I am not saying that egcs should necessarily cater to libc, but I
  > think you may be underestimating the magnitude of the problem we face
  > if you take the patch back out.
Possibly.  In the same light, you may be underestimating the problems the
patch is causing and will cause in the future.  And it's simply absurd to
have these binary incompatibilities between egcs releases.

jeff

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 13 Apr 1999 21:57:56 -0600, Jeffrey A Law wrote:
>
>  In message < 199904121315.JAA13984@blastula.phys.columbia.edu >you write:
>  > >True.  But we've made not attempt to make that setup work and have no
>  > >intention of making that setup work.
>  > 
>  > You may not have, but Mark, HJ, Ulrich and I spent about a month
>  > trying to fix exactly this setup.  The conclusion was that the patch
>  > you don't like was the best solution.
>The egcs project has never considered this kind of compatibility with
>gcc-2.7 compiled binaries a priority.
>
>Giving up compatibility with egcs itself to achieve compatibility with
>an outdated, pre-EH compiler from the FSF is not acceptable.  Sorry.

I've been arguing entirely from the point of view of the libc
project.  And there, `compatibility with an outdated, pre-EH compiler
from the FSF' is critical, because like it or not, >90% of Linux
installations are still using gcc 2.7.  Furthermore, this is no small
bug; if you install a libc compiled by gcc 2.7 over a system compiled
by egcs 1.0/1.1.[01], you hose the entire system.

Libc is in a somewhat unique position, in that we don't have to worry
about intersystem portability unless you count Hurd, but we do have to
make it work for all combinations of compiler, kernel, and source
package - even the ones you don't care to support.

I am not saying that egcs should necessarily cater to libc, but I
think you may be underestimating the magnitude of the problem we face
if you take the patch back out.

zw

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904121315.JAA13984@blastula.phys.columbia.edu >you write:
  > >True.  But we've made not attempt to make that setup work and have no
  > >intention of making that setup work.
  > 
  > You may not have, but Mark, HJ, Ulrich and I spent about a month
  > trying to fix exactly this setup.  The conclusion was that the patch
  > you don't like was the best solution.
The egcs project has never considered this kind of compatibility with
gcc-2.7 compiled binaries a priority.

Giving up compatibility with egcs itself to achieve compatibility with
an outdated, pre-EH compiler from the FSF is not acceptable.  Sorry.

  > 
  > >But it is unacceptable when a library that was originally compiled
  > >with egcs-1.1.1, after compiling with egcs-1.1.2 to cause libraries
  > >to stop working.  Plain and simple, it is unacceptable.
  > 
  > The library should not have been compiled with egcs 1.1.1 in the first
  > place.
  > 
  > Yes, I realize this is an unrealistic attitude.  Would you at least
  > consider an -mcompat-1.0 switch?
The default must be to be compatible with prior egcs releases.  I'd much
prefer not add a switch to enable the losing behavior because it just adds
to the long term maintenance and confusion.


  > # When I compile glibc 2.0.{6,7} with egcs, glibc ends up defining the
  > # symbols __register_frame_info & __deregister_frame_info. I noticed
  > # that these are also defined in egcs' libgcc.a Should this be
  > # happening?
  > #
  > # The real problem happens when I upgrade to a glibc that was not
  > # built with egcs.  Some of the programs I compiled with egcs under a
  > # egcs built glibc also have a dependency on these symbols.  If it
  > # wasn't /bin/login, I might not have cared so much. :-/
  > 
  > Please note the wording: "upgrade to a glibc that was not built with
  > egcs".  As long as the official libc binaries shipped by the
  > distributions are built by gcc 2.7, egcs MUST match gcc 2.7's ABI for
  > C programs.  That is what the patch achieves.  And we told the
  > distributions to use gcc 2.7 until they switch to glibc 2.1, so this
  > will continue to be a problem.
OK.  This kind of compatibility is not a priority and never has been.  We've
always known this would not work.

If you can find a solution that works in both worlds, then great.  But I do
not believe one exists.  And in the absence of a perfect solution we must
continue be compatible with our own releases over compatibility with gcc-2.7.

At this point I'm still planning to remove the patch and make an egcs-1.1.3
release.

jeff

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Sun, 11 Apr 1999 22:19:53 -0600, Jeffrey A Law wrote:
>
>
>  > GNU libc has its own versions of these C startup files, and I thought
>  > they had been changed to do something similar as the weak
>  > __register_frame_info patch in egcs-1.1.2, but it turns out that this
>  > is not true.  So you're right.
>Only in releases most people do not have and are not using.

Every single user of Redhat 5.2 and Debian 2.0 has those changes.  I
wouldn't call that `not most people'

>  >    This problem can be solved by making the reference to the
>  >    frame function in the crt files a normal (non-weak) reference
>  >    again.  This will not break existing code.
>  > 
>  > But it doesn't solve the problems with a program built with egcs-1.1,
>  > linked against shared libraries built with egcs-1.1, running on a
>  > system where the shared libraries were built with gcc 2.7.  Because in
>  > that case both the program and the shared library do not contain
>  > __register_frame_info but it is refernced by the program.
>True.  But we've made not attempt to make that setup work and have no
>intention of making that setup work.

You may not have, but Mark, HJ, Ulrich and I spent about a month
trying to fix exactly this setup.  The conclusion was that the patch
you don't like was the best solution.

>But it is unacceptable when a library that was originally compiled
>with egcs-1.1.1, after compiling with egcs-1.1.2 to cause libraries
>to stop working.  Plain and simple, it is unacceptable.

The library should not have been compiled with egcs 1.1.1 in the first
place.

Yes, I realize this is an unrealistic attitude.  Would you at least
consider an -mcompat-1.0 switch?

>  > We both know that the original submitter of the patch is not very
>  > good at explaining the exact technical reasons why his patches should
>  > go in, but I remember him writing that "it fixes a bug".  I would not
>  > call generating slightly bigger binaries as bug, so I am under the
>  > impression that he was trying to solve the problem illustrated in the
>  > previous paragraph.  And I know for sure
>  > that some of people that advocated the patch were trying to solve that
>  > problem. That was the case Zack was mentioning a few messages back.
>Then I would strongly recommend that HJ explain in painful detail why this
>fixed a bug.

Here's the original bug report glibc development got:

# When I compile glibc 2.0.{6,7} with egcs, glibc ends up defining the
# symbols __register_frame_info & __deregister_frame_info. I noticed
# that these are also defined in egcs' libgcc.a Should this be
# happening?
#
# The real problem happens when I upgrade to a glibc that was not
# built with egcs.  Some of the programs I compiled with egcs under a
# egcs built glibc also have a dependency on these symbols.  If it
# wasn't /bin/login, I might not have cared so much. :-/

Please note the wording: "upgrade to a glibc that was not built with
egcs".  As long as the official libc binaries shipped by the
distributions are built by gcc 2.7, egcs MUST match gcc 2.7's ABI for
C programs.  That is what the patch achieves.  And we told the
distributions to use gcc 2.7 until they switch to glibc 2.1, so this
will continue to be a problem.

The date on that, incidentally, is last June.

Here's one of several painfully long explanations of the problem which
I wrote back in December.

# The bug is a subtle interaction between the way EGCS (all versions)
# implements exception handling, and the way shared libraries are
# constructed.
# 
# EGCS' exception handling uses a bunch of internal library functions.
# They are in libgcc.a.  Those functions are referenced from every
# translation unit compiled without -fno-exceptions, whether or not it
# has exceptions in it.  gcc 2.7/2.8 does not have these functions in
# its libgcc.a, and doesn't refer to them in programs it compiles.
# 
# When libc is built, it is linked with libgcc.  We have to do that
# because libc may need libgcc functions that some executable that uses
# it doesn't need.  The 2.0 libc then turns around and re-exports all
# the libgcc functions that it pulled in.  If 2.0 libc is compiled by
# egcs, that includes the exception handling functions.  In 2.1 we can
# prevent this but we've chosen not to; see below for why.
# 
# When executables are built, gcc links first with libgcc, second with
# libc.  The executable gets static copies of the exception functions,
# and all is well.
# 
# When other shared libraries are built, they are linked first with 
# libc and second with libgcc.  That means they don't get static copies
# of the eh functions; they refer to the ones in libc.  That
# reference will only be satisfied at runtime if the libc.so in use then
# was compiled by egcs.
# 
# My patch for 2.0 changed the link ordering for extra-libs provided by
# libc, so that they would get their own static copy of the eh
# functions.  Now there is no dependency of libm (for example) on an
# egcs-compiled libc.  But this only shifts the problem, because if you
# give gcc any -l options, it links with those before it links with
# libgcc.  Therefore the executable will want the eh functions to be in
# libm.  That will only be true at runtime if the libm.so in use then
# was compiled by egcs.
# 
# In 2.1, we can ensure that all our shared libraries have version maps
# that don't export the eh functions.  That doesn't fix the general
# problem, because most shared libraries are not provided by glibc.  A
# program linked with libX11-compiled-by-egcs will have a dependency
# which is not satisfied by libX11-compiled-by-gcc-2.8.  Instead, Ulrich
# has chosen to explicitly export the eh functions from libc.  That
# means they are part of the official 2.1 ABI, and everyone is supposed
# to reference our copy and not the one in libgcc.  This works as long
# as libc 2.1 is always compiled by egcs.  If it is ever compiled by gcc
# 2.8, it will be binary incompatible with the version compiled by egcs.
# 
# The right fix to the problem is to change gcc such that libgcc is
# always linked ahead of all libraries specified on the command line -
# i.e. if I write
# 
# gcc -o prog prog.o -lfoo -lbar
# 
# the linker command line must wind up like this:
# 
# ld -o prog crt1.o ... prog.o -lgcc -lfoo -lbar -lc -lgcc ... crtn.o
# 
# and similarly when shared libraries are linked (using gcc -shared).
# Then everyone will get a private copy of the eh functions and the
# problem will not arise.
# 
# Until this is done, the right thing to do is:
# 
# never compile glibc 2.0 with egcs
# always compile glibc 2.1 with egcs.

zw

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  > GNU libc has its own versions of these C startup files, and I thought
  > they had been changed to do something similar as the weak
  > __register_frame_info patch in egcs-1.1.2, but it turns out that this
  > is not true.  So you're right.
Only in releases most people do not have and are not using.

  >    This problem can be solved by making the reference to the frame function
  >    in the crt files a normal (non-weak) reference again.  This will not bre
  > ak
  >    existing code.
  > 
  > But it doesn't solve the problems with a program built with egcs-1.1,
  > linked against shared libraries built with egcs-1.1, running on a
  > system where the shared libraries were built with gcc 2.7.  Because in
  > that case both the program and the shared library do not contain
  > __register_frame_info but it is refernced by the program.
True.  But we've made not attempt to make that setup work and have no intention
of making that setup work.  But it is unacceptable when a library that was
originally compiled with egcs-1.1.1, after compiling with egcs-1.1.2 to cause
libraries to stop working.  Plain and simple, it is unacceptable.


  > We both know that the original submitter of the patch is not very
  > good at explaining the exact technical reasons why his patches should
  > go in, but I remember him writing that "it fixes a bug".  I would not
  > call generating slightly bigger binaries as bug, so I am under the
  > impression that he was trying to solve the problem illustrated in the
  > previous paragraph.  And I know for sure
  > that some of people that advocated the patch were trying to solve that
  > problem. That was the case Zack was mentioning a few messages back.
Then I would strongly recommend that HJ explain in painful detail why this
fixed a bug.  As soon as I can find a few free minutes I'm planning to revert
the busted patch (both in the mainline tree and in the egcs-1.1.x release
tree).

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904101341.JAA27027@blastula.phys.columbia.edu >you write:
  > >This problem can be solved by making the reference to the frame function
  > >in the crt files a normal (non-weak) reference again.  This will not break
  > >existing code.
  > 
  > This is technically true, however you will break people's
  > expectations.  Libc has been going around telling everyone that they
  > need to compile their libraries with gcc 2.7 or else apply the
  > controversial patch to egcs.  Oh, and the patch is official in egcs
  > 1.1.2.  From libc's point of view, any library that was not compiled
  > as above is broken; it fails to conform to the de facto ABI.
It does not matter if you and the glibc maintainers consider this a glibc
problem.  It is NOT ACCEPTABLE for egcs to break binary compatibility in
the manner that happened with egcs-1.1.2.  Period.

jeff

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   Date: Sat, 10 Apr 1999 02:28:42 -0600
   From: Jeffrey A Law <law@upchuck.cygnus.com>

     > * GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
     >   reference __register_frame_info [1].
   Incorrect.  If you compile glibc with egcs-1.1, then the resulting library
   will have __register_frame_info.  This happens because the C startup files
   have a reference to __register_frame_info, which in turn cause
   __register_frame_info to be brought in from libgcc::frame.o.

GNU libc has its own versions of these C startup files, and I thought
they had been changed to do something similar as the weak
__register_frame_info patch in egcs-1.1.2, but it turns out that this
is not true.  So you're right.

   If you then recompile that library with egcs-1.1.2, then that library may
   not get a copy of the frame functions if the only reference to them is
   the weak symbol in the crt*.o files.

Therefore the crt*.o files used in building shared libraries must have
normal (non-weak) references to __register_frame_info.

   Now the progrm which was originally linked against that library will fail
   because there is no definition of the frame functions.

Except when the crt*.o files used in building the program had only
weak references to __register_frame_info.  That's why it is a good
idea to do this if the target has two sets of crt*.o files, as Linux does.

   This problem can be solved by making the reference to the frame function
   in the crt files a normal (non-weak) reference again.  This will not break
   existing code.

But it doesn't solve the problems with a program built with egcs-1.1,
linked against shared libraries built with egcs-1.1, running on a
system where the shared libraries were built with gcc 2.7.  Because in
that case both the program and the shared library do not contain
__register_frame_info but it is refernced by the program.

     >    I don't believe there is a right answer.
   The right answer is to put the normal reference back.  It will not break
   programs, just make them a little larger.  That was the only reason those
   symbosl were make weak to begin with -- to reduce the size of programs
   which did not actually use exception handling.

We both know that the original submitter of the patch is not very
good at explaining the exact technical reasons why his patches should
go in, but I remember him writing that "it fixes a bug".  I would not
call generating slightly bigger binaries as bug, so I am under the
impression that he was trying to solve the problem illustrated in the
previous paragraph.  And I know for sure
that some of people that advocated the patch were trying to solve that
problem. That was the case Zack was mentioning a few messages back.

Mark

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Sat, 10 Apr 1999 02:28:42 -0600, Jeffrey A Law wrote:
>
>  > * GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
>  >   reference __register_frame_info [1].
>Incorrect.  If you compile glibc with egcs-1.1, then the resulting library
>will have __register_frame_info.  This happens because the C startup files
>have a reference to __register_frame_info, which in turn cause
>__register_frame_info to be brought in from libgcc::frame.o.
>
>Furthermore, the definition in the library will have external visibility.

This is true or not true depending on which point revision of glibc
2.0 you have.  Patches were added somewhere in there which prevent
libc.so from exporting __register_frame_info - it uses its own private
crtbeginS.o (effectively) which has the weak symbol refs.

Unfortunately, that just means the externally visible definition of
__register_frame_info moves over to libm.so.

To make matters worse, I think this particular patch was added to libc
CVS after Ulrich stopped making official point releases.  That means
Redhat and Debian have it and no one else does.

>  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked
>  >   only against libc contain and reference __register_frame_info.
>Depends on whether or not they link against any libraries which provide
>register_frame_info.

He said `only against libc'.  We always do -lgcc -lc -lgcc, so the
above assertion is correct.

>  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked against an
>  >   other shared library besides libc do not contain
>  >   __register_frame_info but do reference __register_frame_info.
>Incorrect.  It depends on what is in the libraries that the program is
>linked again.

In practice, this assertion is correct if and only if the `other
shared library' was compiled with egcs with a version number in
[1.0, 1.1.1] or with gcc 2.8.

>  > Now there are a few possibilities where programs will not run:
>I've already explained the cases where program will stop running.
>
>If you have a program which is linked against a shared library and the
>shared library originally contained the frame functions, then that program
>will expect to get the frame functions from the library and will not have
>its own copies.
Yes.

>If you then recompile that library with egcs-1.1.2, then that library may
>not get a copy of the frame functions if the only reference to them is
>the weak symbol in the crt*.o files.
Yes.

>Now the progrm which was originally linked against that library will fail
>because there is no definition of the frame functions.
Yes.

>This problem can be solved by making the reference to the frame function
>in the crt files a normal (non-weak) reference again.  This will not break
>existing code.

This is technically true, however you will break people's
expectations.  Libc has been going around telling everyone that they
need to compile their libraries with gcc 2.7 or else apply the
controversial patch to egcs.  Oh, and the patch is official in egcs
1.1.2.  From libc's point of view, any library that was not compiled
as above is broken; it fails to conform to the de facto ABI.

zw

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904091149.NAA16737@landau.wins.uva.nl >you write:
  > * All shared libraries compiled with gcc 2.7 do not contain or
  >   reference __register_frame_info.
Correct.


  > * GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
  >   reference __register_frame_info [1].
Incorrect.  If you compile glibc with egcs-1.1, then the resulting library
will have __register_frame_info.  This happens because the C startup files
have a reference to __register_frame_info, which in turn cause
__register_frame_info to be brought in from libgcc::frame.o.

Furthermore, the definition in the library will have external visibility.

  > * All other shared libraries compiled with gcc 2.8 or egcs-1.1 do
  >   contain __register_frame_info, except when they are linked against
  >   an other shared library that contains __register_frame_info.  They
  >   will all reference __register_frame_info.
Correct.

  > * All shared libraries compiled with egcs-1.1.2 do not contain
  >   __register_frame_info, but have a weak reference to
  >   __register_frame_info.
Correct.


  > * All programs compiled with gcc 2.7 do not contain or reference
  >   __register_frame_info.
Correct.

  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked
  >   only against libc contain and reference __register_frame_info.
Depends on whether or not they link against any libraries which provide
register_frame_info.


  > * All programs compiled with gcc 2.8 or egcs-1.1 and linked against an
  >   other shared library besides libc do not contain
  >   __register_frame_info but do reference __register_frame_info.
Incorrect.  It depends on what is in the libraries that the program is
linked again.


  > * All programs compiled with egcs 1.1.2 do not contain
  >   __register_frame_info but have a weak reference to
  >   __register_frame_info.
Correct.

  > Now there are a few possibilities where programs will not run:
I've already explained the cases where program will stop running.

If you have a program which is linked against a shared library and the
shared library originally contained the frame functions, then that program
will expect to get the frame functions from the library and will not have
its own copies.

If you then recompile that library with egcs-1.1.2, then that library may
not get a copy of the frame functions if the only reference to them is
the weak symbol in the crt*.o files.

Now the progrm which was originally linked against that library will fail
because there is no definition of the frame functions.

This problem can be solved by making the reference to the frame function
in the crt files a normal (non-weak) reference again.  This will not break
existing code.

  >    I don't believe there is a right answer.
The right answer is to put the normal reference back.  It will not break
programs, just make them a little larger.  That was the only reason those
symbosl were make weak to begin with -- to reduce the size of programs
which did not actually use exception handling.

jeff

Re: __register_frame_info & shared library compatibility

[Philipp Thomas]

On Thu, 08 Apr 1999 14:55:48 -0700 (PDT), you wrote:

>If libgcc were shared, then this problem wouldn't exist ... right?

Yea, but you would trade one problem with a whole bunch of new ones a
shared libgcc would introduce as the discussion some time ago showed.

Philipp Thomas

-- 
caffeine low .... brain halted

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

Here is a rather long mail, but I hope that people who want a real
solution to the __register_frame_info & shared library compatibility
problem will take the time to read it.  If you think I'm wrong, please
tell me!

   Leaving the patch out causes a different set of people problems,
   namely the ones that want to compile stuff with egcs-1.1.x and have it
   port over to systems with gcc 2.7-compiled libc.

OK, here are the facts as I understand them.  I only look at C
programs (that should not explicitly be using exceptions).  Binary
compatibility for C++ should be OK if they use the right shared
libstdc++.

gcc 2.7 = all sub-versions of gcc 2.7

gcc 2.8 = all sub-versions of gcc 2.8

egcs 1.1 = egcs-1.1 or egcs-1.1.1 and probably egcs-1.0.3

* All shared libraries compiled with gcc 2.7 do not contain or
  reference __register_frame_info.

* GNU libc compiled with gcc 2.8, egcs-1.1 should not contain or
  reference __register_frame_info [1].

* All other shared libraries compiled with gcc 2.8 or egcs-1.1 do
  contain __register_frame_info, except when they are linked against
  an other shared library that contains __register_frame_info.  They
  will all reference __register_frame_info.

* All shared libraries compiled with egcs-1.1.2 do not contain
  __register_frame_info, but have a weak reference to
  __register_frame_info.

* All programs compiled with gcc 2.7 do not contain or reference
  __register_frame_info.

* All programs compiled with gcc 2.8 or egcs-1.1 and linked
  only against libc contain and reference __register_frame_info.

* All programs compiled with gcc 2.8 or egcs-1.1 and linked against an
  other shared library besides libc do not contain
  __register_frame_info but do reference __register_frame_info.

* All programs compiled with egcs 1.1.2 do not contain
  __register_frame_info but have a weak reference to
  __register_frame_info.

Now there are a few possibilities where programs will not run:

* Any program built with gcc 2.8 or egcs-1.1 linked against an other
  shared library besides libc will not run on a system where this
  shared library was compiled with gcc 2.7 or egcs-1.1.2.

* Any program built with gcc 2.8 or egcs-1.1 linked against lib A
  where lib A was linked against lib B will not run on a system where
  lib A was not linked against lib B.

The last case is a bit far-fetched and I don't think that we should
care about it.  The first case is an unacceptable problem.

How can it be solved:

* Revert the weak __register_frame_info patch.  This solves the
  problem where the shared library was compiled with egcs-1.1.3, but
  does not solve the problem for shared libraries compiled with gcc
  2.7.  Moreover programs built with egcs-1.1.3 linked against a
  library other than libc compiled with gcc 2.8 or egcs will also not
  work with shared libraries compiled with gcc 2.7.

* Keep the weak __register_frame_info patch.  Tell people to link
  their shared libraries with the -u __register_frame_info linker
  option, or put it in the specs.

* Make crtbegin.o have a weak reference to __register_frame_info, and
  crtbeginS.o have an ordinary reference to __register_frame_info.

The third solution is probably the best one, but I don't know if all
ELF targets have a separate crtbegin.o and crtbeginS.o.

The second solution would also work.  Putting the -u option in the
specs has the advantage that it does not require any user
interaction.  However this does not work if people invoke the linker
directly.  Not putting it in the specs makes it possible for people
to omit it when they are changing the shared libraries ABI anyway.  It
should be throughly documented, but since a lot of people will not
read the documentation it will lead to a lot of "bug"-reports.

The first option keeps things as they were before egcs 1.1.2.

Note that this discussion is rather specific to Linux with glibc 2.0.
Linux with glibc 2.1 has no problems with __register_frame_info,
because __register_frame_info is included in glibc's ABI.  I don't
know how things work for the old ELF libc (libc5).

Other targets that use ELF will have similar problems, although they
might not be detected because there are less people distributing
binaries for these targets.  (The various BSD variants come to mind
where one typically builds the entire tree with only one compiler).
If it is no problem for such a target to break binary compatibility,
using the weak __register_frame_info patch as in egcs-1.1.2 is the
right thing to do.  Otherwise one of the options mentioned above
should probably be used there too.  For new ELF targets the weak
__register_frame_info patch is the right thing to do, both for
programs and shared libraries.

   I don't believe there is a right answer.

It is probably not possible to make absolutely sure that nobody will
have __register_frame_info problems anymore, but I think it is
possible to avoid problems in the more common cases.

Mark

[1] I'm not sure if this is really the case, because people always
tend to find ways to break things, but a lot of effort was made to
make sure that this is the case.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < XFMail.990408145548.jdp@polstra.com >you write:
  > It seems like this is an instance of a more general problem that often
  > crops up when you try to mix archive libraries with shared libraries.
  > If libgcc were shared, then this problem wouldn't exist ... right?
Yes.  But there's a whole different can of worms if you make libgcc
a shared library :-)


jeff

Re: __register_frame_info & shared library compatibility

[John Polstra]

Jeffrey A Law wrote:
>   > I don't understand why recompiling a shared library with the
>   > weak-referencing version of egcs would cause programs linked with it
>   > to stop working.
...
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Thanks for the explanation.  That clears it up.

It seems like this is an instance of a more general problem that often
crops up when you try to mix archive libraries with shared libraries.
If libgcc were shared, then this problem wouldn't exist ... right?

John
---
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Thu, Apr 08, 1999 at 10:24:18PM +0200, Jamie Lokier wrote:
> [BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]
> 
> > No, this is an egcs problem.  We can't have these kinds of problems show
> > up when folks rebuild their libraries with newer versions of the compiler.
> > 
> > Can I be any clearer?
> 
> The policy is pretty clear.
> 
> The danger is that we get locked in to including bloat in all binaries,
> that can't ever be removed.  I'm interested in a solution to that long
> term problem.  If we have to include the bloat in the interim to provide
> binary compatibility, that is fine.

It's better to have bloated libraries than broken libraries. ;-) Binary
compatibility issues affect thousands of projects and millions of users.

I build binary packages for a number of projects, and maintain Linux
systems for various people. My life is easiest when the C compiler and libc
adhere to well-defined rules. When things get confusing, I make mistakes
and cause trouble for downstream users. Other people make mistakes, too,
causing me grief. (Right now, both my desktop and latest Mozilla build are
broken because somebody at RHAD labs thought it would be fun to compile
glib and gtk with egcs. I'll fix this when I have nothing better to do.)

Now, whether egcs requires updates to libc is your business, not mine.
What *is* essential, though, is that all of us downstream packagers and
users understand the rules. Otherwise, we'll screw up and make a mess.

So please keep the rules as simple as possible, and document all breakages
carefully. Legacy installations are a pain in the neck, but they're also a
fact of life.

Thank you for letting me get this off my chest. :-)

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408224135.A21864@pcep-jamie.cern.ch >you write:

  > If you're prepared to break the C shared libraries at the same time,
  > your proposal sounds perfect.
I'd _much_ prefer not to, but if people are going to demand removal of those
dead procedures then the only time that will be possible is when we already
have to break binary compatibility for C++.



  > Sorry, you're right.  I have seen big ones before, and I just checked
  > .eh_frame and found a small one on a recent C binary.  Even a trivial
  > program has a tiny .eh_frame of 4 bytes.
Hmmm, I wonder where the tiny .eh_frame came from..  Probably  something in
libgcc :-)

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408223252.A21853@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Later that developer installs egcs-1.1.2 and rebuilds the shared
  > > libraries.  He should be able to reasonably assume that the clients
  > > would still work.  But because of your incompatible change they will
  > > not.
  > 
  > The error on the part of the developer was to install egcs-1.1.2 without
  > upgrading glibc.
Installing egcs-1.1.2 should never require updating the system library. 
Policy, not subject for debate.

  > Could we not guard against that error, by failing at link time against
  > the older glibc?
No.  egcs-1.1.2 has to work with older libcs.

jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> We already know at some point we will have to break binary compatibility in
> the future to get some new C++ features.  That would be the time to drop the
> old interfaces.

If you're prepared to break the C shared libraries at the same time,
your proposal sounds perfect.

>   > Though frankly, the bloat due to the redundant .eh_frame in C binaries
>   > much more significant.  Is throwing an exception through C really
>   > supposed to have defined behaviour?
> ?!?  I don't think we create eh_frames for C code by default now.  We do
> if one specifies -fexceptions though.

Sorry, you're right.  I have seen big ones before, and I just checked
.eh_frame and found a small one on a recent C binary.  Even a trivial
program has a tiny .eh_frame of 4 bytes.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Later that developer installs egcs-1.1.2 and rebuilds the shared
> libraries.  He should be able to reasonably assume that the clients
> would still work.  But because of your incompatible change they will
> not.

The error on the part of the developer was to install egcs-1.1.2 without
upgrading glibc.

Could we not guard against that error, by failing at link time against
the older glibc?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408222418.C21807@pcep-jamie.cern.ch >you write:
  > The danger is that we get locked in to including bloat in all binaries,
  > that can't ever be removed.  I'm interested in a solution to that long
  > term problem.  If we have to include the bloat in the interim to provide
  > binary compatibility, that is fine.
We already know at some point we will have to break binary compatibility in
the future to get some new C++ features.  That would be the time to drop the
old interfaces.

But in general, if an interface is added with external scope in libgcc, then it
has to stay indefinitely (until the next planned binary breakage).  Thus we
have to be a *LOT* more careful about what interfaces we put in that library --
we don't want to plan many binary breakages.


  > Though frankly, the bloat due to the redundant .eh_frame in C binaries
  > much more significant.  Is throwing an exception through C really
  > supposed to have defined behaviour?
?!?  I don't think we create eh_frames for C code by default now.  We do
if one specifies -fexceptions though.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKyt-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Is is possible to include a stub weak *definition* of
  > > __register_frame_info in crtbeginS.o, which is nothing more than a
  > > return instruction?
  > > 
  > > Would that solve these temporary binary compatibility problems, without
  > > the permanent bloat?
  > > 
  > 
  > That means we have to do more testing without much benefit. I don't
  > think it is wise for egcs people to spend any more time on this. It is
  > a glibc problem and let glibc people deal with it. If some glibc users
  > won't listen to the glibc people, they get what they deserved.
It's not just a glibc problem.

Consider a system where all the system libraries were compiled with gcc-2.7
(like an older RedHat distribution).

A developer later installed egcs-1.1.1 and compiled some package that
installed some new shared libraries (let's say X11 libraries).    Then he
built a bunch of apps that link against those libraries (lots of X11 clients)

Later that developer installs egcs-1.1.2 and rebuilds the shared libraries.
He should be able to reasonably assume that the clients would still work.  But
because of your incompatible change they will not.



jeff

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

[BTW Jeffrey, your from: address <law@upchuck.cygnus.com> is bouncing]

> No, this is an egcs problem.  We can't have these kinds of problems show
> up when folks rebuild their libraries with newer versions of the compiler.
> 
> Can I be any clearer?

The policy is pretty clear.

The danger is that we get locked in to including bloat in all binaries,
that can't ever be removed.  I'm interested in a solution to that long
term problem.  If we have to include the bloat in the interim to provide
binary compatibility, that is fine.

Though frankly, the bloat due to the redundant .eh_frame in C binaries
much more significant.  Is throwing an exception through C really
supposed to have defined behaviour?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VKvO-000ErNC@ocean.lucon.org >you write:
  > Exactly. That is a glibc problem. not an egcs problem. I don't think
  > egcs should get involved with it. Maybe we should put it in the glibc
  > FAQ.
No, this is an egcs problem.  We can't have these kinds of problems show
up when folks rebuild their libraries with newer versions of the compiler.

Can I be any clearer?

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408215451.B21750@pcep-jamie.cern.ch >you write:
  > Is is possible to include a stub weak *definition* of
  > __register_frame_info in crtbeginS.o, which is nothing more than a
  > return instruction?
No, because if the user binary actually expected to use the
__register_frame_info that was originally satisfied by the library, then it'll
get the useless stub instead of the real implementation.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408214924.A21750@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up 
  > a
  > >   > ftp site with tar files.
  > > What am I not being clear about?
  > > 
  > > This kind of binary breakage is not acceptable.  It doesn't matter if you
  > > put a new glibc somewhere.  Breaking binaries like that patch did is not
  > > acceptable.
  > 
  > The patch does *not break binaries*.
It does if one rebuilds shared libraries after installing that patch.  Existing
binaries may no longer run.

  > Installing EGCS and using it to compile new applications does
  > *not break binaries*.
Agreed.


  > But recompiling some shared libraries using EGCS with the patch breaks some
  > binaries.  Is it unreasonable to request a user upgrades to the latest
  > Glibc 2.0, or any Glibc 2.1 before doing this?
Yes, it is unreasonable.    This kind of breakage is not acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Is is possible to include a stub weak *definition* of
> __register_frame_info in crtbeginS.o, which is nothing more than a
> return instruction?
> 
> Would that solve these temporary binary compatibility problems, without
> the permanent bloat?
> 

That means we have to do more testing without much benefit. I don't
think it is wise for egcs people to spend any more time on this. It is
a glibc problem and let glibc people deal with it. If some glibc users
won't listen to the glibc people, they get what they deserved.


-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> >   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
> >   > ftp site with tar files.
> > What am I not being clear about?
> > 
> > This kind of binary breakage is not acceptable.  It doesn't matter if you
> > put a new glibc somewhere.  Breaking binaries like that patch did is not
> > acceptable.
> 
> The patch does *not break binaries*.
> 
> Installing EGCS and using it to compile new applications does
> *not break binaries*.
> 
> But recompiling some shared libraries using EGCS with the patch breaks some
> binaries.  Is it unreasonable to request a user upgrades to the latest
> Glibc 2.0, or any Glibc 2.1 before doing this?
> 

Exactly. That is a glibc problem. not an egcs problem. I don't think
egcs should get involved with it. Maybe we should put it in the glibc
FAQ.

-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Is is possible to include a stub weak *definition* of
__register_frame_info in crtbeginS.o, which is nothing more than a
return instruction?

Would that solve these temporary binary compatibility problems, without
the permanent bloat?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
>   > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
>   > ftp site with tar files.
> What am I not being clear about?
> 
> This kind of binary breakage is not acceptable.  It doesn't matter if you
> put a new glibc somewhere.  Breaking binaries like that patch did is not
> acceptable.

The patch does *not break binaries*.

Installing EGCS and using it to compile new applications does
*not break binaries*.

But recompiling some shared libraries using EGCS with the patch breaks some
binaries.  Is it unreasonable to request a user upgrades to the latest
Glibc 2.0, or any Glibc 2.1 before doing this?

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < m10VIdz-000ErMC@ocean.lucon.org >you write:
  > > 
  > > Jeffrey A Law wrote:
  > > > Previously the library would always have those routines, even if it did
  > > > not use EH.
  > > > 
  > > > And any program which referenced that library on the link line would
  > > > get those routines via the library intead of getting its own copies.
  > > > 
  > > > Now you recompile the library.  It no longer gets the EH routines.  Bla
  > m
  > > > your user program stops working.
  > > 
  > > Presumably this is all solved by using Glibc 2.1?
  > > 
  > 
  > Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
  > ftp site with tar files.
What am I not being clear about?

This kind of binary breakage is not acceptable.  It doesn't matter if you
put a new glibc somewhere.  Breaking binaries like that patch did is not
acceptable.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990408185632.A21668@pcep-jamie.cern.ch >you write:
  > Jeffrey A Law wrote:
  > > Previously the library would always have those routines, even if it did
  > > not use EH.
  > > 
  > > And any program which referenced that library on the link line would
  > > get those routines via the library intead of getting its own copies.
  > > 
  > > Now you recompile the library.  It no longer gets the EH routines.  Blam
  > > your user program stops working.
  > 
  > Presumably this is all solved by using Glibc 2.1?
  > 
  > In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
  > the weak references can stay, and executables avoid pointless bloat.
No.  It is not an acceptable solution.  The binary incompatability brought on
by egcs-1.1.2 is not acceptable and must be fixed.  Requiring folks to
upgrade to glibc-2.1 is not an acceptable "solution".

This is precisely why we have to think much harder about the interfaces that
we put into libgcc.  Once there, they have to remain forever.


jeff

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> Jeffrey A Law wrote:
> > Previously the library would always have those routines, even if it did
> > not use EH.
> > 
> > And any program which referenced that library on the link line would
> > get those routines via the library intead of getting its own copies.
> > 
> > Now you recompile the library.  It no longer gets the EH routines.  Blam
> > your user program stops working.
> 
> Presumably this is all solved by using Glibc 2.1?
> 

Or they can use the latest glibc 2.0 via CVS. I am willing to set up a
ftp site with tar files.


H.J.

Re: __register_frame_info & shared library compatibility

[Jamie Lokier]

Jeffrey A Law wrote:
> Previously the library would always have those routines, even if it did
> not use EH.
> 
> And any program which referenced that library on the link line would
> get those routines via the library intead of getting its own copies.
> 
> Now you recompile the library.  It no longer gets the EH routines.  Blam
> your user program stops working.

Presumably this is all solved by using Glibc 2.1?

In which case, simply recommend that EGCS users upgrade to Glibc 2.1,
the weak references can stay, and executables avoid pointless bloat.

Long term I would like to see the weak references long term because
anything else is a waste of space.  Short term, if there's a migration
path where the weak references patch is omitted for a while until
"everyone" has upgraded to Glibc 2.1, that would be fine.

-- Jamie

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904080111.SAA76942@vashon.polstra.com >you write:
  > In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote
  > :
  > 
  > >   > The problem is indeed that egcs 1.1.2 now puts only a weak
  > >   > reference to "__register_frame_info" in crtstuff.  This means
  > >   > that "du moment" one recompiles a shared library that previously
  > >   > contained "__register_frame_info" it probably does not anymore,
  > >   > and programs linked with that library stop running.
  > > Yup.  And I consider this a horrible breakage.  I trusted a couple
  > > of folks with more experience in this area to guide the decision to
  > > include that patch.  They made a mistake.  We need to rectify it.
  > 
  > I don't understand why recompiling a shared library with the
  > weak-referencing version of egcs would cause programs linked with it
  > to stop working.  The library either uses exceptions, or it doesn't.
  > If it uses exceptions, it contains calls to __throw and/or other
  > functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
  > contains a strong reference to __frame_state_for. __frame_state_for
  > is defined in frame.o, so that will also be pulled in from libgcc.a.
  > Presto, the library has __register_frame_info just the same as always.
Previously the library would always have those routines, even if it did
not use EH.

And any program which referenced that library on the link line would
get those routines via the library intead of getting its own copies.

Now you recompile the library.  It no longer gets the EH routines.  Blam
your user program stops working.

jeff

Re: __register_frame_info & shared library compatibility

[John Polstra]

In article < 15609.923487695@upchuck >, Jeffrey A Law  <law@cygnus.com> wrote:

>   > The problem is indeed that egcs 1.1.2 now puts only a weak
>   > reference to "__register_frame_info" in crtstuff.  This means
>   > that "du moment" one recompiles a shared library that previously
>   > contained "__register_frame_info" it probably does not anymore,
>   > and programs linked with that library stop running.
> Yup.  And I consider this a horrible breakage.  I trusted a couple
> of folks with more experience in this area to guide the decision to
> include that patch.  They made a mistake.  We need to rectify it.

I don't understand why recompiling a shared library with the
weak-referencing version of egcs would cause programs linked with it
to stop working.  The library either uses exceptions, or it doesn't.
If it uses exceptions, it contains calls to __throw and/or other
functions which will cause _eh.o to be pulled in from libgcc.a. _eh.o
contains a strong reference to __frame_state_for. __frame_state_for
is defined in frame.o, so that will also be pulled in from libgcc.a.
Presto, the library has __register_frame_info just the same as always.

If the library doesn't use exceptions, then it won't pull in frame.o
and it won't have __register_frame_info.  That won't matter for the
library itself, because of the weak references in crtbeginS.o.  Now,
the executable and any other shared libraries that use exceptions
(if any) will each contain __register_frame_info, and the weak
references to __register_frame_info from the shared library in
question will presumably be resolved by the dynamic linker to the
first of those copies.  So the library will be calling a copy of
__register_frame_info that resides in the executable or in a different
shared library.  Is that a problem?  I.e., does each shared library
need to have its own copy of __register_frame_info and the associated
static variables?

My usual context is FreeBSD rather than Linux, so maybe I'm missing
something fundamental here.  If so, please enlighten me, as we are
working on switching to egcs-1.1.2 in the FreeBSD project as we speak.

John
-- 
  John Polstra                                               jdp@polstra.com
  John D. Polstra & Co., Inc.                        Seattle, Washington USA
  "Self-interest is the aphrodisiac of belief."           -- James V. DeLong

Re: __register_frame_info & shared library compatibility

[alex.buell]

On Wed, 7 Apr 1999, Zack Weinberg wrote:

> I don't believe there is a right answer.  Mark understands the issues
> better, maybe he has one.

Jump to glibc-2.1. I think this is what saved my butt when moving from
egcs-1.0.3 through to egcs-1.1.2. I have built lots of libraries under
egcs-1.1.1 that didn't break when I moved over to egcs-1.1.2, so whoever
wrote glibc-2.1 deserves a medal. 
 
Cheers,
Alex
--
 /_/\  Legalise cannabis now! 
( o.o ) Grow some cannabis today!
 > ^ <  Peace, Love, Unity and Respect to all.

http://www.tahallah.demon.co.uk - Now back in the United Kingdom!

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On 07 Apr 1999 10:33:55 +0200, Andreas Schwab wrote:
>Zack Weinberg <zack@rabi.columbia.edu> writes:
>
>|> Using an explicit -lc is always recommended, but it has nothing to do
>|> with the above - it just ensures that the dynamic linker has a full
>|> dependency graph for the shared libraries.
>
>Note that you don't have to specify -lc explicitly, the gcc driver already
>does this by default, even with -shared, since gcc 2.8.1.

Not on all platforms - possibly only Linux.

zw

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Wed, 07 Apr 1999 02:57:47 -0600, Jeffrey A Law wrote:
>
>  In message < 199904070035.UAA17069@blastula.phys.columbia.edu >you write:
>  > That's strange.  That patch should theoretically make shared libs
>  > compiled by egcs act like shared libs compiled by gcc.
>But that's not necessarily good!  gcc2's handling of shared libraries is
>bad in a variety of ways.  Most notably the lack of backwards compatible
>EH interfaces

`act like' in the sense of `compatible ABI'.  The EH support should
still be present, just hiding.

>  > The correct binary interface for libc 2.0 is the one you get if you
>  > compile all purely C libraries with gcc 2.7.  Programs that expect
>  > something else are broken, sorry.  (There's no reason to prefer this
>  > rule except that that's what all the major distributions shipped.)
>No, the problem was soemone recompiled a library previously compiled with
>egcs-1.1.1 with egcs-1.1.2 and applications started breaking due to the
>weak handling.  At least that was my impression.  And that's horrible.

From one point of view, the libraries compiled with egcs-1.1.1 were at
fault, because they violated the (de facto) ABI.  This doesn't get us
any points for customer service though.

Leaving the patch out causes a different set of people problems,
namely the ones that want to compile stuff with egcs-1.1.x and have it
port over to systems with gcc 2.7-compiled libc.

I don't believe there is a right answer.  Mark understands the issues
better, maybe he has one.

zw

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904071214.OAA13954@landau.wins.uva.nl >you write:
  > I think that you mean "__register_frame" instead of
  > "__register_frame_info".
Yea.  Minor details :-)


  >    Against our recommendations, the interface was removed from gcc2.8.
  > 
  > The "__register_frame" interface was never included in any official
  > FSF release of gcc before 2.8.
I'm well aware of that.  It had been the interface that had been in the gcc
tree for nearly 2 years though.  It was changed about 2 weeks before gcc-2.8
went out the door and just after egcs-1.0 had been released.


  > Back then, egcs was still
  > "experimental" which was probably the reason why Kenner didn't want to
  > include the (broken) "__register_frame" interface that was part of
  > egcs 1.0 in gcc 2.8.
There were other reasons.  Some technical, but mostly political.


  > However, compatibility for the old "__register_frame" was included in
  > gcc 2.8.1.  This has not been noticed by a lot of people since,
  > where the "__register_frame_info" and "__register_frame" functions are
  > part of the same object module in egcs's libgcc.a, they are in different
  > modules in gcc's libgcc.a. This was exactly why 
I didn't know they'd included it in gcc-2.8.1.   Good.  However keeping them
in different modules is still not necessarily right, particularly when one
considers shared libraries and dynamically loaded code.

  > However, this issue is irrelevant for the current
  > "__register_frame_info & shared library compatibility" discussion.
  > The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
  > and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
  > indeed that egcs 1.1.2 now puts only a weak reference to
  > "__register_frame_info" in crtstuff.  This means that "du moment" one
  > recompiles a shared library that previously contained
  > "__register_frame_info" it probably does not anymore, and programs
  > linked with that library stop running.
Yup.  And I consider this a horrible breakage.  I trusted a couple of folks
with more experience in this area to guide the decision to include that
patch.  They made a mistake.  We need to rectify it.

  > Now, things get really complicated because of the egcs/Linux
  > ``releases''.  In an attempt to fix problems the author has messed
  > around with the way "__register_frame_info" is used in the varied
  > egcs-1.1.1/Linux ``releases''.  This might mean that some people that
  > use these versions will experience the same problems, but report that
  > they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
  > prevent that this happens again for egcs-1.1.2, and discourage the
  > author to release any "fixed" version of egcs-1.1.2.
Or fix this (and only this) problem and make egcs-1.1.3.

  > It is probably not a bad idea to revert the weak reference to
  > "__register_frame_info" patch in egcs-1.1.2.  It may result in larger
  > executables with glibc 2.0, but at least we get rid of the shared
  > library compatibility problem.  For glibc 2.1 things shouldn't matter
  > since "__register_frame_info" is exported from libc, which by default
  > every program and shared library is linked against.
jeff

Re: __register_frame_info & shared library compatibility

[Mark Kettenis]

   From: Jeffrey A Law (law@cygnus.com)

   Back in 1997 we tried _very_ hard to convince Richard Kenner and Richard
   Stallman to keep the old "__register_frame_info" interface in libgcc.a for
   gcc2 for precisely this problem.

I think that you mean "__register_frame" instead of
"__register_frame_info".

   Against our recommendations, the interface was removed from gcc2.8.

The "__register_frame" interface was never included in any official
FSF release of gcc before 2.8.  Back then, egcs was still
"experimental" which was probably the reason why Kenner didn't want to
include the (broken) "__register_frame" interface that was part of
egcs 1.0 in gcc 2.8.  However, compatibility for the old
"__register_frame" was included in gcc 2.8.1.  This has not been
noticed by a lot of people since, where the "__register_frame_info"
and "__register_frame" functions are part of the same object module in
egcs's libgcc.a, they are in different modules in gcc's libgcc.a.
This was exactly why 

However, this issue is irrelevant for the current
"__register_frame_info & shared library compatibility" discussion.
The way "__register_frame_info" is referenced in 2.8 <= gcc <= 2.8.1
and 1.0.3 <= egcs <= 1.1.1 is in principle the same.  The problem is
indeed that egcs 1.1.2 now puts only a weak reference to
"__register_frame_info" in crtstuff.  This means that "du moment" one
recompiles a shared library that previously contained
"__register_frame_info" it probably does not anymore, and programs
linked with that library stop running.

Now, things get really complicated because of the egcs/Linux
``releases''.  In an attempt to fix problems the author has messed
around with the way "__register_frame_info" is used in the varied
egcs-1.1.1/Linux ``releases''.  This might mean that some people that
use these versions will experience the same problems, but report that
they're using egcs-1.1.1 instead of egcs-1.1.2.  IMHO we have to
prevent that this happens again for egcs-1.1.2, and discourage the
author to release any "fixed" version of egcs-1.1.2.

It is probably not a bad idea to revert the weak reference to
"__register_frame_info" patch in egcs-1.1.2.  It may result in larger
executables with glibc 2.0, but at least we get rid of the shared
library compatibility problem.  For glibc 2.1 things shouldn't matter
since "__register_frame_info" is exported from libc, which by default
every program and shared library is linked against.

Mark

P.S. I haven't tested this, but using

   "-Wl,-u __register_frame_info"

when rebuilding a shared library with egcs-1.1.2 (or egcs-1.1.1/Linux)
will probably avoid the problems discussed in this thread.

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904070035.UAA17069@blastula.phys.columbia.edu >you write:
  > That's strange.  That patch should theoretically make shared libs
  > compiled by egcs act like shared libs compiled by gcc.
But that's not necessarily good!  gcc2's handling of shared libraries is
bad in a variety of ways.  Most notably the lack of backwards compatible
EH interfaces

  > The correct binary interface for libc 2.0 is the one you get if you
  > compile all purely C libraries with gcc 2.7.  Programs that expect
  > something else are broken, sorry.  (There's no reason to prefer this
  > rule except that that's what all the major distributions shipped.)
No, the problem was soemone recompiled a library previously compiled with
egcs-1.1.1 with egcs-1.1.2 and applications started breaking due to the
weak handling.  At least that was my impression.  And that's horrible.

jeff

Re: __register_frame_info & shared library compatibility

[Andreas Schwab]

Zack Weinberg <zack@rabi.columbia.edu> writes:

|> Using an explicit -lc is always recommended, but it has nothing to do
|> with the above - it just ensures that the dynamic linker has a full
|> dependency graph for the shared libraries.

Note that you don't have to specify -lc explicitly, the gcc driver already
does this by default, even with -shared, since gcc 2.8.1.

-- 
Andreas Schwab                                      "And now for something
schwab@issan.cs.uni-dortmund.de                      completely different"
schwab@gnu.org

Re: __register_frame_info & shared library compatibility

[H.J. Lu]

> 
> The correct binary interface for libc 2.0 is the one you get if you
> compile all purely C libraries with gcc 2.7.  Programs that expect

That is why we have a build machine for RedHat 5.2 with no changes to
libraries and compilers. All our binaries for RedHat 5.2 are compiled
on the build machine.

> something else are broken, sorry.  (There's no reason to prefer this
> rule except that that's what all the major distributions shipped.)
> 

You can use egcs 1.1.2 if you upgrade your glibc 2.0 to glibc 2.0
1998-12-11 or above via CVS. That should make all C shared libraries
compatible with the old binaries.


-- 
H.J. Lu (hjl@gnu.org)

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 06 Apr 1999 17:46:23 -0600, Jeffrey A Law wrote:
>
>  In message < 199904062115.RAA15222@blastula.phys.columbia.edu >you write:
>  > The best suggestion I can make is:  Get glibc 2.1 and compile it with
>  > egcs.  When you do that, glibc absorbs and re-exports the runtime
>  > functions that are causing the problem.  Everything else will pick
>  > them up from there instead of libgcc.a.  You'll have to recompile any
>  > libraries which are currently exporting __register_frame_info.  You
>  > will not be able to run binaries built in this environment on
>  > non-glibc-2.1 systems.
>  > 
>  > You can not do this with glibc 2.0 because then you lose binary
>  > compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
>  > as you describe).
>This points out the terrible problem with the patch to make the *_frame_info
>functions weakly referenced in crtstuff.
>
>We're already getting reports of programs that no longer run because of this
>change.  We may have to revert it.

That's strange.  That patch should theoretically make shared libs
compiled by egcs act like shared libs compiled by gcc.

The correct binary interface for libc 2.0 is the one you get if you
compile all purely C libraries with gcc 2.7.  Programs that expect
something else are broken, sorry.  (There's no reason to prefer this
rule except that that's what all the major distributions shipped.)

zw

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 19990406170202.J29505@moebius.dartmouth.edu >you write:
  >  Object      Soname           Compiler
  >  ------      ------           --------
  >  glibc 2.0   libc.so.6        gcc 2.7.2.3
  >  gtk+ 1.2    libgtk-1.2.so.0  gcc 2.7.2.3
  >  randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)
  > 
  > Now, as I understand it, the purpose of sonames is to manage binary
  > compatibility of shared objects. Any file providing a soname
  > (libgtk-1.2.so.0, for example) should be compatible with all other files
  > that ever provided that soname. If somebody breaks that invariant, then my
  > system supposedly dies in flames.
  > 
  > As far as I can tell, breaking sonames is (1) incorrect from a technical
  > perspective, (2) disasterous for users and (3) really unprofessional.
  > 
  > Now, let's pretend that my vendor issues some updated gtk+ packages. These
  > are a bug fix release--and don't change any APIs--so they still use the
  > soname 'libgtk-1.2.so.0'. However, my vendor has chosen to compile this
  > library with egcs:
  > 
  >  Object      Soname           Compiler
  >  ------      ------           --------
  >  glibc 2.0   libc.so.6        gcc 2.7.2.3
  >  gtk+ 1.2    libgtk-1.2.so.0  egcs-1.0.3
  >  randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)
  > 
  > In theory, randomapp should still run. After all, ld.so can still find
  > libraries with the appropriate sonames. But when I try to run randomapp,
  > things get ugly:
  > 
  >   undefined symbol: __register_frame_info
Yup.  As Zack and others have mentioned, this is a well known problem.

Back in 1997 we tried _very_ hard to convince Richard Kenner and Richard
Stallman to keep the old "__register_frame_info" interface in libgcc.a for
gcc2 for precisely this problem.

Against our recommendations, the interface was removed from gcc2.8.

jeff

Re: __register_frame_info & shared library compatibility

[Jeffrey A Law]

  In message < 199904062115.RAA15222@blastula.phys.columbia.edu >you write:
  > The best suggestion I can make is:  Get glibc 2.1 and compile it with
  > egcs.  When you do that, glibc absorbs and re-exports the runtime
  > functions that are causing the problem.  Everything else will pick
  > them up from there instead of libgcc.a.  You'll have to recompile any
  > libraries which are currently exporting __register_frame_info.  You
  > will not be able to run binaries built in this environment on
  > non-glibc-2.1 systems.
  > 
  > You can not do this with glibc 2.0 because then you lose binary
  > compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
  > as you describe).
This points out the terrible problem with the patch to make the *_frame_info
functions weakly referenced in crtstuff.

We're already getting reports of programs that no longer run because of this
change.  We may have to revert it.



jeff

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Tue, Apr 06, 1999 at 06:51:23PM -0400, Zack Weinberg wrote:
> On Tue, 6 Apr 1999 18:41:17 -0400, Eric Kidd wrote:
> >I help maintain a compiler that's hard to bootstrap. Currently, I must be
> >able to compile glibc-2.0/gcc-2.7 binaries on my system. So my upgrade
> >options are very limited. :-(
> 
> Can you give details?  Maybe I can help disentangle it.

I build glibc2 binaries and RedHat 5.2 RPMS for the Gwydion Dylan compiler:

  http://www.gwydiondylan.org/

It generates ANSI C code, which we then run through a system-dependent
compiler (normally GCC). This works better than you might expect, because
the optimizer is aggressive enough to generate reasonable C code.

The public releases of the compiler link against the C library dynamically,
but use a statically-linked Dylan runtime. So we're pretty well insulated
from many common problems. But I still need to support glibc 2.0 for all
the RedHat 5.x users out there.

> Unfortunately, 2.0 and 2.1 do not coexist well.  You would almost need a
> complete chroot environment to retain the ability to build 2.0 binaries.
> Better use two machines...

I've got diskspace to spare. Is it possible to install a chrooted RedHat or
Debian build environment, given enough patience?

> You will definitely lose if you have any references to stdin/out/err
> inside the library.  If you don't, you might survive, but you might
> not.  Best advice is `try it and see'.

OK, but existing *.a files should be safe, since they haven't been linked
yet? If so, our users are OK--none of our stdio code is in shared
libraries.
 
> > * If you compile glibc with egcs, you can often use egcs-compiled shared
> >   libraries with gcc-compiled applications.
> 
> Yes - and if it doesn't work, that's a bug.

Good. So there's hope. :-) My many thanks for helping me sort through this
problem; I'll pass your advice along to other folks.

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 6 Apr 1999 18:41:17 -0400, Eric Kidd wrote:
>On Tue, Apr 06, 1999 at 06:15:49PM -0400, Zack Weinberg wrote:
>> Make sure you have a current binutils (2.9.1.0.2x).  You may want to
>> upgrade to kernel 2.2; glibc 2.1 wants to use the new features (it does
>> fall back gracefully) and you won't have to keep gcc 2.7 around.
>
>I help maintain a compiler that's hard to bootstrap. Currently, I must be
>able to compile glibc-2.0/gcc-2.7 binaries on my system. So my upgrade
>options are very limited. :-(

Can you give details?  Maybe I can help disentangle it.

Unfortunately, 2.0 and 2.1 do not coexist well.  You would almost need
a complete chroot environment to retain the ability to build 2.0
binaries.  Better use two machines...

>> Any library that uses stdio internally may need to be recompiled in
>> order to work with binaries built with glibc 2.1.  libstdc++,
>> libncurses, libslang, and libglib are known to need this.
>
>Oooh. We've got a library which uses stdio in a big way. What qualifies as
>"using stdio internally"? Where can I find documentation on this, so I
>don't screw over our users?

You will definitely lose if you have any references to stdin/out/err
inside the library.  If you don't, you might survive, but you might
not.  Best advice is `try it and see'.

Executables are unaffected - a program that uses your library does not
need to be recompiled to migrate to 2.1, although the library may.

>> glibc 2.1 can't be compiled with gcc 2.7 (it miscompiles the math
>> library), so we make you use egcs, so __register_frame_info will
>> always be present in libc.so.6.  2.0 could be compiled with gcc 2.7,
>> and most distributions shipped it that way.
>
>Let me try to summarize the rules in a nutshell:
>
> * glibc 2.1 is always compiled with egcs

Technically you can use gcc 2.8 too.  I don't recommend it.

> * glibc 2.0 can be compiled with egcs, but is normally compiled with gcc

Correct.

> * If you compile glibc with gcc, you must use -lc when compiling shared
>   libraries with egcs. Otherwise, gcc-compiled applications will die
>   horrible, screaming deaths.

If you compile glibc 2.0 with gcc, you must also compile all other
libraries with gcc.  Otherwise, egcs-compiled applications may die
horribly.  It probably does not matter which compiler you use to
compile other libraries on a 2.1 system, but I would use egcs just to
be safe.

Using an explicit -lc is always recommended, but it has nothing to do
with the above - it just ensures that the dynamic linker has a full
dependency graph for the shared libraries.  If you have two shared
libraries and libA uses functions in libB, link libA with -lB too.

> * If you compile glibc with egcs, you can often use egcs-compiled shared
>   libraries with gcc-compiled applications.

Yes - and if it doesn't work, that's a bug.

zw

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Tue, Apr 06, 1999 at 06:15:49PM -0400, Zack Weinberg wrote:
> Make sure you have a current binutils (2.9.1.0.2x).  You may want to
> upgrade to kernel 2.2; glibc 2.1 wants to use the new features (it does
> fall back gracefully) and you won't have to keep gcc 2.7 around.

I help maintain a compiler that's hard to bootstrap. Currently, I must be
able to compile glibc-2.0/gcc-2.7 binaries on my system. So my upgrade
options are very limited. :-(
 
> Any library that uses stdio internally may need to be recompiled in
> order to work with binaries built with glibc 2.1.  libstdc++,
> libncurses, libslang, and libglib are known to need this.

Oooh. We've got a library which uses stdio in a big way. What qualifies as
"using stdio internally"? Where can I find documentation on this, so I
don't screw over our users?
 
> You can't use binaries or libraries compiled against 2.1 on a system that
> has libc 2.0.  Hopefully everyone will be using 2.1 soon and this won't
> be a problem.

OK, this is a reasonably normal (and expected) constraint. Understood.

> glibc 2.1 can't be compiled with gcc 2.7 (it miscompiles the math
> library), so we make you use egcs, so __register_frame_info will
> always be present in libc.so.6.  2.0 could be compiled with gcc 2.7,
> and most distributions shipped it that way.

Let me try to summarize the rules in a nutshell:

 * glibc 2.1 is always compiled with egcs
 * glibc 2.0 can be compiled with egcs, but is normally compiled with gcc
 * If you compile glibc with gcc, you must use -lc when compiling shared
   libraries with egcs. Otherwise, gcc-compiled applications will die
   horrible, screaming deaths.
 * If you compile glibc with egcs, you can often use egcs-compiled shared
   libraries with gcc-compiled applications.

Am I getting close?

> >Is there any hope that Red Hat and other distribution vendors will get
> >this right, or will I have to spend a week or two cleaning up the mess?
> 
> I know Debian's got it right, I think Redhat does too.  SuSE is having
> problems, but will hopefully straighten out.  I don't know about the
> others.

I really, really hope that Red Hat doesn't screw this up. The most recent
Gnome RPMS out of RHAD Labs have lots of issues in this area.

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 6 Apr 1999 17:54:09 -0400, Eric Kidd wrote:
>On Tue, Apr 06, 1999 at 05:15:31PM -0400, Zack Weinberg wrote:
>> The best suggestion I can make is: Get glibc 2.1 and compile it with
>> egcs.  When you do that, glibc absorbs and re-exports the runtime
>> functions that are causing the problem.  Everything else will pick them
>> up from there instead of libgcc.a.  You'll have to recompile any
>> libraries which are currently exporting __register_frame_info.  You will
>> not be able to run binaries built in this environment on non-glibc-2.1
>> systems.
>
>OK, this is starting to make sense. If __register_frame_info gets linked
>into glibc-2.1, then any library linked against glibc will pick up the new
>runtime functions? This seems like a fairly safe workaround, assuming that
>everything using shared libraries links dynamically against glibc.

Yes.  To make sure it works, when you link shared libraries, put an
explicit -lc on the command line.  It is wise to use the same compiler
for all the libraries, but it should work to use gcc 2.7 to compile
executables.  (Modulo all the bugs in gcc 2.7...)

Make sure you have a current binutils (2.9.1.0.2x).  You may want to
upgrade to kernel 2.2; glibc 2.1 wants to use the new features (it
does fall back gracefully) and you won't have to keep gcc 2.7 around.

>Does glibc 2.1 provide the soname libc.so.6? Does your solution somehow
>rely on this fact, or are you advising me to break my system into two
>parts: glibc 2.0 with gcc, and glibc 2.1 with egcs?

glibc 2.1 has the same soname as 2.0.  It is backward compatible as
long as your program does not use internal functions such as __write.
For some reason the most common mistake people make in this area is
to go groveling through the internals of the dynamic linker.  If you
get errors about symbols named _dl_somethingorother, that's the
problem.

Any library that uses stdio internally may need to be recompiled in
order to work with binaries built with glibc 2.1.  libstdc++,
libncurses, libslang, and libglib are known to need this.

Some C++ programs built with 2.0 break mysteriously when run on a 2.1
system.  This is due to bugs in the stdio backward compatibility
code.  We think we've fixed them all in 2.1.1, but there *is* always
one more bug :)

You can't use binaries or libraries compiled against 2.1 on a system
that has libc 2.0.  Hopefully everyone will be using 2.1 soon and this
won't be a problem.

>How does this solution affect pre-existing applications that were compiled
>with egcs and linked against a glibc 2.0 that was compiled with gcc?

Programs should work.  Libraries might not.

>> You can not do this with glibc 2.0 because then you lose binary
>> compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
>> as you describe).
>
>I still don't understand something. Why should linking
>__register_frame_info into glibc 2.1 solve the problem, but linking it into
>glibc 2.0 *cause* the problem?

glibc 2.1 can't be compiled with gcc 2.7 (it miscompiles the math
library), so we make you use egcs, so __register_frame_info will
always be present in libc.so.6.  2.0 could be compiled with gcc 2.7,
and most distributions shipped it that way.

>Is there any hope that Red Hat and other distribution vendors will get this
>right, or will I have to spend a week or two cleaning up the mess?

I know Debian's got it right, I think Redhat does too.  SuSE is having
problems, but will hopefully straighten out.  I don't know about the
others.

zw

Re: __register_frame_info & shared library compatibility

[Eric Kidd]

On Tue, Apr 06, 1999 at 05:15:31PM -0400, Zack Weinberg wrote:
> This is a well known problem to which there is, unfortunately, no
> clean solution.

I was afraid of this. :-( Unless the distribution vendors are very careful,
migrating from gcc to egcs will cause as many problems as the libc5 to
glibc2 upgrade. Ugh.
 
> The best suggestion I can make is: Get glibc 2.1 and compile it with
> egcs.  When you do that, glibc absorbs and re-exports the runtime
> functions that are causing the problem.  Everything else will pick them
> up from there instead of libgcc.a.  You'll have to recompile any
> libraries which are currently exporting __register_frame_info.  You will
> not be able to run binaries built in this environment on non-glibc-2.1
> systems.

OK, this is starting to make sense. If __register_frame_info gets linked
into glibc-2.1, then any library linked against glibc will pick up the new
runtime functions? This seems like a fairly safe workaround, assuming that
everything using shared libraries links dynamically against glibc.

Does glibc 2.1 provide the soname libc.so.6? Does your solution somehow
rely on this fact, or are you advising me to break my system into two
parts: glibc 2.0 with gcc, and glibc 2.1 with egcs?

How does this solution affect pre-existing applications that were compiled
with egcs and linked against a glibc 2.0 that was compiled with gcc?

> You can not do this with glibc 2.0 because then you lose binary
> compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
> as you describe).

I still don't understand something. Why should linking
__register_frame_info into glibc 2.1 solve the problem, but linking it into
glibc 2.0 *cause* the problem?

Is there any hope that Red Hat and other distribution vendors will get this
right, or will I have to spend a week or two cleaning up the mess?

Cheers,
Eric

Re: __register_frame_info & shared library compatibility

[Zack Weinberg]

On Tue, 6 Apr 1999 17:02:02 -0400, Eric Kidd wrote:
>First, some background. I run Red Hat Linux 5.2, and build binaries for a
>couple of projects involving shared libraries. I also maintain and upgrade
>a number of Linux systems with locally compiled software.
>
>egcs is starting to cause problems for me--or more precisely, the way
>people *use* egcs is starting to cause problems for me. Let's start with
>the latest problem I've encountered:
[...]
>In theory, randomapp should still run. After all, ld.so can still find
>libraries with the appropriate sonames. But when I try to run randomapp,
>things get ugly:
>
>  undefined symbol: __register_frame_info
>
>A little bit of research discovers something nasty: egcs and gcc have
>incompatible runtimes, and ld.so can't deal with this gracefully. When you
>compile a program with egcs, it pulls in a runtime providing
>__register_frame_info. When you compile an application with gcc 2.7, it
>pulls in an older runtime that lacks this symbol.

This is a well known problem to which there is, unfortunately, no
clean solution.

The best suggestion I can make is:  Get glibc 2.1 and compile it with
egcs.  When you do that, glibc absorbs and re-exports the runtime
functions that are causing the problem.  Everything else will pick
them up from there instead of libgcc.a.  You'll have to recompile any
libraries which are currently exporting __register_frame_info.  You
will not be able to run binaries built in this environment on
non-glibc-2.1 systems.

You can not do this with glibc 2.0 because then you lose binary
compatibility with glibc 2.0 compiled by gcc 2.7 (exact same problem
as you describe).

If your program and libraries are purely C, you can compile
*everything* with -fno-exceptions, which prevents the offending
functions from being linked in the first place.  You must not do this
if you care about C++.  Even if a library is written in C, it has to
use these functions to be compatible with C++ programs.  [I personally
think it's a mistake to use C++ for anything, but YMMV.]

zw

__register_frame_info & shared library compatibility

[Eric Kidd]

First, some background. I run Red Hat Linux 5.2, and build binaries for a
couple of projects involving shared libraries. I also maintain and upgrade
a number of Linux systems with locally compiled software.

egcs is starting to cause problems for me--or more precisely, the way
people *use* egcs is starting to cause problems for me. Let's start with
the latest problem I've encountered:

On Tue, Apr 06, 1999 at 11:23:23AM -0400, Owen Taylor wrote:
> Not true. As has been explained a couple of times here before, these
> RPM's are linked against glibc2.0. However, they are compiled against
> egcs. (The version of egcs shipped with 5.2, in fact)
>
> The consequence of the second thing is that you can't use those libraries
> with applications compiled with gcc, so if you want to compile other
> GNOME applications against these libraries, you'll temporarily have to
> make egcs your default compiler.

So here's the scenario, as best I understand it. My hypothetical system
initially contains:

 Object      Soname           Compiler
 ------      ------           --------
 glibc 2.0   libc.so.6        gcc 2.7.2.3
 gtk+ 1.2    libgtk-1.2.so.0  gcc 2.7.2.3
 randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)

Now, as I understand it, the purpose of sonames is to manage binary
compatibility of shared objects. Any file providing a soname
(libgtk-1.2.so.0, for example) should be compatible with all other files
that ever provided that soname. If somebody breaks that invariant, then my
system supposedly dies in flames.

As far as I can tell, breaking sonames is (1) incorrect from a technical
perspective, (2) disasterous for users and (3) really unprofessional.

Now, let's pretend that my vendor issues some updated gtk+ packages. These
are a bug fix release--and don't change any APIs--so they still use the
soname 'libgtk-1.2.so.0'. However, my vendor has chosen to compile this
library with egcs:

 Object      Soname           Compiler
 ------      ------           --------
 glibc 2.0   libc.so.6        gcc 2.7.2.3
 gtk+ 1.2    libgtk-1.2.so.0  egcs-1.0.3
 randomapp   N/A              gcc 2.7.2.3 (linked against glibc and gtk+)

In theory, randomapp should still run. After all, ld.so can still find
libraries with the appropriate sonames. But when I try to run randomapp,
things get ugly:

  undefined symbol: __register_frame_info

A little bit of research discovers something nasty: egcs and gcc have
incompatible runtimes, and ld.so can't deal with this gracefully. When you
compile a program with egcs, it pulls in a runtime providing
__register_frame_info. When you compile an application with gcc 2.7, it
pulls in an older runtime that lacks this symbol.

This apparently means that you can't use an egcs-compiled shared library
with a gcc-compiled application. Or am I missing something?

If my understanding is correct, however, my hypothetical system is now
broken. The bugfix upgrade to gtk+ broke binary compatibility for the
soname 'libgtk+-1.2.so.0', with potentially disasterous consequences.

Three questions for the audience:

 1) Am I badly confused? (I often am.)
 2) What happens if I recompile glibc 2.0 with ecgs and install it under
    the soname 'libc.so.6'?
 3) Will any of this cause bad problems in the future, and if so, how
    can these problems be prevented?

Thank you for your time!

Cheers,
Eric