forestalling GNU incompatibility - proposal for binary relative dynamic linking

forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

hey all,

Forgive the crosspost in advance, but I had an idea that touched many 
areas, and would need input from multiple groups associated with the 
gnu build chain, and perhaps the kernel itself.

After spending *two weeks* on various ways of building glibc, 
I'm convinced that the gnu/linux toolchain is in great danger of 
losing interoperability.

The main problem is that the glibc's supplied with each commercial 
system are *heavily* patched. My Suse 9.2 system has a rpm for 
glibc with fifty patches.  Fifty patches! 

Fifty patches which make the SuSE glibc binarily incompatible 
with the redhat, and so on.  And everything is incompatible
with the vanilla flavor.

All this makes me very skeptical that I'm ever going to get 
to a 'standard' out of the box glibc build. I build a standard 
glibc, and then all the supplied programs that come with SuSE 
break. And each vendor has the same problem - new that they are 
stuck with a heavily patched glibc, what chance do *they* have 
of getting back to a standard with the need to have old programs 
still work with the upgrades?









What IMO is desperately needed is a backwards compatibility 
hack or hacks. And of course the will of the different linux 
distribution providers to migrate back to the standard gnu 
toolchain.

Below is such a hack that lets providers do this, and 
solves quite a few problems in the process - the basic problem 
being that one currently can't get from a nonstandard glibc to
a standard one without quite a bit of pain - for migrating back 
breaks all the legacy binaries out there.


What is needed is the ability to reference multiple versions of the 
glibc WITHOUT changing my environment to do so.


Ie: If I set up my LD_LIBRARY_PATH to reference:

	setenv LD_LIBRARY_PATH /system/path/to/libc:.....


then the SuSE executables work fine but my new executables break, 
and if I set up my LD_LIBRARY_PATH to reference:

	setenv LD_LIBRARY_PATH /new/path/to/libc:.....

then my new executables work, but my *old* executables break.


What I'd like to do is be able to set up my LD_LIBRARY_PATH 
so that I can reference it from the point of view of the 
*executable*:

	setenv LD_LIBRARY_PATH   "*/../lib:....."




Here, read "* == full path of dirname of executable".

So if gcc was installed in say, "/opt/tools/bin/gcc", 
LD_LIBRARY_PATH would become at runtime "/opt/tools/bin/../lib" 
or "/opt/tools/lib". And hence if I had a libc.so installed 
there, it would pick it up and use it. 



Anyways, I have no idea whether or not this idea is being considered
or has been considered in the past, but AFAICT it would save me the 
trouble of having hundreds of wrapper scripts. And I would like to 
get an idea of what adding something like this to the gnu toolset would 
require. Discussion is welcome..


Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

(
    ps -

    just to forestall a particular objection that was pointed out to me 
    - simply use chroot - I don't want a chroot environment. I want to 
    use both old binaries and new binaries seamlessly.

    Ed
)

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Mike Frysinger]

On Monday 24 January 2005 05:24 pm, Edward Peschko wrote:
> After spending *two weeks* on various ways of building glibc,
> I'm convinced that the gnu/linux toolchain is in great danger of
> losing interoperability.

sounds like what you want is already being tackled by OSDL and their Binary 
Regression Testing group ...
http://groups.osdl.org/apps/group_public/workgroup.php?wg_abbrev=binary_sig
http://www.osdl.org/docs/isv_issues_and_binary_testing.pdf
-mike

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Richard Henderson]

On Mon, Jan 24, 2005 at 02:24:49PM -0800, Edward Peschko wrote:
> What I'd like to do is be able to set up my LD_LIBRARY_PATH 
> so that I can reference it from the point of view of the 
> *executable*:
> 
> 	setenv LD_LIBRARY_PATH   "*/../lib:....."
> 
> Here, read "* == full path of dirname of executable".

See -Wl,-rpath,'$ORIGIN/../lib/'


r~

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 06:08:52PM -0500, Mike Frysinger wrote:
> On Monday 24 January 2005 05:24 pm, Edward Peschko wrote:
> > After spending *two weeks* on various ways of building glibc,
> > I'm convinced that the gnu/linux toolchain is in great danger of
> > losing interoperability.
> 
> sounds like what you want is already being tackled by OSDL and their Binary 
> Regression Testing group ...
> http://groups.osdl.org/apps/group_public/workgroup.php?wg_abbrev=binary_sig
> http://www.osdl.org/docs/isv_issues_and_binary_testing.pdf
> -mike

well of course the osdl is going to focus on this, but they need tools
and functionality to do it correctly.. 

In particular, the statement 'Vendor lock-in (at any level) is not 
desirable' is false - there *is* vendor lock-in, and the suggestion 
of relative pathing for LD_LIBRARY_PATH is just one way to migrate 
back to doing things the right way.


Distributions are basically hoist by their own petard - they need to 
support old legacy executables, which have nonstandard glibc's. And since
they need to support legacy executables in the past, they need 
to support them in the future.


What I'd envision is that the distributions split basically into two: 
executables using the old style glibc/libraries, and executables using the 
new, standard glibcs. There would be two paths,


	/usr/bin

and

	/usr/standard/bin


The first directory would contain old executables which haven't been ported to
the standard glibc. The second would contain executables that have. The relative
pathing in LD_LIBRARY_PATH would insure that each tree used the correct libraries.


In the process of making /usr/standard/bin, linux vendors would need
to make their rpms both path-neutral and build clean. However, they would
*not* need to hold up their release process until everything is ported - 
they could pick and choose which applications were the most important to 
port. Ultimately, /usr/standard/bin would go away, and be moved back to 
/usr/bin, and perhaps the cycle could begin again, with upgrades going into 
a new /usr/standard/bin.

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 03:10:47PM -0800, Richard Henderson wrote:
> On Mon, Jan 24, 2005 at 02:24:49PM -0800, Edward Peschko wrote:
> > What I'd like to do is be able to set up my LD_LIBRARY_PATH 
> > so that I can reference it from the point of view of the 
> > *executable*:
> > 
> > 	setenv LD_LIBRARY_PATH   "*/../lib:....."
> > 
> > Here, read "* == full path of dirname of executable".
> 
> See -Wl,-rpath,'$ORIGIN/../lib/'
> 
> 
> r~

cool.. any chance for some syntactic sugar so me (and other 
users/vendors) wouldn't need to change any of their build scripts 
and compilation processes?

The only thing I would see as a drawback would be backwards compatibility,
but how often do people have directories named '*'?

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Richard Henderson]

On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> cool.. any chance for some syntactic sugar so me (and other 
> users/vendors) wouldn't need to change any of their build scripts 
> and compilation processes?

Uh, like what?  That's about as simple as you can get.


r~

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 03:38:49PM -0800, Richard Henderson wrote:
> On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> > cool.. any chance for some syntactic sugar so me (and other 
> > users/vendors) wouldn't need to change any of their build scripts 
> > and compilation processes?
> 
> Uh, like what?  That's about as simple as you can get.
> 
> 
> r~

I don't understand. 

Which is simpler, changing an environmental variable, or adding extra 
CFLAGS to every single compile and recompiling?

In addition, in your --rpath example, the relative pathing is hardcoded
into the executable, wheras with "*" you could modify the runtime behavior
of the executable at runtime. I suppose you could change this with chrpath,
but why bother? What if you want to test out two versions of relative
libraries side by side? 

And in any case, I'm not even sure if you can change the runtime path 
to something longer than what currently exists in the executable using
chrunpath.


And finally, certain programs (glibc, for example) seem to get into to 
trouble (ie: not compile) when you use --rpath flags. So, what's the 
issue with "*"?

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Daniel Jacobowitz]

On Mon, Jan 24, 2005 at 03:53:11PM -0800, Edward Peschko wrote:
> On Mon, Jan 24, 2005 at 03:38:49PM -0800, Richard Henderson wrote:
> > On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> > > cool.. any chance for some syntactic sugar so me (and other 
> > > users/vendors) wouldn't need to change any of their build scripts 
> > > and compilation processes?
> > 
> > Uh, like what?  That's about as simple as you can get.
> > 
> > 
> > r~
> 
> I don't understand. 
> 
> Which is simpler, changing an environmental variable, or adding extra 
> CFLAGS to every single compile and recompiling?
> 
> In addition, in your --rpath example, the relative pathing is hardcoded
> into the executable, wheras with "*" you could modify the runtime behavior
> of the executable at runtime. I suppose you could change this with chrpath,
> but why bother? What if you want to test out two versions of relative
> libraries side by side? 

You might want to take a look at Richard's suggestion again.  The
string '$ORIGIN' gets hardcoded into the binary and handled by the
dynamic linker.

But really, RPATH is a good solution to almost no problems.

-- 
Daniel Jacobowitz

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 07:29:39PM -0500, Daniel Jacobowitz wrote:
> On Mon, Jan 24, 2005 at 03:53:11PM -0800, Edward Peschko wrote:
> > On Mon, Jan 24, 2005 at 03:38:49PM -0800, Richard Henderson wrote:
> > > On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> > > > cool.. any chance for some syntactic sugar so me (and other 
> > > > users/vendors) wouldn't need to change any of their build scripts 
> > > > and compilation processes?
> > > 
> > > Uh, like what?  That's about as simple as you can get.
> > > 
> > > 
> > > r~
> > 
> > I don't understand. 
> > 
> > Which is simpler, changing an environmental variable, or adding extra 
> > CFLAGS to every single compile and recompiling?
> > 
> > In addition, in your --rpath example, the relative pathing is hardcoded
> > into the executable, wheras with "*" you could modify the runtime behavior
> > of the executable at runtime. I suppose you could change this with chrpath,
> > but why bother? What if you want to test out two versions of relative
> > libraries side by side? 
> 


> You might want to take a look at Richard's suggestion again.  The
> string '$ORIGIN' gets hardcoded into the binary and handled by the
> dynamic linker.

Ok, mea culpa, I've been talking faster than my brain is working.. 
True enough, I see that now. But with that solution you still have the 
issues with changing your build processes and a recompile. Plus it isn't
exactly the simplest solution out there.

Anyways, I'm toning this down a notch. I wasn't sure at first which 
group it would belong to, but it looks like it would be involving gcc 
and libc so I'll limit it to there.

> But really, RPATH is a good solution to almost no problems.

A fair enough statement to make, but why? AFAICT, it would allow me to 
have two truly separate trees and two truly separate libc.sos simultaneously.
Given how configure works, the lib corresponding to a bin for a given system 
is almost always equal to "*/../lib"..

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Mike Stump]

On Jan 26, 2005, at 8:21 PM, Edward Peschko wrote:

> On Thu, Jan 27, 2005 at 05:06:34AM +0100, Marcin Dalecki wrote:
>> Uninformed that you are you are misguiding people to waist their time.
>> You have no clue. There is no "official" glibc release
>> out there remotely compatible with kernel 2.6 and gcc 3.4. Thus there
>> isn't
>> anything out there SuSE could be incompatible with.
> man... where did that piece of venom come from? I've got to ask you a
> question - if there is 'no standard', how can LSB is the answer?

Off topic flame wars don't belong here, please, go find another sandbox 
to play it.  This list for gcc flame wars only.  :-)

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Alfred M. Szmidt]

   What I'm fighting with is a true clean bootstrap of it in a cross
   OS build of the whole tool-chain.

You gotta be one hell of a smart guy then, cause I do it all the time
from GNU/Linux to GNU, and OpenBSD to GNU.  And since I'm lazy, I have
a recipie that does that for me.

   > And even if it wouldn't compile with 3.4, you can always use a
   > older version of gcc, and still be able to not introduce any
   > incompatibilities.

   You are joking? Aren't you? Again this is another idea you spread
   which results only in a waste of time. I wish you a lot of fun with
   NPTL and old GCC versions.

You should compile things with what they were tested with unless you
are prepeared to fix things, which you are obviously not.  But you
don't get this is because you are such a smart guy who knows
everything.


Please go troll with else, since this will be my last message to you;
and proboly this thread as a whole.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Marcin Dalecki]

On 2005-01-27, at 17:17, Alfred M. Szmidt wrote:

>> So point it out to the SuSE folks, because it isn't the problem
>> of glibc or gcc if SuSE applies patches that make things
>> incompatible.
>
>    Uninformed that you are you are misguiding people to waist their
>    time.  You have no clue. There is no "official" glibc release out
>    there remotely compatible with kernel 2.6 and gcc 3.4. Thus there
>    isn't anything out there SuSE could be incompatible with.
>
> Yup, uninformed, misguided and no clue whatsoever.  That is why I have
> glibc 2.3.x nicely working with gcc 3.4 without introducing any silly
> incompatibilities, and you don't.

I have not a problem with cross Linux to Linux build of this piece. 
What I'm
fighting with is a true clean bootstrap of it in a cross OS build of the
whole tool-chain.

> And even if it wouldn't compile
> with 3.4, you can always use a older version of gcc, and still be able
> to not introduce any incompatibilities.

You are joking? Aren't you? Again this is another idea you spread which
results only in a waste of time. I wish you a lot of fun with NPTL and 
old GCC
versions.

> But obviously, I don't have a clue what so ever, so I will just go
> back to hacking.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Marcin Dalecki]

On 2005-01-27, at 15:35, Dan Kegel wrote:

> Jakub Jelinek wrote:
>> glibc-2.3.4 has been widely tested and is certainly stable.
>
> But not, alas, well announced:
> http://sources.redhat.com/ml/libc-announce/2005/
> is empty.

Yep. And glibc-2.3.4 is still ours and not days old.
However it still contains the same mistakes as ever for a true cross OS 
build...

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Alfred M. Szmidt]

   > So point it out to the SuSE folks, because it isn't the problem
   > of glibc or gcc if SuSE applies patches that make things
   > incompatible.

   Uninformed that you are you are misguiding people to waist their
   time.  You have no clue. There is no "official" glibc release out
   there remotely compatible with kernel 2.6 and gcc 3.4. Thus there
   isn't anything out there SuSE could be incompatible with.

Yup, uninformed, misguided and no clue whatsoever.  That is why I have
glibc 2.3.x nicely working with gcc 3.4 without introducing any silly
incompatibilities, and you don't.  And even if it wouldn't compile
with 3.4, you can always use a older version of gcc, and still be able
to not introduce any incompatibilities.

But obviously, I don't have a clue what so ever, so I will just go
back to hacking.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Tommy Vercetti]

On Thursday 27 January 2005 15:35, Dan Kegel wrote:
> Jakub Jelinek wrote:
> > glibc-2.3.4 has been widely tested and is certainly stable.
>
> But not, alas, well announced:
> http://sources.redhat.com/ml/libc-announce/2005/
http://sources.redhat.com/ml/libc-announce/2004/
no glibcs in 2004 either ??

-- 
Vercetti

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Dan Kegel]

Jakub Jelinek wrote:
> glibc-2.3.4 has been widely tested and is certainly stable.

But not, alas, well announced:
http://sources.redhat.com/ml/libc-announce/2005/
is empty.
- Dan


-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Rene Rebe]

Hi,

Jakub Jelinek wrote:

> Please back that up with facts, otherwise this claim is completely useless.
> Red Hat glibc is definitely compatible with Fedora one and to my knowledge
> other distributions when applying patches to glibc are just putting in backports
> of glibc TRUNK changes, at least if it involves symbol version changes.
> 
> glibc-2.3.4 has been widely tested and is certainly stable.

When we are at that ;-) Does it include 64bit user-space on sparc64 (did 
anyone ever actually used that)? I ask, because I get some segfaults and 
SIGBUS using glibc-2.3.4 and gcc-3.4.3 ... (still tracking them).

Yours,

-- 
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
             http://www.exactcode.de | http://www.exactcode.de/t2
             +49 (0)30  255 897 45

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Jakub Jelinek]

On Wed, Jan 26, 2005 at 08:21:16PM -0800, Edward Peschko wrote:
> SuSE is incompatible with redhat, debian, fedora, etc. Redhat is 
> incompatible with SuSE, debian, fedora, etc. And so forth. You need
> a lot of extra effort to make a cross-linux executable distribution.

Please back that up with facts, otherwise this claim is completely useless.
Red Hat glibc is definitely compatible with Fedora one and to my knowledge
other distributions when applying patches to glibc are just putting in backports
of glibc TRUNK changes, at least if it involves symbol version changes.

glibc-2.3.4 has been widely tested and is certainly stable.

	Jakub

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Thu, Jan 27, 2005 at 05:06:34AM +0100, Marcin Dalecki wrote:
> 
> On 2005-01-26, at 23:20, Alfred M. Szmidt wrote:
> 
> >>   With the same version of gcc, and glibc even - *without* the
> >>   patches that SuSE supplies.
> >>
> >>Then it is SuSE's fault that they apply patches that make things
> >>incompatible.  So go blame them instead.
> >
> >   I'm not 'blaming' anybody, just pointing out observations.
> >
> >So point it out to the SuSE folks, because it isn't the problem of
> >glibc or gcc if SuSE applies patches that make things incompatible.
> >
> Uninformed that you are you are misguiding people to waist their time.
> You have no clue. There is no "official" glibc release
> out there remotely compatible with kernel 2.6 and gcc 3.4. Thus there 
> isn't
> anything out there SuSE could be incompatible with.
> 

man... where did that piece of venom come from? I've got to ask you a 
question - if there is 'no standard', how can LSB is the answer?

SuSE is incompatible with redhat, debian, fedora, etc. Redhat is 
incompatible with SuSE, debian, fedora, etc. And so forth. You need
a lot of extra effort to make a cross-linux executable distribution.

As for official releases, the fact that there is a glibc-2.3.4 and 
glibc-2.3.3 itself is misleading thing. libc shouldn't give them milestone
numbers unless they consider them stable enough for use. People are not mind
readers; you shouldn't expect me to be one.

Ed

( ps - its 'waste', not 'waist'... )

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Marcin Dalecki]

On 2005-01-26, at 23:20, Alfred M. Szmidt wrote:

>>    With the same version of gcc, and glibc even - *without* the
>>    patches that SuSE supplies.
>>
>> Then it is SuSE's fault that they apply patches that make things
>> incompatible.  So go blame them instead.
>
>    I'm not 'blaming' anybody, just pointing out observations.
>
> So point it out to the SuSE folks, because it isn't the problem of
> glibc or gcc if SuSE applies patches that make things incompatible.
>
Uninformed that you are you are misguiding people to waist their time.
You have no clue. There is no "official" glibc release
out there remotely compatible with kernel 2.6 and gcc 3.4. Thus there 
isn't
anything out there SuSE could be incompatible with.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Ian Lance Taylor]

Edward Peschko <esp5@pge.com> writes:

> I was told that the two projects were basically almost 'joined at the hip'; 
> that in order to make changes in one, that you often needed to make 
> changes in the other.

That is an exaggeration.  In fact, gcc and glibc routinely change
without affecting the other.

> So, going back to my original point, is this true? In order to make 
> LD_LIBRARY_PATH rpath aware, would the changes be localized in glibc, 
> or would hooks be needed in binutils and in gcc? What work would be
> involved?

As I understood your original proposal, it involved changing how the
dynamic linker interprets LD_LIBRARY_PATH.  That change would be to
the dynamic linker, which is part of glibc.  No change to gcc would be
required at all.

If such a change is adopted into glibc, then there is a part of the
program linker which also looks at LD_LIBRARY_PATH and should probably
also be changed.  This is not a very important part of the linker,
though, and mainly exists to give better warnings.  The linker is, of
course, part of the binutils, not part of gcc.

Ian

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Dan Kegel]

Edward Peschko wrote:
> All this started because I needed to change some system headers for files
> distributed by glibc.  I don't have exclusive rights on the machine, nor do 
> I have root on the machine

No problem, then!  Simply build a private copy of glibc along with gcc
(perhaps using the infamous crosstool script).  Build your apps
using that toolchain.  Use cross-compiling techniques to avoid
referencing any of the main system headers.  I do this all the time;
it's a piece of cake.  As long as you build with an older glibc
than the target system, you should be good to go.

You might want to take further discussion to the crossgcc mailing list
(see http://sources.redhat.com/ml/crossgcc/ ), since the folks
there are familiar with these techniques.
- Dan

-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Wed, Jan 26, 2005 at 02:21:20PM -0800, Joe Buck wrote:
> On Wed, Jan 26, 2005 at 01:51:02PM -0800, Edward Peschko wrote:
> > On Wed, Jan 26, 2005 at 01:54:02PM -0800, Joe Buck wrote:
> > > This discussion is off-topic for the gcc list.  Whether there are problems
> > > with glibc or the LSB is a matter for other lists, not for gcc@gcc.gnu.org.
> > 
> > You are correct, in that this particular complaint is not directly on-topic, 
> > and for that I apologize.
> > 
> > However, IMO the thread is definitely not off topic - my original post was 
> > an idea to amend the gcc toolchain to make running two separate glibcs
> > in parallel possible. And since glibc and gcc seem to be joined at the hip,
> > this is IMO definitely on topic.
> 
> If you identify something in gcc that needs to change in gcc to make this
> easier, yes.  But it seems to me that the work is all on the glibc end of
> things.

I was told that the two projects were basically almost 'joined at the hip'; 
that in order to make changes in one, that you often needed to make 
changes in the other.

So, going back to my original point, is this true? In order to make 
LD_LIBRARY_PATH rpath aware, would the changes be localized in glibc, 
or would hooks be needed in binutils and in gcc? What work would be
involved?

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Jakub Jelinek]

On Wed, Jan 26, 2005 at 01:15:49PM -0800, Edward Peschko wrote:
> On Tue, Jan 25, 2005 at 09:49:47PM -0800, Dan Kegel wrote:
> 
> > Edward Peschko wrote:
> > >I build glibc out of the box, ie: no patches on a SuSE machine, same
> > >version as the OS (glibc-2.3.3), using latest gcc(gcc-3.4.3)
> > >get segmentation faults every time when I try to run it against 
> > >system binaries. 
> > 
> > Why are you replacing the system glibc?  That's never a good idea.
> 
> I'm not replacing the system glibc - all I am doing is making the point
> that *if* you compile glibc and run it against system binaries, the 
> system binaries break.

How do you do that?
If you are using ld.so from one glibc build with libc.so from a completely
different one, there is no guarantee that will work.  There is a private
interface between them that keeps changing.  But glibc as the whole set
ought to be backwards compatible for programs and shared libraries.

	Jakub

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Randy.Dunlap]

Alfred M. Szmidt wrote:
>    >    With the same version of gcc, and glibc even - *without* the
>    >    patches that SuSE supplies.
>    > 
>    > Then it is SuSE's fault that they apply patches that make things
>    > incompatible.  So go blame them instead.
> 
>    I'm not 'blaming' anybody, just pointing out observations.
> 
> So point it out to the SuSE folks, because it isn't the problem of
> glibc or gcc if SuSE applies patches that make things incompatible.
> 
>    >    If your argument is that building glibc isn't for the faint of
>    >    heart and should be left to the experts, my question is why?
>    > 
>    > Because it is a integral part of the system, it can break your
>    > system badly, it can make your system unbootable (due to a broken
>    > /lib/ld.so.1 or whatever), etc, etc.
> 
>    Well of course, but my point is that's exactly why you should be
>    able to run two glibcs simultaneously on the same system.
> 
> You can run two glibcs of different versions simultaneously on the
> same system, that SuSE happens to break things so that x.y.z-SuSE
> doesn't work with programs linked against x.y.z or the other way
> around is their fault.
> 
> Cheers.

I agree that this is mostly OT for gcc, but I don't see how this is
just a SUSE problem.  IOW, I doubt that they are the only distro
that modifies glibc.  However, this would be better addressed via
LSB or LCC or some OSDL projects IMO.

-- 
~Randy

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Joe Buck]

On Wed, Jan 26, 2005 at 01:51:02PM -0800, Edward Peschko wrote:
> On Wed, Jan 26, 2005 at 01:54:02PM -0800, Joe Buck wrote:
> > This discussion is off-topic for the gcc list.  Whether there are problems
> > with glibc or the LSB is a matter for other lists, not for gcc@gcc.gnu.org.
> 
> You are correct, in that this particular complaint is not directly on-topic, 
> and for that I apologize.
> 
> However, IMO the thread is definitely not off topic - my original post was 
> an idea to amend the gcc toolchain to make running two separate glibcs
> in parallel possible. And since glibc and gcc seem to be joined at the hip,
> this is IMO definitely on topic.

If you identify something in gcc that needs to change in gcc to make this
easier, yes.  But it seems to me that the work is all on the glibc end of
things.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Alfred M. Szmidt]

   >    With the same version of gcc, and glibc even - *without* the
   >    patches that SuSE supplies.
   > 
   > Then it is SuSE's fault that they apply patches that make things
   > incompatible.  So go blame them instead.

   I'm not 'blaming' anybody, just pointing out observations.

So point it out to the SuSE folks, because it isn't the problem of
glibc or gcc if SuSE applies patches that make things incompatible.

   >    If your argument is that building glibc isn't for the faint of
   >    heart and should be left to the experts, my question is why?
   > 
   > Because it is a integral part of the system, it can break your
   > system badly, it can make your system unbootable (due to a broken
   > /lib/ld.so.1 or whatever), etc, etc.

   Well of course, but my point is that's exactly why you should be
   able to run two glibcs simultaneously on the same system.

You can run two glibcs of different versions simultaneously on the
same system, that SuSE happens to break things so that x.y.z-SuSE
doesn't work with programs linked against x.y.z or the other way
around is their fault.

Cheers.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Wed, Jan 26, 2005 at 10:55:57PM +0100, Alfred M. Szmidt wrote:
>    With the same version of gcc, and glibc even - *without* the
>    patches that SuSE supplies.
> 
> Then it is SuSE's fault that they apply patches that make things
> incompatible.  So go blame them instead.

I'm not 'blaming' anybody, just pointing out observations.

>    If your argument is that building glibc isn't for the faint of
>    heart and should be left to the experts, my question is why?
> 
> Because it is a integral part of the system, it can break your system
> badly, it can make your system unbootable (due to a broken
> /lib/ld.so.1 or whatever), etc, etc.

Well of course, but my point is that's exactly why you should be 
able to run two glibcs simultaneously on the same system.

Can I ask why this is such a hot-button issue?

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Wed, Jan 26, 2005 at 01:54:02PM -0800, Joe Buck wrote:
> On Wed, Jan 26, 2005 at 01:15:49PM -0800, Edward Peschko wrote:
> > On Tue, Jan 25, 2005 at 09:49:47PM -0800, Dan Kegel wrote:
> > > Why are you replacing the system glibc?  That's never a good idea.
> > 
> > I'm not replacing the system glibc - all I am doing is making the point
> > that *if* you compile glibc and run it against system binaries, the 
> > system binaries break.
> 
> This discussion is off-topic for the gcc list.  Whether there are problems
> with glibc or the LSB is a matter for other lists, not for gcc@gcc.gnu.org.


You are correct, in that this particular complaint is not directly on-topic, 
and for that I apologize.

However, IMO the thread is definitely not off topic - my original post was 
an idea to amend the gcc toolchain to make running two separate glibcs
in parallel possible. And since glibc and gcc seem to be joined at the hip,
this is IMO definitely on topic.

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Alfred M. Szmidt]

   With the same version of gcc, and glibc even - *without* the
   patches that SuSE supplies.

Then it is SuSE's fault that they apply patches that make things
incompatible.  So go blame them instead.

   If your argument is that building glibc isn't for the faint of
   heart and should be left to the experts, my question is why?

Because it is a integral part of the system, it can break your system
badly, it can make your system unbootable (due to a broken
/lib/ld.so.1 or whatever), etc, etc.

Cheers!

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Joe Buck]

On Wed, Jan 26, 2005 at 01:15:49PM -0800, Edward Peschko wrote:
> On Tue, Jan 25, 2005 at 09:49:47PM -0800, Dan Kegel wrote:
> > Why are you replacing the system glibc?  That's never a good idea.
> 
> I'm not replacing the system glibc - all I am doing is making the point
> that *if* you compile glibc and run it against system binaries, the 
> system binaries break.

This discussion is off-topic for the gcc list.  Whether there are problems
with glibc or the LSB is a matter for other lists, not for gcc@gcc.gnu.org.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Tue, Jan 25, 2005 at 09:49:47PM -0800, Dan Kegel wrote:

> Edward Peschko wrote:
> >I build glibc out of the box, ie: no patches on a SuSE machine, same
> >version as the OS (glibc-2.3.3), using latest gcc(gcc-3.4.3)
> >get segmentation faults every time when I try to run it against 
> >system binaries. 
> 
> Why are you replacing the system glibc?  That's never a good idea.

I'm not replacing the system glibc - all I am doing is making the point
that *if* you compile glibc and run it against system binaries, the 
system binaries break.


With the same version of gcc, and glibc even - *without* the patches
that SuSE supplies. 

Hence, the system glibc and vanilla glibc are binarily incompatible. If the
LSB's goal is to prevent this, it isn't working - at least not in this
simple experiment.


> >First, the real world situation is that I compile my own programs, 
> >libraries,
> >etc - sort of a 'distribution within a distribution' which I maintain
> >and make portable to win32,solaris,linux,os390, etc. I don't need
> >root because I can configure the prefix, and everything is run out
> >of source control. I can also control environments and migrate 
> >executables from one environment to the next (dev -- test -- prod)
> >after they go through enough testing.  This setup allows me to 
> >version in new binaries very easily, because I can test them 
> >simultaneously, and promote them when they are ready.. and thie problem 
> >arose when I tried to add glibc to the list of packages I support.
> 
> There's the problem: you're trying to distribute a copy of
> glibc.  Don't do that.  Glibc is an integral part of the system.

I'm not trying to 'distribute it'; I'm trying to make a vanilla version.
If I need to go from system to system and compile glibc for that system, I'll 
do so. I just want the freedom to build everything from source control.

All this started because I needed to change some system headers for files
distributed by glibc. I don't have exclusive rights on the machine, nor do 
I have root on the machine. And I don't want to make the hack of copying 
just modified headers. I'd rather keep everything clean and separate - the best 
way to do this would be to compile my own distribution, which I attempted to do.
I suggested the 'rpath-like' functionality in the LD_LIBRARY_PATH as a way
to have two glibcs coexist on the machine.


If your argument is that building glibc isn't for the faint of heart
and should be left to the experts, my question is why? *Why* should it be left
for the experts, and why should it be a black art? Isn't it the fact that it 
is a black art retard its advancement, and make progress go slower? Isn't 
it the fact that it makes systems stuck in the mud because they can't 
easily experiment with it?

I'm not trying to be inflammatory here, I'm just noting what I see, and the
results of my experiment; I can't think of one good reason why building a 
glibc should be so difficult.

And surely there should be a method to build a separate glibc *outside 
the system glibc* - if anything to make sure that the developer 
doesn't trash his system unintentionally in playing around with it. And 
glibc just might be as fragile as it is because it doesn't get enough
hands experimenting with it.

Ed

(
    ps - for those who are saying the LSB is the only answer, here's 
    another experiment. I just took the 'ls' executable from my SuSE 
    machine (2.4.18) and tried to run it on a Fedora box (2.4.21-27). 
    I got a 'missing library (libselinux.so.1). Seeing SElinux wasn't 
    installed on that machine, I transferred it over, and got a 
    'no versioning info' error from libacl, and finally a segfault.

    Now, if 'ls' doesn't have a chance of being cross-distribution
    compatible, what chance do other, more complicated programs have?

    If I understand the LSB right - that it is a bunch of policies that
    vendors are meant to follow but don't *have* to follow in putting
    together their distributions, and that programmers have to follow
    in writing them (and if they don't do so things will break in 
    mysterious ways), well then its chances of true success are fairly
    slim.  It forces developers to have multiple distribs handy *just
    to be sure* their apps will run on all of them. 
)

(
    pps - admittedly I could be reading the LSB's intent wrong. 
    Feel free to correct me.
)

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Alfred M. Szmidt]

   The glibc setup system is deeply broken by leak of clue and
   concept. I stand by those words.

It is nice that you stand by the fact that you are a ignorant fool
without a remote clue about glibc, but could you please keep it this
off topic drivel to yourself?

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Marcin Dalecki]

On 2005-01-26, at 08:13, Marcus Meissner wrote:
>>
>> Well, that might be the case with your fifty patches, but it sure
>> isn't the case with mine..
>
> We try to be binary compatible with upstream glibc.

Just a question: What is the upstream glibc? I suppose you are talking
about the tarball found on ftp://ftp.fsf.org? Well that has not much in 
common
with what the vendors out there are using and distributing. It isn't
even remotely compatible. (NPTL anyone for example?) Please don't 
misguide
people to try using it on even remotely recent kernels or with remotely
GCC versions. It isn't going to work! Don't guide the people to waist 
their
time!

> Very likely you did something wrong, building glibc is not for
> the faint at heart.

No. The glibc setup system is deeply broken by leak of clue and 
concept. I stand
by those words. Those are not flames but just the adequate description 
of the
situation, It is looking for runtime properties of the host system. It 
greps/seds/awka
like an insane monkey through the installed system header hierarchy to 
decide what to
enable or not. Thus there is no sane way to just control in imperative 
way what it should
choose and what not. Usually the mistakes it does just slip silently 
through
your hands. But building glibc in a TRUE cross build environment, like 
building glibc on
MacOSX for Linux, will immediately reveal how contorted this all is.

A nice example how this will bite you even if working on Linux is the 
wait() system call.
It's implemented on some CPU targets by the kernel on some others there 
is just a
stub saying that it's invalid and on even different ones it is not 
there at all.
Depending on where you build and what you target, glibc setup will make 
different
mistakes, since it is looking at the system headers. Fortunately this 
syscall is
supposed to be obsolete, so those mistakes don't hurt. Building on 2.6 
kernel
system and on 2.4 kernel system will give you utterly different 
results. Why doesn't
this break even more frequently? Well simply:

1. Distros are bootstrapped on themself step by step apparently.
2. Nobody is using the new system calls, and Linus is careful to never 
remove old
    ones.
3. Well, point 2 isn't true. But nobody cares about Oracle breaking 
apart if moved to
    another distro or after upgrading ;-).

You think I sound too harsh? Then please be aware that a glibc build 
will spend
most of it's time in Makefile dependency generation onanism. Yes it 
takes it much
longer to decide what to do then actually doing it. Drepper found a way 
to make make
run slower then gcc, which as we all know is applying far more 
sophistication to
it's input then the make command. Or just try to use a true sed or awk 
instead of the ones
coming with glibc... It's a sad sad world...

At some point in time I have discovered a place where it was even doing 
"runtime setup", thus behaving different whatever it found a particular 
idiotic
useless broken interface file in /proc or not (stat("/proc/blah").
I discovered this by crafting an embedded system with a kernel which 
didn't contain this
particular crap in /proc. And suddenly threading did fall apart because 
glibc was thinking
it was running on a different kernel then it actually did. Nice and 
cute isn't it?
Never mind such "tricks" are scattered through the whole source code of 
glibc...

Nothing of the above is even remotely documented... Not that I would 
expect it after
looking in to the innards of this ugly beast.

<SARCASM>
Yes it is that BAD! It's like the 3rd LIBC for the next 1000 years...
Great promises, awful start, but then came Stalingrad and the rest we 
all know.
</SARCASM>

> The SUSE included glibcs are LSB compliant, so there is no need to
> migrate away from them.

And what does give LSB? LSB isn't even a near shoot from being a proper 
formal
specification (in the sense as first defined by Alfred Tarski),
with proper full coverage of functionality and so on. Formal 
verification anybody?
LSB "compliance" is just a marketing joke for good feelings. Not much 
more.
It shouldn't call itself a standard if taking the meaning of standard 
by word.

>> But there doesn't seem to be a good way to do this.  The best solution
>> someone has come up with so far is run stuff chrooted, which isn't
>> really a solution at all, because it doesn't allow for 
>> interoperability.
>
> Just create your own Linux distribution.

The entropy of kernel variants and libraries is in fact quite 
effectively
preventing the existence of a true vendor software market for Linux. I 
just don't
expect the situation to change. Vendors will simply stick to a single 
distro as
qualified to run their software for the foreseeable future. Small 
vendors will be
locked out because they can't afford to maintain releases for 
bazillions of
variants of systems out there. There will thus be only server 
applications or
trivial statically linked console apps for linux out there.

However embedded systems will be fine.
But you need a lot of expertise and suffering through Dreppers bad 
habits
to really make such a thing going... I really miss the times of the
very fine HJ Lu libc5 work. He did an awesome job! Yes it wasn't 
perfect.
But it wasn't artificially contorted. Hey at least we have
quite well working binutils nowadays. But Drepper? Give ma an
aspirin please. I remember the pain I suffered from it just too well...

PS. I have a full system cross build for divers intel/PowerPC/MIPS/ARM 
platforms
with gcc-3.4.3 (glibc or uClibc or newlib) and recent binutils running 
for myself
rebuildable from *scratch* without attention... No it isn't crosstool 
based.
And I'm not far away from making the cross environment fully functional 
on
MacOSX and Win32. Actually the only not working part here is indeed the 
libc of
shame aka glibc.

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Marcus Meissner]

[-- Attachment #1: Type: text/plain, Size: 2550 bytes --]

On Tue, Jan 25, 2005 at 11:56:06AM -0800, Edward Peschko wrote:
> On Mon, Jan 24, 2005 at 10:34:10PM -0800, Dan Kegel wrote:
> > Edward Peschko wrote:
> > >After spending *two weeks* on various ways of building glibc, 
> > >I'm convinced that the gnu/linux toolchain is in great danger of 
> > >losing interoperability.
> > >
> > >The main problem is that the glibc's supplied with each commercial 
> > >system are *heavily* patched. My Suse 9.2 system has a rpm for 
> > >glibc with fifty patches.  Fifty patches! 
> > >
> > >Fifty patches which make the SuSE glibc binarily incompatible 
> > >with the redhat, and so on.  And everything is incompatible
> > >with the vanilla flavor.
> > 
> > I can sympathize with you.  I've spent several months
> > of my life building gcc and glibc.  (That's why I wrote
> > http://kegel.com/crosstool, so fewer people would ever have to go
> > through that hell.)
> > 
> > But fifty patches doesn't neccessarily mean binary incompatibility.
> > http://kegel.com/crosstool/crosstool-0.28-rc37/patches/glibc-2.3.2/
> > contains about 45 patches, but as far as I know, none of them
> > cause incompatibilities; most are neccessary to get glibc-2.3.2
> > to *build*.
> 
> Well, that might be the case with your fifty patches, but it sure
> isn't the case with mine..

We try to be binary compatible with upstream glibc.
 
> I build glibc out of the box, ie: no patches on a SuSE machine, same
> version as the OS (glibc-2.3.3), using latest gcc(gcc-3.4.3)
>  get segmentation faults every time when I try to run it against 
> system binaries. I set LD_LIBRARY_PATH back, and the segfaults go away. 

Very likely you did something wrong, building glibc is not for
the faint at heart.

Without more exact debug information we cannot say.

> Second - as far as the LSB standard is concerned, migrating to it 
> from incompatible glibc's is a trick in itself; you need to support 
> the legacy applications whilst you are migrating.  

> And hence to facilitate supporting the LSB it makes a lot of sense 
> to be able to support 2 incompatible libcs on the same machine at the 
> same time.

The SUSE included glibcs are LSB compliant, so there is no need to
migrate away from them.
 
> But there doesn't seem to be a good way to do this.  The best solution 
> someone has come up with so far is run stuff chrooted, which isn't 
> really a solution at all, because it doesn't allow for interoperability.

Just create your own Linux distribution.

Ciao, Marcus

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Dan Kegel]

Edward Peschko wrote:
> I build glibc out of the box, ie: no patches on a SuSE machine, same
> version as the OS (glibc-2.3.3), using latest gcc(gcc-3.4.3)
> get segmentation faults every time when I try to run it against 
> system binaries. 

Why are you replacing the system glibc?  That's never a good idea.

> First, the real world situation is that I compile my own programs, libraries,
> etc - sort of a 'distribution within a distribution' which I maintain
> and make portable to win32,solaris,linux,os390, etc. I don't need
> root because I can configure the prefix, and everything is run out
> of source control. I can also control environments and migrate 
> executables from one environment to the next (dev -- test -- prod)
> after they go through enough testing.  This setup allows me to 
> version in new binaries very easily, because I can test them 
> simultaneously, and promote them when they are ready.. and thie problem 
> arose when I tried to add glibc to the list of packages I support.

There's the problem: you're trying to distribute a copy of
glibc.  Don't do that.  Glibc is an integral part of the system.

> Second - as far as the LSB standard is concerned, migrating to it 
> from incompatible glibc's is a trick in itself; you need to support 
> the legacy applications whilst you are migrating.  
> 
> And hence to facilitate supporting the LSB it makes a lot of sense 
> to be able to support 2 incompatible libcs on the same machine at the 
> same time.

No.  If you build your apps to obey the LSB, you don't need
a second libc.
- Dan

-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 10:34:10PM -0800, Dan Kegel wrote:
> Edward Peschko wrote:
> >After spending *two weeks* on various ways of building glibc, 
> >I'm convinced that the gnu/linux toolchain is in great danger of 
> >losing interoperability.
> >
> >The main problem is that the glibc's supplied with each commercial 
> >system are *heavily* patched. My Suse 9.2 system has a rpm for 
> >glibc with fifty patches.  Fifty patches! 
> >
> >Fifty patches which make the SuSE glibc binarily incompatible 
> >with the redhat, and so on.  And everything is incompatible
> >with the vanilla flavor.
> 
> I can sympathize with you.  I've spent several months
> of my life building gcc and glibc.  (That's why I wrote
> http://kegel.com/crosstool, so fewer people would ever have to go
> through that hell.)
> 
> But fifty patches doesn't neccessarily mean binary incompatibility.
> http://kegel.com/crosstool/crosstool-0.28-rc37/patches/glibc-2.3.2/
> contains about 45 patches, but as far as I know, none of them
> cause incompatibilities; most are neccessary to get glibc-2.3.2
> to *build*.

Well, that might be the case with your fifty patches, but it sure
isn't the case with mine..

I build glibc out of the box, ie: no patches on a SuSE machine, same
version as the OS (glibc-2.3.3), using latest gcc(gcc-3.4.3)
 get segmentation faults every time when I try to run it against 
system binaries. I set LD_LIBRARY_PATH back, and the segfaults go away. 



> Also, it's very much worth noting that LSB is designed to
> eliminate a largish class of binary incompatibilities;
> as time goes on, I expect it to actually be used for that purpose.
> It might be the best way to solve the problems you're
> thinking about (but I can't tell, since you haven't described
> the real-world situation involved).
> - Dan

First, the real world situation is that I compile my own programs, libraries,
etc - sort of a 'distribution within a distribution' which I maintain
and make portable to win32,solaris,linux,os390, etc. I don't need
root because I can configure the prefix, and everything is run out
of source control. I can also control environments and migrate 
executables from one environment to the next (dev -- test -- prod)
after they go through enough testing.  This setup allows me to 
version in new binaries very easily, because I can test them 
simultaneously, and promote them when they are ready.. and thie problem 
arose when I tried to add glibc to the list of packages I support.


Second - as far as the LSB standard is concerned, migrating to it 
from incompatible glibc's is a trick in itself; you need to support 
the legacy applications whilst you are migrating.  

And hence to facilitate supporting the LSB it makes a lot of sense 
to be able to support 2 incompatible libcs on the same machine at the 
same time.


But there doesn't seem to be a good way to do this.  The best solution 
someone has come up with so far is run stuff chrooted, which isn't 
really a solution at all, because it doesn't allow for interoperability.


So - there is work being done to be able to swap between two kernels, 
why not two glibcs? I'm open to other ideas aside from mine..


Ed


(
ps - as far as "* == path to binary" suggestion is concerned, the 
one potential drawback that was mentioned was that binaries that didn't
match "/bin/?$" as a regex would be vulnerable if they picked up ../lib
inadvertently.

I'm trying to think of a case where this is true..
)

re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Dan Kegel]

Edward Peschko wrote:
> After spending *two weeks* on various ways of building glibc, 
> I'm convinced that the gnu/linux toolchain is in great danger of 
> losing interoperability.
> 
> The main problem is that the glibc's supplied with each commercial 
> system are *heavily* patched. My Suse 9.2 system has a rpm for 
> glibc with fifty patches.  Fifty patches! 
> 
> Fifty patches which make the SuSE glibc binarily incompatible 
> with the redhat, and so on.  And everything is incompatible
> with the vanilla flavor.

I can sympathize with you.  I've spent several months
of my life building gcc and glibc.  (That's why I wrote
http://kegel.com/crosstool, so fewer people would ever have to go
through that hell.)

But fifty patches doesn't neccessarily mean binary incompatibility.
http://kegel.com/crosstool/crosstool-0.28-rc37/patches/glibc-2.3.2/
contains about 45 patches, but as far as I know, none of them
cause incompatibilities; most are neccessary to get glibc-2.3.2
to *build*.

It might be worth documenting at least one of the incompatibilities
you hint at.  What harm, in particular, have you found?
What use cases are involved?

Also, it's very much worth noting that LSB is designed to
eliminate a largish class of binary incompatibilities;
as time goes on, I expect it to actually be used for that purpose.
It might be the best way to solve the problems you're
thinking about (but I can't tell, since you haven't described
the real-world situation involved).
- Dan

-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html