Building _statically linked_ crosscompiler toolchain.

Building _statically linked_ crosscompiler toolchain.

[John Carter]

Ok, so I can build the latest gcc,binutils i686 linux hosted, embedded
sparc-elf newlib target crosscompiler tool chain.

Great.

Unfortunately some of the team (bless their smelly sockses) are on
different Linux distributions, older distributions etc. Thus different
shareable libraries.

ie. The tools don't run on many of my colleagues machines.

Obviously the answer is to statically link the tools.

The question is "What is The One True Way (or any blooming way) to
build a _completely_ statically linked gcc crosscompiler toolchain?"

The obvious way and several of the less obvious ways have failed
miserably and The Fine Manual has not been nearly as explicit as my
poor aching noddle requires.

Thank you for you patience and forebearance,



John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Re: Building _statically linked_ crosscompiler toolchain.

[David Daney]

John Carter wrote:
> Ok, so I can build the latest gcc,binutils i686 linux hosted, embedded
> sparc-elf newlib target crosscompiler tool chain.
> 
> Great.
> 
> Unfortunately some of the team (bless their smelly sockses) are on
> different Linux distributions, older distributions etc. Thus different
> shareable libraries.
> 
> ie. The tools don't run on many of my colleagues machines.
> 
> Obviously the answer is to statically link the tools.
> 
> The question is "What is The One True Way (or any blooming way) to
> build a _completely_ statically linked gcc crosscompiler toolchain?"
> 

glibc does not work very well statically linked.  So I don't think that 
is a good idea.

Perhaps you should build the toolchain on the oldest distribution that 
will have to be supported.  An alternative is to have a build for each 
incompatible host system.

David Daney

Re: Building _statically linked_ crosscompiler toolchain.

[Brian Dessent]

John Carter wrote:

> The question is "What is The One True Way (or any blooming way) to
> build a _completely_ statically linked gcc crosscompiler toolchain?"

I think you'll find that there is never a "One True Way" for anything
when dealing with toolchains.

I could be wrong, but if you configure with --disable-shared, that will
apply both to the toolchain as well as the target libraries, which is
certainly not what you want.  I.e. you don't want to deprive your target
machine of a shared libstdc++, libgcc, etc. just because of host machine
environment variances.

It sounds like attempting to make the entire toolchain static is a bit
of an overcompensation.  It's not like gcc and binutils have many
host-library dependencies.  Can't you just identify whatever it is that
is causing the problem and build that library static?  For example, if
you drop the gmp/mpfr sources into the gcc tree it will be built and
linked statically with gcc, rather than using the system copy which is
probably shared and which probably doesn't exist on most systems by
default.  And you can --disable-nls to remove any dependencies on
libiconv et al., although on linux systems that stuff should all be
handled by glibc rather than discrete libs.

If it's simply a matter of glibc symbol versioning then the solution is
simply to build on the oldest system you intend to support.

Brian

Re: Building _statically linked_ crosscompiler toolchain.

[John Carter]

On Thu, 7 Jun 2007, David Daney wrote:

> glibc does not work very well statically linked.  So I don't think that is a 
> good idea.

People keep saying that....but never saying in what way doesn't it work or why.

The only hint I have seen as to why that may be is Internationalized
Error messages may get screwed up. If that is so that is the very
least of my concerns since the team is all english.

> Perhaps you should build the toolchain on the oldest distribution that will 
> have to be supported.  An alternative is to have a build for each 
> incompatible host system.

So do something that takes several hours to build on my latest and
greatest and fastest 3.4Ghz 1Gb ram dual core system.....

And build it on every system in the team...

Including the bloke with the 100Mhz celeron  & 256 MB ram.

Hmm.

Hmm. One of the reasons for having a single "blessed" build of the
compiler is it is one less variable to check & account for when the
inevitable "Works for Joe, but Not For John" class of bugs arises.

Sigh! Why is this so hard?

Why have we taken a leap back into the dark ages where we cannot share
a user space program without either recompiling or having identical systems.



John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Re: Building _statically linked_ crosscompiler toolchain.

[John Carter]

On Thu, 7 Jun 2007, Brian Dessent wrote:

> John Carter wrote:
>
>> The question is "What is The One True Way (or any blooming way) to
>> build a _completely_ statically linked gcc crosscompiler toolchain?"
>
> I think you'll find that there is never a "One True Way" for anything
> when dealing with toolchains.

Sigh! Too Horribly True.


> I could be wrong, but if you configure with --disable-shared, that will
> apply both to the toolchain as well as the target libraries, which is
> certainly not what you want.  I.e. you don't want to deprive your target
> machine of a shared libstdc++, libgcc, etc. just because of host machine
> environment variances.

a. It certainly _is_ what I want... The target system is a tiny
    embedded device where everything _must_ be statically linked.

b. No, it doesn't solve the problem of sharing the built toolchain
    with my colleagues.

> It sounds like attempting to make the entire toolchain static is a bit
> of an overcompensation.  It's not like gcc and binutils have many
> host-library dependencies.

You only need one.

libc.

> If it's simply a matter of glibc symbol versioning then the solution is
> simply to build on the oldest system you intend to support.

ie. Build the biggest meanest package on the gnu planet on the oldest
slowest, smallest disk, ram system in the company. Sigh!

Even then, looking at the libc version numbers on the latest
greatest... I don't think that will work.

Hmm.

John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Re: Building _statically linked_ crosscompiler toolchain.

[Brian Dessent]

John Carter wrote:

> People keep saying that....but never saying in what way doesn't it work or why.

http://people.redhat.com/drepper/no_static_linking.html

> a. It certainly _is_ what I want... The target system is a tiny
>     embedded device where everything _must_ be statically linked.

It's hard to guess what you want when you didn't state it in your
original message.

> b. No, it doesn't solve the problem of sharing the built toolchain
>     with my colleagues.

Right, because --disable-shared does not mean statically link against
libc, it means don't build shared libraries.

> You only need one.
> 
> libc.

If you mean that differing libc versions can be a problem when
distributing binaries, then sure.

If you mean that gcc only needs libc, then you're not using a recent
version as gmp and mpfr have been host library requirements since 4.0
for gfortran and since 4.2 (or 4.1?) for the entire compiler.

> ie. Build the biggest meanest package on the gnu planet on the oldest
> slowest, smallest disk, ram system in the company. Sigh!

You can do it with a chroot on a fast machine.  You can do it with
Xen/Vmware/etc on a fast machine.  It's usually an infrequent event that
you need to rebuild the toolchain itself.

Besides, it has nothing to do with it being gcc or being a large
package, this is the general method to achieve binary compatibility for
any package on a wide range of systems.

> Even then, looking at the libc version numbers on the latest
> greatest... I don't think that will work.

That's not true.  Glibc symbol versioning goes to farily great lengths
to assure backwards binary compatibility, so don't write it off.  But
this means only that binaries compiled against an older version of the
library work correctly with a newer version, and never the reverse.

If really want to statically link the toolchain, you can start with:

path/to/configure --foo=bar ... LDFLAGS="-static"

I just tried this on gcc trunk with a C-only non-bootstrap native and it
worked fine, but you might have to make further adjustments.  Although
even after stripping cc1 is nearly 9MB so I can't imagine that this is
very efficient memory-wise or performance-wise.

Brian

Re: Building _statically linked_ crosscompiler toolchain.

[David Daney]

John Carter wrote:
> On Thu, 7 Jun 2007, David Daney wrote:
>
>> glibc does not work very well statically linked.  So I don't think 
>> that is a good idea.
>
> People keep saying that....but never saying in what way doesn't it 
> work or why.

Well perhaps we are all crazy.  You asked for advice, and you got some.  
If you want to learn for yourself go right ahead.  Try doing something 
like building everything with CFLAGS=-static (or you could temporarily 
remove *.so from your /usr/lib directory when building the toolchain).

>> Perhaps you should build the toolchain on the oldest distribution 
>> that will have to be supported.  An alternative is to have a build 
>> for each incompatible host system.
>
> So do something that takes several hours to build on my latest and
> greatest and fastest 3.4Ghz 1Gb ram dual core system.....
>
> And build it on every system in the team...
>
> Including the bloke with the 100Mhz celeron  & 256 MB ram.
>
No.  Just build on this system (it could take a day or two if you are 
serious about the specifications) and use the result everywhere.

> Hmm.
>
> Hmm. One of the reasons for having a single "blessed" build of the
> compiler is it is one less variable to check & account for when the
> inevitable "Works for Joe, but Not For John" class of bugs arises.
>
> Sigh! Why is this so hard?
>
Try compiling a program for Windows Vista and then run it on MS-DOS, 
Windows 3.1 and WindowsME.  What do you think would happen?

> Why have we taken a leap back into the dark ages where we cannot share
> a user space program without either recompiling or having identical 
> systems.
>
I have programs that I built on RedHat 7, that run fine on FC6 x86_64.  
Doing things the opposite way just does not work.

If you want to build something that will run most places install the 
oldest OS version you will have to support and build it there.  That 
usually works.  The fact is that most Linux distributions move forward 
fairly rapidly, but the major ones try to maintain binary compatibility 
with previous versions.

Re: Building _statically linked_ crosscompiler toolchain.

[John Carter]

On Thu, 7 Jun 2007, Brian Dessent wrote:

> John Carter wrote:
>
>> People keep saying that....but never saying in what way doesn't it work or why.
>
> http://people.redhat.com/drepper/no_static_linking.html


Ah! Thanks. Interesting....

Hmm, although.... not single one of his arguments is applicable to the
problem of distributing a cross compiler to a team of coworkers using
different versions of different distributions of Linux.

ie. In this case statically linking the tool chain _is_ the right solution.

On the otherhand, if there was a way of deploying...

$ ldd sparc-elf-gcc
         linux-gate.so.1 =>  (0xffffe000)
         libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e86000)
         /lib/ld-linux.so.2 (0xb7fdb000)

the relevant .so's _without_ creating gnarly conflicts with the
distro's resident versions that may be a solution.

Could I ship my versions of the .so's into a sub-directory of the gcc
package and use LD_PRELOAD or something to load those in preference to
the system ones?

> If really want to statically link the toolchain, you can start with:
>
> path/to/configure --foo=bar ... LDFLAGS="-static"
>
> I just tried this on gcc trunk with a C-only non-bootstrap native and it
> worked fine, but you might have to make further adjustments.  Although
> even after stripping cc1 is nearly 9MB so I can't imagine that this is
> very efficient memory-wise or performance-wise.


Hmm. I seem to remember that didn't work. ie. it compiled and linked
but the system libraries were still sharable. I think libtool requires
-all-static if you want the libc statically linked as well, but the
gcc build barfs if you try that.


John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Re: Building _statically linked_ crosscompiler toolchain.

[Brian Dessent]

John Carter wrote:

> ie. In this case statically linking the tool chain _is_ the right solution.

In your opinion.  Not in the opinion of others.

> On the otherhand, if there was a way of deploying...
> 
> $ ldd sparc-elf-gcc
>          linux-gate.so.1 =>  (0xffffe000)
>          libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e86000)
>          /lib/ld-linux.so.2 (0xb7fdb000)
> 
> the relevant .so's _without_ creating gnarly conflicts with the
> distro's resident versions that may be a solution.

Sure.  <http://gcc.gnu.org/ml/gcc-help/2006-07/msg00126.html>.  However,
I don't know what will happen when the compiler driver goes to invoke
the various subprocesses: cc1, cc1plus, collect2, as, ld, ...  You might
have to use such a wrapper for every binary in the toolchain, which
would be incredibly ugly.  At what point does this become more work than
just installing a vmware copy of an old distro?

> Hmm. I seem to remember that didn't work. ie. it compiled and linked
> but the system libraries were still sharable. I think libtool requires
> -all-static if you want the libc statically linked as well, but the
> gcc build barfs if you try that.

That doesn't make any sense.  For one thing, gcc itself doesn't use
libtool at all, it's only used for target libraries.  And since this is
a crosscompiler, target libraries wouldn't have anything to do with the
host libc anyway.  And secondly, LDFLAGS won't even be considered when
building target libraries, only binaries that run on the host; there is
a whole parallel set of variables (*FLAGS_FOR_TARGET IIRC) used for
building target code. 

Brian

Re: Building _statically linked_ crosscompiler toolchain.

[John Carter]

On Fri, 8 Jun 2007, John Carter wrote:

> On Thu, 7 Jun 2007, Brian Dessent wrote:
>
>> path/to/configure --foo=bar ... LDFLAGS="-static"
>> 
>> I just tried this on gcc trunk with a C-only non-bootstrap native and it
>> worked fine, but you might have to make further adjustments.  Although
>> even after stripping cc1 is nearly 9MB so I can't imagine that this is
>> very efficient memory-wise or performance-wise.


Just checked that over the weekend. Didn't work.

It worked in the sense the thing built, but still isn't statically linked to libc.



John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand