Re: Why is building a cross compiler "out-of-the-box" always broken?

Re: Why is building a cross compiler "out-of-the-box" always broken?

[NightStrike]

Stephen,

I have been working on the x86_64-pc-mingw32 toolchain with Kai Tietz
(Kai is the main person, I am doing much more learning than helping).
I put together a built script that requires no tweaking whatsoever of
any of the projects incorporated in the toolchain.  It is very
straightforward.  You can check it out here:

http://mingw-w64.svn.sourceforge.net/viewvc/*checkout*/mingw-w64/experimental/buildsystem/makebuildroot.sh?revision=56

The project itself is here:
https://sourceforge.net/projects/mingw-w64

If you make a new directory (like /tmp/rt) and change into it, then
run the script with the single argument "build", it will download
binutils, gcc, the crt, and the headers, then proceed to compile
everything using the most default set of options and create a
buildroot starting at the present working directory.

The options to binutils are:
--prefix=$PF --with-sysroot=$PF
--target=x86_64-pc-mingw32
--disable-nls

gcc only adds the option --enable-languages=c.

The disable-nls option is just there to make the build faster -- it is
not needed.  with-sysroot has replaced with-headers and with-includes,
and it allows for easy creation of buildroots.  I set the prefix equal
to the same thing because it works and I don't really understand it
enough to do differently.

The biggest difference between this and normal toolchains is the
absence of glibc.

Anyway, tell me what you think.

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Paolo Bonzini]

Rask Ingemann Lambertsen wrote:
> On Sun, Aug 19, 2007 at 10:09:25AM +0200, Paolo Bonzini wrote:
>>>   Thanks for reminding me to ping that patch.
>> Could you try moving this case statement
>>
>> +	case "$target" in
>> +	  i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
>> +	    libgloss_dir=i386
>> +	    ;;
>> +	  m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
>> +	    libgloss_dir=m68hc11
>> +	    ;;
>>
>> above in another "case $target in" statement?
> 
>    I don't understand what you want it to look like. Your explanation sounds
> to me like
> 
> 	case "$target" in
> 	  i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
> 	    libgloss_dir=i386
> 	    ;;
> 	  m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
> 	    libgloss_dir=m68hc11
> 	    ;;
> 	esac

Sorry, I meant "all the case statement".  There are many case "$target" 
statements and I would rather avoid trying to avoid one.  Setting 
libgloss_dir=${cpu} as a default can be done above the case "$target" 
you choose.

Paolo

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Rask Ingemann Lambertsen]

On Sun, Aug 19, 2007 at 10:09:25AM +0200, Paolo Bonzini wrote:
> 
> >   Thanks for reminding me to ping that patch.
> 
> Could you try moving this case statement
> 
> +	case "$target" in
> +	  i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
> +	    libgloss_dir=i386
> +	    ;;
> +	  m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
> +	    libgloss_dir=m68hc11
> +	    ;;
> 
> above in another "case $target in" statement?

   I don't understand what you want it to look like. Your explanation sounds
to me like

	case "$target" in
	  i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
	    libgloss_dir=i386
	    ;;
	  m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
	    libgloss_dir=m68hc11
	    ;;
	esac

	case "$target" in
	... rest of cases go here ...

	  *)
	    libgloss_dir=${cpu}
	    ;;
	esac

but then the default of the second case construct will override libgloss_dir
from the first one, if I'm not missing something here.

-- 
Rask Ingemann Lambertsen

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Daniel Jacobowitz]

On Mon, Aug 20, 2007 at 12:06:03PM +0300, Kai Ruottu wrote:
> If one tries to produce everything in the 'gcc' subdirectory, including 
> 'libgcc',
> then headers may be needed.  But if producing only the 'xgcc', 'cc1', 
> 'collect2'
> etc binaries for the host is enough, then nothing for the target will be 
> required.

This is part of the reason why libgcc no longer lives in the gcc
directory.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Kai Ruottu]

Segher Boessenkool wrote:
> The manual explicitly says you need to have target headers.  With
> all those --disable-XXX and especially --enable-sjlj-exceptions it
> all works fine without, though.
If one tries to produce everything in the 'gcc' subdirectory, including 
'libgcc',
then headers may be needed.  But if producing only the 'xgcc', 'cc1', 
'collect2'
etc binaries for the host is enough, then nothing for the target will be 
required.
Everything is totally dependent on how that "cross compiler" will  be 
defined....

Years ago in Berkeley university there was a system called "Nachos" which
required a "crosscompiler for DECstation/MIPS" but without any DEC's
target headers  or libraries and even without  any 'libgcc'.

So if one needs a "Nachos-like" crosscompiler for some target on one's host,
that could always succeed!  Only problems totally related to the $host could
be seen....

Meanwhile a cross GCC for AIX, HP-UX, and even for the quite common
Mac OS X and for the commercial "Enterprise Linux" distros and for many
other systems could be either impossible or very hard if no free access 
to the
target stuff (copyrighted prebuilt libraries) is provided.  And when one 
thinks
a cross GCC being something similar to the native GCC, with all the base C
libraries, all the extra termcap, curses, X11 etc. libraries...  Or when 
there is
not much or not enough support in the GNU binutils for the target.  For 
instance
a cross GCC for the MS's "Interix" aka "Services for Unix" would fail 
with the
GNU ld not working for this target :-(  So if one defines a "cross 
compiler" to
include the "as", "ld" etc. binutils and all the target libraries, then 
there can be
very serious problems, neither the GNU binutils, nor the GNU libc are 
supporting
all the targets GCC is seemingly "supporting"....  Someone recently 
mentioned
that the GNU binutils shouldn't work for AIX and quite surely there 
isn't any
free access to the  AIX C libraries...

Quite many crosscompiler builders don't have any clue about what GCC is and
what components are belonging to it...  Some will always mix producing a 
cross
GCC with producing a self-made Linux distro, where that GCC is only a tool
used to produce the final product, Linux :-(   These things just should 
be much
more clear in people's minds...


 

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Segher Boessenkool]

> Thanks, I git cloned buildall and it sure looks simple (now that you 
> have made it work) :-!

Good to see it is useful to more people than just me :-)

> Although, that combination of options was not exactly obvious at first 
> glance.

Heh.  Almost all of it is just "disable what we don't need".  OTOH,
that's not how I ended up with those options: I added one every time
something broke.

> I'll try it on my build system at work tomorrow and see what bites me 
> next.

What target(s) are you building?

> I expect that I am
> having old nightmares from when I was trying to do this under Solaris 
> and made it harder than
> I needed to on a Linux host.  Apparently, I'm not the only one who got 
> lead astray though.

The manual explicitly says you need to have target headers.  With
all those --disable-XXX and especially --enable-sjlj-exceptions it
all works fine without, though.


Let me know how it goes,


Segher

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Steve Kenton]

Thanks, I git cloned buildall and it sure looks simple (now that you 
have made it work) :-!
Although, that combination of options was not exactly obvious at first 
glance.  Time to re-read
all the config --help stuff.

I'll try it on my build system at work tomorrow and see what bites me 
next. I expect that I am
having old nightmares from when I was trying to do this under Solaris 
and made it harder than
I needed to on a Linux host.  Apparently, I'm not the only one who got 
lead astray though.

Thank again - Steve

Segher Boessenkool wrote:
>>> 4. Use the minimal cross-compiler produced in step 3 to configure 
>>> glibc and do "make install-headers" to get glibc headers. At this 
>>> point you should be able to delete the minimal cross-compiler, it's 
>>> job has been done.
>>>
>>> 5. Make distclean and then configure, make, and install a normal gcc 
>>> cross-compiler for the target using the glibc headers produced in 
>>> step 4.
>>>
>>> 6. Make distclean and then configure, make, and install the complete 
>>> glibc header and libraries etc. for the target.
>
> Sometimes you need to repeat 4,5,6 (but using the full toolchain
> produced in the previous run, not the minimal one obviously).
>
> It's a good sanity check anyway ;-)
>
>>> 1. Seems to be a good reliable source of sanitized headers for 
>>> recent kernels. I know of no problems with the current release.
>
> There still are problems so you're advised to use the separate kernel
> headers package instead.
>
>>> 2. Seems to build and work out-of-the-box using the headers from 
>>> step 1.  I know of no problems with current release.
>
> Binutils doesn't need any kernel headers AFAIK.
>
>>> 3. Figuring out the correct combination of --with-newlib, 
>>> --with-sysroot, and --with-headers (or --without, or =no etc.) to 
>>> get inhibit_libc set and then figuring out which of the myriad of 
>>> --with-this-that-and-the-other options can or must be disabled to 
>>> get a functional minimal compiler is a real pain in the neck.
>
> configure --verbose --help
>
> I posted the full set you currently need.
>
>>>   Further more, the make file may try (and fail) to make additional 
>>> things like crt0 in EXTRA_PARTS=
>
> Don't use --with-newlib if you don't want to build newlib.
>
>>> Right now it's a question of "wtf caused that to die?"
>
> Yeah, it's a fun voyage, isn't it :-)
>
>>> Also, should the inhibit_libc hack also disable things like the 
>>> unwind support that requires kernel headers?  That not scrictly 
>>> libc, but --without-headers sort of assume neither are present.
>
> Yeah, that's a good point.  I use --enable-sjlj-exceptions to
> avoid the need for those headers, but that's not enough for
> building ia64-linux (it _is_ enough for all other Linux targets).
>
>>> Some people advocate combining steps 3,4,5 by either cribbing a set 
>>> of headers from someone elses sysroot,
>
> That's the "normal" procedure for building a cross-toolchain, yes.
>
>>> or by fooling glibc configure in to producing them without having a 
>>> minimal cross compiler.
>
> You should only need this if you're porting to a new target.  And
> then this all is the least of your troubles ;-)
>
>>> However, it's probably not viable for someone trying to build 20 or 
>>> so tools chains so they can check for kernel build failure tree wide 
>>> etc.
>
> That's exactly what I'm doing.  Feel free to take a look at my scripts:
>
>     git://git.infradead.org/~segher/buildall.git
>
> or browse it at:
>
>     http://git.infradead.org/?p=users/segher/buildall.git
>
>>> And besides, it would sure be nice if gcc would build a 
>>> cross-compiler out-of-the-box if there is no problem with some other 
>>> part of the tool chain.  That does not seem to be the case for all 
>>> targets at this point.
>
> Bugs happen :-)
>
>>> So, that's the situation as I see it. How should things work in an 
>>> ideal world and where do we go from here? Or, are other people happy 
>>> enough with the current ecosystem and think nothing needs to change?
>
> It would be nice if things were a bit easier; maybe a single configure
> flag "--from-scratch" or something like that.  Someone needs to create
> that though.
>
>
> Segher

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Segher Boessenkool]

>> 4. Use the minimal cross-compiler produced in step 3 to configure 
>> glibc and do "make install-headers" to get glibc headers. At this 
>> point you should be able to delete the minimal cross-compiler, it's 
>> job has been done.
>>
>> 5. Make distclean and then configure, make, and install a normal gcc 
>> cross-compiler for the target using the glibc headers produced in 
>> step 4.
>>
>> 6. Make distclean and then configure, make, and install the complete 
>> glibc header and libraries etc. for the target.

Sometimes you need to repeat 4,5,6 (but using the full toolchain
produced in the previous run, not the minimal one obviously).

It's a good sanity check anyway ;-)

>> 1. Seems to be a good reliable source of sanitized headers for recent 
>> kernels. I know of no problems with the current release.

There still are problems so you're advised to use the separate kernel
headers package instead.

>> 2. Seems to build and work out-of-the-box using the headers from step 
>> 1.  I know of no problems with current release.

Binutils doesn't need any kernel headers AFAIK.

>> 3. Figuring out the correct combination of --with-newlib, 
>> --with-sysroot, and --with-headers (or --without, or =no etc.) to get 
>> inhibit_libc set and then figuring out which of the myriad of 
>> --with-this-that-and-the-other options can or must be disabled to get 
>> a functional minimal compiler is a real pain in the neck.

configure --verbose --help

I posted the full set you currently need.

>>   Further more, the make file may try (and fail) to make additional 
>> things like crt0 in EXTRA_PARTS=

Don't use --with-newlib if you don't want to build newlib.

>> Right now it's a question of "wtf caused that to die?"

Yeah, it's a fun voyage, isn't it :-)

>> Also, should the inhibit_libc hack also disable things like the 
>> unwind support that requires kernel headers?  That not scrictly libc, 
>> but --without-headers sort of assume neither are present.

Yeah, that's a good point.  I use --enable-sjlj-exceptions to
avoid the need for those headers, but that's not enough for
building ia64-linux (it _is_ enough for all other Linux targets).

>> Some people advocate combining steps 3,4,5 by either cribbing a set 
>> of headers from someone elses sysroot,

That's the "normal" procedure for building a cross-toolchain, yes.

>> or by fooling glibc configure in to producing them without having a 
>> minimal cross compiler.

You should only need this if you're porting to a new target.  And
then this all is the least of your troubles ;-)

>> However, it's probably not viable for someone trying to build 20 or 
>> so tools chains so they can check for kernel build failure tree wide 
>> etc.

That's exactly what I'm doing.  Feel free to take a look at my scripts:

	git://git.infradead.org/~segher/buildall.git

or browse it at:

	http://git.infradead.org/?p=users/segher/buildall.git

>> And besides, it would sure be nice if gcc would build a 
>> cross-compiler out-of-the-box if there is no problem with some other 
>> part of the tool chain.  That does not seem to be the case for all 
>> targets at this point.

Bugs happen :-)

>> So, that's the situation as I see it. How should things work in an 
>> ideal world and where do we go from here? Or, are other people happy 
>> enough with the current ecosystem and think nothing needs to change?

It would be nice if things were a bit easier; maybe a single configure
flag "--from-scratch" or something like that.  Someone needs to create
that though.


Segher

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Steve Kenton]

> OK, let me recap as I understand the discussion up to now, realizing 
> that this is the gcc list we can really only hope to fix things, if 
> needed, by making changes to the gcc project.  Other project will have 
> other priorities.
>
> It sounds like the consensus is that bootstrapping a cross-compiler 
> still requires the use of inhibit_libc and for a Linux target *should* 
> work like this. And, apparently is does work like this for some people 
> for some targets, just not for everyone for everything.
>
> 1. Configure the Linux kernel and "make headers_install".
>
> 2. Configure and make binutils using the headers produced in step 1.
>
> 3. Configure and make a minimal gcc cross-compiler for the target by 
> getting configure to set inhibit_libc and disabling any complicating 
> or conflicting options such as threads and multilib not strictly 
> needed to build the minimal compiler. This compiler should not be 
> generally used because of the restrictions placed on it during the 
> build process.
>
> 4. Use the minimal cross-compiler produced in step 3 to configure 
> glibc and do "make install-headers" to get glibc headers. At this 
> point you should be able to delete the minimal cross-compiler, it's 
> job has been done.
>
> 5. Make distclean and then configure, make, and install a normal gcc 
> cross-compiler for the target using the glibc headers produced in step 4.
>
> 6. Make distclean and then configure, make, and install the complete 
> glibc header and libraries etc. for the target.
>
> 7. Use the tools produced in steps 5 and 6 to do whatever 
> cross-compiling project you need them for.
>
> Now for some thoughts about this. I assume we are working with current 
> releases of the kernel and tool chain. There are lots of problems with 
> various older combinations of things, let's not go there please right now.
>
> 1. Seems to be a good reliable source of sanitized headers for recent 
> kernels. I know of no problems with the current release.
>
> 2. Seems to build and work out-of-the-box using the headers from step 
> 1.  I know of no problems with current release.
>
> 3. Figuring out the correct combination of --with-newlib, 
> --with-sysroot, and --with-headers (or --without, or =no etc.) to get 
> inhibit_libc set and then figuring out which of the myriad of 
> --with-this-that-and-the-other options can or must be disabled to get 
> a functional minimal compiler is a real pain in the neck.  Further 
> more, the make file may try (and fail) to make additional things like 
> crt0 in EXTRA_PARTS= which require the use of "make -k" and/or  manual 
> copying of the needed bits and pieces. It does not always work and the 
> failures seem to be in different places on different targets. Maybe 
> the make files need to be modified some depending on the configure 
> setting of inhibit_libc?
>
> If this is *supposed* to work, would it be worth while having a 
> --force-bootstrap or some such option so that configure could force 
> inhibit_libc and then short circuit lots of configuration stuff and 
> disable everything appropriate on a per-target basis so we can 
> reliably generate the minimal comiler needed for the next step?  With 
> that, additional places in the source that really need inhibit_libc 
> special casing would be much clearer.  Right now it's a question of 
> "wtf caused that to die?", since bootstrapping a cross-compiler is 
> such a black art to most people. Also, should the inhibit_libc hack 
> also disable things like the unwind support that requires kernel 
> headers?  That not scrictly libc, but --without-headers sort of assume 
> neither are present. It would also be nice if there were a clear 
> statement about what needs to be where in the headers and/or sysroot 
> directory structures.  Or, should be just require some sort of 
> combined tree build in this case?  In which case it would be nice if 
> there were a clear statement about what needs to be copied or 
> symlinked from where to where to make it work. I'm a bit out of my 
> depth since I have *not* gotten it to work for many of the cases I 
> have tried, yet. It's not that I'm unwilling to try and contribute 
> patches, but I'd like to know where we are heading, please.
>
> Some people advocate combining steps 3,4,5 by either cribbing a set of 
> headers from someone elses sysroot, cobbling them together by hand, or 
> by fooling glibc configure in to producing them without having a 
> minimal cross compiler. They build their cross-compiler using these 
> headers and then they proceed to step 6.  That's great if you are only 
> working with one, common, target where you can expect to find someone 
> to crib from or you have better luck than I did trying to understand 
> and fool glibc configure. In any case that leaves us as the mercy of 
> changes to the glibc package, and the glibc maintainers can reasonably 
> assume the existance of a compiler and may use that compiler to check 
> things like sizeof(long double) etc. so it's fragile at best to rely 
> on. (BTW - why do we have both -m128bit-long-double and 
> -mlong-double-128 options for different targets?)  However, it's 
> probably not viable for someone trying to build 20 or so tools chains 
> so they can check for kernel build failure tree wide etc.  And 
> besides, it would sure be nice if gcc would build a cross-compiler 
> out-of-the-box if there is no problem with some other part of the tool 
> chain.  That does not seem to be the case for all targets at this point.
>
> Once you get to step 6, I expect you are home free, or at least are 
> out of the domain off gcc and need to talk to the glibc folks for any 
> problems that pop up at that point.
>
> So, that's the situation as I see it. How should things work in an 
> ideal world and where do we go from here? Or, are other people happy 
> enough with the current ecosystem and think nothing needs to change?
>
> Steve

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Paolo Bonzini]

>    Thanks for reminding me to ping that patch.

Could you try moving this case statement

+	case "$target" in
+	  i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
+	    libgloss_dir=i386
+	    ;;
+	  m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
+	    libgloss_dir=m68hc11
+	    ;;

above in another "case $target in" statement?

Thanks,

Paolo

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Paolo Bonzini]

>    Thanks for reminding me to ping that patch.

Could you try moving this case statement

+	case "$target" in
+	  i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
+	    libgloss_dir=i386
+	    ;;
+	  m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
+	    libgloss_dir=m68hc11
+	    ;;

above in another "case $target in" statement?

Thanks,

Paolo

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Rask Ingemann Lambertsen]

On Sat, Aug 18, 2007 at 08:18:08PM +0200, Segher Boessenkool wrote:

> p.s. Any advice on how to get more targets working would be
> more than welcome :-)

   With patch three from bug 32154
<URL:http://gcc.gnu.org/bugzilla/attachment.cgi?id=13669>, the newlib
targets build out of the box in a combined tree:

.../configure --target=xxx --with-newlib --enable-sim --disable-gdb
--disable-nls --enable-checking=yes,rtl
make

   See also <URL:http://gcc.gnu.org/ml/gcc-help/2007-06/msg00230.html>.

   Thanks for reminding me to ping that patch.

-- 
Rask Ingemann Lambertsen

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Segher Boessenkool]

> So, my open questions to the list are, what is/should be the preferred 
> way to bootstrap a cross compiler/glibc environment?

This likely isn't the preferred way, but the following builds
a cross toolchain for all but a few Linux targets:


$SRC/src/configure \
         --target=$TARGET --prefix=$PREFIX
make
make install


$SRC/gcc/configure \
         --target=$TARGET --enable-targets=all \
         --prefix=$PREFIX \
         --enable-languages=c --without-headers \
         --disable-nls --disable-multilib --disable-threads 
--disable-shared \
         --disable-libmudflap --disable-libssp --disable-libgomp \
         --disable-decimal-float \
         --enable-sjlj-exceptions
make
make install


(blackfin compiler needs to be configured as bfin-linux-uclibc,
cris compiler won't build, h8300 compiler won't build, ia64
compiler won't build -- some embedded targets seem to need
unmerged patches to work for building Linux but I don't bother).

All "normal" targets work fine.

People on the build farm can find this stuff in gcc13:~segher/build,
and installed toolchains (build off mainline) in ~segher/cross.
I do some fresh builds almost every day.


Segher

p.s. Any advice on how to get more targets working would be
more than welcome :-)

Re: Why is building a cross compiler "out-of-the-box" always broken?

[René Rebe]

On Friday 17 August 2007 23:56:30 Ian Lance Taylor wrote:
> "Stephen M. Kenton" <skenton@ou.edu> writes:
> 
> > However, the question
> > remains, why is the problem still there to be circumvented?  Is there
> > some secret opposition to easy use of these tools, is there some law
> > of nature that prevents them from building, is there some good
> > technical reason that is hard to implement, or has it just not been a
> > big enough pain for anyone to beat it into submission?
> 
> The central problem is that gcc, binutils, glibc, and the kernel are
> all separate projects which are distributed and maintained separately
> by different people.  Thus there are mismatches and confusions and
> difficulties which result from the different release cycles and
> different agendas.
> 
> Thus there is a place in the ecosystem for people to write the scripts
> needed to smooth over those differences.  And indeed that ecosystem is
> filled by tools like crosstool and buildtool.

Or the T2 SDE (http://www.t2-project.org).

> This is certainly not ideal.  But the organizational differences make
> it quite difficult to fix in any other way.
> 
> Ian

-- 
  René Rebe - ExactCODE GmbH - Europe, Germany, Berlin
  http://exactcode.de | http://t2-project.org | http://rene.rebe.name

Re: Why is building a cross compiler "out-of-the-box" always broken?

[David Daney]

Andrew Haley wrote:
> David Daney writes:
>  > Stephen M. Kenton wrote:
>  > > Hello all,
>  > > 
>  > .
>  > .
>  > .
>  > >  I realize that there are various "solutions" for specific 
>  > > platforms.  Dan Kegel's excellent crosstool and the cross-lfs website, 
>  > >
>  > .
>  > .
>  > .
>  > > 
>  > > So, my open questions to the list are, what is/should be the preferred 
>  > > way to bootstrap a cross compiler/glibc environment?
>  > 
>  > Don't bootstrap.
>  
> That's right.  Unless you're building a new operating system -- in
> which case building the toolchain is the least of your problems -- you
> don't need to boostrap.  Just point the sysroot at the root filesystem
> of your target OS when configuring the cross compiler.
>   
It is not always that simple.  Sometimes you have everything you need 
for an existing OS except a sysroot.  While it theory one might be able 
to obtain a suitable pre-built sysroot, often it is easier to bootstrap.

Recently I bootstrapped a mips64-linux sysroot.  I installed Debian mips 
which is 32 bits, but the kernel is 64 bits and can handle a 
mips64-linux userspace also.  For me it seemed the easiest path to 
obtaining a sysroot was to crib off of the cross-lfs instructions.  
However I will never do it again now that I have a working sysroot.

David Daney

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Andrew Haley]

David Daney writes:
 > Stephen M. Kenton wrote:
 > > Hello all,
 > > 
 > .
 > .
 > .
 > >  I realize that there are various "solutions" for specific 
 > > platforms.  Dan Kegel's excellent crosstool and the cross-lfs website, 
 > >
 > .
 > .
 > .
 > > 
 > > So, my open questions to the list are, what is/should be the preferred 
 > > way to bootstrap a cross compiler/glibc environment?
 > 
 > Don't bootstrap.
 
That's right.  Unless you're building a new operating system -- in
which case building the toolchain is the least of your problems -- you
don't need to boostrap.  Just point the sysroot at the root filesystem
of your target OS when configuring the cross compiler.

Andrew.

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Stephen M. Kenton]

Andrew Pinski wrote:
> On 8/17/07, Stephen M. Kenton <skenton@ou.edu> wrote:
>
> Cross compiling works for me out of the box if done correctly.  Yes I
> have to compile GCC and glibc (or newlib) twice but I don't care as it
> is all scripted.
>
> Thanks,
> Andrew Pinski
>   
Great! Scripting is wonderful and I don't object to multiple cycles 
either. Care to hit me with a clue-bat and post your script so I can 
find what I did wrong?  It sound's like other people have 
"out-of-the-box" problems too. I'll patch my scripts and turn them loose 
on ~20 tool chain builds.

Steve

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Ian Lance Taylor]

"Stephen M. Kenton" <skenton@ou.edu> writes:

> However, the question
> remains, why is the problem still there to be circumvented?  Is there
> some secret opposition to easy use of these tools, is there some law
> of nature that prevents them from building, is there some good
> technical reason that is hard to implement, or has it just not been a
> big enough pain for anyone to beat it into submission?

The central problem is that gcc, binutils, glibc, and the kernel are
all separate projects which are distributed and maintained separately
by different people.  Thus there are mismatches and confusions and
difficulties which result from the different release cycles and
different agendas.

Thus there is a place in the ecosystem for people to write the scripts
needed to smooth over those differences.  And indeed that ecosystem is
filled by tools like crosstool and buildtool.

This is certainly not ideal.  But the organizational differences make
it quite difficult to fix in any other way.

Ian

Re: Why is building a cross compiler "out-of-the-box" always broken?

[David Daney]

Stephen M. Kenton wrote:
> Hello all,
> 
.
.
.
>  I realize that there are various "solutions" for specific 
> platforms.  Dan Kegel's excellent crosstool and the cross-lfs website, 
>
.
.
.
> 
> So, my open questions to the list are, what is/should be the preferred 
> way to bootstrap a cross compiler/glibc environment?

Don't bootstrap.

Use one of Kegal's crosstool or the cross-lfs instructions exactly once.

Then keep a good build of glibc around for future build cycles.

That way you never bootstrap.  You just build a cross compiler in the 
standard manner, which *always* works.

David Daney

Re: Why is building a cross compiler "out-of-the-box" always broken?

[Andrew Pinski]

On 8/17/07, Stephen M. Kenton <skenton@ou.edu> wrote:

Cross compiling works for me out of the box if done correctly.  Yes I
have to compile GCC and glibc (or newlib) twice but I don't care as it
is all scripted.

Thanks,
Andrew Pinski

Why is building a cross compiler "out-of-the-box" always broken?

[Stephen M. Kenton]

Hello all,

Several years ago in the gcc 3.3 time frame I looked into building cross 
compilers using the current versions of gcc, glibc etc. for a number of 
different systems.  I quickly found that it was a quagmire.  I inquired 
of this list at that time and was told that the glibc hack was 
problematic and that things would not be fixed in gcc 3.4 but might get 
better by gcc 3.5 (which eventually became gcc 4.0).  I worked around my 
need at that time and went on to other things.  Recently out of 
curiosity I dug out the problem and took another look.  Except for the 
fact that the Linux kernel build system now supports headers_install to 
provide a reliable set of sanitized header, not much seems to have 
changed.  I realize that there are various "solutions" for specific 
platforms.  Dan Kegel's excellent crosstool and the cross-lfs website, 
for example, are testaments to the people who have labored to circumvent 
this problem.  However, the question remains, why is the problem still 
there to be circumvented?  Is there some secret opposition to easy use 
of these tools, is there some law of nature that prevents them from 
building, is there some good technical reason that is hard to implement, 
or has it just not been a big enough pain for anyone to beat it into 
submission?  As it stands currently, it looks like gcc needs both the 
kernel (for unwind support) and glibc  (for generic C reasons) headers 
*for the target* to build  a cross compiler for that target, but glibc 
needs a functioning cross compiler for that target before it will even 
configure so you can try to "make install-headers".  Mean while, back at 
the ranch, the inhibit_libc hack does not seem to be implemented 
everywhere needed for a successful build and the gcc configure seems to 
have a curious need for the --with-newlib  flag to really mean we want 
inhibit_libc as well as handling with_sysroot and with_headers rather 
differently. (See the fragment below from gcc 4.2.1) As an aside, I 
suspect that there are subtle assumptions that the build/host glibc 
headers are the same as the target glibc headers in some respects.  Way 
back, I was trying to cross compile on Solaris using a complete self 
hosting gnu tool chain and I saw some very strange failure.  In the 
current experiment I am using a more-or-less up to date Linux system, 
but if/when building a cross compiler works I could resurrect the 
Solaris test environment to check for portability issues of that type.  
For now I'm just looking at current Linux build/host/targets.

So, my open questions to the list are, what is/should be the preferred 
way to bootstrap a cross compiler/glibc environment?
What needed to be done to get there from here?  Does the inhibit_libc 
hack need to be extended, ripped out and replaced, or just better 
documented?  I will follow this discussion on the gcc mailing list 
archive since I'm not subscribed, but I'm willing to test stuff if 
anyone wants me to.  Or, if it really does work currently and I'm just 
an idiot, could some kind soul hit me with a clue-bat and point me to a 
descriptions of the solution.

# inhibit_libc
 
# If this is using newlib, without having the headers available now,
# then define inhibit_libc in LIBGCC2_CFLAGS.
# This prevents libgcc2 from containing any code which requires libc
# support.
inhibit_libc=false
if { { test x$host != x$target && test "x$with_sysroot" = x ; } ||
       test x$with_newlib = xyes ; } &&
     { test "x$with_headers" = x || test "x$with_headers" = xno ; } ; then
       inhibit_libc=true
fi

Thanks - Steve