More questions on sysroots

More questions on sysroots

[NightStrike]

I thought I was doing the right thing for a while, but apparently
there are still issues.  If I want to build a complete toolchain as a
canadian cross, tar it up, and give it to someone else, I have been
doing the following for configuring binutils/gcc:

../gcc-svn/gcc/configure --prefix=/tmp/rt/root
--with-sysroot=/tmp/rt/root --target=$TARGET

Will this combinations of options, after tarring up /tmp/rt/root,
allow that tar to be distributed anywhere, untarred into any
directory, and run as normal with nothing but a simple change to PATH?
 I thought it would.  I thought I could do this:

cd /tmp/rt
tar cjf tarfile root --owner 0 --group 0

Then on any system:
cd /home/user/opt
tar xf tarfile
export PATH=/home/user/opt/root/bin:$PATH

And finally, use that gcc seemlessly.  Will that work?

RE: More questions on sysroots

[Kaz Kylheku]

gcc-help-owner@gcc.gnu.org wrote:
> I thought I was doing the right thing for a while, but apparently
> there are still issues.  If I want to build a complete toolchain as a
> canadian cross, tar it up, and give it to someone else, I have been
> doing the following for configuring binutils/gcc:
> 
> ../gcc-svn/gcc/configure --prefix=/tmp/rt/root
> --with-sysroot=/tmp/rt/root --target=$TARGET

Using the same value for sysroot and prefix is not such a hot idea.

The sysroot corresponds to the root of the target filesystem that you
are building, or a least its toolchain materials, like usr/include and
usr/lib.

This is not where you are installing the cross-compiler (which isn't
even compiled to run on that architecture).

> Will this combinations of options, after tarring up /tmp/rt/root,
> allow that tar to be distributed anywhere, untarred into any
> directory, and run as normal with nothing but a simple change to PATH?

To build a relocatable toolchain (runs in any directory), the sysroot
has to be a subdirectory of the prefix. It doesn't have to correspond to
the prefix itself. It can be anywhere underneath. Exactly where, that's
up to you.

You can just make up some subdirectory and stick it there.

I use the scheme $prefix/$arch_string/root

The reason for adding the architecture to the path is that the toolchain
can, in principle, be built for multiple targets, all installing to the
same prefix. (This is is supported). Yet each target needs a distinct
sysroot under there.

So say you built for arm and mips. You'd end up with
$prefix/bin/mips-*-gcc and $prefix/bin/arm-*-gcc. Your sysroots in this
combo toolchain would be $prefix/arm-*/root and $prefix/mips-*/root.
Here * represents some additional components of the architecture tuple.
In my case this is ``linux'', since that's the target OS; the toolchain
is coupled with a Linux distro. So my C compiler would be:


  /path/to/toolchain/bin/mips64-linux-gcc

or alternately:

  /path/to/toolchain/mips64-linux/bin/gcc

the sysroot is

  /path/to/toolchain/mips64-linux/root

where we find

  /path/to/toolchain/mips64-linux/root/usr/include
  /path/to/toolchain/mips64-linux/root/usr/lib32

etc. Note that the toolchain itself has lib and include directories:

  /path/to/toolchain/{include,lib}

If we made that the sysroot, these would get mixed into the root.

Re: More questions on sysroots

[NightStrike]

On 12/20/07, Kaz Kylheku <kaz@zeugmasystems.com> wrote:
> gcc-help-owner@gcc.gnu.org wrote:
> > I thought I was doing the right thing for a while, but apparently
> > there are still issues.  If I want to build a complete toolchain as a
> > canadian cross, tar it up, and give it to someone else, I have been
> > doing the following for configuring binutils/gcc:
> >
> > ../gcc-svn/gcc/configure --prefix=/tmp/rt/root
> > --with-sysroot=/tmp/rt/root --target=$TARGET
>
> Using the same value for sysroot and prefix is not such a hot idea.
>
> The sysroot corresponds to the root of the target filesystem that you
> are building, or a least its toolchain materials, like usr/include and
> usr/lib.
>
> This is not where you are installing the cross-compiler (which isn't
> even compiled to run on that architecture).

When you say "the root of the target filesystem", do you mean that
it's the root of a directory structure containing target files that
will be used while creating the toolchain, or that it's the root of a
directory structure where the final toolchain, once built, will look
for target files?  I'm guessing the answer is "both".....

> > Will this combinations of options, after tarring up /tmp/rt/root,
> > allow that tar to be distributed anywhere, untarred into any
> > directory, and run as normal with nothing but a simple change to PATH?
>
> To build a relocatable toolchain (runs in any directory), the sysroot
> has to be a subdirectory of the prefix. It doesn't have to correspond to
> the prefix itself. It can be anywhere underneath. Exactly where, that's
> up to you.
>
> You can just make up some subdirectory and stick it there.
>
> I use the scheme $prefix/$arch_string/root
>
> The reason for adding the architecture to the path is that the toolchain
> can, in principle, be built for multiple targets, all installing to the
> same prefix. (This is is supported). Yet each target needs a distinct
> sysroot under there.
>
> So say you built for arm and mips. You'd end up with
> $prefix/bin/mips-*-gcc and $prefix/bin/arm-*-gcc. Your sysroots in this
> combo toolchain would be $prefix/arm-*/root and $prefix/mips-*/root.
> Here * represents some additional components of the architecture tuple.
> In my case this is ``linux'', since that's the target OS; the toolchain
> is coupled with a Linux distro. So my C compiler would be:
>
>
>  /path/to/toolchain/bin/mips64-linux-gcc
>
> or alternately:
>
>  /path/to/toolchain/mips64-linux/bin/gcc
>
> the sysroot is
>
>  /path/to/toolchain/mips64-linux/root
>
> where we find
>
>  /path/to/toolchain/mips64-linux/root/usr/include
>  /path/to/toolchain/mips64-linux/root/usr/lib32
>
> etc. Note that the toolchain itself has lib and include directories:
>
>  /path/to/toolchain/{include,lib}
>
> If we made that the sysroot, these would get mixed into the root.

Is it absolutely *required* that the sysroot be a subdirectory of
prefix to make the final toolchain relocatable, or is this where
with-build-sysroot comes into play?  I will definitely adapt my
scripts as you suggest (I like that idea of the directory for the
target arch), but I am still curious about how with-build-sysroot fits
into this.  The few sentences in the manual don't clear it up for me,
nor does reading configure.ac.

Also, is there an option that's just plain "--sysroot"?  Does that
have any meaning?  Should it ever be used?

Re: More questions on sysroots

[Kaz Kylheku]

"NightStrike" <nightstrike@gmail.com> wrote:
> When you say "the root of the target filesystem", do you mean that
> it's the root of a directory structure containing target files that
> will be used while creating the toolchain, or that it's the root of a
> directory structure where the final toolchain, once built, will look
> for target files?  I'm guessing the answer is "both".....

The asnwer is both. For instance, suppose you're making a cross-compiled 
LInux system, and building a final user-space GCC which is integrated with 
the GNU C library.

You would already have glibc built into the sysroot (as well, you'd of 
course spin it into a package to install on the target system).

>> If we made that the sysroot, these would get mixed into the root.
>
> Is it absolutely *required* that the sysroot be a subdirectory of
> prefix to make the final toolchain relocatable, or is this where
> with-build-sysroot comes into play?

More precisely, from http://gcc.gnu.org/install/configure.html:

`` If the specified directory is a subdirectory of ${exec_prefix}, then it 
will be found relative to the GCC binaries if the installation tree is 
moved. ''

I think where --with-build-sysroot comes into play is that it allows these 
target system materials (like glibc) to be pulled from somewhere other than 
the sysroot location during the building of the cross compiler.

This provides some flexibility as to how to structure the cross compiler 
build process, in certain situations.

I've built an entire Linux distro, including a cross toolchain that runs on 
a build systems /and/ a native toolchain that runs on the target, without 
ever needing to use --with-build-sysroot.  It wouldn't be that helpful 
because as soon as the compiler is built with the glibc materials, I want to 
run it. And at that point, those materials must be in the sysroot location. 
But I can imagine some situations where you don't want to run the compiler 
that is built, but perhaps package it for someone else to run somewhere 
else.

> Also, is there an option that's just plain "--sysroot"?  Does that
> have any meaning?  Should it ever be used?

That's not a build configuration option, but a run-time switch supported by 
the gcc front end to override the sysroot location. There is probably a 
situation in which this comes in handy, just not the normal case.  I'm only 
guessing, but it's probably useful during the staging of gcc. If you have a 
partially built gcc which doesn't know anything about any sysroot, 
then --sysroot can tell it where to look.

Re: More questions on sysroots

[NightStrike]

On 12/21/07, Kaz Kylheku <kaz@zeugmasystems.com> wrote:
> "NightStrike" <nightstrike@gmail.com> wrote:
> > When you say "the root of the target filesystem", do you mean that
> > it's the root of a directory structure containing target files that
> > will be used while creating the toolchain, or that it's the root of a
> > directory structure where the final toolchain, once built, will look
> > for target files?  I'm guessing the answer is "both".....
>
> The asnwer is both. For instance, suppose you're making a cross-compiled
> LInux system, and building a final user-space GCC which is integrated with
> the GNU C library.
>
> You would already have glibc built into the sysroot (as well, you'd of
> course spin it into a package to install on the target system).
>
> >> If we made that the sysroot, these would get mixed into the root.
> >
> > Is it absolutely *required* that the sysroot be a subdirectory of
> > prefix to make the final toolchain relocatable, or is this where
> > with-build-sysroot comes into play?
>
> More precisely, from http://gcc.gnu.org/install/configure.html:
>
> `` If the specified directory is a subdirectory of ${exec_prefix}, then it
> will be found relative to the GCC binaries if the installation tree is
> moved. ''
>
> I think where --with-build-sysroot comes into play is that it allows these
> target system materials (like glibc) to be pulled from somewhere other than
> the sysroot location during the building of the cross compiler.
>
> This provides some flexibility as to how to structure the cross compiler
> build process, in certain situations.
>
> I've built an entire Linux distro, including a cross toolchain that runs on
> a build systems /and/ a native toolchain that runs on the target, without
> ever needing to use --with-build-sysroot.  It wouldn't be that helpful
> because as soon as the compiler is built with the glibc materials, I want to
> run it. And at that point, those materials must be in the sysroot location.
> But I can imagine some situations where you don't want to run the compiler
> that is built, but perhaps package it for someone else to run somewhere
> else.

That is what I am trying to do.  To build a cross compiler, I have
thus far been doing this:

configure binutils with prefix=with-sysroot=/tmp/rt, having preloaded
/tmp/rt/mingw/include with all of the system headers.
make / install
configure gcc the same way
make all-gcc / make install-gcc
build the runtime crt and libs, putting all of those files into
$(prefix)/$(arch)/lib
build the full gcc doing 'make && make install'

Then I'd tar up the rt/ directory and give it to someone.  This worked
fine, and was even relocatable on all the systems I tried it on.  It
wasn't as portable as I thought, however, and as far as I understand
the documentation, it shouldn't be portable at all.  So, I try your
method:

configure binutils with prefix=/tmp/rt with-sysroot=/tmp/rt/sys-root
(making sure to preload /tmp/rt/sys-root/mingw/include with all of the
system headers, like I did before.)
make / install
configure gcc the same way
make all-gcc / make install-gcc
build the runtime crt and libs, putting all of those files into
/tmp/rt/sys-root/mingw/lib
build the full gcc doing 'make && make install'

This is where it dies as follows:
In file included from ../../../gcc/libgcc/../gcc/libgcc2.c:33:
../../../gcc/libgcc/../gcc/tsystem.h:90:19: error: stdio.h: No such
file or directory
../../../gcc/libgcc/../gcc/tsystem.h:93:23: error: sys/types.h: No
such file or directory
../../../gcc/libgcc/../gcc/tsystem.h:96:19: error: errno.h: No such
file or directory
../../../gcc/libgcc/../gcc/tsystem.h:103:20: error: string.h: No such
file or directory
../../../gcc/libgcc/../gcc/tsystem.h:104:20: error: stdlib.h: No such
file or directory
../../../gcc/libgcc/../gcc/tsystem.h:105:20: error: unistd.h: No such
file or directory
In file included from
/var/tmp/build/gcc-svn/build-x86_64-pc-linux/./gcc/include-fixed/syslimits.h:7,
                 from
/var/tmp/build/gcc-svn/build-x86_64-pc-linux/./gcc/include-fixed/limits.h:11,
                 from ../../../gcc/libgcc/../gcc/tsystem.h:108,
                 from ../../../gcc/libgcc/../gcc/libgcc2.c:33:
/var/tmp/build/gcc-svn/build-x86_64-pc-linux/./gcc/include-fixed/limits.h:122:61:
error: no include path in which to search for limits.h
In file included from ../../../gcc/libgcc/../gcc/libgcc2.c:33:
../../../gcc/libgcc/../gcc/tsystem.h:111:18: error: time.h: No such
file or directory
make[2]: *** [_muldi3.o] Error 1
make[1]: *** [all-target-libgcc] Error 2
make: *** [all] Error 2


I don't understand that error.  It says, for instance, that limits.h
can't include limits.h.  How is that possible?  It also can't find any
of the system headers that are in place.

Why is there such drastic changes in the outcome of a toolchain build
when all I did was move the location of the sysroot from being equal
to prefix to being a subdirectory of prefix?

Re: More questions on sysroots

[Brian Dessent]

NightStrike wrote:

> This is where it dies as follows:
> In file included from ../../../gcc/libgcc/../gcc/libgcc2.c:33:
> ../../../gcc/libgcc/../gcc/tsystem.h:90:19: error: stdio.h: No such
> file or directory
> ../../../gcc/libgcc/../gcc/tsystem.h:93:23: error: sys/types.h: No
> such file or directory
> ../../../gcc/libgcc/../gcc/tsystem.h:96:19: error: errno.h: No such
> file or directory
> ../../../gcc/libgcc/../gcc/tsystem.h:103:20: error: string.h: No such
> file or directory
> ../../../gcc/libgcc/../gcc/tsystem.h:104:20: error: stdlib.h: No such
> file or directory
> ../../../gcc/libgcc/../gcc/tsystem.h:105:20: error: unistd.h: No such
> file or directory

You cut out the part that would have been useful, namely the -B and -I
arguments passed to xgcc.

> I don't understand that error.  It says, for instance, that limits.h
> can't include limits.h.  How is that possible?  It also can't find any
> of the system headers that are in place.

The limits.h that was found is gcc's private copy, not the actual system
limits.h.  gcc uses this technique when it needs to amend the contents
of a system header, by installing its own limits.h in its private
include directory that contains the gcc-specific parts, and at the end
it does #include_next <limits.h> so that the actual system limits.h gets
picked up at that point.  It also does something similar when it needs
to deal with buggy/broken/noncompliant system headers, by making a copy
of the offending header in its private include dir and then fixincluding
it there, obviously without the include_next.  The details don't matter,
the error is materially equivalent to all of the above, namely, a system
header could not be found.

Brian

Re: More questions on sysroots

[NightStrike]

On 12/22/07, Brian Dessent <brian@dessent.net> wrote:
> NightStrike wrote:
>
> > This is where it dies as follows:
> > In file included from ../../../gcc/libgcc/../gcc/libgcc2.c:33:
> > ../../../gcc/libgcc/../gcc/tsystem.h:90:19: error: stdio.h: No such
> > file or directory
> > ../../../gcc/libgcc/../gcc/tsystem.h:93:23: error: sys/types.h: No
> > such file or directory
> > ../../../gcc/libgcc/../gcc/tsystem.h:96:19: error: errno.h: No such
> > file or directory
> > ../../../gcc/libgcc/../gcc/tsystem.h:103:20: error: string.h: No such
> > file or directory
> > ../../../gcc/libgcc/../gcc/tsystem.h:104:20: error: stdlib.h: No such
> > file or directory
> > ../../../gcc/libgcc/../gcc/tsystem.h:105:20: error: unistd.h: No such
> > file or directory
>
> You cut out the part that would have been useful, namely the -B and -I
> arguments passed to xgcc.

Not really cut out.. I just redirected stdout to /dev/null and only
had the stderr to look at.  Here is the part that I lost:

/var/tmp/build/gcc-svn/build-x86_64-pc-linux/./gcc/xgcc
-B/var/tmp/build/gcc-svn/build-x86_64-pc-linux/./gcc/
-L/var/tmp/build/gcc-svn/build-x86_64-pc-linux/x86_64-pc-mingw32/winsup/mingw
-L/var/tmp/build/gcc-svn/build-x86_64-pc-linux/x86_64-pc-mingw32/winsup/w32api/lib
-isystem /var/tmp/build/gcc-svn/gcc/winsup/mingw/include -isystem
/var/tmp/build/gcc-svn/gcc/winsup/w32api/include
-B/var/tmp/rt/x86_64-pc-mingw32/bin/
-B/var/tmp/rt/x86_64-pc-mingw32/lib/ -isystem
/var/tmp/rt/x86_64-pc-mingw32/include -isystem
/var/tmp/rt/x86_64-pc-mingw32/sys-include -O2 -g -g -O2 -O2
-I../../gcc/gcc/../winsup/w32api/include -O2 -g -g -O2   -DIN_GCC
-DCROSS_DIRECTORY_STRUCTURE   -W -Wall -Wwrite-strings
-Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition
-isystem ./include   -g -DHAVE_GTHR_DEFAULT -DIN_LIBGCC2
-D__GCC_FLOAT_NOT_NEEDED   -I. -I. -I../.././gcc -I../../../gcc/libgcc
-I../../../gcc/libgcc/. -I../../../gcc/libgcc/../gcc
-I../../../gcc/libgcc/../include  -DHAVE_CC_TLS -o _muldi3.o -MT
_muldi3.o -MD -MP -MF _muldi3.dep -DL_muldi3 -c
../../../gcc/libgcc/../gcc/libgcc2.c \