Compile GCC using only tools isolated from host environment

Compile GCC using only tools isolated from host environment

[Benjamin Lovy]

Hello,

I am attempting to build GCC 11.2.0 in an isolated environment.  First, I
grabbed the Linux headers:

make headers_install \
ARCH=x86_64 \
INSTALL_HDR_PATH=/bootstrap/usr

Then, I built binutils:

mkdir binutils-build && \
  cd binutils-build &&
  /build/binutils-2.38/configure \
--prefix=/bootstrap \
--with-sysroot=/bootstrap \
--disable-nls \
--disable-werror && \
make -j16 && \
make install

Then, I built GCC using the host toolchain:

mkdir /build/gcc-build && \
  cd gcc-build &&
  /build/gcc-11.2.0/configure \
--prefix=/bootstrap/ \
--host=x86_64-pc-linux-gnu \
--enable-languages=c,c++ \
--disable-bootstrap \
--disable-decimal-float \
--disable-fixed-point \
--disable-libatomic \
--disable-libgomp \
--disable-libitm \
--disable-libmpx \
--disable-libquadmath \
--disable-libsanitizer \
--disable-libssp \
--disable-libvtv \
--disable-lto \
--disable-multilib \
--disable-nls && \
make -j16 && \
make install

Then, I built glibc using this GCC:

mkdir glibc-build && \
  cd glibc-build && \
  /build/glibc-2.34/configure \
CC=/bootstrap/bin/gcc \
CFLAGS="-fno-stack-protector -O2" \
CPPFLAGS="-U_FORTIFY_SOURCE" \
--prefix=/bootstrap \
--with-sysroot=/bootstrap \
--with-headers=/bootstrap/usr/include &&\
make -j16 && \
make install

This all works fine.  I then build the following software:

* GMP 6.21
* MPFR 4.10
* MPC 1.21
* tar 1.34
* bash 5.16
* coreutils 9.0
* curl 7.82
* zlib 1.2.11
* gzip 1.11
* sed 4.8
* grep 2.28
* gawk 5.11
* make 4.3
* xz 5.25

I use invocations similar to this for each entry on that list:

./configure \
CC=/bootstrap/bin/gcc \
CFLAGS="-Wl,-dynamic-linker=/bootstrap/lib/ld-linux-x86-64.so.2
-Wl,-rpath,/bootstrap/lib -I/bootstrap/include" \
--with-sysroot=/bootstrap \
--prefix=/bootstrap && \
make -j16 && \
make check && \
make install

Now, I would like to build a fresh GCC using the previously build GCC, this
glib, and the GMP/MPC/MPFR libraries I just built:

/build/gcc-11.2.0/configure \
CFLAGS="-Wl,-dynamic-linker=/lib/ld-linux-x86-64.so.2 -Wl,-rpath,/lib
-I/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include -I/usr/include
-I/usr/include/linux -I/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include-fixed
-B/bin" \
  --prefix=/bootstrap/gcc \
--host=x86_64-pc-linux-gnu \
--with-sysroot=/bootstrap \
--enable-languages=c,c++ \
--disable-bootstrap \
--disable-decimal-float \
--disable-fixed-point \
--disable-libatomic \
--disable-libgomp \
--disable-libitm \
--disable-libmpx \
--disable-libquadmath \
--disable-libsanitizer \
--disable-libssp \
--disable-libvtv \
--disable-lto \
--disable-multilib \
--disable-nls && \
make -j16 && \
make install;

Configure fails, complaining it cannot find `gmp.h`.  In `config.log`, I
observe the configure process appears to actually read the contents of this
header from `/bootstrap/include/gmp.h`, but the preprocessor fails to
expand `stddef.h` and `limits.h`, both of which I've verified are present
in the provided include paths, along with gmp.h.

From my understanding, all the extra paths shouldn't be necessary if I'm
using `--with-sysroot`, but I have failed to find a set of options that
produces a working buiild.

Have I misconfigured this environment?  What steps am I missing to build
GCC using only tools present in my bootstrap directory tree?

If it's relevant, this build is happening in a fresh Ubuntu 20.04 docker
container.

Thanks in advance,
Ben

Re: Compile GCC using only tools isolated from host environment

[Xi Ruoyao]

On Mon, 2022-03-14 at 18:43 -0400, Benjamin Lovy wrote:

/* snip */

> Now, I would like to build a fresh GCC using the previously build GCC,
> this
> glib, and the GMP/MPC/MPFR libraries I just built:
> 
> /build/gcc-11.2.0/configure \
> CFLAGS="-Wl,-dynamic-linker=/lib/ld-linux-x86-64.so.2 -Wl,-rpath,/lib
> -I/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include -I/usr/include
> -I/usr/include/linux -I/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include-
> fixed
> -B/bin" \

I think these FLAGS are mistaken.  --sysroot won't expand -I/usr/include
to search $SYSROOT/usr/include for example.

And, Without "-O2" your GCC will be built without optimization, which
will make a stupidly slow compiler.

>   --prefix=/bootstrap/gcc \
> --host=x86_64-pc-linux-gnu \
> --with-sysroot=/bootstrap \

You need --with-build-sysroot instead of --with-sysroot.  Read
https://gcc.gnu.org/install/configure.html for their difference.

Please note --sysroot is mostly for cross-compilation.  If you are not
building a GCC which is meant to run in a new OS tree (chroot or
container, or even reboot into a new OS tree), it's likely you shouldn't
use sysroot at all.
-- 
Xi Ruoyao <xry111@mengyan1223.wang>
School of Aerospace Science and Technology, Xidian University

Re: Compile GCC using only tools isolated from host environment

[Benjamin Lovy]

HI,

Thank you for your suggestions.  I've tried the CFLAGS with fixed paths,
and am still not successful.

I think I am trying to do what you describe at the end - I want to produce
a GCC that only depends on artifacts found in my /bootstrap tree.  If I
move this whole tree to a new environment, I'd like to still be able to use
this compiler to build arbitrary software.

> you are not
building a GCC which is meant to run in a new OS tree (chroot or
container, or even reboot into a new OS tree), it's likely you shouldn't
use sysroot at all.

On Tue, Mar 15, 2022 at 3:19 AM Xi Ruoyao <xry111@mengyan1223.wang> wrote:

> On Mon, 2022-03-14 at 18:43 -0400, Benjamin Lovy wrote:
>
> /* snip */
>
> > Now, I would like to build a fresh GCC using the previously build GCC,
> > this
> > glib, and the GMP/MPC/MPFR libraries I just built:
> >
> > /build/gcc-11.2.0/configure \
> > CFLAGS="-Wl,-dynamic-linker=/lib/ld-linux-x86-64.so.2 -Wl,-rpath,/lib
> > -I/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include -I/usr/include
> > -I/usr/include/linux -I/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include-
> > fixed
> > -B/bin" \
>
> I think these FLAGS are mistaken.  --sysroot won't expand -I/usr/include
> to search $SYSROOT/usr/include for example.
>
> And, Without "-O2" your GCC will be built without optimization, which
> will make a stupidly slow compiler.
>
> >   --prefix=/bootstrap/gcc \
> > --host=x86_64-pc-linux-gnu \
> > --with-sysroot=/bootstrap \
>
> You need --with-build-sysroot instead of --with-sysroot.  Read
> https://gcc.gnu.org/install/configure.html for their difference.
>
> Please note --sysroot is mostly for cross-compilation.  If you are not
> building a GCC which is meant to run in a new OS tree (chroot or
> container, or even reboot into a new OS tree), it's likely you shouldn't
> use sysroot at all.
> --
> Xi Ruoyao <xry111@mengyan1223.wang>
> School of Aerospace Science and Technology, Xidian University
>

Re: Compile GCC using only tools isolated from host environment

[Jonathan Wakely]

On Mon, 14 Mar 2022, 22:44 Benjamin Lovy, <ben@tangram.dev> wrote:

> Hello,
>
> I am attempting to build GCC 11.2.0 in an isolated environment.  First, I
> grabbed the Linux headers:
>
> make headers_install \
> ARCH=x86_64 \
> INSTALL_HDR_PATH=/bootstrap/usr
>
> Then, I built binutils:
>
> mkdir binutils-build && \
>   cd binutils-build &&
>   /build/binutils-2.38/configure \
> --prefix=/bootstrap \
> --with-sysroot=/bootstrap \
> --disable-nls \
> --disable-werror && \
> make -j16 && \
> make install
>
> Then, I built GCC using the host toolchain:
>
> mkdir /build/gcc-build && \
>   cd gcc-build &&
>   /build/gcc-11.2.0/configure \
> --prefix=/bootstrap/ \
> --host=x86_64-pc-linux-gnu \
> --enable-languages=c,c++ \
> --disable-bootstrap \
> --disable-decimal-float \
> --disable-fixed-point \
> --disable-libatomic \
> --disable-libgomp \
> --disable-libitm \
> --disable-libmpx \
> --disable-libquadmath \
> --disable-libsanitizer \
> --disable-libssp \
> --disable-libvtv \
> --disable-lto \
> --disable-multilib \
> --disable-nls && \
> make -j16 && \
> make install
>
> Then, I built glibc using this GCC:
>
> mkdir glibc-build && \
>   cd glibc-build && \
>   /build/glibc-2.34/configure \
> CC=/bootstrap/bin/gcc \
> CFLAGS="-fno-stack-protector -O2" \
> CPPFLAGS="-U_FORTIFY_SOURCE" \
> --prefix=/bootstrap \
> --with-sysroot=/bootstrap \
> --with-headers=/bootstrap/usr/include &&\
> make -j16 && \
> make install
>
> This all works fine.  I then build the following software:
>
> * GMP 6.21
> * MPFR 4.10
> * MPC 1.21
> * tar 1.34
> * bash 5.16
> * coreutils 9.0
> * curl 7.82
> * zlib 1.2.11
> * gzip 1.11
> * sed 4.8
> * grep 2.28
> * gawk 5.11
> * make 4.3
> * xz 5.25
>
> I use invocations similar to this for each entry on that list:
>
> ./configure \
> CC=/bootstrap/bin/gcc \
> CFLAGS="-Wl,-dynamic-linker=/bootstrap/lib/ld-linux-x86-64.so.2
> -Wl,-rpath,/bootstrap/lib -I/bootstrap/include" \
> --with-sysroot=/bootstrap \
> --prefix=/bootstrap && \
> make -j16 && \
> make check && \
> make install
>
> Now, I would like to build a fresh GCC using the previously build GCC, this
> glib, and the GMP/MPC/MPFR libraries I just built:
>

I recommend you don't do that.

Use the contrib/download_prerequisites script instead, see
https://gcc.gnu.org/wiki/InstallingGCC

This doesn't solve your header path issue, but it's much simpler and
cleaner anyway.

Question related to -fPIC behaviour across architectures

[vincent Dupaquis]

[-- Attachment #1.1: Type: text/plain, Size: 1387 bytes --]

Hello,

I am using -fPIC option on several architectures, and have experienced 
some strange differences in behavior over 2 aspects :

     - Call of routines

     - Access to constants (eg. constant strings, constant tables, 
larger to fit into one single instruction).

When compiling with -fPIC on an arm platform (I am using Cortex-Mx), the 
jumps and calls are all relative, and constant access are also relative 
if you are cautious enough. You may even end-up having the compiler not 
needing the GOT. Bottom line, for me -fPIC on ARM means portable and 
really Position Independant Code. (I am using GCC 10.3).

When compiling with -fPIC on a xtensa platform, the compiler generates 
CALLX instructions, which effectively make an access at an absolute 
address. The same issue appears about the constants, which are accessed 
at the address you compiled your binary for, which is not the definition 
I have (and have experienced on ARM platforms). (I am using in this case 
8.4).

When NOT specifying -fPIC on a xtensa platform, the compiler generates 
CALL instructions, which are relative calls, and which is the base for 
me for having a portable/relocable executable.

My questions are :

- Did I miss anything ?

- Is there somewhere a common definition of what mean PIC for the 
different architectures ?

Regards,

Vincent.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Question related to -fPIC behaviour across architectures

[Xi Ruoyao]

On Tue, 2022-05-03 at 10:26 +0200, vincent Dupaquis wrote:


> - Is there somewhere a common definition of what mean PIC for the 
> different architectures ?

Generally -fpic/-fPIC does not means only position-independant code, but
position-independant code **suitable for dynamic linking**.

Consider the code:

void callee(void)
{
  /* ... */
}

void caller(void)
{
  callee();
}

Without -fPIC caller may call callee with a PC-relative call
instruction.  But with -fPIC it's not allowed because the symbol callee
may be interposed.  For more info:
https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic

(You may argue that the ELF interposition rule is strange and known to
slow down programs, but there are still many programs depending on the
rule in 2022.)

So my guess is w/o -fPIC the compiler just calls callee with a PC-rel
call, but with -fPIC it needs to either:

(1) Load the address of callee from GOT.

or

(2) Call the PLT stub ("callee@PLT") which is resolved to "jump callee"
at runtime.

For (1), the address of the callee is loaded into a register then a
"call register" instruction is used.  It seems callx8 is such an
instruction on Xtensa (I know nothing about Xtensa so it's from Google).
For (2), the compiler and the assembler cannot determine if the PLT stub
is out-of-range for the PC-rel call instruction (as the PLT stubs are
generated by the linker).  So the only approach legal for the worst case
is to assume the PLT stub may be far away from the call site.  Then a
PC-relative address load instruction will be used to load the address of
the PLT stub into a register, then callx8 is used to perform the call.


For some of other targets, a code model is defined to guarantee the PLT
stubs to be in-range of the PC-rel call instruction.  Those targets can
simply use PC-rel call to invoke callee@PLT.  But again I know nothing
about Xtensa and I can't reproduce the behavior you mentioned with GCC
trunk.  It seems always generating "l32r/callx8" pairs for calls on
xtensa-linux-gnu, unless the callee is `static`.  And it makes sense to
me: "l32r", as a PC-relative address loading instruction, will load the
address of callee@PLT correctly.
-- 
Xi Ruoyao <xry111@mengyan1223.wang>
School of Aerospace Science and Technology, Xidian University

Re: Question related to -fPIC behaviour across architectures

[vincent Dupaquis]

[-- Attachment #1.1: Type: text/plain, Size: 3165 bytes --]

Thanks for your response !

Ok, it makes sense in the context of dynamic libraries.

But in the case of embedded code, not running on a OS like linux (which 
is the case of Cortex-Mx and a lot of xtensa devices), the definition of 
Position Independant Code is quite different, and in this case, only 
relative jumps/calls and relative access to constants should be allowed.

I have the feeling that because some xtensa chips are able to run linux, 
there may be some confusion there.

Regards,

Vincent.

Le 03/05/2022 à 12:24, Xi Ruoyao a écrit :
> On Tue, 2022-05-03 at 10:26 +0200, vincent Dupaquis wrote:
>
>
>> - Is there somewhere a common definition of what mean PIC for the
>> different architectures ?
> Generally -fpic/-fPIC does not means only position-independant code, but
> position-independant code **suitable for dynamic linking**.
>
> Consider the code:
>
> void callee(void)
> {
>    /* ... */
> }
>
> void caller(void)
> {
>    callee();
> }
>
> Without -fPIC caller may call callee with a PC-relative call
> instruction.  But with -fPIC it's not allowed because the symbol callee
> may be interposed.  For more info:
> https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic
>
> (You may argue that the ELF interposition rule is strange and known to
> slow down programs, but there are still many programs depending on the
> rule in 2022.)
>
> So my guess is w/o -fPIC the compiler just calls callee with a PC-rel
> call, but with -fPIC it needs to either:
>
> (1) Load the address of callee from GOT.
>
> or
>
> (2) Call the PLT stub ("callee@PLT") which is resolved to "jump callee"
> at runtime.
>
> For (1), the address of the callee is loaded into a register then a
> "call register" instruction is used.  It seems callx8 is such an
> instruction on Xtensa (I know nothing about Xtensa so it's from Google).
> For (2), the compiler and the assembler cannot determine if the PLT stub
> is out-of-range for the PC-rel call instruction (as the PLT stubs are
> generated by the linker).  So the only approach legal for the worst case
> is to assume the PLT stub may be far away from the call site.  Then a
> PC-relative address load instruction will be used to load the address of
> the PLT stub into a register, then callx8 is used to perform the call.
>
>
> For some of other targets, a code model is defined to guarantee the PLT
> stubs to be in-range of the PC-rel call instruction.  Those targets can
> simply use PC-rel call to invoke callee@PLT.  But again I know nothing
> about Xtensa and I can't reproduce the behavior you mentioned with GCC
> trunk.  It seems always generating "l32r/callx8" pairs for calls on
> xtensa-linux-gnu, unless the callee is `static`.  And it makes sense to
> me: "l32r", as a PC-relative address loading instruction, will load the
> address of callee@PLT correctly.
-- 

*Vincent Dupaquis*
Software security & Cryptography expert
06 24 58 17 05
/Europarc de Pichaury Bâtiment B8 1330 rue Guillibert Gautier de la 
Lauzière 13290 Aix-en-Provence/

www.trusted-objects.com <http://www.trusted-objects.com>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Question related to -fPIC behaviour across architectures

[Florian Weimer]

* vincent Dupaquis:

> - Did I miss anything ?

The linker can perform relaxations (e.g., elimination of GOT
indirection) based on whole-program analysis, something the compiler
cannot do.  Such optimizations are very much target-dependent, and they
often need some previous ABI work to define new relaxable relocations
for relocatable object files.

> - Is there somewhere a common definition of what mean PIC for the
>   different architectures ?

Not really, not even for ELF.  There are some common assumptions in the
background for ELF implementations (e.g., one canonical function
address), but how you get there varies somewhat.

Thanks,
Florian

Re: Question related to -fPIC behaviour across architectures

[vincent Dupaquis]

[-- Attachment #1.1: Type: text/plain, Size: 1146 bytes --]

Thanks for taking the time writing this down, I understand better now. 
It also matches gcc's documentation.

Regards,
     Vincent.

Le 03/05/2022 à 16:29, Florian Weimer a écrit :

> * vincent Dupaquis:
>
>> - Did I miss anything ?
> The linker can perform relaxations (e.g., elimination of GOT
> indirection) based on whole-program analysis, something the compiler
> cannot do.  Such optimizations are very much target-dependent, and they
> often need some previous ABI work to define new relaxable relocations
> for relocatable object files.
>
>> - Is there somewhere a common definition of what mean PIC for the
>>    different architectures ?
> Not really, not even for ELF.  There are some common assumptions in the
> background for ELF implementations (e.g., one canonical function
> address), but how you get there varies somewhat.
>
> Thanks,
> Florian
>
-- 

*Vincent Dupaquis*
Software security & Cryptography expert
06 24 58 17 05
/Europarc de Pichaury Bâtiment B8 1330 rue Guillibert Gautier de la 
Lauzière 13290 Aix-en-Provence/

www.trusted-objects.com <http://www.trusted-objects.com>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]