[PATCH 0 of 2] documentation update

[PATCH 0 of 2] documentation update

[Trevor Woerner]

In this patch series I have attempted to capture
the details of a recent email exchange in the hopes
others will find it (and find it useful).

--
For unsubscribe information see http://sourceware.org/lists.html#faq

[PATCH 2 of 2] docs: Add strategies for assembling root filesystems

[Trevor Woerner]

# HG changeset patch
# User Trevor Woerner <twoerner@gmail.com>
# Date 1321482977 18000
# Node ID 8bd0d6bdba97ba1f9a80be56daf1e6b4161cb9b4
# Parent  c1c4d9174fbfd7d76e0e520dc6657cf239a7d718
docs: Add strategies for assembling root filesystems.

Expand the documentation for using a crosstool-NG-generated toolchain for
building a root filesystem for a target device.

Signed-off-by: "Trevor Woerner" <twoerner@gmail.com>

diff -r c1c4d9174fbf -r 8bd0d6bdba97 docs/5 - Using the toolchain.txt
--- a/docs/5 - Using the toolchain.txt	Wed Nov 16 16:06:42 2011 -0500
+++ b/docs/5 - Using the toolchain.txt	Wed Nov 16 17:36:17 2011 -0500
@@ -21,16 +21,137 @@
   make CROSS_COMPILE=your-host-tuple-
 and so on...
 
+
+Assembling a root filesystem  /
+_____________________________/
+
+
+Assembling a root filesystem for a target device requires the successive
+building of a set of software packages for the target architecture. Building
+a package potentially requires artifacts which were generated as part of an
+earlier build. Note that not all artifacts which are installed as part of a
+package are desirable on a target's root filesystem (e.g. man/info files,
+include files, etc.). Therefore we must distinguish between a 'staging'
+directory and a 'rootfs' directory.
+
+A 'staging' directory is a location into which we install all the build
+artifacts. We can then point future builds to this location so they can find
+the appropriate header and library files. A 'rootfs' directory is a location
+into which we place only the files we want to have on our target.
+
+There are four schools of thought here:
+
+1) Install directly into the sysroot of the toolchain.
+
+   By default (i.e. if you don't pass any arguments to the tools which
+   would change this behaviour) the toolchain that is built by
+   crosstool-NG will only look in its toolchain directories for system
+   header and library files:
+
+#include "..." search starts here:
+#include <...> search starts here:
+<ct-ng install path>/lib/gcc/<host tuple>/4.5.2/include
+<ct-ng install path>/lib/gcc/<host tuple>/4.5.2/include-fixed
+<ct-ng install path>/lib/gcc/<host tuple>/4.5.2/../../../../<host tuple>/include
+<ct-ng install path>/<host tuple>/sysroot/usr/include
+
+   In other words, the compiler will automagically find headers and
+   libraries without extra flags if they are installed under the
+   toolchain's sysroot directory.
+   
+   However, this is bad because the toolchain gets poluted, and can
+   not be re-used.
+
+   $ ./configure --build=<build tuple> --host=<host tuple> \
+         --prefix=/usr --enable-foo-bar...
+   $ make
+   $ make DESTDIR=/<ct-ng install path>/<host tuple>/sysroot install
+
+2) Copy the toolchain's sysroot to the 'staging' area.
+
+   If you start off by copying the toolchain's sysroot directory to your
+   staging area, you can simply proceed to install all your packages'
+   artifacts to the same staging area. You then only need to specify a
+   '--sysroot=<staging area>' option to the compiler of any subsequent
+   builds and all your required header and library files will be found/used.
+
+   This is a viable option, but requires the user to always specify CFLAGS
+   in order to include --sysroot=<staging area>, or requires the use of a
+   wrapper to a few select tools (gcc, ld...) to pass this flag.
+
+   Instead of polluting the toolchain's sysroot you are copying its contents
+   to a new location and polluting the contents in that new location. By
+   specifying the --sysroot option you're effectively abandoning the default
+   sysroot in favour of your own.
+   
+   Incidentally this is what buildroot does using a wrapper, when using an
+   external toolchain.
+
+   $ cp -a $(<host tuple>-gcc --your-cflags-except-sysroot -print-sysroot) \
+         /path/to/staging
+   $ ./configure --prefix=/usr --enable-foo-bar...          \
+         CC="<host tuple>-gcc --syroot=/path/to/staging"    \
+         CXX="<host tuple>-g++ --sysroot=/path/to/staging"  \
+         LD="<host tuple>-ld --sysroot=/path/to/staging"    \
+         AND_SO_ON=tuple-andsoon --sysroot=/path/to/staging"
+   $ make
+   $ make DESTDIR=/path/to/staging install
+
+3) Use separate staging and sysroot directories.
+
+   In this scenario you use a staging area to install programs, but you do
+   not pre-fill that staging area with the toolchain's sysroot. In this case
+   the compiler will find the system includes and libraries in its sysroot
+   area but you have to pass appropriate CPPFLAGS and LDFLAGS to tell it
+   where to find your headers and libraries from your staging area (or use
+   a wrapper).
+
+   $ CPPFLAGS="-I/path/to/staging/usr/include" \
+         LDFLAGS="-L/path/to/staging/lib -L/path/to/staging/usr/lib" \
+         ./configure --prefix=/usr --enable-foo-bar...
+   $ make
+   $ make DESTDIR=/path/to/staging install
+
+4) A mix of 2) and 3), using carefully crafted union mounts.
+
+   The staging area is a union mount of:
+      - the sysroot as a read-only branch
+      - the real staging area as a read-write branch
+   This also requires passing --sysroot to point to the union mount, but has
+   other advantages, such as allowing per-package staging, and a few more
+   obscure pros. It also has its disadvantages, as it potentially requires
+   non-root users to create union mounts. Additionally, union mounts are not
+   yet mainstream in the Linux kernel, so it requires patching. There is a
+   FUSE-based unionfs implementation, but development is almost stalled,
+   and there are a few gotchas...
+
+   $ (good luck!)
+
+
 It is strongly advised not to use the toolchain sysroot directory as an
-install directory for your programs/packages. If you do so, you will not be
-able to use your toolchain for another project. It is even strongly advised
-that your toolchain is chmod-ed to read-only once successfully build, so that
-you don't go polluting your toolchain with your programs/packages' files.
+install directory (i.e. option 1) for your programs/packages. If you do so,
+you will not be able to use your toolchain for another project. It is even
+strongly advised that your toolchain is chmod-ed to read-only once
+successfully install, so that you don't go polluting your toolchain with
+your programs'/packages' files. This can be achieved by selecting the
+"Render the toolchain read-only" from crosstool-NG's "Paths and misc options"
+configuration page.
 
-Thus, when you build a program/package, install it in a separate directory,
-eg. /your/root. This directory is the /image/ of what would be in the root file
-system of your target, and will contain all that your programs/packages have
-installed.
+Thus, when you build a program/package, install it in a separate, staging,
+directory and let the cross-toolchain continue to use its own, pristine,
+sysroot directory.
+
+When you are done building and want to assemble your rootfs you could simply
+take the full contents of your staging directory and use the 'populate'
+script to add in the necessary files from the sysroot. However, the staging
+area you have created will include lots of build artifacts that you won't
+necessarily want/need on your target. For example: static libraries, header
+files, linking helper files, man/info pages. You'll also need to add various
+configuration files, scripts, and directories to the rootfs so it will boot.
+
+Therefore you'll probably end up creating a separate rootfs directory which
+you will populate from the staging area, necessary extras, and then use
+crosstool-NG's populate script to add the necessary sysroot libraries.
 
 
 The 'populate' script |

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 2 of 2] docs: Add strategies for assembling root filesystems

[Yann E. MORIN]

Trevor, All,

On Wednesday 16 November 2011 23:45:27 Trevor Woerner wrote:
> # HG changeset patch
> # User Trevor Woerner <twoerner@gmail.com>
> # Date 1321482977 18000
> # Node ID 8bd0d6bdba97ba1f9a80be56daf1e6b4161cb9b4
> # Parent  c1c4d9174fbfd7d76e0e520dc6657cf239a7d718
> docs: Add strategies for assembling root filesystems.
> 
> Expand the documentation for using a crosstool-NG-generated toolchain for
> building a root filesystem for a target device.

Not much to say, I wrote much of it! :-)
But still, a few comments below...

> Signed-off-by: "Trevor Woerner" <twoerner@gmail.com>
> 
> diff -r c1c4d9174fbf -r 8bd0d6bdba97 docs/5 - Using the toolchain.txt
> --- a/docs/5 - Using the toolchain.txt	Wed Nov 16 16:06:42 2011 -0500
> +++ b/docs/5 - Using the toolchain.txt	Wed Nov 16 17:36:17 2011 -0500
[--SNIP--]
> +2) Copy the toolchain's sysroot to the 'staging' area.
[--SNIP--]
> +   $ ./configure --prefix=/usr --enable-foo-bar...          \
> +         CC="<host tuple>-gcc --syroot=/path/to/staging"    \
> +         CXX="<host tuple>-g++ --sysroot=/path/to/staging"  \
> +         LD="<host tuple>-ld --sysroot=/path/to/staging"    \
> +         AND_SO_ON=tuple-andsoon --sysroot=/path/to/staging"

It is still required to pass --build and --host.

[--SNIP--]
> +3) Use separate staging and sysroot directories.
[--SNIP--]
> +   $ CPPFLAGS="-I/path/to/staging/usr/include" \
> +         LDFLAGS="-L/path/to/staging/lib -L/path/to/staging/usr/lib" \
> +         ./configure --prefix=/usr --enable-foo-bar...

Ditto.

> +4) A mix of 2) and 3), using carefully crafted union mounts.

This one is really coplex, convoluted, and not really possible without
much trouble and expertise. It could be confusing for newcomers (if it
was not confusing enough as it is!), and experts would probably be able
to come up with that solution on their own...

Otherwise, pretty nice. I'll fix the few nit-picks here and there, and
do a re-read pass, then I'll push.

I'm glad you took the time to format it properly, I was going to suggest
this went to the docs, you read my mind! :-)

Thank you!

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 2 of 2] docs: Add strategies for assembling root filesystems

[Thomas Petazzoni]

Le Thu, 17 Nov 2011 00:29:27 +0100,
"Yann E. MORIN" <yann.morin.1998@anciens.enib.fr> a écrit :

> > Expand the documentation for using a crosstool-NG-generated
> > toolchain for building a root filesystem for a target device.
> 
> Not much to say, I wrote much of it! :-)
> But still, a few comments below...

I think this part of the documentation should point to existing
embedded Linux build systems, which already do this job, with several
advantages :

 * Well-tested and proven packages for the most common open-source
   components needed in embedded Linux systems

 * Cross-compilation issues that arise with the most common packages
   are already fixed by those build systems. For example, try to
   cross-compile Python. It is basically impossible without heavily
   patching its build system. Which is already done by OpenEmbedded,
   Buildroot, OpenBricks and all those projects.

 * The build process is reproducible. I have encountered way too many
   embedded Linux systems built manually, for which nobody knows the
   build procedure and nobody is capable of replacing a given component
   by a newer version of that component to fix a bug.

To me, suggesting people to manually create their root filesystem is a
strong mistake.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 2 of 2] docs: Add strategies for assembling root filesystems

[David Wuertele]

Thomas, this is why I created Crossplex.  It is a library of make
macros that let you specify arbitrary numbers of glibc or uclibc
toolchain configurations and target filesystem configurations, and it
is designed from the ground up to support efficient rebuilding and
parallel building.  It has recipes for many different popular
packages.

It is still a work in progress, but Motorola (and another large
company I'm not allowed to mention) are using it to build some of
their products.

I keep the github repo (https://github.com/wuertele/crossplex) more up
to date than the website (crossplex.org).

Dave Wuertele

On Thu, Nov 17, 2011 at 11:28 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Thu, 17 Nov 2011 00:29:27 +0100,
> "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr> a écrit :
>
>> > Expand the documentation for using a crosstool-NG-generated
>> > toolchain for building a root filesystem for a target device.
>>
>> Not much to say, I wrote much of it! :-)
>> But still, a few comments below...
>
> I think this part of the documentation should point to existing
> embedded Linux build systems, which already do this job, with several
> advantages :
>
>  * Well-tested and proven packages for the most common open-source
>   components needed in embedded Linux systems
>
>  * Cross-compilation issues that arise with the most common packages
>   are already fixed by those build systems. For example, try to
>   cross-compile Python. It is basically impossible without heavily
>   patching its build system. Which is already done by OpenEmbedded,
>   Buildroot, OpenBricks and all those projects.
>
>  * The build process is reproducible. I have encountered way too many
>   embedded Linux systems built manually, for which nobody knows the
>   build procedure and nobody is capable of replacing a given component
>   by a newer version of that component to fix a bug.
>
> To me, suggesting people to manually create their root filesystem is a
> strong mistake.
>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
>
> --
> For unsubscribe information see http://sourceware.org/lists.html#faq
>
>

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 2 of 2] docs: Add strategies for assembling root filesystems

[Thomas Petazzoni]

Hello David,

Le Thu, 17 Nov 2011 12:14:46 -0800,
David Wuertele <dave@crossplex.org> a écrit :

> Thomas, this is why I created Crossplex.  It is a library of make
> macros that let you specify arbitrary numbers of glibc or uclibc
> toolchain configurations and target filesystem configurations, and it
> is designed from the ground up to support efficient rebuilding and
> parallel building.  It has recipes for many different popular
> packages.
> 
> It is still a work in progress, but Motorola (and another large
> company I'm not allowed to mention) are using it to build some of
> their products.
> 
> I keep the github repo (https://github.com/wuertele/crossplex) more up
> to date than the website (crossplex.org).

Interesting.

However, the point I was making is not to advertise one particular
build system, but rather to advertise the fact that *a* build system
should be used rather than manual and unreproducible methods for
building the root filesystem.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 2 of 2] docs: Add strategies for assembling root filesystems

[Trevor Woerner]

On Thu, Nov 17, 2011 at 2:28 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> To me, suggesting people to manually create their root filesystem is a
> strong mistake.

Personally, I benefited greatly and learned many new things from this
"mistake" :-)

--
For unsubscribe information see http://sourceware.org/lists.html#faq

[PATCH 1 of 2] docs: --target versus --host

[Trevor Woerner]

# HG changeset patch
# User Trevor Woerner <twoerner@gmail.com>
# Date 1321477602 18000
# Node ID c1c4d9174fbfd7d76e0e520dc6657cf239a7d718
# Parent  c4452a1a18fb5c57b991eb75f7ff480d33e91704
docs: --target versus --host

In the very beginnings, eons ago, autotools also got confused by this
whole build vs. host vs. target, and got it wrong. Now they fixed it,
but they want to keep backward compatibility, so the --target is still
recongised, although ./configure will complain if you do so. It is
better to use --host.

Signed-off-by: "Trevor Woerner" <twoerner@gmail.com>

diff -r c4452a1a18fb -r c1c4d9174fbf docs/5 - Using the toolchain.txt
--- a/docs/5 - Using the toolchain.txt	Tue Nov 15 19:11:57 2011 +0100
+++ b/docs/5 - Using the toolchain.txt	Wed Nov 16 16:06:42 2011 -0500
@@ -11,13 +11,14 @@
 your PATH, such as:
   export PATH="${PATH}:/your/toolchain/path/bin"
 
-and then using the target tuple to tell the build systems to use your
-toolchain:
-  ./configure --target=your-target-tuple
+and then using the '--host' tuple to tell the build systems to use your
+toolchain (if the software package uses the autotools system you should
+also pass --build, for completeness):
+  ./configure --host=your-host-tuple --build=your-build-tuple
 or
-  make CC=your-target-tuple-gcc
+  make CC=your-host-tuple-gcc
 or
-  make CROSS_COMPILE=your-target-tuple-
+  make CROSS_COMPILE=your-host-tuple-
 and so on...
 
 It is strongly advised not to use the toolchain sysroot directory as an

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 1 of 2] docs: --target versus --host

[Yann E. MORIN]

Trevor, All,

On Wednesday 16 November 2011 23:45:26 Trevor Woerner wrote:
> # HG changeset patch
> # User Trevor Woerner <twoerner@gmail.com>
> # Date 1321477602 18000
> # Node ID c1c4d9174fbfd7d76e0e520dc6657cf239a7d718
> # Parent  c4452a1a18fb5c57b991eb75f7ff480d33e91704
> docs: --target versus --host
> 
> In the very beginnings, eons ago, autotools also got confused by this
> whole build vs. host vs. target, and got it wrong. Now they fixed it,
> but they want to keep backward compatibility, so the --target is still
> recongised, although ./configure will complain if you do so. It is
      ^^

"recognised" (this orignally is my typo), I'll fix.

> better to use --host.
> 
> Signed-off-by: "Trevor Woerner" <twoerner@gmail.com>
> 
> diff -r c4452a1a18fb -r c1c4d9174fbf docs/5 - Using the toolchain.txt
> --- a/docs/5 - Using the toolchain.txt	Tue Nov 15 19:11:57 2011 +0100
> +++ b/docs/5 - Using the toolchain.txt	Wed Nov 16 16:06:42 2011 -0500
> @@ -11,13 +11,14 @@
>  your PATH, such as:
>    export PATH="${PATH}:/your/toolchain/path/bin"
>  
> -and then using the target tuple to tell the build systems to use your
> -toolchain:
> -  ./configure --target=your-target-tuple
> +and then using the '--host' tuple to tell the build systems to use your
> +toolchain (if the software package uses the autotools system you should
> +also pass --build, for completeness):
> +  ./configure --host=your-host-tuple --build=your-build-tuple

Usually, I use --build before --host, it seems more logical to me.
But the order has no impact whatsoever on the result.

>  or
> -  make CC=your-target-tuple-gcc
> +  make CC=your-host-tuple-gcc
>  or
> -  make CROSS_COMPILE=your-target-tuple-
> +  make CROSS_COMPILE=your-host-tuple-
>  and so on...

Aren't all these build/host/target things sneaky and confusing? ;-)

To make things clear, I'd like to add this snippet:

    (Note: in the above examples, 'host' refers to the host of your program,
    not the host of the toolchain; and 'build' refers to the machine where
    you build your program, that is the host of the toolchain.)

What do you think?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 1 of 2] docs: --target versus --host

[Trevor Woerner]

Yann,

On Wed, Nov 16, 2011 at 6:18 PM, Yann E. MORIN
<yann.morin.1998@anciens.enib.fr> wrote:
>>  or
>> -  make CC=your-target-tuple-gcc
>> +  make CC=your-host-tuple-gcc
>>  or
>> -  make CROSS_COMPILE=your-target-tuple-
>> +  make CROSS_COMPILE=your-host-tuple-
>>  and so on...
>
> Aren't all these build/host/target things sneaky and confusing? ;-)
>
> To make things clear, I'd like to add this snippet:
>
>    (Note: in the above examples, 'host' refers to the host of your program,
>    not the host of the toolchain; and 'build' refers to the machine where
>    you build your program, that is the host of the toolchain.)
>
> What do you think?

Haha, yes! I sat there looking at it for a while and wondering which
would be correct :-)

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 1 of 2] docs: --target versus --host

[Thomas Petazzoni]

Le Thu, 17 Nov 2011 00:18:27 +0100,
"Yann E. MORIN" <yann.morin.1998@anciens.enib.fr> a écrit :

> > your +toolchain (if the software package uses the autotools system
> > you should +also pass --build, for completeness):

config.guess makes a very good job at finding what is the tuple of the
current machine, which is the build machine. And the guess done by
config.guess is probably better than the random tuple the developer
would pass to --build.

Therefore, I would rather state that if the software package uses the
autotools system, then you should let autotools guess what's the tuple
of the build system.

Any reason not to do so ?

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: [PATCH 1 of 2] docs: --target versus --host

[Trevor Woerner]

On Thu, Nov 17, 2011 at 2:26 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> config.guess makes a very good job at finding what is the tuple of the
> current machine, which is the build machine. And the guess done by
> config.guess is probably better than the random tuple the developer
> would pass to --build.
>
> Therefore, I would rather state that if the software package uses the
> autotools system, then you should let autotools guess what's the tuple
> of the build system.

That's a good point.

Additionally I have found one tiny corner case where not specifying
the --build option was a good thing.

When using a 64-bit x86 system to generate a rootfs for a 32-bit x86
system, if you specify --build you are telling the ./configure script
you are cross compiling and it will therefore not attempt any tests
which require a running binary. In this case bash-4.2 (for example)
will automatically assume conservative values if you don't tell it
otherwise e.g.:

     checking for printf floating point output in hex notation...
configure: WARNING: cannot check printf if cross compiling --
defaulting to no

whereas if you don't specify --build one of the first things the
./configure script will do is to check if it can run a simple binary.
In this situation it will discover that it can, in fact, run 32-bit
binaries and will run any such tests that require running a test
program:

     checking for printf floating point output in hex notation... yes

The relevant part of the config.log for the case where I specify --build is:

     configure:29762: checking for printf floating point output in hex notation
     configure:29768: WARNING: cannot check printf if cross compiling
-- defaulting to no
     configure:29832: result: no

whereas the relevant part of the config.log for when I only specify --host is:

     configure:29762: checking for printf floating point output in hex notation
     configure:29800: i686-nptl-linux-gnu-gcc -o conftest -g -O2
conftest.c -ldl  >&5
     conftest.c: In function 'main':
     conftest.c:356:2: warning: incompatible implicit declaration of
built-in function 'exit'
     configure:29804: $? = 0
     configure:29810: ./conftest
     configure:29814: $? = 0
     configure:29832: result: yes

Of course if you are targeting a platform whose binaries you can't run
it makes no difference (provided you've specified your --build tuple
accurately).

So considering a person might specify the tuple incorrectly and the
./configure script is more robust it sounds like good advice to not
specify the --build tuple.

--
For unsubscribe information see http://sourceware.org/lists.html#faq