Lightning talk notes on glibc

Lightning talk notes on glibc

[Fangrui Song]

Greetings, I will give a lightning talk in a small meetup on some GNU
toolchain projects and I am writing notes:)

Here is that I have now for glibc. Happy to see more points from you experts:)


## glibc

Repository: <https://sourceware.org/git/gitweb.cgi?p=glibc.git>
Wiki: <https://sourceware.org/glibc/wiki/>
Bugzilla: <https://sourceware.org/bugzilla/>
Mailing lists: `{libc-announce,libc-alpha,libc-locale,libc-stable,libc-help}@sourceware.org`

A very unfortunate fact: glibc can only be built with `-O2`, not `-O0` or `-O1`.
If you want to have an un-optimized debug build, deleting an object file and recompiling it with `-g` usually works.
Another workaround is `#pragma GCC optimize ("O0")`.

The `-O2` issue is probably related to (1) expected inlining and (2) avoiding dynamic relocations.

Run the following commands to populate `/tmp/glibc-many` with toolchains.
Caution: please make sure the target file system has tens of gigabytes.

Preparation:
```sh
scripts/build-many-glibcs.py /tmp/glibc-many checkout
scripts/build-many-glibcs.py /tmp/glibc-many host-libraries

scripts/build-many-glibcs.py /tmp/glibc-many compilers aarch64-linux-gnu
scripts/build-many-glibcs.py /tmp/glibc-many compilers powerpc64le-linux-gnu
scripts/build-many-glibcs.py /tmp/glibc-many compilers sparc64-linux-gnu
```

The `glibcs` command will delete the glibc build directory, build glibc, and run `make check`.

```sh
scripts/build-many-glibcs.py /tmp/glibc-many glibcs aarch64-linux-gnu
# Find the logs and test results under /tmp/glibc-many/logs/glibcs/aarch64-linux-gnu/

scripts/build-many-glibcs.py /tmp/glibc-many glibcs powerpc64le-linux-gnu

scripts/build-many-glibcs.py /tmp/glibc-many glibcs sparc64-linux-gnu
```

During development, some interesting targets:

((((((It'd be nice to list more targets here))))))))
```sh
make -C Debug lib  # meaning?
make -C Debug objs  # meaning?
make -C Debug others  # meaning?
make -C Debug check-abi
```

Re: Lightning talk notes on glibc

[Adhemerval Zanella]

On 20/01/2021 04:48, Fangrui Song wrote:
> Greetings, I will give a lightning talk in a small meetup on some GNU
> toolchain projects and I am writing notes:)
> 
> Here is that I have now for glibc. Happy to see more points from you experts:)
> 
> 
> ## glibc
> 
> Repository: <https://sourceware.org/git/gitweb.cgi?p=glibc.git>
> Wiki: <https://sourceware.org/glibc/wiki/>
> Bugzilla: <https://sourceware.org/bugzilla/>
> Mailing lists: `{libc-announce,libc-alpha,libc-locale,libc-stable,libc-help}@sourceware.org`
> 
> A very unfortunate fact: glibc can only be built with `-O2`, not `-O0` or `-O1`.
> If you want to have an un-optimized debug build, deleting an object file and recompiling it with `-g` usually works.
> Another workaround is `#pragma GCC optimize ("O0")`.

Unfortunately this trick might still fail depending whether the routine
is also used on loader, although for library code it does help.

> 
> The `-O2` issue is probably related to (1) expected inlining and (2) avoiding dynamic relocations.

Yes and I started to take a look if it would be possible.  I could build
a un-optimized libc.so, but it still fails at runtime for some reason.

I think a better way would be to try to simplify the bootstrap and document
the requirements of the function call and arch-specific code.  The idea is
to move all the required bootstrap code to maybe a single file and just
build it in an optimized manner.

As static-pie and ifunc has show, the bootstrap code is still somewhat 
complex.

> 
> Run the following commands to populate `/tmp/glibc-many` with toolchains.
> Caution: please make sure the target file system has tens of gigabytes.
> 
> Preparation:
> ```sh
> scripts/build-many-glibcs.py /tmp/glibc-many checkout
> scripts/build-many-glibcs.py /tmp/glibc-many host-libraries
> 
> scripts/build-many-glibcs.py /tmp/glibc-many compilers aarch64-linux-gnu
> scripts/build-many-glibcs.py /tmp/glibc-many compilers powerpc64le-linux-gnu
> scripts/build-many-glibcs.py /tmp/glibc-many compilers sparc64-linux-gnu
> ```
> 

You might want to use the '--shallow' option on checkout to minimize both
the disk and network usage (since at principle you won't need to full git
history) and the --strip on compiler option.

> The `glibcs` command will delete the glibc build directory, build glibc, and run `make check`.

If you want all the build directories to further inspection you might use
'--keep all' option on either compilers or glibcs command.

> 
> ```sh
> scripts/build-many-glibcs.py /tmp/glibc-many glibcs aarch64-linux-gnu
> # Find the logs and test results under /tmp/glibc-many/logs/glibcs/aarch64-linux-gnu/
> 
> scripts/build-many-glibcs.py /tmp/glibc-many glibcs powerpc64le-linux-gnu
> 
> scripts/build-many-glibcs.py /tmp/glibc-many glibcs sparc64-linux-gnu
> ```
> 
> During development, some interesting targets:
> 
> ((((((It'd be nice to list more targets here))))))))
> ```sh
> make -C Debug lib  # meaning?
> make -C Debug objs  # meaning?
> make -C Debug others  # meaning?
> make -C Debug check-abi
> ```

Re: Lightning talk notes on glibc

[Carlos O'Donell]

On 1/20/21 2:48 AM, Fangrui Song wrote:
> The `-O2` issue is probably related to (1) expected inlining and (2) avoiding dynamic relocations.
Not "probably related to" but absolutely about (2).

And (1) is directly related to (2). If a function is not inlined you will
generate relocations that cannot be handled during the bootstrap.

The core bootstrap must be free of relocations in this sense.

As Adhemerval points out there is value in minimizing the bootstrap set
to allow for more complex debugging tools be used *after* the bootstrap.
Like being able to run with sanitizers turned on.

-- 
Cheers,
Carlos.

Re: Lightning talk notes on glibc

[Carlos O'Donell]

On 1/20/21 2:48 AM, Fangrui Song wrote:
> During development, some interesting targets:
> 
> ((((((It'd be nice to list more targets here))))))))
> ```sh
> make -C Debug lib  # meaning?
> make -C Debug objs  # meaning?
> make -C Debug others  # meaning?
> make -C Debug check-abi
> ```
 
Note 'make help' which should be updated to list useful
targets that developer want to use.

Note debugglibc.sh for debugging test and attaching to tests
that run in containers.

Lastly 'make test' with options to run one test (which usually
works depending on the kind of test to execute).

-- 
Cheers,
Carlos.

Re: Lightning talk notes on glibc

[Carlos O'Donell]

On 1/20/21 2:48 AM, Fangrui Song wrote:
> Another workaround is `#pragma GCC optimize ("O0")`.

I use '__attribute__ ((optimize("-O0")))' on a per-function
basis to debug more deeply only specific functions and avoid
possible consequences of the resulting changes.

-- 
Cheers,
Carlos.