Modern gcc on old embedded system (AKA libgcc and crtstuff)

Modern gcc on old embedded system (AKA libgcc and crtstuff)

[Cristian Morales Vega]

Hi,

I have an old OpenWRT system (gcc 4.5 / uClibc 0.9.32) which I can't
update. I would like to use a modern toolchain (i.e. C++11) to build
binaries for it, and since the storage space is limited, reusing as
much as possible from the existing libraries. I understand I am going
to need the updated libstdc++.so.6 library, and I ___guess___ I should
also be using the updated libgcc (it's not very clear to me how gcc
decides if it can use, for example, __divmodti4, which in my computer
is versioned in libgcc with GCC_7.0.0).

I have been trying to do this with a limited understanding of how all
the stack unwinding stuff for exceptions works. But it has actually
(only apparently?) work quite well with other systems, even with gcc
4.6 and uClibc 0.9.33. It actually works well with this gcc 4.5 /
uClibc 0.9.32 until I start throwing exceptions.

The original OpenWRT, even using those ancient tools/libraries, did
use modern enough binutils to have a linker with --eh-frame-hdr
support. The existing ELF files do have the GNU_EH_FRAME program
header / .eh_frame_hdr section. And the C library does implement
dl_iterate_phdr.

I see gcc doesn't make use of the GNU_EH_FRAME program header if using
uClibc (https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libgcc/crtstuff.c;h=3f769a1c6603bbafb5c85d9fb83a57bd187f4d98;hb=HEAD#l114).
I guess because of this, even if the dl_iterate_phdr/GNU_EH_FRAME
support is there, the gcc driver seems to have used --shared-libgcc in
every single shared library in the OpenWRT system. But here is where I
get a bit lost:

- As far as I understand all the unwinding logic is in libgcc, so why
does the C library matter?

- I see glibc does implement some unwind logic using PT_GNU_EH_FRAME
(https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/generic/unwind-dw2-fde-glibc.c;h=f74296d361f2806c5b98ae841fe5e1ab3d9e736e;hb=HEAD).
Again, no idea why, there is no reference to PT_GNU_EH_FRAME in either
uClibc-ng or musl source code.

- OpenWRT did actually patch gcc to remove the uClibc check
(https://github.com/openwrt/archive/blob/v15.05/toolchain/gcc/patches/4.6-linaro/860-uclibc_use_eh_frame.patch).
I have needed to apply the same patch to my modern gcc build to make
exceptions work in my originally gcc 4.6 / uClibc 0.9.33 system.

My understanding of what happens in my gcc 4.6 / uClibc 0.9.33 system
is the following: since OpenWRT patched gcc it does
USE_PT_GNU_EH_FRAME. Meaning the libraries contain crtbegin/crtend
code that doesn't register the stack unwinding information, they rely
on libgcc finding the information via PT_GNU_EH_FRAME. If I don't
patch my modern gcc build, libgcc is going to try to find the unwind
information via the pre-PT_GNU_EH_FRAME method and fail to find it.
Once I patch my modern gcc build to USE_PT_GNU_EH_FRAME everything
will use dl_iterate_phdr/PT_GNU_EH_FRAME and work fine.
Does that make sense?

Now, I guess the non-PT_GNU_EH_FRAME method of registering unwind
information has changed over the years. So my gcc 4.5 / uClibc 0.9.32
system doesn't work because of this. So I guess my options would be:
- Replace the crtbegin.o/crtend.o files from my modern gcc toolchain
with the ones from gcc 4.5 and use the libgcc from gcc 4.5. New and
old shared libraries will use the same, old, method to register the
unwind information and hopefully exceptions will work.

- Patch my modern gcc build to remove the UCLIBC check. Even if the
shared libraries built with gcc 4.5 are going to be using the old
crtstuff, my updated libgcc will just ignore their registration of the
unwind information and find it via dl_iterate_phdr/PT_GNU_EH_FRAME.
Not sure what other stuff there is in crtsuff that could make things
break if some shared libraries use a different version than others.

I am going to try anyway. But does any of this make sense? Any gotcha
I should keep in mind?

And thanks, I'm sure you were hoping not having to think about the
pre-PT_GNU_EH_FRAME world in 2020 any more.

Re: Modern gcc on old embedded system (AKA libgcc and crtstuff)

[Richard Sandiford]

Cristian Morales Vega via Gcc-help <gcc-help@gcc.gnu.org> writes:
> Hi,
>
> I have an old OpenWRT system (gcc 4.5 / uClibc 0.9.32) which I can't
> update. I would like to use a modern toolchain (i.e. C++11) to build
> binaries for it, and since the storage space is limited, reusing as
> much as possible from the existing libraries. I understand I am going
> to need the updated libstdc++.so.6 library, and I ___guess___ I should
> also be using the updated libgcc (it's not very clear to me how gcc
> decides if it can use, for example, __divmodti4, which in my computer
> is versioned in libgcc with GCC_7.0.0).

Using the updated libgcc.so is best, including for old binaries.
It's supposed to be backward-compatible.

However, if OpenWRT patched libgcc to support PT_GNU_EH_FRAME for
uClibc, then yeah, you'll need to make your new libgcc.so do the same,
otherwise the new version won't be backwards compatible.

> I have been trying to do this with a limited understanding of how all
> the stack unwinding stuff for exceptions works. But it has actually
> (only apparently?) work quite well with other systems, even with gcc
> 4.6 and uClibc 0.9.33. It actually works well with this gcc 4.5 /
> uClibc 0.9.32 until I start throwing exceptions.
>
> The original OpenWRT, even using those ancient tools/libraries, did
> use modern enough binutils to have a linker with --eh-frame-hdr
> support. The existing ELF files do have the GNU_EH_FRAME program
> header / .eh_frame_hdr section. And the C library does implement
> dl_iterate_phdr.
>
> I see gcc doesn't make use of the GNU_EH_FRAME program header if using
> uClibc (https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libgcc/crtstuff.c;h=3f769a1c6603bbafb5c85d9fb83a57bd187f4d98;hb=HEAD#l114).
> I guess because of this, even if the dl_iterate_phdr/GNU_EH_FRAME
> support is there, the gcc driver seems to have used --shared-libgcc in
> every single shared library in the OpenWRT system. But here is where I
> get a bit lost:
>
> - As far as I understand all the unwinding logic is in libgcc, so why
> does the C library matter?

I think the purpose of the C library check is simply to make sure
that there's a compatible implementation dl_iterate_phdr.  This can't
be done via configure-time link tests since it would create a
chicken-and-egg problem.

> - I see glibc does implement some unwind logic using PT_GNU_EH_FRAME
> (https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/generic/unwind-dw2-fde-glibc.c;h=f74296d361f2806c5b98ae841fe5e1ab3d9e736e;hb=HEAD).
> Again, no idea why, there is no reference to PT_GNU_EH_FRAME in either
> uClibc-ng or musl source code.

This is there to maintain ABI compatibility with ancient versions of glibc:

#if !defined _LIBC || SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_2_5)

> - OpenWRT did actually patch gcc to remove the uClibc check
> (https://github.com/openwrt/archive/blob/v15.05/toolchain/gcc/patches/4.6-linaro/860-uclibc_use_eh_frame.patch).
> I have needed to apply the same patch to my modern gcc build to make
> exceptions work in my originally gcc 4.6 / uClibc 0.9.33 system.
>
> My understanding of what happens in my gcc 4.6 / uClibc 0.9.33 system
> is the following: since OpenWRT patched gcc it does
> USE_PT_GNU_EH_FRAME. Meaning the libraries contain crtbegin/crtend
> code that doesn't register the stack unwinding information, they rely
> on libgcc finding the information via PT_GNU_EH_FRAME. If I don't
> patch my modern gcc build, libgcc is going to try to find the unwind
> information via the pre-PT_GNU_EH_FRAME method and fail to find it.
> Once I patch my modern gcc build to USE_PT_GNU_EH_FRAME everything
> will use dl_iterate_phdr/PT_GNU_EH_FRAME and work fine.
> Does that make sense?

Yeah, that sounds right.

> Now, I guess the non-PT_GNU_EH_FRAME method of registering unwind
> information has changed over the years. So my gcc 4.5 / uClibc 0.9.32
> system doesn't work because of this. So I guess my options would be:

The registration scheme ABI hasn't changed since GCC 3.0 as far as I know,
so…

> - Replace the crtbegin.o/crtend.o files from my modern gcc toolchain
> with the ones from gcc 4.5 and use the libgcc from gcc 4.5. New and
> old shared libraries will use the same, old, method to register the
> unwind information and hopefully exceptions will work.

…this shouldn't be necessary.

> - Patch my modern gcc build to remove the UCLIBC check. Even if the
> shared libraries built with gcc 4.5 are going to be using the old
> crtstuff, my updated libgcc will just ignore their registration of the
> unwind information and find it via dl_iterate_phdr/PT_GNU_EH_FRAME.
> Not sure what other stuff there is in crtsuff that could make things
> break if some shared libraries use a different version than others.

Yeah, that sounds like the way to go.  Note that it is possible to
build libgcc to support both the registration and PT_GNU_EH_FRAME
scheme simultaneously via --enable-explicit-exception-frame-registration,
but that would only work here if you remove the UCLIBC check.  Since
removing the check should be enough on its own, I don't think the
configuration option is going to help.

Thanks,
Richard