Missing .gnu_debugdata section on ARM platform when libdw is used by systemd-coredump

Missing .gnu_debugdata section on ARM platform when libdw is used by systemd-coredump

[Tino Mettler]

Hi,

I have a system running on 2 different architectures: AMD64 and ARM. When a coredump happens, I want systemd-coredump to generate a stack trace of the crashing application. Systemd depends on elfutils for this feature. I use binaries with minidebuginfo (the LZMA compressed symbol table in the .gnu_debugdata section) on both architectures.

I get a stack trace on AMD64, but not on ARM. While debugging this, I saw that find_aux_sym() from dwfl_module_getdwarf.c does not find a .gnu_debugdata section when iterating through the ELF using elf_nextscn().

However, it finds a .gnu_debuglink section. I inspected the executable and verified that it contains a .gnu_debugdata section and it looks fine. Interestingly, there is no .gnu_debuglink section in the executable despite elf_nextscn() seems to find one.

It looks like libdw does not process the actual executable, but a modified variant, and the .gnu_debugdata section gets lost at some point on my ARM device. Can you give me a hint where I need to look at? Both devices run a different kernel with a different configuration. Could that be related?

I also tried gdb using the coredump file from systemd-coredump and the same executable, and a stack trace worked as expected.

Regards,
Tino

Re: Missing .gnu_debugdata section on ARM platform when libdw is used by systemd-coredump

[Mark Wielaard]

Hi Tino,

On Tue, 2021-05-04 at 14:15 +0200, Tino Mettler via Elfutils-devel
wrote:
> I have a system running on 2 different architectures: AMD64 and ARM.
> When a coredump happens, I want systemd-coredump to generate a stack
> trace of the crashing application. Systemd depends on elfutils for
> this feature. I use binaries with minidebuginfo (the LZMA compressed
> symbol table in the .gnu_debugdata section) on both architectures.
> 
> I get a stack trace on AMD64, but not on ARM. While debugging this, I
> saw that find_aux_sym() from dwfl_module_getdwarf.c does not find a
> .gnu_debugdata section when iterating through the ELF using
> elf_nextscn().
> 
> However, it finds a .gnu_debuglink section. I inspected the
> executable and verified that it contains a .gnu_debugdata section and
> it looks fine. Interestingly, there is no .gnu_debuglink section in
> the executable despite elf_nextscn() seems to find one.
> 
> It looks like libdw does not process the actual executable, but a
> modified variant, and the .gnu_debugdata section gets lost at some
> point on my ARM device. Can you give me a hint where I need to look
> at? Both devices run a different kernel with a different
> configuration. Could that be related?
> 
> I also tried gdb using the coredump file from systemd-coredump and
> the same executable, and a stack trace worked as expected.

If possible you might want to post the core file and/or executables
somewhere so others can inspect them. I am not completely sure how the
.gnu_debugdata (aux symbols) is related. Are you getting a backtrace,
but not function symbols for the addresses?

For ARM it might depend on whether the executable contains an .eh_frame
sections. If it has (or the .debug file has an .debug_frame section)
then libdw should be able to produce a backtrace.

But there are also ARM binaries which only come with IDX data, which
libdw doesn't handle.

Cheers,

Mark

Re: Missing .gnu_debugdata section on ARM platform when libdw is used by systemd-coredump

[Tino Mettler]

Hi Mark,

Am 06.05.2021 um 02:49 schrieb Mark Wielaard <mark@klomp.org>:
> 
> Hi Tino,
> 
> On Tue, 2021-05-04 at 14:15 +0200, Tino Mettler via Elfutils-devel
> wrote:
>> I have a system running on 2 different architectures: AMD64 and ARM.
>> When a coredump happens, I want systemd-coredump to generate a stack
>> trace of the crashing application. Systemd depends on elfutils for
>> this feature. I use binaries with minidebuginfo (the LZMA compressed
>> symbol table in the .gnu_debugdata section) on both architectures.
>> 
>> I get a stack trace on AMD64, but not on ARM. While debugging this, I
>> saw that find_aux_sym() from dwfl_module_getdwarf.c does not find a
>> .gnu_debugdata section when iterating through the ELF using
>> elf_nextscn().
>> 
>> However, it finds a .gnu_debuglink section. I inspected the
>> executable and verified that it contains a .gnu_debugdata section and
>> it looks fine. Interestingly, there is no .gnu_debuglink section in
>> the executable despite elf_nextscn() seems to find one.
>> 
>> It looks like libdw does not process the actual executable, but a
>> modified variant, and the .gnu_debugdata section gets lost at some
>> point on my ARM device. Can you give me a hint where I need to look
>> at? Both devices run a different kernel with a different
>> configuration. Could that be related?
> 
> If possible you might want to post the core file and/or executables
> somewhere so others can inspect them. I am not completely sure how the
> .gnu_debugdata (aux symbols) is related. Are you getting a backtrace,
> but not function symbols for the addresses?

I'll try to provide a minimal example for this. I get only frame #0 of the
stack and no complete stack trace.

> 
> For ARM it might depend on whether the executable contains an .eh_frame
> sections. If it has (or the .debug file has an .debug_frame section)
> then libdw should be able to produce a backtrace.
> 
> But there are also ARM binaries which only come with IDX data, which
> libdw doesn't handle.

There is a .eh_frame in the binary.

Regards,
Tino