[Bug backtrace/27987] New: aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[Bug backtrace/27987] New: aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[ctice42 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

            Bug ID: 27987
           Summary: aarch64 prologue unwinder does not correctly handle
                    -fpatchable-function-entry=2 flag
           Product: gdb
           Version: HEAD
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: backtrace
          Assignee: unassigned at sourceware dot org
          Reporter: ctice42 at gmail dot com
  Target Milestone: ---

While debugging kgdb on a linux kernel built for aarch64 (inside chromiumos),
we found that if we built the kernel with -fpatchable-function-entry=2 and
tried to do a backtrace, the aarch64 prologue unwinder would not properly
handle/ignore the nops at the start of the function prologue, and would then
get confused about the stack, and would hit an assertion failure and crash gdb:

~/trunk/src/third_party/kernel/v5.4 $ aarch64-cros-linux-gnu-gdb         
vmlinux          -ex "target remote localhost:1234"
GNU gdb (Chromium OS 9.2.20200923 vanilla) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
…
Reading symbols from vmlinux...
Remote debugging using localhost:1234
arch_kgdb_breakpoint ()
    at
/home/cmtice/trunk/src/third_party/kernel/v5.4/./arch/arm64/include/asm/kgdb.h:21
21              asm ("brk %0" : : "I" (KGDB_COMPILED_DBG_BRK_IMM));
(gdb) bt
#0  arch_kgdb_breakpoint ()
    at
/home/cmtice/trunk/src/third_party/kernel/v5.4/./arch/arm64/include/asm/kgdb.h:21
#1  kgdb_breakpoint () at kernel/debug/debug_core.c:1208
/var/tmp/portage/cross-aarch64-cros-linux-gnu/gdb-9.2.20200923-r5/work/gdb-9.2/gdb/inline-frame.c:155:
internal-error: void inline_frame_this_id(struct frame_info *, void **, struct
frame_id *): Assertion `frame_id_p (*this_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) y



If we build the kernel WITHOUT the -fpatchable-function-entry=2 flag and do the
same backtrace as the same location, we do get a corrupted stack error message,
but we don't hit the assert or the crash, we get a different (more correct)
backtrace, and because GDB doesn't actually crash, we can continue to do some
useful debugging:

~/trunk/src/third_party/kernel/v5.4 $ aarch64-cros-linux-gnu-gdb         
vmlinux          -ex "target remote localhost:1234"
GNU gdb (Chromium OS 9.2.20200923 vanilla) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
…
Reading symbols from vmlinux...
Remote debugging using localhost:1234
arch_kgdb_breakpoint ()
    at
/home/cmtice/trunk/src/third_party/kernel/v5.4/./arch/arm64/include/asm/kgdb.h:21
21              asm ("brk %0" : : "I" (KGDB_COMPILED_DBG_BRK_IMM));
(gdb) bt
#0  arch_kgdb_breakpoint ()
    at
/home/cmtice/trunk/src/third_party/kernel/v5.4/./arch/arm64/include/asm/kgdb.h:21
#1  kgdb_breakpoint () at kernel/debug/debug_core.c:1208
#2  0xffff8000131d0cd0 in kgdb_initial_breakpoint ()
    at kernel/debug/debug_core.c:1011
#3  dbg_late_init () at kernel/debug/debug_core.c:1026
#4  0xffff8000131b0bc4 in start_kernel () at init/main.c:1044
#5  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) 


Sadly, the only test case I have been able to find is complex to reproduce, as
it
involves downloading and setting up a chromiumos chroot, then downloading,
configuring and building a copy of the linux kernel, then running the kernel in
qemu in the chromiumos chroot and running gdb in a separate terminal window. 
If someone really wants/needs the complete set of reproduction steps, let me
know and I will supply them.

I did debug this issue enough to verify that it is indeed the aarch64 prologue
unwinder that has the issue, but my knowledge of aarch64 is not good enough to
allow me to fix the issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[ctice42 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

ctice42 at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alan.hayward at arm dot com,
                   |                            |ctice42 at gmail dot com,
                   |                            |luis.machado at linaro dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[luis.machado at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

--- Comment #1 from Luis Machado <luis.machado at linaro dot org> ---
If you could share the prologues that AArch64's GDB is failing to recognize, we
can try to improve the heuristics a little to cover such cases.

A simple disassemble dump for those prologue should be enough for an
investigation to proceed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[ctice42 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

--- Comment #2 from ctice42 at gmail dot com ---
Here is the disassembly of the first few instructions for the first four
functions on the stack:

Dump of assembler code for function kgdb_breakpoint:
   0xffff800010323a50 <+0>:             bti     c
   0xffff800010323a54 <+4>:             mov     x9, x30
   0xffff800010323a58 <+8>:             nop
   0xffff800010323a5c <+12>:    paciasp
   0xffff800010323a60 <+16>:    stp     x29, x30, [sp, #-16]!
   0xffff800010323a64 <+20>:    mov     x29, sp

+++++++++++++++++++++++++++++++++++++++

 Dump of assembler code for function sysrq_handle_dbg:
  0xffff80001032404c <+0>:              bti     c
   0xffff800010324050 <+4>:             mov     x9, x30
   0xffff800010324054 <+8>:             nop
   0xffff800010324058 <+12>:    paciasp
   0xffff80001032405c <+16>:    stp     x29, x30, [sp, #-16]!
   0xffff800010324060 <+20>:    mov     x29, sp

+++++++++++++++++++++++++++++++++++++++

Dump of assembler code for function __handle_sysrq:
   0xffff8000111ef174 <+0>:     bti     c
   0xffff8000111ef178 <+4>:     mov     x9, x30
   0xffff8000111ef17c <+8>:     nop
   0xffff8000111ef180 <+12>:    paciasp
   0xffff8000111ef184 <+16>:    sub     sp, sp, #0x70
   0xffff8000111ef188 <+20>:    stp     x29, x30, [sp, #16]

+++++++++++++++++++++++++++++++++++++++
Dump of assembler code for function __handle_sysrq:
   0xffff8000111ef174 <+0>:     bti     c
   0xffff8000111ef178 <+4>:     mov     x9, x30
   0xffff8000111ef17c <+8>:     nop
   0xffff8000111ef180 <+12>:    paciasp
   0xffff8000111ef184 <+16>:    sub     sp, sp, #0x70
   0xffff8000111ef188 <+20>:    stp     x29, x30, [sp, #16]


Please let me know if you need more.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[luis.machado at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

--- Comment #3 from Luis Machado <luis.machado at linaro dot org> ---
Thanks. I'll take a look at this. I wonder if the BTI instructions will also be
a problem in this case. I don't remember GDB handling those.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[luis.machado at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

Luis Machado <luis.machado at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2021-06-29

--- Comment #4 from Luis Machado <luis.machado at linaro dot org> ---
On a brief look, it may not be the prologue analyzer's fault. When using
"-fpatchable-function-entry=N,M", the entry point of the function is populated
with N nops, and the entry point will point at the (M - 1)-th nop. The function
symbol's address is adjusted to reflect this, but the DWARF information for the
function symbol is not.

objdump output:

00000000000010ec <t_small_values>:
    10ec:       d503201f        nop
    10f0:       d503201f        nop
    10f4:       d503201f        nop
    10f8:       d503201f        nop
    10fc:       d100c3ff        sub     sp, sp, #0x30


DWARF:

 <1><c63>: Abbrev Number: 26 (DW_TAG_subprogram)
    <c64>   DW_AT_external    : 1
    <c64>   DW_AT_name        : (indirect string, offset: 0x1e77):
t_small_values
    <c68>   DW_AT_decl_file   : 1
    <c69>   DW_AT_decl_line   : 288
    <c6b>   DW_AT_decl_column : 1
    <c6c>   DW_AT_type        : <0x3f>
    <c70>   DW_AT_low_pc      : 0x10fc
    <c78>   DW_AT_high_pc     : 0xb8
    <c80>   DW_AT_frame_base  : 1 byte block: 9c        (DW_OP_call_frame_cfa)
    <c82>   DW_AT_GNU_all_call_sites: 1
    <c82>   DW_AT_sibling     : <0xd27>


Notice how DW_AT_low_pc points at 0x10fc, the first real instruction of the
function.

I'm not sure if this is by design, but anything coming before the real entry
point to the function will be ignored by GDB's prologue analyzer.

GDB may need to handle patchable functions in a better way, but this may hint
at a problem in GCC's DWARF generation. If GCC didn't choose to not emit the
adjusted DT_AT_low_pc by design, that is.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[luis.machado at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

--- Comment #5 from Luis Machado <luis.machado at linaro dot org> ---
I have confirmed with GCC developers that this is not by design. So we have a
bug in GCC for not emitting correct DWARF information for patchable functions.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[luis.machado at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

--- Comment #6 from Luis Machado <luis.machado at linaro dot org> ---
This has been reported before:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98776

Sounds like we need a bit more pressure to get GCC to fix it.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug backtrace/27987] aarch64 prologue unwinder does not correctly handle -fpatchable-function-entry=2 flag

[luis.machado at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27987

Luis Machado <luis.machado at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |NOTABUG
             Status|WAITING                     |RESOLVED

--- Comment #7 from Luis Machado <luis.machado at linaro dot org> ---
Closing as NOTABUG, since this is not a GDB bug.

Please open a new ticket if the BTI instructions are confusing the prologue
though, and I'll take a look at that.

-- 
You are receiving this mail because:
You are on the CC list for the bug.