x64 machine code and stack frames

x64 machine code and stack frames

[Dov Grobgeld]

Hello,

I've inherited some clever x64 machine code for linux that creates an
machine code wrapper around a c-function call. I guess that in higher
language terms the code might be called a decorator or a closure. The
code is functioning well, but with the unfortunate artifact that when
the wrapper is called, it gobbles the stack trace in gdb.

From what I have learned from the net gdb uses
https://en.wikipedia.org/wiki/DWARF as a guide for separating the
stack frames in the stack. This works well for static code, but
obviously code generated and called at run time isn't registered in
the DWARF framework.

My question is if there is any way to rescue the stack trace in this situation?

Here is a similar code that shows the problem.

     1      typedef int (*ftype)(int x);
     2      int wuz(int x) { return x + 7; }
     3      int wbar(int x) { return wuz(x)+5; }
     4      int main(int argc, char **argv)
     5      {
     6        const unsigned char wbarcode[] = {
     7          0x55 ,                            //  push   %rbp
     8          0x48,0x89,0xe5 ,                  //  mov    %rsp,%rbp
     9          0x48,0x83,0xec,0x08 ,             //  sub    $0x8,%rsp
    10          0x89,0x7d,0xfc ,                  //  mov    %edi,-0x4(%rbp)
    11          0x8b,0x45,0xfc ,                  //  mov    -0x4(%rbp),%eax
    12          0x89,0xc7 ,                       //  mov    %eax,%edi
    13          0x48,0xc7,0xc0,0xf6,0x04,0x40,00, // mov    $0x4004f6,%rax
    14          0xff,0xd0,                        //  callq  *%rax
    15          0x83,0xc0,0x05 ,                  //  add    $0x5,%eax
    16          0xc9 ,                            //  leaveq
    17          0xc3                              //  retq
    18        };
    19
    20        int wb = wbar(5);
    21        ftype wf = (ftype)wbarcode;
    22        int fwb = wf(5);
    23      }

Compile it by:

    gcc -o mcode mcode.c
    execstack -s mcode

and run it in gdb by:

    $ gdb mcode
    (gdb) break wuz

If we disassemble wbar we get something very similar to the byte
sequence in wbarcode[]. The only difference is that I changed the
calling convention for calling wuz().

    (gdb) disas/r wbar
    Dump of assembler code for function wbar:
       0x0000000000400505 <+0>: 55      push   %rbp
       0x0000000000400506 <+1>: 48 89 e5        mov    %rsp,%rbp
       0x0000000000400509 <+4>: 48 83 ec 08     sub    $0x8,%rsp
       0x000000000040050d <+8>: 89 7d fc        mov    %edi,-0x4(%rbp)
       0x0000000000400510 <+11>:        8b 45 fc        mov    -0x4(%rbp),%eax
       0x0000000000400513 <+14>:        89 c7   mov    %eax,%edi
       0x0000000000400515 <+16>:        e8 dc ff ff ff  callq  0x4004f6 <wuz>
       0x000000000040051a <+21>:        83 c0 05        add    $0x5,%eax
       0x000000000040051d <+24>:        c9      leaveq
       0x000000000040051e <+25>:        c3      retq
    End of assembler dump.

If we now run the program it will stop twice in wuz(). The first time
through our c-call and we can ask for a stack trace through bt.

    Breakpoint 3, wuz (x=5) at mcode.c:2
    => 0x00000000004004fd <wuz+7>:    8b 45 fc    mov    -0x4(%rbp),%eax
       0x0000000000400500 <wuz+10>:    83 c0 07    add    $0x7,%eax
       0x0000000000400503 <wuz+13>:    5d    pop    %rbp
       0x0000000000400504 <wuz+14>:    c3    retq
    (gdb) bt
    #0  wuz (x=5) at mcode.c:2
    #1  0x000000000040051a in wbar (x=5) at mcode.c:3
    #2  0x00000000004005b0 in main (argc=1, argv=0x7fffffffe528) at mcode.c:20

This is a normal stack trace showing that we got from main->wbar->wuz().

But if we now continue we reach wuz() a second time, and we again
request a stack trace:

    (gdb) c
    Continuing.

    Breakpoint 3, wuz (x=5) at mcode.c:2
    => 0x00000000004004fd <wuz+7>:    8b 45 fc    mov    -0x4(%rbp),%eax
       0x0000000000400500 <wuz+10>:    83 c0 07    add    $0x7,%eax
       0x0000000000400503 <wuz+13>:    5d    pop    %rbp
       0x0000000000400504 <wuz+14>:    c3    retq
    (gdb) bt
    #0  wuz (x=5) at mcode.c:2
    #1  0x00007fffffffe419 in ?? ()
    #2  0x0000000500000001 in ?? ()
    #3  0x00007fffffffe440 in ?? ()
    #4  0x00000000004005c6 in main (argc=0, argv=0xffffffff) at mcode.c:22
    Backtrace stopped: frame did not save the PC

Even though we have done the same two hierarchical calls, we get a
stack trace that contains the wrong frames. In my original inherited
wrapper code the situation was even worse, as the the stack trace
ended after 5 frames with the top level having address 0.

So the question is again, is there any extra code that can be added to
wbarcode[] that will cause gdb to output a valid stacktrace? Or is
there any other run time technique that may be used to make gdb
recognize the stack frames?

Thanks!
Dov

Re: x64 machine code and stack frames

[Dov Grobgeld]

Thanks. Indeed it sounds like the right direction. I have to figure
out how it works in a mixed environment with both static DWARF based
code as well as dynamically allocated code.

On Fri, Jan 22, 2016 at 4:27 AM, Matt Rice <ratmice@gmail.com> wrote:
>
>
> On Thu, Jan 21, 2016 at 11:45 AM, Dov Grobgeld <dov.grobgeld@gmail.com>
> wrote:
>>
>> Hello,
>>
>> I've inherited some clever x64 machine code for linux that creates an
>> machine code wrapper around a c-function call. I guess that in higher
>> language terms the code might be called a decorator or a closure. The
>> code is functioning well, but with the unfortunate artifact that when
>> the wrapper is called, it gobbles the stack trace in gdb.
>>
>> From what I have learned from the net gdb uses
>> https://en.wikipedia.org/wiki/DWARF as a guide for separating the
>> stack frames in the stack. This works well for static code, but
>> obviously code generated and called at run time isn't registered in
>> the DWARF framework.
>>
>> My question is if there is any way to rescue the stack trace in this
>> situation?
>
>
> While i haven't really used it before & can't comment on the particulars,
> It sounds like you should be using the jit interface to make gdb aware of
> the symbols generated at runtime.
>
> https://sourceware.org/gdb/onlinedocs/gdb/JIT-Interface.html

Re: x64 machine code and stack frames

[Pedro Alves]

On 01/22/2016 06:46 AM, Dov Grobgeld wrote:
> Thanks. Indeed it sounds like the right direction. I have to figure
> out how it works in a mixed environment with both static DWARF based
> code as well as dynamically allocated code.
> 
> On Fri, Jan 22, 2016 at 4:27 AM, Matt Rice <ratmice@gmail.com> wrote:
>>
>>
>> On Thu, Jan 21, 2016 at 11:45 AM, Dov Grobgeld <dov.grobgeld@gmail.com>
>> wrote:
>>>
>>> Hello,
>>>
>>> I've inherited some clever x64 machine code for linux that creates an
>>> machine code wrapper around a c-function call. I guess that in higher
>>> language terms the code might be called a decorator or a closure. The
>>> code is functioning well, but with the unfortunate artifact that when
>>> the wrapper is called, it gobbles the stack trace in gdb.
>>>
>>> From what I have learned from the net gdb uses
>>> https://en.wikipedia.org/wiki/DWARF as a guide for separating the
>>> stack frames in the stack. This works well for static code, but
>>> obviously code generated and called at run time isn't registered in
>>> the DWARF framework.
>>>
>>> My question is if there is any way to rescue the stack trace in this
>>> situation?
>>
>>
>> While i haven't really used it before & can't comment on the particulars,
>> It sounds like you should be using the jit interface to make gdb aware of
>> the symbols generated at runtime.
>>
>> https://sourceware.org/gdb/onlinedocs/gdb/JIT-Interface.html

And for unwinding itself, the new Python unwinder API is something to look at
too, if you want to try to avoid DWARF:

  https://sourceware.org/gdb/current/onlinedocs/gdb/Unwinding-Frames-in-Python.html

Thanks,
Pedro Alves