Re: Stack unwinding for green threads

[Andrew Burgess <andrew.burgess@embecosm.com>]

* Botond Dénes <bdenes@scylladb.com> [2020-07-03 12:50:54 +0300]:

> Hi,
> 
> I'm working on Scylla [1], an application which is built using the
> seastar framework [2]. This framework provides green threads [3] that
> have their own stacks. These threads are created with `setcontext()`
> and later we switch in/out using `setjmp()`/`longjmp()`.
> 
> We have a collection of python scripts [4] to help debug Scylla, among
> these we have a utility command which allows switching in/out of these
> green threads in gdb. This command basically (tries) to emulate
> `setjmp()`/`longjmp()` in python, saving and restoring registers. There
> are several problems with this method. For starters it crashes gdb for
> some time now, and also it doesn't work in coredumps, where gdb refuses
> to write to registers (even after `set write on`). So for these reasons
> I started to look for alternative ways to unwind the stacks of our
> green threads. However after going through the gdb documentation [5],
> reading all I could find about threads, stacks, frames and the Python
> API I haven't found an entry point to call which would unwind a stack
> located at a custom address (not the current $rsp). Is there such an
> API that I haven't found? Is there another way to achive what I want?
> Alternatively, how hard would it be to implement such an API?

No, there's currently nothing in GDB for having a look at a different
stack.

There is 'frame view ...' [1] which I'm sure someone might mention, as
it almost, sort-of, kind-of feels like this should solve your problem,
but it wont.

I did do some work on this area, years ago now, and at the time my
plan was to add a set of commands that solved exactly this problem, my
intention was to have some kind of 'stack create' command that would
allow you to supply an entire register set (if you wanted) and then
GDB would unwind assuming that this was the "current" register
values.  Of course, the only "required" registers would be $sp, and
probably $pc too, but you would have been able to provide others if
you wanted.  Anyway I digress, my need for this work disappeared and
this kind of fell of my radar, though I would like to find time to
revisit this ... one day.

However, thinking about the above kind of gave me an idea, it's not
perfect, but it might work.  How about writing a custom Python
Unwinder[2]?

Here's how I think this might help you:

  (1) By default the custom unwinder is "off", it doesn't claim any
  frames, your stack unwinds in the "normal" way.

  (2) Add a command 'magic-stack-unwind <address-of-register-set>',
  this turns "on" (sets a flag) the unwinder, and stores the address
  of the register set you want to unwind with.

  (3) The unwinder, now "on" would always claim frames at depth 0, and
  would return all of the registers from your stored register set.

I think that this would present you with your alternative stack, but
offset by 1 frame, so I think you'd still get the original "current"
frame, but everything below that would be from your alternative
register stack.  Not ideal maybe, but it might work.

It might even be possible to make use of frame filters[3] to hide
frame 0, though again, I've never tried hiding frame 0, only even
frames > 0, so I don't know how that would work for you.

Good luck,
Andrew



[1] https://sourceware.org/gdb/current/onlinedocs/gdb/Selection.html#Selection
[2] https://sourceware.org/gdb/current/onlinedocs/gdb/Unwinding-Frames-in-Python.html#Unwinding-Frames-in-Python
[3] https://sourceware.org/gdb/current/onlinedocs/gdb/Frame-Filter-API.html#Frame-Filter-API
> 
> Thanks,
> Botond
> 
> [1] https://github.com/scylladb/scylla
> [2] https://github.com/scylladb/seastar
> [3] http://docs.seastar.io/master/group__thread-module.html
> [4] https://github.com/scylladb/scylla/blob/master/scylla-gdb.py
> [5] https://sourceware.org/gdb/onlinedocs/gdb/index.html
>