Re: dwarf unwinder (only works on i386/x86_64) - now with eh_frame and debug_frame fallback

[Mark Wielaard <mjw@redhat.com>]

Hi,

Yesterday I pushed some commits to make the dwarf unwinder use both
debug_frame and eh_frame tables. At first I had wanted to just used
debug_frame tables for the kernel and eh_frame for user space, but
depending on the gcc version, architecture (and apparently GNU/Linux
distro defaults), either table can be missing or only have partial
coverage (see gcc options -fexceptions, -fnon-call-exceptions,
-funwind-tables and -fasynchronous-unwind-tables). So currently we
default to the debug_frame table, and if that fails to unwind for a
particular location we fall back and retry using the eh_frame table.

This make the uprobes_ustack.exp testcase (and hopefully user stack
traces in general) work against gcc 4.4 (which is the default compiler
for fedora 11). Please do test and let me know of any situations where
things don't seem to work (especially if the uprobes_ustack.exp testcase
fails). Currently the dwarf unwinder is only enabled on i386 and x86_64.
It would be interesting to see if it can easily be enabled on other
architectures.

The "ugly" code in these patches is in adjustStartAddress() in
runtime/unwind.c. This really should go into _stp_module_relocate or
read_pointer. One tricky issue here is that we read the eh_frame section
during translation time and then load it in kernel space at module init
time. eh_frame tables can use pointer encodings that are absolute or
pc_relative (actually data relative), so we need to readjust for the new
load location of the eh_frame.

Some optimizations that could be done:
- Use the eh_frame_hdr binary search table
  (needs careful auditing of adjustStartAddress -> read_pointer).
- Try to read eh_frame in-place from user space
  (risks tricky page fault issues if not available)
- Merge debug_frame and eh_frame at runtime and build our own
  binary search hdr.

But for now I won't be working on those, unless the backtraces become a
bottleneck for actual code using them.

Next steps to make stacktraces better are:
- Add more tests (in particular ones that test prelinking
  and missing or split-file debuginfo).
- Make vma-tracker more robust (_stp_tf_mmap_cb)
  Wenji send me some notes on things it seems to miss. If we cannot
  track a location to a _stp_module we cannot unwind.
- Track vdso for process symbols/backtraces. PR10080.
- Simplify unwind interface. Architecture dependent code has too
  much duplication. Need to just handle address, not function symbol
  printing.
- Nicer fallback to in-kernel unwinder/backtrace, in particular
  for backtracing from non-pt_regs probe context. PR6961.
- unwind through kretprobes. PR6436/PR9999.
- Better tapset functions for handling stacks. PR6580.

I'll be away for a couple of days, but will be back early next week.

Cheers,

Mark