On Wed, Feb 4, 2015 at 2:35 PM, Doug Evans <dje@google.com> wrote:
> High level comments:
>
> Is it possible to see the code, and example usage, of a real-life use-case
> of this? That will help folks not familiar with this project to understand
> the problem we are trying to solve.
The case in question is Java Virtual Machine. It is a JIT compiler for Java,
and it is part of the OpenJDK (Java Development Kit). It compiles Java
methods on the fly, and the emitted code omits frame pointers (e.g., on
x86_64 platform RBP is used as a general purpose register rather than
as frame pointer). And, the emitted code does not have unwind info
expected by GDB, so standard sniffers fail and the traceback stops when
it encounters a frame for the JIT-compiled code.
If we know how JVM works, we know where to find the descriptors of the
currently compiled code, and once we locate the descriptor for a given PC,
we can extract the frame size, and unwind the frame.
It's easier to have a custom sniffer than to make JVM maintain DWARF
unwind info. Besides, most of the code for the sniffer is reused by the
corresponding frame decorator.
The full implementation of the combined sniffer/frame filter for OpenJDK
is about 2500 lines and will eventually become part of it. I am waiting for
this GDB patch to be reviewed before I can present it to be reviewed by
the JDK community :-)

> I'm still not sure what kind of performance cost we're looking at here as
> it scales up, I can imagine most times there'll be no Python sniffers,
> or at most one or two.  But it would be good to collect some perf data
> (e.g., install 1,10,100 no-op sniffers and see if there's any measurable
> difference in backtrace performance).
I ran a test that walks a 100-frame stack calling 100 Python sniffers
per frame, and it executes (throught the dejagnu checker) in 500ms on
Xeon E5-1650 0 @ 3.20GHz.
Please let me know if you would like it to be added to gdb/testsuite.

> Exposing frame id implementation details (sp,pc,special), and the
> form of how to do that, is something the community needs to decide on.
> I think we can come up with something suitable, though perhaps not
> the current form.

The revised patch is attached. The important differences are as follows:
* A sniffer is now an object, using the pattern similar to xmethods
and pretty printers.
* Register values are properly types (that is, based on a register type)

I am still not certain whether it's worth changing SnifferInfo.read_register to
have registers retrieved by name rather than by its number. Perhaps
adding gdb.Architecture.register_name_to_number method will be
a reasonable tradeoff?

The documentation is obviously unfinished (to be done once the design
issues are resolved), and what exists needs to be put into proper
English.

Here's take two:

gdb/ChangeLog:
2015-02-28  Sasha Smundak  <asmundak@google.com>

        * Makefile.in (SUBDIR_PYTHON_OBJS): Add py-unwind.o.
        (SUBDIR_PYTHON_SRCS): Add py-unwind.c.
        (py-unwind.o): New recipe.
        * NEWS: mention Python frame unwinding.
        * data-directory/Makefile.in (PYTHON_FILE_LIST):  Add sniffers.py.
        * doc/python.texi (Writing a Frame Unwinder in Python): Add
        section.
        * python/lib/gdb/__init__.py (packages): Add frame_sniffers list.
        * python/lib/gdb/command/sniffers.py: New file, implements GDB
        commands to list/enable/disable Python sniffers.
        * python/lib/gdb/function/sniffers.py: New file, implements
        execute_sniffers function.
        * python/lib/gdb/sniffer.py: New file, contains Sniffer class and
        register_sniffer function.
        * python/py-objfile.c (objfile_object): Add frame_sniffers field.
        (objfpy_dealloc): Decrement frame_sniffers reference count.
        (objfpy_initialize): Create frame_sniffers list.
        (objfpy_get_frame_sniffers): Implement Objfile.frame_sniffers
        getter.
        (objfpy_set_frame_sniffers): Implement Objfile.frame_sniffers
        setter.
        (objfile_getset): Add frame_sniffers attribute to Objfile.
        * python/py-progspace.c (pspace_object): Add frame_sniffers field.
        (pspy_dealloc): Decrement frame_sniffers reference count.
        (pspy_initialize): Create frame_sniffers list.
        (pspy_get_frame_sniffers): Implement gdb.Progspace.frame_sniffers
        getter.
        (pspy_set_frame_sniffers): Implement gdb.Progspace.frame_sniffers
        setter.
        (pspy_getset): Add frame_sniffers attribute to gdb.Progspace.
        * python/py-unwind.c: New file, implements Python frame sniffers
        interface.
        * python/python-internal.h (pspy_get_name_sniffers): New prototype.
        (objpy_get_frame_sniffers): New prototype.
        (gdbpy_initialize_unwind): New prototype.
        * python/python.c (gdbpy_apply_type_printers): Call
        gdbpy_initialize_unwind.

gdb/testsuite/ChangeLog:
2014-02-30  Sasha Smundak  <asmundak@google.com>

        * gdb.python/py-unwind-maint.c: Test program for py-unwind-maint.
        * gdb.python/py-unwind-maint.exp: Tests sniffer-related GDB
        commands.
        * gdb.python/py-unwind-maint.py: Pythons sniffers for the test.
        * gdb.python/py-unwind.c: Test program for the py-unwind test.
        * gdb.python/py-unwind.exp: Python frame sniffers test.
        * gdb.python/py-unwind.py: Frame sniffer in Python tested by
        py-unwind test.