Tips for improving performance of Python pretty-printer?

Tips for improving performance of Python pretty-printer?

[Antoine Pitrou]

Hello,

I'm implementing a bunch of pretty-printers (in Python) to improve the
debuggability of C++ types in Apache Arrow C++:
https://github.com/apache/arrow/pull/12092

However, I'm seeing performance issues where inspecting even relatively
simple information is quite slow as soon as I use the "natural" way, by
calling public C++ APIs (e.g. object methods) using
`gdb.parse_and_eval()`.

So for now I'm resorting to inspect private implementation details,
even for types I don't control such as `std::string` or `std::vector`.
As an example, I have the following helper code (simplified below for
clarity):

```
class SharedPtr:

    def __init__(self, val):
        self.val = val
        try:
            # libstdc++ internals
            self._ptr = val['_M_ptr']
        except gdb.error:
            # fallback for other C++ standard libraries
            self._ptr = gdb.parse_and_eval(
                f"{for_evaluation(val)}.get()")

    def get(self):
        return self._ptr


def for_evaluation(val):
    """
    Return a parsable form of gdb.Value `val`
    """
    ty = gdb.types.get_basic_type(val.type)
    if ty.code == gdb.TYPE_CODE_PTR:
        # It's already a pointer, can represent it directly
        return f"(({ty}) ({val}))"
    if val.address is None:
        raise ValueError(f"Cannot further evaluate rvalue: {val}")
    return f"(* ({ty}*) ({val.address}))"
```

This works fine but:

1) I need to maintain a generic fallback for non-GNU libstdc++
implementations of the C++ standard library
2) The generic fallback (obviously) suffers from the original
performance problem

Is it expected that `gdb.parse_and_eval` has such a poor performance?
(I didn't run any timings, but as a rough estimate I estimate that it
takes around 100 ms for a relatively simple expression)

Regards

Antoine.

Re: Tips for improving performance of Python pretty-printer?

[David Blaikie]

One side note: Generally pretty printers should not call into the code
under test - doing so means the pretty printer can't be used when debugging
a core dump, and risks not being usable when the process is corrupted in
some ways, or risks creating corruption/disturbing a reproduction by
executing code in the test program.

On Tue, Jan 11, 2022 at 10:05 AM Antoine Pitrou via Gdb <gdb@sourceware.org>
wrote:

>
> Hello,
>
> I'm implementing a bunch of pretty-printers (in Python) to improve the
> debuggability of C++ types in Apache Arrow C++:
> https://github.com/apache/arrow/pull/12092
>
> However, I'm seeing performance issues where inspecting even relatively
> simple information is quite slow as soon as I use the "natural" way, by
> calling public C++ APIs (e.g. object methods) using
> `gdb.parse_and_eval()`.
>
> So for now I'm resorting to inspect private implementation details,
> even for types I don't control such as `std::string` or `std::vector`.
> As an example, I have the following helper code (simplified below for
> clarity):
>
> ```
> class SharedPtr:
>
>     def __init__(self, val):
>         self.val = val
>         try:
>             # libstdc++ internals
>             self._ptr = val['_M_ptr']
>         except gdb.error:
>             # fallback for other C++ standard libraries
>             self._ptr = gdb.parse_and_eval(
>                 f"{for_evaluation(val)}.get()")
>
>     def get(self):
>         return self._ptr
>
>
> def for_evaluation(val):
>     """
>     Return a parsable form of gdb.Value `val`
>     """
>     ty = gdb.types.get_basic_type(val.type)
>     if ty.code == gdb.TYPE_CODE_PTR:
>         # It's already a pointer, can represent it directly
>         return f"(({ty}) ({val}))"
>     if val.address is None:
>         raise ValueError(f"Cannot further evaluate rvalue: {val}")
>     return f"(* ({ty}*) ({val.address}))"
> ```
>
> This works fine but:
>
> 1) I need to maintain a generic fallback for non-GNU libstdc++
> implementations of the C++ standard library
> 2) The generic fallback (obviously) suffers from the original
> performance problem
>
> Is it expected that `gdb.parse_and_eval` has such a poor performance?
> (I didn't run any timings, but as a rough estimate I estimate that it
> takes around 100 ms for a relatively simple expression)
>
> Regards
>
> Antoine.
>
>
>