[PATCH 0/5] Reimplement DAP backtrace using frame filters

[PATCH 0/5] Reimplement DAP backtrace using frame filters

[Tom Tromey]

This series reimplements DAP backtraces using frame filters.  This
seemed nice to have, because it would present users with a friendlier
view into the stack.  It also turned out to simplify the code a bit.

---
Tom Tromey (5):
      Fix execute_frame_filters doc string
      Add new interface to frame filter iteration
      Fix oversights in frame decorator code
      Simplify FrameVars
      Reimplement DAP stack traces using frame filters

 gdb/python/lib/gdb/FrameDecorator.py | 44 ++++++-------------
 gdb/python/lib/gdb/dap/bt.py         | 81 +++++++++++++++--------------------
 gdb/python/lib/gdb/dap/evaluate.py   | 11 ++---
 gdb/python/lib/gdb/dap/frames.py     |  7 +++
 gdb/python/lib/gdb/dap/scopes.py     | 83 ++++++++++++++----------------------
 gdb/python/lib/gdb/frames.py         | 83 +++++++++++++++++++++++++-----------
 6 files changed, 152 insertions(+), 157 deletions(-)
---
base-commit: 63224e96d0cd9e726bcd1d6d512255d92b7c5317
change-id: 20230614-dap-frame-decor-5cccd7f34dd3

Best regards,
-- 
Tom Tromey <tromey@adacore.com>

[PATCH 1/5] Fix execute_frame_filters doc string

[Tom Tromey]

When reading the doc string for execute_frame_filters, I wasn't sure
if the ranges were inclusive or exclusive.  This patch updates the doc
string to reflect my findings, and also fixes an existing typo.
---
 gdb/python/lib/gdb/frames.py | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/gdb/python/lib/gdb/frames.py b/gdb/python/lib/gdb/frames.py
index 5f8119c22a0..52f4191a8e8 100644
--- a/gdb/python/lib/gdb/frames.py
+++ b/gdb/python/lib/gdb/frames.py
@@ -165,18 +165,20 @@ def execute_frame_filters(frame, frame_low, frame_high):
     Arguments:
         frame: The initial frame.
 
-        frame_low: The low range of the slice.  If this is a negative
-        integer then it indicates a backward slice (ie bt -4) which
-        counts backward from the last frame in the backtrace.
+        frame_low: The low range of the slice, counting from 0.  If
+        this is a negative integer then it indicates a backward slice
+        (ie bt -4) which counts backward from the last frame in the
+        backtrace.
 
-        frame_high: The high range of the slice.  If this is -1 then
-        it indicates all frames until the end of the stack from
-        frame_low.
+        frame_high: The high range of the slice, inclusive.  If this
+        is -1 then it indicates all frames until the end of the stack
+        from frame_low.
 
     Returns:
         frame_iterator: The sliced iterator after all frame
-        filters have had a change to execute, or None if no frame
+        filters have had a chance to execute, or None if no frame
         filters are registered.
+
     """
 
     # Get a sorted list of frame filters.

-- 
2.40.1

[PATCH 2/5] Add new interface to frame filter iteration

[Tom Tromey]

This patch adds a new function, frame_iterator, that wraps the
existing code to find and execute the frame filters.  However, unlike
execute_frame_filters, it will always return an iterator -- whereas
execute_frame_filters will return None if no frame filters apply.

Nothing uses this new function yet, but it will used by a subsequent
DAP patch.
---
 gdb/python/lib/gdb/frames.py | 85 ++++++++++++++++++++++++++++++--------------
 1 file changed, 59 insertions(+), 26 deletions(-)

diff --git a/gdb/python/lib/gdb/frames.py b/gdb/python/lib/gdb/frames.py
index 52f4191a8e8..01e7cff6c70 100644
--- a/gdb/python/lib/gdb/frames.py
+++ b/gdb/python/lib/gdb/frames.py
@@ -156,37 +156,16 @@ def _sort_list():
     return sorted_frame_filters
 
 
-def execute_frame_filters(frame, frame_low, frame_high):
-    """Internal function called from GDB that will execute the chain
-    of frame filters.  Each filter is executed in priority order.
-    After the execution completes, slice the iterator to frame_low -
-    frame_high range.
-
-    Arguments:
-        frame: The initial frame.
-
-        frame_low: The low range of the slice, counting from 0.  If
-        this is a negative integer then it indicates a backward slice
-        (ie bt -4) which counts backward from the last frame in the
-        backtrace.
-
-        frame_high: The high range of the slice, inclusive.  If this
-        is -1 then it indicates all frames until the end of the stack
-        from frame_low.
-
-    Returns:
-        frame_iterator: The sliced iterator after all frame
-        filters have had a chance to execute, or None if no frame
-        filters are registered.
-
-    """
-
+# Internal function that implements frame_iterator and
+# execute_frame_filters.  If ALWAYS is True, then this will always
+# return an iterator.
+def _frame_iterator(frame, frame_low, frame_high, always):
     # Get a sorted list of frame filters.
     sorted_list = list(_sort_list())
 
     # Check to see if there are any frame-filters.  If not, just
     # return None and let default backtrace printing occur.
-    if len(sorted_list) == 0:
+    if not always and len(sorted_list) == 0:
         return None
 
     frame_iterator = FrameIterator(frame)
@@ -233,3 +212,57 @@ def execute_frame_filters(frame, frame_low, frame_high):
     sliced = itertools.islice(frame_iterator, frame_low, frame_high)
 
     return sliced
+
+
+def frame_iterator(frame, frame_low, frame_high):
+    """Helper function that will execute the chain of frame filters.
+    Each filter is executed in priority order.  After the execution
+    completes, slice the iterator to frame_low - frame_high range.  An
+    iterator is always returned.
+
+    Arguments:
+        frame: The initial frame.
+
+        frame_low: The low range of the slice, counting from 0.  If
+        this is a negative integer then it indicates a backward slice
+        (ie bt -4) which counts backward from the last frame in the
+        backtrace.
+
+        frame_high: The high range of the slice, inclusive.  If this
+        is -1 then it indicates all frames until the end of the stack
+        from frame_low.
+
+    Returns:
+        frame_iterator: The sliced iterator after all frame
+        filters have had a chance to execute.
+    """
+
+    return _frame_iterator(frame, frame_low, frame_high, True)
+
+
+def execute_frame_filters(frame, frame_low, frame_high):
+    """Internal function called from GDB that will execute the chain
+    of frame filters.  Each filter is executed in priority order.
+    After the execution completes, slice the iterator to frame_low -
+    frame_high range.
+
+    Arguments:
+        frame: The initial frame.
+
+        frame_low: The low range of the slice, counting from 0.  If
+        this is a negative integer then it indicates a backward slice
+        (ie bt -4) which counts backward from the last frame in the
+        backtrace.
+
+        frame_high: The high range of the slice, inclusive.  If this
+        is -1 then it indicates all frames until the end of the stack
+        from frame_low.
+
+    Returns:
+        frame_iterator: The sliced iterator after all frame
+        filters have had a chance to execute, or None if no frame
+        filters are registered.
+
+    """
+
+    return _frame_iterator(frame, frame_low, frame_high, False)

-- 
2.40.1

[PATCH 3/5] Fix oversights in frame decorator code

[Tom Tromey]

The frame decorator "FrameVars" code misses a couple of cases,
discovered when working on related DAP changes.

First, fetch_frame_locals does not stop when reaching a function
boundary.  This means it would return locals from any enclosing
functions.

Second, fetch_frame_args assumes that all arguments are at the
outermost scope, but this doesn't seem to be required by gdb.
---
 gdb/python/lib/gdb/FrameDecorator.py | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/gdb/python/lib/gdb/FrameDecorator.py b/gdb/python/lib/gdb/FrameDecorator.py
index 6773780735b..7293be86185 100644
--- a/gdb/python/lib/gdb/FrameDecorator.py
+++ b/gdb/python/lib/gdb/FrameDecorator.py
@@ -269,6 +269,11 @@ class FrameVars(object):
                 if self.fetch_b(sym):
                     lvars.append(SymValueWrapper(sym, None))
 
+            # Stop when the function itself is seen, to avoid showing
+            # variables from outer functions in a nested function.
+            if block.function is not None:
+                break
+
             block = block.superblock
 
         return lvars
@@ -286,14 +291,18 @@ class FrameVars(object):
             block = None
 
         while block is not None:
-            if block.function is not None:
+            if block.is_global or block.is_static:
                 break
-            block = block.superblock
-
-        if block is not None:
             for sym in block:
                 if not sym.is_argument:
                     continue
                 args.append(SymValueWrapper(sym, None))
 
+            # Stop when the function itself is seen, to avoid showing
+            # variables from outer functions in a nested function.
+            if block.function is not None:
+                break
+
+            block = block.superblock
+
         return args

-- 
2.40.1

[PATCH 4/5] Simplify FrameVars

[Tom Tromey]

FrameVars implements its own variant of Symbol.is_variable and/or
Symbol.is_control.  This patch replaces this code.
---
 gdb/python/lib/gdb/FrameDecorator.py | 27 +--------------------------
 1 file changed, 1 insertion(+), 26 deletions(-)

diff --git a/gdb/python/lib/gdb/FrameDecorator.py b/gdb/python/lib/gdb/FrameDecorator.py
index 7293be86185..fc3aa51c7c4 100644
--- a/gdb/python/lib/gdb/FrameDecorator.py
+++ b/gdb/python/lib/gdb/FrameDecorator.py
@@ -223,31 +223,6 @@ class FrameVars(object):
 
     def __init__(self, frame):
         self.frame = frame
-        self.symbol_class = {
-            gdb.SYMBOL_LOC_STATIC: True,
-            gdb.SYMBOL_LOC_REGISTER: True,
-            gdb.SYMBOL_LOC_ARG: True,
-            gdb.SYMBOL_LOC_REF_ARG: True,
-            gdb.SYMBOL_LOC_LOCAL: True,
-            gdb.SYMBOL_LOC_REGPARM_ADDR: True,
-            gdb.SYMBOL_LOC_COMPUTED: True,
-        }
-
-    def fetch_b(self, sym):
-        """Local utility method to determine if according to Symbol
-        type whether it should be included in the iterator.  Not all
-        symbols are fetched, and only symbols that return
-        True from this method should be fetched."""
-
-        # SYM may be a string instead of a symbol in the case of
-        # synthetic local arguments or locals.  If that is the case,
-        # always fetch.
-        if isinstance(sym, str):
-            return True
-
-        sym_type = sym.addr_class
-
-        return self.symbol_class.get(sym_type, False)
 
     def fetch_frame_locals(self):
         """Public utility method to fetch frame local variables for
@@ -266,7 +241,7 @@ class FrameVars(object):
             for sym in block:
                 if sym.is_argument:
                     continue
-                if self.fetch_b(sym):
+                if sym.is_variable or sym.is_constant:
                     lvars.append(SymValueWrapper(sym, None))
 
             # Stop when the function itself is seen, to avoid showing

-- 
2.40.1

[PATCH 5/5] Reimplement DAP stack traces using frame filters

[Tom Tromey]

This reimplements DAP stack traces using frame filters.  This slightly
simplifies the code, because frame filters and DAP were already doing
some similar work.  This also renames RegisterReference and
ScopeReference to make it clear that these are private (and so changes
don't have to worry about other files).

Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30468
---
 gdb/python/lib/gdb/dap/bt.py       | 81 ++++++++++++++++---------------------
 gdb/python/lib/gdb/dap/evaluate.py | 11 ++---
 gdb/python/lib/gdb/dap/frames.py   |  7 ++++
 gdb/python/lib/gdb/dap/scopes.py   | 83 +++++++++++++++-----------------------
 4 files changed, 79 insertions(+), 103 deletions(-)

diff --git a/gdb/python/lib/gdb/dap/bt.py b/gdb/python/lib/gdb/dap/bt.py
index 4439b428926..0350a3bb6d5 100644
--- a/gdb/python/lib/gdb/dap/bt.py
+++ b/gdb/python/lib/gdb/dap/bt.py
@@ -15,67 +15,56 @@
 
 import gdb
 import os
+import itertools
 
+from gdb.frames import frame_iterator
+from gdb.FrameIterator import FrameIterator
+from gdb.FrameDecorator import FrameDecorator
 from .frames import frame_id
 from .server import request, capability
 from .startup import send_gdb_with_response, in_gdb_thread
 from .state import set_thread
 
 
-# Helper function to safely get the name of a frame as a string.
-@in_gdb_thread
-def _frame_name(frame):
-    name = frame.name()
-    if name is None:
-        name = "???"
-    return name
-
-
-# Helper function to get a frame's SAL without an error.
-@in_gdb_thread
-def _safe_sal(frame):
-    try:
-        return frame.find_sal()
-    except gdb.error:
-        return None
-
-
 # Helper function to compute a stack trace.
 @in_gdb_thread
 def _backtrace(thread_id, levels, startFrame):
     set_thread(thread_id)
     frames = []
-    current_number = 0
+    if levels == 0:
+        # Zero means all remaining frames.
+        high = -1
+    else:
+        # frame_iterator uses an inclusive range, so subtract one.
+        high = startFrame + levels - 1
     try:
-        current_frame = gdb.newest_frame()
+        frame_iter = frame_iterator(gdb.newest_frame(), startFrame, high)
     except gdb.error:
-        current_frame = None
-    while current_frame is not None and (levels == 0 or len(frames) < levels):
-        # This condition handles the startFrame==0 case as well.
-        if current_number >= startFrame:
-            newframe = {
-                "id": frame_id(current_frame),
-                "name": _frame_name(current_frame),
-                # This must always be supplied, but we will set it
-                # correctly later if that is possible.
-                "line": 0,
-                # GDB doesn't support columns.
-                "column": 0,
-                "instructionPointerReference": hex(current_frame.pc()),
+        frame_iter = ()
+    for current_frame in frame_iter:
+        newframe = {
+            "id": frame_id(current_frame),
+            "name": current_frame.function(),
+            # This must always be supplied, but we will set it
+            # correctly later if that is possible.
+            "line": 0,
+            # GDB doesn't support columns.
+            "column": 0,
+            "instructionPointerReference": hex(current_frame.address()),
+        }
+        line = current_frame.line()
+        if line is not None:
+            newframe["line"] = line
+        filename = current_frame.filename()
+        if filename is not None:
+            newframe["source"] = {
+                "name": os.path.basename(filename),
+                "path": filename,
+                # We probably don't need this but it doesn't hurt
+                # to be explicit.
+                "sourceReference": 0,
             }
-            sal = _safe_sal(current_frame)
-            if sal is not None and sal.symtab is not None:
-                newframe["source"] = {
-                    "name": os.path.basename(sal.symtab.filename),
-                    "path": sal.symtab.filename,
-                    # We probably don't need this but it doesn't hurt
-                    # to be explicit.
-                    "sourceReference": 0,
-                }
-                newframe["line"] = sal.line
-            frames.append(newframe)
-        current_number = current_number + 1
-        current_frame = current_frame.older()
+        frames.append(newframe)
     # Note that we do not calculate totalFrames here.  Its absence
     # tells the client that it may simply ask for frames until a
     # response yields fewer frames than requested.
diff --git a/gdb/python/lib/gdb/dap/evaluate.py b/gdb/python/lib/gdb/dap/evaluate.py
index af7bf43afd0..01b9eb77202 100644
--- a/gdb/python/lib/gdb/dap/evaluate.py
+++ b/gdb/python/lib/gdb/dap/evaluate.py
@@ -19,7 +19,7 @@ import gdb.printing
 # This is deprecated in 3.9, but required in older versions.
 from typing import Optional
 
-from .frames import frame_for_id
+from .frames import select_frame
 from .server import capability, request
 from .startup import send_gdb_with_response, in_gdb_thread
 from .varref import find_variable, VariableReference
@@ -35,8 +35,7 @@ class EvaluateResult(VariableReference):
 def _evaluate(expr, frame_id):
     global_context = True
     if frame_id is not None:
-        frame = frame_for_id(frame_id)
-        frame.select()
+        select_frame(frame_id)
         global_context = False
     val = gdb.parse_and_eval(expr, global_context=global_context)
     ref = EvaluateResult(val)
@@ -48,8 +47,7 @@ def _evaluate(expr, frame_id):
 def _set_expression(expression, value, frame_id):
     global_context = True
     if frame_id is not None:
-        frame = frame_for_id(frame_id)
-        frame.select()
+        select_frame(frame_id)
         global_context = False
     lhs = gdb.parse_and_eval(expression, global_context=global_context)
     rhs = gdb.parse_and_eval(value, global_context=global_context)
@@ -61,8 +59,7 @@ def _set_expression(expression, value, frame_id):
 @in_gdb_thread
 def _repl(command, frame_id):
     if frame_id is not None:
-        frame = frame_for_id(frame_id)
-        frame.select()
+        select_frame(frame_id)
     val = gdb.execute(command, from_tty=True, to_string=True)
     return {
         "result": val,
diff --git a/gdb/python/lib/gdb/dap/frames.py b/gdb/python/lib/gdb/dap/frames.py
index 08209d0b361..1d2d1371354 100644
--- a/gdb/python/lib/gdb/dap/frames.py
+++ b/gdb/python/lib/gdb/dap/frames.py
@@ -52,3 +52,10 @@ def frame_for_id(id):
     """Given a frame identifier ID, return the corresponding frame."""
     global _all_frames
     return _all_frames[id]
+
+
+@in_gdb_thread
+def select_frame(id):
+    """Given a frame identifier ID, select the corresponding frame."""
+    frame = frame_for_id(id)
+    frame.inferior_frame().select()
diff --git a/gdb/python/lib/gdb/dap/scopes.py b/gdb/python/lib/gdb/dap/scopes.py
index 9b80dd9ce80..1687094c4ce 100644
--- a/gdb/python/lib/gdb/dap/scopes.py
+++ b/gdb/python/lib/gdb/dap/scopes.py
@@ -21,41 +21,17 @@ from .server import request
 from .varref import BaseReference
 
 
-# Helper function to return a frame's block without error.
-@in_gdb_thread
-def _safe_block(frame):
-    try:
-        return frame.block()
-    except gdb.error:
-        return None
-
-
-# Helper function to return two lists of variables of block, up to the
-# enclosing function.  The result is of the form (ARGS, LOCALS), where
-# each element is itself a list.
-@in_gdb_thread
-def _block_vars(block):
-    args = []
-    locs = []
-    while True:
-        for var in block:
-            if var.is_argument:
-                args.append(var)
-            elif var.is_variable or var.is_constant:
-                locs.append(var)
-        if block.function is not None:
-            break
-        block = block.superblock
-    return (args, locs)
-
-
-class ScopeReference(BaseReference):
+class _ScopeReference(BaseReference):
     def __init__(self, name, hint, frame, var_list):
         super().__init__(name)
         self.hint = hint
         self.frame = frame
+        self.inf_frame = frame.inferior_frame()
         self.func = frame.function()
-        self.var_list = var_list
+        self.line = frame.line()
+        # VAR_LIST might be any kind of iterator, but it's convenient
+        # here if it is just a collection.
+        self.var_list = tuple(var_list)
 
     def to_object(self):
         result = super().to_object()
@@ -63,8 +39,8 @@ class ScopeReference(BaseReference):
         # How would we know?
         result["expensive"] = False
         result["namedVariables"] = len(self.var_list)
-        if self.func is not None:
-            result["line"] = self.func.line
+        if self.line is not None:
+            result["line"] = self.line
             # FIXME construct a Source object
         return result
 
@@ -73,38 +49,45 @@ class ScopeReference(BaseReference):
 
     @in_gdb_thread
     def fetch_one_child(self, idx):
+        # Here SYM will conform to the SymValueWrapper interface.
         sym = self.var_list[idx]
-        if sym.needs_frame:
-            val = sym.value(self.frame)
-        else:
-            val = sym.value()
-        return (sym.print_name, val)
-
-
-class RegisterReference(ScopeReference):
+        name = str(sym.symbol())
+        val = sym.value()
+        if val is None:
+            # No synthetic value, so must read the symbol value
+            # ourselves.
+            val = sym.symbol().value(self.inf_frame)
+        elif not isinstance(val, gdb.Value):
+            val = gdb.Value(val)
+        return (name, val)
+
+
+class _RegisterReference(_ScopeReference):
     def __init__(self, name, frame):
         super().__init__(
-            name, "registers", frame, list(frame.architecture().registers())
+            name, "registers", frame, frame.inferior_frame().architecture().registers()
         )
 
     @in_gdb_thread
     def fetch_one_child(self, idx):
-        return (self.var_list[idx].name, self.frame.read_register(self.var_list[idx]))
+        return (
+            self.var_list[idx].name,
+            self.inf_frame.read_register(self.var_list[idx]),
+        )
 
 
 # Helper function to create a DAP scopes for a given frame ID.
 @in_gdb_thread
 def _get_scope(id):
     frame = frame_for_id(id)
-    block = _safe_block(frame)
     scopes = []
-    if block is not None:
-        (args, locs) = _block_vars(block)
-        if args:
-            scopes.append(ScopeReference("Arguments", "arguments", frame, args))
-        if locs:
-            scopes.append(ScopeReference("Locals", "locals", frame, locs))
-    scopes.append(RegisterReference("Registers", frame))
+    args = frame.frame_args()
+    if args:
+        scopes.append(_ScopeReference("Arguments", "arguments", frame, args))
+    locs = frame.frame_locals()
+    if locs:
+        scopes.append(_ScopeReference("Locals", "locals", frame, locs))
+    scopes.append(_RegisterReference("Registers", frame))
     return [x.to_object() for x in scopes]
 
 

-- 
2.40.1

Re: [PATCH 4/5] Simplify FrameVars

[Tom Tromey]

>>>>> "Tom" == Tom Tromey via Gdb-patches <gdb-patches@sourceware.org> writes:

Tom> FrameVars implements its own variant of Symbol.is_variable and/or
Tom> Symbol.is_control.  This patch replaces this code.

Tom> -                if self.fetch_b(sym):
Tom> +                if sym.is_variable or sym.is_constant:

I found out that using is_constant here is wrong, as this picks up local
enum constants, which aren't really interesting as variables.

In v2 of the series I fix this, and I tack on a test case at the end of
the series.

Tom