[RFA] Make "bt N" print correct number of frames when using a frame filter

[Tom Tromey <tom@tromey.com>]

PR python/16497 notes that using "bt" with a positive argument prints
the wrong number of frames when a frame filter is in use.  Also, in this
case, the non-frame-filter path will print a message about "More stack
frames" when there are more; but this is not done in the frame-filter
case.

The first problem is that backtrace_command_1 passes the wrong value
to apply_ext_lang_frame_filter -- that function takes the final
frame's number as an argument, but backtrace_command_1 passes the
count, which is off by one.

The solution to the second problem is to have the C stack-printing
code stop at the correct number of frames and then print the message.

Tested using the buildbot.

ChangeLog
2017-04-22  Tom Tromey  <tom@tromey.com>

	PR python/16497:
	* stack.c (backtrace_command_1): Set PRINT_MORE_FRAMES flag.  Fix
	off-by-one in py_end computation.
	* python/py-framefilter.c (gdbpy_apply_frame_filter): Handle
	PRINT_MORE_FRAMES.
	* extension.h (enum frame_filter_flags) <PRINT_MORE_FRAMES>: New
	constant.

testsuite/ChangeLog
2017-04-22  Tom Tromey  <tom@tromey.com>

	PR python/16497:
	* gdb.python/py-framefilter.exp: Update test.
---
 gdb/ChangeLog                               | 10 ++++++++++
 gdb/extension.h                             |  3 +++
 gdb/python/py-framefilter.c                 | 25 +++++++++++++++++++++++++
 gdb/stack.c                                 |  6 +++++-
 gdb/testsuite/ChangeLog                     |  5 +++++
 gdb/testsuite/gdb.python/py-framefilter.exp |  2 +-
 6 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 28ae0a7..85d5da5 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,5 +1,15 @@
 2017-04-22  Tom Tromey  <tom@tromey.com>
 
+	PR python/16497:
+	* stack.c (backtrace_command_1): Set PRINT_MORE_FRAMES flag.  Fix
+	off-by-one in py_end computation.
+	* python/py-framefilter.c (gdbpy_apply_frame_filter): Handle
+	PRINT_MORE_FRAMES.
+	* extension.h (enum frame_filter_flags) <PRINT_MORE_FRAMES>: New
+	constant.
+
+2017-04-22  Tom Tromey  <tom@tromey.com>
+
 	* mi/mi-cmd-file.c (mi_cmd_file_list_shared_libraries): Use
 	ui_out_emit_list.
 	* stack.c (print_frame): Use ui_out_emit_list.
diff --git a/gdb/extension.h b/gdb/extension.h
index 2c79411..cda2ebf 100644
--- a/gdb/extension.h
+++ b/gdb/extension.h
@@ -100,6 +100,9 @@ enum frame_filter_flags
 
     /* Set this flag if frame locals are to be printed.  */
     PRINT_LOCALS = 8,
+
+    /* Set this flag if a "More frames" message is to be printed.  */
+    PRINT_MORE_FRAMES = 16,
   };
 
 /* A choice of the different frame argument printing strategies that
diff --git a/gdb/python/py-framefilter.c b/gdb/python/py-framefilter.c
index 75b055c..b604d51 100644
--- a/gdb/python/py-framefilter.c
+++ b/gdb/python/py-framefilter.c
@@ -1355,6 +1355,18 @@ gdbpy_apply_frame_filter (const struct extension_language_defn *extlang,
 
   gdbpy_enter enter_py (gdbarch, current_language);
 
+  /* When we're limiting the number of frames, be careful to request
+     one extra frame, so that we can print a message if there are more
+     frames.  */
+  int frame_countdown = -1;
+  if ((flags & PRINT_MORE_FRAMES) != 0 && frame_low >= 0 && frame_high >= 0)
+    {
+      ++frame_high;
+      /* This has an extra +1 because it is checked before a frame is
+	 printed.  */
+      frame_countdown = frame_high - frame_low + 1;
+    }
+
   gdbpy_ref<> iterable (bootstrap_python_frame_filters (frame, frame_low,
 							frame_high));
 
@@ -1402,6 +1414,19 @@ gdbpy_apply_frame_filter (const struct extension_language_defn *extlang,
 	  break;
 	}
 
+      if (frame_countdown != -1)
+	{
+	  assert ((flags & PRINT_MORE_FRAMES) != 0);
+	  --frame_countdown;
+	  if (frame_countdown == 0)
+	    {
+	      /* We've printed all the frames we were asked to
+		 print, but more frames existed.  */
+	      printf_filtered (_("(More stack frames follow...)\n"));
+	      break;
+	    }
+	}
+
       success = py_print_frame (item.get (), flags, args_type, out, 0,
 				levels_printed.get ());
 
diff --git a/gdb/stack.c b/gdb/stack.c
index 7f8a51c..37e8767 100644
--- a/gdb/stack.c
+++ b/gdb/stack.c
@@ -1766,7 +1766,9 @@ backtrace_command_1 (char *count_exp, int show_locals, int no_filters,
       else
 	{
 	  py_start = 0;
-	  py_end = count;
+	  /* The argument to apply_ext_lang_frame_filter is the number
+	     of the final frame to print, and frames start at 0.  */
+	  py_end = count - 1;
 	}
     }
   else
@@ -1800,6 +1802,8 @@ backtrace_command_1 (char *count_exp, int show_locals, int no_filters,
 
       if (show_locals)
 	flags |= PRINT_LOCALS;
+      if (from_tty)
+	flags |= PRINT_MORE_FRAMES;
 
       if (!strcmp (print_frame_arguments, "scalars"))
 	arg_type = CLI_SCALAR_VALUES;
diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index c4d5b79..df883da 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,8 @@
+2017-04-22  Tom Tromey  <tom@tromey.com>
+
+	PR python/16497:
+	* gdb.python/py-framefilter.exp: Update test.
+
 2017-04-19  Pedro Alves  <palves@redhat.com>
 
 	* gdb.threads/threadapply.exp (kill_and_remove_inferior): New
diff --git a/gdb/testsuite/gdb.python/py-framefilter.exp b/gdb/testsuite/gdb.python/py-framefilter.exp
index bbec48d..d81d144 100644
--- a/gdb/testsuite/gdb.python/py-framefilter.exp
+++ b/gdb/testsuite/gdb.python/py-framefilter.exp
@@ -149,7 +149,7 @@ gdb_test "bt -2" \
     ".*#26.*func5.*#27.*in main \\(\\).*" \
     "bt -2 with frame-filter Reverse disabled"
 gdb_test "bt 3" \
-    ".*#0.*end_func.*#1.*in funca \\(\\).*#2.*in funcb \\(j=10\\).*" \
+    ".*#0.*end_func.*#1.*in funca \\(\\).*#2.*in funcb \\(j=10\\)\[^#\]*More stack frames follow.*" \
     "bt 3 with frame-filter Reverse disabled"
 gdb_test "bt no-filter full" \
     ".*#0.*end_func.*str = $hex \"The End\".*st2 = $hex \"Is Near\".*b = 12.*c = 5.*#1.*in funca \\(\\).*#2.*in funcb \\(j=10\\).*bar = \{a = 42, b = 84\}.*" \
-- 
2.9.3