[PATCHv3 12/15] gdb/tui: support placing the cmd window into a horizontal layout

[Andrew Burgess <aburgess@redhat.com>]

This commit allows the user to place the cmd window within horizontal
tui layouts.  Consider this set of steps, carried out in an 80 columns
by 24 lines terminal, using current master gdb:

  (gdb) tui new-layout hsrc { -horizontal src 1 cmd 1 } 1 status 1
  (gdb) tui layout hsrc

What you end up with is a full width cmd window with the status bar
beneath.  Where's the src window gone?  We then try:

  (gdb) info win
  Name       Lines Columns Focus
  src           23       3 (has focus)
  cmd           23      80
  status         1      80
  (gdb)

Something weird has gone on, gdb has overlapped the cmd window with
the src window.  If we trigger the src window to redraw is content,
for example, 'list main', then we see corruption in the cmd window as
the src window overwrites it.

So, what's going on?

The problem is some code in tui_layout_split::apply, in tui-layout.c.
Within 'Step 1', there is a loop that calculates the min/max window
sizes for all windows within a tui_layout_split.  However, there's a
special case for the cmd window.

This special case is trying to have the cmd window retain its current
size when a layout is re-applied, or a new layout is applied.  This
makes sense, consider moving from the 'src' layout to the 'asm'
layout, this looks something like this (status window removed):

    .-------.         .-------.
    |  src  |         |  asm  |
    |-------|  ====>  |-------|
    |  cmd  |         |  cmd  |
    '-------'         '-------'

If the user has gone to the effort of adjusting the cmd window size,
then, the thinking goes, we shouldn't reset the cmd window size when
switching layouts like this.

The problem though, is that when we do a switch more like this:

    .-----------.         .-----------.
    |    src    |         |     |     |
    |-----------|  ====>  | asm | cmd |
    |    cmd    |         |     |     |
    '-----------'         '-----------'

Now retaining the cmd window width makes no sense; the new layout has
a completely different placement for the cmd window, instead of sizing
by height, we're now sizing by width.  The existing code doesn't
understand this though, and tried to retain the full width for the cmd
window.

To solve this problem, I propose we introduce the idea of a layout
"fingerprint".  The fingerprint tries to capture, in an abstract way,
where the cmd window lives within the layout.

Only when two layouts have the same fingerprint will we attempt to
retain the cmd window size.

The fingerprint for a layout is represented as a string, the string is
a series of 'V' or 'H' characters, ending with a single 'C'
character.  The series of 'V' and 'H' characters represent the
vertical or horizontal layouts that must be passed through to find the
cmd window.

Here are a few examples:

  # This layout is equivalent to the builtin 'src' layout.
  # Fingerprint: VC
  tui new-layout example1 src 2 status 0 cmd 1

  # This layout is equivalent to the builtin 'split' layout.
  # Fingerprint: VC
  tui new-layout example2 src 1 asm 1 status 0 cmd 1

  # This is the same layout that was given at the top.
  # Fingerprint: VHC
  tui new-layout hsrc { -horizontal src 1 cmd 1 } 1 status 1

And so, when switching between example1 and example2, gdb knows that
the cmd window is, basically, in the same sort of position within the
layout, and will retain the cmd window size.

In contrast, when switching to the hsrc layout, gdb understands that
the position of the cmd window is different, and does not try to
retain the cmd window size.
---
 gdb/testsuite/gdb.tui/new-layout.exp |  5 ++
 gdb/tui/tui-layout.c                 | 69 ++++++++++++++++++++++------
 gdb/tui/tui-layout.h                 | 47 +++++++++++++++----
 gdb/tui/tui-win.c                    |  6 ++-
 4 files changed, 102 insertions(+), 25 deletions(-)

diff --git a/gdb/testsuite/gdb.tui/new-layout.exp b/gdb/testsuite/gdb.tui/new-layout.exp
index 0d49d033be5..883c7601be7 100644
--- a/gdb/testsuite/gdb.tui/new-layout.exp
+++ b/gdb/testsuite/gdb.tui/new-layout.exp
@@ -71,6 +71,11 @@ set layouts \
 	 [list h "{-horizontal asm 1 src 1} 1 status 0 cmd 1" \
 	      {{0 0 40 15} {39 0 41 15}} \
 	      "$hex <main>.*21.*return 0"] \
+	 [list example3 "{-horizontal src 1 cmd 1} 1 status 0 asm 1" \
+	      {{0 0 40 11} {0 12 80 12}} \
+	      "21.*return 0.*$hex <main>"] \
+	 [list example4 "src 1 status 0 {-horizontal cmd 1 regs 1} 1" \
+	      {{0 0 80 11} {40 12 40 12}} ""] \
 	 [list cmd_only "cmd 1" {} ""]]
 
 # Helper function to verify a list of boxes.
diff --git a/gdb/tui/tui-layout.c b/gdb/tui/tui-layout.c
index 4742f65f6cf..481a67827f9 100644
--- a/gdb/tui/tui-layout.c
+++ b/gdb/tui/tui-layout.c
@@ -67,7 +67,7 @@ std::vector<tui_win_info *> tui_windows;
 /* See tui-layout.h.  */
 
 void
-tui_apply_current_layout ()
+tui_apply_current_layout (bool preserve_cmd_win_size_p)
 {
   struct gdbarch *gdbarch;
   CORE_ADDR addr;
@@ -77,7 +77,8 @@ tui_apply_current_layout ()
   for (tui_win_info *win_info : tui_windows)
     win_info->make_visible (false);
 
-  applied_layout->apply (0, 0, tui_term_width (), tui_term_height ());
+  applied_layout->apply (0, 0, tui_term_width (), tui_term_height (),
+			 preserve_cmd_win_size_p);
 
   /* Keep the list of internal windows up-to-date.  */
   for (int win_type = SRC_WIN; (win_type < MAX_MAJOR_WINDOWS); win_type++)
@@ -134,9 +135,18 @@ tui_adjust_window_width (struct tui_win_info *win, int new_width)
 static void
 tui_set_layout (tui_layout_split *layout)
 {
+  std::string old_fingerprint;
+  if (applied_layout != nullptr)
+    old_fingerprint = applied_layout->layout_fingerprint ();
+
   applied_skeleton = layout;
   applied_layout = layout->clone ();
-  tui_apply_current_layout ();
+
+  std::string new_fingerprint = applied_layout->layout_fingerprint ();
+  bool preserve_command_window_size
+    = (TUI_CMD_WIN != nullptr && old_fingerprint == new_fingerprint);
+
+  tui_apply_current_layout (preserve_command_window_size);
 }
 
 /* See tui-layout.h.  */
@@ -158,7 +168,7 @@ tui_add_win_to_layout (enum tui_win_type type)
 
   const char *name = type == SRC_WIN ? SRC_NAME : DISASSEM_NAME;
   applied_layout->replace_window (tui_win_list[other]->name (), name);
-  tui_apply_current_layout ();
+  tui_apply_current_layout (true);
 }
 
 /* Find LAYOUT in the "layouts" global and return its index.  */
@@ -271,7 +281,7 @@ tui_remove_some_windows ()
     }
 
   applied_layout->remove_windows (focus->name ());
-  tui_apply_current_layout ();
+  tui_apply_current_layout (true);
 }
 
 static void
@@ -411,7 +421,8 @@ tui_layout_window::clone () const
 /* See tui-layout.h.  */
 
 void
-tui_layout_window::apply (int x_, int y_, int width_, int height_)
+tui_layout_window::apply (int x_, int y_, int width_, int height_,
+			  bool preserve_cmd_win_size_p)
 {
   x = x_;
   y = y_;
@@ -492,6 +503,17 @@ tui_layout_window::specification (ui_file *output, int depth)
 
 /* See tui-layout.h.  */
 
+std::string
+tui_layout_window::layout_fingerprint () const
+{
+  if (strcmp (get_name (), "cmd") == 0)
+    return "C";
+  else
+    return "";
+}
+
+/* See tui-layout.h.  */
+
 void
 tui_layout_split::add_split (std::unique_ptr<tui_layout_split> &&layout,
 			     int weight)
@@ -716,8 +738,11 @@ tui_layout_split::set_size (const char *name, int new_size, bool set_width_p)
     }
   else
     {
-      /* Simply re-apply the updated layout.  */
-      apply (x, y, width, height);
+      /* Simply re-apply the updated layout.  We pass false here so that
+	 the cmd window can be resized.  However, we should have already
+	 resized everything above to be "just right", so the apply call
+	 here should not end up changing the sizes at all.  */
+      apply (x, y, width, height, false);
     }
 
   return HANDLED;
@@ -726,7 +751,8 @@ tui_layout_split::set_size (const char *name, int new_size, bool set_width_p)
 /* See tui-layout.h.  */
 
 void
-tui_layout_split::apply (int x_, int y_, int width_, int height_)
+tui_layout_split::apply (int x_, int y_, int width_, int height_,
+			 bool preserve_cmd_win_size_p)
 {
   TUI_SCOPED_DEBUG_ENTER_EXIT;
 
@@ -781,7 +807,7 @@ tui_layout_split::apply (int x_, int y_, int width_, int height_)
       m_splits[i].layout->get_sizes (m_vertical, &info[i].min_size,
 				     &info[i].max_size);
 
-      if (!m_applied
+      if (preserve_cmd_win_size_p
 	  && cmd_win_already_exists
 	  && m_splits[i].layout->get_name () != nullptr
 	  && strcmp (m_splits[i].layout->get_name (), "cmd") == 0)
@@ -964,13 +990,13 @@ tui_layout_split::apply (int x_, int y_, int width_, int height_)
       else if (info[i].share_box)
 	--size_accum;
       if (m_vertical)
-	m_splits[i].layout->apply (x, y + size_accum, width, info[i].size);
+	m_splits[i].layout->apply (x, y + size_accum, width, info[i].size,
+				   preserve_cmd_win_size_p);
       else
-	m_splits[i].layout->apply (x + size_accum, y, info[i].size, height);
+	m_splits[i].layout->apply (x + size_accum, y, info[i].size, height,
+				   preserve_cmd_win_size_p);
       size_accum += info[i].size;
     }
-
-  m_applied = true;
 }
 
 /* See tui-layout.h.  */
@@ -1031,6 +1057,21 @@ tui_layout_split::specification (ui_file *output, int depth)
     fputs_unfiltered ("}", output);
 }
 
+/* See tui-layout.h.  */
+
+std::string
+tui_layout_split::layout_fingerprint () const
+{
+  for (auto &item : m_splits)
+    {
+      std::string fp = item.layout->layout_fingerprint ();
+      if (!fp.empty ())
+	return std::string (m_vertical ? "V" : "H") + fp;
+    }
+
+  return "";
+}
+
 /* Destroy the layout associated with SELF.  */
 
 static void
diff --git a/gdb/tui/tui-layout.h b/gdb/tui/tui-layout.h
index 9cfcfb8e81f..32bc95171bf 100644
--- a/gdb/tui/tui-layout.h
+++ b/gdb/tui/tui-layout.h
@@ -55,8 +55,12 @@ class tui_layout_base
      "skeleton" layout.  */
   virtual std::unique_ptr<tui_layout_base> clone () const = 0;
 
-  /* Change the size and location of this layout.  */
-  virtual void apply (int x, int y, int width, int height) = 0;
+  /* Change the size and location of this layout.  When
+     PRESERVE_CMD_WIN_SIZE_P is true the current size of the TUI_CMD_WIN
+     is preserved, otherwise, the TUI_CMD_WIN will resize just like any
+     other window.  */
+  virtual void apply (int x, int y, int width, int height,
+		      bool preserve_cmd_win_size_p) = 0;
 
   /* Return the minimum and maximum height or width of this layout.
      HEIGHT is true to fetch height, false to fetch width.  */
@@ -97,6 +101,26 @@ class tui_layout_base
      depth of this layout in the hierarchy (zero-based).  */
   virtual void specification (ui_file *output, int depth) = 0;
 
+  /* Return a FINGERPRINT string containing an abstract representation of
+     the location of the cmd window in this layout.
+
+     When called on a complete, top-level layout, the fingerprint will be a
+     non-empty string made of 'V' and 'H' characters, followed by a single
+     'C' character.  Each 'V' and 'H' represents a vertical or horizontal
+     layout that must be passed through in order to find the cmd
+     window.
+
+     Of course, layouts are built recursively, so, when called on a partial
+     layout, if this object represents a single window, then either the
+     empty string is returned (for non-cmd windows), or a string
+     containing a single 'C' is returned.
+
+     For object representing layouts, if the layout contains the cmd
+     window then we will get back a valid fingerprint string (contains 'V'
+     and 'H', ends with 'C'), or, if this layout doesn't contain the cmd
+     window, an empty string is returned.  */
+  virtual std::string layout_fingerprint () const = 0;
+
   /* Add all windows to the WINDOWS vector.  */
   virtual void get_windows (std::vector<tui_win_info *> *windows) = 0;
 
@@ -126,7 +150,8 @@ class tui_layout_window : public tui_layout_base
 
   std::unique_ptr<tui_layout_base> clone () const override;
 
-  void apply (int x, int y, int width, int height) override;
+  void apply (int x, int y, int width, int height,
+	      bool preserve_cmd_win_size_p) override;
 
   const char *get_name () const override
   {
@@ -155,6 +180,8 @@ class tui_layout_window : public tui_layout_base
 
   void specification (ui_file *output, int depth) override;
 
+  std::string layout_fingerprint () const override;
+
   /* See tui_layout_base::get_windows.  */
   void get_windows (std::vector<tui_win_info *> *windows) override
   {
@@ -201,7 +228,8 @@ class tui_layout_split : public tui_layout_base
 
   std::unique_ptr<tui_layout_base> clone () const override;
 
-  void apply (int x, int y, int width, int height) override;
+  void apply (int x, int y, int width, int height,
+	      bool preserve_cmd_win_size_p) override;
 
   tui_adjust_result set_height (const char *name, int new_height) override
   {
@@ -225,6 +253,8 @@ class tui_layout_split : public tui_layout_base
 
   void specification (ui_file *output, int depth) override;
 
+  std::string layout_fingerprint () const override;
+
   /* See tui_layout_base::get_windows.  */
   void get_windows (std::vector<tui_win_info *> *windows) override
   {
@@ -289,9 +319,6 @@ class tui_layout_split : public tui_layout_base
 
   /* True if the windows in this split are arranged vertically.  */
   bool m_vertical;
-
-  /* True if this layout has already been applied at least once.  */
-  bool m_applied = false;
 };
 
 /* Add the specified window to the layout in a logical way.  This
@@ -314,8 +341,10 @@ extern void tui_regs_layout ();
    some other window is chosen to remain.  */
 extern void tui_remove_some_windows ();
 
-/* Apply the current layout.  */
-extern void tui_apply_current_layout ();
+/* Apply the current layout.  When PRESERVE_CMD_WIN_SIZE_P is true the
+   current size of the TUI_CMD_WIN is preserved, otherwise, the TUI_CMD_WIN
+   will resize just like any other window.  */
+extern void tui_apply_current_layout (bool);
 
 /* Adjust the window height of WIN to NEW_HEIGHT.  */
 extern void tui_adjust_window_height (struct tui_win_info *win,
diff --git a/gdb/tui/tui-win.c b/gdb/tui/tui-win.c
index 512ea52b694..98dfcc76534 100644
--- a/gdb/tui/tui-win.c
+++ b/gdb/tui/tui-win.c
@@ -511,8 +511,10 @@ tui_resize_all (void)
 	 AIX 5.3 does not define clear.  */
       erase ();
       clearok (curscr, TRUE);
-      tui_apply_current_layout ();
-      /* Turn keypad back on.  */
+      /* Apply the current layout.  The 'false' here allows the command
+	 window to resize proportionately with containing terminal, rather
+	 than maintaining a fixed size.  */
+      tui_apply_current_layout (false); /* Turn keypad back on.  */
       keypad (TUI_CMD_WIN->handle.get (), TRUE);
     }
 }
-- 
2.25.4