[binutils-gdb] gdb/tui: avoid fp exception when applying layouts

[binutils-gdb] gdb/tui: avoid fp exception when applying layouts

[Andrew Burgess]

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=47b8e12ffd0ecf924836aed55eb63b9e5d9e8ea1

commit 47b8e12ffd0ecf924836aed55eb63b9e5d9e8ea1
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Tue Feb 1 16:53:56 2022 +0000

    gdb/tui: avoid fp exception when applying layouts
    
    Consider:
    
      (gdb) tui enable
      (gdb) layout src
      (gdb) tui new-layout conly cmd 1
      (gdb) layout conly
    
    After this, with current master, gdb crashes with a floating-point
    exception.
    
    The problem is that in tui_layout_split::apply, when we switch from
    'src' to 'conly', we will try to retain the cmd window height.  As
    such, the cmd window will become a fixed size window, which decreases
    the available_size, but doesn't count towards the total_weight.
    
    As the cmd window is the only window, the total_weight stays at zero,
    and, when we move into step 2, where we attempt to size the windows,
    we perform a divide by zero, and crash.
    
    After this commit we avoid the divide by zero, and just directly set
    the window size based on the fixed size.
    
    There is still a problem after this commit, when the conly layout is
    selected the cmd window retains its original height, which will only
    be part of the terminal.  The rest of the terminal is left unused.
    This issue will be addressed in a later commit, this commit is just
    about the floating-point exception.

Diff:
---
 gdb/testsuite/gdb.tui/new-layout.exp |  3 ++-
 gdb/tui/tui-layout.c                 | 30 +++++++++++++++++++-----------
 2 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/gdb/testsuite/gdb.tui/new-layout.exp b/gdb/testsuite/gdb.tui/new-layout.exp
index d3d34af7439..66048e65238 100644
--- a/gdb/testsuite/gdb.tui/new-layout.exp
+++ b/gdb/testsuite/gdb.tui/new-layout.exp
@@ -70,7 +70,8 @@ set layouts \
 	      {{0 0 80 15}} ""] \
 	 [list h "{-horizontal asm 1 src 1} 1 status 0 cmd 1" \
 	      {{0 0 40 15} {39 0 41 15}} \
-	      "$hex <main>.*21.*return 0"]]
+	      "$hex <main>.*21.*return 0"] \
+	 [list cmd_only "cmd 1" {} ""]]
 
 # Helper function to verify a list of boxes.
 proc check_boxes {boxes} {
diff --git a/gdb/tui/tui-layout.c b/gdb/tui/tui-layout.c
index c890b86b026..52105cca612 100644
--- a/gdb/tui/tui-layout.c
+++ b/gdb/tui/tui-layout.c
@@ -790,29 +790,37 @@ tui_layout_split::apply (int x_, int y_, int width_, int height_)
 	info[i].share_box = true;
     }
 
+  /* If last_index is set then we have a window that is not of a fixed
+     size.  This window will have its size calculated below, which requires
+     that the total_weight not be zero (we divide by total_weight, so don't
+     want a floating-point exception).  */
+  gdb_assert (last_index == -1 || total_weight > 0);
+
   /* Step 2: Compute the size of each sub-layout.  Fixed-sized items
      are given their fixed size, while others are resized according to
      their weight.  */
   int used_size = 0;
   for (int i = 0; i < m_splits.size (); ++i)
     {
-      /* Compute the height and clamp to the allowable range.  */
-      info[i].size = available_size * m_splits[i].weight / total_weight;
-      if (info[i].size > info[i].max_size)
-	info[i].size = info[i].max_size;
-      if (info[i].size < info[i].min_size)
-	info[i].size = info[i].min_size;
-      /* If there is any leftover size, just redistribute it to the
-	 last resizeable window, by dropping it from the allocated
-	 size.  We could try to be fancier here perhaps, by
-	 redistributing this size among all windows, not just the
-	 last window.  */
       if (info[i].min_size != info[i].max_size)
 	{
+	  /* Compute the height and clamp to the allowable range.  */
+	  info[i].size = available_size * m_splits[i].weight / total_weight;
+	  if (info[i].size > info[i].max_size)
+	    info[i].size = info[i].max_size;
+	  if (info[i].size < info[i].min_size)
+	    info[i].size = info[i].min_size;
+	  /* If there is any leftover size, just redistribute it to the
+	     last resizeable window, by dropping it from the allocated
+	     size.  We could try to be fancier here perhaps, by
+	     redistributing this size among all windows, not just the
+	     last window.  */
 	  used_size += info[i].size;
 	  if (info[i].share_box)
 	    --used_size;
 	}
+      else
+	info[i].size = info[i].min_size;
     }
 
   if (debug_tui)