[PATCH] Fix Windows sharing_input_terminal hang

[PATCH] Fix Windows sharing_input_terminal hang

[Pedro Alves]

After running a number of programs under Windows gdb and detaching
them, I typed run in gdb, and got a hang, here:

 (top-gdb) bt
 #0  sharing_input_terminal (pid=4672) at /home/pedro/gdb/src/gdb/mingw-hdep.c:388
 #1  0x00007ff71a2d8678 in sharing_input_terminal (inf=0x23bf23dafb0) at /home/pedro/gdb/src/gdb/inflow.c:269
 #2  0x00007ff71a2d887b in child_terminal_save_inferior (self=0x23bf23de060) at /home/pedro/gdb/src/gdb/inflow.c:423
 #3  0x00007ff71a2c80c0 in inf_child_target::terminal_save_inferior (this=0x23bf23de060) at /home/pedro/gdb/src/gdb/inf-child.c:111
 #4  0x00007ff71a429c0f in target_terminal_is_ours_kind (desired_state=target_terminal_state::is_ours_for_output) at /home/pedro/gdb/src/gdb/target.c:1037
 #5  0x00007ff71a429e02 in target_terminal::ours_for_output () at /home/pedro/gdb/src/gdb/target.c:1094
 #6  0x00007ff71a2ccc8e in post_create_inferior (from_tty=0) at /home/pedro/gdb/src/gdb/infcmd.c:245
 #7  0x00007ff71a2cd431 in run_command_1 (args=0x0, from_tty=0, run_how=RUN_NORMAL) at /home/pedro/gdb/src/gdb/infcmd.c:502
 #8  0x00007ff71a2cd58b in run_command (args=0x0, from_tty=0) at /home/pedro/gdb/src/gdb/infcmd.c:527

The problem is that the loop around GetConsoleProcessList looped
forever, because there were exactly 10 processes to return.
GetConsoleProcessList's documentation says:

  If the buffer is too small to hold all the valid process identifiers,
  the return value is the required number of array elements. The
  function will have stored no identifiers in the buffer. In this
  situation, use the return value to allocate a buffer that is large
  enough to store the entire list and call the function again.

In this case, the buffer wasn't too small, it was exactly the right
size, so we should have broken out of the loop.  We didn't due to a
"<" check that should have been "<=".  That is fixed by this patch.

Change-Id: I14e4909f2ac2fa83d0d9b6e64418b5831ac4e4e3
---
 gdb/mingw-hdep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gdb/mingw-hdep.c b/gdb/mingw-hdep.c
index 75f63a3fe17..137a63eb244 100644
--- a/gdb/mingw-hdep.c
+++ b/gdb/mingw-hdep.c
@@ -385,7 +385,7 @@ sharing_input_terminal (int pid)
       len = GetConsoleProcessList (results.data (), results.size ());
       /* Note that LEN == 0 is a failure, but we can treat it the same
 	 as a "no".  */
-      if (len < results.size ())
+      if (len <= results.size ())
 	break;
 
       results.resize (len);

base-commit: 11e3488d3f787d51196a115fd9c6085e57bd9626
-- 
2.34.1

Re: [PATCH] Fix Windows sharing_input_terminal hang

[Tom Tromey]

>>>>> "Pedro" == Pedro Alves <pedro@palves.net> writes:

Pedro> In this case, the buffer wasn't too small, it was exactly the right
Pedro> size, so we should have broken out of the loop.  We didn't due to a
Pedro> "<" check that should have been "<=".  That is fixed by this patch.

Looks good to me.
Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCH] Fix Windows sharing_input_terminal hang

[Eli Zaretskii]

> From: Pedro Alves <pedro@palves.net>
> Date: Tue, 22 Aug 2023 16:13:32 +0100
> 
> After running a number of programs under Windows gdb and detaching
> them, I typed run in gdb, and got a hang, here:
> 
>  (top-gdb) bt
>  #0  sharing_input_terminal (pid=4672) at /home/pedro/gdb/src/gdb/mingw-hdep.c:388
>  #1  0x00007ff71a2d8678 in sharing_input_terminal (inf=0x23bf23dafb0) at /home/pedro/gdb/src/gdb/inflow.c:269
>  #2  0x00007ff71a2d887b in child_terminal_save_inferior (self=0x23bf23de060) at /home/pedro/gdb/src/gdb/inflow.c:423
>  #3  0x00007ff71a2c80c0 in inf_child_target::terminal_save_inferior (this=0x23bf23de060) at /home/pedro/gdb/src/gdb/inf-child.c:111
>  #4  0x00007ff71a429c0f in target_terminal_is_ours_kind (desired_state=target_terminal_state::is_ours_for_output) at /home/pedro/gdb/src/gdb/target.c:1037
>  #5  0x00007ff71a429e02 in target_terminal::ours_for_output () at /home/pedro/gdb/src/gdb/target.c:1094
>  #6  0x00007ff71a2ccc8e in post_create_inferior (from_tty=0) at /home/pedro/gdb/src/gdb/infcmd.c:245
>  #7  0x00007ff71a2cd431 in run_command_1 (args=0x0, from_tty=0, run_how=RUN_NORMAL) at /home/pedro/gdb/src/gdb/infcmd.c:502
>  #8  0x00007ff71a2cd58b in run_command (args=0x0, from_tty=0) at /home/pedro/gdb/src/gdb/infcmd.c:527
> 
> The problem is that the loop around GetConsoleProcessList looped
> forever, because there were exactly 10 processes to return.
> GetConsoleProcessList's documentation says:
> 
>   If the buffer is too small to hold all the valid process identifiers,
>   the return value is the required number of array elements. The
>   function will have stored no identifiers in the buffer. In this
>   situation, use the return value to allocate a buffer that is large
>   enough to store the entire list and call the function again.
> 
> In this case, the buffer wasn't too small, it was exactly the right
> size, so we should have broken out of the loop.  We didn't due to a
> "<" check that should have been "<=".

Such a lucky hang... ;-)

> That is fixed by this patch.

Thanks, LGTM.

Reviewed-By: Eli Zaretskii <eliz@gnu.org>