[binutils-gdb] gdb/infrun: handle already-exited threads when attempting to stop

[binutils-gdb] gdb/infrun: handle already-exited threads when attempting to stop

[gdb-buildbot]

*** TEST RESULTS FOR COMMIT a05575d39a5348bd9979fc09e658a03ff22722b9 ***

commit a05575d39a5348bd9979fc09e658a03ff22722b9
Author:     Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
AuthorDate: Thu May 14 13:59:54 2020 +0200
Commit:     Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
CommitDate: Thu May 14 13:59:54 2020 +0200

    gdb/infrun: handle already-exited threads when attempting to stop
    
    In stop_all_threads, GDB sends signals to other threads in an attempt
    to stop them.  While in a typical scenario the expected wait status is
    TARGET_WAITKIND_STOPPED, it is possible that the thread GDB attempted
    to stop has already terminated.  If so, a waitstatus other than
    TARGET_WAITKIND_STOPPED would be received.  Handle this case
    appropriately.
    
    If a wait status that denotes thread termination is ignored, GDB goes
    into an infinite loop in stop_all_threads.
    E.g.:
    
      $ gdb ./a.out
      (gdb) start
      ...
      (gdb) add-inferior -exec ./a.out
      ...
      (gdb) inferior 2
      ...
      (gdb) start
      ...
      (gdb) set schedule-multiple on
      (gdb) set debug infrun 2
      (gdb) continue
      Continuing.
      infrun: clear_proceed_status_thread (process 10449)
      infrun: clear_proceed_status_thread (process 10453)
      infrun: proceed (addr=0xffffffffffffffff, signal=GDB_SIGNAL_DEFAULT)
      infrun: proceed: resuming process 10449
      infrun: resume (step=0, signal=GDB_SIGNAL_0), trap_expected=0, current thread [process 10449] at 0x55555555514e
      infrun: infrun_async(1)
      infrun: prepare_to_wait
      infrun: proceed: resuming process 10453
      infrun: resume (step=0, signal=GDB_SIGNAL_0), trap_expected=0, current thread [process 10453] at 0x55555555514e
      infrun: prepare_to_wait
      infrun: Found 2 inferiors, starting at #0
      infrun: target_wait (-1.0.0, status) =
      infrun:   10449.10449.0 [process 10449],
      infrun:   status->kind = exited, status = 0
      infrun: handle_inferior_event status->kind = exited, status = 0
      [Inferior 1 (process 10449) exited normally]
      infrun: stop_waiting
      infrun: stop_all_threads
      infrun: stop_all_threads, pass=0, iterations=0
      infrun:   process 10453 executing, need stop
      infrun: target_wait (-1.0.0, status) =
      infrun:   10453.10453.0 [process 10453],
      infrun:   status->kind = exited, status = 0
      infrun: stop_all_threads status->kind = exited, status = 0 process 10453
      infrun:   process 10453 executing, already stopping
      infrun: target_wait (-1.0.0, status) =
      infrun:   -1.0.0 [process -1],
      infrun:   status->kind = no-resumed
      infrun: infrun_async(0)
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      infrun: stop_all_threads status->kind = no-resumed process -1
      infrun:   process 10453 executing, already stopping
      ...
    
    And this polling goes on forever.  This patch prevents the infinite
    looping behavior.  For the same scenario above, we obtain the
    following behavior:
    
      ...
      (gdb) continue
      Continuing.
      infrun: clear_proceed_status_thread (process 31229)
      infrun: clear_proceed_status_thread (process 31233)
      infrun: proceed (addr=0xffffffffffffffff, signal=GDB_SIGNAL_DEFAULT)
      infrun: proceed: resuming process 31229
      infrun: resume (step=0, signal=GDB_SIGNAL_0), trap_expected=0, current thread [process 31229] at 0x55555555514e
      infrun: infrun_async(1)
      infrun: prepare_to_wait
      infrun: proceed: resuming process 31233
      infrun: resume (step=0, signal=GDB_SIGNAL_0), trap_expected=0, current thread [process 31233] at 0x55555555514e
      infrun: prepare_to_wait
      infrun: Found 2 inferiors, starting at #0
      infrun: target_wait (-1.0.0, status) =
      infrun:   31229.31229.0 [process 31229],
      infrun:   status->kind = exited, status = 0
      infrun: handle_inferior_event status->kind = exited, status = 0
      [Inferior 1 (process 31229) exited normally]
      infrun: stop_waiting
      infrun: stop_all_threads
      infrun: stop_all_threads, pass=0, iterations=0
      infrun:   process 31233 executing, need stop
      infrun: target_wait (-1.0.0, status) =
      infrun:   31233.31233.0 [process 31233],
      infrun:   status->kind = exited, status = 0
      infrun: stop_all_threads status->kind = exited, status = 0 process 31233
      infrun: saving status status->kind = exited, status = 0 for 31233.31233.0
      infrun:   process 31233 not executing
      infrun: stop_all_threads, pass=1, iterations=1
      infrun:   process 31233 not executing
      infrun: stop_all_threads done
      (gdb)
    
    The exit event from Inferior 1 is received and shown to the user.
    The exit event from Inferior 2 is not displayed, but kept pending.
    
      (gdb) info inferiors
        Num  Description       Connection           Executable
      * 1    <null>                                 a.out
        2    process 31233     1 (native)           a.out
      (gdb) inferior 2
      [Switching to inferior 2 [process 31233] (a.out)]
      [Switching to thread 2.1 (process 31233)]
      Couldn't get registers: No such process.
      (gdb) continue
      Continuing.
      infrun: clear_proceed_status_thread (process 31233)
      infrun: clear_proceed_status_thread: thread process 31233 has pending wait status status->kind = exited, status = 0 (currently_stepping=0).
      infrun: proceed (addr=0xffffffffffffffff, signal=GDB_SIGNAL_DEFAULT)
      infrun: proceed: resuming process 31233
      infrun: resume: thread process 31233 has pending wait status status->kind = exited, status = 0 (currently_stepping=0).
      infrun: prepare_to_wait
      infrun: Using pending wait status status->kind = exited, status = 0 for process 31233.
      infrun: target_wait (-1.0.0, status) =
      infrun:   31233.31233.0 [process 31233],
      infrun:   status->kind = exited, status = 0
      infrun: handle_inferior_event status->kind = exited, status = 0
      [Inferior 2 (process 31233) exited normally]
      infrun: stop_waiting
      (gdb) info inferiors
        Num  Description       Connection           Executable
        1    <null>                                 a.out
      * 2    <null>                                 a.out
      (gdb)
    
    When a process exits and we leave the process exit event pending, we
    need to make sure that at least one thread is left listed in the
    inferior's thread list.  This is necessary in order to make sure we
    have a thread that we can later resume, so the process exit event can
    be collected/reported.
    
    When native debugging, the GNU/Linux back end already makes sure that
    the last LWP isn't deleted.
    
    When remote debugging against GNU/Linux GDBserver, the GNU/Linux
    GDBserver backend also makes sure that the last thread isn't deleted
    until the process exit event is reported to GDBserver core.
    
    However, between the backend reporting the process exit event to
    GDBserver core, and GDB consuming the event, GDB may update the thread
    list and find no thread left in the process.  The process exit event
    will be pending somewhere in GDBserver's stop reply queue, or
    gdb/remote.c's queue, or whathever other event queue inbetween
    GDBserver and infrun.c's handle_inferior_event.
    
    This patch tweaks remote.c's target_update_thread_list implementation
    to avoid deleting the last thread of an inferior.
    
    In the past, this case of inferior-with-no-threads led to a special
    case at the bottom of handle_no_resumed, where it reads:
    
      /* Note however that we may find no resumed thread because the whole
         process exited meanwhile (thus updating the thread list results
         in an empty thread list).  In this case we know we'll be getting
         a process exit event shortly.  */
      for (inferior *inf : all_non_exited_inferiors (ecs->target))
    
    In current master, that code path is still reachable with the
    gdb.threads/continue-pending-after-query.exp testcase, when tested
    against GDBserver, with "maint set target-non-stop" forced "on".
    
    With this patch, the scenario that loop was concerned about is still
    properly handled, because the loop above it finds the process's last
    thread with "executing" set to true, and thus the handle_no_resumed
    function still returns true.
    
    Since GNU/Linux native and remote are the only targets that support
    non-stop mode, and with this patch, we always make sure the inferior
    has at least one thread, this patch also removes that "inferior with
    no threads" special case handling from handle_no_resumed.
    
    Since remote.c now has a special case where we treat a thread that has
    already exited as if it was still alive, we might need to tweak
    remote.c's target_thread_alive implementation to return true for that
    thread without querying the remote side (which would say "no, not
    alive").  After inspecting all the target_thread_alive calls in the
    codebase, it seems that only the one from prune_threads could result
    in that thread being accidentally deleted.  There's only one call to
    prune_threads in GDB's common code, so this patch handles this by
    replacing the prune_threads call with a delete_exited_threads call.
    This seems like an improvement anyway, because we'll still be doing
    what the comment suggests we want to do, and, we avoid remote protocol
    traffic.
    
    Regression-tested on X86_64 Linux.
    
    gdb/ChangeLog:
    2020-05-14  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
                Tom de Vries  <tdevries@suse.de>
                Pedro Alves  <palves@redhat.com>
    
            PR threads/25478
            * infrun.c (stop_all_threads): Do NOT ignore
            TARGET_WAITKIND_NO_RESUMED, TARGET_WAITKIND_THREAD_EXITED,
            TARGET_WAITKIND_EXITED, TARGET_WAITKIND_SIGNALLED wait statuses
            received.
            (handle_no_resumed): Remove code handling a live inferior with no
            threads.
            * remote.c (has_single_non_exited_thread): New.
            (remote_target::update_thread_list): Do not delete a thread if is
            the last thread of the process.
            * thread.c (thread_select): Call delete_exited_threads instead of
            prune_threads.
    
    gdb/testsuite/ChangeLog:
    2020-05-14  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
                Pedro Alves  <palves@redhat.com>
    
            * gdb.multi/multi-exit.c: New file.
            * gdb.multi/multi-exit.exp: New file.
            * gdb.multi/multi-kill.c: New file.
            * gdb.multi/multi-kill.exp: New file.

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 6d56c04bc3..c4da2a92f9 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,3 +1,20 @@
+2020-05-14  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
+	    Tom de Vries  <tdevries@suse.de>
+	    Pedro Alves  <palves@redhat.com>
+
+	PR threads/25478
+	* infrun.c (stop_all_threads): Do NOT ignore
+	TARGET_WAITKIND_NO_RESUMED, TARGET_WAITKIND_THREAD_EXITED,
+	TARGET_WAITKIND_EXITED, TARGET_WAITKIND_SIGNALLED wait statuses
+	received.
+	(handle_no_resumed): Remove code handling a live inferior with no
+	threads.
+	* remote.c (has_single_non_exited_thread): New.
+	(remote_target::update_thread_list): Do not delete a thread if is
+	the last thread of the process.
+	* thread.c (thread_select): Call delete_exited_threads instead of
+	prune_threads.
+
 2020-05-14  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
 
 	* infrun.c (stop_all_threads): Enable/disable thread events of all
diff --git a/gdb/infrun.c b/gdb/infrun.c
index 2a8e73c111..c3e23a28bd 100644
--- a/gdb/infrun.c
+++ b/gdb/infrun.c
@@ -4804,7 +4804,11 @@ stop_all_threads (void)
 	{
 	  int need_wait = 0;
 
-	  update_thread_list ();
+	  for (auto *target : all_non_exited_process_targets ())
+	    {
+	      switch_to_target_no_thread (target);
+	      update_thread_list ();
+	    }
 
 	  /* Go through all threads looking for threads that we need
 	     to tell the target to stop.  */
@@ -4879,13 +4883,63 @@ stop_all_threads (void)
 				  target_pid_to_str (event.ptid).c_str ());
 	    }
 
-	  if (event.ws.kind == TARGET_WAITKIND_NO_RESUMED
-	      || event.ws.kind == TARGET_WAITKIND_THREAD_EXITED
-	      || event.ws.kind == TARGET_WAITKIND_EXITED
-	      || event.ws.kind == TARGET_WAITKIND_SIGNALLED)
+	  if (event.ws.kind == TARGET_WAITKIND_NO_RESUMED)
+	    {
+	      /* All resumed threads exited.  */
+	    }
+	  else if (event.ws.kind == TARGET_WAITKIND_THREAD_EXITED
+		   || event.ws.kind == TARGET_WAITKIND_EXITED
+		   || event.ws.kind == TARGET_WAITKIND_SIGNALLED)
 	    {
-	      /* All resumed threads exited
-		 or one thread/process exited/signalled.  */
+	      /* One thread/process exited/signalled.  */
+
+	      thread_info *t = nullptr;
+
+	      /* The target may have reported just a pid.  If so, try
+		 the first non-exited thread.  */
+	      if (event.ptid.is_pid ())
+		{
+		  int pid  = event.ptid.pid ();
+		  inferior *inf = find_inferior_pid (event.target, pid);
+		  for (thread_info *tp : inf->non_exited_threads ())
+		    {
+		      t = tp;
+		      break;
+		    }
+
+		  /* If there is no available thread, the event would
+		     have to be appended to a per-inferior event list,
+		     which does not exist (and if it did, we'd have
+		     to adjust run control command to be able to
+		     resume such an inferior).  We assert here instead
+		     of going into an infinite loop.  */
+		  gdb_assert (t != nullptr);
+
+		  if (debug_infrun)
+		    fprintf_unfiltered (gdb_stdlog,
+					"infrun: stop_all_threads, using %s\n",
+					target_pid_to_str (t->ptid).c_str ());
+		}
+	      else
+		{
+		  t = find_thread_ptid (event.target, event.ptid);
+		  /* Check if this is the first time we see this thread.
+		     Don't bother adding if it individually exited.  */
+		  if (t == nullptr
+		      && event.ws.kind != TARGET_WAITKIND_THREAD_EXITED)
+		    t = add_thread (event.target, event.ptid);
+		}
+
+	      if (t != nullptr)
+		{
+		  /* Set the threads as non-executing to avoid
+		     another stop attempt on them.  */
+		  switch_to_thread_no_regs (t);
+		  mark_non_executing_threads (event.target, event.ptid,
+					      event.ws);
+		  save_waitstatus (t, &event.ws);
+		  t->stop_requested = false;
+		}
 	    }
 	  else
 	    {
@@ -5063,24 +5117,6 @@ handle_no_resumed (struct execution_control_state *ecs)
 	}
     }
 
-  /* Note however that we may find no resumed thread because the whole
-     process exited meanwhile (thus updating the thread list results
-     in an empty thread list).  In this case we know we'll be getting
-     a process exit event shortly.  */
-  for (inferior *inf : all_non_exited_inferiors (ecs->target))
-    {
-      thread_info *thread = any_live_thread_of_inferior (inf);
-      if (thread == NULL)
-	{
-	  if (debug_infrun)
-	    fprintf_unfiltered (gdb_stdlog,
-				"infrun: TARGET_WAITKIND_NO_RESUMED "
-				"(expect process exit)\n");
-	  prepare_to_wait (ecs);
-	  return 1;
-	}
-    }
-
   /* Go ahead and report the event.  */
   return 0;
 }
diff --git a/gdb/remote.c b/gdb/remote.c
index 5db406e045..5b1fa84853 100644
--- a/gdb/remote.c
+++ b/gdb/remote.c
@@ -3785,6 +3785,18 @@ remote_target::remote_get_threads_with_qthreadinfo (threads_listing_context *con
   return 0;
 }
 
+/* Return true if INF only has one non-exited thread.  */
+
+static bool
+has_single_non_exited_thread (inferior *inf)
+{
+  int count = 0;
+  for (thread_info *tp ATTRIBUTE_UNUSED : inf->non_exited_threads ())
+    if (++count > 1)
+      break;
+  return count == 1;
+}
+
 /* Implement the to_update_thread_list function for the remote
    targets.  */
 
@@ -3824,6 +3836,14 @@ remote_target::update_thread_list ()
 
 	  if (!context.contains_thread (tp->ptid))
 	    {
+	      /* Do not remove the thread if it is the last thread in
+		 the inferior.  This situation happens when we have a
+		 pending exit process status to process.  Otherwise we
+		 may end up with a seemingly live inferior (i.e.  pid
+		 != 0) that has no threads.  */
+	      if (has_single_non_exited_thread (tp->inf))
+		continue;
+
 	      /* Not found.  */
 	      delete_thread (tp);
 	    }
diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 3ce59bc1c5..c987c829c4 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,11 @@
+2020-05-14  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
+	    Pedro Alves  <palves@redhat.com>
+
+	* gdb.multi/multi-exit.c: New file.
+	* gdb.multi/multi-exit.exp: New file.
+	* gdb.multi/multi-kill.c: New file.
+	* gdb.multi/multi-kill.exp: New file.
+
 2020-05-14  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
 
 	* gdb.base/annota1.exp: Update the expected output.
diff --git a/gdb/testsuite/gdb.multi/multi-exit.c b/gdb/testsuite/gdb.multi/multi-exit.c
new file mode 100644
index 0000000000..f4825c8a7c
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/multi-exit.c
@@ -0,0 +1,22 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2020 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+int
+main ()
+{
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.multi/multi-exit.exp b/gdb/testsuite/gdb.multi/multi-exit.exp
new file mode 100644
index 0000000000..393093b378
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/multi-exit.exp
@@ -0,0 +1,134 @@
+# This testcase is part of GDB, the GNU debugger.
+
+# Copyright 2020 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test receiving TARGET_WAITKIND_EXITED events from multiple
+# inferiors.  In all stop-mode, upon receiving the exit event from one
+# of the inferiors, GDB will try to stop the other inferior, too.  So,
+# a stop request will be sent.  Receiving a TARGET_WAITKIND_EXITED
+# status kind as a response to that stop request instead of a
+# TARGET_WAITKIND_STOPPED should be handled by GDB without problems.
+
+standard_testfile
+
+if {[use_gdb_stub]} {
+    return 0
+}
+
+if {[build_executable "failed to prepare" $testfile $srcfile]} {
+    return -1
+}
+
+# We are testing GDB's ability to stop all threads.
+# Hence, go with the all-stop-on-top-of-non-stop mode.
+save_vars { GDBFLAGS } {
+    append GDBFLAGS " -ex \"maint set target-non-stop on\""
+    clean_restart ${binfile}
+}
+
+# Start inferior NUM.
+
+proc start_inferior {num} {
+    with_test_prefix "start_inferior $num" {
+	global srcfile binfile
+
+	if {$num != 1} {
+	    gdb_test "add-inferior" "Added inferior $num.*" \
+		"add empty inferior"
+	    gdb_test "inferior $num" "Switching to inferior $num.*" \
+		"switch to inferior"
+	}
+
+	gdb_load $binfile
+
+	if {[gdb_start_cmd] < 0} {
+	    fail "could not start"
+	    return -1
+	}
+	gdb_test "" ".*reakpoint .*, main .*${srcfile}.*" "start"
+    }
+
+    return 0
+}
+
+# Sufficient inferiors to make sure that at least some other inferior
+# exits while we're handling a process exit event.
+set NUM_INFS 10
+
+for {set i 1} {$i <= $NUM_INFS} {incr i} {
+    if {[start_inferior $i] < 0} {
+	return -1
+    }
+}
+
+# We want to continue all processes.
+gdb_test_no_output "set schedule-multiple on"
+
+# Check that "continue" continues to the end of an inferior, as many
+# times as we have inferiors.
+
+for {set i 1} {$i <= $NUM_INFS} {incr i} {
+    with_test_prefix "inf $i" {
+	set live_inferior ""
+
+	# Pick any live inferior.
+	gdb_test_multiple "info inferiors" "" {
+	    -re "($decimal) *process.*$gdb_prompt $" {
+		set live_inferior $expect_out(1,string)
+	    }
+	}
+
+	if {$live_inferior == ""} {
+	    return -1
+	}
+
+	gdb_test "inferior $live_inferior" \
+	    ".*Switching to inferior $live_inferior.*" \
+	    "switch to another inferior"
+
+	set exited_inferior ""
+
+	# We want GDB to complete the command and return the prompt
+	# instead of going into an infinite loop.
+	gdb_test_multiple "continue" "continue" {
+	    -re "Inferior ($decimal) \[^\n\r\]+ exited normally.*$gdb_prompt $" {
+		set exited_inferior $expect_out(1,string)
+		pass $gdb_test_name
+	    }
+	}
+
+	if {$exited_inferior == ""} {
+	    return -1
+	}
+    }
+}
+
+# Finally, check that we can re-run all inferiors.  Note that if any
+# inferior was still alive this would catch it, as "run" would query
+# "Start it from the beginning?".
+
+delete_breakpoints
+
+for {set i 1} {$i <= $NUM_INFS} {incr i} {
+    with_test_prefix "inf $i" {
+	gdb_test "inferior $i" \
+	    ".*Switching to inferior $i.*" \
+	    "switch to inferior for re-run"
+
+	gdb_test "run" "$inferior_exited_re normally\]" \
+	    "re-run inferior"
+    }
+}
diff --git a/gdb/testsuite/gdb.multi/multi-kill.c b/gdb/testsuite/gdb.multi/multi-kill.c
new file mode 100644
index 0000000000..66642bbb0e
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/multi-kill.c
@@ -0,0 +1,42 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2020 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <sys/types.h>
+#include <unistd.h>
+
+static pid_t pid;
+
+static void
+initialized ()
+{
+}
+
+int
+main ()
+{
+  pid = getpid ();
+  initialized ();
+
+  /* Don't run forever in case GDB crashes and DejaGNU fails to kill
+     this program.  */
+  alarm (10);
+
+  while (1)
+    ;
+
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.multi/multi-kill.exp b/gdb/testsuite/gdb.multi/multi-kill.exp
new file mode 100644
index 0000000000..ce6075045f
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/multi-kill.exp
@@ -0,0 +1,127 @@
+# This testcase is part of GDB, the GNU debugger.
+
+# Copyright 2020 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test receiving TARGET_WAITKIND_SIGNALLED events from multiple
+# inferiors.  In all stop-mode, upon receiving the exit event from one
+# of the inferiors, GDB will try to stop the other inferior, too.  So,
+# a stop request will be sent.  Receiving a TARGET_WAITKIND_SIGNALLED
+# status kind as a response to that stop request instead of a
+# TARGET_WAITKIND_STOPPED should be handled by GDB without problems.
+
+standard_testfile
+
+if {[use_gdb_stub]} {
+    return 0
+}
+
+if {[build_executable "failed to prepare" $testfile $srcfile {debug}]} {
+    return -1
+}
+
+# We are testing GDB's ability to stop all threads.
+# Hence, go with the all-stop-on-top-of-non-stop mode.
+save_vars { GDBFLAGS } {
+    append GDBFLAGS " -ex \"maint set target-non-stop on\""
+    clean_restart ${binfile}
+}
+
+# Start inferior NUM and record its PID in the TESTPID array.
+
+proc start_inferior {num} {
+    with_test_prefix "start_inferior $num" {
+	global testpid binfile srcfile
+
+	if {$num != 1} {
+	    gdb_test "add-inferior" "Added inferior .*" \
+		"add empty inferior"
+	    gdb_test "inferior $num" "Switching to inferior .*" \
+		"switch to inferior"
+	}
+
+	gdb_load $binfile
+
+	gdb_breakpoint "initialized" {temporary}
+	gdb_run_cmd
+	gdb_test "" ".*reakpoint .*, initialized .*${srcfile}.*" "run"
+
+	set testpid($num) [get_integer_valueof "pid" -1]
+	if {$testpid($num) == -1} {
+	    return -1
+	}
+
+	return 0
+    }
+}
+
+# Sufficient inferiors to make sure that at least some other inferior
+# is killed while we're handling a killed event.
+set NUM_INFS 10
+
+for {set i 1} {$i <= $NUM_INFS} {incr i} {
+    if {[start_inferior $i] < 0} {
+	return -1
+    }
+}
+
+# We want to continue all processes.
+gdb_test_no_output "set schedule-multiple on"
+
+# Resume, but then kill all from outside.
+gdb_test_multiple "continue" "continue processes" {
+    -re "Continuing.\[\r\n\]+" {
+	# Kill all processes at once.
+
+	set kill_cmd "kill -9"
+	for {set i 1} {$i <= $NUM_INFS} {incr i} {
+	    append kill_cmd " $testpid($i)"
+	}
+
+	remote_exec target $kill_cmd
+	exp_continue
+    }
+    -re "Program terminated with signal.*$gdb_prompt $" {
+	pass $gdb_test_name
+    }
+}
+
+# Check that "continue" collects the process kill event, as many times
+# as we have inferiors left.
+
+for {set i 2} {$i <= $NUM_INFS} {incr i} {
+    with_test_prefix "inf $i" {
+	set live_inferior ""
+
+	# Pick any live inferior.
+	gdb_test_multiple "info inferiors" "" {
+	    -re "($decimal) *process.*$gdb_prompt $" {
+		set live_inferior $expect_out(1,string)
+	    }
+	}
+
+	if {$live_inferior == ""} {
+	    return -1
+	}
+
+	gdb_test "inferior $live_inferior" \
+	    ".*Switching to inferior $live_inferior.*" \
+	    "switch to inferior"
+
+	gdb_test "continue" \
+	    "Program terminated with signal SIGKILL, .*" \
+	    "continue to SIGKILL"
+    }
+}
diff --git a/gdb/thread.c b/gdb/thread.c
index 03805bd256..02672f01fc 100644
--- a/gdb/thread.c
+++ b/gdb/thread.c
@@ -2043,7 +2043,7 @@ thread_select (const char *tidstr, thread_info *tp)
 
   /* Since the current thread may have changed, see if there is any
      exited thread we can now delete.  */
-  prune_threads ();
+  delete_exited_threads ();
 }
 
 /* Print thread and frame switch command response.  */

Failures on Fedora-x86_64-cc-with-index, branch master

[gdb-buildbot]

Buildername:
        Fedora-x86_64-cc-with-index

Worker:
        fedora-x86-64-3

Full Build URL:
	https://gdb-buildbot.osci.io/#builders/20/builds/3087

Author:
        Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>

Commit tested:
        a05575d39a5348bd9979fc09e658a03ff22722b9

Subject of commit:
        gdb/infrun: handle already-exited threads when attempting to stop

Testsuite logs (gdb.sum, gdb.log and others):
        https://gdb-buildbot.osci.io/results/Fedora-x86_64-cc-with-index/a0/a05575d39a5348bd9979fc09e658a03ff22722b9/

*** Diff to previous build ***
==============================================
PASS -> KFAIL: gdb.threads/non-ldr-exit.exp: program exits normally
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: multi_scope: second thread: print i02
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: multi_scope: second thread: print i12
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: multi_scope: second thread: print i22
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: inner_threads: 2nd stop: print i
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: inner_threads: 2nd stop: print j
==============================================

*** Complete list of XFAILs for this builder ***

To obtain the list of XFAIL tests for this builder, go to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-cc-with-index/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.gz>

You can also see a pretty-printed version of the list, with more information
about each XFAIL, by going to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-cc-with-index/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.table.gz>

Failures on Fedora-x86_64-native-extended-gdbserver-m32, branch master

[gdb-buildbot]

Buildername:
        Fedora-x86_64-native-extended-gdbserver-m32

Worker:
        fedora-x86-64-4

Full Build URL:
	https://gdb-buildbot.osci.io/#builders/4/builds/3029

Author:
        Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>

Commit tested:
        a05575d39a5348bd9979fc09e658a03ff22722b9

Subject of commit:
        gdb/infrun: handle already-exited threads when attempting to stop

Testsuite logs (gdb.sum, gdb.log and others):
        https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-extended-gdbserver-m32/a0/a05575d39a5348bd9979fc09e658a03ff22722b9/

*** Diff to previous build ***
==============================================
PASS -> FAIL: gdb.linespec/explicit.exp: complete unique label name reversed: cmd complete "b -label top -function myfunction"
PASS -> FAIL: gdb.linespec/explicit.exp: complete unique label name reversed: tab complete "b -label top -function myfunction"
PASS -> FAIL: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=1: continue until exit
PASS -> FAIL: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=1: print re_run_var_1
new UNRESOLVED: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=2: delete all breakpoints in delete_breakpoints
PASS -> UNRESOLVED: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=2: setting breakpoint at all_started
PASS -> FAIL: gdb.multi/multi-re-run.exp: re_run_inf=2: iter=2: continue until exit
PASS -> FAIL: gdb.multi/multi-re-run.exp: re_run_inf=2: iter=2: print re_run_var_2
UNRESOLVED -> FAIL: gdb.threads/fork-plus-threads.exp: detach-on-fork=off: inferior 1 exited
new FAIL: gdb.threads/fork-plus-threads.exp: detach-on-fork=off: only inferior 1 left
==============================================

*** Complete list of XFAILs for this builder ***

To obtain the list of XFAIL tests for this builder, go to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-extended-gdbserver-m32/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.gz>

You can also see a pretty-printed version of the list, with more information
about each XFAIL, by going to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-extended-gdbserver-m32/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.table.gz>

Failures on Fedora-x86_64-m64, branch master

[gdb-buildbot]

Buildername:
        Fedora-x86_64-m64

Worker:
        fedora-x86-64-3

Full Build URL:
	https://gdb-buildbot.osci.io/#builders/3/builds/3193

Author:
        Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>

Commit tested:
        a05575d39a5348bd9979fc09e658a03ff22722b9

Subject of commit:
        gdb/infrun: handle already-exited threads when attempting to stop

Testsuite logs (gdb.sum, gdb.log and others):
        https://gdb-buildbot.osci.io/results/Fedora-x86_64-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9/

*** Diff to previous build ***
==============================================
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 1st call: 1st thread: print k
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 1st call: 1st thread: print r
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 1st call: 1st thread: print z
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: outer_threads: outer stop: print i
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: outer_threads: outer stop: print j
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: single_scope: second thread: print i3
==============================================

*** Complete list of XFAILs for this builder ***

To obtain the list of XFAIL tests for this builder, go to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.gz>

You can also see a pretty-printed version of the list, with more information
about each XFAIL, by going to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.table.gz>

Failures on Fedora-x86_64-native-extended-gdbserver-m64, branch master

[gdb-buildbot]

Buildername:
        Fedora-x86_64-native-extended-gdbserver-m64

Worker:
        fedora-x86-64-4

Full Build URL:
	https://gdb-buildbot.osci.io/#builders/2/builds/3030

Author:
        Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>

Commit tested:
        a05575d39a5348bd9979fc09e658a03ff22722b9

Subject of commit:
        gdb/infrun: handle already-exited threads when attempting to stop

Testsuite logs (gdb.sum, gdb.log and others):
        https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-extended-gdbserver-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9/

*** Diff to previous build ***
==============================================
PASS -> FAIL: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=1: continue until exit
PASS -> FAIL: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=1: print re_run_var_1
new UNRESOLVED: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=2: delete all breakpoints in delete_breakpoints
PASS -> UNRESOLVED: gdb.multi/multi-re-run.exp: re_run_inf=1: iter=2: setting breakpoint at all_started
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 10: break at break_fn: 1
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 10: break at break_fn: 2
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 10: break at break_fn: 3
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 5: break at break_fn: 1
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 5: break at break_fn: 2
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 5: break at break_fn: 3
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 5: detach
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 5: reset timer in the inferior
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 6: break at break_fn: 1
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 6: break at break_fn: 2
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 6: break at break_fn: 3
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 6: detach
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 6: reset timer in the inferior
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 7: break at break_fn: 1
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 7: break at break_fn: 2
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 7: break at break_fn: 3
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 7: detach
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 7: reset timer in the inferior
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 8: break at break_fn: 1
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 8: break at break_fn: 2
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 8: break at break_fn: 3
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 8: detach
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 8: reset timer in the inferior
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 9: break at break_fn: 1
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 9: break at break_fn: 2
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 9: break at break_fn: 3
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 9: detach
PASS -> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 9: reset timer in the inferior
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: multi_scope: second thread: print i02
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: multi_scope: second thread: print i12
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: multi_scope: second thread: print i22
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 2nd call: 1st thread: print k
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 2nd call: 1st thread: print r
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 2nd call: 1st thread: print z
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: inner_threads: 2nd stop: print i
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: inner_threads: 2nd stop: print j
PASS -> KFAIL: gdb.threads/process-dies-while-handling-bp.exp: non_stop=off: cond_bp_target=0: inferior 1 exited
==============================================

*** Complete list of XFAILs for this builder ***

To obtain the list of XFAIL tests for this builder, go to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-extended-gdbserver-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.gz>

You can also see a pretty-printed version of the list, with more information
about each XFAIL, by going to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-extended-gdbserver-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.table.gz>

Failures on Fedora-x86_64-native-gdbserver-m32, branch master

[gdb-buildbot]

Buildername:
        Fedora-x86_64-native-gdbserver-m32

Worker:
        fedora-x86-64-3

Full Build URL:
	https://gdb-buildbot.osci.io/#builders/24/builds/3042

Author:
        Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>

Commit tested:
        a05575d39a5348bd9979fc09e658a03ff22722b9

Subject of commit:
        gdb/infrun: handle already-exited threads when attempting to stop

Testsuite logs (gdb.sum, gdb.log and others):
        https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-gdbserver-m32/a0/a05575d39a5348bd9979fc09e658a03ff22722b9/

*** Diff to previous build ***
==============================================
new FAIL: gdb.base/coredump-filter.exp: disassembling function main for non-Private-Anonymous: no binary: loading /home/gdb-buildbot/fedora-x86-64-3/fedora-x86-64-native-gdbserver-m32/build/gdb/testsuite/outputs/gdb.base/coredump-filter/non-private-anon.gcore
new FAIL: gdb.base/coredump-filter.exp: disassembling function main for non-Shared-Anonymous: no binary: loading /home/gdb-buildbot/fedora-x86-64-3/fedora-x86-64-native-gdbserver-m32/build/gdb/testsuite/outputs/gdb.base/coredump-filter/non-shared-anon.gcore
new FAIL: gdb.base/coredump-filter.exp: loading and testing corefile for DoNotDump: loading /home/gdb-buildbot/fedora-x86-64-3/fedora-x86-64-native-gdbserver-m32/build/gdb/testsuite/outputs/gdb.base/coredump-filter/dont-dump.gcore
new FAIL: gdb.base/coredump-filter.exp: loading and testing corefile for non-Shared-Anonymous: loading /home/gdb-buildbot/fedora-x86-64-3/fedora-x86-64-native-gdbserver-m32/build/gdb/testsuite/outputs/gdb.base/coredump-filter/non-shared-anon.gcore
==============================================

*** Complete list of XFAILs for this builder ***

To obtain the list of XFAIL tests for this builder, go to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-gdbserver-m32/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.gz>

You can also see a pretty-printed version of the list, with more information
about each XFAIL, by going to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-gdbserver-m32/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.table.gz>

Failures on Fedora-x86_64-native-gdbserver-m64, branch master

[gdb-buildbot]

Buildername:
        Fedora-x86_64-native-gdbserver-m64

Worker:
        fedora-x86-64-4

Full Build URL:
	https://gdb-buildbot.osci.io/#builders/22/builds/3040

Author:
        Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>

Commit tested:
        a05575d39a5348bd9979fc09e658a03ff22722b9

Subject of commit:
        gdb/infrun: handle already-exited threads when attempting to stop

Testsuite logs (gdb.sum, gdb.log and others):
        https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-gdbserver-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9/

*** Diff to previous build ***
==============================================
new FAIL: gdb.base/gdb-caching-proc.exp: show print elements
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 1st call: 2nd thread: print k
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 1st call: 2nd thread: print r
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_func: 1st call: 2nd thread: print z
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: inner_threads: 4th stop: print i
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: inner_threads: 4th stop: print j
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: outer_threads: outer stop: print i
PASS -> KFAIL: gdb.threads/omp-par-scope.exp: nested_parallel: outer_threads: outer stop: print j
PASS -> FAIL: gdb.trace/mi-tsv-changed.exp: create delete modify: tvariable $tvar3 modified
==============================================

*** Complete list of XFAILs for this builder ***

To obtain the list of XFAIL tests for this builder, go to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-gdbserver-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.gz>

You can also see a pretty-printed version of the list, with more information
about each XFAIL, by going to:

        <https://gdb-buildbot.osci.io/results/Fedora-x86_64-native-gdbserver-m64/a0/a05575d39a5348bd9979fc09e658a03ff22722b9//xfail.table.gz>