GDB 10.2 Release -- 2021-04-11 update

GDB 10.2 Release -- 2021-04-11 update

[Joel Brobecker]

Hello everyone,

Here is an update on the 10.2 release process. We have one issue
that we have identified as being blocking. I'm proposing below
that we allow ourselves one or two weeks max to fix this issue,
and then release without the fix if needed. See below for the details
and rationale. Please let me know what you guys think.


Fixed Since the Previous Update:
--------------------------------

  * [Simon] <PR symtab/27541>
    gdb crashes on "file -readnow"
    https://sourceware.org/bugzilla/show_bug.cgi?id=27541

Added Since the Last Update:
----------------------------

  * [Simon/Kevin] <PR gdb/27526>
    Attaching to threaded process on glibc 2.33: libthread_db fails to initialize with "generic error"

    Patch proposed here:
    https://sourceware.org/pipermail/gdb-patches/2021-March/177369.html

    Discussed there:
    https://sourceware.org/pipermail/gdb-patches/2021-April/177477.html

    The issue is about GDB triggering an error when trying to attach
    to a process on systems that have upgraded to Glibc 2.33. So,
    theoretically, users in that situation could also try to run
    their program from GDB and debug that instead.

    Do we know how widely deployed glibc-2.33 is?

    This looks like a difficult issue to solve, with both GDB *and*
    GDBserver needing a fix, and no clear indication that I could see
    that they would be getting a similar fix. With that in mind,
    I'm wondering if we shouldn't give ourselves a deadline after which
    we simply accept this as a known limitation of the 10.2 release,
    and work on fixing it for the GDB 11 release instead.

    Speaking of the GDB 11 release, my thinking was that we should be
    starting the process immediately after the 10.2 release is out.
    So we'd be looking at maybe a couple of months between now and
    the GDB 11 release. In other words, if we manage to fix this glibc-2.33
    issues soon, and we don't have too many difficulties stabilizing
    the GDB 11 release, the fix should be in a release pretty quickly.

    If people agree, I suggest we give this problem 1 or 2 weeks max,
    and then we release with what we have.

    Thoughts?

Other Ongoing Items:
--------------------

  < none :) >

Not Critical, but Requested:
----------------------------

  < none :) >

Thank you!
-- 
Joel

Re: GDB 10.2 Release -- 2021-04-11 update

[Kevin Buettner]

On Sun, 11 Apr 2021 09:39:20 +0400
Joel Brobecker <brobecker@adacore.com> wrote:

>   * [Simon/Kevin] <PR gdb/27526>
>     Attaching to threaded process on glibc 2.33: libthread_db fails to initialize with "generic error"
> 
>     Patch proposed here:
>     https://sourceware.org/pipermail/gdb-patches/2021-March/177369.html
> 
>     Discussed there:
>     https://sourceware.org/pipermail/gdb-patches/2021-April/177477.html
> 
>     The issue is about GDB triggering an error when trying to attach
>     to a process on systems that have upgraded to Glibc 2.33. So,
>     theoretically, users in that situation could also try to run
>     their program from GDB and debug that instead.
> 
>     Do we know how widely deployed glibc-2.33 is?

glibc-2.33 will be used in Fedora 34, which is in beta now.  I would
guess that other distributions will use glibc-2.33 for their upcoming
releases too.

>     This looks like a difficult issue to solve, with both GDB *and*
>     GDBserver needing a fix, and no clear indication that I could see
>     that they would be getting a similar fix. With that in mind,
>     I'm wondering if we shouldn't give ourselves a deadline after which
>     we simply accept this as a known limitation of the 10.2 release,
>     and work on fixing it for the GDB 11 release instead.

So... I have the beginnings of a patch for gdbserver.  I'm tracking
down some regressions though, so don't have anything I can share
quite yet.

I agree with giving ourselves a short deadline for fixing this problem
(and the rest of your approach).

> 
>     Speaking of the GDB 11 release, my thinking was that we should be
>     starting the process immediately after the 10.2 release is out.
>     So we'd be looking at maybe a couple of months between now and
>     the GDB 11 release. In other words, if we manage to fix this glibc-2.33
>     issues soon, and we don't have too many difficulties stabilizing
>     the GDB 11 release, the fix should be in a release pretty quickly.
> 
>     If people agree, I suggest we give this problem 1 or 2 weeks max,
>     and then we release with what we have.
> 
>     Thoughts?

Sounds good too me.

Kevin

Re: GDB 10.2 Release -- 2021-04-11 update

[Simon Marchi]

On 2021-04-12 2:00 a.m., Kevin Buettner wrote:
> On Sun, 11 Apr 2021 09:39:20 +0400
> Joel Brobecker <brobecker@adacore.com> wrote:
> 
>>   * [Simon/Kevin] <PR gdb/27526>
>>     Attaching to threaded process on glibc 2.33: libthread_db fails to initialize with "generic error"
>>
>>     Patch proposed here:
>>     https://sourceware.org/pipermail/gdb-patches/2021-March/177369.html
>>
>>     Discussed there:
>>     https://sourceware.org/pipermail/gdb-patches/2021-April/177477.html
>>
>>     The issue is about GDB triggering an error when trying to attach
>>     to a process on systems that have upgraded to Glibc 2.33. So,
>>     theoretically, users in that situation could also try to run
>>     their program from GDB and debug that instead.
>>
>>     Do we know how widely deployed glibc-2.33 is?
> 
> glibc-2.33 will be used in Fedora 34, which is in beta now.  I would
> guess that other distributions will use glibc-2.33 for their upcoming
> releases too.

Ubuntu 21.04 (Hirsute) will have that combo of GDB 10 and glibc 2.33
too:

  https://packages.ubuntu.com/hirsute/gdb
  https://packages.ubuntu.com/hirsute/libc6

>>     This looks like a difficult issue to solve, with both GDB *and*
>>     GDBserver needing a fix, and no clear indication that I could see
>>     that they would be getting a similar fix. With that in mind,
>>     I'm wondering if we shouldn't give ourselves a deadline after which
>>     we simply accept this as a known limitation of the 10.2 release,
>>     and work on fixing it for the GDB 11 release instead.
> 
> So... I have the beginnings of a patch for gdbserver.  I'm tracking
> down some regressions though, so don't have anything I can share
> quite yet.

I didn't have time to look at GDBserver yet.  Could you share your patch
(it could be just a link to an external git repo) even if it's not ready
yet?

Simon

Re: GDB 10.2 Release -- 2021-04-11 update

[Kevin Buettner]

On Mon, 12 Apr 2021 10:04:44 -0400
Simon Marchi <simon.marchi@polymtl.ca> wrote:

> >>     This looks like a difficult issue to solve, with both GDB *and*
> >>     GDBserver needing a fix, and no clear indication that I could see
> >>     that they would be getting a similar fix. With that in mind,
> >>     I'm wondering if we shouldn't give ourselves a deadline after which
> >>     we simply accept this as a known limitation of the 10.2 release,
> >>     and work on fixing it for the GDB 11 release instead.  
> > 
> > So... I have the beginnings of a patch for gdbserver.  I'm tracking
> > down some regressions though, so don't have anything I can share
> > quite yet.  
> 
> I didn't have time to look at GDBserver yet.  Could you share your patch
> (it could be just a link to an external git repo) even if it's not ready
> yet?

My current patch is below.  It's small, so I decided to post it in
this reply.

It no longer works for the case which originally got me interested in
this problem (which is gdb.threads/fork-plus-threads.exp w/
--target_board=native-extended-gdbserver), but it also no longer
causes regressions in gdb.threads/tls.exp.  Things are getting further
though - i.e. _rtld_global is now being found.

Things are going astray further along, but I'm not quite sure where
yet.  I'm still working on it...

diff --git a/gdbserver/thread-db.cc b/gdbserver/thread-db.cc
index 055a0fa970f..9403b419710 100644
--- a/gdbserver/thread-db.cc
+++ b/gdbserver/thread-db.cc
@@ -44,10 +44,14 @@ struct thread_db
   /* Connection to the libthread_db library.  */
   td_thragent_t *thread_agent;
 
-  /* If this flag has been set, we've already asked GDB for all
-     symbols we might need; assume symbol cache misses are
-     failures.  */
-  int all_symbols_looked_up;
+  /* This flag is set after asking GDB for symbols that we need.  It
+     does not, however, indicate that all necessary symbols have found
+     as these may be split among several shared objects.  */
+  bool initialized;
+
+  /* Flag which indicates that it's okay to ask GDB for symbols via
+     a qSymbol reply.  */
+  bool qsymbol_reply_okay;
 
 #ifndef USE_LIBTHREAD_DB_DIRECTLY
   /* Handle of the libthread_db from dlopen.  */
@@ -359,19 +363,20 @@ thread_db_look_up_symbols (void)
   const char **sym_list;
   CORE_ADDR unused;
 
+  thread_db->qsymbol_reply_okay = true;
+
   for (sym_list = thread_db->td_symbol_list_p (); *sym_list; sym_list++)
     look_up_one_symbol (*sym_list, &unused, 1);
 
-  /* We're not interested in any other libraries loaded after this
-     point, only in symbols in libpthread.so.  */
-  thread_db->all_symbols_looked_up = 1;
+  thread_db->qsymbol_reply_okay = false;
+  thread_db->initialized = true;
 }
 
 int
 thread_db_look_up_one_symbol (const char *name, CORE_ADDR *addrp)
 {
   struct thread_db *thread_db = current_process ()->priv->thread_db;
-  int may_ask_gdb = !thread_db->all_symbols_looked_up;
+  int may_ask_gdb = thread_db->qsymbol_reply_okay || !thread_db->initialized;
 
   /* If we've passed the call to thread_db_look_up_symbols, then
      anything not in the cache must not exist; we're not interested
@@ -396,7 +401,7 @@ thread_db_get_tls_address (struct thread_info *thread, CORE_ADDR offset,
   thread_db = proc->priv->thread_db;
 
   /* If the thread layer is not (yet) initialized, fail.  */
-  if (thread_db == NULL || !thread_db->all_symbols_looked_up)
+  if (thread_db == NULL || !thread_db->initialized)
     return TD_ERR;
 
   /* If td_thr_tls_get_addr is missing rather do not expect td_thr_tlsbase
@@ -888,7 +893,7 @@ thread_db_notice_clone (struct thread_info *parent_thr, ptid_t child_ptid)
 
   /* If the thread layer isn't initialized, return.  It may just
      be that the program uses clone, but does not use libthread_db.  */
-  if (thread_db == NULL || !thread_db->all_symbols_looked_up)
+  if (thread_db == NULL || !thread_db->initialized)
     return;
 
   /* find_one_thread calls into libthread_db which accesses memory via

Re: GDB 10.2 Release -- 2021-04-11 update

[Simon Marchi]

On 2021-04-13 12:18 a.m., Kevin Buettner wrote:> On Mon, 12 Apr 2021 10:04:44 -0400
> Simon Marchi <simon.marchi@polymtl.ca> wrote:
> 
>>>>     This looks like a difficult issue to solve, with both GDB *and*
>>>>     GDBserver needing a fix, and no clear indication that I could see
>>>>     that they would be getting a similar fix. With that in mind,
>>>>     I'm wondering if we shouldn't give ourselves a deadline after which
>>>>     we simply accept this as a known limitation of the 10.2 release,
>>>>     and work on fixing it for the GDB 11 release instead.  
>>>
>>> So... I have the beginnings of a patch for gdbserver.  I'm tracking
>>> down some regressions though, so don't have anything I can share
>>> quite yet.  
>>
>> I didn't have time to look at GDBserver yet.  Could you share your patch
>> (it could be just a link to an external git repo) even if it's not ready
>> yet?
> 
> My current patch is below.  It's small, so I decided to post it in
> this reply.
> 
> It no longer works for the case which originally got me interested in
> this problem (which is gdb.threads/fork-plus-threads.exp w/
> --target_board=native-extended-gdbserver), but it also no longer
> causes regressions in gdb.threads/tls.exp.  Things are getting further
> though - i.e. _rtld_global is now being found.
> 
> Things are going astray further along, but I'm not quite sure where
> yet.  I'm still working on it...
> 
> diff --git a/gdbserver/thread-db.cc b/gdbserver/thread-db.cc
> index 055a0fa970f..9403b419710 100644
> --- a/gdbserver/thread-db.cc
> +++ b/gdbserver/thread-db.cc
> @@ -44,10 +44,14 @@ struct thread_db
>    /* Connection to the libthread_db library.  */
>    td_thragent_t *thread_agent;
>  
> -  /* If this flag has been set, we've already asked GDB for all
> -     symbols we might need; assume symbol cache misses are
> -     failures.  */
> -  int all_symbols_looked_up;
> +  /* This flag is set after asking GDB for symbols that we need.  It
> +     does not, however, indicate that all necessary symbols have found
> +     as these may be split among several shared objects.  */
> +  bool initialized;
> +
> +  /* Flag which indicates that it's okay to ask GDB for symbols via
> +     a qSymbol reply.  */
> +  bool qsymbol_reply_okay;
>  
>  #ifndef USE_LIBTHREAD_DB_DIRECTLY
>    /* Handle of the libthread_db from dlopen.  */
> @@ -359,19 +363,20 @@ thread_db_look_up_symbols (void)
>    const char **sym_list;
>    CORE_ADDR unused;
>  
> +  thread_db->qsymbol_reply_okay = true;
> +
>    for (sym_list = thread_db->td_symbol_list_p (); *sym_list; sym_list++)
>      look_up_one_symbol (*sym_list, &unused, 1);
>  
> -  /* We're not interested in any other libraries loaded after this
> -     point, only in symbols in libpthread.so.  */
> -  thread_db->all_symbols_looked_up = 1;
> +  thread_db->qsymbol_reply_okay = false;
> +  thread_db->initialized = true;
>  }
>  
>  int
>  thread_db_look_up_one_symbol (const char *name, CORE_ADDR *addrp)
>  {
>    struct thread_db *thread_db = current_process ()->priv->thread_db;
> -  int may_ask_gdb = !thread_db->all_symbols_looked_up;
> +  int may_ask_gdb = thread_db->qsymbol_reply_okay || !thread_db->initialized;
>  
>    /* If we've passed the call to thread_db_look_up_symbols, then
>       anything not in the cache must not exist; we're not interested
> @@ -396,7 +401,7 @@ thread_db_get_tls_address (struct thread_info *thread, CORE_ADDR offset,
>    thread_db = proc->priv->thread_db;
>  
>    /* If the thread layer is not (yet) initialized, fail.  */
> -  if (thread_db == NULL || !thread_db->all_symbols_looked_up)
> +  if (thread_db == NULL || !thread_db->initialized)
>      return TD_ERR;
>  
>    /* If td_thr_tls_get_addr is missing rather do not expect td_thr_tlsbase
> @@ -888,7 +893,7 @@ thread_db_notice_clone (struct thread_info *parent_thr, ptid_t child_ptid)
>  
>    /* If the thread layer isn't initialized, return.  It may just
>       be that the program uses clone, but does not use libthread_db.  */
> -  if (thread_db == NULL || !thread_db->all_symbols_looked_up)
> +  if (thread_db == NULL || !thread_db->initialized)
>      return;
>  
>    /* find_one_thread calls into libthread_db which accesses memory via
> 

Thanks!

So from what I understand, the problem is that GDB offers GDBserver the
opportunity to look up some symbols in between each objfile read in the
initial library scan.  After GDB has loaded lipthread (but not ld-linux
yet), libthread_db fails to work because GDB doesn't know the ld-linux
symbols yet.  It sounds very much like the non-GDBserver case.

I see that you are trying to fix that GDBserver-side (although I don't
quite get what you are trying to do, perhaps a quick explanation would
help).  Could we instead tackle it GDB-side, with the same idea as what
I proposed the native debugging?  We could suppress the qsymbol packets
sent in between each objfile loaded from the initial library scan.  We
would have a single qsymbol sent after finishing loading all the initial
libraries.

For example, I just hacked this, this makes it work for the simple
non-stop attach case with gdbserver, where you run gdbserver with
--attach and then connect with GDB (I haven't tried the other modes).
The final qsymbol, that lets GDBserver load libthread_db, is sent in
remote_target::start_remote.


From b6f8afc01e0e6377e27da640b809d6013e02cc3f Mon Sep 17 00:00:00 2001
From: Simon Marchi <simon.marchi@polymtl.ca>
Date: Tue, 13 Apr 2021 21:33:37 -0400
Subject: [PATCH] blah

Change-Id: I601aaf0b9bf3040b85ce7ba675afdd8986c56a34
---
 gdb/remote.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gdb/remote.c b/gdb/remote.c
index 7429e1a86b38..282f4cfafbff 100644
--- a/gdb/remote.c
+++ b/gdb/remote.c
@@ -252,6 +252,8 @@ class remote_state
      about the remote side's threads, relocating symbols, etc.).  */
   bool starting_up = false;
 
+  bool some_other_boolean = false;
+
   /* If we negotiated packet size explicitly (and thus can bypass
      heuristics for the largest packet size that will not overflow
      a buffer in the stub), this will be set to that packet size.
@@ -4886,7 +4888,9 @@ remote_target::start_remote (int from_tty, int extended_p)
       strcpy (rs->buf.data (), wait_status);
       rs->cached_wait_status = 1;
 
+      rs->some_other_boolean = true;
       ::start_remote (from_tty); /* Initialize gdb process mechanisms.  */
+      rs->some_other_boolean = false;
     }
   else
     {
@@ -14515,7 +14519,7 @@ remote_new_objfile (struct objfile *objfile)
 {
   remote_target *remote = get_current_remote_target ();
 
-  if (remote != NULL)			/* Have a remote connection.  */
+  if (remote != NULL && !remote->get_remote_state()->some_other_boolean)			/* Have a remote connection.  */
     remote->remote_check_symbols ();
 }
 
-- 
2.30.1

Re: GDB 10.2 Release -- 2021-04-11 update

[Kevin Buettner]

On Tue, 13 Apr 2021 21:45:53 -0400
Simon Marchi <simon.marchi@polymtl.ca> wrote:

> On 2021-04-13 12:18 a.m., Kevin Buettner wrote:> On Mon, 12 Apr 2021 10:04:44 -0400
[...]
> > 
> > diff --git a/gdbserver/thread-db.cc b/gdbserver/thread-db.cc
> > index 055a0fa970f..9403b419710 100644
> > --- a/gdbserver/thread-db.cc
> > +++ b/gdbserver/thread-db.cc
> > @@ -44,10 +44,14 @@ struct thread_db
> >    /* Connection to the libthread_db library.  */
> >    td_thragent_t *thread_agent;
> >  
> > -  /* If this flag has been set, we've already asked GDB for all
> > -     symbols we might need; assume symbol cache misses are
> > -     failures.  */
> > -  int all_symbols_looked_up;
> > +  /* This flag is set after asking GDB for symbols that we need.  It
> > +     does not, however, indicate that all necessary symbols have found
> > +     as these may be split among several shared objects.  */
> > +  bool initialized;
> > +
> > +  /* Flag which indicates that it's okay to ask GDB for symbols via
> > +     a qSymbol reply.  */
> > +  bool qsymbol_reply_okay;
> >  
> >  #ifndef USE_LIBTHREAD_DB_DIRECTLY
> >    /* Handle of the libthread_db from dlopen.  */
> > @@ -359,19 +363,20 @@ thread_db_look_up_symbols (void)
> >    const char **sym_list;
> >    CORE_ADDR unused;
> >  
> > +  thread_db->qsymbol_reply_okay = true;
> > +
> >    for (sym_list = thread_db->td_symbol_list_p (); *sym_list; sym_list++)
> >      look_up_one_symbol (*sym_list, &unused, 1);
> >  
> > -  /* We're not interested in any other libraries loaded after this
> > -     point, only in symbols in libpthread.so.  */
> > -  thread_db->all_symbols_looked_up = 1;
> > +  thread_db->qsymbol_reply_okay = false;
> > +  thread_db->initialized = true;
> >  }
> >  
> >  int
> >  thread_db_look_up_one_symbol (const char *name, CORE_ADDR *addrp)
> >  {
> >    struct thread_db *thread_db = current_process ()->priv->thread_db;
> > -  int may_ask_gdb = !thread_db->all_symbols_looked_up;
> > +  int may_ask_gdb = thread_db->qsymbol_reply_okay || !thread_db->initialized;
> >  
> >    /* If we've passed the call to thread_db_look_up_symbols, then
> >       anything not in the cache must not exist; we're not interested
> > @@ -396,7 +401,7 @@ thread_db_get_tls_address (struct thread_info *thread, CORE_ADDR offset,
> >    thread_db = proc->priv->thread_db;
> >  
> >    /* If the thread layer is not (yet) initialized, fail.  */
> > -  if (thread_db == NULL || !thread_db->all_symbols_looked_up)
> > +  if (thread_db == NULL || !thread_db->initialized)
> >      return TD_ERR;
> >  
> >    /* If td_thr_tls_get_addr is missing rather do not expect td_thr_tlsbase
> > @@ -888,7 +893,7 @@ thread_db_notice_clone (struct thread_info *parent_thr, ptid_t child_ptid)
> >  
> >    /* If the thread layer isn't initialized, return.  It may just
> >       be that the program uses clone, but does not use libthread_db.  */
> > -  if (thread_db == NULL || !thread_db->all_symbols_looked_up)
> > +  if (thread_db == NULL || !thread_db->initialized)
> >      return;
> >  
> >    /* find_one_thread calls into libthread_db which accesses memory via
> >   
> 
> Thanks!
> 
> So from what I understand, the problem is that GDB offers GDBserver the
> opportunity to look up some symbols in between each objfile read in the
> initial library scan.  After GDB has loaded lipthread (but not ld-linux
> yet), libthread_db fails to work because GDB doesn't know the ld-linux
> symbols yet.  It sounds very much like the non-GDBserver case.
> 
> I see that you are trying to fix that GDBserver-side (although I don't
> quite get what you are trying to do, perhaps a quick explanation would
> help).

On the gdbserver side of things, I noticed that there's a flag
named "all_symbols_looked_up".  This flag is used for two purposes:

1) It's used by thread_db_get_tls_address() and
thread_db_notice_clone() to test whether the thread_db layer is
initialized enough to proceed.

2) It's used by thread_db_look_up_one_symbol() to find out whether
it's okay to send a qSymbol reply.  Regarding this particular change,
it currently looks like this:

-  int may_ask_gdb = !thread_db->all_symbols_looked_up;
+  int may_ask_gdb = thread_db->qsymbol_reply_okay || !thread_db->initialized;

However, I had originally (re)written that line as follows:

+  int may_ask_gdb = thread_db->qsymbol_reply_okay;

This looks correct to me, and it caused
gdb.threads/fork-plus-threads.exp to work as well as it does on a
glibc 2.32 machine.  Unfortunately, it caused gdb.threads/tls.exp
(and other tests too) to regress.

So, anyway, my thought was to split that one flag which serves
two purposes into separate flags.  This would allow us to process
additional qSymbol requests from GDB when new shared libraries
are loaded.

I'm not confident that this approach can be made to work, though
it looked promising for a while.


> Could we instead tackle it GDB-side, with the same idea as what
> I proposed the native debugging?  We could suppress the qsymbol packets
> sent in between each objfile loaded from the initial library scan.  We
> would have a single qsymbol sent after finishing loading all the initial
> libraries.

Sure, I'm willing to work on this, or at least help out...

> For example, I just hacked this, this makes it work for the simple
> non-stop attach case with gdbserver, where you run gdbserver with
> --attach and then connect with GDB (I haven't tried the other modes).
> The final qsymbol, that lets GDBserver load libthread_db, is sent in
> remote_target::start_remote.

I tested your patch today (against the test suite) using both
--target_board=native-extended-gdbserver/-m64 and
--target_board=native-gdbserver.  Unfortunately, I didn't see
a test which was helped by your patch.  (Though there was the
usual noise from racy tests.)

For the test that I've been focused on, I see the following results
on a glibc 2.32 machine:

[kev@f33-1 gdb]$ make check RUNTESTFLAGS="--target_board=native-extended-gdbserver/-m64" TESTS=gdb.threads/fork-plus-threads.exp
...
FAIL: gdb.threads/fork-plus-threads.exp: detach-on-fork=off: only inferior 1 left

		=== gdb Summary ===

# of expected passes		15
# of unexpected failures	1

With or without your patch, I see this result on a glibc 2.33 machine:

[kev@f33-3 gdb]$ make check RUNTESTFLAGS="--target_board=native-extended-gdbserver/-m64" TESTS=gdb.threads/fork-plus-threads.exp
...
FAIL: gdb.threads/fork-plus-threads.exp: detach-on-fork=off: inferior 1 exited
FAIL: gdb.threads/fork-plus-threads.exp: detach-on-fork=off: no threads left
FAIL: gdb.threads/fork-plus-threads.exp: detach-on-fork=off: only inferior 1 left
WARNING: Timed out waiting for EOF in server after monitor exit

		=== gdb Summary ===

# of expected passes		13
# of unexpected failures	3

(It also leaves a gdbserver running which must be killed with -KILL.)

The other thing that I should mention is that Florian Weimer has told
me that (he thinks) glibc 2.33 can be changed so that no GDB changes
are needed.  So, if we can't figure it out, we might want to ask
Florian to do this.

Florian has also said that things will change yet again in glibc 2.34. 
It's not clear to me that the fix we're investigating for glibc 2.33
will still be required.  Florian told me:  "So ideally we'd figure out
a way to load libthread_db even if there is no libpthread in the
inferior."  This sounds like a different problem than the one we're
currently investigating.

Kevin