Re: [PATCH] 0001-Fix-multi-thread-debug-bug-in-AIX.patch

[Aditya Kamath1 <Aditya.Kamath1@ibm.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4241 bytes --]

Hi Ulrich and community,

Please find attached the patch {See: 0001-Fix-multi-thread-debug-bug-in-AIX.patch}


So, I took into account the information you gave me the last time. Having said that, it turned out to be a different issue that is causing such a swap of threads as shown in the outputs of the previous email {pasted again below this email}.

Our top target is getting set correctly and symbols are being read in the right way on any new-born inferior. I am thankful to you all for pointing that shared Library issue to me.

While setting the top target for a new inferior we go to sync_threadlist () from the pd_enable () -> pthdb_session_pthreaded () -> push the top target -> pd_activate () -> pthdb_session_init ()-> pd_update () -> sync_threadlists (), our sync_threadlists uses a variable called gcount which represents the number of threads in the GDB. pcount is the number of pthreads. gcount is got from iterate_over_threads ().

Honestly from my understanding we do not take into account the process ID while we iterate over threads. So, this will work in a single inferior case whereas in multi-inferior one with multi thread things mess up. Here is how: So, if we have two threads in our first inferior
{main thread and a task thread say}, and one main thread in the second inferior and we are syncing our second inferior, though our pcount will correctly be one but our gcount will 3. Which means by our current logic we will end up deleting two threads {first two of the first inferior}. This is exactly what happened in my precious mail output if you check and post this, we will swap the threads as this is how the code flow.

This is now one of the causes of the bug.

Inorder to resolve the same I request for one information. How can we iterate_over_threads of a particular process. What is that function. Is there any built-in available?? Kindly let me know and that should solve this issue.

Also kindly give me feedback on this patch if I need to change anything. I thank you for the guidance so far.

Hoping for a reply soon.

Have a nice day ahead.

Thanks and regards,
Aditya.

--------------------------------------------
Output 2:-

Reading symbols from /home/aditya/gdb_tests/ultimate-multi-thread-fork...

(gdb) set detach-on-fork off

(gdb) r

Starting program: /home/aditya/gdb_tests/ultimate-multi-thread-fork

[New Thread 258]

[New inferior 2 (process 16122342)]

I am parent

^C[New process 11665700]


Thread 1.3 received signal SIGINT, Interrupt.

0xd0595fb0 in _p_nsleep () from /usr/lib/libpthread.a(shr_xpg5.o)

(gdb) info threads

  Id   Target Id                          Frame

* 1.3  process 11665700                   0xd0595fb0 in _p_nsleep ()

   from /usr/lib/libpthread.a(shr_xpg5.o)

  2.1  Thread 258 (tid 28115287, running) 0xd0594fc8 in _sigsetmask ()

   from /usr/lib/libpthread.a(shr_xpg5.o)

(gdb)

________________________________
From: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Sent: 11 January 2023 19:01
To: simark@simark.ca <simark@simark.ca>; Aditya Kamath1 <Aditya.Kamath1@ibm.com>; gdb-patches@sourceware.org <gdb-patches@sourceware.org>
Cc: Sangamesh Mallayya <sangamesh.swamy@in.ibm.com>
Subject: Re: [PATCH] 0001-Fix-multi-thread-debug-bug-in-AIX.patch

Aditya Kamath1 <Aditya.Kamath1@ibm.com> wrote:

>So, looking at this, I have missed out something may be minor or
>major causing the bug which I am unaware of in the code base.

The one problem I see is that there are still global variables
that seem to get clobbered if there are multiple inferiors,
in particular pd_session and pd_brk_addr.  Those will get
overwritten by each pd_activate call for each inferior.

Note that pthdb_session_init registers the pid, so the resulting
pd_session should be different for each inferior.  If you just
overwrite it, then the pid values passed to all callbacks will
always reflect only the latest inferior.

Similarly, pd_brk_addr is potentially different in each inferior,
if libpthread is loaded at different addresses.

You should create a struct containing all per-inferior thread-
related data members, and then allocate one such struct per
inferior.

Bye,
Ulrich


[-- Attachment #2: 0001-Fix-multi-thread-debug-bug-in-AIX.patch --]
[-- Type: application/octet-stream, Size: 15474 bytes --]

From 7dae4fa7e37323b355308edb8b13c118b6a2ae8a Mon Sep 17 00:00:00 2001
From: Aditya Vidyadhar Kamath <Aditya.Kamath1@ibm.com>
Date: Fri, 13 Jan 2023 06:43:14 -0600
Subject: [PATCH] Fix multi thread debug bug in AIX

---
 gdb/aix-thread.c | 148 ++++++++++++++++++++++++++---------------------
 gdb/solib-aix.c  |   6 ++
 2 files changed, 89 insertions(+), 65 deletions(-)

diff --git a/gdb/aix-thread.c b/gdb/aix-thread.c
index e556c153576..5894f3ecc7d 100644
--- a/gdb/aix-thread.c
+++ b/gdb/aix-thread.c
@@ -55,6 +55,8 @@
 #include <sys/reg.h>
 #include <sched.h>
 #include <sys/pthdebug.h>
+#include <vector>
+#include <map>
 
 #if !HAVE_DECL_GETTHRDS
 extern int getthrds (pid_t, struct thrdsinfo64 *, int, tid_t *, int);
@@ -70,7 +72,7 @@ static bool debug_aix_thread;
 
 /* Return whether to treat PID as a debuggable thread id.  */
 
-#define PD_TID(ptid)	(pd_active && ptid.tid () != 0)
+#define PD_TID(ptid)	(std::find (pd_active.begin (), pd_active.end (), ptid.pid ()) != pd_active.end () && ptid.tid () != 0)
 
 /* Success and failure values returned by pthdb callbacks.  */
 
@@ -142,20 +144,20 @@ class aix_thread_target final : public target_ops
   ptid_t get_ada_task_ptid (long lwp, ULONGEST thread) override;
 };
 
-static aix_thread_target aix_thread_ops;
+  static aix_thread_target aix_thread_ops;
 
 /* Address of the function that libpthread will call when libpthdebug
    is ready to be initialized.  */
 
-static CORE_ADDR pd_brk_addr;
+ static std::map<pid_t, CORE_ADDR> pd_brk_addr;
 
 /* Whether the current application is debuggable by pthdb.  */
 
-static int pd_able = 0;
+static std::vector<pid_t> pd_able;
 
 /* Whether a threaded application is being debugged.  */
 
-static int pd_active = 0;
+static std::vector<pid_t> pd_active;
 
 /* Whether the current architecture is 64-bit.  
    Only valid when pd_able is true.  */
@@ -193,7 +195,7 @@ static pthdb_callbacks_t pd_callbacks = {
 
 /* Current pthdb session.  */
 
-static pthdb_session_t pd_session;
+static std::map<pid_t, pthdb_session_t> pd_session;
 
 /* Return a printable representation of pthdebug function return
    STATUS.  */
@@ -331,6 +333,9 @@ pdc_symbol_addrs (pthdb_user_t user_current_pid, pthdb_symbol_t *symbols, int co
   struct bound_minimal_symbol ms;
   int i;
   char *name;
+  scoped_restore_current_program_space restore_pspace; 
+  inferior *inf = find_inferior_pid (current_inferior ()->process_target (), user_current_pid);
+  set_current_program_space (inf->pspace);
 
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog,
@@ -508,14 +513,13 @@ pdc_read_data (pthdb_user_t user_current_pid, void *buf,
   /* This is needed to eliminate the dependency of current thread
      which is null so that thread reads the correct target memory.  */
   {
-    scoped_restore_current_thread restore_current_thread;
+    scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid);
     /* Before the first inferior is added, we pass inferior_ptid.pid ()
        from pd_enable () which is 0.  There is no need to switch threads
        during first initialisation.  In the rest of the callbacks the
        current thread needs to be correct.  */
     if (user_current_pid != 0)
-      switch_to_thread (current_inferior ()->process_target (),
-			ptid_t (user_current_pid));
+      inferior_ptid = ptid_t (user_current_pid);
     status = target_read_memory (addr, (gdb_byte *) buf, len);
   }
   ret = status == 0 ? PDC_SUCCESS : PDC_FAILURE;
@@ -639,36 +643,23 @@ pcmp (const void *p1v, const void *p2v)
   return p1->pthid < p2->pthid ? -1 : p1->pthid > p2->pthid;
 }
 
-/* iterate_over_threads() callback for counting GDB threads.
-
-   Do not count the main thread (whose tid is zero).  This matches
-   the list of threads provided by the pthreaddebug library, which
-   does not include that main thread either, and thus allows us
-   to compare the two lists.  */
+/* iterate_over_threads() callback for counting GDB threads.  */
 
 static int
 giter_count (struct thread_info *thread, void *countp)
 {
-  if (PD_TID (thread->ptid))
-    (*(int *) countp)++;
+  (*(int *) countp)++;
   return 0;
 }
 
-/* iterate_over_threads() callback for accumulating GDB thread pids.
-
-   Do not include the main thread (whose tid is zero).  This matches
-   the list of threads provided by the pthreaddebug library, which
-   does not include that main thread either, and thus allows us
-   to compare the two lists.  */
+/* iterate_over_threads() callback for accumulating GDB thread pids.  */
 
 static int
 giter_accum (struct thread_info *thread, void *bufp)
 {
-  if (PD_TID (thread->ptid))
-    {
-      **(struct thread_info ***) bufp = thread;
-      (*(struct thread_info ***) bufp)++;
-    }
+  **(struct thread_info ***) bufp = thread;
+  (*(struct thread_info ***) bufp)++;
+
   return 0;
 }
 
@@ -719,7 +710,7 @@ get_signaled_thread (int pid)
 		    sizeof (thrinf), &ktid, 1) != 1)
 	break;
 
-      if (thrinf.ti_cursig == SIGTRAP)
+      if (thrinf.ti_cursig)
 	return thrinf.ti_tid;
     }
 
@@ -750,6 +741,9 @@ sync_threadlists (int pid)
   pthdb_pthread_t pdtid;
   pthread_t pthid;
   pthdb_tid_t tid;
+  process_stratum_target *proc_target
+            = current_inferior ()->process_target ();
+  thread_info  *tp;
 
   /* Accumulate an array of libpthdebug threads sorted by pthread id.  */
 
@@ -759,11 +753,11 @@ sync_threadlists (int pid)
 
   for (cmd = PTHDB_LIST_FIRST;; cmd = PTHDB_LIST_NEXT)
     {
-      status = pthdb_pthread (pd_session, &pdtid, cmd);
+      status = pthdb_pthread (pd_session[pid], &pdtid, cmd);
       if (status != PTHDB_SUCCESS || pdtid == PTHDB_INVALID_PTHREAD)
 	break;
 
-      status = pthdb_pthread_ptid (pd_session, pdtid, &pthid);
+      status = pthdb_pthread_ptid (pd_session[pid], pdtid, &pthid);
       if (status != PTHDB_SUCCESS || pthid == PTHDB_INVALID_PTID)
 	continue;
 
@@ -780,7 +774,7 @@ sync_threadlists (int pid)
 
   for (pi = 0; pi < pcount; pi++)
     {
-      status = pthdb_pthread_tid (pd_session, pbuf[pi].pdtid, &tid);
+      status = pthdb_pthread_tid (pd_session[pid], pbuf[pi].pdtid, &tid);
       if (status != PTHDB_SUCCESS)
 	tid = PTHDB_INVALID_TID;
       pbuf[pi].tid = tid;
@@ -810,8 +804,6 @@ sync_threadlists (int pid)
 	  priv->pdtid = pbuf[pi].pdtid;
 	  priv->tid = pbuf[pi].tid;
 
-	  process_stratum_target *proc_target
-	    = current_inferior ()->process_target ();
 	  thread = add_thread_with_info (proc_target,
 					 ptid_t (pid, 0, pbuf[pi].pthid),
 					 priv);
@@ -841,8 +833,22 @@ sync_threadlists (int pid)
 	    }
 	  else if (cmp_result > 0)
 	    {
-	      delete_thread (gbuf[gi]);
-	      gi++;
+              if (gptid.is_pid () && gptid.pid () == pptid.pid ())
+                {
+                  thread_change_ptid (proc_target, gptid, pptid);
+                  aix_thread_info *priv = new aix_thread_info;
+                  priv->pdtid = pbuf[pi].pdtid;
+                  priv->tid = pbuf[pi].tid;
+                  tp = find_thread_ptid (proc_target, pptid);
+                  tp->priv.reset (priv);
+                  pi++;
+                  gi++;
+                }
+              else
+                {
+                  delete_thread (gbuf[gi]);
+                  gi++;
+                }
 	    }
 	  else
 	    {
@@ -888,10 +894,11 @@ pd_update (int pid)
   pthdb_tid_t tid;
   struct thread_info *thread = NULL;
 
-  if (!pd_active)
+  if (std::find (pd_active.begin (), pd_active.end (), pid)
+    == pd_active.end ())
     return ptid_t (pid);
 
-  status = pthdb_session_update (pd_session);
+  status = pthdb_session_update (pd_session[pid]);
   if (status != PTHDB_SUCCESS)
     return ptid_t (pid);
 
@@ -918,15 +925,17 @@ static ptid_t
 pd_activate (int pid)
 {
   int status;
-		
+	
   status = pthdb_session_init (pid, arch64 ? PEM_64BIT : PEM_32BIT,
 			       PTHDB_FLAG_REGS, &pd_callbacks, 
-			       &pd_session);
+			       &pd_session[pid]);
   if (status != PTHDB_SUCCESS)
     {
+      pd_session.erase (pid);
       return ptid_t (pid);
     }
-  pd_active = 1;
+  printf ("pd_activated successfully for pid %d\n", pid);
+  pd_active.push_back (pid);
   return pd_update (pid);
 }
 
@@ -935,26 +944,29 @@ pd_activate (int pid)
 static void
 pd_deactivate (void)
 {
-  if (!pd_active)
+  if (std::find (pd_active.begin (), pd_active.end (), inferior_ptid.pid ())
+    == pd_active.end ())
     return;
-  pthdb_session_destroy (pd_session);
+  pthdb_session_destroy (pd_session.erase (inferior_ptid.pid ()));
   
   pid_to_prc (&inferior_ptid);
-  pd_active = 0;
+  pd_active.erase (std::find (pd_active.begin (), pd_active.end (), inferior_ptid.pid ()));
 }
 
 /* An object file has just been loaded.  Check whether the current
    application is pthreaded, and if so, prepare for thread debugging.  */
 
 static void
-pd_enable (void)
+pd_enable (inferior *inf)
 {
   int status;
   char *stub_name;
   struct bound_minimal_symbol ms;
+  pid_t pid = (inf == NULL?inferior_ptid.pid ():inf->pid);
 
   /* Don't initialize twice.  */
-  if (pd_able)
+  if (std::find (pd_able.begin (), pd_able.end (), pid) 
+	!= pd_able.end ())
     return;
 
   /* Check application word size.  */
@@ -962,7 +974,7 @@ pd_enable (void)
 
   /* Check whether the application is pthreaded.  */
   stub_name = NULL;
-  status = pthdb_session_pthreaded (inferior_ptid.pid (), PTHDB_FLAG_REGS,
+  status = pthdb_session_pthreaded (pid, PTHDB_FLAG_REGS,
 				    &pd_callbacks, &stub_name);
   if ((status != PTHDB_SUCCESS
        && status != PTHDB_NOT_PTHREADED) || !stub_name)
@@ -972,18 +984,20 @@ pd_enable (void)
   ms = lookup_minimal_symbol (stub_name, NULL, NULL);
   if (ms.minsym == NULL)
     return;
-  pd_brk_addr = ms.value_address ();
-  if (!create_thread_event_breakpoint (target_gdbarch (), pd_brk_addr))
+  pd_brk_addr.insert ({pid, ms.value_address ()});
+
+  if (!create_thread_event_breakpoint (target_gdbarch (), pd_brk_addr[pid]))
     return;
 
   /* Prepare for thread debugging.  */
   current_inferior ()->push_target (&aix_thread_ops);
-  pd_able = 1;
+
+  pd_able.push_back (pid);
 
   /* If we're debugging a core file or an attached inferior, the
      pthread library may already have been initialized, so try to
      activate thread debugging.  */
-  pd_activate (inferior_ptid.pid ());
+  pd_activate (pid);
 }
 
 /* Undo the effects of pd_enable().  */
@@ -991,11 +1005,14 @@ pd_enable (void)
 static void
 pd_disable (void)
 {
-  if (!pd_able)
+  if (std::find (pd_able.begin (), pd_able.end (), 
+	inferior_ptid.pid ()) == pd_able.end ())
     return;
-  if (pd_active)
+  if (std::find (pd_active.begin (), pd_active.end (),
+        inferior_ptid.pid ()) != pd_active.end ())
     pd_deactivate ();
-  pd_able = 0;
+  pd_able.erase (std::find (pd_able.begin (), pd_able.end (), 
+	inferior_ptid.pid ()));
   current_inferior ()->unpush_target (&aix_thread_ops);
 }
 
@@ -1010,7 +1027,7 @@ static void
 new_objfile (struct objfile *objfile)
 {
   if (objfile)
-    pd_enable ();
+    pd_enable (NULL);
   else
     pd_disable ();
 }
@@ -1020,7 +1037,7 @@ new_objfile (struct objfile *objfile)
 static void
 aix_thread_inferior_created (inferior *inf)
 {
-  pd_enable ();
+  pd_enable (inf);
 }
 
 /* Detach from the process attached to by aix_thread_attach().  */
@@ -1096,7 +1113,8 @@ aix_thread_target::wait (ptid_t ptid, struct target_waitstatus *status,
   gdb_assert (ptid.is_pid ());
 
   /* Check whether libpthdebug might be ready to be initialized.  */
-  if (!pd_active && status->kind () == TARGET_WAITKIND_STOPPED
+  if (std::find (pd_active.begin (), pd_active.end (), ptid.pid ()) == pd_active.end ()
+      && status->kind () == TARGET_WAITKIND_STOPPED
       && status->sig () == GDB_SIGNAL_TRAP)
     {
       process_stratum_target *proc_target
@@ -1105,7 +1123,7 @@ aix_thread_target::wait (ptid_t ptid, struct target_waitstatus *status,
       struct gdbarch *gdbarch = regcache->arch ();
 
       if (regcache_read_pc (regcache)
-	  - gdbarch_decr_pc_after_break (gdbarch) == pd_brk_addr)
+	  - gdbarch_decr_pc_after_break (gdbarch) == pd_brk_addr[ptid.pid ()])
 	return pd_activate (ptid.pid ());
     }
 
@@ -1233,7 +1251,7 @@ fetch_regs_user_thread (struct regcache *regcache, pthdb_pthread_t pdtid)
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog, 
 		"fetch_regs_user_thread %lx\n", (long) pdtid);
-  status = pthdb_pthread_context (pd_session, pdtid, &ctx);
+  status = pthdb_pthread_context (pd_session [inferior_ptid.pid ()], pdtid, &ctx);
   if (status != PTHDB_SUCCESS)
     error (_("aix-thread: fetch_registers: pthdb_pthread_context returned %s"),
 	   pd_status2str (status));
@@ -1518,7 +1536,7 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
 
   /* Retrieve the thread's current context for its non-register
      values.  */
-  status = pthdb_pthread_context (pd_session, pdtid, &ctx);
+  status = pthdb_pthread_context (pd_session [inferior_ptid.pid ()], pdtid, &ctx);
   if (status != PTHDB_SUCCESS)
     error (_("aix-thread: store_registers: pthdb_pthread_context returned %s"),
 	   pd_status2str (status));
@@ -1576,7 +1594,7 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
 	ctx.fpscr = tmp_fpscr;
     }
 
-  status = pthdb_pthread_setcontext (pd_session, pdtid, &ctx);
+  status = pthdb_pthread_setcontext (pd_session[inferior_ptid.pid ()], pdtid, &ctx);
   if (status != PTHDB_SUCCESS)
     error (_("aix-thread: store_registers: "
 	     "pthdb_pthread_setcontext returned %s"),
@@ -1800,24 +1818,24 @@ aix_thread_target::extra_thread_info (struct thread_info *thread)
     /* i18n: Like "thread-identifier %d, [state] running, suspended" */
     buf.printf (_("tid %d"), (int)tid);
 
-  status = pthdb_pthread_state (pd_session, pdtid, &state);
+  status = pthdb_pthread_state (pd_session[inferior_ptid.pid ()], pdtid, &state);
   if (status != PTHDB_SUCCESS)
     state = PST_NOTSUP;
   buf.printf (", %s", state2str (state));
 
-  status = pthdb_pthread_suspendstate (pd_session, pdtid, 
+  status = pthdb_pthread_suspendstate (pd_session[inferior_ptid.pid ()], pdtid, 
 				       &suspendstate);
   if (status == PTHDB_SUCCESS && suspendstate == PSS_SUSPENDED)
     /* i18n: Like "Thread-Id %d, [state] running, suspended" */
     buf.printf (_(", suspended"));
 
-  status = pthdb_pthread_detachstate (pd_session, pdtid, 
+  status = pthdb_pthread_detachstate (pd_session[inferior_ptid.pid ()], pdtid, 
 				      &detachstate);
   if (status == PTHDB_SUCCESS && detachstate == PDS_DETACHED)
     /* i18n: Like "Thread-Id %d, [state] running, detached" */
     buf.printf (_(", detached"));
 
-  pthdb_pthread_cancelpend (pd_session, pdtid, &cancelpend);
+  pthdb_pthread_cancelpend (pd_session[inferior_ptid.pid ()], pdtid, &cancelpend);
   if (status == PTHDB_SUCCESS && cancelpend)
     /* i18n: Like "Thread-Id %d, [state] running, cancel pending" */
     buf.printf (_(", cancel pending"));
diff --git a/gdb/solib-aix.c b/gdb/solib-aix.c
index f483f54de13..181140b3345 100644
--- a/gdb/solib-aix.c
+++ b/gdb/solib-aix.c
@@ -618,6 +618,12 @@ solib_aix_bfd_open (const char *pathname)
       if (member_name == bfd_get_filename (object_bfd.get ()))
 	break;
 
+      std::string s = bfd_get_filename (object_bfd.get ());
+      if (s.find (member_name) != std::string::npos)
+      {
+	return object_bfd;
+      }
+
       object_bfd = gdb_bfd_openr_next_archived_file (archive_bfd.get (),
 						     object_bfd.get ());
     }
-- 
2.31.1