[RFC][gdbsupport] Add PARALLEL_FOR_EACH_DEBUG

[RFC][gdbsupport] Add PARALLEL_FOR_EACH_DEBUG

[Tom de Vries]

Hi,

Add a PARALLEL_FOR_EACH_DEBUG macro, set to 0 by default.

With an a.out compiled from hello world, we get with
PARALLEL_FOR_EACH_DEBUG == 1:
...
$ gdb -q -batch a.out -ex start
  ...
Parallel for: n_elements: 7271
Parallel for: minimum elements per thread: 10
Parallel for: elts_per_thread: 1817
Parallel for: worker threads: 4
Parallel for: worker threads used: 3
Parallel for: elements 0-1816 on thread 0: 1817
Parallel for: elements 1817-3633 on thread 1: 1817
Parallel for: elements 3634-5450 on thread 2: 1817
Parallel for: elements 5451-7270 on main thread: 1820

Temporary breakpoint 1, main () at /home/vries/hello.c:6
6         printf ("hello\n");
...

Note that thread usage isn't ideal.  We end up using only three out of four
worker threads here.

Any comments?

Thanks,
- Tom

[gdbsupport] Add PARALLEL_FOR_EACH_DEBUG

---
 gdbsupport/parallel-for.h | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/gdbsupport/parallel-for.h b/gdbsupport/parallel-for.h
index a614fc35766..358ed9e8f7f 100644
--- a/gdbsupport/parallel-for.h
+++ b/gdbsupport/parallel-for.h
@@ -113,6 +113,8 @@ struct par_for_accumulator<void>
 
 }
 
+#define PARALLEL_FOR_EACH_DEBUG 0
+
 /* A very simple "parallel for".  This splits the range of iterators
    into subranges, and then passes each subrange to the callback.  The
    work may or may not be done in separate threads.
@@ -139,6 +141,10 @@ parallel_for_each (unsigned n, RandomIt first, RandomIt last,
   using result_type
     = typename std::result_of<RangeFunction (RandomIt, RandomIt)>::type;
 
+#if PARALLEL_FOR_EACH_DEBUG
+  RandomIt orig_first = first;
+#endif
+
   size_t n_threads = thread_pool::g_thread_pool->thread_count ();
   size_t n_elements = last - first;
   size_t elts_per_thread = 0;
@@ -155,9 +161,25 @@ parallel_for_each (unsigned n, RandomIt first, RandomIt last,
   size_t count = n_threads == 0 ? 0 : n_threads - 1;
   gdb::detail::par_for_accumulator<result_type> results (count);
 
+#if PARALLEL_FOR_EACH_DEBUG
+  fprintf (stderr, "Parallel for: n_elements: %zu\n", n_elements);
+  fprintf (stderr, "Parallel for: minimum elements per thread: %u\n", n);
+  fprintf (stderr, "Parallel for: elts_per_thread: %zu\n", elts_per_thread);
+  fprintf (stderr, "Parallel for: worker threads: %zu\n",
+	   thread_pool::g_thread_pool->thread_count ());
+  fprintf (stderr, "Parallel for: worker threads used: %zu\n", count);
+#endif
+
   for (int i = 0; i < count; ++i)
     {
       RandomIt end = first + elts_per_thread;
+#if PARALLEL_FOR_EACH_DEBUG
+      fprintf (stderr, "Parallel for: elements %u-%u on thread %d: %u\n",
+	       (unsigned)(first - orig_first),
+	       (unsigned)(end - 1 - orig_first),
+	       i,
+	       (unsigned)(end - first));
+#endif
       results.post (i, [=] ()
         {
 	  return callback (first, end);
@@ -166,6 +188,12 @@ parallel_for_each (unsigned n, RandomIt first, RandomIt last,
     }
 
   /* Process all the remaining elements in the main thread.  */
+#if PARALLEL_FOR_EACH_DEBUG
+      fprintf (stderr, "Parallel for: elements %u-%u on main thread: %u\n",
+	       (unsigned)(first - orig_first),
+	       (unsigned)(last - 1 - orig_first),
+	       (unsigned)(last - first));
+#endif
   return results.finish ([=] ()
     {
       return callback (first, last);

Re: [RFC][gdbsupport] Add PARALLEL_FOR_EACH_DEBUG

[Tom Tromey]

>>>>> "Tom" == Tom de Vries via Gdb-patches <gdb-patches@sourceware.org> writes:

Tom> +#define PARALLEL_FOR_EACH_DEBUG 0

Probably needs a short comment.

Tom> +#if PARALLEL_FOR_EACH_DEBUG
Tom> +  fprintf (stderr, "Parallel for: n_elements: %zu\n", n_elements);
Tom> +  fprintf (stderr, "Parallel for: minimum elements per thread: %u\n", n);

I'm meh on this kind of logging but if it helped you it seems fine.

Tom

Re: [RFC][gdbsupport] Add PARALLEL_FOR_EACH_DEBUG

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

On 7/15/22 21:01, Tom Tromey wrote:
>>>>>> "Tom" == Tom de Vries via Gdb-patches <gdb-patches@sourceware.org> writes:
> 
> Tom> +#define PARALLEL_FOR_EACH_DEBUG 0
> 
> Probably needs a short comment.
> 

Done.

Also changed this into a const variable.

> Tom> +#if PARALLEL_FOR_EACH_DEBUG
> Tom> +  fprintf (stderr, "Parallel for: n_elements: %zu\n", n_elements);
> Tom> +  fprintf (stderr, "Parallel for: minimum elements per thread: %u\n", n);
> 
> I'm meh on this kind of logging but if it helped you it seems fine.
> 

I've reworked the output to the slightly more clear:
...
     Parallel for: n_elements: 7271
     Parallel for: minimum elements per thread: 10
     Parallel for: elts_per_thread: 1817
     Parallel for: elements on worker thread 0       : 1817
     Parallel for: elements on worker thread 1       : 1817
     Parallel for: elements on worker thread 2       : 1817
     Parallel for: elements on worker thread 3       : 0
     Parallel for: elements on main thread           : 1820
...

It was a surprise for me to find that worker thread 3 is idle, so I 
think the information is useful enough.

Committed as attached.

Thanks,
- Tom

[-- Attachment #2: 0001-gdbsupport-Add-parallel_for_each_debug.patch --]
[-- Type: text/x-patch, Size: 2972 bytes --]

[gdbsupport] Add parallel_for_each_debug

Add a parallel_for_each_debug variable, set to false by default.

With an a.out compiled from hello world, we get with
parallel_for_each_debug == true:
...
$ gdb -q -batch a.out -ex start
  ...
Parallel for: n_elements: 7271
Parallel for: minimum elements per thread: 10
Parallel for: elts_per_thread: 1817
Parallel for: elements on worker thread 0       : 1817
Parallel for: elements on worker thread 1       : 1817
Parallel for: elements on worker thread 2       : 1817
Parallel for: elements on worker thread 3       : 0
Parallel for: elements on main thread           : 1820

Temporary breakpoint 1, main () at /home/vries/hello.c:6
6         printf ("hello\n");
...

Tested on x86_64-linux.

---
 gdbsupport/parallel-for.h | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/gdbsupport/parallel-for.h b/gdbsupport/parallel-for.h
index a614fc35766..cfe8a6e4f09 100644
--- a/gdbsupport/parallel-for.h
+++ b/gdbsupport/parallel-for.h
@@ -139,7 +139,12 @@ parallel_for_each (unsigned n, RandomIt first, RandomIt last,
   using result_type
     = typename std::result_of<RangeFunction (RandomIt, RandomIt)>::type;
 
-  size_t n_threads = thread_pool::g_thread_pool->thread_count ();
+  /* If enabled, print debug info about how the work is distributed across
+     the threads.  */
+  const int parallel_for_each_debug = false;
+
+  size_t n_worker_threads = thread_pool::g_thread_pool->thread_count ();
+  size_t n_threads = n_worker_threads;
   size_t n_elements = last - first;
   size_t elts_per_thread = 0;
   if (n_threads > 1)
@@ -155,9 +160,19 @@ parallel_for_each (unsigned n, RandomIt first, RandomIt last,
   size_t count = n_threads == 0 ? 0 : n_threads - 1;
   gdb::detail::par_for_accumulator<result_type> results (count);
 
+  if (parallel_for_each_debug)
+    {
+      debug_printf (_("Parallel for: n_elements: %zu\n"), n_elements);
+      debug_printf (_("Parallel for: minimum elements per thread: %u\n"), n);
+      debug_printf (_("Parallel for: elts_per_thread: %zu\n"), elts_per_thread);
+    }
+
   for (int i = 0; i < count; ++i)
     {
       RandomIt end = first + elts_per_thread;
+      if (parallel_for_each_debug)
+	debug_printf (_("Parallel for: elements on worker thread %i\t: %zu\n"),
+		      i, (size_t)(end - first));
       results.post (i, [=] ()
         {
 	  return callback (first, end);
@@ -165,7 +180,14 @@ parallel_for_each (unsigned n, RandomIt first, RandomIt last,
       first = end;
     }
 
+  for (int i = count; i < n_worker_threads; ++i)
+    if (parallel_for_each_debug)
+      debug_printf (_("Parallel for: elements on worker thread %i\t: 0\n"), i);
+
   /* Process all the remaining elements in the main thread.  */
+  if (parallel_for_each_debug)
+    debug_printf (_("Parallel for: elements on main thread\t\t: %zu\n"),
+		  (size_t)(last - first));
   return results.finish ([=] ()
     {
       return callback (first, last);