[gomp4.1] fix dependency scheduling problem

[gomp4.1] fix dependency scheduling problem

[Aldy Hernandez]

[-- Attachment #1: Type: text/plain, Size: 2246 bytes --]

Hi.

As I may have mentioned earlier, I'm implementing OpenMP 4.1's task 
priorities for GCC's libgomp.  This has had me looking at how we 
currently implement scheduling, and I believe we have a problem in how 
we rearrange dependencies.

Currently in gomp_task_maybe_wait_for_dependencies(), we have the 
following code for rearranging the first dependency (tsk) from a list of 
dependencies.

		else if (tsk != task->children)
                       {
                         /* Remove tsk from the sibling list...  */
                         tsk->prev_child->next_child = tsk->next_child;
                         tsk->next_child->prev_child = tsk->prev_child;
                         /* ...and insert it into the running task's
                            children.  */
-->BAD                  tsk->prev_child = task->children;
-->BAD                  tsk->next_child = task->children->next_child;
                         task->children = tsk;
                         tsk->prev_child->next_child = tsk;
                         tsk->next_child->prev_child = tsk;
                       }

If say, you have a parent_depends_on task PD1 that is in the following 
children queue:

	C1 -> C2 -> C3 -> PD1 -> C4

(Where parent->children points to C1 and C4 wraps around to C1 as per 
the circular list.)

The above code will transform the children queue into:

	PD1 -> C2 -> C3 -> C4 -> C1

The surrounding code looks sane, but this particular snippet quoted 
above has us moving the first child to the end of the queue, which is 
probably not what we want.  However, at least we're still keeping the 
parent_depends_on tasks first, just that we moved other previously 
unaffected children to the end of the queue.

What we really want is:

	PD1 -> C1 -> C2 -> C3 -> C4

The attached patch does just that.  I have also rewritten the comments 
now that I actually understand what's going on :).  Eventually this will 
all become clearer with my upcoming/proposed API for dealing with all 
these queues.

As discussed on IRC, there is another issue you pointed out in 
gomp_task_run_pre(), which I will address in a followup patch.

Tested on x86-64 Linux by running "make check-target-libgomp" with 
OMP_NUM_THREADS={1,2,55}.

OK for branch?

[-- Attachment #2: curr --]
[-- Type: text/plain, Size: 2708 bytes --]

commit 6d8f6db0583326d803c7c7abd8ea26cc842643fc
Author: Aldy Hernandez <aldyh@redhat.com>
Date:   Fri Oct 2 15:40:30 2015 -0700

    	* task.c (gomp_task_maybe_wait_for_dependencies): Fix scheduling
    	problem such that the first non parent_depends_on task does not
    	end up at the end of the children queue.

diff --git a/libgomp/task.c b/libgomp/task.c
index f6a67eb..5c412fc 100644
--- a/libgomp/task.c
+++ b/libgomp/task.c
@@ -1140,17 +1140,37 @@ gomp_task_maybe_wait_for_dependencies (void **depend)
 	      {
 		tsk->parent_depends_on = true;
 		++num_awaited;
+		/* If a task we need to wait for is not already
+		   running and is ready to be scheduled, move it to
+		   front, so that we run it as soon as possible.
+
+		   We rearrange the children queue such that all
+		   parent_depends_on tasks are first, and
+		   last_parent_depends_on points to the last such task
+		   we rearranged.  For example, given the following
+		   children where PD[123] are the parent_depends_on
+		   tasks:
+
+		   	task->children
+			|
+			V
+			C1 -> C2 -> C3 -> PD1 -> PD2 -> PD3 -> C4
+
+		   We rearrange such that:
+
+			task->children
+			|	       +--- last_parent_depends_on
+			|	       |
+			V	       V
+			PD1 -> PD2 -> PD3 -> C1 -> C2 -> C3 -> C4
+		*/
+
 		if (tsk->num_dependees == 0 && tsk->kind == GOMP_TASK_WAITING)
 		  {
-		    /* If a task we need to wait for is not already
-		       running and is ready to be scheduled, move it
-		       to front, so that we run it as soon as possible.  */
 		    if (last_parent_depends_on)
 		      {
-			/* Remove tsk from the sibling list...  */
 			tsk->prev_child->next_child = tsk->next_child;
 			tsk->next_child->prev_child = tsk->prev_child;
-			/* ...and insert it into last_parent_depends_on.  */
 			tsk->prev_child = last_parent_depends_on;
 			tsk->next_child = last_parent_depends_on->next_child;
 			tsk->prev_child->next_child = tsk;
@@ -1158,21 +1178,14 @@ gomp_task_maybe_wait_for_dependencies (void **depend)
 		      }
 		    else if (tsk != task->children)
 		      {
-			/* Remove tsk from the sibling list...  */
 			tsk->prev_child->next_child = tsk->next_child;
 			tsk->next_child->prev_child = tsk->prev_child;
-			/* ...and insert it into the running task's
-			   children.  */
-			tsk->prev_child = task->children;
-			tsk->next_child = task->children->next_child;
+			tsk->prev_child = task->children->prev_child;
+			tsk->next_child = task->children;
 			task->children = tsk;
 			tsk->prev_child->next_child = tsk;
 			tsk->next_child->prev_child = tsk;
 		      }
-		    else
-		      {
-			/* It's already in task->children.  Nothing to do.  */;
-		      }
 		    last_parent_depends_on = tsk;
 		  }
 	      }

Re: [gomp4.1] fix dependency scheduling problem

[Jakub Jelinek]

On Fri, Oct 02, 2015 at 03:46:59PM -0700, Aldy Hernandez wrote:
> commit 6d8f6db0583326d803c7c7abd8ea26cc842643fc
> Author: Aldy Hernandez <aldyh@redhat.com>
> Date:   Fri Oct 2 15:40:30 2015 -0700
> 
>     	* task.c (gomp_task_maybe_wait_for_dependencies): Fix scheduling
>     	problem such that the first non parent_depends_on task does not
>     	end up at the end of the children queue.

Ok for the branch.

	Jakub