[Bug libgomp/51298] New: libgomp team_barrier locking failures

[Bug libgomp/51298] New: libgomp team_barrier locking failures

[amodra at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

             Bug #: 51298
           Summary: libgomp team_barrier locking failures
    Classification: Unclassified
           Product: gcc
           Version: 4.7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libgomp
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: amodra@gmail.com


There seems to be a locking related failure in the linux barrier
implementation, because libgomp testcases hang on power7.  This is both before
and after the fix for http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51249

Here's a dump of some info from one of the hung tests that might help pin down
the problem.  Then again, it might not.  power7 quite aggressively speculates
reads, so I suspect a missing read barrier somewhere.

(gdb) bt
#0  0x00000fff91ebf36c in sys_futex0 (val=0, op=<optimized out>,
addr=0x100210d4) at
/home/amodra/src/gcc-current/libgomp/config/linux/powerpc/futex.h:48
#1  futex_wait (val=0, addr=0x100210d4) at
/home/amodra/src/gcc-current/libgomp/config/linux/powerpc/futex.h:61
#2  do_wait (val=0, addr=<optimized out>) at
/home/amodra/src/gcc-current/libgomp/config/linux/wait.h:64
#3  gomp_team_barrier_wait_end (bar=0x100210d0, state=0) at
/home/amodra/src/gcc-current/libgomp/config/linux/bar.c:109
#4  0x00000fff91eb7540 in GOMP_barrier () at
/home/amodra/src/gcc-current/libgomp/barrier.c:40
#5  0x0000000010000f10 in .test_barrier.822._omp_fn.2 () at
/home/amodra/src/gcc-current/libgomp/testsuite/libgomp.fortran/omp_parse2.f90:59
#6  0x0000000010000b90 in test_barrier () at
/home/amodra/src/gcc-current/libgomp/testsuite/libgomp.fortran/omp_parse2.f90:56
#7  MAIN__ () at
/home/amodra/src/gcc-current/libgomp/testsuite/libgomp.fortran/omp_parse2.f90:5
#8  main (argc=<optimized out>, argv=<optimized out>) at
/home/amodra/src/gcc-current/libgomp/testsuite/libgomp.fortran/omp_parse2.f90:2
#9  0x00000fff91ccf05c in .generic_start_main () from /lib64/power7/libc.so.6
#10 0x00000fff91ccf27c in .__libc_start_main () from /lib64/power7/libc.so.6
#11 0x0000000000000000 in ?? ()
(gdb) up 3
#3  gomp_team_barrier_wait_end (bar=0x100210d0, state=0) at
/home/amodra/src/gcc-current/libgomp/config/linux/bar.c:109
(gdb) p *bar
$1 = {total = 4, generation = 0, awaited = 1}
(gdb) p state
$2 = 0
(gdb) p generation
$3 = 0
(gdb) p gomp_tls_data
$4 = {fn = 0, data = 0x0, ts = {team = 0x10021050, work_share = 0x10021150,
last_work_share = 0x0, team_id = 0, level = 1, active_level = 1, single_count =
0, static_trip = 0}, task = 0x10021568, release = {count = 0}, thread_pool =
0x10021760}
(gdb) p *gomp_tls_data.ts.team
$5 = {nthreads = 4, work_share_chunk = 8, prev_ts = {team = 0x0, work_share =
0x0, last_work_share = 0x0, team_id = 0, level = 0, active_level = 0,
single_count = 0, static_trip = 0}, master_release = {count = 0},
ordered_release = 0x10021708, work_share_list_alloc = 0x100211d0,
work_share_list_free = 0x0, single_count = 0, barrier = {total = 4, generation
= 0, awaited = 1}, work_shares = {{sched = GFS_RUNTIME, mode = 0, {{chunk_size
= 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0, incr_ull = 0}},
ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner = 0, ordered_cur =
0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0, {next = 0,
next_ull = 0, copyprivate = 0x0}, {next_ws = 0x0, next_free = 0x0},
inline_ordered_team_ids = 0x100211a8}, {sched = GFS_RUNTIME, mode = 0,
{{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x10021250, next_free =
0x10021250}, inline_ordered_team_ids = 0x10021228}, {sched = GFS_RUNTIME, mode
= 0, {{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x100212d0, next_free =
0x100212d0}, inline_ordered_team_ids = 0x100212a8}, {sched = GFS_RUNTIME, mode
= 0, {{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x10021350, next_free =
0x10021350}, inline_ordered_team_ids = 0x10021328}, {sched = GFS_RUNTIME, mode
= 0, {{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x100213d0, next_free =
0x100213d0}, inline_ordered_team_ids = 0x100213a8}, {sched = GFS_RUNTIME, mode
= 0, {{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x10021450, next_free =
0x10021450}, inline_ordered_team_ids = 0x10021428}, {sched = GFS_RUNTIME, mode
= 0, {{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x100214d0, next_free =
0x100214d0}, inline_ordered_team_ids = 0x100214a8}, {sched = GFS_RUNTIME, mode
= 0, {{chunk_size = 0, end = 0, incr = 0}, {chunk_size_ull = 0, end_ull = 0,
incr_ull = 0}}, ordered_team_ids = 0x0, ordered_num_used = 0, ordered_owner =
0, ordered_cur = 0, next_alloc = 0x0, lock = {flag = 0}, threads_completed = 0,
{next = 0, next_ull = 0, copyprivate = 0x0}, {next_ws = 0x0, next_free = 0x0},
inline_ordered_team_ids = 0x10021528}}, task_lock = {flag = 0}, task_queue =
0x0, task_count = 0, task_running_count = 0, implicit_task = 0x10021568}
(gdb) p *gomp_tls_data.ts.work_share
$6 = {sched = GFS_RUNTIME, mode = 0, {{chunk_size = 0, end = 0, incr = 0},
{chunk_size_ull = 0, end_ull = 0, incr_ull = 0}}, ordered_team_ids = 0x0,
ordered_num_used = 0, ordered_owner = 0, ordered_cur = 0, next_alloc = 0x0,
lock = {flag = 0}, threads_completed = 0, {next = 0, next_ull = 0, copyprivate
= 0x0}, {next_ws = 0x0, next_free = 0x0}, inline_ordered_team_ids = 0x100211a8}

[Bug libgomp/51298] libgomp team_barrier locking failures

[amodra at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

Alan Modra <amodra at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2011-11-27
         AssignedTo|unassigned at gcc dot       |amodra at gmail dot com
                   |gnu.org                     |
     Ever Confirmed|0                           |1

[Bug libgomp/51298] libgomp team_barrier locking failures

[amodra at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

--- Comment #1 from Alan Modra <amodra at gcc dot gnu.org> 2011-11-30 04:44:18 UTC ---
Author: amodra
Date: Wed Nov 30 04:44:13 2011
New Revision: 181833

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=181833
Log:
    PR libgomp/51298
    * config/linux/bar.h: Use atomic rather than sync builtins.
    * config/linux/bar.c: Likewise.  Add missing acquire
    synchronisation on generation field.
    * task.c (gomp_barrier_handle_tasks): Regain lock so as to not
    double unlock.


Modified:
    trunk/libgomp/ChangeLog
    trunk/libgomp/config/linux/bar.c
    trunk/libgomp/config/linux/bar.h
    trunk/libgomp/task.c

[Bug libgomp/51298] libgomp team_barrier locking failures

[amodra at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

Alan Modra <amodra at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |4.7.0

--- Comment #2 from Alan Modra <amodra at gmail dot com> 2011-11-30 04:49:11 UTC ---
fixed

[Bug libgomp/51298] libgomp team_barrier locking failures

[abenson at caltech dot edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

Andrew Benson <abenson at caltech dot edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |abenson at caltech dot edu

--- Comment #3 from Andrew Benson <abenson at caltech dot edu> 2011-12-15 07:09:32 UTC ---
Is/was this bug also present in 4.6.0?

[Bug libgomp/51298] libgomp team_barrier locking failures

[amodra at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

--- Comment #4 from Alan Modra <amodra at gmail dot com> 2011-12-15 07:57:44 UTC ---
Yes, this bug and more are still in gcc-4.6.  I haven't had time to look at
backporting any of my libgomp fixes.

[Bug libgomp/51298] libgomp team_barrier locking failures

[Joost.VandeVondele at mat dot ethz.ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

--- Comment #5 from Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> 2011-12-15 09:44:46 UTC ---
similarly, does this only affect power7, or potentially also other targets such
as x86_64 (interlagos?)

[Bug libgomp/51298] libgomp team_barrier locking failures

[amodra at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51298

--- Comment #6 from Alan Modra <amodra at gmail dot com> 2011-12-15 13:45:20 UTC ---
I expect all of the libgomp bugs I recently fixed will affect some other
processors, eg. Alpha, IA64, and Sparc in some memory modes.  I know PR51376
won't affect x86, and I expect PR51249 will affect all processors.  Beyond
that, I haven't analyzed which bugs will affect a given processor.  BTW,
there's a good article here http://www.linuxjournal.com/article/8211 that
explains how and why memory accesses are reordered, and the extent that various
processors do reordering.  The table in that article is why I named Alpha, IA64
and Sparc.