Re: [PATCH] Unify pthread_once (bug 15215)

[Torvald Riegel <triegel@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2971 bytes --]

On Thu, 2013-05-23 at 00:15 -0400, Carlos O'Donell wrote:
> On 05/08/2013 10:43 AM, Torvald Riegel wrote:
> > See http://sourceware.org/bugzilla/show_bug.cgi?id=15215 for background.
> 
> You've already hashed out the details of these changes with Rich
> and he has no objection with this first phase of the patch which
> is to unify the implementations.
>  
> > I1 and I2 follow essentially the same algorithm, and we can replace it
> > with a unified variant, as the bug suggests.  See the attached patch for
> > a modified version of the sparc instance.  The differences between both
> > are either cosmetic, or are unnecessary changes (ie, how the
> > init-finished state is set (atomic_inc vs. store), or how the fork
> > generations are compared).
> > 
> > Both I1 and I2 were missing a release memory order (MO) when marking
> > once_control as finished initialization.  If the particular arch doesn't
> > need a HW barrier for release, we at least need a compiler barrier; if
> > it's needed, the original I1 and I2 are not guaranteed to work.
> > 
> > Both I1 and I2 were missing acquire MO on the very first load of
> > once_control.  This needs to synchronize with the release MO on setting
> > the state to init-finished, so without it it's not guaranteed to work
> > either.
> > Note that this will make a call to pthread_once that doesn't need to
> > actually run the init routine slightly slower due to the additional
> > acquire barrier.  If you're really concerned about this overhead, speak
> > up.  There are ways to avoid it, but it comes with additional complexity
> > and bookkeeping.
> 
> We want correctness. This is a place where correctness is infinitely
> more important than speed. We should be correct first and then we
> should argue about how to make it fast.
> 
> > I'm currently also using the existing atomic_{read/write}_barrier
> > functions instead of not-yet-existing load_acq or store_rel functions.
> > I'm not sure whether the latter can have somewhat more efficient
> > implementations on Power and ARM; if so, and if you're concerned about
> > the overhead, we can add load_acq and store_rel to atomic.h and start
> > using it.  This would be in line with C11, where we should eventually be
> > heading to anyways, IMO.
> 
> Agreed.
> 
> > Both I1 and I2 have an ABA issue on __fork_generation, as explained in
> > the comments that the patch adds.  How do you all feel about this?
> > I can't present a simple fix right now, but I believe it could be fixed
> > with additional bookkeeping.
> > 
> > If there's no objection to the essence of this patch, I'll post another
> > patch that actually replaces I1 and I2 with the modified variant in the
> > attached patch.
> 
> Please repost.

See attached patch.  This has been tested on ppc64 but not on the other
archs that are affected.  Nonetheless, ppc has a weak memory model, so,
for example, having an acquire barrier on a load or not having it does
make a difference.

OK?

[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 31090 bytes --]

commit 287d2b33853dec608439521df049b6d818b0b8c1
Author: Torvald Riegel <triegel@redhat.com>
Date:   Wed May 8 16:35:10 2013 +0200

    Fixed and unified pthread_once.
    
    	PR nptl/15215
    	nptl/
    	* sysdeps/unix/sysv/linux/pthread_once.c: New file.
    	* sysdeps/unix/sysv/linux/sparc/pthread_once.c: Remove file.
    
    	ports/
    	* sysdeps/unix/sysv/linux/aarch64/nptl/pthread_once.c: Remove file.
    	* sysdeps/unix/sysv/linux/arm/nptl/pthread_once.c: Remove file.
    	* sysdeps/unix/sysv/linux/hppa/nptl/pthread_once.c: Remove file.
    	* sysdeps/unix/sysv/linux/ia64/nptl/pthread_once.c: Remove file.
    	* sysdeps/unix/sysv/linux/m68k/nptl/pthread_once.c: Remove file.
    	* sysdeps/unix/sysv/linux/mips/nptl/pthread_once.c: Remove file.
    	* sysdeps/unix/sysv/linux/tile/nptl/pthread_once.c: Remove file.

diff --git a/nptl/ChangeLog b/nptl/ChangeLog
index a089153..7e87334 100644
--- a/nptl/ChangeLog
+++ b/nptl/ChangeLog
@@ -1,3 +1,8 @@
+2013-10-06  Torvald Riegel  <triegel@redhat.com>
+
+	* sysdeps/unix/sysv/linux/pthread_once.c: New file.
+	* sysdeps/unix/sysv/linux/sparc/pthread_once.c: Remove file.
+
 2013-10-04  Maciej W. Rozycki  <macro@codesourcery.com>
 
 	* tst-mutex8.c (check_type) [ENABLE_PI]: Handle ENOTSUP failure
diff --git a/nptl/sysdeps/unix/sysv/linux/pthread_once.c b/nptl/sysdeps/unix/sysv/linux/pthread_once.c
new file mode 100644
index 0000000..f9b0953
--- /dev/null
+++ b/nptl/sysdeps/unix/sysv/linux/pthread_once.c
@@ -0,0 +1,128 @@
+/* Copyright (C) 2003-2013 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+   Contributed by Jakub Jelinek <jakub@redhat.com>, 2003.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include "pthreadP.h"
+#include <lowlevellock.h>
+
+
+unsigned long int __fork_generation attribute_hidden;
+
+
+static void
+clear_once_control (void *arg)
+{
+  pthread_once_t *once_control = (pthread_once_t *) arg;
+
+  /* Reset to the uninitialized state here (see __pthread_once).  Also, we
+     don't need a stronger memory order because we do not need to make any
+     other of our writes visible to other threads that see this value.  */
+  *once_control = 0;
+  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
+}
+
+
+/* This is similar to a lock implementation, but we distinguish between three
+   states: not yet initialized (0), initialization finished (2), and
+   initialization in progress (__fork_generation | 1).  If in the first state,
+   threads will try to run the initialization by moving to the second state;
+   the first thread to do so via a CAS on once_control runs init_routine,
+   other threads block.
+   When forking the process, some threads can be interrupted during the second
+   state; they won't be present in the forked child, so we need to restart
+   initialization in the child.  To distinguish an in-progress initialization
+   from an interrupted initialization (in which case we need to reclaim the
+   lock), we look at the fork generation that's part of the second state: We
+   can reclaim iff it differs from the current fork generation.
+   XXX: This algorithm has an ABA issue on the fork generation: If an
+   initialization is interrupted, we then fork 2^30 times (30b of once_control
+   are used for the fork generation), and try to initialize again, we can
+   deadlock because we can't distinguish the in-progress and interrupted cases
+   anymore.  */
+int
+__pthread_once (once_control, init_routine)
+     pthread_once_t *once_control;
+     void (*init_routine) (void);
+{
+  while (1)
+    {
+      int oldval, val, newval;
+
+      /* We need acquire memory order for this load because if the value
+         signals that initialization has finished, we need to be see any
+         data modifications done during initialization.  */
+      val = *once_control;
+      atomic_read_barrier();
+      do
+	{
+	  /* Check if the initialization has already been done.  */
+	  if (__builtin_expect ((val & 2) != 0, 1))
+	    return 0;
+
+	  oldval = val;
+	  /* We try to set the state to in-progress and having the current
+	     fork generation.  We don't need atomic accesses for the fork
+	     generation because it's immutable in a particular process, and
+	     forked child processes start with a single thread that modified
+	     the generation.  */
+	  newval = __fork_generation | 1;
+	  /* We need acquire memory order here for the same reason as for the
+	     load from once_control above.  */
+	  val = atomic_compare_and_exchange_val_acq (once_control, newval,
+						     oldval);
+	}
+      while (__builtin_expect (val != oldval, 0));
+
+      /* Check if another thread already runs the initializer.	*/
+      if ((oldval & 1) != 0)
+	{
+	  /* Check whether the initializer execution was interrupted by a
+	     fork. (We know that for both values, bit 0 is set and bit 1 is
+	     not.)  */
+	  if (oldval == newval)
+	    {
+	      /* Same generation, some other thread was faster. Wait.  */
+	      lll_futex_wait (once_control, newval, LLL_PRIVATE);
+	      continue;
+	    }
+	}
+
+      /* This thread is the first here.  Do the initialization.
+	 Register a cleanup handler so that in case the thread gets
+	 interrupted the initialization can be restarted.  */
+      pthread_cleanup_push (clear_once_control, once_control);
+
+      init_routine ();
+
+      pthread_cleanup_pop (0);
+
+
+      /* Mark *once_control as having finished the initialization.  We need
+         release memory order here because we need to synchronize with other
+         threads that want to use the initialized data.  */
+      atomic_write_barrier();
+      *once_control = 2;
+
+      /* Wake up all other threads.  */
+      lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
+      break;
+    }
+
+  return 0;
+}
+weak_alias (__pthread_once, pthread_once)
+hidden_def (__pthread_once)
diff --git a/nptl/sysdeps/unix/sysv/linux/sparc/pthread_once.c b/nptl/sysdeps/unix/sysv/linux/sparc/pthread_once.c
deleted file mode 100644
index 5879f44..0000000
--- a/nptl/sysdeps/unix/sysv/linux/sparc/pthread_once.c
+++ /dev/null
@@ -1,93 +0,0 @@
-/* Copyright (C) 2003-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Jakub Jelinek <jakub@redhat.com>, 2003.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library; if not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-
-unsigned long int __fork_generation attribute_hidden;
-
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-
-int
-__pthread_once (once_control, init_routine)
-     pthread_once_t *once_control;
-     void (*init_routine) (void);
-{
-  while (1)
-    {
-      int oldval, val, newval;
-
-      val = *once_control;
-      do
-	{
-	  /* Check if the initialized has already been done.  */
-	  if ((val & 2) != 0)
-	    return 0;
-
-	  oldval = val;
-	  newval = (oldval & 3) | __fork_generation | 1;
-	  val = atomic_compare_and_exchange_val_acq (once_control, newval,
-						     oldval);
-	}
-      while (__builtin_expect (val != oldval, 0));
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) != 0)
-	{
-	  /* Check whether the initializer execution was interrupted
-	     by a fork.	 */
-	  if (((oldval ^ newval) & -4) == 0)
-	    {
-	      /* Same generation, some other thread was faster. Wait.  */
-	      lll_futex_wait (once_control, newval, LLL_PRIVATE);
-	      continue;
-	    }
-	}
-
-      /* This thread is the first here.  Do the initialization.
-	 Register a cleanup handler so that in case the thread gets
-	 interrupted the initialization can be restarted.  */
-      pthread_cleanup_push (clear_once_control, once_control);
-
-      init_routine ();
-
-      pthread_cleanup_pop (0);
-
-
-      /* Add one to *once_control.  */
-      atomic_increment (once_control);
-
-      /* Wake up all other threads.  */
-      lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-      break;
-    }
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/ChangeLog b/ports/ChangeLog
index a1dd8c3..13cefd2 100644
--- a/ports/ChangeLog
+++ b/ports/ChangeLog
@@ -1,3 +1,13 @@
+2013-10-06  Torvald Riegel  <triegel@redhat.com>
+
+	* sysdeps/unix/sysv/linux/aarch64/nptl/pthread_once.c: Remove file.
+	* sysdeps/unix/sysv/linux/arm/nptl/pthread_once.c: Remove file.
+	* sysdeps/unix/sysv/linux/hppa/nptl/pthread_once.c: Remove file.
+	* sysdeps/unix/sysv/linux/ia64/nptl/pthread_once.c: Remove file.
+	* sysdeps/unix/sysv/linux/m68k/nptl/pthread_once.c: Remove file.
+	* sysdeps/unix/sysv/linux/mips/nptl/pthread_once.c: Remove file.
+	* sysdeps/unix/sysv/linux/tile/nptl/pthread_once.c: Remove file.
+
 2013-10-04  Alan Modra  <amodra@gmail.com>
 
 	* sysdeps/ia64/fpu/printf_fphex.c: Adjust for fpnum change.
diff --git a/ports/sysdeps/unix/sysv/linux/aarch64/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/aarch64/nptl/pthread_once.c
deleted file mode 100644
index 2fb9b85..0000000
--- a/ports/sysdeps/unix/sysv/linux/aarch64/nptl/pthread_once.c
+++ /dev/null
@@ -1,90 +0,0 @@
-/* Copyright (C) 2004-2013 Free Software Foundation, Inc.
-
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public License as
-   published by the Free Software Foundation; either version 2.1 of the
-   License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library; if not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-unsigned long int __fork_generation attribute_hidden;
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-int
-__pthread_once (pthread_once_t *once_control, void (*init_routine) (void))
-{
-  for (;;)
-    {
-      int oldval;
-      int newval;
-
-      /* Pseudo code:
-	 newval = __fork_generation | 1;
-	 oldval = *once_control;
-	 if ((oldval & 2) == 0)
-	   *once_control = newval;
-	 Do this atomically.
-      */
-      do
-	{
-	  newval = __fork_generation | 1;
-	  oldval = *once_control;
-	  if (oldval & 2)
-	    break;
-	} while (atomic_compare_and_exchange_val_acq (once_control, newval, oldval) != oldval);
-
-      /* Check if the initializer has already been done.  */
-      if ((oldval & 2) != 0)
-	return 0;
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) == 0)
-	break;
-
-      /* Check whether the initializer execution was interrupted by a fork.  */
-      if (oldval != newval)
-	break;
-
-      /* Same generation, some other thread was faster. Wait.  */
-      lll_futex_wait (once_control, oldval, LLL_PRIVATE);
-    }
-
-  /* This thread is the first here.  Do the initialization.
-     Register a cleanup handler so that in case the thread gets
-     interrupted the initialization can be restarted.  */
-  pthread_cleanup_push (clear_once_control, once_control);
-
-  init_routine ();
-
-  pthread_cleanup_pop (0);
-
-  /* Say that the initialisation is done.  */
-  *once_control = __fork_generation | 2;
-
-  /* Wake up all other threads.  */
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/sysdeps/unix/sysv/linux/arm/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/arm/nptl/pthread_once.c
deleted file mode 100644
index 0c897ab..0000000
--- a/ports/sysdeps/unix/sysv/linux/arm/nptl/pthread_once.c
+++ /dev/null
@@ -1,89 +0,0 @@
-/* Copyright (C) 2004-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-unsigned long int __fork_generation attribute_hidden;
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-int
-__pthread_once (pthread_once_t *once_control, void (*init_routine) (void))
-{
-  for (;;)
-    {
-      int oldval;
-      int newval;
-
-      /* Pseudo code:
-	 newval = __fork_generation | 1;
-	 oldval = *once_control;
-	 if ((oldval & 2) == 0)
-	   *once_control = newval;
-	 Do this atomically.
-      */
-      do
-	{
-	  newval = __fork_generation | 1;
-	  oldval = *once_control;
-	  if (oldval & 2)
-	    break;
-	} while (atomic_compare_and_exchange_val_acq (once_control, newval, oldval) != oldval);
-
-      /* Check if the initializer has already been done.  */
-      if ((oldval & 2) != 0)
-	return 0;
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) == 0)
-	break;
-
-      /* Check whether the initializer execution was interrupted by a fork.  */
-      if (oldval != newval)
-	break;
-
-      /* Same generation, some other thread was faster. Wait.  */
-      lll_futex_wait (once_control, oldval, LLL_PRIVATE);
-    }
-
-  /* This thread is the first here.  Do the initialization.
-     Register a cleanup handler so that in case the thread gets
-     interrupted the initialization can be restarted.  */
-  pthread_cleanup_push (clear_once_control, once_control);
-
-  init_routine ();
-
-  pthread_cleanup_pop (0);
-
-  /* Say that the initialisation is done.  */
-  *once_control = __fork_generation | 2;
-
-  /* Wake up all other threads.  */
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/sysdeps/unix/sysv/linux/hppa/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/hppa/nptl/pthread_once.c
deleted file mode 100644
index b032f29..0000000
--- a/ports/sysdeps/unix/sysv/linux/hppa/nptl/pthread_once.c
+++ /dev/null
@@ -1,93 +0,0 @@
-/* Copyright (C) 2003-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Jakub Jelinek <jakub@redhat.com>, 2003.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-
-unsigned long int __fork_generation attribute_hidden;
-
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_private_futex_wake (once_control, INT_MAX);
-}
-
-
-int
-__pthread_once (once_control, init_routine)
-     pthread_once_t *once_control;
-     void (*init_routine) (void);
-{
-  while (1)
-    {
-      int oldval, val, newval;
-
-      val = *once_control;
-      do
-	{
-	  /* Check if the initialized has already been done.  */
-	  if ((val & 2) != 0)
-	    return 0;
-
-	  oldval = val;
-	  newval = (oldval & 3) | __fork_generation | 1;
-	  val = atomic_compare_and_exchange_val_acq (once_control, newval,
-						     oldval);
-	}
-      while (__builtin_expect (val != oldval, 0));
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) != 0)
-	{
-	  /* Check whether the initializer execution was interrupted
-	     by a fork.	 */
-	  if (((oldval ^ newval) & -4) == 0)
-	    {
-	      /* Same generation, some other thread was faster. Wait.  */
-	      lll_private_futex_wait (once_control, newval);
-	      continue;
-	    }
-	}
-
-      /* This thread is the first here.  Do the initialization.
-	 Register a cleanup handler so that in case the thread gets
-	 interrupted the initialization can be restarted.  */
-      pthread_cleanup_push (clear_once_control, once_control);
-
-      init_routine ();
-
-      pthread_cleanup_pop (0);
-
-
-      /* Add one to *once_control.  */
-      atomic_increment (once_control);
-
-      /* Wake up all other threads.  */
-      lll_private_futex_wake (once_control, INT_MAX);
-      break;
-    }
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/sysdeps/unix/sysv/linux/ia64/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/ia64/nptl/pthread_once.c
deleted file mode 100644
index 5879f44..0000000
--- a/ports/sysdeps/unix/sysv/linux/ia64/nptl/pthread_once.c
+++ /dev/null
@@ -1,93 +0,0 @@
-/* Copyright (C) 2003-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Jakub Jelinek <jakub@redhat.com>, 2003.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library; if not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-
-unsigned long int __fork_generation attribute_hidden;
-
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-
-int
-__pthread_once (once_control, init_routine)
-     pthread_once_t *once_control;
-     void (*init_routine) (void);
-{
-  while (1)
-    {
-      int oldval, val, newval;
-
-      val = *once_control;
-      do
-	{
-	  /* Check if the initialized has already been done.  */
-	  if ((val & 2) != 0)
-	    return 0;
-
-	  oldval = val;
-	  newval = (oldval & 3) | __fork_generation | 1;
-	  val = atomic_compare_and_exchange_val_acq (once_control, newval,
-						     oldval);
-	}
-      while (__builtin_expect (val != oldval, 0));
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) != 0)
-	{
-	  /* Check whether the initializer execution was interrupted
-	     by a fork.	 */
-	  if (((oldval ^ newval) & -4) == 0)
-	    {
-	      /* Same generation, some other thread was faster. Wait.  */
-	      lll_futex_wait (once_control, newval, LLL_PRIVATE);
-	      continue;
-	    }
-	}
-
-      /* This thread is the first here.  Do the initialization.
-	 Register a cleanup handler so that in case the thread gets
-	 interrupted the initialization can be restarted.  */
-      pthread_cleanup_push (clear_once_control, once_control);
-
-      init_routine ();
-
-      pthread_cleanup_pop (0);
-
-
-      /* Add one to *once_control.  */
-      atomic_increment (once_control);
-
-      /* Wake up all other threads.  */
-      lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-      break;
-    }
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/sysdeps/unix/sysv/linux/m68k/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/m68k/nptl/pthread_once.c
deleted file mode 100644
index 8a1f307..0000000
--- a/ports/sysdeps/unix/sysv/linux/m68k/nptl/pthread_once.c
+++ /dev/null
@@ -1,90 +0,0 @@
-/* Copyright (C) 2010-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Maxim Kuvyrkov <maxim@codesourcery.com>, 2010.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-unsigned long int __fork_generation attribute_hidden;
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-int
-__pthread_once (pthread_once_t *once_control, void (*init_routine) (void))
-{
-  for (;;)
-    {
-      int oldval;
-      int newval;
-
-      /* Pseudo code:
-	 newval = __fork_generation | 1;
-	 oldval = *once_control;
-	 if ((oldval & 2) == 0)
-	   *once_control = newval;
-	 Do this atomically.
-      */
-      do
-	{
-	  newval = __fork_generation | 1;
-	  oldval = *once_control;
-	  if (oldval & 2)
-	    break;
-	} while (atomic_compare_and_exchange_val_acq (once_control, newval, oldval) != oldval);
-
-      /* Check if the initializer has already been done.  */
-      if ((oldval & 2) != 0)
-	return 0;
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) == 0)
-	break;
-
-      /* Check whether the initializer execution was interrupted by a fork.  */
-      if (oldval != newval)
-	break;
-
-      /* Same generation, some other thread was faster. Wait.  */
-      lll_futex_wait (once_control, oldval, LLL_PRIVATE);
-    }
-
-  /* This thread is the first here.  Do the initialization.
-     Register a cleanup handler so that in case the thread gets
-     interrupted the initialization can be restarted.  */
-  pthread_cleanup_push (clear_once_control, once_control);
-
-  init_routine ();
-
-  pthread_cleanup_pop (0);
-
-  /* Say that the initialisation is done.  */
-  *once_control = __fork_generation | 2;
-
-  /* Wake up all other threads.  */
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/sysdeps/unix/sysv/linux/mips/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/mips/nptl/pthread_once.c
deleted file mode 100644
index 97f1ddf..0000000
--- a/ports/sysdeps/unix/sysv/linux/mips/nptl/pthread_once.c
+++ /dev/null
@@ -1,93 +0,0 @@
-/* Copyright (C) 2003-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Jakub Jelinek <jakub@redhat.com>, 2003.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include "pthreadP.h"
-#include <lowlevellock.h>
-
-
-unsigned long int __fork_generation attribute_hidden;
-
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-
-int
-__pthread_once (once_control, init_routine)
-     pthread_once_t *once_control;
-     void (*init_routine) (void);
-{
-  while (1)
-    {
-      int oldval, val, newval;
-
-      val = *once_control;
-      do
-	{
-	  /* Check if the initialized has already been done.  */
-	  if ((val & 2) != 0)
-	    return 0;
-
-	  oldval = val;
-	  newval = (oldval & 3) | __fork_generation | 1;
-	  val = atomic_compare_and_exchange_val_acq (once_control, newval,
-						     oldval);
-	}
-      while (__builtin_expect (val != oldval, 0));
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) != 0)
-	{
-	  /* Check whether the initializer execution was interrupted
-	     by a fork.	 */
-	  if (((oldval ^ newval) & -4) == 0)
-	    {
-	      /* Same generation, some other thread was faster. Wait.  */
-	      lll_futex_wait (once_control, newval, LLL_PRIVATE);
-	      continue;
-	    }
-	}
-
-      /* This thread is the first here.  Do the initialization.
-	 Register a cleanup handler so that in case the thread gets
-	 interrupted the initialization can be restarted.  */
-      pthread_cleanup_push (clear_once_control, once_control);
-
-      init_routine ();
-
-      pthread_cleanup_pop (0);
-
-
-      /* Add one to *once_control.  */
-      atomic_increment (once_control);
-
-      /* Wake up all other threads.  */
-      lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-      break;
-    }
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)
diff --git a/ports/sysdeps/unix/sysv/linux/tile/nptl/pthread_once.c b/ports/sysdeps/unix/sysv/linux/tile/nptl/pthread_once.c
deleted file mode 100644
index 68456f0..0000000
--- a/ports/sysdeps/unix/sysv/linux/tile/nptl/pthread_once.c
+++ /dev/null
@@ -1,94 +0,0 @@
-/* Copyright (C) 2011-2013 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Chris Metcalf <cmetcalf@tilera.com>, 2011.
-   Based on work contributed by Jakub Jelinek <jakub@redhat.com>, 2003.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library.  If not, see
-   <http://www.gnu.org/licenses/>.  */
-
-#include <nptl/pthreadP.h>
-#include <lowlevellock.h>
-
-
-unsigned long int __fork_generation attribute_hidden;
-
-
-static void
-clear_once_control (void *arg)
-{
-  pthread_once_t *once_control = (pthread_once_t *) arg;
-
-  *once_control = 0;
-  lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-}
-
-
-int
-__pthread_once (once_control, init_routine)
-     pthread_once_t *once_control;
-     void (*init_routine) (void);
-{
-  while (1)
-    {
-      int oldval, val, newval;
-
-      val = *once_control;
-      do
-	{
-	  /* Check if the initialized has already been done.  */
-	  if ((val & 2) != 0)
-	    return 0;
-
-	  oldval = val;
-	  newval = (oldval & 3) | __fork_generation | 1;
-	  val = atomic_compare_and_exchange_val_acq (once_control, newval,
-						     oldval);
-	}
-      while (__builtin_expect (val != oldval, 0));
-
-      /* Check if another thread already runs the initializer.	*/
-      if ((oldval & 1) != 0)
-	{
-	  /* Check whether the initializer execution was interrupted
-	     by a fork.	 */
-	  if (((oldval ^ newval) & -4) == 0)
-	    {
-	      /* Same generation, some other thread was faster. Wait.  */
-	      lll_futex_wait (once_control, newval, LLL_PRIVATE);
-	      continue;
-	    }
-	}
-
-      /* This thread is the first here.  Do the initialization.
-	 Register a cleanup handler so that in case the thread gets
-	 interrupted the initialization can be restarted.  */
-      pthread_cleanup_push (clear_once_control, once_control);
-
-      init_routine ();
-
-      pthread_cleanup_pop (0);
-
-
-      /* Add one to *once_control.  */
-      atomic_increment (once_control);
-
-      /* Wake up all other threads.  */
-      lll_futex_wake (once_control, INT_MAX, LLL_PRIVATE);
-      break;
-    }
-
-  return 0;
-}
-weak_alias (__pthread_once, pthread_once)
-hidden_def (__pthread_once)