[PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Julian Brown]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

Hi,

This patch implements the atomic builtins described at:

  http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/Atomic-Builtins.html

for ARM EABI Linux. This implementation uses the kernel helpers
__kernel_cmpxchg and __kernel_dmb, and so should work on any
architecture which supports those. (More-efficient versions are possible
using ldrex/strex on architectures >=v6, but those are not written yet.)

Atomic operations are provided for data sizes of 1, 2 and 4 bytes (but
not 8 bytes). The implementation uses actual functions
(__sync_fetch_and_add_2, etc.) rather than expanding code inline.

Tested with cross to arm-none-linux-gnueabi, and with some additional
hand-written tests which hopefully exercised the atomicity of the
operations sufficiently.

OK for mainline?

Julian

ChangeLog

    gcc/
    * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
    config/arm/linux-atomic.c.
    * config/arm/linux-atomic.c: New.

[-- Attachment #2: arm-atomics-fsf-1.diff --]
[-- Type: text/x-patch, Size: 9494 bytes --]

Index: gcc/config/arm/linux-atomic.c
===================================================================
--- gcc/config/arm/linux-atomic.c	(revision 0)
+++ gcc/config/arm/linux-atomic.c	(revision 0)
@@ -0,0 +1,280 @@
+/* Linux-specific atomic operations for ARM EABI.
+   Copyright (C) 2008 Free Software Foundation, Inc.
+   Contributed by CodeSourcery.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 2, or (at your option) any later
+version.
+
+In addition to the permissions in the GNU General Public License, the
+Free Software Foundation gives you unlimited permission to link the
+compiled version of this file into combinations with other programs,
+and to distribute those combinations without any restriction coming
+from the use of this file.  (The General Public License restrictions
+do apply in other respects; for example, they cover modification of
+the file, and distribution when not linked into a combine
+executable.)
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING.  If not, write to the Free
+Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
+02110-1301, USA.  */
+
+/* Kernel helper for compare-and-exchange.  */
+typedef int (__kernel_cmpxchg_t) (int oldval, int newval, int *ptr);
+#define __kernel_cmpxchg (*(__kernel_cmpxchg_t *) 0xffff0fc0)
+
+/* Kernel helper for memory barrier.  */
+typedef void (__kernel_dmb_t) (void);
+#define __kernel_dmb (*(__kernel_dmb_t *) 0xffff0fa0)
+
+/* Note: we implement byte, short and int versions of atomic operations using
+   the above kernel helpers, but there is no support for "long long" (64-bit)
+   operations as yet.  */
+
+#define HIDDEN __attribute__ ((visibility ("hidden")))
+
+#ifdef __ARMEL__
+#define INVERT_MASK_1 0
+#define INVERT_MASK_2 0
+#else
+#define INVERT_MASK_1 24
+#define INVERT_MASK_2 16
+#endif
+
+#define MASK_1 0xffu
+#define MASK_2 0xffffu
+
+#define FETCH_AND_OP_WORD(OP, PFX_OP, INF_OP)				\
+  int HIDDEN								\
+  __sync_fetch_and_##OP##_4 (int *ptr, int val)				\
+  {									\
+    int failure, tmp;							\
+									\
+    do {								\
+      tmp = *ptr;							\
+      failure = __kernel_cmpxchg (tmp, PFX_OP tmp INF_OP val, ptr);	\
+    } while (failure != 0);						\
+									\
+    return tmp;								\
+  }
+
+FETCH_AND_OP_WORD (add,   , +)
+FETCH_AND_OP_WORD (sub,   , -)
+FETCH_AND_OP_WORD (or,    , |)
+FETCH_AND_OP_WORD (and,   , &)
+FETCH_AND_OP_WORD (xor,   , ^)
+FETCH_AND_OP_WORD (nand, ~, &)
+
+#define NAME_oldval(OP, WIDTH) __sync_fetch_and_##OP##_##WIDTH
+#define NAME_newval(OP, WIDTH) __sync_##OP##_and_fetch_##WIDTH
+
+/* Implement both __sync_<op>_and_fetch and __sync_fetch_and_<op> for
+   subword-sized quantities.  */
+
+#define SUBWORD_SYNC_OP(OP, PFX_OP, INF_OP, TYPE, WIDTH, RETURN)	\
+  TYPE HIDDEN								\
+  NAME##_##RETURN (OP, WIDTH) (TYPE *ptr, TYPE val)			\
+  {									\
+    int *wordptr = (int *) ((unsigned int) ptr & ~3);			\
+    unsigned int mask, shift, oldval, newval;				\
+    int failure;							\
+									\
+    shift = (((unsigned int) ptr & 3) << 3) ^ INVERT_MASK_##WIDTH;	\
+    mask = MASK_##WIDTH << shift;					\
+									\
+    do {								\
+      oldval = *wordptr;						\
+      newval = ((PFX_OP ((oldval & mask) >> shift)			\
+                 INF_OP (unsigned int) val) << shift) & mask;		\
+      newval |= oldval & ~mask;						\
+      failure = __kernel_cmpxchg (oldval, newval, wordptr);		\
+    } while (failure != 0);						\
+									\
+    return (RETURN & mask) >> shift;					\
+  }
+
+SUBWORD_SYNC_OP (add,   , +, short, 2, oldval)
+SUBWORD_SYNC_OP (sub,   , -, short, 2, oldval)
+SUBWORD_SYNC_OP (or,    , |, short, 2, oldval)
+SUBWORD_SYNC_OP (and,   , &, short, 2, oldval)
+SUBWORD_SYNC_OP (xor,   , ^, short, 2, oldval)
+SUBWORD_SYNC_OP (nand, ~, &, short, 2, oldval)
+
+SUBWORD_SYNC_OP (add,   , +, char, 1, oldval)
+SUBWORD_SYNC_OP (sub,   , -, char, 1, oldval)
+SUBWORD_SYNC_OP (or,    , |, char, 1, oldval)
+SUBWORD_SYNC_OP (and,   , &, char, 1, oldval)
+SUBWORD_SYNC_OP (xor,   , ^, char, 1, oldval)
+SUBWORD_SYNC_OP (nand, ~, &, char, 1, oldval)
+
+#define OP_AND_FETCH_WORD(OP, PFX_OP, INF_OP)				\
+  int HIDDEN								\
+  __sync_##OP##_and_fetch_4 (int *ptr, int val)				\
+  {									\
+    int tmp, failure;							\
+									\
+    do {								\
+      tmp = *ptr;							\
+      failure = __kernel_cmpxchg (tmp, PFX_OP tmp INF_OP val, ptr);	\
+    } while (failure != 0);						\
+									\
+    return PFX_OP tmp INF_OP val;					\
+  }
+
+OP_AND_FETCH_WORD (add,   , +)
+OP_AND_FETCH_WORD (sub,   , -)
+OP_AND_FETCH_WORD (or,    , |)
+OP_AND_FETCH_WORD (and,   , &)
+OP_AND_FETCH_WORD (xor,   , ^)
+OP_AND_FETCH_WORD (nand, ~, &)
+
+SUBWORD_SYNC_OP (add,   , +, short, 2, newval)
+SUBWORD_SYNC_OP (sub,   , -, short, 2, newval)
+SUBWORD_SYNC_OP (or,    , |, short, 2, newval)
+SUBWORD_SYNC_OP (and,   , &, short, 2, newval)
+SUBWORD_SYNC_OP (xor,   , ^, short, 2, newval)
+SUBWORD_SYNC_OP (nand, ~, &, short, 2, newval)
+
+SUBWORD_SYNC_OP (add,   , +, char, 1, newval)
+SUBWORD_SYNC_OP (sub,   , -, char, 1, newval)
+SUBWORD_SYNC_OP (or,    , |, char, 1, newval)
+SUBWORD_SYNC_OP (and,   , &, char, 1, newval)
+SUBWORD_SYNC_OP (xor,   , ^, char, 1, newval)
+SUBWORD_SYNC_OP (nand, ~, &, char, 1, newval)
+
+int HIDDEN
+__sync_val_compare_and_swap_4 (int *ptr, int oldval, int newval)
+{
+  int actual_oldval, fail;
+    
+  while (1)
+    {
+      actual_oldval = *ptr;
+
+      if (oldval != actual_oldval)
+	return actual_oldval;
+
+      fail = __kernel_cmpxchg (actual_oldval, newval, ptr);
+  
+      if (!fail)
+        return oldval;
+    }
+}
+
+#define SUBWORD_VAL_CAS(TYPE, WIDTH)					\
+  TYPE HIDDEN								\
+  __sync_val_compare_and_swap_##WIDTH (TYPE *ptr, TYPE oldval,		\
+				       TYPE newval)			\
+  {									\
+    int *wordptr = (int *)((unsigned int) ptr & ~3), fail;		\
+    unsigned int mask, shift, actual_oldval, actual_newval;		\
+									\
+    shift = (((unsigned int) ptr & 3) << 3) ^ INVERT_MASK_##WIDTH;	\
+    mask = MASK_##WIDTH << shift;					\
+									\
+    while (1)								\
+      {									\
+	actual_oldval = *wordptr;					\
+									\
+	if (((actual_oldval & mask) >> shift) != (unsigned int) oldval)	\
+          return (actual_oldval & mask) >> shift;			\
+									\
+	actual_newval = (actual_oldval & ~mask)				\
+			| (((unsigned int) newval << shift) & mask);	\
+									\
+	fail = __kernel_cmpxchg (actual_oldval, actual_newval,		\
+				 wordptr);				\
+									\
+	if (!fail)							\
+          return oldval;						\
+      }									\
+  }
+
+SUBWORD_VAL_CAS (short, 2)
+SUBWORD_VAL_CAS (char,  1)
+
+typedef unsigned char bool;
+
+bool HIDDEN
+__sync_bool_compare_and_swap_4 (int *ptr, int oldval, int newval)
+{
+  int failure = __kernel_cmpxchg (oldval, newval, ptr);
+  return (failure == 0);
+}
+
+#define SUBWORD_BOOL_CAS(TYPE, WIDTH)					\
+  bool HIDDEN								\
+  __sync_bool_compare_and_swap_##WIDTH (TYPE *ptr, TYPE oldval,		\
+					TYPE newval)			\
+  {									\
+    TYPE actual_oldval							\
+      = __sync_val_compare_and_swap_##WIDTH (ptr, oldval, newval);	\
+    return (oldval == actual_oldval);					\
+  }
+
+SUBWORD_BOOL_CAS (short, 2)
+SUBWORD_BOOL_CAS (char,  1)
+
+void HIDDEN
+__sync_synchronize (void)
+{
+  __kernel_dmb ();
+}
+
+int HIDDEN
+__sync_lock_test_and_set_4 (int *ptr, int val)
+{
+  int failure, oldval;
+
+  do {
+    oldval = *ptr;
+    failure = __kernel_cmpxchg (oldval, val, ptr);
+  } while (failure != 0);
+
+  return oldval;
+}
+
+#define SUBWORD_TEST_AND_SET(TYPE, WIDTH)				\
+  TYPE HIDDEN								\
+  __sync_lock_test_and_set_##WIDTH (TYPE *ptr, TYPE val)		\
+  {									\
+    int failure;							\
+    unsigned int oldval, newval, shift, mask;				\
+    int *wordptr = (int *) ((unsigned int) ptr & ~3);			\
+									\
+    shift = (((unsigned int) ptr & 3) << 3) ^ INVERT_MASK_##WIDTH;	\
+    mask = MASK_##WIDTH << shift;					\
+									\
+    do {								\
+      oldval = *wordptr;						\
+      newval = (oldval & ~mask)						\
+	       | (((unsigned int) val << shift) & mask);		\
+      failure = __kernel_cmpxchg (oldval, newval, wordptr);		\
+    } while (failure != 0);						\
+									\
+    return (oldval & mask) >> shift;					\
+  }
+
+SUBWORD_TEST_AND_SET (short, 2)
+SUBWORD_TEST_AND_SET (char,  1)
+
+#define SYNC_LOCK_RELEASE(TYPE, WIDTH)					\
+  void HIDDEN								\
+  __sync_lock_release_##WIDTH (TYPE *ptr)				\
+  {									\
+    *ptr = 0;								\
+    __kernel_dmb ();							\
+  }
+
+SYNC_LOCK_RELEASE (int,   4)
+SYNC_LOCK_RELEASE (short, 2)
+SYNC_LOCK_RELEASE (char,  1)
Index: gcc/config/arm/t-linux-eabi
===================================================================
--- gcc/config/arm/t-linux-eabi	(revision 136167)
+++ gcc/config/arm/t-linux-eabi	(working copy)
@@ -12,3 +12,5 @@ LIB1ASMFUNCS := $(filter-out _dvmd_tls,$
 # Multilib the standard Linux files.  Don't include crti.o or crtn.o,
 # which are provided by glibc.
 EXTRA_MULTILIB_PARTS=crtbegin.o crtend.o crtbeginS.o crtendS.o crtbeginT.o
+
+LIB2FUNCS_STATIC_EXTRA += $(srcdir)/config/arm/linux-atomic.c

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Helge Deller]

Julian Brown wrote:
> This patch implements the atomic builtins described at:
> 
>   http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/Atomic-Builtins.html
> 
> for ARM EABI Linux. This implementation uses the kernel helpers
> __kernel_cmpxchg and __kernel_dmb, and so should work on any
> architecture which supports those. (More-efficient versions are possible
> using ldrex/strex on architectures >=v6, but those are not written yet.)
> 
> Atomic operations are provided for data sizes of 1, 2 and 4 bytes (but
> not 8 bytes). The implementation uses actual functions
> (__sync_fetch_and_add_2, etc.) rather than expanding code inline.
> 
> Tested with cross to arm-none-linux-gnueabi, and with some additional
> hand-written tests which hopefully exercised the atomicity of the
> operations sufficiently.

Hi Julian,

I like the code, since I was just about to put something similar together
for the atomic builtins for the hppa/parisc architecture.

Basically I think I would be able to reuse all of your code, with only two
exceptions:
a) __kernel_cmpxchg() and __kernel_dmb() are different for hppa
b) You use "#ifdef __ARMEL__" to differentiate the endianess. Needs changing
to a generic solution.

But else, everything seems usable.
So, why not finding a way to share the code for various architectures? I'm
sure not only ARMEL and HPPA would need this?

I talked to Carlos O'Donell on IRC and he proposed:
- You could move it into a gcc/config/atomic.h internal header, which
defines all higher up sync operations as a function of lower level ops.
- Then gcc/config/target/linux-atomic.c can include that. 

Maybe there are other options as well?

Helge

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Andrew Haley]

Helge Deller wrote:
> Julian Brown wrote:
>> This patch implements the atomic builtins described at:
>>
>>   http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/Atomic-Builtins.html
>>
>> for ARM EABI Linux. This implementation uses the kernel helpers
>> __kernel_cmpxchg and __kernel_dmb, and so should work on any
>> architecture which supports those. (More-efficient versions are possible
>> using ldrex/strex on architectures >=v6, but those are not written yet.)
>>
>> Atomic operations are provided for data sizes of 1, 2 and 4 bytes (but
>> not 8 bytes). The implementation uses actual functions
>> (__sync_fetch_and_add_2, etc.) rather than expanding code inline.
>>
>> Tested with cross to arm-none-linux-gnueabi, and with some additional
>> hand-written tests which hopefully exercised the atomicity of the
>> operations sufficiently.
> 
> Hi Julian,
> 
> I like the code, since I was just about to put something similar together
> for the atomic builtins for the hppa/parisc architecture.
> 
> Basically I think I would be able to reuse all of your code, with only two
> exceptions:
> a) __kernel_cmpxchg() and __kernel_dmb() are different for hppa
> b) You use "#ifdef __ARMEL__" to differentiate the endianess. Needs changing
> to a generic solution.
> 
> But else, everything seems usable.
> So, why not finding a way to share the code for various architectures? I'm
> sure not only ARMEL and HPPA would need this?

Hmm.  This could turn into a horrible mess of #ifdefs for the various
architectures.

Andrew.

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Carlos O'Donell]

Andrew Haley wrote:
> Helge Deller wrote:
>> But else, everything seems usable.
>> So, why not finding a way to share the code for various architectures? 
>> I'm sure not only ARMEL and HPPA would need this?
> 
> Hmm.  This could turn into a horrible mess of #ifdefs for the various
> architectures.

There would be set of cpp conditionals per sync operation. You need to 
detect the following:

1. If the target implements the sync operation, do nothing.
2. If the target doesn't implement the sync operation, implement it 
using another sync operation e.g. __sync_val_compare_and_swap.

With the possibility of one more cpp conditional to disable #2.

Do you think that will get messy?

Cheers,
Carlos.
-- 
Carlos O'Donell
CodeSourcery
carlos@codesourcery.com
(650) 331-3385 x716

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Daniel Jacobowitz]

On Thu, Jul 10, 2008 at 10:36:13AM -0400, Carlos O'Donell wrote:
> Do you think that will get messy?

Yes.  I also think it's of limited use.  Out of line calls make sense
for the architectures which need kernel helpers, which on Linux I
think is ARM pre-v6K, hppa, and maybe coldfire.  Anyone with
instructions is going to want to use them inline.

-- 
Daniel Jacobowitz
CodeSourcery

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Mark Mitchell]

Daniel Jacobowitz wrote:
> On Thu, Jul 10, 2008 at 10:36:13AM -0400, Carlos O'Donell wrote:
>> Do you think that will get messy?
> 
> Yes.  I also think it's of limited use.  Out of line calls make sense
> for the architectures which need kernel helpers, which on Linux I
> think is ARM pre-v6K, hppa, and maybe coldfire.  Anyone with
> instructions is going to want to use them inline.

I think that if Julian's patch is OK for ARM, then we should put it in. 
  If parts of it can be reused on HPPA, ColdFire, etc., then we can 
refactor at that point.

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: [PATCH, HPPA] Atomic builtins using kernel helpers for Linux

[Helge Deller]

Mark Mitchell wrote:

> Daniel Jacobowitz wrote:
>> On Thu, Jul 10, 2008 at 10:36:13AM -0400, Carlos O'Donell wrote:
>>> Do you think that will get messy?
>> 
>> Yes.  I also think it's of limited use.  Out of line calls make sense
>> for the architectures which need kernel helpers, which on Linux I
>> think is ARM pre-v6K, hppa, and maybe coldfire.  Anyone with
>> instructions is going to want to use them inline.
> 
> I think that if Julian's patch is OK for ARM, then we should put it in.
>   If parts of it can be reused on HPPA, ColdFire, etc., then we can
> refactor at that point.

Below is the equivalent patch for HPPA.
As mentioned before, there are not much differences between this version and
the ARM version.

OK for mainline as well ?

Helge

ChangeLog

    gcc/
    * config/pa/t-linux (LIB2FUNCS_STATIC_EXTRA): Add
    config/pa/linux-atomic.c.
    * config/pa/t-linux64 (LIB2FUNCS_STATIC_EXTRA): Add
    config/pa/linux-atomic.c.
    * config/pa/linux-atomic.c: New.


Index: gcc/config/pa/linux-atomic.c
===================================================================
--- gcc/config/pa/linux-atomic.c        (revision 0)
+++ gcc/config/pa/linux-atomic.c        (revision 0)
@@ -0,0 +1,299 @@
+/* Linux-specific atomic operations for PA Linux.
+   Copyright (C) 2008 Free Software Foundation, Inc.
+   Based on code contributed by CodeSourcery for ARM EABI Linux.
+   Modifications for PA Linux by Helge Deller <deller@gmx.de>
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 2, or (at your option) any later
+version.
+
+In addition to the permissions in the GNU General Public License, the
+Free Software Foundation gives you unlimited permission to link the
+compiled version of this file into combinations with other programs,
+and to distribute those combinations without any restriction coming
+from the use of this file.  (The General Public License restrictions
+do apply in other respects; for example, they cover modification of
+the file, and distribution when not linked into a combine
+executable.)
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING.  If not, write to the Free
+Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
+02110-1301, USA.  */
+
+/* Determine kernel LWS function call (0=32bit, 1=64bit userspace)  */
+#define LWS_CAS (sizeof(unsigned long) == 4 ? 0 : 1)
+
+/* Kernel helper for compare-and-exchange.  */
+#define __kernel_cmpxchg( oldval, newval, mem )                                \
+  ({                                                                   \
+    register long lws_ret   asm("r28");                                        \
+    register long lws_errno asm("r21");                                        \
+    register unsigned long lws_mem asm("r26") = (unsigned long) (mem); \
+    register long lws_old asm("r25") = (oldval);                       \
+    register long lws_new asm("r24") = (newval);                       \
+    asm volatile(      "ble    0xb0(%%sr2, %%r0)       \n\t"           \
+                       "ldi    %5, %%r20               \n\t"           \
+       : "=r" (lws_ret), "=r" (lws_errno), "=r" (lws_mem),             \
+         "=r" (lws_old), "=r" (lws_new)                                \
+       : "i" (LWS_CAS), "2" (lws_mem), "3" (lws_old), "4" (lws_new)    \
+       : "r1", "r20", "r22", "r23", "r31", "memory"                    \
+    );                                                                         \
+    lws_errno;                                                         \
+   })
+
+/* Kernel helper for memory barrier.  */
+#define __kernel_dmb() asm volatile ( "" : : : "memory" );
+
+/* Note: we implement byte, short and int versions of atomic operations
using
+   the above kernel helpers, but there is no support for "long long"
(64-bit)
+   operations as yet.  */
+
+#define HIDDEN __attribute__ ((visibility ("hidden")))
+
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define INVERT_MASK_1 0
+#define INVERT_MASK_2 0
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#define INVERT_MASK_1 24
+#define INVERT_MASK_2 16
+#else
+#error "Endianess missing"
+#endif
+
+#define MASK_1 0xffu
+#define MASK_2 0xffffu
+
+#define FETCH_AND_OP_WORD(OP, PFX_OP, INF_OP)                          \
+  int HIDDEN                                                           \
+  __sync_fetch_and_##OP##_4 (int *ptr, int val)                                \
+  {                                                                    \
+    int failure, tmp;                                                  \
+                                                                       \
+    do {                                                               \
+      tmp = *ptr;                                                      \
+      failure = __kernel_cmpxchg (tmp, PFX_OP tmp INF_OP val, ptr);    \
+    } while (failure != 0);                                            \
+                                                                       \
+    return tmp;                                                                \
+  }
+
+FETCH_AND_OP_WORD (add,   , +)
+FETCH_AND_OP_WORD (sub,   , -)
+FETCH_AND_OP_WORD (or,    , |)
+FETCH_AND_OP_WORD (and,   , &)
+FETCH_AND_OP_WORD (xor,   , ^)
+FETCH_AND_OP_WORD (nand, ~, &)
+
+#define NAME_oldval(OP, WIDTH) __sync_fetch_and_##OP##_##WIDTH
+#define NAME_newval(OP, WIDTH) __sync_##OP##_and_fetch_##WIDTH
+
+/* Implement both __sync_<op>_and_fetch and __sync_fetch_and_<op> for
+   subword-sized quantities.  */
+
+#define SUBWORD_SYNC_OP(OP, PFX_OP, INF_OP, TYPE, WIDTH, RETURN)       \
+  TYPE HIDDEN                                                          \
+  NAME##_##RETURN (OP, WIDTH) (TYPE *ptr, TYPE val)                    \
+  {                                                                    \
+    int *wordptr = (int *) ((unsigned int) ptr & ~3);                  \
+    unsigned int mask, shift, oldval, newval;                          \
+    int failure;                                                       \
+                                                                       \
+    shift = (((unsigned int) ptr & 3) << 3) ^ INVERT_MASK_##WIDTH;     \
+    mask = MASK_##WIDTH << shift;                                      \
+                                                                       \
+    do {                                                               \
+      oldval = *wordptr;                                               \
+      newval = ((PFX_OP ((oldval & mask) >> shift)                     \
+                 INF_OP (unsigned int) val) << shift) & mask;          \
+      newval |= oldval & ~mask;                                                \
+      failure = __kernel_cmpxchg (oldval, newval, wordptr);            \
+    } while (failure != 0);                                            \
+                                                                       \
+    return (RETURN & mask) >> shift;                                   \
+  }
+
+SUBWORD_SYNC_OP (add,   , +, short, 2, oldval)
+SUBWORD_SYNC_OP (sub,   , -, short, 2, oldval)
+SUBWORD_SYNC_OP (or,    , |, short, 2, oldval)
+SUBWORD_SYNC_OP (and,   , &, short, 2, oldval)
+SUBWORD_SYNC_OP (xor,   , ^, short, 2, oldval)
+SUBWORD_SYNC_OP (nand, ~, &, short, 2, oldval)
+
+SUBWORD_SYNC_OP (add,   , +, char, 1, oldval)
+SUBWORD_SYNC_OP (sub,   , -, char, 1, oldval)
+SUBWORD_SYNC_OP (or,    , |, char, 1, oldval)
+SUBWORD_SYNC_OP (and,   , &, char, 1, oldval)
+SUBWORD_SYNC_OP (xor,   , ^, char, 1, oldval)
+SUBWORD_SYNC_OP (nand, ~, &, char, 1, oldval)
+
+#define OP_AND_FETCH_WORD(OP, PFX_OP, INF_OP)                          \
+  int HIDDEN                                                           \
+  __sync_##OP##_and_fetch_4 (int *ptr, int val)                                \
+  {                                                                    \
+    int tmp, failure;                                                  \
+                                                                       \
+    do {                                                               \
+      tmp = *ptr;                                                      \
+      failure = __kernel_cmpxchg (tmp, PFX_OP tmp INF_OP val, ptr);    \
+    } while (failure != 0);                                            \
+                                                                       \
+    return PFX_OP tmp INF_OP val;                                      \
+  }
+
+OP_AND_FETCH_WORD (add,   , +)
+OP_AND_FETCH_WORD (sub,   , -)
+OP_AND_FETCH_WORD (or,    , |)
+OP_AND_FETCH_WORD (and,   , &)
+OP_AND_FETCH_WORD (xor,   , ^)
+OP_AND_FETCH_WORD (nand, ~, &)
+
+SUBWORD_SYNC_OP (add,   , +, short, 2, newval)
+SUBWORD_SYNC_OP (sub,   , -, short, 2, newval)
+SUBWORD_SYNC_OP (or,    , |, short, 2, newval)
+SUBWORD_SYNC_OP (and,   , &, short, 2, newval)
+SUBWORD_SYNC_OP (xor,   , ^, short, 2, newval)
+SUBWORD_SYNC_OP (nand, ~, &, short, 2, newval)
+
+SUBWORD_SYNC_OP (add,   , +, char, 1, newval)
+SUBWORD_SYNC_OP (sub,   , -, char, 1, newval)
+SUBWORD_SYNC_OP (or,    , |, char, 1, newval)
+SUBWORD_SYNC_OP (and,   , &, char, 1, newval)
+SUBWORD_SYNC_OP (xor,   , ^, char, 1, newval)
+SUBWORD_SYNC_OP (nand, ~, &, char, 1, newval)
+
+int HIDDEN
+__sync_val_compare_and_swap_4 (int *ptr, int oldval, int newval)
+{
+  int actual_oldval, fail;
+    
+  while (1)
+    {
+      actual_oldval = *ptr;
+
+      if (oldval != actual_oldval)
+       return actual_oldval;
+
+      fail = __kernel_cmpxchg (actual_oldval, newval, ptr);
+  
+      if (!fail)
+        return oldval;
+    }
+}
+
+#define SUBWORD_VAL_CAS(TYPE, WIDTH)                                   \
+  TYPE HIDDEN                                                          \
+  __sync_val_compare_and_swap_##WIDTH (TYPE *ptr, TYPE oldval,         \
+                                      TYPE newval)                     \
+  {                                                                    \
+    int *wordptr = (int *)((unsigned int) ptr & ~3), fail;             \
+    unsigned int mask, shift, actual_oldval, actual_newval;            \
+                                                                       \
+    shift = (((unsigned int) ptr & 3) << 3) ^ INVERT_MASK_##WIDTH;     \
+    mask = MASK_##WIDTH << shift;                                      \
+                                                                       \
+    while (1)                                                          \
+      {                                                                        \
+       actual_oldval = *wordptr;                                       \
+                                                                       \
+       if (((actual_oldval & mask) >> shift) != (unsigned int) oldval) \
+          return (actual_oldval & mask) >> shift;                      \
+                                                                       \
+       actual_newval = (actual_oldval & ~mask)                         \
+                       | (((unsigned int) newval << shift) & mask);    \
+                                                                       \
+       fail = __kernel_cmpxchg (actual_oldval, actual_newval,          \
+                                wordptr);                              \
+                                                                       \
+       if (!fail)                                                      \
+          return oldval;                                               \
+      }                                                                        \
+  }
+
+SUBWORD_VAL_CAS (short, 2)
+SUBWORD_VAL_CAS (char,  1)
+
+typedef unsigned char bool;
+
+bool HIDDEN
+__sync_bool_compare_and_swap_4 (int *ptr, int oldval, int newval)
+{
+  int failure = __kernel_cmpxchg (oldval, newval, ptr);
+  return (failure == 0);
+}
+
+#define SUBWORD_BOOL_CAS(TYPE, WIDTH)                                  \
+  bool HIDDEN                                                          \
+  __sync_bool_compare_and_swap_##WIDTH (TYPE *ptr, TYPE oldval,                \
+                                       TYPE newval)                    \
+  {                                                                    \
+    TYPE actual_oldval                                                 \
+      = __sync_val_compare_and_swap_##WIDTH (ptr, oldval, newval);     \
+    return (oldval == actual_oldval);                                  \
+  }
+
+SUBWORD_BOOL_CAS (short, 2)
+SUBWORD_BOOL_CAS (char,  1)
+
+void HIDDEN
+__sync_synchronize (void)
+{
+  __kernel_dmb ();
+}
+
+int HIDDEN
+__sync_lock_test_and_set_4 (int *ptr, int val)
+{
+  int failure, oldval;
+
+  do {
+    oldval = *ptr;
+    failure = __kernel_cmpxchg (oldval, val, ptr);
+  } while (failure != 0);
+
+  return oldval;
+}
+
+#define SUBWORD_TEST_AND_SET(TYPE, WIDTH)                              \
+  TYPE HIDDEN                                                          \
+  __sync_lock_test_and_set_##WIDTH (TYPE *ptr, TYPE val)               \
+  {                                                                    \
+    int failure;                                                       \
+    unsigned int oldval, newval, shift, mask;                          \
+    int *wordptr = (int *) ((unsigned int) ptr & ~3);                  \
+                                                                       \
+    shift = (((unsigned int) ptr & 3) << 3) ^ INVERT_MASK_##WIDTH;     \
+    mask = MASK_##WIDTH << shift;                                      \
+                                                                       \
+    do {                                                               \
+      oldval = *wordptr;                                               \
+      newval = (oldval & ~mask)                                                \
+              | (((unsigned int) val << shift) & mask);                \
+      failure = __kernel_cmpxchg (oldval, newval, wordptr);            \
+    } while (failure != 0);                                            \
+                                                                       \
+    return (oldval & mask) >> shift;                                   \
+  }
+
+SUBWORD_TEST_AND_SET (short, 2)
+SUBWORD_TEST_AND_SET (char,  1)
+
+#define SYNC_LOCK_RELEASE(TYPE, WIDTH)                                 \
+  void HIDDEN                                                          \
+  __sync_lock_release_##WIDTH (TYPE *ptr)                              \
+  {                                                                    \
+    *ptr = 0;                                                          \
+    __kernel_dmb ();                                                   \
+  }
+
+SYNC_LOCK_RELEASE (int,   4)
+SYNC_LOCK_RELEASE (short, 2)
+SYNC_LOCK_RELEASE (char,  1)
Index: gcc/config/pa/t-linux64
===================================================================
--- gcc/config/pa/t-linux64     (revision 137753)
+++ gcc/config/pa/t-linux64     (working copy)
@@ -8,5 +8,7 @@
 # Actually, hppa64 is always PIC but adding -fPIC does no harm.
 CRTSTUFF_T_CFLAGS_S = -fPIC
 
+LIB2FUNCS_STATIC_EXTRA = $(srcdir)/config/pa/linux-atomic.c
+
 # Compile libgcc2.a as PIC.
 TARGET_LIBGCC2_CFLAGS = -fPIC -Dpa64=1 -DELF=1
Index: gcc/config/pa/t-linux
===================================================================
--- gcc/config/pa/t-linux       (revision 137753)
+++ gcc/config/pa/t-linux       (working copy)
@@ -9,6 +9,7 @@
 TARGET_LIBGCC2_CFLAGS = -fPIC -DELF=1 -DLINUX=1
 
 LIB2FUNCS_EXTRA=fptr.c
+LIB2FUNCS_STATIC_EXTRA = $(srcdir)/config/pa/linux-atomic.c
 
 fptr.c: $(srcdir)/config/pa/fptr.c
        rm -f fptr.c

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Julian Brown]

On Tue, 1 Jul 2008 17:46:32 +0100
Julian Brown <julian@codesourcery.com> wrote:

> Hi,
> 
> This patch implements the atomic builtins described at:
> 
>   http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/Atomic-Builtins.html

Ping? (This patch never received ARM-maintainer approval).

Julian

PING Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Andrew Haley]

Julian Brown wrote:
> Hi,
> 
> This patch implements the atomic builtins described at:
> 
>   http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/Atomic-Builtins.html
> 
> for ARM EABI Linux. This implementation uses the kernel helpers
> __kernel_cmpxchg and __kernel_dmb, and so should work on any
> architecture which supports those. (More-efficient versions are possible
> using ldrex/strex on architectures >=v6, but those are not written yet.)
> 
> Atomic operations are provided for data sizes of 1, 2 and 4 bytes (but
> not 8 bytes). The implementation uses actual functions
> (__sync_fetch_and_add_2, etc.) rather than expanding code inline.
> 
> Tested with cross to arm-none-linux-gnueabi, and with some additional
> hand-written tests which hopefully exercised the atomicity of the
> operations sufficiently.
> 
> OK for mainline?

http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html

What happened to this?

Andrew.

PING #2 [Was PING Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI]

[Andrew Haley]

> Julian Brown wrote:
>> Hi,
>>
>> This patch implements the atomic builtins described at:
>>
>>   http://gcc.gnu.org/onlinedocs/gcc-4.3.0/gcc/Atomic-Builtins.html
>>
>> for ARM EABI Linux. This implementation uses the kernel helpers
>> __kernel_cmpxchg and __kernel_dmb, and so should work on any
>> architecture which supports those. (More-efficient versions are possible
>> using ldrex/strex on architectures >=v6, but those are not written yet.)
>>
>> Atomic operations are provided for data sizes of 1, 2 and 4 bytes (but
>> not 8 bytes). The implementation uses actual functions
>> (__sync_fetch_and_add_2, etc.) rather than expanding code inline.
>>
>> Tested with cross to arm-none-linux-gnueabi, and with some additional
>> hand-written tests which hopefully exercised the atomicity of the
>> operations sufficiently.
>>
>> OK for mainline?
> 
> http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html
> 
> What happened to this?

Anyone?  Still not there...

Andrew.

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Paul Brook]

[http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html]
>     gcc/
>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
>     config/arm/linux-atomic.c.
>     * config/arm/linux-atomic.c: New.

Ok.

Paul

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Andrew Haley]

Paul Brook wrote:
> [http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html]
>>     gcc/
>>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
>>     config/arm/linux-atomic.c.
>>     * config/arm/linux-atomic.c: New.
> 
> Ok.

Ta for looking.  What's the status of this now?  Will it
be committed in 4.4 (since it was posted in July) or will it
be held back?

Andrew.

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Mark Mitchell]

Andrew Haley wrote:
> Paul Brook wrote:
>> [http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html]
>>>     gcc/
>>>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
>>>     config/arm/linux-atomic.c.
>>>     * config/arm/linux-atomic.c: New.
>> Ok.
> 
> Ta for looking.  What's the status of this now?  Will it
> be committed in 4.4 (since it was posted in July) or will it
> be held back?

That's at the discretion of the ARM maintainers (Paul et al.).  If they
feel it's sufficiently safe, it's OK to put it in now.

Thanks,

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Julian Brown]

On Tue, 02 Dec 2008 11:33:45 -0800
Mark Mitchell <mark@codesourcery.com> wrote:

> Andrew Haley wrote:
> > Paul Brook wrote:
> >> [http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html]
> >>>     gcc/
> >>>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
> >>>     config/arm/linux-atomic.c.
> >>>     * config/arm/linux-atomic.c: New.
> >> Ok.
> > 
> > Ta for looking.  What's the status of this now?  Will it
> > be committed in 4.4 (since it was posted in July) or will it
> > be held back?
> 
> That's at the discretion of the ARM maintainers (Paul et al.).  If
> they feel it's sufficiently safe, it's OK to put it in now.

Apologies, I've let this patch slip horribly. For avoidance of doubt,
do we (still?) want this on 4.4, or should I hold off until 4.5 now?

Thanks,

Julian

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Andrew Haley]

Julian Brown wrote:
> On Tue, 02 Dec 2008 11:33:45 -0800
> Mark Mitchell <mark@codesourcery.com> wrote:
> 
>> Andrew Haley wrote:
>>> Paul Brook wrote:
>>>> [http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html]
>>>>>     gcc/
>>>>>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
>>>>>     config/arm/linux-atomic.c.
>>>>>     * config/arm/linux-atomic.c: New.
>>>> Ok.
>>> Ta for looking.  What's the status of this now?  Will it
>>> be committed in 4.4 (since it was posted in July) or will it
>>> be held back?
>> That's at the discretion of the ARM maintainers (Paul et al.).  If
>> they feel it's sufficiently safe, it's OK to put it in now.
> 
> Apologies, I've let this patch slip horribly. For avoidance of doubt,
> do we (still?) want this on 4.4, or should I hold off until 4.5 now?

Since it's additional code, so nothing depends on it, and there were no
implementations of these routines before now, I suspect there is little
risk.

Andrew.

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Richard Earnshaw]

On Thu, 2009-01-22 at 15:01 +0000, Andrew Haley wrote:
> Julian Brown wrote:
> > On Tue, 02 Dec 2008 11:33:45 -0800
> > Mark Mitchell <mark@codesourcery.com> wrote:
> > 
> >> Andrew Haley wrote:
> >>> Paul Brook wrote:
> >>>> [http://gcc.gnu.org/ml/gcc-patches/2008-07/msg00025.html]
> >>>>>     gcc/
> >>>>>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
> >>>>>     config/arm/linux-atomic.c.
> >>>>>     * config/arm/linux-atomic.c: New.
> >>>> Ok.
> >>> Ta for looking.  What's the status of this now?  Will it
> >>> be committed in 4.4 (since it was posted in July) or will it
> >>> be held back?
> >> That's at the discretion of the ARM maintainers (Paul et al.).  If
> >> they feel it's sufficiently safe, it's OK to put it in now.
> > 
> > Apologies, I've let this patch slip horribly. For avoidance of doubt,
> > do we (still?) want this on 4.4, or should I hold off until 4.5 now?
> 
> Since it's additional code, so nothing depends on it, and there were no
> implementations of these routines before now, I suspect there is little
> risk.

On that basis: OK.

R.

Re: [PATCH, ARM] Atomic builtins using kernel helpers for Linux/EABI

[Mark Mitchell]

Richard Earnshaw wrote:

>>>>>>>     gcc/
>>>>>>>     * config/arm/t-linux-eabi (LIB2FUNCS_STATIC_EXTRA): Add
>>>>>>>     config/arm/linux-atomic.c.
>>>>>>>     * config/arm/linux-atomic.c: New.

> On that basis: OK.
> 
> R.

For avoidance of doubt, it's OK with me too, as RM.  (Richard doesn't
need my sign-off, but just in case anyone is concerned.)

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713