From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 20922 invoked by alias); 14 Jun 2012 04:27:08 -0000 Received: (qmail 20796 invoked by uid 22791); 14 Jun 2012 04:27:04 -0000 X-SWARE-Spam-Status: No, hits=-3.9 required=5.0 tests=AWL,BAYES_00,KHOP_RCVD_UNTRUST,RCVD_IN_HOSTKARMA_W,RCVD_IN_HOSTKARMA_WL X-Spam-Check-By: sourceware.org Received: from relay1.mentorg.com (HELO relay1.mentorg.com) (192.94.38.131) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 14 Jun 2012 04:26:50 +0000 Received: from svr-orw-fem-01.mgc.mentorg.com ([147.34.98.93]) by relay1.mentorg.com with esmtp id 1Sf1dx-0007ef-8C from Maxim_Kuvyrkov@mentor.com ; Wed, 13 Jun 2012 21:26:49 -0700 Received: from SVR-IES-FEM-01.mgc.mentorg.com ([137.202.0.104]) by svr-orw-fem-01.mgc.mentorg.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Wed, 13 Jun 2012 21:26:48 -0700 Received: from [127.0.0.1] (137.202.0.76) by SVR-IES-FEM-01.mgc.mentorg.com (137.202.0.104) with Microsoft SMTP Server id 14.1.289.1; Thu, 14 Jun 2012 05:26:46 +0100 From: Maxim Kuvyrkov Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: [PATCH 1/3, MIPS] Rewrite MIPS' atomic.h to use __atomic_* builtins. Date: Thu, 14 Jun 2012 04:27:00 -0000 Message-ID: CC: , Richard Sandiford To: "Joseph S. Myers" MIME-Version: 1.0 (Apple Message framework v1278) Mailing-List: contact libc-ports-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: libc-ports-owner@sourceware.org X-SW-Source: 2012-06/txt/msg00028.txt.bz2 This patch rewrites MIPS' atomic.h to use __atomic_* builtins instead of in= line assembly. These builtins are available in recent version of GCC and c= orrespond to C++11 memory model support, they also map very well to GLIBC's= atomic_* macros. With the GCC patches posted here [*] applied, the compiler will generate sa= me, or better, assembly code for the atomic macros. XLP processors in part= icular will see a significant boost as GCC will use XLP-specific SWAP and L= DADD instructions for some of the macros instead of LL/SC sequences. This patch was tested on XLP with no regressions; testing on a non-XLP plat= form is in progress. Testing was done using GCC mainline with [*] patches = applied. OK to apply once 2.16 branches? Thank you, -- Maxim Kuvyrkov CodeSourcery / Mentor Graphics 2012-06-14 Tom de Vries Maxim Kuvyrkov * sysdeps/mips/bit/atomic.h: Rewrite using __atomic_* builtins. --- sysdeps/mips/bits/atomic.h | 297 +++++++++++++++++-----------------------= ---- 1 files changed, 114 insertions(+), 183 deletions(-) diff --git a/sysdeps/mips/bits/atomic.h b/sysdeps/mips/bits/atomic.h index 4d51d7f..99d5db1 100644 --- a/sysdeps/mips/bits/atomic.h +++ b/sysdeps/mips/bits/atomic.h @@ -1,5 +1,5 @@ /* Low-level functions for atomic operations. Mips version. - Copyright (C) 2005 Free Software Foundation, Inc. + Copyright (C) 2005-2012 Free Software Foundation, Inc. This file is part of the GNU C Library. =20 The GNU C Library is free software; you can redistribute it and/or @@ -78,243 +78,174 @@ typedef uintmax_t uatomic_max_t; #define MIPS_SYNC_STR_1(X) MIPS_SYNC_STR_2(X) #define MIPS_SYNC_STR MIPS_SYNC_STR_1(MIPS_SYNC) =20 -/* Compare and exchange. For all of the "xxx" routines, we expect a - "__prev" and a "__cmp" variable to be provided by the enclosing scope, - in which values are returned. */ - -#define __arch_compare_and_exchange_xxx_8_int(mem, newval, oldval, rel, ac= q) \ - (abort (), __prev =3D __cmp =3D 0) - -#define __arch_compare_and_exchange_xxx_16_int(mem, newval, oldval, rel, a= cq) \ - (abort (), __prev =3D __cmp =3D 0) - -#define __arch_compare_and_exchange_xxx_32_int(mem, newval, oldval, rel, a= cq) \ - __asm__ __volatile__ ( \ - ".set push\n\t" \ - MIPS_PUSH_MIPS2 \ - rel "\n" \ - "1:\t" \ - "ll %0,%5\n\t" \ - "move %1,$0\n\t" \ - "bne %0,%3,2f\n\t" \ - "move %1,%4\n\t" \ - "sc %1,%2\n\t" \ - R10K_BEQZ_INSN" %1,1b\n" \ - acq "\n\t" \ - ".set pop\n" \ - "2:\n\t" \ - : "=3D&r" (__prev), "=3D&r" (__cmp), "=3Dm" (*mem) \ - : "r" (oldval), "r" (newval), "m" (*mem) \ - : "memory") +/* Compare and exchange. + For all "bool" routines, we return FALSE if exchange succesful. */ =20 -#if _MIPS_SIM =3D=3D _ABIO32 -/* We can't do an atomic 64-bit operation in O32. */ -#define __arch_compare_and_exchange_xxx_64_int(mem, newval, oldval, rel, a= cq) \ - (abort (), __prev =3D __cmp =3D 0) -#else -#define __arch_compare_and_exchange_xxx_64_int(mem, newval, oldval, rel, a= cq) \ - __asm__ __volatile__ ("\n" \ - ".set push\n\t" \ - MIPS_PUSH_MIPS2 \ - rel "\n" \ - "1:\t" \ - "lld %0,%5\n\t" \ - "move %1,$0\n\t" \ - "bne %0,%3,2f\n\t" \ - "move %1,%4\n\t" \ - "scd %1,%2\n\t" \ - R10K_BEQZ_INSN" %1,1b\n" \ - acq "\n\t" \ - ".set pop\n" \ - "2:\n\t" \ - : "=3D&r" (__prev), "=3D&r" (__cmp), "=3Dm" (*mem) \ - : "r" (oldval), "r" (newval), "m" (*mem) \ - : "memory") -#endif +#define __arch_compare_and_exchange_bool_acq_8_int(mem, newval, oldval) \ + (abort (), 0) =20 -/* For all "bool" routines, we return FALSE if exchange succesful. */ +#define __arch_compare_and_exchange_bool_rel_8_int(mem, newval, oldval) \ + (abort (), 0) =20 -#define __arch_compare_and_exchange_bool_8_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_8_int(mem, new, old, rel, acq); \ - !__cmp; }) +#define __arch_compare_and_exchange_bool_acq_16_int(mem, newval, oldval) \ + (abort (), 0) =20 -#define __arch_compare_and_exchange_bool_16_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_16_int(mem, new, old, rel, acq); \ - !__cmp; }) +#define __arch_compare_and_exchange_bool_rel_16_int(mem, newval, oldval) \ + (abort (), 0) =20 -#define __arch_compare_and_exchange_bool_32_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_32_int(mem, new, old, rel, acq); \ - !__cmp; }) +#define __arch_compare_and_exchange_bool_acq_32_int(mem, newval, oldval) \ + ({ \ + typeof (*mem) __oldval =3D (oldval); \ + !__atomic_compare_exchange_n (mem, &__oldval, newval, 0, \ + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED); \ + }) + +#define __arch_compare_and_exchange_bool_rel_32_int(mem, newval, oldval) \ + ({ \ + typeof (*mem) __oldval =3D (oldval); \ + !__atomic_compare_exchange_n (mem, &__oldval, newval, 0, \ + __ATOMIC_RELEASE, __ATOMIC_RELAXED); \ + }) + +#define __arch_compare_and_exchange_val_acq_8_int(mem, newval, oldval) \ + (abort (), 0) + +#define __arch_compare_and_exchange_val_rel_8_int(mem, newval, oldval) \ + (abort (), 0) + +#define __arch_compare_and_exchange_val_acq_16_int(mem, newval, oldval) \ + (abort (), 0) + +#define __arch_compare_and_exchange_val_rel_16_int(mem, newval, oldval) \ + (abort (), 0) =20 -#define __arch_compare_and_exchange_bool_64_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_64_int(mem, new, old, rel, acq); \ - !__cmp; }) +#define __arch_compare_and_exchange_val_acq_32_int(mem, newval, oldval) \ + ({ \ + typeof (*mem) __oldval =3D (oldval); \ + __atomic_compare_exchange_n (mem, &__oldval, newval, 0, \ + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED); \ + __oldval; \ + }) + +#define __arch_compare_and_exchange_val_rel_32_int(mem, newval, oldval) \ + ({ \ + typeof (*mem) __oldval =3D (oldval); \ + __atomic_compare_exchange_n (mem, &__oldval, newval, 0, \ + __ATOMIC_RELEASE, __ATOMIC_RELAXED); \ + __oldval; \ + }) =20 -/* For all "val" routines, return the old value whether exchange - successful or not. */ +#if _MIPS_SIM =3D=3D _ABIO32 + /* We can't do an atomic 64-bit operation in O32. */ +# define __arch_compare_and_exchange_bool_acq_64_int(mem, newval, oldval) \ + (abort (), 0) +# define __arch_compare_and_exchange_bool_rel_64_int(mem, newval, oldval) \ + (abort (), 0) +# define __arch_compare_and_exchange_val_acq_64_int(mem, newval, oldval) \ + (abort (), 0) +# define __arch_compare_and_exchange_val_rel_64_int(mem, newval, oldval) \ + (abort (), 0) +#else +# define __arch_compare_and_exchange_bool_acq_64_int(mem, newval, oldval) \ + __arch_compare_and_exchange_bool_acq_32_int (mem, newval, oldval) =20 -#define __arch_compare_and_exchange_val_8_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_8_int(mem, new, old, rel, acq); \ - (typeof (*mem))__prev; }) +# define __arch_compare_and_exchange_bool_rel_64_int(mem, newval, oldval) \ + __arch_compare_and_exchange_bool_rel_32_int (mem, newval, oldval) =20 -#define __arch_compare_and_exchange_val_16_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_16_int(mem, new, old, rel, acq); \ - (typeof (*mem))__prev; }) +# define __arch_compare_and_exchange_val_acq_64_int(mem, newval, oldval) \ + __arch_compare_and_exchange_val_acq_32_int (mem, newval, oldval) =20 -#define __arch_compare_and_exchange_val_32_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_32_int(mem, new, old, rel, acq); \ - (typeof (*mem))__prev; }) +# define __arch_compare_and_exchange_val_rel_64_int(mem, newval, oldval) \ + __arch_compare_and_exchange_val_rel_32_int (mem, newval, oldval) =20 -#define __arch_compare_and_exchange_val_64_int(mem, new, old, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __arch_compare_and_exchange_xxx_64_int(mem, new, old, rel, acq); \ - (typeof (*mem))__prev; }) +#endif =20 /* Compare and exchange with "acquire" semantics, ie barrier after. */ =20 -#define atomic_compare_and_exchange_bool_acq(mem, new, old) \ - __atomic_bool_bysize (__arch_compare_and_exchange_bool, int, \ - mem, new, old, "", MIPS_SYNC_STR) +#define atomic_compare_and_exchange_bool_acq(mem, new, old) \ + (__atomic_bool_bysize (__arch_compare_and_exchange_bool_acq, int, \ + mem, new, old)) =20 -#define atomic_compare_and_exchange_val_acq(mem, new, old) \ - __atomic_val_bysize (__arch_compare_and_exchange_val, int, \ - mem, new, old, "", MIPS_SYNC_STR) +#define atomic_compare_and_exchange_val_acq(mem, new, old) \ + __atomic_val_bysize (__arch_compare_and_exchange_val_acq, int, \ + mem, new, old) =20 /* Compare and exchange with "release" semantics, ie barrier before. */ =20 -#define atomic_compare_and_exchange_bool_rel(mem, new, old) \ - __atomic_bool_bysize (__arch_compare_and_exchange_bool, int, \ - mem, new, old, MIPS_SYNC_STR, "") - -#define atomic_compare_and_exchange_val_rel(mem, new, old) \ - __atomic_val_bysize (__arch_compare_and_exchange_val, int, \ - mem, new, old, MIPS_SYNC_STR, "") +#define atomic_compare_and_exchange_bool_rel(mem, new, old) \ + (__atomic_bool_bysize (__arch_compare_and_exchange_bool_rel, int, \ + mem, new, old)) =20 +#define atomic_compare_and_exchange_val_rel(mem, new, old) \ + __atomic_val_bysize (__arch_compare_and_exchange_val_rel, int, \ + mem, new, old) =20 =20 /* Atomic exchange (without compare). */ =20 -#define __arch_exchange_xxx_8_int(mem, newval, rel, acq) \ +#define __arch_exchange_acq_8_int(mem, newval) \ (abort (), 0) =20 -#define __arch_exchange_xxx_16_int(mem, newval, rel, acq) \ +#define __arch_exchange_rel_8_int(mem, newval) \ (abort (), 0) =20 -#define __arch_exchange_xxx_32_int(mem, newval, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __asm__ __volatile__ ("\n" \ - ".set push\n\t" \ - MIPS_PUSH_MIPS2 \ - rel "\n" \ - "1:\t" \ - "ll %0,%4\n\t" \ - "move %1,%3\n\t" \ - "sc %1,%2\n\t" \ - R10K_BEQZ_INSN" %1,1b\n" \ - acq "\n\t" \ - ".set pop\n" \ - "2:\n\t" \ - : "=3D&r" (__prev), "=3D&r" (__cmp), "=3Dm" (*mem) \ - : "r" (newval), "m" (*mem) \ - : "memory"); \ - __prev; }) +#define __arch_exchange_acq_16_int(mem, newval) \ + (abort (), 0) + +#define __arch_exchange_rel_16_int(mem, newval) \ + (abort (), 0) + +#define __arch_exchange_acq_32_int(mem, newval) \ + __atomic_exchange_n (mem, newval, __ATOMIC_ACQUIRE) + +#define __arch_exchange_rel_32_int(mem, newval) \ + __atomic_exchange_n (mem, newval, __ATOMIC_RELEASE) =20 #if _MIPS_SIM =3D=3D _ABIO32 /* We can't do an atomic 64-bit operation in O32. */ -#define __arch_exchange_xxx_64_int(mem, newval, rel, acq) \ +# define __arch_exchange_acq_64_int(mem, newval) \ + (abort (), 0) +# define __arch_exchange_rel_64_int(mem, newval) \ (abort (), 0) #else -#define __arch_exchange_xxx_64_int(mem, newval, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __asm__ __volatile__ ("\n" \ - ".set push\n\t" \ - MIPS_PUSH_MIPS2 \ - rel "\n" \ - "1:\n" \ - "lld %0,%4\n\t" \ - "move %1,%3\n\t" \ - "scd %1,%2\n\t" \ - R10K_BEQZ_INSN" %1,1b\n" \ - acq "\n\t" \ - ".set pop\n" \ - "2:\n\t" \ - : "=3D&r" (__prev), "=3D&r" (__cmp), "=3Dm" (*mem) \ - : "r" (newval), "m" (*mem) \ - : "memory"); \ - __prev; }) +# define __arch_exchange_acq_64_int(mem, newval) \ + __atomic_exchange_n (mem, newval, __ATOMIC_ACQUIRE) + +# define __arch_exchange_rel_64_int(mem, newval) \ + __atomic_exchange_n (mem, newval, __ATOMIC_RELEASE) #endif =20 #define atomic_exchange_acq(mem, value) \ - __atomic_val_bysize (__arch_exchange_xxx, int, mem, value, "", MIPS_SYNC= _STR) + __atomic_val_bysize (__arch_exchange_acq, int, mem, value) =20 #define atomic_exchange_rel(mem, value) \ - __atomic_val_bysize (__arch_exchange_xxx, int, mem, value, MIPS_SYNC_STR= , "") + __atomic_val_bysize (__arch_exchange_rel, int, mem, value) =20 =20 /* Atomically add value and return the previous (unincremented) value. */ =20 -#define __arch_exchange_and_add_8_int(mem, newval, rel, acq) \ +#define __arch_exchange_and_add_8_int(mem, newval) \ (abort (), (typeof(*mem)) 0) =20 -#define __arch_exchange_and_add_16_int(mem, newval, rel, acq) \ +#define __arch_exchange_and_add_16_int(mem, newval) \ (abort (), (typeof(*mem)) 0) =20 -#define __arch_exchange_and_add_32_int(mem, value, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __asm__ __volatile__ ("\n" \ - ".set push\n\t" \ - MIPS_PUSH_MIPS2 \ - rel "\n" \ - "1:\t" \ - "ll %0,%4\n\t" \ - "addu %1,%0,%3\n\t" \ - "sc %1,%2\n\t" \ - R10K_BEQZ_INSN" %1,1b\n" \ - acq "\n\t" \ - ".set pop\n" \ - "2:\n\t" \ - : "=3D&r" (__prev), "=3D&r" (__cmp), "=3Dm" (*mem) \ - : "r" (value), "m" (*mem) \ - : "memory"); \ - __prev; }) +#define __arch_exchange_and_add_32_int(mem, value) \ + __atomic_fetch_add (mem, value, __ATOMIC_ACQ_REL) =20 #if _MIPS_SIM =3D=3D _ABIO32 /* We can't do an atomic 64-bit operation in O32. */ -#define __arch_exchange_and_add_64_int(mem, value, rel, acq) \ +# define __arch_exchange_and_add_64_int(mem, value) \ (abort (), (typeof(*mem)) 0) #else -#define __arch_exchange_and_add_64_int(mem, value, rel, acq) \ -({ typeof (*mem) __prev; int __cmp; \ - __asm__ __volatile__ ( \ - ".set push\n\t" \ - MIPS_PUSH_MIPS2 \ - rel "\n" \ - "1:\t" \ - "lld %0,%4\n\t" \ - "daddu %1,%0,%3\n\t" \ - "scd %1,%2\n\t" \ - R10K_BEQZ_INSN" %1,1b\n" \ - acq "\n\t" \ - ".set pop\n" \ - "2:\n\t" \ - : "=3D&r" (__prev), "=3D&r" (__cmp), "=3Dm" (*mem) \ - : "r" (value), "m" (*mem) \ - : "memory"); \ - __prev; }) +# define __arch_exchange_and_add_64_int(mem, value) \ + __atomic_fetch_add (mem, value, __ATOMIC_ACQ_REL) #endif =20 /* ??? Barrier semantics for atomic_exchange_and_add appear to be=20 undefined. Use full barrier for now, as that's safe. */ #define atomic_exchange_and_add(mem, value) \ - __atomic_val_bysize (__arch_exchange_and_add, int, mem, value, \ - MIPS_SYNC_STR, MIPS_SYNC_STR) + __atomic_val_bysize (__arch_exchange_and_add, int, mem, value) =20 /* TODO: More atomic operations could be implemented efficiently; only the basic requirements are done. */ --=20 1.7.4.1