From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 1039) id DD7A83858C20; Mon, 14 Aug 2023 15:15:09 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DD7A83858C20 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1692026109; bh=RZLpoL4dhIqT4oteadG8u0ZISC6QMAMCrnyNPb+pleM=; h=From:To:Subject:Date:From; b=oJuE9Aa8VmDP7/rkB1Woz9UT0aT9i7tgmS+iyeyvXnlZTc68jsiIjFjU8D/UX7ybk IEs+ociIFCwiN9iNUXI/pBHsKpXvwkUF8drunE3CEpqtEe2bTrGEMEffJJ7nzKLglL 9IjSpPARQkxqqb2MC8z9RRUqaQdZ2mwPxeCczdfg= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: H.J. Lu To: glibc-cvs@sourceware.org Subject: [glibc] x86_64: Add expm1 with FMA X-Act-Checkin: glibc X-Git-Author: H.J. Lu X-Git-Refname: refs/heads/master X-Git-Oldrev: dd2794d6680d322754464448a25916ee199ce678 X-Git-Newrev: 1b214630ce6f7e0099b8b6f87246246739b079cf Message-Id: <20230814151509.DD7A83858C20@sourceware.org> Date: Mon, 14 Aug 2023 15:15:09 +0000 (GMT) List-Id: https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=1b214630ce6f7e0099b8b6f87246246739b079cf commit 1b214630ce6f7e0099b8b6f87246246739b079cf Author: H.J. Lu Date: Fri Aug 11 08:04:08 2023 -0700 x86_64: Add expm1 with FMA On Skylake, it improves expm1 bench performance by: Before After Improvement max 70.204 68.054 3% min 20.709 16.2 22% mean 22.1221 16.7367 24% NB: Add extern long double __expm1l (long double); extern long double __expm1f128 (long double); for __typeof (__expm1l) and __typeof (__expm1f128) when __expm1 is defined since __expm1 may be expanded in their declarations which causes the build failure. Diff: --- sysdeps/ieee754/dbl-64/s_expm1.c | 7 ++++++ sysdeps/x86_64/fpu/multiarch/Makefile | 2 ++ sysdeps/x86_64/fpu/multiarch/s_expm1-fma.c | 10 +++++++++ sysdeps/x86_64/fpu/multiarch/s_expm1.c | 36 ++++++++++++++++++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/sysdeps/ieee754/dbl-64/s_expm1.c b/sysdeps/ieee754/dbl-64/s_expm1.c index 8f1c95bd04..1cafeca9c0 100644 --- a/sysdeps/ieee754/dbl-64/s_expm1.c +++ b/sysdeps/ieee754/dbl-64/s_expm1.c @@ -130,6 +130,11 @@ static const double 4.00821782732936239552e-06, /* 3ED0CFCA 86E65239 */ -2.01099218183624371326e-07 }; /* BE8AFDB7 6E09C32D */ +#ifndef SECTION +# define SECTION +#endif + +SECTION double __expm1 (double x) { @@ -258,4 +263,6 @@ __expm1 (double x) } return y; } +#ifndef __expm1 libm_alias_double (__expm1, expm1) +#endif diff --git a/sysdeps/x86_64/fpu/multiarch/Makefile b/sysdeps/x86_64/fpu/multiarch/Makefile index f773255721..add339a876 100644 --- a/sysdeps/x86_64/fpu/multiarch/Makefile +++ b/sysdeps/x86_64/fpu/multiarch/Makefile @@ -37,6 +37,7 @@ libm-sysdep_routines += \ e_log2-fma \ e_pow-fma \ s_atan-fma \ + s_expm1-fma \ s_sin-fma \ s_sincos-fma \ s_tan-fma \ @@ -49,6 +50,7 @@ CFLAGS-e_log-fma.c = -mfma -mavx2 CFLAGS-e_log2-fma.c = -mfma -mavx2 CFLAGS-e_pow-fma.c = -mfma -mavx2 CFLAGS-s_atan-fma.c = -mfma -mavx2 +CFLAGS-s_expm1-fma.c = -mfma -mavx2 CFLAGS-s_sin-fma.c = -mfma -mavx2 CFLAGS-s_tan-fma.c = -mfma -mavx2 CFLAGS-s_sincos-fma.c = -mfma -mavx2 diff --git a/sysdeps/x86_64/fpu/multiarch/s_expm1-fma.c b/sysdeps/x86_64/fpu/multiarch/s_expm1-fma.c new file mode 100644 index 0000000000..3ee2bd804e --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/s_expm1-fma.c @@ -0,0 +1,10 @@ +#define __expm1 __expm1_fma + +/* NB: __expm1 may be expanded to __expm1_fma in the following + prototypes. */ +extern long double __expm1l (long double); +extern long double __expm1f128 (long double); + +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/s_expm1.c b/sysdeps/x86_64/fpu/multiarch/s_expm1.c new file mode 100644 index 0000000000..2cae83fb7f --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/s_expm1.c @@ -0,0 +1,36 @@ +/* Multiple versions of expm1. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +extern double __redirect_expm1 (double); + +#define SYMBOL_NAME expm1 +#include "ifunc-fma.h" + +libc_ifunc_redirected (__redirect_expm1, __expm1, IFUNC_SELECTOR ()); +libm_alias_double (__expm1, expm1) + +#define __expm1 __expm1_sse2 + +/* NB: __expm1 may be expanded to __expm1_sse2 in the following + prototypes. */ +extern long double __expm1l (long double); +extern long double __expm1f128 (long double); + +#include