* [PATCH] x86_64: Add sinh with FMA
@ 2023-08-14 15:16 H.J. Lu
2023-08-14 15:50 ` Andreas Schwab
2023-08-16 14:45 ` Paul Zimmermann
0 siblings, 2 replies; 10+ messages in thread
From: H.J. Lu @ 2023-08-14 15:16 UTC (permalink / raw)
To: libc-alpha
On Skylake, it improves sinh bench performance by:
Before After Improvement
max 64.06 60.416 7%
min 10.874 9.674 11%
mean 14.0546 12.2018 13%
NB: Add
extern long double __expm1l (long double);
extern long double __expm1f128 (long double);
for __typeof (__expm1l) and __typeof (__expm1f128) when __expm1 is
defined since __expm1 may be expanded in their declarations which
causes the build failure.
---
sysdeps/ieee754/dbl-64/e_sinh.c | 7 +++++
sysdeps/x86_64/fpu/multiarch/Makefile | 2 ++
sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c | 12 +++++++++
sysdeps/x86_64/fpu/multiarch/e_sinh.c | 32 +++++++++++++++++++++++
4 files changed, 53 insertions(+)
create mode 100644 sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c
create mode 100644 sysdeps/x86_64/fpu/multiarch/e_sinh.c
diff --git a/sysdeps/ieee754/dbl-64/e_sinh.c b/sysdeps/ieee754/dbl-64/e_sinh.c
index b4b5857ddd..08611c94df 100644
--- a/sysdeps/ieee754/dbl-64/e_sinh.c
+++ b/sysdeps/ieee754/dbl-64/e_sinh.c
@@ -41,6 +41,11 @@ static char rcsid[] = "$NetBSD: e_sinh.c,v 1.7 1995/05/10 20:46:13 jtc Exp $";
static const double one = 1.0, shuge = 1.0e307;
+#ifndef SECTION
+# define SECTION
+#endif
+
+SECTION
double
__ieee754_sinh (double x)
{
@@ -90,4 +95,6 @@ __ieee754_sinh (double x)
/* |x| > overflowthresold, sinh(x) overflow */
return math_narrow_eval (x * shuge);
}
+#ifndef __ieee754_sinh
libm_alias_finite (__ieee754_sinh, __sinh)
+#endif
diff --git a/sysdeps/x86_64/fpu/multiarch/Makefile b/sysdeps/x86_64/fpu/multiarch/Makefile
index add339a876..15c44fc343 100644
--- a/sysdeps/x86_64/fpu/multiarch/Makefile
+++ b/sysdeps/x86_64/fpu/multiarch/Makefile
@@ -36,6 +36,7 @@ libm-sysdep_routines += \
e_log-fma \
e_log2-fma \
e_pow-fma \
+ e_sinh-fma \
s_atan-fma \
s_expm1-fma \
s_sin-fma \
@@ -49,6 +50,7 @@ CFLAGS-e_exp-fma.c = -mfma -mavx2
CFLAGS-e_log-fma.c = -mfma -mavx2
CFLAGS-e_log2-fma.c = -mfma -mavx2
CFLAGS-e_pow-fma.c = -mfma -mavx2
+CFLAGS-e_sinh-fma.c = -mfma -mavx2 -march=x86-64-v3
CFLAGS-s_atan-fma.c = -mfma -mavx2
CFLAGS-s_expm1-fma.c = -mfma -mavx2
CFLAGS-s_sin-fma.c = -mfma -mavx2
diff --git a/sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c b/sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c
new file mode 100644
index 0000000000..e0e1e39a7a
--- /dev/null
+++ b/sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c
@@ -0,0 +1,12 @@
+#define __ieee754_sinh __ieee754_sinh_fma
+#define __ieee754_exp __ieee754_exp_fma
+#define __expm1 __expm1_fma
+
+/* NB: __expm1 may be expanded to __expm1_fma in the following
+ prototypes. */
+extern long double __expm1l (long double);
+extern long double __expm1f128 (long double);
+
+#define SECTION __attribute__ ((section (".text.fma")))
+
+#include <sysdeps/ieee754/dbl-64/e_sinh.c>
diff --git a/sysdeps/x86_64/fpu/multiarch/e_sinh.c b/sysdeps/x86_64/fpu/multiarch/e_sinh.c
new file mode 100644
index 0000000000..4c42adeb21
--- /dev/null
+++ b/sysdeps/x86_64/fpu/multiarch/e_sinh.c
@@ -0,0 +1,32 @@
+/* Multiple versions of sinh.
+ Copyright (C) 2023 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <libm-alias-double.h>
+#include <libm-alias-finite.h>
+
+extern double __redirect_ieee754_sinh (double);
+
+#define SYMBOL_NAME ieee754_sinh
+#include "ifunc-fma.h"
+
+libc_ifunc_redirected (__redirect_ieee754_sinh, __ieee754_sinh,
+ IFUNC_SELECTOR ());
+libm_alias_finite (__ieee754_sinh, __sinh)
+
+#define __ieee754_sinh __ieee754_sinh_sse2
+#include <sysdeps/ieee754/dbl-64/e_sinh.c>
--
2.41.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-14 15:16 [PATCH] x86_64: Add sinh with FMA H.J. Lu
@ 2023-08-14 15:50 ` Andreas Schwab
2023-08-14 15:57 ` H.J. Lu
2023-08-16 14:45 ` Paul Zimmermann
1 sibling, 1 reply; 10+ messages in thread
From: Andreas Schwab @ 2023-08-14 15:50 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha; +Cc: H.J. Lu
On Aug 14 2023, H.J. Lu via Libc-alpha wrote:
> +/* NB: __expm1 may be expanded to __expm1_fma in the following
> + prototypes. */
Did you mean __expm1l?
> +extern long double __expm1l (long double);
> +extern long double __expm1f128 (long double);
--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-14 15:50 ` Andreas Schwab
@ 2023-08-14 15:57 ` H.J. Lu
2023-08-14 16:06 ` Andreas Schwab
0 siblings, 1 reply; 10+ messages in thread
From: H.J. Lu @ 2023-08-14 15:57 UTC (permalink / raw)
To: Andreas Schwab; +Cc: H.J. Lu via Libc-alpha
On Mon, Aug 14, 2023 at 8:50 AM Andreas Schwab <schwab@linux-m68k.org> wrote:
>
> On Aug 14 2023, H.J. Lu via Libc-alpha wrote:
>
> > +/* NB: __expm1 may be expanded to __expm1_fma in the following
> > + prototypes. */
>
> Did you mean __expm1l?
>
> > +extern long double __expm1l (long double);
> > +extern long double __expm1f128 (long double);
It looks like
extern long double expm1l (long double __x) __attribute__
((__nothrow__ )); extern long double __expm1_sse2l (long double __x)
__attribute__ ((__nothrow__ ));
extern _Float128 expm1f128 (_Float128 __x) __attribute__
((__nothrow__ )); extern _Float128 __expm1_sse2f128 (_Float128 __x)
__attribute__ ((__nothrow__ ));
where __expm1 is expanded to __expm1_sse2.
--
H.J.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-14 15:57 ` H.J. Lu
@ 2023-08-14 16:06 ` Andreas Schwab
0 siblings, 0 replies; 10+ messages in thread
From: Andreas Schwab @ 2023-08-14 16:06 UTC (permalink / raw)
To: H.J. Lu; +Cc: H.J. Lu via Libc-alpha
On Aug 14 2023, H.J. Lu wrote:
> On Mon, Aug 14, 2023 at 8:50 AM Andreas Schwab <schwab@linux-m68k.org> wrote:
>>
>> On Aug 14 2023, H.J. Lu via Libc-alpha wrote:
>>
>> > +/* NB: __expm1 may be expanded to __expm1_fma in the following
>> > + prototypes. */
>>
>> Did you mean __expm1l?
>>
>> > +extern long double __expm1l (long double);
>> > +extern long double __expm1f128 (long double);
>
> It looks like
>
> extern long double expm1l (long double __x) __attribute__
> ((__nothrow__ )); extern long double __expm1_sse2l (long double __x)
> __attribute__ ((__nothrow__ ));
> extern _Float128 expm1f128 (_Float128 __x) __attribute__
> ((__nothrow__ )); extern _Float128 __expm1_sse2f128 (_Float128 __x)
> __attribute__ ((__nothrow__ ));
>
> where __expm1 is expanded to __expm1_sse2.
Ok, expanded is confusing here since this is a term associated with the
preprocessor, but __expm1 doesn't occur as an identifier.
--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-14 15:16 [PATCH] x86_64: Add sinh with FMA H.J. Lu
2023-08-14 15:50 ` Andreas Schwab
@ 2023-08-16 14:45 ` Paul Zimmermann
2023-08-16 17:23 ` H.J. Lu
1 sibling, 1 reply; 10+ messages in thread
From: Paul Zimmermann @ 2023-08-16 14:45 UTC (permalink / raw)
To: H.J. Lu; +Cc: libc-alpha
Hi H.J.,
I get no improvement in accuracy. The largest known ulp error is still
the same:
sinh 0 -1 -0x1.633c654fee2bap+9 [2] [1.93] 1.92222 1.922214006544865
Maybe the FMA is not used for large inputs?
Best regards,
Paul
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-16 14:45 ` Paul Zimmermann
@ 2023-08-16 17:23 ` H.J. Lu
2023-08-17 19:15 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: H.J. Lu @ 2023-08-16 17:23 UTC (permalink / raw)
To: Paul Zimmermann; +Cc: libc-alpha
On Wed, Aug 16, 2023 at 7:45 AM Paul Zimmermann
<Paul.Zimmermann@inria.fr> wrote:
>
> Hi H.J.,
>
> I get no improvement in accuracy. The largest known ulp error is still
> the same:
>
> sinh 0 -1 -0x1.633c654fee2bap+9 [2] [1.93] 1.92222 1.922214006544865
>
> Maybe the FMA is not used for large inputs?
>
There is only one FMA insn in __ieee754_sinh_fma:
114: c4 e2 e9 9b 05 00 00 00 00 vfmsub132sd 0x0(%rip),%xmm2,%xmm0
Since it calls __ieee754_exp and __expm1, with
commit 1b214630ce6f7e0099b8b6f87246246739b079cf (master)
Author: H.J. Lu <hjl.tools@gmail.com>
Date: Fri Aug 11 08:04:08 2023 -0700
x86_64: Add expm1 with FMA
On Skylake, it improves expm1 bench performance by:
Before After Improvement
max 70.204 68.054 3%
min 20.709 16.2 22%
mean 22.1221 16.7367 24%
There is very little improvement with a single FMA insn.
I will drop this patch.
Thanks.
--
H.J.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-16 17:23 ` H.J. Lu
@ 2023-08-17 19:15 ` Florian Weimer
2023-08-17 19:21 ` H.J. Lu
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer @ 2023-08-17 19:15 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha; +Cc: Paul Zimmermann, H.J. Lu
* H. J. Lu via Libc-alpha:
> On Wed, Aug 16, 2023 at 7:45 AM Paul Zimmermann
> <Paul.Zimmermann@inria.fr> wrote:
>>
>> Hi H.J.,
>>
>> I get no improvement in accuracy. The largest known ulp error is still
>> the same:
>>
>> sinh 0 -1 -0x1.633c654fee2bap+9 [2] [1.93] 1.92222 1.922214006544865
>>
>> Maybe the FMA is not used for large inputs?
>>
>
> There is only one FMA insn in __ieee754_sinh_fma:
>
> 114: c4 e2 e9 9b 05 00 00 00 00 vfmsub132sd 0x0(%rip),%xmm2,%xmm0
>
> Since it calls __ieee754_exp and __expm1, with
>
> commit 1b214630ce6f7e0099b8b6f87246246739b079cf (master)
> Author: H.J. Lu <hjl.tools@gmail.com>
> Date: Fri Aug 11 08:04:08 2023 -0700
>
> x86_64: Add expm1 with FMA
>
> On Skylake, it improves expm1 bench performance by:
>
> Before After Improvement
> max 70.204 68.054 3%
> min 20.709 16.2 22%
> mean 22.1221 16.7367 24%
>
> There is very little improvement with a single FMA insn.
If you specialize the implementation for FMA, could you avoid the
indirect call to __ieee754_exp and __expm1?
Thanks,
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-17 19:15 ` Florian Weimer
@ 2023-08-17 19:21 ` H.J. Lu
2023-08-20 14:28 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: H.J. Lu @ 2023-08-17 19:21 UTC (permalink / raw)
To: Florian Weimer; +Cc: H.J. Lu via Libc-alpha, Paul Zimmermann
On Thu, Aug 17, 2023 at 12:15 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * H. J. Lu via Libc-alpha:
>
> > On Wed, Aug 16, 2023 at 7:45 AM Paul Zimmermann
> > <Paul.Zimmermann@inria.fr> wrote:
> >>
> >> Hi H.J.,
> >>
> >> I get no improvement in accuracy. The largest known ulp error is still
> >> the same:
> >>
> >> sinh 0 -1 -0x1.633c654fee2bap+9 [2] [1.93] 1.92222 1.922214006544865
> >>
> >> Maybe the FMA is not used for large inputs?
> >>
> >
> > There is only one FMA insn in __ieee754_sinh_fma:
> >
> > 114: c4 e2 e9 9b 05 00 00 00 00 vfmsub132sd 0x0(%rip),%xmm2,%xmm0
> >
> > Since it calls __ieee754_exp and __expm1, with
> >
> > commit 1b214630ce6f7e0099b8b6f87246246739b079cf (master)
> > Author: H.J. Lu <hjl.tools@gmail.com>
> > Date: Fri Aug 11 08:04:08 2023 -0700
> >
> > x86_64: Add expm1 with FMA
> >
> > On Skylake, it improves expm1 bench performance by:
> >
> > Before After Improvement
> > max 70.204 68.054 3%
> > min 20.709 16.2 22%
> > mean 22.1221 16.7367 24%
> >
> > There is very little improvement with a single FMA insn.
>
> If you specialize the implementation for FMA, could you avoid the
> indirect call to __ieee754_exp and __expm1?
>
That is true. Performance differences are
1. Non-FMA
"sinh": {
"": {
"duration": 1.60856e+09,
"iterations": 1.248e+08,
"max": 173.747,
"min": 10.706,
"mean": 12.8891
}
}%
2. FMA
"sinh": {
"": {
"duration": 1.61017e+09,
"iterations": 1.275e+08,
"max": 100.218,
"min": 9.678,
"mean": 12.6288
}
}%
FMA is a little faster. Should I repost it with updated performance
numbers?
--
H.J.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-17 19:21 ` H.J. Lu
@ 2023-08-20 14:28 ` Florian Weimer
2023-08-20 14:29 ` H.J. Lu
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer @ 2023-08-20 14:28 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha; +Cc: Florian Weimer, H.J. Lu, Paul Zimmermann
* H. J. Lu via Libc-alpha:
> On Thu, Aug 17, 2023 at 12:15 PM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * H. J. Lu via Libc-alpha:
>> > There is only one FMA insn in __ieee754_sinh_fma:
>> >
>> > 114: c4 e2 e9 9b 05 00 00 00 00 vfmsub132sd 0x0(%rip),%xmm2,%xmm0
>> >
>> > Since it calls __ieee754_exp and __expm1, with
>> >
>> > commit 1b214630ce6f7e0099b8b6f87246246739b079cf (master)
>> > Author: H.J. Lu <hjl.tools@gmail.com>
>> > Date: Fri Aug 11 08:04:08 2023 -0700
>> >
>> > x86_64: Add expm1 with FMA
>> >
>> > On Skylake, it improves expm1 bench performance by:
>> >
>> > Before After Improvement
>> > max 70.204 68.054 3%
>> > min 20.709 16.2 22%
>> > mean 22.1221 16.7367 24%
>> >
>> > There is very little improvement with a single FMA insn.
>>
>> If you specialize the implementation for FMA, could you avoid the
>> indirect call to __ieee754_exp and __expm1?
>>
>
> That is true. Performance differences are
>
> 1. Non-FMA
>
> "sinh": {
> "": {
> "duration": 1.60856e+09,
> "iterations": 1.248e+08,
> "max": 173.747,
> "min": 10.706,
> "mean": 12.8891
> }
> }%
>
> 2. FMA
>
> "sinh": {
> "": {
> "duration": 1.61017e+09,
> "iterations": 1.275e+08,
> "max": 100.218,
> "min": 9.678,
> "mean": 12.6288
> }
> }%
>
> FMA is a little faster. Should I repost it with updated performance
> numbers?
So the baseline is the before column from the original patch?
> On Skylake, it improves sinh bench performance by:
>
> Before After Improvement
> max 64.06 60.416 7%
> min 10.874 9.674 11%
> mean 14.0546 12.2018 13%
So we go from 14.0546 to 12.6288 for the mean case?
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] x86_64: Add sinh with FMA
2023-08-20 14:28 ` Florian Weimer
@ 2023-08-20 14:29 ` H.J. Lu
0 siblings, 0 replies; 10+ messages in thread
From: H.J. Lu @ 2023-08-20 14:29 UTC (permalink / raw)
To: Florian Weimer; +Cc: H.J. Lu via Libc-alpha, Florian Weimer, Paul Zimmermann
On Sun, Aug 20, 2023 at 7:28 AM Florian Weimer <fw@deneb.enyo.de> wrote:
>
> * H. J. Lu via Libc-alpha:
>
> > On Thu, Aug 17, 2023 at 12:15 PM Florian Weimer <fweimer@redhat.com> wrote:
> >>
> >> * H. J. Lu via Libc-alpha:
> >> > There is only one FMA insn in __ieee754_sinh_fma:
> >> >
> >> > 114: c4 e2 e9 9b 05 00 00 00 00 vfmsub132sd 0x0(%rip),%xmm2,%xmm0
> >> >
> >> > Since it calls __ieee754_exp and __expm1, with
> >> >
> >> > commit 1b214630ce6f7e0099b8b6f87246246739b079cf (master)
> >> > Author: H.J. Lu <hjl.tools@gmail.com>
> >> > Date: Fri Aug 11 08:04:08 2023 -0700
> >> >
> >> > x86_64: Add expm1 with FMA
> >> >
> >> > On Skylake, it improves expm1 bench performance by:
> >> >
> >> > Before After Improvement
> >> > max 70.204 68.054 3%
> >> > min 20.709 16.2 22%
> >> > mean 22.1221 16.7367 24%
> >> >
> >> > There is very little improvement with a single FMA insn.
> >>
> >> If you specialize the implementation for FMA, could you avoid the
> >> indirect call to __ieee754_exp and __expm1?
> >>
> >
> > That is true. Performance differences are
> >
> > 1. Non-FMA
> >
> > "sinh": {
> > "": {
> > "duration": 1.60856e+09,
> > "iterations": 1.248e+08,
> > "max": 173.747,
> > "min": 10.706,
> > "mean": 12.8891
> > }
> > }%
> >
> > 2. FMA
> >
> > "sinh": {
> > "": {
> > "duration": 1.61017e+09,
> > "iterations": 1.275e+08,
> > "max": 100.218,
> > "min": 9.678,
> > "mean": 12.6288
> > }
> > }%
> >
> > FMA is a little faster. Should I repost it with updated performance
> > numbers?
>
> So the baseline is the before column from the original patch?
>
> > On Skylake, it improves sinh bench performance by:
> >
> > Before After Improvement
> > max 64.06 60.416 7%
> > min 10.874 9.674 11%
> > mean 14.0546 12.2018 13%
>
> So we go from 14.0546 to 12.6288 for the mean case?
No. It is from 12.8891 to 12.6288.
--
H.J.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-08-20 14:30 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-14 15:16 [PATCH] x86_64: Add sinh with FMA H.J. Lu
2023-08-14 15:50 ` Andreas Schwab
2023-08-14 15:57 ` H.J. Lu
2023-08-14 16:06 ` Andreas Schwab
2023-08-16 14:45 ` Paul Zimmermann
2023-08-16 17:23 ` H.J. Lu
2023-08-17 19:15 ` Florian Weimer
2023-08-17 19:21 ` H.J. Lu
2023-08-20 14:28 ` Florian Weimer
2023-08-20 14:29 ` H.J. Lu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).