* [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686
@ 2022-05-31 21:34 Adhemerval Zanella
2022-05-31 21:34 ` [PATCH 1/5] benchtests: Add workload name for cosf Adhemerval Zanella
` (5 more replies)
0 siblings, 6 replies; 19+ messages in thread
From: Adhemerval Zanella @ 2022-05-31 21:34 UTC (permalink / raw)
To: libc-alpha, H . J . Lu
Performance seems to be similar, with generic implementation showing
slight better performance for sinf large inputs and for sincosf.
Although generic implementation shows slight lower precision (the
ulp data required some adjustments), the code size shows a good
improvement:
i686-linux-gnu-master$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
math/s_sincosf-sse2.os
text data bss dec hex filename
1555 0 0 1555 613 math/s_sinf-sse2.os
1551 0 0 1551 60f math/s_cosf-sse2.os
1754 0 0 1754 6da math/s_sincosf-sse2.os
i686-linux-gnu-patched$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
math/s_sincosf-sse2.os
text data bss dec hex filename
1059 0 0 1059 423 math/s_sinf-sse2.os
1067 0 0 1067 42b math/s_cosf-sse2.os
993 0 0 993 3e1 math/s_sincosf-sse2.os
And it also simplifies the code base a lot with less assembly
implementations.
Adhemerval Zanella (5):
benchtests: Add workload name for cosf
i686: Use generic cosf implementation for SSE2 version
i686: Use generic sinf implementation for SSE2 version
benchtests: Add workload name for sincosf
math: Use generic sinf implementation for SSE2 version
benchtests/cosf-inputs | 2 +-
benchtests/sincosf-inputs | 1 +
sysdeps/i386/i686/fpu/multiarch/Makefile | 3 +
.../i386/i686/fpu/multiarch/libm-test-ulps | 11 +
sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S | 552 -----------------
sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c | 3 +
.../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
.../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S | 565 -----------------
sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c | 3 +
sysdeps/ieee754/flt-32/s_cosf.c | 5 +
sysdeps/ieee754/flt-32/s_sinf.c | 5 +
12 files changed, 35 insertions(+), 1703 deletions(-)
delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
--
2.34.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 1/5] benchtests: Add workload name for cosf
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
@ 2022-05-31 21:34 ` Adhemerval Zanella
2022-05-31 21:47 ` H.J. Lu
2022-05-31 21:34 ` [PATCH 2/5] i686: Use generic cosf implementation for SSE2 version Adhemerval Zanella
` (4 subsequent siblings)
5 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-05-31 21:34 UTC (permalink / raw)
To: libc-alpha, H . J . Lu
So it can show both reciprocal-throughput and latency.
---
benchtests/cosf-inputs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/benchtests/cosf-inputs b/benchtests/cosf-inputs
index 73d6991cbc..6fba27833a 100644
--- a/benchtests/cosf-inputs
+++ b/benchtests/cosf-inputs
@@ -1,7 +1,7 @@
## args: float
## ret: float
## includes: math.h
-
+## name: workload-random
-0x1.d24c56p-1
-0x1.50ab58p-1
0x1.112339p-6
--
2.34.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 2/5] i686: Use generic cosf implementation for SSE2 version
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
2022-05-31 21:34 ` [PATCH 1/5] benchtests: Add workload name for cosf Adhemerval Zanella
@ 2022-05-31 21:34 ` Adhemerval Zanella
2022-05-31 21:46 ` H.J. Lu
2022-05-31 21:35 ` [PATCH 3/5] i686: Use generic sinf " Adhemerval Zanella
` (3 subsequent siblings)
5 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-05-31 21:34 UTC (permalink / raw)
To: libc-alpha, H . J . Lu
Performance seems to be similar (gcc 11.2.1 on a Ryzen 9 5900X):
* s_cosf-sse2.S:
"cosf": {
"workload-random": {
"duration": 3.74987e+09,
"iterations": 9.616e+07,
"reciprocal-throughput": 15.8141,
"latency": 62.1782,
"max-throughput": 6.32346e+07,
"min-throughput": 1.60828e+07
}
}
* generic s_cosf.c:
"cosf": {
"workload-random": {
"duration": 3.87298e+09,
"iterations": 1.00968e+08,
"reciprocal-throughput": 18.3448,
"latency": 58.3722,
"max-throughput": 5.45113e+07,
"min-throughput": 1.71314e+07
}
}
Checked on i686-linux-gnu.
---
sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
.../i386/i686/fpu/multiarch/libm-test-ulps | 4 +
sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S | 552 ------------------
sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c | 3 +
sysdeps/ieee754/flt-32/s_cosf.c | 5 +
5 files changed, 13 insertions(+), 552 deletions(-)
delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
index c0fa9761d3..b8f52c311f 100644
--- a/sysdeps/i386/i686/fpu/multiarch/Makefile
+++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
@@ -7,4 +7,5 @@ CFLAGS-e_expf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-e_log2f-sse2.c = -msse2 -mfpmath=sse
CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
+CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
endif
diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
index cc21e6907f..1bfad77adc 100644
--- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
+++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
@@ -746,21 +746,25 @@ ldouble: 1
Function: "cos":
double: 1
+float: 1
float128: 2
ldouble: 1
Function: "cos_downward":
double: 1
+float: 1
float128: 3
ldouble: 3
Function: "cos_towardzero":
double: 1
+float: 1
float128: 1
ldouble: 2
Function: "cos_upward":
double: 1
+float: 1
float128: 2
ldouble: 2
diff --git a/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
deleted file mode 100644
index 28cc68dbc7..0000000000
--- a/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
+++ /dev/null
@@ -1,552 +0,0 @@
-/* Optimized with sse2 version of cosf
- Copyright (C) 2012-2022 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <https://www.gnu.org/licenses/>. */
-
-#include <sysdep.h>
-#include <errno.h>
-
-/* Short algorithm description:
- *
- * 1) if |x| == 0: return 1.0-|x|.
- * 2) if |x| < 2^-27: return 1.0-|x|.
- * 3) if |x| < 2^-5 : return 1.0+x^2*DP_COS2_0+x^5*DP_COS2_1.
- * 4) if |x| < Pi/4: return 1.0+x^2*(C0+x^2*(C1+x^2*(C2+x^2*(C3+x^2*C4)))).
- * 5) if |x| < 9*Pi/4:
- * 5.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+3,
- * t=|x|-j*Pi/4.
- * 5.2) Reconstruction:
- * s = (-1.0)^((n>>2)&1)
- * if(n&2 != 0) {
- * using cos(t) polynomial for |t|<Pi/4, result is
- * s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4))))).
- * } else {
- * using sin(t) polynomial for |t|<Pi/4, result is
- * s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4))))).
- * }
- * 6) if |x| < 2^23, large args:
- * 6.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+3,
- * t=|x|-j*Pi/4.
- * 6.2) Reconstruction same as (5.2).
- * 7) if |x| >= 2^23, very large args:
- * 7.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+3,
- * t=|x|-j*Pi/4.
- * 7.2) Reconstruction same as (5.2).
- * 8) if x is Inf, return x-x, and set errno=EDOM.
- * 9) if x is NaN, return x-x.
- *
- * Special cases:
- * cos(+-0) = 1 not raising inexact,
- * cos(subnormal) raises inexact,
- * cos(min_normalized) raises inexact,
- * cos(normalized) raises inexact,
- * cos(Inf) = NaN, raises invalid, sets errno to EDOM,
- * cos(NaN) = NaN.
- */
-
-#ifdef PIC
-# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
-# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
-# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
-# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
-# define PUSH(REG) pushl REG; CFI_PUSH(REG)
-# define POP(REG) popl REG; CFI_POP(REG)
-# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
-# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
-# define ARG_X 8(%esp)
-#else
-# define MO1(symbol) L(symbol)
-# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
-# define ENTRANCE
-# define RETURN ret
-# define ARG_X 4(%esp)
-#endif
-
- .text
-ENTRY(__cosf_sse2)
- /* Input: single precision x on stack at address ARG_X */
-
- ENTRANCE
- movl ARG_X, %eax /* Bits of x */
- cvtss2sd ARG_X, %xmm0 /* DP x */
- andl $0x7fffffff, %eax /* |x| */
-
- cmpl $0x3f490fdb, %eax /* |x|<Pi/4? */
- jb L(arg_less_pio4)
-
- /* Here if |x|>=Pi/4 */
- movd %eax, %xmm3 /* SP |x| */
- andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
- movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
-
- cmpl $0x40e231d6, %eax /* |x|<9*Pi/4? */
- jae L(large_args)
-
- /* Here if Pi/4<=|x|<9*Pi/4 */
- mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
- cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
- addl $1, %eax /* k+1 */
- movl $0x0e, %edx
- andl %eax, %edx /* j = (k+1)&0x0e */
- addl $2, %eax /* n */
- subsd MO2(PIO4J,%edx,8), %xmm0 /* t = |x| - j * Pi/4 */
-
-L(reconstruction):
- /* Input: %eax=n, %xmm0=t */
- testl $2, %eax /* n&2 != 0? */
- jz L(sin_poly)
-
-/*L(cos_poly):*/
- /* Here if cos(x) calculated using cos(t) polynomial for |t|<Pi/4:
- * y = t*t; z = y*y;
- * s = sign(x) * (-1.0)^((n>>2)&1)
- * result = s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))))
- */
- shrl $2, %eax /* n>>2 */
- mulsd %xmm0, %xmm0 /* y=t^2 */
- andl $1, %eax /* (n>>2)&1 */
- movaps %xmm0, %xmm1 /* y */
- mulsd %xmm0, %xmm0 /* z=t^4 */
-
- movsd MO1(DP_C4), %xmm4 /* C4 */
- mulsd %xmm0, %xmm4 /* z*C4 */
- movsd MO1(DP_C3), %xmm3 /* C3 */
- mulsd %xmm0, %xmm3 /* z*C3 */
- addsd MO1(DP_C2), %xmm4 /* C2+z*C4 */
- mulsd %xmm0, %xmm4 /* z*(C2+z*C4) */
- lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
- addsd MO1(DP_C1), %xmm3 /* C1+z*C3 */
- mulsd %xmm0, %xmm3 /* z*(C1+z*C3) */
- addsd MO1(DP_C0), %xmm4 /* C0+z*(C2+z*C4) */
- mulsd %xmm1, %xmm4 /* y*(C0+z*(C2+z*C4)) */
-
- addsd %xmm4, %xmm3 /* y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
- /* 1.0+y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
- addsd MO1(DP_ONES), %xmm3
-
- mulsd MO2(DP_ONES,%eax,8), %xmm3 /* DP result */
- movsd %xmm3, 0(%esp) /* Move result from sse... */
- fldl 0(%esp) /* ...to FPU. */
- /* Return back 4 bytes of stack frame */
- lea 8(%esp), %esp
- RETURN
-
- .p2align 4
-L(sin_poly):
- /* Here if cos(x) calculated using sin(t) polynomial for |t|<Pi/4:
- * y = t*t; z = y*y;
- * s = sign(x) * (-1.0)^((n>>2)&1)
- * result = s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))))
- */
-
- movaps %xmm0, %xmm4 /* t */
- shrl $2, %eax /* n>>2 */
- mulsd %xmm0, %xmm0 /* y=t^2 */
- andl $1, %eax /* (n>>2)&1 */
- movaps %xmm0, %xmm1 /* y */
- mulsd %xmm0, %xmm0 /* z=t^4 */
-
- movsd MO1(DP_S4), %xmm2 /* S4 */
- mulsd %xmm0, %xmm2 /* z*S4 */
- movsd MO1(DP_S3), %xmm3 /* S3 */
- mulsd %xmm0, %xmm3 /* z*S3 */
- lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
- addsd MO1(DP_S2), %xmm2 /* S2+z*S4 */
- mulsd %xmm0, %xmm2 /* z*(S2+z*S4) */
- addsd MO1(DP_S1), %xmm3 /* S1+z*S3 */
- mulsd %xmm0, %xmm3 /* z*(S1+z*S3) */
- addsd MO1(DP_S0), %xmm2 /* S0+z*(S2+z*S4) */
- mulsd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
- /* t*s, where s = sign(x) * (-1.0)^((n>>2)&1) */
- mulsd MO2(DP_ONES,%eax,8), %xmm4
- addsd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- /* t*s*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- mulsd %xmm4, %xmm3
- /* t*s*(1.0+y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- addsd %xmm4, %xmm3
- movsd %xmm3, 0(%esp) /* Move result from sse... */
- fldl 0(%esp) /* ...to FPU. */
- /* Return back 4 bytes of stack frame */
- lea 8(%esp), %esp
- RETURN
-
- .p2align 4
-L(large_args):
- /* Here if |x|>=9*Pi/4 */
- cmpl $0x7f800000, %eax /* x is Inf or NaN? */
- jae L(arg_inf_or_nan)
-
- /* Here if finite |x|>=9*Pi/4 */
- cmpl $0x4b000000, %eax /* |x|<2^23? */
- jae L(very_large_args)
-
- /* Here if 9*Pi/4<=|x|<2^23 */
- movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
- mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
- cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
- addl $1, %eax /* k+1 */
- movl %eax, %edx
- andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
- cvtsi2sdl %edx, %xmm4 /* DP j */
- movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
- mulsd %xmm4, %xmm2 /* -j*PIO4HI */
- movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
- addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
- addl $2, %eax /* n */
- mulsd %xmm3, %xmm4 /* j*PIO4LO */
- addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
- jmp L(reconstruction)
-
- .p2align 4
-L(very_large_args):
- /* Here if finite |x|>=2^23 */
-
- /* bitpos = (ix>>23) - BIAS_32 + 59; */
- shrl $23, %eax /* eb = biased exponent of x */
- /* bitpos = eb - 0x7f + 59, where 0x7f is exponent bias */
- subl $68, %eax
- movl $28, %ecx /* %cl=28 */
- movl %eax, %edx /* bitpos copy */
-
- /* j = bitpos/28; */
- div %cl /* j in register %al=%ax/%cl */
- movapd %xmm0, %xmm3 /* |x| */
- /* clear unneeded remainder from %ah */
- andl $0xff, %eax
-
- imull $28, %eax, %ecx /* j*28 */
- movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
- movapd %xmm0, %xmm5 /* |x| */
- mulsd -2*8+MO2(_FPI,%eax,8), %xmm3 /* tmp3 = FPI[j-2]*|x| */
- movapd %xmm0, %xmm1 /* |x| */
- mulsd -1*8+MO2(_FPI,%eax,8), %xmm5 /* tmp2 = FPI[j-1]*|x| */
- mulsd 0*8+MO2(_FPI,%eax,8), %xmm0 /* tmp0 = FPI[j]*|x| */
- addl $19, %ecx /* j*28+19 */
- mulsd 1*8+MO2(_FPI,%eax,8), %xmm1 /* tmp1 = FPI[j+1]*|x| */
- cmpl %ecx, %edx /* bitpos>=j*28+19? */
- jl L(very_large_skip1)
-
- /* Here if bitpos>=j*28+19 */
- andpd %xmm3, %xmm4 /* HI(tmp3) */
- subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
-L(very_large_skip1):
-
- movsd MO1(DP_2POW52), %xmm6
- movapd %xmm5, %xmm2 /* tmp2 copy */
- addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
- movl $1, %edx
- addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
- movsd 8+MO1(DP_2POW52), %xmm4
- movd %xmm6, %eax /* k = I64_LO(tmp6); */
- addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
- comisd %xmm5, %xmm4 /* tmp4 > tmp5? */
- jbe L(very_large_skip2)
-
- /* Here if tmp4 > tmp5 */
- subl $1, %eax /* k-- */
- addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
-L(very_large_skip2):
-
- andl %eax, %edx /* k&1 */
- subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
- addsd MO2(DP_ZERONE,%edx,8), %xmm3 /* t = DP_ZERONE[k&1] + tmp3 */
- addsd %xmm2, %xmm3 /* t += tmp2 */
- addsd %xmm3, %xmm0 /* t += tmp0 */
- addl $3, %eax /* n=k+3 */
- addsd %xmm1, %xmm0 /* t += tmp1 */
- mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
-
- jmp L(reconstruction) /* end of very_large_args peth */
-
- .p2align 4
-L(arg_less_pio4):
- /* Here if |x|<Pi/4 */
- cmpl $0x3d000000, %eax /* |x|<2^-5? */
- jl L(arg_less_2pn5)
-
- /* Here if 2^-5<=|x|<Pi/4 */
- mulsd %xmm0, %xmm0 /* y=x^2 */
- movaps %xmm0, %xmm1 /* y */
- mulsd %xmm0, %xmm0 /* z=x^4 */
- movsd MO1(DP_C4), %xmm3 /* C4 */
- mulsd %xmm0, %xmm3 /* z*C4 */
- movsd MO1(DP_C3), %xmm5 /* C3 */
- mulsd %xmm0, %xmm5 /* z*C3 */
- addsd MO1(DP_C2), %xmm3 /* C2+z*C4 */
- mulsd %xmm0, %xmm3 /* z*(C2+z*C4) */
- addsd MO1(DP_C1), %xmm5 /* C1+z*C3 */
- mulsd %xmm0, %xmm5 /* z*(C1+z*C3) */
- addsd MO1(DP_C0), %xmm3 /* C0+z*(C2+z*C4) */
- mulsd %xmm1, %xmm3 /* y*(C0+z*(C2+z*C4)) */
- addsd %xmm5, %xmm3 /* y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
- /* 1.0 + y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
- addsd MO1(DP_ONES), %xmm3
- cvtsd2ss %xmm3, %xmm3 /* SP result */
-
-L(epilogue):
- lea -4(%esp), %esp /* Borrow 4 bytes of stack frame */
- movss %xmm3, 0(%esp) /* Move result from sse... */
- flds 0(%esp) /* ...to FPU. */
- /* Return back 4 bytes of stack frame */
- lea 4(%esp), %esp
- RETURN
-
- .p2align 4
-L(arg_less_2pn5):
- /* Here if |x|<2^-5 */
- cmpl $0x32000000, %eax /* |x|<2^-27? */
- jl L(arg_less_2pn27)
-
- /* Here if 2^-27<=|x|<2^-5 */
- mulsd %xmm0, %xmm0 /* DP x^2 */
- movsd MO1(DP_COS2_1), %xmm3 /* DP DP_COS2_1 */
- mulsd %xmm0, %xmm3 /* DP x^2*DP_COS2_1 */
- addsd MO1(DP_COS2_0), %xmm3 /* DP DP_COS2_0+x^2*DP_COS2_1 */
- mulsd %xmm0, %xmm3 /* DP x^2*DP_COS2_0+x^4*DP_COS2_1 */
- /* DP 1.0+x^2*DP_COS2_0+x^4*DP_COS2_1 */
- addsd MO1(DP_ONES), %xmm3
- cvtsd2ss %xmm3, %xmm3 /* SP result */
- jmp L(epilogue)
-
- .p2align 4
-L(arg_less_2pn27):
- /* Here if |x|<2^-27 */
- movss ARG_X, %xmm0 /* x */
- andps MO1(SP_ABS_MASK),%xmm0 /* |x| */
- movss MO1(SP_ONE), %xmm3 /* 1.0 */
- subss %xmm0, %xmm3 /* result is 1.0-|x| */
- jmp L(epilogue)
-
- .p2align 4
-L(arg_inf_or_nan):
- /* Here if |x| is Inf or NAN */
- jne L(skip_errno_setting) /* in case of x is NaN */
-
- /* Here if x is Inf. Set errno to EDOM. */
- call JUMPTARGET(__errno_location)
- movl $EDOM, (%eax)
-
- .p2align 4
-L(skip_errno_setting):
- /* Here if |x| is Inf or NAN. Continued. */
- movss ARG_X, %xmm3 /* load x */
- subss %xmm3, %xmm3 /* Result is NaN */
- jmp L(epilogue)
-END(__cosf_sse2)
-
- .section .rodata, "a"
- .p2align 3
-L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
- .long 0x00000000,0x00000000
- .long 0x54442d18,0x3fe921fb
- .long 0x54442d18,0x3ff921fb
- .long 0x7f3321d2,0x4002d97c
- .long 0x54442d18,0x400921fb
- .long 0x2955385e,0x400f6a7a
- .long 0x7f3321d2,0x4012d97c
- .long 0xe9bba775,0x4015fdbb
- .long 0x54442d18,0x401921fb
- .long 0xbeccb2bb,0x401c463a
- .long 0x2955385e,0x401f6a7a
- .type L(PIO4J), @object
- ASM_SIZE_DIRECTIVE(L(PIO4J))
-
- .p2align 3
-L(_FPI): /* 4/Pi broken into sum of positive DP values */
- .long 0x00000000,0x00000000
- .long 0x6c000000,0x3ff45f30
- .long 0x2a000000,0x3e3c9c88
- .long 0xa8000000,0x3c54fe13
- .long 0xd0000000,0x3aaf47d4
- .long 0x6c000000,0x38fbb81b
- .long 0xe0000000,0x3714acc9
- .long 0x7c000000,0x3560e410
- .long 0x56000000,0x33bca2c7
- .long 0xac000000,0x31fbd778
- .long 0xe0000000,0x300b7246
- .long 0xe8000000,0x2e5d2126
- .long 0x48000000,0x2c970032
- .long 0xe8000000,0x2ad77504
- .long 0xe0000000,0x290921cf
- .long 0xb0000000,0x274deb1c
- .long 0xe0000000,0x25829a73
- .long 0xbe000000,0x23fd1046
- .long 0x10000000,0x2224baed
- .long 0x8e000000,0x20709d33
- .long 0x80000000,0x1e535a2f
- .long 0x64000000,0x1cef904e
- .long 0x30000000,0x1b0d6398
- .long 0x24000000,0x1964ce7d
- .long 0x16000000,0x17b908bf
- .type L(_FPI), @object
- ASM_SIZE_DIRECTIVE(L(_FPI))
-
-/* Coefficients of polynomial
- for cos(x)~=1.0+x^2*DP_COS2_0+x^4*DP_COS2_1, |x|<2^-5. */
- .p2align 3
-L(DP_COS2_0):
- .long 0xff5cc6fd,0xbfdfffff
- .type L(DP_COS2_0), @object
- ASM_SIZE_DIRECTIVE(L(DP_COS2_0))
-
- .p2align 3
-L(DP_COS2_1):
- .long 0xb178dac5,0x3fa55514
- .type L(DP_COS2_1), @object
- ASM_SIZE_DIRECTIVE(L(DP_COS2_1))
-
- .p2align 3
-L(DP_ZERONE):
- .long 0x00000000,0x00000000 /* 0.0 */
- .long 0x00000000,0xbff00000 /* 1.0 */
- .type L(DP_ZERONE),@object
- ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
-
- .p2align 3
-L(DP_ONES):
- .long 0x00000000,0x3ff00000 /* +1.0 */
- .long 0x00000000,0xbff00000 /* -1.0 */
- .type L(DP_ONES), @object
- ASM_SIZE_DIRECTIVE(L(DP_ONES))
-
-/* Coefficients of polynomial
- for sin(t)~=t+t^3*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))), |t|<Pi/4. */
- .p2align 3
-L(DP_S3):
- .long 0x64e6b5b4,0x3ec71d72
- .type L(DP_S3), @object
- ASM_SIZE_DIRECTIVE(L(DP_S3))
-
- .p2align 3
-L(DP_S1):
- .long 0x10c2688b,0x3f811111
- .type L(DP_S1), @object
- ASM_SIZE_DIRECTIVE(L(DP_S1))
-
- .p2align 3
-L(DP_S4):
- .long 0x1674b58a,0xbe5a947e
- .type L(DP_S4), @object
- ASM_SIZE_DIRECTIVE(L(DP_S4))
-
- .p2align 3
-L(DP_S2):
- .long 0x8b4bd1f9,0xbf2a019f
- .type L(DP_S2), @object
- ASM_SIZE_DIRECTIVE(L(DP_S2))
-
- .p2align 3
-L(DP_S0):
- .long 0x55551cd9,0xbfc55555
- .type L(DP_S0), @object
- ASM_SIZE_DIRECTIVE(L(DP_S0))
-
-/* Coefficients of polynomial
- for cos(t)~=1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))), |t|<Pi/4. */
- .p2align 3
-L(DP_C3):
- .long 0x9ac43cc0,0x3efa00eb
- .type L(DP_C3), @object
- ASM_SIZE_DIRECTIVE(L(DP_C3))
-
- .p2align 3
-L(DP_C1):
- .long 0x545c50c7,0x3fa55555
- .type L(DP_C1), @object
- ASM_SIZE_DIRECTIVE(L(DP_C1))
-
- .p2align 3
-L(DP_C4):
- .long 0xdd8844d7,0xbe923c97
- .type L(DP_C4), @object
- ASM_SIZE_DIRECTIVE(L(DP_C4))
-
- .p2align 3
-L(DP_C2):
- .long 0x348b6874,0xbf56c16b
- .type L(DP_C2), @object
- ASM_SIZE_DIRECTIVE(L(DP_C2))
-
- .p2align 3
-L(DP_C0):
- .long 0xfffe98ae,0xbfdfffff
- .type L(DP_C0), @object
- ASM_SIZE_DIRECTIVE(L(DP_C0))
-
- .p2align 3
-L(DP_PIO4):
- .long 0x54442d18,0x3fe921fb /* Pi/4 */
- .type L(DP_PIO4), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4))
-
- .p2align 3
-L(DP_2POW52):
- .long 0x00000000,0x43300000 /* +2^52 */
- .long 0x00000000,0xc3300000 /* -2^52 */
- .type L(DP_2POW52), @object
- ASM_SIZE_DIRECTIVE(L(DP_2POW52))
-
- .p2align 3
-L(DP_INVPIO4):
- .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
- .type L(DP_INVPIO4), @object
- ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
-
- .p2align 3
-L(DP_PIO4HI):
- .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
- .type L(DP_PIO4HI), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
-
- .p2align 3
-L(DP_PIO4LO):
- .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
- .type L(DP_PIO4LO), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
-
- .p2align 2
-L(SP_INVPIO4):
- .long 0x3fa2f983 /* 4/Pi */
- .type L(SP_INVPIO4), @object
- ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
-
- .p2align 4
-L(DP_ABS_MASK): /* Mask for getting DP absolute value */
- .long 0xffffffff,0x7fffffff
- .long 0xffffffff,0x7fffffff
- .type L(DP_ABS_MASK), @object
- ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
-
- .p2align 3
-L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
- .long 0x00000000,0xffffffff
- .type L(DP_HI_MASK), @object
- ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
-
- .p2align 4
-L(SP_ABS_MASK): /* Mask for getting SP absolute value */
- .long 0x7fffffff,0x7fffffff
- .long 0x7fffffff,0x7fffffff
- .type L(SP_ABS_MASK), @object
- ASM_SIZE_DIRECTIVE(L(SP_ABS_MASK))
-
- .p2align 2
-L(SP_ONE):
- .long 0x3f800000 /* 1.0 */
- .type L(SP_ONE), @object
- ASM_SIZE_DIRECTIVE(L(SP_ONE))
-
-weak_alias (__cosf, cosf)
diff --git a/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
new file mode 100644
index 0000000000..3db7a2474f
--- /dev/null
+++ b/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
@@ -0,0 +1,3 @@
+#define COSF __cosf_sse2
+#define SECTION __attribute__ ((section (".text.sse2")))
+#include <sysdeps/ieee754/flt-32/s_cosf.c>
diff --git a/sysdeps/ieee754/flt-32/s_cosf.c b/sysdeps/ieee754/flt-32/s_cosf.c
index ee9fa8565b..1236244244 100644
--- a/sysdeps/ieee754/flt-32/s_cosf.c
+++ b/sysdeps/ieee754/flt-32/s_cosf.c
@@ -23,6 +23,10 @@
#include "math_config.h"
#include "s_sincosf.h"
+#ifndef SECTION
+# define SECTION
+#endif
+
#ifndef COSF
# define COSF_FUNC __cosf
#else
@@ -35,6 +39,7 @@
arithmetic.
*/
float
+SECTION
COSF_FUNC (float y)
{
double x = y;
--
2.34.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 3/5] i686: Use generic sinf implementation for SSE2 version
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
2022-05-31 21:34 ` [PATCH 1/5] benchtests: Add workload name for cosf Adhemerval Zanella
2022-05-31 21:34 ` [PATCH 2/5] i686: Use generic cosf implementation for SSE2 version Adhemerval Zanella
@ 2022-05-31 21:35 ` Adhemerval Zanella
2022-05-31 21:47 ` H.J. Lu
2022-05-31 21:35 ` [PATCH 4/5] benchtests: Add workload name for sincosf Adhemerval Zanella
` (2 subsequent siblings)
5 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-05-31 21:35 UTC (permalink / raw)
To: libc-alpha, H . J . Lu
Performance seems to be similar (gcc 11.2.1 on a Ryzen 9 5900X),
the generic algorithm shows slight better performance for
the 'workload-huge.wrf' input set.
* s_sinf-sse2.S:
"sinf": {
"": {
"duration": 3.72405e+09,
"iterations": 2.38374e+08,
"max": 63.973,
"min": 11.211,
"mean": 15.6227
},
"workload-random.wrf": {
"duration": 3.76923e+09,
"iterations": 8.4e+07,
"reciprocal-throughput": 17.6355,
"latency": 72.108,
"max-throughput": 5.67037e+07,
"min-throughput": 1.38681e+07
},
"workload-huge.wrf": {
"duration": 3.76943e+09,
"iterations": 6e+07,
"reciprocal-throughput": 29.3493,
"latency": 96.2985,
"max-throughput": 3.40724e+07,
"min-throughput": 1.03844e+07
}
}
* generic s_sinf.c:
"sinf": {
"": {
"duration": 3.70989e+09,
"iterations": 2.18025e+08,
"max": 69.782,
"min": 11.1,
"mean": 17.0159
},
"workload-random.wrf": {
"duration": 3.77213e+09,
"iterations": 9.6e+07,
"reciprocal-throughput": 17.5402,
"latency": 61.0459,
"max-throughput": 5.70119e+07,
"min-throughput": 1.63811e+07
},
"workload-huge.wrf": {
"duration": 3.81576e+09,
"iterations": 5.6e+07,
"reciprocal-throughput": 38.2111,
"latency": 98.0659,
"max-throughput": 2.61704e+07,
"min-throughput": 1.01972e+07
}
}
Checked on i686-linux-gnu.
---
sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
.../i386/i686/fpu/multiarch/libm-test-ulps | 4 +
sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S | 565 ------------------
sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c | 3 +
sysdeps/ieee754/flt-32/s_sinf.c | 5 +
5 files changed, 13 insertions(+), 565 deletions(-)
delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
index b8f52c311f..c2cd5cefef 100644
--- a/sysdeps/i386/i686/fpu/multiarch/Makefile
+++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
@@ -8,4 +8,5 @@ CFLAGS-e_log2f-sse2.c = -msse2 -mfpmath=sse
CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
+CFLAGS-s_sinf-sse2.c = -msse2 -mfpmath=sse
endif
diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
index 1bfad77adc..0357d7bd14 100644
--- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
+++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
@@ -1463,21 +1463,25 @@ ldouble: 4
Function: "sin":
double: 1
+float: 1
float128: 2
ldouble: 2
Function: "sin_downward":
double: 1
+float: 1
float128: 3
ldouble: 3
Function: "sin_towardzero":
double: 1
+float: 1
float128: 2
ldouble: 2
Function: "sin_upward":
double: 1
+float: 1
float128: 3
ldouble: 3
diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
deleted file mode 100644
index c473eeb4f7..0000000000
--- a/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
+++ /dev/null
@@ -1,565 +0,0 @@
-/* Optimized with sse2 version of sinf
- Copyright (C) 2012-2022 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <https://www.gnu.org/licenses/>. */
-
-#include <sysdep.h>
-#include <errno.h>
-
-/* Short algorithm description:
- *
- * 1) if |x| == 0: return x.
- * 2) if |x| < 2^-27: return x-x*DP_SMALL, raise underflow only when needed.
- * 3) if |x| < 2^-5 : return x+x^3*DP_SIN2_0+x^5*DP_SIN2_1.
- * 4) if |x| < Pi/4: return x+x^3*(S0+x^2*(S1+x^2*(S2+x^2*(S3+x^2*S4)))).
- * 5) if |x| < 9*Pi/4:
- * 5.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+1,
- * t=|x|-j*Pi/4.
- * 5.2) Reconstruction:
- * s = sign(x) * (-1.0)^((n>>2)&1)
- * if(n&2 != 0) {
- * using cos(t) polynomial for |t|<Pi/4, result is
- * s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4))))).
- * } else {
- * using sin(t) polynomial for |t|<Pi/4, result is
- * s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4))))).
- * }
- * 6) if |x| < 2^23, large args:
- * 6.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1,
- * t=|x|-j*Pi/4.
- * 6.2) Reconstruction same as (5.2).
- * 7) if |x| >= 2^23, very large args:
- * 7.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1,
- * t=|x|-j*Pi/4.
- * 7.2) Reconstruction same as (5.2).
- * 8) if x is Inf, return x-x, and set errno=EDOM.
- * 9) if x is NaN, return x-x.
- *
- * Special cases:
- * sin(+-0) = +-0 not raising inexact/underflow,
- * sin(subnormal) raises inexact/underflow,
- * sin(min_normalized) raises inexact/underflow,
- * sin(normalized) raises inexact,
- * sin(Inf) = NaN, raises invalid, sets errno to EDOM,
- * sin(NaN) = NaN.
- */
-
-#ifdef PIC
-# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
-# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
-# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
-# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
-# define PUSH(REG) pushl REG; CFI_PUSH(REG)
-# define POP(REG) popl REG; CFI_POP(REG)
-# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
-# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
-# define ARG_X 8(%esp)
-#else
-# define MO1(symbol) L(symbol)
-# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
-# define ENTRANCE
-# define RETURN ret
-# define ARG_X 4(%esp)
-#endif
-
- .text
-ENTRY(__sinf_sse2)
- /* Input: single precision x on stack at address ARG_X */
-
- ENTRANCE
- movl ARG_X, %eax /* Bits of x */
- cvtss2sd ARG_X, %xmm0 /* DP x */
- andl $0x7fffffff, %eax /* |x| */
-
- cmpl $0x3f490fdb, %eax /* |x|<Pi/4? */
- jb L(arg_less_pio4)
-
- /* Here if |x|>=Pi/4 */
- movd %eax, %xmm3 /* SP |x| */
- andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
- movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
-
- cmpl $0x40e231d6, %eax /* |x|<9*Pi/4? */
- jae L(large_args)
-
- /* Here if Pi/4<=|x|<9*Pi/4 */
- mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
- movl ARG_X, %ecx /* Load x */
- cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
- shrl $31, %ecx /* sign of x */
- addl $1, %eax /* k+1 */
- movl $0x0e, %edx
- andl %eax, %edx /* j = (k+1)&0x0e */
- subsd MO2(PIO4J,%edx,8), %xmm0 /* t = |x| - j * Pi/4 */
-
-L(reconstruction):
- /* Input: %eax=n, %xmm0=t, %ecx=sign(x) */
- testl $2, %eax /* n&2 != 0? */
- jz L(sin_poly)
-
-/*L(cos_poly):*/
- /* Here if sin(x) calculated using cos(t) polynomial for |t|<Pi/4:
- * y = t*t; z = y*y;
- * s = sign(x) * (-1.0)^((n>>2)&1)
- * result = s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))))
- */
- shrl $2, %eax /* n>>2 */
- mulsd %xmm0, %xmm0 /* y=t^2 */
- andl $1, %eax /* (n>>2)&1 */
- movaps %xmm0, %xmm1 /* y */
- mulsd %xmm0, %xmm0 /* z=t^4 */
-
- movsd MO1(DP_C4), %xmm4 /* C4 */
- mulsd %xmm0, %xmm4 /* z*C4 */
- xorl %eax, %ecx /* (-1.0)^((n>>2)&1) XOR sign(x) */
- movsd MO1(DP_C3), %xmm3 /* C3 */
- mulsd %xmm0, %xmm3 /* z*C3 */
- addsd MO1(DP_C2), %xmm4 /* C2+z*C4 */
- mulsd %xmm0, %xmm4 /* z*(C2+z*C4) */
- lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
- addsd MO1(DP_C1), %xmm3 /* C1+z*C3 */
- mulsd %xmm0, %xmm3 /* z*(C1+z*C3) */
- addsd MO1(DP_C0), %xmm4 /* C0+z*(C2+z*C4) */
- mulsd %xmm1, %xmm4 /* y*(C0+z*(C2+z*C4)) */
-
- addsd %xmm4, %xmm3 /* y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
- /* 1.0+y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
- addsd MO1(DP_ONES), %xmm3
-
- mulsd MO2(DP_ONES,%ecx,8), %xmm3 /* DP result */
- movsd %xmm3, 0(%esp) /* Move result from sse... */
- fldl 0(%esp) /* ...to FPU. */
- /* Return back 4 bytes of stack frame */
- lea 8(%esp), %esp
- RETURN
-
- .p2align 4
-L(sin_poly):
- /* Here if sin(x) calculated using sin(t) polynomial for |t|<Pi/4:
- * y = t*t; z = y*y;
- * s = sign(x) * (-1.0)^((n>>2)&1)
- * result = s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))))
- */
-
- movaps %xmm0, %xmm4 /* t */
- shrl $2, %eax /* n>>2 */
- mulsd %xmm0, %xmm0 /* y=t^2 */
- andl $1, %eax /* (n>>2)&1 */
- movaps %xmm0, %xmm1 /* y */
- xorl %eax, %ecx /* (-1.0)^((n>>2)&1) XOR sign(x) */
- mulsd %xmm0, %xmm0 /* z=t^4 */
-
- movsd MO1(DP_S4), %xmm2 /* S4 */
- mulsd %xmm0, %xmm2 /* z*S4 */
- movsd MO1(DP_S3), %xmm3 /* S3 */
- mulsd %xmm0, %xmm3 /* z*S3 */
- lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
- addsd MO1(DP_S2), %xmm2 /* S2+z*S4 */
- mulsd %xmm0, %xmm2 /* z*(S2+z*S4) */
- addsd MO1(DP_S1), %xmm3 /* S1+z*S3 */
- mulsd %xmm0, %xmm3 /* z*(S1+z*S3) */
- addsd MO1(DP_S0), %xmm2 /* S0+z*(S2+z*S4) */
- mulsd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
- /* t*s, where s = sign(x) * (-1.0)^((n>>2)&1) */
- mulsd MO2(DP_ONES,%ecx,8), %xmm4
- addsd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- /* t*s*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- mulsd %xmm4, %xmm3
- /* t*s*(1.0+y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- addsd %xmm4, %xmm3
- movsd %xmm3, 0(%esp) /* Move result from sse... */
- fldl 0(%esp) /* ...to FPU. */
- /* Return back 4 bytes of stack frame */
- lea 8(%esp), %esp
- RETURN
-
- .p2align 4
-L(large_args):
- /* Here if |x|>=9*Pi/4 */
- cmpl $0x7f800000, %eax /* x is Inf or NaN? */
- jae L(arg_inf_or_nan)
-
- /* Here if finite |x|>=9*Pi/4 */
- cmpl $0x4b000000, %eax /* |x|<2^23? */
- jae L(very_large_args)
-
- /* Here if 9*Pi/4<=|x|<2^23 */
- movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
- mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
- cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
- addl $1, %eax /* k+1 */
- movl %eax, %edx
- andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
- cvtsi2sdl %edx, %xmm4 /* DP j */
- movl ARG_X, %ecx /* Load x */
- movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
- shrl $31, %ecx /* sign bit of x */
- mulsd %xmm4, %xmm2 /* -j*PIO4HI */
- movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
- addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
- mulsd %xmm3, %xmm4 /* j*PIO4LO */
- addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
- jmp L(reconstruction)
-
- .p2align 4
-L(very_large_args):
- /* Here if finite |x|>=2^23 */
-
- /* bitpos = (ix>>23) - BIAS_32 + 59; */
- shrl $23, %eax /* eb = biased exponent of x */
- /* bitpos = eb - 0x7f + 59, where 0x7f is exponent bias */
- subl $68, %eax
- movl $28, %ecx /* %cl=28 */
- movl %eax, %edx /* bitpos copy */
-
- /* j = bitpos/28; */
- div %cl /* j in register %al=%ax/%cl */
- movapd %xmm0, %xmm3 /* |x| */
- /* clear unneeded remainder from %ah */
- andl $0xff, %eax
-
- imull $28, %eax, %ecx /* j*28 */
- movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
- movapd %xmm0, %xmm5 /* |x| */
- mulsd -2*8+MO2(_FPI,%eax,8), %xmm3 /* tmp3 = FPI[j-2]*|x| */
- movapd %xmm0, %xmm1 /* |x| */
- mulsd -1*8+MO2(_FPI,%eax,8), %xmm5 /* tmp2 = FPI[j-1]*|x| */
- mulsd 0*8+MO2(_FPI,%eax,8), %xmm0 /* tmp0 = FPI[j]*|x| */
- addl $19, %ecx /* j*28+19 */
- mulsd 1*8+MO2(_FPI,%eax,8), %xmm1 /* tmp1 = FPI[j+1]*|x| */
- cmpl %ecx, %edx /* bitpos>=j*28+19? */
- jl L(very_large_skip1)
-
- /* Here if bitpos>=j*28+19 */
- andpd %xmm3, %xmm4 /* HI(tmp3) */
- subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
-L(very_large_skip1):
-
- movsd MO1(DP_2POW52), %xmm6
- movapd %xmm5, %xmm2 /* tmp2 copy */
- addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
- movl $1, %edx
- addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
- movsd 8+MO1(DP_2POW52), %xmm4
- movd %xmm6, %eax /* k = I64_LO(tmp6); */
- addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
- movl ARG_X, %ecx /* Load x */
- comisd %xmm5, %xmm4 /* tmp4 > tmp5? */
- jbe L(very_large_skip2)
-
- /* Here if tmp4 > tmp5 */
- subl $1, %eax /* k-- */
- addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
-L(very_large_skip2):
-
- andl %eax, %edx /* k&1 */
- subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
- addsd MO2(DP_ZERONE,%edx,8), %xmm3 /* t = DP_ZERONE[k&1] + tmp3 */
- addsd %xmm2, %xmm3 /* t += tmp2 */
- shrl $31, %ecx /* sign of x */
- addsd %xmm3, %xmm0 /* t += tmp0 */
- addl $1, %eax /* n=k+1 */
- addsd %xmm1, %xmm0 /* t += tmp1 */
- mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
-
- jmp L(reconstruction) /* end of very_large_args peth */
-
- .p2align 4
-L(arg_less_pio4):
- /* Here if |x|<Pi/4 */
- cmpl $0x3d000000, %eax /* |x|<2^-5? */
- jl L(arg_less_2pn5)
-
- /* Here if 2^-5<=|x|<Pi/4 */
- movaps %xmm0, %xmm3 /* x */
- mulsd %xmm0, %xmm0 /* y=x^2 */
- movaps %xmm0, %xmm1 /* y */
- mulsd %xmm0, %xmm0 /* z=x^4 */
- movsd MO1(DP_S4), %xmm4 /* S4 */
- mulsd %xmm0, %xmm4 /* z*S4 */
- movsd MO1(DP_S3), %xmm5 /* S3 */
- mulsd %xmm0, %xmm5 /* z*S3 */
- addsd MO1(DP_S2), %xmm4 /* S2+z*S4 */
- mulsd %xmm0, %xmm4 /* z*(S2+z*S4) */
- addsd MO1(DP_S1), %xmm5 /* S1+z*S3 */
- mulsd %xmm0, %xmm5 /* z*(S1+z*S3) */
- addsd MO1(DP_S0), %xmm4 /* S0+z*(S2+z*S4) */
- mulsd %xmm1, %xmm4 /* y*(S0+z*(S2+z*S4)) */
- mulsd %xmm3, %xmm5 /* x*z*(S1+z*S3) */
- mulsd %xmm3, %xmm4 /* x*y*(S0+z*(S2+z*S4)) */
- /* x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- addsd %xmm5, %xmm4
- /* x + x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- addsd %xmm4, %xmm3
- cvtsd2ss %xmm3, %xmm3 /* SP result */
-
-L(epilogue):
- lea -4(%esp), %esp /* Borrow 4 bytes of stack frame */
- movss %xmm3, 0(%esp) /* Move result from sse... */
- flds 0(%esp) /* ...to FPU. */
- /* Return back 4 bytes of stack frame */
- lea 4(%esp), %esp
- RETURN
-
- .p2align 4
-L(arg_less_2pn5):
- /* Here if |x|<2^-5 */
- cmpl $0x32000000, %eax /* |x|<2^-27? */
- jl L(arg_less_2pn27)
-
- /* Here if 2^-27<=|x|<2^-5 */
- movaps %xmm0, %xmm1 /* DP x */
- mulsd %xmm0, %xmm0 /* DP x^2 */
- movsd MO1(DP_SIN2_1), %xmm3 /* DP DP_SIN2_1 */
- mulsd %xmm0, %xmm3 /* DP x^2*DP_SIN2_1 */
- addsd MO1(DP_SIN2_0), %xmm3 /* DP DP_SIN2_0+x^2*DP_SIN2_1 */
- mulsd %xmm0, %xmm3 /* DP x^2*DP_SIN2_0+x^4*DP_SIN2_1 */
- mulsd %xmm1, %xmm3 /* DP x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
- addsd %xmm1, %xmm3 /* DP x+x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
- cvtsd2ss %xmm3, %xmm3 /* SP result */
- jmp L(epilogue)
-
- .p2align 4
-L(arg_less_2pn27):
- movss ARG_X, %xmm3 /* SP x */
- cmpl $0, %eax /* x=0? */
- je L(epilogue) /* in case x=0 return sin(+-0)==+-0 */
- /* Here if |x|<2^-27 */
- /*
- * Special cases here:
- * sin(subnormal) raises inexact/underflow
- * sin(min_normalized) raises inexact/underflow
- * sin(normalized) raises inexact
- */
- movaps %xmm0, %xmm3 /* Copy of DP x */
- mulsd MO1(DP_SMALL), %xmm0 /* x*DP_SMALL */
- subsd %xmm0, %xmm3 /* Result is x-x*DP_SMALL */
- cvtsd2ss %xmm3, %xmm3 /* Result converted to SP */
- jmp L(epilogue)
-
- .p2align 4
-L(arg_inf_or_nan):
- /* Here if |x| is Inf or NAN */
- jne L(skip_errno_setting) /* in case of x is NaN */
-
- /* Here if x is Inf. Set errno to EDOM. */
- call JUMPTARGET(__errno_location)
- movl $EDOM, (%eax)
-
- .p2align 4
-L(skip_errno_setting):
- /* Here if |x| is Inf or NAN. Continued. */
- movss ARG_X, %xmm3 /* load x */
- subss %xmm3, %xmm3 /* Result is NaN */
- jmp L(epilogue)
-END(__sinf_sse2)
-
- .section .rodata, "a"
- .p2align 3
-L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
- .long 0x00000000,0x00000000
- .long 0x54442d18,0x3fe921fb
- .long 0x54442d18,0x3ff921fb
- .long 0x7f3321d2,0x4002d97c
- .long 0x54442d18,0x400921fb
- .long 0x2955385e,0x400f6a7a
- .long 0x7f3321d2,0x4012d97c
- .long 0xe9bba775,0x4015fdbb
- .long 0x54442d18,0x401921fb
- .long 0xbeccb2bb,0x401c463a
- .long 0x2955385e,0x401f6a7a
- .type L(PIO4J), @object
- ASM_SIZE_DIRECTIVE(L(PIO4J))
-
- .p2align 3
-L(_FPI): /* 4/Pi broken into sum of positive DP values */
- .long 0x00000000,0x00000000
- .long 0x6c000000,0x3ff45f30
- .long 0x2a000000,0x3e3c9c88
- .long 0xa8000000,0x3c54fe13
- .long 0xd0000000,0x3aaf47d4
- .long 0x6c000000,0x38fbb81b
- .long 0xe0000000,0x3714acc9
- .long 0x7c000000,0x3560e410
- .long 0x56000000,0x33bca2c7
- .long 0xac000000,0x31fbd778
- .long 0xe0000000,0x300b7246
- .long 0xe8000000,0x2e5d2126
- .long 0x48000000,0x2c970032
- .long 0xe8000000,0x2ad77504
- .long 0xe0000000,0x290921cf
- .long 0xb0000000,0x274deb1c
- .long 0xe0000000,0x25829a73
- .long 0xbe000000,0x23fd1046
- .long 0x10000000,0x2224baed
- .long 0x8e000000,0x20709d33
- .long 0x80000000,0x1e535a2f
- .long 0x64000000,0x1cef904e
- .long 0x30000000,0x1b0d6398
- .long 0x24000000,0x1964ce7d
- .long 0x16000000,0x17b908bf
- .type L(_FPI), @object
- ASM_SIZE_DIRECTIVE(L(_FPI))
-
-/* Coefficients of polynomial
- for sin(x)~=x+x^3*DP_SIN2_0+x^5*DP_SIN2_1, |x|<2^-5. */
- .p2align 3
-L(DP_SIN2_0):
- .long 0x5543d49d,0xbfc55555
- .type L(DP_SIN2_0), @object
- ASM_SIZE_DIRECTIVE(L(DP_SIN2_0))
-
- .p2align 3
-L(DP_SIN2_1):
- .long 0x75cec8c5,0x3f8110f4
- .type L(DP_SIN2_1), @object
- ASM_SIZE_DIRECTIVE(L(DP_SIN2_1))
-
- .p2align 3
-L(DP_ZERONE):
- .long 0x00000000,0x00000000 /* 0.0 */
- .long 0x00000000,0xbff00000 /* 1.0 */
- .type L(DP_ZERONE), @object
- ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
-
- .p2align 3
-L(DP_ONES):
- .long 0x00000000,0x3ff00000 /* +1.0 */
- .long 0x00000000,0xbff00000 /* -1.0 */
- .type L(DP_ONES), @object
- ASM_SIZE_DIRECTIVE(L(DP_ONES))
-
-/* Coefficients of polynomial
- for sin(t)~=t+t^3*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))), |t|<Pi/4. */
- .p2align 3
-L(DP_S3):
- .long 0x64e6b5b4,0x3ec71d72
- .type L(DP_S3), @object
- ASM_SIZE_DIRECTIVE(L(DP_S3))
-
- .p2align 3
-L(DP_S1):
- .long 0x10c2688b,0x3f811111
- .type L(DP_S1), @object
- ASM_SIZE_DIRECTIVE(L(DP_S1))
-
- .p2align 3
-L(DP_S4):
- .long 0x1674b58a,0xbe5a947e
- .type L(DP_S4), @object
- ASM_SIZE_DIRECTIVE(L(DP_S4))
-
- .p2align 3
-L(DP_S2):
- .long 0x8b4bd1f9,0xbf2a019f
- .type L(DP_S2), @object
- ASM_SIZE_DIRECTIVE(L(DP_S2))
-
- .p2align 3
-L(DP_S0):
- .long 0x55551cd9,0xbfc55555
- .type L(DP_S0), @object
- ASM_SIZE_DIRECTIVE(L(DP_S0))
-
- .p2align 3
-L(DP_SMALL):
- .long 0x00000000,0x3cd00000 /* 2^(-50) */
- .type L(DP_SMALL), @object
- ASM_SIZE_DIRECTIVE(L(DP_SMALL))
-
-/* Coefficients of polynomial
- for cos(t)~=1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))), |t|<Pi/4. */
- .p2align 3
-L(DP_C3):
- .long 0x9ac43cc0,0x3efa00eb
- .type L(DP_C3), @object
- ASM_SIZE_DIRECTIVE(L(DP_C3))
-
- .p2align 3
-L(DP_C1):
- .long 0x545c50c7,0x3fa55555
- .type L(DP_C1), @object
- ASM_SIZE_DIRECTIVE(L(DP_C1))
-
- .p2align 3
-L(DP_C4):
- .long 0xdd8844d7,0xbe923c97
- .type L(DP_C4), @object
- ASM_SIZE_DIRECTIVE(L(DP_C4))
-
- .p2align 3
-L(DP_C2):
- .long 0x348b6874,0xbf56c16b
- .type L(DP_C2), @object
- ASM_SIZE_DIRECTIVE(L(DP_C2))
-
- .p2align 3
-L(DP_C0):
- .long 0xfffe98ae,0xbfdfffff
- .type L(DP_C0), @object
- ASM_SIZE_DIRECTIVE(L(DP_C0))
-
- .p2align 3
-L(DP_PIO4):
- .long 0x54442d18,0x3fe921fb /* Pi/4 */
- .type L(DP_PIO4), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4))
-
- .p2align 3
-L(DP_2POW52):
- .long 0x00000000,0x43300000 /* +2^52 */
- .long 0x00000000,0xc3300000 /* -2^52 */
- .type L(DP_2POW52), @object
- ASM_SIZE_DIRECTIVE(L(DP_2POW52))
-
- .p2align 3
-L(DP_INVPIO4):
- .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
- .type L(DP_INVPIO4), @object
- ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
-
- .p2align 3
-L(DP_PIO4HI):
- .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
- .type L(DP_PIO4HI), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
-
- .p2align 3
-L(DP_PIO4LO):
- .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
- .type L(DP_PIO4LO), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
-
- .p2align 2
-L(SP_INVPIO4):
- .long 0x3fa2f983 /* 4/Pi */
- .type L(SP_INVPIO4), @object
- ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
-
- .p2align 4
-L(DP_ABS_MASK): /* Mask for getting DP absolute value */
- .long 0xffffffff,0x7fffffff
- .long 0xffffffff,0x7fffffff
- .type L(DP_ABS_MASK), @object
- ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
-
- .p2align 3
-L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
- .long 0x00000000,0xffffffff
- .type L(DP_HI_MASK), @object
- ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
-
-weak_alias (__sinf, sinf)
diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
new file mode 100644
index 0000000000..e7dc0dc1a6
--- /dev/null
+++ b/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
@@ -0,0 +1,3 @@
+#define SINF __sinf_sse2
+#define SECTION __attribute__ ((section (".text.sse2")))
+#include <sysdeps/ieee754/flt-32/s_sinf.c>
diff --git a/sysdeps/ieee754/flt-32/s_sinf.c b/sysdeps/ieee754/flt-32/s_sinf.c
index ad54b1c7e6..e157ff071d 100644
--- a/sysdeps/ieee754/flt-32/s_sinf.c
+++ b/sysdeps/ieee754/flt-32/s_sinf.c
@@ -23,6 +23,10 @@
#include "math_config.h"
#include "s_sincosf.h"
+#ifndef SECTION
+# define SECTION
+#endif
+
#ifndef SINF
# define SINF_FUNC __sinf
#else
@@ -35,6 +39,7 @@
arithmetic.
*/
float
+SECTION
SINF_FUNC (float y)
{
double x = y;
--
2.34.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 4/5] benchtests: Add workload name for sincosf
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
` (2 preceding siblings ...)
2022-05-31 21:35 ` [PATCH 3/5] i686: Use generic sinf " Adhemerval Zanella
@ 2022-05-31 21:35 ` Adhemerval Zanella
2022-05-31 21:48 ` H.J. Lu
2022-05-31 21:35 ` [PATCH 5/5] math: Use generic sinf implementation for SSE2 version Adhemerval Zanella
2022-05-31 21:48 ` [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Noah Goldstein
5 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-05-31 21:35 UTC (permalink / raw)
To: libc-alpha, H . J . Lu
So it can show both reciprocal-throughput and latency.
---
benchtests/sincosf-inputs | 1 +
1 file changed, 1 insertion(+)
diff --git a/benchtests/sincosf-inputs b/benchtests/sincosf-inputs
index ef5668a38c..0a8495a67c 100644
--- a/benchtests/sincosf-inputs
+++ b/benchtests/sincosf-inputs
@@ -1,5 +1,6 @@
## includes: math.h
## args: float:<float *>:<float *>
+## name: workload-random
0x1.759424p-1
-0x1.324949p0
-0x1.855528p8
--
2.34.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 5/5] math: Use generic sinf implementation for SSE2 version
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
` (3 preceding siblings ...)
2022-05-31 21:35 ` [PATCH 4/5] benchtests: Add workload name for sincosf Adhemerval Zanella
@ 2022-05-31 21:35 ` Adhemerval Zanella
2022-05-31 21:51 ` H.J. Lu
2022-05-31 21:48 ` [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Noah Goldstein
5 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-05-31 21:35 UTC (permalink / raw)
To: libc-alpha, H . J . Lu
The generic implementation shows slight better performance
(gcc 11.2.1 on a Ryzen 9 5900X):
* s_sincosf-sse2.S:
"sincosf": {
"workload-random": {
"duration": 3.89961e+09,
"iterations": 9.5472e+07,
"reciprocal-throughput": 40.8429,
"latency": 40.8483,
"max-throughput": 2.4484e+07,
"min-throughput": 2.44808e+07
}
}
* generic s_cossinf.c:
"sincosf": {
"workload-random": {
"duration": 3.71953e+09,
"iterations": 1.48512e+08,
"reciprocal-throughput": 25.0515,
"latency": 25.0391,
"max-throughput": 3.99177e+07,
"min-throughput": 3.99375e+07
}
}
Checked on i686-linux-gnu.
---
sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
.../i386/i686/fpu/multiarch/libm-test-ulps | 3 +
.../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
.../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
4 files changed, 7 insertions(+), 585 deletions(-)
delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
index c2cd5cefef..c8b0a6528a 100644
--- a/sysdeps/i386/i686/fpu/multiarch/Makefile
+++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
@@ -9,4 +9,5 @@ CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
CFLAGS-s_sinf-sse2.c = -msse2 -mfpmath=sse
+CFLAGS-s_sincosf-sse2.c = -msse2 -mfpmath=sse
endif
diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
index 0357d7bd14..284e3fa275 100644
--- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
+++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
@@ -1492,16 +1492,19 @@ ldouble: 1
Function: "sincos_downward":
double: 1
+float: 1
float128: 3
ldouble: 3
Function: "sincos_towardzero":
double: 1
+float: 1
float128: 2
ldouble: 2
Function: "sincos_upward":
double: 1
+float: 1
float128: 3
ldouble: 3
diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
deleted file mode 100644
index 71e222b930..0000000000
--- a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
+++ /dev/null
@@ -1,585 +0,0 @@
-/* Optimized with sse2 version of sincosf
- Copyright (C) 2012-2022 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <https://www.gnu.org/licenses/>. */
-
-#include <sysdep.h>
-#include <errno.h>
-
-/* Short algorithm description:
- *
- * 1) if |x|==0: sin(x)=x,
- * cos(x)=1.
- * 2) if |x|<2^-27: sin(x)=x-x*DP_SMALL, raising underflow only when needed,
- * cos(x)=1-|x|.
- * 3) if |x|<2^-5 : sin(x)=x+x*x^2*DP_SIN2_0+x^5*DP_SIN2_1,
- * cos(x)=1+1*x^2*DP_COS2_0+x^5*DP_COS2_1
- * 4) if |x|< Pi/4: sin(x)=x+x*x^2*(S0+x^2*(S1+x^2*(S2+x^2*(S3+x^2*S4)))),
- * cos(x)=1+1*x^2*(C0+x^2*(C1+x^2*(C2+x^2*(C3+x^2*C4)))).
- * 5) if |x| < 9*Pi/4:
- * 5.1) Range reduction:
- * k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+1, t=|x|-j*Pi/4.
- * 5.2) Reconstruction:
- * sign_sin = sign(x) * (-1.0)^(( n >>2)&1)
- * sign_cos = (-1.0)^(((n+2)>>2)&1)
- * poly_sin = ((((S4*t^2 + S3)*t^2 + S2)*t^2 + S1)*t^2 + S0)*t^2*t+t
- * poly_cos = ((((C4*t^2 + C3)*t^2 + C2)*t^2 + C1)*t^2 + C0)*t^2*s+s
- * if(n&2 != 0) {
- * using cos(t) and sin(t) polynomials for |t|<Pi/4, results are
- * cos(x) = poly_sin * sign_cos
- * sin(x) = poly_cos * sign_sin
- * } else {
- * sin(x) = poly_sin * sign_sin
- * cos(x) = poly_cos * sign_cos
- * }
- * 6) if |x| < 2^23, large args:
- * 6.1) Range reduction:
- * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4
- * 6.2) Reconstruction same as (5.2).
- * 7) if |x| >= 2^23, very large args:
- * 7.1) Range reduction:
- * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4.
- * 7.2) Reconstruction same as (5.2).
- * 8) if x is Inf, return x-x, and set errno=EDOM.
- * 9) if x is NaN, return x-x.
- *
- * Special cases:
- * sin/cos(+-0) = +-0/1 not raising inexact/underflow,
- * sin/cos(subnormal) raises inexact/underflow,
- * sin/cos(min_normalized) raises inexact/underflow,
- * sin/cos(normalized) raises inexact,
- * sin/cos(Inf) = NaN, raises invalid, sets errno to EDOM,
- * sin/cos(NaN) = NaN.
- */
-
-#ifdef PIC
-# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
-# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
-# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
-# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
-# define PUSH(REG) pushl REG; CFI_PUSH(REG)
-# define POP(REG) popl REG; CFI_POP(REG)
-# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
-# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
-# define ARG_X 8(%esp)
-# define ARG_SIN_PTR 12(%esp)
-# define ARG_COS_PTR 16(%esp)
-#else
-# define MO1(symbol) L(symbol)
-# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
-# define ENTRANCE
-# define RETURN ret
-# define ARG_X 4(%esp)
-# define ARG_SIN_PTR 8(%esp)
-# define ARG_COS_PTR 12(%esp)
-#endif
-
- .text
-ENTRY(__sincosf_sse2)
- /* Input: single precision x on stack at address ARG_X */
- /* pointer to sin result on stack at address ARG_SIN_PTR */
- /* pointer to cos result on stack at address ARG_COS_PTR */
-
- ENTRANCE
- movl ARG_X, %eax /* Bits of x */
- cvtss2sd ARG_X, %xmm0 /* DP x */
- andl $0x7fffffff, %eax /* |x| */
-
- cmpl $0x3f490fdb, %eax /* |x|<Pi/4 ? */
- jb L(arg_less_pio4)
-
- /* Here if |x|>=Pi/4 */
- movd %eax, %xmm3 /* SP |x| */
- andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
- movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
-
- cmpl $0x40e231d6, %eax /* |x|<9*Pi/4 ? */
- jae L(large_args)
-
- /* Here if Pi/4<=|x|<9*Pi/4 */
- mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
- movl ARG_X, %ecx /* Load x */
- cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
- shrl $29, %ecx /* (sign of x) << 2 */
- addl $1, %eax /* k+1 */
- movl $0x0e, %edx
- andl %eax, %edx /* j = (k+1)&0x0e */
- subsd MO2(PIO4J,%edx,8), %xmm0/* t = |x| - j * Pi/4 */
-
-L(reconstruction):
- /* Input: %eax=n, %xmm0=t, %ecx=sign(x) */
-
- movaps %xmm0, %xmm4 /* t */
- movhpd MO1(DP_ONES), %xmm4 /* 1|t */
- mulsd %xmm0, %xmm0 /* y=t^2 */
- movl $2, %edx
- unpcklpd %xmm0, %xmm0 /* y|y */
- addl %eax, %edx /* k+2 */
- movaps %xmm0, %xmm1 /* y|y */
- mulpd %xmm0, %xmm0 /* z=t^4|z=t^4 */
-
- movaps MO1(DP_SC4), %xmm2 /* S4 */
- mulpd %xmm0, %xmm2 /* z*S4 */
- movaps MO1(DP_SC3), %xmm3 /* S3 */
- mulpd %xmm0, %xmm3 /* z*S3 */
- xorl %eax, %ecx /* (sign_x ^ (k>>2))<<2 */
- addpd MO1(DP_SC2), %xmm2 /* S2+z*S4 */
- mulpd %xmm0, %xmm2 /* z*(S2+z*S4) */
- shrl $2, %edx /* (k+2)>>2 */
- addpd MO1(DP_SC1), %xmm3 /* S1+z*S3 */
- mulpd %xmm0, %xmm3 /* z*(S1+z*S3) */
- shrl $2, %ecx /* sign_x ^ k>>2 */
- addpd MO1(DP_SC0), %xmm2 /* S0+z*(S2+z*S4) */
- andl $1, %edx /* sign_cos = ((k+2)>>2)&1 */
- mulpd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
- andl $1, %ecx /* sign_sin = sign_x ^ ((k>>2)&1) */
- addpd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
- mulpd %xmm4, %xmm3 /*t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
- testl $2, %eax /* n&2 != 0 ? */
- addpd %xmm4, %xmm3 /*t+t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
- jnz L(sin_result_sin_poly)
-
-/*L(sin_result_cos_poly):*/
- /*
- * Here if
- * cos(x) = poly_sin * sign_cos
- * sin(x) = poly_cos * sign_sin
- */
- movsd MO2(DP_ONES,%ecx,8), %xmm4/* 0|sign_sin */
- movhpd MO2(DP_ONES,%edx,8), %xmm4/* sign_cos|sign_sin */
- mulpd %xmm4, %xmm3 /* result_cos|result_sin */
- movl ARG_SIN_PTR, %eax
- cvtpd2ps %xmm3, %xmm0 /* SP results */
- movl ARG_COS_PTR, %ecx
- movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
- shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
- movss %xmm0, (%ecx) /* store cos(x) */
- RETURN
-
- .p2align 4
-L(sin_result_sin_poly):
- /*
- * Here if
- * sin(x) = poly_sin * sign_sin
- * cos(x) = poly_cos * sign_cos
- */
- movsd MO2(DP_ONES,%edx,8), %xmm4/* 0|sign_cos */
- movhpd MO2(DP_ONES,%ecx,8), %xmm4/* sign_sin|sign_cos */
- mulpd %xmm4, %xmm3 /* result_sin|result_cos */
- movl ARG_SIN_PTR, %eax
- cvtpd2ps %xmm3, %xmm0 /* SP results */
- movl ARG_COS_PTR, %ecx
- movss %xmm0, (%ecx) /* store cos(x) from xmm0[0] */
- shufps $1, %xmm0, %xmm0 /* move sin(x) to xmm0[0] */
- movss %xmm0, (%eax) /* store sin(x) */
- RETURN
-
- .p2align 4
-L(large_args):
- /* Here if |x|>=9*Pi/4 */
- cmpl $0x7f800000, %eax /* x is Inf or NaN ? */
- jae L(arg_inf_or_nan)
-
- /* Here if finite |x|>=9*Pi/4 */
- cmpl $0x4b000000, %eax /* |x|<2^23 ? */
- jae L(very_large_args)
-
- /* Here if 9*Pi/4<=|x|<2^23 */
- movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
- mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
- cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
- addl $1, %eax /* k+1 */
- movl %eax, %edx
- andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
- cvtsi2sdl %edx, %xmm4 /* DP j */
- movl ARG_X, %ecx /* Load x */
- movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
- shrl $29, %ecx /* (sign of x) << 2 */
- mulsd %xmm4, %xmm2 /* -j*PIO4HI */
- movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
- addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
- mulsd %xmm3, %xmm4 /* j*PIO4LO */
- addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
- jmp L(reconstruction)
-
- .p2align 4
-L(very_large_args):
- /* Here if finite |x|>=2^23 */
-
- /* bitpos = (ix>>23) - BIAS_32 + 59; */
- shrl $23, %eax /* eb = biased exponent of x */
- subl $68, %eax /* bitpos=eb-0x7f+59, where 0x7f */
- /*is exponent bias */
- movl $28, %ecx /* %cl=28 */
- movl %eax, %edx /* bitpos copy */
-
- /* j = bitpos/28; */
- div %cl /* j in register %al=%ax/%cl */
- movapd %xmm0, %xmm3 /* |x| */
- andl $0xff, %eax /* clear unneeded remainder from %ah*/
-
- imull $28, %eax, %ecx /* j*28 */
- movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
- movapd %xmm0, %xmm5 /* |x| */
- mulsd -2*8+MO2(_FPI,%eax,8), %xmm3/* tmp3 = FPI[j-2]*|x| */
- movapd %xmm0, %xmm1 /* |x| */
- mulsd -1*8+MO2(_FPI,%eax,8), %xmm5/* tmp2 = FPI[j-1]*|x| */
- mulsd 0*8+MO2(_FPI,%eax,8), %xmm0/* tmp0 = FPI[j]*|x| */
- addl $19, %ecx /* j*28+19 */
- mulsd 1*8+MO2(_FPI,%eax,8), %xmm1/* tmp1 = FPI[j+1]*|x| */
- cmpl %ecx, %edx /* bitpos>=j*28+19 ? */
- jl L(very_large_skip1)
-
- /* Here if bitpos>=j*28+19 */
- andpd %xmm3, %xmm4 /* HI(tmp3) */
- subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
-L(very_large_skip1):
-
- movsd MO1(DP_2POW52), %xmm6
- movapd %xmm5, %xmm2 /* tmp2 copy */
- addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
- movl $1, %edx
- addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
- movsd 8+MO1(DP_2POW52), %xmm4
- movd %xmm6, %eax /* k = I64_LO(tmp6); */
- addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
- movl ARG_X, %ecx /* Load x */
- comisd %xmm5, %xmm4 /* tmp4 > tmp5 ? */
- jbe L(very_large_skip2)
-
- /* Here if tmp4 > tmp5 */
- subl $1, %eax /* k-- */
- addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
-L(very_large_skip2):
-
- andl %eax, %edx /* k&1 */
- subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
- addsd MO2(DP_ZERONE,%edx,8), %xmm3/* t = DP_ZERONE[k&1] + tmp3 */
- addsd %xmm2, %xmm3 /* t += tmp2 */
- shrl $29, %ecx /* (sign of x) << 2 */
- addsd %xmm3, %xmm0 /* t += tmp0 */
- addl $1, %eax /* n=k+1 */
- addsd %xmm1, %xmm0 /* t += tmp1 */
- mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
-
- jmp L(reconstruction) /* end of very_large_args peth */
-
- .p2align 4
-L(arg_less_pio4):
- /* Here if |x|<Pi/4 */
- cmpl $0x3d000000, %eax /* |x|<2^-5 ? */
- jl L(arg_less_2pn5)
-
- /* Here if 2^-5<=|x|<Pi/4 */
- movaps %xmm0, %xmm3 /* DP x */
- movhpd MO1(DP_ONES), %xmm3 /* DP 1|x */
- mulsd %xmm0, %xmm0 /* DP y=x^2 */
- unpcklpd %xmm0, %xmm0 /* DP y|y */
- movaps %xmm0, %xmm1 /* y|y */
- mulpd %xmm0, %xmm0 /* z=x^4|z=x^4 */
-
- movapd MO1(DP_SC4), %xmm4 /* S4 */
- mulpd %xmm0, %xmm4 /* z*S4 */
- movapd MO1(DP_SC3), %xmm5 /* S3 */
- mulpd %xmm0, %xmm5 /* z*S3 */
- addpd MO1(DP_SC2), %xmm4 /* S2+z*S4 */
- mulpd %xmm0, %xmm4 /* z*(S2+z*S4) */
- addpd MO1(DP_SC1), %xmm5 /* S1+z*S3 */
- mulpd %xmm0, %xmm5 /* z*(S1+z*S3) */
- addpd MO1(DP_SC0), %xmm4 /* S0+z*(S2+z*S4) */
- mulpd %xmm1, %xmm4 /* y*(S0+z*(S2+z*S4)) */
- mulpd %xmm3, %xmm5 /* x*z*(S1+z*S3) */
- mulpd %xmm3, %xmm4 /* x*y*(S0+z*(S2+z*S4)) */
- addpd %xmm5, %xmm4 /*x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
- movl ARG_SIN_PTR, %eax
- addpd %xmm4, %xmm3 /*x+x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
- movl ARG_COS_PTR, %ecx
- cvtpd2ps %xmm3, %xmm0 /* SP results */
- movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
- shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
- movss %xmm0, (%ecx) /* store cos(x) */
- RETURN
-
- .p2align 4
-L(arg_less_2pn5):
- /* Here if |x|<2^-5 */
- cmpl $0x32000000, %eax /* |x|<2^-27 ? */
- jl L(arg_less_2pn27)
-
- /* Here if 2^-27<=|x|<2^-5 */
- movaps %xmm0, %xmm1 /* DP x */
- movhpd MO1(DP_ONES), %xmm1 /* DP 1|x */
- mulsd %xmm0, %xmm0 /* DP x^2 */
- unpcklpd %xmm0, %xmm0 /* DP x^2|x^2 */
-
- movaps MO1(DP_SINCOS2_1), %xmm3/* DP DP_SIN2_1 */
- mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_1 */
- addpd MO1(DP_SINCOS2_0), %xmm3/* DP DP_SIN2_0+x^2*DP_SIN2_1 */
- mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_0+x^4*DP_SIN2_1 */
- mulpd %xmm1, %xmm3 /* DP x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
- addpd %xmm1, %xmm3 /* DP x+x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
- movl ARG_SIN_PTR, %eax
- cvtpd2ps %xmm3, %xmm0 /* SP results */
- movl ARG_COS_PTR, %ecx
- movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
- shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
- movss %xmm0, (%ecx) /* store cos(x) */
- RETURN
-
- .p2align 4
-L(arg_less_2pn27):
- movss ARG_X, %xmm7 /* SP x */
- cmpl $0, %eax /* x=0 ? */
- je L(arg_zero) /* in case x=0 return sin(+-0)==+-0 */
- /* Here if |x|<2^-27 */
- /*
- * Special cases here:
- * sin(subnormal) raises inexact/underflow
- * sin(min_normalized) raises inexact/underflow
- * sin(normalized) raises inexact
- * cos(here)=1-|x| (raising inexact)
- */
- movaps %xmm0, %xmm3 /* DP x */
- mulsd MO1(DP_SMALL), %xmm0 /* DP x*DP_SMALL */
- subsd %xmm0, %xmm3 /* DP sin result is x-x*DP_SMALL */
- andps MO1(SP_ABS_MASK), %xmm7 /* SP |x| */
- cvtsd2ss %xmm3, %xmm0 /* sin(x) */
- movl ARG_SIN_PTR, %eax
- movss MO1(SP_ONE), %xmm1 /* SP 1.0 */
- movss %xmm0, (%eax) /* sin(x) store */
- movl ARG_COS_PTR, %ecx
- subss %xmm7, %xmm1 /* cos(x) */
- movss %xmm1, (%ecx) /* cos(x) store */
- RETURN
-
- .p2align 4
-L(arg_zero):
- movss MO1(SP_ONE), %xmm0 /* 1.0 */
- movl ARG_SIN_PTR, %eax
- movl ARG_COS_PTR, %ecx
- movss %xmm7, (%eax) /* sin(+-0)==x */
- movss %xmm0, (%ecx) /* cos(+-0)==1 */
- RETURN
-
- .p2align 4
-L(arg_inf_or_nan):
- movss ARG_X, %xmm7 /* SP x */
- /* Here if |x| is Inf or NAN */
- jne L(skip_errno_setting) /* in case of x is NaN */
-
- /* Here if x is Inf. Set errno to EDOM. */
- call JUMPTARGET(__errno_location)
- movl $EDOM, (%eax)
-
- .p2align 4
-L(skip_errno_setting):
- /* Here if |x| is Inf or NAN. Continued. */
- subss %xmm7, %xmm7 /* x-x, result is NaN */
- movl ARG_SIN_PTR, %eax
- movl ARG_COS_PTR, %ecx
- movss %xmm7, (%eax)
- movss %xmm7, (%ecx)
- RETURN
-END(__sincosf_sse2)
-
- .section .rodata, "a"
- .p2align 3
-L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
- .long 0x00000000,0x00000000
- .long 0x54442d18,0x3fe921fb
- .long 0x54442d18,0x3ff921fb
- .long 0x7f3321d2,0x4002d97c
- .long 0x54442d18,0x400921fb
- .long 0x2955385e,0x400f6a7a
- .long 0x7f3321d2,0x4012d97c
- .long 0xe9bba775,0x4015fdbb
- .long 0x54442d18,0x401921fb
- .long 0xbeccb2bb,0x401c463a
- .long 0x2955385e,0x401f6a7a
- .type L(PIO4J), @object
- ASM_SIZE_DIRECTIVE(L(PIO4J))
-
- .p2align 3
-L(_FPI): /* 4/Pi broken into sum of positive DP values */
- .long 0x00000000,0x00000000
- .long 0x6c000000,0x3ff45f30
- .long 0x2a000000,0x3e3c9c88
- .long 0xa8000000,0x3c54fe13
- .long 0xd0000000,0x3aaf47d4
- .long 0x6c000000,0x38fbb81b
- .long 0xe0000000,0x3714acc9
- .long 0x7c000000,0x3560e410
- .long 0x56000000,0x33bca2c7
- .long 0xac000000,0x31fbd778
- .long 0xe0000000,0x300b7246
- .long 0xe8000000,0x2e5d2126
- .long 0x48000000,0x2c970032
- .long 0xe8000000,0x2ad77504
- .long 0xe0000000,0x290921cf
- .long 0xb0000000,0x274deb1c
- .long 0xe0000000,0x25829a73
- .long 0xbe000000,0x23fd1046
- .long 0x10000000,0x2224baed
- .long 0x8e000000,0x20709d33
- .long 0x80000000,0x1e535a2f
- .long 0x64000000,0x1cef904e
- .long 0x30000000,0x1b0d6398
- .long 0x24000000,0x1964ce7d
- .long 0x16000000,0x17b908bf
- .type L(_FPI), @object
- ASM_SIZE_DIRECTIVE(L(_FPI))
-
-/* Coefficients of polynomials for */
-/* sin(x)~=x+x*x^2*(DP_SIN2_0+x^2*DP_SIN2_1) in low DP part, */
-/* cos(x)~=1+1*x^2*(DP_COS2_0+x^2*DP_COS2_1) in high DP part, */
-/* for |x|<2^-5. */
- .p2align 4
-L(DP_SINCOS2_0):
- .long 0x5543d49d,0xbfc55555
- .long 0xff5cc6fd,0xbfdfffff
- .type L(DP_SINCOS2_0), @object
- ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_0))
-
- .p2align 4
-L(DP_SINCOS2_1):
- .long 0x75cec8c5,0x3f8110f4
- .long 0xb178dac5,0x3fa55514
- .type L(DP_SINCOS2_1), @object
- ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_1))
-
- .p2align 3
-L(DP_ZERONE):
- .long 0x00000000,0x00000000 /* 0.0 */
- .long 0x00000000,0xbff00000 /* 1.0 */
- .type L(DP_ZERONE), @object
- ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
-
- .p2align 3
-L(DP_ONES):
- .long 0x00000000,0x3ff00000 /* +1.0 */
- .long 0x00000000,0xbff00000 /* -1.0 */
- .type L(DP_ONES), @object
- ASM_SIZE_DIRECTIVE(L(DP_ONES))
-
-/* Coefficients of polynomials for */
-/* sin(t)~=t+t*t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))) in low DP part, */
-/* cos(t)~=1+1*t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))) in high DP part, */
-/* for |t|<Pi/4. */
- .p2align 4
-L(DP_SC4):
- .long 0x1674b58a,0xbe5a947e
- .long 0xdd8844d7,0xbe923c97
- .type L(DP_SC4), @object
- ASM_SIZE_DIRECTIVE(L(DP_SC4))
-
- .p2align 4
-L(DP_SC3):
- .long 0x64e6b5b4,0x3ec71d72
- .long 0x9ac43cc0,0x3efa00eb
- .type L(DP_SC3), @object
- ASM_SIZE_DIRECTIVE(L(DP_SC3))
-
- .p2align 4
-L(DP_SC2):
- .long 0x8b4bd1f9,0xbf2a019f
- .long 0x348b6874,0xbf56c16b
- .type L(DP_SC2), @object
- ASM_SIZE_DIRECTIVE(L(DP_SC2))
-
- .p2align 4
-L(DP_SC1):
- .long 0x10c2688b,0x3f811111
- .long 0x545c50c7,0x3fa55555
- .type L(DP_SC1), @object
- ASM_SIZE_DIRECTIVE(L(DP_SC1))
-
- .p2align 4
-L(DP_SC0):
- .long 0x55551cd9,0xbfc55555
- .long 0xfffe98ae,0xbfdfffff
- .type L(DP_SC0), @object
- ASM_SIZE_DIRECTIVE(L(DP_SC0))
-
- .p2align 3
-L(DP_SMALL):
- .long 0x00000000,0x3cd00000 /* 2^(-50) */
- .type L(DP_SMALL), @object
- ASM_SIZE_DIRECTIVE(L(DP_SMALL))
-
- .p2align 3
-L(DP_PIO4):
- .long 0x54442d18,0x3fe921fb /* Pi/4 */
- .type L(DP_PIO4), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4))
-
- .p2align 3
-L(DP_2POW52):
- .long 0x00000000,0x43300000 /* +2^52 */
- .long 0x00000000,0xc3300000 /* -2^52 */
- .type L(DP_2POW52), @object
- ASM_SIZE_DIRECTIVE(L(DP_2POW52))
-
- .p2align 3
-L(DP_INVPIO4):
- .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
- .type L(DP_INVPIO4), @object
- ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
-
- .p2align 3
-L(DP_PIO4HI):
- .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
- .type L(DP_PIO4HI), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
-
- .p2align 3
-L(DP_PIO4LO):
- .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
- .type L(DP_PIO4LO), @object
- ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
-
- .p2align 2
-L(SP_INVPIO4):
- .long 0x3fa2f983 /* 4/Pi */
- .type L(SP_INVPIO4), @object
- ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
-
- .p2align 4
-L(DP_ABS_MASK): /* Mask for getting DP absolute value */
- .long 0xffffffff,0x7fffffff
- .long 0xffffffff,0x7fffffff
- .type L(DP_ABS_MASK), @object
- ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
-
- .p2align 3
-L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
- .long 0x00000000,0xffffffff
- .type L(DP_HI_MASK), @object
- ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
-
- .p2align 4
-L(SP_ABS_MASK): /* Mask for getting SP absolute value */
- .long 0x7fffffff,0x7fffffff
- .long 0x7fffffff,0x7fffffff
- .type L(SP_ABS_MASK), @object
- ASM_SIZE_DIRECTIVE(L(SP_ABS_MASK))
-
- .p2align 2
-L(SP_ONE):
- .long 0x3f800000 /* 1.0 */
- .type L(SP_ONE), @object
- ASM_SIZE_DIRECTIVE(L(SP_ONE))
-
-weak_alias(__sincosf, sincosf)
diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
new file mode 100644
index 0000000000..46363fc11a
--- /dev/null
+++ b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
@@ -0,0 +1,3 @@
+#define SINCOSF __sincosf_sse2
+#define SECTION __attribute__ ((section (".text.sse2")))
+#include <sysdeps/ieee754/flt-32/s_sincosf.c>
--
2.34.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 2/5] i686: Use generic cosf implementation for SSE2 version
2022-05-31 21:34 ` [PATCH 2/5] i686: Use generic cosf implementation for SSE2 version Adhemerval Zanella
@ 2022-05-31 21:46 ` H.J. Lu
0 siblings, 0 replies; 19+ messages in thread
From: H.J. Lu @ 2022-05-31 21:46 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library
On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
> Performance seems to be similar (gcc 11.2.1 on a Ryzen 9 5900X):
>
> * s_cosf-sse2.S:
> "cosf": {
> "workload-random": {
> "duration": 3.74987e+09,
> "iterations": 9.616e+07,
> "reciprocal-throughput": 15.8141,
> "latency": 62.1782,
> "max-throughput": 6.32346e+07,
> "min-throughput": 1.60828e+07
> }
> }
>
> * generic s_cosf.c:
> "cosf": {
> "workload-random": {
> "duration": 3.87298e+09,
> "iterations": 1.00968e+08,
> "reciprocal-throughput": 18.3448,
> "latency": 58.3722,
> "max-throughput": 5.45113e+07,
> "min-throughput": 1.71314e+07
> }
> }
>
> Checked on i686-linux-gnu.
> ---
> sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
> .../i386/i686/fpu/multiarch/libm-test-ulps | 4 +
> sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S | 552 ------------------
> sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c | 3 +
> sysdeps/ieee754/flt-32/s_cosf.c | 5 +
> 5 files changed, 13 insertions(+), 552 deletions(-)
> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
>
> diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
> index c0fa9761d3..b8f52c311f 100644
> --- a/sysdeps/i386/i686/fpu/multiarch/Makefile
> +++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
> @@ -7,4 +7,5 @@ CFLAGS-e_expf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-e_log2f-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
> +CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
> endif
> diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> index cc21e6907f..1bfad77adc 100644
> --- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> +++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> @@ -746,21 +746,25 @@ ldouble: 1
>
> Function: "cos":
> double: 1
> +float: 1
> float128: 2
> ldouble: 1
>
> Function: "cos_downward":
> double: 1
> +float: 1
> float128: 3
> ldouble: 3
>
> Function: "cos_towardzero":
> double: 1
> +float: 1
> float128: 1
> ldouble: 2
>
> Function: "cos_upward":
> double: 1
> +float: 1
> float128: 2
> ldouble: 2
>
> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
> deleted file mode 100644
> index 28cc68dbc7..0000000000
> --- a/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
> +++ /dev/null
> @@ -1,552 +0,0 @@
> -/* Optimized with sse2 version of cosf
> - Copyright (C) 2012-2022 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <sysdep.h>
> -#include <errno.h>
> -
> -/* Short algorithm description:
> - *
> - * 1) if |x| == 0: return 1.0-|x|.
> - * 2) if |x| < 2^-27: return 1.0-|x|.
> - * 3) if |x| < 2^-5 : return 1.0+x^2*DP_COS2_0+x^5*DP_COS2_1.
> - * 4) if |x| < Pi/4: return 1.0+x^2*(C0+x^2*(C1+x^2*(C2+x^2*(C3+x^2*C4)))).
> - * 5) if |x| < 9*Pi/4:
> - * 5.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+3,
> - * t=|x|-j*Pi/4.
> - * 5.2) Reconstruction:
> - * s = (-1.0)^((n>>2)&1)
> - * if(n&2 != 0) {
> - * using cos(t) polynomial for |t|<Pi/4, result is
> - * s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4))))).
> - * } else {
> - * using sin(t) polynomial for |t|<Pi/4, result is
> - * s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4))))).
> - * }
> - * 6) if |x| < 2^23, large args:
> - * 6.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+3,
> - * t=|x|-j*Pi/4.
> - * 6.2) Reconstruction same as (5.2).
> - * 7) if |x| >= 2^23, very large args:
> - * 7.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+3,
> - * t=|x|-j*Pi/4.
> - * 7.2) Reconstruction same as (5.2).
> - * 8) if x is Inf, return x-x, and set errno=EDOM.
> - * 9) if x is NaN, return x-x.
> - *
> - * Special cases:
> - * cos(+-0) = 1 not raising inexact,
> - * cos(subnormal) raises inexact,
> - * cos(min_normalized) raises inexact,
> - * cos(normalized) raises inexact,
> - * cos(Inf) = NaN, raises invalid, sets errno to EDOM,
> - * cos(NaN) = NaN.
> - */
> -
> -#ifdef PIC
> -# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
> -# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
> -# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
> -# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
> -# define PUSH(REG) pushl REG; CFI_PUSH(REG)
> -# define POP(REG) popl REG; CFI_POP(REG)
> -# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
> -# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
> -# define ARG_X 8(%esp)
> -#else
> -# define MO1(symbol) L(symbol)
> -# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
> -# define ENTRANCE
> -# define RETURN ret
> -# define ARG_X 4(%esp)
> -#endif
> -
> - .text
> -ENTRY(__cosf_sse2)
> - /* Input: single precision x on stack at address ARG_X */
> -
> - ENTRANCE
> - movl ARG_X, %eax /* Bits of x */
> - cvtss2sd ARG_X, %xmm0 /* DP x */
> - andl $0x7fffffff, %eax /* |x| */
> -
> - cmpl $0x3f490fdb, %eax /* |x|<Pi/4? */
> - jb L(arg_less_pio4)
> -
> - /* Here if |x|>=Pi/4 */
> - movd %eax, %xmm3 /* SP |x| */
> - andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
> - movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
> -
> - cmpl $0x40e231d6, %eax /* |x|<9*Pi/4? */
> - jae L(large_args)
> -
> - /* Here if Pi/4<=|x|<9*Pi/4 */
> - mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
> - cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
> - addl $1, %eax /* k+1 */
> - movl $0x0e, %edx
> - andl %eax, %edx /* j = (k+1)&0x0e */
> - addl $2, %eax /* n */
> - subsd MO2(PIO4J,%edx,8), %xmm0 /* t = |x| - j * Pi/4 */
> -
> -L(reconstruction):
> - /* Input: %eax=n, %xmm0=t */
> - testl $2, %eax /* n&2 != 0? */
> - jz L(sin_poly)
> -
> -/*L(cos_poly):*/
> - /* Here if cos(x) calculated using cos(t) polynomial for |t|<Pi/4:
> - * y = t*t; z = y*y;
> - * s = sign(x) * (-1.0)^((n>>2)&1)
> - * result = s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))))
> - */
> - shrl $2, %eax /* n>>2 */
> - mulsd %xmm0, %xmm0 /* y=t^2 */
> - andl $1, %eax /* (n>>2)&1 */
> - movaps %xmm0, %xmm1 /* y */
> - mulsd %xmm0, %xmm0 /* z=t^4 */
> -
> - movsd MO1(DP_C4), %xmm4 /* C4 */
> - mulsd %xmm0, %xmm4 /* z*C4 */
> - movsd MO1(DP_C3), %xmm3 /* C3 */
> - mulsd %xmm0, %xmm3 /* z*C3 */
> - addsd MO1(DP_C2), %xmm4 /* C2+z*C4 */
> - mulsd %xmm0, %xmm4 /* z*(C2+z*C4) */
> - lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
> - addsd MO1(DP_C1), %xmm3 /* C1+z*C3 */
> - mulsd %xmm0, %xmm3 /* z*(C1+z*C3) */
> - addsd MO1(DP_C0), %xmm4 /* C0+z*(C2+z*C4) */
> - mulsd %xmm1, %xmm4 /* y*(C0+z*(C2+z*C4)) */
> -
> - addsd %xmm4, %xmm3 /* y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
> - /* 1.0+y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
> - addsd MO1(DP_ONES), %xmm3
> -
> - mulsd MO2(DP_ONES,%eax,8), %xmm3 /* DP result */
> - movsd %xmm3, 0(%esp) /* Move result from sse... */
> - fldl 0(%esp) /* ...to FPU. */
> - /* Return back 4 bytes of stack frame */
> - lea 8(%esp), %esp
> - RETURN
> -
> - .p2align 4
> -L(sin_poly):
> - /* Here if cos(x) calculated using sin(t) polynomial for |t|<Pi/4:
> - * y = t*t; z = y*y;
> - * s = sign(x) * (-1.0)^((n>>2)&1)
> - * result = s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))))
> - */
> -
> - movaps %xmm0, %xmm4 /* t */
> - shrl $2, %eax /* n>>2 */
> - mulsd %xmm0, %xmm0 /* y=t^2 */
> - andl $1, %eax /* (n>>2)&1 */
> - movaps %xmm0, %xmm1 /* y */
> - mulsd %xmm0, %xmm0 /* z=t^4 */
> -
> - movsd MO1(DP_S4), %xmm2 /* S4 */
> - mulsd %xmm0, %xmm2 /* z*S4 */
> - movsd MO1(DP_S3), %xmm3 /* S3 */
> - mulsd %xmm0, %xmm3 /* z*S3 */
> - lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
> - addsd MO1(DP_S2), %xmm2 /* S2+z*S4 */
> - mulsd %xmm0, %xmm2 /* z*(S2+z*S4) */
> - addsd MO1(DP_S1), %xmm3 /* S1+z*S3 */
> - mulsd %xmm0, %xmm3 /* z*(S1+z*S3) */
> - addsd MO1(DP_S0), %xmm2 /* S0+z*(S2+z*S4) */
> - mulsd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
> - /* t*s, where s = sign(x) * (-1.0)^((n>>2)&1) */
> - mulsd MO2(DP_ONES,%eax,8), %xmm4
> - addsd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - /* t*s*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - mulsd %xmm4, %xmm3
> - /* t*s*(1.0+y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - addsd %xmm4, %xmm3
> - movsd %xmm3, 0(%esp) /* Move result from sse... */
> - fldl 0(%esp) /* ...to FPU. */
> - /* Return back 4 bytes of stack frame */
> - lea 8(%esp), %esp
> - RETURN
> -
> - .p2align 4
> -L(large_args):
> - /* Here if |x|>=9*Pi/4 */
> - cmpl $0x7f800000, %eax /* x is Inf or NaN? */
> - jae L(arg_inf_or_nan)
> -
> - /* Here if finite |x|>=9*Pi/4 */
> - cmpl $0x4b000000, %eax /* |x|<2^23? */
> - jae L(very_large_args)
> -
> - /* Here if 9*Pi/4<=|x|<2^23 */
> - movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
> - mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
> - cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
> - addl $1, %eax /* k+1 */
> - movl %eax, %edx
> - andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
> - cvtsi2sdl %edx, %xmm4 /* DP j */
> - movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
> - mulsd %xmm4, %xmm2 /* -j*PIO4HI */
> - movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
> - addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
> - addl $2, %eax /* n */
> - mulsd %xmm3, %xmm4 /* j*PIO4LO */
> - addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
> - jmp L(reconstruction)
> -
> - .p2align 4
> -L(very_large_args):
> - /* Here if finite |x|>=2^23 */
> -
> - /* bitpos = (ix>>23) - BIAS_32 + 59; */
> - shrl $23, %eax /* eb = biased exponent of x */
> - /* bitpos = eb - 0x7f + 59, where 0x7f is exponent bias */
> - subl $68, %eax
> - movl $28, %ecx /* %cl=28 */
> - movl %eax, %edx /* bitpos copy */
> -
> - /* j = bitpos/28; */
> - div %cl /* j in register %al=%ax/%cl */
> - movapd %xmm0, %xmm3 /* |x| */
> - /* clear unneeded remainder from %ah */
> - andl $0xff, %eax
> -
> - imull $28, %eax, %ecx /* j*28 */
> - movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
> - movapd %xmm0, %xmm5 /* |x| */
> - mulsd -2*8+MO2(_FPI,%eax,8), %xmm3 /* tmp3 = FPI[j-2]*|x| */
> - movapd %xmm0, %xmm1 /* |x| */
> - mulsd -1*8+MO2(_FPI,%eax,8), %xmm5 /* tmp2 = FPI[j-1]*|x| */
> - mulsd 0*8+MO2(_FPI,%eax,8), %xmm0 /* tmp0 = FPI[j]*|x| */
> - addl $19, %ecx /* j*28+19 */
> - mulsd 1*8+MO2(_FPI,%eax,8), %xmm1 /* tmp1 = FPI[j+1]*|x| */
> - cmpl %ecx, %edx /* bitpos>=j*28+19? */
> - jl L(very_large_skip1)
> -
> - /* Here if bitpos>=j*28+19 */
> - andpd %xmm3, %xmm4 /* HI(tmp3) */
> - subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
> -L(very_large_skip1):
> -
> - movsd MO1(DP_2POW52), %xmm6
> - movapd %xmm5, %xmm2 /* tmp2 copy */
> - addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
> - movl $1, %edx
> - addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
> - movsd 8+MO1(DP_2POW52), %xmm4
> - movd %xmm6, %eax /* k = I64_LO(tmp6); */
> - addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
> - comisd %xmm5, %xmm4 /* tmp4 > tmp5? */
> - jbe L(very_large_skip2)
> -
> - /* Here if tmp4 > tmp5 */
> - subl $1, %eax /* k-- */
> - addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
> -L(very_large_skip2):
> -
> - andl %eax, %edx /* k&1 */
> - subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
> - addsd MO2(DP_ZERONE,%edx,8), %xmm3 /* t = DP_ZERONE[k&1] + tmp3 */
> - addsd %xmm2, %xmm3 /* t += tmp2 */
> - addsd %xmm3, %xmm0 /* t += tmp0 */
> - addl $3, %eax /* n=k+3 */
> - addsd %xmm1, %xmm0 /* t += tmp1 */
> - mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
> -
> - jmp L(reconstruction) /* end of very_large_args peth */
> -
> - .p2align 4
> -L(arg_less_pio4):
> - /* Here if |x|<Pi/4 */
> - cmpl $0x3d000000, %eax /* |x|<2^-5? */
> - jl L(arg_less_2pn5)
> -
> - /* Here if 2^-5<=|x|<Pi/4 */
> - mulsd %xmm0, %xmm0 /* y=x^2 */
> - movaps %xmm0, %xmm1 /* y */
> - mulsd %xmm0, %xmm0 /* z=x^4 */
> - movsd MO1(DP_C4), %xmm3 /* C4 */
> - mulsd %xmm0, %xmm3 /* z*C4 */
> - movsd MO1(DP_C3), %xmm5 /* C3 */
> - mulsd %xmm0, %xmm5 /* z*C3 */
> - addsd MO1(DP_C2), %xmm3 /* C2+z*C4 */
> - mulsd %xmm0, %xmm3 /* z*(C2+z*C4) */
> - addsd MO1(DP_C1), %xmm5 /* C1+z*C3 */
> - mulsd %xmm0, %xmm5 /* z*(C1+z*C3) */
> - addsd MO1(DP_C0), %xmm3 /* C0+z*(C2+z*C4) */
> - mulsd %xmm1, %xmm3 /* y*(C0+z*(C2+z*C4)) */
> - addsd %xmm5, %xmm3 /* y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
> - /* 1.0 + y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
> - addsd MO1(DP_ONES), %xmm3
> - cvtsd2ss %xmm3, %xmm3 /* SP result */
> -
> -L(epilogue):
> - lea -4(%esp), %esp /* Borrow 4 bytes of stack frame */
> - movss %xmm3, 0(%esp) /* Move result from sse... */
> - flds 0(%esp) /* ...to FPU. */
> - /* Return back 4 bytes of stack frame */
> - lea 4(%esp), %esp
> - RETURN
> -
> - .p2align 4
> -L(arg_less_2pn5):
> - /* Here if |x|<2^-5 */
> - cmpl $0x32000000, %eax /* |x|<2^-27? */
> - jl L(arg_less_2pn27)
> -
> - /* Here if 2^-27<=|x|<2^-5 */
> - mulsd %xmm0, %xmm0 /* DP x^2 */
> - movsd MO1(DP_COS2_1), %xmm3 /* DP DP_COS2_1 */
> - mulsd %xmm0, %xmm3 /* DP x^2*DP_COS2_1 */
> - addsd MO1(DP_COS2_0), %xmm3 /* DP DP_COS2_0+x^2*DP_COS2_1 */
> - mulsd %xmm0, %xmm3 /* DP x^2*DP_COS2_0+x^4*DP_COS2_1 */
> - /* DP 1.0+x^2*DP_COS2_0+x^4*DP_COS2_1 */
> - addsd MO1(DP_ONES), %xmm3
> - cvtsd2ss %xmm3, %xmm3 /* SP result */
> - jmp L(epilogue)
> -
> - .p2align 4
> -L(arg_less_2pn27):
> - /* Here if |x|<2^-27 */
> - movss ARG_X, %xmm0 /* x */
> - andps MO1(SP_ABS_MASK),%xmm0 /* |x| */
> - movss MO1(SP_ONE), %xmm3 /* 1.0 */
> - subss %xmm0, %xmm3 /* result is 1.0-|x| */
> - jmp L(epilogue)
> -
> - .p2align 4
> -L(arg_inf_or_nan):
> - /* Here if |x| is Inf or NAN */
> - jne L(skip_errno_setting) /* in case of x is NaN */
> -
> - /* Here if x is Inf. Set errno to EDOM. */
> - call JUMPTARGET(__errno_location)
> - movl $EDOM, (%eax)
> -
> - .p2align 4
> -L(skip_errno_setting):
> - /* Here if |x| is Inf or NAN. Continued. */
> - movss ARG_X, %xmm3 /* load x */
> - subss %xmm3, %xmm3 /* Result is NaN */
> - jmp L(epilogue)
> -END(__cosf_sse2)
> -
> - .section .rodata, "a"
> - .p2align 3
> -L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
> - .long 0x00000000,0x00000000
> - .long 0x54442d18,0x3fe921fb
> - .long 0x54442d18,0x3ff921fb
> - .long 0x7f3321d2,0x4002d97c
> - .long 0x54442d18,0x400921fb
> - .long 0x2955385e,0x400f6a7a
> - .long 0x7f3321d2,0x4012d97c
> - .long 0xe9bba775,0x4015fdbb
> - .long 0x54442d18,0x401921fb
> - .long 0xbeccb2bb,0x401c463a
> - .long 0x2955385e,0x401f6a7a
> - .type L(PIO4J), @object
> - ASM_SIZE_DIRECTIVE(L(PIO4J))
> -
> - .p2align 3
> -L(_FPI): /* 4/Pi broken into sum of positive DP values */
> - .long 0x00000000,0x00000000
> - .long 0x6c000000,0x3ff45f30
> - .long 0x2a000000,0x3e3c9c88
> - .long 0xa8000000,0x3c54fe13
> - .long 0xd0000000,0x3aaf47d4
> - .long 0x6c000000,0x38fbb81b
> - .long 0xe0000000,0x3714acc9
> - .long 0x7c000000,0x3560e410
> - .long 0x56000000,0x33bca2c7
> - .long 0xac000000,0x31fbd778
> - .long 0xe0000000,0x300b7246
> - .long 0xe8000000,0x2e5d2126
> - .long 0x48000000,0x2c970032
> - .long 0xe8000000,0x2ad77504
> - .long 0xe0000000,0x290921cf
> - .long 0xb0000000,0x274deb1c
> - .long 0xe0000000,0x25829a73
> - .long 0xbe000000,0x23fd1046
> - .long 0x10000000,0x2224baed
> - .long 0x8e000000,0x20709d33
> - .long 0x80000000,0x1e535a2f
> - .long 0x64000000,0x1cef904e
> - .long 0x30000000,0x1b0d6398
> - .long 0x24000000,0x1964ce7d
> - .long 0x16000000,0x17b908bf
> - .type L(_FPI), @object
> - ASM_SIZE_DIRECTIVE(L(_FPI))
> -
> -/* Coefficients of polynomial
> - for cos(x)~=1.0+x^2*DP_COS2_0+x^4*DP_COS2_1, |x|<2^-5. */
> - .p2align 3
> -L(DP_COS2_0):
> - .long 0xff5cc6fd,0xbfdfffff
> - .type L(DP_COS2_0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_COS2_0))
> -
> - .p2align 3
> -L(DP_COS2_1):
> - .long 0xb178dac5,0x3fa55514
> - .type L(DP_COS2_1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_COS2_1))
> -
> - .p2align 3
> -L(DP_ZERONE):
> - .long 0x00000000,0x00000000 /* 0.0 */
> - .long 0x00000000,0xbff00000 /* 1.0 */
> - .type L(DP_ZERONE),@object
> - ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
> -
> - .p2align 3
> -L(DP_ONES):
> - .long 0x00000000,0x3ff00000 /* +1.0 */
> - .long 0x00000000,0xbff00000 /* -1.0 */
> - .type L(DP_ONES), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ONES))
> -
> -/* Coefficients of polynomial
> - for sin(t)~=t+t^3*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))), |t|<Pi/4. */
> - .p2align 3
> -L(DP_S3):
> - .long 0x64e6b5b4,0x3ec71d72
> - .type L(DP_S3), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S3))
> -
> - .p2align 3
> -L(DP_S1):
> - .long 0x10c2688b,0x3f811111
> - .type L(DP_S1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S1))
> -
> - .p2align 3
> -L(DP_S4):
> - .long 0x1674b58a,0xbe5a947e
> - .type L(DP_S4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S4))
> -
> - .p2align 3
> -L(DP_S2):
> - .long 0x8b4bd1f9,0xbf2a019f
> - .type L(DP_S2), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S2))
> -
> - .p2align 3
> -L(DP_S0):
> - .long 0x55551cd9,0xbfc55555
> - .type L(DP_S0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S0))
> -
> -/* Coefficients of polynomial
> - for cos(t)~=1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))), |t|<Pi/4. */
> - .p2align 3
> -L(DP_C3):
> - .long 0x9ac43cc0,0x3efa00eb
> - .type L(DP_C3), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C3))
> -
> - .p2align 3
> -L(DP_C1):
> - .long 0x545c50c7,0x3fa55555
> - .type L(DP_C1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C1))
> -
> - .p2align 3
> -L(DP_C4):
> - .long 0xdd8844d7,0xbe923c97
> - .type L(DP_C4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C4))
> -
> - .p2align 3
> -L(DP_C2):
> - .long 0x348b6874,0xbf56c16b
> - .type L(DP_C2), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C2))
> -
> - .p2align 3
> -L(DP_C0):
> - .long 0xfffe98ae,0xbfdfffff
> - .type L(DP_C0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C0))
> -
> - .p2align 3
> -L(DP_PIO4):
> - .long 0x54442d18,0x3fe921fb /* Pi/4 */
> - .type L(DP_PIO4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4))
> -
> - .p2align 3
> -L(DP_2POW52):
> - .long 0x00000000,0x43300000 /* +2^52 */
> - .long 0x00000000,0xc3300000 /* -2^52 */
> - .type L(DP_2POW52), @object
> - ASM_SIZE_DIRECTIVE(L(DP_2POW52))
> -
> - .p2align 3
> -L(DP_INVPIO4):
> - .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
> - .type L(DP_INVPIO4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
> -
> - .p2align 3
> -L(DP_PIO4HI):
> - .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
> - .type L(DP_PIO4HI), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
> -
> - .p2align 3
> -L(DP_PIO4LO):
> - .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
> - .type L(DP_PIO4LO), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
> -
> - .p2align 2
> -L(SP_INVPIO4):
> - .long 0x3fa2f983 /* 4/Pi */
> - .type L(SP_INVPIO4), @object
> - ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
> -
> - .p2align 4
> -L(DP_ABS_MASK): /* Mask for getting DP absolute value */
> - .long 0xffffffff,0x7fffffff
> - .long 0xffffffff,0x7fffffff
> - .type L(DP_ABS_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
> -
> - .p2align 3
> -L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
> - .long 0x00000000,0xffffffff
> - .type L(DP_HI_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
> -
> - .p2align 4
> -L(SP_ABS_MASK): /* Mask for getting SP absolute value */
> - .long 0x7fffffff,0x7fffffff
> - .long 0x7fffffff,0x7fffffff
> - .type L(SP_ABS_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(SP_ABS_MASK))
> -
> - .p2align 2
> -L(SP_ONE):
> - .long 0x3f800000 /* 1.0 */
> - .type L(SP_ONE), @object
> - ASM_SIZE_DIRECTIVE(L(SP_ONE))
> -
> -weak_alias (__cosf, cosf)
> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
> new file mode 100644
> index 0000000000..3db7a2474f
> --- /dev/null
> +++ b/sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
> @@ -0,0 +1,3 @@
> +#define COSF __cosf_sse2
> +#define SECTION __attribute__ ((section (".text.sse2")))
> +#include <sysdeps/ieee754/flt-32/s_cosf.c>
> diff --git a/sysdeps/ieee754/flt-32/s_cosf.c b/sysdeps/ieee754/flt-32/s_cosf.c
> index ee9fa8565b..1236244244 100644
> --- a/sysdeps/ieee754/flt-32/s_cosf.c
> +++ b/sysdeps/ieee754/flt-32/s_cosf.c
> @@ -23,6 +23,10 @@
> #include "math_config.h"
> #include "s_sincosf.h"
>
> +#ifndef SECTION
> +# define SECTION
> +#endif
> +
> #ifndef COSF
> # define COSF_FUNC __cosf
> #else
> @@ -35,6 +39,7 @@
> arithmetic.
> */
> float
> +SECTION
> COSF_FUNC (float y)
> {
> double x = y;
> --
> 2.34.1
>
LGTM.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Thanks.
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/5] benchtests: Add workload name for cosf
2022-05-31 21:34 ` [PATCH 1/5] benchtests: Add workload name for cosf Adhemerval Zanella
@ 2022-05-31 21:47 ` H.J. Lu
0 siblings, 0 replies; 19+ messages in thread
From: H.J. Lu @ 2022-05-31 21:47 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library
On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
> So it can show both reciprocal-throughput and latency.
> ---
> benchtests/cosf-inputs | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/benchtests/cosf-inputs b/benchtests/cosf-inputs
> index 73d6991cbc..6fba27833a 100644
> --- a/benchtests/cosf-inputs
> +++ b/benchtests/cosf-inputs
> @@ -1,7 +1,7 @@
> ## args: float
> ## ret: float
> ## includes: math.h
> -
> +## name: workload-random
> -0x1.d24c56p-1
> -0x1.50ab58p-1
> 0x1.112339p-6
> --
> 2.34.1
>
LGTM.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Thanks.
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 3/5] i686: Use generic sinf implementation for SSE2 version
2022-05-31 21:35 ` [PATCH 3/5] i686: Use generic sinf " Adhemerval Zanella
@ 2022-05-31 21:47 ` H.J. Lu
0 siblings, 0 replies; 19+ messages in thread
From: H.J. Lu @ 2022-05-31 21:47 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library
On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
> Performance seems to be similar (gcc 11.2.1 on a Ryzen 9 5900X),
> the generic algorithm shows slight better performance for
> the 'workload-huge.wrf' input set.
>
> * s_sinf-sse2.S:
> "sinf": {
> "": {
> "duration": 3.72405e+09,
> "iterations": 2.38374e+08,
> "max": 63.973,
> "min": 11.211,
> "mean": 15.6227
> },
> "workload-random.wrf": {
> "duration": 3.76923e+09,
> "iterations": 8.4e+07,
> "reciprocal-throughput": 17.6355,
> "latency": 72.108,
> "max-throughput": 5.67037e+07,
> "min-throughput": 1.38681e+07
> },
> "workload-huge.wrf": {
> "duration": 3.76943e+09,
> "iterations": 6e+07,
> "reciprocal-throughput": 29.3493,
> "latency": 96.2985,
> "max-throughput": 3.40724e+07,
> "min-throughput": 1.03844e+07
> }
> }
>
> * generic s_sinf.c:
> "sinf": {
> "": {
> "duration": 3.70989e+09,
> "iterations": 2.18025e+08,
> "max": 69.782,
> "min": 11.1,
> "mean": 17.0159
> },
> "workload-random.wrf": {
> "duration": 3.77213e+09,
> "iterations": 9.6e+07,
> "reciprocal-throughput": 17.5402,
> "latency": 61.0459,
> "max-throughput": 5.70119e+07,
> "min-throughput": 1.63811e+07
> },
> "workload-huge.wrf": {
> "duration": 3.81576e+09,
> "iterations": 5.6e+07,
> "reciprocal-throughput": 38.2111,
> "latency": 98.0659,
> "max-throughput": 2.61704e+07,
> "min-throughput": 1.01972e+07
> }
> }
>
> Checked on i686-linux-gnu.
> ---
> sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
> .../i386/i686/fpu/multiarch/libm-test-ulps | 4 +
> sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S | 565 ------------------
> sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c | 3 +
> sysdeps/ieee754/flt-32/s_sinf.c | 5 +
> 5 files changed, 13 insertions(+), 565 deletions(-)
> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
>
> diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
> index b8f52c311f..c2cd5cefef 100644
> --- a/sysdeps/i386/i686/fpu/multiarch/Makefile
> +++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
> @@ -8,4 +8,5 @@ CFLAGS-e_log2f-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
> +CFLAGS-s_sinf-sse2.c = -msse2 -mfpmath=sse
> endif
> diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> index 1bfad77adc..0357d7bd14 100644
> --- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> +++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> @@ -1463,21 +1463,25 @@ ldouble: 4
>
> Function: "sin":
> double: 1
> +float: 1
> float128: 2
> ldouble: 2
>
> Function: "sin_downward":
> double: 1
> +float: 1
> float128: 3
> ldouble: 3
>
> Function: "sin_towardzero":
> double: 1
> +float: 1
> float128: 2
> ldouble: 2
>
> Function: "sin_upward":
> double: 1
> +float: 1
> float128: 3
> ldouble: 3
>
> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
> deleted file mode 100644
> index c473eeb4f7..0000000000
> --- a/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
> +++ /dev/null
> @@ -1,565 +0,0 @@
> -/* Optimized with sse2 version of sinf
> - Copyright (C) 2012-2022 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <sysdep.h>
> -#include <errno.h>
> -
> -/* Short algorithm description:
> - *
> - * 1) if |x| == 0: return x.
> - * 2) if |x| < 2^-27: return x-x*DP_SMALL, raise underflow only when needed.
> - * 3) if |x| < 2^-5 : return x+x^3*DP_SIN2_0+x^5*DP_SIN2_1.
> - * 4) if |x| < Pi/4: return x+x^3*(S0+x^2*(S1+x^2*(S2+x^2*(S3+x^2*S4)))).
> - * 5) if |x| < 9*Pi/4:
> - * 5.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+1,
> - * t=|x|-j*Pi/4.
> - * 5.2) Reconstruction:
> - * s = sign(x) * (-1.0)^((n>>2)&1)
> - * if(n&2 != 0) {
> - * using cos(t) polynomial for |t|<Pi/4, result is
> - * s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4))))).
> - * } else {
> - * using sin(t) polynomial for |t|<Pi/4, result is
> - * s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4))))).
> - * }
> - * 6) if |x| < 2^23, large args:
> - * 6.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1,
> - * t=|x|-j*Pi/4.
> - * 6.2) Reconstruction same as (5.2).
> - * 7) if |x| >= 2^23, very large args:
> - * 7.1) Range reduction: k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1,
> - * t=|x|-j*Pi/4.
> - * 7.2) Reconstruction same as (5.2).
> - * 8) if x is Inf, return x-x, and set errno=EDOM.
> - * 9) if x is NaN, return x-x.
> - *
> - * Special cases:
> - * sin(+-0) = +-0 not raising inexact/underflow,
> - * sin(subnormal) raises inexact/underflow,
> - * sin(min_normalized) raises inexact/underflow,
> - * sin(normalized) raises inexact,
> - * sin(Inf) = NaN, raises invalid, sets errno to EDOM,
> - * sin(NaN) = NaN.
> - */
> -
> -#ifdef PIC
> -# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
> -# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
> -# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
> -# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
> -# define PUSH(REG) pushl REG; CFI_PUSH(REG)
> -# define POP(REG) popl REG; CFI_POP(REG)
> -# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
> -# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
> -# define ARG_X 8(%esp)
> -#else
> -# define MO1(symbol) L(symbol)
> -# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
> -# define ENTRANCE
> -# define RETURN ret
> -# define ARG_X 4(%esp)
> -#endif
> -
> - .text
> -ENTRY(__sinf_sse2)
> - /* Input: single precision x on stack at address ARG_X */
> -
> - ENTRANCE
> - movl ARG_X, %eax /* Bits of x */
> - cvtss2sd ARG_X, %xmm0 /* DP x */
> - andl $0x7fffffff, %eax /* |x| */
> -
> - cmpl $0x3f490fdb, %eax /* |x|<Pi/4? */
> - jb L(arg_less_pio4)
> -
> - /* Here if |x|>=Pi/4 */
> - movd %eax, %xmm3 /* SP |x| */
> - andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
> - movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
> -
> - cmpl $0x40e231d6, %eax /* |x|<9*Pi/4? */
> - jae L(large_args)
> -
> - /* Here if Pi/4<=|x|<9*Pi/4 */
> - mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
> - movl ARG_X, %ecx /* Load x */
> - cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
> - shrl $31, %ecx /* sign of x */
> - addl $1, %eax /* k+1 */
> - movl $0x0e, %edx
> - andl %eax, %edx /* j = (k+1)&0x0e */
> - subsd MO2(PIO4J,%edx,8), %xmm0 /* t = |x| - j * Pi/4 */
> -
> -L(reconstruction):
> - /* Input: %eax=n, %xmm0=t, %ecx=sign(x) */
> - testl $2, %eax /* n&2 != 0? */
> - jz L(sin_poly)
> -
> -/*L(cos_poly):*/
> - /* Here if sin(x) calculated using cos(t) polynomial for |t|<Pi/4:
> - * y = t*t; z = y*y;
> - * s = sign(x) * (-1.0)^((n>>2)&1)
> - * result = s * (1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))))
> - */
> - shrl $2, %eax /* n>>2 */
> - mulsd %xmm0, %xmm0 /* y=t^2 */
> - andl $1, %eax /* (n>>2)&1 */
> - movaps %xmm0, %xmm1 /* y */
> - mulsd %xmm0, %xmm0 /* z=t^4 */
> -
> - movsd MO1(DP_C4), %xmm4 /* C4 */
> - mulsd %xmm0, %xmm4 /* z*C4 */
> - xorl %eax, %ecx /* (-1.0)^((n>>2)&1) XOR sign(x) */
> - movsd MO1(DP_C3), %xmm3 /* C3 */
> - mulsd %xmm0, %xmm3 /* z*C3 */
> - addsd MO1(DP_C2), %xmm4 /* C2+z*C4 */
> - mulsd %xmm0, %xmm4 /* z*(C2+z*C4) */
> - lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
> - addsd MO1(DP_C1), %xmm3 /* C1+z*C3 */
> - mulsd %xmm0, %xmm3 /* z*(C1+z*C3) */
> - addsd MO1(DP_C0), %xmm4 /* C0+z*(C2+z*C4) */
> - mulsd %xmm1, %xmm4 /* y*(C0+z*(C2+z*C4)) */
> -
> - addsd %xmm4, %xmm3 /* y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
> - /* 1.0+y*(C0+y*(C1+y*(C2+y*(C3+y*C4)))) */
> - addsd MO1(DP_ONES), %xmm3
> -
> - mulsd MO2(DP_ONES,%ecx,8), %xmm3 /* DP result */
> - movsd %xmm3, 0(%esp) /* Move result from sse... */
> - fldl 0(%esp) /* ...to FPU. */
> - /* Return back 4 bytes of stack frame */
> - lea 8(%esp), %esp
> - RETURN
> -
> - .p2align 4
> -L(sin_poly):
> - /* Here if sin(x) calculated using sin(t) polynomial for |t|<Pi/4:
> - * y = t*t; z = y*y;
> - * s = sign(x) * (-1.0)^((n>>2)&1)
> - * result = s * t * (1.0+t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))))
> - */
> -
> - movaps %xmm0, %xmm4 /* t */
> - shrl $2, %eax /* n>>2 */
> - mulsd %xmm0, %xmm0 /* y=t^2 */
> - andl $1, %eax /* (n>>2)&1 */
> - movaps %xmm0, %xmm1 /* y */
> - xorl %eax, %ecx /* (-1.0)^((n>>2)&1) XOR sign(x) */
> - mulsd %xmm0, %xmm0 /* z=t^4 */
> -
> - movsd MO1(DP_S4), %xmm2 /* S4 */
> - mulsd %xmm0, %xmm2 /* z*S4 */
> - movsd MO1(DP_S3), %xmm3 /* S3 */
> - mulsd %xmm0, %xmm3 /* z*S3 */
> - lea -8(%esp), %esp /* Borrow 4 bytes of stack frame */
> - addsd MO1(DP_S2), %xmm2 /* S2+z*S4 */
> - mulsd %xmm0, %xmm2 /* z*(S2+z*S4) */
> - addsd MO1(DP_S1), %xmm3 /* S1+z*S3 */
> - mulsd %xmm0, %xmm3 /* z*(S1+z*S3) */
> - addsd MO1(DP_S0), %xmm2 /* S0+z*(S2+z*S4) */
> - mulsd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
> - /* t*s, where s = sign(x) * (-1.0)^((n>>2)&1) */
> - mulsd MO2(DP_ONES,%ecx,8), %xmm4
> - addsd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - /* t*s*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - mulsd %xmm4, %xmm3
> - /* t*s*(1.0+y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - addsd %xmm4, %xmm3
> - movsd %xmm3, 0(%esp) /* Move result from sse... */
> - fldl 0(%esp) /* ...to FPU. */
> - /* Return back 4 bytes of stack frame */
> - lea 8(%esp), %esp
> - RETURN
> -
> - .p2align 4
> -L(large_args):
> - /* Here if |x|>=9*Pi/4 */
> - cmpl $0x7f800000, %eax /* x is Inf or NaN? */
> - jae L(arg_inf_or_nan)
> -
> - /* Here if finite |x|>=9*Pi/4 */
> - cmpl $0x4b000000, %eax /* |x|<2^23? */
> - jae L(very_large_args)
> -
> - /* Here if 9*Pi/4<=|x|<2^23 */
> - movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
> - mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
> - cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
> - addl $1, %eax /* k+1 */
> - movl %eax, %edx
> - andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
> - cvtsi2sdl %edx, %xmm4 /* DP j */
> - movl ARG_X, %ecx /* Load x */
> - movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
> - shrl $31, %ecx /* sign bit of x */
> - mulsd %xmm4, %xmm2 /* -j*PIO4HI */
> - movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
> - addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
> - mulsd %xmm3, %xmm4 /* j*PIO4LO */
> - addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
> - jmp L(reconstruction)
> -
> - .p2align 4
> -L(very_large_args):
> - /* Here if finite |x|>=2^23 */
> -
> - /* bitpos = (ix>>23) - BIAS_32 + 59; */
> - shrl $23, %eax /* eb = biased exponent of x */
> - /* bitpos = eb - 0x7f + 59, where 0x7f is exponent bias */
> - subl $68, %eax
> - movl $28, %ecx /* %cl=28 */
> - movl %eax, %edx /* bitpos copy */
> -
> - /* j = bitpos/28; */
> - div %cl /* j in register %al=%ax/%cl */
> - movapd %xmm0, %xmm3 /* |x| */
> - /* clear unneeded remainder from %ah */
> - andl $0xff, %eax
> -
> - imull $28, %eax, %ecx /* j*28 */
> - movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
> - movapd %xmm0, %xmm5 /* |x| */
> - mulsd -2*8+MO2(_FPI,%eax,8), %xmm3 /* tmp3 = FPI[j-2]*|x| */
> - movapd %xmm0, %xmm1 /* |x| */
> - mulsd -1*8+MO2(_FPI,%eax,8), %xmm5 /* tmp2 = FPI[j-1]*|x| */
> - mulsd 0*8+MO2(_FPI,%eax,8), %xmm0 /* tmp0 = FPI[j]*|x| */
> - addl $19, %ecx /* j*28+19 */
> - mulsd 1*8+MO2(_FPI,%eax,8), %xmm1 /* tmp1 = FPI[j+1]*|x| */
> - cmpl %ecx, %edx /* bitpos>=j*28+19? */
> - jl L(very_large_skip1)
> -
> - /* Here if bitpos>=j*28+19 */
> - andpd %xmm3, %xmm4 /* HI(tmp3) */
> - subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
> -L(very_large_skip1):
> -
> - movsd MO1(DP_2POW52), %xmm6
> - movapd %xmm5, %xmm2 /* tmp2 copy */
> - addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
> - movl $1, %edx
> - addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
> - movsd 8+MO1(DP_2POW52), %xmm4
> - movd %xmm6, %eax /* k = I64_LO(tmp6); */
> - addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
> - movl ARG_X, %ecx /* Load x */
> - comisd %xmm5, %xmm4 /* tmp4 > tmp5? */
> - jbe L(very_large_skip2)
> -
> - /* Here if tmp4 > tmp5 */
> - subl $1, %eax /* k-- */
> - addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
> -L(very_large_skip2):
> -
> - andl %eax, %edx /* k&1 */
> - subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
> - addsd MO2(DP_ZERONE,%edx,8), %xmm3 /* t = DP_ZERONE[k&1] + tmp3 */
> - addsd %xmm2, %xmm3 /* t += tmp2 */
> - shrl $31, %ecx /* sign of x */
> - addsd %xmm3, %xmm0 /* t += tmp0 */
> - addl $1, %eax /* n=k+1 */
> - addsd %xmm1, %xmm0 /* t += tmp1 */
> - mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
> -
> - jmp L(reconstruction) /* end of very_large_args peth */
> -
> - .p2align 4
> -L(arg_less_pio4):
> - /* Here if |x|<Pi/4 */
> - cmpl $0x3d000000, %eax /* |x|<2^-5? */
> - jl L(arg_less_2pn5)
> -
> - /* Here if 2^-5<=|x|<Pi/4 */
> - movaps %xmm0, %xmm3 /* x */
> - mulsd %xmm0, %xmm0 /* y=x^2 */
> - movaps %xmm0, %xmm1 /* y */
> - mulsd %xmm0, %xmm0 /* z=x^4 */
> - movsd MO1(DP_S4), %xmm4 /* S4 */
> - mulsd %xmm0, %xmm4 /* z*S4 */
> - movsd MO1(DP_S3), %xmm5 /* S3 */
> - mulsd %xmm0, %xmm5 /* z*S3 */
> - addsd MO1(DP_S2), %xmm4 /* S2+z*S4 */
> - mulsd %xmm0, %xmm4 /* z*(S2+z*S4) */
> - addsd MO1(DP_S1), %xmm5 /* S1+z*S3 */
> - mulsd %xmm0, %xmm5 /* z*(S1+z*S3) */
> - addsd MO1(DP_S0), %xmm4 /* S0+z*(S2+z*S4) */
> - mulsd %xmm1, %xmm4 /* y*(S0+z*(S2+z*S4)) */
> - mulsd %xmm3, %xmm5 /* x*z*(S1+z*S3) */
> - mulsd %xmm3, %xmm4 /* x*y*(S0+z*(S2+z*S4)) */
> - /* x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - addsd %xmm5, %xmm4
> - /* x + x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - addsd %xmm4, %xmm3
> - cvtsd2ss %xmm3, %xmm3 /* SP result */
> -
> -L(epilogue):
> - lea -4(%esp), %esp /* Borrow 4 bytes of stack frame */
> - movss %xmm3, 0(%esp) /* Move result from sse... */
> - flds 0(%esp) /* ...to FPU. */
> - /* Return back 4 bytes of stack frame */
> - lea 4(%esp), %esp
> - RETURN
> -
> - .p2align 4
> -L(arg_less_2pn5):
> - /* Here if |x|<2^-5 */
> - cmpl $0x32000000, %eax /* |x|<2^-27? */
> - jl L(arg_less_2pn27)
> -
> - /* Here if 2^-27<=|x|<2^-5 */
> - movaps %xmm0, %xmm1 /* DP x */
> - mulsd %xmm0, %xmm0 /* DP x^2 */
> - movsd MO1(DP_SIN2_1), %xmm3 /* DP DP_SIN2_1 */
> - mulsd %xmm0, %xmm3 /* DP x^2*DP_SIN2_1 */
> - addsd MO1(DP_SIN2_0), %xmm3 /* DP DP_SIN2_0+x^2*DP_SIN2_1 */
> - mulsd %xmm0, %xmm3 /* DP x^2*DP_SIN2_0+x^4*DP_SIN2_1 */
> - mulsd %xmm1, %xmm3 /* DP x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
> - addsd %xmm1, %xmm3 /* DP x+x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
> - cvtsd2ss %xmm3, %xmm3 /* SP result */
> - jmp L(epilogue)
> -
> - .p2align 4
> -L(arg_less_2pn27):
> - movss ARG_X, %xmm3 /* SP x */
> - cmpl $0, %eax /* x=0? */
> - je L(epilogue) /* in case x=0 return sin(+-0)==+-0 */
> - /* Here if |x|<2^-27 */
> - /*
> - * Special cases here:
> - * sin(subnormal) raises inexact/underflow
> - * sin(min_normalized) raises inexact/underflow
> - * sin(normalized) raises inexact
> - */
> - movaps %xmm0, %xmm3 /* Copy of DP x */
> - mulsd MO1(DP_SMALL), %xmm0 /* x*DP_SMALL */
> - subsd %xmm0, %xmm3 /* Result is x-x*DP_SMALL */
> - cvtsd2ss %xmm3, %xmm3 /* Result converted to SP */
> - jmp L(epilogue)
> -
> - .p2align 4
> -L(arg_inf_or_nan):
> - /* Here if |x| is Inf or NAN */
> - jne L(skip_errno_setting) /* in case of x is NaN */
> -
> - /* Here if x is Inf. Set errno to EDOM. */
> - call JUMPTARGET(__errno_location)
> - movl $EDOM, (%eax)
> -
> - .p2align 4
> -L(skip_errno_setting):
> - /* Here if |x| is Inf or NAN. Continued. */
> - movss ARG_X, %xmm3 /* load x */
> - subss %xmm3, %xmm3 /* Result is NaN */
> - jmp L(epilogue)
> -END(__sinf_sse2)
> -
> - .section .rodata, "a"
> - .p2align 3
> -L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
> - .long 0x00000000,0x00000000
> - .long 0x54442d18,0x3fe921fb
> - .long 0x54442d18,0x3ff921fb
> - .long 0x7f3321d2,0x4002d97c
> - .long 0x54442d18,0x400921fb
> - .long 0x2955385e,0x400f6a7a
> - .long 0x7f3321d2,0x4012d97c
> - .long 0xe9bba775,0x4015fdbb
> - .long 0x54442d18,0x401921fb
> - .long 0xbeccb2bb,0x401c463a
> - .long 0x2955385e,0x401f6a7a
> - .type L(PIO4J), @object
> - ASM_SIZE_DIRECTIVE(L(PIO4J))
> -
> - .p2align 3
> -L(_FPI): /* 4/Pi broken into sum of positive DP values */
> - .long 0x00000000,0x00000000
> - .long 0x6c000000,0x3ff45f30
> - .long 0x2a000000,0x3e3c9c88
> - .long 0xa8000000,0x3c54fe13
> - .long 0xd0000000,0x3aaf47d4
> - .long 0x6c000000,0x38fbb81b
> - .long 0xe0000000,0x3714acc9
> - .long 0x7c000000,0x3560e410
> - .long 0x56000000,0x33bca2c7
> - .long 0xac000000,0x31fbd778
> - .long 0xe0000000,0x300b7246
> - .long 0xe8000000,0x2e5d2126
> - .long 0x48000000,0x2c970032
> - .long 0xe8000000,0x2ad77504
> - .long 0xe0000000,0x290921cf
> - .long 0xb0000000,0x274deb1c
> - .long 0xe0000000,0x25829a73
> - .long 0xbe000000,0x23fd1046
> - .long 0x10000000,0x2224baed
> - .long 0x8e000000,0x20709d33
> - .long 0x80000000,0x1e535a2f
> - .long 0x64000000,0x1cef904e
> - .long 0x30000000,0x1b0d6398
> - .long 0x24000000,0x1964ce7d
> - .long 0x16000000,0x17b908bf
> - .type L(_FPI), @object
> - ASM_SIZE_DIRECTIVE(L(_FPI))
> -
> -/* Coefficients of polynomial
> - for sin(x)~=x+x^3*DP_SIN2_0+x^5*DP_SIN2_1, |x|<2^-5. */
> - .p2align 3
> -L(DP_SIN2_0):
> - .long 0x5543d49d,0xbfc55555
> - .type L(DP_SIN2_0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SIN2_0))
> -
> - .p2align 3
> -L(DP_SIN2_1):
> - .long 0x75cec8c5,0x3f8110f4
> - .type L(DP_SIN2_1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SIN2_1))
> -
> - .p2align 3
> -L(DP_ZERONE):
> - .long 0x00000000,0x00000000 /* 0.0 */
> - .long 0x00000000,0xbff00000 /* 1.0 */
> - .type L(DP_ZERONE), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
> -
> - .p2align 3
> -L(DP_ONES):
> - .long 0x00000000,0x3ff00000 /* +1.0 */
> - .long 0x00000000,0xbff00000 /* -1.0 */
> - .type L(DP_ONES), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ONES))
> -
> -/* Coefficients of polynomial
> - for sin(t)~=t+t^3*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))), |t|<Pi/4. */
> - .p2align 3
> -L(DP_S3):
> - .long 0x64e6b5b4,0x3ec71d72
> - .type L(DP_S3), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S3))
> -
> - .p2align 3
> -L(DP_S1):
> - .long 0x10c2688b,0x3f811111
> - .type L(DP_S1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S1))
> -
> - .p2align 3
> -L(DP_S4):
> - .long 0x1674b58a,0xbe5a947e
> - .type L(DP_S4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S4))
> -
> - .p2align 3
> -L(DP_S2):
> - .long 0x8b4bd1f9,0xbf2a019f
> - .type L(DP_S2), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S2))
> -
> - .p2align 3
> -L(DP_S0):
> - .long 0x55551cd9,0xbfc55555
> - .type L(DP_S0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_S0))
> -
> - .p2align 3
> -L(DP_SMALL):
> - .long 0x00000000,0x3cd00000 /* 2^(-50) */
> - .type L(DP_SMALL), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SMALL))
> -
> -/* Coefficients of polynomial
> - for cos(t)~=1.0+t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))), |t|<Pi/4. */
> - .p2align 3
> -L(DP_C3):
> - .long 0x9ac43cc0,0x3efa00eb
> - .type L(DP_C3), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C3))
> -
> - .p2align 3
> -L(DP_C1):
> - .long 0x545c50c7,0x3fa55555
> - .type L(DP_C1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C1))
> -
> - .p2align 3
> -L(DP_C4):
> - .long 0xdd8844d7,0xbe923c97
> - .type L(DP_C4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C4))
> -
> - .p2align 3
> -L(DP_C2):
> - .long 0x348b6874,0xbf56c16b
> - .type L(DP_C2), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C2))
> -
> - .p2align 3
> -L(DP_C0):
> - .long 0xfffe98ae,0xbfdfffff
> - .type L(DP_C0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_C0))
> -
> - .p2align 3
> -L(DP_PIO4):
> - .long 0x54442d18,0x3fe921fb /* Pi/4 */
> - .type L(DP_PIO4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4))
> -
> - .p2align 3
> -L(DP_2POW52):
> - .long 0x00000000,0x43300000 /* +2^52 */
> - .long 0x00000000,0xc3300000 /* -2^52 */
> - .type L(DP_2POW52), @object
> - ASM_SIZE_DIRECTIVE(L(DP_2POW52))
> -
> - .p2align 3
> -L(DP_INVPIO4):
> - .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
> - .type L(DP_INVPIO4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
> -
> - .p2align 3
> -L(DP_PIO4HI):
> - .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
> - .type L(DP_PIO4HI), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
> -
> - .p2align 3
> -L(DP_PIO4LO):
> - .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
> - .type L(DP_PIO4LO), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
> -
> - .p2align 2
> -L(SP_INVPIO4):
> - .long 0x3fa2f983 /* 4/Pi */
> - .type L(SP_INVPIO4), @object
> - ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
> -
> - .p2align 4
> -L(DP_ABS_MASK): /* Mask for getting DP absolute value */
> - .long 0xffffffff,0x7fffffff
> - .long 0xffffffff,0x7fffffff
> - .type L(DP_ABS_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
> -
> - .p2align 3
> -L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
> - .long 0x00000000,0xffffffff
> - .type L(DP_HI_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
> -
> -weak_alias (__sinf, sinf)
> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
> new file mode 100644
> index 0000000000..e7dc0dc1a6
> --- /dev/null
> +++ b/sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
> @@ -0,0 +1,3 @@
> +#define SINF __sinf_sse2
> +#define SECTION __attribute__ ((section (".text.sse2")))
> +#include <sysdeps/ieee754/flt-32/s_sinf.c>
> diff --git a/sysdeps/ieee754/flt-32/s_sinf.c b/sysdeps/ieee754/flt-32/s_sinf.c
> index ad54b1c7e6..e157ff071d 100644
> --- a/sysdeps/ieee754/flt-32/s_sinf.c
> +++ b/sysdeps/ieee754/flt-32/s_sinf.c
> @@ -23,6 +23,10 @@
> #include "math_config.h"
> #include "s_sincosf.h"
>
> +#ifndef SECTION
> +# define SECTION
> +#endif
> +
> #ifndef SINF
> # define SINF_FUNC __sinf
> #else
> @@ -35,6 +39,7 @@
> arithmetic.
> */
> float
> +SECTION
> SINF_FUNC (float y)
> {
> double x = y;
> --
> 2.34.1
>
LGTM.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Thanks.
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 4/5] benchtests: Add workload name for sincosf
2022-05-31 21:35 ` [PATCH 4/5] benchtests: Add workload name for sincosf Adhemerval Zanella
@ 2022-05-31 21:48 ` H.J. Lu
0 siblings, 0 replies; 19+ messages in thread
From: H.J. Lu @ 2022-05-31 21:48 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library
On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
> So it can show both reciprocal-throughput and latency.
> ---
> benchtests/sincosf-inputs | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/benchtests/sincosf-inputs b/benchtests/sincosf-inputs
> index ef5668a38c..0a8495a67c 100644
> --- a/benchtests/sincosf-inputs
> +++ b/benchtests/sincosf-inputs
> @@ -1,5 +1,6 @@
> ## includes: math.h
> ## args: float:<float *>:<float *>
> +## name: workload-random
> 0x1.759424p-1
> -0x1.324949p0
> -0x1.855528p8
> --
> 2.34.1
>
LGTM.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Thanks.
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
` (4 preceding siblings ...)
2022-05-31 21:35 ` [PATCH 5/5] math: Use generic sinf implementation for SSE2 version Adhemerval Zanella
@ 2022-05-31 21:48 ` Noah Goldstein
2022-05-31 21:49 ` H.J. Lu
5 siblings, 1 reply; 19+ messages in thread
From: Noah Goldstein @ 2022-05-31 21:48 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library, H . J . Lu
On Tue, May 31, 2022 at 4:35 PM Adhemerval Zanella via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> Performance seems to be similar, with generic implementation showing
> slight better performance for sinf large inputs and for sincosf.
>
> Although generic implementation shows slight lower precision (the
> ulp data required some adjustments), the code size shows a good
> improvement:
>
How much loss of precision?
> i686-linux-gnu-master$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
> math/s_sincosf-sse2.os
> text data bss dec hex filename
> 1555 0 0 1555 613 math/s_sinf-sse2.os
> 1551 0 0 1551 60f math/s_cosf-sse2.os
> 1754 0 0 1754 6da math/s_sincosf-sse2.os
>
> i686-linux-gnu-patched$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
> math/s_sincosf-sse2.os
> text data bss dec hex filename
> 1059 0 0 1059 423 math/s_sinf-sse2.os
> 1067 0 0 1067 42b math/s_cosf-sse2.os
> 993 0 0 993 3e1 math/s_sincosf-sse2.os
>
> And it also simplifies the code base a lot with less assembly
> implementations.
>
> Adhemerval Zanella (5):
> benchtests: Add workload name for cosf
> i686: Use generic cosf implementation for SSE2 version
> i686: Use generic sinf implementation for SSE2 version
> benchtests: Add workload name for sincosf
> math: Use generic sinf implementation for SSE2 version
>
> benchtests/cosf-inputs | 2 +-
> benchtests/sincosf-inputs | 1 +
> sysdeps/i386/i686/fpu/multiarch/Makefile | 3 +
> .../i386/i686/fpu/multiarch/libm-test-ulps | 11 +
> sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S | 552 -----------------
> sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c | 3 +
> .../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
> .../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
> sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S | 565 -----------------
> sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c | 3 +
> sysdeps/ieee754/flt-32/s_cosf.c | 5 +
> sysdeps/ieee754/flt-32/s_sinf.c | 5 +
> 12 files changed, 35 insertions(+), 1703 deletions(-)
> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686
2022-05-31 21:48 ` [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Noah Goldstein
@ 2022-05-31 21:49 ` H.J. Lu
2022-05-31 21:53 ` Noah Goldstein
0 siblings, 1 reply; 19+ messages in thread
From: H.J. Lu @ 2022-05-31 21:49 UTC (permalink / raw)
To: Noah Goldstein; +Cc: Adhemerval Zanella, GNU C Library
On Tue, May 31, 2022 at 2:48 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
>
> On Tue, May 31, 2022 at 4:35 PM Adhemerval Zanella via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
> >
> > Performance seems to be similar, with generic implementation showing
> > slight better performance for sinf large inputs and for sincosf.
> >
> > Although generic implementation shows slight lower precision (the
> > ulp data required some adjustments), the code size shows a good
> > improvement:
> >
>
> How much loss of precision?
About 1ULP.
> > i686-linux-gnu-master$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
> > math/s_sincosf-sse2.os
> > text data bss dec hex filename
> > 1555 0 0 1555 613 math/s_sinf-sse2.os
> > 1551 0 0 1551 60f math/s_cosf-sse2.os
> > 1754 0 0 1754 6da math/s_sincosf-sse2.os
> >
> > i686-linux-gnu-patched$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
> > math/s_sincosf-sse2.os
> > text data bss dec hex filename
> > 1059 0 0 1059 423 math/s_sinf-sse2.os
> > 1067 0 0 1067 42b math/s_cosf-sse2.os
> > 993 0 0 993 3e1 math/s_sincosf-sse2.os
> >
> > And it also simplifies the code base a lot with less assembly
> > implementations.
> >
> > Adhemerval Zanella (5):
> > benchtests: Add workload name for cosf
> > i686: Use generic cosf implementation for SSE2 version
> > i686: Use generic sinf implementation for SSE2 version
> > benchtests: Add workload name for sincosf
> > math: Use generic sinf implementation for SSE2 version
> >
> > benchtests/cosf-inputs | 2 +-
> > benchtests/sincosf-inputs | 1 +
> > sysdeps/i386/i686/fpu/multiarch/Makefile | 3 +
> > .../i386/i686/fpu/multiarch/libm-test-ulps | 11 +
> > sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S | 552 -----------------
> > sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c | 3 +
> > .../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
> > .../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
> > sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S | 565 -----------------
> > sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c | 3 +
> > sysdeps/ieee754/flt-32/s_cosf.c | 5 +
> > sysdeps/ieee754/flt-32/s_sinf.c | 5 +
> > 12 files changed, 35 insertions(+), 1703 deletions(-)
> > delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
> > create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
> > delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> > create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> > delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
> > create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
> >
> > --
> > 2.34.1
> >
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] math: Use generic sinf implementation for SSE2 version
2022-05-31 21:35 ` [PATCH 5/5] math: Use generic sinf implementation for SSE2 version Adhemerval Zanella
@ 2022-05-31 21:51 ` H.J. Lu
2022-06-01 12:42 ` Adhemerval Zanella
0 siblings, 1 reply; 19+ messages in thread
From: H.J. Lu @ 2022-05-31 21:51 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library
? On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
> The generic implementation shows slight better performance
> (gcc 11.2.1 on a Ryzen 9 5900X):
>
> * s_sincosf-sse2.S:
> "sincosf": {
> "workload-random": {
> "duration": 3.89961e+09,
> "iterations": 9.5472e+07,
> "reciprocal-throughput": 40.8429,
> "latency": 40.8483,
> "max-throughput": 2.4484e+07,
> "min-throughput": 2.44808e+07
> }
> }
>
> * generic s_cossinf.c:
> "sincosf": {
> "workload-random": {
> "duration": 3.71953e+09,
> "iterations": 1.48512e+08,
> "reciprocal-throughput": 25.0515,
> "latency": 25.0391,
> "max-throughput": 3.99177e+07,
> "min-throughput": 3.99375e+07
> }
> }
>
> Checked on i686-linux-gnu.
> ---
> sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
> .../i386/i686/fpu/multiarch/libm-test-ulps | 3 +
> .../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
> .../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
> 4 files changed, 7 insertions(+), 585 deletions(-)
> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
>
> diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
> index c2cd5cefef..c8b0a6528a 100644
> --- a/sysdeps/i386/i686/fpu/multiarch/Makefile
> +++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
> @@ -9,4 +9,5 @@ CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
> CFLAGS-s_sinf-sse2.c = -msse2 -mfpmath=sse
> +CFLAGS-s_sincosf-sse2.c = -msse2 -mfpmath=sse
> endif
> diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> index 0357d7bd14..284e3fa275 100644
> --- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> +++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> @@ -1492,16 +1492,19 @@ ldouble: 1
>
> Function: "sincos_downward":
> double: 1
> +float: 1
> float128: 3
> ldouble: 3
>
> Function: "sincos_towardzero":
> double: 1
> +float: 1
> float128: 2
> ldouble: 2
>
> Function: "sincos_upward":
> double: 1
> +float: 1
> float128: 3
> ldouble: 3
>
> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> deleted file mode 100644
> index 71e222b930..0000000000
> --- a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> +++ /dev/null
> @@ -1,585 +0,0 @@
> -/* Optimized with sse2 version of sincosf
> - Copyright (C) 2012-2022 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <sysdep.h>
> -#include <errno.h>
> -
> -/* Short algorithm description:
> - *
> - * 1) if |x|==0: sin(x)=x,
> - * cos(x)=1.
> - * 2) if |x|<2^-27: sin(x)=x-x*DP_SMALL, raising underflow only when needed,
> - * cos(x)=1-|x|.
> - * 3) if |x|<2^-5 : sin(x)=x+x*x^2*DP_SIN2_0+x^5*DP_SIN2_1,
> - * cos(x)=1+1*x^2*DP_COS2_0+x^5*DP_COS2_1
> - * 4) if |x|< Pi/4: sin(x)=x+x*x^2*(S0+x^2*(S1+x^2*(S2+x^2*(S3+x^2*S4)))),
> - * cos(x)=1+1*x^2*(C0+x^2*(C1+x^2*(C2+x^2*(C3+x^2*C4)))).
> - * 5) if |x| < 9*Pi/4:
> - * 5.1) Range reduction:
> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+1, t=|x|-j*Pi/4.
> - * 5.2) Reconstruction:
> - * sign_sin = sign(x) * (-1.0)^(( n >>2)&1)
> - * sign_cos = (-1.0)^(((n+2)>>2)&1)
> - * poly_sin = ((((S4*t^2 + S3)*t^2 + S2)*t^2 + S1)*t^2 + S0)*t^2*t+t
> - * poly_cos = ((((C4*t^2 + C3)*t^2 + C2)*t^2 + C1)*t^2 + C0)*t^2*s+s
> - * if(n&2 != 0) {
> - * using cos(t) and sin(t) polynomials for |t|<Pi/4, results are
> - * cos(x) = poly_sin * sign_cos
> - * sin(x) = poly_cos * sign_sin
> - * } else {
> - * sin(x) = poly_sin * sign_sin
> - * cos(x) = poly_cos * sign_cos
> - * }
> - * 6) if |x| < 2^23, large args:
> - * 6.1) Range reduction:
> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4
> - * 6.2) Reconstruction same as (5.2).
> - * 7) if |x| >= 2^23, very large args:
> - * 7.1) Range reduction:
> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4.
> - * 7.2) Reconstruction same as (5.2).
> - * 8) if x is Inf, return x-x, and set errno=EDOM.
> - * 9) if x is NaN, return x-x.
> - *
> - * Special cases:
> - * sin/cos(+-0) = +-0/1 not raising inexact/underflow,
> - * sin/cos(subnormal) raises inexact/underflow,
> - * sin/cos(min_normalized) raises inexact/underflow,
> - * sin/cos(normalized) raises inexact,
> - * sin/cos(Inf) = NaN, raises invalid, sets errno to EDOM,
> - * sin/cos(NaN) = NaN.
> - */
> -
> -#ifdef PIC
> -# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
> -# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
> -# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
> -# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
> -# define PUSH(REG) pushl REG; CFI_PUSH(REG)
> -# define POP(REG) popl REG; CFI_POP(REG)
> -# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
> -# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
> -# define ARG_X 8(%esp)
> -# define ARG_SIN_PTR 12(%esp)
> -# define ARG_COS_PTR 16(%esp)
> -#else
> -# define MO1(symbol) L(symbol)
> -# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
> -# define ENTRANCE
> -# define RETURN ret
> -# define ARG_X 4(%esp)
> -# define ARG_SIN_PTR 8(%esp)
> -# define ARG_COS_PTR 12(%esp)
> -#endif
> -
> - .text
> -ENTRY(__sincosf_sse2)
> - /* Input: single precision x on stack at address ARG_X */
> - /* pointer to sin result on stack at address ARG_SIN_PTR */
> - /* pointer to cos result on stack at address ARG_COS_PTR */
> -
> - ENTRANCE
> - movl ARG_X, %eax /* Bits of x */
> - cvtss2sd ARG_X, %xmm0 /* DP x */
> - andl $0x7fffffff, %eax /* |x| */
> -
> - cmpl $0x3f490fdb, %eax /* |x|<Pi/4 ? */
> - jb L(arg_less_pio4)
> -
> - /* Here if |x|>=Pi/4 */
> - movd %eax, %xmm3 /* SP |x| */
> - andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
> - movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
> -
> - cmpl $0x40e231d6, %eax /* |x|<9*Pi/4 ? */
> - jae L(large_args)
> -
> - /* Here if Pi/4<=|x|<9*Pi/4 */
> - mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
> - movl ARG_X, %ecx /* Load x */
> - cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
> - shrl $29, %ecx /* (sign of x) << 2 */
> - addl $1, %eax /* k+1 */
> - movl $0x0e, %edx
> - andl %eax, %edx /* j = (k+1)&0x0e */
> - subsd MO2(PIO4J,%edx,8), %xmm0/* t = |x| - j * Pi/4 */
> -
> -L(reconstruction):
> - /* Input: %eax=n, %xmm0=t, %ecx=sign(x) */
> -
> - movaps %xmm0, %xmm4 /* t */
> - movhpd MO1(DP_ONES), %xmm4 /* 1|t */
> - mulsd %xmm0, %xmm0 /* y=t^2 */
> - movl $2, %edx
> - unpcklpd %xmm0, %xmm0 /* y|y */
> - addl %eax, %edx /* k+2 */
> - movaps %xmm0, %xmm1 /* y|y */
> - mulpd %xmm0, %xmm0 /* z=t^4|z=t^4 */
> -
> - movaps MO1(DP_SC4), %xmm2 /* S4 */
> - mulpd %xmm0, %xmm2 /* z*S4 */
> - movaps MO1(DP_SC3), %xmm3 /* S3 */
> - mulpd %xmm0, %xmm3 /* z*S3 */
> - xorl %eax, %ecx /* (sign_x ^ (k>>2))<<2 */
> - addpd MO1(DP_SC2), %xmm2 /* S2+z*S4 */
> - mulpd %xmm0, %xmm2 /* z*(S2+z*S4) */
> - shrl $2, %edx /* (k+2)>>2 */
> - addpd MO1(DP_SC1), %xmm3 /* S1+z*S3 */
> - mulpd %xmm0, %xmm3 /* z*(S1+z*S3) */
> - shrl $2, %ecx /* sign_x ^ k>>2 */
> - addpd MO1(DP_SC0), %xmm2 /* S0+z*(S2+z*S4) */
> - andl $1, %edx /* sign_cos = ((k+2)>>2)&1 */
> - mulpd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
> - andl $1, %ecx /* sign_sin = sign_x ^ ((k>>2)&1) */
> - addpd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> - mulpd %xmm4, %xmm3 /*t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
> - testl $2, %eax /* n&2 != 0 ? */
> - addpd %xmm4, %xmm3 /*t+t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
> - jnz L(sin_result_sin_poly)
> -
> -/*L(sin_result_cos_poly):*/
> - /*
> - * Here if
> - * cos(x) = poly_sin * sign_cos
> - * sin(x) = poly_cos * sign_sin
> - */
> - movsd MO2(DP_ONES,%ecx,8), %xmm4/* 0|sign_sin */
> - movhpd MO2(DP_ONES,%edx,8), %xmm4/* sign_cos|sign_sin */
> - mulpd %xmm4, %xmm3 /* result_cos|result_sin */
> - movl ARG_SIN_PTR, %eax
> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> - movl ARG_COS_PTR, %ecx
> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
> - movss %xmm0, (%ecx) /* store cos(x) */
> - RETURN
> -
> - .p2align 4
> -L(sin_result_sin_poly):
> - /*
> - * Here if
> - * sin(x) = poly_sin * sign_sin
> - * cos(x) = poly_cos * sign_cos
> - */
> - movsd MO2(DP_ONES,%edx,8), %xmm4/* 0|sign_cos */
> - movhpd MO2(DP_ONES,%ecx,8), %xmm4/* sign_sin|sign_cos */
> - mulpd %xmm4, %xmm3 /* result_sin|result_cos */
> - movl ARG_SIN_PTR, %eax
> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> - movl ARG_COS_PTR, %ecx
> - movss %xmm0, (%ecx) /* store cos(x) from xmm0[0] */
> - shufps $1, %xmm0, %xmm0 /* move sin(x) to xmm0[0] */
> - movss %xmm0, (%eax) /* store sin(x) */
> - RETURN
> -
> - .p2align 4
> -L(large_args):
> - /* Here if |x|>=9*Pi/4 */
> - cmpl $0x7f800000, %eax /* x is Inf or NaN ? */
> - jae L(arg_inf_or_nan)
> -
> - /* Here if finite |x|>=9*Pi/4 */
> - cmpl $0x4b000000, %eax /* |x|<2^23 ? */
> - jae L(very_large_args)
> -
> - /* Here if 9*Pi/4<=|x|<2^23 */
> - movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
> - mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
> - cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
> - addl $1, %eax /* k+1 */
> - movl %eax, %edx
> - andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
> - cvtsi2sdl %edx, %xmm4 /* DP j */
> - movl ARG_X, %ecx /* Load x */
> - movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
> - shrl $29, %ecx /* (sign of x) << 2 */
> - mulsd %xmm4, %xmm2 /* -j*PIO4HI */
> - movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
> - addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
> - mulsd %xmm3, %xmm4 /* j*PIO4LO */
> - addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
> - jmp L(reconstruction)
> -
> - .p2align 4
> -L(very_large_args):
> - /* Here if finite |x|>=2^23 */
> -
> - /* bitpos = (ix>>23) - BIAS_32 + 59; */
> - shrl $23, %eax /* eb = biased exponent of x */
> - subl $68, %eax /* bitpos=eb-0x7f+59, where 0x7f */
> - /*is exponent bias */
> - movl $28, %ecx /* %cl=28 */
> - movl %eax, %edx /* bitpos copy */
> -
> - /* j = bitpos/28; */
> - div %cl /* j in register %al=%ax/%cl */
> - movapd %xmm0, %xmm3 /* |x| */
> - andl $0xff, %eax /* clear unneeded remainder from %ah*/
> -
> - imull $28, %eax, %ecx /* j*28 */
> - movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
> - movapd %xmm0, %xmm5 /* |x| */
> - mulsd -2*8+MO2(_FPI,%eax,8), %xmm3/* tmp3 = FPI[j-2]*|x| */
> - movapd %xmm0, %xmm1 /* |x| */
> - mulsd -1*8+MO2(_FPI,%eax,8), %xmm5/* tmp2 = FPI[j-1]*|x| */
> - mulsd 0*8+MO2(_FPI,%eax,8), %xmm0/* tmp0 = FPI[j]*|x| */
> - addl $19, %ecx /* j*28+19 */
> - mulsd 1*8+MO2(_FPI,%eax,8), %xmm1/* tmp1 = FPI[j+1]*|x| */
> - cmpl %ecx, %edx /* bitpos>=j*28+19 ? */
> - jl L(very_large_skip1)
> -
> - /* Here if bitpos>=j*28+19 */
> - andpd %xmm3, %xmm4 /* HI(tmp3) */
> - subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
> -L(very_large_skip1):
> -
> - movsd MO1(DP_2POW52), %xmm6
> - movapd %xmm5, %xmm2 /* tmp2 copy */
> - addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
> - movl $1, %edx
> - addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
> - movsd 8+MO1(DP_2POW52), %xmm4
> - movd %xmm6, %eax /* k = I64_LO(tmp6); */
> - addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
> - movl ARG_X, %ecx /* Load x */
> - comisd %xmm5, %xmm4 /* tmp4 > tmp5 ? */
> - jbe L(very_large_skip2)
> -
> - /* Here if tmp4 > tmp5 */
> - subl $1, %eax /* k-- */
> - addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
> -L(very_large_skip2):
> -
> - andl %eax, %edx /* k&1 */
> - subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
> - addsd MO2(DP_ZERONE,%edx,8), %xmm3/* t = DP_ZERONE[k&1] + tmp3 */
> - addsd %xmm2, %xmm3 /* t += tmp2 */
> - shrl $29, %ecx /* (sign of x) << 2 */
> - addsd %xmm3, %xmm0 /* t += tmp0 */
> - addl $1, %eax /* n=k+1 */
> - addsd %xmm1, %xmm0 /* t += tmp1 */
> - mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
> -
> - jmp L(reconstruction) /* end of very_large_args peth */
> -
> - .p2align 4
> -L(arg_less_pio4):
> - /* Here if |x|<Pi/4 */
> - cmpl $0x3d000000, %eax /* |x|<2^-5 ? */
> - jl L(arg_less_2pn5)
> -
> - /* Here if 2^-5<=|x|<Pi/4 */
> - movaps %xmm0, %xmm3 /* DP x */
> - movhpd MO1(DP_ONES), %xmm3 /* DP 1|x */
> - mulsd %xmm0, %xmm0 /* DP y=x^2 */
> - unpcklpd %xmm0, %xmm0 /* DP y|y */
> - movaps %xmm0, %xmm1 /* y|y */
> - mulpd %xmm0, %xmm0 /* z=x^4|z=x^4 */
> -
> - movapd MO1(DP_SC4), %xmm4 /* S4 */
> - mulpd %xmm0, %xmm4 /* z*S4 */
> - movapd MO1(DP_SC3), %xmm5 /* S3 */
> - mulpd %xmm0, %xmm5 /* z*S3 */
> - addpd MO1(DP_SC2), %xmm4 /* S2+z*S4 */
> - mulpd %xmm0, %xmm4 /* z*(S2+z*S4) */
> - addpd MO1(DP_SC1), %xmm5 /* S1+z*S3 */
> - mulpd %xmm0, %xmm5 /* z*(S1+z*S3) */
> - addpd MO1(DP_SC0), %xmm4 /* S0+z*(S2+z*S4) */
> - mulpd %xmm1, %xmm4 /* y*(S0+z*(S2+z*S4)) */
> - mulpd %xmm3, %xmm5 /* x*z*(S1+z*S3) */
> - mulpd %xmm3, %xmm4 /* x*y*(S0+z*(S2+z*S4)) */
> - addpd %xmm5, %xmm4 /*x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
> - movl ARG_SIN_PTR, %eax
> - addpd %xmm4, %xmm3 /*x+x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
> - movl ARG_COS_PTR, %ecx
> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
> - movss %xmm0, (%ecx) /* store cos(x) */
> - RETURN
> -
> - .p2align 4
> -L(arg_less_2pn5):
> - /* Here if |x|<2^-5 */
> - cmpl $0x32000000, %eax /* |x|<2^-27 ? */
> - jl L(arg_less_2pn27)
> -
> - /* Here if 2^-27<=|x|<2^-5 */
> - movaps %xmm0, %xmm1 /* DP x */
> - movhpd MO1(DP_ONES), %xmm1 /* DP 1|x */
> - mulsd %xmm0, %xmm0 /* DP x^2 */
> - unpcklpd %xmm0, %xmm0 /* DP x^2|x^2 */
> -
> - movaps MO1(DP_SINCOS2_1), %xmm3/* DP DP_SIN2_1 */
> - mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_1 */
> - addpd MO1(DP_SINCOS2_0), %xmm3/* DP DP_SIN2_0+x^2*DP_SIN2_1 */
> - mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_0+x^4*DP_SIN2_1 */
> - mulpd %xmm1, %xmm3 /* DP x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
> - addpd %xmm1, %xmm3 /* DP x+x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
> - movl ARG_SIN_PTR, %eax
> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> - movl ARG_COS_PTR, %ecx
> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
> - movss %xmm0, (%ecx) /* store cos(x) */
> - RETURN
> -
> - .p2align 4
> -L(arg_less_2pn27):
> - movss ARG_X, %xmm7 /* SP x */
> - cmpl $0, %eax /* x=0 ? */
> - je L(arg_zero) /* in case x=0 return sin(+-0)==+-0 */
> - /* Here if |x|<2^-27 */
> - /*
> - * Special cases here:
> - * sin(subnormal) raises inexact/underflow
> - * sin(min_normalized) raises inexact/underflow
> - * sin(normalized) raises inexact
> - * cos(here)=1-|x| (raising inexact)
> - */
> - movaps %xmm0, %xmm3 /* DP x */
> - mulsd MO1(DP_SMALL), %xmm0 /* DP x*DP_SMALL */
> - subsd %xmm0, %xmm3 /* DP sin result is x-x*DP_SMALL */
> - andps MO1(SP_ABS_MASK), %xmm7 /* SP |x| */
> - cvtsd2ss %xmm3, %xmm0 /* sin(x) */
> - movl ARG_SIN_PTR, %eax
> - movss MO1(SP_ONE), %xmm1 /* SP 1.0 */
> - movss %xmm0, (%eax) /* sin(x) store */
> - movl ARG_COS_PTR, %ecx
> - subss %xmm7, %xmm1 /* cos(x) */
> - movss %xmm1, (%ecx) /* cos(x) store */
> - RETURN
> -
> - .p2align 4
> -L(arg_zero):
> - movss MO1(SP_ONE), %xmm0 /* 1.0 */
> - movl ARG_SIN_PTR, %eax
> - movl ARG_COS_PTR, %ecx
> - movss %xmm7, (%eax) /* sin(+-0)==x */
> - movss %xmm0, (%ecx) /* cos(+-0)==1 */
> - RETURN
> -
> - .p2align 4
> -L(arg_inf_or_nan):
> - movss ARG_X, %xmm7 /* SP x */
> - /* Here if |x| is Inf or NAN */
> - jne L(skip_errno_setting) /* in case of x is NaN */
> -
> - /* Here if x is Inf. Set errno to EDOM. */
> - call JUMPTARGET(__errno_location)
> - movl $EDOM, (%eax)
> -
> - .p2align 4
> -L(skip_errno_setting):
> - /* Here if |x| is Inf or NAN. Continued. */
> - subss %xmm7, %xmm7 /* x-x, result is NaN */
> - movl ARG_SIN_PTR, %eax
> - movl ARG_COS_PTR, %ecx
> - movss %xmm7, (%eax)
> - movss %xmm7, (%ecx)
> - RETURN
> -END(__sincosf_sse2)
> -
> - .section .rodata, "a"
> - .p2align 3
> -L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
> - .long 0x00000000,0x00000000
> - .long 0x54442d18,0x3fe921fb
> - .long 0x54442d18,0x3ff921fb
> - .long 0x7f3321d2,0x4002d97c
> - .long 0x54442d18,0x400921fb
> - .long 0x2955385e,0x400f6a7a
> - .long 0x7f3321d2,0x4012d97c
> - .long 0xe9bba775,0x4015fdbb
> - .long 0x54442d18,0x401921fb
> - .long 0xbeccb2bb,0x401c463a
> - .long 0x2955385e,0x401f6a7a
> - .type L(PIO4J), @object
> - ASM_SIZE_DIRECTIVE(L(PIO4J))
> -
> - .p2align 3
> -L(_FPI): /* 4/Pi broken into sum of positive DP values */
> - .long 0x00000000,0x00000000
> - .long 0x6c000000,0x3ff45f30
> - .long 0x2a000000,0x3e3c9c88
> - .long 0xa8000000,0x3c54fe13
> - .long 0xd0000000,0x3aaf47d4
> - .long 0x6c000000,0x38fbb81b
> - .long 0xe0000000,0x3714acc9
> - .long 0x7c000000,0x3560e410
> - .long 0x56000000,0x33bca2c7
> - .long 0xac000000,0x31fbd778
> - .long 0xe0000000,0x300b7246
> - .long 0xe8000000,0x2e5d2126
> - .long 0x48000000,0x2c970032
> - .long 0xe8000000,0x2ad77504
> - .long 0xe0000000,0x290921cf
> - .long 0xb0000000,0x274deb1c
> - .long 0xe0000000,0x25829a73
> - .long 0xbe000000,0x23fd1046
> - .long 0x10000000,0x2224baed
> - .long 0x8e000000,0x20709d33
> - .long 0x80000000,0x1e535a2f
> - .long 0x64000000,0x1cef904e
> - .long 0x30000000,0x1b0d6398
> - .long 0x24000000,0x1964ce7d
> - .long 0x16000000,0x17b908bf
> - .type L(_FPI), @object
> - ASM_SIZE_DIRECTIVE(L(_FPI))
> -
> -/* Coefficients of polynomials for */
> -/* sin(x)~=x+x*x^2*(DP_SIN2_0+x^2*DP_SIN2_1) in low DP part, */
> -/* cos(x)~=1+1*x^2*(DP_COS2_0+x^2*DP_COS2_1) in high DP part, */
> -/* for |x|<2^-5. */
> - .p2align 4
> -L(DP_SINCOS2_0):
> - .long 0x5543d49d,0xbfc55555
> - .long 0xff5cc6fd,0xbfdfffff
> - .type L(DP_SINCOS2_0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_0))
> -
> - .p2align 4
> -L(DP_SINCOS2_1):
> - .long 0x75cec8c5,0x3f8110f4
> - .long 0xb178dac5,0x3fa55514
> - .type L(DP_SINCOS2_1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_1))
> -
> - .p2align 3
> -L(DP_ZERONE):
> - .long 0x00000000,0x00000000 /* 0.0 */
> - .long 0x00000000,0xbff00000 /* 1.0 */
> - .type L(DP_ZERONE), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
> -
> - .p2align 3
> -L(DP_ONES):
> - .long 0x00000000,0x3ff00000 /* +1.0 */
> - .long 0x00000000,0xbff00000 /* -1.0 */
> - .type L(DP_ONES), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ONES))
> -
> -/* Coefficients of polynomials for */
> -/* sin(t)~=t+t*t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))) in low DP part, */
> -/* cos(t)~=1+1*t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))) in high DP part, */
> -/* for |t|<Pi/4. */
> - .p2align 4
> -L(DP_SC4):
> - .long 0x1674b58a,0xbe5a947e
> - .long 0xdd8844d7,0xbe923c97
> - .type L(DP_SC4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SC4))
> -
> - .p2align 4
> -L(DP_SC3):
> - .long 0x64e6b5b4,0x3ec71d72
> - .long 0x9ac43cc0,0x3efa00eb
> - .type L(DP_SC3), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SC3))
> -
> - .p2align 4
> -L(DP_SC2):
> - .long 0x8b4bd1f9,0xbf2a019f
> - .long 0x348b6874,0xbf56c16b
> - .type L(DP_SC2), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SC2))
> -
> - .p2align 4
> -L(DP_SC1):
> - .long 0x10c2688b,0x3f811111
> - .long 0x545c50c7,0x3fa55555
> - .type L(DP_SC1), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SC1))
> -
> - .p2align 4
> -L(DP_SC0):
> - .long 0x55551cd9,0xbfc55555
> - .long 0xfffe98ae,0xbfdfffff
> - .type L(DP_SC0), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SC0))
> -
> - .p2align 3
> -L(DP_SMALL):
> - .long 0x00000000,0x3cd00000 /* 2^(-50) */
> - .type L(DP_SMALL), @object
> - ASM_SIZE_DIRECTIVE(L(DP_SMALL))
> -
> - .p2align 3
> -L(DP_PIO4):
> - .long 0x54442d18,0x3fe921fb /* Pi/4 */
> - .type L(DP_PIO4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4))
> -
> - .p2align 3
> -L(DP_2POW52):
> - .long 0x00000000,0x43300000 /* +2^52 */
> - .long 0x00000000,0xc3300000 /* -2^52 */
> - .type L(DP_2POW52), @object
> - ASM_SIZE_DIRECTIVE(L(DP_2POW52))
> -
> - .p2align 3
> -L(DP_INVPIO4):
> - .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
> - .type L(DP_INVPIO4), @object
> - ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
> -
> - .p2align 3
> -L(DP_PIO4HI):
> - .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
> - .type L(DP_PIO4HI), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
> -
> - .p2align 3
> -L(DP_PIO4LO):
> - .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
> - .type L(DP_PIO4LO), @object
> - ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
> -
> - .p2align 2
> -L(SP_INVPIO4):
> - .long 0x3fa2f983 /* 4/Pi */
> - .type L(SP_INVPIO4), @object
> - ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
> -
> - .p2align 4
> -L(DP_ABS_MASK): /* Mask for getting DP absolute value */
> - .long 0xffffffff,0x7fffffff
> - .long 0xffffffff,0x7fffffff
> - .type L(DP_ABS_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
> -
> - .p2align 3
> -L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
> - .long 0x00000000,0xffffffff
> - .type L(DP_HI_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
> -
> - .p2align 4
> -L(SP_ABS_MASK): /* Mask for getting SP absolute value */
> - .long 0x7fffffff,0x7fffffff
> - .long 0x7fffffff,0x7fffffff
> - .type L(SP_ABS_MASK), @object
> - ASM_SIZE_DIRECTIVE(L(SP_ABS_MASK))
t> -
> - .p2align 2
> -L(SP_ONE):
> - .long 0x3f800000 /* 1.0 */
> - .type L(SP_ONE), @object
> - ASM_SIZE_DIRECTIVE(L(SP_ONE))
> -
> -weak_alias(__sincosf, sincosf)
> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> new file mode 100644
> index 0000000000..46363fc11a
> --- /dev/null
> +++ b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> @@ -0,0 +1,3 @@
> +#define SINCOSF __sincosf_sse2
> +#define SECTION __attribute__ ((section (".text.sse2")))
> +#include <sysdeps/ieee754/flt-32/s_sincosf.c>
> --
> 2.34.1
>
Don't you need to add SECTION to sysdeps/ieee754/flt-32/s_sincosf.c?
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686
2022-05-31 21:49 ` H.J. Lu
@ 2022-05-31 21:53 ` Noah Goldstein
2022-06-01 12:36 ` Adhemerval Zanella
0 siblings, 1 reply; 19+ messages in thread
From: Noah Goldstein @ 2022-05-31 21:53 UTC (permalink / raw)
To: H.J. Lu; +Cc: Adhemerval Zanella, GNU C Library
On Tue, May 31, 2022 at 4:50 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Tue, May 31, 2022 at 2:48 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> >
> > On Tue, May 31, 2022 at 4:35 PM Adhemerval Zanella via Libc-alpha
> > <libc-alpha@sourceware.org> wrote:
> > >
> > > Performance seems to be similar, with generic implementation showing
> > > slight better performance for sinf large inputs and for sincosf.
> > >
> > > Although generic implementation shows slight lower precision (the
> > > ulp data required some adjustments), the code size shows a good
> > > improvement:
> > >
> >
> > How much loss of precision?
>
> About 1ULP.
Should be okay then. Does:
https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html
need to be updated?
>
> > > i686-linux-gnu-master$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
> > > math/s_sincosf-sse2.os
> > > text data bss dec hex filename
> > > 1555 0 0 1555 613 math/s_sinf-sse2.os
> > > 1551 0 0 1551 60f math/s_cosf-sse2.os
> > > 1754 0 0 1754 6da math/s_sincosf-sse2.os
> > >
> > > i686-linux-gnu-patched$ size math/s_sinf-sse2.os math/s_cosf-sse2.os
> > > math/s_sincosf-sse2.os
> > > text data bss dec hex filename
> > > 1059 0 0 1059 423 math/s_sinf-sse2.os
> > > 1067 0 0 1067 42b math/s_cosf-sse2.os
> > > 993 0 0 993 3e1 math/s_sincosf-sse2.os
> > >
> > > And it also simplifies the code base a lot with less assembly
> > > implementations.
> > >
> > > Adhemerval Zanella (5):
> > > benchtests: Add workload name for cosf
> > > i686: Use generic cosf implementation for SSE2 version
> > > i686: Use generic sinf implementation for SSE2 version
> > > benchtests: Add workload name for sincosf
> > > math: Use generic sinf implementation for SSE2 version
> > >
> > > benchtests/cosf-inputs | 2 +-
> > > benchtests/sincosf-inputs | 1 +
> > > sysdeps/i386/i686/fpu/multiarch/Makefile | 3 +
> > > .../i386/i686/fpu/multiarch/libm-test-ulps | 11 +
> > > sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S | 552 -----------------
> > > sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c | 3 +
> > > .../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
> > > .../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
> > > sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S | 565 -----------------
> > > sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c | 3 +
> > > sysdeps/ieee754/flt-32/s_cosf.c | 5 +
> > > sysdeps/ieee754/flt-32/s_sinf.c | 5 +
> > > 12 files changed, 35 insertions(+), 1703 deletions(-)
> > > delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S
> > > create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.c
> > > delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> > > create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> > > delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S
> > > create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.c
> > >
> > > --
> > > 2.34.1
> > >
>
>
>
> --
> H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686
2022-05-31 21:53 ` Noah Goldstein
@ 2022-06-01 12:36 ` Adhemerval Zanella
2022-06-01 13:12 ` Adhemerval Zanella
0 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-06-01 12:36 UTC (permalink / raw)
To: Noah Goldstein, H.J. Lu; +Cc: GNU C Library
On 31/05/2022 18:53, Noah Goldstein wrote:
> On Tue, May 31, 2022 at 4:50 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>>
>> On Tue, May 31, 2022 at 2:48 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
>>>
>>> On Tue, May 31, 2022 at 4:35 PM Adhemerval Zanella via Libc-alpha
>>> <libc-alpha@sourceware.org> wrote:
>>>>
>>>> Performance seems to be similar, with generic implementation showing
>>>> slight better performance for sinf large inputs and for sincosf.
>>>>
>>>> Although generic implementation shows slight lower precision (the
>>>> ulp data required some adjustments), the code size shows a good
>>>> improvement:
>>>>
>>>
>>> How much loss of precision?
>>
>> About 1ULP.
>
> Should be okay then. Does:
> https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html
> need to be updated?
Good call, I will send a patch to update the documentation.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] math: Use generic sinf implementation for SSE2 version
2022-05-31 21:51 ` H.J. Lu
@ 2022-06-01 12:42 ` Adhemerval Zanella
2022-06-01 15:14 ` H.J. Lu
0 siblings, 1 reply; 19+ messages in thread
From: Adhemerval Zanella @ 2022-06-01 12:42 UTC (permalink / raw)
To: H.J. Lu; +Cc: GNU C Library
On 31/05/2022 18:51, H.J. Lu wrote:
> ? On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
> <adhemerval.zanella@linaro.org> wrote:
>>
>> The generic implementation shows slight better performance
>> (gcc 11.2.1 on a Ryzen 9 5900X):
>>
>> * s_sincosf-sse2.S:
>> "sincosf": {
>> "workload-random": {
>> "duration": 3.89961e+09,
>> "iterations": 9.5472e+07,
>> "reciprocal-throughput": 40.8429,
>> "latency": 40.8483,
>> "max-throughput": 2.4484e+07,
>> "min-throughput": 2.44808e+07
>> }
>> }
>>
>> * generic s_cossinf.c:
>> "sincosf": {
>> "workload-random": {
>> "duration": 3.71953e+09,
>> "iterations": 1.48512e+08,
>> "reciprocal-throughput": 25.0515,
>> "latency": 25.0391,
>> "max-throughput": 3.99177e+07,
>> "min-throughput": 3.99375e+07
>> }
>> }
>>
>> Checked on i686-linux-gnu.
>> ---
>> sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
>> .../i386/i686/fpu/multiarch/libm-test-ulps | 3 +
>> .../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
>> .../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
>> 4 files changed, 7 insertions(+), 585 deletions(-)
>> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
>> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
>>
>> diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
>> index c2cd5cefef..c8b0a6528a 100644
>> --- a/sysdeps/i386/i686/fpu/multiarch/Makefile
>> +++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
>> @@ -9,4 +9,5 @@ CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
>> CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
>> CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
>> CFLAGS-s_sinf-sse2.c = -msse2 -mfpmath=sse
>> +CFLAGS-s_sincosf-sse2.c = -msse2 -mfpmath=sse
>> endif
>> diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
>> index 0357d7bd14..284e3fa275 100644
>> --- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
>> +++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
>> @@ -1492,16 +1492,19 @@ ldouble: 1
>>
>> Function: "sincos_downward":
>> double: 1
>> +float: 1
>> float128: 3
>> ldouble: 3
>>
>> Function: "sincos_towardzero":
>> double: 1
>> +float: 1
>> float128: 2
>> ldouble: 2
>>
>> Function: "sincos_upward":
>> double: 1
>> +float: 1
>> float128: 3
>> ldouble: 3
>>
>> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
>> deleted file mode 100644
>> index 71e222b930..0000000000
>> --- a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
>> +++ /dev/null
>> @@ -1,585 +0,0 @@
>> -/* Optimized with sse2 version of sincosf
>> - Copyright (C) 2012-2022 Free Software Foundation, Inc.
>> - This file is part of the GNU C Library.
>> -
>> - The GNU C Library is free software; you can redistribute it and/or
>> - modify it under the terms of the GNU Lesser General Public
>> - License as published by the Free Software Foundation; either
>> - version 2.1 of the License, or (at your option) any later version.
>> -
>> - The GNU C Library is distributed in the hope that it will be useful,
>> - but WITHOUT ANY WARRANTY; without even the implied warranty of
>> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> - Lesser General Public License for more details.
>> -
>> - You should have received a copy of the GNU Lesser General Public
>> - License along with the GNU C Library; if not, see
>> - <https://www.gnu.org/licenses/>. */
>> -
>> -#include <sysdep.h>
>> -#include <errno.h>
>> -
>> -/* Short algorithm description:
>> - *
>> - * 1) if |x|==0: sin(x)=x,
>> - * cos(x)=1.
>> - * 2) if |x|<2^-27: sin(x)=x-x*DP_SMALL, raising underflow only when needed,
>> - * cos(x)=1-|x|.
>> - * 3) if |x|<2^-5 : sin(x)=x+x*x^2*DP_SIN2_0+x^5*DP_SIN2_1,
>> - * cos(x)=1+1*x^2*DP_COS2_0+x^5*DP_COS2_1
>> - * 4) if |x|< Pi/4: sin(x)=x+x*x^2*(S0+x^2*(S1+x^2*(S2+x^2*(S3+x^2*S4)))),
>> - * cos(x)=1+1*x^2*(C0+x^2*(C1+x^2*(C2+x^2*(C3+x^2*C4)))).
>> - * 5) if |x| < 9*Pi/4:
>> - * 5.1) Range reduction:
>> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+1, t=|x|-j*Pi/4.
>> - * 5.2) Reconstruction:
>> - * sign_sin = sign(x) * (-1.0)^(( n >>2)&1)
>> - * sign_cos = (-1.0)^(((n+2)>>2)&1)
>> - * poly_sin = ((((S4*t^2 + S3)*t^2 + S2)*t^2 + S1)*t^2 + S0)*t^2*t+t
>> - * poly_cos = ((((C4*t^2 + C3)*t^2 + C2)*t^2 + C1)*t^2 + C0)*t^2*s+s
>> - * if(n&2 != 0) {
>> - * using cos(t) and sin(t) polynomials for |t|<Pi/4, results are
>> - * cos(x) = poly_sin * sign_cos
>> - * sin(x) = poly_cos * sign_sin
>> - * } else {
>> - * sin(x) = poly_sin * sign_sin
>> - * cos(x) = poly_cos * sign_cos
>> - * }
>> - * 6) if |x| < 2^23, large args:
>> - * 6.1) Range reduction:
>> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4
>> - * 6.2) Reconstruction same as (5.2).
>> - * 7) if |x| >= 2^23, very large args:
>> - * 7.1) Range reduction:
>> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4.
>> - * 7.2) Reconstruction same as (5.2).
>> - * 8) if x is Inf, return x-x, and set errno=EDOM.
>> - * 9) if x is NaN, return x-x.
>> - *
>> - * Special cases:
>> - * sin/cos(+-0) = +-0/1 not raising inexact/underflow,
>> - * sin/cos(subnormal) raises inexact/underflow,
>> - * sin/cos(min_normalized) raises inexact/underflow,
>> - * sin/cos(normalized) raises inexact,
>> - * sin/cos(Inf) = NaN, raises invalid, sets errno to EDOM,
>> - * sin/cos(NaN) = NaN.
>> - */
>> -
>> -#ifdef PIC
>> -# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
>> -# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
>> -# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
>> -# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
>> -# define PUSH(REG) pushl REG; CFI_PUSH(REG)
>> -# define POP(REG) popl REG; CFI_POP(REG)
>> -# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
>> -# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
>> -# define ARG_X 8(%esp)
>> -# define ARG_SIN_PTR 12(%esp)
>> -# define ARG_COS_PTR 16(%esp)
>> -#else
>> -# define MO1(symbol) L(symbol)
>> -# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
>> -# define ENTRANCE
>> -# define RETURN ret
>> -# define ARG_X 4(%esp)
>> -# define ARG_SIN_PTR 8(%esp)
>> -# define ARG_COS_PTR 12(%esp)
>> -#endif
>> -
>> - .text
>> -ENTRY(__sincosf_sse2)
>> - /* Input: single precision x on stack at address ARG_X */
>> - /* pointer to sin result on stack at address ARG_SIN_PTR */
>> - /* pointer to cos result on stack at address ARG_COS_PTR */
>> -
>> - ENTRANCE
>> - movl ARG_X, %eax /* Bits of x */
>> - cvtss2sd ARG_X, %xmm0 /* DP x */
>> - andl $0x7fffffff, %eax /* |x| */
>> -
>> - cmpl $0x3f490fdb, %eax /* |x|<Pi/4 ? */
>> - jb L(arg_less_pio4)
>> -
>> - /* Here if |x|>=Pi/4 */
>> - movd %eax, %xmm3 /* SP |x| */
>> - andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
>> - movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
>> -
>> - cmpl $0x40e231d6, %eax /* |x|<9*Pi/4 ? */
>> - jae L(large_args)
>> -
>> - /* Here if Pi/4<=|x|<9*Pi/4 */
>> - mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
>> - movl ARG_X, %ecx /* Load x */
>> - cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
>> - shrl $29, %ecx /* (sign of x) << 2 */
>> - addl $1, %eax /* k+1 */
>> - movl $0x0e, %edx
>> - andl %eax, %edx /* j = (k+1)&0x0e */
>> - subsd MO2(PIO4J,%edx,8), %xmm0/* t = |x| - j * Pi/4 */
>> -
>> -L(reconstruction):
>> - /* Input: %eax=n, %xmm0=t, %ecx=sign(x) */
>> -
>> - movaps %xmm0, %xmm4 /* t */
>> - movhpd MO1(DP_ONES), %xmm4 /* 1|t */
>> - mulsd %xmm0, %xmm0 /* y=t^2 */
>> - movl $2, %edx
>> - unpcklpd %xmm0, %xmm0 /* y|y */
>> - addl %eax, %edx /* k+2 */
>> - movaps %xmm0, %xmm1 /* y|y */
>> - mulpd %xmm0, %xmm0 /* z=t^4|z=t^4 */
>> -
>> - movaps MO1(DP_SC4), %xmm2 /* S4 */
>> - mulpd %xmm0, %xmm2 /* z*S4 */
>> - movaps MO1(DP_SC3), %xmm3 /* S3 */
>> - mulpd %xmm0, %xmm3 /* z*S3 */
>> - xorl %eax, %ecx /* (sign_x ^ (k>>2))<<2 */
>> - addpd MO1(DP_SC2), %xmm2 /* S2+z*S4 */
>> - mulpd %xmm0, %xmm2 /* z*(S2+z*S4) */
>> - shrl $2, %edx /* (k+2)>>2 */
>> - addpd MO1(DP_SC1), %xmm3 /* S1+z*S3 */
>> - mulpd %xmm0, %xmm3 /* z*(S1+z*S3) */
>> - shrl $2, %ecx /* sign_x ^ k>>2 */
>> - addpd MO1(DP_SC0), %xmm2 /* S0+z*(S2+z*S4) */
>> - andl $1, %edx /* sign_cos = ((k+2)>>2)&1 */
>> - mulpd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
>> - andl $1, %ecx /* sign_sin = sign_x ^ ((k>>2)&1) */
>> - addpd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
>> - mulpd %xmm4, %xmm3 /*t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
>> - testl $2, %eax /* n&2 != 0 ? */
>> - addpd %xmm4, %xmm3 /*t+t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
>> - jnz L(sin_result_sin_poly)
>> -
>> -/*L(sin_result_cos_poly):*/
>> - /*
>> - * Here if
>> - * cos(x) = poly_sin * sign_cos
>> - * sin(x) = poly_cos * sign_sin
>> - */
>> - movsd MO2(DP_ONES,%ecx,8), %xmm4/* 0|sign_sin */
>> - movhpd MO2(DP_ONES,%edx,8), %xmm4/* sign_cos|sign_sin */
>> - mulpd %xmm4, %xmm3 /* result_cos|result_sin */
>> - movl ARG_SIN_PTR, %eax
>> - cvtpd2ps %xmm3, %xmm0 /* SP results */
>> - movl ARG_COS_PTR, %ecx
>> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
>> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
>> - movss %xmm0, (%ecx) /* store cos(x) */
>> - RETURN
>> -
>> - .p2align 4
>> -L(sin_result_sin_poly):
>> - /*
>> - * Here if
>> - * sin(x) = poly_sin * sign_sin
>> - * cos(x) = poly_cos * sign_cos
>> - */
>> - movsd MO2(DP_ONES,%edx,8), %xmm4/* 0|sign_cos */
>> - movhpd MO2(DP_ONES,%ecx,8), %xmm4/* sign_sin|sign_cos */
>> - mulpd %xmm4, %xmm3 /* result_sin|result_cos */
>> - movl ARG_SIN_PTR, %eax
>> - cvtpd2ps %xmm3, %xmm0 /* SP results */
>> - movl ARG_COS_PTR, %ecx
>> - movss %xmm0, (%ecx) /* store cos(x) from xmm0[0] */
>> - shufps $1, %xmm0, %xmm0 /* move sin(x) to xmm0[0] */
>> - movss %xmm0, (%eax) /* store sin(x) */
>> - RETURN
>> -
>> - .p2align 4
>> -L(large_args):
>> - /* Here if |x|>=9*Pi/4 */
>> - cmpl $0x7f800000, %eax /* x is Inf or NaN ? */
>> - jae L(arg_inf_or_nan)
>> -
>> - /* Here if finite |x|>=9*Pi/4 */
>> - cmpl $0x4b000000, %eax /* |x|<2^23 ? */
>> - jae L(very_large_args)
>> -
>> - /* Here if 9*Pi/4<=|x|<2^23 */
>> - movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
>> - mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
>> - cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
>> - addl $1, %eax /* k+1 */
>> - movl %eax, %edx
>> - andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
>> - cvtsi2sdl %edx, %xmm4 /* DP j */
>> - movl ARG_X, %ecx /* Load x */
>> - movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
>> - shrl $29, %ecx /* (sign of x) << 2 */
>> - mulsd %xmm4, %xmm2 /* -j*PIO4HI */
>> - movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
>> - addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
>> - mulsd %xmm3, %xmm4 /* j*PIO4LO */
>> - addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
>> - jmp L(reconstruction)
>> -
>> - .p2align 4
>> -L(very_large_args):
>> - /* Here if finite |x|>=2^23 */
>> -
>> - /* bitpos = (ix>>23) - BIAS_32 + 59; */
>> - shrl $23, %eax /* eb = biased exponent of x */
>> - subl $68, %eax /* bitpos=eb-0x7f+59, where 0x7f */
>> - /*is exponent bias */
>> - movl $28, %ecx /* %cl=28 */
>> - movl %eax, %edx /* bitpos copy */
>> -
>> - /* j = bitpos/28; */
>> - div %cl /* j in register %al=%ax/%cl */
>> - movapd %xmm0, %xmm3 /* |x| */
>> - andl $0xff, %eax /* clear unneeded remainder from %ah*/
>> -
>> - imull $28, %eax, %ecx /* j*28 */
>> - movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
>> - movapd %xmm0, %xmm5 /* |x| */
>> - mulsd -2*8+MO2(_FPI,%eax,8), %xmm3/* tmp3 = FPI[j-2]*|x| */
>> - movapd %xmm0, %xmm1 /* |x| */
>> - mulsd -1*8+MO2(_FPI,%eax,8), %xmm5/* tmp2 = FPI[j-1]*|x| */
>> - mulsd 0*8+MO2(_FPI,%eax,8), %xmm0/* tmp0 = FPI[j]*|x| */
>> - addl $19, %ecx /* j*28+19 */
>> - mulsd 1*8+MO2(_FPI,%eax,8), %xmm1/* tmp1 = FPI[j+1]*|x| */
>> - cmpl %ecx, %edx /* bitpos>=j*28+19 ? */
>> - jl L(very_large_skip1)
>> -
>> - /* Here if bitpos>=j*28+19 */
>> - andpd %xmm3, %xmm4 /* HI(tmp3) */
>> - subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
>> -L(very_large_skip1):
>> -
>> - movsd MO1(DP_2POW52), %xmm6
>> - movapd %xmm5, %xmm2 /* tmp2 copy */
>> - addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
>> - movl $1, %edx
>> - addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
>> - movsd 8+MO1(DP_2POW52), %xmm4
>> - movd %xmm6, %eax /* k = I64_LO(tmp6); */
>> - addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
>> - movl ARG_X, %ecx /* Load x */
>> - comisd %xmm5, %xmm4 /* tmp4 > tmp5 ? */
>> - jbe L(very_large_skip2)
>> -
>> - /* Here if tmp4 > tmp5 */
>> - subl $1, %eax /* k-- */
>> - addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
>> -L(very_large_skip2):
>> -
>> - andl %eax, %edx /* k&1 */
>> - subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
>> - addsd MO2(DP_ZERONE,%edx,8), %xmm3/* t = DP_ZERONE[k&1] + tmp3 */
>> - addsd %xmm2, %xmm3 /* t += tmp2 */
>> - shrl $29, %ecx /* (sign of x) << 2 */
>> - addsd %xmm3, %xmm0 /* t += tmp0 */
>> - addl $1, %eax /* n=k+1 */
>> - addsd %xmm1, %xmm0 /* t += tmp1 */
>> - mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
>> -
>> - jmp L(reconstruction) /* end of very_large_args peth */
>> -
>> - .p2align 4
>> -L(arg_less_pio4):
>> - /* Here if |x|<Pi/4 */
>> - cmpl $0x3d000000, %eax /* |x|<2^-5 ? */
>> - jl L(arg_less_2pn5)
>> -
>> - /* Here if 2^-5<=|x|<Pi/4 */
>> - movaps %xmm0, %xmm3 /* DP x */
>> - movhpd MO1(DP_ONES), %xmm3 /* DP 1|x */
>> - mulsd %xmm0, %xmm0 /* DP y=x^2 */
>> - unpcklpd %xmm0, %xmm0 /* DP y|y */
>> - movaps %xmm0, %xmm1 /* y|y */
>> - mulpd %xmm0, %xmm0 /* z=x^4|z=x^4 */
>> -
>> - movapd MO1(DP_SC4), %xmm4 /* S4 */
>> - mulpd %xmm0, %xmm4 /* z*S4 */
>> - movapd MO1(DP_SC3), %xmm5 /* S3 */
>> - mulpd %xmm0, %xmm5 /* z*S3 */
>> - addpd MO1(DP_SC2), %xmm4 /* S2+z*S4 */
>> - mulpd %xmm0, %xmm4 /* z*(S2+z*S4) */
>> - addpd MO1(DP_SC1), %xmm5 /* S1+z*S3 */
>> - mulpd %xmm0, %xmm5 /* z*(S1+z*S3) */
>> - addpd MO1(DP_SC0), %xmm4 /* S0+z*(S2+z*S4) */
>> - mulpd %xmm1, %xmm4 /* y*(S0+z*(S2+z*S4)) */
>> - mulpd %xmm3, %xmm5 /* x*z*(S1+z*S3) */
>> - mulpd %xmm3, %xmm4 /* x*y*(S0+z*(S2+z*S4)) */
>> - addpd %xmm5, %xmm4 /*x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
>> - movl ARG_SIN_PTR, %eax
>> - addpd %xmm4, %xmm3 /*x+x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
>> - movl ARG_COS_PTR, %ecx
>> - cvtpd2ps %xmm3, %xmm0 /* SP results */
>> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
>> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
>> - movss %xmm0, (%ecx) /* store cos(x) */
>> - RETURN
>> -
>> - .p2align 4
>> -L(arg_less_2pn5):
>> - /* Here if |x|<2^-5 */
>> - cmpl $0x32000000, %eax /* |x|<2^-27 ? */
>> - jl L(arg_less_2pn27)
>> -
>> - /* Here if 2^-27<=|x|<2^-5 */
>> - movaps %xmm0, %xmm1 /* DP x */
>> - movhpd MO1(DP_ONES), %xmm1 /* DP 1|x */
>> - mulsd %xmm0, %xmm0 /* DP x^2 */
>> - unpcklpd %xmm0, %xmm0 /* DP x^2|x^2 */
>> -
>> - movaps MO1(DP_SINCOS2_1), %xmm3/* DP DP_SIN2_1 */
>> - mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_1 */
>> - addpd MO1(DP_SINCOS2_0), %xmm3/* DP DP_SIN2_0+x^2*DP_SIN2_1 */
>> - mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_0+x^4*DP_SIN2_1 */
>> - mulpd %xmm1, %xmm3 /* DP x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
>> - addpd %xmm1, %xmm3 /* DP x+x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
>> - movl ARG_SIN_PTR, %eax
>> - cvtpd2ps %xmm3, %xmm0 /* SP results */
>> - movl ARG_COS_PTR, %ecx
>> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
>> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
>> - movss %xmm0, (%ecx) /* store cos(x) */
>> - RETURN
>> -
>> - .p2align 4
>> -L(arg_less_2pn27):
>> - movss ARG_X, %xmm7 /* SP x */
>> - cmpl $0, %eax /* x=0 ? */
>> - je L(arg_zero) /* in case x=0 return sin(+-0)==+-0 */
>> - /* Here if |x|<2^-27 */
>> - /*
>> - * Special cases here:
>> - * sin(subnormal) raises inexact/underflow
>> - * sin(min_normalized) raises inexact/underflow
>> - * sin(normalized) raises inexact
>> - * cos(here)=1-|x| (raising inexact)
>> - */
>> - movaps %xmm0, %xmm3 /* DP x */
>> - mulsd MO1(DP_SMALL), %xmm0 /* DP x*DP_SMALL */
>> - subsd %xmm0, %xmm3 /* DP sin result is x-x*DP_SMALL */
>> - andps MO1(SP_ABS_MASK), %xmm7 /* SP |x| */
>> - cvtsd2ss %xmm3, %xmm0 /* sin(x) */
>> - movl ARG_SIN_PTR, %eax
>> - movss MO1(SP_ONE), %xmm1 /* SP 1.0 */
>> - movss %xmm0, (%eax) /* sin(x) store */
>> - movl ARG_COS_PTR, %ecx
>> - subss %xmm7, %xmm1 /* cos(x) */
>> - movss %xmm1, (%ecx) /* cos(x) store */
>> - RETURN
>> -
>> - .p2align 4
>> -L(arg_zero):
>> - movss MO1(SP_ONE), %xmm0 /* 1.0 */
>> - movl ARG_SIN_PTR, %eax
>> - movl ARG_COS_PTR, %ecx
>> - movss %xmm7, (%eax) /* sin(+-0)==x */
>> - movss %xmm0, (%ecx) /* cos(+-0)==1 */
>> - RETURN
>> -
>> - .p2align 4
>> -L(arg_inf_or_nan):
>> - movss ARG_X, %xmm7 /* SP x */
>> - /* Here if |x| is Inf or NAN */
>> - jne L(skip_errno_setting) /* in case of x is NaN */
>> -
>> - /* Here if x is Inf. Set errno to EDOM. */
>> - call JUMPTARGET(__errno_location)
>> - movl $EDOM, (%eax)
>> -
>> - .p2align 4
>> -L(skip_errno_setting):
>> - /* Here if |x| is Inf or NAN. Continued. */
>> - subss %xmm7, %xmm7 /* x-x, result is NaN */
>> - movl ARG_SIN_PTR, %eax
>> - movl ARG_COS_PTR, %ecx
>> - movss %xmm7, (%eax)
>> - movss %xmm7, (%ecx)
>> - RETURN
>> -END(__sincosf_sse2)
>> -
>> - .section .rodata, "a"
>> - .p2align 3
>> -L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
>> - .long 0x00000000,0x00000000
>> - .long 0x54442d18,0x3fe921fb
>> - .long 0x54442d18,0x3ff921fb
>> - .long 0x7f3321d2,0x4002d97c
>> - .long 0x54442d18,0x400921fb
>> - .long 0x2955385e,0x400f6a7a
>> - .long 0x7f3321d2,0x4012d97c
>> - .long 0xe9bba775,0x4015fdbb
>> - .long 0x54442d18,0x401921fb
>> - .long 0xbeccb2bb,0x401c463a
>> - .long 0x2955385e,0x401f6a7a
>> - .type L(PIO4J), @object
>> - ASM_SIZE_DIRECTIVE(L(PIO4J))
>> -
>> - .p2align 3
>> -L(_FPI): /* 4/Pi broken into sum of positive DP values */
>> - .long 0x00000000,0x00000000
>> - .long 0x6c000000,0x3ff45f30
>> - .long 0x2a000000,0x3e3c9c88
>> - .long 0xa8000000,0x3c54fe13
>> - .long 0xd0000000,0x3aaf47d4
>> - .long 0x6c000000,0x38fbb81b
>> - .long 0xe0000000,0x3714acc9
>> - .long 0x7c000000,0x3560e410
>> - .long 0x56000000,0x33bca2c7
>> - .long 0xac000000,0x31fbd778
>> - .long 0xe0000000,0x300b7246
>> - .long 0xe8000000,0x2e5d2126
>> - .long 0x48000000,0x2c970032
>> - .long 0xe8000000,0x2ad77504
>> - .long 0xe0000000,0x290921cf
>> - .long 0xb0000000,0x274deb1c
>> - .long 0xe0000000,0x25829a73
>> - .long 0xbe000000,0x23fd1046
>> - .long 0x10000000,0x2224baed
>> - .long 0x8e000000,0x20709d33
>> - .long 0x80000000,0x1e535a2f
>> - .long 0x64000000,0x1cef904e
>> - .long 0x30000000,0x1b0d6398
>> - .long 0x24000000,0x1964ce7d
>> - .long 0x16000000,0x17b908bf
>> - .type L(_FPI), @object
>> - ASM_SIZE_DIRECTIVE(L(_FPI))
>> -
>> -/* Coefficients of polynomials for */
>> -/* sin(x)~=x+x*x^2*(DP_SIN2_0+x^2*DP_SIN2_1) in low DP part, */
>> -/* cos(x)~=1+1*x^2*(DP_COS2_0+x^2*DP_COS2_1) in high DP part, */
>> -/* for |x|<2^-5. */
>> - .p2align 4
>> -L(DP_SINCOS2_0):
>> - .long 0x5543d49d,0xbfc55555
>> - .long 0xff5cc6fd,0xbfdfffff
>> - .type L(DP_SINCOS2_0), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_0))
>> -
>> - .p2align 4
>> -L(DP_SINCOS2_1):
>> - .long 0x75cec8c5,0x3f8110f4
>> - .long 0xb178dac5,0x3fa55514
>> - .type L(DP_SINCOS2_1), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_1))
>> -
>> - .p2align 3
>> -L(DP_ZERONE):
>> - .long 0x00000000,0x00000000 /* 0.0 */
>> - .long 0x00000000,0xbff00000 /* 1.0 */
>> - .type L(DP_ZERONE), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
>> -
>> - .p2align 3
>> -L(DP_ONES):
>> - .long 0x00000000,0x3ff00000 /* +1.0 */
>> - .long 0x00000000,0xbff00000 /* -1.0 */
>> - .type L(DP_ONES), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_ONES))
>> -
>> -/* Coefficients of polynomials for */
>> -/* sin(t)~=t+t*t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))) in low DP part, */
>> -/* cos(t)~=1+1*t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))) in high DP part, */
>> -/* for |t|<Pi/4. */
>> - .p2align 4
>> -L(DP_SC4):
>> - .long 0x1674b58a,0xbe5a947e
>> - .long 0xdd8844d7,0xbe923c97
>> - .type L(DP_SC4), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SC4))
>> -
>> - .p2align 4
>> -L(DP_SC3):
>> - .long 0x64e6b5b4,0x3ec71d72
>> - .long 0x9ac43cc0,0x3efa00eb
>> - .type L(DP_SC3), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SC3))
>> -
>> - .p2align 4
>> -L(DP_SC2):
>> - .long 0x8b4bd1f9,0xbf2a019f
>> - .long 0x348b6874,0xbf56c16b
>> - .type L(DP_SC2), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SC2))
>> -
>> - .p2align 4
>> -L(DP_SC1):
>> - .long 0x10c2688b,0x3f811111
>> - .long 0x545c50c7,0x3fa55555
>> - .type L(DP_SC1), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SC1))
>> -
>> - .p2align 4
>> -L(DP_SC0):
>> - .long 0x55551cd9,0xbfc55555
>> - .long 0xfffe98ae,0xbfdfffff
>> - .type L(DP_SC0), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SC0))
>> -
>> - .p2align 3
>> -L(DP_SMALL):
>> - .long 0x00000000,0x3cd00000 /* 2^(-50) */
>> - .type L(DP_SMALL), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_SMALL))
>> -
>> - .p2align 3
>> -L(DP_PIO4):
>> - .long 0x54442d18,0x3fe921fb /* Pi/4 */
>> - .type L(DP_PIO4), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_PIO4))
>> -
>> - .p2align 3
>> -L(DP_2POW52):
>> - .long 0x00000000,0x43300000 /* +2^52 */
>> - .long 0x00000000,0xc3300000 /* -2^52 */
>> - .type L(DP_2POW52), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_2POW52))
>> -
>> - .p2align 3
>> -L(DP_INVPIO4):
>> - .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
>> - .type L(DP_INVPIO4), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
>> -
>> - .p2align 3
>> -L(DP_PIO4HI):
>> - .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
>> - .type L(DP_PIO4HI), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
>> -
>> - .p2align 3
>> -L(DP_PIO4LO):
>> - .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
>> - .type L(DP_PIO4LO), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
>> -
>> - .p2align 2
>> -L(SP_INVPIO4):
>> - .long 0x3fa2f983 /* 4/Pi */
>> - .type L(SP_INVPIO4), @object
>> - ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
>> -
>> - .p2align 4
>> -L(DP_ABS_MASK): /* Mask for getting DP absolute value */
>> - .long 0xffffffff,0x7fffffff
>> - .long 0xffffffff,0x7fffffff
>> - .type L(DP_ABS_MASK), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
>> -
>> - .p2align 3
>> -L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
>> - .long 0x00000000,0xffffffff
>> - .type L(DP_HI_MASK), @object
>> - ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
>> -
>> - .p2align 4
>> -L(SP_ABS_MASK): /* Mask for getting SP absolute value */
>> - .long 0x7fffffff,0x7fffffff
>> - .long 0x7fffffff,0x7fffffff
>> - .type L(SP_ABS_MASK), @object
>> - ASM_SIZE_DIRECTIVE(L(SP_ABS_MASK))
> t> -
>> - .p2align 2
>> -L(SP_ONE):
>> - .long 0x3f800000 /* 1.0 */
>> - .type L(SP_ONE), @object
>> - ASM_SIZE_DIRECTIVE(L(SP_ONE))
>> -
>> -weak_alias(__sincosf, sincosf)
>> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
>> new file mode 100644
>> index 0000000000..46363fc11a
>> --- /dev/null
>> +++ b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
>> @@ -0,0 +1,3 @@
>> +#define SINCOSF __sincosf_sse2
>> +#define SECTION __attribute__ ((section (".text.sse2")))
>> +#include <sysdeps/ieee754/flt-32/s_sincosf.c>
>> --
>> 2.34.1
>>
>
> Don't you need to add SECTION to sysdeps/ieee754/flt-32/s_sincosf.c?
>
It does, thanks for catching it. I will fix it before commit.
diff --git a/sysdeps/ieee754/flt-32/s_sincosf.c b/sysdeps/ieee754/flt-32/s_sincosf.c
index 918f6a655a..4fab42d974 100644
--- a/sysdeps/ieee754/flt-32/s_sincosf.c
+++ b/sysdeps/ieee754/flt-32/s_sincosf.c
@@ -24,6 +24,10 @@
#include "math_config.h"
#include "s_sincosf.h"
+#ifndef SECTION
+# define SECTION
+#endif
+
#ifndef SINCOSF
# define SINCOSF_FUNC __sincosf
#else
@@ -35,6 +39,7 @@
small values. Large inputs have their range reduced using fast integer
arithmetic. */
void
+SECTION
SINCOSF_FUNC (float y, float *sinp, float *cosp)
{
double x = y;
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686
2022-06-01 12:36 ` Adhemerval Zanella
@ 2022-06-01 13:12 ` Adhemerval Zanella
0 siblings, 0 replies; 19+ messages in thread
From: Adhemerval Zanella @ 2022-06-01 13:12 UTC (permalink / raw)
To: Noah Goldstein, H.J. Lu; +Cc: GNU C Library
On 01/06/2022 09:36, Adhemerval Zanella wrote:
>
>
> On 31/05/2022 18:53, Noah Goldstein wrote:
>> On Tue, May 31, 2022 at 4:50 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>>>
>>> On Tue, May 31, 2022 at 2:48 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
>>>>
>>>> On Tue, May 31, 2022 at 4:35 PM Adhemerval Zanella via Libc-alpha
>>>> <libc-alpha@sourceware.org> wrote:
>>>>>
>>>>> Performance seems to be similar, with generic implementation showing
>>>>> slight better performance for sinf large inputs and for sincosf.
>>>>>
>>>>> Although generic implementation shows slight lower precision (the
>>>>> ulp data required some adjustments), the code size shows a good
>>>>> improvement:
>>>>>
>>>>
>>>> How much loss of precision?
>>>
>>> About 1ULP.
>>
>> Should be okay then. Does:
>> https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html
>> need to be updated?
>
> Good call, I will send a patch to update the documentation.
>
In fact the table is already auto-generated from libm-ulps files, so there is
no need to change anything.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] math: Use generic sinf implementation for SSE2 version
2022-06-01 12:42 ` Adhemerval Zanella
@ 2022-06-01 15:14 ` H.J. Lu
2022-06-01 16:06 ` Adhemerval Zanella
0 siblings, 1 reply; 19+ messages in thread
From: H.J. Lu @ 2022-06-01 15:14 UTC (permalink / raw)
To: Adhemerval Zanella; +Cc: GNU C Library
On Wed, Jun 1, 2022 at 5:42 AM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 31/05/2022 18:51, H.J. Lu wrote:
> > ? On Tue, May 31, 2022 at 2:35 PM Adhemerval Zanella
> > <adhemerval.zanella@linaro.org> wrote:
> >>
> >> The generic implementation shows slight better performance
> >> (gcc 11.2.1 on a Ryzen 9 5900X):
> >>
> >> * s_sincosf-sse2.S:
> >> "sincosf": {
> >> "workload-random": {
> >> "duration": 3.89961e+09,
> >> "iterations": 9.5472e+07,
> >> "reciprocal-throughput": 40.8429,
> >> "latency": 40.8483,
> >> "max-throughput": 2.4484e+07,
> >> "min-throughput": 2.44808e+07
> >> }
> >> }
> >>
> >> * generic s_cossinf.c:
> >> "sincosf": {
> >> "workload-random": {
> >> "duration": 3.71953e+09,
> >> "iterations": 1.48512e+08,
> >> "reciprocal-throughput": 25.0515,
> >> "latency": 25.0391,
> >> "max-throughput": 3.99177e+07,
> >> "min-throughput": 3.99375e+07
> >> }
> >> }
> >>
> >> Checked on i686-linux-gnu.
> >> ---
> >> sysdeps/i386/i686/fpu/multiarch/Makefile | 1 +
> >> .../i386/i686/fpu/multiarch/libm-test-ulps | 3 +
> >> .../i386/i686/fpu/multiarch/s_sincosf-sse2.S | 585 ------------------
> >> .../i386/i686/fpu/multiarch/s_sincosf-sse2.c | 3 +
> >> 4 files changed, 7 insertions(+), 585 deletions(-)
> >> delete mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> >> create mode 100644 sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> >>
> >> diff --git a/sysdeps/i386/i686/fpu/multiarch/Makefile b/sysdeps/i386/i686/fpu/multiarch/Makefile
> >> index c2cd5cefef..c8b0a6528a 100644
> >> --- a/sysdeps/i386/i686/fpu/multiarch/Makefile
> >> +++ b/sysdeps/i386/i686/fpu/multiarch/Makefile
> >> @@ -9,4 +9,5 @@ CFLAGS-e_logf-sse2.c = -msse2 -mfpmath=sse
> >> CFLAGS-e_powf-sse2.c = -msse2 -mfpmath=sse
> >> CFLAGS-s_cosf-sse2.c = -msse2 -mfpmath=sse
> >> CFLAGS-s_sinf-sse2.c = -msse2 -mfpmath=sse
> >> +CFLAGS-s_sincosf-sse2.c = -msse2 -mfpmath=sse
> >> endif
> >> diff --git a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> >> index 0357d7bd14..284e3fa275 100644
> >> --- a/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> >> +++ b/sysdeps/i386/i686/fpu/multiarch/libm-test-ulps
> >> @@ -1492,16 +1492,19 @@ ldouble: 1
> >>
> >> Function: "sincos_downward":
> >> double: 1
> >> +float: 1
> >> float128: 3
> >> ldouble: 3
> >>
> >> Function: "sincos_towardzero":
> >> double: 1
> >> +float: 1
> >> float128: 2
> >> ldouble: 2
> >>
> >> Function: "sincos_upward":
> >> double: 1
> >> +float: 1
> >> float128: 3
> >> ldouble: 3
> >>
> >> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> >> deleted file mode 100644
> >> index 71e222b930..0000000000
> >> --- a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.S
> >> +++ /dev/null
> >> @@ -1,585 +0,0 @@
> >> -/* Optimized with sse2 version of sincosf
> >> - Copyright (C) 2012-2022 Free Software Foundation, Inc.
> >> - This file is part of the GNU C Library.
> >> -
> >> - The GNU C Library is free software; you can redistribute it and/or
> >> - modify it under the terms of the GNU Lesser General Public
> >> - License as published by the Free Software Foundation; either
> >> - version 2.1 of the License, or (at your option) any later version.
> >> -
> >> - The GNU C Library is distributed in the hope that it will be useful,
> >> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> >> - Lesser General Public License for more details.
> >> -
> >> - You should have received a copy of the GNU Lesser General Public
> >> - License along with the GNU C Library; if not, see
> >> - <https://www.gnu.org/licenses/>. */
> >> -
> >> -#include <sysdep.h>
> >> -#include <errno.h>
> >> -
> >> -/* Short algorithm description:
> >> - *
> >> - * 1) if |x|==0: sin(x)=x,
> >> - * cos(x)=1.
> >> - * 2) if |x|<2^-27: sin(x)=x-x*DP_SMALL, raising underflow only when needed,
> >> - * cos(x)=1-|x|.
> >> - * 3) if |x|<2^-5 : sin(x)=x+x*x^2*DP_SIN2_0+x^5*DP_SIN2_1,
> >> - * cos(x)=1+1*x^2*DP_COS2_0+x^5*DP_COS2_1
> >> - * 4) if |x|< Pi/4: sin(x)=x+x*x^2*(S0+x^2*(S1+x^2*(S2+x^2*(S3+x^2*S4)))),
> >> - * cos(x)=1+1*x^2*(C0+x^2*(C1+x^2*(C2+x^2*(C3+x^2*C4)))).
> >> - * 5) if |x| < 9*Pi/4:
> >> - * 5.1) Range reduction:
> >> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0x0e, n=k+1, t=|x|-j*Pi/4.
> >> - * 5.2) Reconstruction:
> >> - * sign_sin = sign(x) * (-1.0)^(( n >>2)&1)
> >> - * sign_cos = (-1.0)^(((n+2)>>2)&1)
> >> - * poly_sin = ((((S4*t^2 + S3)*t^2 + S2)*t^2 + S1)*t^2 + S0)*t^2*t+t
> >> - * poly_cos = ((((C4*t^2 + C3)*t^2 + C2)*t^2 + C1)*t^2 + C0)*t^2*s+s
> >> - * if(n&2 != 0) {
> >> - * using cos(t) and sin(t) polynomials for |t|<Pi/4, results are
> >> - * cos(x) = poly_sin * sign_cos
> >> - * sin(x) = poly_cos * sign_sin
> >> - * } else {
> >> - * sin(x) = poly_sin * sign_sin
> >> - * cos(x) = poly_cos * sign_cos
> >> - * }
> >> - * 6) if |x| < 2^23, large args:
> >> - * 6.1) Range reduction:
> >> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4
> >> - * 6.2) Reconstruction same as (5.2).
> >> - * 7) if |x| >= 2^23, very large args:
> >> - * 7.1) Range reduction:
> >> - * k=trunc(|x|/(Pi/4)), j=(k+1)&0xfffffffe, n=k+1, t=|x|-j*Pi/4.
> >> - * 7.2) Reconstruction same as (5.2).
> >> - * 8) if x is Inf, return x-x, and set errno=EDOM.
> >> - * 9) if x is NaN, return x-x.
> >> - *
> >> - * Special cases:
> >> - * sin/cos(+-0) = +-0/1 not raising inexact/underflow,
> >> - * sin/cos(subnormal) raises inexact/underflow,
> >> - * sin/cos(min_normalized) raises inexact/underflow,
> >> - * sin/cos(normalized) raises inexact,
> >> - * sin/cos(Inf) = NaN, raises invalid, sets errno to EDOM,
> >> - * sin/cos(NaN) = NaN.
> >> - */
> >> -
> >> -#ifdef PIC
> >> -# define MO1(symbol) L(symbol)##@GOTOFF(%ebx)
> >> -# define MO2(symbol,reg2,_scale) L(symbol)##@GOTOFF(%ebx,reg2,_scale)
> >> -# define CFI_PUSH(REG) cfi_adjust_cfa_offset(4); cfi_rel_offset(REG,0)
> >> -# define CFI_POP(REG) cfi_adjust_cfa_offset(-4); cfi_restore(REG)
> >> -# define PUSH(REG) pushl REG; CFI_PUSH(REG)
> >> -# define POP(REG) popl REG; CFI_POP(REG)
> >> -# define ENTRANCE PUSH(%ebx); LOAD_PIC_REG(bx)
> >> -# define RETURN POP(%ebx); ret; CFI_PUSH(%ebx)
> >> -# define ARG_X 8(%esp)
> >> -# define ARG_SIN_PTR 12(%esp)
> >> -# define ARG_COS_PTR 16(%esp)
> >> -#else
> >> -# define MO1(symbol) L(symbol)
> >> -# define MO2(symbol,reg2,_scale) L(symbol)(,reg2,_scale)
> >> -# define ENTRANCE
> >> -# define RETURN ret
> >> -# define ARG_X 4(%esp)
> >> -# define ARG_SIN_PTR 8(%esp)
> >> -# define ARG_COS_PTR 12(%esp)
> >> -#endif
> >> -
> >> - .text
> >> -ENTRY(__sincosf_sse2)
> >> - /* Input: single precision x on stack at address ARG_X */
> >> - /* pointer to sin result on stack at address ARG_SIN_PTR */
> >> - /* pointer to cos result on stack at address ARG_COS_PTR */
> >> -
> >> - ENTRANCE
> >> - movl ARG_X, %eax /* Bits of x */
> >> - cvtss2sd ARG_X, %xmm0 /* DP x */
> >> - andl $0x7fffffff, %eax /* |x| */
> >> -
> >> - cmpl $0x3f490fdb, %eax /* |x|<Pi/4 ? */
> >> - jb L(arg_less_pio4)
> >> -
> >> - /* Here if |x|>=Pi/4 */
> >> - movd %eax, %xmm3 /* SP |x| */
> >> - andpd MO1(DP_ABS_MASK),%xmm0 /* DP |x| */
> >> - movss MO1(SP_INVPIO4), %xmm2 /* SP 1/(Pi/4) */
> >> -
> >> - cmpl $0x40e231d6, %eax /* |x|<9*Pi/4 ? */
> >> - jae L(large_args)
> >> -
> >> - /* Here if Pi/4<=|x|<9*Pi/4 */
> >> - mulss %xmm3, %xmm2 /* SP |x|/(Pi/4) */
> >> - movl ARG_X, %ecx /* Load x */
> >> - cvttss2si %xmm2, %eax /* k, number of Pi/4 in x */
> >> - shrl $29, %ecx /* (sign of x) << 2 */
> >> - addl $1, %eax /* k+1 */
> >> - movl $0x0e, %edx
> >> - andl %eax, %edx /* j = (k+1)&0x0e */
> >> - subsd MO2(PIO4J,%edx,8), %xmm0/* t = |x| - j * Pi/4 */
> >> -
> >> -L(reconstruction):
> >> - /* Input: %eax=n, %xmm0=t, %ecx=sign(x) */
> >> -
> >> - movaps %xmm0, %xmm4 /* t */
> >> - movhpd MO1(DP_ONES), %xmm4 /* 1|t */
> >> - mulsd %xmm0, %xmm0 /* y=t^2 */
> >> - movl $2, %edx
> >> - unpcklpd %xmm0, %xmm0 /* y|y */
> >> - addl %eax, %edx /* k+2 */
> >> - movaps %xmm0, %xmm1 /* y|y */
> >> - mulpd %xmm0, %xmm0 /* z=t^4|z=t^4 */
> >> -
> >> - movaps MO1(DP_SC4), %xmm2 /* S4 */
> >> - mulpd %xmm0, %xmm2 /* z*S4 */
> >> - movaps MO1(DP_SC3), %xmm3 /* S3 */
> >> - mulpd %xmm0, %xmm3 /* z*S3 */
> >> - xorl %eax, %ecx /* (sign_x ^ (k>>2))<<2 */
> >> - addpd MO1(DP_SC2), %xmm2 /* S2+z*S4 */
> >> - mulpd %xmm0, %xmm2 /* z*(S2+z*S4) */
> >> - shrl $2, %edx /* (k+2)>>2 */
> >> - addpd MO1(DP_SC1), %xmm3 /* S1+z*S3 */
> >> - mulpd %xmm0, %xmm3 /* z*(S1+z*S3) */
> >> - shrl $2, %ecx /* sign_x ^ k>>2 */
> >> - addpd MO1(DP_SC0), %xmm2 /* S0+z*(S2+z*S4) */
> >> - andl $1, %edx /* sign_cos = ((k+2)>>2)&1 */
> >> - mulpd %xmm1, %xmm2 /* y*(S0+z*(S2+z*S4)) */
> >> - andl $1, %ecx /* sign_sin = sign_x ^ ((k>>2)&1) */
> >> - addpd %xmm2, %xmm3 /* y*(S0+y*(S1+y*(S2+y*(S3+y*S4)))) */
> >> - mulpd %xmm4, %xmm3 /*t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
> >> - testl $2, %eax /* n&2 != 0 ? */
> >> - addpd %xmm4, %xmm3 /*t+t*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
> >> - jnz L(sin_result_sin_poly)
> >> -
> >> -/*L(sin_result_cos_poly):*/
> >> - /*
> >> - * Here if
> >> - * cos(x) = poly_sin * sign_cos
> >> - * sin(x) = poly_cos * sign_sin
> >> - */
> >> - movsd MO2(DP_ONES,%ecx,8), %xmm4/* 0|sign_sin */
> >> - movhpd MO2(DP_ONES,%edx,8), %xmm4/* sign_cos|sign_sin */
> >> - mulpd %xmm4, %xmm3 /* result_cos|result_sin */
> >> - movl ARG_SIN_PTR, %eax
> >> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> >> - movl ARG_COS_PTR, %ecx
> >> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
> >> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
> >> - movss %xmm0, (%ecx) /* store cos(x) */
> >> - RETURN
> >> -
> >> - .p2align 4
> >> -L(sin_result_sin_poly):
> >> - /*
> >> - * Here if
> >> - * sin(x) = poly_sin * sign_sin
> >> - * cos(x) = poly_cos * sign_cos
> >> - */
> >> - movsd MO2(DP_ONES,%edx,8), %xmm4/* 0|sign_cos */
> >> - movhpd MO2(DP_ONES,%ecx,8), %xmm4/* sign_sin|sign_cos */
> >> - mulpd %xmm4, %xmm3 /* result_sin|result_cos */
> >> - movl ARG_SIN_PTR, %eax
> >> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> >> - movl ARG_COS_PTR, %ecx
> >> - movss %xmm0, (%ecx) /* store cos(x) from xmm0[0] */
> >> - shufps $1, %xmm0, %xmm0 /* move sin(x) to xmm0[0] */
> >> - movss %xmm0, (%eax) /* store sin(x) */
> >> - RETURN
> >> -
> >> - .p2align 4
> >> -L(large_args):
> >> - /* Here if |x|>=9*Pi/4 */
> >> - cmpl $0x7f800000, %eax /* x is Inf or NaN ? */
> >> - jae L(arg_inf_or_nan)
> >> -
> >> - /* Here if finite |x|>=9*Pi/4 */
> >> - cmpl $0x4b000000, %eax /* |x|<2^23 ? */
> >> - jae L(very_large_args)
> >> -
> >> - /* Here if 9*Pi/4<=|x|<2^23 */
> >> - movsd MO1(DP_INVPIO4), %xmm1 /* 1/(Pi/4) */
> >> - mulsd %xmm0, %xmm1 /* |x|/(Pi/4) */
> >> - cvttsd2si %xmm1, %eax /* k=trunc(|x|/(Pi/4)) */
> >> - addl $1, %eax /* k+1 */
> >> - movl %eax, %edx
> >> - andl $0xfffffffe, %edx /* j=(k+1)&0xfffffffe */
> >> - cvtsi2sdl %edx, %xmm4 /* DP j */
> >> - movl ARG_X, %ecx /* Load x */
> >> - movsd MO1(DP_PIO4HI), %xmm2 /* -PIO4HI = high part of -Pi/4 */
> >> - shrl $29, %ecx /* (sign of x) << 2 */
> >> - mulsd %xmm4, %xmm2 /* -j*PIO4HI */
> >> - movsd MO1(DP_PIO4LO), %xmm3 /* -PIO4LO = low part of -Pi/4 */
> >> - addsd %xmm2, %xmm0 /* |x| - j*PIO4HI */
> >> - mulsd %xmm3, %xmm4 /* j*PIO4LO */
> >> - addsd %xmm4, %xmm0 /* t = |x| - j*PIO4HI - j*PIO4LO */
> >> - jmp L(reconstruction)
> >> -
> >> - .p2align 4
> >> -L(very_large_args):
> >> - /* Here if finite |x|>=2^23 */
> >> -
> >> - /* bitpos = (ix>>23) - BIAS_32 + 59; */
> >> - shrl $23, %eax /* eb = biased exponent of x */
> >> - subl $68, %eax /* bitpos=eb-0x7f+59, where 0x7f */
> >> - /*is exponent bias */
> >> - movl $28, %ecx /* %cl=28 */
> >> - movl %eax, %edx /* bitpos copy */
> >> -
> >> - /* j = bitpos/28; */
> >> - div %cl /* j in register %al=%ax/%cl */
> >> - movapd %xmm0, %xmm3 /* |x| */
> >> - andl $0xff, %eax /* clear unneeded remainder from %ah*/
> >> -
> >> - imull $28, %eax, %ecx /* j*28 */
> >> - movsd MO1(DP_HI_MASK), %xmm4 /* DP_HI_MASK */
> >> - movapd %xmm0, %xmm5 /* |x| */
> >> - mulsd -2*8+MO2(_FPI,%eax,8), %xmm3/* tmp3 = FPI[j-2]*|x| */
> >> - movapd %xmm0, %xmm1 /* |x| */
> >> - mulsd -1*8+MO2(_FPI,%eax,8), %xmm5/* tmp2 = FPI[j-1]*|x| */
> >> - mulsd 0*8+MO2(_FPI,%eax,8), %xmm0/* tmp0 = FPI[j]*|x| */
> >> - addl $19, %ecx /* j*28+19 */
> >> - mulsd 1*8+MO2(_FPI,%eax,8), %xmm1/* tmp1 = FPI[j+1]*|x| */
> >> - cmpl %ecx, %edx /* bitpos>=j*28+19 ? */
> >> - jl L(very_large_skip1)
> >> -
> >> - /* Here if bitpos>=j*28+19 */
> >> - andpd %xmm3, %xmm4 /* HI(tmp3) */
> >> - subsd %xmm4, %xmm3 /* tmp3 = tmp3 - HI(tmp3) */
> >> -L(very_large_skip1):
> >> -
> >> - movsd MO1(DP_2POW52), %xmm6
> >> - movapd %xmm5, %xmm2 /* tmp2 copy */
> >> - addsd %xmm3, %xmm5 /* tmp5 = tmp3 + tmp2 */
> >> - movl $1, %edx
> >> - addsd %xmm5, %xmm6 /* tmp6 = tmp5 + 2^52 */
> >> - movsd 8+MO1(DP_2POW52), %xmm4
> >> - movd %xmm6, %eax /* k = I64_LO(tmp6); */
> >> - addsd %xmm6, %xmm4 /* tmp4 = tmp6 - 2^52 */
> >> - movl ARG_X, %ecx /* Load x */
> >> - comisd %xmm5, %xmm4 /* tmp4 > tmp5 ? */
> >> - jbe L(very_large_skip2)
> >> -
> >> - /* Here if tmp4 > tmp5 */
> >> - subl $1, %eax /* k-- */
> >> - addsd 8+MO1(DP_ONES), %xmm4 /* tmp4 -= 1.0 */
> >> -L(very_large_skip2):
> >> -
> >> - andl %eax, %edx /* k&1 */
> >> - subsd %xmm4, %xmm3 /* tmp3 -= tmp4 */
> >> - addsd MO2(DP_ZERONE,%edx,8), %xmm3/* t = DP_ZERONE[k&1] + tmp3 */
> >> - addsd %xmm2, %xmm3 /* t += tmp2 */
> >> - shrl $29, %ecx /* (sign of x) << 2 */
> >> - addsd %xmm3, %xmm0 /* t += tmp0 */
> >> - addl $1, %eax /* n=k+1 */
> >> - addsd %xmm1, %xmm0 /* t += tmp1 */
> >> - mulsd MO1(DP_PIO4), %xmm0 /* t *= PI04 */
> >> -
> >> - jmp L(reconstruction) /* end of very_large_args peth */
> >> -
> >> - .p2align 4
> >> -L(arg_less_pio4):
> >> - /* Here if |x|<Pi/4 */
> >> - cmpl $0x3d000000, %eax /* |x|<2^-5 ? */
> >> - jl L(arg_less_2pn5)
> >> -
> >> - /* Here if 2^-5<=|x|<Pi/4 */
> >> - movaps %xmm0, %xmm3 /* DP x */
> >> - movhpd MO1(DP_ONES), %xmm3 /* DP 1|x */
> >> - mulsd %xmm0, %xmm0 /* DP y=x^2 */
> >> - unpcklpd %xmm0, %xmm0 /* DP y|y */
> >> - movaps %xmm0, %xmm1 /* y|y */
> >> - mulpd %xmm0, %xmm0 /* z=x^4|z=x^4 */
> >> -
> >> - movapd MO1(DP_SC4), %xmm4 /* S4 */
> >> - mulpd %xmm0, %xmm4 /* z*S4 */
> >> - movapd MO1(DP_SC3), %xmm5 /* S3 */
> >> - mulpd %xmm0, %xmm5 /* z*S3 */
> >> - addpd MO1(DP_SC2), %xmm4 /* S2+z*S4 */
> >> - mulpd %xmm0, %xmm4 /* z*(S2+z*S4) */
> >> - addpd MO1(DP_SC1), %xmm5 /* S1+z*S3 */
> >> - mulpd %xmm0, %xmm5 /* z*(S1+z*S3) */
> >> - addpd MO1(DP_SC0), %xmm4 /* S0+z*(S2+z*S4) */
> >> - mulpd %xmm1, %xmm4 /* y*(S0+z*(S2+z*S4)) */
> >> - mulpd %xmm3, %xmm5 /* x*z*(S1+z*S3) */
> >> - mulpd %xmm3, %xmm4 /* x*y*(S0+z*(S2+z*S4)) */
> >> - addpd %xmm5, %xmm4 /*x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))))*/
> >> - movl ARG_SIN_PTR, %eax
> >> - addpd %xmm4, %xmm3 /*x+x*y*(S0+y*(S1+y*(S2+y*(S3+y*S4))*/
> >> - movl ARG_COS_PTR, %ecx
> >> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> >> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
> >> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
> >> - movss %xmm0, (%ecx) /* store cos(x) */
> >> - RETURN
> >> -
> >> - .p2align 4
> >> -L(arg_less_2pn5):
> >> - /* Here if |x|<2^-5 */
> >> - cmpl $0x32000000, %eax /* |x|<2^-27 ? */
> >> - jl L(arg_less_2pn27)
> >> -
> >> - /* Here if 2^-27<=|x|<2^-5 */
> >> - movaps %xmm0, %xmm1 /* DP x */
> >> - movhpd MO1(DP_ONES), %xmm1 /* DP 1|x */
> >> - mulsd %xmm0, %xmm0 /* DP x^2 */
> >> - unpcklpd %xmm0, %xmm0 /* DP x^2|x^2 */
> >> -
> >> - movaps MO1(DP_SINCOS2_1), %xmm3/* DP DP_SIN2_1 */
> >> - mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_1 */
> >> - addpd MO1(DP_SINCOS2_0), %xmm3/* DP DP_SIN2_0+x^2*DP_SIN2_1 */
> >> - mulpd %xmm0, %xmm3 /* DP x^2*DP_SIN2_0+x^4*DP_SIN2_1 */
> >> - mulpd %xmm1, %xmm3 /* DP x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
> >> - addpd %xmm1, %xmm3 /* DP x+x^3*DP_SIN2_0+x^5*DP_SIN2_1 */
> >> - movl ARG_SIN_PTR, %eax
> >> - cvtpd2ps %xmm3, %xmm0 /* SP results */
> >> - movl ARG_COS_PTR, %ecx
> >> - movss %xmm0, (%eax) /* store sin(x) from xmm0[0] */
> >> - shufps $1, %xmm0, %xmm0 /* move cos(x) to xmm0[0] */
> >> - movss %xmm0, (%ecx) /* store cos(x) */
> >> - RETURN
> >> -
> >> - .p2align 4
> >> -L(arg_less_2pn27):
> >> - movss ARG_X, %xmm7 /* SP x */
> >> - cmpl $0, %eax /* x=0 ? */
> >> - je L(arg_zero) /* in case x=0 return sin(+-0)==+-0 */
> >> - /* Here if |x|<2^-27 */
> >> - /*
> >> - * Special cases here:
> >> - * sin(subnormal) raises inexact/underflow
> >> - * sin(min_normalized) raises inexact/underflow
> >> - * sin(normalized) raises inexact
> >> - * cos(here)=1-|x| (raising inexact)
> >> - */
> >> - movaps %xmm0, %xmm3 /* DP x */
> >> - mulsd MO1(DP_SMALL), %xmm0 /* DP x*DP_SMALL */
> >> - subsd %xmm0, %xmm3 /* DP sin result is x-x*DP_SMALL */
> >> - andps MO1(SP_ABS_MASK), %xmm7 /* SP |x| */
> >> - cvtsd2ss %xmm3, %xmm0 /* sin(x) */
> >> - movl ARG_SIN_PTR, %eax
> >> - movss MO1(SP_ONE), %xmm1 /* SP 1.0 */
> >> - movss %xmm0, (%eax) /* sin(x) store */
> >> - movl ARG_COS_PTR, %ecx
> >> - subss %xmm7, %xmm1 /* cos(x) */
> >> - movss %xmm1, (%ecx) /* cos(x) store */
> >> - RETURN
> >> -
> >> - .p2align 4
> >> -L(arg_zero):
> >> - movss MO1(SP_ONE), %xmm0 /* 1.0 */
> >> - movl ARG_SIN_PTR, %eax
> >> - movl ARG_COS_PTR, %ecx
> >> - movss %xmm7, (%eax) /* sin(+-0)==x */
> >> - movss %xmm0, (%ecx) /* cos(+-0)==1 */
> >> - RETURN
> >> -
> >> - .p2align 4
> >> -L(arg_inf_or_nan):
> >> - movss ARG_X, %xmm7 /* SP x */
> >> - /* Here if |x| is Inf or NAN */
> >> - jne L(skip_errno_setting) /* in case of x is NaN */
> >> -
> >> - /* Here if x is Inf. Set errno to EDOM. */
> >> - call JUMPTARGET(__errno_location)
> >> - movl $EDOM, (%eax)
> >> -
> >> - .p2align 4
> >> -L(skip_errno_setting):
> >> - /* Here if |x| is Inf or NAN. Continued. */
> >> - subss %xmm7, %xmm7 /* x-x, result is NaN */
> >> - movl ARG_SIN_PTR, %eax
> >> - movl ARG_COS_PTR, %ecx
> >> - movss %xmm7, (%eax)
> >> - movss %xmm7, (%ecx)
> >> - RETURN
> >> -END(__sincosf_sse2)
> >> -
> >> - .section .rodata, "a"
> >> - .p2align 3
> >> -L(PIO4J): /* Table of j*Pi/4, for j=0,1,..,10 */
> >> - .long 0x00000000,0x00000000
> >> - .long 0x54442d18,0x3fe921fb
> >> - .long 0x54442d18,0x3ff921fb
> >> - .long 0x7f3321d2,0x4002d97c
> >> - .long 0x54442d18,0x400921fb
> >> - .long 0x2955385e,0x400f6a7a
> >> - .long 0x7f3321d2,0x4012d97c
> >> - .long 0xe9bba775,0x4015fdbb
> >> - .long 0x54442d18,0x401921fb
> >> - .long 0xbeccb2bb,0x401c463a
> >> - .long 0x2955385e,0x401f6a7a
> >> - .type L(PIO4J), @object
> >> - ASM_SIZE_DIRECTIVE(L(PIO4J))
> >> -
> >> - .p2align 3
> >> -L(_FPI): /* 4/Pi broken into sum of positive DP values */
> >> - .long 0x00000000,0x00000000
> >> - .long 0x6c000000,0x3ff45f30
> >> - .long 0x2a000000,0x3e3c9c88
> >> - .long 0xa8000000,0x3c54fe13
> >> - .long 0xd0000000,0x3aaf47d4
> >> - .long 0x6c000000,0x38fbb81b
> >> - .long 0xe0000000,0x3714acc9
> >> - .long 0x7c000000,0x3560e410
> >> - .long 0x56000000,0x33bca2c7
> >> - .long 0xac000000,0x31fbd778
> >> - .long 0xe0000000,0x300b7246
> >> - .long 0xe8000000,0x2e5d2126
> >> - .long 0x48000000,0x2c970032
> >> - .long 0xe8000000,0x2ad77504
> >> - .long 0xe0000000,0x290921cf
> >> - .long 0xb0000000,0x274deb1c
> >> - .long 0xe0000000,0x25829a73
> >> - .long 0xbe000000,0x23fd1046
> >> - .long 0x10000000,0x2224baed
> >> - .long 0x8e000000,0x20709d33
> >> - .long 0x80000000,0x1e535a2f
> >> - .long 0x64000000,0x1cef904e
> >> - .long 0x30000000,0x1b0d6398
> >> - .long 0x24000000,0x1964ce7d
> >> - .long 0x16000000,0x17b908bf
> >> - .type L(_FPI), @object
> >> - ASM_SIZE_DIRECTIVE(L(_FPI))
> >> -
> >> -/* Coefficients of polynomials for */
> >> -/* sin(x)~=x+x*x^2*(DP_SIN2_0+x^2*DP_SIN2_1) in low DP part, */
> >> -/* cos(x)~=1+1*x^2*(DP_COS2_0+x^2*DP_COS2_1) in high DP part, */
> >> -/* for |x|<2^-5. */
> >> - .p2align 4
> >> -L(DP_SINCOS2_0):
> >> - .long 0x5543d49d,0xbfc55555
> >> - .long 0xff5cc6fd,0xbfdfffff
> >> - .type L(DP_SINCOS2_0), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_0))
> >> -
> >> - .p2align 4
> >> -L(DP_SINCOS2_1):
> >> - .long 0x75cec8c5,0x3f8110f4
> >> - .long 0xb178dac5,0x3fa55514
> >> - .type L(DP_SINCOS2_1), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SINCOS2_1))
> >> -
> >> - .p2align 3
> >> -L(DP_ZERONE):
> >> - .long 0x00000000,0x00000000 /* 0.0 */
> >> - .long 0x00000000,0xbff00000 /* 1.0 */
> >> - .type L(DP_ZERONE), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_ZERONE))
> >> -
> >> - .p2align 3
> >> -L(DP_ONES):
> >> - .long 0x00000000,0x3ff00000 /* +1.0 */
> >> - .long 0x00000000,0xbff00000 /* -1.0 */
> >> - .type L(DP_ONES), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_ONES))
> >> -
> >> -/* Coefficients of polynomials for */
> >> -/* sin(t)~=t+t*t^2*(S0+t^2*(S1+t^2*(S2+t^2*(S3+t^2*S4)))) in low DP part, */
> >> -/* cos(t)~=1+1*t^2*(C0+t^2*(C1+t^2*(C2+t^2*(C3+t^2*C4)))) in high DP part, */
> >> -/* for |t|<Pi/4. */
> >> - .p2align 4
> >> -L(DP_SC4):
> >> - .long 0x1674b58a,0xbe5a947e
> >> - .long 0xdd8844d7,0xbe923c97
> >> - .type L(DP_SC4), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SC4))
> >> -
> >> - .p2align 4
> >> -L(DP_SC3):
> >> - .long 0x64e6b5b4,0x3ec71d72
> >> - .long 0x9ac43cc0,0x3efa00eb
> >> - .type L(DP_SC3), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SC3))
> >> -
> >> - .p2align 4
> >> -L(DP_SC2):
> >> - .long 0x8b4bd1f9,0xbf2a019f
> >> - .long 0x348b6874,0xbf56c16b
> >> - .type L(DP_SC2), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SC2))
> >> -
> >> - .p2align 4
> >> -L(DP_SC1):
> >> - .long 0x10c2688b,0x3f811111
> >> - .long 0x545c50c7,0x3fa55555
> >> - .type L(DP_SC1), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SC1))
> >> -
> >> - .p2align 4
> >> -L(DP_SC0):
> >> - .long 0x55551cd9,0xbfc55555
> >> - .long 0xfffe98ae,0xbfdfffff
> >> - .type L(DP_SC0), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SC0))
> >> -
> >> - .p2align 3
> >> -L(DP_SMALL):
> >> - .long 0x00000000,0x3cd00000 /* 2^(-50) */
> >> - .type L(DP_SMALL), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_SMALL))
> >> -
> >> - .p2align 3
> >> -L(DP_PIO4):
> >> - .long 0x54442d18,0x3fe921fb /* Pi/4 */
> >> - .type L(DP_PIO4), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_PIO4))
> >> -
> >> - .p2align 3
> >> -L(DP_2POW52):
> >> - .long 0x00000000,0x43300000 /* +2^52 */
> >> - .long 0x00000000,0xc3300000 /* -2^52 */
> >> - .type L(DP_2POW52), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_2POW52))
> >> -
> >> - .p2align 3
> >> -L(DP_INVPIO4):
> >> - .long 0x6dc9c883,0x3ff45f30 /* 4/Pi */
> >> - .type L(DP_INVPIO4), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_INVPIO4))
> >> -
> >> - .p2align 3
> >> -L(DP_PIO4HI):
> >> - .long 0x54000000,0xbfe921fb /* High part of Pi/4 */
> >> - .type L(DP_PIO4HI), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_PIO4HI))
> >> -
> >> - .p2align 3
> >> -L(DP_PIO4LO):
> >> - .long 0x11A62633,0xbe010b46 /* Low part of Pi/4 */
> >> - .type L(DP_PIO4LO), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_PIO4LO))
> >> -
> >> - .p2align 2
> >> -L(SP_INVPIO4):
> >> - .long 0x3fa2f983 /* 4/Pi */
> >> - .type L(SP_INVPIO4), @object
> >> - ASM_SIZE_DIRECTIVE(L(SP_INVPIO4))
> >> -
> >> - .p2align 4
> >> -L(DP_ABS_MASK): /* Mask for getting DP absolute value */
> >> - .long 0xffffffff,0x7fffffff
> >> - .long 0xffffffff,0x7fffffff
> >> - .type L(DP_ABS_MASK), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_ABS_MASK))
> >> -
> >> - .p2align 3
> >> -L(DP_HI_MASK): /* Mask for getting high 21 bits of DP value */
> >> - .long 0x00000000,0xffffffff
> >> - .type L(DP_HI_MASK), @object
> >> - ASM_SIZE_DIRECTIVE(L(DP_HI_MASK))
> >> -
> >> - .p2align 4
> >> -L(SP_ABS_MASK): /* Mask for getting SP absolute value */
> >> - .long 0x7fffffff,0x7fffffff
> >> - .long 0x7fffffff,0x7fffffff
> >> - .type L(SP_ABS_MASK), @object
> >> - ASM_SIZE_DIRECTIVE(L(SP_ABS_MASK))
> > t> -
> >> - .p2align 2
> >> -L(SP_ONE):
> >> - .long 0x3f800000 /* 1.0 */
> >> - .type L(SP_ONE), @object
> >> - ASM_SIZE_DIRECTIVE(L(SP_ONE))
> >> -
> >> -weak_alias(__sincosf, sincosf)
> >> diff --git a/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> >> new file mode 100644
> >> index 0000000000..46363fc11a
> >> --- /dev/null
> >> +++ b/sysdeps/i386/i686/fpu/multiarch/s_sincosf-sse2.c
> >> @@ -0,0 +1,3 @@
> >> +#define SINCOSF __sincosf_sse2
> >> +#define SECTION __attribute__ ((section (".text.sse2")))
> >> +#include <sysdeps/ieee754/flt-32/s_sincosf.c>
> >> --
> >> 2.34.1
> >>
> >
> > Don't you need to add SECTION to sysdeps/ieee754/flt-32/s_sincosf.c?
> >
>
> It does, thanks for catching it. I will fix it before commit.
The commit subject should be sincosf, not sinf.
> diff --git a/sysdeps/ieee754/flt-32/s_sincosf.c b/sysdeps/ieee754/flt-32/s_sincosf.c
> index 918f6a655a..4fab42d974 100644
> --- a/sysdeps/ieee754/flt-32/s_sincosf.c
> +++ b/sysdeps/ieee754/flt-32/s_sincosf.c
> @@ -24,6 +24,10 @@
> #include "math_config.h"
> #include "s_sincosf.h"
>
> +#ifndef SECTION
> +# define SECTION
> +#endif
> +
> #ifndef SINCOSF
> # define SINCOSF_FUNC __sincosf
> #else
> @@ -35,6 +39,7 @@
> small values. Large inputs have their range reduced using fast integer
> arithmetic. */
> void
> +SECTION
> SINCOSF_FUNC (float y, float *sinp, float *cosp)
> {
> double x = y;
--
H.J.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] math: Use generic sinf implementation for SSE2 version
2022-06-01 15:14 ` H.J. Lu
@ 2022-06-01 16:06 ` Adhemerval Zanella
0 siblings, 0 replies; 19+ messages in thread
From: Adhemerval Zanella @ 2022-06-01 16:06 UTC (permalink / raw)
To: H.J. Lu; +Cc: GNU C Library
On 01/06/2022 12:14, H.J. Lu wrote:
> On Wed, Jun 1, 2022 at 5:42 AM Adhemerval Zanella
>>> Don't you need to add SECTION to sysdeps/ieee754/flt-32/s_sincosf.c?
>>>
>>
>> It does, thanks for catching it. I will fix it before commit.
>
> The commit subject should be sincosf, not sinf.
Yeah, I fixed it as well.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2022-06-01 16:06 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-31 21:34 [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Adhemerval Zanella
2022-05-31 21:34 ` [PATCH 1/5] benchtests: Add workload name for cosf Adhemerval Zanella
2022-05-31 21:47 ` H.J. Lu
2022-05-31 21:34 ` [PATCH 2/5] i686: Use generic cosf implementation for SSE2 version Adhemerval Zanella
2022-05-31 21:46 ` H.J. Lu
2022-05-31 21:35 ` [PATCH 3/5] i686: Use generic sinf " Adhemerval Zanella
2022-05-31 21:47 ` H.J. Lu
2022-05-31 21:35 ` [PATCH 4/5] benchtests: Add workload name for sincosf Adhemerval Zanella
2022-05-31 21:48 ` H.J. Lu
2022-05-31 21:35 ` [PATCH 5/5] math: Use generic sinf implementation for SSE2 version Adhemerval Zanella
2022-05-31 21:51 ` H.J. Lu
2022-06-01 12:42 ` Adhemerval Zanella
2022-06-01 15:14 ` H.J. Lu
2022-06-01 16:06 ` Adhemerval Zanella
2022-05-31 21:48 ` [PATCH 0/5] Use generic sinf, cosf, and sincosf for i686 Noah Goldstein
2022-05-31 21:49 ` H.J. Lu
2022-05-31 21:53 ` Noah Goldstein
2022-06-01 12:36 ` Adhemerval Zanella
2022-06-01 13:12 ` Adhemerval Zanella
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).