* [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2
@ 2022-06-28 15:27 Noah Goldstein
2022-06-28 15:27 ` [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
` (7 more replies)
0 siblings, 8 replies; 36+ messages in thread
From: Noah Goldstein @ 2022-06-28 15:27 UTC (permalink / raw)
To: libc-alpha
Just for clarities sake and so that if a future implementation is
added we remember to add the check.
---
sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h
index ee36525bcf..204c4b5406 100644
--- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h
+++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h
@@ -27,7 +27,11 @@ IFUNC_SELECTOR (void)
{
const struct cpu_features* cpu_features = __get_cpu_features ();
- if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2))
+ /* This function uses slow sse4.2 instructions (pcmpstri) but since
+ there is no other optimized implementation keep using. If an
+ optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P
+ (cpu_features, SSE4_2) check. */
+ if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2))
return OPTIMIZE (sse42);
return OPTIMIZE (generic);
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein @ 2022-06-28 15:27 ` Noah Goldstein 2022-06-29 19:20 ` H.J. Lu 2022-06-29 18:53 ` [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu ` (6 subsequent siblings) 7 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-06-28 15:27 UTC (permalink / raw) To: libc-alpha The changes for these functions are different than the others because the best implementation (sse4_2) requires the generic implementation as a fallback to be built as well. Changes are: 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically select the best implementation based on the configured ISA build level. 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the sysdeps/x86_64 directory so that the the sse4 implementation will have all of its dependencies for the non-multiarch / rtld build when ISA level >= 3. Add new multiarch/rtld-strcspn.c that just include the non-multiarch strcspn.c which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch. --- sysdeps/x86_64/Makefile | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-impl-list.c | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ sysdeps/x86_64/multiarch/strcspn-generic.c | 11 +++++---- sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- sysdeps/x86_64/multiarch/strpbrk-generic.c | 9 +++++--- sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- sysdeps/x86_64/multiarch/strspn-generic.c | 8 ++++--- sysdeps/x86_64/multiarch/strspn-sse4.c | 6 ++++- sysdeps/x86_64/strcspn-generic.c | 22 ++++++++++++++++++ sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strpbrk-generic.c | 22 ++++++++++++++++++ sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strspn-generic.c | 23 ++++++++++++++++++ sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/varshift.c | 23 ++++++++++++++++++ 17 files changed, 238 insertions(+), 17 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c create mode 100644 sysdeps/x86_64/strcspn-generic.c create mode 100644 sysdeps/x86_64/strcspn.c create mode 100644 sysdeps/x86_64/strpbrk-generic.c create mode 100644 sysdeps/x86_64/strpbrk.c create mode 100644 sysdeps/x86_64/strspn-generic.c create mode 100644 sysdeps/x86_64/strspn.c create mode 100644 sysdeps/x86_64/varshift.c diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile index 79365aff2a..c130d56342 100644 --- a/sysdeps/x86_64/Makefile +++ b/sysdeps/x86_64/Makefile @@ -14,7 +14,14 @@ sysdep_noprof += _mcount endif ifeq ($(subdir),string) -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii +sysdep_routines += \ + strcasecmp_l-nonascii \ + strcspn-generic \ + strncase_l-nonascii \ + strpbrk-generic \ + strspn-generic \ + varshift \ +#sysdep_routines gen-as-const-headers += locale-defines.sym tests += \ tst-rsi-strlen diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c index 772b4ace6c..251b0022d9 100644 --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c @@ -540,6 +540,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strcspn.c. */ IFUNC_IMPL (i, name, strcspn, + /* All implementations of strcspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), __strcspn_sse42) IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) @@ -616,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ IFUNC_IMPL (i, name, strpbrk, + /* All implementations of strpbrk are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), __strpbrk_sse42) IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) @@ -623,13 +627,16 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strspn.c. */ IFUNC_IMPL (i, name, strspn, + /* All implementations of strspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), __strspn_sse42) IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) /* Support sysdeps/x86_64/multiarch/strstr.c. */ IFUNC_IMPL (i, name, strstr, - /* All implementations of strstr are built at all ISA levels. */ + /* All implementations of strstr are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strstr, (CPU_FEATURE_USABLE (AVX512VL) && CPU_FEATURE_USABLE (AVX512BW) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index 204c4b5406..d642f54fac 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) there is no other optimized implementation keep using. If an optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2) check. */ - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c new file mode 100644 index 0000000000..99e3c59e00 --- /dev/null +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c @@ -0,0 +1,18 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include "../strcspn.c" diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c index 423de2e2b2..64e90a26bb 100644 --- a/sysdeps/x86_64/multiarch/strcspn-generic.c +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strcspn-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRCSPN __strcspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRCSPN) -#endif -#include <string/strcspn.c> +# include <string/strcspn.c> + +#endif diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c index 59f64f9fe8..becdaf05f3 100644 --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c @@ -52,9 +52,11 @@ when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset X for case 1. */ +#ifndef STRCSPN +# define STRCSPN __strcspn_sse42 +#endif #ifndef STRCSPN_GENERIC # define STRCSPN_GENERIC __strcspn_generic -# define STRCSPN_SSE42 __strcspn_sse42 #endif #ifdef USE_AS_STRPBRK @@ -78,7 +80,7 @@ char * size_t #endif __attribute__ ((section (".text.sse4.2"))) -STRCSPN_SSE42 (const char *s, const char *a) +STRCSPN (const char *s, const char *a) { if (*a == 0) RETURN (NULL, strlen (s)); diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c index d31acfe495..7f3ea86e91 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +/* We always need to build this implementation as strpbrk-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRPBRK __strpbrk_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRPBRK) -#endif -#include <string/strpbrk.c> +# include <string/strpbrk.c> +#endif diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c index bf74d660d5..0adb577955 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c @@ -16,7 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ +#ifndef STRPBRK +# define STRPBRK __strpbrk_sse42 +#endif + #define USE_AS_STRPBRK #define STRCSPN_GENERIC __strpbrk_generic -#define STRCSPN_SSE42 __strpbrk_sse42 +#define STRCSPN STRPBRK #include "strcspn-sse4.c" diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c index 6b50c36432..f5632ad3b2 100644 --- a/sysdeps/x86_64/multiarch/strspn-generic.c +++ b/sysdeps/x86_64/multiarch/strspn-generic.c @@ -16,13 +16,15 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strspn-sse4 needs to + be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRSPN __strspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRSPN) -#endif #include <string/strspn.c> +#endif diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c index d044916688..6263669b54 100644 --- a/sysdeps/x86_64/multiarch/strspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c @@ -51,12 +51,16 @@ We exit from the loop for case 1. */ + extern size_t __strspn_generic (const char *, const char *) attribute_hidden; +#ifndef STRSPN +# define STRSPN __strspn_sse42 +#endif size_t __attribute__ ((section (".text.sse4.2"))) -__strspn_sse42 (const char *s, const char *a) +STRSPN (const char *s, const char *a) { if (*a == 0) return 0; diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c new file mode 100644 index 0000000000..082324e793 --- /dev/null +++ b/sysdeps/x86_64/strcspn-generic.c @@ -0,0 +1,22 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strcspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c new file mode 100644 index 0000000000..cd54eed869 --- /dev/null +++ b/sysdeps/x86_64/strcspn.c @@ -0,0 +1,27 @@ +/* strcspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strcspn.c> +#else +#define STRCSPN strcspn +#include "multiarch/strcspn-sse4.c" +libc_hidden_builtin_def (strcspn) +#endif diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c new file mode 100644 index 0000000000..6bdbb2d164 --- /dev/null +++ b/sysdeps/x86_64/strpbrk-generic.c @@ -0,0 +1,22 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strpbrk-generic.c" +#endif diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c new file mode 100644 index 0000000000..e7ea1b334a --- /dev/null +++ b/sysdeps/x86_64/strpbrk.c @@ -0,0 +1,27 @@ +/* strpbrk hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strpbrk.c> +#else +#define STRPBRK strpbrk +#include "multiarch/strpbrk-sse4.c" +libc_hidden_builtin_def (strpbrk) +#endif diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c new file mode 100644 index 0000000000..0730972bbd --- /dev/null +++ b/sysdeps/x86_64/strspn-generic.c @@ -0,0 +1,23 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c new file mode 100644 index 0000000000..7b9ede26d9 --- /dev/null +++ b/sysdeps/x86_64/strspn.c @@ -0,0 +1,27 @@ +/* strspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strspn.c> +#else +#define STRSPN strspn +#include "multiarch/strspn-sse4.c" +libc_hidden_builtin_def (strspn) +#endif diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c new file mode 100644 index 0000000000..ab3c506378 --- /dev/null +++ b/sysdeps/x86_64/varshift.c @@ -0,0 +1,23 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/varshift.c" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-28 15:27 ` [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-06-29 19:20 ` H.J. Lu 2022-06-29 22:07 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-06-29 19:20 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Tue, Jun 28, 2022 at 8:27 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > The changes for these functions are different than the others because > the best implementation (sse4_2) requires the generic > implementation as a fallback to be built as well. > > Changes are: > > 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically > select the best implementation based on the configured ISA build > level. > > 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the > sysdeps/x86_64 directory so that the the sse4 implementation will > have all of its dependencies for the non-multiarch / rtld build > when ISA level >= Something is missing. > 3. Add new multiarch/rtld-strcspn.c that just include the > non-multiarch strcspn.c which will in turn select the best > implementation based on the compiled ISA level. > > 4. Refactor the ifunc selector and ifunc implementation list to use > the ISA level aware wrapper macros that allow functions below the > compiled ISA level (with a guranteed replacement) to be skipped. > > Tested with and without multiarch on x86_64 for ISA levels: > {generic, x86-64-v2, x86-64-v3, x86-64-v4} > > And m32 with and without multiarch. > --- > sysdeps/x86_64/Makefile | 9 +++++++- > sysdeps/x86_64/multiarch/ifunc-impl-list.c | 9 +++++++- > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- > sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ > sysdeps/x86_64/multiarch/strcspn-generic.c | 11 +++++---- > sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- > sysdeps/x86_64/multiarch/strpbrk-generic.c | 9 +++++--- > sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- > sysdeps/x86_64/multiarch/strspn-generic.c | 8 ++++--- > sysdeps/x86_64/multiarch/strspn-sse4.c | 6 ++++- > sysdeps/x86_64/strcspn-generic.c | 22 ++++++++++++++++++ > sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/strpbrk-generic.c | 22 ++++++++++++++++++ > sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/strspn-generic.c | 23 ++++++++++++++++++ > sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/varshift.c | 23 ++++++++++++++++++ > 17 files changed, 238 insertions(+), 17 deletions(-) > create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c > create mode 100644 sysdeps/x86_64/strcspn-generic.c > create mode 100644 sysdeps/x86_64/strcspn.c > create mode 100644 sysdeps/x86_64/strpbrk-generic.c > create mode 100644 sysdeps/x86_64/strpbrk.c > create mode 100644 sysdeps/x86_64/strspn-generic.c > create mode 100644 sysdeps/x86_64/strspn.c > create mode 100644 sysdeps/x86_64/varshift.c > > diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile > index 79365aff2a..c130d56342 100644 > --- a/sysdeps/x86_64/Makefile > +++ b/sysdeps/x86_64/Makefile > @@ -14,7 +14,14 @@ sysdep_noprof += _mcount > endif > > ifeq ($(subdir),string) > -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii > +sysdep_routines += \ > + strcasecmp_l-nonascii \ > + strcspn-generic \ > + strncase_l-nonascii \ > + strpbrk-generic \ > + strspn-generic \ > + varshift \ > +#sysdep_routines Add a space after #. > gen-as-const-headers += locale-defines.sym > tests += \ > tst-rsi-strlen > diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > index 772b4ace6c..251b0022d9 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c > +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > @@ -540,6 +540,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strcspn.c. */ > IFUNC_IMPL (i, name, strcspn, > + /* All implementations of strcspn are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), > __strcspn_sse42) > IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) > @@ -616,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ > IFUNC_IMPL (i, name, strpbrk, > + /* All implementations of strpbrk are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), > __strpbrk_sse42) > IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) > @@ -623,13 +627,16 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strspn.c. */ > IFUNC_IMPL (i, name, strspn, > + /* All implementations of strspn are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), > __strspn_sse42) > IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) > > /* Support sysdeps/x86_64/multiarch/strstr.c. */ > IFUNC_IMPL (i, name, strstr, > - /* All implementations of strstr are built at all ISA levels. */ > + /* All implementations of strstr are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strstr, > (CPU_FEATURE_USABLE (AVX512VL) > && CPU_FEATURE_USABLE (AVX512BW) > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index 204c4b5406..d642f54fac 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) > there is no other optimized implementation keep using. If an > optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P > (cpu_features, SSE4_2) check. */ > - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > return OPTIMIZE (generic); > diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c > new file mode 100644 > index 0000000000..99e3c59e00 > --- /dev/null > +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c > @@ -0,0 +1,18 @@ > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include "../strcspn.c" > diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c > index 423de2e2b2..64e90a26bb 100644 > --- a/sysdeps/x86_64/multiarch/strcspn-generic.c > +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c > @@ -16,13 +16,16 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > - > +/* We always need to build this implementation as strcspn-sse4 needs > + to be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRCSPN __strcspn_generic > > # undef libc_hidden_builtin_def > # define libc_hidden_builtin_def(STRCSPN) > -#endif > > -#include <string/strcspn.c> > +# include <string/strcspn.c> <string/strcspn.c> will always be included. The only change should be #if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 which controls __strcspn_generic vs strcspn. > + > +#endif > diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c > index 59f64f9fe8..becdaf05f3 100644 > --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c > +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c > @@ -52,9 +52,11 @@ > when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset > X for case 1. */ > > +#ifndef STRCSPN > +# define STRCSPN __strcspn_sse42 > +#endif > #ifndef STRCSPN_GENERIC > # define STRCSPN_GENERIC __strcspn_generic > -# define STRCSPN_SSE42 __strcspn_sse42 > #endif > > #ifdef USE_AS_STRPBRK > @@ -78,7 +80,7 @@ char * > size_t > #endif > __attribute__ ((section (".text.sse4.2"))) > -STRCSPN_SSE42 (const char *s, const char *a) > +STRCSPN (const char *s, const char *a) > { > if (*a == 0) > RETURN (NULL, strlen (s)); > diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c > index d31acfe495..7f3ea86e91 100644 > --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c > +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c > @@ -16,13 +16,16 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > > +/* We always need to build this implementation as strpbrk-sse4 needs > + to be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 Only this change is needed. > # include <sysdep.h> > # define STRPBRK __strpbrk_generic > > # undef libc_hidden_builtin_def > # define libc_hidden_builtin_def(STRPBRK) > -#endif > > -#include <string/strpbrk.c> > +# include <string/strpbrk.c> > +#endif > diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > index bf74d660d5..0adb577955 100644 > --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c > +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > @@ -16,7 +16,11 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > +#ifndef STRPBRK > +# define STRPBRK __strpbrk_sse42 > +#endif > + > #define USE_AS_STRPBRK > #define STRCSPN_GENERIC __strpbrk_generic > -#define STRCSPN_SSE42 __strpbrk_sse42 > +#define STRCSPN STRPBRK > #include "strcspn-sse4.c" > diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c > index 6b50c36432..f5632ad3b2 100644 > --- a/sysdeps/x86_64/multiarch/strspn-generic.c > +++ b/sysdeps/x86_64/multiarch/strspn-generic.c > @@ -16,13 +16,15 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > - > +/* We always need to build this implementation as strspn-sse4 needs to > + be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 Only this change is needed. > # include <sysdep.h> > # define STRSPN __strspn_generic > > # undef libc_hidden_builtin_def > # define libc_hidden_builtin_def(STRSPN) > -#endif > > #include <string/strspn.c> > +#endif > diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c > index d044916688..6263669b54 100644 > --- a/sysdeps/x86_64/multiarch/strspn-sse4.c > +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c > @@ -51,12 +51,16 @@ > > We exit from the loop for case 1. */ > > + > extern size_t __strspn_generic (const char *, const char *) attribute_hidden; > > +#ifndef STRSPN > +# define STRSPN __strspn_sse42 > +#endif > > size_t > __attribute__ ((section (".text.sse4.2"))) > -__strspn_sse42 (const char *s, const char *a) > +STRSPN (const char *s, const char *a) > { > if (*a == 0) > return 0; > diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c > new file mode 100644 > index 0000000000..082324e793 > --- /dev/null > +++ b/sysdeps/x86_64/strcspn-generic.c > @@ -0,0 +1,22 @@ Need a comment for its purpose. > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 Do we need to check USE_MULTIARCH here? multiartch build will use multiarch/strcspn-generic.c. > +# include "multiarch/strcspn-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c > new file mode 100644 > index 0000000000..cd54eed869 > --- /dev/null > +++ b/sysdeps/x86_64/strcspn.c > @@ -0,0 +1,27 @@ > +/* strcspn hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strcspn.c> > +#else > +#define STRCSPN strcspn > +#include "multiarch/strcspn-sse4.c" > +libc_hidden_builtin_def (strcspn) > +#endif > diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c > new file mode 100644 > index 0000000000..6bdbb2d164 > --- /dev/null > +++ b/sysdeps/x86_64/strpbrk-generic.c > @@ -0,0 +1,22 @@ Need a comment for its purpose. > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strpbrk-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c > new file mode 100644 > index 0000000000..e7ea1b334a > --- /dev/null > +++ b/sysdeps/x86_64/strpbrk.c > @@ -0,0 +1,27 @@ > +/* strpbrk hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strpbrk.c> > +#else > +#define STRPBRK strpbrk > +#include "multiarch/strpbrk-sse4.c" > +libc_hidden_builtin_def (strpbrk) > +#endif > diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c > new file mode 100644 > index 0000000000..0730972bbd > --- /dev/null > +++ b/sysdeps/x86_64/strspn-generic.c > @@ -0,0 +1,23 @@ Need a comment for its purpose. > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > + > +#include <isa-level.h> > + > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strspn-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c > new file mode 100644 > index 0000000000..7b9ede26d9 > --- /dev/null > +++ b/sysdeps/x86_64/strspn.c > @@ -0,0 +1,27 @@ > +/* strspn hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strspn.c> > +#else > +#define STRSPN strspn > +#include "multiarch/strspn-sse4.c" > +libc_hidden_builtin_def (strspn) > +#endif > diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c > new file mode 100644 > index 0000000000..ab3c506378 > --- /dev/null > +++ b/sysdeps/x86_64/varshift.c > @@ -0,0 +1,23 @@ > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > + > +#include <isa-level.h> > + > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/varshift.c" > +#endif > -- > 2.34.1 > -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-29 19:20 ` H.J. Lu @ 2022-06-29 22:07 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-29 22:07 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 12:21 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Tue, Jun 28, 2022 at 8:27 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > The changes for these functions are different than the others because > > the best implementation (sse4_2) requires the generic > > implementation as a fallback to be built as well. > > > > Changes are: > > > > 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically > > select the best implementation based on the configured ISA build > > level. > > > > 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the > > sysdeps/x86_64 directory so that the the sse4 implementation will > > have all of its dependencies for the non-multiarch / rtld build > > when ISA level >= > Something is missing. > > 3. Add new multiarch/rtld-strcspn.c that just include the > > non-multiarch strcspn.c which will in turn select the best > > implementation based on the compiled ISA level. > > > > 4. Refactor the ifunc selector and ifunc implementation list to use > > the ISA level aware wrapper macros that allow functions below the > > compiled ISA level (with a guranteed replacement) to be skipped. > > > > Tested with and without multiarch on x86_64 for ISA levels: > > {generic, x86-64-v2, x86-64-v3, x86-64-v4} > > > > And m32 with and without multiarch. > > --- > > sysdeps/x86_64/Makefile | 9 +++++++- > > sysdeps/x86_64/multiarch/ifunc-impl-list.c | 9 +++++++- > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- > > sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ > > sysdeps/x86_64/multiarch/strcspn-generic.c | 11 +++++---- > > sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- > > sysdeps/x86_64/multiarch/strpbrk-generic.c | 9 +++++--- > > sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- > > sysdeps/x86_64/multiarch/strspn-generic.c | 8 ++++--- > > sysdeps/x86_64/multiarch/strspn-sse4.c | 6 ++++- > > sysdeps/x86_64/strcspn-generic.c | 22 ++++++++++++++++++ > > sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ > > sysdeps/x86_64/strpbrk-generic.c | 22 ++++++++++++++++++ > > sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ > > sysdeps/x86_64/strspn-generic.c | 23 ++++++++++++++++++ > > sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ > > sysdeps/x86_64/varshift.c | 23 ++++++++++++++++++ > > 17 files changed, 238 insertions(+), 17 deletions(-) > > create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c > > create mode 100644 sysdeps/x86_64/strcspn-generic.c > > create mode 100644 sysdeps/x86_64/strcspn.c > > create mode 100644 sysdeps/x86_64/strpbrk-generic.c > > create mode 100644 sysdeps/x86_64/strpbrk.c > > create mode 100644 sysdeps/x86_64/strspn-generic.c > > create mode 100644 sysdeps/x86_64/strspn.c > > create mode 100644 sysdeps/x86_64/varshift.c > > > > diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile > > index 79365aff2a..c130d56342 100644 > > --- a/sysdeps/x86_64/Makefile > > +++ b/sysdeps/x86_64/Makefile > > @@ -14,7 +14,14 @@ sysdep_noprof += _mcount > > endif > > > > ifeq ($(subdir),string) > > -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii > > +sysdep_routines += \ > > + strcasecmp_l-nonascii \ > > + strcspn-generic \ > > + strncase_l-nonascii \ > > + strpbrk-generic \ > > + strspn-generic \ > > + varshift \ > > +#sysdep_routines > > Add a space after #. Fixed in V2. > > > gen-as-const-headers += locale-defines.sym > > tests += \ > > tst-rsi-strlen > > diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > index 772b4ace6c..251b0022d9 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > @@ -540,6 +540,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > /* Support sysdeps/x86_64/multiarch/strcspn.c. */ > > IFUNC_IMPL (i, name, strcspn, > > + /* All implementations of strcspn are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), > > __strcspn_sse42) > > IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) > > @@ -616,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ > > IFUNC_IMPL (i, name, strpbrk, > > + /* All implementations of strpbrk are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), > > __strpbrk_sse42) > > IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) > > @@ -623,13 +627,16 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > /* Support sysdeps/x86_64/multiarch/strspn.c. */ > > IFUNC_IMPL (i, name, strspn, > > + /* All implementations of strspn are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), > > __strspn_sse42) > > IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) > > > > /* Support sysdeps/x86_64/multiarch/strstr.c. */ > > IFUNC_IMPL (i, name, strstr, > > - /* All implementations of strstr are built at all ISA levels. */ > > + /* All implementations of strstr are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strstr, > > (CPU_FEATURE_USABLE (AVX512VL) > > && CPU_FEATURE_USABLE (AVX512BW) > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > index 204c4b5406..d642f54fac 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) > > there is no other optimized implementation keep using. If an > > optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P > > (cpu_features, SSE4_2) check. */ > > - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > return OPTIMIZE (sse42); > > > > return OPTIMIZE (generic); > > diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c > > new file mode 100644 > > index 0000000000..99e3c59e00 > > --- /dev/null > > +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c > > @@ -0,0 +1,18 @@ > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include "../strcspn.c" > > diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c > > index 423de2e2b2..64e90a26bb 100644 > > --- a/sysdeps/x86_64/multiarch/strcspn-generic.c > > +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c > > @@ -16,13 +16,16 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > -#if IS_IN (libc) > > - > > +/* We always need to build this implementation as strcspn-sse4 needs > > + to be able to fallback to it. */ > > +#include <isa-level.h> > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > # include <sysdep.h> > > # define STRCSPN __strcspn_generic > > > > # undef libc_hidden_builtin_def > > # define libc_hidden_builtin_def(STRCSPN) > > -#endif > > > > -#include <string/strcspn.c> > > +# include <string/strcspn.c> > > <string/strcspn.c> will always be included. The only change > should be > > #if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > which controls __strcspn_generic vs strcspn. > Fixed in V2. > > + > > +#endif > > diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c > > index 59f64f9fe8..becdaf05f3 100644 > > --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c > > +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c > > @@ -52,9 +52,11 @@ > > when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset > > X for case 1. */ > > > > +#ifndef STRCSPN > > +# define STRCSPN __strcspn_sse42 > > +#endif > > #ifndef STRCSPN_GENERIC > > # define STRCSPN_GENERIC __strcspn_generic > > -# define STRCSPN_SSE42 __strcspn_sse42 > > #endif > > > > #ifdef USE_AS_STRPBRK > > @@ -78,7 +80,7 @@ char * > > size_t > > #endif > > __attribute__ ((section (".text.sse4.2"))) > > -STRCSPN_SSE42 (const char *s, const char *a) > > +STRCSPN (const char *s, const char *a) > > { > > if (*a == 0) > > RETURN (NULL, strlen (s)); > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c > > index d31acfe495..7f3ea86e91 100644 > > --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c > > +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c > > @@ -16,13 +16,16 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > -#if IS_IN (libc) > > > > +/* We always need to build this implementation as strpbrk-sse4 needs > > + to be able to fallback to it. */ > > +#include <isa-level.h> > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > Only this change is needed. > Fixed in V2. > > # include <sysdep.h> > > # define STRPBRK __strpbrk_generic > > > > # undef libc_hidden_builtin_def > > # define libc_hidden_builtin_def(STRPBRK) > > -#endif > > > > -#include <string/strpbrk.c> > > +# include <string/strpbrk.c> > > +#endif > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > index bf74d660d5..0adb577955 100644 > > --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > @@ -16,7 +16,11 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > +#ifndef STRPBRK > > +# define STRPBRK __strpbrk_sse42 > > +#endif > > + > > #define USE_AS_STRPBRK > > #define STRCSPN_GENERIC __strpbrk_generic > > -#define STRCSPN_SSE42 __strpbrk_sse42 > > +#define STRCSPN STRPBRK > > #include "strcspn-sse4.c" > > diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c > > index 6b50c36432..f5632ad3b2 100644 > > --- a/sysdeps/x86_64/multiarch/strspn-generic.c > > +++ b/sysdeps/x86_64/multiarch/strspn-generic.c > > @@ -16,13 +16,15 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > -#if IS_IN (libc) > > - > > +/* We always need to build this implementation as strspn-sse4 needs to > > + be able to fallback to it. */ > > +#include <isa-level.h> > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > Only this change is needed. > > > # include <sysdep.h> > > # define STRSPN __strspn_generic > > > > # undef libc_hidden_builtin_def > > # define libc_hidden_builtin_def(STRSPN) > > -#endif > > > > #include <string/strspn.c> > > +#endif > > diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c > > index d044916688..6263669b54 100644 > > --- a/sysdeps/x86_64/multiarch/strspn-sse4.c > > +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c > > @@ -51,12 +51,16 @@ > > > > We exit from the loop for case 1. */ > > > > + > > extern size_t __strspn_generic (const char *, const char *) attribute_hidden; > > > > +#ifndef STRSPN > > +# define STRSPN __strspn_sse42 > > +#endif > > > > size_t > > __attribute__ ((section (".text.sse4.2"))) > > -__strspn_sse42 (const char *s, const char *a) > > +STRSPN (const char *s, const char *a) > > { > > if (*a == 0) > > return 0; > > diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c > > new file mode 100644 > > index 0000000000..082324e793 > > --- /dev/null > > +++ b/sysdeps/x86_64/strcspn-generic.c > > @@ -0,0 +1,22 @@ > > Need a comment for its purpose. > Done in V2. > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > > Do we need to check USE_MULTIARCH here? multiartch build will > use multiarch/strcspn-generic.c. > No. Removed in V2. (All builds succeed) > > +# include "multiarch/strcspn-generic.c" > > +#endif > > diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c > > new file mode 100644 > > index 0000000000..cd54eed869 > > --- /dev/null > > +++ b/sysdeps/x86_64/strcspn.c > > @@ -0,0 +1,27 @@ > > +/* strcspn hook for non-multiarch and RTLD build. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > +#include <string/strcspn.c> > > +#else > > +#define STRCSPN strcspn > > +#include "multiarch/strcspn-sse4.c" > > +libc_hidden_builtin_def (strcspn) > > +#endif > > diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c > > new file mode 100644 > > index 0000000000..6bdbb2d164 > > --- /dev/null > > +++ b/sysdeps/x86_64/strpbrk-generic.c > > @@ -0,0 +1,22 @@ > > Need a comment for its purpose. > Done in V2. > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/strpbrk-generic.c" > > +#endif > > diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c > > new file mode 100644 > > index 0000000000..e7ea1b334a > > --- /dev/null > > +++ b/sysdeps/x86_64/strpbrk.c > > @@ -0,0 +1,27 @@ > > +/* strpbrk hook for non-multiarch and RTLD build. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > +#include <string/strpbrk.c> > > +#else > > +#define STRPBRK strpbrk > > +#include "multiarch/strpbrk-sse4.c" > > +libc_hidden_builtin_def (strpbrk) > > +#endif > > diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c > > new file mode 100644 > > index 0000000000..0730972bbd > > --- /dev/null > > +++ b/sysdeps/x86_64/strspn-generic.c > > @@ -0,0 +1,23 @@ > > Need a comment for its purpose. Done in V2. > > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > + > > +#include <isa-level.h> > > + > > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/strspn-generic.c" > > +#endif > > diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c > > new file mode 100644 > > index 0000000000..7b9ede26d9 > > --- /dev/null > > +++ b/sysdeps/x86_64/strspn.c > > @@ -0,0 +1,27 @@ > > +/* strspn hook for non-multiarch and RTLD build. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > +#include <string/strspn.c> > > +#else > > +#define STRSPN strspn > > +#include "multiarch/strspn-sse4.c" > > +libc_hidden_builtin_def (strspn) > > +#endif > > diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c > > new file mode 100644 > > index 0000000000..ab3c506378 > > --- /dev/null > > +++ b/sysdeps/x86_64/varshift.c > > @@ -0,0 +1,23 @@ > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > + > > +#include <isa-level.h> > > + > > +#if !defined USE_MULTIARCH && MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/varshift.c" > > +#endif > > -- > > 2.34.1 > > > > > -- > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein 2022-06-28 15:27 ` [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-06-29 18:53 ` H.J. Lu 2022-06-29 19:19 ` Noah Goldstein 2022-06-29 22:05 ` [PATCH v2 " Noah Goldstein ` (5 subsequent siblings) 7 siblings, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-06-29 18:53 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Tue, Jun 28, 2022 at 8:27 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > Just for clarities sake and so that if a future implementation is > added we remember to add the check. > --- > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index ee36525bcf..204c4b5406 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -27,7 +27,11 @@ IFUNC_SELECTOR (void) > { > const struct cpu_features* cpu_features = __get_cpu_features (); > > - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > + there is no other optimized implementation keep using. If an > + optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P > + (cpu_features, SSE4_2) check. */ Did you mean Slow_SSE4_2? > + if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > return OPTIMIZE (generic); > -- > 2.34.1 > -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-29 18:53 ` [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu @ 2022-06-29 19:19 ` Noah Goldstein 2022-06-29 22:06 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-06-29 19:19 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 11:54 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Tue, Jun 28, 2022 at 8:27 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > Just for clarities sake and so that if a future implementation is > > added we remember to add the check. > > --- > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++- > > 1 file changed, 5 insertions(+), 1 deletion(-) > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > index ee36525bcf..204c4b5406 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > @@ -27,7 +27,11 @@ IFUNC_SELECTOR (void) > > { > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > + there is no other optimized implementation keep using. If an > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P > > + (cpu_features, SSE4_2) check. */ > > Did you mean Slow_SSE4_2? Yes. Will fix for V2. > > > + if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > return OPTIMIZE (sse42); > > > > return OPTIMIZE (generic); > > -- > > 2.34.1 > > > > > -- > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-29 19:19 ` Noah Goldstein @ 2022-06-29 22:06 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-29 22:06 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 12:19 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > On Wed, Jun 29, 2022 at 11:54 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > On Tue, Jun 28, 2022 at 8:27 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > Just for clarities sake and so that if a future implementation is > > > added we remember to add the check. > > > --- > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++- > > > 1 file changed, 5 insertions(+), 1 deletion(-) > > > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > index ee36525bcf..204c4b5406 100644 > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > @@ -27,7 +27,11 @@ IFUNC_SELECTOR (void) > > > { > > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > > > - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > > + there is no other optimized implementation keep using. If an > > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_USABLE_P > > > + (cpu_features, SSE4_2) check. */ > > > > Did you mean Slow_SSE4_2? > > Yes. Will fix for V2. Fixed in V2. > > > > > + if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > return OPTIMIZE (sse42); > > > > > > return OPTIMIZE (generic); > > > -- > > > 2.34.1 > > > > > > > > > -- > > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein 2022-06-28 15:27 ` [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein 2022-06-29 18:53 ` [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu @ 2022-06-29 22:05 ` Noah Goldstein 2022-06-29 22:05 ` [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein 2022-06-29 22:12 ` [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu 2022-06-30 3:08 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein ` (4 subsequent siblings) 7 siblings, 2 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-29 22:05 UTC (permalink / raw) To: libc-alpha Just for clarities sake and so that if a future implementation is added we remember to add the check. --- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index ee36525bcf..973041d23b 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -27,7 +27,11 @@ IFUNC_SELECTOR (void) { const struct cpu_features* cpu_features = __get_cpu_features (); - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + /* This function uses slow sse4.2 instructions (pcmpstri) but since + there is no other optimized implementation keep using. If an + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P + (cpu_features, Slow_SSE4_2) check. */ + if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-29 22:05 ` [PATCH v2 " Noah Goldstein @ 2022-06-29 22:05 ` Noah Goldstein 2022-06-29 22:15 ` H.J. Lu 2022-06-29 22:12 ` [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu 1 sibling, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-06-29 22:05 UTC (permalink / raw) To: libc-alpha The changes for these functions are different than the others because the best implementation (sse4_2) requires the generic implementation as a fallback to be built as well. Changes are: 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically select the best implementation based on the configured ISA build level. 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the sysdeps/x86_64 directory so that the the sse4 implementation will have all of its dependencies for the non-multiarch / rtld build when ISA level >= 2. 3. Add new multiarch/rtld-strcspn.c that just include the non-multiarch strcspn.c which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch. --- sysdeps/x86_64/Makefile | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ 17 files changed, 242 insertions(+), 11 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c create mode 100644 sysdeps/x86_64/strcspn-generic.c create mode 100644 sysdeps/x86_64/strcspn.c create mode 100644 sysdeps/x86_64/strpbrk-generic.c create mode 100644 sysdeps/x86_64/strpbrk.c create mode 100644 sysdeps/x86_64/strspn-generic.c create mode 100644 sysdeps/x86_64/strspn.c create mode 100644 sysdeps/x86_64/varshift.c diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile index 79365aff2a..e597a4855f 100644 --- a/sysdeps/x86_64/Makefile +++ b/sysdeps/x86_64/Makefile @@ -14,7 +14,14 @@ sysdep_noprof += _mcount endif ifeq ($(subdir),string) -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii +sysdep_routines += \ + strcasecmp_l-nonascii \ + strcspn-generic \ + strncase_l-nonascii \ + strpbrk-generic \ + strspn-generic \ + varshift \ +# sysdep_routines gen-as-const-headers += locale-defines.sym tests += \ tst-rsi-strlen diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c index 0d28319905..6c03f53a8b 100644 --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c @@ -531,6 +531,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strcspn.c. */ IFUNC_IMPL (i, name, strcspn, + /* All implementations of strcspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), __strcspn_sse42) IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) @@ -607,6 +609,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ IFUNC_IMPL (i, name, strpbrk, + /* All implementations of strpbrk are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), __strpbrk_sse42) IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) @@ -614,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strspn.c. */ IFUNC_IMPL (i, name, strspn, + /* All implementations of strspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), __strspn_sse42) IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index 973041d23b..f8943c8210 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) there is no other optimized implementation keep using. If an optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, Slow_SSE4_2) check. */ - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c new file mode 100644 index 0000000000..99e3c59e00 --- /dev/null +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c @@ -0,0 +1,18 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include "../strcspn.c" diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c index 423de2e2b2..86cda7b037 100644 --- a/sysdeps/x86_64/multiarch/strcspn-generic.c +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c @@ -16,8 +16,10 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strcspn-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRCSPN __strcspn_generic diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c index 59f64f9fe8..becdaf05f3 100644 --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c @@ -52,9 +52,11 @@ when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset X for case 1. */ +#ifndef STRCSPN +# define STRCSPN __strcspn_sse42 +#endif #ifndef STRCSPN_GENERIC # define STRCSPN_GENERIC __strcspn_generic -# define STRCSPN_SSE42 __strcspn_sse42 #endif #ifdef USE_AS_STRPBRK @@ -78,7 +80,7 @@ char * size_t #endif __attribute__ ((section (".text.sse4.2"))) -STRCSPN_SSE42 (const char *s, const char *a) +STRCSPN (const char *s, const char *a) { if (*a == 0) RETURN (NULL, strlen (s)); diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c index d31acfe495..058f635774 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c @@ -16,8 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +/* We always need to build this implementation as strpbrk-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRPBRK __strpbrk_generic diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c index bf74d660d5..0adb577955 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c @@ -16,7 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ +#ifndef STRPBRK +# define STRPBRK __strpbrk_sse42 +#endif + #define USE_AS_STRPBRK #define STRCSPN_GENERIC __strpbrk_generic -#define STRCSPN_SSE42 __strpbrk_sse42 +#define STRCSPN STRPBRK #include "strcspn-sse4.c" diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c index 6b50c36432..0f485ab4da 100644 --- a/sysdeps/x86_64/multiarch/strspn-generic.c +++ b/sysdeps/x86_64/multiarch/strspn-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strspn-sse4 needs to + be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRSPN __strspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRSPN) + #endif #include <string/strspn.c> diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c index d044916688..4d50507a0d 100644 --- a/sysdeps/x86_64/multiarch/strspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c @@ -53,10 +53,13 @@ extern size_t __strspn_generic (const char *, const char *) attribute_hidden; +#ifndef STRSPN +# define STRSPN __strspn_sse42 +#endif size_t __attribute__ ((section (".text.sse4.2"))) -__strspn_sse42 (const char *s, const char *a) +STRSPN (const char *s, const char *a) { if (*a == 0) return 0; diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c new file mode 100644 index 0000000000..51fd70b63d --- /dev/null +++ b/sysdeps/x86_64/strcspn-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strcspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strcspn-sse4 has a dependency on + strcspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strcspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c new file mode 100644 index 0000000000..cd54eed869 --- /dev/null +++ b/sysdeps/x86_64/strcspn.c @@ -0,0 +1,27 @@ +/* strcspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strcspn.c> +#else +#define STRCSPN strcspn +#include "multiarch/strcspn-sse4.c" +libc_hidden_builtin_def (strcspn) +#endif diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c new file mode 100644 index 0000000000..a7d8b11216 --- /dev/null +++ b/sysdeps/x86_64/strpbrk-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strpbrk-generic for non-multiarch build. Needed for + the ISA level >= 2 because strpbrk-sse4 has a dependency on + strpbrk-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strpbrk-generic.c" +#endif diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c new file mode 100644 index 0000000000..e7ea1b334a --- /dev/null +++ b/sysdeps/x86_64/strpbrk.c @@ -0,0 +1,27 @@ +/* strpbrk hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strpbrk.c> +#else +#define STRPBRK strpbrk +#include "multiarch/strpbrk-sse4.c" +libc_hidden_builtin_def (strpbrk) +#endif diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c new file mode 100644 index 0000000000..1a2a576e49 --- /dev/null +++ b/sysdeps/x86_64/strspn-generic.c @@ -0,0 +1,26 @@ +/* Hook for build strspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strspn-sse4 has a dependency on + strspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c new file mode 100644 index 0000000000..7b9ede26d9 --- /dev/null +++ b/sysdeps/x86_64/strspn.c @@ -0,0 +1,27 @@ +/* strspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strspn.c> +#else +#define STRSPN strspn +#include "multiarch/strspn-sse4.c" +libc_hidden_builtin_def (strspn) +#endif diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c new file mode 100644 index 0000000000..8f9eb13547 --- /dev/null +++ b/sysdeps/x86_64/varshift.c @@ -0,0 +1,26 @@ +/* Hook for build varshift for non-multiarch build. Needed for the + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 + all have a dependency on varshift.c. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/varshift.c" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-29 22:05 ` [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-06-29 22:15 ` H.J. Lu 2022-06-30 3:11 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-06-29 22:15 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 3:05 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > The changes for these functions are different than the others because > the best implementation (sse4_2) requires the generic > implementation as a fallback to be built as well. > > Changes are: > > 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically > select the best implementation based on the configured ISA build > level. > > 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the > sysdeps/x86_64 directory so that the the sse4 implementation will > have all of its dependencies for the non-multiarch / rtld build > when ISA level >= 2. > > 3. Add new multiarch/rtld-strcspn.c that just include the > non-multiarch strcspn.c which will in turn select the best > implementation based on the compiled ISA level. > > 4. Refactor the ifunc selector and ifunc implementation list to use > the ISA level aware wrapper macros that allow functions below the > compiled ISA level (with a guranteed replacement) to be skipped. > > Tested with and without multiarch on x86_64 for ISA levels: > {generic, x86-64-v2, x86-64-v3, x86-64-v4} > > And m32 with and without multiarch. > --- > sysdeps/x86_64/Makefile | 9 +++++++- > sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- > sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ > sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- > sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- > sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- > sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- > sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- > sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- > sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ > sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ > sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ > sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ > 17 files changed, 242 insertions(+), 11 deletions(-) > create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c > create mode 100644 sysdeps/x86_64/strcspn-generic.c > create mode 100644 sysdeps/x86_64/strcspn.c > create mode 100644 sysdeps/x86_64/strpbrk-generic.c > create mode 100644 sysdeps/x86_64/strpbrk.c > create mode 100644 sysdeps/x86_64/strspn-generic.c > create mode 100644 sysdeps/x86_64/strspn.c > create mode 100644 sysdeps/x86_64/varshift.c > > diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile > index 79365aff2a..e597a4855f 100644 > --- a/sysdeps/x86_64/Makefile > +++ b/sysdeps/x86_64/Makefile > @@ -14,7 +14,14 @@ sysdep_noprof += _mcount > endif > > ifeq ($(subdir),string) > -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii > +sysdep_routines += \ > + strcasecmp_l-nonascii \ > + strcspn-generic \ > + strncase_l-nonascii \ > + strpbrk-generic \ > + strspn-generic \ > + varshift \ > +# sysdep_routines > gen-as-const-headers += locale-defines.sym > tests += \ > tst-rsi-strlen > diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > index 0d28319905..6c03f53a8b 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c > +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > @@ -531,6 +531,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strcspn.c. */ > IFUNC_IMPL (i, name, strcspn, > + /* All implementations of strcspn are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), > __strcspn_sse42) > IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) > @@ -607,6 +609,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ > IFUNC_IMPL (i, name, strpbrk, > + /* All implementations of strpbrk are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), > __strpbrk_sse42) > IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) > @@ -614,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strspn.c. */ > IFUNC_IMPL (i, name, strspn, > + /* All implementations of strspn are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), > __strspn_sse42) > IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index 973041d23b..f8943c8210 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) > there is no other optimized implementation keep using. If an > optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > (cpu_features, Slow_SSE4_2) check. */ > - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > return OPTIMIZE (generic); > diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c > new file mode 100644 > index 0000000000..99e3c59e00 > --- /dev/null > +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c > @@ -0,0 +1,18 @@ > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include "../strcspn.c" > diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c > index 423de2e2b2..86cda7b037 100644 > --- a/sysdeps/x86_64/multiarch/strcspn-generic.c > +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c > @@ -16,8 +16,10 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > - > +/* We always need to build this implementation as strcspn-sse4 needs > + to be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRCSPN __strcspn_generic > > diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c > index 59f64f9fe8..becdaf05f3 100644 > --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c > +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c > @@ -52,9 +52,11 @@ > when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset > X for case 1. */ > > +#ifndef STRCSPN > +# define STRCSPN __strcspn_sse42 > +#endif > #ifndef STRCSPN_GENERIC > # define STRCSPN_GENERIC __strcspn_generic > -# define STRCSPN_SSE42 __strcspn_sse42 > #endif > > #ifdef USE_AS_STRPBRK > @@ -78,7 +80,7 @@ char * > size_t > #endif > __attribute__ ((section (".text.sse4.2"))) > -STRCSPN_SSE42 (const char *s, const char *a) > +STRCSPN (const char *s, const char *a) > { > if (*a == 0) > RETURN (NULL, strlen (s)); > diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c > index d31acfe495..058f635774 100644 > --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c > +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c > @@ -16,8 +16,11 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > > +/* We always need to build this implementation as strpbrk-sse4 needs > + to be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRPBRK __strpbrk_generic > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > index bf74d660d5..0adb577955 100644 > --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c > +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > @@ -16,7 +16,11 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > +#ifndef STRPBRK > +# define STRPBRK __strpbrk_sse42 > +#endif > + > #define USE_AS_STRPBRK > #define STRCSPN_GENERIC __strpbrk_generic > -#define STRCSPN_SSE42 __strpbrk_sse42 > +#define STRCSPN STRPBRK > #include "strcspn-sse4.c" > diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c > index 6b50c36432..0f485ab4da 100644 > --- a/sysdeps/x86_64/multiarch/strspn-generic.c > +++ b/sysdeps/x86_64/multiarch/strspn-generic.c > @@ -16,13 +16,16 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > - > +/* We always need to build this implementation as strspn-sse4 needs to > + be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRSPN __strspn_generic > > # undef libc_hidden_builtin_def > # define libc_hidden_builtin_def(STRSPN) > + > #endif > > #include <string/strspn.c> > diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c > index d044916688..4d50507a0d 100644 > --- a/sysdeps/x86_64/multiarch/strspn-sse4.c > +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c > @@ -53,10 +53,13 @@ > > extern size_t __strspn_generic (const char *, const char *) attribute_hidden; > > +#ifndef STRSPN > +# define STRSPN __strspn_sse42 > +#endif > > size_t > __attribute__ ((section (".text.sse4.2"))) > -__strspn_sse42 (const char *s, const char *a) > +STRSPN (const char *s, const char *a) > { > if (*a == 0) > return 0; > diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c > new file mode 100644 > index 0000000000..51fd70b63d > --- /dev/null > +++ b/sysdeps/x86_64/strcspn-generic.c > @@ -0,0 +1,25 @@ > +/* Hook for build strcspn-generic for non-multiarch build. Needed for > + the ISA level >= 2 because strcspn-sse4 has a dependency on > + strcspn-generic. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strcspn-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c > new file mode 100644 > index 0000000000..cd54eed869 > --- /dev/null > +++ b/sysdeps/x86_64/strcspn.c > @@ -0,0 +1,27 @@ > +/* strcspn hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strcspn.c> > +#else > +#define STRCSPN strcspn > +#include "multiarch/strcspn-sse4.c" > +libc_hidden_builtin_def (strcspn) > +#endif > diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c > new file mode 100644 > index 0000000000..a7d8b11216 > --- /dev/null > +++ b/sysdeps/x86_64/strpbrk-generic.c > @@ -0,0 +1,25 @@ > +/* Hook for build strpbrk-generic for non-multiarch build. Needed for > + the ISA level >= 2 because strpbrk-sse4 has a dependency on > + strpbrk-generic. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strpbrk-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c > new file mode 100644 > index 0000000000..e7ea1b334a > --- /dev/null > +++ b/sysdeps/x86_64/strpbrk.c > @@ -0,0 +1,27 @@ > +/* strpbrk hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strpbrk.c> > +#else > +#define STRPBRK strpbrk > +#include "multiarch/strpbrk-sse4.c" > +libc_hidden_builtin_def (strpbrk) > +#endif > diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c > new file mode 100644 > index 0000000000..1a2a576e49 > --- /dev/null > +++ b/sysdeps/x86_64/strspn-generic.c > @@ -0,0 +1,26 @@ > +/* Hook for build strspn-generic for non-multiarch build. Needed for > + the ISA level >= 2 because strspn-sse4 has a dependency on > + strspn-generic. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strspn-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c > new file mode 100644 > index 0000000000..7b9ede26d9 > --- /dev/null > +++ b/sysdeps/x86_64/strspn.c > @@ -0,0 +1,27 @@ > +/* strspn hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strspn.c> > +#else > +#define STRSPN strspn > +#include "multiarch/strspn-sse4.c" > +libc_hidden_builtin_def (strspn) > +#endif > diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c > new file mode 100644 > index 0000000000..8f9eb13547 > --- /dev/null > +++ b/sysdeps/x86_64/varshift.c > @@ -0,0 +1,26 @@ > +/* Hook for build varshift for non-multiarch build. Needed for the > + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 > + all have a dependency on varshift.c. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/varshift.c" > +#endif > -- > 2.34.1 > LGTM. Thanks. -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-29 22:15 ` H.J. Lu @ 2022-06-30 3:11 ` Noah Goldstein 2022-06-30 3:27 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:11 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 3:16 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Wed, Jun 29, 2022 at 3:05 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > The changes for these functions are different than the others because > > the best implementation (sse4_2) requires the generic > > implementation as a fallback to be built as well. > > > > Changes are: > > > > 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically > > select the best implementation based on the configured ISA build > > level. > > > > 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the > > sysdeps/x86_64 directory so that the the sse4 implementation will > > have all of its dependencies for the non-multiarch / rtld build > > when ISA level >= 2. > > > > 3. Add new multiarch/rtld-strcspn.c that just include the > > non-multiarch strcspn.c which will in turn select the best > > implementation based on the compiled ISA level. > > > > 4. Refactor the ifunc selector and ifunc implementation list to use > > the ISA level aware wrapper macros that allow functions below the > > compiled ISA level (with a guranteed replacement) to be skipped. > > > > Tested with and without multiarch on x86_64 for ISA levels: > > {generic, x86-64-v2, x86-64-v3, x86-64-v4} > > > > And m32 with and without multiarch. > > --- > > sysdeps/x86_64/Makefile | 9 +++++++- > > sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- > > sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ > > sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- > > sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- > > sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- > > sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- > > sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- > > sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- > > sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ > > sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ > > sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ > > sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ > > sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ > > sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ > > sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ > > 17 files changed, 242 insertions(+), 11 deletions(-) > > create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c > > create mode 100644 sysdeps/x86_64/strcspn-generic.c > > create mode 100644 sysdeps/x86_64/strcspn.c > > create mode 100644 sysdeps/x86_64/strpbrk-generic.c > > create mode 100644 sysdeps/x86_64/strpbrk.c > > create mode 100644 sysdeps/x86_64/strspn-generic.c > > create mode 100644 sysdeps/x86_64/strspn.c > > create mode 100644 sysdeps/x86_64/varshift.c > > > > diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile > > index 79365aff2a..e597a4855f 100644 > > --- a/sysdeps/x86_64/Makefile > > +++ b/sysdeps/x86_64/Makefile > > @@ -14,7 +14,14 @@ sysdep_noprof += _mcount > > endif > > > > ifeq ($(subdir),string) > > -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii > > +sysdep_routines += \ > > + strcasecmp_l-nonascii \ > > + strcspn-generic \ > > + strncase_l-nonascii \ > > + strpbrk-generic \ > > + strspn-generic \ > > + varshift \ > > +# sysdep_routines > > gen-as-const-headers += locale-defines.sym > > tests += \ > > tst-rsi-strlen > > diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > index 0d28319905..6c03f53a8b 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > @@ -531,6 +531,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > /* Support sysdeps/x86_64/multiarch/strcspn.c. */ > > IFUNC_IMPL (i, name, strcspn, > > + /* All implementations of strcspn are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), > > __strcspn_sse42) > > IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) > > @@ -607,6 +609,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ > > IFUNC_IMPL (i, name, strpbrk, > > + /* All implementations of strpbrk are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), > > __strpbrk_sse42) > > IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) > > @@ -614,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > /* Support sysdeps/x86_64/multiarch/strspn.c. */ > > IFUNC_IMPL (i, name, strspn, > > + /* All implementations of strspn are built at all ISA > > + levels. */ > > IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), > > __strspn_sse42) > > IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > index 973041d23b..f8943c8210 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) > > there is no other optimized implementation keep using. If an > > optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > (cpu_features, Slow_SSE4_2) check. */ > > - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > return OPTIMIZE (sse42); > > > > return OPTIMIZE (generic); > > diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c > > new file mode 100644 > > index 0000000000..99e3c59e00 > > --- /dev/null > > +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c > > @@ -0,0 +1,18 @@ > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include "../strcspn.c" > > diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c > > index 423de2e2b2..86cda7b037 100644 > > --- a/sysdeps/x86_64/multiarch/strcspn-generic.c > > +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c > > @@ -16,8 +16,10 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > -#if IS_IN (libc) > > - > > +/* We always need to build this implementation as strcspn-sse4 needs > > + to be able to fallback to it. */ > > +#include <isa-level.h> > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > # include <sysdep.h> > > # define STRCSPN __strcspn_generic > > > > diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c > > index 59f64f9fe8..becdaf05f3 100644 > > --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c > > +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c > > @@ -52,9 +52,11 @@ > > when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset > > X for case 1. */ > > > > +#ifndef STRCSPN > > +# define STRCSPN __strcspn_sse42 > > +#endif > > #ifndef STRCSPN_GENERIC > > # define STRCSPN_GENERIC __strcspn_generic > > -# define STRCSPN_SSE42 __strcspn_sse42 > > #endif > > > > #ifdef USE_AS_STRPBRK > > @@ -78,7 +80,7 @@ char * > > size_t > > #endif > > __attribute__ ((section (".text.sse4.2"))) > > -STRCSPN_SSE42 (const char *s, const char *a) > > +STRCSPN (const char *s, const char *a) > > { > > if (*a == 0) > > RETURN (NULL, strlen (s)); > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c > > index d31acfe495..058f635774 100644 > > --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c > > +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c > > @@ -16,8 +16,11 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > -#if IS_IN (libc) > > > > +/* We always need to build this implementation as strpbrk-sse4 needs > > + to be able to fallback to it. */ > > +#include <isa-level.h> > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > # include <sysdep.h> > > # define STRPBRK __strpbrk_generic > > > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > index bf74d660d5..0adb577955 100644 > > --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > @@ -16,7 +16,11 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > +#ifndef STRPBRK > > +# define STRPBRK __strpbrk_sse42 > > +#endif > > + > > #define USE_AS_STRPBRK > > #define STRCSPN_GENERIC __strpbrk_generic > > -#define STRCSPN_SSE42 __strpbrk_sse42 > > +#define STRCSPN STRPBRK > > #include "strcspn-sse4.c" > > diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c > > index 6b50c36432..0f485ab4da 100644 > > --- a/sysdeps/x86_64/multiarch/strspn-generic.c > > +++ b/sysdeps/x86_64/multiarch/strspn-generic.c > > @@ -16,13 +16,16 @@ > > License along with the GNU C Library; if not, see > > <https://www.gnu.org/licenses/>. */ > > > > -#if IS_IN (libc) > > - > > +/* We always need to build this implementation as strspn-sse4 needs to > > + be able to fallback to it. */ > > +#include <isa-level.h> > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > # include <sysdep.h> > > # define STRSPN __strspn_generic > > > > # undef libc_hidden_builtin_def > > # define libc_hidden_builtin_def(STRSPN) > > + > > #endif > > > > #include <string/strspn.c> > > diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c > > index d044916688..4d50507a0d 100644 > > --- a/sysdeps/x86_64/multiarch/strspn-sse4.c > > +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c > > @@ -53,10 +53,13 @@ > > > > extern size_t __strspn_generic (const char *, const char *) attribute_hidden; > > > > +#ifndef STRSPN > > +# define STRSPN __strspn_sse42 > > +#endif > > > > size_t > > __attribute__ ((section (".text.sse4.2"))) > > -__strspn_sse42 (const char *s, const char *a) > > +STRSPN (const char *s, const char *a) > > { > > if (*a == 0) > > return 0; > > diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c > > new file mode 100644 > > index 0000000000..51fd70b63d > > --- /dev/null > > +++ b/sysdeps/x86_64/strcspn-generic.c > > @@ -0,0 +1,25 @@ > > +/* Hook for build strcspn-generic for non-multiarch build. Needed for > > + the ISA level >= 2 because strcspn-sse4 has a dependency on > > + strcspn-generic. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/strcspn-generic.c" > > +#endif > > diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c > > new file mode 100644 > > index 0000000000..cd54eed869 > > --- /dev/null > > +++ b/sysdeps/x86_64/strcspn.c > > @@ -0,0 +1,27 @@ > > +/* strcspn hook for non-multiarch and RTLD build. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > +#include <string/strcspn.c> > > +#else > > +#define STRCSPN strcspn > > +#include "multiarch/strcspn-sse4.c" > > +libc_hidden_builtin_def (strcspn) > > +#endif > > diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c > > new file mode 100644 > > index 0000000000..a7d8b11216 > > --- /dev/null > > +++ b/sysdeps/x86_64/strpbrk-generic.c > > @@ -0,0 +1,25 @@ > > +/* Hook for build strpbrk-generic for non-multiarch build. Needed for > > + the ISA level >= 2 because strpbrk-sse4 has a dependency on > > + strpbrk-generic. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/strpbrk-generic.c" > > +#endif > > diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c > > new file mode 100644 > > index 0000000000..e7ea1b334a > > --- /dev/null > > +++ b/sysdeps/x86_64/strpbrk.c > > @@ -0,0 +1,27 @@ > > +/* strpbrk hook for non-multiarch and RTLD build. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > +#include <string/strpbrk.c> > > +#else > > +#define STRPBRK strpbrk > > +#include "multiarch/strpbrk-sse4.c" > > +libc_hidden_builtin_def (strpbrk) > > +#endif > > diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c > > new file mode 100644 > > index 0000000000..1a2a576e49 > > --- /dev/null > > +++ b/sysdeps/x86_64/strspn-generic.c > > @@ -0,0 +1,26 @@ > > +/* Hook for build strspn-generic for non-multiarch build. Needed for > > + the ISA level >= 2 because strspn-sse4 has a dependency on > > + strspn-generic. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/strspn-generic.c" > > +#endif > > diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c > > new file mode 100644 > > index 0000000000..7b9ede26d9 > > --- /dev/null > > +++ b/sysdeps/x86_64/strspn.c > > @@ -0,0 +1,27 @@ > > +/* strspn hook for non-multiarch and RTLD build. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > +#include <string/strspn.c> > > +#else > > +#define STRSPN strspn > > +#include "multiarch/strspn-sse4.c" > > +libc_hidden_builtin_def (strspn) > > +#endif > > diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c > > new file mode 100644 > > index 0000000000..8f9eb13547 > > --- /dev/null > > +++ b/sysdeps/x86_64/varshift.c > > @@ -0,0 +1,26 @@ > > +/* Hook for build varshift for non-multiarch build. Needed for the > > + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 > > + all have a dependency on varshift.c. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + <https://www.gnu.org/licenses/>. */ > > + > > + > > +#include <isa-level.h> > > + > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > +# include "multiarch/varshift.c" > > +#endif > > -- > > 2.34.1 > > > > LGTM. > > Thanks. > > -- > H.J. Rebsubmitted in V5 after rebase ontop of: commit 0aa294fb887bb5aae4cdfa4b764325466a329131 Author: Noah Goldstein <goldstein.w.n@gmail.com> Date: Wed Jun 29 18:56:17 2022 -0700 x86: Add missing IS_IN (libc) check to strcspn-sse4.c ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-30 3:11 ` Noah Goldstein @ 2022-06-30 3:27 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:27 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 8:11 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > On Wed, Jun 29, 2022 at 3:16 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > On Wed, Jun 29, 2022 at 3:05 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > The changes for these functions are different than the others because > > > the best implementation (sse4_2) requires the generic > > > implementation as a fallback to be built as well. > > > > > > Changes are: > > > > > > 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically > > > select the best implementation based on the configured ISA build > > > level. > > > > > > 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the > > > sysdeps/x86_64 directory so that the the sse4 implementation will > > > have all of its dependencies for the non-multiarch / rtld build > > > when ISA level >= 2. > > > > > > 3. Add new multiarch/rtld-strcspn.c that just include the > > > non-multiarch strcspn.c which will in turn select the best > > > implementation based on the compiled ISA level. > > > > > > 4. Refactor the ifunc selector and ifunc implementation list to use > > > the ISA level aware wrapper macros that allow functions below the > > > compiled ISA level (with a guranteed replacement) to be skipped. > > > > > > Tested with and without multiarch on x86_64 for ISA levels: > > > {generic, x86-64-v2, x86-64-v3, x86-64-v4} > > > > > > And m32 with and without multiarch. > > > --- > > > sysdeps/x86_64/Makefile | 9 +++++++- > > > sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- > > > sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ > > > sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- > > > sysdeps/x86_64/multiarch/strcspn-sse4.c | 6 +++-- > > > sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- > > > sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- > > > sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- > > > sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- > > > sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ > > > sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ > > > sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ > > > sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ > > > sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ > > > sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ > > > sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ > > > 17 files changed, 242 insertions(+), 11 deletions(-) > > > create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c > > > create mode 100644 sysdeps/x86_64/strcspn-generic.c > > > create mode 100644 sysdeps/x86_64/strcspn.c > > > create mode 100644 sysdeps/x86_64/strpbrk-generic.c > > > create mode 100644 sysdeps/x86_64/strpbrk.c > > > create mode 100644 sysdeps/x86_64/strspn-generic.c > > > create mode 100644 sysdeps/x86_64/strspn.c > > > create mode 100644 sysdeps/x86_64/varshift.c > > > > > > diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile > > > index 79365aff2a..e597a4855f 100644 > > > --- a/sysdeps/x86_64/Makefile > > > +++ b/sysdeps/x86_64/Makefile > > > @@ -14,7 +14,14 @@ sysdep_noprof += _mcount > > > endif > > > > > > ifeq ($(subdir),string) > > > -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii > > > +sysdep_routines += \ > > > + strcasecmp_l-nonascii \ > > > + strcspn-generic \ > > > + strncase_l-nonascii \ > > > + strpbrk-generic \ > > > + strspn-generic \ > > > + varshift \ > > > +# sysdep_routines > > > gen-as-const-headers += locale-defines.sym > > > tests += \ > > > tst-rsi-strlen > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > > index 0d28319905..6c03f53a8b 100644 > > > --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > > +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > > > @@ -531,6 +531,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > > > /* Support sysdeps/x86_64/multiarch/strcspn.c. */ > > > IFUNC_IMPL (i, name, strcspn, > > > + /* All implementations of strcspn are built at all ISA > > > + levels. */ > > > IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), > > > __strcspn_sse42) > > > IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) > > > @@ -607,6 +609,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > > > /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ > > > IFUNC_IMPL (i, name, strpbrk, > > > + /* All implementations of strpbrk are built at all ISA > > > + levels. */ > > > IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), > > > __strpbrk_sse42) > > > IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) > > > @@ -614,6 +618,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > > > > > /* Support sysdeps/x86_64/multiarch/strspn.c. */ > > > IFUNC_IMPL (i, name, strspn, > > > + /* All implementations of strspn are built at all ISA > > > + levels. */ > > > IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), > > > __strspn_sse42) > > > IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > index 973041d23b..f8943c8210 100644 > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) > > > there is no other optimized implementation keep using. If an > > > optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > > (cpu_features, Slow_SSE4_2) check. */ > > > - if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > return OPTIMIZE (sse42); > > > > > > return OPTIMIZE (generic); > > > diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c > > > new file mode 100644 > > > index 0000000000..99e3c59e00 > > > --- /dev/null > > > +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c > > > @@ -0,0 +1,18 @@ > > > +/* Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > +#include "../strcspn.c" > > > diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c > > > index 423de2e2b2..86cda7b037 100644 > > > --- a/sysdeps/x86_64/multiarch/strcspn-generic.c > > > +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c > > > @@ -16,8 +16,10 @@ > > > License along with the GNU C Library; if not, see > > > <https://www.gnu.org/licenses/>. */ > > > > > > -#if IS_IN (libc) > > > - > > > +/* We always need to build this implementation as strcspn-sse4 needs > > > + to be able to fallback to it. */ > > > +#include <isa-level.h> > > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > > # include <sysdep.h> > > > # define STRCSPN __strcspn_generic > > > > > > diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c > > > index 59f64f9fe8..becdaf05f3 100644 > > > --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c > > > +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c > > > @@ -52,9 +52,11 @@ > > > when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset > > > X for case 1. */ > > > > > > +#ifndef STRCSPN > > > +# define STRCSPN __strcspn_sse42 > > > +#endif > > > #ifndef STRCSPN_GENERIC > > > # define STRCSPN_GENERIC __strcspn_generic > > > -# define STRCSPN_SSE42 __strcspn_sse42 > > > #endif > > > > > > #ifdef USE_AS_STRPBRK > > > @@ -78,7 +80,7 @@ char * > > > size_t > > > #endif > > > __attribute__ ((section (".text.sse4.2"))) > > > -STRCSPN_SSE42 (const char *s, const char *a) > > > +STRCSPN (const char *s, const char *a) > > > { > > > if (*a == 0) > > > RETURN (NULL, strlen (s)); > > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c > > > index d31acfe495..058f635774 100644 > > > --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c > > > +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c > > > @@ -16,8 +16,11 @@ > > > License along with the GNU C Library; if not, see > > > <https://www.gnu.org/licenses/>. */ > > > > > > -#if IS_IN (libc) > > > > > > +/* We always need to build this implementation as strpbrk-sse4 needs > > > + to be able to fallback to it. */ > > > +#include <isa-level.h> > > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > > # include <sysdep.h> > > > # define STRPBRK __strpbrk_generic > > > > > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > > index bf74d660d5..0adb577955 100644 > > > --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > > +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > > > @@ -16,7 +16,11 @@ > > > License along with the GNU C Library; if not, see > > > <https://www.gnu.org/licenses/>. */ > > > > > > +#ifndef STRPBRK > > > +# define STRPBRK __strpbrk_sse42 > > > +#endif > > > + > > > #define USE_AS_STRPBRK > > > #define STRCSPN_GENERIC __strpbrk_generic > > > -#define STRCSPN_SSE42 __strpbrk_sse42 > > > +#define STRCSPN STRPBRK > > > #include "strcspn-sse4.c" > > > diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c > > > index 6b50c36432..0f485ab4da 100644 > > > --- a/sysdeps/x86_64/multiarch/strspn-generic.c > > > +++ b/sysdeps/x86_64/multiarch/strspn-generic.c > > > @@ -16,13 +16,16 @@ > > > License along with the GNU C Library; if not, see > > > <https://www.gnu.org/licenses/>. */ > > > > > > -#if IS_IN (libc) > > > - > > > +/* We always need to build this implementation as strspn-sse4 needs to > > > + be able to fallback to it. */ > > > +#include <isa-level.h> > > > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > > # include <sysdep.h> > > > # define STRSPN __strspn_generic > > > > > > # undef libc_hidden_builtin_def > > > # define libc_hidden_builtin_def(STRSPN) > > > + > > > #endif > > > > > > #include <string/strspn.c> > > > diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c > > > index d044916688..4d50507a0d 100644 > > > --- a/sysdeps/x86_64/multiarch/strspn-sse4.c > > > +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c > > > @@ -53,10 +53,13 @@ > > > > > > extern size_t __strspn_generic (const char *, const char *) attribute_hidden; > > > > > > +#ifndef STRSPN > > > +# define STRSPN __strspn_sse42 > > > +#endif > > > > > > size_t > > > __attribute__ ((section (".text.sse4.2"))) > > > -__strspn_sse42 (const char *s, const char *a) > > > +STRSPN (const char *s, const char *a) > > > { > > > if (*a == 0) > > > return 0; > > > diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c > > > new file mode 100644 > > > index 0000000000..51fd70b63d > > > --- /dev/null > > > +++ b/sysdeps/x86_64/strcspn-generic.c > > > @@ -0,0 +1,25 @@ > > > +/* Hook for build strcspn-generic for non-multiarch build. Needed for > > > + the ISA level >= 2 because strcspn-sse4 has a dependency on > > > + strcspn-generic. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > > +# include "multiarch/strcspn-generic.c" > > > +#endif > > > diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c > > > new file mode 100644 > > > index 0000000000..cd54eed869 > > > --- /dev/null > > > +++ b/sysdeps/x86_64/strcspn.c > > > @@ -0,0 +1,27 @@ > > > +/* strcspn hook for non-multiarch and RTLD build. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > > +#include <string/strcspn.c> > > > +#else > > > +#define STRCSPN strcspn > > > +#include "multiarch/strcspn-sse4.c" > > > +libc_hidden_builtin_def (strcspn) > > > +#endif > > > diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c > > > new file mode 100644 > > > index 0000000000..a7d8b11216 > > > --- /dev/null > > > +++ b/sysdeps/x86_64/strpbrk-generic.c > > > @@ -0,0 +1,25 @@ > > > +/* Hook for build strpbrk-generic for non-multiarch build. Needed for > > > + the ISA level >= 2 because strpbrk-sse4 has a dependency on > > > + strpbrk-generic. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > > +# include "multiarch/strpbrk-generic.c" > > > +#endif > > > diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c > > > new file mode 100644 > > > index 0000000000..e7ea1b334a > > > --- /dev/null > > > +++ b/sysdeps/x86_64/strpbrk.c > > > @@ -0,0 +1,27 @@ > > > +/* strpbrk hook for non-multiarch and RTLD build. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > > +#include <string/strpbrk.c> > > > +#else > > > +#define STRPBRK strpbrk > > > +#include "multiarch/strpbrk-sse4.c" > > > +libc_hidden_builtin_def (strpbrk) > > > +#endif > > > diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c > > > new file mode 100644 > > > index 0000000000..1a2a576e49 > > > --- /dev/null > > > +++ b/sysdeps/x86_64/strspn-generic.c > > > @@ -0,0 +1,26 @@ > > > +/* Hook for build strspn-generic for non-multiarch build. Needed for > > > + the ISA level >= 2 because strspn-sse4 has a dependency on > > > + strspn-generic. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > > +# include "multiarch/strspn-generic.c" > > > +#endif > > > diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c > > > new file mode 100644 > > > index 0000000000..7b9ede26d9 > > > --- /dev/null > > > +++ b/sysdeps/x86_64/strspn.c > > > @@ -0,0 +1,27 @@ > > > +/* strspn hook for non-multiarch and RTLD build. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL == 1 > > > +#include <string/strspn.c> > > > +#else > > > +#define STRSPN strspn > > > +#include "multiarch/strspn-sse4.c" > > > +libc_hidden_builtin_def (strspn) > > > +#endif > > > diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c > > > new file mode 100644 > > > index 0000000000..8f9eb13547 > > > --- /dev/null > > > +++ b/sysdeps/x86_64/varshift.c > > > @@ -0,0 +1,26 @@ > > > +/* Hook for build varshift for non-multiarch build. Needed for the > > > + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 > > > + all have a dependency on varshift.c. > > > + Copyright (C) 2022 Free Software Foundation, Inc. > > > + This file is part of the GNU C Library. > > > + > > > + The GNU C Library is free software; you can redistribute it and/or > > > + modify it under the terms of the GNU Lesser General Public > > > + License as published by the Free Software Foundation; either > > > + version 2.1 of the License, or (at your option) any later version. > > > + > > > + The GNU C Library is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > > + Lesser General Public License for more details. > > > + > > > + You should have received a copy of the GNU Lesser General Public > > > + License along with the GNU C Library; if not, see > > > + <https://www.gnu.org/licenses/>. */ > > > + > > > + > > > +#include <isa-level.h> > > > + > > > +#if MINIMUM_X86_ISA_LEVEL >= 2 > > > +# include "multiarch/varshift.c" > > > +#endif > > > -- > > > 2.34.1 > > > > > > > LGTM. > > > > Thanks. > > > > -- > > H.J. > > Rebsubmitted in V5 after rebase ontop of: > > commit 0aa294fb887bb5aae4cdfa4b764325466a329131 > Author: Noah Goldstein <goldstein.w.n@gmail.com> > Date: Wed Jun 29 18:56:17 2022 -0700 > > x86: Add missing IS_IN (libc) check to strcspn-sse4.c There is an issue with this patch. Ignore it. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-29 22:05 ` [PATCH v2 " Noah Goldstein 2022-06-29 22:05 ` [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-06-29 22:12 ` H.J. Lu 2022-06-30 3:09 ` Noah Goldstein 1 sibling, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-06-29 22:12 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 3:05 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > Just for clarities sake and so that if a future implementation is > added we remember to add the check. > --- > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index ee36525bcf..973041d23b 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -27,7 +27,11 @@ IFUNC_SELECTOR (void) > { > const struct cpu_features* cpu_features = __get_cpu_features (); > > - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > + there is no other optimized implementation keep using. If an > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > + (cpu_features, Slow_SSE4_2) check. */ > + if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > return OPTIMIZE (generic); > -- > 2.34.1 > LGTM. Thanks. -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-29 22:12 ` [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu @ 2022-06-30 3:09 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:09 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Wed, Jun 29, 2022 at 3:13 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Wed, Jun 29, 2022 at 3:05 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > Just for clarities sake and so that if a future implementation is > > added we remember to add the check. > > --- > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 +++++- > > 1 file changed, 5 insertions(+), 1 deletion(-) > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > index ee36525bcf..973041d23b 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > @@ -27,7 +27,11 @@ IFUNC_SELECTOR (void) > > { > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > + there is no other optimized implementation keep using. If an > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > + (cpu_features, Slow_SSE4_2) check. */ > > + if (ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) This was buggy as standalone patch (hidden by the next in series). Resubmitted with fix in V4. > > return OPTIMIZE (sse42); > > > > return OPTIMIZE (generic); > > -- > > 2.34.1 > > > > LGTM. > > Thanks. > > -- > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (2 preceding siblings ...) 2022-06-29 22:05 ` [PATCH v2 " Noah Goldstein @ 2022-06-30 3:08 ` Noah Goldstein 2022-06-30 3:08 ` [PATCH v4 2/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein 2022-06-30 3:10 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein 2022-06-30 3:10 ` [PATCH v5 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (3 subsequent siblings) 7 siblings, 2 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:08 UTC (permalink / raw) To: libc-alpha Was missing to for the multiarch build rtld-strncmp-sse4_2.os was being built and exporting symbols: build/glibc/string/rtld-strncmp-sse4_2.os: 0000000000000000 T __strncmp_sse42 Introduced in: commit 11ffcacb64a939c10cfc713746b8ec88837f5c4a Author: H.J. Lu <hjl.tools@gmail.com> Date: Wed Jun 21 12:10:50 2017 -0700 x86-64: Implement strcmp family IFUNC selectors in C --- sysdeps/x86_64/multiarch/strncmp-sse4_2.S | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/sysdeps/x86_64/multiarch/strncmp-sse4_2.S b/sysdeps/x86_64/multiarch/strncmp-sse4_2.S index 9773e5fc09..310a6dbe77 100644 --- a/sysdeps/x86_64/multiarch/strncmp-sse4_2.S +++ b/sysdeps/x86_64/multiarch/strncmp-sse4_2.S @@ -16,6 +16,8 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#define STRCMP_SSE42 __strncmp_sse42 -#define USE_AS_STRNCMP -#include "strcmp-sse42.S" +#if IS_IN (libc) +# define STRCMP_SSE42 __strncmp_sse42 +# define USE_AS_STRNCMP +# include "strcmp-sse42.S" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v4 2/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-30 3:08 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein @ 2022-06-30 3:08 ` Noah Goldstein 2022-06-30 3:10 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein 1 sibling, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:08 UTC (permalink / raw) To: libc-alpha Just for clarities sake and so that if a future implementation is added we remember to add the check. --- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index ee36525bcf..4eb9cca7f0 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) { const struct cpu_features* cpu_features = __get_cpu_features (); + /* This function uses slow sse4.2 instructions (pcmpstri) but since + there is no other optimized implementation keep using. If an + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P + (cpu_features, Slow_SSE4_2) check. */ if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S 2022-06-30 3:08 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein 2022-06-30 3:08 ` [PATCH v4 2/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein @ 2022-06-30 3:10 ` Noah Goldstein 1 sibling, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:10 UTC (permalink / raw) To: GNU C Library On Wed, Jun 29, 2022 at 8:08 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > Was missing to for the multiarch build rtld-strncmp-sse4_2.os was > being built and exporting symbols: > > build/glibc/string/rtld-strncmp-sse4_2.os: > 0000000000000000 T __strncmp_sse42 > > Introduced in: > > commit 11ffcacb64a939c10cfc713746b8ec88837f5c4a > Author: H.J. Lu <hjl.tools@gmail.com> > Date: Wed Jun 21 12:10:50 2017 -0700 > > x86-64: Implement strcmp family IFUNC selectors in C > --- > sysdeps/x86_64/multiarch/strncmp-sse4_2.S | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/sysdeps/x86_64/multiarch/strncmp-sse4_2.S b/sysdeps/x86_64/multiarch/strncmp-sse4_2.S > index 9773e5fc09..310a6dbe77 100644 > --- a/sysdeps/x86_64/multiarch/strncmp-sse4_2.S > +++ b/sysdeps/x86_64/multiarch/strncmp-sse4_2.S > @@ -16,6 +16,8 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#define STRCMP_SSE42 __strncmp_sse42 > -#define USE_AS_STRNCMP > -#include "strcmp-sse42.S" > +#if IS_IN (libc) > +# define STRCMP_SSE42 __strncmp_sse42 > +# define USE_AS_STRNCMP > +# include "strcmp-sse42.S" > +#endif > -- > 2.34.1 > Ignore this patch ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v5 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (3 preceding siblings ...) 2022-06-30 3:08 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein @ 2022-06-30 3:10 ` Noah Goldstein 2022-06-30 3:10 ` [PATCH v5 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein 2022-06-30 3:44 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (2 subsequent siblings) 7 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:10 UTC (permalink / raw) To: libc-alpha Just for clarities sake and so that if a future implementation is added we remember to add the check. --- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index ee36525bcf..4eb9cca7f0 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) { const struct cpu_features* cpu_features = __get_cpu_features (); + /* This function uses slow sse4.2 instructions (pcmpstri) but since + there is no other optimized implementation keep using. If an + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P + (cpu_features, Slow_SSE4_2) check. */ if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v5 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-30 3:10 ` [PATCH v5 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein @ 2022-06-30 3:10 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:10 UTC (permalink / raw) To: libc-alpha The changes for these functions are different than the others because the best implementation (sse4_2) requires the generic implementation as a fallback to be built as well. Changes are: 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically select the best implementation based on the configured ISA build level. 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the sysdeps/x86_64 directory so that the the sse4 implementation will have all of its dependencies for the non-multiarch / rtld build when ISA level >= 2. 3. Add new multiarch/rtld-strcspn.c that just include the non-multiarch strcspn.c which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch. --- sysdeps/x86_64/Makefile | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- sysdeps/x86_64/multiarch/strcspn-sse4.c | 8 ++++--- sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ 17 files changed, 243 insertions(+), 12 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c create mode 100644 sysdeps/x86_64/strcspn-generic.c create mode 100644 sysdeps/x86_64/strcspn.c create mode 100644 sysdeps/x86_64/strpbrk-generic.c create mode 100644 sysdeps/x86_64/strpbrk.c create mode 100644 sysdeps/x86_64/strspn-generic.c create mode 100644 sysdeps/x86_64/strspn.c create mode 100644 sysdeps/x86_64/varshift.c diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile index 79365aff2a..e597a4855f 100644 --- a/sysdeps/x86_64/Makefile +++ b/sysdeps/x86_64/Makefile @@ -14,7 +14,14 @@ sysdep_noprof += _mcount endif ifeq ($(subdir),string) -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii +sysdep_routines += \ + strcasecmp_l-nonascii \ + strcspn-generic \ + strncase_l-nonascii \ + strpbrk-generic \ + strspn-generic \ + varshift \ +# sysdep_routines gen-as-const-headers += locale-defines.sym tests += \ tst-rsi-strlen diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c index 9e07283f86..b84acfead2 100644 --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c @@ -533,6 +533,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strcspn.c. */ IFUNC_IMPL (i, name, strcspn, + /* All implementations of strcspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), __strcspn_sse42) IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) @@ -609,6 +611,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ IFUNC_IMPL (i, name, strpbrk, + /* All implementations of strpbrk are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), __strpbrk_sse42) IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) @@ -616,6 +620,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strspn.c. */ IFUNC_IMPL (i, name, strspn, + /* All implementations of strspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), __strspn_sse42) IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index 4eb9cca7f0..f8943c8210 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) there is no other optimized implementation keep using. If an optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, Slow_SSE4_2) check. */ - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c new file mode 100644 index 0000000000..99e3c59e00 --- /dev/null +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c @@ -0,0 +1,18 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include "../strcspn.c" diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c index 423de2e2b2..86cda7b037 100644 --- a/sysdeps/x86_64/multiarch/strcspn-generic.c +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c @@ -16,8 +16,10 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strcspn-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRCSPN __strcspn_generic diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c index d11e87ab6c..ff88dfdb55 100644 --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c @@ -16,7 +16,7 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <nmmintrin.h> # include <string.h> @@ -54,9 +54,11 @@ when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset X for case 1. */ +# ifndef STRCSPN +# define STRCSPN __strcspn_sse42 +# endif # ifndef STRCSPN_GENERIC # define STRCSPN_GENERIC __strcspn_generic -# define STRCSPN_SSE42 __strcspn_sse42 # endif # ifdef USE_AS_STRPBRK @@ -80,7 +82,7 @@ char * size_t # endif __attribute__ ((section (".text.sse4.2"))) -STRCSPN_SSE42 (const char *s, const char *a) +STRCSPN (const char *s, const char *a) { if (*a == 0) RETURN (NULL, strlen (s)); diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c index d31acfe495..058f635774 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c @@ -16,8 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +/* We always need to build this implementation as strpbrk-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRPBRK __strpbrk_generic diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c index bf74d660d5..0adb577955 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c @@ -16,7 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ +#ifndef STRPBRK +# define STRPBRK __strpbrk_sse42 +#endif + #define USE_AS_STRPBRK #define STRCSPN_GENERIC __strpbrk_generic -#define STRCSPN_SSE42 __strpbrk_sse42 +#define STRCSPN STRPBRK #include "strcspn-sse4.c" diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c index 6b50c36432..0f485ab4da 100644 --- a/sysdeps/x86_64/multiarch/strspn-generic.c +++ b/sysdeps/x86_64/multiarch/strspn-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strspn-sse4 needs to + be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRSPN __strspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRSPN) + #endif #include <string/strspn.c> diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c index d044916688..4d50507a0d 100644 --- a/sysdeps/x86_64/multiarch/strspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c @@ -53,10 +53,13 @@ extern size_t __strspn_generic (const char *, const char *) attribute_hidden; +#ifndef STRSPN +# define STRSPN __strspn_sse42 +#endif size_t __attribute__ ((section (".text.sse4.2"))) -__strspn_sse42 (const char *s, const char *a) +STRSPN (const char *s, const char *a) { if (*a == 0) return 0; diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c new file mode 100644 index 0000000000..51fd70b63d --- /dev/null +++ b/sysdeps/x86_64/strcspn-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strcspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strcspn-sse4 has a dependency on + strcspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strcspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c new file mode 100644 index 0000000000..cd54eed869 --- /dev/null +++ b/sysdeps/x86_64/strcspn.c @@ -0,0 +1,27 @@ +/* strcspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strcspn.c> +#else +#define STRCSPN strcspn +#include "multiarch/strcspn-sse4.c" +libc_hidden_builtin_def (strcspn) +#endif diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c new file mode 100644 index 0000000000..a7d8b11216 --- /dev/null +++ b/sysdeps/x86_64/strpbrk-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strpbrk-generic for non-multiarch build. Needed for + the ISA level >= 2 because strpbrk-sse4 has a dependency on + strpbrk-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strpbrk-generic.c" +#endif diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c new file mode 100644 index 0000000000..e7ea1b334a --- /dev/null +++ b/sysdeps/x86_64/strpbrk.c @@ -0,0 +1,27 @@ +/* strpbrk hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strpbrk.c> +#else +#define STRPBRK strpbrk +#include "multiarch/strpbrk-sse4.c" +libc_hidden_builtin_def (strpbrk) +#endif diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c new file mode 100644 index 0000000000..1a2a576e49 --- /dev/null +++ b/sysdeps/x86_64/strspn-generic.c @@ -0,0 +1,26 @@ +/* Hook for build strspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strspn-sse4 has a dependency on + strspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c new file mode 100644 index 0000000000..7b9ede26d9 --- /dev/null +++ b/sysdeps/x86_64/strspn.c @@ -0,0 +1,27 @@ +/* strspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strspn.c> +#else +#define STRSPN strspn +#include "multiarch/strspn-sse4.c" +libc_hidden_builtin_def (strspn) +#endif diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c new file mode 100644 index 0000000000..8f9eb13547 --- /dev/null +++ b/sysdeps/x86_64/varshift.c @@ -0,0 +1,26 @@ +/* Hook for build varshift for non-multiarch build. Needed for the + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 + all have a dependency on varshift.c. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/varshift.c" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (4 preceding siblings ...) 2022-06-30 3:10 ` [PATCH v5 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein @ 2022-06-30 3:44 ` Noah Goldstein 2022-06-30 3:44 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein 2022-06-30 7:47 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Andreas Schwab 2022-06-30 20:13 ` Noah Goldstein 2022-07-04 4:28 ` [PATCH v7 " Noah Goldstein 7 siblings, 2 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:44 UTC (permalink / raw) To: libc-alpha Just for clarities sake and so that if a future implementation is added we remember to add the check. --- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index ee36525bcf..4eb9cca7f0 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) { const struct cpu_features* cpu_features = __get_cpu_features (); + /* This function uses slow sse4.2 instructions (pcmpstri) but since + there is no other optimized implementation keep using. If an + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P + (cpu_features, Slow_SSE4_2) check. */ if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-30 3:44 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein @ 2022-06-30 3:44 ` Noah Goldstein 2022-06-30 7:47 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Andreas Schwab 1 sibling, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 3:44 UTC (permalink / raw) To: libc-alpha The changes for these functions are different than the others because the best implementation (sse4_2) requires the generic implementation as a fallback to be built as well. Changes are: 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically select the best implementation based on the configured ISA build level. 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the sysdeps/x86_64 directory so that the the sse4 implementation will have all of its dependencies for the non-multiarch / rtld build when ISA level >= 2. 3. Add new multiarch/rtld-strcspn.c that just include the non-multiarch strcspn.c which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch. --- sysdeps/x86_64/Makefile | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- sysdeps/x86_64/multiarch/strcspn-sse4.c | 9 +++++--- sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- sysdeps/x86_64/multiarch/varshift.c | 4 +++- sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ 18 files changed, 247 insertions(+), 13 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c create mode 100644 sysdeps/x86_64/strcspn-generic.c create mode 100644 sysdeps/x86_64/strcspn.c create mode 100644 sysdeps/x86_64/strpbrk-generic.c create mode 100644 sysdeps/x86_64/strpbrk.c create mode 100644 sysdeps/x86_64/strspn-generic.c create mode 100644 sysdeps/x86_64/strspn.c create mode 100644 sysdeps/x86_64/varshift.c diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile index 79365aff2a..e597a4855f 100644 --- a/sysdeps/x86_64/Makefile +++ b/sysdeps/x86_64/Makefile @@ -14,7 +14,14 @@ sysdep_noprof += _mcount endif ifeq ($(subdir),string) -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii +sysdep_routines += \ + strcasecmp_l-nonascii \ + strcspn-generic \ + strncase_l-nonascii \ + strpbrk-generic \ + strspn-generic \ + varshift \ +# sysdep_routines gen-as-const-headers += locale-defines.sym tests += \ tst-rsi-strlen diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c index 9e07283f86..b84acfead2 100644 --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c @@ -533,6 +533,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strcspn.c. */ IFUNC_IMPL (i, name, strcspn, + /* All implementations of strcspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), __strcspn_sse42) IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) @@ -609,6 +611,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ IFUNC_IMPL (i, name, strpbrk, + /* All implementations of strpbrk are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), __strpbrk_sse42) IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) @@ -616,6 +620,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strspn.c. */ IFUNC_IMPL (i, name, strspn, + /* All implementations of strspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), __strspn_sse42) IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index 4eb9cca7f0..f8943c8210 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) there is no other optimized implementation keep using. If an optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, Slow_SSE4_2) check. */ - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c new file mode 100644 index 0000000000..99e3c59e00 --- /dev/null +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c @@ -0,0 +1,18 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include "../strcspn.c" diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c index 423de2e2b2..86cda7b037 100644 --- a/sysdeps/x86_64/multiarch/strcspn-generic.c +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c @@ -16,8 +16,10 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strcspn-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRCSPN __strcspn_generic diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c index d11e87ab6c..873a416cef 100644 --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c @@ -16,7 +16,8 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <nmmintrin.h> # include <string.h> @@ -54,9 +55,11 @@ when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset X for case 1. */ +# ifndef STRCSPN +# define STRCSPN __strcspn_sse42 +# endif # ifndef STRCSPN_GENERIC # define STRCSPN_GENERIC __strcspn_generic -# define STRCSPN_SSE42 __strcspn_sse42 # endif # ifdef USE_AS_STRPBRK @@ -80,7 +83,7 @@ char * size_t # endif __attribute__ ((section (".text.sse4.2"))) -STRCSPN_SSE42 (const char *s, const char *a) +STRCSPN (const char *s, const char *a) { if (*a == 0) RETURN (NULL, strlen (s)); diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c index d31acfe495..058f635774 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c @@ -16,8 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +/* We always need to build this implementation as strpbrk-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRPBRK __strpbrk_generic diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c index bf74d660d5..0adb577955 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c @@ -16,7 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ +#ifndef STRPBRK +# define STRPBRK __strpbrk_sse42 +#endif + #define USE_AS_STRPBRK #define STRCSPN_GENERIC __strpbrk_generic -#define STRCSPN_SSE42 __strpbrk_sse42 +#define STRCSPN STRPBRK #include "strcspn-sse4.c" diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c index 6b50c36432..0f485ab4da 100644 --- a/sysdeps/x86_64/multiarch/strspn-generic.c +++ b/sysdeps/x86_64/multiarch/strspn-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strspn-sse4 needs to + be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRSPN __strspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRSPN) + #endif #include <string/strspn.c> diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c index d044916688..4d50507a0d 100644 --- a/sysdeps/x86_64/multiarch/strspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c @@ -53,10 +53,13 @@ extern size_t __strspn_generic (const char *, const char *) attribute_hidden; +#ifndef STRSPN +# define STRSPN __strspn_sse42 +#endif size_t __attribute__ ((section (".text.sse4.2"))) -__strspn_sse42 (const char *s, const char *a) +STRSPN (const char *s, const char *a) { if (*a == 0) return 0; diff --git a/sysdeps/x86_64/multiarch/varshift.c b/sysdeps/x86_64/multiarch/varshift.c index 5e2f2bdf51..a98311d749 100644 --- a/sysdeps/x86_64/multiarch/varshift.c +++ b/sysdeps/x86_64/multiarch/varshift.c @@ -16,7 +16,9 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 + # include <stdint.h> const int8_t ___m128i_shift_right[31] attribute_hidden diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c new file mode 100644 index 0000000000..51fd70b63d --- /dev/null +++ b/sysdeps/x86_64/strcspn-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strcspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strcspn-sse4 has a dependency on + strcspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strcspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c new file mode 100644 index 0000000000..cd54eed869 --- /dev/null +++ b/sysdeps/x86_64/strcspn.c @@ -0,0 +1,27 @@ +/* strcspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strcspn.c> +#else +#define STRCSPN strcspn +#include "multiarch/strcspn-sse4.c" +libc_hidden_builtin_def (strcspn) +#endif diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c new file mode 100644 index 0000000000..a7d8b11216 --- /dev/null +++ b/sysdeps/x86_64/strpbrk-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strpbrk-generic for non-multiarch build. Needed for + the ISA level >= 2 because strpbrk-sse4 has a dependency on + strpbrk-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strpbrk-generic.c" +#endif diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c new file mode 100644 index 0000000000..e7ea1b334a --- /dev/null +++ b/sysdeps/x86_64/strpbrk.c @@ -0,0 +1,27 @@ +/* strpbrk hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strpbrk.c> +#else +#define STRPBRK strpbrk +#include "multiarch/strpbrk-sse4.c" +libc_hidden_builtin_def (strpbrk) +#endif diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c new file mode 100644 index 0000000000..1a2a576e49 --- /dev/null +++ b/sysdeps/x86_64/strspn-generic.c @@ -0,0 +1,26 @@ +/* Hook for build strspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strspn-sse4 has a dependency on + strspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c new file mode 100644 index 0000000000..7b9ede26d9 --- /dev/null +++ b/sysdeps/x86_64/strspn.c @@ -0,0 +1,27 @@ +/* strspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strspn.c> +#else +#define STRSPN strspn +#include "multiarch/strspn-sse4.c" +libc_hidden_builtin_def (strspn) +#endif diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c new file mode 100644 index 0000000000..8f9eb13547 --- /dev/null +++ b/sysdeps/x86_64/varshift.c @@ -0,0 +1,26 @@ +/* Hook for build varshift for non-multiarch build. Needed for the + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 + all have a dependency on varshift.c. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/varshift.c" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-30 3:44 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein 2022-06-30 3:44 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-06-30 7:47 ` Andreas Schwab 2022-06-30 20:13 ` Noah Goldstein 1 sibling, 1 reply; 36+ messages in thread From: Andreas Schwab @ 2022-06-30 7:47 UTC (permalink / raw) To: Noah Goldstein via Libc-alpha On Jun 29 2022, Noah Goldstein via Libc-alpha wrote: > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index ee36525bcf..4eb9cca7f0 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > { > const struct cpu_features* cpu_features = __get_cpu_features (); > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > + there is no other optimized implementation keep using. If an I think there is something missing at the end of the sentence. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-30 7:47 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Andreas Schwab @ 2022-06-30 20:13 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 20:13 UTC (permalink / raw) To: Andreas Schwab; +Cc: Noah Goldstein via Libc-alpha On Thu, Jun 30, 2022 at 12:47 AM Andreas Schwab <schwab@suse.de> wrote: > > On Jun 29 2022, Noah Goldstein via Libc-alpha wrote: > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > index ee36525bcf..4eb9cca7f0 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > { > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > + there is no other optimized implementation keep using. If an > > I think there is something missing at the end of the sentence. Fixed in V6. > > -- > Andreas Schwab, SUSE Labs, schwab@suse.de > GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 > "And now for something completely different." ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (5 preceding siblings ...) 2022-06-30 3:44 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein @ 2022-06-30 20:13 ` Noah Goldstein 2022-06-30 20:13 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein 2022-06-30 23:20 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu 2022-07-04 4:28 ` [PATCH v7 " Noah Goldstein 7 siblings, 2 replies; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 20:13 UTC (permalink / raw) To: libc-alpha Just for clarities sake and so that if a future implementation is added we remember to add the check. --- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index ee36525bcf..752798278c 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) { const struct cpu_features* cpu_features = __get_cpu_features (); + /* This function uses slow sse4.2 instructions (pcmpstri) but since + there is no other optimized implementation keep using it. If an + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P + (cpu_features, Slow_SSE4_2) check. */ if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-30 20:13 ` Noah Goldstein @ 2022-06-30 20:13 ` Noah Goldstein 2022-07-01 22:35 ` H.J. Lu 2022-06-30 23:20 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu 1 sibling, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-06-30 20:13 UTC (permalink / raw) To: libc-alpha The changes for these functions are different than the others because the best implementation (sse4_2) requires the generic implementation as a fallback to be built as well. Changes are: 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically select the best implementation based on the configured ISA build level. 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the sysdeps/x86_64 directory so that the the sse4 implementation will have all of its dependencies for the non-multiarch / rtld build when ISA level >= 2. 3. Add new multiarch/rtld-strcspn.c that just include the non-multiarch strcspn.c which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch. --- sysdeps/x86_64/Makefile | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- sysdeps/x86_64/multiarch/strcspn-sse4.c | 9 +++++--- sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- sysdeps/x86_64/multiarch/varshift.c | 4 +++- sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ 18 files changed, 247 insertions(+), 13 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c create mode 100644 sysdeps/x86_64/strcspn-generic.c create mode 100644 sysdeps/x86_64/strcspn.c create mode 100644 sysdeps/x86_64/strpbrk-generic.c create mode 100644 sysdeps/x86_64/strpbrk.c create mode 100644 sysdeps/x86_64/strspn-generic.c create mode 100644 sysdeps/x86_64/strspn.c create mode 100644 sysdeps/x86_64/varshift.c diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile index 79365aff2a..e597a4855f 100644 --- a/sysdeps/x86_64/Makefile +++ b/sysdeps/x86_64/Makefile @@ -14,7 +14,14 @@ sysdep_noprof += _mcount endif ifeq ($(subdir),string) -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii +sysdep_routines += \ + strcasecmp_l-nonascii \ + strcspn-generic \ + strncase_l-nonascii \ + strpbrk-generic \ + strspn-generic \ + varshift \ +# sysdep_routines gen-as-const-headers += locale-defines.sym tests += \ tst-rsi-strlen diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c index 9e07283f86..b84acfead2 100644 --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c @@ -533,6 +533,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strcspn.c. */ IFUNC_IMPL (i, name, strcspn, + /* All implementations of strcspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), __strcspn_sse42) IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) @@ -609,6 +611,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ IFUNC_IMPL (i, name, strpbrk, + /* All implementations of strpbrk are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), __strpbrk_sse42) IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) @@ -616,6 +620,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strspn.c. */ IFUNC_IMPL (i, name, strspn, + /* All implementations of strspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), __strspn_sse42) IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index 752798278c..9baf1bc1b3 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) there is no other optimized implementation keep using it. If an optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, Slow_SSE4_2) check. */ - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c new file mode 100644 index 0000000000..99e3c59e00 --- /dev/null +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c @@ -0,0 +1,18 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include "../strcspn.c" diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c index 423de2e2b2..86cda7b037 100644 --- a/sysdeps/x86_64/multiarch/strcspn-generic.c +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c @@ -16,8 +16,10 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strcspn-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRCSPN __strcspn_generic diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c index d11e87ab6c..873a416cef 100644 --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c @@ -16,7 +16,8 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <nmmintrin.h> # include <string.h> @@ -54,9 +55,11 @@ when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset X for case 1. */ +# ifndef STRCSPN +# define STRCSPN __strcspn_sse42 +# endif # ifndef STRCSPN_GENERIC # define STRCSPN_GENERIC __strcspn_generic -# define STRCSPN_SSE42 __strcspn_sse42 # endif # ifdef USE_AS_STRPBRK @@ -80,7 +83,7 @@ char * size_t # endif __attribute__ ((section (".text.sse4.2"))) -STRCSPN_SSE42 (const char *s, const char *a) +STRCSPN (const char *s, const char *a) { if (*a == 0) RETURN (NULL, strlen (s)); diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c index d31acfe495..058f635774 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c @@ -16,8 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +/* We always need to build this implementation as strpbrk-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRPBRK __strpbrk_generic diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c index bf74d660d5..0adb577955 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c @@ -16,7 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ +#ifndef STRPBRK +# define STRPBRK __strpbrk_sse42 +#endif + #define USE_AS_STRPBRK #define STRCSPN_GENERIC __strpbrk_generic -#define STRCSPN_SSE42 __strpbrk_sse42 +#define STRCSPN STRPBRK #include "strcspn-sse4.c" diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c index 6b50c36432..0f485ab4da 100644 --- a/sysdeps/x86_64/multiarch/strspn-generic.c +++ b/sysdeps/x86_64/multiarch/strspn-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strspn-sse4 needs to + be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRSPN __strspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRSPN) + #endif #include <string/strspn.c> diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c index d044916688..4d50507a0d 100644 --- a/sysdeps/x86_64/multiarch/strspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c @@ -53,10 +53,13 @@ extern size_t __strspn_generic (const char *, const char *) attribute_hidden; +#ifndef STRSPN +# define STRSPN __strspn_sse42 +#endif size_t __attribute__ ((section (".text.sse4.2"))) -__strspn_sse42 (const char *s, const char *a) +STRSPN (const char *s, const char *a) { if (*a == 0) return 0; diff --git a/sysdeps/x86_64/multiarch/varshift.c b/sysdeps/x86_64/multiarch/varshift.c index 5e2f2bdf51..a98311d749 100644 --- a/sysdeps/x86_64/multiarch/varshift.c +++ b/sysdeps/x86_64/multiarch/varshift.c @@ -16,7 +16,9 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 + # include <stdint.h> const int8_t ___m128i_shift_right[31] attribute_hidden diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c new file mode 100644 index 0000000000..51fd70b63d --- /dev/null +++ b/sysdeps/x86_64/strcspn-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strcspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strcspn-sse4 has a dependency on + strcspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strcspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c new file mode 100644 index 0000000000..cd54eed869 --- /dev/null +++ b/sysdeps/x86_64/strcspn.c @@ -0,0 +1,27 @@ +/* strcspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strcspn.c> +#else +#define STRCSPN strcspn +#include "multiarch/strcspn-sse4.c" +libc_hidden_builtin_def (strcspn) +#endif diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c new file mode 100644 index 0000000000..a7d8b11216 --- /dev/null +++ b/sysdeps/x86_64/strpbrk-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strpbrk-generic for non-multiarch build. Needed for + the ISA level >= 2 because strpbrk-sse4 has a dependency on + strpbrk-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strpbrk-generic.c" +#endif diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c new file mode 100644 index 0000000000..e7ea1b334a --- /dev/null +++ b/sysdeps/x86_64/strpbrk.c @@ -0,0 +1,27 @@ +/* strpbrk hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strpbrk.c> +#else +#define STRPBRK strpbrk +#include "multiarch/strpbrk-sse4.c" +libc_hidden_builtin_def (strpbrk) +#endif diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c new file mode 100644 index 0000000000..1a2a576e49 --- /dev/null +++ b/sysdeps/x86_64/strspn-generic.c @@ -0,0 +1,26 @@ +/* Hook for build strspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strspn-sse4 has a dependency on + strspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c new file mode 100644 index 0000000000..7b9ede26d9 --- /dev/null +++ b/sysdeps/x86_64/strspn.c @@ -0,0 +1,27 @@ +/* strspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strspn.c> +#else +#define STRSPN strspn +#include "multiarch/strspn-sse4.c" +libc_hidden_builtin_def (strspn) +#endif diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c new file mode 100644 index 0000000000..8f9eb13547 --- /dev/null +++ b/sysdeps/x86_64/varshift.c @@ -0,0 +1,26 @@ +/* Hook for build varshift for non-multiarch build. Needed for the + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 + all have a dependency on varshift.c. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/varshift.c" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-06-30 20:13 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-07-01 22:35 ` H.J. Lu 0 siblings, 0 replies; 36+ messages in thread From: H.J. Lu @ 2022-07-01 22:35 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > The changes for these functions are different than the others because > the best implementation (sse4_2) requires the generic > implementation as a fallback to be built as well. > > Changes are: > > 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically > select the best implementation based on the configured ISA build > level. > > 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the > sysdeps/x86_64 directory so that the the sse4 implementation will > have all of its dependencies for the non-multiarch / rtld build > when ISA level >= 2. > > 3. Add new multiarch/rtld-strcspn.c that just include the > non-multiarch strcspn.c which will in turn select the best > implementation based on the compiled ISA level. > > 4. Refactor the ifunc selector and ifunc implementation list to use > the ISA level aware wrapper macros that allow functions below the > compiled ISA level (with a guranteed replacement) to be skipped. > > Tested with and without multiarch on x86_64 for ISA levels: > {generic, x86-64-v2, x86-64-v3, x86-64-v4} > > And m32 with and without multiarch. > --- > sysdeps/x86_64/Makefile | 9 +++++++- > sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- > sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ > sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- > sysdeps/x86_64/multiarch/strcspn-sse4.c | 9 +++++--- > sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- > sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- > sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- > sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- > sysdeps/x86_64/multiarch/varshift.c | 4 +++- > sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ > sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ > sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ > sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ > sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ > 18 files changed, 247 insertions(+), 13 deletions(-) > create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c > create mode 100644 sysdeps/x86_64/strcspn-generic.c > create mode 100644 sysdeps/x86_64/strcspn.c > create mode 100644 sysdeps/x86_64/strpbrk-generic.c > create mode 100644 sysdeps/x86_64/strpbrk.c > create mode 100644 sysdeps/x86_64/strspn-generic.c > create mode 100644 sysdeps/x86_64/strspn.c > create mode 100644 sysdeps/x86_64/varshift.c > > diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile > index 79365aff2a..e597a4855f 100644 > --- a/sysdeps/x86_64/Makefile > +++ b/sysdeps/x86_64/Makefile > @@ -14,7 +14,14 @@ sysdep_noprof += _mcount > endif > > ifeq ($(subdir),string) > -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii > +sysdep_routines += \ > + strcasecmp_l-nonascii \ > + strcspn-generic \ > + strncase_l-nonascii \ > + strpbrk-generic \ > + strspn-generic \ > + varshift \ > +# sysdep_routines > gen-as-const-headers += locale-defines.sym > tests += \ > tst-rsi-strlen > diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > index 9e07283f86..b84acfead2 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c > +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c > @@ -533,6 +533,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strcspn.c. */ > IFUNC_IMPL (i, name, strcspn, > + /* All implementations of strcspn are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), > __strcspn_sse42) > IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) > @@ -609,6 +611,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ > IFUNC_IMPL (i, name, strpbrk, > + /* All implementations of strpbrk are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), > __strpbrk_sse42) > IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) > @@ -616,6 +620,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > > /* Support sysdeps/x86_64/multiarch/strspn.c. */ > IFUNC_IMPL (i, name, strspn, > + /* All implementations of strspn are built at all ISA > + levels. */ > IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), > __strspn_sse42) > IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index 752798278c..9baf1bc1b3 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -31,7 +31,7 @@ IFUNC_SELECTOR (void) > there is no other optimized implementation keep using it. If an > optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > (cpu_features, Slow_SSE4_2) check. */ > - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > return OPTIMIZE (generic); > diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c > new file mode 100644 > index 0000000000..99e3c59e00 > --- /dev/null > +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c > @@ -0,0 +1,18 @@ > +/* Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include "../strcspn.c" > diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c > index 423de2e2b2..86cda7b037 100644 > --- a/sysdeps/x86_64/multiarch/strcspn-generic.c > +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c > @@ -16,8 +16,10 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > - > +/* We always need to build this implementation as strcspn-sse4 needs > + to be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRCSPN __strcspn_generic > > diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c > index d11e87ab6c..873a416cef 100644 > --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c > +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c > @@ -16,7 +16,8 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > > # include <nmmintrin.h> > # include <string.h> > @@ -54,9 +55,11 @@ > when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset > X for case 1. */ > > +# ifndef STRCSPN > +# define STRCSPN __strcspn_sse42 > +# endif > # ifndef STRCSPN_GENERIC > # define STRCSPN_GENERIC __strcspn_generic > -# define STRCSPN_SSE42 __strcspn_sse42 > # endif > > # ifdef USE_AS_STRPBRK > @@ -80,7 +83,7 @@ char * > size_t > # endif > __attribute__ ((section (".text.sse4.2"))) > -STRCSPN_SSE42 (const char *s, const char *a) > +STRCSPN (const char *s, const char *a) > { > if (*a == 0) > RETURN (NULL, strlen (s)); > diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c > index d31acfe495..058f635774 100644 > --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c > +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c > @@ -16,8 +16,11 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > > +/* We always need to build this implementation as strpbrk-sse4 needs > + to be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRPBRK __strpbrk_generic > > diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > index bf74d660d5..0adb577955 100644 > --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c > +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c > @@ -16,7 +16,11 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > +#ifndef STRPBRK > +# define STRPBRK __strpbrk_sse42 > +#endif > + > #define USE_AS_STRPBRK > #define STRCSPN_GENERIC __strpbrk_generic > -#define STRCSPN_SSE42 __strpbrk_sse42 > +#define STRCSPN STRPBRK > #include "strcspn-sse4.c" > diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c > index 6b50c36432..0f485ab4da 100644 > --- a/sysdeps/x86_64/multiarch/strspn-generic.c > +++ b/sysdeps/x86_64/multiarch/strspn-generic.c > @@ -16,13 +16,16 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > - > +/* We always need to build this implementation as strspn-sse4 needs to > + be able to fallback to it. */ > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > # include <sysdep.h> > # define STRSPN __strspn_generic > > # undef libc_hidden_builtin_def > # define libc_hidden_builtin_def(STRSPN) > + > #endif > > #include <string/strspn.c> > diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c > index d044916688..4d50507a0d 100644 > --- a/sysdeps/x86_64/multiarch/strspn-sse4.c > +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c > @@ -53,10 +53,13 @@ > > extern size_t __strspn_generic (const char *, const char *) attribute_hidden; > > +#ifndef STRSPN > +# define STRSPN __strspn_sse42 > +#endif > > size_t > __attribute__ ((section (".text.sse4.2"))) > -__strspn_sse42 (const char *s, const char *a) > +STRSPN (const char *s, const char *a) > { > if (*a == 0) > return 0; > diff --git a/sysdeps/x86_64/multiarch/varshift.c b/sysdeps/x86_64/multiarch/varshift.c > index 5e2f2bdf51..a98311d749 100644 > --- a/sysdeps/x86_64/multiarch/varshift.c > +++ b/sysdeps/x86_64/multiarch/varshift.c > @@ -16,7 +16,9 @@ > License along with the GNU C Library; if not, see > <https://www.gnu.org/licenses/>. */ > > -#if IS_IN (libc) > +#include <isa-level.h> > +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 > + > # include <stdint.h> > > const int8_t ___m128i_shift_right[31] attribute_hidden > diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c > new file mode 100644 > index 0000000000..51fd70b63d > --- /dev/null > +++ b/sysdeps/x86_64/strcspn-generic.c > @@ -0,0 +1,25 @@ > +/* Hook for build strcspn-generic for non-multiarch build. Needed for > + the ISA level >= 2 because strcspn-sse4 has a dependency on > + strcspn-generic. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strcspn-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c > new file mode 100644 > index 0000000000..cd54eed869 > --- /dev/null > +++ b/sysdeps/x86_64/strcspn.c > @@ -0,0 +1,27 @@ > +/* strcspn hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strcspn.c> > +#else > +#define STRCSPN strcspn > +#include "multiarch/strcspn-sse4.c" > +libc_hidden_builtin_def (strcspn) > +#endif > diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c > new file mode 100644 > index 0000000000..a7d8b11216 > --- /dev/null > +++ b/sysdeps/x86_64/strpbrk-generic.c > @@ -0,0 +1,25 @@ > +/* Hook for build strpbrk-generic for non-multiarch build. Needed for > + the ISA level >= 2 because strpbrk-sse4 has a dependency on > + strpbrk-generic. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strpbrk-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c > new file mode 100644 > index 0000000000..e7ea1b334a > --- /dev/null > +++ b/sysdeps/x86_64/strpbrk.c > @@ -0,0 +1,27 @@ > +/* strpbrk hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strpbrk.c> > +#else > +#define STRPBRK strpbrk > +#include "multiarch/strpbrk-sse4.c" > +libc_hidden_builtin_def (strpbrk) > +#endif > diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c > new file mode 100644 > index 0000000000..1a2a576e49 > --- /dev/null > +++ b/sysdeps/x86_64/strspn-generic.c > @@ -0,0 +1,26 @@ > +/* Hook for build strspn-generic for non-multiarch build. Needed for > + the ISA level >= 2 because strspn-sse4 has a dependency on > + strspn-generic. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/strspn-generic.c" > +#endif > diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c > new file mode 100644 > index 0000000000..7b9ede26d9 > --- /dev/null > +++ b/sysdeps/x86_64/strspn.c > @@ -0,0 +1,27 @@ > +/* strspn hook for non-multiarch and RTLD build. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL == 1 > +#include <string/strspn.c> > +#else > +#define STRSPN strspn > +#include "multiarch/strspn-sse4.c" > +libc_hidden_builtin_def (strspn) > +#endif > diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c > new file mode 100644 > index 0000000000..8f9eb13547 > --- /dev/null > +++ b/sysdeps/x86_64/varshift.c > @@ -0,0 +1,26 @@ > +/* Hook for build varshift for non-multiarch build. Needed for the > + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 > + all have a dependency on varshift.c. > + Copyright (C) 2022 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > + > +#include <isa-level.h> > + > +#if MINIMUM_X86_ISA_LEVEL >= 2 > +# include "multiarch/varshift.c" > +#endif > -- > 2.34.1 > LGTM. Thanks. -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-30 20:13 ` Noah Goldstein 2022-06-30 20:13 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-06-30 23:20 ` H.J. Lu 2022-07-01 0:01 ` Noah Goldstein 1 sibling, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-06-30 23:20 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > Just for clarities sake and so that if a future implementation is > added we remember to add the check. > --- > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index ee36525bcf..752798278c 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > { > const struct cpu_features* cpu_features = __get_cpu_features (); > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > + there is no other optimized implementation keep using it. If an > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > + (cpu_features, Slow_SSE4_2) check. */ This function always uses sse4.2 instructions (pcmpstri) since there is no other optimized implementation. If an ... > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > -- > 2.34.1 > -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-30 23:20 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu @ 2022-07-01 0:01 ` Noah Goldstein 2022-07-01 0:01 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-07-01 0:01 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Thu, Jun 30, 2022 at 4:20 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > Just for clarities sake and so that if a future implementation is > > added we remember to add the check. > > --- > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > index ee36525bcf..752798278c 100644 > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > { > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > + there is no other optimized implementation keep using it. If an > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > + (cpu_features, Slow_SSE4_2) check. */ > > This function always uses sse4.2 instructions (pcmpstri) since there > is no other optimized implementation. If an ... Is it all sse4.2 instructions that count for Slow_SSE4.2 or other the microcode string ones? > > > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > return OPTIMIZE (sse42); > > > > -- > > 2.34.1 > > > > > -- > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-07-01 0:01 ` Noah Goldstein @ 2022-07-01 0:01 ` Noah Goldstein 2022-07-01 22:38 ` H.J. Lu 0 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-07-01 0:01 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > On Thu, Jun 30, 2022 at 4:20 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > Just for clarities sake and so that if a future implementation is > > > added we remember to add the check. > > > --- > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > > > 1 file changed, 4 insertions(+) > > > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > index ee36525bcf..752798278c 100644 > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > > { > > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > > + there is no other optimized implementation keep using it. If an > > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > > + (cpu_features, Slow_SSE4_2) check. */ > > > > This function always uses sse4.2 instructions (pcmpstri) since there > > is no other optimized implementation. If an ... > > Is it all sse4.2 instructions that count for Slow_SSE4.2 or other the > microcode string ones? other->only* > > > > > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > return OPTIMIZE (sse42); > > > > > > -- > > > 2.34.1 > > > > > > > > > -- > > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-07-01 0:01 ` Noah Goldstein @ 2022-07-01 22:38 ` H.J. Lu 2022-07-01 22:52 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-07-01 22:38 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > On Thu, Jun 30, 2022 at 4:20 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > Just for clarities sake and so that if a future implementation is > > > > added we remember to add the check. > > > > --- > > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > > > > 1 file changed, 4 insertions(+) > > > > > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > index ee36525bcf..752798278c 100644 > > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > > > { > > > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > > > + there is no other optimized implementation keep using it. If an > > > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > > > + (cpu_features, Slow_SSE4_2) check. */ > > > > > > This function always uses sse4.2 instructions (pcmpstri) since there > > > is no other optimized implementation. If an ... > > > > Is it all sse4.2 instructions that count for Slow_SSE4.2 or other the > > microcode string ones? > other->only* No. Only the string instructions are slow. > > > > > > > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > > return OPTIMIZE (sse42); > > > > > > > > -- > > > > 2.34.1 > > > > > > > > > > > > > -- > > > H.J. -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-07-01 22:38 ` H.J. Lu @ 2022-07-01 22:52 ` Noah Goldstein 2022-07-03 18:00 ` H.J. Lu 0 siblings, 1 reply; 36+ messages in thread From: Noah Goldstein @ 2022-07-01 22:52 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Fri, Jul 1, 2022 at 3:38 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > On Thu, Jun 30, 2022 at 4:20 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > > > On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > > > Just for clarities sake and so that if a future implementation is > > > > > added we remember to add the check. > > > > > --- > > > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > > > > > 1 file changed, 4 insertions(+) > > > > > > > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > index ee36525bcf..752798278c 100644 > > > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > > > > { > > > > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > > > > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > > > > + there is no other optimized implementation keep using it. If an > > > > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > > > > + (cpu_features, Slow_SSE4_2) check. */ > > > > > > > > This function always uses sse4.2 instructions (pcmpstri) since there > > > > is no other optimized implementation. If an ... Then always uses slow sse4.2... Also I think there needs to be a 'but' before since otherwise the sentence sounds like it's saying the function uses sse4.2 because there are no other implementations. > > > > > > Is it all sse4.2 instructions that count for Slow_SSE4.2 or other the > > > microcode string ones? > > other->only* > > No. Only the string instructions are slow. > > > > > > > > > > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > > > return OPTIMIZE (sse42); > > > > > > > > > > -- > > > > > 2.34.1 > > > > > > > > > > > > > > > > > -- > > > > H.J. > > > > -- > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-07-01 22:52 ` Noah Goldstein @ 2022-07-03 18:00 ` H.J. Lu 2022-07-04 4:28 ` Noah Goldstein 0 siblings, 1 reply; 36+ messages in thread From: H.J. Lu @ 2022-07-03 18:00 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Fri, Jul 1, 2022 at 3:52 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > On Fri, Jul 1, 2022 at 3:38 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > On Thu, Jun 30, 2022 at 4:20 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > > > > > On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > > > > > Just for clarities sake and so that if a future implementation is > > > > > > added we remember to add the check. > > > > > > --- > > > > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > > > > > > 1 file changed, 4 insertions(+) > > > > > > > > > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > > index ee36525bcf..752798278c 100644 > > > > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > > > > > { > > > > > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > > > > > > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > > > > > + there is no other optimized implementation keep using it. If an > > > > > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > > > > > + (cpu_features, Slow_SSE4_2) check. */ > > > > > > > > > > This function always uses sse4.2 instructions (pcmpstri) since there > > > > > is no other optimized implementation. If an ... > > Then always uses slow sse4.2... SSE4.2 isn't always slow. > Also I think there needs to be a 'but' before since otherwise the sentence > sounds like it's saying the function uses sse4.2 because there are no > other implementations. because there are no other "optimized" implementations. > > > > > > > > Is it all sse4.2 instructions that count for Slow_SSE4.2 or other the > > > > microcode string ones? > > > other->only* > > > > No. Only the string instructions are slow. > > > > > > > > > > > > > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > > > > return OPTIMIZE (sse42); > > > > > > > > > > > > -- > > > > > > 2.34.1 > > > > > > > > > > > > > > > > > > > > > -- > > > > > H.J. > > > > > > > > -- > > H.J. -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-07-03 18:00 ` H.J. Lu @ 2022-07-04 4:28 ` Noah Goldstein 0 siblings, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-07-04 4:28 UTC (permalink / raw) To: H.J. Lu; +Cc: GNU C Library, Carlos O'Donell On Sun, Jul 3, 2022 at 11:01 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Fri, Jul 1, 2022 at 3:52 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > On Fri, Jul 1, 2022 at 3:38 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > On Thu, Jun 30, 2022 at 5:01 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > > > On Thu, Jun 30, 2022 at 4:20 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > > > > > > > On Thu, Jun 30, 2022 at 1:13 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > > > > > > > > > > > Just for clarities sake and so that if a future implementation is > > > > > > > added we remember to add the check. > > > > > > > --- > > > > > > > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 4 ++++ > > > > > > > 1 file changed, 4 insertions(+) > > > > > > > > > > > > > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > > > index ee36525bcf..752798278c 100644 > > > > > > > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > > > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > > > > > > > @@ -27,6 +27,10 @@ IFUNC_SELECTOR (void) > > > > > > > { > > > > > > > const struct cpu_features* cpu_features = __get_cpu_features (); > > > > > > > > > > > > > > + /* This function uses slow sse4.2 instructions (pcmpstri) but since > > > > > > > + there is no other optimized implementation keep using it. If an > > > > > > > + optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P > > > > > > > + (cpu_features, Slow_SSE4_2) check. */ > > > > > > > > > > > > This function always uses sse4.2 instructions (pcmpstri) since there > > > > > > is no other optimized implementation. If an ... > > > > Then always uses slow sse4.2... > > SSE4.2 isn't always slow. > Think made the comment reflect that in V7. > > Also I think there needs to be a 'but' before since otherwise the sentence > > sounds like it's saying the function uses sse4.2 because there are no > > other implementations. > > because there are no other "optimized" implementations. > Made clearer in V7. > > > > > > > > > > Is it all sse4.2 instructions that count for Slow_SSE4.2 or other the > > > > > microcode string ones? > > > > other->only* > > > > > > No. Only the string instructions are slow. > > > > > > > > > > > > > > > > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > > > > > > > return OPTIMIZE (sse42); > > > > > > > > > > > > > > -- > > > > > > > 2.34.1 > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > H.J. > > > > > > > > > > > > -- > > > H.J. > > > > -- > H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v7 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein ` (6 preceding siblings ...) 2022-06-30 20:13 ` Noah Goldstein @ 2022-07-04 4:28 ` Noah Goldstein 2022-07-04 4:28 ` [PATCH v7 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein 2022-07-05 15:41 ` [PATCH v7 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu 7 siblings, 2 replies; 36+ messages in thread From: Noah Goldstein @ 2022-07-04 4:28 UTC (permalink / raw) To: libc-alpha Just for clarities sake and so that if a future implementation is added we remember to add the check. --- sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index ee36525bcf..f8b56936ec 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -27,6 +27,12 @@ IFUNC_SELECTOR (void) { const struct cpu_features* cpu_features = __get_cpu_features (); + /* This function uses the `pcmpstri` sse4.2 instruction which can be + slow on some CPUs. This normally would be guarded by a + Slow_SSE4_2 check, but since there is no other optimized + implementation its best to keep it regardless. If an optimized + fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, + Slow_SSE4_2) check. */ if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v7 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level 2022-07-04 4:28 ` [PATCH v7 " Noah Goldstein @ 2022-07-04 4:28 ` Noah Goldstein 2022-07-05 15:41 ` [PATCH v7 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu 1 sibling, 0 replies; 36+ messages in thread From: Noah Goldstein @ 2022-07-04 4:28 UTC (permalink / raw) To: libc-alpha The changes for these functions are different than the others because the best implementation (sse4_2) requires the generic implementation as a fallback to be built as well. Changes are: 1. Add non-multiarch functions for str{c|p}{brk|spn}.c to statically select the best implementation based on the configured ISA build level. 2. Add stubs for str{c|p}{brk|spn}-generic and varshift.c to in the sysdeps/x86_64 directory so that the the sse4 implementation will have all of its dependencies for the non-multiarch / rtld build when ISA level >= 2. 3. Add new multiarch/rtld-strcspn.c that just include the non-multiarch strcspn.c which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch. --- sysdeps/x86_64/Makefile | 9 +++++++- sysdeps/x86_64/multiarch/ifunc-impl-list.c | 6 +++++ sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 2 +- sysdeps/x86_64/multiarch/rtld-strcspn.c | 18 +++++++++++++++ sysdeps/x86_64/multiarch/strcspn-generic.c | 6 +++-- sysdeps/x86_64/multiarch/strcspn-sse4.c | 9 +++++--- sysdeps/x86_64/multiarch/strpbrk-generic.c | 5 +++- sysdeps/x86_64/multiarch/strpbrk-sse4.c | 6 ++++- sysdeps/x86_64/multiarch/strspn-generic.c | 7 ++++-- sysdeps/x86_64/multiarch/strspn-sse4.c | 5 +++- sysdeps/x86_64/multiarch/varshift.c | 4 +++- sysdeps/x86_64/strcspn-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strcspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strpbrk-generic.c | 25 ++++++++++++++++++++ sysdeps/x86_64/strpbrk.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/strspn-generic.c | 26 +++++++++++++++++++++ sysdeps/x86_64/strspn.c | 27 ++++++++++++++++++++++ sysdeps/x86_64/varshift.c | 26 +++++++++++++++++++++ 18 files changed, 247 insertions(+), 13 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/rtld-strcspn.c create mode 100644 sysdeps/x86_64/strcspn-generic.c create mode 100644 sysdeps/x86_64/strcspn.c create mode 100644 sysdeps/x86_64/strpbrk-generic.c create mode 100644 sysdeps/x86_64/strpbrk.c create mode 100644 sysdeps/x86_64/strspn-generic.c create mode 100644 sysdeps/x86_64/strspn.c create mode 100644 sysdeps/x86_64/varshift.c diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile index 79365aff2a..e597a4855f 100644 --- a/sysdeps/x86_64/Makefile +++ b/sysdeps/x86_64/Makefile @@ -14,7 +14,14 @@ sysdep_noprof += _mcount endif ifeq ($(subdir),string) -sysdep_routines += strcasecmp_l-nonascii strncase_l-nonascii +sysdep_routines += \ + strcasecmp_l-nonascii \ + strcspn-generic \ + strncase_l-nonascii \ + strpbrk-generic \ + strspn-generic \ + varshift \ +# sysdep_routines gen-as-const-headers += locale-defines.sym tests += \ tst-rsi-strlen diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c index 9e07283f86..b84acfead2 100644 --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c @@ -533,6 +533,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strcspn.c. */ IFUNC_IMPL (i, name, strcspn, + /* All implementations of strcspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strcspn, CPU_FEATURE_USABLE (SSE4_2), __strcspn_sse42) IFUNC_IMPL_ADD (array, i, strcspn, 1, __strcspn_generic)) @@ -609,6 +611,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strpbrk.c. */ IFUNC_IMPL (i, name, strpbrk, + /* All implementations of strpbrk are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strpbrk, CPU_FEATURE_USABLE (SSE4_2), __strpbrk_sse42) IFUNC_IMPL_ADD (array, i, strpbrk, 1, __strpbrk_generic)) @@ -616,6 +620,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/x86_64/multiarch/strspn.c. */ IFUNC_IMPL (i, name, strspn, + /* All implementations of strspn are built at all ISA + levels. */ IFUNC_IMPL_ADD (array, i, strspn, CPU_FEATURE_USABLE (SSE4_2), __strspn_sse42) IFUNC_IMPL_ADD (array, i, strspn, 1, __strspn_generic)) diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h index f8b56936ec..9a819ee1d2 100644 --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h @@ -33,7 +33,7 @@ IFUNC_SELECTOR (void) implementation its best to keep it regardless. If an optimized fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, Slow_SSE4_2) check. */ - if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) return OPTIMIZE (sse42); return OPTIMIZE (generic); diff --git a/sysdeps/x86_64/multiarch/rtld-strcspn.c b/sysdeps/x86_64/multiarch/rtld-strcspn.c new file mode 100644 index 0000000000..99e3c59e00 --- /dev/null +++ b/sysdeps/x86_64/multiarch/rtld-strcspn.c @@ -0,0 +1,18 @@ +/* Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include "../strcspn.c" diff --git a/sysdeps/x86_64/multiarch/strcspn-generic.c b/sysdeps/x86_64/multiarch/strcspn-generic.c index 423de2e2b2..86cda7b037 100644 --- a/sysdeps/x86_64/multiarch/strcspn-generic.c +++ b/sysdeps/x86_64/multiarch/strcspn-generic.c @@ -16,8 +16,10 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strcspn-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRCSPN __strcspn_generic diff --git a/sysdeps/x86_64/multiarch/strcspn-sse4.c b/sysdeps/x86_64/multiarch/strcspn-sse4.c index d11e87ab6c..873a416cef 100644 --- a/sysdeps/x86_64/multiarch/strcspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strcspn-sse4.c @@ -16,7 +16,8 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <nmmintrin.h> # include <string.h> @@ -54,9 +55,11 @@ when either CFlag or ZFlag is 1. If CFlag == 1, ECX has the offset X for case 1. */ +# ifndef STRCSPN +# define STRCSPN __strcspn_sse42 +# endif # ifndef STRCSPN_GENERIC # define STRCSPN_GENERIC __strcspn_generic -# define STRCSPN_SSE42 __strcspn_sse42 # endif # ifdef USE_AS_STRPBRK @@ -80,7 +83,7 @@ char * size_t # endif __attribute__ ((section (".text.sse4.2"))) -STRCSPN_SSE42 (const char *s, const char *a) +STRCSPN (const char *s, const char *a) { if (*a == 0) RETURN (NULL, strlen (s)); diff --git a/sysdeps/x86_64/multiarch/strpbrk-generic.c b/sysdeps/x86_64/multiarch/strpbrk-generic.c index d31acfe495..058f635774 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-generic.c +++ b/sysdeps/x86_64/multiarch/strpbrk-generic.c @@ -16,8 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +/* We always need to build this implementation as strpbrk-sse4 needs + to be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRPBRK __strpbrk_generic diff --git a/sysdeps/x86_64/multiarch/strpbrk-sse4.c b/sysdeps/x86_64/multiarch/strpbrk-sse4.c index bf74d660d5..0adb577955 100644 --- a/sysdeps/x86_64/multiarch/strpbrk-sse4.c +++ b/sysdeps/x86_64/multiarch/strpbrk-sse4.c @@ -16,7 +16,11 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ +#ifndef STRPBRK +# define STRPBRK __strpbrk_sse42 +#endif + #define USE_AS_STRPBRK #define STRCSPN_GENERIC __strpbrk_generic -#define STRCSPN_SSE42 __strpbrk_sse42 +#define STRCSPN STRPBRK #include "strcspn-sse4.c" diff --git a/sysdeps/x86_64/multiarch/strspn-generic.c b/sysdeps/x86_64/multiarch/strspn-generic.c index 6b50c36432..0f485ab4da 100644 --- a/sysdeps/x86_64/multiarch/strspn-generic.c +++ b/sysdeps/x86_64/multiarch/strspn-generic.c @@ -16,13 +16,16 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) - +/* We always need to build this implementation as strspn-sse4 needs to + be able to fallback to it. */ +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 # include <sysdep.h> # define STRSPN __strspn_generic # undef libc_hidden_builtin_def # define libc_hidden_builtin_def(STRSPN) + #endif #include <string/strspn.c> diff --git a/sysdeps/x86_64/multiarch/strspn-sse4.c b/sysdeps/x86_64/multiarch/strspn-sse4.c index d044916688..4d50507a0d 100644 --- a/sysdeps/x86_64/multiarch/strspn-sse4.c +++ b/sysdeps/x86_64/multiarch/strspn-sse4.c @@ -53,10 +53,13 @@ extern size_t __strspn_generic (const char *, const char *) attribute_hidden; +#ifndef STRSPN +# define STRSPN __strspn_sse42 +#endif size_t __attribute__ ((section (".text.sse4.2"))) -__strspn_sse42 (const char *s, const char *a) +STRSPN (const char *s, const char *a) { if (*a == 0) return 0; diff --git a/sysdeps/x86_64/multiarch/varshift.c b/sysdeps/x86_64/multiarch/varshift.c index 5e2f2bdf51..a98311d749 100644 --- a/sysdeps/x86_64/multiarch/varshift.c +++ b/sysdeps/x86_64/multiarch/varshift.c @@ -16,7 +16,9 @@ License along with the GNU C Library; if not, see <https://www.gnu.org/licenses/>. */ -#if IS_IN (libc) +#include <isa-level.h> +#if IS_IN (libc) || MINIMUM_X86_ISA_LEVEL >= 2 + # include <stdint.h> const int8_t ___m128i_shift_right[31] attribute_hidden diff --git a/sysdeps/x86_64/strcspn-generic.c b/sysdeps/x86_64/strcspn-generic.c new file mode 100644 index 0000000000..51fd70b63d --- /dev/null +++ b/sysdeps/x86_64/strcspn-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strcspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strcspn-sse4 has a dependency on + strcspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strcspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strcspn.c b/sysdeps/x86_64/strcspn.c new file mode 100644 index 0000000000..cd54eed869 --- /dev/null +++ b/sysdeps/x86_64/strcspn.c @@ -0,0 +1,27 @@ +/* strcspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strcspn.c> +#else +#define STRCSPN strcspn +#include "multiarch/strcspn-sse4.c" +libc_hidden_builtin_def (strcspn) +#endif diff --git a/sysdeps/x86_64/strpbrk-generic.c b/sysdeps/x86_64/strpbrk-generic.c new file mode 100644 index 0000000000..a7d8b11216 --- /dev/null +++ b/sysdeps/x86_64/strpbrk-generic.c @@ -0,0 +1,25 @@ +/* Hook for build strpbrk-generic for non-multiarch build. Needed for + the ISA level >= 2 because strpbrk-sse4 has a dependency on + strpbrk-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strpbrk-generic.c" +#endif diff --git a/sysdeps/x86_64/strpbrk.c b/sysdeps/x86_64/strpbrk.c new file mode 100644 index 0000000000..e7ea1b334a --- /dev/null +++ b/sysdeps/x86_64/strpbrk.c @@ -0,0 +1,27 @@ +/* strpbrk hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strpbrk.c> +#else +#define STRPBRK strpbrk +#include "multiarch/strpbrk-sse4.c" +libc_hidden_builtin_def (strpbrk) +#endif diff --git a/sysdeps/x86_64/strspn-generic.c b/sysdeps/x86_64/strspn-generic.c new file mode 100644 index 0000000000..1a2a576e49 --- /dev/null +++ b/sysdeps/x86_64/strspn-generic.c @@ -0,0 +1,26 @@ +/* Hook for build strspn-generic for non-multiarch build. Needed for + the ISA level >= 2 because strspn-sse4 has a dependency on + strspn-generic. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/strspn-generic.c" +#endif diff --git a/sysdeps/x86_64/strspn.c b/sysdeps/x86_64/strspn.c new file mode 100644 index 0000000000..7b9ede26d9 --- /dev/null +++ b/sysdeps/x86_64/strspn.c @@ -0,0 +1,27 @@ +/* strspn hook for non-multiarch and RTLD build. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL == 1 +#include <string/strspn.c> +#else +#define STRSPN strspn +#include "multiarch/strspn-sse4.c" +libc_hidden_builtin_def (strspn) +#endif diff --git a/sysdeps/x86_64/varshift.c b/sysdeps/x86_64/varshift.c new file mode 100644 index 0000000000..8f9eb13547 --- /dev/null +++ b/sysdeps/x86_64/varshift.c @@ -0,0 +1,26 @@ +/* Hook for build varshift for non-multiarch build. Needed for the + ISA level >= 2 because strspn-sse4, strcspn-sse4, and strpbrk-sse4 + all have a dependency on varshift.c. + Copyright (C) 2022 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + + +#include <isa-level.h> + +#if MINIMUM_X86_ISA_LEVEL >= 2 +# include "multiarch/varshift.c" +#endif -- 2.34.1 ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v7 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 2022-07-04 4:28 ` [PATCH v7 " Noah Goldstein 2022-07-04 4:28 ` [PATCH v7 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein @ 2022-07-05 15:41 ` H.J. Lu 1 sibling, 0 replies; 36+ messages in thread From: H.J. Lu @ 2022-07-05 15:41 UTC (permalink / raw) To: Noah Goldstein; +Cc: GNU C Library, Carlos O'Donell On Sun, Jul 3, 2022 at 9:28 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > Just for clarities sake and so that if a future implementation is > added we remember to add the check. > --- > sysdeps/x86_64/multiarch/ifunc-sse4_2.h | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > index ee36525bcf..f8b56936ec 100644 > --- a/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > +++ b/sysdeps/x86_64/multiarch/ifunc-sse4_2.h > @@ -27,6 +27,12 @@ IFUNC_SELECTOR (void) > { > const struct cpu_features* cpu_features = __get_cpu_features (); > > + /* This function uses the `pcmpstri` sse4.2 instruction which can be > + slow on some CPUs. This normally would be guarded by a > + Slow_SSE4_2 check, but since there is no other optimized > + implementation its best to keep it regardless. If an optimized > + fallback is added add a X86_ISA_CPU_FEATURE_ARCH_P (cpu_features, > + Slow_SSE4_2) check. */ > if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2)) > return OPTIMIZE (sse42); > > -- > 2.34.1 > LGTM. Thanks. -- H.J. ^ permalink raw reply [flat|nested] 36+ messages in thread
end of thread, other threads:[~2022-07-05 15:41 UTC | newest]
Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-28 15:27 [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein
2022-06-28 15:27 ` [PATCH v1 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
2022-06-29 19:20 ` H.J. Lu
2022-06-29 22:07 ` Noah Goldstein
2022-06-29 18:53 ` [PATCH v1 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu
2022-06-29 19:19 ` Noah Goldstein
2022-06-29 22:06 ` Noah Goldstein
2022-06-29 22:05 ` [PATCH v2 " Noah Goldstein
2022-06-29 22:05 ` [PATCH v2 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
2022-06-29 22:15 ` H.J. Lu
2022-06-30 3:11 ` Noah Goldstein
2022-06-30 3:27 ` Noah Goldstein
2022-06-29 22:12 ` [PATCH v2 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu
2022-06-30 3:09 ` Noah Goldstein
2022-06-30 3:08 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein
2022-06-30 3:08 ` [PATCH v4 2/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein
2022-06-30 3:10 ` [PATCH v4 1/2] x86: Add missing IS_IN (libc) check to strncmp-sse4_2.S Noah Goldstein
2022-06-30 3:10 ` [PATCH v5 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein
2022-06-30 3:10 ` [PATCH v5 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
2022-06-30 3:44 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Noah Goldstein
2022-06-30 3:44 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
2022-06-30 7:47 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 Andreas Schwab
2022-06-30 20:13 ` Noah Goldstein
2022-06-30 20:13 ` Noah Goldstein
2022-06-30 20:13 ` [PATCH v6 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
2022-07-01 22:35 ` H.J. Lu
2022-06-30 23:20 ` [PATCH v6 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu
2022-07-01 0:01 ` Noah Goldstein
2022-07-01 0:01 ` Noah Goldstein
2022-07-01 22:38 ` H.J. Lu
2022-07-01 22:52 ` Noah Goldstein
2022-07-03 18:00 ` H.J. Lu
2022-07-04 4:28 ` Noah Goldstein
2022-07-04 4:28 ` [PATCH v7 " Noah Goldstein
2022-07-04 4:28 ` [PATCH v7 2/2] x86: Add support for building str{c|p}{brk|spn} with explicit ISA level Noah Goldstein
2022-07-05 15:41 ` [PATCH v7 1/2] x86: Add comment explaining no Slow_SSE4_2 check in ifunc-sse4_2 H.J. Lu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).