public inbox for libc-alpha@sourceware.org
 help / color / mirror / Atom feed
From: "H.J. Lu" <hjl.tools@gmail.com>
To: Noah Goldstein <goldstein.w.n@gmail.com>
Cc: libc-alpha@sourceware.org, carlos@systemhalted.org
Subject: Re: [PATCH v2 4/4] x86: Add optimized functions for the wide-character strcpy family
Date: Fri, 4 Nov 2022 09:47:01 -0700	[thread overview]
Message-ID: <CAMe9rOpnw6MoFhUCWfFcYVMO2t-VU5wiCLsF9e01gb0P1KN1QQ@mail.gmail.com> (raw)
In-Reply-To: <20221104082024.205104-4-goldstein.w.n@gmail.com>

On Fri, Nov 4, 2022 at 1:20 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
>
> Implemented:
>     wcscat-avx2{+rtm}
>     wcscpy-avx2{+rtm}
>     wcpcpy-avx2{+rtm}
>     wcsncpy-avx2{+rtm}
>     wcpncpy-avx2{+rtm}
>     wcsncat-avx2{+rtm}
>     wcscat-evex
>     wcscpy-evex
>     wcpcpy-evex
>     wcsncpy-evex
>     wcpncpy-evex
>     wcsncat-evex
>
> Performance Changes:
>     Times are from N = 10 runs of the benchmark suite and are reported
>     as geometric mean of all ratios of New Implementation / Best Old
>     Implementation. Best Old Implementation was determined with the
>     highest ISA implementation.
>
>     wcscat-avx2     -> 0.975
>     wcscpy-avx2     -> 0.591
>     wcpcpy-avx2     -> 0.698
>     wcsncpy-avx2    -> 0.730
>     wcpncpy-avx2    -> 0.711
>     wcsncat-avx2    -> 0.954
>     wcscat-evex     -> 0.991
>     wcscpy-evex     -> 0.587
>     wcpcpy-evex     -> 0.695
>     wcsncpy-evex    -> 0.719
>     wcpncpy-evex    -> 0.694
>     wcsncat-evex    -> 0.979
>
> Code Size Changes:
>     This change (compared with the last two commits without it)
>     increase the size of libc.so by ~19kb bytes. For reference this
>     entire patchset increases libc.so by ~2.5kb (so without the
>     wide-character functions libc.so would decrease by 16.5kb).
>
> Full check passes on x86-64 and build succeeds for all ISA levels w/
> and w/o multiarch.

Please separate AVX2 and EVEX to reduce the patch size.

> ---
>  sysdeps/x86_64/Makefile                     |   5 +
>  sysdeps/x86_64/multiarch/Makefile           |  26 +++-
>  sysdeps/x86_64/multiarch/ifunc-impl-list.c  | 135 +++++++++++++++++++-
>  sysdeps/x86_64/multiarch/ifunc-wcs.h        |  60 +++++++++
>  sysdeps/x86_64/multiarch/wcpcpy-avx2-rtm.S  |   3 +
>  sysdeps/x86_64/multiarch/wcpcpy-avx2.S      |   8 ++
>  sysdeps/x86_64/multiarch/wcpcpy-evex.S      |   8 ++
>  sysdeps/x86_64/multiarch/wcpcpy-generic.c   |  27 ++++
>  sysdeps/x86_64/multiarch/wcpcpy.c           |  37 ++++++
>  sysdeps/x86_64/multiarch/wcpncpy-avx2-rtm.S |   3 +
>  sysdeps/x86_64/multiarch/wcpncpy-avx2.S     |   8 ++
>  sysdeps/x86_64/multiarch/wcpncpy-evex.S     |   8 ++
>  sysdeps/x86_64/multiarch/wcpncpy-generic.c  |  27 ++++
>  sysdeps/x86_64/multiarch/wcpncpy.c          |  37 ++++++
>  sysdeps/x86_64/multiarch/wcscat-avx2-rtm.S  |   3 +
>  sysdeps/x86_64/multiarch/wcscat-avx2.S      |  10 ++
>  sysdeps/x86_64/multiarch/wcscat-evex.S      |   9 ++
>  sysdeps/x86_64/multiarch/wcscat-generic.c   |  27 ++++
>  sysdeps/x86_64/multiarch/wcscat.c           |  37 ++++++
>  sysdeps/x86_64/multiarch/wcscpy-avx2-rtm.S  |   3 +
>  sysdeps/x86_64/multiarch/wcscpy-avx2.S      |   7 +
>  sysdeps/x86_64/multiarch/wcscpy-evex.S      |   7 +
>  sysdeps/x86_64/multiarch/wcscpy-generic.c   |   3 +-
>  sysdeps/x86_64/multiarch/wcscpy.c           |  21 +++
>  sysdeps/x86_64/multiarch/wcsncat-avx2-rtm.S |   3 +
>  sysdeps/x86_64/multiarch/wcsncat-avx2.S     |   9 ++
>  sysdeps/x86_64/multiarch/wcsncat-evex.S     |   9 ++
>  sysdeps/x86_64/multiarch/wcsncat-generic.c  |  27 ++++
>  sysdeps/x86_64/multiarch/wcsncat.c          |  34 +++++
>  sysdeps/x86_64/multiarch/wcsncpy-avx2-rtm.S |   3 +
>  sysdeps/x86_64/multiarch/wcsncpy-avx2.S     |   7 +
>  sysdeps/x86_64/multiarch/wcsncpy-evex.S     |   7 +
>  sysdeps/x86_64/multiarch/wcsncpy-generic.c  |  27 ++++
>  sysdeps/x86_64/multiarch/wcsncpy.c          |  37 ++++++
>  sysdeps/x86_64/wcpcpy-generic.c             |  31 +++++
>  sysdeps/x86_64/wcpcpy.S                     |  41 ++++++
>  sysdeps/x86_64/wcpncpy-generic.c            |  31 +++++
>  sysdeps/x86_64/wcpncpy.S                    |  41 ++++++
>  sysdeps/x86_64/wcscat-generic.c             |  31 +++++
>  sysdeps/x86_64/wcscat.S                     |  41 ++++++
>  sysdeps/x86_64/wcscpy.S                     |   2 +
>  sysdeps/x86_64/wcsncat-generic.c            |  31 +++++
>  sysdeps/x86_64/wcsncat.S                    |  39 ++++++
>  sysdeps/x86_64/wcsncpy-generic.c            |  31 +++++
>  sysdeps/x86_64/wcsncpy.S                    |  41 ++++++
>  45 files changed, 1036 insertions(+), 6 deletions(-)
>  create mode 100644 sysdeps/x86_64/multiarch/ifunc-wcs.h
>  create mode 100644 sysdeps/x86_64/multiarch/wcpcpy-avx2-rtm.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcpcpy-avx2.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcpcpy-evex.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcpcpy-generic.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcpcpy.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcpncpy-avx2-rtm.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcpncpy-avx2.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcpncpy-evex.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcpncpy-generic.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcpncpy.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcscat-avx2-rtm.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcscat-avx2.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcscat-evex.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcscat-generic.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcscat.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcscpy-avx2-rtm.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcscpy-avx2.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcscpy-evex.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncat-avx2-rtm.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncat-avx2.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncat-evex.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncat-generic.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncat.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncpy-avx2-rtm.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncpy-avx2.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncpy-evex.S
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncpy-generic.c
>  create mode 100644 sysdeps/x86_64/multiarch/wcsncpy.c
>  create mode 100644 sysdeps/x86_64/wcpcpy-generic.c
>  create mode 100644 sysdeps/x86_64/wcpcpy.S
>  create mode 100644 sysdeps/x86_64/wcpncpy-generic.c
>  create mode 100644 sysdeps/x86_64/wcpncpy.S
>  create mode 100644 sysdeps/x86_64/wcscat-generic.c
>  create mode 100644 sysdeps/x86_64/wcscat.S
>  create mode 100644 sysdeps/x86_64/wcsncat-generic.c
>  create mode 100644 sysdeps/x86_64/wcsncat.S
>  create mode 100644 sysdeps/x86_64/wcsncpy-generic.c
>  create mode 100644 sysdeps/x86_64/wcsncpy.S
>
> diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile
> index 3627c5659f..688eb2d7c4 100644
> --- a/sysdeps/x86_64/Makefile
> +++ b/sysdeps/x86_64/Makefile
> @@ -188,8 +188,13 @@ endif
>  ifeq ($(subdir),wcsmbs)
>
>  sysdep_routines += \
> +  wcpcpy-generic \
> +  wcpncpy-generic \
> +  wcscat-generic \
>    wcscpy-generic \
> +  wcsncat-generic \
>    wcsncmp-generic \
> +  wcsncpy-generic \
>    wcsnlen-generic \
>  # sysdep_routines
>
> diff --git a/sysdeps/x86_64/multiarch/Makefile b/sysdeps/x86_64/multiarch/Makefile
> index 066bfa48d9..f848fc0e28 100644
> --- a/sysdeps/x86_64/multiarch/Makefile
> +++ b/sysdeps/x86_64/multiarch/Makefile
> @@ -131,6 +131,18 @@ endif
>
>  ifeq ($(subdir),wcsmbs)
>  sysdep_routines += \
> +  wcpcpy-avx2 \
> +  wcpcpy-avx2-rtm \
> +  wcpcpy-evex \
> +  wcpcpy-generic \
> +  wcpncpy-avx2 \
> +  wcpncpy-avx2-rtm \
> +  wcpncpy-evex \
> +  wcpncpy-generic \
> +  wcscat-avx2 \
> +  wcscat-avx2-rtm \
> +  wcscat-evex \
> +  wcscat-generic \
>    wcschr-avx2 \
>    wcschr-avx2-rtm \
>    wcschr-evex \
> @@ -140,6 +152,10 @@ sysdep_routines += \
>    wcscmp-avx2-rtm \
>    wcscmp-evex \
>    wcscmp-sse2 \
> +  wcscpy-avx2 \
> +  wcscpy-avx2-rtm \
> +  wcscpy-evex \
> +  wcscpy-generic \
>    wcscpy-ssse3 \
>    wcslen-avx2 \
>    wcslen-avx2-rtm \
> @@ -147,9 +163,17 @@ sysdep_routines += \
>    wcslen-evex512 \
>    wcslen-sse2 \
>    wcslen-sse4_1 \
> +  wcsncat-avx2 \
> +  wcsncat-avx2-rtm \
> +  wcsncat-evex \
> +  wcsncat-generic \
>    wcsncmp-avx2 \
>    wcsncmp-avx2-rtm \
>    wcsncmp-evex \
> +  wcsncpy-avx2 \
> +  wcsncpy-avx2-rtm \
> +  wcsncpy-evex \
> +  wcsncpy-generic \
>    wcsnlen-avx2 \
>    wcsnlen-avx2-rtm \
>    wcsnlen-evex \
> @@ -163,8 +187,8 @@ sysdep_routines += \
>    wmemchr-avx2 \
>    wmemchr-avx2-rtm \
>    wmemchr-evex \
> -  wmemchr-evex512 \
>    wmemchr-evex-rtm \
> +  wmemchr-evex512 \
>    wmemchr-sse2 \
>    wmemcmp-avx2-movbe \
>    wmemcmp-avx2-movbe-rtm \
> diff --git a/sysdeps/x86_64/multiarch/ifunc-impl-list.c b/sysdeps/x86_64/multiarch/ifunc-impl-list.c
> index 7cebee7ec7..71e8953e91 100644
> --- a/sysdeps/x86_64/multiarch/ifunc-impl-list.c
> +++ b/sysdeps/x86_64/multiarch/ifunc-impl-list.c
> @@ -901,16 +901,145 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
>
>    /* Support sysdeps/x86_64/multiarch/wcscpy.c.  */
>    IFUNC_IMPL (i, name, wcscpy,
> -             /* ISA V4 wrapper for SSSE3 implementation because
> -                the SSSE3 implementation is also used at ISA
> -                level 3/4.  */
>               X86_IFUNC_IMPL_ADD_V4 (array, i, wcscpy,
> +                                    (CPU_FEATURE_USABLE (AVX512VL)
> +                                     && CPU_FEATURE_USABLE (AVX512BW)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcscpy_evex)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcscpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcscpy_avx2)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcscpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)
> +                                     && CPU_FEATURE_USABLE (RTM)),
> +                                    __wcscpy_avx2_rtm)
> +             X86_IFUNC_IMPL_ADD_V2 (array, i, wcscpy,
>                                      CPU_FEATURE_USABLE (SSSE3),
>                                      __wcscpy_ssse3)
>               X86_IFUNC_IMPL_ADD_V1 (array, i, wcscpy,
>                                      1,
>                                      __wcscpy_generic))
>
> +  /* Support sysdeps/x86_64/multiarch/wcsncpy.c.  */
> +  IFUNC_IMPL (i, name, wcsncpy,
> +             X86_IFUNC_IMPL_ADD_V4 (array, i, wcsncpy,
> +                                    (CPU_FEATURE_USABLE (AVX512VL)
> +                                     && CPU_FEATURE_USABLE (AVX512BW)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcsncpy_evex)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcsncpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcsncpy_avx2)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcsncpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)
> +                                     && CPU_FEATURE_USABLE (RTM)),
> +                                    __wcsncpy_avx2_rtm)
> +             X86_IFUNC_IMPL_ADD_V1 (array, i, wcpncpy,
> +                                    1,
> +                                    __wcsncpy_generic))
> +
> +  /* Support sysdeps/x86_64/multiarch/wcpcpy.c.  */
> +  IFUNC_IMPL (i, name, wcpcpy,
> +             X86_IFUNC_IMPL_ADD_V4 (array, i, wcpcpy,
> +                                    (CPU_FEATURE_USABLE (AVX512VL)
> +                                     && CPU_FEATURE_USABLE (AVX512BW)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcpcpy_evex)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcpcpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcpcpy_avx2)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcpcpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)
> +                                     && CPU_FEATURE_USABLE (RTM)),
> +                                    __wcpcpy_avx2_rtm)
> +             X86_IFUNC_IMPL_ADD_V1 (array, i, wcpcpy,
> +                                    1,
> +                                    __wcpcpy_generic))
> +
> +  /* Support sysdeps/x86_64/multiarch/wcpncpy.c.  */
> +  IFUNC_IMPL (i, name, wcpncpy,
> +             X86_IFUNC_IMPL_ADD_V4 (array, i, wcpncpy,
> +                                    (CPU_FEATURE_USABLE (AVX512VL)
> +                                     && CPU_FEATURE_USABLE (AVX512BW)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcpncpy_evex)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcpncpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcpncpy_avx2)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcpncpy,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)
> +                                     && CPU_FEATURE_USABLE (RTM)),
> +                                    __wcpncpy_avx2_rtm)
> +             X86_IFUNC_IMPL_ADD_V1 (array, i, wcsncpy,
> +                                    1,
> +                                    __wcpncpy_generic))
> +
> +  /* Support sysdeps/x86_64/multiarch/wcscat.c.  */
> +  IFUNC_IMPL (i, name, wcscat,
> +             X86_IFUNC_IMPL_ADD_V4 (array, i, wcscat,
> +                                    (CPU_FEATURE_USABLE (AVX512VL)
> +                                     && CPU_FEATURE_USABLE (AVX512BW)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcscat_evex)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcscat,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcscat_avx2)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcscat,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)
> +                                     && CPU_FEATURE_USABLE (RTM)),
> +                                    __wcscat_avx2_rtm)
> +             X86_IFUNC_IMPL_ADD_V1 (array, i, wcscat,
> +                                    1,
> +                                    __wcscat_generic))
> +
> +  /* Support sysdeps/x86_64/multiarch/wcsncat.c.  */
> +  IFUNC_IMPL (i, name, wcsncat,
> +             X86_IFUNC_IMPL_ADD_V4 (array, i, wcsncat,
> +                                    (CPU_FEATURE_USABLE (AVX512VL)
> +                                     && CPU_FEATURE_USABLE (AVX512BW)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcsncat_evex)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcsncat,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)),
> +                                    __wcsncat_avx2)
> +             X86_IFUNC_IMPL_ADD_V3 (array, i, wcsncat,
> +                                    (CPU_FEATURE_USABLE (AVX2)
> +                                     && CPU_FEATURE_USABLE (BMI1)
> +                                     && CPU_FEATURE_USABLE (BMI2)
> +                                     && CPU_FEATURE_USABLE (RTM)),
> +                                    __wcsncat_avx2_rtm)
> +             X86_IFUNC_IMPL_ADD_V1 (array, i, wcsncat,
> +                                    1,
> +                                    __wcsncat_generic))
> +
>    /* Support sysdeps/x86_64/multiarch/wcslen.c.  */
>    IFUNC_IMPL (i, name, wcslen,
>               X86_IFUNC_IMPL_ADD_V4 (array, i, wcslen,
> diff --git a/sysdeps/x86_64/multiarch/ifunc-wcs.h b/sysdeps/x86_64/multiarch/ifunc-wcs.h
> new file mode 100644
> index 0000000000..cda633d8fb
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/ifunc-wcs.h
> @@ -0,0 +1,60 @@
> +/* Common definition for ifunc selections optimized wide-character
> +   string copy functions.
> +
> +   All versions must be listed in ifunc-impl-list.c.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <init-arch.h>
> +
> +#ifndef GENERIC
> +# define GENERIC generic
> +#endif
> +
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (evex) attribute_hidden;
> +
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2) attribute_hidden;
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2_rtm) attribute_hidden;
> +
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (GENERIC) attribute_hidden;
> +
> +static inline void *
> +IFUNC_SELECTOR (void)
> +{
> +  const struct cpu_features *cpu_features = __get_cpu_features ();
> +
> +  if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2)
> +      && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, BMI1)
> +      && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, BMI2)
> +      && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features,
> +                                     AVX_Fast_Unaligned_Load, ))
> +    {
> +      if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX512VL)
> +         && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX512BW))
> +       return OPTIMIZE (evex);
> +
> +      if (CPU_FEATURE_USABLE_P (cpu_features, RTM))
> +       return OPTIMIZE (avx2_rtm);
> +
> +      if (X86_ISA_CPU_FEATURES_ARCH_P (cpu_features,
> +                                      Prefer_No_VZEROUPPER, !))
> +       return OPTIMIZE (avx2);
> +
> +    }
> +
> +  return OPTIMIZE (GENERIC);
> +}
> diff --git a/sysdeps/x86_64/multiarch/wcpcpy-avx2-rtm.S b/sysdeps/x86_64/multiarch/wcpcpy-avx2-rtm.S
> new file mode 100644
> index 0000000000..756280a3ab
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpcpy-avx2-rtm.S
> @@ -0,0 +1,3 @@
> +#define WCPCPY __wcpcpy_avx2_rtm
> +#include "x86-avx2-rtm-vecs.h"
> +#include "wcpcpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcpcpy-avx2.S b/sysdeps/x86_64/multiarch/wcpcpy-avx2.S
> new file mode 100644
> index 0000000000..0fffd912d3
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpcpy-avx2.S
> @@ -0,0 +1,8 @@
> +#ifndef WCPCPY
> +# define WCPCPY        __wcpcpy_avx2
> +#endif
> +
> +#define USE_AS_STPCPY
> +#define USE_AS_WCSCPY
> +#define STRCPY WCPCPY
> +#include "strcpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcpcpy-evex.S b/sysdeps/x86_64/multiarch/wcpcpy-evex.S
> new file mode 100644
> index 0000000000..ac6429cc07
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpcpy-evex.S
> @@ -0,0 +1,8 @@
> +#ifndef WCPCPY
> +# define WCPCPY        __wcpcpy_evex
> +#endif
> +
> +#define USE_AS_STPCPY
> +#define USE_AS_WCSCPY
> +#define STRCPY WCPCPY
> +#include "strcpy-evex.S"
> diff --git a/sysdeps/x86_64/multiarch/wcpcpy-generic.c b/sysdeps/x86_64/multiarch/wcpcpy-generic.c
> new file mode 100644
> index 0000000000..0ba29b081f
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpcpy-generic.c
> @@ -0,0 +1,27 @@
> +/* wcpcpy.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* We always need to build this implementation as strspn-sse4 needs to
> +   be able to fallback to it.  */
> +#include <isa-level.h>
> +#if ISA_SHOULD_BUILD (2)
> +
> +# define WCPCPY __wcpcpy_generic
> +# include <wcsmbs/wcpcpy.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcpcpy.c b/sysdeps/x86_64/multiarch/wcpcpy.c
> new file mode 100644
> index 0000000000..8f96ddbc99
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpcpy.c
> @@ -0,0 +1,37 @@
> +/* Multiple versions of wcpcpy.
> +   All versions must be listed in ifunc-impl-list.c.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* Define multiple versions only for the definition in libc.  */
> +#if IS_IN (libc)
> +# define __wcpcpy __redirect_wcpcpy
> +# include <wchar.h>
> +# undef __wcpcpy
> +
> +# define SYMBOL_NAME wcpcpy
> +# include <init-arch.h>
> +
> +# include "ifunc-wcs.h"
> +
> +libc_ifunc_redirected (__redirect_wcpcpy, __wcpcpy, IFUNC_SELECTOR ());
> +weak_alias (__wcpcpy, wcpcpy)
> +# ifdef SHARED
> +__hidden_ver1 (__wcpcpy, __GI___wcpcpy, __redirect_wcpcpy)
> +  __attribute__((visibility ("hidden"))) __attribute_copy__ (wcpcpy);
> +# endif
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcpncpy-avx2-rtm.S b/sysdeps/x86_64/multiarch/wcpncpy-avx2-rtm.S
> new file mode 100644
> index 0000000000..80600d6b01
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpncpy-avx2-rtm.S
> @@ -0,0 +1,3 @@
> +#define WCPNCPY        __wcpncpy_avx2_rtm
> +#include "x86-avx2-rtm-vecs.h"
> +#include "wcpncpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcpncpy-avx2.S b/sysdeps/x86_64/multiarch/wcpncpy-avx2.S
> new file mode 100644
> index 0000000000..b7e594f7b7
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpncpy-avx2.S
> @@ -0,0 +1,8 @@
> +#ifndef WCPNCPY
> +# define WCPNCPY       __wcpncpy_avx2
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define USE_AS_STPCPY
> +#define STRNCPY        WCPNCPY
> +#include "strncpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcpncpy-evex.S b/sysdeps/x86_64/multiarch/wcpncpy-evex.S
> new file mode 100644
> index 0000000000..62ddb694fe
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpncpy-evex.S
> @@ -0,0 +1,8 @@
> +#ifndef WCPNCPY
> +# define WCPNCPY       __wcpncpy_evex
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define USE_AS_STPCPY
> +#define STRNCPY        WCPNCPY
> +#include "strncpy-evex.S"
> diff --git a/sysdeps/x86_64/multiarch/wcpncpy-generic.c b/sysdeps/x86_64/multiarch/wcpncpy-generic.c
> new file mode 100644
> index 0000000000..4aab4ecdd2
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpncpy-generic.c
> @@ -0,0 +1,27 @@
> +/* wcpncpy.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* We always need to build this implementation as strspn-sse4 needs to
> +   be able to fallback to it.  */
> +#include <isa-level.h>
> +#if ISA_SHOULD_BUILD (2)
> +
> +# define WCPNCPY __wcpncpy_generic
> +# include <wcsmbs/wcpncpy.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcpncpy.c b/sysdeps/x86_64/multiarch/wcpncpy.c
> new file mode 100644
> index 0000000000..ed8f307e07
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcpncpy.c
> @@ -0,0 +1,37 @@
> +/* Multiple versions of wcpncpy.
> +   All versions must be listed in ifunc-impl-list.c.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* Define multiple versions only for the definition in libc.  */
> +#if IS_IN (libc)
> +# define __wcpncpy __redirect_wcpncpy
> +# include <wchar.h>
> +# undef __wcpncpy
> +
> +# define SYMBOL_NAME wcpncpy
> +# include <init-arch.h>
> +
> +# include "ifunc-wcs.h"
> +
> +libc_ifunc_redirected (__redirect_wcpncpy, __wcpncpy, IFUNC_SELECTOR ());
> +weak_alias (__wcpncpy, wcpncpy)
> +# ifdef SHARED
> +__hidden_ver1 (__wcpncpy, __GI___wcpncpy, __redirect_wcpncpy)
> +  __attribute__((visibility ("hidden"))) __attribute_copy__ (wcpncpy);
> +# endif
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcscat-avx2-rtm.S b/sysdeps/x86_64/multiarch/wcscat-avx2-rtm.S
> new file mode 100644
> index 0000000000..e99449a2dc
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscat-avx2-rtm.S
> @@ -0,0 +1,3 @@
> +#define WCSCAT __wcscat_avx2_rtm
> +#include "x86-avx2-rtm-vecs.h"
> +#include "wcscat-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcscat-avx2.S b/sysdeps/x86_64/multiarch/wcscat-avx2.S
> new file mode 100644
> index 0000000000..a20f23c09d
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscat-avx2.S
> @@ -0,0 +1,10 @@
> +#ifndef WCSCAT
> +# define WCSCAT        __wcscat_avx2
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define USE_AS_STRCAT
> +
> +#define STRCPY WCSCAT
> +
> +#include "strcpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcscat-evex.S b/sysdeps/x86_64/multiarch/wcscat-evex.S
> new file mode 100644
> index 0000000000..1d017e4899
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscat-evex.S
> @@ -0,0 +1,9 @@
> +#ifndef WCSCAT
> +# define WCSCAT        __wcscat_evex
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define USE_AS_STRCAT
> +
> +#define STRCPY WCSCAT
> +#include "strcpy-evex.S"
> diff --git a/sysdeps/x86_64/multiarch/wcscat-generic.c b/sysdeps/x86_64/multiarch/wcscat-generic.c
> new file mode 100644
> index 0000000000..6476f85bbb
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscat-generic.c
> @@ -0,0 +1,27 @@
> +/* wcscat.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* We always need to build this implementation as strspn-sse4 needs to
> +   be able to fallback to it.  */
> +#include <isa-level.h>
> +#if ISA_SHOULD_BUILD (2)
> +
> +# define WCSCAT __wcscat_generic
> +# include <wcsmbs/wcscat.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcscat.c b/sysdeps/x86_64/multiarch/wcscat.c
> new file mode 100644
> index 0000000000..3277c44561
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscat.c
> @@ -0,0 +1,37 @@
> +/* Multiple versions of wcscat.
> +   All versions must be listed in ifunc-impl-list.c.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* Define multiple versions only for the definition in libc.  */
> +#if IS_IN (libc)
> +# define __wcscat __redirect_wcscat
> +# include <wchar.h>
> +# undef __wcscat
> +
> +# define SYMBOL_NAME wcscat
> +# include <init-arch.h>
> +
> +# include "ifunc-wcs.h"
> +
> +libc_ifunc_redirected (__redirect_wcscat, __wcscat, IFUNC_SELECTOR ());
> +weak_alias (__wcscat, wcscat)
> +# ifdef SHARED
> +__hidden_ver1 (__wcscat, __GI___wcscat, __redirect_wcscat)
> +  __attribute__((visibility ("hidden"))) __attribute_copy__ (wcscat);
> +# endif
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcscpy-avx2-rtm.S b/sysdeps/x86_64/multiarch/wcscpy-avx2-rtm.S
> new file mode 100644
> index 0000000000..2f800c8d3e
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscpy-avx2-rtm.S
> @@ -0,0 +1,3 @@
> +#define WCSCPY __wcscpy_avx2_rtm
> +#include "x86-avx2-rtm-vecs.h"
> +#include "wcscpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcscpy-avx2.S b/sysdeps/x86_64/multiarch/wcscpy-avx2.S
> new file mode 100644
> index 0000000000..6bc509da07
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscpy-avx2.S
> @@ -0,0 +1,7 @@
> +#ifndef WCSCPY
> +# define WCSCPY        __wcscpy_avx2
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define STRCPY WCSCPY
> +#include "strcpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcscpy-evex.S b/sysdeps/x86_64/multiarch/wcscpy-evex.S
> new file mode 100644
> index 0000000000..1069a8e224
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcscpy-evex.S
> @@ -0,0 +1,7 @@
> +#ifndef WCSCPY
> +# define WCSCPY        __wcscpy_evex
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define STRCPY WCSCPY
> +#include "strcpy-evex.S"
> diff --git a/sysdeps/x86_64/multiarch/wcscpy-generic.c b/sysdeps/x86_64/multiarch/wcscpy-generic.c
> index 93d314aaad..600d606c45 100644
> --- a/sysdeps/x86_64/multiarch/wcscpy-generic.c
> +++ b/sysdeps/x86_64/multiarch/wcscpy-generic.c
> @@ -18,8 +18,7 @@
>
>
>  #include <isa-level.h>
> -
> -#if ISA_SHOULD_BUILD (1)
> +#if ISA_SHOULD_BUILD (2)
>
>  # define WCSCPY  __wcscpy_generic
>  # include <wcsmbs/wcscpy.c>
> diff --git a/sysdeps/x86_64/multiarch/wcscpy.c b/sysdeps/x86_64/multiarch/wcscpy.c
> index 92c917b6b4..7f6387817b 100644
> --- a/sysdeps/x86_64/multiarch/wcscpy.c
> +++ b/sysdeps/x86_64/multiarch/wcscpy.c
> @@ -26,6 +26,11 @@
>  # define SYMBOL_NAME wcscpy
>  # include <init-arch.h>
>
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (evex) attribute_hidden;
> +
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2) attribute_hidden;
> +extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2_rtm) attribute_hidden;
> +
>  extern __typeof (REDIRECT_NAME) OPTIMIZE (ssse3) attribute_hidden;
>
>  extern __typeof (REDIRECT_NAME) OPTIMIZE (generic) attribute_hidden;
> @@ -35,6 +40,22 @@ IFUNC_SELECTOR (void)
>  {
>    const struct cpu_features* cpu_features = __get_cpu_features ();
>
> +  if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2)
> +      && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, BMI1)
> +      && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, BMI2)
> +      && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, AVX_Fast_Unaligned_Load, ))
> +    {
> +      if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX512VL)
> +         && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX512BW))
> +       return OPTIMIZE (evex);
> +
> +      if (CPU_FEATURE_USABLE_P (cpu_features, RTM))
> +       return OPTIMIZE (avx2_rtm);
> +
> +      if (X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER, !))
> +       return OPTIMIZE (avx2);
> +    }
> +
>    if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, SSSE3))
>      return OPTIMIZE (ssse3);
>
> diff --git a/sysdeps/x86_64/multiarch/wcsncat-avx2-rtm.S b/sysdeps/x86_64/multiarch/wcsncat-avx2-rtm.S
> new file mode 100644
> index 0000000000..609d6e69c0
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncat-avx2-rtm.S
> @@ -0,0 +1,3 @@
> +#define WCSNCAT        __wcsncat_avx2_rtm
> +#include "x86-avx2-rtm-vecs.h"
> +#include "wcsncat-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcsncat-avx2.S b/sysdeps/x86_64/multiarch/wcsncat-avx2.S
> new file mode 100644
> index 0000000000..a72105b7e9
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncat-avx2.S
> @@ -0,0 +1,9 @@
> +#ifndef WCSNCAT
> +# define WCSNCAT       __wcsncat_avx2
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define USE_AS_STRCAT
> +
> +#define STRNCAT        WCSNCAT
> +#include "strncat-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcsncat-evex.S b/sysdeps/x86_64/multiarch/wcsncat-evex.S
> new file mode 100644
> index 0000000000..392215950a
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncat-evex.S
> @@ -0,0 +1,9 @@
> +#ifndef WCSCAT
> +# define WCSCAT        __wcsncat_evex
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define USE_AS_STRCAT
> +
> +#define STRNCAT        WCSCAT
> +#include "strncat-evex.S"
> diff --git a/sysdeps/x86_64/multiarch/wcsncat-generic.c b/sysdeps/x86_64/multiarch/wcsncat-generic.c
> new file mode 100644
> index 0000000000..9ced02b35e
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncat-generic.c
> @@ -0,0 +1,27 @@
> +/* wcsncat.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* We always need to build this implementation as strspn-sse4 needs to
> +   be able to fallback to it.  */
> +#include <isa-level.h>
> +#if ISA_SHOULD_BUILD (2)
> +
> +# define WCSNCAT __wcsncat_generic
> +# include <wcsmbs/wcsncat.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcsncat.c b/sysdeps/x86_64/multiarch/wcsncat.c
> new file mode 100644
> index 0000000000..49c46aef08
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncat.c
> @@ -0,0 +1,34 @@
> +/* Multiple versions of wcsncat.
> +   All versions must be listed in ifunc-impl-list.c.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* Define multiple versions only for the definition in libc.  */
> +#if IS_IN (libc)
> +# define wcsncat __redirect_wcsncat
> +# include <wchar.h>
> +# undef wcsncat
> +
> +# define SYMBOL_NAME wcsncat
> +# include "ifunc-wcs.h"
> +
> +libc_ifunc_redirected (__redirect_wcsncat, wcsncat, IFUNC_SELECTOR ());
> +# ifdef SHARED
> +__hidden_ver1 (wcsncat, __GI_wcsncat, __redirect_wcsncat)
> +  __attribute__((visibility ("hidden"))) __attribute_copy__ (wcsncat);
> +# endif
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcsncpy-avx2-rtm.S b/sysdeps/x86_64/multiarch/wcsncpy-avx2-rtm.S
> new file mode 100644
> index 0000000000..cab5a6b820
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncpy-avx2-rtm.S
> @@ -0,0 +1,3 @@
> +#define WCSNCPY        __wcsncpy_avx2_rtm
> +#include "x86-avx2-rtm-vecs.h"
> +#include "wcsncpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcsncpy-avx2.S b/sysdeps/x86_64/multiarch/wcsncpy-avx2.S
> new file mode 100644
> index 0000000000..3a1a8a372c
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncpy-avx2.S
> @@ -0,0 +1,7 @@
> +#ifndef WCSNCPY
> +# define WCSNCPY       __wcsncpy_avx2
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define STRNCPY        WCSNCPY
> +#include "strncpy-avx2.S"
> diff --git a/sysdeps/x86_64/multiarch/wcsncpy-evex.S b/sysdeps/x86_64/multiarch/wcsncpy-evex.S
> new file mode 100644
> index 0000000000..2debb8fd6b
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncpy-evex.S
> @@ -0,0 +1,7 @@
> +#ifndef WCSNCPY
> +# define WCSNCPY       __wcsncpy_evex
> +#endif
> +
> +#define USE_AS_WCSCPY
> +#define STRNCPY        WCSNCPY
> +#include "strncpy-evex.S"
> diff --git a/sysdeps/x86_64/multiarch/wcsncpy-generic.c b/sysdeps/x86_64/multiarch/wcsncpy-generic.c
> new file mode 100644
> index 0000000000..693521713b
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncpy-generic.c
> @@ -0,0 +1,27 @@
> +/* wcsncpy.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* We always need to build this implementation as strspn-sse4 needs to
> +   be able to fallback to it.  */
> +#include <isa-level.h>
> +#if ISA_SHOULD_BUILD (2)
> +
> +# define WCSNCPY __wcsncpy_generic
> +# include <wcsmbs/wcsncpy.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/multiarch/wcsncpy.c b/sysdeps/x86_64/multiarch/wcsncpy.c
> new file mode 100644
> index 0000000000..5b89dd4d27
> --- /dev/null
> +++ b/sysdeps/x86_64/multiarch/wcsncpy.c
> @@ -0,0 +1,37 @@
> +/* Multiple versions of wcsncpy.
> +   All versions must be listed in ifunc-impl-list.c.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* Define multiple versions only for the definition in libc.  */
> +#if IS_IN (libc)
> +# define __wcsncpy __redirect_wcsncpy
> +# include <wchar.h>
> +# undef __wcsncpy
> +
> +# define SYMBOL_NAME wcsncpy
> +# include <init-arch.h>
> +
> +# include "ifunc-wcs.h"
> +
> +libc_ifunc_redirected (__redirect_wcsncpy, __wcsncpy, IFUNC_SELECTOR ());
> +weak_alias (__wcsncpy, wcsncpy)
> +# ifdef SHARED
> +__hidden_ver1 (__wcsncpy, __GI___wcsncpy, __redirect_wcsncpy)
> +  __attribute__((visibility ("hidden"))) __attribute_copy__ (wcsncpy);
> +# endif
> +#endif
> diff --git a/sysdeps/x86_64/wcpcpy-generic.c b/sysdeps/x86_64/wcpcpy-generic.c
> new file mode 100644
> index 0000000000..d52525f288
> --- /dev/null
> +++ b/sysdeps/x86_64/wcpcpy-generic.c
> @@ -0,0 +1,31 @@
> +/* wcpcpy dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcpcpy non-multiarch build is split into two files,
> +   wcpcpy-generic.c and wcpcpy.S. The wcpcpy-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcpcpy-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL <= 3
> +
> +# include <wcsmbs/wcpcpy.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/wcpcpy.S b/sysdeps/x86_64/wcpcpy.S
> new file mode 100644
> index 0000000000..ec32dc070a
> --- /dev/null
> +++ b/sysdeps/x86_64/wcpcpy.S
> @@ -0,0 +1,41 @@
> +/* wcpcpy dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcpcpy non-multiarch build is split into two files,
> +   wcpcpy-generic.c and wcpcpy.S. The wcpcpy-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcpcpy-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL >= 4
> +
> +# define WCPCPY        __wcpcpy
> +
> +# define DEFAULT_IMPL_V4       "multiarch/wcpcpy-evex.S"
> +# define DEFAULT_IMPL_V3       "multiarch/wcpcpy-avx2.S"
> +/* isa-default-impl.h expects DEFAULT_IMPL_V1 to be defined but it
> +   should never be used from here.  */
> +# define DEFAULT_IMPL_V1       "ERROR -- Invalid ISA IMPL"
> +
> +# include "isa-default-impl.h"
> +
> +weak_alias (__wcpcpy, wcpcpy)
> +libc_hidden_def (__wcpcpy)
> +#endif
> diff --git a/sysdeps/x86_64/wcpncpy-generic.c b/sysdeps/x86_64/wcpncpy-generic.c
> new file mode 100644
> index 0000000000..871219a445
> --- /dev/null
> +++ b/sysdeps/x86_64/wcpncpy-generic.c
> @@ -0,0 +1,31 @@
> +/* wcpncpy dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcpncpy non-multiarch build is split into two files,
> +   wcpncpy-generic.c and wcpncpy.S. The wcpncpy-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcpncpy-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL <= 3
> +
> +# include <wcsmbs/wcpncpy.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/wcpncpy.S b/sysdeps/x86_64/wcpncpy.S
> new file mode 100644
> index 0000000000..68e6ff1836
> --- /dev/null
> +++ b/sysdeps/x86_64/wcpncpy.S
> @@ -0,0 +1,41 @@
> +/* wcpncpy dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcpncpy non-multiarch build is split into two files,
> +   wcpncpy-generic.c and wcpncpy.S. The wcpncpy-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcpncpy-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL >= 4
> +
> +# define WCPNCPY       __wcpncpy
> +
> +# define DEFAULT_IMPL_V4       "multiarch/wcpncpy-evex.S"
> +# define DEFAULT_IMPL_V3       "multiarch/wcpncpy-avx2.S"
> +/* isa-default-impl.h expects DEFAULT_IMPL_V1 to be defined but it
> +   should never be used from here.  */
> +# define DEFAULT_IMPL_V1       "ERROR -- Invalid ISA IMPL"
> +
> +# include "isa-default-impl.h"
> +
> +weak_alias (__wcpncpy, wcpncpy)
> +libc_hidden_def (__wcpncpy)
> +#endif
> diff --git a/sysdeps/x86_64/wcscat-generic.c b/sysdeps/x86_64/wcscat-generic.c
> new file mode 100644
> index 0000000000..85f981a81f
> --- /dev/null
> +++ b/sysdeps/x86_64/wcscat-generic.c
> @@ -0,0 +1,31 @@
> +/* wcscat dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcscat non-multiarch build is split into two files,
> +   wcscat-generic.c and wcscat.S. The wcscat-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcscat-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL <= 3
> +
> +# include <wcsmbs/wcscat.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/wcscat.S b/sysdeps/x86_64/wcscat.S
> new file mode 100644
> index 0000000000..007de3c40c
> --- /dev/null
> +++ b/sysdeps/x86_64/wcscat.S
> @@ -0,0 +1,41 @@
> +/* wcscat dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcscat non-multiarch build is split into two files,
> +   wcscat-generic.c and wcscat.S. The wcscat-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcscat-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL >= 4
> +
> +# define WCSCAT        __wcscat
> +
> +# define DEFAULT_IMPL_V4       "multiarch/wcscat-evex.S"
> +# define DEFAULT_IMPL_V3       "multiarch/wcscat-avx2.S"
> +/* isa-default-impl.h expects DEFAULT_IMPL_V1 to be defined but it
> +   should never be used from here.  */
> +# define DEFAULT_IMPL_V1       "ERROR -- Invalid ISA IMPL"
> +
> +# include "isa-default-impl.h"
> +
> +weak_alias (__wcscat, wcscat)
> +libc_hidden_def (__wcscat)
> +#endif
> diff --git a/sysdeps/x86_64/wcscpy.S b/sysdeps/x86_64/wcscpy.S
> index 11d0bb4bab..ab9288ed74 100644
> --- a/sysdeps/x86_64/wcscpy.S
> +++ b/sysdeps/x86_64/wcscpy.S
> @@ -28,6 +28,8 @@
>
>  # define WCSCPY        __wcscpy
>
> +# define DEFAULT_IMPL_V4       "multiarch/wcscpy-evex.S"
> +# define DEFAULT_IMPL_V3       "multiarch/wcscpy-avx2.S"
>  # define DEFAULT_IMPL_V2       "multiarch/wcscpy-ssse3.S"
>  /* isa-default-impl.h expects DEFAULT_IMPL_V1 to be defined but it
>     should never be used from here.  */
> diff --git a/sysdeps/x86_64/wcsncat-generic.c b/sysdeps/x86_64/wcsncat-generic.c
> new file mode 100644
> index 0000000000..2cc0f7b11a
> --- /dev/null
> +++ b/sysdeps/x86_64/wcsncat-generic.c
> @@ -0,0 +1,31 @@
> +/* wcsncat dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcsncat non-multiarch build is split into two files,
> +   wcsncat-generic.c and wcsncat.S. The wcsncat-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcsncat-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL <= 3
> +
> +# include <wcsmbs/wcsncat.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/wcsncat.S b/sysdeps/x86_64/wcsncat.S
> new file mode 100644
> index 0000000000..3f4c7948db
> --- /dev/null
> +++ b/sysdeps/x86_64/wcsncat.S
> @@ -0,0 +1,39 @@
> +/* wcsncat dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcsncat non-multiarch build is split into two files,
> +   wcsncat-generic.c and wcsncat.S. The wcsncat-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcsncat-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL >= 4
> +
> +# define WCSNCAT       wcsncat
> +
> +# define DEFAULT_IMPL_V4       "multiarch/wcsncat-evex.S"
> +# define DEFAULT_IMPL_V3       "multiarch/wcsncat-avx2.S"
> +/* isa-default-impl.h expects DEFAULT_IMPL_V1 to be defined but it
> +   should never be used from here.  */
> +# define DEFAULT_IMPL_V1       "ERROR -- Invalid ISA IMPL"
> +
> +# include "isa-default-impl.h"
> +
> +#endif
> diff --git a/sysdeps/x86_64/wcsncpy-generic.c b/sysdeps/x86_64/wcsncpy-generic.c
> new file mode 100644
> index 0000000000..49d06b8ae8
> --- /dev/null
> +++ b/sysdeps/x86_64/wcsncpy-generic.c
> @@ -0,0 +1,31 @@
> +/* wcsncpy dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcsncpy non-multiarch build is split into two files,
> +   wcsncpy-generic.c and wcsncpy.S. The wcsncpy-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcsncpy-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL <= 3
> +
> +# include <wcsmbs/wcsncpy.c>
> +
> +#endif
> diff --git a/sysdeps/x86_64/wcsncpy.S b/sysdeps/x86_64/wcsncpy.S
> new file mode 100644
> index 0000000000..e1428fd4c1
> --- /dev/null
> +++ b/sysdeps/x86_64/wcsncpy.S
> @@ -0,0 +1,41 @@
> +/* wcsncpy dispatch for RTLD and non-multiarch .c ISA level 1 build.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +/* wcsncpy non-multiarch build is split into two files,
> +   wcsncpy-generic.c and wcsncpy.S. The wcsncpy-generic.c build is
> +   for ISA level <= 1 and just uses multiarch/wcsncpy-generic.c.
> +   This must be split into two files because we cannot include C
> +   code from assembly or vice versa.  */
> +
> +#include <isa-level.h>
> +
> +#if MINIMUM_X86_ISA_LEVEL >= 4
> +
> +# define WCSNCPY       __wcsncpy
> +
> +# define DEFAULT_IMPL_V4       "multiarch/wcsncpy-evex.S"
> +# define DEFAULT_IMPL_V3       "multiarch/wcsncpy-avx2.S"
> +/* isa-default-impl.h expects DEFAULT_IMPL_V1 to be defined but it
> +   should never be used from here.  */
> +# define DEFAULT_IMPL_V1       "ERROR -- Invalid ISA IMPL"
> +
> +# include "isa-default-impl.h"
> +
> +weak_alias (__wcsncpy, wcsncpy)
> +libc_hidden_def (__wcsncpy)
> +#endif
> --
> 2.34.1
>


-- 
H.J.

  reply	other threads:[~2022-11-04 16:47 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-03  8:53 [PATCH v1 1/4] benchtests: Make str{n}{cat|cpy} benchmarks output json Noah Goldstein
2022-11-03  8:53 ` [PATCH v1 2/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-evex functions Noah Goldstein
2022-11-03  8:55   ` Noah Goldstein
2022-11-04 23:04   ` [PATCH v4 1/4] " Noah Goldstein
2022-11-04 23:04     ` [PATCH v4 2/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-avx2 functions Noah Goldstein
2022-11-04 23:04     ` [PATCH v4 3/4] x86: Add evex optimized functions for the wchar_t strcpy family Noah Goldstein
2022-11-04 23:04     ` [PATCH v4 4/4] x86: Add avx2 " Noah Goldstein
2022-11-04 23:34     ` [PATCH v4 1/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-evex functions H.J. Lu
2022-11-09  1:38   ` [PATCH v5 " Noah Goldstein
2022-11-09  1:38     ` [PATCH v5 2/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-avx2 functions Noah Goldstein
2022-11-09  3:00       ` H.J. Lu
2022-11-09  1:38     ` [PATCH v5 3/4] x86: Add evex optimized functions for the wchar_t strcpy family Noah Goldstein
2022-11-09  3:01       ` H.J. Lu
2022-11-09  1:38     ` [PATCH v5 4/4] x86: Add avx2 " Noah Goldstein
2022-11-09  3:01       ` H.J. Lu
2022-11-09  3:00     ` [PATCH v5 1/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-evex functions H.J. Lu
2022-11-03  8:53 ` [PATCH v1 3/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-avx2 functions Noah Goldstein
2022-11-03  8:55   ` Noah Goldstein
2022-11-03  8:53 ` [PATCH v1 4/4] x86: Add optimized functions for the wide-character strcpy family Noah Goldstein
2022-11-03  9:06 ` [PATCH v1 1/4] benchtests: Make str{n}{cat|cpy} benchmarks output json Noah Goldstein
2022-11-04  8:20 ` [PATCH v2 " Noah Goldstein
2022-11-04  8:20   ` [PATCH v2 2/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-evex functions Noah Goldstein
2022-11-04 16:33     ` H.J. Lu
2022-11-04 20:20       ` Noah Goldstein
2022-11-04  8:20   ` [PATCH v2 3/4] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-avx2 functions Noah Goldstein
2022-11-04 16:45     ` H.J. Lu
2022-11-04 20:21       ` Noah Goldstein
2022-11-04  8:20   ` [PATCH v2 4/4] x86: Add optimized functions for the wide-character strcpy family Noah Goldstein
2022-11-04 16:47     ` H.J. Lu [this message]
2022-11-04 20:22       ` Noah Goldstein
2022-11-04 16:26   ` [PATCH v2 1/4] benchtests: Make str{n}{cat|cpy} benchmarks output json H.J. Lu
2022-11-04 20:13 ` [PATCH v3 1/5] " Noah Goldstein
2022-11-04 20:13   ` [PATCH v3 2/5] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-evex functions Noah Goldstein
2022-11-04 21:46     ` H.J. Lu
2022-11-04 22:27       ` Noah Goldstein
2022-11-04 22:47         ` H.J. Lu
2022-11-04 23:06           ` Noah Goldstein
2022-11-04 20:13   ` [PATCH v3 3/5] x86: Optimize and shrink st{r|p}{n}{cat|cpy}-avx2 functions Noah Goldstein
2022-11-04 20:13   ` [PATCH v3 4/5] x86: Add evex optimized functions for the wchar_t strcpy family Noah Goldstein
2022-11-04 20:13   ` [PATCH v3 5/5] x86: Add avx2 " Noah Goldstein
2022-11-04 21:01   ` [PATCH v3 1/5] benchtests: Make str{n}{cat|cpy} benchmarks output json H.J. Lu
2022-11-04 21:24     ` Noah Goldstein

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAMe9rOpnw6MoFhUCWfFcYVMO2t-VU5wiCLsF9e01gb0P1KN1QQ@mail.gmail.com \
    --to=hjl.tools@gmail.com \
    --cc=carlos@systemhalted.org \
    --cc=goldstein.w.n@gmail.com \
    --cc=libc-alpha@sourceware.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).