From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by sourceware.org (Postfix) with ESMTPS id 3408D3858D35 for ; Mon, 6 May 2024 13:46:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3408D3858D35 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=vrull.eu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=vrull.eu ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 3408D3858D35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::1036 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715003184; cv=none; b=LITBFV2xaIIG7SuRGj+HtdksI3bT/OjFW28hZ+Ocqr1/FsB2cKggaoVcjkFFr1fDYCXCglrZnVuOxkTR6bx5R7UjRzljxoBx7XVaMHyeeu3LyqFvrZLej5U22SPemigZNujXroBODl4b+S2qzbFM8MJ33kldYbXbETzJfKi8pto= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715003184; c=relaxed/simple; bh=DFWNF7r8duA9dcIlZHs1YkfzbJaG0TgCYcLS4V5FtKw=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=IKMMt7/o16qX6TBTy1C1Zt8b8krnMMtyIJ0HB46F+00SYpxsrYV493tWIYfLN9ipq3bPFeE8GtruzMZQN3ZmPzyIpkLSl918vd3kNenAoILIQZXkBKCu98W127cXv3UZc1k/Oi4JhOGTO6AxtSIlojcmfJwpZBqCkZLhf2jBGDw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-2b5388087f9so694349a91.0 for ; Mon, 06 May 2024 06:46:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1715003180; x=1715607980; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=cQkB+pIVHKBhSH1kGN37EXXzgXThiJT6k3Iezijwj9E=; b=rNZ35vLtRvqvg1xbu4xjG9M2TX/WOz5/eaHdSMnd9XZJdYKuZFNqm4v3Fpp2RJCIZG jFFHV92g99X/b/LDnO5egR0JMMfYFtCDI1OPAr07x6+vXtn469gM27LPoC3pAJiV4cQw xLm3vTLzJyjulFLz+/H2mHJ9ys63pYzMjntTN6JowB35SieHMJOw3+yT9WaKgq4Nj8gu +KqebRLVy0iIcqf/Ah0I6q2IoclK1mposl+O4w4MGSBwapZ3cb4SxnYtGVNWJFAmt9be KPFyTMPtChPg255/TNuhUjmbyvVS4RITsgDwUz54PMrLnIV88yqWHZtkEFrROshnARiC 261g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715003180; x=1715607980; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cQkB+pIVHKBhSH1kGN37EXXzgXThiJT6k3Iezijwj9E=; b=ZgcRWxeUpiLR7GtOfS+Iii/yh81tWQ1lhN+ye4Z4/HLmO43i2mnd/XAvDwXgXClhbB 2J9fir6Q/iyAwe37q0ztHaTNTMht7JAbEW5NuKw4m0t1/mCJR1ukvhl3k85r2Dc+yCyG Y8ZkAe5GK2QbxqNql2txi+J7V9NKMkZuf/fJO6M9MXuB7UPChbnRMI5y488Ejvv/un+i IuE+JDCDpo22hKGyOAGtMkP1EOoYdmrGtHkg/sfzt65gpACQW8Flmik2JW3pvYvF27Mm lDxnpGBAC1b2lXJ6g7cClSXacmncA4TWFNBNrPoWRnSEBy7klqLMdQkJAMAgok9owcIR bcGA== X-Forwarded-Encrypted: i=1; AJvYcCWDvzGiF8Wh9biUEkpmz6uMx72BELUHCDqpm09rHIxCgK4+1sjzfSquog1qLl0vmfWA8GxCInVhgndcDBuegYWRiFbOxYvHe6yq X-Gm-Message-State: AOJu0YwDaxKcFBAvj/lzkGWkVXn+pQ1HGbGbcRqkpxs2uWZXxP3xrt3v Q8o+7yiJtXbn5WYOv2SHaQKmWT+C8q7+5VBMaCMu5+bOAgi56IqxxU0xwVrQdGmlxg+h0Phc8ND 3vV2w/tABL5kiv2T7JVr9rDWk8cAFgmYwXab3GQ== X-Google-Smtp-Source: AGHT+IFd0eKJdL1NhvG9B/9eEgYHbxXeizUOovTPpjorTmV1VgC2WOWpneDh/rWEwvyiAM+HPQ8I4wIKa4QahgnQzac= X-Received: by 2002:a17:90b:38c9:b0:2b4:e4d2:cfb4 with SMTP id nn9-20020a17090b38c900b002b4e4d2cfb4mr4344601pjb.18.1715003179756; Mon, 06 May 2024 06:46:19 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?Q?Christoph_M=C3=BCllner?= Date: Mon, 6 May 2024 15:46:08 +0200 Message-ID: Subject: Re: [PATCH 2/7] RISC-V: Add Zbb optimized memchr as ifunc To: Kito Cheng Cc: Palmer Dabbelt , adhemerval.zanella@linaro.org, libc-alpha@sourceware.org, Darius Rad , Andrew Waterman , philipp.tomsich@vrull.eu, Evan Green , DJ Delorie , Vineet Gupta , jeffreyalaw@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-10.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Mon, May 6, 2024 at 3:32=E2=80=AFPM Kito Cheng w= rote: > > `target` attribute is provided by GCC 14, and `-march=3D+ext` will at > least require GCC 15 (if we add), > also we relaxed the canonical order requirement for `-march=3D`, that > means we can relatively easy to manipulate the ISA > string by just concat that? so I don't really think we need `-march=3D+e= xt`. Then we require the Makefile author to: 1) parse if the current compiler flags include march (repeat for all variables that may be used) 2a) if so, then amend to that 2b) if not, parse if there is a configured default ISA (via "gcc -v") 2ba) if so, then create -march accordingly 2bb) if not, then Bonus problem: If there are multiple variables that might set -march=3D... this gets even trickier because the order how these variables are integrated in the compiler invocation command is unknown. Using "-march=3D+zfoo" seems simpler and cleaner. > > On Mon, May 6, 2024 at 9:20=E2=80=AFPM Christoph M=C3=BCllner > wrote: > > > > On Tue, Apr 30, 2024 at 7:54=E2=80=AFPM Palmer Dabbelt wrote: > > > > > > On Tue, 30 Apr 2024 10:45:12 PDT (-0700), adhemerval.zanella@linaro.o= rg wrote: > > > > > > > > > > > > On 30/04/24 12:13, Palmer Dabbelt wrote: > > > >> On Wed, 24 Apr 2024 06:36:43 PDT (-0700), adhemerval.zanella@linar= o.org wrote: > > > >>> > > > >>> > > > >>> On 24/04/24 10:16, Christoph M=C3=BCllner wrote: > > > >>>> On Wed, Apr 24, 2024 at 2:53=E2=80=AFPM Adhemerval Zanella Netto > > > >>>> wrote: > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> On 22/04/24 04:43, Christoph M=C3=BCllner wrote: > > > >>>>>> When building with Zbb enabled, memchr benefits from using orc= .b in > > > >>>>>> find_zero_all(). This patch changes the build system such, th= at a > > > >>>>>> non-Zbb version as well as a Zbb version of this routine is bu= ilt. > > > >>>>>> Further, a ifunc resolver is provided that selects the right r= outine > > > >>>>>> based on the outcome of extension probing via hwprobe(). > > > >>>>>> > > > >>>>>> Signed-off-by: Christoph M=C3=BCllner > > > >>>>>> --- > > > >>>>>> sysdeps/riscv/multiarch/memchr-generic.c | 26 +++++++++ > > > >>>>>> sysdeps/riscv/multiarch/memchr-zbb.c | 30 ++++++++++ > > > >>>>>> .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + > > > >>>>>> .../linux/riscv/multiarch/ifunc-impl-list.c | 31 ++++++++-- > > > >>>>>> .../unix/sysv/linux/riscv/multiarch/memchr.c | 57 ++++++++++= +++++++++ > > > >>>>>> 5 files changed, 142 insertions(+), 5 deletions(-) > > > >>>>>> create mode 100644 sysdeps/riscv/multiarch/memchr-generic.c > > > >>>>>> create mode 100644 sysdeps/riscv/multiarch/memchr-zbb.c > > > >>>>>> create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/me= mchr.c > > > >>>>>> > > > >>>>>> diff --git a/sysdeps/riscv/multiarch/memchr-generic.c b/sysdep= s/riscv/multiarch/memchr-generic.c > > > >>>>>> new file mode 100644 > > > >>>>>> index 0000000000..a96c36398b > > > >>>>>> --- /dev/null > > > >>>>>> +++ b/sysdeps/riscv/multiarch/memchr-generic.c > > > >>>>>> @@ -0,0 +1,26 @@ > > > >>>>>> +/* Re-include the default memchr implementation. > > > >>>>>> + Copyright (C) 2024 Free Software Foundation, Inc. > > > >>>>>> + This file is part of the GNU C Library. > > > >>>>>> + > > > >>>>>> + The GNU C Library is free software; you can redistribute i= t and/or > > > >>>>>> + modify it under the terms of the GNU Lesser General Public > > > >>>>>> + License as published by the Free Software Foundation; eith= er > > > >>>>>> + version 2.1 of the License, or (at your option) any later = version. > > > >>>>>> + > > > >>>>>> + The GNU C Library is distributed in the hope that it will = be useful, > > > >>>>>> + but WITHOUT ANY WARRANTY; without even the implied warrant= y of > > > >>>>>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See = the GNU > > > >>>>>> + Lesser General Public License for more details. > > > >>>>>> + > > > >>>>>> + You should have received a copy of the GNU Lesser General = Public > > > >>>>>> + License along with the GNU C Library; if not, see > > > >>>>>> + . */ > > > >>>>>> + > > > >>>>>> +#include > > > >>>>>> + > > > >>>>>> +#if IS_IN(libc) > > > >>>>>> +# define MEMCHR __memchr_generic > > > >>>>>> +# undef libc_hidden_builtin_def > > > >>>>>> +# define libc_hidden_builtin_def(x) > > > >>>>>> +#endif > > > >>>>>> +#include > > > >>>>>> diff --git a/sysdeps/riscv/multiarch/memchr-zbb.c b/sysdeps/ri= scv/multiarch/memchr-zbb.c > > > >>>>>> new file mode 100644 > > > >>>>>> index 0000000000..bead0335ae > > > >>>>>> --- /dev/null > > > >>>>>> +++ b/sysdeps/riscv/multiarch/memchr-zbb.c > > > >>>>>> @@ -0,0 +1,30 @@ > > > >>>>>> +/* Re-include the default memchr implementation for Zbb. > > > >>>>>> + Copyright (C) 2024 Free Software Foundation, Inc. > > > >>>>>> + This file is part of the GNU C Library. > > > >>>>>> + > > > >>>>>> + The GNU C Library is free software; you can redistribute i= t and/or > > > >>>>>> + modify it under the terms of the GNU Lesser General Public > > > >>>>>> + License as published by the Free Software Foundation; eith= er > > > >>>>>> + version 2.1 of the License, or (at your option) any later = version. > > > >>>>>> + > > > >>>>>> + The GNU C Library is distributed in the hope that it will = be useful, > > > >>>>>> + but WITHOUT ANY WARRANTY; without even the implied warrant= y of > > > >>>>>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See = the GNU > > > >>>>>> + Lesser General Public License for more details. > > > >>>>>> + > > > >>>>>> + You should have received a copy of the GNU Lesser General = Public > > > >>>>>> + License along with the GNU C Library; if not, see > > > >>>>>> + . */ > > > >>>>>> + > > > >>>>>> +#include > > > >>>>>> + > > > >>>>>> +#if IS_IN(libc) > > > >>>>>> +# define MEMCHR __memchr_zbb > > > >>>>>> +# undef libc_hidden_builtin_def > > > >>>>>> +# define libc_hidden_builtin_def(x) > > > >>>>>> +#endif > > > >>>>>> +/* Convince preprocessor to have Zbb instructions. */ > > > >>>>>> +#ifndef __riscv_zbb > > > >>>>>> +# define __riscv_zbb > > > >>>>>> +#endif > > > >>>>> > > > >>>>> Is there a way to specific the compiler to enable a extension, = like aarch64 > > > >>>>> -march=3Darch{+[no]feature}? I think ideally this should be ena= bled as CFLAGS > > > >>>>> instead of messing with compiler defined pre-processor. > > > >>>> > > > >>>> The tools expect a list of all extensions as parameter to the -m= arch=3D option. > > > >>>> But there is no way to append extensions to an existing march st= ring > > > >>>> on the command line. > > > >>>> > > > >>>> And if we would add this feature today, it would take many years= until we could > > > >>>> use it here, because we want to remain compatible with old tools= . > > > >>>> Or we enable the optimization only when being built with new too= ls, but that > > > >>>> adds even more complexity and build/test configurations. > > > >>>> > > > >>>> What we have is: > > > >>>> * Preprocessor (since forever): Extension test macros (__riscv_E= XTENSION) > > > >>>> * Command line (since forever): -march=3DBASE_EXTENSIONLIST > > > >>>> * GAS (since Nov 21): .option arch, +EXTENSION (in combination w= ith > > > >>>> option push/pop) > > > >>>> * GCC (since Nov 23): __attribute__((target("arch=3D+EXTENSION")= )) > > > >>>> > > > >>>> I was not sure about using __riscv_zbb as well, but I considered= it safe within > > > >>>> ifdef tests that ensure the macro won't be set twice. > > > >>>> If that's a concern, I could change to use something like this: > > > >>>> #define __riscv_force_zbb > > > >>>> #include > > > >>>> #undef __riscv_force_zbb > > > >>>> ... and change string-fza.h like this: > > > >>>> #if defined(__riscv_zbb) || defined(__riscv_force_zbb) > > > >>>> // orc.b > > > >>>> #endif > > > >>>> > > > >>>> BR > > > >>>> Christoph > > > >>> > > > >>> Another options would to parse the current march and add the exte= nsion if required, > > > >>> something like: > > > >>> > > > >>> abi=3D$(riscv64-linux-gnu-gcc -Q --help=3Dtarget | grep march | c= ut -d '=3D' -f2 | xargs) > > > >>> if [[ ! "$abi" =3D~ "_zbb" ]] > > > >>> then > > > >>> abi=3D"$abi"_zbb > > > >>> fi > > > >> > > > >> That alone likely won't do it, there's a bunch of ordering rules i= n the ISA string handling so we might get tripped up on them. We've got a = fairly relaxed version of the rules in GCC to try and match the various old= er rules, though, so it might be possible to make something similar work. > > > >> > > > >> We should probably just add some sort of -march=3D+zbb type argume= nt. IIRC Kito was going to do it at some point, not sure if he got around = to it? > > > > > > > > I am just pointing this out because I think the way RISCV extension= selection > > > > is currently implemented makes it awkward to provide ifunc implemen= tation in > > > > a agnostic way (specially now that RISCV has dozens of extensions) = without > > > > knowing the current target compiler is generating. > > > > > > > > Some other ABI allows to either specify a ISA/chip reference (like = powerpc > > > > with -mcpu=3DpowerX) or a ABI extension directly (like aarch64 with= -march=3D+xxx). > > > > > > We have -mcpu, but for RISC-V that's even more fragmented than -march > > > (-mcpu essentially just rolls together -march and -mtune, and only > > > allows a curated list). The `-march=3D+`-type stuff has been a prett= y > > > common request, I think someone just needst oget around to actually > > > implementing it. > > > > I agree that `-march=3D+` would be the best solution. > > As we don't have it, I've created a PR for the riscv-toolchain-conventi= ons: > > https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/47 > > This will have to depend on GCC 15+ (assuming there will be consensus > > in the next 10 months about it). > > > > Thanks for the proposal with the function-attributes. > > I'm working on a patchset for that and will send it out after some test= ing. > > This will have to depend on GCC 14+. > > > > > >>> I don't have a strong preference, it is just that by not using th= e compiler flag > > > >>> we won't be able to either use the builtin (__builtin_riscv_orc_b= _32) and/or get > > > >>> a possible better code generation from compiler. > > > >> > > > >> I think we'd likely get slightly better codgen from telling the co= mpiler about the bitmanip extensions. Maybe we want something like > > > >> > > > >> diff --git a/string/memchr.c b/string/memchr.c > > > >> index 08b5c41667..1b62dce8d8 100644 > > > >> --- a/string/memchr.c > > > >> +++ b/string/memchr.c > > > >> @@ -29,15 +29,19 @@ > > > >> # define __memchr MEMCHR > > > >> #endif > > > >> +#ifndef __MEMCHR_CODEGEN_ATTRIBUTE > > > >> +#define __MEMCHR_CODEGEN_ATTRIBUTE > > > >> +#endif > > > >> + > > > >> static __always_inline const char * > > > >> -sadd (uintptr_t x, uintptr_t y) > > > >> +sadd (uintptr_t x, uintptr_t y) __MEMCHR_CODEGEN_ATTRIBUTE > > > >> { > > > >> return (const char *)(y > UINTPTR_MAX - x ? UINTPTR_MAX : x = + y); > > > >> } > > > >> /* Search no more than N bytes of S for C. */ > > > >> void * > > > >> -__memchr (void const *s, int c_in, size_t n) > > > >> +__memchr (void const *s, int c_in, size_t n) __MEMCHR_CODEGEN_= ATTRIBUTE > > > >> { > > > >> if (__glibc_unlikely (n =3D=3D 0)) > > > >> return NULL; > > > >> > > > >> in the generic versions, so we can add a > > > >> > > > >> #define __MEMCHR_CODEGEN_ATTRIBUTE __attribuet__((target("+zbb"= ))) > > > >> > > > >> (or whatever the syntax is) to the Zbb-flavored versions of these = routines? > > > > > > > > Yeah, this might work and it is clear than messing with compiler-de= fined > > > > macros. > > > > > > > >> > > > >> It might also be worth just jumping to the fast-misaligned version= s for these routines, too --the slow-misaligned stuff is there for compatib= ility with old stuff (though memchr aligns the pointer, so it doesn't matte= r so much here). > > > > > > > > I was hopping that ABIs that would like to provide unaligned varian= ts > > > > for mem* routines to improve the generic code, but it seems that fo= r some > > > > it is easier to just add an assembly routine (as loongarch did). > > > > > > > > For memchr, I think it should be easy to provide a unaligned versio= n. > > > > Something like (completely untested): > > > > > > > > /* Search no more than N bytes of S for C. */ > > > > void * > > > > __memchr (void const *s, int c_in, size_t n) > > > > { > > > > if (__glibc_unlikely (n =3D=3D 0)) > > > > return NULL; > > > > > > > > #ifdef USE_MEMCHR_UNALIGNED > > > > /* Read the first word, but munge it so that bytes before the arr= ay > > > > will not match goal. */ > > > > const op_t *word_ptr =3D PTR_ALIGN_DOWN (s, sizeof (op_t)); > > > > uintptr_t s_int =3D (uintptr_t) s; > > > > > > > > op_t word =3D *word_ptr; > > > > op_t repeated_c =3D repeat_bytes (c_in); > > > > /* Compute the address of the last byte taking in consideration p= ossible > > > > overflow. */ > > > > const char *lbyte =3D sadd (s_int, n - 1); > > > > /* And also the address of the word containing the last byte. */ > > > > const op_t *lword =3D (const op_t *) PTR_ALIGN_DOWN (lbyte, sizeo= f (op_t)); > > > > > > > > find_t mask =3D shift_find (find_eq_all (word, repeated_c), s_int= ); > > > > if (mask !=3D 0) > > > > { > > > > char *ret =3D (char *) s + index_first (mask); > > > > return (ret <=3D lbyte) ? ret : NULL; > > > > } > > > > if (word_ptr =3D=3D lword) > > > > return NULL; > > > > #endif > > > > > > > > word =3D *++word_ptr; > > > > while (word_ptr !=3D lword) > > > > { > > > > if (has_eq (word, repeated_c)) > > > > return (char *) word_ptr + index_first_eq (word, repeated_c= ); > > > > word =3D *++word_ptr; > > > > } > > > > > > > > if (has_eq (word, repeated_c)) > > > > { > > > > /* We found a match, but it might be in a byte past the end o= f the > > > > array. */ > > > > char *ret =3D (char *) word_ptr + index_first_eq (word, repea= ted_c); > > > > if (ret <=3D lbyte) > > > > return ret; > > > > } > > > > return NULL; > > > > } > > > > > > > >> > > > >>>>>> +#include > > > >>>>>> diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile = b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile > > > >>>>>> index fcef5659d4..5586d11c89 100644 > > > >>>>>> --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile > > > >>>>>> +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile > > > >>>>>> @@ -1,5 +1,8 @@ > > > >>>>>> ifeq ($(subdir),string) > > > >>>>>> sysdep_routines +=3D \ > > > >>>>>> + memchr \ > > > >>>>>> + memchr-generic \ > > > >>>>>> + memchr-zbb \ > > > >>>>>> memcpy \ > > > >>>>>> memcpy-generic \ > > > >>>>>> memcpy_noalignment \ > > > >>>>>> diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-imp= l-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c > > > >>>>>> index 9f806d7a9e..7321144a32 100644 > > > >>>>>> --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.= c > > > >>>>>> +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.= c > > > >>>>>> @@ -20,19 +20,40 @@ > > > >>>>>> #include > > > >>>>>> #include > > > >>>>>> > > > >>>>>> +#define ARRAY_SIZE(A) (sizeof (A) / sizeof ((A)[0])) > > > >>>>>> + > > > >>>>>> size_t > > > >>>>>> __libc_ifunc_impl_list (const char *name, struct libc_ifunc_i= mpl *array, > > > >>>>>> size_t max) > > > >>>>>> { > > > >>>>>> size_t i =3D max; > > > >>>>>> + struct riscv_hwprobe pairs[] =3D { > > > >>>>>> + { .key =3D RISCV_HWPROBE_KEY_IMA_EXT_0 }, > > > >>>>>> + { .key =3D RISCV_HWPROBE_KEY_CPUPERF_0 }, > > > >>>>>> + }; > > > >>>>>> > > > >>>>>> + bool has_zbb =3D false; > > > >>>>>> bool fast_unaligned =3D false; > > > >>>>>> > > > >>>>>> - struct riscv_hwprobe pair =3D { .key =3D RISCV_HWPROBE_KEY_= CPUPERF_0 }; > > > >>>>>> - if (__riscv_hwprobe (&pair, 1, 0, NULL, 0) =3D=3D 0 > > > >>>>>> - && (pair.value & RISCV_HWPROBE_MISALIGNED_MASK) > > > >>>>>> - =3D=3D RISCV_HWPROBE_MISALIGNED_FAST) > > > >>>>>> - fast_unaligned =3D true; > > > >>>>>> + if (__riscv_hwprobe (pairs, ARRAY_SIZE (pairs), 0, NULL, 0)= =3D=3D 0) > > > >>>>>> + { > > > >>>>>> + struct riscv_hwprobe *pair; > > > >>>>>> + > > > >>>>>> + /* RISCV_HWPROBE_KEY_IMA_EXT_0 */ > > > >>>>>> + pair =3D &pairs[0]; > > > >>>>>> + if (pair->value & RISCV_HWPROBE_EXT_ZBB) > > > >>>>>> + has_zbb =3D true; > > > >>>>>> + > > > >>>>>> + /* RISCV_HWPROBE_KEY_CPUPERF_0 */ > > > >>>>>> + pair =3D &pairs[1]; > > > >>>>>> + if ((pair->value & RISCV_HWPROBE_MISALIGNED_MASK) > > > >>>>>> + =3D=3D RISCV_HWPROBE_MISALIGNED_FAST) > > > >>>>>> + fast_unaligned =3D true; > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + IFUNC_IMPL (i, name, memchr, > > > >>>>>> + IFUNC_IMPL_ADD (array, i, memchr, has_zbb, __memch= r_zbb) > > > >>>>>> + IFUNC_IMPL_ADD (array, i, memchr, 1, __memchr_gene= ric)) > > > >>>>>> > > > >>>>>> IFUNC_IMPL (i, name, memcpy, > > > >>>>>> IFUNC_IMPL_ADD (array, i, memcpy, fast_unaligned, > > > >>>>>> diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c = b/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c > > > >>>>>> new file mode 100644 > > > >>>>>> index 0000000000..bc076cbf24 > > > >>>>>> --- /dev/null > > > >>>>>> +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c > > > >>>>>> @@ -0,0 +1,57 @@ > > > >>>>>> +/* Multiple versions of memchr. > > > >>>>>> + All versions must be listed in ifunc-impl-list.c. > > > >>>>>> + Copyright (C) 2017-2024 Free Software Foundation, Inc. > > > >>>>>> + This file is part of the GNU C Library. > > > >>>>>> + > > > >>>>>> + The GNU C Library is free software; you can redistribute i= t and/or > > > >>>>>> + modify it under the terms of the GNU Lesser General Public > > > >>>>>> + License as published by the Free Software Foundation; eith= er > > > >>>>>> + version 2.1 of the License, or (at your option) any later = version. > > > >>>>>> + > > > >>>>>> + The GNU C Library is distributed in the hope that it will = be useful, > > > >>>>>> + but WITHOUT ANY WARRANTY; without even the implied warrant= y of > > > >>>>>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See = the GNU > > > >>>>>> + Lesser General Public License for more details. > > > >>>>>> + > > > >>>>>> + You should have received a copy of the GNU Lesser General = Public > > > >>>>>> + License along with the GNU C Library; if not, see > > > >>>>>> + . */ > > > >>>>>> + > > > >>>>>> +#if IS_IN (libc) > > > >>>>>> +/* Redefine memchr so that the compiler won't complain about = the type > > > >>>>>> + mismatch with the IFUNC selector in strong_alias, below. = */ > > > >>>>>> +# undef memchr > > > >>>>>> +# define memchr __redirect_memchr > > > >>>>>> +# include > > > >>>>>> +# include > > > >>>>>> +# include > > > >>>>>> +# include > > > >>>>>> +# include > > > >>>>>> + > > > >>>>>> +extern __typeof (__redirect_memchr) __libc_memchr; > > > >>>>>> + > > > >>>>>> +extern __typeof (__redirect_memchr) __memchr_generic attribut= e_hidden; > > > >>>>>> +extern __typeof (__redirect_memchr) __memchr_zbb attribute_hi= dden; > > > >>>>>> + > > > >>>>>> +static inline __typeof (__redirect_memchr) * > > > >>>>>> +select_memchr_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwp= robe_func) > > > >>>>>> +{ > > > >>>>>> + unsigned long long int v; > > > >>>>>> + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IM= A_EXT_0, &v) =3D=3D 0 > > > >>>>>> + && (v & RISCV_HWPROBE_EXT_ZBB)) > > > >>>>>> + return __memchr_zbb; > > > >>>>>> + > > > >>>>>> + return __memchr_generic; > > > >>>>>> +} > > > >>>>>> + > > > >>>>>> +riscv_libc_ifunc (__libc_memchr, select_memchr_ifunc); > > > >>>>>> + > > > >>>>>> +# undef memchr > > > >>>>>> +strong_alias (__libc_memchr, memchr); > > > >>>>>> +# ifdef SHARED > > > >>>>>> +__hidden_ver1 (memchr, __GI_memchr, __redirect_memchr) > > > >>>>>> + __attribute__ ((visibility ("hidden"))) __attribute_copy__ = (memchr); > > > >>>>>> +# endif > > > >>>>>> +#else > > > >>>>>> +# include > > > >>>>>> +#endif