From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oa1-x31.google.com (mail-oa1-x31.google.com [IPv6:2001:4860:4864:20::31]) by sourceware.org (Postfix) with ESMTPS id 314243849AD7 for ; Wed, 24 Apr 2024 12:53:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 314243849AD7 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 314243849AD7 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2001:4860:4864:20::31 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713963210; cv=none; b=wKRvFzNoMPd9py/3nxYVeHsQ8HfYNq4wd8cif5HKUvPSRswGSUUCuwE004ipIgNzEMxqXIW788epIKyd4kp5qjamNZbae9zfzDQO07AdjHL9ZXfDrryQxEwpZFoyJjHDgPydeHcscD6DZGN7knwBL3VbByrRjsGoTLgAYn4QpeY= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713963210; c=relaxed/simple; bh=TvdvAdwGkA1Re0fUz2BJhA/R6gGvgSq4/gCc/d2pYcE=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=cFnrHjfxjgvowCQ/69YN5CV7WakVZkk5h3MqUOgdtb03z8D+3JukdTMI71w6Wjqs0mywVgSHiNfZ5qtVPV6IpogQzgWJQFbXB8YZSQelxIDEgasIIE+FlR2JjpIgiGBDft+z9RhC6A7H6CkUHMSXbvH4aUnJZJvdUR3arFSZa1w= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oa1-x31.google.com with SMTP id 586e51a60fabf-2343ae31a9bso2800706fac.1 for ; Wed, 24 Apr 2024 05:53:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713963207; x=1714568007; darn=sourceware.org; h=content-transfer-encoding:in-reply-to:organization:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=gzv4culj7pnNxhKx6w7OhNIZyf+9je7Bk1mpMokfpGg=; b=QMsrM4O0SUMBb/VQMBfCS2Oi9mAmtyeF2BuQ1peuqxt0dFbdQ5eF6u/GPQOYj68KTB /wyAUoNAw4MohF9VpKzb5eLItP8MxCgIAgCsnHgyVlE8NpCyS1kUGOUPyLULRXO5QW9e k1Gu0XOu1G7H5o7Cbwz8QQwBpa9vvTUJoA+mKkbD8Rm7D+f6lWaY0aoshCyhCVQ+KB1x bOJU08YosvoM5BSowNOt80xQYchVLcto2byqlajQz1WgfHfgDI+pLkP2hwDWq9CGOx/E juHF52iXCeW67+5nzclSxQG+3hqDU+U8z8LwDb/17F15V5gj/Y/AbjxlMuq5ERnJSRfM a/WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713963207; x=1714568007; h=content-transfer-encoding:in-reply-to:organization:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gzv4culj7pnNxhKx6w7OhNIZyf+9je7Bk1mpMokfpGg=; b=FVW0dvtlMAbykg4MnSmGg2csuhr+5yCiyQmh0pgTenZf8Tyf/mkG0GxvWg9eVbyJ3w aWTB44tEoHZTQFuQfBX/DPDDWv6IHPQ1aFr/oFaLGiA6eWGof1ssJklfLbvvTcIG4k0J i1/U8p/jEv15aF5GSDi0Dn4Mflhba+/TuU2M0Ur7WLWpBKSTMocbczhgfOjVQXNbQBuE 8h/PxBEPYljFwe2q+sM30klKEEePttwt3CrAKkEKhjH8KhqxCWRBOaDqDMPAt67f/BM3 A50tf9FOoFWHdxaOBq+2kjKep/2nPbA66RP1EBZypvv9VarZ0lmofqfFfK2UVq7lu8z/ y3RA== X-Forwarded-Encrypted: i=1; AJvYcCXZmA9LO9+OrJuBxrjknArYjnrH6oXRrkqrkZvG2hgnOV7ngenCpcuvdKTAXcrPT8WKqaNJbGanCWwJBr+I7JGg6C3Xs425dfvC X-Gm-Message-State: AOJu0Yy2TJZ3MBUNtPuOF85+H65L8aVRZQRqAYzV3nf6OiD0+/DuM1Ww X1UhUEI+LYGeRuLKOt73xiLWsSeNJHGoZMjFsnoxkjPhFeBG050xk6yQJQA2nLqhDrB0cIVqE0F d X-Google-Smtp-Source: AGHT+IFcANgCPk9VRlTHsxy+7Mzw+XMPbU9ZvEC5ksTZeOxR/+zxqFJBRfGkbbircdSVqzQ6FxiI0A== X-Received: by 2002:a05:6870:a3d1:b0:238:fbea:9c2f with SMTP id h17-20020a056870a3d100b00238fbea9c2fmr2310468oak.4.1713963207307; Wed, 24 Apr 2024 05:53:27 -0700 (PDT) Received: from ?IPV6:2804:1b3:a7c1:6157:e153:bb61:86b2:ad5c? ([2804:1b3:a7c1:6157:e153:bb61:86b2:ad5c]) by smtp.gmail.com with ESMTPSA id v28-20020a63481c000000b005f7536fbebfsm10847653pga.11.2024.04.24.05.53.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 24 Apr 2024 05:53:26 -0700 (PDT) Message-ID: Date: Wed, 24 Apr 2024 09:53:22 -0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/7] RISC-V: Add Zbb optimized memchr as ifunc To: =?UTF-8?Q?Christoph_M=C3=BCllner?= , libc-alpha@sourceware.org, Palmer Dabbelt , Darius Rad , Andrew Waterman , Philipp Tomsich , Evan Green , DJ Delorie , Vineet Gupta , Kito Cheng , Jeff Law References: <20240422074403.2399529-1-christoph.muellner@vrull.eu> <20240422074403.2399529-3-christoph.muellner@vrull.eu> Content-Language: en-US From: Adhemerval Zanella Netto Organization: Linaro In-Reply-To: <20240422074403.2399529-3-christoph.muellner@vrull.eu> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,KAM_MANYTO,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 22/04/24 04:43, Christoph Müllner wrote: > When building with Zbb enabled, memchr benefits from using orc.b in > find_zero_all(). This patch changes the build system such, that a > non-Zbb version as well as a Zbb version of this routine is built. > Further, a ifunc resolver is provided that selects the right routine > based on the outcome of extension probing via hwprobe(). > > Signed-off-by: Christoph Müllner > --- > sysdeps/riscv/multiarch/memchr-generic.c | 26 +++++++++ > sysdeps/riscv/multiarch/memchr-zbb.c | 30 ++++++++++ > .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + > .../linux/riscv/multiarch/ifunc-impl-list.c | 31 ++++++++-- > .../unix/sysv/linux/riscv/multiarch/memchr.c | 57 +++++++++++++++++++ > 5 files changed, 142 insertions(+), 5 deletions(-) > create mode 100644 sysdeps/riscv/multiarch/memchr-generic.c > create mode 100644 sysdeps/riscv/multiarch/memchr-zbb.c > create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c > > diff --git a/sysdeps/riscv/multiarch/memchr-generic.c b/sysdeps/riscv/multiarch/memchr-generic.c > new file mode 100644 > index 0000000000..a96c36398b > --- /dev/null > +++ b/sysdeps/riscv/multiarch/memchr-generic.c > @@ -0,0 +1,26 @@ > +/* Re-include the default memchr implementation. > + Copyright (C) 2024 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + . */ > + > +#include > + > +#if IS_IN(libc) > +# define MEMCHR __memchr_generic > +# undef libc_hidden_builtin_def > +# define libc_hidden_builtin_def(x) > +#endif > +#include > diff --git a/sysdeps/riscv/multiarch/memchr-zbb.c b/sysdeps/riscv/multiarch/memchr-zbb.c > new file mode 100644 > index 0000000000..bead0335ae > --- /dev/null > +++ b/sysdeps/riscv/multiarch/memchr-zbb.c > @@ -0,0 +1,30 @@ > +/* Re-include the default memchr implementation for Zbb. > + Copyright (C) 2024 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + . */ > + > +#include > + > +#if IS_IN(libc) > +# define MEMCHR __memchr_zbb > +# undef libc_hidden_builtin_def > +# define libc_hidden_builtin_def(x) > +#endif > +/* Convince preprocessor to have Zbb instructions. */ > +#ifndef __riscv_zbb > +# define __riscv_zbb > +#endif Is there a way to specific the compiler to enable a extension, like aarch64 -march=arch{+[no]feature}? I think ideally this should be enabled as CFLAGS instead of messing with compiler defined pre-processor. > +#include > diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile > index fcef5659d4..5586d11c89 100644 > --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile > +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile > @@ -1,5 +1,8 @@ > ifeq ($(subdir),string) > sysdep_routines += \ > + memchr \ > + memchr-generic \ > + memchr-zbb \ > memcpy \ > memcpy-generic \ > memcpy_noalignment \ > diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c > index 9f806d7a9e..7321144a32 100644 > --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c > +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c > @@ -20,19 +20,40 @@ > #include > #include > > +#define ARRAY_SIZE(A) (sizeof (A) / sizeof ((A)[0])) > + > size_t > __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > size_t max) > { > size_t i = max; > + struct riscv_hwprobe pairs[] = { > + { .key = RISCV_HWPROBE_KEY_IMA_EXT_0 }, > + { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }, > + }; > > + bool has_zbb = false; > bool fast_unaligned = false; > > - struct riscv_hwprobe pair = { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }; > - if (__riscv_hwprobe (&pair, 1, 0, NULL, 0) == 0 > - && (pair.value & RISCV_HWPROBE_MISALIGNED_MASK) > - == RISCV_HWPROBE_MISALIGNED_FAST) > - fast_unaligned = true; > + if (__riscv_hwprobe (pairs, ARRAY_SIZE (pairs), 0, NULL, 0) == 0) > + { > + struct riscv_hwprobe *pair; > + > + /* RISCV_HWPROBE_KEY_IMA_EXT_0 */ > + pair = &pairs[0]; > + if (pair->value & RISCV_HWPROBE_EXT_ZBB) > + has_zbb = true; > + > + /* RISCV_HWPROBE_KEY_CPUPERF_0 */ > + pair = &pairs[1]; > + if ((pair->value & RISCV_HWPROBE_MISALIGNED_MASK) > + == RISCV_HWPROBE_MISALIGNED_FAST) > + fast_unaligned = true; > + } > + > + IFUNC_IMPL (i, name, memchr, > + IFUNC_IMPL_ADD (array, i, memchr, has_zbb, __memchr_zbb) > + IFUNC_IMPL_ADD (array, i, memchr, 1, __memchr_generic)) > > IFUNC_IMPL (i, name, memcpy, > IFUNC_IMPL_ADD (array, i, memcpy, fast_unaligned, > diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c > new file mode 100644 > index 0000000000..bc076cbf24 > --- /dev/null > +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c > @@ -0,0 +1,57 @@ > +/* Multiple versions of memchr. > + All versions must be listed in ifunc-impl-list.c. > + Copyright (C) 2017-2024 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + . */ > + > +#if IS_IN (libc) > +/* Redefine memchr so that the compiler won't complain about the type > + mismatch with the IFUNC selector in strong_alias, below. */ > +# undef memchr > +# define memchr __redirect_memchr > +# include > +# include > +# include > +# include > +# include > + > +extern __typeof (__redirect_memchr) __libc_memchr; > + > +extern __typeof (__redirect_memchr) __memchr_generic attribute_hidden; > +extern __typeof (__redirect_memchr) __memchr_zbb attribute_hidden; > + > +static inline __typeof (__redirect_memchr) * > +select_memchr_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) > +{ > + unsigned long long int v; > + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 > + && (v & RISCV_HWPROBE_EXT_ZBB)) > + return __memchr_zbb; > + > + return __memchr_generic; > +} > + > +riscv_libc_ifunc (__libc_memchr, select_memchr_ifunc); > + > +# undef memchr > +strong_alias (__libc_memchr, memchr); > +# ifdef SHARED > +__hidden_ver1 (memchr, __GI_memchr, __redirect_memchr) > + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memchr); > +# endif > +#else > +# include > +#endif