From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) by sourceware.org (Postfix) with ESMTPS id 9D329385DC3C for ; Thu, 2 Feb 2023 18:12:47 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9D329385DC3C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org Received: by mail-ot1-x32d.google.com with SMTP id r34-20020a05683044a200b0068d4a8a8d2dso685982otv.12 for ; Thu, 02 Feb 2023 10:12:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=IZae/T37iLL1MGDMx4SxF7or+Kfq+KjDIyUEiRzWEvU=; b=tedc+ZtKOQgCmLJ49NpTEAiAx04gig7iorUvR/Ol2xrLmiax999o99EH1r5MUDUP+C oR0O3z5jEWHCt9LKTyuSk1T7dHHSicDEpueFVupHASVQtK5y/GM+J2qvabTdiPYcKDDT OhRGmwZbZgr6QjTIz1HS/aOwDMI3MCEUUEaJzrE0NTDU+3WdBeQrYDe+SZjGaPzvuIQP BGHAJ0i6Zd3t1F3ErnFNSTYaj9GDd6ld6WbJWkHxjnyxTG5JoEANrYiE23Caaiwu1kaQ MnJ8Pt7gQJclHFOiJxEEdNV4tpk7XQdMA7dwyQf4XLSAXP2lRLOxgJHOQeJFE2EEP/WT 9R6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IZae/T37iLL1MGDMx4SxF7or+Kfq+KjDIyUEiRzWEvU=; b=yS1GqwzL9yMz+7BZKIuLsyI4ntTM8oarObOWyIQYZohOckAL/DM2SgeM/NiQgQMs1U +HWfl3eUSaC7Uy5BwRq9VaBmGOMMaXRaud0W+bEVP1ovUqHx4TmQbSVlijs14IZkrGPU 8G06vOT3KxA0EBhyYyjozPP5y6jmaNqetWz2V0TS6jsRa3UozNCFkUP75hbH/yfFUaQ6 n+5q9VkSFw0mWSgApIFxk7uagxN8G1K06bLY3X3zQIcZnIeM50VfChPNBLZ+PkLqKXyh drm+n3aInA3pWOtJgWDyR5/fPwlyFvw4liiwdMNBZQ3pcNDzjuz3kMKnSzAftV2jSGD1 ZI7g== X-Gm-Message-State: AO0yUKXiV2uqUfVB52/UBjHmPdVzM2eCcgOwzT74BTKe2BUQ2Bxwoiwd ybLGCXnk8eJF23fJRgirgVsCLVmC/nRUhAZ1QpQ= X-Google-Smtp-Source: AK7set+0mZjiag4rxtjFkOcwzVwNhFD61pHn4yjA5K92M6a59SY3+qyaW4n4G3E4xEmCbqLrwFL8Qw== X-Received: by 2002:a05:6830:4109:b0:68b:c8fc:bfcc with SMTP id w9-20020a056830410900b0068bc8fcbfccmr4387372ott.9.1675361566410; Thu, 02 Feb 2023 10:12:46 -0800 (PST) Received: from mandiga.. ([2804:1b3:a7c2:1887:da12:b9d3:2162:a28c]) by smtp.gmail.com with ESMTPSA id ci10-20020a05683063ca00b00684a10970adsm126689otb.16.2023.02.02.10.12.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Feb 2023 10:12:45 -0800 (PST) From: Adhemerval Zanella To: libc-alpha@sourceware.org, Richard Henderson , Jeff Law , Xi Ruoyao , Noah Goldstein Subject: [PATCH v12 21/31] riscv: Add string-fza.h and string-fzi.h Date: Thu, 2 Feb 2023 15:11:39 -0300 Message-Id: <20230202181149.2181553-22-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230202181149.2181553-1-adhemerval.zanella@linaro.org> References: <20230202181149.2181553-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: It uses the bitmanip extension to optimize index_fist and index_last with clz/ctz (using generic implementation that routes to compiler builtin) and orc.b to check null bytes. Checked the string test on riscv64 user mode. Reviewed-by: Richard Henderson --- sysdeps/riscv/string-fza.h | 70 ++++++++++++++++++++++++++++++++++ sysdeps/riscv/string-fzi.h | 77 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) create mode 100644 sysdeps/riscv/string-fza.h create mode 100644 sysdeps/riscv/string-fzi.h diff --git a/sysdeps/riscv/string-fza.h b/sysdeps/riscv/string-fza.h new file mode 100644 index 0000000000..9c7a6efba2 --- /dev/null +++ b/sysdeps/riscv/string-fza.h @@ -0,0 +1,70 @@ +/* Zero byte detection; basics. RISCV version. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _RISCV_STRING_FZA_H +#define _RISCV_STRING_FZA_H 1 + +#ifdef __riscv_zbb +/* With bitmap extension we can use orc.b to find all zero bytes. */ +# include +# include + +/* The functions return a byte mask. */ +typedef op_t find_t; + +/* This function returns 0xff for each byte that is zero in X. */ +static __always_inline find_t +find_zero_all (op_t x) +{ + find_t r; + asm ("orc.b %0, %1" : "=r" (r) : "r" (x)); + return ~r; +} + +/* This function returns 0xff for each byte that is equal between X1 and + X2. */ +static __always_inline find_t +find_eq_all (op_t x1, op_t x2) +{ + return find_zero_all (x1 ^ x2); +} + +/* Identify zero bytes in X1 or equality between X1 and X2. */ +static __always_inline find_t +find_zero_eq_all (op_t x1, op_t x2) +{ + return find_zero_all (x1) | find_eq_all (x1, x2); +} + +/* Identify zero bytes in X1 or inequality between X1 and X2. */ +static __always_inline find_t +find_zero_ne_all (op_t x1, op_t x2) +{ + return find_zero_all (x1) | ~find_eq_all (x1, x2); +} + +/* Define the "inexact" versions in terms of the exact versions. */ +# define find_zero_low find_zero_all +# define find_eq_low find_eq_all +# define find_zero_eq_low find_zero_eq_all +# define find_zero_ne_low find_zero_ne_all +#else +#include +#endif + +#endif /* _RISCV_STRING_FZA_H */ diff --git a/sysdeps/riscv/string-fzi.h b/sysdeps/riscv/string-fzi.h new file mode 100644 index 0000000000..3cde113afb --- /dev/null +++ b/sysdeps/riscv/string-fzi.h @@ -0,0 +1,77 @@ +/* Zero byte detection; indexes. RISCV version. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _STRING_RISCV_FZI_H +#define _STRING_RISCV_FZI_H 1 + +#ifdef __riscv_zbb +# include +#else +/* Without bitmap clz/ctz extensions, it is faster to direct test the bits + instead of calling compiler auxiliary functions. */ +# include + +static __always_inline unsigned int +index_first (find_t c) +{ + if (c & 0x80U) + return 0; + if (c & 0x8000U) + return 1; + if (c & 0x800000U) + return 2; + + if (sizeof (op_t) == 4) + return 3; + + if (c & 0x80000000U) + return 3; + if (c & 0x8000000000UL) + return 4; + if (c & 0x800000000000UL) + return 5; + if (c & 0x80000000000000UL) + return 6; + return 7; +} + +static __always_inline unsigned int +index_last (find_t c) +{ + if (sizeof (op_t) == 8) + { + if (c & 0x8000000000000000UL) + return 7; + if (c & 0x80000000000000UL) + return 6; + if (c & 0x800000000000UL) + return 5; + if (c & 0x8000000000UL) + return 4; + } + if (c & 0x80000000U) + return 3; + if (c & 0x800000U) + return 2; + if (c & 0x8000U) + return 1; + return 0; +} +#endif + +#endif /* STRING_FZI_H */ -- 2.34.1