From: Noah Goldstein <goldstein.w.n@gmail.com>
To: libc-alpha@sourceware.org
Subject: [PATCH v3] x86: Remove __mmask intrinsics in strstr-avx512.c
Date: Tue, 12 Jul 2022 11:48:04 -0700 [thread overview]
Message-ID: <20220712184804.2374822-1-goldstein.w.n@gmail.com> (raw)
In-Reply-To: <20220711220730.1968923-1-goldstein.w.n@gmail.com>
The intrinsics are not available before GCC7 and using standard
operators generates code of equivilent or better quality.
Removed:
_cvtmask64_u64
_kshiftri_mask64
_kand_mask64
Geometric Mean of 5 Runs of Full Benchmark Suite New / Old: 0.958
---
sysdeps/x86_64/multiarch/strstr-avx512.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/sysdeps/x86_64/multiarch/strstr-avx512.c b/sysdeps/x86_64/multiarch/strstr-avx512.c
index 2ab9e96db8..e44c1a05dc 100644
--- a/sysdeps/x86_64/multiarch/strstr-avx512.c
+++ b/sysdeps/x86_64/multiarch/strstr-avx512.c
@@ -26,6 +26,10 @@
#define ZMM_SIZE_IN_BYTES 64
#define PAGESIZE 4096
+#define cvtmask64_u64(...) (uint64_t) (__VA_ARGS__)
+#define kshiftri_mask64(x, y) ((x) >> (y))
+#define kand_mask64(x, y) ((x) & (y))
+
/*
Returns the index of the first edge within the needle, returns 0 if no edge
is found. Example: 'ab' is the first edge in 'aaaaaaaaaabaarddg'
@@ -133,15 +137,15 @@ __strstr_avx512 (const char *haystack, const char *ned)
__m512i hay0 = _mm512_maskz_loadu_epi8 (loadmask, haystack + hay_index);
/* Search for NULL and compare only till null char */
uint64_t nullmask
- = _cvtmask64_u64 (_mm512_mask_testn_epi8_mask (loadmask, hay0, hay0));
+ = cvtmask64_u64 (_mm512_mask_testn_epi8_mask (loadmask, hay0, hay0));
uint64_t cmpmask = nullmask ^ (nullmask - ONE_64BIT);
- cmpmask = cmpmask & _cvtmask64_u64 (loadmask);
+ cmpmask = cmpmask & cvtmask64_u64 (loadmask);
/* Search for the 2 charaters of needle */
__mmask64 k0 = _mm512_cmpeq_epi8_mask (hay0, ned0);
__mmask64 k1 = _mm512_cmpeq_epi8_mask (hay0, ned1);
- k1 = _kshiftri_mask64 (k1, 1);
+ k1 = kshiftri_mask64 (k1, 1);
/* k2 masks tell us if both chars from needle match */
- uint64_t k2 = _cvtmask64_u64 (_kand_mask64 (k0, k1)) & cmpmask;
+ uint64_t k2 = cvtmask64_u64 (kand_mask64 (k0, k1)) & cmpmask;
/* For every match, search for the entire needle for a full match */
while (k2)
{
@@ -178,13 +182,13 @@ __strstr_avx512 (const char *haystack, const char *ned)
hay0 = _mm512_loadu_si512 (haystack + hay_index);
hay1 = _mm512_load_si512 (haystack + hay_index
+ 1); // Always 64 byte aligned
- nullmask = _cvtmask64_u64 (_mm512_testn_epi8_mask (hay1, hay1));
+ nullmask = cvtmask64_u64 (_mm512_testn_epi8_mask (hay1, hay1));
/* Compare only till null char */
cmpmask = nullmask ^ (nullmask - ONE_64BIT);
k0 = _mm512_cmpeq_epi8_mask (hay0, ned0);
k1 = _mm512_cmpeq_epi8_mask (hay1, ned1);
/* k2 masks tell us if both chars from needle match */
- k2 = _cvtmask64_u64 (_kand_mask64 (k0, k1)) & cmpmask;
+ k2 = cvtmask64_u64 (kand_mask64 (k0, k1)) & cmpmask;
/* For every match, compare full strings for potential match */
while (k2)
{
--
2.34.1
next prev parent reply other threads:[~2022-07-12 18:48 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-11 22:07 [PATCH v1] x86: Use regular casting instead of _cvtmask64_u64 in strstr-avx512 Noah Goldstein
2022-07-11 22:08 ` Noah Goldstein
2022-07-11 22:17 ` Sunil Pandey
2022-07-11 22:32 ` Noah Goldstein
2022-07-11 22:38 ` [PATCH v2] x86: Remove __mmask intrinsics in strstr-avx512.c Noah Goldstein
2022-07-11 22:52 ` Noah Goldstein
2022-07-11 23:33 ` Sunil Pandey
2022-07-12 2:26 ` H.J. Lu
2022-07-12 2:37 ` Noah Goldstein
2022-07-12 15:56 ` H.J. Lu
2022-07-12 17:11 ` Noah Goldstein
2022-07-12 18:40 ` Noah Goldstein
2022-07-12 18:43 ` H.J. Lu
2022-07-12 18:48 ` Noah Goldstein
2022-07-11 23:32 ` Sunil Pandey
2022-07-12 18:48 ` Noah Goldstein [this message]
2022-07-12 19:07 ` [PATCH v3] " H.J. Lu
2022-07-12 19:48 ` Andreas Schwab
2022-07-12 20:12 ` Noah Goldstein
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220712184804.2374822-1-goldstein.w.n@gmail.com \
--to=goldstein.w.n@gmail.com \
--cc=libc-alpha@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).