From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 7844) id 256C3385AC3E; Tue, 12 Jul 2022 23:02:17 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 256C3385AC3E Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Noah Goldstein To: glibc-cvs@sourceware.org Subject: [glibc] x86: Remove __mmask intrinsics in strstr-avx512.c X-Act-Checkin: glibc X-Git-Author: Noah Goldstein X-Git-Refname: refs/heads/master X-Git-Oldrev: 9c38deec96ebe1b052a0c1bef137b90967755f18 X-Git-Newrev: f2698954ff9c2f9626d4bcb5a30eb5729714e0b0 Message-Id: <20220712230217.256C3385AC3E@sourceware.org> Date: Tue, 12 Jul 2022 23:02:17 +0000 (GMT) X-BeenThere: glibc-cvs@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Glibc-cvs mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2022 23:02:17 -0000 https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=f2698954ff9c2f9626d4bcb5a30eb5729714e0b0 commit f2698954ff9c2f9626d4bcb5a30eb5729714e0b0 Author: Noah Goldstein Date: Tue Jul 12 11:48:04 2022 -0700 x86: Remove __mmask intrinsics in strstr-avx512.c The intrinsics are not available before GCC7 and using standard operators generates code of equivalent or better quality. Removed: _cvtmask64_u64 _kshiftri_mask64 _kand_mask64 Geometric Mean of 5 Runs of Full Benchmark Suite New / Old: 0.958 Diff: --- sysdeps/x86_64/multiarch/strstr-avx512.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/sysdeps/x86_64/multiarch/strstr-avx512.c b/sysdeps/x86_64/multiarch/strstr-avx512.c index 2ab9e96db8..e44c1a05dc 100644 --- a/sysdeps/x86_64/multiarch/strstr-avx512.c +++ b/sysdeps/x86_64/multiarch/strstr-avx512.c @@ -26,6 +26,10 @@ #define ZMM_SIZE_IN_BYTES 64 #define PAGESIZE 4096 +#define cvtmask64_u64(...) (uint64_t) (__VA_ARGS__) +#define kshiftri_mask64(x, y) ((x) >> (y)) +#define kand_mask64(x, y) ((x) & (y)) + /* Returns the index of the first edge within the needle, returns 0 if no edge is found. Example: 'ab' is the first edge in 'aaaaaaaaaabaarddg' @@ -133,15 +137,15 @@ __strstr_avx512 (const char *haystack, const char *ned) __m512i hay0 = _mm512_maskz_loadu_epi8 (loadmask, haystack + hay_index); /* Search for NULL and compare only till null char */ uint64_t nullmask - = _cvtmask64_u64 (_mm512_mask_testn_epi8_mask (loadmask, hay0, hay0)); + = cvtmask64_u64 (_mm512_mask_testn_epi8_mask (loadmask, hay0, hay0)); uint64_t cmpmask = nullmask ^ (nullmask - ONE_64BIT); - cmpmask = cmpmask & _cvtmask64_u64 (loadmask); + cmpmask = cmpmask & cvtmask64_u64 (loadmask); /* Search for the 2 charaters of needle */ __mmask64 k0 = _mm512_cmpeq_epi8_mask (hay0, ned0); __mmask64 k1 = _mm512_cmpeq_epi8_mask (hay0, ned1); - k1 = _kshiftri_mask64 (k1, 1); + k1 = kshiftri_mask64 (k1, 1); /* k2 masks tell us if both chars from needle match */ - uint64_t k2 = _cvtmask64_u64 (_kand_mask64 (k0, k1)) & cmpmask; + uint64_t k2 = cvtmask64_u64 (kand_mask64 (k0, k1)) & cmpmask; /* For every match, search for the entire needle for a full match */ while (k2) { @@ -178,13 +182,13 @@ __strstr_avx512 (const char *haystack, const char *ned) hay0 = _mm512_loadu_si512 (haystack + hay_index); hay1 = _mm512_load_si512 (haystack + hay_index + 1); // Always 64 byte aligned - nullmask = _cvtmask64_u64 (_mm512_testn_epi8_mask (hay1, hay1)); + nullmask = cvtmask64_u64 (_mm512_testn_epi8_mask (hay1, hay1)); /* Compare only till null char */ cmpmask = nullmask ^ (nullmask - ONE_64BIT); k0 = _mm512_cmpeq_epi8_mask (hay0, ned0); k1 = _mm512_cmpeq_epi8_mask (hay1, ned1); /* k2 masks tell us if both chars from needle match */ - k2 = _cvtmask64_u64 (_kand_mask64 (k0, k1)) & cmpmask; + k2 = cvtmask64_u64 (kand_mask64 (k0, k1)) & cmpmask; /* For every match, compare full strings for potential match */ while (k2) {