From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oa1-x33.google.com (mail-oa1-x33.google.com [IPv6:2001:4860:4864:20::33]) by sourceware.org (Postfix) with ESMTPS id 872AB3858CD1 for ; Mon, 18 Mar 2024 19:33:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 872AB3858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 872AB3858CD1 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2001:4860:4864:20::33 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710790409; cv=none; b=WCvc1WQlpWS9O1rNUAh6UUcG9YpKJziZU0F39hikh9AnfXCuS2GKa25Ql7As2xDksclaOM2yJUnI7/TtCFZmYzILrjuNIGtlEeRhElcYM72+Qdtj7yem4zmLwBzSkug3IGDtCAzo4vqWLU8cLoqE/iH8bAzFWt/zBONi07LEf6k= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710790409; c=relaxed/simple; bh=rgVtOh5PK7fodCxdVCJiav9mhNJ1OxPZYMNB7fzGH1U=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=UfPkwVPIWXLWNjysvJZ00FBZa/b/wQIWodtXxcYx9apAt7o0EfcXLrByuwHGs8qhSdev1kLG+5a2H7qQGj0ISEhWGP3KQonKsTVX8NjFdnhPE4zDsaHn+tNljmcQrM4Hfomv0hs9PlNwtxg10PqBijMHZlUjdXjBfgUT9wLpn3w= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oa1-x33.google.com with SMTP id 586e51a60fabf-221830f6643so2523710fac.2 for ; Mon, 18 Mar 2024 12:33:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710790398; x=1711395198; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=zjPhJzl4S1dWg7XrF+VCatlps3fLlwiI010fUx41fWs=; b=dBRKAaTgsHZMRuOQw4LK1aX57zIGYaKVskr40LTGlD0drcx0cV+IAJ9asapQWWRI/C X7mw1vTv2cce1hC22bWoBTteR80nth4BLTCO98PKerb8k9LUtBb0u5jzB0ZDuye6QH9G uf3D0WROX3KQjBuB3cB5HP6yAkZEJgiSRM9dVLw9CGUPIz/QLICI2AIbCXedIWtqSHOE l3n/LD3oS/vbSc8LyPx745581LQ/q4pBEBW9d7S3aSJT0Q9XsmNZm1Tnd7iU/gU11W6D W+H025uay03huQC0znaOsOsy4jLhu8V76y2V3G/ySuRfDQd10A4UKBbqSKiNLbpWIrsu mtNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710790398; x=1711395198; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zjPhJzl4S1dWg7XrF+VCatlps3fLlwiI010fUx41fWs=; b=XSWr6IDCXelcng1jEToX5RGBPI/Lex4h7MXZ1rtHvAP8WPpNDDMNun1GBTs+gTHGC3 yJCp6VyWpebFsv+RYU6gCYoYj4qm8IR7iuERm/aPplGT8u0BArFep7GOKGnTBZlCLzLp A/xHCZFbcIdj+qgVF9Jy/meeUMJnmMGee88FvWqDVnAdawwCm6KHZUk9HMCMKMmKa6jo yTJIshsCIttHtrkNoigkUdYlMMiV/U/oqSW6qBSkfuhDykVUYqbQmyfaec/X123BsZm7 qCqOFtqDBCemQA+kC36sa13FZpCFRkxhzmLyy9gH4rMbdsA4lgBwQDoOWUYqGoK7a1yQ siJw== X-Gm-Message-State: AOJu0YyK4TD7G3iPo5OYcGzEQqpNyoFvyUcXWME7/Hm0bZuwdcsCDREd 1OodRSA6/OaHUkgNFoV7HRHXy/4TPsNpmqmk5n8O5deUCKD+ouhlYrUdCEehb8BxIfLXSRAKPTM xY0rsMdOtxkOeGjbNivGN1VHkaBFtHzei X-Google-Smtp-Source: AGHT+IFHcg57MCcRli/PPxMTC/5//PeWv7sSZxWnduvRRJruHwJ++0QO7UoViWOnqSDSHPh3gyEk7xlgrXlhUs46Wl0= X-Received: by 2002:a05:6870:c34e:b0:21e:e5db:7964 with SMTP id e14-20020a056870c34e00b0021ee5db7964mr665551oak.23.1710790397760; Mon, 18 Mar 2024 12:33:17 -0700 (PDT) MIME-Version: 1.0 References: <20240318122752.197208-1-adhemerval.zanella@linaro.org> In-Reply-To: <20240318122752.197208-1-adhemerval.zanella@linaro.org> From: Noah Goldstein Date: Mon, 18 Mar 2024 14:33:06 -0500 Message-ID: Subject: Re: [PATCH] benchtests: Improve benchtests for strstr To: Adhemerval Zanella Cc: libc-alpha@sourceware.org, Peter Bergner Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-9.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Mon, Mar 18, 2024 at 7:28=E2=80=AFAM Adhemerval Zanella wrote: > > Use same strategy as bench-strstr.c (93eebae5168e5cf2 and 80b2bfb53504) > and use json_ctx for output to help standardize format across all > benchtests. > --- > benchtests/bench-strcasestr.c | 349 ++++++++++++++++++++++++++-------- > 1 file changed, 273 insertions(+), 76 deletions(-) > > diff --git a/benchtests/bench-strcasestr.c b/benchtests/bench-strcasestr.= c > index f6d1a78fba..84a0bef38f 100644 > --- a/benchtests/bench-strcasestr.c > +++ b/benchtests/bench-strcasestr.c > @@ -16,10 +16,36 @@ > License along with the GNU C Library; if not, see > . */ > > +#define MIN_PAGE_SIZE 131072 > #define TEST_MAIN > #define TEST_NAME "strcasestr" > #include "bench-string.h" > > +#include "json-lib.h" > + > +static const char input[] =3D > +"This manual is written with the assumption that you are at least " > +"somewhat familiar with the C programming language and basic programming= " > +"concepts. Specifically, familiarity with ISO standard C (*note ISO " > +"C::), rather than =E2=80=9Ctraditional=E2=80=9D pre-ISO C dialects, is = assumed.\n" > + > +" The GNU C Library includes several =E2=80=9Cheader files=E2=80=9D, e= ach of which " > +"provides definitions and declarations for a group of related facilities= ; " > +"this information is used by the C compiler when processing your program= . " > +"For example, the header file =E2=80=98stdio.h=E2=80=99 declares facilit= ies for " > +"performing input and output, and the header file =E2=80=98string.h=E2= =80=99 declares " > +"string processing utilities. The organization of this manual generally= " > +"follows the same division as the header files.\n" > + > +" If you are reading this manual for the first time, you should read " > +"all of the introductory material and skim the remaining chapters. Ther= e " > +"are a _lot_ of functions in the GNU C Library and it=E2=80=99s not real= istic to " > +"expect that you will be able to remember exactly _how_ to use each and = " > +"every one of them. It=E2=80=99s more important to become generally fam= iliar " > +"with the kinds of facilities that the library provides, so that when yo= u " > +"are writing your programs you can recognize _when_ to make use of " > +"library functions, and _where_ in this manual you can find more specifi= c " > +"information about them.\n"; > > #define STRCASESTR simple_strcasestr > #define NO_ALIAS > @@ -32,123 +58,294 @@ typedef char *(*proto_t) (const char *, const char = *); > IMPL (simple_strcasestr, 0) > IMPL (strcasestr, 1) > > - > static void > -do_one_test (impl_t *impl, const char *s1, const char *s2, char *exp_res= ult) > +do_one_test (json_ctx_t *json_ctx, impl_t *impl, const char *s1, > + const char *s2, char *exp_result) > { > - size_t i, iters =3D INNER_LOOP_ITERS_SMALL; > + size_t i, iters =3D INNER_LOOP_ITERS_SMALL / 8; was it taking to long? 256 doesn't seem like so much for this. > timing_t start, stop, cur; > + char *res; > > TIMING_NOW (start); > for (i =3D 0; i < iters; ++i) > - { > - CALL (impl, s1, s2); > - } > + res =3D CALL (impl, s1, s2); > TIMING_NOW (stop); > > TIMING_DIFF (cur, start, stop); > > - TIMING_PRINT_MEAN ((double) cur, (double) iters); > -} > + json_element_double (json_ctx, (double) cur / (double) iters); > > + if (res !=3D exp_result) > + { > + error (0, 0, "Wrong result in function %s %s %s", impl->name, > + (res =3D=3D NULL) ? "(null)" : res, > + (exp_result =3D=3D NULL) ? "(null)" : exp_result); > + ret =3D 1; > + } > +} > > static void > -do_test (size_t align1, size_t align2, size_t len1, size_t len2, > - int fail) > +do_test (json_ctx_t *json_ctx, size_t align1, size_t align2, size_t len1= , > + size_t len2, int fail) > { > char *s1 =3D (char *) (buf1 + align1); > char *s2 =3D (char *) (buf2 + align2); > > - static const char d[] =3D "1234567890abcxyz"; > -#define dl (sizeof (d) - 1) > - char *ss2 =3D s2; > - for (size_t l =3D len2; l > 0; l =3D l > dl ? l - dl : 0) > - { > - size_t t =3D l > dl ? dl : l; > - ss2 =3D mempcpy (ss2, d, t); > - } > - s2[len2] =3D '\0'; > + size_t size =3D sizeof (input) - 1; > + size_t pos =3D (len1 + len2) % size; > > - if (fail) > + char *ss2 =3D s2; > + for (size_t l =3D len2; l > 0; l =3D l > size ? l - size : 0) > { > - char *ss1 =3D s1; > - for (size_t l =3D len1; l > 0; l =3D l > dl ? l - dl : 0) > + size_t t =3D l > size ? size : l; > + if (pos + t <=3D size) > + ss2 =3D mempcpy (ss2, input + pos, t); > + else > { > - size_t t =3D l > dl ? dl : l; > - memcpy (ss1, d, t); > - ++ss1[len2 > 7 ? 7 : len2 - 1]; > - ss1 +=3D t; > + ss2 =3D mempcpy (ss2, input + pos, size - pos); > + ss2 =3D mempcpy (ss2, input, t - (size - pos)); > } > } > - else > + s2[len2] =3D '\0'; > + > + char *ss1 =3D s1; > + for (size_t l =3D len1; l > 0; l =3D l > size ? l - size : 0) > { > - memset (s1, '0', len1); > - for (size_t i =3D 0; i < len2; ++i) > - s1[len1 - len2 + i] =3D toupper (s2[i]); > + size_t t =3D l > size ? size : l; > + memcpy (ss1, input, t); > + ss1 +=3D t; > } > + > + if (!fail) > + memcpy (s1 + len1 - len2, s2, len2); > s1[len1] =3D '\0'; > > - printf ("Length %4zd/%zd, alignment %2zd/%2zd, %s:", > - len1, len2, align1, align2, fail ? "fail" : "found"); > + /* Remove any accidental matches except for the last if !fail. */ > + for (ss1 =3D simple_strcasestr (s1, s2); > + ss1 !=3D NULL; > + ss1 =3D simple_strcasestr (ss1 + 1, s2)) > + if (fail || ss1 !=3D s1 + len1 - len2) > + ++ss1[len2 / 2]; > + > + json_element_object_begin (json_ctx); > + json_attr_uint (json_ctx, "len_haystack", len1); > + json_attr_uint (json_ctx, "len_needle", len2); > + json_attr_uint (json_ctx, "align_haystack", align1); > + json_attr_uint (json_ctx, "align_needle", align2); > + json_attr_uint (json_ctx, "fail", fail); > + > + json_array_begin (json_ctx, "timings"); > > FOR_EACH_IMPL (impl, 0) > - do_one_test (impl, s1, s2, fail ? NULL : s1 + len1 - len2); > + do_one_test (json_ctx, impl, s1, s2, fail ? NULL : s1 + len1 - len2)= ; > + > + json_array_end (json_ctx); > + json_element_object_end (json_ctx); > > - putchar ('\n'); > +} > + > +/* Test needles which exhibit worst-case performance for naive quadradic > + implementations. */ > + > +static void > +test_hard_needle (json_ctx_t *json_ctx, size_t ne_len, size_t hs_len) > +{ > + char *ne =3D (char *) buf1; > + char *hs =3D (char *) buf2; > + > + /* Hard needle for strstr algorithm using skip table. This results in= many > + memcmp calls comparing most of the needle. */ > + { > + memset (ne, 'a', ne_len); > + ne[ne_len] =3D '\0'; > + ne[ne_len - 14] =3D 'b'; > + > + memset (hs, 'a', hs_len); > + for (size_t i =3D ne_len; i <=3D hs_len; i +=3D ne_len) > + { > + hs[i - 5] =3D 'b'; > + hs[i - 62] =3D 'b'; > + } > + > + json_element_object_begin (json_ctx); > + json_attr_uint (json_ctx, "len_haystack", hs_len); > + json_attr_uint (json_ctx, "len_needle", ne_len); > + json_attr_uint (json_ctx, "align_haystack", 0); > + json_attr_uint (json_ctx, "align_needle", 0); > + json_attr_uint (json_ctx, "fail", 1); > + json_attr_string (json_ctx, "desc", "Difficult skiptable(0)"); > + > + json_array_begin (json_ctx, "timings"); > + > + FOR_EACH_IMPL (impl, 0) > + do_one_test (json_ctx, impl, hs, ne, NULL); > + > + json_array_end (json_ctx); > + json_element_object_end (json_ctx); > + } > + > + /* 2nd hard needle for strstr algorithm using skip table. This result= s in > + many memcmp calls comparing most of the needle. */ > + { > + memset (ne, 'a', ne_len); > + ne[ne_len] =3D '\0'; > + ne[ne_len - 6] =3D 'b'; > + > + memset (hs, 'a', hs_len); > + for (size_t i =3D ne_len; i <=3D hs_len; i +=3D ne_len) > + { > + hs[i - 5] =3D 'b'; > + hs[i - 6] =3D 'b'; > + } > + > + json_element_object_begin (json_ctx); > + json_attr_uint (json_ctx, "len_haystack", hs_len); > + json_attr_uint (json_ctx, "len_needle", ne_len); > + json_attr_uint (json_ctx, "align_haystack", 0); > + json_attr_uint (json_ctx, "align_needle", 0); > + json_attr_uint (json_ctx, "fail", 1); > + json_attr_string (json_ctx, "desc", "Difficult skiptable(1)"); > + > + json_array_begin (json_ctx, "timings"); > + > + FOR_EACH_IMPL (impl, 0) > + do_one_test (json_ctx, impl, hs, ne, NULL); > + > + json_array_end (json_ctx); > + json_element_object_end (json_ctx); > + } > + > + /* Hard needle for Two-way algorithm - the random input causes a large= number > + of branch mispredictions which significantly reduces performance on= modern > + micro architectures. */ > + { > + for (int i =3D 0; i < hs_len; i++) > + hs[i] =3D (rand () & 255) > 155 ? 'a' : 'b'; > + hs[hs_len] =3D 0; > + > + memset (ne, 'a', ne_len); > + ne[ne_len - 2] =3D 'b'; > + ne[0] =3D 'b'; > + ne[ne_len] =3D 0; > + > + json_element_object_begin (json_ctx); > + json_attr_uint (json_ctx, "len_haystack", hs_len); > + json_attr_uint (json_ctx, "len_needle", ne_len); > + json_attr_uint (json_ctx, "align_haystack", 0); > + json_attr_uint (json_ctx, "align_needle", 0); > + json_attr_uint (json_ctx, "fail", 1); > + json_attr_string (json_ctx, "desc", "Difficult 2-way"); > + > + json_array_begin (json_ctx, "timings"); > + > + FOR_EACH_IMPL (impl, 0) > + do_one_test (json_ctx, impl, hs, ne, NULL); > + > + json_array_end (json_ctx); > + json_element_object_end (json_ctx); > + } > + > + /* Hard needle for standard algorithm testing first few characters of > + * needle. */ > + { > + for (int i =3D 0; i < hs_len; i++) > + hs[i] =3D (rand () & 255) >=3D 128 ? 'a' : 'b'; > + hs[hs_len] =3D 0; > + > + for (int i =3D 0; i < ne_len; i++) > + { > + if (i % 3 =3D=3D 0) > + ne[i] =3D 'a'; > + else if (i % 3 =3D=3D 1) > + ne[i] =3D 'b'; > + else > + ne[i] =3D 'c'; > + } > + ne[ne_len] =3D 0; > + > + json_element_object_begin (json_ctx); > + json_attr_uint (json_ctx, "len_haystack", hs_len); > + json_attr_uint (json_ctx, "len_needle", ne_len); > + json_attr_uint (json_ctx, "align_haystack", 0); > + json_attr_uint (json_ctx, "align_needle", 0); > + json_attr_uint (json_ctx, "fail", 1); > + json_attr_string (json_ctx, "desc", "Difficult testing first 2"); > + > + json_array_begin (json_ctx, "timings"); > + > + FOR_EACH_IMPL (impl, 0) > + do_one_test (json_ctx, impl, hs, ne, NULL); > + > + json_array_end (json_ctx); > + json_element_object_end (json_ctx); > + } > } > > static int > test_main (void) > { > + json_ctx_t json_ctx; > test_init (); > > - printf ("%23s", ""); > + json_init (&json_ctx, 0, stdout); > + > + json_document_begin (&json_ctx); > + json_attr_string (&json_ctx, "timing_type", TIMING_TYPE); > + > + json_attr_object_begin (&json_ctx, "functions"); > + json_attr_object_begin (&json_ctx, TEST_NAME); > + json_attr_string (&json_ctx, "bench-variant", ""); > + > + json_array_begin (&json_ctx, "ifuncs"); > FOR_EACH_IMPL (impl, 0) > - printf ("\t%s", impl->name); > - putchar ('\n'); > + json_element_string (&json_ctx, impl->name); > + json_array_end (&json_ctx); > > - for (size_t klen =3D 2; klen < 32; ++klen) > - for (size_t hlen =3D 2 * klen; hlen < 16 * klen; hlen +=3D klen) > + json_array_begin (&json_ctx, "results"); > + > + for (size_t hlen =3D 8; hlen <=3D 256;) > + for (size_t klen =3D 1; klen <=3D 16; klen++) > { > - do_test (0, 0, hlen, klen, 0); > - do_test (0, 0, hlen, klen, 1); > - do_test (0, 3, hlen, klen, 0); > - do_test (0, 3, hlen, klen, 1); > - do_test (0, 9, hlen, klen, 0); > - do_test (0, 9, hlen, klen, 1); > - do_test (0, 15, hlen, klen, 0); > - do_test (0, 15, hlen, klen, 1); > - > - do_test (3, 0, hlen, klen, 0); > - do_test (3, 0, hlen, klen, 1); > - do_test (3, 3, hlen, klen, 0); > - do_test (3, 3, hlen, klen, 1); > - do_test (3, 9, hlen, klen, 0); > - do_test (3, 9, hlen, klen, 1); > - do_test (3, 15, hlen, klen, 0); > - do_test (3, 15, hlen, klen, 1); > - > - do_test (9, 0, hlen, klen, 0); > - do_test (9, 0, hlen, klen, 1); > - do_test (9, 3, hlen, klen, 0); > - do_test (9, 3, hlen, klen, 1); > - do_test (9, 9, hlen, klen, 0); > - do_test (9, 9, hlen, klen, 1); > - do_test (9, 15, hlen, klen, 0); > - do_test (9, 15, hlen, klen, 1); > - > - do_test (15, 0, hlen, klen, 0); > - do_test (15, 0, hlen, klen, 1); > - do_test (15, 3, hlen, klen, 0); > - do_test (15, 3, hlen, klen, 1); > - do_test (15, 9, hlen, klen, 0); > - do_test (15, 9, hlen, klen, 1); > - do_test (15, 15, hlen, klen, 0); > - do_test (15, 15, hlen, klen, 1); > + do_test (&json_ctx, 1, 3, hlen, klen, 0); > + do_test (&json_ctx, 0, 9, hlen, klen, 1); > + > + do_test (&json_ctx, 1, 3, hlen + 1, klen, 0); > + do_test (&json_ctx, 0, 9, hlen + 1, klen, 1); > + > + do_test (&json_ctx, getpagesize () - 15, 9, hlen, klen, 1); > + if (hlen < 64) > + { > + hlen +=3D 8; > + } > + else > + { > + hlen +=3D 32; > + } > + } > + > + for (size_t hlen =3D 256; hlen <=3D 65536; hlen *=3D 2) > + for (size_t klen =3D 4; klen <=3D 256; klen *=3D 2) > + { > + do_test (&json_ctx, 1, 11, hlen, klen, 0); > + do_test (&json_ctx, 14, 5, hlen, klen, 1); > + > + do_test (&json_ctx, 1, 11, hlen + 1, klen + 1, 0); > + do_test (&json_ctx, 14, 5, hlen + 1, klen + 1, 1); > + > + do_test (&json_ctx, 1, 11, hlen + 1, klen, 0); > + do_test (&json_ctx, 14, 5, hlen + 1, klen, 1); > + > + do_test (&json_ctx, getpagesize () - 15, 5, hlen + 1, klen, 1); > } > > - do_test (0, 0, page_size - 1, 16, 0); > - do_test (0, 0, page_size - 1, 16, 1); > + test_hard_needle (&json_ctx, 64, 65536); > + test_hard_needle (&json_ctx, 256, 65536); > + test_hard_needle (&json_ctx, 1024, 65536); > + > + json_array_end (&json_ctx); > + json_attr_object_end (&json_ctx); > + json_attr_object_end (&json_ctx); > + json_document_end (&json_ctx); > > return ret; > } > -- > 2.34.1 >