From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [IPv6:2607:f8b0:4864:20::1130]) by sourceware.org (Postfix) with ESMTPS id CFC023856252 for ; Wed, 18 May 2022 17:33:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org CFC023856252 Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-2ff39b44b06so31925737b3.13 for ; Wed, 18 May 2022 10:33:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oCIMvQR4d+cWTiiXmkBBey34efVF/vdLyF6pGeMWMi8=; b=7sSQz650AbvcNTVRZpKcpYlvWbSWNV3bWDaFmaqjBnI0wvquvx1NZpvH/ojq4zlK8L XUkQP5+lurk07jFQjRt2yChRfJJkIcOaRJQXYuoJjr8nY/v8xLjiypQxO+lZgSWxZC5z aOSowx4ifuj7SXTFA+WKEQseu/EYElksVYF/pywV/zkjSb7gakc7riPfIsziLprwJjNT 4NMkRlHlRYPUBWWN2a66ADsJmkaqrkWDu7Id+uSrQLFTWhyBkNq7N3xtQUxcdvVI0LqV Y8XYjcS0gahn0ih+BRLpD4ZhTGL09m+2Rz6eXQWrBLCVfF0QDif2dzh20kB74fvpyMm1 zPhg== X-Gm-Message-State: AOAM533dz3LW4c86FKK0gHZLQZObf5v1Rp96XBTkIevFKjfLtOAUTlMA V2ctOwy6owQg9MLl/p/lgTJVvPb0y8h49xzhBSjihqte X-Google-Smtp-Source: ABdhPJxvYgfV7GRC8EVY1DjNAJC3dBZMfp+aXuXeHEk4Mt8Er8kiKt4fcT7jDqHlAx0JrFJQGXa3k3f3nEQFHzmc/0A= X-Received: by 2002:a81:25d3:0:b0:2ff:111c:a5ce with SMTP id l202-20020a8125d3000000b002ff111ca5cemr522062ywl.372.1652895203066; Wed, 18 May 2022 10:33:23 -0700 (PDT) MIME-Version: 1.0 References: <20220414041231.926415-1-goldstein.w.n@gmail.com> <20220516203004.38687-1-goldstein.w.n@gmail.com> <20220516203004.38687-4-goldstein.w.n@gmail.com> <9ae00701-cd7f-165d-66ed-b1a3bf2e2633@gotplt.org> In-Reply-To: <9ae00701-cd7f-165d-66ed-b1a3bf2e2633@gotplt.org> From: Noah Goldstein Date: Wed, 18 May 2022 12:33:11 -0500 Message-ID: Subject: Re: [PATCH v9 4/6] benchtests: Add benchtests for dl_elf_hash, dl_new_hash and nss_hash To: Siddhesh Poyarekar Cc: GNU C Library Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2022 17:33:26 -0000 On Mon, May 16, 2022 at 11:52 PM Siddhesh Poyarekar wrote: > > On 17/05/2022 02:00, Noah Goldstein via Libc-alpha wrote: > > Benchtests are for throughput and include random / fixed size > > benchmarks. > > --- > > benchtests/Makefile | 25 ++++- > > benchtests/README | 9 +- > > benchtests/bench-dl-elf-hash.c | 23 ++++ > > benchtests/bench-dl-new-hash.c | 23 ++++ > > benchtests/bench-hash-funcs.c | 196 +++++++++++++++++++++++++++++++++ > > benchtests/bench-nss-hash.c | 24 ++++ > > 6 files changed, 292 insertions(+), 8 deletions(-) > > create mode 100644 benchtests/bench-dl-elf-hash.c > > create mode 100644 benchtests/bench-dl-new-hash.c > > create mode 100644 benchtests/bench-hash-funcs.c > > create mode 100644 benchtests/bench-nss-hash.c > > > > diff --git a/benchtests/Makefile b/benchtests/Makefile > > index de9de5cf58..c279041e19 100644 > > --- a/benchtests/Makefile > > +++ b/benchtests/Makefile > > @@ -227,6 +227,12 @@ LOCALES := \ > > include ../gen-locales.mk > > endif > > > > +hash-benchset := \ > > + dl-elf-hash \ > > + dl-new-hash \ > > + nss-hash \ > > +# hash-benchset > > + > > stdlib-benchset := strtod > > > > stdio-common-benchset := sprintf > > @@ -235,7 +241,7 @@ math-benchset := math-inlines > > > > ifeq (${BENCHSET},) > > benchset := $(string-benchset-all) $(stdlib-benchset) $(stdio-common-benchset) \ > > - $(math-benchset) > > + $(math-benchset) $(hash-benchset) > > else > > benchset := $(foreach B,$(filter %-benchset,${BENCHSET}), ${${B}}) > > endif > > @@ -363,9 +369,20 @@ bench-clean: > > > > # Validate the passed in BENCHSET > > ifneq ($(strip ${BENCHSET}),) > > -VALIDBENCHSETNAMES := bench-pthread bench-math bench-string string-benchset \ > > - wcsmbs-benchset stdlib-benchset stdio-common-benchset math-benchset \ > > - malloc-thread malloc-simple > > +VALIDBENCHSETNAMES := \ > > + bench-math \ > > + bench-pthread \ > > + bench-string \ > > + hash-benchset \ > > + malloc-simple \ > > + malloc-thread \ > > + math-benchset \ > > + stdio-common-benchset \ > > + stdlib-benchset \ > > + string-benchset \ > > + wcsmbs-benchset \ > > +# VALIDBENCHSETNAMES > > + > > INVALIDBENCHSETNAMES := $(filter-out ${VALIDBENCHSETNAMES},${BENCHSET}) > > ifneq (${INVALIDBENCHSETNAMES},) > > $(info The following values in BENCHSET are invalid: ${INVALIDBENCHSETNAMES}) > > OK. > > > diff --git a/benchtests/README b/benchtests/README > > index 4d83a05b4b..998ba9b2b4 100644 > > --- a/benchtests/README > > +++ b/benchtests/README > > @@ -84,12 +84,13 @@ where BENCHSET may be a space-separated list of the following values: > > bench-math > > bench-pthread > > bench-string > > + hash-benchset > > + malloc-thread > > + math-benchset > > + stdio-common-benchset > > + stdlib-benchset > > string-benchset > > wcsmbs-benchset > > - stdlib-benchset > > - stdio-common-benchset > > - math-benchset > > - malloc-thread > > > > OK. > > > Adding a function to benchtests: > > =============================== > > diff --git a/benchtests/bench-dl-elf-hash.c b/benchtests/bench-dl-elf-hash.c > > new file mode 100644 > > index 0000000000..5ca5116ad3 > > --- /dev/null > > +++ b/benchtests/bench-dl-elf-hash.c > > @@ -0,0 +1,23 @@ > > +/* Measure __dl_new_hash runtime > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + . */ > > + > > +#include > > +#define TEST_FUNC(x, y) _dl_elf_hash (x) > > +#define TEST_NAME "_dl_elf_hash" > > + > > +#include "bench-hash-funcs.c" > > Reusing infrastructure. OK. > > > diff --git a/benchtests/bench-dl-new-hash.c b/benchtests/bench-dl-new-hash.c > > new file mode 100644 > > index 0000000000..f5be528960 > > --- /dev/null > > +++ b/benchtests/bench-dl-new-hash.c > > @@ -0,0 +1,23 @@ > > +/* Measure __dl_new_hash runtime > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + . */ > > + > > +#include > > +#define TEST_FUNC(x, y) _dl_new_hash (x) > > +#define TEST_NAME "_dl_new_hash" > > + > > +#include "bench-hash-funcs.c" > > Same. OK. > > > diff --git a/benchtests/bench-hash-funcs.c b/benchtests/bench-hash-funcs.c > > new file mode 100644 > > index 0000000000..85cf7de8bc > > --- /dev/null > > +++ b/benchtests/bench-hash-funcs.c > > @@ -0,0 +1,196 @@ > > +/* Measure hash functions runtime. > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + . */ > > + > > +#define TEST_MAIN > > +#ifndef TEST_FUNC > > +# error "No TEST_FUNC provided!" > > +#endif > > + > > +#ifndef TEST_NAME > > +# define STRINGIFY_PRIMITIVE(x) # x > > +# define STRINGIFY(x) STRINGIFY_PRIMITIVE (x) > > + > > +# define TEST_NAME STRINGIFY (TEST_FUNC) > > +#endif > > + > > +#include "json-lib.h" > > +#include "bench-timing.h" > > + > > +#include > > +#include > > +#include > > + > > +#define DO_NOT_OPTIMIZE_OUT(x) __asm__ volatile("" : : "r,m"(x) : "memory") > > + > > +enum > > +{ > > + NFIXED_ITERS = 1048576, > > + NRAND_BUFS = 16384, > > + NRAND_ITERS = 2048, > > + RAND_BENCH_MAX_LEN = 256 > > +}; > > + > > +static double __attribute__ ((noinline, noclone)) > > +do_one_test_kernel (const char *s, size_t len) > > +{ > > + > > + unsigned int iters; > > + timing_t start, stop, cur; > > + > > + /* Warmup. */ > > + for (iters = NFIXED_ITERS / 32; iters; --iters) > > + { > > + DO_NOT_OPTIMIZE_OUT (TEST_FUNC (s, len)); > > + } > > + > > + TIMING_NOW (start); > > + for (iters = NFIXED_ITERS; iters; --iters) > > + { > > + DO_NOT_OPTIMIZE_OUT (TEST_FUNC (s, len)); > > + } > > + TIMING_NOW (stop); > > + > > + TIMING_DIFF (cur, start, stop); > > + > > + (void) (len); > > + return (double) cur / (double) NFIXED_ITERS; > > +} > > + > > +static void > > +do_one_test (json_ctx_t *json_ctx, size_t len) > > +{ > > + char buf[len + 1]; > > + memset (buf, -1, len); > > + buf[len] = '\0'; > > + > > + json_element_object_begin (json_ctx); > > + > > + json_attr_string (json_ctx, "type", "fixed"); > > + json_attr_uint (json_ctx, "length", len); > > + json_attr_double (json_ctx, "time", do_one_test_kernel (buf, len)); > > + > > + json_element_object_end (json_ctx); > > +} > > +static double > > +do_rand_test_kernel (char const *bufs, unsigned int const *sizes) > > +{ > > + unsigned int i, iters; > > + size_t offset; > > + timing_t start, stop, cur; > > + > > + /* Warmup. */ > > + for (i = 0, offset = 0; i < NRAND_BUFS; ++i, offset += RAND_BENCH_MAX_LEN) > > + { > > + DO_NOT_OPTIMIZE_OUT (TEST_FUNC (bufs + offset, sizes[i])); > > + } > > + > > + TIMING_NOW (start); > > + for (iters = NRAND_ITERS; iters; --iters) > > + { > > + for (i = 0, offset = 0; i < NRAND_BUFS; > > + ++i, offset += RAND_BENCH_MAX_LEN) > > + { > > + DO_NOT_OPTIMIZE_OUT (TEST_FUNC (bufs + offset, sizes[i])); > > + } > > + } > > + TIMING_NOW (stop); > > + > > + TIMING_DIFF (cur, start, stop); > > + > > + (void) (sizes); > > + return (double) cur / (double) (NRAND_ITERS * NRAND_BUFS); > > +} > > + > > +static void __attribute__ ((noinline, noclone)) > > +do_rand_test (json_ctx_t *json_ctx) > > +{ > > + size_t i, sz, offset; > > + char *bufs; > > + unsigned int *sizes; > > + > > + bufs = (char *) calloc (NRAND_BUFS, RAND_BENCH_MAX_LEN); > > + sizes = (unsigned int *) calloc (NRAND_BUFS, sizeof (unsigned int)); > > + if (bufs == NULL || sizes == NULL) > > + { > > + fprintf (stderr, "Failed to allocate bufs for random test\n"); > > + goto done; > > + } > > + > > + for (sz = 2; sz <= RAND_BENCH_MAX_LEN; sz += sz) > > + { > > + json_element_object_begin (json_ctx); > > + json_attr_string (json_ctx, "type", "random"); > > + json_attr_uint (json_ctx, "length", sz); > > + > > + for (i = 0, offset = 0; i < NRAND_BUFS; > > + ++i, offset += RAND_BENCH_MAX_LEN) > > + { > > + sizes[i] = random () % sz; > > + memset (bufs + offset, -1, sizes[i]); > > + bufs[offset + sizes[i]] = '\0'; > > + } > > + > > + json_attr_double (json_ctx, "time", do_rand_test_kernel (bufs, sizes)); > > + json_element_object_end (json_ctx); > > + } > > + > > +done: > > + if (bufs) > > + { > > + free (bufs); > > + } > > + if (sizes) > > + { > > + free (sizes); > > + } > > +} > > + > > +static int > > +do_test (void) > > +{ > > + int i; > > + json_ctx_t json_ctx; > > + > > + json_init (&json_ctx, 0, stdout); > > + json_document_begin (&json_ctx); > > + json_attr_string (&json_ctx, "timing_type", TIMING_TYPE); > > + json_attr_object_begin (&json_ctx, "functions"); > > + json_attr_object_begin (&json_ctx, TEST_NAME); > > + json_array_begin (&json_ctx, "results"); > > + > > + for (i = 0; i < 16; ++i) > > + { > > + do_one_test (&json_ctx, i); > > + } > > + > > + for (i = 16; i <= 256; i += i) > > + { > > + do_one_test (&json_ctx, i); > > + } > > + > > + do_rand_test (&json_ctx); > > + > > + json_array_end (&json_ctx); > > + json_attr_object_end (&json_ctx); > > + json_attr_object_end (&json_ctx); > > + json_document_end (&json_ctx); > > + > > + return 0; > > +} > > Please remove all redundant parantheses. The benchmark looks OK, but > how about also benchmarking the reference implementation in > elf/dl-hash.h and elf/dl-new-hash.h so that we always have a comparison > point, similar to the string benchmarks? Added the __simple_* defs in V10. Its a bit ugly because I don't think its quite fair to benchmark these the way we do strings because we need to keep the dl_*hash functions inlined for a fair comparison. Have a comment expressing that. Also fixed all redundant parens (sorry, thats generally my stylistic preference but shoulda gone with style of the project). > > > + > > +#include > > diff --git a/benchtests/bench-nss-hash.c b/benchtests/bench-nss-hash.c > > new file mode 100644 > > index 0000000000..085e1f8ee2 > > --- /dev/null > > +++ b/benchtests/bench-nss-hash.c > > @@ -0,0 +1,24 @@ > > +/* Measure __nss_hash runtime > > + Copyright (C) 2022 Free Software Foundation, Inc. > > + This file is part of the GNU C Library. > > + > > + The GNU C Library is free software; you can redistribute it and/or > > + modify it under the terms of the GNU Lesser General Public > > + License as published by the Free Software Foundation; either > > + version 2.1 of the License, or (at your option) any later version. > > + > > + The GNU C Library is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + Lesser General Public License for more details. > > + > > + You should have received a copy of the GNU Lesser General Public > > + License along with the GNU C Library; if not, see > > + . */ > > + > > +#include > > +#define TEST_FUNC __nss_hash > > + > > +uint32_t __nss_hash (const void *__key, size_t __length); > > + > > +#include "bench-hash-funcs.c" > > Reusing infrastructure. OK. > > Thanks, > Siddhesh