Re: [PATCH 2/2] Add simd testsuite

[Jonathan Wakely <jwakely@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 13861 bytes --]

On 18/12/20 16:49 +0100, Matthias Kretz wrote:
>Resending squashed patch after addressing Jonathan's comments.
>
>From: Matthias Kretz <kretz@kde.org>
>
>Add a new check-simd target to the testsuite. The new target creates a
>subdirectory, generates the necessary Makefiles, and spawns submakes to
>build and run the tests. Running this testsuite with defaults on my
>machine takes half of the time the dejagnu testsuite required to only
>determine whether to run tests. Since the simd testsuite integrated in
>dejagnu increased the time of the whole libstdc++ testsuite by ~100%
>this approach is a compromise for speed while not sacrificing coverage
>too much. Since the test driver is invoked individually per test
>executable from a Makefile, make's jobserver (-j) trivially parallelizes
>testing.
>
>Testing different flags and with simulator (or remote execution) is
>possible. E.g. `make check-simd DRIVEROPTS=-q
>target_list="unix{-m64,-m32}{-march=sandybridge,-march=skylake-avx512}{,-
>ffast-math}"`
>runs the testsuite 8 times in different subdirectories, using 8
>different combinations of compiler flags, only outputs failing tests
>(-q), and prints all summaries at the end. It skips most ABI tags by
>default unless --run-expensive is passed to DRIVEROPTS or
>GCC_TEST_RUN_EXPENSIVE is not empty.
>
>To use a simulator, the CHECK_SIMD_CONFIG variable needs to point to a
>shell script which calls `define_target <name> <flags> <simulator>` and
>set target_list as needed. E.g.:
>case "$target_triplet" in
>x86_64-*)
>  target_list="unix{-march=sandybridge,-march=skylake-avx512}
>  ;;
>powerpc64le-*)
>  define_target power8 "-static -mcpu=power8" "/usr/bin/qemu-ppc64le -cpu
>power8"
>  define_target power9 -mcpu=power9 "$HOME/bin/run_on_gcc135"
>  target_list="power8 power9{,-ffast-math}"
>  ;;
>esac
>
>libstdc++-v3/ChangeLog:
>	* scripts/check_simd: New file. This script is called from the
>	the check-simd target. It determines a set of compiler flags and
>	simulator setups for calling generate_makefile.sh and passes the
>	information back to the check-simd target, which recurses to the
>	generated Makefiles.
>	* scripts/create_testsuite_files: Remove files below simd/tests/
>	from testsuite_files and place them in testsuite_files_simd.
>	* testsuite/Makefile.am: Add testsuite_files_simd. Add
>	check-simd target.
>	* testsuite/Makefile.in: Regenerate.
>	* testsuite/experimental/simd/driver.sh: New file. This script
>	compiles and runs a given simd test, logging its output and
>	status. It uses the timeout command to implement compile and
>	test timeouts.
>	* testsuite/experimental/simd/generate_makefile.sh: New file.
>	This script generates a Makefile which uses driver.sh to compile
>	and run the tests and collect the logs into a single log file.
>	* testsuite/experimental/simd/tests/abs.cc: New file. Tests
>	abs(simd).
>	* testsuite/experimental/simd/tests/algorithms.cc: New file.
>	Tests min/max(simd, simd).
>	* testsuite/experimental/simd/tests/bits/conversions.h: New
>	file. Contains functions to support tests involving conversions.
>	* testsuite/experimental/simd/tests/bits/make_vec.h: New file.
>	Support functions make_mask and make_vec.
>	* testsuite/experimental/simd/tests/bits/mathreference.h: New
>	file. Support functions to supply precomputed math function
>	reference data.
>	* testsuite/experimental/simd/tests/bits/metahelpers.h: New
>	file. Support code for SFINAE testing.
>	* testsuite/experimental/simd/tests/bits/simd_view.h: New file.
>	* testsuite/experimental/simd/tests/bits/test_values.h: New
>	file. Test functions to easily drive a test with simd objects
>	initialized from a given list of values and a range of random
>	values.
>	* testsuite/experimental/simd/tests/bits/ulp.h: New file.
>	Support code to determine the ULP distance of simd objects.
>	* testsuite/experimental/simd/tests/bits/verify.h: New file.
>	Test framework for COMPARE'ing simd objects and instantiating
>	the test templates with value_type and ABI tag.
>	* testsuite/experimental/simd/tests/broadcast.cc: New file. Test
>	simd broadcasts.
>	* testsuite/experimental/simd/tests/casts.cc: New file. Test
>	simd casts.
>	* testsuite/experimental/simd/tests/fpclassify.cc: New file.
>	Test floating-point classification functions.
>	* testsuite/experimental/simd/tests/frexp.cc: New file. Test
>	frexp(simd).
>	* testsuite/experimental/simd/tests/generator.cc: New file. Test
>	simd generator constructor.
>	* testsuite/experimental/simd/tests/hypot3_fma.cc: New file.
>	Test 3-arg hypot(simd,simd,simd) and fma(simd,simd,sim).
>	* testsuite/experimental/simd/tests/integer_operators.cc: New
>	file. Test integer operators.
>	* testsuite/experimental/simd/tests/ldexp_scalbn_scalbln_modf.cc:
>	New file. Test ldexp(simd), scalbn(simd), scalbln(simd), and
>	modf(simd).
>	* testsuite/experimental/simd/tests/loadstore.cc: New file. Test
>	(converting) simd loads and stores.
>	* testsuite/experimental/simd/tests/logarithm.cc: New file. Test
>	log*(simd).
>	* testsuite/experimental/simd/tests/mask_broadcast.cc: New file.
>	Test simd_mask broadcasts.
>	* testsuite/experimental/simd/tests/mask_conversions.cc: New
>	file. Test simd_mask conversions.
>	* testsuite/experimental/simd/tests/mask_implicit_cvt.cc: New
>	file. Test simd_mask implicit conversions.
>	* testsuite/experimental/simd/tests/mask_loadstore.cc: New file.
>	Test simd_mask loads and stores.
>	* testsuite/experimental/simd/tests/mask_operator_cvt.cc: New
>	file. Test simd_mask operators convert as specified.
>	* testsuite/experimental/simd/tests/mask_operators.cc: New file.
>	Test simd_mask compares, subscripts, and negation.
>	* testsuite/experimental/simd/tests/mask_reductions.cc: New
>	file. Test simd_mask reductions.
>	* testsuite/experimental/simd/tests/math_1arg.cc: New file. Test
>	1-arg math functions on simd.
>	* testsuite/experimental/simd/tests/math_2arg.cc: New file. Test
>	2-arg math functions on simd.
>	* testsuite/experimental/simd/tests/operator_cvt.cc: New file.
>	Test implicit conversions on simd binary operators behave as
>	specified.
>	* testsuite/experimental/simd/tests/operators.cc: New file. Test
>	simd compares, subscripts, not, unary minus, plus, minus,
>	multiplies, divides, increment, and decrement.
>	* testsuite/experimental/simd/tests/reductions.cc: New file.
>	Test reduce(simd).
>	* testsuite/experimental/simd/tests/remqo.cc: New file. Test
>	remqo(simd).
>	* testsuite/experimental/simd/tests/simd.cc: New file. Basic
>	sanity checks of simd types.
>	* testsuite/experimental/simd/tests/sincos.cc: New file. Test
>	sin(simd) and cos(simd).
>	* testsuite/experimental/simd/tests/split_concat.cc: New file.
>	Test split(simd) and concat(simd, simd).
>	* testsuite/experimental/simd/tests/splits.cc: New file. Test
>	split(simd_mask).
>	* testsuite/experimental/simd/tests/trigonometric.cc: New file.
>	Test remaining trigonometric functions on simd.
>	* testsuite/experimental/simd/tests/trunc_ceil_floor.cc: New
>	file. Test trunc(simd), ceil(simd), and floor(simd).
>	* testsuite/experimental/simd/tests/where.cc: New file. Test
>	masked operations using where.
>---
> libstdc++-v3/scripts/check_simd               |   75 ++
> libstdc++-v3/scripts/create_testsuite_files   |    6 +-
> libstdc++-v3/testsuite/Makefile.am            |   18 +-
> libstdc++-v3/testsuite/Makefile.in            |   18 +-
> .../testsuite/experimental/simd/driver.sh     |  249 ++++
> .../experimental/simd/generate_makefile.sh    |  261 ++++
> .../testsuite/experimental/simd/tests/abs.cc  |   41 +
> .../experimental/simd/tests/algorithms.cc     |   30 +
> .../simd/tests/bits/conversions.h             |  184 +++
> .../experimental/simd/tests/bits/make_vec.h   |   59 +
> .../simd/tests/bits/mathreference.h           |  160 +++
> .../simd/tests/bits/metahelpers.h             |  164 +++
> .../experimental/simd/tests/bits/simd_view.h  |  121 ++
> .../simd/tests/bits/test_values.h             |  383 ++++++
> .../experimental/simd/tests/bits/ulp.h        |  101 ++
> .../experimental/simd/tests/bits/verify.h     |  353 ++++++
> .../experimental/simd/tests/broadcast.cc      |  104 ++
> .../experimental/simd/tests/casts.cc          |  169 +++
> .../experimental/simd/tests/fpclassify.cc     |  106 ++
> .../experimental/simd/tests/frexp.cc          |   85 ++
> .../experimental/simd/tests/generator.cc      |   58 +
> .../experimental/simd/tests/hypot3_fma.cc     |  151 +++
> .../simd/tests/integer_operators.cc           |  218 ++++
> .../simd/tests/ldexp_scalbn_scalbln_modf.cc   |  169 +++
> .../experimental/simd/tests/loadstore.cc      |  229 ++++
> .../experimental/simd/tests/logarithm.cc      |   83 ++
> .../experimental/simd/tests/mask_broadcast.cc |   67 ++
> .../simd/tests/mask_conversions.cc            |  113 ++
> .../simd/tests/mask_implicit_cvt.cc           |  102 ++
> .../experimental/simd/tests/mask_loadstore.cc |  161 +++
> .../simd/tests/mask_operator_cvt.cc           |  111 ++
> .../experimental/simd/tests/mask_operators.cc |   57 +
> .../simd/tests/mask_reductions.cc             |  226 ++++
> .../experimental/simd/tests/math_1arg.cc      |  107 ++
> .../experimental/simd/tests/math_2arg.cc      |   79 ++
> .../experimental/simd/tests/operator_cvt.cc   | 1072 +++++++++++++++++
> .../experimental/simd/tests/operators.cc      |  297 +++++
> .../experimental/simd/tests/reductions.cc     |   97 ++
> .../experimental/simd/tests/remqo.cc          |   70 ++
> .../testsuite/experimental/simd/tests/simd.cc |   46 +
> .../experimental/simd/tests/sincos.cc         |   44 +
> .../experimental/simd/tests/split_concat.cc   |  183 +++
> .../experimental/simd/tests/splits.cc         |   38 +
> .../experimental/simd/tests/trigonometric.cc  |   41 +
> .../simd/tests/trunc_ceil_floor.cc            |  109 ++
> .../experimental/simd/tests/where.cc          |  136 +++
> 46 files changed, 6746 insertions(+), 5 deletions(-)
> create mode 100755 libstdc++-v3/scripts/check_simd
> create mode 100755 libstdc++-v3/testsuite/experimental/simd/driver.sh
> create mode 100755 libstdc++-v3/testsuite/experimental/simd/
>generate_makefile.sh
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/abs.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>algorithms.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>conversions.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>make_vec.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>mathreference.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>metahelpers.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>simd_view.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>test_values.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/ulp.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/
>verify.h
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>broadcast.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/casts.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>fpclassify.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/frexp.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>generator.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>hypot3_fma.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>integer_operators.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>ldexp_scalbn_scalbln_modf.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>loadstore.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>logarithm.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_broadcast.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_conversions.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_implicit_cvt.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_loadstore.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_operator_cvt.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_operators.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>mask_reductions.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>math_1arg.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>math_2arg.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>operator_cvt.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>operators.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>reductions.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/remqo.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/simd.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/sincos.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>split_concat.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/splits.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>trigonometric.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/
>trunc_ceil_floor.cc
> create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/where.cc

Also pushed to master, with the attached patch to fix some
non-portable shell commands (as discussed with Matthias on IRC).

Thanks for this very significant contribution, and for your patience
over the two years since your first patch (and three since we started
talking about copyright assignments).

I'll regen the docs and add something to the release notes too.



[-- Attachment #2: patch.txt --]
[-- Type: text/x-patch, Size: 3139 bytes --]

commit c31a633e13ff5222e9285a2c2b7897094ab061d9
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Thu Jan 21 13:15:42 2021

    libstdc++: Use printf to print control characters
    
    Bash and GNU echo do not interpret backslash escapes by default, so use
    printf when printing \n or \t in strings.
    
    libstdc++-v3/ChangeLog:
    
            * testsuite/experimental/simd/generate_makefile.sh: Use printf
            instead of echo when printing escape characters.

diff --git a/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh b/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh
index 85a7f87271e..ab5970554c3 100755
--- a/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh
+++ b/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh
@@ -97,10 +97,10 @@ DRIVEROPTS ?=
 all: simd_testsuite.sum
 
 simd_testsuite.sum: simd_testsuite.log
-	@echo "\n\t\t=== simd_testsuite \$(TESTFLAGS) Summary ===\n\n"\\
+	@printf "\n\t\t=== simd_testsuite \$(TESTFLAGS) Summary ===\n\n"\\
 	"# of expected passes:\t\t\$(shell grep -c '^PASS:' \$@)\n"\\
 	"# of unexpected failures:\t\$(shell grep -c '^FAIL:' \$@)\n"\\
-	"# of unsupported tests:\t\t\$(shell grep -c '^UNSUPPORTED:' \$@)"\\
+	"# of unsupported tests:\t\t\$(shell grep -c '^UNSUPPORTED:' \$@)\n"\\
 	  | tee -a \$@
 
 EOF
@@ -208,7 +208,7 @@ EOF
 EOF
     done
   done
-  echo 'run-%: export GCC_TEST_RUN_EXPENSIVE=yes\n'
+  echo 'run-%: export GCC_TEST_RUN_EXPENSIVE=yes'
   all_tests | while read file && read name; do
     echo "run-$name: $name.log"
     all_types "$file" | while read t && read type; do
@@ -221,7 +221,7 @@ EOF
   done
   cat <<EOF
 help:
-	@echo "use DRIVEROPTS=<options> to pass the following options:\n"\\
+	@printf "use DRIVEROPTS=<options> to pass the following options:\n"\\
 	"-q, --quiet         Only print failures.\n"\\
 	"-v, --verbose       Print compiler and test output on failure.\n"\\
 	"-k, --keep-failed   Keep executables of failed tests.\n"\\
@@ -232,19 +232,20 @@ help:
 	"                    Multiply the default timeout with x.\n"\\
 	"--run-expensive     Compile and run tests marked as expensive (default:\n"\\
 	"                    true if GCC_TEST_RUN_EXPENSIVE is set, false otherwise).\n"\\
-	"--only <pattern>    Compile and run only tests matching the given pattern.\n"
-	@echo "use TESTFLAGS=<flags> to pass additional compiler flags\n"
+	"--only <pattern>    Compile and run only tests matching the given pattern.\n\n"
+	@echo "use TESTFLAGS=<flags> to pass additional compiler flags"
+	@echo
 	@echo "The following are some of the valid targets for this Makefile:"
 	@echo "... all"
 	@echo "... clean"
 	@echo "... help"
 EOF
   all_tests | while read file && read name; do
-    echo "\t@echo '... run-${name}'"
+    printf "\t@echo '... run-${name}'\n"
     all_types | while read t && read type; do
-      echo "\t@echo '... run-${name}-${type}'"
+      printf "\t@echo '... run-${name}-${type}'\n"
       for i in $(seq 0 9); do
-        echo "\t@echo '... run-${name}-${type}-$i'"
+        printf "\t@echo '... run-${name}-${type}-$i'\n"
       done
     done
   done