From: Richard Sandiford <richard.sandiford@arm.com>
To: gcc-patches@gcc.gnu.org
Subject: [12/n] [AArch64] Support vectorising with multiple vector sizes
Date: Fri, 25 Oct 2019 12:49:00 -0000 [thread overview]
Message-ID: <mpttv7x802l.fsf@arm.com> (raw)
In-Reply-To: <mpto8y59fe7.fsf@arm.com> (Richard Sandiford's message of "Fri, 25 Oct 2019 13:30:08 +0100")
This patch makes the vectoriser try mixtures of 64-bit and 128-bit
vector modes on AArch64. It fixes some existing XFAILs and allows
kernel 24 from the Livermore Loops test to be vectorised (by using
a mixture of V2DF and V2SI).
I'll apply this if the prerequisites are approved.
2019-10-24 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* config/aarch64/aarch64.c (aarch64_vectorize_related_mode): New
function.
(aarch64_autovectorize_vector_modes): Also add V4HImode and V2SImode.
(TARGET_VECTORIZE_RELATED_MODE): Define.
gcc/testsuite/
* gcc.dg/vect/vect-outer-4f.c: Expect the test to pass on aarch64
targets.
* gcc.dg/vect/vect-outer-4g.c: Likewise.
* gcc.dg/vect/vect-outer-4k.c: Likewise.
* gcc.dg/vect/vect-outer-4l.c: Likewise.
* gfortran.dg/vect/vect-8.f90: Expect kernel 24 to be vectorized
for aarch64.
* gcc.target/aarch64/sve/reduc_strict_3.c: Update the number of
times that "Detected double reduction" is printed.
* gcc.target/aarch64/vect_mixed_sizes_1.c: New test.
* gcc.target/aarch64/vect_mixed_sizes_2.c: Likewise.
* gcc.target/aarch64/vect_mixed_sizes_3.c: Likewise.
* gcc.target/aarch64/vect_mixed_sizes_4.c: Likewise.
Index: gcc/config/aarch64/aarch64.c
===================================================================
--- gcc/config/aarch64/aarch64.c 2019-10-25 13:27:15.505763118 +0100
+++ gcc/config/aarch64/aarch64.c 2019-10-25 13:27:29.685662922 +0100
@@ -1767,6 +1767,30 @@ aarch64_sve_int_mode (machine_mode mode)
return aarch64_sve_data_mode (int_mode, GET_MODE_NUNITS (mode)).require ();
}
+/* Implement TARGET_VECTORIZE_RELATED_MODE. */
+
+static opt_machine_mode
+aarch64_vectorize_related_mode (machine_mode vector_mode,
+ scalar_mode element_mode,
+ poly_uint64 nunits)
+{
+ unsigned int vec_flags = aarch64_classify_vector_mode (vector_mode);
+
+ /* Prefer to use 1 128-bit vector instead of 2 64-bit vectors. */
+ if ((vec_flags & VEC_ADVSIMD)
+ && known_eq (nunits, 0U)
+ && known_eq (GET_MODE_BITSIZE (vector_mode), 64U)
+ && maybe_ge (GET_MODE_BITSIZE (element_mode)
+ * GET_MODE_NUNITS (vector_mode), 128U))
+ {
+ machine_mode res = aarch64_simd_container_mode (element_mode, 128);
+ if (VECTOR_MODE_P (res))
+ return res;
+ }
+
+ return default_vectorize_related_mode (vector_mode, element_mode, nunits);
+}
+
/* Implement TARGET_PREFERRED_ELSE_VALUE. For binary operations,
prefer to use the first arithmetic operand as the else value if
the else value doesn't matter, since that exactly matches the SVE
@@ -15207,8 +15231,27 @@ aarch64_autovectorize_vector_modes (vect
{
if (TARGET_SVE)
modes->safe_push (VNx16QImode);
+
+ /* Try using 128-bit vectors for all element types. */
modes->safe_push (V16QImode);
+
+ /* Try using 64-bit vectors for 8-bit elements and 128-bit vectors
+ for wider elements. */
modes->safe_push (V8QImode);
+
+ /* Try using 64-bit vectors for 16-bit elements and 128-bit vectors
+ for wider elements.
+
+ TODO: We could support a limited form of V4QImode too, so that
+ we use 32-bit vectors for 8-bit elements. */
+ modes->safe_push (V4HImode);
+
+ /* Try using 64-bit vectors for 32-bit elements and 128-bit vectors
+ for 64-bit elements.
+
+ TODO: We could similarly support limited forms of V2QImode and V2HImode
+ for this case. */
+ modes->safe_push (V2SImode);
}
/* Implement TARGET_MANGLE_TYPE. */
@@ -20950,6 +20993,8 @@ #define TARGET_VECTORIZE_VECTOR_ALIGNMEN
#define TARGET_VECTORIZE_VEC_PERM_CONST \
aarch64_vectorize_vec_perm_const
+#undef TARGET_VECTORIZE_RELATED_MODE
+#define TARGET_VECTORIZE_RELATED_MODE aarch64_vectorize_related_mode
#undef TARGET_VECTORIZE_GET_MASK_MODE
#define TARGET_VECTORIZE_GET_MASK_MODE aarch64_get_mask_mode
#undef TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE
Index: gcc/testsuite/gcc.dg/vect/vect-outer-4f.c
===================================================================
--- gcc/testsuite/gcc.dg/vect/vect-outer-4f.c 2019-03-08 18:15:02.304871094 +0000
+++ gcc/testsuite/gcc.dg/vect/vect-outer-4f.c 2019-10-25 13:27:29.685662922 +0100
@@ -65,4 +65,4 @@ int main (void)
return 0;
}
-/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail *-*-* } } } */
+/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail { ! aarch64*-*-* } } } } */
Index: gcc/testsuite/gcc.dg/vect/vect-outer-4g.c
===================================================================
--- gcc/testsuite/gcc.dg/vect/vect-outer-4g.c 2019-03-08 18:15:02.268871230 +0000
+++ gcc/testsuite/gcc.dg/vect/vect-outer-4g.c 2019-10-25 13:27:29.685662922 +0100
@@ -65,4 +65,4 @@ int main (void)
return 0;
}
-/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail *-*-* } } } */
+/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail { ! aarch64*-*-* } } } } */
Index: gcc/testsuite/gcc.dg/vect/vect-outer-4k.c
===================================================================
--- gcc/testsuite/gcc.dg/vect/vect-outer-4k.c 2019-03-08 18:15:02.280871184 +0000
+++ gcc/testsuite/gcc.dg/vect/vect-outer-4k.c 2019-10-25 13:27:29.685662922 +0100
@@ -65,4 +65,4 @@ int main (void)
return 0;
}
-/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail *-*-* } } } */
+/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail { ! aarch64*-*-* } } } } */
Index: gcc/testsuite/gcc.dg/vect/vect-outer-4l.c
===================================================================
--- gcc/testsuite/gcc.dg/vect/vect-outer-4l.c 2019-03-08 18:15:02.240871337 +0000
+++ gcc/testsuite/gcc.dg/vect/vect-outer-4l.c 2019-10-25 13:27:29.685662922 +0100
@@ -65,4 +65,4 @@ int main (void)
return 0;
}
-/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail *-*-* } } } */
+/* { dg-final { scan-tree-dump-times "OUTER LOOP VECTORIZED" 1 "vect" { xfail { ! aarch64*-*-* } } } }*/
Index: gcc/testsuite/gfortran.dg/vect/vect-8.f90
===================================================================
--- gcc/testsuite/gfortran.dg/vect/vect-8.f90 2019-10-07 09:33:30.357955705 +0100
+++ gcc/testsuite/gfortran.dg/vect/vect-8.f90 2019-10-25 13:27:29.689662894 +0100
@@ -704,5 +704,6 @@ CALL track('KERNEL ')
RETURN
END SUBROUTINE kernel
-! { dg-final { scan-tree-dump-times "vectorized 22 loops" 1 "vect" { target vect_intdouble_cvt } } }
-! { dg-final { scan-tree-dump-times "vectorized 17 loops" 1 "vect" { target { ! vect_intdouble_cvt } } } }
+! { dg-final { scan-tree-dump-times "vectorized 23 loops" 1 "vect" { target aarch64*-*-* } } }
+! { dg-final { scan-tree-dump-times "vectorized 22 loops" 1 "vect" { target { vect_intdouble_cvt && { ! aarch64*-*-* } } } } }
+! { dg-final { scan-tree-dump-times "vectorized 17 loops" 1 "vect" { target { { ! vect_intdouble_cvt } && { ! aarch64*-*-* } } } } }
Index: gcc/testsuite/gcc.target/aarch64/sve/reduc_strict_3.c
===================================================================
--- gcc/testsuite/gcc.target/aarch64/sve/reduc_strict_3.c 2019-10-25 10:16:43.638748945 +0100
+++ gcc/testsuite/gcc.target/aarch64/sve/reduc_strict_3.c 2019-10-25 13:27:29.685662922 +0100
@@ -122,8 +122,7 @@ double_reduc3 (float *restrict i, float
/* { dg-final { scan-assembler-times {\tfadda\ts[0-9]+, p[0-7], s[0-9]+, z[0-9]+\.s} 4 } } */
/* { dg-final { scan-assembler-times {\tfadda\td[0-9]+, p[0-7], d[0-9]+, z[0-9]+\.d} 9 } } */
/* 1 reduction each for double_reduc{1,2} and 2 for double_reduc3. Each one
- is reported three times, once for SVE, once for 128-bit AdvSIMD and once
- for 64-bit AdvSIMD. */
-/* { dg-final { scan-tree-dump-times "Detected double reduction" 12 "vect" } } */
+ is reported five times, once for each of the autovectorize_vector_modes. */
+/* { dg-final { scan-tree-dump-times "Detected double reduction" 20 "vect" } } */
/* double_reduc2 has 2 reductions and slp_non_chained_reduc has 3. */
/* { dg-final { scan-tree-dump-times "Detected reduction" 10 "vect" } } */
Index: gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_1.c
===================================================================
--- /dev/null 2019-09-17 11:41:18.176664108 +0100
+++ gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_1.c 2019-10-25 13:27:29.685662922 +0100
@@ -0,0 +1,18 @@
+/* { dg-options "-O2 -ftree-vectorize" } */
+
+#pragma GCC target "+nosve"
+
+#include <stdint.h>
+
+void
+f (int64_t *x, int64_t *y, int32_t *z, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ x[i] += y[i];
+ z[i] += z[i - 2];
+ }
+}
+
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.2d,} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.2s,} 1 } } */
Index: gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_2.c
===================================================================
--- /dev/null 2019-09-17 11:41:18.176664108 +0100
+++ gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_2.c 2019-10-25 13:27:29.685662922 +0100
@@ -0,0 +1,19 @@
+/* { dg-options "-O2 -ftree-vectorize" } */
+
+#pragma GCC target "+nosve"
+
+#include <stdint.h>
+
+void
+f (int32_t *x, int32_t *y, int16_t *z, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ x[i] += y[i];
+ z[i] += z[i - 4];
+ }
+}
+
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.4s,} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.4h,} 1 } } */
+/* { dg-final { scan-assembler-not {\tadd\tv[0-9]+\.2s,} } } */
Index: gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_3.c
===================================================================
--- /dev/null 2019-09-17 11:41:18.176664108 +0100
+++ gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_3.c 2019-10-25 13:27:29.689662894 +0100
@@ -0,0 +1,19 @@
+/* { dg-options "-O2 -ftree-vectorize" } */
+
+#pragma GCC target "+nosve"
+
+#include <stdint.h>
+
+void
+f (int16_t *x, int16_t *y, int8_t *z, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ x[i] += y[i];
+ z[i] += z[i - 8];
+ }
+}
+
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.8h,} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.8b,} 1 } } */
+/* { dg-final { scan-assembler-not {\tadd\tv[0-9]+\.4h,} } } */
Index: gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_4.c
===================================================================
--- /dev/null 2019-09-17 11:41:18.176664108 +0100
+++ gcc/testsuite/gcc.target/aarch64/vect_mixed_sizes_4.c 2019-10-25 13:27:29.689662894 +0100
@@ -0,0 +1,18 @@
+/* { dg-options "-O2 -ftree-vectorize" } */
+
+#pragma GCC target "+nosve"
+
+#include <stdint.h>
+
+void
+f (int64_t *x, int64_t *y, int8_t *z, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ x[i] += y[i];
+ z[i] += z[i - 8];
+ }
+}
+
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.2d,} 4 } } */
+/* { dg-final { scan-assembler-times {\tadd\tv[0-9]+\.8b,} 1 } } */
next prev parent reply other threads:[~2019-10-25 12:46 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-25 12:32 [0/n] Support multiple vector sizes for vectorisation Richard Sandiford
2019-10-25 12:34 ` [6/n] Use build_vector_type_for_mode in get_vectype_for_scalar_type_and_size Richard Sandiford
2019-10-30 14:32 ` Richard Biener
2019-10-25 12:34 ` [7/n] Use consistent compatibility checks in vectorizable_shift Richard Sandiford
2019-10-30 14:33 ` Richard Biener
2019-10-25 12:39 ` [8/n] Replace autovectorize_vector_sizes with autovectorize_vector_modes Richard Sandiford
2019-10-30 14:48 ` Richard Biener
2019-10-30 16:33 ` Richard Sandiford
2019-11-11 10:30 ` Richard Sandiford
2019-11-11 14:33 ` Richard Biener
2019-11-12 17:55 ` Richard Sandiford
2019-11-13 14:32 ` Richard Biener
2019-11-13 16:16 ` Richard Sandiford
2019-10-25 12:41 ` [9/n] Replace vec_info::vector_size with vec_info::vector_mode Richard Sandiford
2019-11-05 12:47 ` Richard Biener
2019-10-25 12:43 ` [10/n] Make less use of get_same_sized_vectype Richard Sandiford
2019-11-05 12:50 ` Richard Biener
2019-11-05 15:34 ` Richard Sandiford
2019-11-05 16:09 ` Richard Biener
2019-10-25 12:44 ` [11/n] Support vectorisation with mixed vector sizes Richard Sandiford
2019-11-05 12:57 ` Richard Biener
2019-11-06 12:38 ` Richard Sandiford
2019-11-12 9:22 ` Richard Biener
2019-10-25 12:49 ` Richard Sandiford [this message]
2019-10-25 12:51 ` [13/n] Allow mixed vector sizes within a single vectorised stmt Richard Sandiford
2019-11-05 12:58 ` Richard Biener
2019-10-25 13:00 ` [14/n] Vectorise conversions between differently-sized integer vectors Richard Sandiford
2019-11-05 13:02 ` Richard Biener
2019-11-06 12:45 ` Richard Sandiford
2019-11-12 9:40 ` Richard Biener
2019-10-29 17:05 ` [15/n] Consider building nodes from scalars in vect_slp_analyze_node_operations Richard Sandiford
2019-11-05 13:07 ` Richard Biener
2019-10-29 17:14 ` [16/n] Apply maximum nunits for BB SLP Richard Sandiford
2019-11-05 13:22 ` Richard Biener
2019-11-05 14:09 ` Richard Sandiford
2019-11-14 12:22 ` Richard Biener
2019-11-05 20:10 ` [10a/n] Require equal type sizes for vectorised calls Richard Sandiford
2019-11-06 9:44 ` Richard Biener
2019-11-05 20:25 ` [11a/n] Avoid retrying with the same vector modes Richard Sandiford
2019-11-06 9:49 ` Richard Biener
2019-11-06 10:21 ` Richard Sandiford
2019-11-06 10:27 ` Richard Biener
2019-11-06 11:02 ` Richard Sandiford
2019-11-06 11:22 ` Richard Biener
2019-11-06 12:47 ` Richard Sandiford
2019-11-12 9:25 ` Richard Biener
2019-11-05 20:45 ` [17/17] Extend can_duplicate_and_interleave_p to mixed-size vectors Richard Sandiford
2019-11-14 12:23 ` Richard Biener
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=mpttv7x802l.fsf@arm.com \
--to=richard.sandiford@arm.com \
--cc=gcc-patches@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).