* [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates
@ 2021-10-13 10:15 Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 01/14] arm: Add new tests for comparison vectorization with Neon and MVE Christophe Lyon
` (14 more replies)
0 siblings, 15 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This is v2 of this patch series, addressing the comments I received.
The changes v1 -> v2 are:
- Patch 3: added an executable test, and updated
check_effective_target_arm_mve_hw
- Patch 4: split into patch 4 and patch 14 (to keep numbering the same
for the other patches)
- Patch 5: updated arm_class_likely_spilled_p as suggested.
- Patch 7: updated test_vector_ops_duplicate in simplify-rtx.c as
suggested.
- Patch 8: added V2DI -> HI/hi mapping in MVE_VPRED/MVE_vpred
iterators, removed now useless mve_vpselq_<supf>v2di, and fixed
mov<mode> expander.
- Patch 9: arm_mode_to_pred_mode now returns opt_machine_mode, removed
useless floating-point checks in vec_cmpu.
- Patch 12: replaced hi with v8bi in v2di load/store instructions
I'll squash patch 2 with patch patch 9 and patch 3 with patch 8.
Original text:
This patch series addresses PR 100757 and 101325 by representing
vectors of predicates (MVE VPR.P0 register) as vectors of booleans
rather than using HImode.
As this implies a lot of mostly mechanical changes, I have tried to
split the patches in a way that should help reviewers, but the split
is a bit artificial.
Patches 1-3 add new tests.
Patches 4-6 are small independent improvements.
Patch 7 implements the predicate qualifier, but does not change any
builtin yet.
Patch 8 is the first of the two main patches, and uses the new
qualifier to describe the vcmp and vpsel builtins that are useful for
auto-vectorization of comparisons.
Patch 9 is the second main patch, which fixes the vcond_mask expander.
Patches 10-13 convert almost all the remaining builtins with HI
operands to use the predicate qualifier. After these, there are still
a few builtins with HI operands left, about which I am not sure: vctp,
vpnot, load-gather and store-scatter with v2di operands. In fact,
patches 11/12 update some STR/LDR qualifiers in a way that breaks
these v2di builtins although existing tests still pass.
Christophe Lyon (14):
arm: Add new tests for comparison vectorization with Neon and MVE
arm: Add tests for PR target/100757
arm: Add tests for PR target/101325
arm: Add GENERAL_AND_VPR_REGS regclass
arm: Add support for VPR_REG in arm_class_likely_spilled_p
arm: Fix mve_vmvnq_n_<supf><mode> argument mode
arm: Implement MVE predicates as vectors of booleans
arm: Implement auto-vectorized MVE comparisons with vectors of boolean
predicates
arm: Fix vcond_mask expander for MVE (PR target/100757)
arm: Convert remaining MVE vcmp builtins to predicate qualifiers
arm: Convert more MVE builtins to predicate qualifiers
arm: Convert more load/store MVE builtins to predicate qualifiers
arm: Convert more MVE/CDE builtins to predicate qualifiers
arm: Add VPR_REG to ALL_REGS
gcc/config/arm/arm-builtins.c | 228 +++--
gcc/config/arm/arm-modes.def | 5 +
gcc/config/arm/arm-protos.h | 3 +-
gcc/config/arm/arm-simd-builtin-types.def | 4 +
gcc/config/arm/arm.c | 130 ++-
gcc/config/arm/arm.h | 5 +-
gcc/config/arm/arm_mve_builtins.def | 746 ++++++++--------
gcc/config/arm/iterators.md | 5 +
gcc/config/arm/mve.md | 832 ++++++++++--------
gcc/config/arm/neon.md | 39 +
gcc/config/arm/vec-common.md | 52 --
gcc/simplify-rtx.c | 26 +-
.../arm/acle/cde-mve-full-assembly.c | 264 +++---
.../gcc.target/arm/simd/mve-vcmp-f32-2.c | 32 +
.../gcc.target/arm/simd/neon-compare-1.c | 78 ++
.../gcc.target/arm/simd/neon-compare-2.c | 13 +
.../gcc.target/arm/simd/neon-compare-3.c | 14 +
.../arm/simd/neon-compare-scalar-1.c | 57 ++
.../gcc.target/arm/simd/neon-vcmp-f16.c | 12 +
.../gcc.target/arm/simd/neon-vcmp-f32-2.c | 15 +
.../gcc.target/arm/simd/neon-vcmp-f32-3.c | 12 +
.../gcc.target/arm/simd/neon-vcmp-f32.c | 12 +
gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c | 22 +
.../gcc.target/arm/simd/pr100757-2.c | 20 +
.../gcc.target/arm/simd/pr100757-3.c | 20 +
.../gcc.target/arm/simd/pr100757-4.c | 19 +
gcc/testsuite/gcc.target/arm/simd/pr100757.c | 19 +
.../gcc.target/arm/simd/pr101325-2.c | 19 +
gcc/testsuite/gcc.target/arm/simd/pr101325.c | 14 +
gcc/testsuite/lib/target-supports.exp | 3 +-
30 files changed, 1611 insertions(+), 1109 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/arm/simd/mve-vcmp-f32-2.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-1.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-2.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-3.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-scalar-1.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f16.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-2.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-3.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757-2.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757-3.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757-4.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr101325-2.c
create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr101325.c
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 01/14] arm: Add new tests for comparison vectorization with Neon and MVE
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 02/14] arm: Add tests for PR target/100757 Christophe Lyon
` (13 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This patch mainly adds Neon tests similar to existing MVE ones,
to make sure we do not break Neon when fixing MVE.
mve-vcmp-f32-2.c is similar to mve-vcmp-f32.c but uses a conditional
with 2.0f and 3.0f constants to help scan-assembler-times.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/testsuite/
* gcc.target/arm/simd/mve-vcmp-f32-2.c: New.
* gcc.target/arm/simd/neon-compare-1.c: New.
* gcc.target/arm/simd/neon-compare-2.c: New.
* gcc.target/arm/simd/neon-compare-3.c: New.
* gcc.target/arm/simd/neon-compare-scalar-1.c: New.
* gcc.target/arm/simd/neon-vcmp-f16.c: New.
* gcc.target/arm/simd/neon-vcmp-f32-2.c: New.
* gcc.target/arm/simd/neon-vcmp-f32-3.c: New.
* gcc.target/arm/simd/neon-vcmp-f32.c: New.
* gcc.target/arm/simd/neon-vcmp.c: New.
diff --git a/gcc/testsuite/gcc.target/arm/simd/mve-vcmp-f32-2.c b/gcc/testsuite/gcc.target/arm/simd/mve-vcmp-f32-2.c
new file mode 100644
index 00000000000..917a95bf141
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/mve-vcmp-f32-2.c
@@ -0,0 +1,32 @@
+/* { dg-do assemble } */
+/* { dg-require-effective-target arm_v8_1m_mve_fp_ok } */
+/* { dg-add-options arm_v8_1m_mve_fp } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+
+#include <stdint.h>
+
+#define NB 4
+
+#define FUNC(OP, NAME) \
+ void test_ ## NAME ##_f (float * __restrict__ dest, float *a, float *b) { \
+ int i; \
+ for (i=0; i<NB; i++) { \
+ dest[i] = (a[i] OP b[i]) ? 2.0f : 3.0f; \
+ } \
+ }
+
+FUNC(==, vcmpeq)
+FUNC(!=, vcmpne)
+FUNC(<, vcmplt)
+FUNC(<=, vcmple)
+FUNC(>, vcmpgt)
+FUNC(>=, vcmpge)
+
+/* { dg-final { scan-assembler-times {\tvcmp.f32\teq, q[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcmp.f32\tne, q[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcmp.f32\tlt, q[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcmp.f32\tle, q[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcmp.f32\tgt, q[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcmp.f32\tge, q[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\t.word\t1073741824\n} 24 } } */ /* Constant 2.0f. */
+/* { dg-final { scan-assembler-times {\t.word\t1077936128\n} 24 } } */ /* Constant 3.0f. */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-compare-1.c b/gcc/testsuite/gcc.target/arm/simd/neon-compare-1.c
new file mode 100644
index 00000000000..2e0222a71f2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-compare-1.c
@@ -0,0 +1,78 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3" } */
+
+#include "mve-compare-1.c"
+
+/* 64-bit vectors. */
+/* vmvn is used by 'ne' comparisons: 3 sizes * 2 (signed/unsigned) * 2
+ (register/zero) = 12. */
+/* { dg-final { scan-assembler-times {\tvmvn\td[0-9]+, d[0-9]+\n} 12 } } */
+
+/* { 8 bits } x { eq, ne, lt, le, gt, ge }. */
+/* ne uses eq, lt/le only apply to comparison with zero, they use gt/ge
+ otherwise. */
+/* { dg-final { scan-assembler-times {\tvceq.i8\td[0-9]+, d[0-9]+, d[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i8\td[0-9]+, d[0-9]+, #0\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvclt.s8\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcle.s8\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s8\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s8\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s8\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s8\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+
+/* { 16 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i16\td[0-9]+, d[0-9]+, d[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i16\td[0-9]+, d[0-9]+, #0\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvclt.s16\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcle.s16\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s16\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s16\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s16\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s16\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+
+/* { 32 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i32\td[0-9]+, d[0-9]+, d[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i32\td[0-9]+, d[0-9]+, #0\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvclt.s32\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcle.s32\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s32\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s32\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s32\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s32\td[0-9]+, d[0-9]+, #0\n} 1 } } */
+
+/* 128-bit vectors. */
+
+/* vmvn is used by 'ne' comparisons. */
+/* { dg-final { scan-assembler-times {\tvmvn\tq[0-9]+, q[0-9]+\n} 12 } } */
+
+/* { 8 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i8\tq[0-9]+, q[0-9]+, #0\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvclt.s8\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcle.s8\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s8\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s8\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+
+/* { 16 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i16\tq[0-9]+, q[0-9]+, #0\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvclt.s16\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcle.s16\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s16\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s16\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+
+/* { 32 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i32\tq[0-9]+, q[0-9]+, #0\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvclt.s32\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcle.s32\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s32\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s32\tq[0-9]+, q[0-9]+, #0\n} 1 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-compare-2.c b/gcc/testsuite/gcc.target/arm/simd/neon-compare-2.c
new file mode 100644
index 00000000000..06f3c14c91e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-compare-2.c
@@ -0,0 +1,13 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+
+#include "mve-compare-2.c"
+
+/* eq, ne, lt, le, gt, ge. */
+/* ne uses eq+vmvn, lt/le use gt/ge with swapped operands. */
+/* { dg-final { scan-assembler-times {\tvceq.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvmvn\tq[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-compare-3.c b/gcc/testsuite/gcc.target/arm/simd/neon-compare-3.c
new file mode 100644
index 00000000000..9c9f108843b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-compare-3.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_2a_fp16_neon_ok } */
+/* { dg-add-options arm_v8_2a_fp16_neon } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+
+#include "mve-compare-3.c"
+
+
+/* eq, ne, lt, le, gt, ge. */
+/* ne uses eq+vmvn, lt/le use gt/ge with swapped operands. */
+/* { dg-final { scan-assembler-times {\tvceq.f16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvmvn\tq[0-9]+, q[0-9]+\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.f16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.f16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-compare-scalar-1.c b/gcc/testsuite/gcc.target/arm/simd/neon-compare-scalar-1.c
new file mode 100644
index 00000000000..0783624a3f2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-compare-scalar-1.c
@@ -0,0 +1,57 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3" } */
+
+#include "mve-compare-scalar-1.c"
+
+/* 64-bit vectors. */
+/* vmvn is used by 'ne' comparisons. */
+/* { dg-final { scan-assembler-times {\tvmvn\td[0-9]+, d[0-9]+\n} 6 } } */
+
+/* { 8 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i8\td[0-9]+, d[0-9]+, d[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s8\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u8\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s8\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u8\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+
+/* { 16 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i16\td[0-9]+, d[0-9]+, d[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s16\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u16\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s16\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u16\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+
+/* { 32 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i32\td[0-9]+, d[0-9]+, d[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s32\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u32\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s32\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u32\td[0-9]+, d[0-9]+, d[0-9]+\n} 2 } } */
+
+/* 128-bit vectors. */
+
+/* vmvn is used by 'ne' comparisons. */
+/* { dg-final { scan-assembler-times {\tvmvn\tq[0-9]+, q[0-9]+\n} 6 } } */
+
+/* { 8 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u8\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+
+/* { 16 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+
+/* { 32 bits } x { eq, ne, lt, le, gt, ge }. */
+/* { dg-final { scan-assembler-times {\tvceq.i32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 4 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f16.c b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f16.c
new file mode 100644
index 00000000000..688fd9a235f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f16.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_2a_fp16_neon_ok } */
+/* { dg-add-options arm_v8_2a_fp16_neon } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+
+#include "mve-vcmp-f16.c"
+
+/* 'ne' uses vceq. */
+/* le and lt use ge and gt with inverted operands. */
+/* { dg-final { scan-assembler-times {\tvceq.f16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.f16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.f16\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-2.c b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-2.c
new file mode 100644
index 00000000000..a22923eb242
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+
+#include "mve-vcmp-f32-2.c"
+
+/* 'ne' uses vceq. */
+/* le and lt use ge and gt with inverted operands. */
+/* { dg-final { scan-assembler-times {\tvceq.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+
+/* { dg-final { scan-assembler-times {\tvmov.f32\tq[0-9]+, #2.0e\+0} 6 } } */
+/* { dg-final { scan-assembler-times {\tvmov.f32\tq[0-9]+, #3.0e\+0} 6 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-3.c b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-3.c
new file mode 100644
index 00000000000..4f12f043d3a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-3.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3" } */
+
+#include "mve-vcmp-f32.c"
+
+/* Should not be vectorized, since we do not use -funsafe-math-optimizations. */
+
+/* { dg-final { scan-assembler-not {\tvceq.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} } } */
+/* { dg-final { scan-assembler-not {\tvcge.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} } } */
+/* { dg-final { scan-assembler-not {\tvcgt.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32.c b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32.c
new file mode 100644
index 00000000000..06e5c4fd1d1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+
+#include "mve-vcmp-f32.c"
+
+/* 'ne' uses vceq. */
+/* le and lt use ge and gt with inverted operands. */
+/* { dg-final { scan-assembler-times {\tvceq.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcge.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.f32\tq[0-9]+, q[0-9]+, q[0-9]+\n} 2 } } */
diff --git a/gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c
new file mode 100644
index 00000000000..f2b92b1be7f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c
@@ -0,0 +1,22 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_neon_ok } */
+/* { dg-add-options arm_neon } */
+/* { dg-additional-options "-O3" } */
+
+#include "mve-vcmp.c"
+
+/* vceq is also used for 'ne' comparisons. */
+/* { dg-final { scan-assembler-times {\tvceq.i[0-9]+\td[0-9]+, d[0-9]+, d[0-9]+\n} 12 } } */
+/* { dg-final { scan-assembler-times {\tvceq.i[0-9]+\tq[0-9]+, q[0-9]+, q[0-9]+\n} 12 } } */
+
+/* lt and le are replaced with the opposite condition, hence the double number
+ of matches for gt and ge. */
+/* { dg-final { scan-assembler-times {\tvcge.s[0-9]+\td[0-9]+, d[0-9]+, d[0-9]+\n} 6 } } */
+/* { dg-final { scan-assembler-times {\tvcge.s[0-9]+\tq[0-9]+, q[0-9]+, q[0-9]+\n} 6 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u[0-9]+\td[0-9]+, d[0-9]+, d[0-9]+\n} 6 } } */
+/* { dg-final { scan-assembler-times {\tvcge.u[0-9]+\tq[0-9]+, q[0-9]+, q[0-9]+\n} 6 } } */
+
+/* { dg-final { scan-assembler-times {\tvcgt.s[0-9]+\td[0-9]+, d[0-9]+, d[0-9]+\n} 6 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.s[0-9]+\tq[0-9]+, q[0-9]+, q[0-9]+\n} 6 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u[0-9]+\td[0-9]+, d[0-9]+, d[0-9]+\n} 6 } } */
+/* { dg-final { scan-assembler-times {\tvcgt.u[0-9]+\tq[0-9]+, q[0-9]+, q[0-9]+\n} 6 } } */
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 02/14] arm: Add tests for PR target/100757
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 01/14] arm: Add new tests for comparison vectorization with Neon and MVE Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 03/14] arm: Add tests for PR target/101325 Christophe Lyon
` (12 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
These tests currently trigger an ICE which is fixed later in the patch
series.
The pr100757*.c testcases are derived from
gcc.c-torture/compile/20160205-1.c, forcing the use of MVE, and using
various types and return values different from 0 and 1 to avoid
commonalization with boolean masks. In addition, since we should not
need these masks, the tests make sure they are not present.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/testsuite/
PR target/100757
* gcc.target/arm/simd/pr100757-2.c: New.
* gcc.target/arm/simd/pr100757-3.c: New.
* gcc.target/arm/simd/pr100757-4.c: New.
* gcc.target/arm/simd/pr100757.c: New.
diff --git a/gcc/testsuite/gcc.target/arm/simd/pr100757-2.c b/gcc/testsuite/gcc.target/arm/simd/pr100757-2.c
new file mode 100644
index 00000000000..c2262b4d81e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/pr100757-2.c
@@ -0,0 +1,20 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_1m_mve_fp_ok } */
+/* { dg-add-options arm_v8_1m_mve_fp } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+/* Derived from gcc.c-torture/compile/20160205-1.c. */
+
+float a[32];
+int fn1(int d) {
+ int c = 4;
+ for (int b = 0; b < 32; b++)
+ if (a[b] != 2.0f)
+ c = 5;
+ return c;
+}
+
+/* { dg-final { scan-assembler-times {\t.word\t1073741824\n} 4 } } */ /* Constant 2.0f. */
+/* { dg-final { scan-assembler-times {\t.word\t4\n} 4 } } */ /* Initial value for c. */
+/* { dg-final { scan-assembler-times {\t.word\t5\n} 4 } } */ /* Possible value for c. */
+/* { dg-final { scan-assembler-not {\t.word\t1\n} } } */ /* 'true' mask. */
+/* { dg-final { scan-assembler-not {\t.word\t0\n} } } */ /* 'false' mask. */
diff --git a/gcc/testsuite/gcc.target/arm/simd/pr100757-3.c b/gcc/testsuite/gcc.target/arm/simd/pr100757-3.c
new file mode 100644
index 00000000000..e604555c04c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/pr100757-3.c
@@ -0,0 +1,20 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_1m_mve_fp_ok } */
+/* { dg-add-options arm_v8_1m_mve_fp } */
+/* { dg-additional-options "-O3 -funsafe-math-optimizations" } */
+/* Copied from gcc.c-torture/compile/20160205-1.c. */
+
+float a[32];
+float fn1(int d) {
+ float c = 4.0f;
+ for (int b = 0; b < 32; b++)
+ if (a[b] != 2.0f)
+ c = 5.0f;
+ return c;
+}
+
+/* { dg-final { scan-assembler-times {\t.word\t1073741824\n} 4 } } */ /* Constant 2.0f. */
+/* { dg-final { scan-assembler-times {\t.word\t1084227584\n} 4 } } */ /* Initial value for c (4.0). */
+/* { dg-final { scan-assembler-times {\t.word\t1082130432\n} 4 } } */ /* Possible value for c (5.0). */
+/* { dg-final { scan-assembler-not {\t.word\t1\n} } } */ /* 'true' mask. */
+/* { dg-final { scan-assembler-not {\t.word\t0\n} } } */ /* 'false' mask. */
diff --git a/gcc/testsuite/gcc.target/arm/simd/pr100757-4.c b/gcc/testsuite/gcc.target/arm/simd/pr100757-4.c
new file mode 100644
index 00000000000..c12040c517f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/pr100757-4.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_1m_mve_ok } */
+/* { dg-add-options arm_v8_1m_mve } */
+/* { dg-additional-options "-O3" } */
+/* Derived from gcc.c-torture/compile/20160205-1.c. */
+
+unsigned int a[32];
+int fn1(int d) {
+ int c = 2;
+ for (int b = 0; b < 32; b++)
+ if (a[b])
+ c = 3;
+ return c;
+}
+
+/* { dg-final { scan-assembler-times {\t.word\t0\n} 4 } } */ /* 'false' mask. */
+/* { dg-final { scan-assembler-not {\t.word\t1\n} } } */ /* 'true' mask. */
+/* { dg-final { scan-assembler-times {\t.word\t2\n} 4 } } */ /* Initial value for c. */
+/* { dg-final { scan-assembler-times {\t.word\t3\n} 4 } } */ /* Possible value for c. */
diff --git a/gcc/testsuite/gcc.target/arm/simd/pr100757.c b/gcc/testsuite/gcc.target/arm/simd/pr100757.c
new file mode 100644
index 00000000000..41d6e4e2d7a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/pr100757.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_1m_mve_ok } */
+/* { dg-add-options arm_v8_1m_mve } */
+/* { dg-additional-options "-O3" } */
+/* Derived from gcc.c-torture/compile/20160205-1.c. */
+
+int a[32];
+int fn1(int d) {
+ int c = 2;
+ for (int b = 0; b < 32; b++)
+ if (a[b])
+ c = 3;
+ return c;
+}
+
+/* { dg-final { scan-assembler-times {\t.word\t0\n} 4 } } */ /* 'false' mask. */
+/* { dg-final { scan-assembler-not {\t.word\t1\n} } } */ /* 'true' mask. */
+/* { dg-final { scan-assembler-times {\t.word\t2\n} 4 } } */ /* Initial value for c. */
+/* { dg-final { scan-assembler-times {\t.word\t3\n} 4 } } */ /* Possible value for c. */
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 03/14] arm: Add tests for PR target/101325
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 01/14] arm: Add new tests for comparison vectorization with Neon and MVE Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 02/14] arm: Add tests for PR target/100757 Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-15 14:49 ` Richard Sandiford
2021-10-13 10:15 ` [PATCH v2 04/14] arm: Add GENERAL_AND_VPR_REGS regclass Christophe Lyon
` (11 subsequent siblings)
14 siblings, 1 reply; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
These tests are derived from the one provided in the PR: there is a
compile-only test because I did not have access to anything that could
execute MVE code until recently.
I have been able to add an executable test since QEMU supports MVE.
Instead of adding arm_v8_1m_mve_hw, I update arm_mve_hw so that it
uses add_options_for_arm_v8_1m_mve_fp, like arm_neon_hw does. This
ensures arm_mve_hw passes even if the toolchain does not generate MVE
code by default.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/testsuite/
PR target/101325
* gcc.target/arm/simd/pr101325.c: New.
* gcc.target/arm/simd/pr101325-2.c: New.
* lib/target-supports.exp (check_effective_target_arm_mve_hw): Use
add_options_for_arm_v8_1m_mve_fp.
add executable test and update check_effective_target_arm_mve_hw
diff --git a/gcc/testsuite/gcc.target/arm/simd/pr101325-2.c b/gcc/testsuite/gcc.target/arm/simd/pr101325-2.c
new file mode 100644
index 00000000000..7907a386385
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/pr101325-2.c
@@ -0,0 +1,19 @@
+/* { dg-do run } */
+/* { dg-require-effective-target arm_mve_hw } */
+/* { dg-options "-O3" } */
+/* { dg-add-options arm_v8_1m_mve } */
+
+#include <arm_mve.h>
+
+
+__attribute((noinline,noipa))
+unsigned foo(int8x16_t v, int8x16_t w)
+{
+ return vcmpeqq (v, w);
+}
+
+int main(void)
+{
+ if (foo (vdupq_n_s8(0), vdupq_n_s8(0)) != 0xffffU)
+ __builtin_abort ();
+}
diff --git a/gcc/testsuite/gcc.target/arm/simd/pr101325.c b/gcc/testsuite/gcc.target/arm/simd/pr101325.c
new file mode 100644
index 00000000000..a466683a0b1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/simd/pr101325.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_v8_1m_mve_ok } */
+/* { dg-add-options arm_v8_1m_mve } */
+/* { dg-additional-options "-O3" } */
+
+#include <arm_mve.h>
+
+unsigned foo(int8x16_t v, int8x16_t w)
+{
+ return vcmpeqq (v, w);
+}
+/* { dg-final { scan-assembler {\tvcmp.i8 eq} } } */
+/* { dg-final { scan-assembler {\tvmrs\t r[0-9]+, P0} } } */
+/* { dg-final { scan-assembler {\tuxth} } } */
diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp
index e030e4f376b..b0e35b602af 100644
--- a/gcc/testsuite/lib/target-supports.exp
+++ b/gcc/testsuite/lib/target-supports.exp
@@ -4889,6 +4889,7 @@ proc check_effective_target_arm_cmse_hw { } {
}
} "-mcmse -Wl,--section-start,.gnu.sgstubs=0x00400000"]
}
+
# Return 1 if the target supports executing MVE instructions, 0
# otherwise.
@@ -4904,7 +4905,7 @@ proc check_effective_target_arm_mve_hw {} {
: "0" (a), "r" (b));
return (a != 2);
}
- } ""]
+ } [add_options_for_arm_v8_1m_mve_fp ""]]
}
# Return 1 if this is an ARM target where ARMv8-M Security Extensions with
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 04/14] arm: Add GENERAL_AND_VPR_REGS regclass
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (2 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 03/14] arm: Add tests for PR target/101325 Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 05/14] arm: Add support for VPR_REG in arm_class_likely_spilled_p Christophe Lyon
` (10 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
At some point during the development of this patch series, it appeared
that in some cases the register allocator wants “VPR or general”
rather than “VPR or general or FP” (which is the same thing as
ALL_REGS). The series does not seem to require this anymore, but it
seems to be a good thing to do anyway, to give the register allocator
more freedom.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
* config/arm/arm.h (reg_class): Add GENERAL_AND_VPR_REGS.
(REG_CLASS_NAMES): Likewise.
(REG_CLASS_CONTENTS): Likewise.
diff --git a/gcc/config/arm/arm.h b/gcc/config/arm/arm.h
index 015299c1534..eae1b1cd0fb 100644
--- a/gcc/config/arm/arm.h
+++ b/gcc/config/arm/arm.h
@@ -1286,6 +1286,7 @@ enum reg_class
SFP_REG,
AFP_REG,
VPR_REG,
+ GENERAL_AND_VPR_REGS,
ALL_REGS,
LIM_REG_CLASSES
};
@@ -1315,6 +1316,7 @@ enum reg_class
"SFP_REG", \
"AFP_REG", \
"VPR_REG", \
+ "GENERAL_AND_VPR_REGS", \
"ALL_REGS" \
}
@@ -1343,6 +1345,7 @@ enum reg_class
{ 0x00000000, 0x00000000, 0x00000000, 0x00000040 }, /* SFP_REG */ \
{ 0x00000000, 0x00000000, 0x00000000, 0x00000080 }, /* AFP_REG */ \
{ 0x00000000, 0x00000000, 0x00000000, 0x00000400 }, /* VPR_REG. */ \
+ { 0x00005FFF, 0x00000000, 0x00000000, 0x00000400 }, /* GENERAL_AND_VPR_REGS. */ \
{ 0xFFFF7FFF, 0xFFFFFFFF, 0xFFFFFFFF, 0x0000000F } /* ALL_REGS. */ \
}
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 05/14] arm: Add support for VPR_REG in arm_class_likely_spilled_p
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (3 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 04/14] arm: Add GENERAL_AND_VPR_REGS regclass Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 06/14] arm: Fix mve_vmvnq_n_<supf><mode> argument mode Christophe Lyon
` (9 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
VPR_REG is the only register in its class, so it should be handled by
TARGET_CLASS_LIKELY_SPILLED_P, which is achieved by calling
default_class_likely_spilled_p. No test fails without this patch, but
it seems it should be implemented.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
* config/arm/arm.c (arm_class_likely_spilled_p): Handle VPR_REG.
diff --git a/gcc/config/arm/arm.c b/gcc/config/arm/arm.c
index 11dafc70067..9f52a152444 100644
--- a/gcc/config/arm/arm.c
+++ b/gcc/config/arm/arm.c
@@ -29307,7 +29307,7 @@ arm_class_likely_spilled_p (reg_class_t rclass)
|| rclass == CC_REG)
return true;
- return false;
+ return default_class_likely_spilled_p (rclass);
}
/* Implements target hook small_register_classes_for_mode_p. */
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 06/14] arm: Fix mve_vmvnq_n_<supf><mode> argument mode
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (4 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 05/14] arm: Add support for VPR_REG in arm_class_likely_spilled_p Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 07/14] arm: Implement MVE predicates as vectors of booleans Christophe Lyon
` (8 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
The vmvnq_n* intrinsics and have [u]int[16|32]_t arguments, so use
<V_elem> iterator instead of HI in mve_vmvnq_n_<supf><mode>.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
* config/arm/mve.md (mve_vmvnq_n_<supf><mode>): Use V_elem mode
for operand 1.
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index e393518ea88..14d17060290 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -617,7 +617,7 @@ (define_insn "mve_vcvtaq_<supf><mode>"
(define_insn "mve_vmvnq_n_<supf><mode>"
[
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
- (unspec:MVE_5 [(match_operand:HI 1 "immediate_operand" "i")]
+ (unspec:MVE_5 [(match_operand:<V_elem> 1 "immediate_operand" "i")]
VMVNQ_N))
]
"TARGET_HAVE_MVE"
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 07/14] arm: Implement MVE predicates as vectors of booleans
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (5 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 06/14] arm: Fix mve_vmvnq_n_<supf><mode> argument mode Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 08/14] arm: Implement auto-vectorized MVE comparisons with vectors of boolean predicates Christophe Lyon
` (7 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This patch implements support for vectors of booleans to support MVE
predicates, instead of HImode. Since the ABI mandates pred16_t (aka
uint16_t) to represent predicates in intrinsics prototypes, we
introduce a new "predicate" type qualifier so that we can map relevant
builtins HImode arguments and return value to the appropriate vector
of booleans (VxBI).
We have to update test_vector_ops_duplicate, because it iterates using
an offset in bytes, where we would need to iterate in bits: we stop
iterating when we reach the end of the vector of booleans.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
PR target/100757
PR target/101325
* config/arm/arm-builtins.c (arm_type_qualifiers): Add qualifier_predicate.
(arm_init_simd_builtin_types): Add new simd types.
(arm_init_builtin): Map predicate vectors arguments to HImode.
(arm_expand_builtin_args): Move HImode predicate arguments to VxBI
rtx. Move return value to HImode rtx.
* config/arm/arm-modes.def (V16BI, V8BI, V4BI): New modes.
* config/arm/arm-simd-builtin-types.def (Pred1x16_t,
Pred2x8_t,Pred4x4_t): New.
* simplify-rtx.c (test_vector_ops_duplicate): Skip vec_merge test
with vectors of booleans.
diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
index 3a9ff8f26b8..771759f0cdd 100644
--- a/gcc/config/arm/arm-builtins.c
+++ b/gcc/config/arm/arm-builtins.c
@@ -92,7 +92,9 @@ enum arm_type_qualifiers
qualifier_lane_pair_index = 0x1000,
/* Lane indices selected in quadtuplets - must be within range of previous
argument = a vector. */
- qualifier_lane_quadtup_index = 0x2000
+ qualifier_lane_quadtup_index = 0x2000,
+ /* MVE vector predicates. */
+ qualifier_predicate = 0x4000
};
/* The qualifier_internal allows generation of a unary builtin from
@@ -1633,6 +1635,13 @@ arm_init_simd_builtin_types (void)
arm_simd_types[Bfloat16x4_t].eltype = arm_bf16_type_node;
arm_simd_types[Bfloat16x8_t].eltype = arm_bf16_type_node;
+ if (TARGET_HAVE_MVE)
+ {
+ arm_simd_types[Pred1x16_t].eltype = unsigned_intHI_type_node;
+ arm_simd_types[Pred2x8_t].eltype = unsigned_intHI_type_node;
+ arm_simd_types[Pred4x4_t].eltype = unsigned_intHI_type_node;
+ }
+
for (i = 0; i < nelts; i++)
{
tree eltype = arm_simd_types[i].eltype;
@@ -1780,6 +1789,11 @@ arm_init_builtin (unsigned int fcode, arm_builtin_datum *d,
if (qualifiers & qualifier_map_mode)
op_mode = d->mode;
+ /* MVE Predicates use HImode as mandated by the ABI: pred16_t is unsigned
+ short. */
+ if (qualifiers & qualifier_predicate)
+ op_mode = HImode;
+
/* For pointers, we want a pointer to the basic type
of the vector. */
if (qualifiers & qualifier_pointer && VECTOR_MODE_P (op_mode))
@@ -3024,6 +3038,11 @@ arm_expand_builtin_args (rtx target, machine_mode map_mode, int fcode,
case ARG_BUILTIN_COPY_TO_REG:
if (POINTER_TYPE_P (TREE_TYPE (arg[argc])))
op[argc] = convert_memory_address (Pmode, op[argc]);
+
+ /* MVE uses mve_pred16_t (aka HImode) for vectors of predicates. */
+ if (GET_MODE_CLASS (mode[argc]) == MODE_VECTOR_BOOL)
+ op[argc] = gen_lowpart (mode[argc], op[argc]);
+
/*gcc_assert (GET_MODE (op[argc]) == mode[argc]); */
if (!(*insn_data[icode].operand[opno].predicate)
(op[argc], mode[argc]))
@@ -3229,6 +3248,13 @@ constant_arg:
else
emit_insn (insn);
+ if (GET_MODE_CLASS (tmode) == MODE_VECTOR_BOOL)
+ {
+ rtx HItarget = gen_reg_rtx (HImode);
+ emit_move_insn (HItarget, gen_lowpart (HImode, target));
+ return HItarget;
+ }
+
return target;
}
diff --git a/gcc/config/arm/arm-modes.def b/gcc/config/arm/arm-modes.def
index a5e74ba3943..b414a709a62 100644
--- a/gcc/config/arm/arm-modes.def
+++ b/gcc/config/arm/arm-modes.def
@@ -84,6 +84,11 @@ VECTOR_MODE (FLOAT, BF, 2); /* V2BF. */
VECTOR_MODE (FLOAT, BF, 4); /* V4BF. */
VECTOR_MODE (FLOAT, BF, 8); /* V8BF. */
+/* Predicates for MVE. */
+VECTOR_BOOL_MODE (V16BI, 16, 2);
+VECTOR_BOOL_MODE (V8BI, 8, 2);
+VECTOR_BOOL_MODE (V4BI, 4, 2);
+
/* Fraction and accumulator vector modes. */
VECTOR_MODES (FRACT, 4); /* V4QQ V2HQ */
VECTOR_MODES (UFRACT, 4); /* V4UQQ V2UHQ */
diff --git a/gcc/config/arm/arm-simd-builtin-types.def b/gcc/config/arm/arm-simd-builtin-types.def
index c19a1b6e3eb..d3987985b4c 100644
--- a/gcc/config/arm/arm-simd-builtin-types.def
+++ b/gcc/config/arm/arm-simd-builtin-types.def
@@ -51,3 +51,7 @@
ENTRY (Bfloat16x2_t, V2BF, none, 32, bfloat16, 20)
ENTRY (Bfloat16x4_t, V4BF, none, 64, bfloat16, 20)
ENTRY (Bfloat16x8_t, V8BF, none, 128, bfloat16, 20)
+
+ ENTRY (Pred1x16_t, V16BI, unsigned, 16, uint16, 21)
+ ENTRY (Pred2x8_t, V8BI, unsigned, 8, uint16, 21)
+ ENTRY (Pred4x4_t, V4BI, unsigned, 4, uint16, 21)
diff --git a/gcc/simplify-rtx.c b/gcc/simplify-rtx.c
index a719f57870f..7b3962339ba 100644
--- a/gcc/simplify-rtx.c
+++ b/gcc/simplify-rtx.c
@@ -7634,17 +7634,23 @@ test_vector_ops_duplicate (machine_mode mode, rtx scalar_reg)
duplicate, last_par));
/* Test a scalar subreg of a VEC_MERGE of a VEC_DUPLICATE. */
- rtx vector_reg = make_test_reg (mode);
- for (unsigned HOST_WIDE_INT i = 0; i < const_nunits; i++)
+ /* Skip this test for vectors of booleans, because offset is in bytes,
+ while vec_merge indices are in elements (usually bits). */
+ if (GET_MODE_CLASS (mode) != MODE_VECTOR_BOOL)
{
- if (i >= HOST_BITS_PER_WIDE_INT)
- break;
- rtx mask = GEN_INT ((HOST_WIDE_INT_1U << i) | (i + 1));
- rtx vm = gen_rtx_VEC_MERGE (mode, duplicate, vector_reg, mask);
- poly_uint64 offset = i * GET_MODE_SIZE (inner_mode);
- ASSERT_RTX_EQ (scalar_reg,
- simplify_gen_subreg (inner_mode, vm,
- mode, offset));
+ rtx vector_reg = make_test_reg (mode);
+ for (unsigned HOST_WIDE_INT i = 0; i < const_nunits; i++)
+ {
+ if (i >= HOST_BITS_PER_WIDE_INT)
+ break;
+ rtx mask = GEN_INT ((HOST_WIDE_INT_1U << i) | (i + 1));
+ rtx vm = gen_rtx_VEC_MERGE (mode, duplicate, vector_reg, mask);
+ poly_uint64 offset = i * GET_MODE_SIZE (inner_mode);
+
+ ASSERT_RTX_EQ (scalar_reg,
+ simplify_gen_subreg (inner_mode, vm,
+ mode, offset));
+ }
}
}
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 08/14] arm: Implement auto-vectorized MVE comparisons with vectors of boolean predicates
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (6 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 07/14] arm: Implement MVE predicates as vectors of booleans Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 09/14] arm: Fix vcond_mask expander for MVE (PR target/100757) Christophe Lyon
` (6 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
We make use of qualifier_predicate to describe MVE builtins
prototypes, restricting to auto-vectorizable vcmp* and vpsel builtins,
as they are exercised by the tests added earlier in the series.
Special handling is needed for mve_vpselq because it has a v2di
variant, which has no natural VPR.P0 representation: we keep HImode
for it.
The vector_compare expansion code is updated to use the right VxBI
mode instead of HI for the result.
New mov patterns are introduced to handle the new modes.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
PR target/100757
PR target/101325
* config/arm/arm-builtins.c (BINOP_PRED_UNONE_UNONE_QUALIFIERS)
(BINOP_PRED_NONE_NONE_QUALIFIERS)
(TERNOP_NONE_NONE_NONE_PRED_QUALIFIERS)
(TERNOP_UNONE_UNONE_UNONE_PRED_QUALIFIERS): New.
* config/arm/arm.c (arm_hard_regno_mode_ok): Handle new VxBI
modes.
(arm_mode_to_pred_mode): New.
(arm_expand_vector_compare): Use the right VxBI mode instead of
HI.
(arm_expand_vcond): Likewise.
* config/arm/arm_mve_builtins.def (vcmpneq_, vcmphiq_, vcmpcsq_)
(vcmpltq_, vcmpleq_, vcmpgtq_, vcmpgeq_, vcmpeqq_, vcmpneq_f)
(vcmpltq_f, vcmpleq_f, vcmpgtq_f, vcmpgeq_f, vcmpeqq_f, vpselq_u)
(vpselq_s, vpselq_f): Use new predicated qualifiers.
* config/arm/iterators.md (MVE_7): New mode iterator.
(MVE_VPRED, MVE_vpred): New attribute iterators.
* config/arm/mve.md (@mve_vcmp<mve_cmp_op>q_<mode>)
(@mve_vcmp<mve_cmp_op>q_f<mode>, @mve_vpselq_<supf><mode>)
(@mve_vpselq_f<mode>): Use MVE_VPRED instead of HI.
(@mve_vpselq_<supf>v2di): Define separately.
(mov<mode>): New expander for VxBI modes.
(mve_mov<mode>): New insn for VxBI modes.
diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
index 771759f0cdd..6e3638869f1 100644
--- a/gcc/config/arm/arm-builtins.c
+++ b/gcc/config/arm/arm-builtins.c
@@ -469,6 +469,12 @@ arm_binop_unone_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define BINOP_UNONE_UNONE_UNONE_QUALIFIERS \
(arm_binop_unone_unone_unone_qualifiers)
+static enum arm_type_qualifiers
+arm_binop_pred_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_predicate, qualifier_unsigned, qualifier_unsigned };
+#define BINOP_PRED_UNONE_UNONE_QUALIFIERS \
+ (arm_binop_pred_unone_unone_qualifiers)
+
static enum arm_type_qualifiers
arm_binop_unone_none_imm_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_none, qualifier_immediate };
@@ -487,6 +493,12 @@ arm_binop_unone_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define BINOP_UNONE_NONE_NONE_QUALIFIERS \
(arm_binop_unone_none_none_qualifiers)
+static enum arm_type_qualifiers
+arm_binop_pred_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_predicate, qualifier_none, qualifier_none };
+#define BINOP_PRED_NONE_NONE_QUALIFIERS \
+ (arm_binop_pred_none_none_qualifiers)
+
static enum arm_type_qualifiers
arm_binop_unone_unone_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_none };
@@ -558,6 +570,12 @@ arm_ternop_none_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define TERNOP_NONE_NONE_NONE_UNONE_QUALIFIERS \
(arm_ternop_none_none_none_unone_qualifiers)
+static enum arm_type_qualifiers
+arm_ternop_none_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_none, qualifier_none, qualifier_none, qualifier_predicate };
+#define TERNOP_NONE_NONE_NONE_PRED_QUALIFIERS \
+ (arm_ternop_none_none_none_pred_qualifiers)
+
static enum arm_type_qualifiers
arm_ternop_none_none_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_immediate, qualifier_unsigned };
@@ -577,6 +595,13 @@ arm_ternop_unone_unone_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define TERNOP_UNONE_UNONE_UNONE_UNONE_QUALIFIERS \
(arm_ternop_unone_unone_unone_unone_qualifiers)
+static enum arm_type_qualifiers
+arm_ternop_unone_unone_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
+ qualifier_predicate };
+#define TERNOP_UNONE_UNONE_UNONE_PRED_QUALIFIERS \
+ (arm_ternop_unone_unone_unone_pred_qualifiers)
+
static enum arm_type_qualifiers
arm_ternop_none_none_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_none, qualifier_none };
diff --git a/gcc/config/arm/arm.c b/gcc/config/arm/arm.c
index 9f52a152444..94696d528f8 100644
--- a/gcc/config/arm/arm.c
+++ b/gcc/config/arm/arm.c
@@ -25304,7 +25304,10 @@ arm_hard_regno_mode_ok (unsigned int regno, machine_mode mode)
return false;
if (IS_VPR_REGNUM (regno))
- return mode == HImode;
+ return mode == HImode
+ || mode == V16BImode
+ || mode == V8BImode
+ || mode == V4BImode;
if (TARGET_THUMB1)
/* For the Thumb we only allow values bigger than SImode in
@@ -30991,6 +30994,19 @@ arm_split_atomic_op (enum rtx_code code, rtx old_out, rtx new_out, rtx mem,
arm_post_atomic_barrier (model);
}
\f
+/* Return the mode for the MVE vector of predicates corresponding to MODE. */
+machine_mode
+arm_mode_to_pred_mode (machine_mode mode)
+{
+ switch (GET_MODE_NUNITS (mode))
+ {
+ case 16: return V16BImode;
+ case 8: return V8BImode;
+ case 4: return V4BImode;
+ }
+ gcc_unreachable ();
+}
+
/* Expand code to compare vectors OP0 and OP1 using condition CODE.
If CAN_INVERT, store either the result or its inverse in TARGET
and return true if TARGET contains the inverse. If !CAN_INVERT,
@@ -31074,7 +31090,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
if (vcond_mve)
vpr_p0 = target;
else
- vpr_p0 = gen_reg_rtx (HImode);
+ vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
switch (GET_MODE_CLASS (cmp_mode))
{
@@ -31116,7 +31132,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
if (vcond_mve)
vpr_p0 = target;
else
- vpr_p0 = gen_reg_rtx (HImode);
+ vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
emit_insn (gen_mve_vcmpq (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
if (!vcond_mve)
@@ -31143,7 +31159,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
if (vcond_mve)
vpr_p0 = target;
else
- vpr_p0 = gen_reg_rtx (HImode);
+ vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
emit_insn (gen_mve_vcmpq (swap_condition (code), cmp_mode, vpr_p0, force_reg (cmp_mode, op1), op0));
if (!vcond_mve)
@@ -31196,7 +31212,7 @@ arm_expand_vcond (rtx *operands, machine_mode cmp_result_mode)
if (TARGET_HAVE_MVE)
{
vcond_mve=true;
- mask = gen_reg_rtx (HImode);
+ mask = gen_reg_rtx (arm_mode_to_pred_mode (cmp_result_mode));
}
else
mask = gen_reg_rtx (cmp_result_mode);
diff --git a/gcc/config/arm/arm_mve_builtins.def b/gcc/config/arm/arm_mve_builtins.def
index e9b5b28f506..58a05e61bd9 100644
--- a/gcc/config/arm/arm_mve_builtins.def
+++ b/gcc/config/arm/arm_mve_builtins.def
@@ -89,7 +89,7 @@ VAR3 (BINOP_UNONE_UNONE_IMM, vshrq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_IMM, vshrq_n_s, v16qi, v8hi, v4si)
VAR1 (BINOP_NONE_NONE_UNONE, vaddlvq_p_s, v4si)
VAR1 (BINOP_UNONE_UNONE_UNONE, vaddlvq_p_u, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpneq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpneq_, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_NONE, vshlq_s, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_NONE, vshlq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vsubq_u, v16qi, v8hi, v4si)
@@ -117,9 +117,9 @@ VAR3 (BINOP_UNONE_UNONE_UNONE, vhsubq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vhaddq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vhaddq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, veorq_u, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_UNONE_UNONE, vcmphiq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_UNONE_UNONE, vcmphiq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vcmphiq_n_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_UNONE_UNONE, vcmpcsq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_UNONE_UNONE, vcmpcsq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vcmpcsq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vbicq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vandq_u, v16qi, v8hi, v4si)
@@ -143,15 +143,15 @@ VAR3 (BINOP_UNONE_UNONE_IMM, vshlq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vrshrq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vqshlq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_NONE, vcmpneq_n_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpltq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpltq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_NONE, vcmpltq_n_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpleq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpleq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_NONE, vcmpleq_n_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpgtq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpgtq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_NONE, vcmpgtq_n_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpgeq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpgeq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_NONE, vcmpgeq_n_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpeqq_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpeqq_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_NONE, vcmpeqq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_IMM, vqshluq_n_s, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_UNONE, vaddvq_p_s, v16qi, v8hi, v4si)
@@ -219,17 +219,17 @@ VAR2 (BINOP_UNONE_UNONE_IMM, vshllbq_n_u, v16qi, v8hi)
VAR2 (BINOP_UNONE_UNONE_IMM, vorrq_n_u, v8hi, v4si)
VAR2 (BINOP_UNONE_UNONE_IMM, vbicq_n_u, v8hi, v4si)
VAR2 (BINOP_UNONE_NONE_NONE, vcmpneq_n_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpneq_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpneq_f, v8hf, v4sf)
VAR2 (BINOP_UNONE_NONE_NONE, vcmpltq_n_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpltq_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpltq_f, v8hf, v4sf)
VAR2 (BINOP_UNONE_NONE_NONE, vcmpleq_n_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpleq_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpleq_f, v8hf, v4sf)
VAR2 (BINOP_UNONE_NONE_NONE, vcmpgtq_n_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpgtq_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpgtq_f, v8hf, v4sf)
VAR2 (BINOP_UNONE_NONE_NONE, vcmpgeq_n_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpgeq_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpgeq_f, v8hf, v4sf)
VAR2 (BINOP_UNONE_NONE_NONE, vcmpeqq_n_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpeqq_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpeqq_f, v8hf, v4sf)
VAR2 (BINOP_NONE_NONE_NONE, vsubq_f, v8hf, v4sf)
VAR2 (BINOP_NONE_NONE_NONE, vqmovntq_s, v8hi, v4si)
VAR2 (BINOP_NONE_NONE_NONE, vqmovnbq_s, v8hi, v4si)
@@ -295,8 +295,8 @@ VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vcvtaq_m_u, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtaq_m_s, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vshlcq_vec_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_UNONE_IMM, vshlcq_vec_s, v16qi, v8hi, v4si)
-VAR4 (TERNOP_UNONE_UNONE_UNONE_UNONE, vpselq_u, v16qi, v8hi, v4si, v2di)
-VAR4 (TERNOP_NONE_NONE_NONE_UNONE, vpselq_s, v16qi, v8hi, v4si, v2di)
+VAR4 (TERNOP_UNONE_UNONE_UNONE_PRED, vpselq_u, v16qi, v8hi, v4si, v2di)
+VAR4 (TERNOP_NONE_NONE_NONE_PRED, vpselq_s, v16qi, v8hi, v4si, v2di)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrev64q_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmvnq_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmlasq_n_u, v16qi, v8hi, v4si)
@@ -426,7 +426,7 @@ VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrev64q_m_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrev32q_m_s, v16qi, v8hi)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vqmovntq_m_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vqmovnbq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vpselq_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vpselq_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vnegq_m_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmovntq_m_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmovnbq_m_s, v8hi, v4si)
diff --git a/gcc/config/arm/iterators.md b/gcc/config/arm/iterators.md
index fafbd2f94b8..b7249f2b189 100644
--- a/gcc/config/arm/iterators.md
+++ b/gcc/config/arm/iterators.md
@@ -272,6 +272,7 @@ (define_mode_iterator MVE_3 [V16QI V8HI])
(define_mode_iterator MVE_2 [V16QI V8HI V4SI])
(define_mode_iterator MVE_5 [V8HI V4SI])
(define_mode_iterator MVE_6 [V8HI V4SI])
+(define_mode_iterator MVE_7 [V16BI V8BI V4BI])
;;----------------------------------------------------------------------------
;; Code iterators
@@ -946,6 +947,10 @@ (define_mode_attr V_extr_elem [(V16QI "u8") (V8HI "u16") (V4SI "32")
(V8HF "u16") (V4SF "32")])
(define_mode_attr earlyclobber_32 [(V16QI "=w") (V8HI "=w") (V4SI "=&w")
(V8HF "=w") (V4SF "=&w")])
+(define_mode_attr MVE_VPRED [(V16QI "V16BI") (V8HI "V8BI") (V4SI "V4BI")
+ (V2DI "HI") (V8HF "V8BI") (V4SF "V4BI")])
+(define_mode_attr MVE_vpred [(V16QI "v16bi") (V8HI "v8bi") (V4SI "v4bi")
+ (V2DI "hi") (V8HF "v8bi") (V4SF "v4bi")])
;;----------------------------------------------------------------------------
;; Code attributes
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index 14d17060290..9da78657798 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -839,8 +839,8 @@ (define_insn "mve_vaddlvq_p_<supf>v4si"
;;
(define_insn "@mve_vcmp<mve_cmp_op>q_<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (MVE_COMPARISONS:HI (match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (MVE_COMPARISONS:<MVE_VPRED> (match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")))
]
"TARGET_HAVE_MVE"
@@ -1929,8 +1929,8 @@ (define_insn "mve_vcaddq<mve_rot><mode>"
;;
(define_insn "@mve_vcmp<mve_cmp_op>q_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (MVE_FP_COMPARISONS:HI (match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (MVE_FP_COMPARISONS:<MVE_VPRED> (match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3324,7 +3324,7 @@ (define_insn "@mve_vpselq_<supf><mode>"
(set (match_operand:MVE_1 0 "s_register_operand" "=w")
(unspec:MVE_1 [(match_operand:MVE_1 1 "s_register_operand" "w")
(match_operand:MVE_1 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VPSELQ))
]
"TARGET_HAVE_MVE"
@@ -4419,7 +4419,7 @@ (define_insn "@mve_vpselq_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VPSELQ_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -10516,3 +10516,27 @@ (define_insn "*movmisalign<mode>_mve_load"
"vldr<V_sz_elem1>.<V_sz_elem>\t%q0, %E1"
[(set_attr "type" "mve_load")]
)
+
+(define_expand "mov<mode>"
+ [(set (match_operand:MVE_7 0 "nonimmediate_operand")
+ (match_operand:MVE_7 1 "nonimmediate_operand"))]
+ "TARGET_HAVE_MVE"
+ {
+ if (!register_operand (operands[0], <MODE>mode))
+ operands[1] = force_reg (<MODE>mode, operands[1]);
+ }
+)
+
+(define_insn "*mve_mov<mode>"
+ [(set (match_operand:MVE_7 0 "nonimmediate_operand" "=rk, m, r, Up, r")
+ (match_operand:MVE_7 1 "nonimmediate_operand" "rk, r, m, r, Up"))]
+ "TARGET_HAVE_MVE
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[1], <MODE>mode))"
+ "@
+ mov%?\t%0, %1
+ strh%?\t%1, %0
+ ldrh%?\t%0, %1
+ vmsr%?\t P0, %1
+ vmrs%?\t %0, P0"
+)
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 09/14] arm: Fix vcond_mask expander for MVE (PR target/100757)
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (7 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 08/14] arm: Implement auto-vectorized MVE comparisons with vectors of boolean predicates Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-15 14:55 ` Richard Sandiford
2021-10-13 10:15 ` [PATCH v2 10/14] arm: Convert remaining MVE vcmp builtins to predicate qualifiers Christophe Lyon
` (5 subsequent siblings)
14 siblings, 1 reply; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
The problem in this PR is that we call VPSEL with a mask of vector
type instead of HImode. This happens because operand 3 in vcond_mask
is the pre-computed vector comparison and has vector type.
This patch fixes it by implementing TARGET_VECTORIZE_GET_MASK_MODE,
returning the appropriate VxBI mode when targeting MVE. In turn, this
implies implementing vec_cmp<mode><MVE_vpred>,
vec_cmpu<mode><MVE_vpred> and vcond_mask_<mode><MVE_vpred>, and we can
move vec_cmp<mode><v_cmp_result>, vec_cmpu<mode><mode> and
vcond_mask_<mode><v_cmp_result> back to neon.md since they are not
used by MVE anymore. The new *<MVE_vpred> patterns listed above are
implemented in mve.md since they are only valid for MVE. However this
may make maintenance/comparison more painful than having all of them
in vec-common.md.
In the process, we can get rid of the recently added vcond_mve
parameter of arm_expand_vector_compare.
Compared to neon.md's vcond_mask_<mode><v_cmp_result> before my "arm:
Auto-vectorization for MVE: vcmp" patch (r12-834), it keeps the VDQWH
iterator added in r12-835 (to have V4HF/V8HF support), as well as the
(!<Is_float_mode> || flag_unsafe_math_optimizations) condition which
was not present before r12-834 although SF modes were enabled by VDQW
(I think this was a bug).
Using TARGET_VECTORIZE_GET_MASK_MODE has the advantage that we no
longer need to generate vpsel with vectors of 0 and 1: the masks are
now merged via scalar 'ands' instructions operating on 16-bit masks
after converting the boolean vectors.
In addition, this patch fixes a problem in arm_expand_vcond() where
the result would be a vector of 0 or 1 instead of operand 1 or 2.
Reducing the number of iterations in pr100757-3.c from 32 to 8, we
generate the code below:
float a[32];
float fn1(int d) {
float c = 4.0f;
for (int b = 0; b < 8; b++)
if (a[b] != 2.0f)
c = 5.0f;
return c;
}
fn1:
ldr r3, .L3+48
vldr.64 d4, .L3 // q2=(2.0,2.0,2.0,2.0)
vldr.64 d5, .L3+8
vldrw.32 q0, [r3] // q0=a(0..3)
adds r3, r3, #16
vcmp.f32 eq, q0, q2 // cmp a(0..3) == (2.0,2.0,2.0,2.0)
vldrw.32 q1, [r3] // q1=a(4..7)
vmrs r3, P0
vcmp.f32 eq, q1, q2 // cmp a(4..7) == (2.0,2.0,2.0,2.0)
vmrs r2, P0 @ movhi
ands r3, r3, r2 // r3=select(a(0..3]) & select(a(4..7))
vldr.64 d4, .L3+16 // q2=(5.0,5.0,5.0,5.0)
vldr.64 d5, .L3+24
vmsr P0, r3
vldr.64 d6, .L3+32 // q3=(4.0,4.0,4.0,4.0)
vldr.64 d7, .L3+40
vpsel q3, q3, q2 // q3=vcond_mask(4.0,5.0)
vmov.32 r2, q3[1] // keep the scalar max
vmov.32 r0, q3[3]
vmov.32 r3, q3[2]
vmov.f32 s11, s12
vmov s15, r2
vmov s14, r3
vmaxnm.f32 s15, s11, s15
vmaxnm.f32 s15, s15, s14
vmov s14, r0
vmaxnm.f32 s15, s15, s14
vmov r0, s15
bx lr
.L4:
.align 3
.L3:
.word 1073741824 // 2.0f
.word 1073741824
.word 1073741824
.word 1073741824
.word 1084227584 // 5.0f
.word 1084227584
.word 1084227584
.word 1084227584
.word 1082130432 // 4.0f
.word 1082130432
.word 1082130432
.word 1082130432
2021-10-13 Christophe Lyon <christophe.lyon@linaro.org>
PR target/100757
gcc/
* config/arm/arm-protos.h (arm_get_mask_mode): New prototype.
(arm_expand_vector_compare): Update prototype.
* config/arm/arm.c (TARGET_VECTORIZE_GET_MASK_MODE): New.
(arm_vector_mode_supported_p): Add support for VxBI modes.
(arm_expand_vector_compare): Remove useless generation of vpsel.
(arm_expand_vcond): Fix select operands.
(arm_get_mask_mode): New.
* config/arm/mve.md (vec_cmp<mode><MVE_vpred>): New.
(vec_cmpu<mode><MVE_vpred>): New.
(vcond_mask_<mode><MVE_vpred>): New.
* config/arm/vec-common.md (vec_cmp<mode><v_cmp_result>)
(vec_cmpu<mode><mode, vcond_mask_<mode><v_cmp_result>): Move to ...
* config/arm/neon.md (vec_cmp<mode><v_cmp_result>)
(vec_cmpu<mode><mode, vcond_mask_<mode><v_cmp_result>): ... here
and disable for MVE.
diff --git a/gcc/config/arm/arm-protos.h b/gcc/config/arm/arm-protos.h
index 9b1f61394ad..9e3d71e0c29 100644
--- a/gcc/config/arm/arm-protos.h
+++ b/gcc/config/arm/arm-protos.h
@@ -201,6 +201,7 @@ extern void arm_init_cumulative_args (CUMULATIVE_ARGS *, tree, rtx, tree);
extern bool arm_pad_reg_upward (machine_mode, tree, int);
#endif
extern int arm_apply_result_size (void);
+extern opt_machine_mode arm_get_mask_mode (machine_mode mode);
#endif /* RTX_CODE */
@@ -372,7 +373,7 @@ extern void arm_emit_coreregs_64bit_shift (enum rtx_code, rtx, rtx, rtx, rtx,
extern bool arm_fusion_enabled_p (tune_params::fuse_ops);
extern bool arm_valid_symbolic_address_p (rtx);
extern bool arm_validize_comparison (rtx *, rtx *, rtx *);
-extern bool arm_expand_vector_compare (rtx, rtx_code, rtx, rtx, bool, bool);
+extern bool arm_expand_vector_compare (rtx, rtx_code, rtx, rtx, bool);
#endif /* RTX_CODE */
extern bool arm_gen_setmem (rtx *);
diff --git a/gcc/config/arm/arm.c b/gcc/config/arm/arm.c
index 94696d528f8..d576dcd1d6c 100644
--- a/gcc/config/arm/arm.c
+++ b/gcc/config/arm/arm.c
@@ -835,6 +835,10 @@ static const struct attribute_spec arm_attribute_table[] =
#undef TARGET_MD_ASM_ADJUST
#define TARGET_MD_ASM_ADJUST arm_md_asm_adjust
+
+#undef TARGET_VECTORIZE_GET_MASK_MODE
+#define TARGET_VECTORIZE_GET_MASK_MODE arm_get_mask_mode
+
\f
/* Obstack for minipool constant handling. */
static struct obstack minipool_obstack;
@@ -29196,7 +29200,8 @@ arm_vector_mode_supported_p (machine_mode mode)
if (TARGET_HAVE_MVE
&& (mode == V2DImode || mode == V4SImode || mode == V8HImode
- || mode == V16QImode))
+ || mode == V16QImode
+ || mode == V16BImode || mode == V8BImode || mode == V4BImode))
return true;
if (TARGET_HAVE_MVE_FLOAT
@@ -30995,7 +31000,7 @@ arm_split_atomic_op (enum rtx_code code, rtx old_out, rtx new_out, rtx mem,
}
\f
/* Return the mode for the MVE vector of predicates corresponding to MODE. */
-machine_mode
+opt_machine_mode
arm_mode_to_pred_mode (machine_mode mode)
{
switch (GET_MODE_NUNITS (mode))
@@ -31004,7 +31009,7 @@ arm_mode_to_pred_mode (machine_mode mode)
case 8: return V8BImode;
case 4: return V4BImode;
}
- gcc_unreachable ();
+ return opt_machine_mode ();
}
/* Expand code to compare vectors OP0 and OP1 using condition CODE.
@@ -31012,16 +31017,12 @@ arm_mode_to_pred_mode (machine_mode mode)
and return true if TARGET contains the inverse. If !CAN_INVERT,
always store the result in TARGET, never its inverse.
- If VCOND_MVE, do not emit the vpsel instruction here, let arm_expand_vcond do
- it with the right destination type to avoid emiting two vpsel, one here and
- one in arm_expand_vcond.
-
Note that the handling of floating-point comparisons is not
IEEE compliant. */
bool
arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
- bool can_invert, bool vcond_mve)
+ bool can_invert)
{
machine_mode cmp_result_mode = GET_MODE (target);
machine_mode cmp_mode = GET_MODE (op0);
@@ -31050,7 +31051,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
and then store its inverse in TARGET. This avoids reusing
TARGET (which for integer NE could be one of the inputs). */
rtx tmp = gen_reg_rtx (cmp_result_mode);
- if (arm_expand_vector_compare (tmp, code, op0, op1, true, vcond_mve))
+ if (arm_expand_vector_compare (tmp, code, op0, op1, true))
gcc_unreachable ();
emit_insn (gen_rtx_SET (target, gen_rtx_NOT (cmp_result_mode, tmp)));
return false;
@@ -31086,36 +31087,20 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
case NE:
if (TARGET_HAVE_MVE)
{
- rtx vpr_p0;
- if (vcond_mve)
- vpr_p0 = target;
- else
- vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
-
switch (GET_MODE_CLASS (cmp_mode))
{
case MODE_VECTOR_INT:
- emit_insn (gen_mve_vcmpq (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
+ emit_insn (gen_mve_vcmpq (code, cmp_mode, target, op0, force_reg (cmp_mode, op1)));
break;
case MODE_VECTOR_FLOAT:
if (TARGET_HAVE_MVE_FLOAT)
- emit_insn (gen_mve_vcmpq_f (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
+ emit_insn (gen_mve_vcmpq_f (code, cmp_mode, target, op0, force_reg (cmp_mode, op1)));
else
gcc_unreachable ();
break;
default:
gcc_unreachable ();
}
-
- /* If we are not expanding a vcond, build the result here. */
- if (!vcond_mve)
- {
- rtx zero = gen_reg_rtx (cmp_result_mode);
- rtx one = gen_reg_rtx (cmp_result_mode);
- emit_move_insn (zero, CONST0_RTX (cmp_result_mode));
- emit_move_insn (one, CONST1_RTX (cmp_result_mode));
- emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, target, one, zero, vpr_p0));
- }
}
else
emit_insn (gen_neon_vc (code, cmp_mode, target, op0, op1));
@@ -31127,23 +31112,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
case GEU:
case GTU:
if (TARGET_HAVE_MVE)
- {
- rtx vpr_p0;
- if (vcond_mve)
- vpr_p0 = target;
- else
- vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
-
- emit_insn (gen_mve_vcmpq (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
- if (!vcond_mve)
- {
- rtx zero = gen_reg_rtx (cmp_result_mode);
- rtx one = gen_reg_rtx (cmp_result_mode);
- emit_move_insn (zero, CONST0_RTX (cmp_result_mode));
- emit_move_insn (one, CONST1_RTX (cmp_result_mode));
- emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, target, one, zero, vpr_p0));
- }
- }
+ emit_insn (gen_mve_vcmpq (code, cmp_mode, target, op0, force_reg (cmp_mode, op1)));
else
emit_insn (gen_neon_vc (code, cmp_mode, target,
op0, force_reg (cmp_mode, op1)));
@@ -31154,23 +31123,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
case LEU:
case LTU:
if (TARGET_HAVE_MVE)
- {
- rtx vpr_p0;
- if (vcond_mve)
- vpr_p0 = target;
- else
- vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
-
- emit_insn (gen_mve_vcmpq (swap_condition (code), cmp_mode, vpr_p0, force_reg (cmp_mode, op1), op0));
- if (!vcond_mve)
- {
- rtx zero = gen_reg_rtx (cmp_result_mode);
- rtx one = gen_reg_rtx (cmp_result_mode);
- emit_move_insn (zero, CONST0_RTX (cmp_result_mode));
- emit_move_insn (one, CONST1_RTX (cmp_result_mode));
- emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, target, one, zero, vpr_p0));
- }
- }
+ emit_insn (gen_mve_vcmpq (swap_condition (code), cmp_mode, target, force_reg (cmp_mode, op1), op0));
else
emit_insn (gen_neon_vc (swap_condition (code), cmp_mode,
target, force_reg (cmp_mode, op1), op0));
@@ -31185,8 +31138,8 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
rtx gt_res = gen_reg_rtx (cmp_result_mode);
rtx alt_res = gen_reg_rtx (cmp_result_mode);
rtx_code alt_code = (code == LTGT ? LT : LE);
- if (arm_expand_vector_compare (gt_res, GT, op0, op1, true, vcond_mve)
- || arm_expand_vector_compare (alt_res, alt_code, op0, op1, true, vcond_mve))
+ if (arm_expand_vector_compare (gt_res, GT, op0, op1, true)
+ || arm_expand_vector_compare (alt_res, alt_code, op0, op1, true))
gcc_unreachable ();
emit_insn (gen_rtx_SET (target, gen_rtx_IOR (cmp_result_mode,
gt_res, alt_res)));
@@ -31206,19 +31159,15 @@ arm_expand_vcond (rtx *operands, machine_mode cmp_result_mode)
{
/* When expanding for MVE, we do not want to emit a (useless) vpsel in
arm_expand_vector_compare, and another one here. */
- bool vcond_mve=false;
rtx mask;
if (TARGET_HAVE_MVE)
- {
- vcond_mve=true;
- mask = gen_reg_rtx (arm_mode_to_pred_mode (cmp_result_mode));
- }
+ mask = gen_reg_rtx (arm_mode_to_pred_mode (cmp_result_mode).require ());
else
mask = gen_reg_rtx (cmp_result_mode);
bool inverted = arm_expand_vector_compare (mask, GET_CODE (operands[3]),
- operands[4], operands[5], true, vcond_mve);
+ operands[4], operands[5], true);
if (inverted)
std::swap (operands[1], operands[2]);
if (TARGET_NEON)
@@ -31226,20 +31175,20 @@ arm_expand_vcond (rtx *operands, machine_mode cmp_result_mode)
mask, operands[1], operands[2]));
else
{
- machine_mode cmp_mode = GET_MODE (operands[4]);
- rtx vpr_p0 = mask;
- rtx zero = gen_reg_rtx (cmp_mode);
- rtx one = gen_reg_rtx (cmp_mode);
- emit_move_insn (zero, CONST0_RTX (cmp_mode));
- emit_move_insn (one, CONST1_RTX (cmp_mode));
+ machine_mode cmp_mode = GET_MODE (operands[0]);
+
switch (GET_MODE_CLASS (cmp_mode))
{
case MODE_VECTOR_INT:
- emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, operands[0], one, zero, vpr_p0));
+ emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_mode, operands[0],
+ operands[1], operands[2], mask));
break;
case MODE_VECTOR_FLOAT:
if (TARGET_HAVE_MVE_FLOAT)
- emit_insn (gen_mve_vpselq_f (cmp_mode, operands[0], one, zero, vpr_p0));
+ emit_insn (gen_mve_vpselq_f (cmp_mode, operands[0],
+ operands[1], operands[2], mask));
+ else
+ gcc_unreachable ();
break;
default:
gcc_unreachable ();
@@ -34149,4 +34098,15 @@ arm_mode_base_reg_class (machine_mode mode)
struct gcc_target targetm = TARGET_INITIALIZER;
+/* Implement TARGET_VECTORIZE_GET_MASK_MODE. */
+
+opt_machine_mode
+arm_get_mask_mode (machine_mode mode)
+{
+ if (TARGET_HAVE_MVE)
+ return arm_mode_to_pred_mode (mode);
+
+ return default_get_mask_mode (mode);
+}
+
#include "gt-arm.h"
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index 9da78657798..fb25cac1cfd 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -10540,3 +10540,57 @@ (define_insn "*mve_mov<mode>"
vmsr%?\t P0, %1
vmrs%?\t %0, P0"
)
+
+;; Expanders for vec_cmp and vcond
+
+(define_expand "vec_cmp<mode><MVE_vpred>"
+ [(set (match_operand:<MVE_VPRED> 0 "s_register_operand")
+ (match_operator:<MVE_VPRED> 1 "comparison_operator"
+ [(match_operand:MVE_VLD_ST 2 "s_register_operand")
+ (match_operand:MVE_VLD_ST 3 "reg_or_zero_operand")]))]
+ "TARGET_HAVE_MVE
+ && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
+{
+ arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
+ operands[2], operands[3], false);
+ DONE;
+})
+
+(define_expand "vec_cmpu<mode><MVE_vpred>"
+ [(set (match_operand:<MVE_VPRED> 0 "s_register_operand")
+ (match_operator:<MVE_VPRED> 1 "comparison_operator"
+ [(match_operand:MVE_2 2 "s_register_operand")
+ (match_operand:MVE_2 3 "reg_or_zero_operand")]))]
+ "TARGET_HAVE_MVE"
+{
+ arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
+ operands[2], operands[3], false);
+ DONE;
+})
+
+(define_expand "vcond_mask_<mode><MVE_vpred>"
+ [(set (match_operand:MVE_VLD_ST 0 "s_register_operand")
+ (if_then_else:MVE_VLD_ST
+ (match_operand:<MVE_VPRED> 3 "s_register_operand")
+ (match_operand:MVE_VLD_ST 1 "s_register_operand")
+ (match_operand:MVE_VLD_ST 2 "s_register_operand")))]
+ "TARGET_HAVE_MVE"
+{
+ switch (GET_MODE_CLASS (<MODE>mode))
+ {
+ case MODE_VECTOR_INT:
+ emit_insn (gen_mve_vpselq (VPSELQ_S, <MODE>mode, operands[0],
+ operands[1], operands[2], operands[3]));
+ break;
+ case MODE_VECTOR_FLOAT:
+ if (TARGET_HAVE_MVE_FLOAT)
+ emit_insn (gen_mve_vpselq_f (<MODE>mode, operands[0],
+ operands[1], operands[2], operands[3]));
+ else
+ gcc_unreachable ();
+ break;
+ default:
+ gcc_unreachable ();
+ }
+ DONE;
+})
diff --git a/gcc/config/arm/neon.md b/gcc/config/arm/neon.md
index 8b0a396947c..28310d93a4e 100644
--- a/gcc/config/arm/neon.md
+++ b/gcc/config/arm/neon.md
@@ -1394,6 +1394,45 @@ (define_insn "*us_sub<mode>_neon"
[(set_attr "type" "neon_qsub<q>")]
)
+(define_expand "vec_cmp<mode><v_cmp_result>"
+ [(set (match_operand:<V_cmp_result> 0 "s_register_operand")
+ (match_operator:<V_cmp_result> 1 "comparison_operator"
+ [(match_operand:VDQWH 2 "s_register_operand")
+ (match_operand:VDQWH 3 "reg_or_zero_operand")]))]
+ "TARGET_NEON
+ && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
+{
+ arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
+ operands[2], operands[3], false);
+ DONE;
+})
+
+(define_expand "vec_cmpu<mode><mode>"
+ [(set (match_operand:VDQIW 0 "s_register_operand")
+ (match_operator:VDQIW 1 "comparison_operator"
+ [(match_operand:VDQIW 2 "s_register_operand")
+ (match_operand:VDQIW 3 "reg_or_zero_operand")]))]
+ "TARGET_NEON"
+{
+ arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
+ operands[2], operands[3], false);
+ DONE;
+})
+
+(define_expand "vcond_mask_<mode><v_cmp_result>"
+ [(set (match_operand:VDQWH 0 "s_register_operand")
+ (if_then_else:VDQWH
+ (match_operand:<V_cmp_result> 3 "s_register_operand")
+ (match_operand:VDQWH 1 "s_register_operand")
+ (match_operand:VDQWH 2 "s_register_operand")))]
+ "TARGET_NEON
+ && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
+{
+ emit_insn (gen_neon_vbsl<mode> (operands[0], operands[3], operands[1],
+ operands[2]));
+ DONE;
+})
+
;; Patterns for builtins.
; good for plain vadd, vaddq.
diff --git a/gcc/config/arm/vec-common.md b/gcc/config/arm/vec-common.md
index 68de4f0f943..9b461a76155 100644
--- a/gcc/config/arm/vec-common.md
+++ b/gcc/config/arm/vec-common.md
@@ -363,33 +363,6 @@ (define_expand "vlshr<mode>3"
}
})
-(define_expand "vec_cmp<mode><v_cmp_result>"
- [(set (match_operand:<V_cmp_result> 0 "s_register_operand")
- (match_operator:<V_cmp_result> 1 "comparison_operator"
- [(match_operand:VDQWH 2 "s_register_operand")
- (match_operand:VDQWH 3 "reg_or_zero_operand")]))]
- "ARM_HAVE_<MODE>_ARITH
- && !TARGET_REALLY_IWMMXT
- && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
-{
- arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
- operands[2], operands[3], false, false);
- DONE;
-})
-
-(define_expand "vec_cmpu<mode><mode>"
- [(set (match_operand:VDQIW 0 "s_register_operand")
- (match_operator:VDQIW 1 "comparison_operator"
- [(match_operand:VDQIW 2 "s_register_operand")
- (match_operand:VDQIW 3 "reg_or_zero_operand")]))]
- "ARM_HAVE_<MODE>_ARITH
- && !TARGET_REALLY_IWMMXT"
-{
- arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
- operands[2], operands[3], false, false);
- DONE;
-})
-
;; Conditional instructions. These are comparisons with conditional moves for
;; vectors. They perform the assignment:
;;
@@ -461,31 +434,6 @@ (define_expand "vcondu<mode><v_cmp_result>"
DONE;
})
-(define_expand "vcond_mask_<mode><v_cmp_result>"
- [(set (match_operand:VDQWH 0 "s_register_operand")
- (if_then_else:VDQWH
- (match_operand:<V_cmp_result> 3 "s_register_operand")
- (match_operand:VDQWH 1 "s_register_operand")
- (match_operand:VDQWH 2 "s_register_operand")))]
- "ARM_HAVE_<MODE>_ARITH
- && !TARGET_REALLY_IWMMXT
- && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
-{
- if (TARGET_NEON)
- {
- emit_insn (gen_neon_vbsl (<MODE>mode, operands[0], operands[3],
- operands[1], operands[2]));
- }
- else if (TARGET_HAVE_MVE)
- {
- emit_insn (gen_mve_vpselq (VPSELQ_S, <MODE>mode, operands[0],
- operands[1], operands[2], operands[3]));
- }
- else
- gcc_unreachable ();
- DONE;
-})
-
(define_expand "vec_load_lanesoi<mode>"
[(set (match_operand:OI 0 "s_register_operand")
(unspec:OI [(match_operand:OI 1 "neon_struct_operand")
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 10/14] arm: Convert remaining MVE vcmp builtins to predicate qualifiers
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (8 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 09/14] arm: Fix vcond_mask expander for MVE (PR target/100757) Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 11/14] arm: Convert more MVE " Christophe Lyon
` (4 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This is mostly a mechanical change, only tested by the intrinsics
expansion tests.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
PR target/100757
PR target/101325
* config/arm/arm-builtins.c (BINOP_UNONE_NONE_NONE_QUALIFIERS):
Delete.
(TERNOP_UNONE_NONE_NONE_UNONE_QUALIFIERS): Change to ...
(TERNOP_PRED_NONE_NONE_PRED_QUALIFIERS): ... this.
(TERNOP_PRED_UNONE_UNONE_PRED_QUALIFIERS): New.
* config/arm/arm_mve_builtins.def (vcmp*q_n_, vcmp*q_m_f): Use new
predicated qualifiers.
* config/arm/mve.md (mve_vcmp<mve_cmp_op>q_n_<mode>)
(mve_vcmp*q_m_f<mode>): Use MVE_VPRED instead of HI.
diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
index 6e3638869f1..b3455d87d4f 100644
--- a/gcc/config/arm/arm-builtins.c
+++ b/gcc/config/arm/arm-builtins.c
@@ -487,12 +487,6 @@ arm_binop_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define BINOP_NONE_NONE_UNONE_QUALIFIERS \
(arm_binop_none_none_unone_qualifiers)
-static enum arm_type_qualifiers
-arm_binop_unone_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_unsigned, qualifier_none, qualifier_none };
-#define BINOP_UNONE_NONE_NONE_QUALIFIERS \
- (arm_binop_unone_none_none_qualifiers)
-
static enum arm_type_qualifiers
arm_binop_pred_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_predicate, qualifier_none, qualifier_none };
@@ -553,10 +547,10 @@ arm_ternop_unone_unone_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
(arm_ternop_unone_unone_imm_unone_qualifiers)
static enum arm_type_qualifiers
-arm_ternop_unone_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_unsigned, qualifier_none, qualifier_none, qualifier_unsigned };
-#define TERNOP_UNONE_NONE_NONE_UNONE_QUALIFIERS \
- (arm_ternop_unone_none_none_unone_qualifiers)
+arm_ternop_pred_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_predicate, qualifier_none, qualifier_none, qualifier_predicate };
+#define TERNOP_PRED_NONE_NONE_PRED_QUALIFIERS \
+ (arm_ternop_pred_none_none_pred_qualifiers)
static enum arm_type_qualifiers
arm_ternop_none_none_none_imm_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -602,6 +596,13 @@ arm_ternop_unone_unone_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define TERNOP_UNONE_UNONE_UNONE_PRED_QUALIFIERS \
(arm_ternop_unone_unone_unone_pred_qualifiers)
+static enum arm_type_qualifiers
+arm_ternop_pred_unone_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_predicate, qualifier_unsigned, qualifier_unsigned,
+ qualifier_predicate };
+#define TERNOP_PRED_UNONE_UNONE_PRED_QUALIFIERS \
+ (arm_ternop_pred_unone_unone_pred_qualifiers)
+
static enum arm_type_qualifiers
arm_ternop_none_none_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_none, qualifier_none };
diff --git a/gcc/config/arm/arm_mve_builtins.def b/gcc/config/arm/arm_mve_builtins.def
index 58a05e61bd9..91ed2073918 100644
--- a/gcc/config/arm/arm_mve_builtins.def
+++ b/gcc/config/arm/arm_mve_builtins.def
@@ -118,9 +118,9 @@ VAR3 (BINOP_UNONE_UNONE_UNONE, vhaddq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vhaddq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, veorq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_UNONE_UNONE, vcmphiq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_UNONE_UNONE, vcmphiq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_UNONE_UNONE, vcmphiq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_UNONE_UNONE, vcmpcsq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_UNONE_UNONE, vcmpcsq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_UNONE_UNONE, vcmpcsq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vbicq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vandq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vaddvq_p_u, v16qi, v8hi, v4si)
@@ -142,17 +142,17 @@ VAR3 (BINOP_UNONE_UNONE_NONE, vbrsrq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vshlq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vrshrq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vqshlq_n_u, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpneq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpneq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpltq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpltq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpltq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpleq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpleq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpleq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpgtq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpgtq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpgtq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpgeq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpgeq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpgeq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpeqq_, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_NONE_NONE, vcmpeqq_n_, v16qi, v8hi, v4si)
+VAR3 (BINOP_PRED_NONE_NONE, vcmpeqq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_IMM, vqshluq_n_s, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_UNONE, vaddvq_p_s, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_NONE, vsubq_s, v16qi, v8hi, v4si)
@@ -218,17 +218,17 @@ VAR2 (BINOP_UNONE_UNONE_IMM, vshlltq_n_u, v16qi, v8hi)
VAR2 (BINOP_UNONE_UNONE_IMM, vshllbq_n_u, v16qi, v8hi)
VAR2 (BINOP_UNONE_UNONE_IMM, vorrq_n_u, v8hi, v4si)
VAR2 (BINOP_UNONE_UNONE_IMM, vbicq_n_u, v8hi, v4si)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpneq_n_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpneq_n_f, v8hf, v4sf)
VAR2 (BINOP_PRED_NONE_NONE, vcmpneq_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpltq_n_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpltq_n_f, v8hf, v4sf)
VAR2 (BINOP_PRED_NONE_NONE, vcmpltq_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpleq_n_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpleq_n_f, v8hf, v4sf)
VAR2 (BINOP_PRED_NONE_NONE, vcmpleq_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpgtq_n_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpgtq_n_f, v8hf, v4sf)
VAR2 (BINOP_PRED_NONE_NONE, vcmpgtq_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpgeq_n_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpgeq_n_f, v8hf, v4sf)
VAR2 (BINOP_PRED_NONE_NONE, vcmpgeq_f, v8hf, v4sf)
-VAR2 (BINOP_UNONE_NONE_NONE, vcmpeqq_n_f, v8hf, v4sf)
+VAR2 (BINOP_PRED_NONE_NONE, vcmpeqq_n_f, v8hf, v4sf)
VAR2 (BINOP_PRED_NONE_NONE, vcmpeqq_f, v8hf, v4sf)
VAR2 (BINOP_NONE_NONE_NONE, vsubq_f, v8hf, v4sf)
VAR2 (BINOP_NONE_NONE_NONE, vqmovntq_s, v8hi, v4si)
@@ -285,7 +285,7 @@ VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlaldavhaq_s, v4si)
VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrmlaldavhaq_u, v4si)
VAR2 (TERNOP_NONE_NONE_UNONE_UNONE, vcvtq_m_to_f_u, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtq_m_to_f_s, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpeqq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_f, v8hf, v4sf)
VAR3 (TERNOP_UNONE_NONE_UNONE_IMM, vshlcq_carry_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vshlcq_carry_u, v16qi, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqrshrunbq_n_s, v8hi, v4si)
@@ -306,14 +306,14 @@ VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmladavaq_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vminvq_p_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmaxvq_p_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vdupq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmpneq_m_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmpneq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmphiq_m_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmphiq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmpeqq_m_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmpeqq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmpcsq_m_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vcmpcsq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpneq_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpneq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmphiq_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmphiq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpeqq_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpeqq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpcsq_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpcsq_m_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vclzq_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vaddvaq_p_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vsriq_n_u, v16qi, v8hi, v4si)
@@ -326,18 +326,18 @@ VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vminavq_p_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vminaq_m_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vmaxavq_p_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vmaxaq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpneq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpneq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpltq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpltq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpleq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpleq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgtq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgtq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgeq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgeq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpeqq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpeqq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpltq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpltq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpleq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpleq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpgtq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpgtq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_n_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vshlq_m_r_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vrshlq_m_n_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vrev64q_m_s, v16qi, v8hi, v4si)
@@ -405,17 +405,17 @@ VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqshrunbq_n_s, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqrshruntq_n_s, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_IMM_UNONE, vorrq_m_n_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_IMM_UNONE, vmvnq_m_n_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpneq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpneq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpltq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpltq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpleq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpleq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgtq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgtq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgeq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpgeq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_UNONE_NONE_NONE_UNONE, vcmpeqq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpltq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpltq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpleq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpleq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgtq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgtq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_n_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndxq_m_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndq_m_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndpq_m_f, v8hf, v4sf)
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index fb25cac1cfd..4766bdd74f7 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -853,8 +853,8 @@ (define_insn "@mve_vcmp<mve_cmp_op>q_<mode>"
;;
(define_insn "mve_vcmp<mve_cmp_op>q_n_<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (MVE_COMPARISONS:HI (match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (MVE_COMPARISONS:<MVE_VPRED> (match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")))
]
"TARGET_HAVE_MVE"
@@ -1943,8 +1943,8 @@ (define_insn "@mve_vcmp<mve_cmp_op>q_f<mode>"
;;
(define_insn "@mve_vcmp<mve_cmp_op>q_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (MVE_FP_COMPARISONS:HI (match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (MVE_FP_COMPARISONS:<MVE_VPRED> (match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -2593,10 +2593,10 @@ (define_insn "mve_vbicq_m_n_<supf><mode>"
;;
(define_insn "mve_vcmpeqq_m_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPEQQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -2809,10 +2809,10 @@ (define_insn "mve_vclzq_m_<supf><mode>"
;;
(define_insn "mve_vcmpcsq_m_n_u<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPCSQ_M_N_U))
]
"TARGET_HAVE_MVE"
@@ -2825,10 +2825,10 @@ (define_insn "mve_vcmpcsq_m_n_u<mode>"
;;
(define_insn "mve_vcmpcsq_m_u<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPCSQ_M_U))
]
"TARGET_HAVE_MVE"
@@ -2841,10 +2841,10 @@ (define_insn "mve_vcmpcsq_m_u<mode>"
;;
(define_insn "mve_vcmpeqq_m_n_<supf><mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPEQQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -2857,10 +2857,10 @@ (define_insn "mve_vcmpeqq_m_n_<supf><mode>"
;;
(define_insn "mve_vcmpeqq_m_<supf><mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPEQQ_M))
]
"TARGET_HAVE_MVE"
@@ -2873,10 +2873,10 @@ (define_insn "mve_vcmpeqq_m_<supf><mode>"
;;
(define_insn "mve_vcmpgeq_m_n_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGEQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -2889,10 +2889,10 @@ (define_insn "mve_vcmpgeq_m_n_s<mode>"
;;
(define_insn "mve_vcmpgeq_m_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGEQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -2905,10 +2905,10 @@ (define_insn "mve_vcmpgeq_m_s<mode>"
;;
(define_insn "mve_vcmpgtq_m_n_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGTQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -2921,10 +2921,10 @@ (define_insn "mve_vcmpgtq_m_n_s<mode>"
;;
(define_insn "mve_vcmpgtq_m_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGTQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -2937,10 +2937,10 @@ (define_insn "mve_vcmpgtq_m_s<mode>"
;;
(define_insn "mve_vcmphiq_m_n_u<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPHIQ_M_N_U))
]
"TARGET_HAVE_MVE"
@@ -2953,10 +2953,10 @@ (define_insn "mve_vcmphiq_m_n_u<mode>"
;;
(define_insn "mve_vcmphiq_m_u<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPHIQ_M_U))
]
"TARGET_HAVE_MVE"
@@ -2969,10 +2969,10 @@ (define_insn "mve_vcmphiq_m_u<mode>"
;;
(define_insn "mve_vcmpleq_m_n_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLEQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -2985,10 +2985,10 @@ (define_insn "mve_vcmpleq_m_n_s<mode>"
;;
(define_insn "mve_vcmpleq_m_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLEQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3001,10 +3001,10 @@ (define_insn "mve_vcmpleq_m_s<mode>"
;;
(define_insn "mve_vcmpltq_m_n_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLTQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -3017,10 +3017,10 @@ (define_insn "mve_vcmpltq_m_n_s<mode>"
;;
(define_insn "mve_vcmpltq_m_s<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLTQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3033,10 +3033,10 @@ (define_insn "mve_vcmpltq_m_s<mode>"
;;
(define_insn "mve_vcmpneq_m_n_<supf><mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPNEQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -3049,10 +3049,10 @@ (define_insn "mve_vcmpneq_m_n_<supf><mode>"
;;
(define_insn "mve_vcmpneq_m_<supf><mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_2 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPNEQ_M))
]
"TARGET_HAVE_MVE"
@@ -3770,10 +3770,10 @@ (define_insn "mve_vcmlaq<mve_rot><mode>"
;;
(define_insn "mve_vcmpeqq_m_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPEQQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3786,10 +3786,10 @@ (define_insn "mve_vcmpeqq_m_n_f<mode>"
;;
(define_insn "mve_vcmpgeq_m_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGEQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3802,10 +3802,10 @@ (define_insn "mve_vcmpgeq_m_f<mode>"
;;
(define_insn "mve_vcmpgeq_m_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGEQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3818,10 +3818,10 @@ (define_insn "mve_vcmpgeq_m_n_f<mode>"
;;
(define_insn "mve_vcmpgtq_m_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGTQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3834,10 +3834,10 @@ (define_insn "mve_vcmpgtq_m_f<mode>"
;;
(define_insn "mve_vcmpgtq_m_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPGTQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3850,10 +3850,10 @@ (define_insn "mve_vcmpgtq_m_n_f<mode>"
;;
(define_insn "mve_vcmpleq_m_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLEQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3866,10 +3866,10 @@ (define_insn "mve_vcmpleq_m_f<mode>"
;;
(define_insn "mve_vcmpleq_m_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLEQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3882,10 +3882,10 @@ (define_insn "mve_vcmpleq_m_n_f<mode>"
;;
(define_insn "mve_vcmpltq_m_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLTQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3898,10 +3898,10 @@ (define_insn "mve_vcmpltq_m_f<mode>"
;;
(define_insn "mve_vcmpltq_m_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPLTQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3914,10 +3914,10 @@ (define_insn "mve_vcmpltq_m_n_f<mode>"
;;
(define_insn "mve_vcmpneq_m_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPNEQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3930,10 +3930,10 @@ (define_insn "mve_vcmpneq_m_f<mode>"
;;
(define_insn "mve_vcmpneq_m_n_f<mode>"
[
- (set (match_operand:HI 0 "vpr_register_operand" "=Up")
- (unspec:HI [(match_operand:MVE_0 1 "s_register_operand" "w")
+ (set (match_operand:<MVE_VPRED> 0 "vpr_register_operand" "=Up")
+ (unspec:<MVE_VPRED> [(match_operand:MVE_0 1 "s_register_operand" "w")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCMPNEQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 11/14] arm: Convert more MVE builtins to predicate qualifiers
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (9 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 10/14] arm: Convert remaining MVE vcmp builtins to predicate qualifiers Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 12/14] arm: Convert more load/store " Christophe Lyon
` (3 subsequent siblings)
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This patch covers all builtins that have an HI operand and use the
<mode> iterator, thus we can replace HI whe <MVE_vpred>.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
PR target/100757
PR target/101325
* config/arm/arm-builtins.c (TERNOP_UNONE_UNONE_NONE_UNONE_QUALIFIERS): Change to ...
(TERNOP_UNONE_UNONE_NONE_PRED_QUALIFIERS): ... this.
(TERNOP_UNONE_UNONE_IMM_UNONE_QUALIFIERS): Change to ...
(TERNOP_UNONE_UNONE_IMM_PRED_QUALIFIERS): ... this.
(TERNOP_NONE_NONE_IMM_UNONE_QUALIFIERS): Change to ...
(TERNOP_NONE_NONE_IMM_PRED_QUALIFIERS): ... this.
(TERNOP_NONE_NONE_UNONE_UNONE_QUALIFIERS): Change to ...
(TERNOP_NONE_NONE_UNONE_PRED_QUALIFIERS): ... this.
(QUADOP_UNONE_UNONE_NONE_NONE_UNONE_QUALIFIERS): Change to ...
(QUADOP_UNONE_UNONE_NONE_NONE_PRED_QUALIFIERS): ... this.
(QUADOP_NONE_NONE_NONE_NONE_PRED_QUALIFIERS): New.
(QUADOP_NONE_NONE_NONE_IMM_UNONE_QUALIFIERS): Change to ...
(QUADOP_NONE_NONE_NONE_IMM_PRED_QUALIFIERS): ... this.
(QUADOP_UNONE_UNONE_UNONE_UNONE_PRED_QUALIFIERS): New.
(QUADOP_UNONE_UNONE_NONE_IMM_UNONE_QUALIFIERS): Change to ...
(QUADOP_UNONE_UNONE_NONE_IMM_PRED_QUALIFIERS): ... this.
(QUADOP_NONE_NONE_UNONE_IMM_UNONE_QUALIFIERS): Change to ...
(QUADOP_NONE_NONE_UNONE_IMM_PRED_QUALIFIERS): ... this.
(QUADOP_UNONE_UNONE_UNONE_IMM_UNONE_QUALIFIERS): Change to ...
(QUADOP_UNONE_UNONE_UNONE_IMM_PRED_QUALIFIERS): ... this.
(QUADOP_UNONE_UNONE_UNONE_NONE_UNONE_QUALIFIERS): Change to ...
(QUADOP_UNONE_UNONE_UNONE_NONE_PRED_QUALIFIERS): ... this.
(STRS_P_QUALIFIERS): Use predicate qualifier.
(STRU_P_QUALIFIERS): Likewise.
(STRSU_P_QUALIFIERS): Likewise.
(STRSS_P_QUALIFIERS): Likewise.
(LDRGS_Z_QUALIFIERS): Likewise.
(LDRGU_Z_QUALIFIERS): Likewise.
(LDRS_Z_QUALIFIERS): Likewise.
(LDRU_Z_QUALIFIERS): Likewise.
(QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_UNONE_QUALIFIERS): Change to ...
(QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_PRED_QUALIFIERS): ... this.
(BINOP_NONE_NONE_PRED_QUALIFIERS): New.
(BINOP_UNONE_UNONE_PRED_QUALIFIERS): New.
* config/arm/arm_mve_builtins.def: Use new predicated qualifiers.
* config/arm/mve.md: Use MVE_VPRED instead of HI.
diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
index b3455d87d4f..06ff9d2278a 100644
--- a/gcc/config/arm/arm-builtins.c
+++ b/gcc/config/arm/arm-builtins.c
@@ -533,18 +533,18 @@ arm_ternop_unone_unone_none_imm_qualifiers[SIMD_MAX_BUILTIN_ARGS]
(arm_ternop_unone_unone_none_imm_qualifiers)
static enum arm_type_qualifiers
-arm_ternop_unone_unone_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_ternop_unone_unone_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_none,
- qualifier_unsigned };
-#define TERNOP_UNONE_UNONE_NONE_UNONE_QUALIFIERS \
- (arm_ternop_unone_unone_none_unone_qualifiers)
+ qualifier_predicate };
+#define TERNOP_UNONE_UNONE_NONE_PRED_QUALIFIERS \
+ (arm_ternop_unone_unone_none_pred_qualifiers)
static enum arm_type_qualifiers
-arm_ternop_unone_unone_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_ternop_unone_unone_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned };
-#define TERNOP_UNONE_UNONE_IMM_UNONE_QUALIFIERS \
- (arm_ternop_unone_unone_imm_unone_qualifiers)
+ qualifier_predicate };
+#define TERNOP_UNONE_UNONE_IMM_PRED_QUALIFIERS \
+ (arm_ternop_unone_unone_imm_pred_qualifiers)
static enum arm_type_qualifiers
arm_ternop_pred_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -571,16 +571,16 @@ arm_ternop_none_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
(arm_ternop_none_none_none_pred_qualifiers)
static enum arm_type_qualifiers
-arm_ternop_none_none_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_none, qualifier_none, qualifier_immediate, qualifier_unsigned };
-#define TERNOP_NONE_NONE_IMM_UNONE_QUALIFIERS \
- (arm_ternop_none_none_imm_unone_qualifiers)
+arm_ternop_none_none_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_none, qualifier_none, qualifier_immediate, qualifier_predicate };
+#define TERNOP_NONE_NONE_IMM_PRED_QUALIFIERS \
+ (arm_ternop_none_none_imm_pred_qualifiers)
static enum arm_type_qualifiers
-arm_ternop_none_none_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_none, qualifier_none, qualifier_unsigned, qualifier_unsigned };
-#define TERNOP_NONE_NONE_UNONE_UNONE_QUALIFIERS \
- (arm_ternop_none_none_unone_unone_qualifiers)
+arm_ternop_none_none_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_none, qualifier_none, qualifier_unsigned, qualifier_predicate };
+#define TERNOP_NONE_NONE_UNONE_PRED_QUALIFIERS \
+ (arm_ternop_none_none_unone_pred_qualifiers)
static enum arm_type_qualifiers
arm_ternop_unone_unone_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -610,11 +610,11 @@ arm_ternop_none_none_none_none_qualifiers[SIMD_MAX_BUILTIN_ARGS]
(arm_ternop_none_none_none_none_qualifiers)
static enum arm_type_qualifiers
-arm_quadop_unone_unone_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quadop_unone_unone_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_none, qualifier_none,
- qualifier_unsigned };
-#define QUADOP_UNONE_UNONE_NONE_NONE_UNONE_QUALIFIERS \
- (arm_quadop_unone_unone_none_none_unone_qualifiers)
+ qualifier_predicate };
+#define QUADOP_UNONE_UNONE_NONE_NONE_PRED_QUALIFIERS \
+ (arm_quadop_unone_unone_none_none_pred_qualifiers)
static enum arm_type_qualifiers
arm_quadop_none_none_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -624,11 +624,18 @@ arm_quadop_none_none_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
(arm_quadop_none_none_none_none_unone_qualifiers)
static enum arm_type_qualifiers
-arm_quadop_none_none_none_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quadop_none_none_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_none, qualifier_none, qualifier_none, qualifier_none,
+ qualifier_predicate };
+#define QUADOP_NONE_NONE_NONE_NONE_PRED_QUALIFIERS \
+ (arm_quadop_none_none_none_none_pred_qualifiers)
+
+static enum arm_type_qualifiers
+arm_quadop_none_none_none_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_none, qualifier_immediate,
- qualifier_unsigned };
-#define QUADOP_NONE_NONE_NONE_IMM_UNONE_QUALIFIERS \
- (arm_quadop_none_none_none_imm_unone_qualifiers)
+ qualifier_predicate };
+#define QUADOP_NONE_NONE_NONE_IMM_PRED_QUALIFIERS \
+ (arm_quadop_none_none_none_imm_pred_qualifiers)
static enum arm_type_qualifiers
arm_quadop_unone_unone_unone_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -638,32 +645,39 @@ arm_quadop_unone_unone_unone_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
(arm_quadop_unone_unone_unone_unone_unone_qualifiers)
static enum arm_type_qualifiers
-arm_quadop_unone_unone_none_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quadop_unone_unone_unone_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
+ qualifier_unsigned, qualifier_predicate };
+#define QUADOP_UNONE_UNONE_UNONE_UNONE_PRED_QUALIFIERS \
+ (arm_quadop_unone_unone_unone_unone_pred_qualifiers)
+
+static enum arm_type_qualifiers
+arm_quadop_unone_unone_none_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_none,
- qualifier_immediate, qualifier_unsigned };
-#define QUADOP_UNONE_UNONE_NONE_IMM_UNONE_QUALIFIERS \
- (arm_quadop_unone_unone_none_imm_unone_qualifiers)
+ qualifier_immediate, qualifier_predicate };
+#define QUADOP_UNONE_UNONE_NONE_IMM_PRED_QUALIFIERS \
+ (arm_quadop_unone_unone_none_imm_pred_qualifiers)
static enum arm_type_qualifiers
-arm_quadop_none_none_unone_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quadop_none_none_unone_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned };
-#define QUADOP_NONE_NONE_UNONE_IMM_UNONE_QUALIFIERS \
- (arm_quadop_none_none_unone_imm_unone_qualifiers)
+ qualifier_predicate };
+#define QUADOP_NONE_NONE_UNONE_IMM_PRED_QUALIFIERS \
+ (arm_quadop_none_none_unone_imm_pred_qualifiers)
static enum arm_type_qualifiers
-arm_quadop_unone_unone_unone_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quadop_unone_unone_unone_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
- qualifier_immediate, qualifier_unsigned };
-#define QUADOP_UNONE_UNONE_UNONE_IMM_UNONE_QUALIFIERS \
- (arm_quadop_unone_unone_unone_imm_unone_qualifiers)
+ qualifier_immediate, qualifier_predicate };
+#define QUADOP_UNONE_UNONE_UNONE_IMM_PRED_QUALIFIERS \
+ (arm_quadop_unone_unone_unone_imm_pred_qualifiers)
static enum arm_type_qualifiers
-arm_quadop_unone_unone_unone_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quadop_unone_unone_unone_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
- qualifier_none, qualifier_unsigned };
-#define QUADOP_UNONE_UNONE_UNONE_NONE_UNONE_QUALIFIERS \
- (arm_quadop_unone_unone_unone_none_unone_qualifiers)
+ qualifier_none, qualifier_predicate };
+#define QUADOP_UNONE_UNONE_UNONE_NONE_PRED_QUALIFIERS \
+ (arm_quadop_unone_unone_unone_none_pred_qualifiers)
static enum arm_type_qualifiers
arm_strs_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -700,25 +714,25 @@ arm_strsbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_strs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_void, qualifier_pointer, qualifier_none, qualifier_unsigned};
+ = { qualifier_void, qualifier_pointer, qualifier_none, qualifier_predicate};
#define STRS_P_QUALIFIERS (arm_strs_p_qualifiers)
static enum arm_type_qualifiers
arm_stru_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_void, qualifier_pointer, qualifier_unsigned,
- qualifier_unsigned};
+ qualifier_predicate};
#define STRU_P_QUALIFIERS (arm_stru_p_qualifiers)
static enum arm_type_qualifiers
arm_strsu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_void, qualifier_pointer, qualifier_unsigned,
- qualifier_unsigned, qualifier_unsigned};
+ qualifier_unsigned, qualifier_predicate};
#define STRSU_P_QUALIFIERS (arm_strsu_p_qualifiers)
static enum arm_type_qualifiers
arm_strss_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_void, qualifier_pointer, qualifier_unsigned,
- qualifier_none, qualifier_unsigned};
+ qualifier_none, qualifier_predicate};
#define STRSS_P_QUALIFIERS (arm_strss_p_qualifiers)
static enum arm_type_qualifiers
@@ -778,31 +792,31 @@ arm_ldrgbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_ldrgs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_pointer, qualifier_unsigned,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGS_Z_QUALIFIERS (arm_ldrgs_z_qualifiers)
static enum arm_type_qualifiers
arm_ldrgu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_pointer, qualifier_unsigned,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGU_Z_QUALIFIERS (arm_ldrgu_z_qualifiers)
static enum arm_type_qualifiers
arm_ldrs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_none, qualifier_pointer, qualifier_unsigned};
+ = { qualifier_none, qualifier_pointer, qualifier_predicate};
#define LDRS_Z_QUALIFIERS (arm_ldrs_z_qualifiers)
static enum arm_type_qualifiers
arm_ldru_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_unsigned, qualifier_pointer, qualifier_unsigned};
+ = { qualifier_unsigned, qualifier_pointer, qualifier_predicate};
#define LDRU_Z_QUALIFIERS (arm_ldru_z_qualifiers)
static enum arm_type_qualifiers
-arm_quinop_unone_unone_unone_unone_imm_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+arm_quinop_unone_unone_unone_unone_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
- qualifier_unsigned, qualifier_immediate, qualifier_unsigned };
-#define QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_UNONE_QUALIFIERS \
- (arm_quinop_unone_unone_unone_unone_imm_unone_qualifiers)
+ qualifier_unsigned, qualifier_immediate, qualifier_predicate };
+#define QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_PRED_QUALIFIERS \
+ (arm_quinop_unone_unone_unone_unone_imm_pred_qualifiers)
static enum arm_type_qualifiers
arm_ldrgbwbxu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
@@ -879,6 +893,18 @@ arm_sqshl_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_const};
#define SQSHL_QUALIFIERS (arm_sqshl_qualifiers)
+static enum arm_type_qualifiers
+arm_binop_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_none, qualifier_none, qualifier_predicate };
+#define BINOP_NONE_NONE_PRED_QUALIFIERS \
+ (arm_binop_none_none_pred_qualifiers)
+
+static enum arm_type_qualifiers
+arm_binop_unone_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
+ = { qualifier_unsigned, qualifier_unsigned, qualifier_predicate };
+#define BINOP_UNONE_UNONE_PRED_QUALIFIERS \
+ (arm_binop_unone_unone_pred_qualifiers)
+
/* End of Qualifier for MVE builtins. */
/* void ([T element type] *, T, immediate). */
diff --git a/gcc/config/arm/arm_mve_builtins.def b/gcc/config/arm/arm_mve_builtins.def
index 91ed2073918..bb79edf83ca 100644
--- a/gcc/config/arm/arm_mve_builtins.def
+++ b/gcc/config/arm/arm_mve_builtins.def
@@ -123,7 +123,7 @@ VAR3 (BINOP_PRED_UNONE_UNONE, vcmpcsq_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_UNONE_UNONE, vcmpcsq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vbicq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vandq_u, v16qi, v8hi, v4si)
-VAR3 (BINOP_UNONE_UNONE_UNONE, vaddvq_p_u, v16qi, v8hi, v4si)
+VAR3 (BINOP_UNONE_UNONE_PRED, vaddvq_p_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vaddvaq_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vaddq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_UNONE, vabdq_u, v16qi, v8hi, v4si)
@@ -154,7 +154,7 @@ VAR3 (BINOP_PRED_NONE_NONE, vcmpgeq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpeqq_, v16qi, v8hi, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpeqq_n_, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_NONE_IMM, vqshluq_n_s, v16qi, v8hi, v4si)
-VAR3 (BINOP_NONE_NONE_UNONE, vaddvq_p_s, v16qi, v8hi, v4si)
+VAR3 (BINOP_NONE_NONE_PRED, vaddvq_p_s, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_NONE, vsubq_s, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_NONE, vsubq_n_s, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_NONE, vshlq_r_s, v16qi, v8hi, v4si)
@@ -277,35 +277,35 @@ VAR1 (BINOP_NONE_NONE_NONE, vrmlaldavhq_s, v4si)
VAR1 (BINOP_NONE_NONE_NONE, vcvttq_f16_f32, v8hf)
VAR1 (BINOP_NONE_NONE_NONE, vcvtbq_f16_f32, v8hf)
VAR1 (BINOP_NONE_NONE_NONE, vaddlvaq_s, v4si)
-VAR2 (TERNOP_NONE_NONE_IMM_UNONE, vbicq_m_n_s, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_IMM_UNONE, vbicq_m_n_u, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_IMM_PRED, vbicq_m_n_s, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_IMM_PRED, vbicq_m_n_u, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_IMM, vqrshrnbq_n_s, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vqrshrnbq_n_u, v8hi, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlaldavhaq_s, v4si)
VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrmlaldavhaq_u, v4si)
-VAR2 (TERNOP_NONE_NONE_UNONE_UNONE, vcvtq_m_to_f_u, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtq_m_to_f_s, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_UNONE_PRED, vcvtq_m_to_f_u, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vcvtq_m_to_f_s, v8hf, v4sf)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_f, v8hf, v4sf)
VAR3 (TERNOP_UNONE_NONE_UNONE_IMM, vshlcq_carry_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vshlcq_carry_u, v16qi, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqrshrunbq_n_s, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_NONE_NONE, vabavq_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vabavq_u, v16qi, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vcvtaq_m_u, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtaq_m_s, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vcvtaq_m_u, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vcvtaq_m_s, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vshlcq_vec_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_UNONE_IMM, vshlcq_vec_s, v16qi, v8hi, v4si)
VAR4 (TERNOP_UNONE_UNONE_UNONE_PRED, vpselq_u, v16qi, v8hi, v4si, v2di)
VAR4 (TERNOP_NONE_NONE_NONE_PRED, vpselq_s, v16qi, v8hi, v4si, v2di)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrev64q_m_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmvnq_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vrev64q_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vmvnq_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmlasq_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmlaq_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmladavq_p_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vmladavq_p_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmladavaq_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vminvq_p_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmaxvq_p_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vdupq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vminvq_p_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vmaxvq_p_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vdupq_m_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpneq_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpneq_m_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmphiq_m_u, v16qi, v8hi, v4si)
@@ -314,18 +314,18 @@ VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpeqq_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpeqq_m_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpcsq_m_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_UNONE_UNONE_PRED, vcmpcsq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vclzq_m_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_UNONE_UNONE, vaddvaq_p_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vclzq_m_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_UNONE_PRED, vaddvaq_p_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vsriq_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vsliq_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vshlq_m_r_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vrshlq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vqshlq_m_r_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vqrshlq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vminavq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vminaq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vmaxavq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_UNONE_UNONE_NONE_UNONE, vmaxaq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vshlq_m_r_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vrshlq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vqshlq_m_r_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vqrshlq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vminavq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vminaq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vmaxavq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_UNONE_UNONE_NONE_PRED, vmaxaq_m_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_n_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpltq_m_s, v16qi, v8hi, v4si)
@@ -338,26 +338,26 @@ VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_n_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vshlq_m_r_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vrshlq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vrev64q_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vqshlq_m_r_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vqrshlq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vqnegq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vqabsq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vnegq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vmvnq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vmlsdavxq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vmlsdavq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vmladavxq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vmladavq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vminvq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vmaxvq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vdupq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vclzq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vclsq_m_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vaddvaq_p_s, v16qi, v8hi, v4si)
-VAR3 (TERNOP_NONE_NONE_NONE_UNONE, vabsq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vshlq_m_r_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vrshlq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vrev64q_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vqshlq_m_r_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vqrshlq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vqnegq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vqabsq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vnegq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vmvnq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vmlsdavxq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vmlsdavq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vmladavxq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vmladavq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vminvq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vmaxvq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vdupq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vclzq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vclsq_m_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vaddvaq_p_s, v16qi, v8hi, v4si)
+VAR3 (TERNOP_NONE_NONE_NONE_PRED, vabsq_m_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_NONE, vqrdmlsdhxq_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_NONE, vqrdmlsdhq_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_NONE, vqrdmlashq_n_s, v16qi, v8hi, v4si)
@@ -378,14 +378,14 @@ VAR3 (TERNOP_NONE_NONE_NONE_NONE, vmladavaxq_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_NONE, vmladavaq_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_IMM, vsriq_n_s, v16qi, v8hi, v4si)
VAR3 (TERNOP_NONE_NONE_NONE_IMM, vsliq_n_s, v16qi, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrev32q_m_u, v16qi, v8hi)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vqmovntq_m_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vqmovnbq_m_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmovntq_m_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmovnbq_m_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmovltq_m_u, v16qi, v8hi)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmovlbq_m_u, v16qi, v8hi)
-VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmlaldavq_p_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vrev32q_m_u, v16qi, v8hi)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vqmovntq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vqmovnbq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vmovntq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vmovnbq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vmovltq_m_u, v16qi, v8hi)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vmovlbq_m_u, v16qi, v8hi)
+VAR2 (TERNOP_UNONE_UNONE_UNONE_PRED, vmlaldavq_p_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_UNONE, vmlaldavaq_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vshrntq_n_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vshrnbq_n_u, v8hi, v4si)
@@ -394,17 +394,17 @@ VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vrshrnbq_n_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vqshrntq_n_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vqshrnbq_n_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_UNONE_IMM, vqrshrntq_n_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vqmovuntq_m_s, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vqmovunbq_m_s, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vcvtq_m_from_f_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vcvtpq_m_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vcvtnq_m_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_NONE_UNONE, vcvtmq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vqmovuntq_m_s, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vqmovunbq_m_s, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vcvtq_m_from_f_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vcvtpq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vcvtnq_m_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_NONE_PRED, vcvtmq_m_u, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqshruntq_n_s, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqshrunbq_n_s, v8hi, v4si)
VAR2 (TERNOP_UNONE_UNONE_NONE_IMM, vqrshruntq_n_s, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_IMM_UNONE, vorrq_m_n_u, v8hi, v4si)
-VAR2 (TERNOP_UNONE_UNONE_IMM_UNONE, vmvnq_m_n_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_IMM_PRED, vorrq_m_n_u, v8hi, v4si)
+VAR2 (TERNOP_UNONE_UNONE_IMM_PRED, vmvnq_m_n_u, v8hi, v4si)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_n_f, v8hf, v4sf)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpneq_m_f, v8hf, v4sf)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpltq_m_n_f, v8hf, v4sf)
@@ -416,38 +416,38 @@ VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgtq_m_f, v8hf, v4sf)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_n_f, v8hf, v4sf)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpgeq_m_f, v8hf, v4sf)
VAR2 (TERNOP_PRED_NONE_NONE_PRED, vcmpeqq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndxq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndpq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndnq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndmq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrndaq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrev64q_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vrev32q_m_s, v16qi, v8hi)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vqmovntq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vqmovnbq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrndxq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrndq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrndpq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrndnq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrndmq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrndaq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrev64q_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vrev32q_m_s, v16qi, v8hi)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vqmovntq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vqmovnbq_m_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_PRED, vpselq_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vnegq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmovntq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmovnbq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmovltq_m_s, v16qi, v8hi)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmovlbq_m_s, v16qi, v8hi)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmlsldavxq_p_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmlsldavq_p_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmlaldavxq_p_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmlaldavq_p_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vminnmvq_p_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vminnmavq_p_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vminnmaq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmaxnmvq_p_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmaxnmavq_p_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vmaxnmaq_m_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vdupq_m_n_f, v8hf, v4sf)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtq_m_from_f_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtpq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtnq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vcvtmq_m_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_NONE_UNONE, vabsq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vnegq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmovntq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmovnbq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmovltq_m_s, v16qi, v8hi)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmovlbq_m_s, v16qi, v8hi)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmlsldavxq_p_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmlsldavq_p_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmlaldavxq_p_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmlaldavq_p_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vminnmvq_p_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vminnmavq_p_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vminnmaq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmaxnmvq_p_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmaxnmavq_p_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vmaxnmaq_m_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vdupq_m_n_f, v8hf, v4sf)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vcvtq_m_from_f_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vcvtpq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vcvtnq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vcvtmq_m_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_NONE_PRED, vabsq_m_f, v8hf, v4sf)
VAR2 (TERNOP_NONE_NONE_NONE_NONE, vmlsldavaxq_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_NONE, vmlsldavaq_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_NONE, vmlaldavaxq_s, v8hi, v4si)
@@ -463,8 +463,8 @@ VAR2 (TERNOP_NONE_NONE_NONE_IMM, vrshrnbq_n_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_IMM, vqshrntq_n_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_IMM, vqshrnbq_n_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_IMM, vqrshrntq_n_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_IMM_UNONE, vorrq_m_n_s, v8hi, v4si)
-VAR2 (TERNOP_NONE_NONE_IMM_UNONE, vmvnq_m_n_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_IMM_PRED, vorrq_m_n_s, v8hi, v4si)
+VAR2 (TERNOP_NONE_NONE_IMM_PRED, vmvnq_m_n_s, v8hi, v4si)
VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrmlaldavhq_p_u, v4si)
VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrev16q_m_u, v16qi)
VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vaddlvaq_p_u, v4si)
@@ -482,189 +482,189 @@ VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vaddlvaq_p_s, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlsldavhaxq_s, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlsldavhaq_s, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlaldavhaxq_s, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vsriq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vsriq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vsubq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vsubq_m_u, v16qi, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_UNONE_IMM_UNONE, vcvtq_m_n_to_f_u, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vcvtq_m_n_to_f_s, v8hf, v4sf)
-VAR3 (QUADOP_UNONE_UNONE_NONE_IMM_UNONE, vqshluq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_NONE_NONE_UNONE, vabavq_p_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vabavq_p_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_UNONE, vshlq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vshlq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vsubq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vrmulhq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vrhaddq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vqsubq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vqsubq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vqaddq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vqaddq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vorrq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vornq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmulq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmulq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmulltq_int_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmullbq_int_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmulhq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmlasq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmlaq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmladavaq_p_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vminq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmaxq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vhsubq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vhsubq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vhaddq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vhaddq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, veorq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vcaddq_rot90_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vcaddq_rot270_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vbicq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vandq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vaddq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vaddq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vabdq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_UNONE, vrshlq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_UNONE, vqshlq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_UNONE, vqrshlq_m_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_UNONE, vbrsrq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vsliq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshrq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshlq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vrshrq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vqshlq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vsubq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrshlq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmulhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrhaddq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqsubq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqsubq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqshlq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrshlq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmulhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmulhq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmlsdhxq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmlsdhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmlashq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmlahq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmladhxq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqrdmladhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmulhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmulhq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmlsdhxq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmlsdhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmlahq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmlashq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmladhxq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmladhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqaddq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqaddq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vorrq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vornq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmulq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmulq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmulltq_int_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmullbq_int_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmulhq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlsdavaxq_p_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlsdavaq_p_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlasq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlaq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmladavaxq_p_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmladavaq_p_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vminq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmaxq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vhsubq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vhsubq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vhcaddq_rot90_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vhcaddq_rot270_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vhaddq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vhaddq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, veorq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcaddq_rot90_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcaddq_rot270_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vbrsrq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vbicq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vandq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vaddq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vaddq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vabdq_m_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vsliq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vshrq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vshlq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vrshrq_m_n_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vqshlq_m_n_s, v16qi, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmulltq_poly_m_p, v16qi, v8hi)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmullbq_poly_m_p, v16qi, v8hi)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vmlaldavaq_p_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshrntq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshrnbq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshlltq_m_n_u, v16qi, v8hi)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshllbq_m_n_u, v16qi, v8hi)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vrshrntq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vrshrnbq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vqshrntq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vqshrnbq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vqrshrntq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vqrshrnbq_m_n_u, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_UNONE, vqshruntq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_UNONE, vqshrunbq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_UNONE, vqrshruntq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_UNONE, vqrshrunbq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmulltq_m_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmulltq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmullbq_m_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vqdmullbq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlsldavaxq_p_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlsldavaq_p_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlaldavaxq_p_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmlaldavaq_p_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vshrntq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vshrnbq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vshlltq_m_n_s, v16qi, v8hi)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vshllbq_m_n_s, v16qi, v8hi)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vrshrntq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vrshrnbq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vqshrntq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vqshrnbq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vqrshrntq_m_n_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vqrshrnbq_m_n_s, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_IMM_PRED, vsriq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vsriq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vsubq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vsubq_m_u, v16qi, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_UNONE_IMM_PRED, vcvtq_m_n_to_f_u, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vcvtq_m_n_to_f_s, v8hf, v4sf)
+VAR3 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vqshluq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_NONE_NONE_PRED, vabavq_p_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vabavq_p_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_PRED, vshlq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vshlq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vsubq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vrmulhq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vrhaddq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vqsubq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vqsubq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vqaddq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vqaddq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vorrq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vornq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmulq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmulq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmulltq_int_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmullbq_int_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmulhq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmlasq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmlaq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmladavaq_p_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vminq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmaxq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vhsubq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vhsubq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vhaddq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vhaddq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, veorq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vcaddq_rot90_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vcaddq_rot270_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vbicq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vandq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vaddq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vaddq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vabdq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_PRED, vrshlq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_PRED, vqshlq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_PRED, vqrshlq_m_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_NONE_PRED, vbrsrq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vsliq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshrq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshlq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vrshrq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vqshlq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vsubq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrshlq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrmulhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrhaddq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqsubq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqsubq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqshlq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrshlq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmulhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmulhq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmlsdhxq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmlsdhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmlashq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmlahq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmladhxq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqrdmladhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmulhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmulhq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmlsdhxq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmlsdhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmlahq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmlashq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmladhxq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmladhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqaddq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqaddq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vorrq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vornq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmulq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmulq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmulltq_int_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmullbq_int_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmulhq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlsdavaxq_p_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlsdavaq_p_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlasq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlaq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmladavaxq_p_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmladavaq_p_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vminq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmaxq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vhsubq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vhsubq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vhcaddq_rot90_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vhcaddq_rot270_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vhaddq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vhaddq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, veorq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcaddq_rot90_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcaddq_rot270_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vbrsrq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vbicq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vandq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vaddq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vaddq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_NONE_PRED, vabdq_m_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_IMM_PRED, vsliq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_IMM_PRED, vshrq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_IMM_PRED, vshlq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_IMM_PRED, vrshrq_m_n_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqshlq_m_n_s, v16qi, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmulltq_poly_m_p, v16qi, v8hi)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmullbq_poly_m_p, v16qi, v8hi)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vmlaldavaq_p_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshrntq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshrnbq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshlltq_m_n_u, v16qi, v8hi)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshllbq_m_n_u, v16qi, v8hi)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vrshrntq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vrshrnbq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vqshrntq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vqshrnbq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vqrshrntq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vqrshrnbq_m_n_u, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vqshruntq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vqshrunbq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vqrshruntq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vqrshrunbq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmulltq_m_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmulltq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmullbq_m_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vqdmullbq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlsldavaxq_p_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlsldavaq_p_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlaldavaxq_p_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmlaldavaq_p_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vshrntq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vshrnbq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vshlltq_m_n_s, v16qi, v8hi)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vshllbq_m_n_s, v16qi, v8hi)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vrshrntq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vrshrnbq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqshrntq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqshrnbq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqrshrntq_m_n_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqrshrnbq_m_n_s, v8hi, v4si)
VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vrmlaldavhaq_p_u, v4si)
VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlsldavhaxq_p_s, v4si)
VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlsldavhaq_p_s, v4si)
VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlaldavhaxq_p_s, v4si)
VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlaldavhaq_p_s, v4si)
-VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_UNONE, vcvtq_m_n_from_f_u, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_IMM_UNONE, vcvtq_m_n_from_f_s, v8hi, v4si)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vbrsrq_m_n_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vsubq_m_n_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vsubq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vorrq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vornq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmulq_m_n_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmulq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vminnmq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vmaxnmq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vfmsq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vfmasq_m_n_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vfmaq_m_n_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vfmaq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, veorq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmulq_rot90_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmulq_rot270_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmulq_rot180_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmulq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmlaq_rot90_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmlaq_rot270_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmlaq_rot180_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcmlaq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcaddq_rot90_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vcaddq_rot270_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vbicq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vandq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vaddq_m_n_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vaddq_m_f, v8hf, v4sf)
-VAR2 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vabdq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vcvtq_m_n_from_f_u, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vcvtq_m_n_from_f_s, v8hi, v4si)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vbrsrq_m_n_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vsubq_m_n_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vsubq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vorrq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vornq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmulq_m_n_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmulq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vminnmq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vmaxnmq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vfmsq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vfmasq_m_n_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vfmaq_m_n_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vfmaq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, veorq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmulq_rot90_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmulq_rot270_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmulq_rot180_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmulq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmlaq_rot90_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmlaq_rot270_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmlaq_rot180_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcmlaq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcaddq_rot90_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vcaddq_rot270_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vbicq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vandq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vaddq_m_n_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vaddq_m_f, v8hf, v4sf)
+VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vabdq_m_f, v8hf, v4sf)
VAR3 (STRS, vstrbq_s, v16qi, v8hi, v4si)
VAR3 (STRU, vstrbq_u, v16qi, v8hi, v4si)
VAR3 (STRSS, vstrbq_scatter_offset_s, v16qi, v8hi, v4si)
@@ -797,14 +797,14 @@ VAR1 (STRSU_P, vstrwq_scatter_offset_p_u, v4si)
VAR1 (STRSU_P, vstrwq_scatter_shifted_offset_p_u, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, viwdupq_wb_u, v16qi, v4si, v8hi)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vdwdupq_wb_u, v16qi, v4si, v8hi)
-VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_UNONE, viwdupq_m_wb_u, v16qi, v8hi, v4si)
-VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_UNONE, vdwdupq_m_wb_u, v16qi, v8hi, v4si)
-VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_UNONE, viwdupq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_UNONE, vdwdupq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_PRED, viwdupq_m_wb_u, v16qi, v8hi, v4si)
+VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_PRED, vdwdupq_m_wb_u, v16qi, v8hi, v4si)
+VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_PRED, viwdupq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUINOP_UNONE_UNONE_UNONE_UNONE_IMM_PRED, vdwdupq_m_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vddupq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_IMM, vidupq_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vddupq_m_n_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vidupq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vddupq_m_n_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vidupq_m_n_u, v16qi, v8hi, v4si)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, vdwdupq_n_u, v16qi, v4si, v8hi)
VAR3 (TERNOP_UNONE_UNONE_UNONE_IMM, viwdupq_n_u, v16qi, v4si, v8hi)
VAR1 (STRSBWBU, vstrwq_scatter_base_wb_u, v4si)
@@ -870,10 +870,10 @@ VAR1 (UQSHL, urshr_, si)
VAR1 (UQSHL, urshrl_, di)
VAR1 (UQSHL, uqshl_, si)
VAR1 (UQSHL, uqshll_, di)
-VAR3 (QUADOP_NONE_NONE_UNONE_IMM_UNONE, vshlcq_m_vec_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_NONE_NONE_UNONE_IMM_UNONE, vshlcq_m_carry_s, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshlcq_m_vec_u, v16qi, v8hi, v4si)
-VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_UNONE, vshlcq_m_carry_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_UNONE_IMM_PRED, vshlcq_m_vec_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_NONE_NONE_UNONE_IMM_PRED, vshlcq_m_carry_s, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshlcq_m_vec_u, v16qi, v8hi, v4si)
+VAR3 (QUADOP_UNONE_UNONE_UNONE_IMM_PRED, vshlcq_m_carry_u, v16qi, v8hi, v4si)
/* optabs without any suffixes. */
VAR5 (BINOP_NONE_NONE_NONE, vcaddq_rot90, v16qi, v8hi, v4si, v8hf, v4sf)
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index 4766bdd74f7..81ad488155d 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -130,7 +130,7 @@ (define_insn "mve_vrndq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRNDQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -918,7 +918,7 @@ (define_insn "mve_vaddvq_p_<supf><mode>"
[
(set (match_operand:SI 0 "s_register_operand" "=Te")
(unspec:SI [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VADDVQ_P))
]
"TARGET_HAVE_MVE"
@@ -2581,7 +2581,7 @@ (define_insn "mve_vbicq_m_n_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:SI 2 "immediate_operand" "i")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VBICQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -2611,7 +2611,7 @@ (define_insn "mve_vcvtaq_m_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTAQ_M))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -2626,7 +2626,7 @@ (define_insn "mve_vcvtq_m_to_f_<supf><mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTQ_M_TO_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -2748,7 +2748,7 @@ (define_insn "mve_vabsq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VABSQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -2764,7 +2764,7 @@ (define_insn "mve_vaddvaq_p_<supf><mode>"
(set (match_operand:SI 0 "s_register_operand" "=Te")
(unspec:SI [(match_operand:SI 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VADDVAQ_P))
]
"TARGET_HAVE_MVE"
@@ -2780,7 +2780,7 @@ (define_insn "mve_vclsq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCLSQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -2796,7 +2796,7 @@ (define_insn "mve_vclzq_m_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCLZQ_M))
]
"TARGET_HAVE_MVE"
@@ -3068,7 +3068,7 @@ (define_insn "mve_vdupq_m_n_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VDUPQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -3084,7 +3084,7 @@ (define_insn "mve_vmaxaq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMAXAQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3100,7 +3100,7 @@ (define_insn "mve_vmaxavq_p_s<mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMAXAVQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -3116,7 +3116,7 @@ (define_insn "mve_vmaxvq_p_<supf><mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMAXVQ_P))
]
"TARGET_HAVE_MVE"
@@ -3132,7 +3132,7 @@ (define_insn "mve_vminaq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMINAQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3148,7 +3148,7 @@ (define_insn "mve_vminavq_p_s<mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMINAVQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -3164,7 +3164,7 @@ (define_insn "mve_vminvq_p_<supf><mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMINVQ_P))
]
"TARGET_HAVE_MVE"
@@ -3196,7 +3196,7 @@ (define_insn "mve_vmladavq_p_<supf><mode>"
(set (match_operand:SI 0 "s_register_operand" "=Te")
(unspec:SI [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLADAVQ_P))
]
"TARGET_HAVE_MVE"
@@ -3212,7 +3212,7 @@ (define_insn "mve_vmladavxq_p_s<mode>"
(set (match_operand:SI 0 "s_register_operand" "=Te")
(unspec:SI [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLADAVXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -3260,7 +3260,7 @@ (define_insn "mve_vmlsdavq_p_s<mode>"
(set (match_operand:SI 0 "s_register_operand" "=Te")
(unspec:SI [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLSDAVQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -3276,7 +3276,7 @@ (define_insn "mve_vmlsdavxq_p_s<mode>"
(set (match_operand:SI 0 "s_register_operand" "=Te")
(unspec:SI [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLSDAVXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -3292,7 +3292,7 @@ (define_insn "mve_vmvnq_m_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMVNQ_M))
]
"TARGET_HAVE_MVE"
@@ -3308,7 +3308,7 @@ (define_insn "mve_vnegq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VNEGQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3340,7 +3340,7 @@ (define_insn "mve_vqabsq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQABSQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3388,7 +3388,7 @@ (define_insn "mve_vqnegq_m_s<mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQNEGQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -3500,7 +3500,7 @@ (define_insn "mve_vqrshlq_m_n_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:SI 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQRSHLQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -3516,7 +3516,7 @@ (define_insn "mve_vqshlq_m_r_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:SI 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQSHLQ_M_R))
]
"TARGET_HAVE_MVE"
@@ -3532,7 +3532,7 @@ (define_insn "mve_vrev64q_m_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VREV64Q_M))
]
"TARGET_HAVE_MVE"
@@ -3548,7 +3548,7 @@ (define_insn "mve_vrshlq_m_n_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:SI 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRSHLQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -3564,7 +3564,7 @@ (define_insn "mve_vshlq_m_r_<supf><mode>"
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:SI 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VSHLQ_M_R))
]
"TARGET_HAVE_MVE"
@@ -3723,7 +3723,7 @@ (define_insn "mve_vabsq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VABSQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4013,7 +4013,7 @@ (define_insn "mve_vdupq_m_n_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:<V_elem> 2 "s_register_operand" "r")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VDUPQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4092,7 +4092,7 @@ (define_insn "mve_vmaxnmaq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMAXNMAQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4107,7 +4107,7 @@ (define_insn "mve_vmaxnmavq_p_f<mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMAXNMAVQ_P_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4123,7 +4123,7 @@ (define_insn "mve_vmaxnmvq_p_f<mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMAXNMVQ_P_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4138,7 +4138,7 @@ (define_insn "mve_vminnmaq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMINNMAQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4154,7 +4154,7 @@ (define_insn "mve_vminnmavq_p_f<mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMINNMAVQ_P_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4169,7 +4169,7 @@ (define_insn "mve_vminnmvq_p_f<mode>"
(set (match_operand:<V_elem> 0 "s_register_operand" "=r")
(unspec:<V_elem> [(match_operand:<V_elem> 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMINNMVQ_P_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4217,7 +4217,7 @@ (define_insn "mve_vmlaldavq_p_<supf><mode>"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:MVE_5 1 "s_register_operand" "w")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLALDAVQ_P))
]
"TARGET_HAVE_MVE"
@@ -4233,7 +4233,7 @@ (define_insn "mve_vmlaldavxq_p_s<mode>"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:MVE_5 1 "s_register_operand" "w")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLALDAVXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -4280,7 +4280,7 @@ (define_insn "mve_vmlsldavq_p_s<mode>"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:MVE_5 1 "s_register_operand" "w")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLSLDAVQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -4296,7 +4296,7 @@ (define_insn "mve_vmlsldavxq_p_s<mode>"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:MVE_5 1 "s_register_operand" "w")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMLSLDAVXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -4311,7 +4311,7 @@ (define_insn "mve_vmovlbq_m_<supf><mode>"
(set (match_operand:<V_double_width> 0 "s_register_operand" "=w")
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMOVLBQ_M))
]
"TARGET_HAVE_MVE"
@@ -4326,7 +4326,7 @@ (define_insn "mve_vmovltq_m_<supf><mode>"
(set (match_operand:<V_double_width> 0 "s_register_operand" "=w")
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMOVLTQ_M))
]
"TARGET_HAVE_MVE"
@@ -4341,7 +4341,7 @@ (define_insn "mve_vmovnbq_m_<supf><mode>"
(set (match_operand:<V_narrow_pack> 0 "s_register_operand" "=w")
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMOVNBQ_M))
]
"TARGET_HAVE_MVE"
@@ -4357,7 +4357,7 @@ (define_insn "mve_vmovntq_m_<supf><mode>"
(set (match_operand:<V_narrow_pack> 0 "s_register_operand" "=w")
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMOVNTQ_M))
]
"TARGET_HAVE_MVE"
@@ -4373,7 +4373,7 @@ (define_insn "mve_vmvnq_m_n_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:SI 2 "immediate_operand" "i")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VMVNQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -4388,7 +4388,7 @@ (define_insn "mve_vnegq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VNEGQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4404,7 +4404,7 @@ (define_insn "mve_vorrq_m_n_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:SI 2 "immediate_operand" "i")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VORRQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -4435,7 +4435,7 @@ (define_insn "mve_vqmovnbq_m_<supf><mode>"
(set (match_operand:<V_narrow_pack> 0 "s_register_operand" "=w")
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQMOVNBQ_M))
]
"TARGET_HAVE_MVE"
@@ -4451,7 +4451,7 @@ (define_insn "mve_vqmovntq_m_<supf><mode>"
(set (match_operand:<V_narrow_pack> 0 "s_register_operand" "=w")
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQMOVNTQ_M))
]
"TARGET_HAVE_MVE"
@@ -4467,7 +4467,7 @@ (define_insn "mve_vqmovunbq_m_s<mode>"
(set (match_operand:<V_narrow_pack> 0 "s_register_operand" "=w")
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQMOVUNBQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -4483,7 +4483,7 @@ (define_insn "mve_vqmovuntq_m_s<mode>"
(set (match_operand:<V_narrow_pack> 0 "s_register_operand" "=w")
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VQMOVUNTQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -4611,7 +4611,7 @@ (define_insn "mve_vrev32q_m_<supf><mode>"
(set (match_operand:MVE_3 0 "s_register_operand" "=w")
(unspec:MVE_3 [(match_operand:MVE_3 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VREV32Q_M))
]
"TARGET_HAVE_MVE"
@@ -4627,7 +4627,7 @@ (define_insn "mve_vrev64q_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VREV64Q_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4723,7 +4723,7 @@ (define_insn "mve_vrndaq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRNDAQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4739,7 +4739,7 @@ (define_insn "mve_vrndmq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRNDMQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4755,7 +4755,7 @@ (define_insn "mve_vrndnq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRNDNQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4771,7 +4771,7 @@ (define_insn "mve_vrndpq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRNDPQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4787,7 +4787,7 @@ (define_insn "mve_vrndxq_m_f<mode>"
(set (match_operand:MVE_0 0 "s_register_operand" "=w")
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRNDXQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4867,7 +4867,7 @@ (define_insn "mve_vcvtmq_m_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTMQ_M))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4883,7 +4883,7 @@ (define_insn "mve_vcvtpq_m_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTPQ_M))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4899,7 +4899,7 @@ (define_insn "mve_vcvtnq_m_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTNQ_M))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4916,7 +4916,7 @@ (define_insn "mve_vcvtq_m_n_from_f_<supf><mode>"
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred2>" "<MVE_constraint2>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCVTQ_M_N_FROM_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4948,7 +4948,7 @@ (define_insn "mve_vcvtq_m_from_f_<supf><mode>"
(set (match_operand:MVE_5 0 "s_register_operand" "=w")
(unspec:MVE_5 [(match_operand:MVE_5 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTQ_M_FROM_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4997,7 +4997,7 @@ (define_insn "mve_vabavq_p_<supf><mode>"
(unspec:SI [(match_operand:SI 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VABAVQ_P))
]
"TARGET_HAVE_MVE"
@@ -5014,7 +5014,7 @@ (define_insn "mve_vqshluq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_7" "Ra")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHLUQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -5030,7 +5030,7 @@ (define_insn "mve_vshlq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHLQ_M))
]
"TARGET_HAVE_MVE"
@@ -5046,7 +5046,7 @@ (define_insn "mve_vsriq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_selective_upto_8" "Rg")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSRIQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5062,7 +5062,7 @@ (define_insn "mve_vsubq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSUBQ_M))
]
"TARGET_HAVE_MVE"
@@ -5078,7 +5078,7 @@ (define_insn "mve_vcvtq_m_n_to_f_<supf><mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:<MVE_CNVT> 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred2>" "<MVE_constraint2>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCVTQ_M_N_TO_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -5094,7 +5094,7 @@ (define_insn "mve_vabdq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VABDQ_M))
]
"TARGET_HAVE_MVE"
@@ -5111,7 +5111,7 @@ (define_insn "mve_vaddq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VADDQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5128,7 +5128,7 @@ (define_insn "mve_vaddq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VADDQ_M))
]
"TARGET_HAVE_MVE"
@@ -5145,7 +5145,7 @@ (define_insn "mve_vandq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VANDQ_M))
]
"TARGET_HAVE_MVE"
@@ -5162,7 +5162,7 @@ (define_insn "mve_vbicq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VBICQ_M))
]
"TARGET_HAVE_MVE"
@@ -5179,7 +5179,7 @@ (define_insn "mve_vbrsrq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VBRSRQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5196,7 +5196,7 @@ (define_insn "mve_vcaddq_rot270_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCADDQ_ROT270_M))
]
"TARGET_HAVE_MVE"
@@ -5213,7 +5213,7 @@ (define_insn "mve_vcaddq_rot90_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCADDQ_ROT90_M))
]
"TARGET_HAVE_MVE"
@@ -5230,7 +5230,7 @@ (define_insn "mve_veorq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VEORQ_M))
]
"TARGET_HAVE_MVE"
@@ -5247,7 +5247,7 @@ (define_insn "mve_vhaddq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VHADDQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5264,7 +5264,7 @@ (define_insn "mve_vhaddq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VHADDQ_M))
]
"TARGET_HAVE_MVE"
@@ -5281,7 +5281,7 @@ (define_insn "mve_vhsubq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VHSUBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5298,7 +5298,7 @@ (define_insn "mve_vhsubq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VHSUBQ_M))
]
"TARGET_HAVE_MVE"
@@ -5315,7 +5315,7 @@ (define_insn "mve_vmaxq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMAXQ_M))
]
"TARGET_HAVE_MVE"
@@ -5332,7 +5332,7 @@ (define_insn "mve_vminq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMINQ_M))
]
"TARGET_HAVE_MVE"
@@ -5349,7 +5349,7 @@ (define_insn "mve_vmladavaq_p_<supf><mode>"
(unspec:SI [(match_operand:SI 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLADAVAQ_P))
]
"TARGET_HAVE_MVE"
@@ -5366,7 +5366,7 @@ (define_insn "mve_vmlaq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLAQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5383,7 +5383,7 @@ (define_insn "mve_vmlasq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLASQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5400,7 +5400,7 @@ (define_insn "mve_vmulhq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULHQ_M))
]
"TARGET_HAVE_MVE"
@@ -5417,7 +5417,7 @@ (define_insn "mve_vmullbq_int_m_<supf><mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULLBQ_INT_M))
]
"TARGET_HAVE_MVE"
@@ -5434,7 +5434,7 @@ (define_insn "mve_vmulltq_int_m_<supf><mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULLTQ_INT_M))
]
"TARGET_HAVE_MVE"
@@ -5451,7 +5451,7 @@ (define_insn "mve_vmulq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5468,7 +5468,7 @@ (define_insn "mve_vmulq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULQ_M))
]
"TARGET_HAVE_MVE"
@@ -5485,7 +5485,7 @@ (define_insn "mve_vornq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VORNQ_M))
]
"TARGET_HAVE_MVE"
@@ -5502,7 +5502,7 @@ (define_insn "mve_vorrq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VORRQ_M))
]
"TARGET_HAVE_MVE"
@@ -5519,7 +5519,7 @@ (define_insn "mve_vqaddq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQADDQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5536,7 +5536,7 @@ (define_insn "mve_vqaddq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQADDQ_M))
]
"TARGET_HAVE_MVE"
@@ -5553,7 +5553,7 @@ (define_insn "mve_vqdmlahq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMLAHQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -5570,7 +5570,7 @@ (define_insn "mve_vqdmlashq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMLASHQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -5587,7 +5587,7 @@ (define_insn "mve_vqrdmlahq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMLAHQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -5604,7 +5604,7 @@ (define_insn "mve_vqrdmlashq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMLASHQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -5621,7 +5621,7 @@ (define_insn "mve_vqrshlq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRSHLQ_M))
]
"TARGET_HAVE_MVE"
@@ -5638,7 +5638,7 @@ (define_insn "mve_vqshlq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "immediate_operand" "i")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHLQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5655,7 +5655,7 @@ (define_insn "mve_vqshlq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHLQ_M))
]
"TARGET_HAVE_MVE"
@@ -5672,7 +5672,7 @@ (define_insn "mve_vqsubq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSUBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5689,7 +5689,7 @@ (define_insn "mve_vqsubq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSUBQ_M))
]
"TARGET_HAVE_MVE"
@@ -5706,7 +5706,7 @@ (define_insn "mve_vrhaddq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRHADDQ_M))
]
"TARGET_HAVE_MVE"
@@ -5723,7 +5723,7 @@ (define_insn "mve_vrmulhq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRMULHQ_M))
]
"TARGET_HAVE_MVE"
@@ -5740,7 +5740,7 @@ (define_insn "mve_vrshlq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRSHLQ_M))
]
"TARGET_HAVE_MVE"
@@ -5757,7 +5757,7 @@ (define_insn "mve_vrshrq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred2>" "<MVE_constraint2>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRSHRQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5774,7 +5774,7 @@ (define_insn "mve_vshlq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "immediate_operand" "i")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHLQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5791,7 +5791,7 @@ (define_insn "mve_vshrq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred2>" "<MVE_constraint2>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHRQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5808,7 +5808,7 @@ (define_insn "mve_vsliq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred>" "<MVE_constraint>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSLIQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5825,7 +5825,7 @@ (define_insn "mve_vsubq_m_n_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSUBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -5842,7 +5842,7 @@ (define_insn "mve_vhcaddq_rot270_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VHCADDQ_ROT270_M_S))
]
"TARGET_HAVE_MVE"
@@ -5859,7 +5859,7 @@ (define_insn "mve_vhcaddq_rot90_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VHCADDQ_ROT90_M_S))
]
"TARGET_HAVE_MVE"
@@ -5876,7 +5876,7 @@ (define_insn "mve_vmladavaxq_p_s<mode>"
(unspec:SI [(match_operand:SI 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLADAVAXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -5893,7 +5893,7 @@ (define_insn "mve_vmlsdavaq_p_s<mode>"
(unspec:SI [(match_operand:SI 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLSDAVAQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -5910,7 +5910,7 @@ (define_insn "mve_vmlsdavaxq_p_s<mode>"
(unspec:SI [(match_operand:SI 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLSDAVAXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -5927,7 +5927,7 @@ (define_insn "mve_vqdmladhq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMLADHQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -5944,7 +5944,7 @@ (define_insn "mve_vqdmladhxq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMLADHXQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -5961,7 +5961,7 @@ (define_insn "mve_vqdmlsdhq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMLSDHQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -5978,7 +5978,7 @@ (define_insn "mve_vqdmlsdhxq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMLSDHXQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -5995,7 +5995,7 @@ (define_insn "mve_vqdmulhq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMULHQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6012,7 +6012,7 @@ (define_insn "mve_vqdmulhq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMULHQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6029,7 +6029,7 @@ (define_insn "mve_vqrdmladhq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMLADHQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6046,7 +6046,7 @@ (define_insn "mve_vqrdmladhxq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMLADHXQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6063,7 +6063,7 @@ (define_insn "mve_vqrdmlsdhq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMLSDHQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6080,7 +6080,7 @@ (define_insn "mve_vqrdmlsdhxq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMLSDHXQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6097,7 +6097,7 @@ (define_insn "mve_vqrdmulhq_m_n_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMULHQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6114,7 +6114,7 @@ (define_insn "mve_vqrdmulhq_m_s<mode>"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:MVE_2 2 "s_register_operand" "w")
(match_operand:MVE_2 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRDMULHQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6131,7 +6131,7 @@ (define_insn "mve_vmlaldavaq_p_<supf><mode>"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:MVE_5 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLALDAVAQ_P))
]
"TARGET_HAVE_MVE"
@@ -6148,7 +6148,7 @@ (define_insn "mve_vmlaldavaxq_p_<supf><mode>"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:MVE_5 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLALDAVAXQ_P))
]
"TARGET_HAVE_MVE"
@@ -6165,7 +6165,7 @@ (define_insn "mve_vqrshrnbq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_8" "Rb")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRSHRNBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6182,7 +6182,7 @@ (define_insn "mve_vqrshrntq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_8" "Rb")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRSHRNTQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6199,7 +6199,7 @@ (define_insn "mve_vqshrnbq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHRNBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6216,7 +6216,7 @@ (define_insn "mve_vqshrntq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHRNTQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6250,7 +6250,7 @@ (define_insn "mve_vrshrnbq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_8" "Rb")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRSHRNBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6267,7 +6267,7 @@ (define_insn "mve_vrshrntq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_8" "Rb")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRSHRNTQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6284,7 +6284,7 @@ (define_insn "mve_vshllbq_m_n_<supf><mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
(match_operand:SI 3 "immediate_operand" "i")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHLLBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6301,7 +6301,7 @@ (define_insn "mve_vshlltq_m_n_<supf><mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
(match_operand:SI 3 "immediate_operand" "i")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHLLTQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6318,7 +6318,7 @@ (define_insn "mve_vshrnbq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHRNBQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6335,7 +6335,7 @@ (define_insn "mve_vshrntq_m_n_<supf><mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSHRNTQ_M_N))
]
"TARGET_HAVE_MVE"
@@ -6352,7 +6352,7 @@ (define_insn "mve_vmlsldavaq_p_s<mode>"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:MVE_5 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLSLDAVAQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -6369,7 +6369,7 @@ (define_insn "mve_vmlsldavaxq_p_s<mode>"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:MVE_5 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMLSLDAVAXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -6386,7 +6386,7 @@ (define_insn "mve_vmullbq_poly_m_p<mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
(match_operand:MVE_3 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULLBQ_POLY_M_P))
]
"TARGET_HAVE_MVE"
@@ -6403,7 +6403,7 @@ (define_insn "mve_vmulltq_poly_m_p<mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_3 2 "s_register_operand" "w")
(match_operand:MVE_3 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULLTQ_POLY_M_P))
]
"TARGET_HAVE_MVE"
@@ -6420,7 +6420,7 @@ (define_insn "mve_vqdmullbq_m_n_s<mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMULLBQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6437,7 +6437,7 @@ (define_insn "mve_vqdmullbq_m_s<mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:MVE_5 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMULLBQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6454,7 +6454,7 @@ (define_insn "mve_vqdmulltq_m_n_s<mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMULLTQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6471,7 +6471,7 @@ (define_insn "mve_vqdmulltq_m_s<mode>"
(unspec:<V_double_width> [(match_operand:<V_double_width> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:MVE_5 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQDMULLTQ_M_S))
]
"TARGET_HAVE_MVE"
@@ -6488,7 +6488,7 @@ (define_insn "mve_vqrshrunbq_m_n_s<mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "mve_imm_8" "Rb")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRSHRUNBQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6505,7 +6505,7 @@ (define_insn "mve_vqrshruntq_m_n_s<mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQRSHRUNTQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6522,7 +6522,7 @@ (define_insn "mve_vqshrunbq_m_n_s<mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHRUNBQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6539,7 +6539,7 @@ (define_insn "mve_vqshruntq_m_n_s<mode>"
(unspec:<V_narrow_pack> [(match_operand:<V_narrow_pack> 1 "s_register_operand" "0")
(match_operand:MVE_5 2 "s_register_operand" "w")
(match_operand:SI 3 "<MVE_pred3>" "<MVE_constraint3>")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VQSHRUNTQ_M_N_S))
]
"TARGET_HAVE_MVE"
@@ -6623,7 +6623,7 @@ (define_insn "mve_vabdq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VABDQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6640,7 +6640,7 @@ (define_insn "mve_vaddq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VADDQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6657,7 +6657,7 @@ (define_insn "mve_vaddq_m_n_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VADDQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6674,7 +6674,7 @@ (define_insn "mve_vandq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VANDQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6691,7 +6691,7 @@ (define_insn "mve_vbicq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VBICQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6708,7 +6708,7 @@ (define_insn "mve_vbrsrq_m_n_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:SI 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VBRSRQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6725,7 +6725,7 @@ (define_insn "mve_vcaddq_rot270_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCADDQ_ROT270_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6742,7 +6742,7 @@ (define_insn "mve_vcaddq_rot90_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCADDQ_ROT90_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6759,7 +6759,7 @@ (define_insn "mve_vcmlaq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMLAQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6776,7 +6776,7 @@ (define_insn "mve_vcmlaq_rot180_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMLAQ_ROT180_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6793,7 +6793,7 @@ (define_insn "mve_vcmlaq_rot270_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMLAQ_ROT270_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6810,7 +6810,7 @@ (define_insn "mve_vcmlaq_rot90_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMLAQ_ROT90_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6827,7 +6827,7 @@ (define_insn "mve_vcmulq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMULQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6844,7 +6844,7 @@ (define_insn "mve_vcmulq_rot180_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMULQ_ROT180_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6861,7 +6861,7 @@ (define_insn "mve_vcmulq_rot270_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMULQ_ROT270_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6878,7 +6878,7 @@ (define_insn "mve_vcmulq_rot90_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VCMULQ_ROT90_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6895,7 +6895,7 @@ (define_insn "mve_veorq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VEORQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6912,7 +6912,7 @@ (define_insn "mve_vfmaq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VFMAQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6929,7 +6929,7 @@ (define_insn "mve_vfmaq_m_n_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VFMAQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6946,7 +6946,7 @@ (define_insn "mve_vfmasq_m_n_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VFMASQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6963,7 +6963,7 @@ (define_insn "mve_vfmsq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VFMSQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6980,7 +6980,7 @@ (define_insn "mve_vmaxnmq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMAXNMQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -6997,7 +6997,7 @@ (define_insn "mve_vminnmq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMINNMQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7014,7 +7014,7 @@ (define_insn "mve_vmulq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7031,7 +7031,7 @@ (define_insn "mve_vmulq_m_n_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VMULQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7048,7 +7048,7 @@ (define_insn "mve_vornq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VORNQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7065,7 +7065,7 @@ (define_insn "mve_vorrq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VORRQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7082,7 +7082,7 @@ (define_insn "mve_vsubq_m_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:MVE_0 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSUBQ_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7099,7 +7099,7 @@ (define_insn "mve_vsubq_m_n_f<mode>"
(unspec:MVE_0 [(match_operand:MVE_0 1 "s_register_operand" "0")
(match_operand:MVE_0 2 "s_register_operand" "w")
(match_operand:<V_elem> 3 "s_register_operand" "r")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VSUBQ_M_N_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7248,7 +7248,7 @@ (define_expand "mve_vstrbq_scatter_offset_p_<supf><mode>"
[(match_operand:<MVE_B_ELEM> 0 "mve_scatter_memory")
(match_operand:MVE_2 1 "s_register_operand")
(match_operand:MVE_2 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand" "Up")
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")
(unspec:V4SI [(const_int 0)] VSTRBSOQ)]
"TARGET_HAVE_MVE"
{
@@ -7267,7 +7267,7 @@ (define_insn "mve_vstrbq_scatter_offset_p_<supf><mode>_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VSTRBSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrbt.<V_sz_elem>\t%q2, [%0, %q1]"
@@ -7302,7 +7302,7 @@ (define_insn "mve_vstrwq_scatter_base_p_<supf>v4si"
(define_insn "mve_vstrbq_p_<supf><mode>"
[(set (match_operand:<MVE_B_ELEM> 0 "mve_memory_operand" "=Ux")
(unspec:<MVE_B_ELEM> [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VSTRBQ))
]
"TARGET_HAVE_MVE"
@@ -7323,7 +7323,7 @@ (define_insn "mve_vldrbq_gather_offset_z_<supf><mode>"
[(set (match_operand:MVE_2 0 "s_register_operand" "=&w")
(unspec:MVE_2 [(match_operand:<MVE_B_ELEM> 1 "memory_operand" "Us")
(match_operand:MVE_2 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VLDRBGOQ))
]
"TARGET_HAVE_MVE"
@@ -7347,7 +7347,7 @@ (define_insn "mve_vldrbq_gather_offset_z_<supf><mode>"
(define_insn "mve_vldrbq_z_<supf><mode>"
[(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:<MVE_B_ELEM> 1 "mve_memory_operand" "Ux")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VLDRBQ))
]
"TARGET_HAVE_MVE"
@@ -7434,7 +7434,7 @@ (define_insn "mve_vldrhq_gather_offset_z_<supf><mode>"
[(set (match_operand:MVE_6 0 "s_register_operand" "=&w")
(unspec:MVE_6 [(match_operand:<MVE_H_ELEM> 1 "memory_operand" "Us")
(match_operand:MVE_6 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")
]VLDRHGOQ))
]
"TARGET_HAVE_MVE"
@@ -7482,7 +7482,7 @@ (define_insn "mve_vldrhq_gather_shifted_offset_z_<supf><mode>"
[(set (match_operand:MVE_6 0 "s_register_operand" "=&w")
(unspec:MVE_6 [(match_operand:<MVE_H_ELEM> 1 "memory_operand" "Us")
(match_operand:MVE_6 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")
]VLDRHGSOQ))
]
"TARGET_HAVE_MVE"
@@ -7548,7 +7548,7 @@ (define_insn "mve_vldrhq_z_fv8hf"
(define_insn "mve_vldrhq_z_<supf><mode>"
[(set (match_operand:MVE_6 0 "s_register_operand" "=w")
(unspec:MVE_6 [(match_operand:<MVE_H_ELEM> 1 "mve_memory_operand" "Ux")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VLDRHQ))
]
"TARGET_HAVE_MVE"
@@ -8124,7 +8124,7 @@ (define_insn "mve_vstrhq_p_fv8hf"
(define_insn "mve_vstrhq_p_<supf><mode>"
[(set (match_operand:<MVE_H_ELEM> 0 "mve_memory_operand" "=Ux")
(unspec:<MVE_H_ELEM> [(match_operand:MVE_6 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VSTRHQ))
]
"TARGET_HAVE_MVE"
@@ -8145,7 +8145,7 @@ (define_expand "mve_vstrhq_scatter_offset_p_<supf><mode>"
[(match_operand:<MVE_H_ELEM> 0 "mve_scatter_memory")
(match_operand:MVE_6 1 "s_register_operand")
(match_operand:MVE_6 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRHSOQ)]
"TARGET_HAVE_MVE"
{
@@ -8164,7 +8164,7 @@ (define_insn "mve_vstrhq_scatter_offset_p_<supf><mode>_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:MVE_6 1 "s_register_operand" "w")
(match_operand:MVE_6 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VSTRHSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrht.<V_sz_elem>\t%q2, [%0, %q1]"
@@ -8205,7 +8205,7 @@ (define_expand "mve_vstrhq_scatter_shifted_offset_p_<supf><mode>"
[(match_operand:<MVE_H_ELEM> 0 "mve_scatter_memory")
(match_operand:MVE_6 1 "s_register_operand")
(match_operand:MVE_6 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRHSSOQ)]
"TARGET_HAVE_MVE"
{
@@ -8224,7 +8224,7 @@ (define_insn "mve_vstrhq_scatter_shifted_offset_p_<supf><mode>_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:MVE_6 1 "s_register_operand" "w")
(match_operand:MVE_6 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VSTRHSSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrht.<V_sz_elem>\t%q2, [%0, %q1, uxtw #1]"
@@ -9011,7 +9011,7 @@ (define_expand "mve_vidupq_m_n_u<mode>"
(match_operand:MVE_2 1 "s_register_operand")
(match_operand:SI 2 "s_register_operand")
(match_operand:SI 3 "mve_imm_selective_upto_8")
- (match_operand:HI 4 "vpr_register_operand")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand")]
"TARGET_HAVE_MVE"
{
rtx temp = gen_reg_rtx (SImode);
@@ -9031,7 +9031,7 @@ (define_insn "mve_vidupq_m_wb_u<mode>_insn"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:SI 3 "s_register_operand" "2")
(match_operand:SI 4 "mve_imm_selective_upto_8" "Rg")
- (match_operand:HI 5 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand" "Up")]
VIDUPQ_M))
(set (match_operand:SI 2 "s_register_operand" "=Te")
(plus:SI (match_dup 3)
@@ -9079,7 +9079,7 @@ (define_expand "mve_vddupq_m_n_u<mode>"
(match_operand:MVE_2 1 "s_register_operand")
(match_operand:SI 2 "s_register_operand")
(match_operand:SI 3 "mve_imm_selective_upto_8")
- (match_operand:HI 4 "vpr_register_operand")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand")]
"TARGET_HAVE_MVE"
{
rtx temp = gen_reg_rtx (SImode);
@@ -9099,7 +9099,7 @@ (define_insn "mve_vddupq_m_wb_u<mode>_insn"
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "0")
(match_operand:SI 3 "s_register_operand" "2")
(match_operand:SI 4 "mve_imm_selective_upto_8" "Rg")
- (match_operand:HI 5 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand" "Up")]
VDDUPQ_M))
(set (match_operand:SI 2 "s_register_operand" "=Te")
(minus:SI (match_dup 3)
@@ -9170,7 +9170,7 @@ (define_expand "mve_vdwdupq_m_n_u<mode>"
(match_operand:SI 2 "s_register_operand")
(match_operand:DI 3 "s_register_operand")
(match_operand:SI 4 "mve_imm_selective_upto_8")
- (match_operand:HI 5 "vpr_register_operand")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand")]
"TARGET_HAVE_MVE"
{
rtx ignore_wb = gen_reg_rtx (SImode);
@@ -9190,7 +9190,7 @@ (define_expand "mve_vdwdupq_m_wb_u<mode>"
(match_operand:SI 2 "s_register_operand")
(match_operand:DI 3 "s_register_operand")
(match_operand:SI 4 "mve_imm_selective_upto_8")
- (match_operand:HI 5 "vpr_register_operand")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand")]
"TARGET_HAVE_MVE"
{
rtx ignore_vec = gen_reg_rtx (<MODE>mode);
@@ -9210,7 +9210,7 @@ (define_insn "mve_vdwdupq_m_wb_u<mode>_insn"
(match_operand:SI 3 "s_register_operand" "1")
(subreg:SI (match_operand:DI 4 "s_register_operand" "r") 4)
(match_operand:SI 5 "mve_imm_selective_upto_8" "Rg")
- (match_operand:HI 6 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 6 "vpr_register_operand" "Up")]
VDWDUPQ_M))
(set (match_operand:SI 1 "s_register_operand" "=Te")
(unspec:SI [(match_dup 2)
@@ -9287,7 +9287,7 @@ (define_expand "mve_viwdupq_m_n_u<mode>"
(match_operand:SI 2 "s_register_operand")
(match_operand:DI 3 "s_register_operand")
(match_operand:SI 4 "mve_imm_selective_upto_8")
- (match_operand:HI 5 "vpr_register_operand")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand")]
"TARGET_HAVE_MVE"
{
rtx ignore_wb = gen_reg_rtx (SImode);
@@ -9307,7 +9307,7 @@ (define_expand "mve_viwdupq_m_wb_u<mode>"
(match_operand:SI 2 "s_register_operand")
(match_operand:DI 3 "s_register_operand")
(match_operand:SI 4 "mve_imm_selective_upto_8")
- (match_operand:HI 5 "vpr_register_operand")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand")]
"TARGET_HAVE_MVE"
{
rtx ignore_vec = gen_reg_rtx (<MODE>mode);
@@ -9327,7 +9327,7 @@ (define_insn "mve_viwdupq_m_wb_u<mode>_insn"
(match_operand:SI 3 "s_register_operand" "1")
(subreg:SI (match_operand:DI 4 "s_register_operand" "r") 4)
(match_operand:SI 5 "mve_imm_selective_upto_8" "Rg")
- (match_operand:HI 6 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 6 "vpr_register_operand" "Up")]
VIWDUPQ_M))
(set (match_operand:SI 1 "s_register_operand" "=Te")
(unspec:SI [(match_dup 2)
@@ -10335,7 +10335,7 @@ (define_expand "mve_vshlcq_m_vec_<supf><mode>"
(match_operand:MVE_2 1 "s_register_operand")
(match_operand:SI 2 "s_register_operand")
(match_operand:SI 3 "mve_imm_32")
- (match_operand:HI 4 "vpr_register_operand")
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand")
(unspec:MVE_2 [(const_int 0)] VSHLCQ_M)]
"TARGET_HAVE_MVE"
{
@@ -10351,7 +10351,7 @@ (define_expand "mve_vshlcq_m_carry_<supf><mode>"
(match_operand:MVE_2 1 "s_register_operand")
(match_operand:SI 2 "s_register_operand")
(match_operand:SI 3 "mve_imm_32")
- (match_operand:HI 4 "vpr_register_operand")
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand")
(unspec:MVE_2 [(const_int 0)] VSHLCQ_M)]
"TARGET_HAVE_MVE"
{
@@ -10367,7 +10367,7 @@ (define_insn "mve_vshlcq_m_<supf><mode>"
(unspec:MVE_2 [(match_operand:MVE_2 2 "s_register_operand" "0")
(match_operand:SI 3 "s_register_operand" "1")
(match_operand:SI 4 "mve_imm_32" "Rf")
- (match_operand:HI 5 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 5 "vpr_register_operand" "Up")]
VSHLCQ_M))
(set (match_operand:SI 1 "s_register_operand" "=r")
(unspec:SI [(match_dup 2)
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 12/14] arm: Convert more load/store MVE builtins to predicate qualifiers
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (10 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 11/14] arm: Convert more MVE " Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-15 15:08 ` Richard Sandiford
2021-10-13 10:15 ` [PATCH v2 13/14] arm: Convert more MVE/CDE " Christophe Lyon
` (2 subsequent siblings)
14 siblings, 1 reply; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This patch covers a few builtins where we do not use the <mode>
iterator and thus we cannot use <MVE_vpred>.
For v2di instructions, we use the V8BI mode for predicates.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
PR target/100757
PR target/101325
* config/arm/arm-builtins.c (STRSBS_P_QUALIFIERS): Use predicate
qualifier.
(STRSBU_P_QUALIFIERS): Likewise.
(LDRGBS_Z_QUALIFIERS): Likewise.
(LDRGBU_Z_QUALIFIERS): Likewise.
(LDRGBWBXU_Z_QUALIFIERS): Likewise.
(LDRGBWBS_Z_QUALIFIERS): Likewise.
(LDRGBWBU_Z_QUALIFIERS): Likewise.
(STRSBWBS_P_QUALIFIERS): Likewise.
(STRSBWBU_P_QUALIFIERS): Likewise.
* config/arm/mve.md: Use VxBI instead of HI.
diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
index 06ff9d2278a..e58580bb828 100644
--- a/gcc/config/arm/arm-builtins.c
+++ b/gcc/config/arm/arm-builtins.c
@@ -738,13 +738,13 @@ arm_strss_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_strsbs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_void, qualifier_unsigned, qualifier_immediate,
- qualifier_none, qualifier_unsigned};
+ qualifier_none, qualifier_predicate};
#define STRSBS_P_QUALIFIERS (arm_strsbs_p_qualifiers)
static enum arm_type_qualifiers
arm_strsbu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_void, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned, qualifier_unsigned};
+ qualifier_unsigned, qualifier_predicate};
#define STRSBU_P_QUALIFIERS (arm_strsbu_p_qualifiers)
static enum arm_type_qualifiers
@@ -780,13 +780,13 @@ arm_ldrgbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_ldrgbs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGBS_Z_QUALIFIERS (arm_ldrgbs_z_qualifiers)
static enum arm_type_qualifiers
arm_ldrgbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGBU_Z_QUALIFIERS (arm_ldrgbu_z_qualifiers)
static enum arm_type_qualifiers
@@ -826,7 +826,7 @@ arm_ldrgbwbxu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_ldrgbwbxu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGBWBXU_Z_QUALIFIERS (arm_ldrgbwbxu_z_qualifiers)
static enum arm_type_qualifiers
@@ -842,13 +842,13 @@ arm_ldrgbwbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_ldrgbwbs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGBWBS_Z_QUALIFIERS (arm_ldrgbwbs_z_qualifiers)
static enum arm_type_qualifiers
arm_ldrgbwbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
- qualifier_unsigned};
+ qualifier_predicate};
#define LDRGBWBU_Z_QUALIFIERS (arm_ldrgbwbu_z_qualifiers)
static enum arm_type_qualifiers
@@ -864,13 +864,13 @@ arm_strsbwbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
static enum arm_type_qualifiers
arm_strsbwbs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_const,
- qualifier_none, qualifier_unsigned};
+ qualifier_none, qualifier_predicate};
#define STRSBWBS_P_QUALIFIERS (arm_strsbwbs_p_qualifiers)
static enum arm_type_qualifiers
arm_strsbwbu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_const,
- qualifier_unsigned, qualifier_unsigned};
+ qualifier_unsigned, qualifier_predicate};
#define STRSBWBU_P_QUALIFIERS (arm_strsbwbu_p_qualifiers)
static enum arm_type_qualifiers
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index 81ad488155d..c07487c0750 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -7282,7 +7282,7 @@ (define_insn "mve_vstrwq_scatter_base_p_<supf>v4si"
[(match_operand:V4SI 0 "s_register_operand" "w")
(match_operand:SI 1 "immediate_operand" "i")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VSTRWSBQ))
]
"TARGET_HAVE_MVE"
@@ -7371,7 +7371,7 @@ (define_insn "mve_vldrwq_gather_base_z_<supf>v4si"
[(set (match_operand:V4SI 0 "s_register_operand" "=&w")
(unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:SI 2 "immediate_operand" "i")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VLDRWGBQ))
]
"TARGET_HAVE_MVE"
@@ -7609,7 +7609,7 @@ (define_insn "mve_vldrwq_<supf>v4si"
(define_insn "mve_vldrwq_z_fv4sf"
[(set (match_operand:V4SF 0 "s_register_operand" "=w")
(unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Ux")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 2 "vpr_register_operand" "Up")]
VLDRWQ_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7629,7 +7629,7 @@ (define_insn "mve_vldrwq_z_fv4sf"
(define_insn "mve_vldrwq_z_<supf>v4si"
[(set (match_operand:V4SI 0 "s_register_operand" "=w")
(unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Ux")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 2 "vpr_register_operand" "Up")]
VLDRWQ))
]
"TARGET_HAVE_MVE"
@@ -7690,7 +7690,7 @@ (define_insn "mve_vldrdq_gather_base_z_<supf>v2di"
[(set (match_operand:V2DI 0 "s_register_operand" "=&w")
(unspec:V2DI [(match_operand:V2DI 1 "s_register_operand" "w")
(match_operand:SI 2 "immediate_operand" "i")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VLDRDGBQ))
]
"TARGET_HAVE_MVE"
@@ -7731,7 +7731,7 @@ (define_insn "mve_vldrdq_gather_offset_z_<supf>v2di"
[(set (match_operand:V2DI 0 "s_register_operand" "=&w")
(unspec:V2DI [(match_operand:V2DI 1 "memory_operand" "Us")
(match_operand:V2DI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VLDRDGOQ))
]
"TARGET_HAVE_MVE"
@@ -7772,7 +7772,7 @@ (define_insn "mve_vldrdq_gather_shifted_offset_z_<supf>v2di"
[(set (match_operand:V2DI 0 "s_register_operand" "=&w")
(unspec:V2DI [(match_operand:V2DI 1 "memory_operand" "Us")
(match_operand:V2DI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VLDRDGSOQ))
]
"TARGET_HAVE_MVE"
@@ -7813,7 +7813,7 @@ (define_insn "mve_vldrhq_gather_offset_z_fv8hf"
[(set (match_operand:V8HF 0 "s_register_operand" "=&w")
(unspec:V8HF [(match_operand:V8HI 1 "memory_operand" "Us")
(match_operand:V8HI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VLDRHQGO_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7855,7 +7855,7 @@ (define_insn "mve_vldrhq_gather_shifted_offset_z_fv8hf"
[(set (match_operand:V8HF 0 "s_register_operand" "=&w")
(unspec:V8HF [(match_operand:V8HI 1 "memory_operand" "Us")
(match_operand:V8HI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VLDRHQGSO_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7897,7 +7897,7 @@ (define_insn "mve_vldrwq_gather_base_z_fv4sf"
[(set (match_operand:V4SF 0 "s_register_operand" "=&w")
(unspec:V4SF [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:SI 2 "immediate_operand" "i")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VLDRWQGB_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7958,7 +7958,7 @@ (define_insn "mve_vldrwq_gather_offset_z_fv4sf"
[(set (match_operand:V4SF 0 "s_register_operand" "=&w")
(unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Us")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VLDRWQGO_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -7980,7 +7980,7 @@ (define_insn "mve_vldrwq_gather_offset_z_<supf>v4si"
[(set (match_operand:V4SI 0 "s_register_operand" "=&w")
(unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Us")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VLDRWGOQ))
]
"TARGET_HAVE_MVE"
@@ -8042,7 +8042,7 @@ (define_insn "mve_vldrwq_gather_shifted_offset_z_fv4sf"
[(set (match_operand:V4SF 0 "s_register_operand" "=&w")
(unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Us")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VLDRWQGSO_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -8064,7 +8064,7 @@ (define_insn "mve_vldrwq_gather_shifted_offset_z_<supf>v4si"
[(set (match_operand:V4SI 0 "s_register_operand" "=&w")
(unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Us")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VLDRWGSOQ))
]
"TARGET_HAVE_MVE"
@@ -8104,7 +8104,7 @@ (define_insn "mve_vstrhq_fv8hf"
(define_insn "mve_vstrhq_p_fv8hf"
[(set (match_operand:V8HI 0 "mve_memory_operand" "=Ux")
(unspec:V8HI [(match_operand:V8HF 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 2 "vpr_register_operand" "Up")]
VSTRHQ_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -8323,7 +8323,7 @@ (define_insn "mve_vstrwq_p_fv4sf"
(define_insn "mve_vstrwq_p_<supf>v4si"
[(set (match_operand:V4SI 0 "memory_operand" "=Ux")
(unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 2 "vpr_register_operand" "Up")]
VSTRWQ))
]
"TARGET_HAVE_MVE"
@@ -8385,7 +8385,7 @@ (define_insn "mve_vstrdq_scatter_base_p_<supf>v2di"
[(match_operand:V2DI 0 "s_register_operand" "w")
(match_operand:SI 1 "mve_vldrd_immediate" "Ri")
(match_operand:V2DI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VSTRDSBQ))
]
"TARGET_HAVE_MVE"
@@ -8428,7 +8428,7 @@ (define_expand "mve_vstrdq_scatter_offset_p_<supf>v2di"
[(match_operand:V2DI 0 "mve_scatter_memory")
(match_operand:V2DI 1 "s_register_operand")
(match_operand:V2DI 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V8BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRDSOQ)]
"TARGET_HAVE_MVE"
{
@@ -8446,7 +8446,7 @@ (define_insn "mve_vstrdq_scatter_offset_p_<supf>v2di_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V2DI 1 "s_register_operand" "w")
(match_operand:V2DI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VSTRDSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrdt.64\t%q2, [%0, %q1]"
@@ -8487,7 +8487,7 @@ (define_expand "mve_vstrdq_scatter_shifted_offset_p_<supf>v2di"
[(match_operand:V2DI 0 "mve_scatter_memory")
(match_operand:V2DI 1 "s_register_operand")
(match_operand:V2DI 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V8BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRDSSOQ)]
"TARGET_HAVE_MVE"
{
@@ -8506,7 +8506,7 @@ (define_insn "mve_vstrdq_scatter_shifted_offset_p_<supf>v2di_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V2DI 1 "s_register_operand" "w")
(match_operand:V2DI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VSTRDSSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrdt.64\t%q2, [%0, %q1, UXTW #3]"
@@ -8576,7 +8576,7 @@ (define_expand "mve_vstrhq_scatter_offset_p_fv8hf"
[(match_operand:V8HI 0 "mve_scatter_memory")
(match_operand:V8HI 1 "s_register_operand")
(match_operand:V8HF 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V8BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRHQSO_F)]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
{
@@ -8594,7 +8594,7 @@ (define_insn "mve_vstrhq_scatter_offset_p_fv8hf_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V8HI 1 "s_register_operand" "w")
(match_operand:V8HF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VSTRHQSO_F))]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
"vpst\;vstrht.16\t%q2, [%0, %q1]"
@@ -8635,7 +8635,7 @@ (define_expand "mve_vstrhq_scatter_shifted_offset_p_fv8hf"
[(match_operand:V8HI 0 "memory_operand" "=Us")
(match_operand:V8HI 1 "s_register_operand" "w")
(match_operand:V8HF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")
(unspec:V4SI [(const_int 0)] VSTRHQSSO_F)]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
{
@@ -8654,7 +8654,7 @@ (define_insn "mve_vstrhq_scatter_shifted_offset_p_fv8hf_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V8HI 1 "s_register_operand" "w")
(match_operand:V8HF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V8BI 3 "vpr_register_operand" "Up")]
VSTRHQSSO_F))]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
"vpst\;vstrht.16\t%q2, [%0, %q1, uxtw #1]"
@@ -8691,7 +8691,7 @@ (define_insn "mve_vstrwq_scatter_base_p_fv4sf"
[(match_operand:V4SI 0 "s_register_operand" "w")
(match_operand:SI 1 "immediate_operand" "i")
(match_operand:V4SF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VSTRWQSB_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -8740,7 +8740,7 @@ (define_expand "mve_vstrwq_scatter_offset_p_fv4sf"
[(match_operand:V4SI 0 "mve_scatter_memory")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:V4SF 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRWQSO_F)]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
{
@@ -8758,7 +8758,7 @@ (define_insn "mve_vstrwq_scatter_offset_p_fv4sf_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VSTRWQSO_F))]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
"vpst\;vstrwt.32\t%q2, [%0, %q1]"
@@ -8771,7 +8771,7 @@ (define_expand "mve_vstrwq_scatter_offset_p_<supf>v4si"
[(match_operand:V4SI 0 "mve_scatter_memory")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:V4SI 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRWSOQ)]
"TARGET_HAVE_MVE"
{
@@ -8789,7 +8789,7 @@ (define_insn "mve_vstrwq_scatter_offset_p_<supf>v4si_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VSTRWSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrwt.32\t%q2, [%0, %q1]"
@@ -8858,7 +8858,7 @@ (define_expand "mve_vstrwq_scatter_shifted_offset_p_fv4sf"
[(match_operand:V4SI 0 "mve_scatter_memory")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:V4SF 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRWQSSO_F)]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
{
@@ -8877,7 +8877,7 @@ (define_insn "mve_vstrwq_scatter_shifted_offset_p_fv4sf_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VSTRWQSSO_F))]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
"vpst\;vstrwt.32\t%q2, [%0, %q1, uxtw #2]"
@@ -8890,7 +8890,7 @@ (define_expand "mve_vstrwq_scatter_shifted_offset_p_<supf>v4si"
[(match_operand:V4SI 0 "mve_scatter_memory")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:V4SI 2 "s_register_operand")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VSTRWSSOQ)]
"TARGET_HAVE_MVE"
{
@@ -8909,7 +8909,7 @@ (define_insn "mve_vstrwq_scatter_shifted_offset_p_<supf>v4si_insn"
[(match_operand:SI 0 "register_operand" "r")
(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VSTRWSSOQ))]
"TARGET_HAVE_MVE"
"vpst\;vstrwt.32\t%q2, [%0, %q1, uxtw #2]"
@@ -9376,7 +9376,7 @@ (define_insn "mve_vstrwq_scatter_base_wb_p_<supf>v4si"
[(match_operand:V4SI 1 "s_register_operand" "0")
(match_operand:SI 2 "mve_vldrd_immediate" "Ri")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand")]
+ (match_operand:V4BI 4 "vpr_register_operand")]
VSTRWSBWBQ))
(set (match_operand:V4SI 0 "s_register_operand" "=w")
(unspec:V4SI [(match_dup 1) (match_dup 2)]
@@ -9427,7 +9427,7 @@ (define_insn "mve_vstrwq_scatter_base_wb_p_fv4sf"
[(match_operand:V4SI 1 "s_register_operand" "0")
(match_operand:SI 2 "mve_vldrd_immediate" "Ri")
(match_operand:V4SF 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand")]
+ (match_operand:V4BI 4 "vpr_register_operand")]
VSTRWQSBWB_F))
(set (match_operand:V4SI 0 "s_register_operand" "=w")
(unspec:V4SI [(match_dup 1) (match_dup 2)]
@@ -9478,7 +9478,7 @@ (define_insn "mve_vstrdq_scatter_base_wb_p_<supf>v2di"
[(match_operand:V2DI 1 "s_register_operand" "0")
(match_operand:SI 2 "mve_vldrd_immediate" "Ri")
(match_operand:V2DI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand")]
+ (match_operand:V8BI 4 "vpr_register_operand")]
VSTRDSBWBQ))
(set (match_operand:V2DI 0 "s_register_operand" "=w")
(unspec:V2DI [(match_dup 1) (match_dup 2)]
@@ -9551,7 +9551,7 @@ (define_expand "mve_vldrwq_gather_base_wb_z_<supf>v4si"
[(match_operand:V4SI 0 "s_register_operand")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:SI 2 "mve_vldrd_immediate")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VLDRWGBWBQ)]
"TARGET_HAVE_MVE"
{
@@ -9566,7 +9566,7 @@ (define_expand "mve_vldrwq_gather_base_nowb_z_<supf>v4si"
[(match_operand:V4SI 0 "s_register_operand")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:SI 2 "mve_vldrd_immediate")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VLDRWGBWBQ)]
"TARGET_HAVE_MVE"
{
@@ -9585,7 +9585,7 @@ (define_insn "mve_vldrwq_gather_base_wb_z_<supf>v4si_insn"
[(set (match_operand:V4SI 0 "s_register_operand" "=&w")
(unspec:V4SI [(match_operand:V4SI 2 "s_register_operand" "1")
(match_operand:SI 3 "mve_vldrd_immediate" "Ri")
- (match_operand:HI 4 "vpr_register_operand" "Up")
+ (match_operand:V4BI 4 "vpr_register_operand" "Up")
(mem:BLK (scratch))]
VLDRWGBWBQ))
(set (match_operand:V4SI 1 "s_register_operand" "=&w")
@@ -9659,7 +9659,7 @@ (define_expand "mve_vldrwq_gather_base_wb_z_fv4sf"
[(match_operand:V4SI 0 "s_register_operand")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:SI 2 "mve_vldrd_immediate")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VLDRWQGBWB_F)]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
{
@@ -9675,7 +9675,7 @@ (define_expand "mve_vldrwq_gather_base_nowb_z_fv4sf"
[(match_operand:V4SF 0 "s_register_operand")
(match_operand:V4SI 1 "s_register_operand")
(match_operand:SI 2 "mve_vldrd_immediate")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V4BI 3 "vpr_register_operand")
(unspec:V4SI [(const_int 0)] VLDRWQGBWB_F)]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
{
@@ -9694,7 +9694,7 @@ (define_insn "mve_vldrwq_gather_base_wb_z_fv4sf_insn"
[(set (match_operand:V4SF 0 "s_register_operand" "=&w")
(unspec:V4SF [(match_operand:V4SI 2 "s_register_operand" "1")
(match_operand:SI 3 "mve_vldrd_immediate" "Ri")
- (match_operand:HI 4 "vpr_register_operand" "Up")
+ (match_operand:V4BI 4 "vpr_register_operand" "Up")
(mem:BLK (scratch))]
VLDRWQGBWB_F))
(set (match_operand:V4SI 1 "s_register_operand" "=&w")
@@ -9769,7 +9769,7 @@ (define_expand "mve_vldrdq_gather_base_wb_z_<supf>v2di"
[(match_operand:V2DI 0 "s_register_operand")
(match_operand:V2DI 1 "s_register_operand")
(match_operand:SI 2 "mve_vldrd_immediate")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V8BI 3 "vpr_register_operand")
(unspec:V2DI [(const_int 0)] VLDRDGBWBQ)]
"TARGET_HAVE_MVE"
{
@@ -9785,7 +9785,7 @@ (define_expand "mve_vldrdq_gather_base_nowb_z_<supf>v2di"
[(match_operand:V2DI 0 "s_register_operand")
(match_operand:V2DI 1 "s_register_operand")
(match_operand:SI 2 "mve_vldrd_immediate")
- (match_operand:HI 3 "vpr_register_operand")
+ (match_operand:V8BI 3 "vpr_register_operand")
(unspec:V2DI [(const_int 0)] VLDRDGBWBQ)]
"TARGET_HAVE_MVE"
{
@@ -9819,7 +9819,7 @@ (define_insn "mve_vldrdq_gather_base_wb_z_<supf>v2di_insn"
[(set (match_operand:V2DI 0 "s_register_operand" "=&w")
(unspec:V2DI [(match_operand:V2DI 2 "s_register_operand" "1")
(match_operand:SI 3 "mve_vldrd_immediate" "Ri")
- (match_operand:HI 4 "vpr_register_operand" "Up")
+ (match_operand:V8BI 4 "vpr_register_operand" "Up")
(mem:BLK (scratch))]
VLDRDGBWBQ))
(set (match_operand:V2DI 1 "s_register_operand" "=&w")
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 13/14] arm: Convert more MVE/CDE builtins to predicate qualifiers
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (11 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 12/14] arm: Convert more load/store " Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 14/14] arm: Add VPR_REG to ALL_REGS Christophe Lyon
2021-10-15 15:10 ` [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Richard Sandiford
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
This patch covers a few non-load/store builtins where we do not use
the <mode> iterator and thus we cannot use <MVE_vpred>.
We need to update the expected code in cde-mve-full-assembly.c because
we now use mve_movv16qi instead of movhi to generate the vmsr
instruction.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
PR target/100757
PR target/101325
* config/arm/arm-builtins.c (CX_UNARY_UNONE_QUALIFIERS): Use
predicate.
(CX_BINARY_UNONE_QUALIFIERS): Likewise.
(CX_TERNARY_UNONE_QUALIFIERS): Likewise.
(TERNOP_NONE_NONE_NONE_UNONE_QUALIFIERS): Delete.
(QUADOP_NONE_NONE_NONE_NONE_UNONE_QUALIFIERS): Delete.
(QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE_QUALIFIERS): Delete.
* config/arm/arm_mve_builtins.def: Use predicated qualifiers.
* config/arm/mve.md: Use VxBI instead of HI.
gcc/testsuite/
* gcc.target/arm/acle/cde-mve-full-assembly.c: Remove expected '@ movhi'.
diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
index e58580bb828..d725458f1ad 100644
--- a/gcc/config/arm/arm-builtins.c
+++ b/gcc/config/arm/arm-builtins.c
@@ -344,7 +344,7 @@ static enum arm_type_qualifiers
arm_cx_unary_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_immediate, qualifier_none,
qualifier_unsigned_immediate,
- qualifier_unsigned };
+ qualifier_predicate };
#define CX_UNARY_UNONE_QUALIFIERS (arm_cx_unary_unone_qualifiers)
/* T (immediate, T, T, unsigned immediate). */
@@ -353,7 +353,7 @@ arm_cx_binary_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_immediate,
qualifier_none, qualifier_none,
qualifier_unsigned_immediate,
- qualifier_unsigned };
+ qualifier_predicate };
#define CX_BINARY_UNONE_QUALIFIERS (arm_cx_binary_unone_qualifiers)
/* T (immediate, T, T, T, unsigned immediate). */
@@ -362,7 +362,7 @@ arm_cx_ternary_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_immediate,
qualifier_none, qualifier_none, qualifier_none,
qualifier_unsigned_immediate,
- qualifier_unsigned };
+ qualifier_predicate };
#define CX_TERNARY_UNONE_QUALIFIERS (arm_cx_ternary_unone_qualifiers)
/* The first argument (return type) of a store should be void type,
@@ -558,12 +558,6 @@ arm_ternop_none_none_none_imm_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define TERNOP_NONE_NONE_NONE_IMM_QUALIFIERS \
(arm_ternop_none_none_none_imm_qualifiers)
-static enum arm_type_qualifiers
-arm_ternop_none_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_none, qualifier_none, qualifier_none, qualifier_unsigned };
-#define TERNOP_NONE_NONE_NONE_UNONE_QUALIFIERS \
- (arm_ternop_none_none_none_unone_qualifiers)
-
static enum arm_type_qualifiers
arm_ternop_none_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_none, qualifier_predicate };
@@ -616,13 +610,6 @@ arm_quadop_unone_unone_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define QUADOP_UNONE_UNONE_NONE_NONE_PRED_QUALIFIERS \
(arm_quadop_unone_unone_none_none_pred_qualifiers)
-static enum arm_type_qualifiers
-arm_quadop_none_none_none_none_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_none, qualifier_none, qualifier_none, qualifier_none,
- qualifier_unsigned };
-#define QUADOP_NONE_NONE_NONE_NONE_UNONE_QUALIFIERS \
- (arm_quadop_none_none_none_none_unone_qualifiers)
-
static enum arm_type_qualifiers
arm_quadop_none_none_none_none_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_none, qualifier_none, qualifier_none, qualifier_none,
@@ -637,13 +624,6 @@ arm_quadop_none_none_none_imm_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
#define QUADOP_NONE_NONE_NONE_IMM_PRED_QUALIFIERS \
(arm_quadop_none_none_none_imm_pred_qualifiers)
-static enum arm_type_qualifiers
-arm_quadop_unone_unone_unone_unone_unone_qualifiers[SIMD_MAX_BUILTIN_ARGS]
- = { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
- qualifier_unsigned, qualifier_unsigned };
-#define QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE_QUALIFIERS \
- (arm_quadop_unone_unone_unone_unone_unone_qualifiers)
-
static enum arm_type_qualifiers
arm_quadop_unone_unone_unone_unone_pred_qualifiers[SIMD_MAX_BUILTIN_ARGS]
= { qualifier_unsigned, qualifier_unsigned, qualifier_unsigned,
diff --git a/gcc/config/arm/arm_mve_builtins.def b/gcc/config/arm/arm_mve_builtins.def
index bb79edf83ca..0fb53d866ec 100644
--- a/gcc/config/arm/arm_mve_builtins.def
+++ b/gcc/config/arm/arm_mve_builtins.def
@@ -87,8 +87,8 @@ VAR4 (BINOP_UNONE_UNONE_UNONE, vcreateq_u, v16qi, v8hi, v4si, v2di)
VAR4 (BINOP_NONE_UNONE_UNONE, vcreateq_s, v16qi, v8hi, v4si, v2di)
VAR3 (BINOP_UNONE_UNONE_IMM, vshrq_n_u, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_IMM, vshrq_n_s, v16qi, v8hi, v4si)
-VAR1 (BINOP_NONE_NONE_UNONE, vaddlvq_p_s, v4si)
-VAR1 (BINOP_UNONE_UNONE_UNONE, vaddlvq_p_u, v4si)
+VAR1 (BINOP_NONE_NONE_PRED, vaddlvq_p_s, v4si)
+VAR1 (BINOP_UNONE_UNONE_PRED, vaddlvq_p_u, v4si)
VAR3 (BINOP_PRED_NONE_NONE, vcmpneq_, v16qi, v8hi, v4si)
VAR3 (BINOP_NONE_NONE_NONE, vshlq_s, v16qi, v8hi, v4si)
VAR3 (BINOP_UNONE_UNONE_NONE, vshlq_u, v16qi, v8hi, v4si)
@@ -465,20 +465,20 @@ VAR2 (TERNOP_NONE_NONE_NONE_IMM, vqshrnbq_n_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_NONE_IMM, vqrshrntq_n_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_IMM_PRED, vorrq_m_n_s, v8hi, v4si)
VAR2 (TERNOP_NONE_NONE_IMM_PRED, vmvnq_m_n_s, v8hi, v4si)
-VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrmlaldavhq_p_u, v4si)
-VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vrev16q_m_u, v16qi)
-VAR1 (TERNOP_UNONE_UNONE_UNONE_UNONE, vaddlvaq_p_u, v4si)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vrmlsldavhxq_p_s, v4si)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vrmlsldavhq_p_s, v4si)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vrmlaldavhxq_p_s, v4si)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vrmlaldavhq_p_s, v4si)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vrev32q_m_f, v8hf)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vrev16q_m_s, v16qi)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vcvttq_m_f32_f16, v4sf)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vcvttq_m_f16_f32, v8hf)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vcvtbq_m_f32_f16, v4sf)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vcvtbq_m_f16_f32, v8hf)
-VAR1 (TERNOP_NONE_NONE_NONE_UNONE, vaddlvaq_p_s, v4si)
+VAR1 (TERNOP_UNONE_UNONE_UNONE_PRED, vrmlaldavhq_p_u, v4si)
+VAR1 (TERNOP_UNONE_UNONE_UNONE_PRED, vrev16q_m_u, v16qi)
+VAR1 (TERNOP_UNONE_UNONE_UNONE_PRED, vaddlvaq_p_u, v4si)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vrmlsldavhxq_p_s, v4si)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vrmlsldavhq_p_s, v4si)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vrmlaldavhxq_p_s, v4si)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vrmlaldavhq_p_s, v4si)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vrev32q_m_f, v8hf)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vrev16q_m_s, v16qi)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vcvttq_m_f32_f16, v4sf)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vcvttq_m_f16_f32, v8hf)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vcvtbq_m_f32_f16, v4sf)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vcvtbq_m_f16_f32, v8hf)
+VAR1 (TERNOP_NONE_NONE_NONE_PRED, vaddlvaq_p_s, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlsldavhaxq_s, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlsldavhaq_s, v4si)
VAR1 (TERNOP_NONE_NONE_NONE_NONE, vrmlaldavhaxq_s, v4si)
@@ -629,11 +629,11 @@ VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqshrntq_m_n_s, v8hi, v4si)
VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqshrnbq_m_n_s, v8hi, v4si)
VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqrshrntq_m_n_s, v8hi, v4si)
VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vqrshrnbq_m_n_s, v8hi, v4si)
-VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vrmlaldavhaq_p_u, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlsldavhaxq_p_s, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlsldavhaq_p_s, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlaldavhaxq_p_s, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vrmlaldavhaq_p_s, v4si)
+VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vrmlaldavhaq_p_u, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrmlsldavhaxq_p_s, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrmlsldavhaq_p_s, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrmlaldavhaxq_p_s, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vrmlaldavhaq_p_s, v4si)
VAR2 (QUADOP_UNONE_UNONE_NONE_IMM_PRED, vcvtq_m_n_from_f_u, v8hi, v4si)
VAR2 (QUADOP_NONE_NONE_NONE_IMM_PRED, vcvtq_m_n_from_f_s, v8hi, v4si)
VAR2 (QUADOP_NONE_NONE_NONE_NONE_PRED, vbrsrq_m_n_f, v8hf, v4sf)
@@ -845,14 +845,14 @@ VAR1 (BINOP_NONE_NONE_NONE, vsbciq_s, v4si)
VAR1 (BINOP_UNONE_UNONE_UNONE, vsbciq_u, v4si)
VAR1 (BINOP_NONE_NONE_NONE, vsbcq_s, v4si)
VAR1 (BINOP_UNONE_UNONE_UNONE, vsbcq_u, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vadciq_m_s, v4si)
-VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vadciq_m_u, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vadcq_m_s, v4si)
-VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vadcq_m_u, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vsbciq_m_s, v4si)
-VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vsbciq_m_u, v4si)
-VAR1 (QUADOP_NONE_NONE_NONE_NONE_UNONE, vsbcq_m_s, v4si)
-VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_UNONE, vsbcq_m_u, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vadciq_m_s, v4si)
+VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vadciq_m_u, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vadcq_m_s, v4si)
+VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vadcq_m_u, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vsbciq_m_s, v4si)
+VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vsbciq_m_u, v4si)
+VAR1 (QUADOP_NONE_NONE_NONE_NONE_PRED, vsbcq_m_s, v4si)
+VAR1 (QUADOP_UNONE_UNONE_UNONE_UNONE_PRED, vsbcq_m_u, v4si)
VAR5 (STORE1, vst2q, v16qi, v8hi, v4si, v8hf, v4sf)
VAR5 (LOAD1, vld4q, v16qi, v8hi, v4si, v8hf, v4sf)
VAR5 (LOAD1, vld2q, v16qi, v8hi, v4si, v8hf, v4sf)
diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
index c07487c0750..2d5dbd05940 100644
--- a/gcc/config/arm/mve.md
+++ b/gcc/config/arm/mve.md
@@ -826,7 +826,7 @@ (define_insn "mve_vaddlvq_p_<supf>v4si"
[
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:V4SI 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 2 "vpr_register_operand" "Up")]
VADDLVQ_P))
]
"TARGET_HAVE_MVE"
@@ -3739,7 +3739,7 @@ (define_insn "mve_vaddlvaq_p_<supf>v4si"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VADDLVAQ_P))
]
"TARGET_HAVE_MVE"
@@ -3949,7 +3949,7 @@ (define_insn "mve_vcvtbq_m_f16_f32v8hf"
(set (match_operand:V8HF 0 "s_register_operand" "=w")
(unspec:V8HF [(match_operand:V8HF 1 "s_register_operand" "0")
(match_operand:V4SF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTBQ_M_F16_F32))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3965,7 +3965,7 @@ (define_insn "mve_vcvtbq_m_f32_f16v4sf"
(set (match_operand:V4SF 0 "s_register_operand" "=w")
(unspec:V4SF [(match_operand:V4SF 1 "s_register_operand" "0")
(match_operand:V8HF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTBQ_M_F32_F16))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3981,7 +3981,7 @@ (define_insn "mve_vcvttq_m_f16_f32v8hf"
(set (match_operand:V8HF 0 "s_register_operand" "=w")
(unspec:V8HF [(match_operand:V8HF 1 "s_register_operand" "0")
(match_operand:V4SF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTTQ_M_F16_F32))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -3997,7 +3997,7 @@ (define_insn "mve_vcvttq_m_f32_f16v4sf"
(set (match_operand:V4SF 0 "s_register_operand" "=w")
(unspec:V4SF [(match_operand:V4SF 1 "s_register_operand" "0")
(match_operand:V8HF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VCVTTQ_M_F32_F16))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4595,7 +4595,7 @@ (define_insn "mve_vrev32q_m_fv8hf"
(set (match_operand:V8HF 0 "s_register_operand" "=w")
(unspec:V8HF [(match_operand:V8HF 1 "s_register_operand" "0")
(match_operand:V8HF 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VREV32Q_M_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -4659,7 +4659,7 @@ (define_insn "mve_vrmlaldavhxq_p_sv4si"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRMLALDAVHXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -4691,7 +4691,7 @@ (define_insn "mve_vrmlsldavhq_p_sv4si"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRMLSLDAVHQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -4707,7 +4707,7 @@ (define_insn "mve_vrmlsldavhxq_p_sv4si"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 3 "vpr_register_operand" "Up")]
VRMLSLDAVHXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -4932,7 +4932,7 @@ (define_insn "mve_vrev16q_m_<supf>v16qi"
(set (match_operand:V16QI 0 "s_register_operand" "=w")
(unspec:V16QI [(match_operand:V16QI 1 "s_register_operand" "0")
(match_operand:V16QI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V16BI 3 "vpr_register_operand" "Up")]
VREV16Q_M))
]
"TARGET_HAVE_MVE"
@@ -4964,7 +4964,7 @@ (define_insn "mve_vrmlaldavhq_p_<supf>v4si"
(set (match_operand:DI 0 "s_register_operand" "=r")
(unspec:DI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
- (match_operand:HI 3 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 3 "vpr_register_operand" "Up")]
VRMLALDAVHQ_P))
]
"TARGET_HAVE_MVE"
@@ -6233,7 +6233,7 @@ (define_insn "mve_vrmlaldavhaq_p_sv4si"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRMLALDAVHAQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -6556,7 +6556,7 @@ (define_insn "mve_vrmlaldavhaq_p_uv4si"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRMLALDAVHAQ_P_U))
]
"TARGET_HAVE_MVE"
@@ -6573,7 +6573,7 @@ (define_insn "mve_vrmlaldavhaxq_p_sv4si"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRMLALDAVHAXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -6590,7 +6590,7 @@ (define_insn "mve_vrmlsldavhaq_p_sv4si"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRMLSLDAVHAQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -6607,7 +6607,7 @@ (define_insn "mve_vrmlsldavhaxq_p_sv4si"
(unspec:DI [(match_operand:DI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 4 "vpr_register_operand" "Up")]
VRMLSLDAVHAXQ_P_S))
]
"TARGET_HAVE_MVE"
@@ -7528,7 +7528,7 @@ (define_insn "mve_vldrhq_<supf><mode>"
(define_insn "mve_vldrhq_z_fv8hf"
[(set (match_operand:V8HF 0 "s_register_operand" "=w")
(unspec:V8HF [(match_operand:V8HI 1 "mve_memory_operand" "Ux")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VLDRHQ_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -8303,7 +8303,7 @@ (define_insn "mve_vstrwq_fv4sf"
(define_insn "mve_vstrwq_p_fv4sf"
[(set (match_operand:V4SI 0 "memory_operand" "=Ux")
(unspec:V4SI [(match_operand:V4SF 1 "s_register_operand" "w")
- (match_operand:HI 2 "vpr_register_operand" "Up")]
+ (match_operand:<MVE_VPRED> 2 "vpr_register_operand" "Up")]
VSTRWQ_F))
]
"TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
@@ -9844,7 +9844,7 @@ (define_insn "mve_vadciq_m_<supf>v4si"
(unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 4 "vpr_register_operand" "Up")]
VADCIQ_M))
(set (reg:SI VFPCC_REGNUM)
(unspec:SI [(const_int 0)]
@@ -9880,7 +9880,7 @@ (define_insn "mve_vadcq_m_<supf>v4si"
(unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "0")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 4 "vpr_register_operand" "Up")]
VADCQ_M))
(set (reg:SI VFPCC_REGNUM)
(unspec:SI [(reg:SI VFPCC_REGNUM)]
@@ -9917,7 +9917,7 @@ (define_insn "mve_vsbciq_m_<supf>v4si"
(unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 4 "vpr_register_operand" "Up")]
VSBCIQ_M))
(set (reg:SI VFPCC_REGNUM)
(unspec:SI [(const_int 0)]
@@ -9953,7 +9953,7 @@ (define_insn "mve_vsbcq_m_<supf>v4si"
(unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
(match_operand:V4SI 2 "s_register_operand" "w")
(match_operand:V4SI 3 "s_register_operand" "w")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:V4BI 4 "vpr_register_operand" "Up")]
VSBCQ_M))
(set (reg:SI VFPCC_REGNUM)
(unspec:SI [(reg:SI VFPCC_REGNUM)]
@@ -10457,7 +10457,7 @@ (define_insn "arm_vcx1q<a>_p_v16qi"
(unspec:V16QI [(match_operand:SI 1 "const_int_coproc_operand" "i")
(match_operand:V16QI 2 "register_operand" "0")
(match_operand:SI 3 "const_int_mve_cde1_operand" "i")
- (match_operand:HI 4 "vpr_register_operand" "Up")]
+ (match_operand:V16BI 4 "vpr_register_operand" "Up")]
CDE_VCX))]
"TARGET_CDE && TARGET_HAVE_MVE"
"vpst\;vcx1<a>t\\tp%c1, %q0, #%c3"
@@ -10471,7 +10471,7 @@ (define_insn "arm_vcx2q<a>_p_v16qi"
(match_operand:V16QI 2 "register_operand" "0")
(match_operand:V16QI 3 "register_operand" "t")
(match_operand:SI 4 "const_int_mve_cde2_operand" "i")
- (match_operand:HI 5 "vpr_register_operand" "Up")]
+ (match_operand:V16BI 5 "vpr_register_operand" "Up")]
CDE_VCX))]
"TARGET_CDE && TARGET_HAVE_MVE"
"vpst\;vcx2<a>t\\tp%c1, %q0, %q3, #%c4"
@@ -10486,7 +10486,7 @@ (define_insn "arm_vcx3q<a>_p_v16qi"
(match_operand:V16QI 3 "register_operand" "t")
(match_operand:V16QI 4 "register_operand" "t")
(match_operand:SI 5 "const_int_mve_cde3_operand" "i")
- (match_operand:HI 6 "vpr_register_operand" "Up")]
+ (match_operand:V16BI 6 "vpr_register_operand" "Up")]
CDE_VCX))]
"TARGET_CDE && TARGET_HAVE_MVE"
"vpst\;vcx3<a>t\\tp%c1, %q0, %q3, %q4, #%c5"
diff --git a/gcc/testsuite/gcc.target/arm/acle/cde-mve-full-assembly.c b/gcc/testsuite/gcc.target/arm/acle/cde-mve-full-assembly.c
index 501cc84da10..77ea2866ad2 100644
--- a/gcc/testsuite/gcc.target/arm/acle/cde-mve-full-assembly.c
+++ b/gcc/testsuite/gcc.target/arm/acle/cde-mve-full-assembly.c
@@ -567,80 +567,80 @@
contain back references). */
/*
** test_cde_vcx1q_mfloat16x8_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_mfloat32x4_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_muint8x16_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_muint16x8_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_muint32x4_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_muint64x2_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_mint8x16_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_mint16x8_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_mint32x4_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1q_mint64x2_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1t p0, q0, #32
** bx lr
@@ -649,80 +649,80 @@
/*
** test_cde_vcx1qa_mfloat16x8_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_mfloat32x4_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_muint8x16_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_muint16x8_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_muint32x4_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_muint64x2_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_mint8x16_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_mint16x8_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_mint32x4_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
*/
/*
** test_cde_vcx1qa_mint64x2_tintint:
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
-** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2 @ movhi)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
+** (?:vldr\.64 d0, \.L[0-9]*\n\tvldr\.64 d1, \.L[0-9]*\+8|vmsr P0, r2)
** vpst
** vcx1at p0, q0, #32
** bx lr
@@ -731,8 +731,8 @@
/*
** test_cde_vcx2q_mfloat16x8_tuint16x8_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -740,8 +740,8 @@
*/
/*
** test_cde_vcx2q_mfloat16x8_tfloat32x4_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -749,8 +749,8 @@
*/
/*
** test_cde_vcx2q_mfloat32x4_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -758,8 +758,8 @@
*/
/*
** test_cde_vcx2q_mint64x2_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -767,8 +767,8 @@
*/
/*
** test_cde_vcx2q_mint8x16_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -776,8 +776,8 @@
*/
/*
** test_cde_vcx2q_muint16x8_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -785,8 +785,8 @@
*/
/*
** test_cde_vcx2q_muint8x16_tint64x2_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -794,8 +794,8 @@
*/
/*
** test_cde_vcx2q_muint8x16_tint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -803,8 +803,8 @@
*/
/*
** test_cde_vcx2q_muint8x16_tuint16x8_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -812,8 +812,8 @@
*/
/*
** test_cde_vcx2q_muint8x16_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2t p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -823,8 +823,8 @@
/*
** test_cde_vcx2qa_mfloat16x8_tuint16x8_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -832,8 +832,8 @@
*/
/*
** test_cde_vcx2qa_mfloat16x8_tfloat32x4_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -841,8 +841,8 @@
*/
/*
** test_cde_vcx2qa_mfloat32x4_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -850,8 +850,8 @@
*/
/*
** test_cde_vcx2qa_mint64x2_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -859,8 +859,8 @@
*/
/*
** test_cde_vcx2qa_mint8x16_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -868,8 +868,8 @@
*/
/*
** test_cde_vcx2qa_muint16x8_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -877,8 +877,8 @@
*/
/*
** test_cde_vcx2qa_muint8x16_tint64x2_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -886,8 +886,8 @@
*/
/*
** test_cde_vcx2qa_muint8x16_tint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -895,8 +895,8 @@
*/
/*
** test_cde_vcx2qa_muint8x16_tuint16x8_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -904,8 +904,8 @@
*/
/*
** test_cde_vcx2qa_muint8x16_tuint8x16_tint:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r1)
** vpst
** vcx2at p0, (q[0-7]), q0, #32
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -915,8 +915,8 @@
/*
** test_cde_vcx3q_muint8x16_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -924,8 +924,8 @@
*/
/*
** test_cde_vcx3q_mfloat16x8_tfloat16x8_tfloat16x8_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -933,8 +933,8 @@
*/
/*
** test_cde_vcx3q_mfloat32x4_tuint64x2_tfloat16x8_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -942,8 +942,8 @@
*/
/*
** test_cde_vcx3q_muint16x8_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -951,8 +951,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tuint16x8_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -960,8 +960,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tuint8x16_tuint16x8_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -969,8 +969,8 @@
*/
/*
** test_cde_vcx3q_mint8x16_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -978,8 +978,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -987,8 +987,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tuint8x16_tint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -996,8 +996,8 @@
*/
/*
** test_cde_vcx3q_mint64x2_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1005,8 +1005,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tint64x2_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1014,8 +1014,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tuint8x16_tint64x2_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1023,8 +1023,8 @@
*/
/*
** test_cde_vcx3q_muint8x16_tint64x2_tint64x2_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3t p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1034,8 +1034,8 @@
/*
** test_cde_vcx3qa_muint8x16_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1043,8 +1043,8 @@
*/
/*
** test_cde_vcx3qa_mfloat16x8_tfloat16x8_tfloat16x8_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1052,8 +1052,8 @@
*/
/*
** test_cde_vcx3qa_mfloat32x4_tuint64x2_tfloat16x8_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1061,8 +1061,8 @@
*/
/*
** test_cde_vcx3qa_muint16x8_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1070,8 +1070,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tuint16x8_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1079,8 +1079,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tuint8x16_tuint16x8_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1088,8 +1088,8 @@
*/
/*
** test_cde_vcx3qa_mint8x16_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1097,8 +1097,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1106,8 +1106,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tuint8x16_tint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1115,8 +1115,8 @@
*/
/*
** test_cde_vcx3qa_mint64x2_tuint8x16_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1124,8 +1124,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tint64x2_tuint8x16_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1133,8 +1133,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tuint8x16_tint64x2_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
@@ -1142,8 +1142,8 @@
*/
/*
** test_cde_vcx3qa_muint8x16_tint64x2_tint64x2_t:
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
-** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0 @ movhi)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
+** (?:vldr\.64 d(?:[02468]|1[024]), \.L[0-9]*\n\tvldr\.64 d(?:[13579]|1[135]), \.L[0-9]*\+8|vmsr P0, r0)
** vpst
** vcx3at p0, (q[0-7]), q0, q1, #15
** vmov q0, \1([[:space:]]+@ [^\n]*)?
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 14/14] arm: Add VPR_REG to ALL_REGS
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (12 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 13/14] arm: Convert more MVE/CDE " Christophe Lyon
@ 2021-10-13 10:15 ` Christophe Lyon
2021-10-15 15:10 ` [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Richard Sandiford
14 siblings, 0 replies; 21+ messages in thread
From: Christophe Lyon @ 2021-10-13 10:15 UTC (permalink / raw)
To: gcc-patches
VPR_REG should be part of ALL_REGS, this patch fixes this omission.
2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
gcc/
* config/arm/arm.h (REG_CLASS_CONTENTS): Add VPR_REG to ALL_REGS.
diff --git a/gcc/config/arm/arm.h b/gcc/config/arm/arm.h
index eae1b1cd0fb..fab39d05916 100644
--- a/gcc/config/arm/arm.h
+++ b/gcc/config/arm/arm.h
@@ -1346,7 +1346,7 @@ enum reg_class
{ 0x00000000, 0x00000000, 0x00000000, 0x00000080 }, /* AFP_REG */ \
{ 0x00000000, 0x00000000, 0x00000000, 0x00000400 }, /* VPR_REG. */ \
{ 0x00005FFF, 0x00000000, 0x00000000, 0x00000400 }, /* GENERAL_AND_VPR_REGS. */ \
- { 0xFFFF7FFF, 0xFFFFFFFF, 0xFFFFFFFF, 0x0000000F } /* ALL_REGS. */ \
+ { 0xFFFF7FFF, 0xFFFFFFFF, 0xFFFFFFFF, 0x0000040F } /* ALL_REGS. */ \
}
#define FP_SYSREGS \
--
2.25.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 03/14] arm: Add tests for PR target/101325
2021-10-13 10:15 ` [PATCH v2 03/14] arm: Add tests for PR target/101325 Christophe Lyon
@ 2021-10-15 14:49 ` Richard Sandiford
0 siblings, 0 replies; 21+ messages in thread
From: Richard Sandiford @ 2021-10-15 14:49 UTC (permalink / raw)
To: Christophe Lyon via Gcc-patches
Christophe Lyon via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
> These tests are derived from the one provided in the PR: there is a
> compile-only test because I did not have access to anything that could
> execute MVE code until recently.
> I have been able to add an executable test since QEMU supports MVE.
>
> Instead of adding arm_v8_1m_mve_hw, I update arm_mve_hw so that it
> uses add_options_for_arm_v8_1m_mve_fp, like arm_neon_hw does. This
> ensures arm_mve_hw passes even if the toolchain does not generate MVE
> code by default.
>
> 2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
>
> gcc/testsuite/
> PR target/101325
> * gcc.target/arm/simd/pr101325.c: New.
> * gcc.target/arm/simd/pr101325-2.c: New.
> * lib/target-supports.exp (check_effective_target_arm_mve_hw): Use
> add_options_for_arm_v8_1m_mve_fp.
>
> add executable test and update check_effective_target_arm_mve_hw
>
> diff --git a/gcc/testsuite/gcc.target/arm/simd/pr101325-2.c b/gcc/testsuite/gcc.target/arm/simd/pr101325-2.c
> new file mode 100644
> index 00000000000..7907a386385
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/arm/simd/pr101325-2.c
> @@ -0,0 +1,19 @@
> +/* { dg-do run } */
> +/* { dg-require-effective-target arm_mve_hw } */
> +/* { dg-options "-O3" } */
> +/* { dg-add-options arm_v8_1m_mve } */
> +
> +#include <arm_mve.h>
> +
> +
> +__attribute((noinline,noipa))
Very minor, but: noinline is redundant with noipa.
Richard
> +unsigned foo(int8x16_t v, int8x16_t w)
> +{
> + return vcmpeqq (v, w);
> +}
> +
> +int main(void)
> +{
> + if (foo (vdupq_n_s8(0), vdupq_n_s8(0)) != 0xffffU)
> + __builtin_abort ();
> +}
> diff --git a/gcc/testsuite/gcc.target/arm/simd/pr101325.c b/gcc/testsuite/gcc.target/arm/simd/pr101325.c
> new file mode 100644
> index 00000000000..a466683a0b1
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/arm/simd/pr101325.c
> @@ -0,0 +1,14 @@
> +/* { dg-do compile } */
> +/* { dg-require-effective-target arm_v8_1m_mve_ok } */
> +/* { dg-add-options arm_v8_1m_mve } */
> +/* { dg-additional-options "-O3" } */
> +
> +#include <arm_mve.h>
> +
> +unsigned foo(int8x16_t v, int8x16_t w)
> +{
> + return vcmpeqq (v, w);
> +}
> +/* { dg-final { scan-assembler {\tvcmp.i8 eq} } } */
> +/* { dg-final { scan-assembler {\tvmrs\t r[0-9]+, P0} } } */
> +/* { dg-final { scan-assembler {\tuxth} } } */
> diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp
> index e030e4f376b..b0e35b602af 100644
> --- a/gcc/testsuite/lib/target-supports.exp
> +++ b/gcc/testsuite/lib/target-supports.exp
> @@ -4889,6 +4889,7 @@ proc check_effective_target_arm_cmse_hw { } {
> }
> } "-mcmse -Wl,--section-start,.gnu.sgstubs=0x00400000"]
> }
> +
> # Return 1 if the target supports executing MVE instructions, 0
> # otherwise.
>
> @@ -4904,7 +4905,7 @@ proc check_effective_target_arm_mve_hw {} {
> : "0" (a), "r" (b));
> return (a != 2);
> }
> - } ""]
> + } [add_options_for_arm_v8_1m_mve_fp ""]]
> }
>
> # Return 1 if this is an ARM target where ARMv8-M Security Extensions with
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 09/14] arm: Fix vcond_mask expander for MVE (PR target/100757)
2021-10-13 10:15 ` [PATCH v2 09/14] arm: Fix vcond_mask expander for MVE (PR target/100757) Christophe Lyon
@ 2021-10-15 14:55 ` Richard Sandiford
0 siblings, 0 replies; 21+ messages in thread
From: Richard Sandiford @ 2021-10-15 14:55 UTC (permalink / raw)
To: Christophe Lyon via Gcc-patches
Christophe Lyon via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
> @@ -31086,36 +31087,20 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
> case NE:
> if (TARGET_HAVE_MVE)
> {
> - rtx vpr_p0;
> - if (vcond_mve)
> - vpr_p0 = target;
> - else
> - vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
> -
> switch (GET_MODE_CLASS (cmp_mode))
> {
> case MODE_VECTOR_INT:
> - emit_insn (gen_mve_vcmpq (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
> + emit_insn (gen_mve_vcmpq (code, cmp_mode, target, op0, force_reg (cmp_mode, op1)));
Pre-existing nit: long line. Same for later calls in the same function.
Richard
> break;
> case MODE_VECTOR_FLOAT:
> if (TARGET_HAVE_MVE_FLOAT)
> - emit_insn (gen_mve_vcmpq_f (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
> + emit_insn (gen_mve_vcmpq_f (code, cmp_mode, target, op0, force_reg (cmp_mode, op1)));
> else
> gcc_unreachable ();
> break;
> default:
> gcc_unreachable ();
> }
> -
> - /* If we are not expanding a vcond, build the result here. */
> - if (!vcond_mve)
> - {
> - rtx zero = gen_reg_rtx (cmp_result_mode);
> - rtx one = gen_reg_rtx (cmp_result_mode);
> - emit_move_insn (zero, CONST0_RTX (cmp_result_mode));
> - emit_move_insn (one, CONST1_RTX (cmp_result_mode));
> - emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, target, one, zero, vpr_p0));
> - }
> }
> else
> emit_insn (gen_neon_vc (code, cmp_mode, target, op0, op1));
> @@ -31127,23 +31112,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
> case GEU:
> case GTU:
> if (TARGET_HAVE_MVE)
> - {
> - rtx vpr_p0;
> - if (vcond_mve)
> - vpr_p0 = target;
> - else
> - vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
> -
> - emit_insn (gen_mve_vcmpq (code, cmp_mode, vpr_p0, op0, force_reg (cmp_mode, op1)));
> - if (!vcond_mve)
> - {
> - rtx zero = gen_reg_rtx (cmp_result_mode);
> - rtx one = gen_reg_rtx (cmp_result_mode);
> - emit_move_insn (zero, CONST0_RTX (cmp_result_mode));
> - emit_move_insn (one, CONST1_RTX (cmp_result_mode));
> - emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, target, one, zero, vpr_p0));
> - }
> - }
> + emit_insn (gen_mve_vcmpq (code, cmp_mode, target, op0, force_reg (cmp_mode, op1)));
> else
> emit_insn (gen_neon_vc (code, cmp_mode, target,
> op0, force_reg (cmp_mode, op1)));
> @@ -31154,23 +31123,7 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
> case LEU:
> case LTU:
> if (TARGET_HAVE_MVE)
> - {
> - rtx vpr_p0;
> - if (vcond_mve)
> - vpr_p0 = target;
> - else
> - vpr_p0 = gen_reg_rtx (arm_mode_to_pred_mode (cmp_mode));
> -
> - emit_insn (gen_mve_vcmpq (swap_condition (code), cmp_mode, vpr_p0, force_reg (cmp_mode, op1), op0));
> - if (!vcond_mve)
> - {
> - rtx zero = gen_reg_rtx (cmp_result_mode);
> - rtx one = gen_reg_rtx (cmp_result_mode);
> - emit_move_insn (zero, CONST0_RTX (cmp_result_mode));
> - emit_move_insn (one, CONST1_RTX (cmp_result_mode));
> - emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, target, one, zero, vpr_p0));
> - }
> - }
> + emit_insn (gen_mve_vcmpq (swap_condition (code), cmp_mode, target, force_reg (cmp_mode, op1), op0));
> else
> emit_insn (gen_neon_vc (swap_condition (code), cmp_mode,
> target, force_reg (cmp_mode, op1), op0));
> @@ -31185,8 +31138,8 @@ arm_expand_vector_compare (rtx target, rtx_code code, rtx op0, rtx op1,
> rtx gt_res = gen_reg_rtx (cmp_result_mode);
> rtx alt_res = gen_reg_rtx (cmp_result_mode);
> rtx_code alt_code = (code == LTGT ? LT : LE);
> - if (arm_expand_vector_compare (gt_res, GT, op0, op1, true, vcond_mve)
> - || arm_expand_vector_compare (alt_res, alt_code, op0, op1, true, vcond_mve))
> + if (arm_expand_vector_compare (gt_res, GT, op0, op1, true)
> + || arm_expand_vector_compare (alt_res, alt_code, op0, op1, true))
> gcc_unreachable ();
> emit_insn (gen_rtx_SET (target, gen_rtx_IOR (cmp_result_mode,
> gt_res, alt_res)));
> @@ -31206,19 +31159,15 @@ arm_expand_vcond (rtx *operands, machine_mode cmp_result_mode)
> {
> /* When expanding for MVE, we do not want to emit a (useless) vpsel in
> arm_expand_vector_compare, and another one here. */
> - bool vcond_mve=false;
> rtx mask;
>
> if (TARGET_HAVE_MVE)
> - {
> - vcond_mve=true;
> - mask = gen_reg_rtx (arm_mode_to_pred_mode (cmp_result_mode));
> - }
> + mask = gen_reg_rtx (arm_mode_to_pred_mode (cmp_result_mode).require ());
> else
> mask = gen_reg_rtx (cmp_result_mode);
>
> bool inverted = arm_expand_vector_compare (mask, GET_CODE (operands[3]),
> - operands[4], operands[5], true, vcond_mve);
> + operands[4], operands[5], true);
> if (inverted)
> std::swap (operands[1], operands[2]);
> if (TARGET_NEON)
> @@ -31226,20 +31175,20 @@ arm_expand_vcond (rtx *operands, machine_mode cmp_result_mode)
> mask, operands[1], operands[2]));
> else
> {
> - machine_mode cmp_mode = GET_MODE (operands[4]);
> - rtx vpr_p0 = mask;
> - rtx zero = gen_reg_rtx (cmp_mode);
> - rtx one = gen_reg_rtx (cmp_mode);
> - emit_move_insn (zero, CONST0_RTX (cmp_mode));
> - emit_move_insn (one, CONST1_RTX (cmp_mode));
> + machine_mode cmp_mode = GET_MODE (operands[0]);
> +
> switch (GET_MODE_CLASS (cmp_mode))
> {
> case MODE_VECTOR_INT:
> - emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_result_mode, operands[0], one, zero, vpr_p0));
> + emit_insn (gen_mve_vpselq (VPSELQ_S, cmp_mode, operands[0],
> + operands[1], operands[2], mask));
> break;
> case MODE_VECTOR_FLOAT:
> if (TARGET_HAVE_MVE_FLOAT)
> - emit_insn (gen_mve_vpselq_f (cmp_mode, operands[0], one, zero, vpr_p0));
> + emit_insn (gen_mve_vpselq_f (cmp_mode, operands[0],
> + operands[1], operands[2], mask));
> + else
> + gcc_unreachable ();
> break;
> default:
> gcc_unreachable ();
> @@ -34149,4 +34098,15 @@ arm_mode_base_reg_class (machine_mode mode)
>
> struct gcc_target targetm = TARGET_INITIALIZER;
>
> +/* Implement TARGET_VECTORIZE_GET_MASK_MODE. */
> +
> +opt_machine_mode
> +arm_get_mask_mode (machine_mode mode)
> +{
> + if (TARGET_HAVE_MVE)
> + return arm_mode_to_pred_mode (mode);
> +
> + return default_get_mask_mode (mode);
> +}
> +
> #include "gt-arm.h"
> diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
> index 9da78657798..fb25cac1cfd 100644
> --- a/gcc/config/arm/mve.md
> +++ b/gcc/config/arm/mve.md
> @@ -10540,3 +10540,57 @@ (define_insn "*mve_mov<mode>"
> vmsr%?\t P0, %1
> vmrs%?\t %0, P0"
> )
> +
> +;; Expanders for vec_cmp and vcond
> +
> +(define_expand "vec_cmp<mode><MVE_vpred>"
> + [(set (match_operand:<MVE_VPRED> 0 "s_register_operand")
> + (match_operator:<MVE_VPRED> 1 "comparison_operator"
> + [(match_operand:MVE_VLD_ST 2 "s_register_operand")
> + (match_operand:MVE_VLD_ST 3 "reg_or_zero_operand")]))]
> + "TARGET_HAVE_MVE
> + && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
> +{
> + arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
> + operands[2], operands[3], false);
> + DONE;
> +})
> +
> +(define_expand "vec_cmpu<mode><MVE_vpred>"
> + [(set (match_operand:<MVE_VPRED> 0 "s_register_operand")
> + (match_operator:<MVE_VPRED> 1 "comparison_operator"
> + [(match_operand:MVE_2 2 "s_register_operand")
> + (match_operand:MVE_2 3 "reg_or_zero_operand")]))]
> + "TARGET_HAVE_MVE"
> +{
> + arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
> + operands[2], operands[3], false);
> + DONE;
> +})
> +
> +(define_expand "vcond_mask_<mode><MVE_vpred>"
> + [(set (match_operand:MVE_VLD_ST 0 "s_register_operand")
> + (if_then_else:MVE_VLD_ST
> + (match_operand:<MVE_VPRED> 3 "s_register_operand")
> + (match_operand:MVE_VLD_ST 1 "s_register_operand")
> + (match_operand:MVE_VLD_ST 2 "s_register_operand")))]
> + "TARGET_HAVE_MVE"
> +{
> + switch (GET_MODE_CLASS (<MODE>mode))
> + {
> + case MODE_VECTOR_INT:
> + emit_insn (gen_mve_vpselq (VPSELQ_S, <MODE>mode, operands[0],
> + operands[1], operands[2], operands[3]));
> + break;
> + case MODE_VECTOR_FLOAT:
> + if (TARGET_HAVE_MVE_FLOAT)
> + emit_insn (gen_mve_vpselq_f (<MODE>mode, operands[0],
> + operands[1], operands[2], operands[3]));
> + else
> + gcc_unreachable ();
> + break;
> + default:
> + gcc_unreachable ();
> + }
> + DONE;
> +})
> diff --git a/gcc/config/arm/neon.md b/gcc/config/arm/neon.md
> index 8b0a396947c..28310d93a4e 100644
> --- a/gcc/config/arm/neon.md
> +++ b/gcc/config/arm/neon.md
> @@ -1394,6 +1394,45 @@ (define_insn "*us_sub<mode>_neon"
> [(set_attr "type" "neon_qsub<q>")]
> )
>
> +(define_expand "vec_cmp<mode><v_cmp_result>"
> + [(set (match_operand:<V_cmp_result> 0 "s_register_operand")
> + (match_operator:<V_cmp_result> 1 "comparison_operator"
> + [(match_operand:VDQWH 2 "s_register_operand")
> + (match_operand:VDQWH 3 "reg_or_zero_operand")]))]
> + "TARGET_NEON
> + && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
> +{
> + arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
> + operands[2], operands[3], false);
> + DONE;
> +})
> +
> +(define_expand "vec_cmpu<mode><mode>"
> + [(set (match_operand:VDQIW 0 "s_register_operand")
> + (match_operator:VDQIW 1 "comparison_operator"
> + [(match_operand:VDQIW 2 "s_register_operand")
> + (match_operand:VDQIW 3 "reg_or_zero_operand")]))]
> + "TARGET_NEON"
> +{
> + arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
> + operands[2], operands[3], false);
> + DONE;
> +})
> +
> +(define_expand "vcond_mask_<mode><v_cmp_result>"
> + [(set (match_operand:VDQWH 0 "s_register_operand")
> + (if_then_else:VDQWH
> + (match_operand:<V_cmp_result> 3 "s_register_operand")
> + (match_operand:VDQWH 1 "s_register_operand")
> + (match_operand:VDQWH 2 "s_register_operand")))]
> + "TARGET_NEON
> + && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
> +{
> + emit_insn (gen_neon_vbsl<mode> (operands[0], operands[3], operands[1],
> + operands[2]));
> + DONE;
> +})
> +
> ;; Patterns for builtins.
>
> ; good for plain vadd, vaddq.
> diff --git a/gcc/config/arm/vec-common.md b/gcc/config/arm/vec-common.md
> index 68de4f0f943..9b461a76155 100644
> --- a/gcc/config/arm/vec-common.md
> +++ b/gcc/config/arm/vec-common.md
> @@ -363,33 +363,6 @@ (define_expand "vlshr<mode>3"
> }
> })
>
> -(define_expand "vec_cmp<mode><v_cmp_result>"
> - [(set (match_operand:<V_cmp_result> 0 "s_register_operand")
> - (match_operator:<V_cmp_result> 1 "comparison_operator"
> - [(match_operand:VDQWH 2 "s_register_operand")
> - (match_operand:VDQWH 3 "reg_or_zero_operand")]))]
> - "ARM_HAVE_<MODE>_ARITH
> - && !TARGET_REALLY_IWMMXT
> - && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
> -{
> - arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
> - operands[2], operands[3], false, false);
> - DONE;
> -})
> -
> -(define_expand "vec_cmpu<mode><mode>"
> - [(set (match_operand:VDQIW 0 "s_register_operand")
> - (match_operator:VDQIW 1 "comparison_operator"
> - [(match_operand:VDQIW 2 "s_register_operand")
> - (match_operand:VDQIW 3 "reg_or_zero_operand")]))]
> - "ARM_HAVE_<MODE>_ARITH
> - && !TARGET_REALLY_IWMMXT"
> -{
> - arm_expand_vector_compare (operands[0], GET_CODE (operands[1]),
> - operands[2], operands[3], false, false);
> - DONE;
> -})
> -
> ;; Conditional instructions. These are comparisons with conditional moves for
> ;; vectors. They perform the assignment:
> ;;
> @@ -461,31 +434,6 @@ (define_expand "vcondu<mode><v_cmp_result>"
> DONE;
> })
>
> -(define_expand "vcond_mask_<mode><v_cmp_result>"
> - [(set (match_operand:VDQWH 0 "s_register_operand")
> - (if_then_else:VDQWH
> - (match_operand:<V_cmp_result> 3 "s_register_operand")
> - (match_operand:VDQWH 1 "s_register_operand")
> - (match_operand:VDQWH 2 "s_register_operand")))]
> - "ARM_HAVE_<MODE>_ARITH
> - && !TARGET_REALLY_IWMMXT
> - && (!<Is_float_mode> || flag_unsafe_math_optimizations)"
> -{
> - if (TARGET_NEON)
> - {
> - emit_insn (gen_neon_vbsl (<MODE>mode, operands[0], operands[3],
> - operands[1], operands[2]));
> - }
> - else if (TARGET_HAVE_MVE)
> - {
> - emit_insn (gen_mve_vpselq (VPSELQ_S, <MODE>mode, operands[0],
> - operands[1], operands[2], operands[3]));
> - }
> - else
> - gcc_unreachable ();
> - DONE;
> -})
> -
> (define_expand "vec_load_lanesoi<mode>"
> [(set (match_operand:OI 0 "s_register_operand")
> (unspec:OI [(match_operand:OI 1 "neon_struct_operand")
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 12/14] arm: Convert more load/store MVE builtins to predicate qualifiers
2021-10-13 10:15 ` [PATCH v2 12/14] arm: Convert more load/store " Christophe Lyon
@ 2021-10-15 15:08 ` Richard Sandiford
2021-10-15 15:33 ` Christophe LYON
0 siblings, 1 reply; 21+ messages in thread
From: Richard Sandiford @ 2021-10-15 15:08 UTC (permalink / raw)
To: Christophe Lyon via Gcc-patches
Christophe Lyon via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
> This patch covers a few builtins where we do not use the <mode>
> iterator and thus we cannot use <MVE_vpred>.
>
> For v2di instructions, we use the V8BI mode for predicates.
Why V8BI though, when VPRED uses HI?
Would it make sense to define a V2BI? Or doesn't that work?
Thanks,
Richard
>
> 2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
>
> gcc/
> PR target/100757
> PR target/101325
> * config/arm/arm-builtins.c (STRSBS_P_QUALIFIERS): Use predicate
> qualifier.
> (STRSBU_P_QUALIFIERS): Likewise.
> (LDRGBS_Z_QUALIFIERS): Likewise.
> (LDRGBU_Z_QUALIFIERS): Likewise.
> (LDRGBWBXU_Z_QUALIFIERS): Likewise.
> (LDRGBWBS_Z_QUALIFIERS): Likewise.
> (LDRGBWBU_Z_QUALIFIERS): Likewise.
> (STRSBWBS_P_QUALIFIERS): Likewise.
> (STRSBWBU_P_QUALIFIERS): Likewise.
> * config/arm/mve.md: Use VxBI instead of HI.
>
> diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
> index 06ff9d2278a..e58580bb828 100644
> --- a/gcc/config/arm/arm-builtins.c
> +++ b/gcc/config/arm/arm-builtins.c
> @@ -738,13 +738,13 @@ arm_strss_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> static enum arm_type_qualifiers
> arm_strsbs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_void, qualifier_unsigned, qualifier_immediate,
> - qualifier_none, qualifier_unsigned};
> + qualifier_none, qualifier_predicate};
> #define STRSBS_P_QUALIFIERS (arm_strsbs_p_qualifiers)
>
> static enum arm_type_qualifiers
> arm_strsbu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_void, qualifier_unsigned, qualifier_immediate,
> - qualifier_unsigned, qualifier_unsigned};
> + qualifier_unsigned, qualifier_predicate};
> #define STRSBU_P_QUALIFIERS (arm_strsbu_p_qualifiers)
>
> static enum arm_type_qualifiers
> @@ -780,13 +780,13 @@ arm_ldrgbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> static enum arm_type_qualifiers
> arm_ldrgbs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_none, qualifier_unsigned, qualifier_immediate,
> - qualifier_unsigned};
> + qualifier_predicate};
> #define LDRGBS_Z_QUALIFIERS (arm_ldrgbs_z_qualifiers)
>
> static enum arm_type_qualifiers
> arm_ldrgbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
> - qualifier_unsigned};
> + qualifier_predicate};
> #define LDRGBU_Z_QUALIFIERS (arm_ldrgbu_z_qualifiers)
>
> static enum arm_type_qualifiers
> @@ -826,7 +826,7 @@ arm_ldrgbwbxu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> static enum arm_type_qualifiers
> arm_ldrgbwbxu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
> - qualifier_unsigned};
> + qualifier_predicate};
> #define LDRGBWBXU_Z_QUALIFIERS (arm_ldrgbwbxu_z_qualifiers)
>
> static enum arm_type_qualifiers
> @@ -842,13 +842,13 @@ arm_ldrgbwbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> static enum arm_type_qualifiers
> arm_ldrgbwbs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_none, qualifier_unsigned, qualifier_immediate,
> - qualifier_unsigned};
> + qualifier_predicate};
> #define LDRGBWBS_Z_QUALIFIERS (arm_ldrgbwbs_z_qualifiers)
>
> static enum arm_type_qualifiers
> arm_ldrgbwbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
> - qualifier_unsigned};
> + qualifier_predicate};
> #define LDRGBWBU_Z_QUALIFIERS (arm_ldrgbwbu_z_qualifiers)
>
> static enum arm_type_qualifiers
> @@ -864,13 +864,13 @@ arm_strsbwbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> static enum arm_type_qualifiers
> arm_strsbwbs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_unsigned, qualifier_unsigned, qualifier_const,
> - qualifier_none, qualifier_unsigned};
> + qualifier_none, qualifier_predicate};
> #define STRSBWBS_P_QUALIFIERS (arm_strsbwbs_p_qualifiers)
>
> static enum arm_type_qualifiers
> arm_strsbwbu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
> = { qualifier_unsigned, qualifier_unsigned, qualifier_const,
> - qualifier_unsigned, qualifier_unsigned};
> + qualifier_unsigned, qualifier_predicate};
> #define STRSBWBU_P_QUALIFIERS (arm_strsbwbu_p_qualifiers)
>
> static enum arm_type_qualifiers
> diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
> index 81ad488155d..c07487c0750 100644
> --- a/gcc/config/arm/mve.md
> +++ b/gcc/config/arm/mve.md
> @@ -7282,7 +7282,7 @@ (define_insn "mve_vstrwq_scatter_base_p_<supf>v4si"
> [(match_operand:V4SI 0 "s_register_operand" "w")
> (match_operand:SI 1 "immediate_operand" "i")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VSTRWSBQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -7371,7 +7371,7 @@ (define_insn "mve_vldrwq_gather_base_z_<supf>v4si"
> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
> (unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
> (match_operand:SI 2 "immediate_operand" "i")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VLDRWGBQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -7609,7 +7609,7 @@ (define_insn "mve_vldrwq_<supf>v4si"
> (define_insn "mve_vldrwq_z_fv4sf"
> [(set (match_operand:V4SF 0 "s_register_operand" "=w")
> (unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Ux")
> - (match_operand:HI 2 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 2 "vpr_register_operand" "Up")]
> VLDRWQ_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -7629,7 +7629,7 @@ (define_insn "mve_vldrwq_z_fv4sf"
> (define_insn "mve_vldrwq_z_<supf>v4si"
> [(set (match_operand:V4SI 0 "s_register_operand" "=w")
> (unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Ux")
> - (match_operand:HI 2 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 2 "vpr_register_operand" "Up")]
> VLDRWQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -7690,7 +7690,7 @@ (define_insn "mve_vldrdq_gather_base_z_<supf>v2di"
> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
> (unspec:V2DI [(match_operand:V2DI 1 "s_register_operand" "w")
> (match_operand:SI 2 "immediate_operand" "i")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VLDRDGBQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -7731,7 +7731,7 @@ (define_insn "mve_vldrdq_gather_offset_z_<supf>v2di"
> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
> (unspec:V2DI [(match_operand:V2DI 1 "memory_operand" "Us")
> (match_operand:V2DI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VLDRDGOQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -7772,7 +7772,7 @@ (define_insn "mve_vldrdq_gather_shifted_offset_z_<supf>v2di"
> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
> (unspec:V2DI [(match_operand:V2DI 1 "memory_operand" "Us")
> (match_operand:V2DI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VLDRDGSOQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -7813,7 +7813,7 @@ (define_insn "mve_vldrhq_gather_offset_z_fv8hf"
> [(set (match_operand:V8HF 0 "s_register_operand" "=&w")
> (unspec:V8HF [(match_operand:V8HI 1 "memory_operand" "Us")
> (match_operand:V8HI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VLDRHQGO_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -7855,7 +7855,7 @@ (define_insn "mve_vldrhq_gather_shifted_offset_z_fv8hf"
> [(set (match_operand:V8HF 0 "s_register_operand" "=&w")
> (unspec:V8HF [(match_operand:V8HI 1 "memory_operand" "Us")
> (match_operand:V8HI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VLDRHQGSO_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -7897,7 +7897,7 @@ (define_insn "mve_vldrwq_gather_base_z_fv4sf"
> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
> (unspec:V4SF [(match_operand:V4SI 1 "s_register_operand" "w")
> (match_operand:SI 2 "immediate_operand" "i")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VLDRWQGB_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -7958,7 +7958,7 @@ (define_insn "mve_vldrwq_gather_offset_z_fv4sf"
> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
> (unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Us")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VLDRWQGO_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -7980,7 +7980,7 @@ (define_insn "mve_vldrwq_gather_offset_z_<supf>v4si"
> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
> (unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Us")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VLDRWGOQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -8042,7 +8042,7 @@ (define_insn "mve_vldrwq_gather_shifted_offset_z_fv4sf"
> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
> (unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Us")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VLDRWQGSO_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -8064,7 +8064,7 @@ (define_insn "mve_vldrwq_gather_shifted_offset_z_<supf>v4si"
> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
> (unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Us")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VLDRWGSOQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -8104,7 +8104,7 @@ (define_insn "mve_vstrhq_fv8hf"
> (define_insn "mve_vstrhq_p_fv8hf"
> [(set (match_operand:V8HI 0 "mve_memory_operand" "=Ux")
> (unspec:V8HI [(match_operand:V8HF 1 "s_register_operand" "w")
> - (match_operand:HI 2 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 2 "vpr_register_operand" "Up")]
> VSTRHQ_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -8323,7 +8323,7 @@ (define_insn "mve_vstrwq_p_fv4sf"
> (define_insn "mve_vstrwq_p_<supf>v4si"
> [(set (match_operand:V4SI 0 "memory_operand" "=Ux")
> (unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
> - (match_operand:HI 2 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 2 "vpr_register_operand" "Up")]
> VSTRWQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -8385,7 +8385,7 @@ (define_insn "mve_vstrdq_scatter_base_p_<supf>v2di"
> [(match_operand:V2DI 0 "s_register_operand" "w")
> (match_operand:SI 1 "mve_vldrd_immediate" "Ri")
> (match_operand:V2DI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VSTRDSBQ))
> ]
> "TARGET_HAVE_MVE"
> @@ -8428,7 +8428,7 @@ (define_expand "mve_vstrdq_scatter_offset_p_<supf>v2di"
> [(match_operand:V2DI 0 "mve_scatter_memory")
> (match_operand:V2DI 1 "s_register_operand")
> (match_operand:V2DI 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V8BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRDSOQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -8446,7 +8446,7 @@ (define_insn "mve_vstrdq_scatter_offset_p_<supf>v2di_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V2DI 1 "s_register_operand" "w")
> (match_operand:V2DI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VSTRDSOQ))]
> "TARGET_HAVE_MVE"
> "vpst\;vstrdt.64\t%q2, [%0, %q1]"
> @@ -8487,7 +8487,7 @@ (define_expand "mve_vstrdq_scatter_shifted_offset_p_<supf>v2di"
> [(match_operand:V2DI 0 "mve_scatter_memory")
> (match_operand:V2DI 1 "s_register_operand")
> (match_operand:V2DI 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V8BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRDSSOQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -8506,7 +8506,7 @@ (define_insn "mve_vstrdq_scatter_shifted_offset_p_<supf>v2di_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V2DI 1 "s_register_operand" "w")
> (match_operand:V2DI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VSTRDSSOQ))]
> "TARGET_HAVE_MVE"
> "vpst\;vstrdt.64\t%q2, [%0, %q1, UXTW #3]"
> @@ -8576,7 +8576,7 @@ (define_expand "mve_vstrhq_scatter_offset_p_fv8hf"
> [(match_operand:V8HI 0 "mve_scatter_memory")
> (match_operand:V8HI 1 "s_register_operand")
> (match_operand:V8HF 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V8BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRHQSO_F)]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> {
> @@ -8594,7 +8594,7 @@ (define_insn "mve_vstrhq_scatter_offset_p_fv8hf_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V8HI 1 "s_register_operand" "w")
> (match_operand:V8HF 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VSTRHQSO_F))]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> "vpst\;vstrht.16\t%q2, [%0, %q1]"
> @@ -8635,7 +8635,7 @@ (define_expand "mve_vstrhq_scatter_shifted_offset_p_fv8hf"
> [(match_operand:V8HI 0 "memory_operand" "=Us")
> (match_operand:V8HI 1 "s_register_operand" "w")
> (match_operand:V8HF 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")
> (unspec:V4SI [(const_int 0)] VSTRHQSSO_F)]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> {
> @@ -8654,7 +8654,7 @@ (define_insn "mve_vstrhq_scatter_shifted_offset_p_fv8hf_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V8HI 1 "s_register_operand" "w")
> (match_operand:V8HF 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
> VSTRHQSSO_F))]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> "vpst\;vstrht.16\t%q2, [%0, %q1, uxtw #1]"
> @@ -8691,7 +8691,7 @@ (define_insn "mve_vstrwq_scatter_base_p_fv4sf"
> [(match_operand:V4SI 0 "s_register_operand" "w")
> (match_operand:SI 1 "immediate_operand" "i")
> (match_operand:V4SF 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VSTRWQSB_F))
> ]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> @@ -8740,7 +8740,7 @@ (define_expand "mve_vstrwq_scatter_offset_p_fv4sf"
> [(match_operand:V4SI 0 "mve_scatter_memory")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:V4SF 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRWQSO_F)]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> {
> @@ -8758,7 +8758,7 @@ (define_insn "mve_vstrwq_scatter_offset_p_fv4sf_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V4SI 1 "s_register_operand" "w")
> (match_operand:V4SF 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VSTRWQSO_F))]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> "vpst\;vstrwt.32\t%q2, [%0, %q1]"
> @@ -8771,7 +8771,7 @@ (define_expand "mve_vstrwq_scatter_offset_p_<supf>v4si"
> [(match_operand:V4SI 0 "mve_scatter_memory")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:V4SI 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRWSOQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -8789,7 +8789,7 @@ (define_insn "mve_vstrwq_scatter_offset_p_<supf>v4si_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V4SI 1 "s_register_operand" "w")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VSTRWSOQ))]
> "TARGET_HAVE_MVE"
> "vpst\;vstrwt.32\t%q2, [%0, %q1]"
> @@ -8858,7 +8858,7 @@ (define_expand "mve_vstrwq_scatter_shifted_offset_p_fv4sf"
> [(match_operand:V4SI 0 "mve_scatter_memory")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:V4SF 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRWQSSO_F)]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> {
> @@ -8877,7 +8877,7 @@ (define_insn "mve_vstrwq_scatter_shifted_offset_p_fv4sf_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V4SI 1 "s_register_operand" "w")
> (match_operand:V4SF 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VSTRWQSSO_F))]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> "vpst\;vstrwt.32\t%q2, [%0, %q1, uxtw #2]"
> @@ -8890,7 +8890,7 @@ (define_expand "mve_vstrwq_scatter_shifted_offset_p_<supf>v4si"
> [(match_operand:V4SI 0 "mve_scatter_memory")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:V4SI 2 "s_register_operand")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VSTRWSSOQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -8909,7 +8909,7 @@ (define_insn "mve_vstrwq_scatter_shifted_offset_p_<supf>v4si_insn"
> [(match_operand:SI 0 "register_operand" "r")
> (match_operand:V4SI 1 "s_register_operand" "w")
> (match_operand:V4SI 2 "s_register_operand" "w")
> - (match_operand:HI 3 "vpr_register_operand" "Up")]
> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
> VSTRWSSOQ))]
> "TARGET_HAVE_MVE"
> "vpst\;vstrwt.32\t%q2, [%0, %q1, uxtw #2]"
> @@ -9376,7 +9376,7 @@ (define_insn "mve_vstrwq_scatter_base_wb_p_<supf>v4si"
> [(match_operand:V4SI 1 "s_register_operand" "0")
> (match_operand:SI 2 "mve_vldrd_immediate" "Ri")
> (match_operand:V4SI 3 "s_register_operand" "w")
> - (match_operand:HI 4 "vpr_register_operand")]
> + (match_operand:V4BI 4 "vpr_register_operand")]
> VSTRWSBWBQ))
> (set (match_operand:V4SI 0 "s_register_operand" "=w")
> (unspec:V4SI [(match_dup 1) (match_dup 2)]
> @@ -9427,7 +9427,7 @@ (define_insn "mve_vstrwq_scatter_base_wb_p_fv4sf"
> [(match_operand:V4SI 1 "s_register_operand" "0")
> (match_operand:SI 2 "mve_vldrd_immediate" "Ri")
> (match_operand:V4SF 3 "s_register_operand" "w")
> - (match_operand:HI 4 "vpr_register_operand")]
> + (match_operand:V4BI 4 "vpr_register_operand")]
> VSTRWQSBWB_F))
> (set (match_operand:V4SI 0 "s_register_operand" "=w")
> (unspec:V4SI [(match_dup 1) (match_dup 2)]
> @@ -9478,7 +9478,7 @@ (define_insn "mve_vstrdq_scatter_base_wb_p_<supf>v2di"
> [(match_operand:V2DI 1 "s_register_operand" "0")
> (match_operand:SI 2 "mve_vldrd_immediate" "Ri")
> (match_operand:V2DI 3 "s_register_operand" "w")
> - (match_operand:HI 4 "vpr_register_operand")]
> + (match_operand:V8BI 4 "vpr_register_operand")]
> VSTRDSBWBQ))
> (set (match_operand:V2DI 0 "s_register_operand" "=w")
> (unspec:V2DI [(match_dup 1) (match_dup 2)]
> @@ -9551,7 +9551,7 @@ (define_expand "mve_vldrwq_gather_base_wb_z_<supf>v4si"
> [(match_operand:V4SI 0 "s_register_operand")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:SI 2 "mve_vldrd_immediate")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VLDRWGBWBQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -9566,7 +9566,7 @@ (define_expand "mve_vldrwq_gather_base_nowb_z_<supf>v4si"
> [(match_operand:V4SI 0 "s_register_operand")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:SI 2 "mve_vldrd_immediate")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VLDRWGBWBQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -9585,7 +9585,7 @@ (define_insn "mve_vldrwq_gather_base_wb_z_<supf>v4si_insn"
> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
> (unspec:V4SI [(match_operand:V4SI 2 "s_register_operand" "1")
> (match_operand:SI 3 "mve_vldrd_immediate" "Ri")
> - (match_operand:HI 4 "vpr_register_operand" "Up")
> + (match_operand:V4BI 4 "vpr_register_operand" "Up")
> (mem:BLK (scratch))]
> VLDRWGBWBQ))
> (set (match_operand:V4SI 1 "s_register_operand" "=&w")
> @@ -9659,7 +9659,7 @@ (define_expand "mve_vldrwq_gather_base_wb_z_fv4sf"
> [(match_operand:V4SI 0 "s_register_operand")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:SI 2 "mve_vldrd_immediate")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VLDRWQGBWB_F)]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> {
> @@ -9675,7 +9675,7 @@ (define_expand "mve_vldrwq_gather_base_nowb_z_fv4sf"
> [(match_operand:V4SF 0 "s_register_operand")
> (match_operand:V4SI 1 "s_register_operand")
> (match_operand:SI 2 "mve_vldrd_immediate")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V4BI 3 "vpr_register_operand")
> (unspec:V4SI [(const_int 0)] VLDRWQGBWB_F)]
> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
> {
> @@ -9694,7 +9694,7 @@ (define_insn "mve_vldrwq_gather_base_wb_z_fv4sf_insn"
> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
> (unspec:V4SF [(match_operand:V4SI 2 "s_register_operand" "1")
> (match_operand:SI 3 "mve_vldrd_immediate" "Ri")
> - (match_operand:HI 4 "vpr_register_operand" "Up")
> + (match_operand:V4BI 4 "vpr_register_operand" "Up")
> (mem:BLK (scratch))]
> VLDRWQGBWB_F))
> (set (match_operand:V4SI 1 "s_register_operand" "=&w")
> @@ -9769,7 +9769,7 @@ (define_expand "mve_vldrdq_gather_base_wb_z_<supf>v2di"
> [(match_operand:V2DI 0 "s_register_operand")
> (match_operand:V2DI 1 "s_register_operand")
> (match_operand:SI 2 "mve_vldrd_immediate")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V8BI 3 "vpr_register_operand")
> (unspec:V2DI [(const_int 0)] VLDRDGBWBQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -9785,7 +9785,7 @@ (define_expand "mve_vldrdq_gather_base_nowb_z_<supf>v2di"
> [(match_operand:V2DI 0 "s_register_operand")
> (match_operand:V2DI 1 "s_register_operand")
> (match_operand:SI 2 "mve_vldrd_immediate")
> - (match_operand:HI 3 "vpr_register_operand")
> + (match_operand:V8BI 3 "vpr_register_operand")
> (unspec:V2DI [(const_int 0)] VLDRDGBWBQ)]
> "TARGET_HAVE_MVE"
> {
> @@ -9819,7 +9819,7 @@ (define_insn "mve_vldrdq_gather_base_wb_z_<supf>v2di_insn"
> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
> (unspec:V2DI [(match_operand:V2DI 2 "s_register_operand" "1")
> (match_operand:SI 3 "mve_vldrd_immediate" "Ri")
> - (match_operand:HI 4 "vpr_register_operand" "Up")
> + (match_operand:V8BI 4 "vpr_register_operand" "Up")
> (mem:BLK (scratch))]
> VLDRDGBWBQ))
> (set (match_operand:V2DI 1 "s_register_operand" "=&w")
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
` (13 preceding siblings ...)
2021-10-13 10:15 ` [PATCH v2 14/14] arm: Add VPR_REG to ALL_REGS Christophe Lyon
@ 2021-10-15 15:10 ` Richard Sandiford
14 siblings, 0 replies; 21+ messages in thread
From: Richard Sandiford @ 2021-10-15 15:10 UTC (permalink / raw)
To: Christophe Lyon via Gcc-patches
Cc: Christophe Lyon, nickc, richard.earnshaw, ramana.radhakrishnan,
kyrylo.tkachov
Christophe Lyon via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
> This is v2 of this patch series, addressing the comments I received.
> The changes v1 -> v2 are:
>
> - Patch 3: added an executable test, and updated
> check_effective_target_arm_mve_hw
> - Patch 4: split into patch 4 and patch 14 (to keep numbering the same
> for the other patches)
> - Patch 5: updated arm_class_likely_spilled_p as suggested.
> - Patch 7: updated test_vector_ops_duplicate in simplify-rtx.c as
> suggested.
> - Patch 8: added V2DI -> HI/hi mapping in MVE_VPRED/MVE_vpred
> iterators, removed now useless mve_vpselq_<supf>v2di, and fixed
> mov<mode> expander.
> - Patch 9: arm_mode_to_pred_mode now returns opt_machine_mode, removed
> useless floating-point checks in vec_cmpu.
> - Patch 12: replaced hi with v8bi in v2di load/store instructions
>
> I'll squash patch 2 with patch patch 9 and patch 3 with patch 8.
This looks good to me part from the question in 12/14 and the couple
of other (very) minor nits.
Thanks,
Richard
> Original text:
>
> This patch series addresses PR 100757 and 101325 by representing
> vectors of predicates (MVE VPR.P0 register) as vectors of booleans
> rather than using HImode.
>
> As this implies a lot of mostly mechanical changes, I have tried to
> split the patches in a way that should help reviewers, but the split
> is a bit artificial.
>
> Patches 1-3 add new tests.
>
> Patches 4-6 are small independent improvements.
>
> Patch 7 implements the predicate qualifier, but does not change any
> builtin yet.
>
> Patch 8 is the first of the two main patches, and uses the new
> qualifier to describe the vcmp and vpsel builtins that are useful for
> auto-vectorization of comparisons.
>
> Patch 9 is the second main patch, which fixes the vcond_mask expander.
>
> Patches 10-13 convert almost all the remaining builtins with HI
> operands to use the predicate qualifier. After these, there are still
> a few builtins with HI operands left, about which I am not sure: vctp,
> vpnot, load-gather and store-scatter with v2di operands. In fact,
> patches 11/12 update some STR/LDR qualifiers in a way that breaks
> these v2di builtins although existing tests still pass.
>
> Christophe Lyon (14):
> arm: Add new tests for comparison vectorization with Neon and MVE
> arm: Add tests for PR target/100757
> arm: Add tests for PR target/101325
> arm: Add GENERAL_AND_VPR_REGS regclass
> arm: Add support for VPR_REG in arm_class_likely_spilled_p
> arm: Fix mve_vmvnq_n_<supf><mode> argument mode
> arm: Implement MVE predicates as vectors of booleans
> arm: Implement auto-vectorized MVE comparisons with vectors of boolean
> predicates
> arm: Fix vcond_mask expander for MVE (PR target/100757)
> arm: Convert remaining MVE vcmp builtins to predicate qualifiers
> arm: Convert more MVE builtins to predicate qualifiers
> arm: Convert more load/store MVE builtins to predicate qualifiers
> arm: Convert more MVE/CDE builtins to predicate qualifiers
> arm: Add VPR_REG to ALL_REGS
>
> gcc/config/arm/arm-builtins.c | 228 +++--
> gcc/config/arm/arm-modes.def | 5 +
> gcc/config/arm/arm-protos.h | 3 +-
> gcc/config/arm/arm-simd-builtin-types.def | 4 +
> gcc/config/arm/arm.c | 130 ++-
> gcc/config/arm/arm.h | 5 +-
> gcc/config/arm/arm_mve_builtins.def | 746 ++++++++--------
> gcc/config/arm/iterators.md | 5 +
> gcc/config/arm/mve.md | 832 ++++++++++--------
> gcc/config/arm/neon.md | 39 +
> gcc/config/arm/vec-common.md | 52 --
> gcc/simplify-rtx.c | 26 +-
> .../arm/acle/cde-mve-full-assembly.c | 264 +++---
> .../gcc.target/arm/simd/mve-vcmp-f32-2.c | 32 +
> .../gcc.target/arm/simd/neon-compare-1.c | 78 ++
> .../gcc.target/arm/simd/neon-compare-2.c | 13 +
> .../gcc.target/arm/simd/neon-compare-3.c | 14 +
> .../arm/simd/neon-compare-scalar-1.c | 57 ++
> .../gcc.target/arm/simd/neon-vcmp-f16.c | 12 +
> .../gcc.target/arm/simd/neon-vcmp-f32-2.c | 15 +
> .../gcc.target/arm/simd/neon-vcmp-f32-3.c | 12 +
> .../gcc.target/arm/simd/neon-vcmp-f32.c | 12 +
> gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c | 22 +
> .../gcc.target/arm/simd/pr100757-2.c | 20 +
> .../gcc.target/arm/simd/pr100757-3.c | 20 +
> .../gcc.target/arm/simd/pr100757-4.c | 19 +
> gcc/testsuite/gcc.target/arm/simd/pr100757.c | 19 +
> .../gcc.target/arm/simd/pr101325-2.c | 19 +
> gcc/testsuite/gcc.target/arm/simd/pr101325.c | 14 +
> gcc/testsuite/lib/target-supports.exp | 3 +-
> 30 files changed, 1611 insertions(+), 1109 deletions(-)
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/mve-vcmp-f32-2.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-1.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-2.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-3.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-compare-scalar-1.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f16.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-2.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32-3.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp-f32.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/neon-vcmp.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757-2.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757-3.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757-4.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr100757.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr101325-2.c
> create mode 100644 gcc/testsuite/gcc.target/arm/simd/pr101325.c
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 12/14] arm: Convert more load/store MVE builtins to predicate qualifiers
2021-10-15 15:08 ` Richard Sandiford
@ 2021-10-15 15:33 ` Christophe LYON
2021-10-15 15:56 ` Richard Sandiford
0 siblings, 1 reply; 21+ messages in thread
From: Christophe LYON @ 2021-10-15 15:33 UTC (permalink / raw)
To: Christophe Lyon via Gcc-patches, richard.sandiford
On 15/10/2021 17:08, Richard Sandiford wrote:
> Christophe Lyon via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
>> This patch covers a few builtins where we do not use the <mode>
>> iterator and thus we cannot use <MVE_vpred>.
>>
>> For v2di instructions, we use the V8BI mode for predicates.
> Why V8BI though, when VPRED uses HI?
Hmm.. I used your suggestion:
https://gcc.gnu.org/pipermail/gcc-patches/2021-October/581362.html
Maybe I misinterpreted it?
>
> Would it make sense to define a V2BI? Or doesn't that work?
I didn't try, I'll have a look. wrt the architecture, I'm not sure what
this would mean?
Thanks,
Christophe
>
> Thanks,
> Richard
>
>> 2021-10-13 Christophe Lyon <christophe.lyon@foss.st.com>
>>
>> gcc/
>> PR target/100757
>> PR target/101325
>> * config/arm/arm-builtins.c (STRSBS_P_QUALIFIERS): Use predicate
>> qualifier.
>> (STRSBU_P_QUALIFIERS): Likewise.
>> (LDRGBS_Z_QUALIFIERS): Likewise.
>> (LDRGBU_Z_QUALIFIERS): Likewise.
>> (LDRGBWBXU_Z_QUALIFIERS): Likewise.
>> (LDRGBWBS_Z_QUALIFIERS): Likewise.
>> (LDRGBWBU_Z_QUALIFIERS): Likewise.
>> (STRSBWBS_P_QUALIFIERS): Likewise.
>> (STRSBWBU_P_QUALIFIERS): Likewise.
>> * config/arm/mve.md: Use VxBI instead of HI.
>>
>> diff --git a/gcc/config/arm/arm-builtins.c b/gcc/config/arm/arm-builtins.c
>> index 06ff9d2278a..e58580bb828 100644
>> --- a/gcc/config/arm/arm-builtins.c
>> +++ b/gcc/config/arm/arm-builtins.c
>> @@ -738,13 +738,13 @@ arm_strss_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> static enum arm_type_qualifiers
>> arm_strsbs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_void, qualifier_unsigned, qualifier_immediate,
>> - qualifier_none, qualifier_unsigned};
>> + qualifier_none, qualifier_predicate};
>> #define STRSBS_P_QUALIFIERS (arm_strsbs_p_qualifiers)
>>
>> static enum arm_type_qualifiers
>> arm_strsbu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_void, qualifier_unsigned, qualifier_immediate,
>> - qualifier_unsigned, qualifier_unsigned};
>> + qualifier_unsigned, qualifier_predicate};
>> #define STRSBU_P_QUALIFIERS (arm_strsbu_p_qualifiers)
>>
>> static enum arm_type_qualifiers
>> @@ -780,13 +780,13 @@ arm_ldrgbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> static enum arm_type_qualifiers
>> arm_ldrgbs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_none, qualifier_unsigned, qualifier_immediate,
>> - qualifier_unsigned};
>> + qualifier_predicate};
>> #define LDRGBS_Z_QUALIFIERS (arm_ldrgbs_z_qualifiers)
>>
>> static enum arm_type_qualifiers
>> arm_ldrgbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
>> - qualifier_unsigned};
>> + qualifier_predicate};
>> #define LDRGBU_Z_QUALIFIERS (arm_ldrgbu_z_qualifiers)
>>
>> static enum arm_type_qualifiers
>> @@ -826,7 +826,7 @@ arm_ldrgbwbxu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> static enum arm_type_qualifiers
>> arm_ldrgbwbxu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
>> - qualifier_unsigned};
>> + qualifier_predicate};
>> #define LDRGBWBXU_Z_QUALIFIERS (arm_ldrgbwbxu_z_qualifiers)
>>
>> static enum arm_type_qualifiers
>> @@ -842,13 +842,13 @@ arm_ldrgbwbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> static enum arm_type_qualifiers
>> arm_ldrgbwbs_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_none, qualifier_unsigned, qualifier_immediate,
>> - qualifier_unsigned};
>> + qualifier_predicate};
>> #define LDRGBWBS_Z_QUALIFIERS (arm_ldrgbwbs_z_qualifiers)
>>
>> static enum arm_type_qualifiers
>> arm_ldrgbwbu_z_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_unsigned, qualifier_unsigned, qualifier_immediate,
>> - qualifier_unsigned};
>> + qualifier_predicate};
>> #define LDRGBWBU_Z_QUALIFIERS (arm_ldrgbwbu_z_qualifiers)
>>
>> static enum arm_type_qualifiers
>> @@ -864,13 +864,13 @@ arm_strsbwbu_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> static enum arm_type_qualifiers
>> arm_strsbwbs_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_unsigned, qualifier_unsigned, qualifier_const,
>> - qualifier_none, qualifier_unsigned};
>> + qualifier_none, qualifier_predicate};
>> #define STRSBWBS_P_QUALIFIERS (arm_strsbwbs_p_qualifiers)
>>
>> static enum arm_type_qualifiers
>> arm_strsbwbu_p_qualifiers[SIMD_MAX_BUILTIN_ARGS]
>> = { qualifier_unsigned, qualifier_unsigned, qualifier_const,
>> - qualifier_unsigned, qualifier_unsigned};
>> + qualifier_unsigned, qualifier_predicate};
>> #define STRSBWBU_P_QUALIFIERS (arm_strsbwbu_p_qualifiers)
>>
>> static enum arm_type_qualifiers
>> diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
>> index 81ad488155d..c07487c0750 100644
>> --- a/gcc/config/arm/mve.md
>> +++ b/gcc/config/arm/mve.md
>> @@ -7282,7 +7282,7 @@ (define_insn "mve_vstrwq_scatter_base_p_<supf>v4si"
>> [(match_operand:V4SI 0 "s_register_operand" "w")
>> (match_operand:SI 1 "immediate_operand" "i")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VSTRWSBQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -7371,7 +7371,7 @@ (define_insn "mve_vldrwq_gather_base_z_<supf>v4si"
>> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
>> (unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
>> (match_operand:SI 2 "immediate_operand" "i")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VLDRWGBQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -7609,7 +7609,7 @@ (define_insn "mve_vldrwq_<supf>v4si"
>> (define_insn "mve_vldrwq_z_fv4sf"
>> [(set (match_operand:V4SF 0 "s_register_operand" "=w")
>> (unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Ux")
>> - (match_operand:HI 2 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 2 "vpr_register_operand" "Up")]
>> VLDRWQ_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -7629,7 +7629,7 @@ (define_insn "mve_vldrwq_z_fv4sf"
>> (define_insn "mve_vldrwq_z_<supf>v4si"
>> [(set (match_operand:V4SI 0 "s_register_operand" "=w")
>> (unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Ux")
>> - (match_operand:HI 2 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 2 "vpr_register_operand" "Up")]
>> VLDRWQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -7690,7 +7690,7 @@ (define_insn "mve_vldrdq_gather_base_z_<supf>v2di"
>> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
>> (unspec:V2DI [(match_operand:V2DI 1 "s_register_operand" "w")
>> (match_operand:SI 2 "immediate_operand" "i")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VLDRDGBQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -7731,7 +7731,7 @@ (define_insn "mve_vldrdq_gather_offset_z_<supf>v2di"
>> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
>> (unspec:V2DI [(match_operand:V2DI 1 "memory_operand" "Us")
>> (match_operand:V2DI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VLDRDGOQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -7772,7 +7772,7 @@ (define_insn "mve_vldrdq_gather_shifted_offset_z_<supf>v2di"
>> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
>> (unspec:V2DI [(match_operand:V2DI 1 "memory_operand" "Us")
>> (match_operand:V2DI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VLDRDGSOQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -7813,7 +7813,7 @@ (define_insn "mve_vldrhq_gather_offset_z_fv8hf"
>> [(set (match_operand:V8HF 0 "s_register_operand" "=&w")
>> (unspec:V8HF [(match_operand:V8HI 1 "memory_operand" "Us")
>> (match_operand:V8HI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VLDRHQGO_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -7855,7 +7855,7 @@ (define_insn "mve_vldrhq_gather_shifted_offset_z_fv8hf"
>> [(set (match_operand:V8HF 0 "s_register_operand" "=&w")
>> (unspec:V8HF [(match_operand:V8HI 1 "memory_operand" "Us")
>> (match_operand:V8HI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VLDRHQGSO_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -7897,7 +7897,7 @@ (define_insn "mve_vldrwq_gather_base_z_fv4sf"
>> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
>> (unspec:V4SF [(match_operand:V4SI 1 "s_register_operand" "w")
>> (match_operand:SI 2 "immediate_operand" "i")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VLDRWQGB_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -7958,7 +7958,7 @@ (define_insn "mve_vldrwq_gather_offset_z_fv4sf"
>> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
>> (unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Us")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VLDRWQGO_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -7980,7 +7980,7 @@ (define_insn "mve_vldrwq_gather_offset_z_<supf>v4si"
>> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
>> (unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Us")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VLDRWGOQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -8042,7 +8042,7 @@ (define_insn "mve_vldrwq_gather_shifted_offset_z_fv4sf"
>> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
>> (unspec:V4SF [(match_operand:V4SI 1 "memory_operand" "Us")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VLDRWQGSO_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -8064,7 +8064,7 @@ (define_insn "mve_vldrwq_gather_shifted_offset_z_<supf>v4si"
>> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
>> (unspec:V4SI [(match_operand:V4SI 1 "memory_operand" "Us")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VLDRWGSOQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -8104,7 +8104,7 @@ (define_insn "mve_vstrhq_fv8hf"
>> (define_insn "mve_vstrhq_p_fv8hf"
>> [(set (match_operand:V8HI 0 "mve_memory_operand" "=Ux")
>> (unspec:V8HI [(match_operand:V8HF 1 "s_register_operand" "w")
>> - (match_operand:HI 2 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 2 "vpr_register_operand" "Up")]
>> VSTRHQ_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -8323,7 +8323,7 @@ (define_insn "mve_vstrwq_p_fv4sf"
>> (define_insn "mve_vstrwq_p_<supf>v4si"
>> [(set (match_operand:V4SI 0 "memory_operand" "=Ux")
>> (unspec:V4SI [(match_operand:V4SI 1 "s_register_operand" "w")
>> - (match_operand:HI 2 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 2 "vpr_register_operand" "Up")]
>> VSTRWQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -8385,7 +8385,7 @@ (define_insn "mve_vstrdq_scatter_base_p_<supf>v2di"
>> [(match_operand:V2DI 0 "s_register_operand" "w")
>> (match_operand:SI 1 "mve_vldrd_immediate" "Ri")
>> (match_operand:V2DI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VSTRDSBQ))
>> ]
>> "TARGET_HAVE_MVE"
>> @@ -8428,7 +8428,7 @@ (define_expand "mve_vstrdq_scatter_offset_p_<supf>v2di"
>> [(match_operand:V2DI 0 "mve_scatter_memory")
>> (match_operand:V2DI 1 "s_register_operand")
>> (match_operand:V2DI 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V8BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRDSOQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -8446,7 +8446,7 @@ (define_insn "mve_vstrdq_scatter_offset_p_<supf>v2di_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V2DI 1 "s_register_operand" "w")
>> (match_operand:V2DI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VSTRDSOQ))]
>> "TARGET_HAVE_MVE"
>> "vpst\;vstrdt.64\t%q2, [%0, %q1]"
>> @@ -8487,7 +8487,7 @@ (define_expand "mve_vstrdq_scatter_shifted_offset_p_<supf>v2di"
>> [(match_operand:V2DI 0 "mve_scatter_memory")
>> (match_operand:V2DI 1 "s_register_operand")
>> (match_operand:V2DI 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V8BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRDSSOQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -8506,7 +8506,7 @@ (define_insn "mve_vstrdq_scatter_shifted_offset_p_<supf>v2di_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V2DI 1 "s_register_operand" "w")
>> (match_operand:V2DI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VSTRDSSOQ))]
>> "TARGET_HAVE_MVE"
>> "vpst\;vstrdt.64\t%q2, [%0, %q1, UXTW #3]"
>> @@ -8576,7 +8576,7 @@ (define_expand "mve_vstrhq_scatter_offset_p_fv8hf"
>> [(match_operand:V8HI 0 "mve_scatter_memory")
>> (match_operand:V8HI 1 "s_register_operand")
>> (match_operand:V8HF 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V8BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRHQSO_F)]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> {
>> @@ -8594,7 +8594,7 @@ (define_insn "mve_vstrhq_scatter_offset_p_fv8hf_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V8HI 1 "s_register_operand" "w")
>> (match_operand:V8HF 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VSTRHQSO_F))]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> "vpst\;vstrht.16\t%q2, [%0, %q1]"
>> @@ -8635,7 +8635,7 @@ (define_expand "mve_vstrhq_scatter_shifted_offset_p_fv8hf"
>> [(match_operand:V8HI 0 "memory_operand" "=Us")
>> (match_operand:V8HI 1 "s_register_operand" "w")
>> (match_operand:V8HF 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")
>> (unspec:V4SI [(const_int 0)] VSTRHQSSO_F)]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> {
>> @@ -8654,7 +8654,7 @@ (define_insn "mve_vstrhq_scatter_shifted_offset_p_fv8hf_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V8HI 1 "s_register_operand" "w")
>> (match_operand:V8HF 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V8BI 3 "vpr_register_operand" "Up")]
>> VSTRHQSSO_F))]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> "vpst\;vstrht.16\t%q2, [%0, %q1, uxtw #1]"
>> @@ -8691,7 +8691,7 @@ (define_insn "mve_vstrwq_scatter_base_p_fv4sf"
>> [(match_operand:V4SI 0 "s_register_operand" "w")
>> (match_operand:SI 1 "immediate_operand" "i")
>> (match_operand:V4SF 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VSTRWQSB_F))
>> ]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> @@ -8740,7 +8740,7 @@ (define_expand "mve_vstrwq_scatter_offset_p_fv4sf"
>> [(match_operand:V4SI 0 "mve_scatter_memory")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:V4SF 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRWQSO_F)]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> {
>> @@ -8758,7 +8758,7 @@ (define_insn "mve_vstrwq_scatter_offset_p_fv4sf_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V4SI 1 "s_register_operand" "w")
>> (match_operand:V4SF 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VSTRWQSO_F))]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> "vpst\;vstrwt.32\t%q2, [%0, %q1]"
>> @@ -8771,7 +8771,7 @@ (define_expand "mve_vstrwq_scatter_offset_p_<supf>v4si"
>> [(match_operand:V4SI 0 "mve_scatter_memory")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:V4SI 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRWSOQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -8789,7 +8789,7 @@ (define_insn "mve_vstrwq_scatter_offset_p_<supf>v4si_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V4SI 1 "s_register_operand" "w")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VSTRWSOQ))]
>> "TARGET_HAVE_MVE"
>> "vpst\;vstrwt.32\t%q2, [%0, %q1]"
>> @@ -8858,7 +8858,7 @@ (define_expand "mve_vstrwq_scatter_shifted_offset_p_fv4sf"
>> [(match_operand:V4SI 0 "mve_scatter_memory")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:V4SF 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRWQSSO_F)]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> {
>> @@ -8877,7 +8877,7 @@ (define_insn "mve_vstrwq_scatter_shifted_offset_p_fv4sf_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V4SI 1 "s_register_operand" "w")
>> (match_operand:V4SF 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VSTRWQSSO_F))]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> "vpst\;vstrwt.32\t%q2, [%0, %q1, uxtw #2]"
>> @@ -8890,7 +8890,7 @@ (define_expand "mve_vstrwq_scatter_shifted_offset_p_<supf>v4si"
>> [(match_operand:V4SI 0 "mve_scatter_memory")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:V4SI 2 "s_register_operand")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VSTRWSSOQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -8909,7 +8909,7 @@ (define_insn "mve_vstrwq_scatter_shifted_offset_p_<supf>v4si_insn"
>> [(match_operand:SI 0 "register_operand" "r")
>> (match_operand:V4SI 1 "s_register_operand" "w")
>> (match_operand:V4SI 2 "s_register_operand" "w")
>> - (match_operand:HI 3 "vpr_register_operand" "Up")]
>> + (match_operand:V4BI 3 "vpr_register_operand" "Up")]
>> VSTRWSSOQ))]
>> "TARGET_HAVE_MVE"
>> "vpst\;vstrwt.32\t%q2, [%0, %q1, uxtw #2]"
>> @@ -9376,7 +9376,7 @@ (define_insn "mve_vstrwq_scatter_base_wb_p_<supf>v4si"
>> [(match_operand:V4SI 1 "s_register_operand" "0")
>> (match_operand:SI 2 "mve_vldrd_immediate" "Ri")
>> (match_operand:V4SI 3 "s_register_operand" "w")
>> - (match_operand:HI 4 "vpr_register_operand")]
>> + (match_operand:V4BI 4 "vpr_register_operand")]
>> VSTRWSBWBQ))
>> (set (match_operand:V4SI 0 "s_register_operand" "=w")
>> (unspec:V4SI [(match_dup 1) (match_dup 2)]
>> @@ -9427,7 +9427,7 @@ (define_insn "mve_vstrwq_scatter_base_wb_p_fv4sf"
>> [(match_operand:V4SI 1 "s_register_operand" "0")
>> (match_operand:SI 2 "mve_vldrd_immediate" "Ri")
>> (match_operand:V4SF 3 "s_register_operand" "w")
>> - (match_operand:HI 4 "vpr_register_operand")]
>> + (match_operand:V4BI 4 "vpr_register_operand")]
>> VSTRWQSBWB_F))
>> (set (match_operand:V4SI 0 "s_register_operand" "=w")
>> (unspec:V4SI [(match_dup 1) (match_dup 2)]
>> @@ -9478,7 +9478,7 @@ (define_insn "mve_vstrdq_scatter_base_wb_p_<supf>v2di"
>> [(match_operand:V2DI 1 "s_register_operand" "0")
>> (match_operand:SI 2 "mve_vldrd_immediate" "Ri")
>> (match_operand:V2DI 3 "s_register_operand" "w")
>> - (match_operand:HI 4 "vpr_register_operand")]
>> + (match_operand:V8BI 4 "vpr_register_operand")]
>> VSTRDSBWBQ))
>> (set (match_operand:V2DI 0 "s_register_operand" "=w")
>> (unspec:V2DI [(match_dup 1) (match_dup 2)]
>> @@ -9551,7 +9551,7 @@ (define_expand "mve_vldrwq_gather_base_wb_z_<supf>v4si"
>> [(match_operand:V4SI 0 "s_register_operand")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:SI 2 "mve_vldrd_immediate")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VLDRWGBWBQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -9566,7 +9566,7 @@ (define_expand "mve_vldrwq_gather_base_nowb_z_<supf>v4si"
>> [(match_operand:V4SI 0 "s_register_operand")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:SI 2 "mve_vldrd_immediate")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VLDRWGBWBQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -9585,7 +9585,7 @@ (define_insn "mve_vldrwq_gather_base_wb_z_<supf>v4si_insn"
>> [(set (match_operand:V4SI 0 "s_register_operand" "=&w")
>> (unspec:V4SI [(match_operand:V4SI 2 "s_register_operand" "1")
>> (match_operand:SI 3 "mve_vldrd_immediate" "Ri")
>> - (match_operand:HI 4 "vpr_register_operand" "Up")
>> + (match_operand:V4BI 4 "vpr_register_operand" "Up")
>> (mem:BLK (scratch))]
>> VLDRWGBWBQ))
>> (set (match_operand:V4SI 1 "s_register_operand" "=&w")
>> @@ -9659,7 +9659,7 @@ (define_expand "mve_vldrwq_gather_base_wb_z_fv4sf"
>> [(match_operand:V4SI 0 "s_register_operand")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:SI 2 "mve_vldrd_immediate")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VLDRWQGBWB_F)]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> {
>> @@ -9675,7 +9675,7 @@ (define_expand "mve_vldrwq_gather_base_nowb_z_fv4sf"
>> [(match_operand:V4SF 0 "s_register_operand")
>> (match_operand:V4SI 1 "s_register_operand")
>> (match_operand:SI 2 "mve_vldrd_immediate")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V4BI 3 "vpr_register_operand")
>> (unspec:V4SI [(const_int 0)] VLDRWQGBWB_F)]
>> "TARGET_HAVE_MVE && TARGET_HAVE_MVE_FLOAT"
>> {
>> @@ -9694,7 +9694,7 @@ (define_insn "mve_vldrwq_gather_base_wb_z_fv4sf_insn"
>> [(set (match_operand:V4SF 0 "s_register_operand" "=&w")
>> (unspec:V4SF [(match_operand:V4SI 2 "s_register_operand" "1")
>> (match_operand:SI 3 "mve_vldrd_immediate" "Ri")
>> - (match_operand:HI 4 "vpr_register_operand" "Up")
>> + (match_operand:V4BI 4 "vpr_register_operand" "Up")
>> (mem:BLK (scratch))]
>> VLDRWQGBWB_F))
>> (set (match_operand:V4SI 1 "s_register_operand" "=&w")
>> @@ -9769,7 +9769,7 @@ (define_expand "mve_vldrdq_gather_base_wb_z_<supf>v2di"
>> [(match_operand:V2DI 0 "s_register_operand")
>> (match_operand:V2DI 1 "s_register_operand")
>> (match_operand:SI 2 "mve_vldrd_immediate")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V8BI 3 "vpr_register_operand")
>> (unspec:V2DI [(const_int 0)] VLDRDGBWBQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -9785,7 +9785,7 @@ (define_expand "mve_vldrdq_gather_base_nowb_z_<supf>v2di"
>> [(match_operand:V2DI 0 "s_register_operand")
>> (match_operand:V2DI 1 "s_register_operand")
>> (match_operand:SI 2 "mve_vldrd_immediate")
>> - (match_operand:HI 3 "vpr_register_operand")
>> + (match_operand:V8BI 3 "vpr_register_operand")
>> (unspec:V2DI [(const_int 0)] VLDRDGBWBQ)]
>> "TARGET_HAVE_MVE"
>> {
>> @@ -9819,7 +9819,7 @@ (define_insn "mve_vldrdq_gather_base_wb_z_<supf>v2di_insn"
>> [(set (match_operand:V2DI 0 "s_register_operand" "=&w")
>> (unspec:V2DI [(match_operand:V2DI 2 "s_register_operand" "1")
>> (match_operand:SI 3 "mve_vldrd_immediate" "Ri")
>> - (match_operand:HI 4 "vpr_register_operand" "Up")
>> + (match_operand:V8BI 4 "vpr_register_operand" "Up")
>> (mem:BLK (scratch))]
>> VLDRDGBWBQ))
>> (set (match_operand:V2DI 1 "s_register_operand" "=&w")
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 12/14] arm: Convert more load/store MVE builtins to predicate qualifiers
2021-10-15 15:33 ` Christophe LYON
@ 2021-10-15 15:56 ` Richard Sandiford
0 siblings, 0 replies; 21+ messages in thread
From: Richard Sandiford @ 2021-10-15 15:56 UTC (permalink / raw)
To: Christophe LYON; +Cc: Christophe Lyon via Gcc-patches
Christophe LYON <christophe.lyon@foss.st.com> writes:
> On 15/10/2021 17:08, Richard Sandiford wrote:
>> Christophe Lyon via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
>>> This patch covers a few builtins where we do not use the <mode>
>>> iterator and thus we cannot use <MVE_vpred>.
>>>
>>> For v2di instructions, we use the V8BI mode for predicates.
>> Why V8BI though, when VPRED uses HI?
>
>
> Hmm.. I used your suggestion:
>
> https://gcc.gnu.org/pipermail/gcc-patches/2021-October/581362.html
>
> Maybe I misinterpreted it?
Gah! Sorry, that was a typo for V2BI :-(
>> Would it make sense to define a V2BI? Or doesn't that work?
>
> I didn't try, I'll have a look. wrt the architecture, I'm not sure what
> this would mean?
Hmm, yeah, I guess V2BI is wrong because every 4th bit matters.
It would need to be V4BI instead.
I guess if the number of predicate elements doesn't match the number
of data elements, there's probably not much point using a dedicated
predicate mode.
So yeah, maybe keeping HImode is better after all. Sorry for the
misdirection.
Richard
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2021-10-15 15:56 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-13 10:15 [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 01/14] arm: Add new tests for comparison vectorization with Neon and MVE Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 02/14] arm: Add tests for PR target/100757 Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 03/14] arm: Add tests for PR target/101325 Christophe Lyon
2021-10-15 14:49 ` Richard Sandiford
2021-10-13 10:15 ` [PATCH v2 04/14] arm: Add GENERAL_AND_VPR_REGS regclass Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 05/14] arm: Add support for VPR_REG in arm_class_likely_spilled_p Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 06/14] arm: Fix mve_vmvnq_n_<supf><mode> argument mode Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 07/14] arm: Implement MVE predicates as vectors of booleans Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 08/14] arm: Implement auto-vectorized MVE comparisons with vectors of boolean predicates Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 09/14] arm: Fix vcond_mask expander for MVE (PR target/100757) Christophe Lyon
2021-10-15 14:55 ` Richard Sandiford
2021-10-13 10:15 ` [PATCH v2 10/14] arm: Convert remaining MVE vcmp builtins to predicate qualifiers Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 11/14] arm: Convert more MVE " Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 12/14] arm: Convert more load/store " Christophe Lyon
2021-10-15 15:08 ` Richard Sandiford
2021-10-15 15:33 ` Christophe LYON
2021-10-15 15:56 ` Richard Sandiford
2021-10-13 10:15 ` [PATCH v2 13/14] arm: Convert more MVE/CDE " Christophe Lyon
2021-10-13 10:15 ` [PATCH v2 14/14] arm: Add VPR_REG to ALL_REGS Christophe Lyon
2021-10-15 15:10 ` [PATCH v2 00/14] ARM/MVE use vectors of boolean for predicates Richard Sandiford
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).