* [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2}
@ 2023-11-15 14:41 Tamar Christina
2023-11-21 10:39 ` Richard Sandiford
2023-11-21 22:43 ` Andrew Pinski
0 siblings, 2 replies; 4+ messages in thread
From: Tamar Christina @ 2023-11-15 14:41 UTC (permalink / raw)
To: gcc-patches
Cc: nd, Richard.Earnshaw, Marcus.Shawcroft, Kyrylo.Tkachov,
richard.sandiford
[-- Attachment #1: Type: text/plain, Size: 13268 bytes --]
Hi All,
This changes unpack instructions to use zip{1,2} when doing a zero-extending
widening operation. Permutes generally have a higher throughput than the
widening operations. Zeros are shuffled into the top half of the registers.
The testcase
void d2 (unsigned * restrict a, unsigned short *b, int n)
{
for (int i = 0; i < (n & -8); i++)
a[i] = b[i];
}
now generates:
movi v1.4s, 0
.L3:
ldr q0, [x1], 16
zip1 v2.8h, v0.8h, v1.8h
zip2 v0.8h, v0.8h, v1.8h
stp q2, q0, [x0]
add x0, x0, 32
cmp x1, x2
bne .L3
instead of:
.L3:
ldr q0, [x1], 16
uxtl v1.4s, v0.4h
uxtl2 v0.4s, v0.8h
stp q1, q0, [x0]
add x0, x0, 32
cmp x1, x2
bne .L3
Since we need the extra 0 register we do this only for the vectorizer's lo/hi
pairs when we know the 0 will be floated outside of the loop.
This gives an 8% speed-up in Imagick in SPECCPU 2017 on Neoverse V2.
Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.
Ok for master?
Thanks,
Tamar
gcc/ChangeLog:
* config/aarch64/aarch64-simd.md (vec_unpack<su>_lo_<mode,
vec_unpack<su>_lo_<mode): Split into...
(vec_unpacku_lo_<mode, vec_unpacks_lo_<mode,
vec_unpacku_lo_<mode, vec_unpacks_lo_<mode): ...These.
(aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
(aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
* config/aarch64/iterators.md (PERM_EXTEND, perm_index): New.
(perm_hilo): Add UNSPEC_ZIP1, UNSPEC_ZIP2.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/simd/vmovl_high_1.c: Update codegen.
* gcc.target/aarch64/uxtl-combine-1.c: New test.
* gcc.target/aarch64/uxtl-combine-2.c: New test.
* gcc.target/aarch64/uxtl-combine-3.c: New test.
* gcc.target/aarch64/uxtl-combine-4.c: New test.
* gcc.target/aarch64/uxtl-combine-5.c: New test.
* gcc.target/aarch64/uxtl-combine-6.c: New test.
--- inline copy of patch --
diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md
index 81ff5bad03d598fa0d48df93d172a28bc0d1d92e..3d811007dd94dcd9176d6021a41a196c12fe9c3f 100644
--- a/gcc/config/aarch64/aarch64-simd.md
+++ b/gcc/config/aarch64/aarch64-simd.md
@@ -1988,26 +1988,60 @@ (define_insn "aarch64_simd_vec_unpack<su>_hi_<mode>"
[(set_attr "type" "neon_shift_imm_long")]
)
-(define_expand "vec_unpack<su>_hi_<mode>"
+(define_expand "vec_unpacku_hi_<mode>"
[(match_operand:<VWIDE> 0 "register_operand")
- (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
+ (match_operand:VQW 1 "register_operand")]
+ "TARGET_SIMD"
+ {
+ rtx res = gen_reg_rtx (<MODE>mode);
+ rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
+ if (BYTES_BIG_ENDIAN)
+ emit_insn (gen_aarch64_zip2<mode> (res, tmp, operands[1]));
+ else
+ emit_insn (gen_aarch64_zip2<mode> (res, operands[1], tmp));
+ emit_move_insn (operands[0],
+ simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
+ DONE;
+ }
+)
+
+(define_expand "vec_unpacks_hi_<mode>"
+ [(match_operand:<VWIDE> 0 "register_operand")
+ (match_operand:VQW 1 "register_operand")]
"TARGET_SIMD"
{
rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, true);
- emit_insn (gen_aarch64_simd_vec_unpack<su>_hi_<mode> (operands[0],
- operands[1], p));
+ emit_insn (gen_aarch64_simd_vec_unpacks_hi_<mode> (operands[0],
+ operands[1], p));
+ DONE;
+ }
+)
+
+(define_expand "vec_unpacku_lo_<mode>"
+ [(match_operand:<VWIDE> 0 "register_operand")
+ (match_operand:VQW 1 "register_operand")]
+ "TARGET_SIMD"
+ {
+ rtx res = gen_reg_rtx (<MODE>mode);
+ rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
+ if (BYTES_BIG_ENDIAN)
+ emit_insn (gen_aarch64_zip1<mode> (res, tmp, operands[1]));
+ else
+ emit_insn (gen_aarch64_zip1<mode> (res, operands[1], tmp));
+ emit_move_insn (operands[0],
+ simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
DONE;
}
)
-(define_expand "vec_unpack<su>_lo_<mode>"
+(define_expand "vec_unpacks_lo_<mode>"
[(match_operand:<VWIDE> 0 "register_operand")
- (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
+ (match_operand:VQW 1 "register_operand")]
"TARGET_SIMD"
{
rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, false);
- emit_insn (gen_aarch64_simd_vec_unpack<su>_lo_<mode> (operands[0],
- operands[1], p));
+ emit_insn (gen_aarch64_simd_vec_unpacks_lo_<mode> (operands[0],
+ operands[1], p));
DONE;
}
)
@@ -4735,6 +4769,34 @@ (define_insn "aarch64_<ANY_EXTEND:su>subw2<mode>_internal"
[(set_attr "type" "neon_sub_widen")]
)
+(define_insn "aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip"
+ [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
+ (minus:<VWIDE>
+ (match_operand:<VWIDE> 1 "register_operand" "w")
+ (subreg:<VWIDE>
+ (unspec:<MODE> [
+ (match_operand:VQW 2 "register_operand" "w")
+ (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
+ ] PERM_EXTEND) 0)))]
+ "TARGET_SIMD"
+ "usubw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
+ [(set_attr "type" "neon_sub_widen")]
+)
+
+(define_insn "aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip"
+ [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
+ (plus:<VWIDE>
+ (subreg:<VWIDE>
+ (unspec:<MODE> [
+ (match_operand:VQW 2 "register_operand" "w")
+ (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
+ ] PERM_EXTEND) 0)
+ (match_operand:<VWIDE> 1 "register_operand" "w")))]
+ "TARGET_SIMD"
+ "uaddw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
+ [(set_attr "type" "neon_add_widen")]
+)
+
(define_insn "aarch64_<ANY_EXTEND:su>addw<mode>"
[(set (match_operand:<VWIDE> 0 "register_operand" "=w")
(plus:<VWIDE>
diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
index f9e2210095ea9d6d9c96971222a7757a2f418c2d..de281671aedc5141c69063f14cf0fbec5adecb04 100644
--- a/gcc/config/aarch64/iterators.md
+++ b/gcc/config/aarch64/iterators.md
@@ -2674,6 +2674,9 @@ (define_int_iterator PERMUTEQ [UNSPEC_ZIP1Q UNSPEC_ZIP2Q
(define_int_iterator OPTAB_PERMUTE [UNSPEC_ZIP1 UNSPEC_ZIP2
UNSPEC_UZP1 UNSPEC_UZP2])
+;; Permutes for zero extends
+(define_int_iterator PERM_EXTEND [UNSPEC_ZIP1 UNSPEC_ZIP2])
+
(define_int_iterator REVERSE [UNSPEC_REV64 UNSPEC_REV32 UNSPEC_REV16])
(define_int_iterator FRINT [UNSPEC_FRINTZ UNSPEC_FRINTP UNSPEC_FRINTM
@@ -3496,7 +3499,10 @@ (define_int_attr rev_op [(UNSPEC_REV64 "64") (UNSPEC_REV32 "32")
(UNSPEC_REV16 "16")])
(define_int_attr perm_hilo [(UNSPEC_UNPACKSHI "hi") (UNSPEC_UNPACKUHI "hi")
- (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")])
+ (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")
+ (UNSPEC_ZIP2 "hi") (UNSPEC_ZIP1 "lo")])
+
+(define_int_attr perm_index [(UNSPEC_ZIP2 "2") (UNSPEC_ZIP1 "")])
;; Return true if the associated optab refers to the high-numbered lanes,
;; false if it refers to the low-numbered lanes. The convention is for
diff --git a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
index d45bb83e3503d512c443f37a446d30d188719a96..a2d09eaee0de5a3d3409330c5c26a3b5315e84eb 100644
--- a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
+++ b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
@@ -22,11 +22,11 @@ FUNC (int32x4_t, int64x2_t, s32)
/* { dg-final { scan-assembler-times {sxtl2\tv0\.2d, v0\.4s} 1} } */
FUNC (uint8x16_t, uint16x8_t, u8)
-/* { dg-final { scan-assembler-times {uxtl2\tv0\.8h, v0\.16b} 1} } */
+/* { dg-final { scan-assembler-times {zip2\tv0\.16b, v0\.16b} 1} } */
FUNC (uint16x8_t, uint32x4_t, u16)
-/* { dg-final { scan-assembler-times {uxtl2\tv0\.4s, v0\.8h} 1} } */
+/* { dg-final { scan-assembler-times {zip2\tv0\.8h, v0\.8h} 1} } */
FUNC (uint32x4_t, uint64x2_t, u32)
-/* { dg-final { scan-assembler-times {uxtl2\tv0\.2d, v0\.4s} 1} } */
+/* { dg-final { scan-assembler-times {zip2\tv0\.4s, v0\.4s} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
new file mode 100755
index 0000000000000000000000000000000000000000..68fa9a09fe55f5a72355e23c90e781a898c5975e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN unsigned
+#define TYPE1 char
+#define TYPE2 short
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
+/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
+/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
+/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
new file mode 100755
index 0000000000000000000000000000000000000000..af8a89085cfca800b41970a8410bc91b84a31d07
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN unsigned
+#define TYPE1 short
+#define TYPE2 int
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
+/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
+/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
+/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
new file mode 100755
index 0000000000000000000000000000000000000000..cdae6d09529b743857a092f53a07111df64775d7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN unsigned
+#define TYPE1 int
+#define TYPE2 long long
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
+/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
+/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
+/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
new file mode 100755
index 0000000000000000000000000000000000000000..e1a9c4f5661a36ec7b2c5dc6f0fd85c42fcaac39
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN signed
+#define TYPE1 char
+#define TYPE2 short
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-not {\tzip1\t} } } */
+/* { dg-final { scan-assembler-not {\tzip2\t} } } */
+/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
+/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
new file mode 100755
index 0000000000000000000000000000000000000000..92b09ba4abba80f240ac175be2ef880968534975
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN signed
+#define TYPE1 short
+#define TYPE2 int
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-not {\tzip1\t} } } */
+/* { dg-final { scan-assembler-not {\tzip2\t} } } */
+/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
+/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
new file mode 100755
index 0000000000000000000000000000000000000000..5c6e635f29d1e52f51f5b75a477f7d8744f32ca3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN signed
+#define TYPE1 int
+#define TYPE2 long long
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-not {\tzip1\t} } } */
+/* { dg-final { scan-assembler-not {\tzip2\t} } } */
+/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
+/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
+
--
[-- Attachment #2: rb15000.patch --]
[-- Type: text/plain, Size: 11268 bytes --]
diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md
index 81ff5bad03d598fa0d48df93d172a28bc0d1d92e..3d811007dd94dcd9176d6021a41a196c12fe9c3f 100644
--- a/gcc/config/aarch64/aarch64-simd.md
+++ b/gcc/config/aarch64/aarch64-simd.md
@@ -1988,26 +1988,60 @@ (define_insn "aarch64_simd_vec_unpack<su>_hi_<mode>"
[(set_attr "type" "neon_shift_imm_long")]
)
-(define_expand "vec_unpack<su>_hi_<mode>"
+(define_expand "vec_unpacku_hi_<mode>"
[(match_operand:<VWIDE> 0 "register_operand")
- (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
+ (match_operand:VQW 1 "register_operand")]
+ "TARGET_SIMD"
+ {
+ rtx res = gen_reg_rtx (<MODE>mode);
+ rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
+ if (BYTES_BIG_ENDIAN)
+ emit_insn (gen_aarch64_zip2<mode> (res, tmp, operands[1]));
+ else
+ emit_insn (gen_aarch64_zip2<mode> (res, operands[1], tmp));
+ emit_move_insn (operands[0],
+ simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
+ DONE;
+ }
+)
+
+(define_expand "vec_unpacks_hi_<mode>"
+ [(match_operand:<VWIDE> 0 "register_operand")
+ (match_operand:VQW 1 "register_operand")]
"TARGET_SIMD"
{
rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, true);
- emit_insn (gen_aarch64_simd_vec_unpack<su>_hi_<mode> (operands[0],
- operands[1], p));
+ emit_insn (gen_aarch64_simd_vec_unpacks_hi_<mode> (operands[0],
+ operands[1], p));
+ DONE;
+ }
+)
+
+(define_expand "vec_unpacku_lo_<mode>"
+ [(match_operand:<VWIDE> 0 "register_operand")
+ (match_operand:VQW 1 "register_operand")]
+ "TARGET_SIMD"
+ {
+ rtx res = gen_reg_rtx (<MODE>mode);
+ rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
+ if (BYTES_BIG_ENDIAN)
+ emit_insn (gen_aarch64_zip1<mode> (res, tmp, operands[1]));
+ else
+ emit_insn (gen_aarch64_zip1<mode> (res, operands[1], tmp));
+ emit_move_insn (operands[0],
+ simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
DONE;
}
)
-(define_expand "vec_unpack<su>_lo_<mode>"
+(define_expand "vec_unpacks_lo_<mode>"
[(match_operand:<VWIDE> 0 "register_operand")
- (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
+ (match_operand:VQW 1 "register_operand")]
"TARGET_SIMD"
{
rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, false);
- emit_insn (gen_aarch64_simd_vec_unpack<su>_lo_<mode> (operands[0],
- operands[1], p));
+ emit_insn (gen_aarch64_simd_vec_unpacks_lo_<mode> (operands[0],
+ operands[1], p));
DONE;
}
)
@@ -4735,6 +4769,34 @@ (define_insn "aarch64_<ANY_EXTEND:su>subw2<mode>_internal"
[(set_attr "type" "neon_sub_widen")]
)
+(define_insn "aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip"
+ [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
+ (minus:<VWIDE>
+ (match_operand:<VWIDE> 1 "register_operand" "w")
+ (subreg:<VWIDE>
+ (unspec:<MODE> [
+ (match_operand:VQW 2 "register_operand" "w")
+ (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
+ ] PERM_EXTEND) 0)))]
+ "TARGET_SIMD"
+ "usubw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
+ [(set_attr "type" "neon_sub_widen")]
+)
+
+(define_insn "aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip"
+ [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
+ (plus:<VWIDE>
+ (subreg:<VWIDE>
+ (unspec:<MODE> [
+ (match_operand:VQW 2 "register_operand" "w")
+ (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
+ ] PERM_EXTEND) 0)
+ (match_operand:<VWIDE> 1 "register_operand" "w")))]
+ "TARGET_SIMD"
+ "uaddw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
+ [(set_attr "type" "neon_add_widen")]
+)
+
(define_insn "aarch64_<ANY_EXTEND:su>addw<mode>"
[(set (match_operand:<VWIDE> 0 "register_operand" "=w")
(plus:<VWIDE>
diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
index f9e2210095ea9d6d9c96971222a7757a2f418c2d..de281671aedc5141c69063f14cf0fbec5adecb04 100644
--- a/gcc/config/aarch64/iterators.md
+++ b/gcc/config/aarch64/iterators.md
@@ -2674,6 +2674,9 @@ (define_int_iterator PERMUTEQ [UNSPEC_ZIP1Q UNSPEC_ZIP2Q
(define_int_iterator OPTAB_PERMUTE [UNSPEC_ZIP1 UNSPEC_ZIP2
UNSPEC_UZP1 UNSPEC_UZP2])
+;; Permutes for zero extends
+(define_int_iterator PERM_EXTEND [UNSPEC_ZIP1 UNSPEC_ZIP2])
+
(define_int_iterator REVERSE [UNSPEC_REV64 UNSPEC_REV32 UNSPEC_REV16])
(define_int_iterator FRINT [UNSPEC_FRINTZ UNSPEC_FRINTP UNSPEC_FRINTM
@@ -3496,7 +3499,10 @@ (define_int_attr rev_op [(UNSPEC_REV64 "64") (UNSPEC_REV32 "32")
(UNSPEC_REV16 "16")])
(define_int_attr perm_hilo [(UNSPEC_UNPACKSHI "hi") (UNSPEC_UNPACKUHI "hi")
- (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")])
+ (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")
+ (UNSPEC_ZIP2 "hi") (UNSPEC_ZIP1 "lo")])
+
+(define_int_attr perm_index [(UNSPEC_ZIP2 "2") (UNSPEC_ZIP1 "")])
;; Return true if the associated optab refers to the high-numbered lanes,
;; false if it refers to the low-numbered lanes. The convention is for
diff --git a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
index d45bb83e3503d512c443f37a446d30d188719a96..a2d09eaee0de5a3d3409330c5c26a3b5315e84eb 100644
--- a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
+++ b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
@@ -22,11 +22,11 @@ FUNC (int32x4_t, int64x2_t, s32)
/* { dg-final { scan-assembler-times {sxtl2\tv0\.2d, v0\.4s} 1} } */
FUNC (uint8x16_t, uint16x8_t, u8)
-/* { dg-final { scan-assembler-times {uxtl2\tv0\.8h, v0\.16b} 1} } */
+/* { dg-final { scan-assembler-times {zip2\tv0\.16b, v0\.16b} 1} } */
FUNC (uint16x8_t, uint32x4_t, u16)
-/* { dg-final { scan-assembler-times {uxtl2\tv0\.4s, v0\.8h} 1} } */
+/* { dg-final { scan-assembler-times {zip2\tv0\.8h, v0\.8h} 1} } */
FUNC (uint32x4_t, uint64x2_t, u32)
-/* { dg-final { scan-assembler-times {uxtl2\tv0\.2d, v0\.4s} 1} } */
+/* { dg-final { scan-assembler-times {zip2\tv0\.4s, v0\.4s} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
new file mode 100755
index 0000000000000000000000000000000000000000..68fa9a09fe55f5a72355e23c90e781a898c5975e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN unsigned
+#define TYPE1 char
+#define TYPE2 short
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
+/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
+/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
+/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
new file mode 100755
index 0000000000000000000000000000000000000000..af8a89085cfca800b41970a8410bc91b84a31d07
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN unsigned
+#define TYPE1 short
+#define TYPE2 int
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
+/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
+/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
+/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
new file mode 100755
index 0000000000000000000000000000000000000000..cdae6d09529b743857a092f53a07111df64775d7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN unsigned
+#define TYPE1 int
+#define TYPE2 long long
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
+/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
+/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
+/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
new file mode 100755
index 0000000000000000000000000000000000000000..e1a9c4f5661a36ec7b2c5dc6f0fd85c42fcaac39
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN signed
+#define TYPE1 char
+#define TYPE2 short
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-not {\tzip1\t} } } */
+/* { dg-final { scan-assembler-not {\tzip2\t} } } */
+/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
+/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
new file mode 100755
index 0000000000000000000000000000000000000000..92b09ba4abba80f240ac175be2ef880968534975
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN signed
+#define TYPE1 short
+#define TYPE2 int
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-not {\tzip1\t} } } */
+/* { dg-final { scan-assembler-not {\tzip2\t} } } */
+/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
+/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
+
diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
new file mode 100755
index 0000000000000000000000000000000000000000..5c6e635f29d1e52f51f5b75a477f7d8744f32ca3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
@@ -0,0 +1,20 @@
+/* { dg-do assemble } */
+/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
+
+#pragma GCC target "+nosve"
+
+#define SIGN signed
+#define TYPE1 int
+#define TYPE2 long long
+
+void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
+{
+ for (int i = 0; i < (n & -8); i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-assembler-not {\tzip1\t} } } */
+/* { dg-final { scan-assembler-not {\tzip2\t} } } */
+/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
+/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
+
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2}
2023-11-15 14:41 [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2} Tamar Christina
@ 2023-11-21 10:39 ` Richard Sandiford
2023-11-21 22:43 ` Andrew Pinski
1 sibling, 0 replies; 4+ messages in thread
From: Richard Sandiford @ 2023-11-21 10:39 UTC (permalink / raw)
To: Tamar Christina
Cc: gcc-patches, nd, Richard.Earnshaw, Marcus.Shawcroft, Kyrylo.Tkachov
Tamar Christina <tamar.christina@arm.com> writes:
> Hi All,
>
> This changes unpack instructions to use zip{1,2} when doing a zero-extending
> widening operation. Permutes generally have a higher throughput than the
> widening operations. Zeros are shuffled into the top half of the registers.
>
> The testcase
>
> void d2 (unsigned * restrict a, unsigned short *b, int n)
> {
> for (int i = 0; i < (n & -8); i++)
> a[i] = b[i];
> }
>
> now generates:
>
> movi v1.4s, 0
> .L3:
> ldr q0, [x1], 16
> zip1 v2.8h, v0.8h, v1.8h
> zip2 v0.8h, v0.8h, v1.8h
> stp q2, q0, [x0]
> add x0, x0, 32
> cmp x1, x2
> bne .L3
>
>
> instead of:
>
> .L3:
> ldr q0, [x1], 16
> uxtl v1.4s, v0.4h
> uxtl2 v0.4s, v0.8h
> stp q1, q0, [x0]
> add x0, x0, 32
> cmp x1, x2
> bne .L3
>
> Since we need the extra 0 register we do this only for the vectorizer's lo/hi
> pairs when we know the 0 will be floated outside of the loop.
The patterns are used by BB SLP as well, so we don't know for certain
that there's a containing loop.
We could provide patterns that match zips with zero, to allow the
zero to be combined back into the pair if the zero doesn't get hoisted.
The zip-with-zero patterns would need to have a higher cost than plain zips,
so that 2 zips + movi has the same cost as 2 zips with zero. I guess
that means that the zips with zero should have a cost of 1.5 insns.
The late combine pass would then be able to get rid of the zero
if it is stuck in the same basic block (or a block with the same
execution frequency), but would keep a hoisted zero.
But if we do one thing all the time or another thing all the time,
I agree it's better to do the zips all the time.
If this turns out to be bad for non-Neoverse cores, we can add some sort
of tuning flag. But I agree that we should use zips unconditionally
until we know of a specific core that doesn't want it.
> This gives an 8% speed-up in Imagick in SPECCPU 2017 on Neoverse V2.
>
> Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.
>
> Ok for master?
>
> Thanks,
> Tamar
>
> gcc/ChangeLog:
>
> * config/aarch64/aarch64-simd.md (vec_unpack<su>_lo_<mode,
> vec_unpack<su>_lo_<mode): Split into...
> (vec_unpacku_lo_<mode, vec_unpacks_lo_<mode,
> vec_unpacku_lo_<mode, vec_unpacks_lo_<mode): ...These.
> (aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
> (aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
> * config/aarch64/iterators.md (PERM_EXTEND, perm_index): New.
> (perm_hilo): Add UNSPEC_ZIP1, UNSPEC_ZIP2.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/aarch64/simd/vmovl_high_1.c: Update codegen.
> * gcc.target/aarch64/uxtl-combine-1.c: New test.
> * gcc.target/aarch64/uxtl-combine-2.c: New test.
> * gcc.target/aarch64/uxtl-combine-3.c: New test.
> * gcc.target/aarch64/uxtl-combine-4.c: New test.
> * gcc.target/aarch64/uxtl-combine-5.c: New test.
> * gcc.target/aarch64/uxtl-combine-6.c: New test.
>
> --- inline copy of patch --
> diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md
> index 81ff5bad03d598fa0d48df93d172a28bc0d1d92e..3d811007dd94dcd9176d6021a41a196c12fe9c3f 100644
> --- a/gcc/config/aarch64/aarch64-simd.md
> +++ b/gcc/config/aarch64/aarch64-simd.md
> @@ -1988,26 +1988,60 @@ (define_insn "aarch64_simd_vec_unpack<su>_hi_<mode>"
> [(set_attr "type" "neon_shift_imm_long")]
> )
>
> -(define_expand "vec_unpack<su>_hi_<mode>"
> +(define_expand "vec_unpacku_hi_<mode>"
> [(match_operand:<VWIDE> 0 "register_operand")
> - (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
> + (match_operand:VQW 1 "register_operand")]
> + "TARGET_SIMD"
> + {
> + rtx res = gen_reg_rtx (<MODE>mode);
> + rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
> + if (BYTES_BIG_ENDIAN)
> + emit_insn (gen_aarch64_zip2<mode> (res, tmp, operands[1]));
> + else
> + emit_insn (gen_aarch64_zip2<mode> (res, operands[1], tmp));
> + emit_move_insn (operands[0],
> + simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
> + DONE;
> + }
> +)
> +
> +(define_expand "vec_unpacks_hi_<mode>"
> + [(match_operand:<VWIDE> 0 "register_operand")
> + (match_operand:VQW 1 "register_operand")]
> "TARGET_SIMD"
> {
> rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, true);
> - emit_insn (gen_aarch64_simd_vec_unpack<su>_hi_<mode> (operands[0],
> - operands[1], p));
> + emit_insn (gen_aarch64_simd_vec_unpacks_hi_<mode> (operands[0],
> + operands[1], p));
> + DONE;
> + }
> +)
> +
> +(define_expand "vec_unpacku_lo_<mode>"
> + [(match_operand:<VWIDE> 0 "register_operand")
> + (match_operand:VQW 1 "register_operand")]
> + "TARGET_SIMD"
> + {
> + rtx res = gen_reg_rtx (<MODE>mode);
> + rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
> + if (BYTES_BIG_ENDIAN)
> + emit_insn (gen_aarch64_zip1<mode> (res, tmp, operands[1]));
> + else
> + emit_insn (gen_aarch64_zip1<mode> (res, operands[1], tmp));
> + emit_move_insn (operands[0],
> + simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
> DONE;
> }
> )
>
> -(define_expand "vec_unpack<su>_lo_<mode>"
> +(define_expand "vec_unpacks_lo_<mode>"
> [(match_operand:<VWIDE> 0 "register_operand")
> - (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
> + (match_operand:VQW 1 "register_operand")]
> "TARGET_SIMD"
> {
> rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, false);
> - emit_insn (gen_aarch64_simd_vec_unpack<su>_lo_<mode> (operands[0],
> - operands[1], p));
> + emit_insn (gen_aarch64_simd_vec_unpacks_lo_<mode> (operands[0],
> + operands[1], p));
> DONE;
> }
> )
> @@ -4735,6 +4769,34 @@ (define_insn "aarch64_<ANY_EXTEND:su>subw2<mode>_internal"
> [(set_attr "type" "neon_sub_widen")]
> )
>
> +(define_insn "aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip"
> + [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> + (minus:<VWIDE>
> + (match_operand:<VWIDE> 1 "register_operand" "w")
> + (subreg:<VWIDE>
> + (unspec:<MODE> [
> + (match_operand:VQW 2 "register_operand" "w")
> + (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
IMO it's better to leave the Dz constraint out. The predicate does
all the work here.
OK with that change, thanks.
Richard
> + ] PERM_EXTEND) 0)))]
> + "TARGET_SIMD"
> + "usubw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
> + [(set_attr "type" "neon_sub_widen")]
> +)
> +
> +(define_insn "aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip"
> + [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> + (plus:<VWIDE>
> + (subreg:<VWIDE>
> + (unspec:<MODE> [
> + (match_operand:VQW 2 "register_operand" "w")
> + (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
> + ] PERM_EXTEND) 0)
> + (match_operand:<VWIDE> 1 "register_operand" "w")))]
> + "TARGET_SIMD"
> + "uaddw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
> + [(set_attr "type" "neon_add_widen")]
> +)
> +
> (define_insn "aarch64_<ANY_EXTEND:su>addw<mode>"
> [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> (plus:<VWIDE>
> diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
> index f9e2210095ea9d6d9c96971222a7757a2f418c2d..de281671aedc5141c69063f14cf0fbec5adecb04 100644
> --- a/gcc/config/aarch64/iterators.md
> +++ b/gcc/config/aarch64/iterators.md
> @@ -2674,6 +2674,9 @@ (define_int_iterator PERMUTEQ [UNSPEC_ZIP1Q UNSPEC_ZIP2Q
> (define_int_iterator OPTAB_PERMUTE [UNSPEC_ZIP1 UNSPEC_ZIP2
> UNSPEC_UZP1 UNSPEC_UZP2])
>
> +;; Permutes for zero extends
> +(define_int_iterator PERM_EXTEND [UNSPEC_ZIP1 UNSPEC_ZIP2])
> +
> (define_int_iterator REVERSE [UNSPEC_REV64 UNSPEC_REV32 UNSPEC_REV16])
>
> (define_int_iterator FRINT [UNSPEC_FRINTZ UNSPEC_FRINTP UNSPEC_FRINTM
> @@ -3496,7 +3499,10 @@ (define_int_attr rev_op [(UNSPEC_REV64 "64") (UNSPEC_REV32 "32")
> (UNSPEC_REV16 "16")])
>
> (define_int_attr perm_hilo [(UNSPEC_UNPACKSHI "hi") (UNSPEC_UNPACKUHI "hi")
> - (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")])
> + (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")
> + (UNSPEC_ZIP2 "hi") (UNSPEC_ZIP1 "lo")])
> +
> +(define_int_attr perm_index [(UNSPEC_ZIP2 "2") (UNSPEC_ZIP1 "")])
>
> ;; Return true if the associated optab refers to the high-numbered lanes,
> ;; false if it refers to the low-numbered lanes. The convention is for
> diff --git a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> index d45bb83e3503d512c443f37a446d30d188719a96..a2d09eaee0de5a3d3409330c5c26a3b5315e84eb 100644
> --- a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> +++ b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> @@ -22,11 +22,11 @@ FUNC (int32x4_t, int64x2_t, s32)
> /* { dg-final { scan-assembler-times {sxtl2\tv0\.2d, v0\.4s} 1} } */
>
> FUNC (uint8x16_t, uint16x8_t, u8)
> -/* { dg-final { scan-assembler-times {uxtl2\tv0\.8h, v0\.16b} 1} } */
> +/* { dg-final { scan-assembler-times {zip2\tv0\.16b, v0\.16b} 1} } */
>
> FUNC (uint16x8_t, uint32x4_t, u16)
> -/* { dg-final { scan-assembler-times {uxtl2\tv0\.4s, v0\.8h} 1} } */
> +/* { dg-final { scan-assembler-times {zip2\tv0\.8h, v0\.8h} 1} } */
>
> FUNC (uint32x4_t, uint64x2_t, u32)
> -/* { dg-final { scan-assembler-times {uxtl2\tv0\.2d, v0\.4s} 1} } */
> +/* { dg-final { scan-assembler-times {zip2\tv0\.4s, v0\.4s} 1} } */
>
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..68fa9a09fe55f5a72355e23c90e781a898c5975e
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN unsigned
> +#define TYPE1 char
> +#define TYPE2 short
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..af8a89085cfca800b41970a8410bc91b84a31d07
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN unsigned
> +#define TYPE1 short
> +#define TYPE2 int
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..cdae6d09529b743857a092f53a07111df64775d7
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN unsigned
> +#define TYPE1 int
> +#define TYPE2 long long
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..e1a9c4f5661a36ec7b2c5dc6f0fd85c42fcaac39
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN signed
> +#define TYPE1 char
> +#define TYPE2 short
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..92b09ba4abba80f240ac175be2ef880968534975
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN signed
> +#define TYPE1 short
> +#define TYPE2 int
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..5c6e635f29d1e52f51f5b75a477f7d8744f32ca3
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN signed
> +#define TYPE1 int
> +#define TYPE2 long long
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> +
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2}
2023-11-15 14:41 [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2} Tamar Christina
2023-11-21 10:39 ` Richard Sandiford
@ 2023-11-21 22:43 ` Andrew Pinski
2023-11-21 23:05 ` Andrew Pinski
1 sibling, 1 reply; 4+ messages in thread
From: Andrew Pinski @ 2023-11-21 22:43 UTC (permalink / raw)
To: Tamar Christina
Cc: gcc-patches, nd, Richard.Earnshaw, Marcus.Shawcroft,
Kyrylo.Tkachov, richard.sandiford, Andrew Pinski (QUIC)
On Wed, Nov 15, 2023 at 6:42 AM Tamar Christina <tamar.christina@arm.com> wrote:
>
> Hi All,
>
> This changes unpack instructions to use zip{1,2} when doing a zero-extending
> widening operation. Permutes generally have a higher throughput than the
> widening operations. Zeros are shuffled into the top half of the registers.
>
> The testcase
>
> void d2 (unsigned * restrict a, unsigned short *b, int n)
> {
> for (int i = 0; i < (n & -8); i++)
> a[i] = b[i];
> }
>
> now generates:
>
> movi v1.4s, 0
> .L3:
> ldr q0, [x1], 16
> zip1 v2.8h, v0.8h, v1.8h
> zip2 v0.8h, v0.8h, v1.8h
> stp q2, q0, [x0]
> add x0, x0, 32
> cmp x1, x2
> bne .L3
>
>
> instead of:
>
> .L3:
> ldr q0, [x1], 16
> uxtl v1.4s, v0.4h
> uxtl2 v0.4s, v0.8h
> stp q1, q0, [x0]
> add x0, x0, 32
> cmp x1, x2
> bne .L3
>
> Since we need the extra 0 register we do this only for the vectorizer's lo/hi
> pairs when we know the 0 will be floated outside of the loop.
>
> This gives an 8% speed-up in Imagick in SPECCPU 2017 on Neoverse V2.
>
> Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.
>
> Ok for master?
>
> Thanks,
> Tamar
>
> gcc/ChangeLog:
>
> * config/aarch64/aarch64-simd.md (vec_unpack<su>_lo_<mode,
> vec_unpack<su>_lo_<mode): Split into...
> (vec_unpacku_lo_<mode, vec_unpacks_lo_<mode,
> vec_unpacku_lo_<mode, vec_unpacks_lo_<mode): ...These.
> (aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
> (aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
> * config/aarch64/iterators.md (PERM_EXTEND, perm_index): New.
> (perm_hilo): Add UNSPEC_ZIP1, UNSPEC_ZIP2.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/aarch64/simd/vmovl_high_1.c: Update codegen.
> * gcc.target/aarch64/uxtl-combine-1.c: New test.
> * gcc.target/aarch64/uxtl-combine-2.c: New test.
> * gcc.target/aarch64/uxtl-combine-3.c: New test.
> * gcc.target/aarch64/uxtl-combine-4.c: New test.
> * gcc.target/aarch64/uxtl-combine-5.c: New test.
> * gcc.target/aarch64/uxtl-combine-6.c: New test.
You have a few typos in the testcases which causes:
ERROR: gcc.target/aarch64/uxtl-combine-4.c: error executing dg-final:
invalid command name "scan-assembler-time"
ERROR: gcc.target/aarch64/uxtl-combine-5.c: error executing dg-final:
invalid command name "scan-assembler-time"
ERROR: gcc.target/aarch64/uxtl-combine-6.c: error executing dg-final:
invalid command name "scan-assembler-time"
They all should be `scan-assembler-times`.
Thanks,
Andrew
>
> --- inline copy of patch --
> diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md
> index 81ff5bad03d598fa0d48df93d172a28bc0d1d92e..3d811007dd94dcd9176d6021a41a196c12fe9c3f 100644
> --- a/gcc/config/aarch64/aarch64-simd.md
> +++ b/gcc/config/aarch64/aarch64-simd.md
> @@ -1988,26 +1988,60 @@ (define_insn "aarch64_simd_vec_unpack<su>_hi_<mode>"
> [(set_attr "type" "neon_shift_imm_long")]
> )
>
> -(define_expand "vec_unpack<su>_hi_<mode>"
> +(define_expand "vec_unpacku_hi_<mode>"
> [(match_operand:<VWIDE> 0 "register_operand")
> - (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
> + (match_operand:VQW 1 "register_operand")]
> + "TARGET_SIMD"
> + {
> + rtx res = gen_reg_rtx (<MODE>mode);
> + rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
> + if (BYTES_BIG_ENDIAN)
> + emit_insn (gen_aarch64_zip2<mode> (res, tmp, operands[1]));
> + else
> + emit_insn (gen_aarch64_zip2<mode> (res, operands[1], tmp));
> + emit_move_insn (operands[0],
> + simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
> + DONE;
> + }
> +)
> +
> +(define_expand "vec_unpacks_hi_<mode>"
> + [(match_operand:<VWIDE> 0 "register_operand")
> + (match_operand:VQW 1 "register_operand")]
> "TARGET_SIMD"
> {
> rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, true);
> - emit_insn (gen_aarch64_simd_vec_unpack<su>_hi_<mode> (operands[0],
> - operands[1], p));
> + emit_insn (gen_aarch64_simd_vec_unpacks_hi_<mode> (operands[0],
> + operands[1], p));
> + DONE;
> + }
> +)
> +
> +(define_expand "vec_unpacku_lo_<mode>"
> + [(match_operand:<VWIDE> 0 "register_operand")
> + (match_operand:VQW 1 "register_operand")]
> + "TARGET_SIMD"
> + {
> + rtx res = gen_reg_rtx (<MODE>mode);
> + rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
> + if (BYTES_BIG_ENDIAN)
> + emit_insn (gen_aarch64_zip1<mode> (res, tmp, operands[1]));
> + else
> + emit_insn (gen_aarch64_zip1<mode> (res, operands[1], tmp));
> + emit_move_insn (operands[0],
> + simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
> DONE;
> }
> )
>
> -(define_expand "vec_unpack<su>_lo_<mode>"
> +(define_expand "vec_unpacks_lo_<mode>"
> [(match_operand:<VWIDE> 0 "register_operand")
> - (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
> + (match_operand:VQW 1 "register_operand")]
> "TARGET_SIMD"
> {
> rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, false);
> - emit_insn (gen_aarch64_simd_vec_unpack<su>_lo_<mode> (operands[0],
> - operands[1], p));
> + emit_insn (gen_aarch64_simd_vec_unpacks_lo_<mode> (operands[0],
> + operands[1], p));
> DONE;
> }
> )
> @@ -4735,6 +4769,34 @@ (define_insn "aarch64_<ANY_EXTEND:su>subw2<mode>_internal"
> [(set_attr "type" "neon_sub_widen")]
> )
>
> +(define_insn "aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip"
> + [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> + (minus:<VWIDE>
> + (match_operand:<VWIDE> 1 "register_operand" "w")
> + (subreg:<VWIDE>
> + (unspec:<MODE> [
> + (match_operand:VQW 2 "register_operand" "w")
> + (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
> + ] PERM_EXTEND) 0)))]
> + "TARGET_SIMD"
> + "usubw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
> + [(set_attr "type" "neon_sub_widen")]
> +)
> +
> +(define_insn "aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip"
> + [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> + (plus:<VWIDE>
> + (subreg:<VWIDE>
> + (unspec:<MODE> [
> + (match_operand:VQW 2 "register_operand" "w")
> + (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
> + ] PERM_EXTEND) 0)
> + (match_operand:<VWIDE> 1 "register_operand" "w")))]
> + "TARGET_SIMD"
> + "uaddw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
> + [(set_attr "type" "neon_add_widen")]
> +)
> +
> (define_insn "aarch64_<ANY_EXTEND:su>addw<mode>"
> [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> (plus:<VWIDE>
> diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
> index f9e2210095ea9d6d9c96971222a7757a2f418c2d..de281671aedc5141c69063f14cf0fbec5adecb04 100644
> --- a/gcc/config/aarch64/iterators.md
> +++ b/gcc/config/aarch64/iterators.md
> @@ -2674,6 +2674,9 @@ (define_int_iterator PERMUTEQ [UNSPEC_ZIP1Q UNSPEC_ZIP2Q
> (define_int_iterator OPTAB_PERMUTE [UNSPEC_ZIP1 UNSPEC_ZIP2
> UNSPEC_UZP1 UNSPEC_UZP2])
>
> +;; Permutes for zero extends
> +(define_int_iterator PERM_EXTEND [UNSPEC_ZIP1 UNSPEC_ZIP2])
> +
> (define_int_iterator REVERSE [UNSPEC_REV64 UNSPEC_REV32 UNSPEC_REV16])
>
> (define_int_iterator FRINT [UNSPEC_FRINTZ UNSPEC_FRINTP UNSPEC_FRINTM
> @@ -3496,7 +3499,10 @@ (define_int_attr rev_op [(UNSPEC_REV64 "64") (UNSPEC_REV32 "32")
> (UNSPEC_REV16 "16")])
>
> (define_int_attr perm_hilo [(UNSPEC_UNPACKSHI "hi") (UNSPEC_UNPACKUHI "hi")
> - (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")])
> + (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")
> + (UNSPEC_ZIP2 "hi") (UNSPEC_ZIP1 "lo")])
> +
> +(define_int_attr perm_index [(UNSPEC_ZIP2 "2") (UNSPEC_ZIP1 "")])
>
> ;; Return true if the associated optab refers to the high-numbered lanes,
> ;; false if it refers to the low-numbered lanes. The convention is for
> diff --git a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> index d45bb83e3503d512c443f37a446d30d188719a96..a2d09eaee0de5a3d3409330c5c26a3b5315e84eb 100644
> --- a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> +++ b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> @@ -22,11 +22,11 @@ FUNC (int32x4_t, int64x2_t, s32)
> /* { dg-final { scan-assembler-times {sxtl2\tv0\.2d, v0\.4s} 1} } */
>
> FUNC (uint8x16_t, uint16x8_t, u8)
> -/* { dg-final { scan-assembler-times {uxtl2\tv0\.8h, v0\.16b} 1} } */
> +/* { dg-final { scan-assembler-times {zip2\tv0\.16b, v0\.16b} 1} } */
>
> FUNC (uint16x8_t, uint32x4_t, u16)
> -/* { dg-final { scan-assembler-times {uxtl2\tv0\.4s, v0\.8h} 1} } */
> +/* { dg-final { scan-assembler-times {zip2\tv0\.8h, v0\.8h} 1} } */
>
> FUNC (uint32x4_t, uint64x2_t, u32)
> -/* { dg-final { scan-assembler-times {uxtl2\tv0\.2d, v0\.4s} 1} } */
> +/* { dg-final { scan-assembler-times {zip2\tv0\.4s, v0\.4s} 1} } */
>
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..68fa9a09fe55f5a72355e23c90e781a898c5975e
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN unsigned
> +#define TYPE1 char
> +#define TYPE2 short
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..af8a89085cfca800b41970a8410bc91b84a31d07
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN unsigned
> +#define TYPE1 short
> +#define TYPE2 int
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..cdae6d09529b743857a092f53a07111df64775d7
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN unsigned
> +#define TYPE1 int
> +#define TYPE2 long long
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..e1a9c4f5661a36ec7b2c5dc6f0fd85c42fcaac39
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN signed
> +#define TYPE1 char
> +#define TYPE2 short
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..92b09ba4abba80f240ac175be2ef880968534975
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN signed
> +#define TYPE1 short
> +#define TYPE2 int
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> +
> diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
> new file mode 100755
> index 0000000000000000000000000000000000000000..5c6e635f29d1e52f51f5b75a477f7d8744f32ca3
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
> @@ -0,0 +1,20 @@
> +/* { dg-do assemble } */
> +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> +
> +#pragma GCC target "+nosve"
> +
> +#define SIGN signed
> +#define TYPE1 int
> +#define TYPE2 long long
> +
> +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> +{
> + for (int i = 0; i < (n & -8); i++)
> + a[i] = b[i];
> +}
> +
> +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> +
>
>
>
>
> --
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2}
2023-11-21 22:43 ` Andrew Pinski
@ 2023-11-21 23:05 ` Andrew Pinski
0 siblings, 0 replies; 4+ messages in thread
From: Andrew Pinski @ 2023-11-21 23:05 UTC (permalink / raw)
To: Tamar Christina
Cc: gcc-patches, nd, Richard.Earnshaw, Marcus.Shawcroft,
Kyrylo.Tkachov, richard.sandiford, Andrew Pinski (QUIC)
On Tue, Nov 21, 2023 at 2:43 PM Andrew Pinski <pinskia@gmail.com> wrote:
>
> On Wed, Nov 15, 2023 at 6:42 AM Tamar Christina <tamar.christina@arm.com> wrote:
> >
> > Hi All,
> >
> > This changes unpack instructions to use zip{1,2} when doing a zero-extending
> > widening operation. Permutes generally have a higher throughput than the
> > widening operations. Zeros are shuffled into the top half of the registers.
> >
> > The testcase
> >
> > void d2 (unsigned * restrict a, unsigned short *b, int n)
> > {
> > for (int i = 0; i < (n & -8); i++)
> > a[i] = b[i];
> > }
> >
> > now generates:
> >
> > movi v1.4s, 0
> > .L3:
> > ldr q0, [x1], 16
> > zip1 v2.8h, v0.8h, v1.8h
> > zip2 v0.8h, v0.8h, v1.8h
> > stp q2, q0, [x0]
> > add x0, x0, 32
> > cmp x1, x2
> > bne .L3
> >
> >
> > instead of:
> >
> > .L3:
> > ldr q0, [x1], 16
> > uxtl v1.4s, v0.4h
> > uxtl2 v0.4s, v0.8h
> > stp q1, q0, [x0]
> > add x0, x0, 32
> > cmp x1, x2
> > bne .L3
> >
> > Since we need the extra 0 register we do this only for the vectorizer's lo/hi
> > pairs when we know the 0 will be floated outside of the loop.
> >
> > This gives an 8% speed-up in Imagick in SPECCPU 2017 on Neoverse V2.
> >
> > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.
> >
> > Ok for master?
> >
> > Thanks,
> > Tamar
> >
> > gcc/ChangeLog:
> >
> > * config/aarch64/aarch64-simd.md (vec_unpack<su>_lo_<mode,
> > vec_unpack<su>_lo_<mode): Split into...
> > (vec_unpacku_lo_<mode, vec_unpacks_lo_<mode,
> > vec_unpacku_lo_<mode, vec_unpacks_lo_<mode): ...These.
> > (aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
> > (aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip): New.
> > * config/aarch64/iterators.md (PERM_EXTEND, perm_index): New.
> > (perm_hilo): Add UNSPEC_ZIP1, UNSPEC_ZIP2.
> >
> > gcc/testsuite/ChangeLog:
> >
> > * gcc.target/aarch64/simd/vmovl_high_1.c: Update codegen.
> > * gcc.target/aarch64/uxtl-combine-1.c: New test.
> > * gcc.target/aarch64/uxtl-combine-2.c: New test.
> > * gcc.target/aarch64/uxtl-combine-3.c: New test.
> > * gcc.target/aarch64/uxtl-combine-4.c: New test.
> > * gcc.target/aarch64/uxtl-combine-5.c: New test.
> > * gcc.target/aarch64/uxtl-combine-6.c: New test.
>
> You have a few typos in the testcases which causes:
> ERROR: gcc.target/aarch64/uxtl-combine-4.c: error executing dg-final:
> invalid command name "scan-assembler-time"
> ERROR: gcc.target/aarch64/uxtl-combine-5.c: error executing dg-final:
> invalid command name "scan-assembler-time"
> ERROR: gcc.target/aarch64/uxtl-combine-6.c: error executing dg-final:
> invalid command name "scan-assembler-time"
You must not have done some good testing since it also caused the following:
FAIL: gcc.dg/vect/slp-widen-mult-half.c (test for excess errors)
FAIL: gcc.dg/vect/slp-widen-mult-half.c -flto -ffat-lto-objects (test
for excess errors)
UNRESOLVED: gcc.dg/vect/slp-widen-mult-half.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/slp-widen-mult-half.c compilation failed to
produce executable
FAIL: gcc.dg/vect/vect-avg-9.c (test for excess errors)
FAIL: gcc.dg/vect/vect-avg-9.c -flto -ffat-lto-objects (test for excess errors)
UNRESOLVED: gcc.dg/vect/vect-avg-9.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-avg-9.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-outer-4f.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4f.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4f.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4f.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-outer-4g.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4g.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4g.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4g.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-outer-4i.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4i.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4i.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4i.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-outer-4k.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4k.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4k.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4k.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-outer-4l.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4l.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4l.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4l.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-outer-4m-big-array.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4m-big-array.c -flto -ffat-lto-objects
(test for excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4m-big-array.c -flto
-ffat-lto-objects compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4m-big-array.c compilation failed
to produce executable
FAIL: gcc.dg/vect/vect-outer-4m.c (test for excess errors)
FAIL: gcc.dg/vect/vect-outer-4m.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-outer-4m.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-outer-4m.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-over-widen-10.c (test for excess errors)
FAIL: gcc.dg/vect/vect-over-widen-10.c -flto -ffat-lto-objects (test
for excess errors)
UNRESOLVED: gcc.dg/vect/vect-over-widen-10.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-over-widen-10.c compilation failed to
produce executable
FAIL: gcc.dg/vect/vect-over-widen-12.c (test for excess errors)
FAIL: gcc.dg/vect/vect-over-widen-12.c -flto -ffat-lto-objects (test
for excess errors)
UNRESOLVED: gcc.dg/vect/vect-over-widen-12.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-over-widen-12.c compilation failed to
produce executable
FAIL: gcc.dg/vect/vect-pr111779.c (test for excess errors)
FAIL: gcc.dg/vect/vect-pr111779.c -flto -ffat-lto-objects (test for
excess errors)
UNRESOLVED: gcc.dg/vect/vect-pr111779.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-pr111779.c compilation failed to produce executable
FAIL: gcc.dg/vect/vect-widen-mult-half.c (test for excess errors)
FAIL: gcc.dg/vect/vect-widen-mult-half.c -flto -ffat-lto-objects (test
for excess errors)
UNRESOLVED: gcc.dg/vect/vect-widen-mult-half.c -flto -ffat-lto-objects
compilation failed to produce executable
UNRESOLVED: gcc.dg/vect/vect-widen-mult-half.c compilation failed to
produce executable
Which are all due to a bad uaddw2 instruction being emitted:
uaddw2 v26.4s, v31.4s, v26.4h
Thanks,
Andrew Pinski
>
>
> They all should be `scan-assembler-times`.
>
> Thanks,
> Andrew
>
> >
> > --- inline copy of patch --
> > diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md
> > index 81ff5bad03d598fa0d48df93d172a28bc0d1d92e..3d811007dd94dcd9176d6021a41a196c12fe9c3f 100644
> > --- a/gcc/config/aarch64/aarch64-simd.md
> > +++ b/gcc/config/aarch64/aarch64-simd.md
> > @@ -1988,26 +1988,60 @@ (define_insn "aarch64_simd_vec_unpack<su>_hi_<mode>"
> > [(set_attr "type" "neon_shift_imm_long")]
> > )
> >
> > -(define_expand "vec_unpack<su>_hi_<mode>"
> > +(define_expand "vec_unpacku_hi_<mode>"
> > [(match_operand:<VWIDE> 0 "register_operand")
> > - (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
> > + (match_operand:VQW 1 "register_operand")]
> > + "TARGET_SIMD"
> > + {
> > + rtx res = gen_reg_rtx (<MODE>mode);
> > + rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
> > + if (BYTES_BIG_ENDIAN)
> > + emit_insn (gen_aarch64_zip2<mode> (res, tmp, operands[1]));
> > + else
> > + emit_insn (gen_aarch64_zip2<mode> (res, operands[1], tmp));
> > + emit_move_insn (operands[0],
> > + simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
> > + DONE;
> > + }
> > +)
> > +
> > +(define_expand "vec_unpacks_hi_<mode>"
> > + [(match_operand:<VWIDE> 0 "register_operand")
> > + (match_operand:VQW 1 "register_operand")]
> > "TARGET_SIMD"
> > {
> > rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, true);
> > - emit_insn (gen_aarch64_simd_vec_unpack<su>_hi_<mode> (operands[0],
> > - operands[1], p));
> > + emit_insn (gen_aarch64_simd_vec_unpacks_hi_<mode> (operands[0],
> > + operands[1], p));
> > + DONE;
> > + }
> > +)
> > +
> > +(define_expand "vec_unpacku_lo_<mode>"
> > + [(match_operand:<VWIDE> 0 "register_operand")
> > + (match_operand:VQW 1 "register_operand")]
> > + "TARGET_SIMD"
> > + {
> > + rtx res = gen_reg_rtx (<MODE>mode);
> > + rtx tmp = aarch64_gen_shareable_zero (<MODE>mode);
> > + if (BYTES_BIG_ENDIAN)
> > + emit_insn (gen_aarch64_zip1<mode> (res, tmp, operands[1]));
> > + else
> > + emit_insn (gen_aarch64_zip1<mode> (res, operands[1], tmp));
> > + emit_move_insn (operands[0],
> > + simplify_gen_subreg (<VWIDE>mode, res, <MODE>mode, 0));
> > DONE;
> > }
> > )
> >
> > -(define_expand "vec_unpack<su>_lo_<mode>"
> > +(define_expand "vec_unpacks_lo_<mode>"
> > [(match_operand:<VWIDE> 0 "register_operand")
> > - (ANY_EXTEND:<VWIDE> (match_operand:VQW 1 "register_operand"))]
> > + (match_operand:VQW 1 "register_operand")]
> > "TARGET_SIMD"
> > {
> > rtx p = aarch64_simd_vect_par_cnst_half (<MODE>mode, <nunits>, false);
> > - emit_insn (gen_aarch64_simd_vec_unpack<su>_lo_<mode> (operands[0],
> > - operands[1], p));
> > + emit_insn (gen_aarch64_simd_vec_unpacks_lo_<mode> (operands[0],
> > + operands[1], p));
> > DONE;
> > }
> > )
> > @@ -4735,6 +4769,34 @@ (define_insn "aarch64_<ANY_EXTEND:su>subw2<mode>_internal"
> > [(set_attr "type" "neon_sub_widen")]
> > )
> >
> > +(define_insn "aarch64_usubw<mode>_<PERM_EXTEND:perm_hilo>_zip"
> > + [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> > + (minus:<VWIDE>
> > + (match_operand:<VWIDE> 1 "register_operand" "w")
> > + (subreg:<VWIDE>
> > + (unspec:<MODE> [
> > + (match_operand:VQW 2 "register_operand" "w")
> > + (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
> > + ] PERM_EXTEND) 0)))]
> > + "TARGET_SIMD"
> > + "usubw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
> > + [(set_attr "type" "neon_sub_widen")]
> > +)
> > +
> > +(define_insn "aarch64_uaddw<mode>_<PERM_EXTEND:perm_hilo>_zip"
> > + [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> > + (plus:<VWIDE>
> > + (subreg:<VWIDE>
> > + (unspec:<MODE> [
> > + (match_operand:VQW 2 "register_operand" "w")
> > + (match_operand:VQW 3 "aarch64_simd_imm_zero" "Dz")
> > + ] PERM_EXTEND) 0)
> > + (match_operand:<VWIDE> 1 "register_operand" "w")))]
> > + "TARGET_SIMD"
> > + "uaddw<PERM_EXTEND:perm_index>\\t%0.<Vwtype>, %1.<Vwtype>, %2.<Vhalftype>"
> > + [(set_attr "type" "neon_add_widen")]
> > +)
> > +
> > (define_insn "aarch64_<ANY_EXTEND:su>addw<mode>"
> > [(set (match_operand:<VWIDE> 0 "register_operand" "=w")
> > (plus:<VWIDE>
> > diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
> > index f9e2210095ea9d6d9c96971222a7757a2f418c2d..de281671aedc5141c69063f14cf0fbec5adecb04 100644
> > --- a/gcc/config/aarch64/iterators.md
> > +++ b/gcc/config/aarch64/iterators.md
> > @@ -2674,6 +2674,9 @@ (define_int_iterator PERMUTEQ [UNSPEC_ZIP1Q UNSPEC_ZIP2Q
> > (define_int_iterator OPTAB_PERMUTE [UNSPEC_ZIP1 UNSPEC_ZIP2
> > UNSPEC_UZP1 UNSPEC_UZP2])
> >
> > +;; Permutes for zero extends
> > +(define_int_iterator PERM_EXTEND [UNSPEC_ZIP1 UNSPEC_ZIP2])
> > +
> > (define_int_iterator REVERSE [UNSPEC_REV64 UNSPEC_REV32 UNSPEC_REV16])
> >
> > (define_int_iterator FRINT [UNSPEC_FRINTZ UNSPEC_FRINTP UNSPEC_FRINTM
> > @@ -3496,7 +3499,10 @@ (define_int_attr rev_op [(UNSPEC_REV64 "64") (UNSPEC_REV32 "32")
> > (UNSPEC_REV16 "16")])
> >
> > (define_int_attr perm_hilo [(UNSPEC_UNPACKSHI "hi") (UNSPEC_UNPACKUHI "hi")
> > - (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")])
> > + (UNSPEC_UNPACKSLO "lo") (UNSPEC_UNPACKULO "lo")
> > + (UNSPEC_ZIP2 "hi") (UNSPEC_ZIP1 "lo")])
> > +
> > +(define_int_attr perm_index [(UNSPEC_ZIP2 "2") (UNSPEC_ZIP1 "")])
> >
> > ;; Return true if the associated optab refers to the high-numbered lanes,
> > ;; false if it refers to the low-numbered lanes. The convention is for
> > diff --git a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> > index d45bb83e3503d512c443f37a446d30d188719a96..a2d09eaee0de5a3d3409330c5c26a3b5315e84eb 100644
> > --- a/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> > +++ b/gcc/testsuite/gcc.target/aarch64/simd/vmovl_high_1.c
> > @@ -22,11 +22,11 @@ FUNC (int32x4_t, int64x2_t, s32)
> > /* { dg-final { scan-assembler-times {sxtl2\tv0\.2d, v0\.4s} 1} } */
> >
> > FUNC (uint8x16_t, uint16x8_t, u8)
> > -/* { dg-final { scan-assembler-times {uxtl2\tv0\.8h, v0\.16b} 1} } */
> > +/* { dg-final { scan-assembler-times {zip2\tv0\.16b, v0\.16b} 1} } */
> >
> > FUNC (uint16x8_t, uint32x4_t, u16)
> > -/* { dg-final { scan-assembler-times {uxtl2\tv0\.4s, v0\.8h} 1} } */
> > +/* { dg-final { scan-assembler-times {zip2\tv0\.8h, v0\.8h} 1} } */
> >
> > FUNC (uint32x4_t, uint64x2_t, u32)
> > -/* { dg-final { scan-assembler-times {uxtl2\tv0\.2d, v0\.4s} 1} } */
> > +/* { dg-final { scan-assembler-times {zip2\tv0\.4s, v0\.4s} 1} } */
> >
> > diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..68fa9a09fe55f5a72355e23c90e781a898c5975e
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-1.c
> > @@ -0,0 +1,20 @@
> > +/* { dg-do assemble } */
> > +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> > +
> > +#pragma GCC target "+nosve"
> > +
> > +#define SIGN unsigned
> > +#define TYPE1 char
> > +#define TYPE2 short
> > +
> > +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> > +{
> > + for (int i = 0; i < (n & -8); i++)
> > + a[i] = b[i];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> > +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> > +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> > +
> > diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..af8a89085cfca800b41970a8410bc91b84a31d07
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-2.c
> > @@ -0,0 +1,20 @@
> > +/* { dg-do assemble } */
> > +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> > +
> > +#pragma GCC target "+nosve"
> > +
> > +#define SIGN unsigned
> > +#define TYPE1 short
> > +#define TYPE2 int
> > +
> > +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> > +{
> > + for (int i = 0; i < (n & -8); i++)
> > + a[i] = b[i];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> > +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> > +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> > +
> > diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..cdae6d09529b743857a092f53a07111df64775d7
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-3.c
> > @@ -0,0 +1,20 @@
> > +/* { dg-do assemble } */
> > +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> > +
> > +#pragma GCC target "+nosve"
> > +
> > +#define SIGN unsigned
> > +#define TYPE1 int
> > +#define TYPE2 long long
> > +
> > +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> > +{
> > + for (int i = 0; i < (n & -8); i++)
> > + a[i] = b[i];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tzip1\t} 1 } } */
> > +/* { dg-final { scan-assembler-times {\tzip2\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tuxtl\t} } } */
> > +/* { dg-final { scan-assembler-not {\tuxtl2\t} } } */
> > +
> > diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..e1a9c4f5661a36ec7b2c5dc6f0fd85c42fcaac39
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-4.c
> > @@ -0,0 +1,20 @@
> > +/* { dg-do assemble } */
> > +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> > +
> > +#pragma GCC target "+nosve"
> > +
> > +#define SIGN signed
> > +#define TYPE1 char
> > +#define TYPE2 short
> > +
> > +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> > +{
> > + for (int i = 0; i < (n & -8); i++)
> > + a[i] = b[i];
> > +}
> > +
> > +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> > +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> > +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> > +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> > +
> > diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..92b09ba4abba80f240ac175be2ef880968534975
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-5.c
> > @@ -0,0 +1,20 @@
> > +/* { dg-do assemble } */
> > +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> > +
> > +#pragma GCC target "+nosve"
> > +
> > +#define SIGN signed
> > +#define TYPE1 short
> > +#define TYPE2 int
> > +
> > +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> > +{
> > + for (int i = 0; i < (n & -8); i++)
> > + a[i] = b[i];
> > +}
> > +
> > +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> > +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> > +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> > +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> > +
> > diff --git a/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..5c6e635f29d1e52f51f5b75a477f7d8744f32ca3
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/uxtl-combine-6.c
> > @@ -0,0 +1,20 @@
> > +/* { dg-do assemble } */
> > +/* { dg-options "-O3 --save-temps --param=vect-epilogues-nomask=0" } */
> > +
> > +#pragma GCC target "+nosve"
> > +
> > +#define SIGN signed
> > +#define TYPE1 int
> > +#define TYPE2 long long
> > +
> > +void d2 (SIGN TYPE2 * restrict a, SIGN TYPE1 *b, int n)
> > +{
> > + for (int i = 0; i < (n & -8); i++)
> > + a[i] = b[i];
> > +}
> > +
> > +/* { dg-final { scan-assembler-not {\tzip1\t} } } */
> > +/* { dg-final { scan-assembler-not {\tzip2\t} } } */
> > +/* { dg-final { scan-assembler-times {\tsxtl\t} 1 } } */
> > +/* { dg-final { scan-assembler-time {\tsxtl2\t} 1 } } */
> > +
> >
> >
> >
> >
> > --
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-11-21 23:05 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-15 14:41 [PATCH]AArch64 Add pattern for unsigned widenings (uxtl) to zip{1,2} Tamar Christina
2023-11-21 10:39 ` Richard Sandiford
2023-11-21 22:43 ` Andrew Pinski
2023-11-21 23:05 ` Andrew Pinski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).