From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 10C5D385801A for ; Fri, 8 Jan 2021 10:52:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 10C5D385801A Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B94B331B for ; Fri, 8 Jan 2021 02:51:59 -0800 (PST) Received: from localhost (e121540-lin.manchester.arm.com [10.32.98.126]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 609DC3F719 for ; Fri, 8 Jan 2021 02:51:59 -0800 (PST) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [pushed] aarch64: Support conditional unpacked UXT on SVE Date: Fri, 08 Jan 2021 10:51:57 +0000 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2021 10:52:02 -0000 This patch extends the conditional UXT patterns from SVE_FULL_I to SVE_I. It doesn't matter in this case whether the type suffix is taken from the element size or the container size. Tested on aarch64-linux-gnu and aarch64_be-elf, pushed to trunk. Richard gcc/ * config/aarch64/aarch64-sve.md (*cond_uxt_2): Extend from SVE_FULL_I to SVE_I. (*cond_uxt_any): Likewise. gcc/testsuite/ * gcc.target/aarch64/sve/cond_uxt_5.c: New test. * gcc.target/aarch64/sve/cond_uxt_5_run.c: Likewise. * gcc.target/aarch64/sve/cond_uxt_6.c: Likewise. * gcc.target/aarch64/sve/cond_uxt_6_run.c: Likewise. * gcc.target/aarch64/sve/cond_uxt_7.c: Likewise. * gcc.target/aarch64/sve/cond_uxt_7_run.c: Likewise. * gcc.target/aarch64/sve/cond_uxt_8.c: Likewise. * gcc.target/aarch64/sve/cond_uxt_8_run.c: Likewise. --- gcc/config/aarch64/aarch64-sve.md | 22 ++++++------- .../gcc.target/aarch64/sve/cond_uxt_5.c | 33 +++++++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_5_run.c | 26 +++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_6.c | 33 +++++++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_6_run.c | 26 +++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_7.c | 29 ++++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_7_run.c | 26 +++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_8.c | 32 ++++++++++++++++++ .../gcc.target/aarch64/sve/cond_uxt_8_run.c | 26 +++++++++++++++ 9 files changed, 242 insertions(+), 11 deletions(-) create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5_run.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6_run.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7_run.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8_run.c diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index 2ec9acbf38d..b83f9912cb6 100644 --- a/gcc/config/aarch64/aarch64-sve.md +++ b/gcc/config/aarch64/aarch64-sve.md @@ -3135,12 +3135,12 @@ (define_insn "@aarch64_cond_sxt" ;; The canonical form of this operation is an AND of a constant rather ;; than (zero_extend (truncate ...)). (define_insn "*cond_uxt_2" - [(set (match_operand:SVE_FULL_I 0 "register_operand" "=w, ?&w") - (unspec:SVE_FULL_I + [(set (match_operand:SVE_I 0 "register_operand" "=w, ?&w") + (unspec:SVE_I [(match_operand: 1 "register_operand" "Upl, Upl") - (and:SVE_FULL_I - (match_operand:SVE_FULL_I 2 "register_operand" "0, w") - (match_operand:SVE_FULL_I 3 "aarch64_sve_uxt_immediate")) + (and:SVE_I + (match_operand:SVE_I 2 "register_operand" "0, w") + (match_operand:SVE_I 3 "aarch64_sve_uxt_immediate")) (match_dup 2)] UNSPEC_SEL))] "TARGET_SVE" @@ -3159,13 +3159,13 @@ (define_insn "*cond_uxt_2" ;; as early-clobber helps to make the instruction more regular to the ;; register allocator. (define_insn "*cond_uxt_any" - [(set (match_operand:SVE_FULL_I 0 "register_operand" "=&w, ?&w, ?&w") - (unspec:SVE_FULL_I + [(set (match_operand:SVE_I 0 "register_operand" "=&w, ?&w, ?&w") + (unspec:SVE_I [(match_operand: 1 "register_operand" "Upl, Upl, Upl") - (and:SVE_FULL_I - (match_operand:SVE_FULL_I 2 "register_operand" "w, w, w") - (match_operand:SVE_FULL_I 3 "aarch64_sve_uxt_immediate")) - (match_operand:SVE_FULL_I 4 "aarch64_simd_reg_or_zero" "0, Dz, w")] + (and:SVE_I + (match_operand:SVE_I 2 "register_operand" "w, w, w") + (match_operand:SVE_I 3 "aarch64_sve_uxt_immediate")) + (match_operand:SVE_I 4 "aarch64_simd_reg_or_zero" "0, Dz, w")] UNSPEC_SEL))] "TARGET_SVE && !rtx_equal_p (operands[2], operands[4])" "@ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5.c new file mode 100644 index 00000000000..18866286b7f --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include + +#define DEF_LOOP(TYPE1, TYPE2, CONST, COUNT) \ + void __attribute__ ((noipa)) \ + test_##CONST##_##TYPE1##_##TYPE2 (TYPE2 *restrict r, \ + TYPE1 *restrict a, \ + TYPE2 *restrict b) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = a[i] > 20 ? b[i] & CONST : b[i]; \ + } + +#define TEST_ALL(T) \ + T (int32_t, uint16_t, 0xff, 3) \ + \ + T (int64_t, uint16_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xffff, 5) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler {\tld1h\t(z[0-9]+)\.s, p[0-7]/z, \[x2[],][^L]*\tuxtb\t\1\.h, p[0-7]/m, \1\.h\n} } } */ + +/* { dg-final { scan-assembler {\tld1h\t(z[0-9]+)\.d, p[0-7]/z, \[x2[],][^L]*\tuxtb\t\1\.h, p[0-7]/m, \1\.h\n} } } */ +/* { dg-final { scan-assembler {\tld1w\t(z[0-9]+)\.d, p[0-7]/z, \[x2[],][^L]*\tuxtb\t\1\.s, p[0-7]/m, \1\.s\n} } } */ +/* { dg-final { scan-assembler {\tld1w\t(z[0-9]+)\.d, p[0-7]/z, \[x2[],][^L]*\tuxth\t\1\.s, p[0-7]/m, \1\.s\n} } } */ + +/* { dg-final { scan-assembler-not {\tmov\tz[^\n]*z} } } */ +/* { dg-final { scan-assembler-not {\tmovprfx\t} } } */ +/* { dg-final { scan-assembler-not {\tsel\t} } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5_run.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5_run.c new file mode 100644 index 00000000000..675fd2be96b --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_5_run.c @@ -0,0 +1,26 @@ +/* { dg-do run { target { aarch64_sve_hw } } } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include "cond_uxt_5.c" + +#define TEST_LOOP(TYPE1, TYPE2, CONST, N) \ + { \ + TYPE1 a[N]; \ + TYPE2 r[N], b[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? -20 - i : 20 + i); \ + b[i] = -5 - i; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##CONST##_##TYPE1##_##TYPE2 (r, a, b); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (TYPE2) (a[i] > 20 ? b[i] & CONST : b[i])) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6.c new file mode 100644 index 00000000000..e47276a3a35 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include + +#define DEF_LOOP(TYPE1, TYPE2, CONST, COUNT) \ + void __attribute__ ((noipa)) \ + test_##CONST##_##TYPE1##_##TYPE2 (TYPE2 *restrict r, \ + TYPE1 *restrict a, \ + TYPE2 *restrict b) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = a[i] > 20 ? b[i] & CONST : a[i]; \ + } + +#define TEST_ALL(T) \ + T (int32_t, uint16_t, 0xff, 3) \ + \ + T (int64_t, uint16_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xffff, 5) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler {\tld1w\t(z[0-9]+)\.s, p[0-7]/z, \[x1[],][^L]*\tld1h\t(z[0-9]+)\.s, p[0-7]/z, \[x2[],][^L]*\tuxtb\t\1\.h, p[0-7]/m, \2\.h\n} } } */ + +/* { dg-final { scan-assembler {\tld1d\t(z[0-9]+)\.d, p[0-7]/z, \[x1[],][^L]*\tld1h\t(z[0-9]+)\.d, p[0-7]/z, \[x2[],][^L]*\tuxtb\t\1\.h, p[0-7]/m, \2\.h\n} } } */ +/* { dg-final { scan-assembler {\tld1d\t(z[0-9]+)\.d, p[0-7]/z, \[x1[],][^L]*\tld1w\t(z[0-9]+)\.d, p[0-7]/z, \[x2[],][^L]*\tuxtb\t\1\.s, p[0-7]/m, \2\.s\n} } } */ +/* { dg-final { scan-assembler {\tld1d\t(z[0-9]+)\.d, p[0-7]/z, \[x1[],][^L]*\tld1w\t(z[0-9]+)\.d, p[0-7]/z, \[x2[],][^L]*\tuxth\t\1\.s, p[0-7]/m, \2\.s\n} } } */ + +/* { dg-final { scan-assembler-not {\tmov\tz[^\n]*z} } } */ +/* { dg-final { scan-assembler-not {\tmovprfx\t} } } */ +/* { dg-final { scan-assembler-not {\tsel\t} } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6_run.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6_run.c new file mode 100644 index 00000000000..72208542b88 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_6_run.c @@ -0,0 +1,26 @@ +/* { dg-do run { target { aarch64_sve_hw } } } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include "cond_uxt_6.c" + +#define TEST_LOOP(TYPE1, TYPE2, CONST, N) \ + { \ + TYPE1 a[N]; \ + TYPE2 r[N], b[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? -20 - i : 20 + i); \ + b[i] = -5 - i; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##CONST##_##TYPE1##_##TYPE2 (r, a, b); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (TYPE2) (a[i] > 20 ? b[i] & CONST : a[i])) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7.c new file mode 100644 index 00000000000..f49915c4ac1 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7.c @@ -0,0 +1,29 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include + +#define DEF_LOOP(TYPE1, TYPE2, CONST, COUNT) \ + void __attribute__ ((noipa)) \ + test_##CONST##_##TYPE1##_##TYPE2 (TYPE2 *restrict r, \ + TYPE1 *restrict a, \ + TYPE2 *restrict b) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = a[i] > 20 ? b[i] & CONST : 127; \ + } + +#define TEST_ALL(T) \ + T (int32_t, uint16_t, 0xff, 3) \ + \ + T (int64_t, uint16_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xffff, 5) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler-times {\tuxtb\tz[0-9]+\.h, p[0-7]/m, z[0-9]+\.h\n} 2 } } */ +/* { dg-final { scan-assembler-times {\tuxtb\tz[0-9]+\.s, p[0-7]/m, z[0-9]+\.s\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tuxth\tz[0-9]+\.s, p[0-7]/m, z[0-9]+\.s\n} 1 } } */ + +/* { dg-final { scan-assembler-not {\tsel\t} } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7_run.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7_run.c new file mode 100644 index 00000000000..4c6dc36583c --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_7_run.c @@ -0,0 +1,26 @@ +/* { dg-do run { target { aarch64_sve_hw } } } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include "cond_uxt_7.c" + +#define TEST_LOOP(TYPE1, TYPE2, CONST, N) \ + { \ + TYPE1 a[N]; \ + TYPE2 r[N], b[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? -20 - i : 20 + i); \ + b[i] = -5 - i; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##CONST##_##TYPE1##_##TYPE2 (r, a, b); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (TYPE2) (a[i] > 20 ? b[i] & CONST : 127)) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8.c new file mode 100644 index 00000000000..42eb4b2661b --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8.c @@ -0,0 +1,32 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include + +#define DEF_LOOP(TYPE1, TYPE2, CONST, COUNT) \ + void __attribute__ ((noipa)) \ + test_##CONST##_##TYPE1##_##TYPE2 (TYPE2 *restrict r, \ + TYPE1 *restrict a, \ + TYPE2 *restrict b) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = a[i] > 20 ? b[i] & CONST : 0; \ + } + +#define TEST_ALL(T) \ + T (int32_t, uint16_t, 0xff, 3) \ + \ + T (int64_t, uint16_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xff, 5) \ + T (int64_t, uint32_t, 0xffff, 5) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler-times {\tuxtb\tz[0-9]+\.h, p[0-7]/m, z[0-9]+\.h\n} 2 } } */ +/* { dg-final { scan-assembler-times {\tuxtb\tz[0-9]+\.s, p[0-7]/m, z[0-9]+\.s\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tuxth\tz[0-9]+\.s, p[0-7]/m, z[0-9]+\.s\n} 1 } } */ + +/* { dg-final { scan-assembler {\tmovprfx\tz[^,]*, p[0-7]/z} } } */ + +/* { dg-final { scan-assembler-not {\tmov\tz[^\n]*z} } } */ +/* { dg-final { scan-assembler-not {\tsel\t} } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8_run.c b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8_run.c new file mode 100644 index 00000000000..cfe3ed64818 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/cond_uxt_8_run.c @@ -0,0 +1,26 @@ +/* { dg-do run { target { aarch64_sve_hw } } } */ +/* { dg-options "-O2 -ftree-vectorize" } */ + +#include "cond_uxt_8.c" + +#define TEST_LOOP(TYPE1, TYPE2, CONST, N) \ + { \ + TYPE1 a[N]; \ + TYPE2 r[N], b[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? -20 - i : 20 + i); \ + b[i] = -5 - i; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##CONST##_##TYPE1##_##TYPE2 (r, a, b); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (TYPE2) (a[i] > 20 ? b[i] & CONST : 0)) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +}