From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by sourceware.org (Postfix) with ESMTPS id 2613F388A43A for ; Mon, 24 May 2021 07:20:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 2613F388A43A Received: by mail-pf1-x431.google.com with SMTP id d78so19241906pfd.10 for ; Mon, 24 May 2021 00:20:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ORH3JMXA7BEjkcrz/SF2dkgB2DKKFd58H8bb0z3MAK0=; b=b/Yx9p8F1FRo+01mz1XeOVhDUY3J7gwqPSZ8ONokvEqAY2wSyfBqBKYJbV3kDoEXOh ZozhMW2Pr7QBBi/n88fiBHAkD6sjJS9WjsEI6i9Dql0ssLzs4/ij7sudMtV+deYkAtFE CNZzgeOGqTlBt+XFTgWQxK/73cBlTS4j3aDu0d1N1leWHCvPNpsKdeKdfiAMpeSGxot4 VOKqr7UyhQ7oHVdTLEVBpnq0vKKwSLAPZ4YsIH4dz7IcnhkEv7Yg+z/SG5NyVXlQ5mBL DJdMl25jespCu6X10t1NCKzR0rvd8r9W8Ct1uIr8I6hpgAO9g7VT8U8Akvqza6Znve9R Fmkw== X-Gm-Message-State: AOAM533JIdrsirrKXhXxDhedbzKAwPabf0W7t9XzHsLPrsIcJGhrz1XI zhCVuc6ZKoBJZ+IEN+5NRmxIGKYJ8FSjMJBx+ckYREHL/j0= X-Google-Smtp-Source: ABdhPJxi9g4g48gVHryuhOVdJ8Ec5e924SGL7NQlpOdCkHhDWusZYiuZwdeq79uGxbwah/HAXFiAQ8oVS11cGDAEfyU= X-Received: by 2002:a63:e709:: with SMTP id b9mr12175481pgi.18.1621840818003; Mon, 24 May 2021 00:20:18 -0700 (PDT) MIME-Version: 1.0 References: <1619791790-628-1-git-send-email-christophe.lyon@linaro.org> <1619791790-628-9-git-send-email-christophe.lyon@linaro.org> <6c622137-dfaa-4a62-5a37-0a1f3f6ef5d5@arm.com> In-Reply-To: From: Christophe Lyon Date: Mon, 24 May 2021 09:20:07 +0200 Message-ID: Subject: Re: [PATCH 9/9] arm: Auto-vectorization for MVE: vld4/vst4 To: gcc Patches Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_NUMSUBJECT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2021 07:20:21 -0000 ping? On Mon, 17 May 2021 at 11:55, Christophe Lyon wrote: > > ping? > > On Tue, 4 May 2021 at 16:57, Christophe Lyon wrote: > > > > On Tue, 4 May 2021 at 14:03, Andre Vieira (lists) > > wrote: > > > > > > Hi Christophe, > > > > > > The series LGTM but you'll need the approval of an arm port maintainer > > > before committing. I only did code-review, did not try to build/run tests. > > > > > > > Hi Andre, > > > > Thanks for the comments! > > > > > Kind regards, > > > Andre > > > > > > On 30/04/2021 15:09, Christophe Lyon via Gcc-patches wrote: > > > > This patch enables MVE vld4/vst4 instructions for auto-vectorization. > > > > We move the existing expanders from neon.md and enable them for MVE, > > > > calling the respective emitter. > > > > > > > > 2021-03-12 Christophe Lyon > > > > > > > > gcc/ > > > > * config/arm/neon.md (vec_load_lanesxi) > > > > (vec_store_lanexoi): Move ... > > > > * config/arm/vec-common.md: here. > > > > > > > > gcc/testsuite/ > > > > * gcc.target/arm/simd/mve-vld4.c: New test, derived from > > > > slp-perm-3.c > > > > --- > > > > gcc/config/arm/neon.md | 20 ---- > > > > gcc/config/arm/vec-common.md | 26 +++++ > > > > gcc/testsuite/gcc.target/arm/simd/mve-vld4.c | 140 +++++++++++++++++++++++++++ > > > > 3 files changed, 166 insertions(+), 20 deletions(-) > > > > create mode 100644 gcc/testsuite/gcc.target/arm/simd/mve-vld4.c > > > > > > > > diff --git a/gcc/config/arm/neon.md b/gcc/config/arm/neon.md > > > > index bc8775c..fb58baf 100644 > > > > --- a/gcc/config/arm/neon.md > > > > +++ b/gcc/config/arm/neon.md > > > > @@ -5617,16 +5617,6 @@ (define_insn "neon_vld4" > > > > (const_string "neon_load4_4reg")))] > > > > ) > > > > > > > > -(define_expand "vec_load_lanesxi" > > > > - [(match_operand:XI 0 "s_register_operand") > > > > - (match_operand:XI 1 "neon_struct_operand") > > > > - (unspec:VQ2 [(const_int 0)] UNSPEC_VSTRUCTDUMMY)] > > > > - "TARGET_NEON" > > > > -{ > > > > - emit_insn (gen_neon_vld4 (operands[0], operands[1])); > > > > - DONE; > > > > -}) > > > > - > > > > (define_expand "neon_vld4" > > > > [(match_operand:XI 0 "s_register_operand") > > > > (match_operand:XI 1 "neon_struct_operand") > > > > @@ -5818,16 +5808,6 @@ (define_insn "neon_vst4" > > > > (const_string "neon_store4_4reg")))] > > > > ) > > > > > > > > -(define_expand "vec_store_lanesxi" > > > > - [(match_operand:XI 0 "neon_struct_operand") > > > > - (match_operand:XI 1 "s_register_operand") > > > > - (unspec:VQ2 [(const_int 0)] UNSPEC_VSTRUCTDUMMY)] > > > > - "TARGET_NEON" > > > > -{ > > > > - emit_insn (gen_neon_vst4 (operands[0], operands[1])); > > > > - DONE; > > > > -}) > > > > - > > > > (define_expand "neon_vst4" > > > > [(match_operand:XI 0 "neon_struct_operand") > > > > (match_operand:XI 1 "s_register_operand") > > > > diff --git a/gcc/config/arm/vec-common.md b/gcc/config/arm/vec-common.md > > > > index 7abefea..d46b78d 100644 > > > > --- a/gcc/config/arm/vec-common.md > > > > +++ b/gcc/config/arm/vec-common.md > > > > @@ -512,3 +512,29 @@ (define_expand "vec_store_lanesoi" > > > > emit_insn (gen_mve_vst2q (operands[0], operands[1])); > > > > DONE; > > > > }) > > > > + > > > > +(define_expand "vec_load_lanesxi" > > > > + [(match_operand:XI 0 "s_register_operand") > > > > + (match_operand:XI 1 "neon_struct_operand") > > > > + (unspec:VQ2 [(const_int 0)] UNSPEC_VSTRUCTDUMMY)] > > > > + "TARGET_NEON || TARGET_HAVE_MVE" > > > > +{ > > > > + if (TARGET_NEON) > > > > + emit_insn (gen_neon_vld4 (operands[0], operands[1])); > > > > + else > > > > + emit_insn (gen_mve_vld4q (operands[0], operands[1])); > > > > + DONE; > > > > +}) > > > > + > > > > +(define_expand "vec_store_lanesxi" > > > > + [(match_operand:XI 0 "neon_struct_operand") > > > > + (match_operand:XI 1 "s_register_operand") > > > > + (unspec:VQ2 [(const_int 0)] UNSPEC_VSTRUCTDUMMY)] > > > > + "TARGET_NEON || TARGET_HAVE_MVE" > > > > +{ > > > > + if (TARGET_NEON) > > > > + emit_insn (gen_neon_vst4 (operands[0], operands[1])); > > > > + else > > > > + emit_insn (gen_mve_vst4q (operands[0], operands[1])); > > > > + DONE; > > > > +}) > > > > diff --git a/gcc/testsuite/gcc.target/arm/simd/mve-vld4.c b/gcc/testsuite/gcc.target/arm/simd/mve-vld4.c > > > > new file mode 100644 > > > > index 0000000..ce3e755 > > > > --- /dev/null > > > > +++ b/gcc/testsuite/gcc.target/arm/simd/mve-vld4.c > > > > @@ -0,0 +1,140 @@ > > > > +/* { dg-do assemble } */ > > > > +/* { dg-require-effective-target arm_v8_1m_mve_fp_ok } */ > > > > +/* { dg-add-options arm_v8_1m_mve_fp } */ > > > > +/* { dg-additional-options "-O3" } */ > > > > + > > > > +#include > > > > + > > > > +#define M00 100 > > > > +#define M10 216 > > > > +#define M20 23 > > > > +#define M30 237 > > > > +#define M01 1322 > > > > +#define M11 13 > > > > +#define M21 27271 > > > > +#define M31 2280 > > > > +#define M02 74 > > > > +#define M12 191 > > > > +#define M22 500 > > > > +#define M32 111 > > > > +#define M03 134 > > > > +#define M13 117 > > > > +#define M23 11 > > > > +#define M33 771 > > > > + > > > > +#define N 128 > > > > + > > > > +/* Integer tests. */ > > > > +#define FUNC(SIGN, TYPE, BITS) \ > > > > + void foo_##SIGN##BITS##x (TYPE##BITS##_t *__restrict__ pInput, \ > > > > + TYPE##BITS##_t *__restrict__ pOutput) \ > > > > + { \ > > > > + unsigned int i; \ > > > > + TYPE##BITS##_t a, b, c, d; \ > > > > + \ > > > > + for (i = 0; i < N / BITS; i++) \ > > > > + { \ > > > > + a = *pInput++; \ > > > > + b = *pInput++; \ > > > > + c = *pInput++; \ > > > > + d = *pInput++; \ > > > > + \ > > > > + *pOutput++ = M00 * a + M01 * b + M02 * c + M03 * d; \ > > > > + *pOutput++ = M10 * a + M11 * b + M12 * c + M13 * d; \ > > > > + *pOutput++ = M20 * a + M21 * b + M22 * c + M23 * d; \ > > > > + *pOutput++ = M30 * a + M31 * b + M32 * c + M33 * d; \ > > > > + } \ > > > > + } > > > > + > > > > +FUNC(s, int, 8) > > > > +FUNC(u, uint, 8) > > > > +FUNC(s, int, 16) > > > > +FUNC(u, uint, 16) > > > > +FUNC(s, int, 32) > > > > +FUNC(u, uint, 32) > > > > + > > > > +/* float test, keep the macro because it's similar to the above, but does not > > > > + need the ##BITS##_t. */ > > > > +#define FUNC_FLOAT(SIGN, TYPE, BITS) \ > > > > + void foo_##SIGN##BITS##x (TYPE *__restrict__ pInput, \ > > > > + TYPE *__restrict__ pOutput) \ > > > > + { \ > > > > + unsigned int i; \ > > > > + TYPE a, b, c, d; \ > > > > + \ > > > > + for (i = 0; i < N / BITS; i++) \ > > > > + { \ > > > > + a = *pInput++; \ > > > > + b = *pInput++; \ > > > > + c = *pInput++; \ > > > > + d = *pInput++; \ > > > > + \ > > > > + *pOutput++ = M00 * a + M01 * b + M02 * c + M03 * d; \ > > > > + *pOutput++ = M10 * a + M11 * b + M12 * c + M13 * d; \ > > > > + *pOutput++ = M20 * a + M21 * b + M22 * c + M23 * d; \ > > > > + *pOutput++ = M30 * a + M31 * b + M32 * c + M33 * d; \ > > > > + } \ > > > > + } > > > > + > > > > +FUNC_FLOAT(f, float, 32) > > > > + > > > > +/* __fp16 test, needs explicit casts to avoid conversions to floating-point and > > > > + failure to vectorize. */ > > > > +__fp16 M00_fp16 = 100.0f16; > > > > +__fp16 M10_fp16 = 216.0f16; > > > > +__fp16 M20_fp16 = 23.0f16; > > > > +__fp16 M30_fp16 = 237.0f16; > > > > +__fp16 M01_fp16 = 1322.0f16; > > > > +__fp16 M11_fp16 = 13.0f16; > > > > +__fp16 M21_fp16 = 27271.0f16; > > > > +__fp16 M31_fp16 = 2280.0f16; > > > > +__fp16 M02_fp16 = 74.0f16; > > > > +__fp16 M12_fp16 = 191.0f16; > > > > +__fp16 M22_fp16 = 500.0f16; > > > > +__fp16 M32_fp16 = 111.0f16; > > > > +__fp16 M03_fp16 = 134.0f16; > > > > +__fp16 M13_fp16 = 117.0f16; > > > > +__fp16 M23_fp16 = 11.0f16; > > > > +__fp16 M33_fp16 = 771.0f16; > > > > + > > > > +#define FUNC_FLOAT_FP16(SIGN, TYPE, BITS) \ > > > > + void foo_##SIGN##BITS##x (TYPE *__restrict__ pInput, \ > > > > + TYPE *__restrict__ pOutput) \ > > > > + { \ > > > > + unsigned int i; \ > > > > + TYPE a, b, c, d; \ > > > > + \ > > > > + for (i = 0; i < N / BITS; i++) \ > > > > + { \ > > > > + a = *pInput++; \ > > > > + b = *pInput++; \ > > > > + c = *pInput++; \ > > > > + d = *pInput++; \ > > > > + \ > > > > + TYPE ab, cd; \ > > > > + ab = (__fp16)(M00_fp16 * a) + (__fp16)(M01_fp16 * b); \ > > > > + cd = (__fp16)(M02_fp16 * c) + (__fp16)(M03_fp16 * d); \ > > > > + *pOutput++ = ab + cd; \ > > > > + ab = (__fp16)(M10_fp16 * a) + (__fp16)(M11_fp16 * b); \ > > > > + cd = (__fp16)(M12_fp16 * c) + (__fp16)(M13_fp16 * d); \ > > > > + *pOutput++ = ab + cd; \ > > > > + ab = (__fp16)(M20_fp16 * a) + (__fp16)(M21_fp16 * b); \ > > > > + cd = (__fp16)(M22_fp16 * c) + (__fp16)(M23_fp16 * d); \ > > > > + *pOutput++ = ab + cd; \ > > > > + ab = (__fp16)(M30_fp16 * a) + (__fp16)(M31_fp16 * b); \ > > > > + cd = (__fp16)(M32_fp16 * c) + (__fp16)(M33_fp16 * d); \ > > > > + *pOutput++ = ab + cd; \ > > > > + } \ > > > > + } > > > > + > > > > +FUNC_FLOAT_FP16(f, __fp16, 16) > > > > + > > > > +/* vld4X.8 is used for signed and unsigned chars: 2 * 4. */ > > > > +/* vld4X.16 is used for signed and unsigned shorts and __fp16: 3 * 4. */ > > > > +/* vld4X.32 is used for signed and unsigned ints and float: 3 * 4. */ > > > > +/* { dg-final { scan-assembler-times {vld4[0123].8\t.q[0-9]+, q[0-9]+, q[0-9]+, q[0-9]+., } 8 } } */ > > > > +/* { dg-final { scan-assembler-times {vld4[0123].16\t.q[0-9]+, q[0-9]+, q[0-9]+, q[0-9]+., } 12 } } */ > > > > +/* { dg-final { scan-assembler-times {vld4[0123].32\t.q[0-9]+, q[0-9]+, q[0-9]+, q[0-9]+., } 12 } } */ > > > > +/* { dg-final { scan-assembler-times {vst4[0123].8\t.q[0-9]+, q[0-9]+, q[0-9]+, q[0-9]+., } 8 } } */ > > > > +/* { dg-final { scan-assembler-times {vst4[0123].16\t.q[0-9]+, q[0-9]+, q[0-9]+, q[0-9]+., } 12 } } */ > > > > +/* { dg-final { scan-assembler-times {vst4[0123].32\t.q[0-9]+, q[0-9]+, q[0-9]+, q[0-9]+., } 12 } } */