From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 5D8DE3858C66 for ; Tue, 25 Jul 2023 09:44:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5D8DE3858C66 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F05C315BF; Tue, 25 Jul 2023 02:44:48 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.110.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A49E3F6C4; Tue, 25 Jul 2023 02:44:05 -0700 (PDT) From: Richard Sandiford To: Hao Liu OS Mail-Followup-To: Hao Liu OS ,"GCC-patches\@gcc.gnu.org" , richard.sandiford@arm.com Cc: "GCC-patches\@gcc.gnu.org" Subject: Re: [PATCH] AArch64: Do not increase the vect reduction latency by multiplying count [PR110625] References: Date: Tue, 25 Jul 2023 10:44:04 +0100 In-Reply-To: (Hao Liu's message of "Tue, 25 Jul 2023 09:10:31 +0000") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-26.5 required=5.0 tests=BAYES_00,GIT_PATCH_0,KAM_DMARC_NONE,KAM_DMARC_STATUS,KAM_LAZY_DOMAIN_SECURITY,KAM_SHORT,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hao Liu OS writes: > Hi, > > Thanks for the suggestion. I tested it and found a gcc_assert failure: > gcc.target/aarch64/sve/cost_model_13.c (internal compiler error: in info_for_reduction, at tree-vect-loop.cc:5473) > > It is caused by empty STMT_VINFO_REDUC_DEF. When was STMT_VINFO_REDUC_DEF empty? I just want to make sure that we're not papering over an issue elsewhere. Thanks, Richard So, I added an extra check before checking single_defuse_cycle. The updated patch is below. Is it OK for trunk? > > --- > > The new costs should only count reduction latency by multiplying count for > single_defuse_cycle. For other situations, this will increase the reduction > latency a lot and miss vectorization opportunities. > > Tested on aarch64-linux-gnu. > > gcc/ChangeLog: > > PR target/110625 > * config/aarch64/aarch64.cc (count_ops): Only '* count' for > single_defuse_cycle while counting reduction_latency. > > gcc/testsuite/ChangeLog: > > * gcc.target/aarch64/pr110625_1.c: New testcase. > * gcc.target/aarch64/pr110625_2.c: New testcase. > --- > gcc/config/aarch64/aarch64.cc | 13 ++++-- > gcc/testsuite/gcc.target/aarch64/pr110625_1.c | 46 +++++++++++++++++++ > gcc/testsuite/gcc.target/aarch64/pr110625_2.c | 14 ++++++ > 3 files changed, 69 insertions(+), 4 deletions(-) > create mode 100644 gcc/testsuite/gcc.target/aarch64/pr110625_1.c > create mode 100644 gcc/testsuite/gcc.target/aarch64/pr110625_2.c > > diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc > index 560e5431636..478a4e00110 100644 > --- a/gcc/config/aarch64/aarch64.cc > +++ b/gcc/config/aarch64/aarch64.cc > @@ -16788,10 +16788,15 @@ aarch64_vector_costs::count_ops (unsigned int count, vect_cost_for_stmt kind, > { > unsigned int base > = aarch64_in_loop_reduction_latency (m_vinfo, stmt_info, m_vec_flags); > - > - /* ??? Ideally we'd do COUNT reductions in parallel, but unfortunately > - that's not yet the case. */ > - ops->reduction_latency = MAX (ops->reduction_latency, base * count); > + if (STMT_VINFO_REDUC_DEF (stmt_info) > + && STMT_VINFO_FORCE_SINGLE_CYCLE ( > + info_for_reduction (m_vinfo, stmt_info))) > + /* ??? Ideally we'd use a tree to reduce the copies down to 1 vector, > + and then accumulate that, but at the moment the loop-carried > + dependency includes all copies. */ > + ops->reduction_latency = MAX (ops->reduction_latency, base * count); > + else > + ops->reduction_latency = MAX (ops->reduction_latency, base); > } > > /* Assume that multiply-adds will become a single operation. */ > diff --git a/gcc/testsuite/gcc.target/aarch64/pr110625_1.c b/gcc/testsuite/gcc.target/aarch64/pr110625_1.c > new file mode 100644 > index 00000000000..0965cac33a0 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/aarch64/pr110625_1.c > @@ -0,0 +1,46 @@ > +/* { dg-do compile } */ > +/* { dg-options "-Ofast -mcpu=neoverse-n2 -fdump-tree-vect-details -fno-tree-slp-vectorize" } */ > +/* { dg-final { scan-tree-dump-not "reduction latency = 8" "vect" } } */ > + > +/* Do not increase the vector body cost due to the incorrect reduction latency > + Original vector body cost = 51 > + Scalar issue estimate: > + ... > + reduction latency = 2 > + estimated min cycles per iteration = 2.000000 > + estimated cycles per vector iteration (for VF 2) = 4.000000 > + Vector issue estimate: > + ... > + reduction latency = 8 <-- Too large > + estimated min cycles per iteration = 8.000000 > + Increasing body cost to 102 because scalar code would issue more quickly > + ... > + missed: cost model: the vector iteration cost = 102 divided by the scalar iteration cost = 44 is greater or equal to the vectorization factor = 2. > + missed: not vectorized: vectorization not profitable. */ > + > +typedef struct > +{ > + unsigned short m1, m2, m3, m4; > +} the_struct_t; > +typedef struct > +{ > + double m1, m2, m3, m4, m5; > +} the_struct2_t; > + > +double > +bar (the_struct2_t *); > + > +double > +foo (double *k, unsigned int n, the_struct_t *the_struct) > +{ > + unsigned int u; > + the_struct2_t result; > + for (u = 0; u < n; u++, k--) > + { > + result.m1 += (*k) * the_struct[u].m1; > + result.m2 += (*k) * the_struct[u].m2; > + result.m3 += (*k) * the_struct[u].m3; > + result.m4 += (*k) * the_struct[u].m4; > + } > + return bar (&result); > +} > diff --git a/gcc/testsuite/gcc.target/aarch64/pr110625_2.c b/gcc/testsuite/gcc.target/aarch64/pr110625_2.c > new file mode 100644 > index 00000000000..7a84aa8355e > --- /dev/null > +++ b/gcc/testsuite/gcc.target/aarch64/pr110625_2.c > @@ -0,0 +1,14 @@ > +/* { dg-do compile } */ > +/* { dg-options "-Ofast -mcpu=neoverse-n2 -fdump-tree-vect-details -fno-tree-slp-vectorize" } */ > +/* { dg-final { scan-tree-dump "reduction latency = 8" "vect" } } */ > + > +/* The reduction latency should be multiplied by the count for > + single_defuse_cycle. */ > + > +long > +f (long res, short *ptr1, short *ptr2, int n) > +{ > + for (int i = 0; i < n; ++i) > + res += (long) ptr1[i] << ptr2[i]; > + return res; > +}