From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 270EC393AC23 for ; Tue, 3 Aug 2021 12:06:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 270EC393AC23 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D88DF11FB for ; Tue, 3 Aug 2021 05:06:24 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.98.126]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7EA213F40C for ; Tue, 3 Aug 2021 05:06:24 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [PATCH 7/8] aarch64: Restrict issue heuristics to inner vector loop References: Date: Tue, 03 Aug 2021 13:06:23 +0100 In-Reply-To: (Richard Sandiford's message of "Tue, 03 Aug 2021 13:03:15 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Aug 2021 12:06:26 -0000 The AArch64 vector costs try to take issue rates into account. However, when vectorising an outer loop, we lumped the inner and outer operations together, which is somewhat meaningless. This patch restricts the heuristic to the inner loop. gcc/ * config/aarch64/aarch64.c (aarch64_add_stmt_cost): Only record issue information for operations that occur in the innermost loop. --- gcc/config/aarch64/aarch64.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index 19045ef6944..19625eb048d 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -15392,6 +15392,10 @@ aarch64_add_stmt_cost (class vec_info *vinfo, void *data, int count, fractional_cost stmt_cost = aarch64_builtin_vectorization_cost (kind, vectype, misalign); + bool in_inner_loop_p = (where == vect_body + && stmt_info + && stmt_in_inner_loop_p (vinfo, stmt_info)); + /* Do one-time initialization based on the vinfo. */ loop_vec_info loop_vinfo = dyn_cast (vinfo); bb_vec_info bb_vinfo = dyn_cast (vinfo); @@ -15438,14 +15442,15 @@ aarch64_add_stmt_cost (class vec_info *vinfo, void *data, int count, stmt_cost = aarch64_adjust_stmt_cost (kind, stmt_info, vectype, stmt_cost); - /* If we're recording a nonzero vector loop body cost, also estimate - the operations that would need to be issued by all relevant - implementations of the loop. */ + /* If we're recording a nonzero vector loop body cost for the + innermost loop, also estimate the operations that would need + to be issued by all relevant implementations of the loop. */ auto *issue_info = aarch64_tune_params.vec_costs->issue_info; if (loop_vinfo && issue_info && costs->vec_flags && where == vect_body + && (!LOOP_VINFO_LOOP (loop_vinfo)->inner || in_inner_loop_p) && vectype && stmt_cost != 0) { @@ -15489,8 +15494,7 @@ aarch64_add_stmt_cost (class vec_info *vinfo, void *data, int count, /* Statements in an inner loop relative to the loop being vectorized are weighted more heavily. The value here is arbitrary and could potentially be improved with analysis. */ - if (where == vect_body && stmt_info - && stmt_in_inner_loop_p (vinfo, stmt_info)) + if (in_inner_loop_p) { gcc_assert (loop_vinfo); count *= LOOP_VINFO_INNER_LOOP_COST_FACTOR (loop_vinfo); /* FIXME */