From: Richard Sandiford <richard.sandiford@arm.com>
To: gcc-patches@gcc.gnu.org
Subject: [PATCH 4/8] aarch64: Add gather_load_xNN_cost tuning fields
Date: Tue, 03 Aug 2021 13:05:17 +0100 [thread overview]
Message-ID: <mptzgtybuo2.fsf@arm.com> (raw)
In-Reply-To: <mptlf5id9bw.fsf@arm.com> (Richard Sandiford's message of "Tue, 03 Aug 2021 13:03:15 +0100")
This patch adds tuning fields for the total cost of a gather load
instruction. Until now, we've costed them as one scalar load
per element instead. Those scalar_load-based values are also
what the patch uses to fill in the new fields for existing
cost structures.
gcc/
* config/aarch64/aarch64-protos.h (sve_vec_cost):
Add gather_load_x32_cost and gather_load_x64_cost.
* config/aarch64/aarch64.c (generic_sve_vector_cost)
(a64fx_sve_vector_cost, neoversev1_sve_vector_cost): Update
accordingly, using the values given by the scalar_load * number
of elements calculation that we used previously.
(aarch64_detect_vector_stmt_subtype): Use the new fields.
---
gcc/config/aarch64/aarch64-protos.h | 9 +++++++++
gcc/config/aarch64/aarch64.c | 19 +++++++++++++++++++
2 files changed, 28 insertions(+)
diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h
index fb4ce8e9f84..b91eeeba101 100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -259,12 +259,16 @@ struct sve_vec_cost : simd_vec_cost
unsigned int fadda_f16_cost,
unsigned int fadda_f32_cost,
unsigned int fadda_f64_cost,
+ unsigned int gather_load_x32_cost,
+ unsigned int gather_load_x64_cost,
unsigned int scatter_store_elt_cost)
: simd_vec_cost (base),
clast_cost (clast_cost),
fadda_f16_cost (fadda_f16_cost),
fadda_f32_cost (fadda_f32_cost),
fadda_f64_cost (fadda_f64_cost),
+ gather_load_x32_cost (gather_load_x32_cost),
+ gather_load_x64_cost (gather_load_x64_cost),
scatter_store_elt_cost (scatter_store_elt_cost)
{}
@@ -279,6 +283,11 @@ struct sve_vec_cost : simd_vec_cost
const int fadda_f32_cost;
const int fadda_f64_cost;
+ /* The cost of a gather load instruction. The x32 value is for loads
+ of 32-bit elements and the x64 value is for loads of 64-bit elements. */
+ const int gather_load_x32_cost;
+ const int gather_load_x64_cost;
+
/* The per-element cost of a scatter store. */
const int scatter_store_elt_cost;
};
diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c
index b14b6f22aec..36f11808916 100644
--- a/gcc/config/aarch64/aarch64.c
+++ b/gcc/config/aarch64/aarch64.c
@@ -675,6 +675,8 @@ static const sve_vec_cost generic_sve_vector_cost =
2, /* fadda_f16_cost */
2, /* fadda_f32_cost */
2, /* fadda_f64_cost */
+ 4, /* gather_load_x32_cost */
+ 2, /* gather_load_x64_cost */
1 /* scatter_store_elt_cost */
};
@@ -744,6 +746,8 @@ static const sve_vec_cost a64fx_sve_vector_cost =
13, /* fadda_f16_cost */
13, /* fadda_f32_cost */
13, /* fadda_f64_cost */
+ 64, /* gather_load_x32_cost */
+ 32, /* gather_load_x64_cost */
1 /* scatter_store_elt_cost */
};
@@ -1739,6 +1743,8 @@ static const sve_vec_cost neoversev1_sve_vector_cost =
19, /* fadda_f16_cost */
11, /* fadda_f32_cost */
8, /* fadda_f64_cost */
+ 32, /* gather_load_x32_cost */
+ 16, /* gather_load_x64_cost */
3 /* scatter_store_elt_cost */
};
@@ -14958,6 +14964,19 @@ aarch64_detect_vector_stmt_subtype (vec_info *vinfo, vect_cost_for_stmt kind,
&& DR_IS_WRITE (STMT_VINFO_DATA_REF (stmt_info)))
return simd_costs->store_elt_extra_cost;
+ /* Detect SVE gather loads, which are costed as a single scalar_load
+ for each element. We therefore need to divide the full-instruction
+ cost by the number of elements in the vector. */
+ if (kind == scalar_load
+ && sve_costs
+ && STMT_VINFO_MEMORY_ACCESS_TYPE (stmt_info) == VMAT_GATHER_SCATTER)
+ {
+ unsigned int nunits = vect_nunits_for_cost (vectype);
+ if (GET_MODE_UNIT_BITSIZE (TYPE_MODE (vectype)) == 64)
+ return { sve_costs->gather_load_x64_cost, nunits };
+ return { sve_costs->gather_load_x32_cost, nunits };
+ }
+
/* Detect cases in which a scalar_store is really storing one element
in a scatter operation. */
if (kind == scalar_store
next prev parent reply other threads:[~2021-08-03 12:05 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-03 12:03 [PATCH 0/8] aarch64 vector cost tweaks Richard Sandiford
2021-08-03 12:03 ` [PATCH 1/8] aarch64: Turn sve_width tuning field into a bitmask Richard Sandiford
2021-08-03 12:04 ` [PATCH 2/8] aarch64: Add a simple fixed-point class for costing Richard Sandiford
2021-08-03 12:04 ` [PATCH 3/8] aarch64: Split out aarch64_adjust_body_cost_sve Richard Sandiford
2021-08-03 12:05 ` Richard Sandiford [this message]
2021-08-03 12:05 ` [PATCH 5/8] aarch64: Tweak the cost of elementwise stores Richard Sandiford
2021-08-04 11:43 ` Richard Biener
2021-08-05 12:04 ` [PATCH] vect: Move costing helpers from aarch64 code Richard Sandiford
2021-08-05 12:11 ` Richard Biener
2021-08-05 13:06 ` Richard Sandiford
2021-08-03 12:06 ` [PATCH 6/8] aarch64: Tweak MLA vector costs Richard Sandiford
2021-08-04 11:45 ` Richard Biener
2021-08-04 12:14 ` Richard Sandiford
2021-08-04 12:19 ` Richard Sandiford
2021-08-03 12:06 ` [PATCH 7/8] aarch64: Restrict issue heuristics to inner vector loop Richard Sandiford
2021-08-03 12:06 ` [PATCH 8/8] aarch64: Add -mtune=neoverse-512tvb Richard Sandiford
2021-08-17 14:37 ` Richard Sandiford
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=mptzgtybuo2.fsf@arm.com \
--to=richard.sandiford@arm.com \
--cc=gcc-patches@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).