From: "H.J. Lu" <hjl.tools@gmail.com>
To: Kirill Yukhin <kirill.yukhin@gmail.com>
Cc: GCC Patches <gcc-patches@gcc.gnu.org>
Subject: Re: [PATCH] x86: Enable FMA in rsqrt<mode>2 expander
Date: Thu, 9 Jul 2020 06:35:27 -0700 [thread overview]
Message-ID: <CAMe9rOpEq_W7FxvrHs0yPM+Z4iwHjo12Bf5THrNoj0tA=WDV6A@mail.gmail.com> (raw)
In-Reply-To: <20200709120449.qkx4o5yhhsd5ujpr@kyukhin>
[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]
On Thu, Jul 9, 2020 at 5:04 AM Kirill Yukhin <kirill.yukhin@gmail.com> wrote:
>
> On 07 июл 09:06, H.J. Lu wrote:
> > On Tue, Jul 7, 2020 at 8:56 AM Kirill Yukhin <kirill.yukhin@gmail.com> wrote:
> > >
> > > Hello HJ,
> > >
> > > On 28 июн 07:19, H.J. Lu via Gcc-patches wrote:
> > > > Enable FMA in rsqrt<mode>2 expander and fold rsqrtv16sf2 expander into
> > > > rsqrt<mode>2 expander which expands to UNSPEC_RSQRT28 for TARGET_AVX512ER.
> > > > Although it doesn't show performance change in our workloads, FMA can
> > > > improve other workloads.
> > > >
> > > > gcc/
> > > >
> > > > PR target/88713
> > > > * config/i386/i386-expand.c (ix86_emit_swsqrtsf): Enable FMA.
> > > > * config/i386/sse.md (VF_AVX512VL_VF1_128_256): New.
> > > > (rsqrt<mode>2): Replace VF1_128_256 with VF_AVX512VL_VF1_128_256.
> > > > (rsqrtv16sf2): Removed.
> > > >
> > > > gcc/testsuite/
> > > >
> > > > PR target/88713
> > > > * gcc.target/i386/pr88713-1.c: New test.
> > > > * gcc.target/i386/pr88713-2.c: Likewise.
> > >
> > > So, you've introduced new rsqrt expanders for DF vectors and relaxed
> > > condition for V16SF. What I didn't get is why did you change unspec
> > > type from RSQRT to RSQRT28 for V16SF expander?
> > >
> >
> > UNSPEC in define_expand is meaningless when the pattern is fully
> > expanded by ix86_emit_swsqrtsf. I believe that UNSPEC in rsqrt<mode>2
> > expander can be removed.
>
> Agree.
I will leave UNSPEC alone here.
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/i386/pr88713-1.c
> @@ -0,0 +1,13 @@
> +/* { dg-do compile } */
> +/* { dg-options "-O2 -Ofast -mno-avx512f -mfma" } */
>
> I gues -O2 is useless here (and in -2.c test).
Fixed.
> Othwerwise LGTM.
>
This is the patch I am checking in.
Thanks.
--
H.J.
[-- Attachment #2: 0001-x86-Enable-FMA-in-rsqrt-mode-2-expander.patch --]
[-- Type: text/x-patch, Size: 4895 bytes --]
From 54d4f6c740ed885e55b0092895ef0986714211b3 Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Wed, 23 Jan 2019 06:33:58 -0800
Subject: [PATCH] x86: Enable FMA in rsqrt<mode>2 expander
Enable FMA in rsqrt<mode>2 expander and fold rsqrtv16sf2 expander into
rsqrt<mode>2 expander which expands to UNSPEC_RSQRT28 for TARGET_AVX512ER.
Although it doesn't show performance change in our workloads, FMA can
improve other workloads.
gcc/
PR target/88713
* config/i386/i386-expand.c (ix86_emit_swsqrtsf): Enable FMA.
* config/i386/sse.md (VF_AVX512VL_VF1_128_256): New.
(rsqrt<mode>2): Replace VF1_128_256 with VF_AVX512VL_VF1_128_256.
(rsqrtv16sf2): Removed.
gcc/testsuite/
PR target/88713
* gcc.target/i386/pr88713-1.c: New test.
* gcc.target/i386/pr88713-2.c: Likewise.
---
gcc/config/i386/i386-expand.c | 18 ++++++++++++-----
gcc/config/i386/sse.md | 24 ++++++++++-------------
gcc/testsuite/gcc.target/i386/pr88713-1.c | 13 ++++++++++++
gcc/testsuite/gcc.target/i386/pr88713-2.c | 6 ++++++
4 files changed, 42 insertions(+), 19 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/i386/pr88713-1.c
create mode 100644 gcc/testsuite/gcc.target/i386/pr88713-2.c
diff --git a/gcc/config/i386/i386-expand.c b/gcc/config/i386/i386-expand.c
index d81dd73f034..49718b7a41c 100644
--- a/gcc/config/i386/i386-expand.c
+++ b/gcc/config/i386/i386-expand.c
@@ -15535,14 +15535,22 @@ void ix86_emit_swsqrtsf (rtx res, rtx a, machine_mode mode, bool recip)
}
}
+ mthree = force_reg (mode, mthree);
+
/* e0 = x0 * a */
emit_insn (gen_rtx_SET (e0, gen_rtx_MULT (mode, x0, a)));
- /* e1 = e0 * x0 */
- emit_insn (gen_rtx_SET (e1, gen_rtx_MULT (mode, e0, x0)));
- /* e2 = e1 - 3. */
- mthree = force_reg (mode, mthree);
- emit_insn (gen_rtx_SET (e2, gen_rtx_PLUS (mode, e1, mthree)));
+ if (TARGET_FMA || TARGET_AVX512F)
+ emit_insn (gen_rtx_SET (e2,
+ gen_rtx_FMA (mode, e0, x0, mthree)));
+ else
+ {
+ /* e1 = e0 * x0 */
+ emit_insn (gen_rtx_SET (e1, gen_rtx_MULT (mode, e0, x0)));
+
+ /* e2 = e1 - 3. */
+ emit_insn (gen_rtx_SET (e2, gen_rtx_PLUS (mode, e1, mthree)));
+ }
mhalf = force_reg (mode, mhalf);
if (recip)
diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md
index 431571a4bc1..d3ad5833e1f 100644
--- a/gcc/config/i386/sse.md
+++ b/gcc/config/i386/sse.md
@@ -326,6 +326,12 @@ (define_mode_iterator VF_AVX512VL
[V16SF (V8SF "TARGET_AVX512VL") (V4SF "TARGET_AVX512VL")
V8DF (V4DF "TARGET_AVX512VL") (V2DF "TARGET_AVX512VL")])
+;; AVX512VL SF/DF plus 128- and 256-bit SF vector modes
+(define_mode_iterator VF_AVX512VL_VF1_128_256
+ [(V16SF "TARGET_AVX512F") (V8SF "TARGET_AVX") V4SF
+ (V8DF "TARGET_AVX512F") (V4DF "TARGET_AVX512VL")
+ (V2DF "TARGET_AVX512VL")])
+
(define_mode_iterator VF2_AVX512VL
[V8DF (V4DF "TARGET_AVX512VL") (V2DF "TARGET_AVX512VL")])
@@ -2070,26 +2076,16 @@ (define_insn "*<sse>_vmsqrt<mode>2<mask_scalar_name><round_scalar_name>"
(set_attr "mode" "<ssescalarmode>")])
(define_expand "rsqrt<mode>2"
- [(set (match_operand:VF1_128_256 0 "register_operand")
- (unspec:VF1_128_256
- [(match_operand:VF1_128_256 1 "vector_operand")] UNSPEC_RSQRT))]
+ [(set (match_operand:VF_AVX512VL_VF1_128_256 0 "register_operand")
+ (unspec:VF_AVX512VL_VF1_128_256
+ [(match_operand:VF_AVX512VL_VF1_128_256 1 "vector_operand")]
+ UNSPEC_RSQRT))]
"TARGET_SSE && TARGET_SSE_MATH"
{
ix86_emit_swsqrtsf (operands[0], operands[1], <MODE>mode, true);
DONE;
})
-(define_expand "rsqrtv16sf2"
- [(set (match_operand:V16SF 0 "register_operand")
- (unspec:V16SF
- [(match_operand:V16SF 1 "vector_operand")]
- UNSPEC_RSQRT28))]
- "TARGET_AVX512ER && TARGET_SSE_MATH"
-{
- ix86_emit_swsqrtsf (operands[0], operands[1], V16SFmode, true);
- DONE;
-})
-
(define_insn "<sse>_rsqrt<mode>2"
[(set (match_operand:VF1_128_256 0 "register_operand" "=x")
(unspec:VF1_128_256
diff --git a/gcc/testsuite/gcc.target/i386/pr88713-1.c b/gcc/testsuite/gcc.target/i386/pr88713-1.c
new file mode 100644
index 00000000000..26a0da57a41
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/pr88713-1.c
@@ -0,0 +1,13 @@
+/* { dg-do compile } */
+/* { dg-options "-Ofast -mno-avx512f -mfma" } */
+
+extern float sqrtf (float);
+
+void
+rsqrt (float* restrict r, float* restrict a)
+{
+ for (int i = 0; i < 64; i++)
+ r[i] = sqrtf(a[i]);
+}
+
+/* { dg-final { scan-assembler "\tvfmadd\[123\]+ps" } } */
diff --git a/gcc/testsuite/gcc.target/i386/pr88713-2.c b/gcc/testsuite/gcc.target/i386/pr88713-2.c
new file mode 100644
index 00000000000..8b55dfcf924
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/pr88713-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-Ofast -march=skylake-avx512 -mno-fma" } */
+
+#include "pr88713-1.c"
+
+/* { dg-final { scan-assembler "\tvfmadd\[123\]+ps" } } */
--
2.26.2
next prev parent reply other threads:[~2020-07-09 13:36 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-28 14:19 H.J. Lu
2020-07-07 15:56 ` Kirill Yukhin
2020-07-07 16:06 ` H.J. Lu
2020-07-09 12:04 ` Kirill Yukhin
2020-07-09 13:35 ` H.J. Lu [this message]
2020-07-09 22:02 ` [PATCH] x86: Check TARGET_AVX512VL when enabling FMA H.J. Lu
2020-07-10 11:08 ` H.J. Lu
2020-07-10 11:18 ` Jakub Jelinek
2020-07-10 11:28 ` H.J. Lu
2020-07-13 16:12 ` [PATCH] x86: Rename VF_AVX512VL_VF1_128_256 to VF1_AVX512ER_128_256 H.J. Lu
2020-07-17 2:50 ` H.J. Lu
2020-07-17 12:14 ` H.J. Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAMe9rOpEq_W7FxvrHs0yPM+Z4iwHjo12Bf5THrNoj0tA=WDV6A@mail.gmail.com' \
--to=hjl.tools@gmail.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=kirill.yukhin@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).