From: "juzhe.zhong@rivai.ai" <juzhe.zhong@rivai.ai>
To: richard.sandiford <richard.sandiford@arm.com>
Cc: gcc-patches <gcc-patches@gcc.gnu.org>, rguenther <rguenther@suse.de>
Subject: Re: Re: [PATCH V2] VECT: Support CALL vectorization for COND_LEN_*
Date: Mon, 31 Jul 2023 19:52:20 +0800 [thread overview]
Message-ID: <B964B3B247FA0D02+2023073119521950775318@rivai.ai> (raw)
In-Reply-To: <mpttttk9ye0.fsf@arm.com>
[-- Attachment #1: Type: text/plain, Size: 5255 bytes --]
Ok . Thanks Richard.
Could you give me a case that SVE can vectorize a reduction with FMA?
Meaning it will go into vectorize_call and vectorize FMA into COND_FMA ?
I tried many times to reproduce such cases but I failed.
Thanks.
juzhe.zhong@rivai.ai
From: Richard Sandiford
Date: 2023-07-31 18:19
To: Juzhe-Zhong
CC: gcc-patches; rguenther
Subject: Re: [PATCH V2] VECT: Support CALL vectorization for COND_LEN_*
Juzhe-Zhong <juzhe.zhong@rivai.ai> writes:
> Hi, Richard and Richi.
>
> Base on the suggestions from Richard:
> https://gcc.gnu.org/pipermail/gcc-patches/2023-July/625396.html
>
> This patch choose (1) approach that Richard provided, meaning:
>
> RVV implements cond_* optabs as expanders. RVV therefore supports
> both IFN_COND_ADD and IFN_COND_LEN_ADD. No dummy length arguments
> are needed at the gimple level.
>
> Such approach can make codes much cleaner and reasonable.
>
> Consider this following case:
> void foo (float * __restrict a, float * __restrict b, int * __restrict cond, int n)
> {
> for (int i = 0; i < n; i++)
> if (cond[i])
> a[i] = b[i] + a[i];
> }
>
>
> Output of RISC-V (32-bits) gcc (trunk) (Compiler #3)
> <source>:5:21: missed: couldn't vectorize loop
> <source>:5:21: missed: not vectorized: control flow in loop.
>
> ARM SVE:
>
> ...
> mask__27.10_51 = vect__4.9_49 != { 0, ... };
> ...
> vec_mask_and_55 = loop_mask_49 & mask__27.10_51;
> ...
> vect__9.17_62 = .COND_ADD (vec_mask_and_55, vect__6.13_56, vect__8.16_60, vect__6.13_56);
>
> For RVV, we want IR as follows:
>
> ...
> _68 = .SELECT_VL (ivtmp_66, POLY_INT_CST [4, 4]);
> ...
> mask__27.10_51 = vect__4.9_49 != { 0, ... };
> ...
> vect__9.17_60 = .COND_LEN_ADD (mask__27.10_51, vect__6.13_55, vect__8.16_59, vect__6.13_55, _68, 0);
> ...
>
> Both len and mask of COND_LEN_ADD are real not dummy.
>
> This patch has been fully tested in RISC-V port with supporting both COND_* and COND_LEN_*.
>
> And also, Bootstrap and Regression on X86 passed.
>
> OK for trunk?
>
> gcc/ChangeLog:
>
> * internal-fn.cc (FOR_EACH_LEN_FN_PAIR): New macro.
> (get_len_internal_fn): New function.
> (CASE): Ditto.
> * internal-fn.h (get_len_internal_fn): Ditto.
> * tree-vect-stmts.cc (vectorizable_call): Support CALL vectorization with COND_LEN_*.
>
> ---
> gcc/internal-fn.cc | 46 ++++++++++++++++++++++
> gcc/internal-fn.h | 1 +
> gcc/tree-vect-stmts.cc | 87 +++++++++++++++++++++++++++++++++++++-----
> 3 files changed, 125 insertions(+), 9 deletions(-)
>
> diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
> index 8e294286388..379220bebc7 100644
> --- a/gcc/internal-fn.cc
> +++ b/gcc/internal-fn.cc
> @@ -4443,6 +4443,52 @@ get_conditional_internal_fn (internal_fn fn)
> }
> }
>
> +/* Invoke T(IFN) for each internal function IFN that also has an
> + IFN_COND_LEN_* or IFN_MASK_LEN_* form. */
> +#define FOR_EACH_LEN_FN_PAIR(T) \
> + T (MASK_LOAD, MASK_LEN_LOAD) \
> + T (MASK_STORE, MASK_LEN_STORE) \
> + T (MASK_GATHER_LOAD, MASK_LEN_GATHER_LOAD) \
> + T (MASK_SCATTER_STORE, MASK_LEN_SCATTER_STORE) \
> + T (COND_ADD, COND_LEN_ADD) \
> + T (COND_SUB, COND_LEN_SUB) \
> + T (COND_MUL, COND_LEN_MUL) \
> + T (COND_DIV, COND_LEN_DIV) \
> + T (COND_MOD, COND_LEN_MOD) \
> + T (COND_RDIV, COND_LEN_RDIV) \
> + T (COND_FMIN, COND_LEN_FMIN) \
> + T (COND_FMAX, COND_LEN_FMAX) \
> + T (COND_MIN, COND_LEN_MIN) \
> + T (COND_MAX, COND_LEN_MAX) \
> + T (COND_AND, COND_LEN_AND) \
> + T (COND_IOR, COND_LEN_IOR) \
> + T (COND_XOR, COND_LEN_XOR) \
> + T (COND_SHL, COND_LEN_SHL) \
> + T (COND_SHR, COND_LEN_SHR) \
> + T (COND_NEG, COND_LEN_NEG) \
> + T (COND_FMA, COND_LEN_FMA) \
> + T (COND_FMS, COND_LEN_FMS) \
> + T (COND_FNMA, COND_LEN_FNMA) \
> + T (COND_FNMS, COND_LEN_FNMS)
With the earlier patch to add DEF_INTERNAL_COND_FN and
DEF_INTERNAL_SIGNED_COND_FN, I think we should use those to handle
the COND_* cases, rather than putting them in this macro.
Thanks,
Richard
next prev parent reply other threads:[~2023-07-31 11:52 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-28 7:10 Juzhe-Zhong
2023-07-31 9:26 ` Richard Biener
2023-07-31 9:52 ` juzhe.zhong
2023-07-31 10:45 ` Richard Biener
2023-07-31 10:59 ` juzhe.zhong
2023-07-31 10:19 ` Richard Sandiford
2023-07-31 11:52 ` juzhe.zhong [this message]
2023-07-31 12:00 ` Richard Biener
2023-07-31 12:09 ` juzhe.zhong
2023-07-31 13:33 ` Richard Biener
2023-07-31 13:43 ` 钟居哲
2023-07-31 13:58 ` Richard Biener
2023-07-31 14:29 ` 钟居哲
2023-08-02 7:49 ` Richard Biener
2023-08-02 8:29 ` juzhe.zhong
2023-08-02 8:33 ` Richard Biener
2023-08-02 8:46 ` juzhe.zhong
2023-08-03 2:38 ` juzhe.zhong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=B964B3B247FA0D02+2023073119521950775318@rivai.ai \
--to=juzhe.zhong@rivai.ai \
--cc=gcc-patches@gcc.gnu.org \
--cc=rguenther@suse.de \
--cc=richard.sandiford@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).