* [PATCH V3] VECT: Change flow of decrement IV
@ 2023-06-01 4:36 juzhe.zhong
2023-06-01 5:00 ` juzhe.zhong
0 siblings, 1 reply; 9+ messages in thread
From: juzhe.zhong @ 2023-06-01 4:36 UTC (permalink / raw)
To: gcc-patches; +Cc: richard.sandiford, rguenther, linkw, Ju-Zhe Zhong
From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
Follow Richi's suggestion, I change current decrement IV flow from:
do {
remain -= MIN (vf, remain);
} while (remain != 0);
into:
do {
old_remain = remain;
len = MIN (vf, remain);
remain -= vf;
} while (old_remain >= vf);
to enhance SCEV.
Include fixes from kewen.
This patch will need to wait for Kewen's test feedback.
Testing on X86 is on-going
Co-Authored by: Kewen Lin <linkw@linux.ibm.com>
PR tree-optimization/109971
gcc/ChangeLog:
* tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow.
(vect_set_loop_condition_partial_vectors): Ditto.
---
gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++-----------
1 file changed, 25 insertions(+), 11 deletions(-)
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index acf3642ceb2..3f735945e67 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
gimple_stmt_iterator loop_cond_gsi,
rgroup_controls *rgc, tree niters,
tree niters_skip, bool might_wrap_p,
- tree *iv_step)
+ tree *iv_step, tree *compare_step)
{
tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo);
tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo);
@@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
...
vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0);
...
- ivtmp_35 = ivtmp_9 - _36;
+ ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4];
...
- if (ivtmp_35 != 0)
+ if (ivtmp_9 > POLY_INT_CST [4, 4])
goto <bb 4>; [83.33%]
else
goto <bb 5>; [16.67%]
@@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
tree step = rgc->controls.length () == 1 ? rgc->controls[0]
: make_ssa_name (iv_type);
/* Create decrement IV. */
- create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi,
- insert_after, &index_before_incr, &index_after_incr);
+ create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop,
+ &incr_gsi, insert_after, &index_before_incr,
+ &index_after_incr);
gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR,
index_before_incr,
nitems_step));
*iv_step = step;
- return index_after_incr;
+ *compare_step = nitems_step;
+ return index_before_incr;
}
/* Create increment IV. */
@@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
arbitrarily pick the last. */
tree test_ctrl = NULL_TREE;
tree iv_step = NULL_TREE;
+ tree compare_step = NULL_TREE;
rgroup_controls *rgc;
rgroup_controls *iv_rgc = nullptr;
unsigned int i;
@@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
&preheader_seq, &header_seq,
loop_cond_gsi, rgc, niters,
niters_skip, might_wrap_p,
- &iv_step);
+ &iv_step, &compare_step);
iv_rgc = rgc;
}
@@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
/* Get a boolean result that tells us whether to iterate. */
edge exit_edge = single_exit (loop);
- tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
- tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
- gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl,
- NULL_TREE, NULL_TREE);
+ gcond *cond_stmt;
+ if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo))
+ {
+ gcc_assert (compare_step);
+ tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR;
+ cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE,
+ NULL_TREE);
+ }
+ else
+ {
+ tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
+ tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
+ cond_stmt
+ = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE);
+ }
gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT);
/* The loop iterates (NITERS - 1) / VF + 1 times.
--
2.36.3
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-01 4:36 [PATCH V3] VECT: Change flow of decrement IV juzhe.zhong
@ 2023-06-01 5:00 ` juzhe.zhong
2023-06-01 5:24 ` Kewen.Lin
2023-06-02 8:56 ` Richard Biener
0 siblings, 2 replies; 9+ messages in thread
From: juzhe.zhong @ 2023-06-01 5:00 UTC (permalink / raw)
To: 钟居哲, gcc-patches; +Cc: richard.sandiford, rguenther, linkw
[-- Attachment #1: Type: text/plain, Size: 4663 bytes --]
This patch is no difference from V2.
Just add PR tree-optimization/109971 as Kewen's suggested.
Already bootstrapped and Regression on X86 no difference.
Ok for trunk ?
juzhe.zhong@rivai.ai
From: juzhe.zhong
Date: 2023-06-01 12:36
To: gcc-patches
CC: richard.sandiford; rguenther; linkw; Ju-Zhe Zhong
Subject: [PATCH V3] VECT: Change flow of decrement IV
From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
Follow Richi's suggestion, I change current decrement IV flow from:
do {
remain -= MIN (vf, remain);
} while (remain != 0);
into:
do {
old_remain = remain;
len = MIN (vf, remain);
remain -= vf;
} while (old_remain >= vf);
to enhance SCEV.
Include fixes from kewen.
This patch will need to wait for Kewen's test feedback.
Testing on X86 is on-going
Co-Authored by: Kewen Lin <linkw@linux.ibm.com>
PR tree-optimization/109971
gcc/ChangeLog:
* tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow.
(vect_set_loop_condition_partial_vectors): Ditto.
---
gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++-----------
1 file changed, 25 insertions(+), 11 deletions(-)
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index acf3642ceb2..3f735945e67 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
gimple_stmt_iterator loop_cond_gsi,
rgroup_controls *rgc, tree niters,
tree niters_skip, bool might_wrap_p,
- tree *iv_step)
+ tree *iv_step, tree *compare_step)
{
tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo);
tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo);
@@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
...
vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0);
...
- ivtmp_35 = ivtmp_9 - _36;
+ ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4];
...
- if (ivtmp_35 != 0)
+ if (ivtmp_9 > POLY_INT_CST [4, 4])
goto <bb 4>; [83.33%]
else
goto <bb 5>; [16.67%]
@@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
tree step = rgc->controls.length () == 1 ? rgc->controls[0]
: make_ssa_name (iv_type);
/* Create decrement IV. */
- create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi,
- insert_after, &index_before_incr, &index_after_incr);
+ create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop,
+ &incr_gsi, insert_after, &index_before_incr,
+ &index_after_incr);
gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR,
index_before_incr,
nitems_step));
*iv_step = step;
- return index_after_incr;
+ *compare_step = nitems_step;
+ return index_before_incr;
}
/* Create increment IV. */
@@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
arbitrarily pick the last. */
tree test_ctrl = NULL_TREE;
tree iv_step = NULL_TREE;
+ tree compare_step = NULL_TREE;
rgroup_controls *rgc;
rgroup_controls *iv_rgc = nullptr;
unsigned int i;
@@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
&preheader_seq, &header_seq,
loop_cond_gsi, rgc, niters,
niters_skip, might_wrap_p,
- &iv_step);
+ &iv_step, &compare_step);
iv_rgc = rgc;
}
@@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
/* Get a boolean result that tells us whether to iterate. */
edge exit_edge = single_exit (loop);
- tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
- tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
- gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl,
- NULL_TREE, NULL_TREE);
+ gcond *cond_stmt;
+ if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo))
+ {
+ gcc_assert (compare_step);
+ tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR;
+ cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE,
+ NULL_TREE);
+ }
+ else
+ {
+ tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
+ tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
+ cond_stmt
+ = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE);
+ }
gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT);
/* The loop iterates (NITERS - 1) / VF + 1 times.
--
2.36.3
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-01 5:00 ` juzhe.zhong
@ 2023-06-01 5:24 ` Kewen.Lin
2023-06-01 8:09 ` juzhe.zhong
2023-06-02 8:56 ` Richard Biener
1 sibling, 1 reply; 9+ messages in thread
From: Kewen.Lin @ 2023-06-01 5:24 UTC (permalink / raw)
To: juzhe.zhong; +Cc: richard.sandiford, rguenther, gcc-patches
Hi,
on 2023/6/1 13:00, juzhe.zhong@rivai.ai wrote:
> This patch is no difference from V2.
I support this patch based on the testing and SPEC2017 evaluation
results on Power (see my comments on patch v2).
> Just add PR tree-optimization/109971 as Kewen's suggested.
Thanks for adding that, I was expecting you will add that when you
are committing it, not really requesting one new version. :) btw,
the PR marker(s) will trigger scripts to comment some commit info
(commit link, commit log) into the specified PR(s), people can
find some connections between PRs and (fixing or progressing forward)
commits easily.
BR,
Kewen
>
> Already bootstrapped and Regression on X86 no difference.
>
> Ok for trunk ?
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> juzhe.zhong@rivai.ai
>
>
> *From:* juzhe.zhong <mailto:juzhe.zhong@rivai.ai>
> *Date:* 2023-06-01 12:36
> *To:* gcc-patches <mailto:gcc-patches@gcc.gnu.org>
> *CC:* richard.sandiford <mailto:richard.sandiford@arm.com>; rguenther <mailto:rguenther@suse.de>; linkw <mailto:linkw@linux.ibm.com>; Ju-Zhe Zhong <mailto:juzhe.zhong@rivai.ai>
> *Subject:* [PATCH V3] VECT: Change flow of decrement IV
> From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
>
> Follow Richi's suggestion, I change current decrement IV flow from:
>
> do {
> remain -= MIN (vf, remain);
> } while (remain != 0);
>
> into:
>
> do {
> old_remain = remain;
> len = MIN (vf, remain);
> remain -= vf;
> } while (old_remain >= vf);
>
> to enhance SCEV.
>
> Include fixes from kewen.
>
>
> This patch will need to wait for Kewen's test feedback.
>
> Testing on X86 is on-going
>
> Co-Authored by: Kewen Lin <linkw@linux.ibm.com>
>
> PR tree-optimization/109971
>
> gcc/ChangeLog:
>
> * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow.
> (vect_set_loop_condition_partial_vectors): Ditto.
>
> ---
> gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++-----------
> 1 file changed, 25 insertions(+), 11 deletions(-)
>
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index acf3642ceb2..3f735945e67 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> gimple_stmt_iterator loop_cond_gsi,
> rgroup_controls *rgc, tree niters,
> tree niters_skip, bool might_wrap_p,
> - tree *iv_step)
> + tree *iv_step, tree *compare_step)
> {
> tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo);
> tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo);
> @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> ...
> vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0);
> ...
> - ivtmp_35 = ivtmp_9 - _36;
> + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4];
> ...
> - if (ivtmp_35 != 0)
> + if (ivtmp_9 > POLY_INT_CST [4, 4])
> goto <bb 4>; [83.33%]
> else
> goto <bb 5>; [16.67%]
> @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> tree step = rgc->controls.length () == 1 ? rgc->controls[0]
> : make_ssa_name (iv_type);
> /* Create decrement IV. */
> - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi,
> - insert_after, &index_before_incr, &index_after_incr);
> + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop,
> + &incr_gsi, insert_after, &index_before_incr,
> + &index_after_incr);
> gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR,
> index_before_incr,
> nitems_step));
> *iv_step = step;
> - return index_after_incr;
> + *compare_step = nitems_step;
> + return index_before_incr;
> }
> /* Create increment IV. */
> @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> arbitrarily pick the last. */
> tree test_ctrl = NULL_TREE;
> tree iv_step = NULL_TREE;
> + tree compare_step = NULL_TREE;
> rgroup_controls *rgc;
> rgroup_controls *iv_rgc = nullptr;
> unsigned int i;
> @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> &preheader_seq, &header_seq,
> loop_cond_gsi, rgc, niters,
> niters_skip, might_wrap_p,
> - &iv_step);
> + &iv_step, &compare_step);
> iv_rgc = rgc;
> }
> @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> /* Get a boolean result that tells us whether to iterate. */
> edge exit_edge = single_exit (loop);
> - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl,
> - NULL_TREE, NULL_TREE);
> + gcond *cond_stmt;
> + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo))
> + {
> + gcc_assert (compare_step);
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR;
> + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE,
> + NULL_TREE);
> + }
> + else
> + {
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> + cond_stmt
> + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE);
> + }
> gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT);
> /* The loop iterates (NITERS - 1) / VF + 1 times.
> --
> 2.36.3
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-01 5:24 ` Kewen.Lin
@ 2023-06-01 8:09 ` juzhe.zhong
0 siblings, 0 replies; 9+ messages in thread
From: juzhe.zhong @ 2023-06-01 8:09 UTC (permalink / raw)
To: linkw; +Cc: richard.sandiford, rguenther, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 7458 bytes --]
Thanks Kewen. Let's wait for Richard and Richi.
juzhe.zhong@rivai.ai
From: Kewen.Lin
Date: 2023-06-01 13:24
To: juzhe.zhong@rivai.ai
CC: richard.sandiford; rguenther; gcc-patches
Subject: Re: [PATCH V3] VECT: Change flow of decrement IV
Hi,
on 2023/6/1 13:00, juzhe.zhong@rivai.ai wrote:
> This patch is no difference from V2.
I support this patch based on the testing and SPEC2017 evaluation
results on Power (see my comments on patch v2).
> Just add PR tree-optimization/109971 as Kewen's suggested.
Thanks for adding that, I was expecting you will add that when you
are committing it, not really requesting one new version. :) btw,
the PR marker(s) will trigger scripts to comment some commit info
(commit link, commit log) into the specified PR(s), people can
find some connections between PRs and (fixing or progressing forward)
commits easily.
BR,
Kewen
>
> Already bootstrapped and Regression on X86 no difference.
>
> Ok for trunk ?
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> juzhe.zhong@rivai.ai
>
>
> *From:* juzhe.zhong <mailto:juzhe.zhong@rivai.ai>
> *Date:* 2023-06-01 12:36
> *To:* gcc-patches <mailto:gcc-patches@gcc.gnu.org>
> *CC:* richard.sandiford <mailto:richard.sandiford@arm.com>; rguenther <mailto:rguenther@suse.de>; linkw <mailto:linkw@linux.ibm.com>; Ju-Zhe Zhong <mailto:juzhe.zhong@rivai.ai>
> *Subject:* [PATCH V3] VECT: Change flow of decrement IV
> From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
>
> Follow Richi's suggestion, I change current decrement IV flow from:
>
> do {
> remain -= MIN (vf, remain);
> } while (remain != 0);
>
> into:
>
> do {
> old_remain = remain;
> len = MIN (vf, remain);
> remain -= vf;
> } while (old_remain >= vf);
>
> to enhance SCEV.
>
> Include fixes from kewen.
>
>
> This patch will need to wait for Kewen's test feedback.
>
> Testing on X86 is on-going
>
> Co-Authored by: Kewen Lin <linkw@linux.ibm.com>
>
> PR tree-optimization/109971
>
> gcc/ChangeLog:
>
> * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow.
> (vect_set_loop_condition_partial_vectors): Ditto.
>
> ---
> gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++-----------
> 1 file changed, 25 insertions(+), 11 deletions(-)
>
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index acf3642ceb2..3f735945e67 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> gimple_stmt_iterator loop_cond_gsi,
> rgroup_controls *rgc, tree niters,
> tree niters_skip, bool might_wrap_p,
> - tree *iv_step)
> + tree *iv_step, tree *compare_step)
> {
> tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo);
> tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo);
> @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> ...
> vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0);
> ...
> - ivtmp_35 = ivtmp_9 - _36;
> + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4];
> ...
> - if (ivtmp_35 != 0)
> + if (ivtmp_9 > POLY_INT_CST [4, 4])
> goto <bb 4>; [83.33%]
> else
> goto <bb 5>; [16.67%]
> @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> tree step = rgc->controls.length () == 1 ? rgc->controls[0]
> : make_ssa_name (iv_type);
> /* Create decrement IV. */
> - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi,
> - insert_after, &index_before_incr, &index_after_incr);
> + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop,
> + &incr_gsi, insert_after, &index_before_incr,
> + &index_after_incr);
> gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR,
> index_before_incr,
> nitems_step));
> *iv_step = step;
> - return index_after_incr;
> + *compare_step = nitems_step;
> + return index_before_incr;
> }
> /* Create increment IV. */
> @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> arbitrarily pick the last. */
> tree test_ctrl = NULL_TREE;
> tree iv_step = NULL_TREE;
> + tree compare_step = NULL_TREE;
> rgroup_controls *rgc;
> rgroup_controls *iv_rgc = nullptr;
> unsigned int i;
> @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> &preheader_seq, &header_seq,
> loop_cond_gsi, rgc, niters,
> niters_skip, might_wrap_p,
> - &iv_step);
> + &iv_step, &compare_step);
> iv_rgc = rgc;
> }
> @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> /* Get a boolean result that tells us whether to iterate. */
> edge exit_edge = single_exit (loop);
> - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl,
> - NULL_TREE, NULL_TREE);
> + gcond *cond_stmt;
> + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo))
> + {
> + gcc_assert (compare_step);
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR;
> + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE,
> + NULL_TREE);
> + }
> + else
> + {
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> + cond_stmt
> + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE);
> + }
> gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT);
> /* The loop iterates (NITERS - 1) / VF + 1 times.
> --
> 2.36.3
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-01 5:00 ` juzhe.zhong
2023-06-01 5:24 ` Kewen.Lin
@ 2023-06-02 8:56 ` Richard Biener
2023-06-02 8:58 ` juzhe.zhong
1 sibling, 1 reply; 9+ messages in thread
From: Richard Biener @ 2023-06-02 8:56 UTC (permalink / raw)
To: juzhe.zhong; +Cc: gcc-patches, richard.sandiford, linkw
On Thu, 1 Jun 2023, juzhe.zhong@rivai.ai wrote:
> This patch is no difference from V2.
> Just add PR tree-optimization/109971 as Kewen's suggested.
>
> Already bootstrapped and Regression on X86 no difference.
>
> Ok for trunk ?
OK.
Richard.
>
> juzhe.zhong@rivai.ai
>
> From: juzhe.zhong
> Date: 2023-06-01 12:36
> To: gcc-patches
> CC: richard.sandiford; rguenther; linkw; Ju-Zhe Zhong
> Subject: [PATCH V3] VECT: Change flow of decrement IV
> From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
>
> Follow Richi's suggestion, I change current decrement IV flow from:
>
> do {
> remain -= MIN (vf, remain);
> } while (remain != 0);
>
> into:
>
> do {
> old_remain = remain;
> len = MIN (vf, remain);
> remain -= vf;
> } while (old_remain >= vf);
>
> to enhance SCEV.
>
> Include fixes from kewen.
>
>
> This patch will need to wait for Kewen's test feedback.
>
> Testing on X86 is on-going
>
> Co-Authored by: Kewen Lin <linkw@linux.ibm.com>
>
> PR tree-optimization/109971
>
> gcc/ChangeLog:
>
> * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow.
> (vect_set_loop_condition_partial_vectors): Ditto.
>
> ---
> gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++-----------
> 1 file changed, 25 insertions(+), 11 deletions(-)
>
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index acf3642ceb2..3f735945e67 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> gimple_stmt_iterator loop_cond_gsi,
> rgroup_controls *rgc, tree niters,
> tree niters_skip, bool might_wrap_p,
> - tree *iv_step)
> + tree *iv_step, tree *compare_step)
> {
> tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo);
> tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo);
> @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> ...
> vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0);
> ...
> - ivtmp_35 = ivtmp_9 - _36;
> + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4];
> ...
> - if (ivtmp_35 != 0)
> + if (ivtmp_9 > POLY_INT_CST [4, 4])
> goto <bb 4>; [83.33%]
> else
> goto <bb 5>; [16.67%]
> @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> tree step = rgc->controls.length () == 1 ? rgc->controls[0]
> : make_ssa_name (iv_type);
> /* Create decrement IV. */
> - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi,
> - insert_after, &index_before_incr, &index_after_incr);
> + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop,
> + &incr_gsi, insert_after, &index_before_incr,
> + &index_after_incr);
> gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR,
> index_before_incr,
> nitems_step));
> *iv_step = step;
> - return index_after_incr;
> + *compare_step = nitems_step;
> + return index_before_incr;
> }
> /* Create increment IV. */
> @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> arbitrarily pick the last. */
> tree test_ctrl = NULL_TREE;
> tree iv_step = NULL_TREE;
> + tree compare_step = NULL_TREE;
> rgroup_controls *rgc;
> rgroup_controls *iv_rgc = nullptr;
> unsigned int i;
> @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> &preheader_seq, &header_seq,
> loop_cond_gsi, rgc, niters,
> niters_skip, might_wrap_p,
> - &iv_step);
> + &iv_step, &compare_step);
> iv_rgc = rgc;
> }
> @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> /* Get a boolean result that tells us whether to iterate. */
> edge exit_edge = single_exit (loop);
> - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl,
> - NULL_TREE, NULL_TREE);
> + gcond *cond_stmt;
> + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo))
> + {
> + gcc_assert (compare_step);
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR;
> + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE,
> + NULL_TREE);
> + }
> + else
> + {
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> + cond_stmt
> + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE);
> + }
> gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT);
> /* The loop iterates (NITERS - 1) / VF + 1 times.
>
--
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-02 8:56 ` Richard Biener
@ 2023-06-02 8:58 ` juzhe.zhong
2023-06-02 11:43 ` Richard Sandiford
0 siblings, 1 reply; 9+ messages in thread
From: juzhe.zhong @ 2023-06-02 8:58 UTC (permalink / raw)
To: rguenther; +Cc: gcc-patches, richard.sandiford, linkw
[-- Attachment #1: Type: text/plain, Size: 5488 bytes --]
Thanks Richi. I am gonna merge it after Richard's final approve.
juzhe.zhong@rivai.ai
From: Richard Biener
Date: 2023-06-02 16:56
To: juzhe.zhong@rivai.ai
CC: gcc-patches; richard.sandiford; linkw
Subject: Re: [PATCH V3] VECT: Change flow of decrement IV
On Thu, 1 Jun 2023, juzhe.zhong@rivai.ai wrote:
> This patch is no difference from V2.
> Just add PR tree-optimization/109971 as Kewen's suggested.
>
> Already bootstrapped and Regression on X86 no difference.
>
> Ok for trunk ?
OK.
Richard.
>
> juzhe.zhong@rivai.ai
>
> From: juzhe.zhong
> Date: 2023-06-01 12:36
> To: gcc-patches
> CC: richard.sandiford; rguenther; linkw; Ju-Zhe Zhong
> Subject: [PATCH V3] VECT: Change flow of decrement IV
> From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
>
> Follow Richi's suggestion, I change current decrement IV flow from:
>
> do {
> remain -= MIN (vf, remain);
> } while (remain != 0);
>
> into:
>
> do {
> old_remain = remain;
> len = MIN (vf, remain);
> remain -= vf;
> } while (old_remain >= vf);
>
> to enhance SCEV.
>
> Include fixes from kewen.
>
>
> This patch will need to wait for Kewen's test feedback.
>
> Testing on X86 is on-going
>
> Co-Authored by: Kewen Lin <linkw@linux.ibm.com>
>
> PR tree-optimization/109971
>
> gcc/ChangeLog:
>
> * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow.
> (vect_set_loop_condition_partial_vectors): Ditto.
>
> ---
> gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++-----------
> 1 file changed, 25 insertions(+), 11 deletions(-)
>
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index acf3642ceb2..3f735945e67 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> gimple_stmt_iterator loop_cond_gsi,
> rgroup_controls *rgc, tree niters,
> tree niters_skip, bool might_wrap_p,
> - tree *iv_step)
> + tree *iv_step, tree *compare_step)
> {
> tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo);
> tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo);
> @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> ...
> vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0);
> ...
> - ivtmp_35 = ivtmp_9 - _36;
> + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4];
> ...
> - if (ivtmp_35 != 0)
> + if (ivtmp_9 > POLY_INT_CST [4, 4])
> goto <bb 4>; [83.33%]
> else
> goto <bb 5>; [16.67%]
> @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo,
> tree step = rgc->controls.length () == 1 ? rgc->controls[0]
> : make_ssa_name (iv_type);
> /* Create decrement IV. */
> - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi,
> - insert_after, &index_before_incr, &index_after_incr);
> + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop,
> + &incr_gsi, insert_after, &index_before_incr,
> + &index_after_incr);
> gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR,
> index_before_incr,
> nitems_step));
> *iv_step = step;
> - return index_after_incr;
> + *compare_step = nitems_step;
> + return index_before_incr;
> }
> /* Create increment IV. */
> @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> arbitrarily pick the last. */
> tree test_ctrl = NULL_TREE;
> tree iv_step = NULL_TREE;
> + tree compare_step = NULL_TREE;
> rgroup_controls *rgc;
> rgroup_controls *iv_rgc = nullptr;
> unsigned int i;
> @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> &preheader_seq, &header_seq,
> loop_cond_gsi, rgc, niters,
> niters_skip, might_wrap_p,
> - &iv_step);
> + &iv_step, &compare_step);
> iv_rgc = rgc;
> }
> @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop,
> /* Get a boolean result that tells us whether to iterate. */
> edge exit_edge = single_exit (loop);
> - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl,
> - NULL_TREE, NULL_TREE);
> + gcond *cond_stmt;
> + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo))
> + {
> + gcc_assert (compare_step);
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR;
> + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE,
> + NULL_TREE);
> + }
> + else
> + {
> + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR;
> + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl));
> + cond_stmt
> + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE);
> + }
> gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT);
> /* The loop iterates (NITERS - 1) / VF + 1 times.
>
--
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-02 8:58 ` juzhe.zhong
@ 2023-06-02 11:43 ` Richard Sandiford
2023-06-02 11:50 ` Li, Pan2
2023-06-06 8:02 ` juzhe.zhong
0 siblings, 2 replies; 9+ messages in thread
From: Richard Sandiford @ 2023-06-02 11:43 UTC (permalink / raw)
To: juzhe.zhong; +Cc: rguenther, gcc-patches, linkw
"juzhe.zhong@rivai.ai" <juzhe.zhong@rivai.ai> writes:
> Thanks Richi. I am gonna merge it after Richard's final approve.
Thanks for checking, but no need to wait for a second ack from me!
Please go ahead and commit.
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [PATCH V3] VECT: Change flow of decrement IV
2023-06-02 11:43 ` Richard Sandiford
@ 2023-06-02 11:50 ` Li, Pan2
2023-06-06 8:02 ` juzhe.zhong
1 sibling, 0 replies; 9+ messages in thread
From: Li, Pan2 @ 2023-06-02 11:50 UTC (permalink / raw)
To: Richard Sandiford, juzhe.zhong; +Cc: rguenther, gcc-patches, linkw
Committed, thanks all.
Pan
-----Original Message-----
From: Gcc-patches <gcc-patches-bounces+pan2.li=intel.com@gcc.gnu.org> On Behalf Of Richard Sandiford via Gcc-patches
Sent: Friday, June 2, 2023 7:44 PM
To: juzhe.zhong@rivai.ai
Cc: rguenther <rguenther@suse.de>; gcc-patches <gcc-patches@gcc.gnu.org>; linkw <linkw@linux.ibm.com>
Subject: Re: [PATCH V3] VECT: Change flow of decrement IV
"juzhe.zhong@rivai.ai" <juzhe.zhong@rivai.ai> writes:
> Thanks Richi. I am gonna merge it after Richard's final approve.
Thanks for checking, but no need to wait for a second ack from me!
Please go ahead and commit.
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re: [PATCH V3] VECT: Change flow of decrement IV
2023-06-02 11:43 ` Richard Sandiford
2023-06-02 11:50 ` Li, Pan2
@ 2023-06-06 8:02 ` juzhe.zhong
1 sibling, 0 replies; 9+ messages in thread
From: juzhe.zhong @ 2023-06-06 8:02 UTC (permalink / raw)
To: richard.sandiford; +Cc: rguenther, gcc-patches, linkw
[-- Attachment #1: Type: text/plain, Size: 830 bytes --]
Hi, Richard and Richi.
Recently, we have some tries on our downstream RVV LLVM.
We change "get_vector_length" (same IR like GCC "select_vl") into "umin", turns out LLVM SCEV analysis succeed.
The unroll PASS works in LLVM.
I think Richard's suggestion is very reasonable.
I didn't have the time take a look at how LLVM do now but I will take a look at it in the future.
Thanks.
juzhe.zhong@rivai.ai
From: Richard Sandiford
Date: 2023-06-02 19:43
To: juzhe.zhong\@rivai.ai
CC: rguenther; gcc-patches; linkw
Subject: Re: [PATCH V3] VECT: Change flow of decrement IV
"juzhe.zhong@rivai.ai" <juzhe.zhong@rivai.ai> writes:
> Thanks Richi. I am gonna merge it after Richard's final approve.
Thanks for checking, but no need to wait for a second ack from me!
Please go ahead and commit.
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-06-06 8:02 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-06-01 4:36 [PATCH V3] VECT: Change flow of decrement IV juzhe.zhong
2023-06-01 5:00 ` juzhe.zhong
2023-06-01 5:24 ` Kewen.Lin
2023-06-01 8:09 ` juzhe.zhong
2023-06-02 8:56 ` Richard Biener
2023-06-02 8:58 ` juzhe.zhong
2023-06-02 11:43 ` Richard Sandiford
2023-06-02 11:50 ` Li, Pan2
2023-06-06 8:02 ` juzhe.zhong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).