[PATCH]middle-end: correct loop bounds for early breaks and peeled vector loops

[PATCH]middle-end: correct loop bounds for early breaks and peeled vector loops

[Tamar Christina]

[-- Attachment #1: Type: text/plain, Size: 1528 bytes --]

Hi All,

While waiting for reviews I've continued to run more test.
In this case this was one found running 32-bit systems.

While we calculate the right latch count for the epilog,
the vectorizer overrides SCEV and so unrolling goes wrong.

This updates the bounds for the case where we've peeled a
vector iteration.

Testcase in early break testsuite adjusted to test for this
and I'll merge this commit in the main one.

Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.

Ok for master?

Thanks,
Tamar

gcc/ChangeLog:

	* tree-vect-loop-manip.cc (vect_do_peeling): Adjust bounds.

--- inline copy of patch -- 
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
       if (bound_scalar.is_constant (&bound))
 	{
 	  gcc_assert (bound != 0);
+	  /* Adjust the upper bound by the extra peeled vector iteration if we
+	     are an epilogue of an peeled vect loop and not VLA.  For VLA the
+	     loop bounds are unknown.  */
+	  if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)
+	      && vf.is_constant ())
+	    bound += vf.to_constant ();
 	  /* -1 to convert loop iterations to latch iterations.  */
 	  record_niter_bound (epilog, bound - 1, false, true);
 	  scale_loop_profile (epilog, profile_probability::always (),




-- 

[-- Attachment #2: rb18055.patch --]
[-- Type: text/plain, Size: 896 bytes --]

diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
       if (bound_scalar.is_constant (&bound))
 	{
 	  gcc_assert (bound != 0);
+	  /* Adjust the upper bound by the extra peeled vector iteration if we
+	     are an epilogue of an peeled vect loop and not VLA.  For VLA the
+	     loop bounds are unknown.  */
+	  if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)
+	      && vf.is_constant ())
+	    bound += vf.to_constant ();
 	  /* -1 to convert loop iterations to latch iterations.  */
 	  record_niter_bound (epilog, bound - 1, false, true);
 	  scale_loop_profile (epilog, profile_probability::always (),

Re: [PATCH]middle-end: correct loop bounds for early breaks and peeled vector loops

[Richard Biener]

On Wed, 6 Dec 2023, Tamar Christina wrote:

> Hi All,
> 
> While waiting for reviews I've continued to run more test.
> In this case this was one found running 32-bit systems.
> 
> While we calculate the right latch count for the epilog,
> the vectorizer overrides SCEV and so unrolling goes wrong.
> 
> This updates the bounds for the case where we've peeled a
> vector iteration.
> 
> Testcase in early break testsuite adjusted to test for this
> and I'll merge this commit in the main one.
> 
> Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.
> 
> Ok for master?

OK.

> Thanks,
> Tamar
> 
> gcc/ChangeLog:
> 
> 	* tree-vect-loop-manip.cc (vect_do_peeling): Adjust bounds.
> 
> --- inline copy of patch -- 
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
>        if (bound_scalar.is_constant (&bound))
>  	{
>  	  gcc_assert (bound != 0);
> +	  /* Adjust the upper bound by the extra peeled vector iteration if we
> +	     are an epilogue of an peeled vect loop and not VLA.  For VLA the
> +	     loop bounds are unknown.  */
> +	  if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)
> +	      && vf.is_constant ())
> +	    bound += vf.to_constant ();
>  	  /* -1 to convert loop iterations to latch iterations.  */
>  	  record_niter_bound (epilog, bound - 1, false, true);
>  	  scale_loop_profile (epilog, profile_probability::always (),
> 
> 
> 
> 
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)