[gcc r12-3107] Adjust inner loop cost scaling

[gcc r12-3107] Adjust inner loop cost scaling

[Richard Biener]

https://gcc.gnu.org/g:8571ff0ae0922bee292161c7fd61dd127d26a4ed

commit r12-3107-g8571ff0ae0922bee292161c7fd61dd127d26a4ed
Author: Richard Biener <rguenther@suse.de>
Date:   Mon Aug 23 14:15:14 2021 +0200

    Adjust inner loop cost scaling
    
    This makes use of the estimated number of iterations of the inner loop
    to limit --param vect-inner-loop-cost-factor scaling.  It also reduces
    the maximum value of vect-inner-loop-cost-factor to 10000 making it
    less likely to cause overflow of costs.
    
    2021-08-23  Richard Biener  <rguenther@suse.de>
    
            * doc/invoke.texi (vect-inner-loop-cost-factor): Adjust.
            * params.opt (--param vect-inner-loop-cost-factor): Adjust
            maximum value.
            * tree-vect-loop.c (vect_analyze_loop_form): Initialize
            inner_loop_cost_factor to the minimum of the estimated number
            of iterations of the inner loop and vect-inner-loop-cost-factor.

Diff:
---
 gcc/doc/invoke.texi  | 7 ++++---
 gcc/params.opt       | 4 ++--
 gcc/tree-vect-loop.c | 7 +++++++
 3 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index c057cc1e4ae..a9d56fecf4e 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -14386,9 +14386,10 @@ The parameter only has an effect on targets that support partial
 vector loads and stores.
 
 @item vect-inner-loop-cost-factor
-The factor which the loop vectorizer applies to the cost of statements
-in an inner loop relative to the loop being vectorized.  The default
-value is 50.
+The maximum factor which the loop vectorizer applies to the cost of statements
+in an inner loop relative to the loop being vectorized.  The factor applied
+is the maximum of the estimated number of iterations of the inner loop and
+this parameter.  The default value of this parameter is 50.
 
 @item avoid-fma-max-bits
 Maximum number of bits for which we avoid creating FMAs.
diff --git a/gcc/params.opt b/gcc/params.opt
index f9264887b40..f414dc1a61c 100644
--- a/gcc/params.opt
+++ b/gcc/params.opt
@@ -1114,7 +1114,7 @@ Common Joined UInteger Var(param_vect_partial_vector_usage) Init(2) IntegerRange
 Controls how loop vectorizer uses partial vectors.  0 means never, 1 means only for loops whose need to iterate can be removed, 2 means for all loops.  The default value is 2.
 
 -param=vect-inner-loop-cost-factor=
-Common Joined UInteger Var(param_vect_inner_loop_cost_factor) Init(50) IntegerRange(1, 999999) Param Optimization
-The factor which the loop vectorizer applies to the cost of statements in an inner loop relative to the loop being vectorized.
+Common Joined UInteger Var(param_vect_inner_loop_cost_factor) Init(50) IntegerRange(1, 10000) Param Optimization
+The maximum factor which the loop vectorizer applies to the cost of statements in an inner loop relative to the loop being vectorized.
 
 ; This comment is to ensure we retain the blank line above.
diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index c521b43a47c..0c8d992624b 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -1519,6 +1519,13 @@ vect_analyze_loop_form (class loop *loop, vec_info_shared *shared)
       stmt_vec_info inner_loop_cond_info
 	= loop_vinfo->lookup_stmt (inner_loop_cond);
       STMT_VINFO_TYPE (inner_loop_cond_info) = loop_exit_ctrl_vec_info_type;
+      /* If we have an estimate on the number of iterations of the inner
+	 loop use that to limit the scale for costing, otherwise use
+	 --param vect-inner-loop-cost-factor literally.  */
+      widest_int nit;
+      if (estimated_stmt_executions (loop->inner, &nit))
+	LOOP_VINFO_INNER_LOOP_COST_FACTOR (loop_vinfo)
+	  = wi::smin (nit, param_vect_inner_loop_cost_factor).to_uhwi ();
     }
 
   gcc_assert (!loop->aux);