[gcc r10-9408] tree-optimization/98758 - fix integer arithmetic in data-ref analysis

[gcc r10-9408] tree-optimization/98758 - fix integer arithmetic in data-ref analysis

[Richard Biener]

https://gcc.gnu.org/g:85977f624a34eac309f9d77a58164553dfc82975

commit r10-9408-g85977f624a34eac309f9d77a58164553dfc82975
Author: Richard Biener <rguenther@suse.de>
Date:   Wed Jan 20 08:48:34 2021 +0100

    tree-optimization/98758 - fix integer arithmetic in data-ref analysis
    
    This fixes some int arithmetic issues and a bogus truncation.
    
    2021-01-20  Richard Biener  <rguenther@suse.de>
    
            PR tree-optimization/98758
            * tree-data-ref.c (int_divides_p): Use lambda_int arguments.
            (lambda_matrix_right_hermite): Avoid undefinedness with
            signed integer abs and multiplication.
            (analyze_subscript_affine_affine): Use lambda_int.
    
            * gcc.dg/torture/pr98758.c: New testcase.
    
    (cherry picked from commit 34599780d0de72faf5719ea08d11a061722b9d19)

Diff:
---
 gcc/testsuite/gcc.dg/torture/pr98758.c | 13 +++++++++++++
 gcc/tree-data-ref.c                    | 12 ++++++------
 2 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/gcc/testsuite/gcc.dg/torture/pr98758.c b/gcc/testsuite/gcc.dg/torture/pr98758.c
new file mode 100644
index 00000000000..7b9fdb2b613
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/torture/pr98758.c
@@ -0,0 +1,13 @@
+/* { dg-do compile } */
+
+long *a, *b;
+long c;
+void d(void)
+{
+  b = a;
+  while (c) {
+    *a = (__INTPTR_TYPE__)(a += (long)1 << (sizeof(long) * 8 - 10));
+    c = b[0];
+    b = a;
+  }
+}
diff --git a/gcc/tree-data-ref.c b/gcc/tree-data-ref.c
index 5505ba46778..2cb54def860 100644
--- a/gcc/tree-data-ref.c
+++ b/gcc/tree-data-ref.c
@@ -141,7 +141,7 @@ tree_fold_divides_p (const_tree a, const_tree b)
 /* Returns true iff A divides B.  */
 
 static inline bool
-int_divides_p (int a, int b)
+int_divides_p (lambda_int a, lambda_int b)
 {
   return ((b % a) == 0);
 }
@@ -4019,10 +4019,10 @@ lambda_matrix_right_hermite (lambda_matrix A, int m, int n,
 
 		  a = S[i-1][j];
 		  b = S[i][j];
-		  sigma = (a * b < 0) ? -1: 1;
-		  a = abs_hwi (a);
-		  b = abs_hwi (b);
-		  factor = sigma * (a / b);
+		  sigma = ((a < 0) ^ (b < 0)) ? -1: 1;
+		  unsigned HOST_WIDE_INT abs_a = absu_hwi (a);
+		  unsigned HOST_WIDE_INT abs_b = absu_hwi (b);
+		  factor = sigma * (lambda_int)(abs_a / abs_b);
 
 		  lambda_matrix_row_add (S, n, i, i-1, -factor);
 		  std::swap (S[i], S[i-1]);
@@ -4048,7 +4048,7 @@ analyze_subscript_affine_affine (tree chrec_a,
 				 tree *last_conflicts)
 {
   unsigned nb_vars_a, nb_vars_b, dim;
-  HOST_WIDE_INT gamma, gcd_alpha_beta;
+  lambda_int gamma, gcd_alpha_beta;
   lambda_matrix A, U, S;
   struct obstack scratch_obstack;