[PATCH] fold-const: Don't consider NaN non-negative [PR97965]

[Jakub Jelinek <jakub@redhat.com>]

Hi!

The testcase in the PR
constexpr bool a = __builtin_nan ("") > 0.0;
constexpr bool b = __builtin_nans ("") > 0.0;
constexpr bool c = __builtin_nan ("") < 0.0;
constexpr bool d = __builtin_nans ("") < 0.0;
constexpr bool e = __builtin_nan ("") >= 0.0;
constexpr bool f = __builtin_nans ("") >= 0.0;
constexpr bool g = __builtin_nan ("") <= 0.0;
constexpr bool h = __builtin_nans ("") <= 0.0;
has inconsistent behavior, we fold c and d initializers to 0 and don't fold
any other comparisons to zero.
Not including the testcase in the testsuite because I really don't know
if it should be accepted or rejected (it is all accepted with
-fno-trapping-math).
The reason we optimize those < 0.0 comparisons to 0 is:
      /* Convert ABS_EXPR<x> < 0 to false.  */
      strict_overflow_p = false;
      if (code == LT_EXPR
          && (integer_zerop (arg1) || real_zerop (arg1))
          && tree_expr_nonnegative_warnv_p (arg0, &strict_overflow_p))
and we return true for __builtin_nan ("") (but not for -__builtin_nan ("").

Now, it just feels wrong to me to say that NaN with sign bit clear is
non-negative, but other than the above inconsistency I haven't been able to
construct a miscompiled testcase, only questionable thing is that
we fold away also comparisons of sNaN < 0.0 - but then while for sNaN >= 0.0
we don't fold that away during gimple optimizations, we fold it during RTL
optimizations at least on x86_64.

So, I really don't know if we want this or not, posting it for discussions.

2020-11-26  Jakub Jelinek  <jakub@redhat.com>

	PR c++/97965
	* fold-const.c (tree_single_nonnegative_warnv_p): Don't return true
	for NaNs with sign bit clear.

--- gcc/fold-const.c.jj	2020-11-24 09:02:25.330419895 +0100
+++ gcc/fold-const.c	2020-11-25 15:37:14.426229476 +0100
@@ -14186,7 +14186,9 @@ tree_single_nonnegative_warnv_p (tree t,
       return tree_int_cst_sgn (t) >= 0;
 
     case REAL_CST:
-      return ! REAL_VALUE_NEGATIVE (TREE_REAL_CST (t));
+      return (! REAL_VALUE_NEGATIVE (TREE_REAL_CST (t))
+	      /* Don't consider NaNs non-negative.  */
+	      && ! REAL_VALUE_ISNAN (TREE_REAL_CST (t)));
 
     case FIXED_CST:
       return ! FIXED_VALUE_NEGATIVE (TREE_FIXED_CST (t));

	Jakub