[Bug middle-end/106819] New: [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[Bug middle-end/106819] New: [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[ibuclaw at gdcproject dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

            Bug ID: 106819
           Summary: [13 Regression] NaN != NaN comparisons return false at
                    -O2 since r13-2338
           Product: gcc
           Version: 13.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: middle-end
          Assignee: unassigned at gcc dot gnu.org
          Reporter: ibuclaw at gdcproject dot org
  Target Milestone: ---

Isolated, and translated to C from the D testsuite, likely can be reduced
further, as I don't think opCmpProper() affects the outcome.

---
static int isNaN(double x)
{
    return x != x;
}

static double opCmpProper(int lhs, double rhs)
{
  return lhs < rhs ? -1.0
       : lhs > rhs ? 1.0
       : lhs == rhs ? 0.0
       : __builtin_nan("");
}

int main()
{
    if (!isNaN(opCmpProper(41, __builtin_nan(""))))
      __builtin_abort();
    return 0;
}

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |13.0
           Priority|P3                          |P1
                 CC|aldyh at redhat dot com            |aldyh at gcc dot gnu.org,
                   |                            |jakub at gcc dot gnu.org

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Global Exported: iftmp.2_6 = [frange] double [0.0, 0.0] !SIGN
Folding PHI node: iftmp.2_6 = PHI <0.0(4),  Nan(5)>
Queued PHI for removal.  Folds to: 0.0

doesn't look correct.  It is a range [0,0] or NaN, so it shouldn't be
singleton_p and shouldn't fold to 0.0.
BTW, regarding sign, generally NaNs can have either sign, though in this
testcase we know the sign is clear (positive NaN).  Not sure how much we can
rely on that though, say if the NaN would be sNaN with known sign bit set, it
could raise an exception and be replaced by canonical qNaN which doesn't have
sign set.

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

Aldy Hernandez <aldyh at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |aldyh at gcc dot gnu.org
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2022-09-03

--- Comment #2 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
NANs can never be a singleton, otherwise they are candidates for propagation. 
That was the original intent, and then I mistakenly put the signed zero check
before it.

Ughh, that's embarrassing.  

Jakub, thanks for analyzing this.

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #3 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
Created attachment 53532
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=53532&action=edit
patch in testing

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #4 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #1)
> Global Exported: iftmp.2_6 = [frange] double [0.0, 0.0] !SIGN
> Folding PHI node: iftmp.2_6 = PHI <0.0(4),  Nan(5)>
> Queued PHI for removal.  Folds to: 0.0
> 
> doesn't look correct.  It is a range [0,0] or NaN, so it shouldn't be
> singleton_p and shouldn't fold to 0.0.
> BTW, regarding sign, generally NaNs can have either sign, though in this
> testcase we know the sign is clear (positive NaN).  Not sure how much we can
> rely on that though, say if the NaN would be sNaN with known sign bit set,
> it could raise an exception and be replaced by canonical qNaN which doesn't
> have sign set.

Hmmm, I haven't really thought about signs on NaNs:

    # iftmp.2_6 = PHI <0.0(4),  Nan(5)>

iftmp.2_6 : [frange] double [0.0, 0.0] !SIGN 

This means we have a known positive 0.0 or a NAN, but the !SIGN only applies to
the range not the NAN.

Out of curiosity, how would be know a NAN is positive or negative?  Is there a
builtin that tells us that?  Perhaps we could use another frange property to
model that (NAN_SIGN??).

I'm way out of my depth here (as usual -)).

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

Aldy Hernandez <aldyh at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #5 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
fixed in mainline

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
You can get a NaN with negative sign e.g. through copysign (__builtin_nan (""),
-1.0).

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #7 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #1)

> BTW, regarding sign, generally NaNs can have either sign, though in this
> testcase we know the sign is clear (positive NaN).  Not sure how much we can
> rely on that though, say if the NaN would be sNaN with known sign bit set,
> it could raise an exception and be replaced by canonical qNaN which doesn't
> have sign set.

I take that back.  There's no reason why we can't keep the signbit up to date
wrt NANs.  For that matter, we're doing pretty good.  I just found one case
where unioning a NAN and a range with a known sign bit clobbers the NAN. 
Everything else is working, so I think we should just fix that and then you can
do whatever you need with signalling nans and what have you.

I still have to implement folding of __builtin_copysign, but that's trivial.

?? And if we keep NAN signs up to date, maybe just maybe, we can return a NAN
from singleton_p() if we're sure about the sign.  ISTM we should be able to
propagate a known NAN with SIGN.  Or is there some funky floating point
property where we can't propagate those?

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #8 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
Created attachment 53534
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=53534&action=edit
Do not clobber signbit when unioning a NAN.

Untested patch to maintain signbit when unioning a NAN with another range.

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
(In reply to Aldy Hernandez from comment #7)
> ?? And if we keep NAN signs up to date, maybe just maybe, we can return a
> NAN from singleton_p() if we're sure about the sign.  ISTM we should be able
> to propagate a known NAN with SIGN.  Or is there some funky floating point
> property where we can't propagate those?

No, NaNs have still many different representations even disregarding the sign
bit differences.
There are e.g. 2^23-1 different sNaNs and 2^23 different qNaNs in binary32
format.

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #10 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
Got it. Less work for me :-). Thanks for the explanation.

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[aldyh at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #11 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
I'll just fix union and implement copysign folding and leave it at that.

[Bug middle-end/106819] [13 Regression] NaN != NaN comparisons return false at -O2 since r13-2338

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106819

--- Comment #12 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #9)
> There are e.g. 2^23-1 different sNaNs and 2^23 different qNaNs in binary32
> format.

2^23-2 sNaNs actually ;)