[Bug math/4997] lround(nexafter(0.5,-1)) has incorrect value

["rsa at us dot ibm dot com" <sourceware-bugzilla@sourceware.org>]

------- Additional Comments From rsa at us dot ibm dot com  2007-11-09 13:45 -------
Francois-Xavier, The bug doesn't manifest for values nextafter(1.5,-1),
nextafter(2.5,-1), etc because nextafter(0.5,-1) just happens to be encoded with
all 1's in the mantissa, e.g. 0x1.fffffffffffffp-2 and the shift and carry
results in an erroneous round-up due to the fadd instruction.  All 1's in the
mantissa is not the resultant encoding for nextafter(1.5,-1)
[0x1.7ffffffffffffp+0] or others.

The other cases where there are all 1's in the mantissa are:
0x1.fffffffffffffp-1 == .999999 which should rightly round up.
0x1.fffffffffffffp+0 == 1.99999 which should rightly round up, etc.

So nextafter(+0.5,-1) is the only case for lround where all 1's in the mantissa
should round down instead of up.

We fixed a couple bugs where a stale register was being passed to fctiwz.  Also
the .Lretzero branch should be 'less than' per the comments and not 'less than
or equal' as in the previous asm code.  Finally we reordered the initial
instructions for data dependence streamlining.

        lis 9,.LC1@ha
        lfs 10,.LC1@l(9)        /* Load constant 0.5 into fpr10.  */
        fabs    2, 1            /* Get the absolute value of x.  */
        fsub    12,10,10        /* Compute 0.0.  */
        fcmpu   6, 2, 10        /* if |x| < 0.5  */
        fcmpu   3, 1, 12        /* x is negative? x < 0.0  */
        blt-    6,.Lretzero
        fadd    3,2,10          /* |x|+=0.5 bias to prepare to round.  */
        bge     3,.Lconvert     /* x is positive so don't negate x.  */
        fnabs   3,3             /* -(|x|+=0.5)  */
.Lconvert:
        fctiwz  4,3             /* Convert to Integer word lround toward 0.  */
        stfd    4,8(1)
        nop
        nop
        nop
        lwz     3,12(1)         /* Load return as integer.  */
.Lout:
        addi    1,1,16
        blr
.Lretzero:                      /* 0.5 > x > -0.5  */
        li      3,0             /* return 0.  */

I still have to add the case where x is between 2^52 and 2^53-1 to cover the
llround case.

Expect  0: Got  0 = lround(nextafter(0.5,-1) [0x1.fffffffffffffp-2]
{0.4999999999999999})
Expect  0: Got  0 = lround(nextafter(-0.5,1) [-0x1.fffffffffffffp-2]
{-0.4999999999999999})
Expect  1: Got  1 = lround(nextafter(1.5,-1) [0x1.7ffffffffffffp+0]
{1.4999999999999998})
Expect -1: Got -1 = lround(nextafter(-1.5,1) [-0x1.7ffffffffffffp+0]
{-1.4999999999999998})
Expect  1: Got  1 = lround(0.5 [0x1.0000000000000p-1] {0.5000000000000000})
Expect  2: Got  2 = lround(1.5 [0x1.8000000000000p+0] {1.5000000000000000})
Expect  3: Got  3 = lround(2.5 [0x1.4000000000000p+1] {2.5000000000000000})
Expect  4: Got  4 = lround(3.5 [0x1.c000000000000p+1] {3.5000000000000000})
Expect  0: Got  0 = lround(0.250000000000000 [0x1.fffffffffffffp-3]
{0.2500000000000000})
Expect  0: Got  0 = lround(-0.250000000000000 [-0x1.fffffffffffffp-3]
{-0.2500000000000000})
Expect  0: Got  0 = lround(0.4999999999999999 [0x1.fffffffffffffp-2]
{0.4999999999999999})
Expect  0: Got  0 = lround(-0.4999999999999999 [-0x1.fffffffffffffp-2]
{-0.4999999999999999})
Expect  1: Got  1 = lround(0.9999999999999999 [0x1.fffffffffffffp-1]
{0.9999999999999999})
Expect -1: Got -1 = lround(-0.9999999999999999 [-0x1.fffffffffffffp-1]
{-0.9999999999999999})
Expect  2: Got  2 = lround(1.9999999999999999 [0x1.fffffffffffffp+0]
{1.9999999999999998})
Expect -2: Got -2 = lround(-1.9999999999999999 [-0x1.fffffffffffffp+0]
{-1.9999999999999998})
Expect  4: Got  4 = lround(3.999999999999999 [0x1.fffffffffffffp+1]
{3.9999999999999996})
Expect -4: Got -4 = lround(-3.999999999999999 [-0x1.fffffffffffffp+1]
{-3.9999999999999996})
Expect  8: Got  8 = lround(7.999999999999999 [0x1.fffffffffffffp+2]
{7.9999999999999991})
Expect -8: Got -8 = lround(-7.999999999999999 [-0x1.fffffffffffffp+2]
{-7.9999999999999991})


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=4997

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.