[Bug libgcj/16122] gij - Incorrect result due to computations in extended precision on x86

["aph at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org>]

------- Comment #4 from aph at gcc dot gnu dot org  2006-02-14 15:45 -------
A bit more explanation.

The problem is caused by the fact that 9007199254740994.0 + 0.9999847412109375
is carried out in extended precision, and the result is rounded to
9007199254740995.  In double precision, the result of this calculation is
rounded to 9007199254740994.

When the extended-precision value is rounded to double for storing, it is
rounded (again) to 9007199254740996.

So, double rounding leads gij to return a value of d that is 2.0, whereas it
should be 0.0.

Note however, that the true accurate value for d, calculated at infinite
precision, is 1-(2^-16).  So, the absolute error for gcj is 1+(2^-16) and the
absolute error with correct rounding is 1-(2^-16).  (I'm not surprised this
hasn't been reported as a problem with any real applications!)

It might be worth setting the floating-point precision of gcj to double, but
that would only fix the double-precision case, and I presume we'd still have
the same double rounding problem for floats.  

And in any case, I do not know if libcalls would be affected by being entered
with the FPU in round-to-double mode.  We might end up breaking things.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16122