Re: FWD: FLOATING-POINT CONSISTENCY, -FFLOAT-STORE, AND X86

[hjstein@bfr.co.il (Harvey J. Stein)]

Stephen L Moshier <moshier@mediaone.net> writes:

 > The extra-precise registers are supposed to be a feature, not a bug.
 > Neither the computer language nor the compiler has a way to say
 > "this is an extra-precise register" so there is some inconvenience
 > using the feature.  It can't be made consistent.  The harder you look,
 > the more contradictions you find.
 > 
 > If you don't believe that, the alternative that makes sense is to
 > ask for straight IEEE behavior.  You can't get IEEE behavior without
 > setting the coprocessor rounding precision.  After you set the
 > rounding precision, all the other bugs disappear except for a rare
 > hardware bug or two.  The hardware bugs are dealt with by a trap
 > handler in the operating system, in the time honored fashion
 > of Intel, Borland, or Microsoft.
 > 
 > So there could be a straightforward plan to make x86 obey IEEE.
 > It's doubtful there could be a workable plan to fix the extra-precise
 > registers; anyway, they are a feature, no fix is needed!

I agree wholeheartedly.

Reasonable floating point code should expect that reordering
operations will produce slightly different results due to round off
error, and should be tolerant of the optimizer doing such.  Especially
given how little control the programmer has over exactly how
computations are ordered.

What floating point code *doesn't* expect is that there are multiple
underlying representations with different precisions, and that
identical computations might yield different results depending on
which representation happens to get used.  For example, most code will
expect that after:

   x = 1.0;
   y = 1.0;

that f(x) and f(y) will always be equal (for an f with no side
effects).

It will also expect that after

   x = 1.0/3.0;
   y = 1.0/3.0;

that x and y will always be equal.

Both of these assumptions break on ix86 because of the extended
precision FPU registers.

These are very reasonable things to expect, and much code breaks when
they're not satisfied.  One would think that this can be fixed by the
programmer with careful use of tolerances in comparisons, but this is
not the case.  Even if *very* carefully (i.e. - with knowledge of the
precision of the underlying representations), it will often just cause
discontinuities which create worse problems elsewhere.  But, it
*can't* be done carefully, because values in registers and values in
memory can accumulate differently, thus affecting more than just the
excess precision bits.

C and Fortran give little control over order of operations and
virtually no control over where such variances in precision might crop
up.  The compiler is free to use registers as it sees fit, and the
programmer has no control over it aside from recourse to assembler
programming.

So, well behaved, well written floating point code might behave well
on motorola, sparc & alpha CPUs but fail miserably on ix86 CPUs.  In
particular, the problems that have been discussed don't have to do
with compiler reordering & optimization.  They only have to do with
the fact that registers can contain excess precision, which AFAIK,
only happens on the ix86 CPUs.

As was pointed out, -ffloat-store only goes part way to fixing the
problem, because it doesn't affect compiler generated temporaries.  It
also makes the code slower.

I think the option of setting the fpu precision to 53 bits is a great
solution.  Even the Intel assember manual says:

   The double precision and single precision settings reduce the size
   of the significand to 53 and 24 bits, respectively.  These settings
   are providied to support the IEEE standard and to allow exact
   replication of calculations which were done using the lower
   precision data types.

I think Intel's comments about how great it is to use the extra
precision is just propaganda.  I think that it maybe helps *slightly*
with badly written code.  But it can make it impossible to write good
code.  Maybe it was just a poorly thought out clever idea for fixing
bad code.  Maybe it's to avoid having to have single precision
and mixed single/double register instructions, coupled with lots of
marketing propaganda to prevent seeing this shortcomming.

I'd think the only drawback to using the FPU in double precision mode
instead of extended precision mode would be for carefully hand
optimized numerical routines in assembler which can possibly do things
quicker by utilizing the 64 bit extended precision mode.  For such
code the programmer could explicitly muck with the FPU control word -
save its value, set it to extended precision, do the computations &
restore the value.  The OS will have to save and restore the FPU
control word when context switching, but it has to do this now anyway
& the ix86 FPU state save & restore instructions include the FPU
control word.

Of course, it's also *not* going to solve problems with single
precision unless the compiler is careful to not mix single precision &
double precision operations on the FPU stack & to always keep the FPU
control word set to the appropriate value.  It'd seem, actually, that
this would be the only way to get IEEE conformance with code that uses
both single and double precision values.  Fortunately, it seems that
people are tending to just use double precision, so maybe this can be
put off for later.

-- 
Harvey J. Stein
BFM Financial Research
hjstein@bfr.co.il