Re: Bug with g77 and -mieee on Alpha Linux

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 3:56:36 PM PST, toon@moene.indiv.nluug.nl writes:

> to the IEEE fans cost (as expressed in slowness of
>  computation) is irrelevant.  
I've been reading all these exchanges and trying not to make too much noise.  
To which costs do you refer?  The cost of trapped gradual underflows, which 
ought to be eliminated by rescaling, not by departing from IEEE?   Probably 
not the cost of extended precision divide and sqrt() on chips which support 
them, since you said you didn't care about those.  The cost of not 
indiscriminately replacing division by inversion and multiplication?  I'm 
happy with writing that specifically into the code where I want it.  It looks 
like Intel plans to make it a bigger issue than it is now.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

N8TM@aol.com wrote:
> 
> In a message dated 7/12/99 3:56:36 PM PST, toon@moene.indiv.nluug.nl writes:
> 
> > to the IEEE fans cost (as expressed in slowness of
> >  computation) is irrelevant.

> I've been reading all these exchanges and trying not to make too much noise.
> To which costs do you refer? 

Well, I was replying to something Jeff wrote:

> Some targets already do this (sh for example).  Is it painful?  Yes.  But
> is the cost coming down?  Yes.

where `this' is the setting of precision mode (and not rounding, as I
incorrectly stated -  thanks Richard) before every change in
computational mode (note - we're talking the ia32 architecture here, not
the Alpha; we've wandered from the subject).

If we take our support for IEEE 754 seriously, we should bear that cost,
because those who want this feature are willing to pay that cost.

>  Probably not the cost of extended precision divide and sqrt() on 
>  chips which support them, since you said you didn't care about those.

Note that we are talking about correctly implementing IEEE conformance. 
The fact that I personally do not need it is orthogonal to that issue.
(but I'm fully to blame for unduly pushing my own needs, for which I
apologize).

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

N8TM@aol.com wrote:
> 
> In a message dated 7/12/99 3:56:36 PM PST, toon@moene.indiv.nluug.nl writes:
> 
> > to the IEEE fans cost (as expressed in slowness of
> >  computation) is irrelevant.

> I've been reading all these exchanges and trying not to make too much noise.
> To which costs do you refer? 

Well, I was replying to something Jeff wrote:

> Some targets already do this (sh for example).  Is it painful?  Yes.  But
> is the cost coming down?  Yes.

where `this' is the setting of precision mode (and not rounding, as I
incorrectly stated -  thanks Richard) before every change in
computational mode (note - we're talking the ia32 architecture here, not
the Alpha; we've wandered from the subject).

If we take our support for IEEE 754 seriously, we should bear that cost,
because those who want this feature are willing to pay that cost.

>  Probably not the cost of extended precision divide and sqrt() on 
>  chips which support them, since you said you didn't care about those.

Note that we are talking about correctly implementing IEEE conformance. 
The fact that I personally do not need it is orthogonal to that issue.
(but I'm fully to blame for unduly pushing my own needs, for which I
apologize).

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 3:56:36 PM PST, toon@moene.indiv.nluug.nl writes:

> to the IEEE fans cost (as expressed in slowness of
>  computation) is irrelevant.  
I've been reading all these exchanges and trying not to make too much noise.  
To which costs do you refer?  The cost of trapped gradual underflows, which 
ought to be eliminated by rescaling, not by departing from IEEE?   Probably 
not the cost of extended precision divide and sqrt() on chips which support 
them, since you said you didn't care about those.  The cost of not 
indiscriminately replacing division by inversion and multiplication?  I'm 
happy with writing that specifically into the code where I want it.  It looks 
like Intel plans to make it a bigger issue than it is now.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>The point I really wanted to make is that in current, Standard
>Fortran, there is no way to deal with the fact that on some hardware,
>floating point numbers in registers have different precision (and range)
>then when stored in memory.  Therefore, IMHO, the "most" correct way of
>dealing with, say, DOUBLE PRECISION calculations is to store *all*
>intermediate values of computations.  Obviously, this would be very
>costly - so we settle for the less "correct" solution of ignoring all
>issues with intermediate extra precision.  Not having 80-bit spills is
>just one of the ways in which we are ignoring these issues.  We could
>spill floating point registers in 80 (or rather 128) bit chunks, but
>that would still not solve this problem completely.

Yes, well, standard Fortran doesn't imply much of anything useful about
precision and range.

So it is indeed permissible to use 80-bit precision for one computation,
64-bit for another, 32-bit for another, and 3-bit for yet another,
even if all computations are DOUBLE PRECISION in the code!

However, having *computed* an approximation of something (like the
result of evaluating an expression), that *one* approximation is the
only one available from that computation and/or the variable in which
it is stored, by my reading of the standard.

So while not forcing all DOUBLE PRECISION computations to 64 instead of
80 bits, we violate a principle that users accustomed to strict IEEE 754
(or even classic strict FP) machines obeyed, which is "never compute
using more precision that declared".

That's (considered) okay, because the result is much faster performance
on some hardware, and, besides, the Fortran standard does not disallow it.
(Further, it's fairly easy to code around, assuming the *next* principle
is obeyed.)

However, having computed *an* approximation to 80 bits, exposed it in
one code-visible way, then spilled it to 64 bits and afterwards used that
*distinct*, separate approximation in *another* code-visible way, we
violate the vastly more encompassing principle that a given computation
produces exactly *one* (code-visible) result.  (The result may indeed
represent an approximation of a value within some interval -- but, like
quantum mechanics, once *observed*, in the sense of being visible to the
code in any way, that observation is *fixed*, especially by assigning
it to a variable.  That variable cannot, by this principle, subsequently
appear to have *multiple* values.)

Now, this principle I *believe* to be mandated by the Fortran standard.

But, even if it isn't, it is a principle a vastly larger audience accepts
as a "given" than the one that accepts the former principle concerning
never using excess precision at all.

Further, the performance penalty of holding to the latter principle is,
AFAIK, never higher, and sometimes vastly *lower*, on all machines than
the penalty of holding to the former principle.

Now, this is all "water under the bridge", of course.

But it illustrates an aspect of what I have been talking about all along.

The attitudes, among so many who make design decisions affecting the
rest of us, that "nobody should care about low-level precision issues",
"speed is always more important than numeric stability", and so on,
make even mere *verification* that a product is doing the right things
vastly more difficult.  (Further, and this is my greater concern about
the 80-bit-spill issue, these attitudes do *not* defend against what
I believe *will* be future desired optimizations that will make code
*supposedly* resistant to truncated spills start failing.  At which point
those who *like* truncating spills will split into camps -- those who stick
to the "party line" that the *code* must be wrong, and various others who
claim that, to various degrees, the *compiler* is wrong to have made *that*
"extreme" a transformation to the code.)

In particular, those attitudes say, to *both* principles (of expectations
users have regarding FP behavior), "we don't care one whit about you,
we'll break you anytime if it gets us a few more percentage points of
performance on the codes *we* care about".

And now we're paying a very significant penalty for those attitudes.

Which cannot, and *will* (likely) not, be changed merely by changing
any of the particular short-sighted decisions under discussion (not
doing 80-bit spills, not making -mieee the default, to *summarize* --
see my earlier emails for more explanation).

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/9/99 8:12:31 PM PST, craig@jcb-sc.com writes:

> >I tried to express that to write useful numerical code one has to have
>  >knowledge of numerical analysis.

 One of the reasons we continue to use Fortran is that it's easier to fix 
code written without attention to such details in Fortran than in other 
languages.  To put it another way, Fortran is the only language where people 
who barely get by with their numerical skills can "write useful numerical 
code."  Many times those people are the only experts in their specialty 
available to put a needed procedure into practice.

>>spilling 80-bit registers assigned to variables to 64-bit
or 32-bit temporaries *does* -- returns an 80-bit result and a 32-bit
result (say), allowing the processor to randomly choose which to use at
any given time

If a code is written portably it shouldn't fail due to numerical differences 
outside the precision of the declarations.  g77 is remarkably reliable as it 
is, but it hasn't solved the problem of alignments on several important 
targets.  Acknowledged experts on egcs have advised against -malign-double, 
although I would consider it a bug if that option caused g77 to fail.  At the 
risk of being overly repetitive, I would be happy with 64-bit spills of 
single precision and 80-bit spills of double if the alignment performance 
problem could be solved.  I've noted also the correctness of Moshier's 
suggestion that many spills are of loop-invariant quantities which are read 
from static storage and thus a wider spill cannot contribute to accuracy.  
It's a shame to have these spills hurt performance, and probably impractical 
to distinguish them from those where there really are some guard bits to save.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

[ Oh, I see I forgot one important issue - Emil, I hope you don't mind
  if I discuss this private e-mail on the list: ]

Emil Hallin wrote:
> 
> Toon Moene wrote:
> 
> > Of course one could always set up a scenario where, using the fact that
> > you cannot know whether a floating point value is 80-, 64- or 32-bits
> > wide, you can generate small differences (which should have been zero)
> > that, suitably multiplied which each other, will generate a denormal
> > (especially if you start close to FLT_MIN).  My point is that you have
> > to cleverly set this up, because it will not show up in normal practice.
> 
> I still think the egcs compiler should do the right thing and spill 80 bit fp
> values when it has to. (IBM thought noone would ever use more than 640K "in
> normal practice" either and look what that did to us!).
> 
> I am a  physicist so I appreciate your arguments; my programs are not likely to
> break on this issue, but I can't *guarantee* that they won't break, and I
> dislike having to rely on results that might be different depending on whether
> or not I clicked on the mouse at the wrong time, forcing a context switch and
> register spills at an inopportune moment.

Ah, ho, keep it right there !

The saving of floating point registers on interrupt is an OS-issue.  The
compiler cannot do anything about that.

[I hope and actually am quite confident that Linux actually saves the
 complete 8x80 bit contents of the ia32 floating point stack]

This, indeed, is not the sort of impredictability one would like to have
(even if your algorithms are robust against small differences in
floating point arithmetic, having two subsequent runs with the same
input differ would be rather scary).

The discussion is whether spills to the stack of floating point
registers *by the compiler* should be done in full 80-bit accuracy, even
if the relevant variable was only declared as REAL (32 bits) or DOUBLE
PRECISION (64 bits).  This is not done now, and that might mean that two
runs with the same code and same input, but a different version of the
compiler, could give you different results.

OTOH, a new version of the compiler could also decide to evaluate your
expressions differently (within the limits required by the Standard, of
course) and causing output to differ that way.

Hope this helps,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:

> On Mon, Jul 12, 1999 at 06:04:04PM -0600, Donn Terry wrote:

> > Rounding mode only affects STORE operations.

> Toon was imprecise.  We were not speaking of rounding
> (bits 10:11 of fpcr) but rather precision (bits 8:9 of fpcr).

Not surprising - I'm operating without the document defining ia32
floating point arithmetic.  Richard, do you have an URL to it ?  I
couldn't find it on Intel's Web site.  Oops, I just realized - it's
probably in our Readings section ...

PS: Rounding mode is something different in IEEE-754 then what I was
referring to.  It doesn't tell at what bit to round, but which method of
rounding you want:  round to nearest, round to zero, round up or round
down.

> And in any case, rounding _does_ affect every operation.
> Just that in 64-bit precision mode, the rounding happens
> for the XFmode value of the register, rather than for the
> SFmode value of a variable.

As long as it's the same as that what happens when storing the SFmode
value in memory, we're safe (well, OK, modulo the range question).

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Lieven Marchand]

craig@jcb-sc.com writes:
 > So it's not that I don't respect that approach -- I do.  I just happen
 > to think that, in practice, the *audience* for that approach -- defined
 > as the audience that *knows what they're doing* -- is vanishingly small
 > compared to the audience for "pedestrian", straightforward, robust
 > IEEE 754 support.

Even the audience that *knows what they're doing* needs a robust
implementation first and foremost. I'm no longer really part of that
audience but I have a degree in Numeral Mathematics and before you put 
a program in production you want to have an implementation that
catches NaNs, deNorms, out of bound references etc to write and debug
your program. Tuning a production version with faster versions of
functions, inlining etc. is only feasible with a large regression set
of known correct results to catch errors.

 > After all, Toon represents the sort of audience I claim does not *need*
 > g77 to default to "gofast" mode -- I mean, they've already rewritten
 > their code to inline transcendentals, to avoid INTEGER*8 (so it could
 > run on $$$ Fujitsu iron), and so on.  To me, it's hard to understand
 > why it's so painful for that audience to specify `-mno-ieee' or some
 > such thing, but apparently it is.
 > 

Ditto.

 > Also, I keep seeing the claim that, effectively, nobody should be using
 > floating-point (at least in Fortran) unless they're solving one of a
 > particular set of problems that have been Declared Properly Solved Using
 > Limited Range (DPSULR) by people like Toon.

Some elder statesmen of Numerical Analysis really suffer from "All the 
world is a Cray"-syndrome and never really accepted IEEE semantics.

-- 
Lieven Marchand <mal@bewoner.dma.be>
If there are aliens, they play Go. -- Lasker

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 09:17:04PM +0100, Joern Rennecke wrote:
> Make keeping 128-bit-alignment the default, and hope that soon crt0.o
> and libraries will provide / keep this alignment too on platforms where
> we care about performance?

Yes.

> > The spill widening can actually be done in such a way that it helps
> > alpha ev4 wrt spilling QImode.  So I want that capability for other
> > reasons.
> 
> Will this design also allow to use the alignment bytes of the 10-byte-values
> to put smaller spills there?

I think that's rather orthogonal.  The biggest part of the reload work
for the target to be able to request that QImode pseudos be spilled in
SImode.  Or SFmode pseudos be spilled in XFmode.

The fact that XFmode slots leave 6 bytes of extra alignment storage to
be use for other spills is simply another problem.  Not that it wouldn't
be good to do that too.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Dave Love]

>>>>> "Tim" == N8TM  <N8TM@aol.com> writes:

 Tim> I'd be happy to resurrect paranoia or some similar code if you
 Tim> can specify it, in whichever version you like, and use that as
 Tim> the standard of whether g77 is dealing adequately with these
 Tim> issues.  It's been quite a few versions since I've run it, 

I've run it a bit recently.  As reported on egcs-bugs, it's fine with
-mieee on current OSF1.  In default mode, there's one flaw, as with
the DEC compiler.

[While I'm on, it looks as though I don't actually agree with what
Toon says, contrary to what I said on egcs-bugs, but I haven't time to
plough through all this stuff right now and I'm not really sure what
his position is.  There are clearly uses for subnormals
(e.g. apparently in complex division) and I have done barrier
penetration calculations involving arbitrarily small dimensionless
numbers, not that that's a big deal or really the same issue as
numerical analysis robustness.  I did see things fall over in minor
ways when IEEE machines appeared.]

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Edward Jason Riedy wrote:
> 
> And Toon Moene writes:

>  - Sure, when run on different architectures, they will give different
>  - results.  The crucial question, however, is:  Are these differences
>  - interesting ?
> 
> Yup.  When you're looking for the smallest eigenvalues, and they're
> clustered together, the differences are _very_ interesting.  This
> is linked to the waterpipe remark: earthquake simulations.  The
> smallest eigenvalues are the ones of interest (not recalling why
> at the moment, too tired).  While _your_ code may not care about
> the nth place, these codes do.  Trying to rely on data that's
> only as good as the input data deserves (which is a favorite
> phrase of Dr. Kahan's) can be very difficult when the best bounds
> on what the input deserves are still dodgy.

Well, if you have an error analysis that supports that (and I am sure
you have) then I am all with you.

>  - Of course there are differences - but they're all meteorologically
>  - irrelevant.
> 
> That's nice.  You're right, you shouldn't be relying on your own
> problem domain.  There are others where it very much matters.

Touche'.

>  - Heck - before the 60's, [...]
> 
> Before the 60's, the idea of solving systems of equations with more
> than 6 unknowns was silly.  The fact that von Neumann's error
> analysis was wrong was not widely known, according to Dr. Parlett.
> And the SVD didn't even exist (iirc).

Yep, after I wrote that (and reread it) it dawned on me that you can
reverse the remark I made:  Getting better defined floating point
arithmetic permits different algorithms to be used.

> [on using extended precision]
>  - But how do you reliably code that ?  Going to assembler ?  There is no
>  - way to Standardly specify IEEE-754 extended precision arithmetic, yet.
> 
> Who said anything about using IEEE-754 extended precision in hardware?
> The reference implementation of the XBLAS (not my project, but one
> of my advisor's) is using a software implementation.  That system
> uses IEEE double (and won't work on x86s without forcing the CPU to
> round to double).  It's also in C, for various reasons.

Sorry, I was jumping to conclusions, here - and down here:

>  - > Yes, it could conceivably be possible to code around
>  - > a lack of IEEE-754.  The algorithms involved are complicated enough,
>  - > thank you.
>  -
>  - Well, OK; that's a choice you want to make.  I think it's rather obvious
>  - that I don't like to live that dangerously.
> 
> Pick your choices:
> 
>         * Live dangerously by relying on a standard that allows you
>           to use, test, and debug a single version of your code, or
> 
>         * Live dangerously by littering your code with a thousand
>           tests for various platforms, many of which are inaccessible
>           to you, and try to test any problems by waving a dead
>           chicken over the code and chanting ``please don't be too
>           wrong'' a thousand times.
> 
> These algorithms are _not_ simple.  Go read Inderjit Dhillon's
> thesis on the new eigen routines in LAPACK.  Then tell me if you'd
> even consider trusting an implementation that cannot be tested or
> verified with any sense of completeness.  Good thing he didn't
> try to use g77 on an Alpha, or he'd still be working on it.

Well, as long as it is expressible in Fortran, we have no choice *but*
to support it.  That means that if it doesn't work with the current
compiler, we should regard that as a bug.

>  - It's a pity, but I agree that one should use a tool that does the job at
>  - hand.  If g77 doesn't cut it, that's fine with me.
> 
> *boggle*
> 
> I thought the idea was to improve it.

Hmmm, yes, that's what I mean - I didn't express myself very clearly: 
If g77 doesn't yet do what you need, you should use another compiler for
the time being (and that's fine with me).

> Craig Burley is very correct.  The relevant issue at hand should not be
> whether or not j. random code to be compiled is numerically correct.
> It should be whether or not the compiler is even known to produce
> correct code on all platforms where it is `supported.'  Since no one
> even knows the definition of correct for g77...

Granted, but we often get bug reports where the code itself makes
assumptions that are simply not valid - e.g. the infamous "root-finder"
that compares the final value and its latest approximation for equality.

But you are right:  If there is code for which there is a single correct
answer (or interval in which the answer should lie) then g77-compiled
code should provide that answer (an answer within that interval).

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 12:20:06 PM PST, amylaar@cygnus.co.uk writes:

> What are we going to do about alignment?
>  Make keeping 128-bit-alignment the default, and hope that soon crt0.o
>  and libraries will provide / keep this alignment too on platforms where
>  we care about performance?
>  
>  
>  Will this design also allow to use the alignment bytes of the 
10-byte-values
>  to put smaller spills there?


I'm still seeing occasional 50% performance loss due to mis-aligned 64-bit 
spills (on targets where the stack is only 32-bit aligned).  So I would 
request again that this project not go ahead without solving this alignment 
problem, and taking Craig up on the suggestion that it may be possible for 
the compiler to avoid widening the spills where the data have already been 
rounded to 32 or 64 bits.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Andy Vaught]

Someone else had a pointer to Prof. Kahan's home page (he is one of the
designers of IEEE-754 arithmetic).  There is an another article there,
"How Java's Floating-Point Hurts Everyone Everywhere",
http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf , which talks much more
about the pitfalls of floating point than simply bashing Java.  Kahan and
his co-author were trying to convince Sun to abandon some bad decisions
they'd made, and ended up writing one of the best papers on numerical
analysis that I've ever seen.

People writing numerical programs (or fortran compilers) should read this
paper. 

DDJ also has a neat interview with Kahan at
http://www.ddj.com/articles/1997/9711/9711a/9711a.htm

        Andy

-----------------                        XOLD(K,IC,I)=
Andy Vaught               ....        DO ITERS=1, 10  XOLD(K,IC,I)
andy@maxwell.la.asu.edu   |  |   /CALLMSOLVE(A,B,X,I,ITERS,TOL)+(RANNYU(0)
Arizona State University  ======|WRITE(6,'(I5,2X,F12.6)')ITERS,TOL -HALF)
Tempe, Arizona USA        OOOOOO \ENDDORETURN PARAMETER(ZERO=1.D0)*TENTH*DELTA

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < b9c6a72c.24bbede7@aol.com >you write:
  > I'm still seeing occasional 50% performance loss due to mis-aligned 64-bit 
  > spills (on targets where the stack is only 32-bit aligned).  So I would 
  > request again that this project not go ahead without solving this alignment
We'd be interested in a testcase for this.  To the best of my knowledge we
should be aligning spill slots relative to the stack frame.

jeff

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>If a code is written portably it shouldn't fail due to numerical differences 
>outside the precision of the declarations.

So you really think it's okay for any arbitrary reference to a variable
X to evaluate, at any given time, to X +/- a sufficiently small random
number?

That's what you're essentially saying -- that even IF (X .LT. X) should
potentially evaluate to .TRUE. on a random basis, which means even that
statement is not "written portably".  Further, you're saying that two
IF (X .LT. 0.) in a row should evaluate differently, if the difference
is due to bits 64-79 (so to speak) or, for single precision, 32-79, so
*that* statement is not "written portably".  For programs that *contain*
those statements to be "written portably", they must somehow cope with
the fact that the statements themselves may behave randomly.

Yet that contradicts any reasonable reading of ANSI FORTRAN 77, as far
as I can tell, which mandates that a *single* approximation is assigned to
a variable.  (It uses the singular, not plural, form of the pertinent
nouns.)

>I've noted also the correctness of Moshier's 
>suggestion that many spills are of loop-invariant quantities which are read 
>from static storage and thus a wider spill cannot contribute to accuracy.  
>It's a shame to have these spills hurt performance, and probably impractical 
>to distinguish them from those where there really are some guard bits to save.

Not impractical at all, though perhaps non-trivial for gcc as current
designed.  A compiler *necessarily* knows from where a value originally
came (static storage, or computed on-the-fly, e.g. in a register).
Whether it tracks that, I don't know.  One thing for sure: if the static
variable is *defined* with a new value initially computed as an 80-bit
result, then if that 80-bit result gets *any* visibility (i.e. if it is
not immediately truncated to the declared size of the static variable),
then, yes, any spill of that 80-bit result, *including committing the
value back to static storage for use by subsequent procedures*, must
preserve the entire 80 bits, to conform to ANSI FORTRAN 77, by my
(current) reading.

But, I didn't intend to re-propose 80-bit spills -- I withdrew my proposal
concerning that last December, and I'm sticking with my decision.  AFAIC,
"we" decided (overriding my concerns, despite the lack of any real
performance analysis showing that my proposal would be as bad as claimed if
implemented) to ignore FORTRAN standards compatibility in favor of the
ad-hoc consensus (which, *months* later, has yet to be clearly documented)
regarding how to write floating-point code for g77.

And defaulting to -mno-ieee, I've *also* already "swallowed" in terms
of what that does to the quality-of-implementation issue at a *design*
level.

What we're now beginning to discover is just how much that decision
adversely affected the quality-of-implementation at the *implementation*
and *packaging* levels -- to wit, random crashes depending on what
narrow range of numerics is involved, and confused users who don't
understand that they shouldn't go anywhere near a Fortran compiler
(much less write Fortran code).

And there's basically *nothing* I can do about this, even as g77 author
and de facto "project leader", since to effectively mandate what I believe
is the robust, properly-engineered design (which would include the
concept of *tolerances*, i.e. have the compiler support IEEE 754 in as
clean a way as possible so not only does Toon-style code work, but so
does independent spec-based testing, etc.), I'd have to either convince
the GCC team to switch everything over (which I will no longer attempt to
do) or re-do all their work in the form of a fork or new compiler (which
probably isn't worth attempting).

The issue now is, am I going to continue working on a compiler that an
extremely small percentage of the *Fortran programmer* population can
use effectively, given the decisions *already made*, and which I concede
will *not* be changed, concerning the use of truncating spills, defaulting
to -mno-ieee (without bothering to first architect, design, implement, and
document the underlying types throughout the infrastructure), and other
decisions that have been, and surely will be, made that are as wrongheaded
and short-sighted (as well as contradictory with each other) as I consider
these to be?

I expect I'll have made a decision by the end of this week, if not sooner.

After all, I've been *seriously* thinking about this issue since this
past December, and now regret that I didn't insist that we make robustness
of code generation and predictability of numerical results the #1 topic
of the Fortran BOF at LinuxExpo, since, in my view, that's the most
important decision vis-a-vis g77's future than anything else we discussed.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>DDJ also has a neat interview with Kahan at
> http://www.ddj.com/articles/1997/9711/9711a/9711a.htm

This, on the other hand, already has a priceless quote from WK that
pertains to the discussion at hand:

   Let's calculate accuracy well out of proportion with what anybody
   actually needs. They don't need it, but it doesn't matter. We'll do it
   that way and then there won't be any argument.

Imagine that.  If g77/gcc delivered to -mieee and 80-bit spills from
the outset, we'd have avoided all *sorts* of arguments, other than
discussions as to how to improve performance (and offer options to
users who insist they don't need certain assurances, like we already
do via -ffast-math).

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Edward Jason Riedy]

And Toon Moene writes:
 - 
 - My point is that every physicist engaging in numerical analysis will
 - choose his/her units to get this effect.  This is not surprising, as
 - people like to talk about the major phenomena they study using small
 - numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
 - measure distance.

Please go to http://www.cise.ufl.edu/~davis/sparse/ and support 
the claim that all of these matrices (most from real problems, 
many from industry) never run into floating-point problems and 
always live happily in your world of `just-right' numbers when 
being factored or when used in an iterative solver.

You can't, because they don't.  Now, they can give useful solutions
using less precision in most places and more in a few, key places 
(see papers by Demmel and Li on parallelizing SuperLU), but that's a 
different story.  

Likewise, from a new eigenvalue routine in LAPACK, a large algorithmic 
improvement:
*  Note 3 : ZSTEGR works only on machines which follow ieee-754
*  floating-point standard in their handling of infinities and NaNs.
*  Normal execution of ZSTEGR may create NaNs and infinities and hence
*  may abort due to a floating point exception in environments which
*  do not conform to the ieee standard.

[No, this doesn't directly apply to denorms, but it does apply to
numbers not in your nice range, even when they were before calling
this routine.]

Mr. Burley's defaults (-mieee and appropriate-width spills) are the
only sane ones possible for a general-purpose compiler to be used by
anyone.  People shouldn't have to read every LAWN and ask every FP
expert to determine what their system's particular quirks are.

BTW, the Alpha-denorm non-implementation caused major problems 
with testing and using ScaLAPACK on a multi-architecture PVM 
machine, anecdotally from the authors.  The old argument about 
denorms is covered in 
http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.html .
Interesting reading.  And IA-64 seems to get things right in spec, 
although generating the `best' division code sequence is going to 
be fun.  Intel's compiler currently only uses one sequence, according 
to a recent presentation.  The support for funky loop unrolling has 
some applications in getting boundary FP exceptions and tests right 
without acrobatics, too.

Jason

Re: Bug with g77 and -mieee on Alpha Linux

[Sylvain Pion]

On Tue, Jul 13, 1999 at 08:54:46AM -0400, Stephen L Moshier wrote:
> But that is what you should do!  If there were an important need for
> "correctly implementing IEEE conformance" then someone would have put
> it in long ago.  We implement the features that we need, and we ignore
> the ones we don't.

Then just stop developping egcs, because all the features we need are
already implemented...

I personnaly would like to have better (correct and fast) IEEE support from
GCC.  I have even started to code some small optimizations specifically for
my IEEE dependent code, but I'm not a compiler guru, so...

The related features I miss are:
- ability to stop constant propagation when you need the evaluation to be done
  at runtime because of different rounding/precision.
- an equivalent to -fforce-mem for intermediate values, where needed (after
  arithmetic operations on ia32).

Of course I can put the workarounds (volatile) in my own code, but the
compiler has more informations, and could perhaps optimize better if it
was aware of the real problem.

-- 
Sylvain

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Thu, Jul 08, 1999 at 04:08:07PM -0400, Richard Hadsell wrote:
> The problem I ran into immediately (with C++) was a call to a math
> function (e.g., exp()) that returns a denormal.  libm must have been
> compiled with -mieee, or its equivalent.  When my code is compiled the
> default way, without -mieee, the denormal eventually causes a
> floating-point exception that crashes my program.

Yes, that is a known problem on Linux.  Digital's math libraries
don't do this.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> > And in any case, rounding _does_ affect every operation.
> > Just that in 64-bit precision mode, the rounding happens
> > for the XFmode value of the register, rather than for the
> > SFmode value of a variable.
> 
> As long as it's the same as that what happens when storing the SFmode
> value in memory, we're safe (well, OK, modulo the range question).

Not addressing the exponent range question means that the code still
remains incorrect, only that you see the failures for a smaller range
of values.  I.e. it remains buggy, but becomes harder to debug.

Re: Bug with g77 and -mieee on Alpha Linux

[Donn Terry]

> 2. [This one is from hearsay, on comp.arch]  One can only set the
>    rounding mode on the ix86, not the range.  That means that the
>    register values still will not have the same non-finite behaviour
>    as stored values.

Based on my imperfect understanding of what you're saying, and on
my potentially imperfect undrstanding of the FP unit, I may be saying
something you already know, but also I may be adding something,
so I'll take the chance...  (I also came into this late.)

Rounding mode only affects STORE operations.
Thus, setting the rounding mode to 53 bits has no effect on the content
of a register (even after a subsequent arithmetic operation).  The only
way I was able to find that caused a round down from 64-bit to 53 bit
mantissa was a store-load sequence.  (One would hope that there's
a way to fix this in the newer versions, but this was a PII architecture
book I was looking at.)

I spent a lot of time looking at the FP registers very carefully, and
couldn't make them round to the right precision any way *I* could
find that didn't involve memory.   Thus, I believe the problem is worse than
just spills; computations exist which, depending on how many registers the
compiler chooses
to use (and this can be expression order sensitive, of course), the
results could differ (potentially substantially) depending on optimization
level and exact code generation strategies.  (Similar differences would
not be visible on a pure 64bit IEEE machine such as most of the RISCs.)

I pushed on this hard enough that I hacked up gdb to print out
the x86 FP registers in both hex and float decimal, along with intepreted
values of the control registers and a TOS marker.  It helped a lot.
(This is for an older gdb, and I haven't converted it to a patch, but
I'll be happy to ship the file to someone to convert it to a newer
gdb; the changes should be easily portable.)

Donn

P.S.  Thinking about it, there may be a way to get that rounding by
extracting exponents and stuffing mantissas, but I don't really want to
think about that, and it's not clear that it would be any faster than a
store-load, allowing for cache.

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Mike Stump wrote:

> Guys, guys...  In the end, the gcc folks get to pick what front end
> they want, and they should be willing to respect the wishes of the
> frontend folks to a large degree...  The frontend folks need to agree
> among themselves what the `right' behavior is. \

Good point.

We're working on it.

:-)

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Stephen L Moshier]

> Note that we are talking about correctly implementing IEEE
> conformance.  The fact that I personally do not need it is orthogonal
> to that issue.  (but I'm fully to blame for unduly pushing my own
> needs, for which I apologize).

But that is what you should do!  If there were an important need for
"correctly implementing IEEE conformance" then someone would have put
it in long ago.  We implement the features that we need, and we ignore
the ones we don't.

Re: Bug with g77 and -mieee on Alpha Linux

[Stephen L Moshier]

On Tue, 13 Jul 1999, Sylvain Pion wrote:
> Then just stop developping egcs, because all the features we need are
> already implemented...

Just so.  GCC is not a new program.  It is an old program.  Consider
that IEEE has been in effect for some 14 years.  During all those
years there have been any number of hot-shots, gurus, and hangers-on
who could have implemented the features.  They have not done so.
People have their priorities.


> The related features I miss are:
> - ability to stop constant propagation when you need the evaluation to be done
>   at runtime because of different rounding/precision.

  C9X treats this question, so you will probably get your wish.


> - an equivalent to -fforce-mem for intermediate values, where needed (after
>   arithmetic operations on ia32).

  If you are talking about the fix for extra-precise registers, I think it
may have been dropped from C9X.  It has already been done, however,
at Next.  The Next work would provide at least a focus of discussion,
for those wanting to work on the problem.

  -fforce-mem, at double precision, would tend to violate IEEE by creating
double-rounding errors.

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> This is a good point.  The reason I keep harping on the range question
> is that I *hope* that documenting it makes it possible for people to
> work around it.

It's pretty similar to trapping on denormalized values.  Except that
you can silently get incorrect results instead.

> If such a mode of working is not possible, we have no choice but to
> revert to storing *all* intermediate values.

No, we may compute intermediate values in a higher precision, as long as
we make sure that a value that is assigned to a variable remains consistently
the same value when it is used more than once.

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 11:57:55PM +0200, Toon Moene wrote:
> I think it's *either* -mieee *or* 80-bit spills.

Eh?  One is proposed for Alpha, the other for IA-32.

> This still has two limitations w.r.t. to true IEEE conformance:
> 
> 1. It only works correctly for doubles, because floats still get
>    extended to 64 bits and are therefore not equal to their stored
>    value.

No, you can set the rounding mode to 24, 53, or 64 bits.

> 2. [This one is from hearsay, on comp.arch]  One can only set the
>    rounding mode on the ix86, not the range.  That means that the
>    register values still will not have the same non-finite behaviour
>    as stored values.

True.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Joern Rennecke wrote:

> > > And in any case, rounding _does_ affect every operation.
> > > Just that in 64-bit precision mode, the rounding happens
> > > for the XFmode value of the register, rather than for the
> > > SFmode value of a variable.

> > As long as it's the same as that what happens when storing the SFmode
> > value in memory, we're safe (well, OK, modulo the range question).

> Not addressing the exponent range question means that the code still
> remains incorrect, only that you see the failures for a smaller range
> of values.  I.e. it remains buggy, but becomes harder to debug.

This is a good point.  The reason I keep harping on the range question
is that I *hope* that documenting it makes it possible for people to
work around it.

If such a mode of working is not possible, we have no choice but to
revert to storing *all* intermediate values.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Hadsell]

Richard Henderson wrote:
> 
> Underflow to zero always ocurrs when software completion (ie -mieee)
> is not requested.  This is still useful for when you need -mieee to
> handle Inf and NaN, but don't care about denormals.

The problem I ran into immediately (with C++) was a call to a math
function (e.g., exp()) that returns a denormal.  libm must have been
compiled with -mieee, or its equivalent.  When my code is compiled the
default way, without -mieee, the denormal eventually causes a
floating-point exception that crashes my program.  Yes, it forced me to
discover and start using -mieee, but the default compilation linked with
the standard libm does not execute gracefully.

Contributors to this list like to blame the programmer for incorrect
code.  I think someone should reconsider the interaction between the
compiled code and the libm functions.  If that means that functions like
exp() should return 0 instead of denormals, it's fine with me.  If
setting a hardware register to force denormals to 0 will have the same
effect, that's also fine.  

Just provide a reasonable way to use the compiler and the math functions
in a reliable way.  At the moment, that's not the default.

-- 
Dick Hadsell			914-381-8400 x5446 Fax: 914-381-9790
Reply-to:			hadsell@blueskystudios.com
Blue Sky Studios                http://www.blueskystudios.com
1 South Road, Harrison, NY 10528

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < 378A77AA.A234987F@moene.indiv.nluug.nl >you write:
  > OK, but the point is you have to set it to *one* value for the complete
  > compilation (unless you want to call this routine after every
  > calculation to set the default right for the next one).
Some targets already do this (sh for example).  Is it painful?  Yes.  But
is the cost coming down?  Yes.

Cygnus wrote a pass based on the lcm routines to find computationally 
optimal locations for FP mode switching on the sh.  That same code can be
used to do mode switching for any system which needed it, including stuff
like mmx/fp mode swtiching on ia32.


jeff

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Jeffrey A Law wrote:
> 
>   In message < 378A77AA.A234987F@moene.indiv.nluug.nl >you write:
>   > OK, but the point is you have to set it to *one* value for the complete
>   > compilation (unless you want to call this routine after every
>   > calculation to set the default right for the next one).

> Some targets already do this (sh for example).  Is it painful?  Yes.  But
> is the cost coming down?  Yes.

Note that to the IEEE fans cost (as expressed in slowness of
computation) is irrelevant.  We should play this by their rules.

> Cygnus wrote a pass based on the lcm routines to find computationally
> optimal locations for FP mode switching on the sh.  That same code can be
> used to do mode switching for any system which needed it, including stuff
> like mmx/fp mode swtiching on ia32.

Well, if this can be done reliably, I am convinced that support for IEEE
conformance (minus the range question, which should be documented quite
clearly) is possible.

Thanks for your input (Richard and Jeff),

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 07:01:10AM -0000, craig@jcb-sc.com wrote:
>    Let's calculate accuracy well out of proportion with what anybody
>    actually needs. They don't need it, but it doesn't matter. We'll do it
>    that way and then there won't be any argument.
> 
> Imagine that.  If g77/gcc delivered to -mieee and 80-bit spills from
> the outset, we'd have avoided all *sorts* of arguments...

I spent a good bit of the afternoon reading through several of Kahan's
papers.  Despite my earlier position, I'm mostly convinced we should do
both -mieee and 80-bit spills by default, despite what other compilers
for the systems in question do by default.

It's the sort of change that would require Large Announcements, so that
someone rebuilding their favourite mpeg widget doesn't scream.  Or at
least not as loudly.

The quality of code generated by -mieee definitely needs attention.
Even given the constraints it must follow, the generated code is abysmal.

The spill widening can actually be done in such a way that it helps
alpha ev4 wrt spilling QImode.  So I want that capability for other
reasons.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>In the end, it is about great software that users like and respect.

Great software doesn't crash and burn on input that is wrong in highly
predictable ways.  g77 *does*, in effect, do that.

>Personally I don't have much of an opinion if gofast or becorrect
>should be the default.

If only that were the choice.  In fact, the choice here is between bewrong
and beabitslowerthantheoreticallypossible.  After all, even Toon agrees
that too many people are writing Fortran code without knowing what they're
doing.  That number is, IMO, vastly higher if we choose Toon's model than
if we choose defaults consistent with what the industry, itself, has
proposed, debated, and standardized upon (and which at least *some*,
if not all, other Fortran compiler have agreed upon as well).

The wonderful thing about choosing Toon's approach is that, we can then
go rewrite all the intrinsics (such as the transcendentals) to use faster
methods that don't bother to compute correct results for inputs beyond
the "accepted" range.  After all, that's what the code Toon works on *does*
-- it doesn't bother calling SIN, COS, etc., it employs faster in-line
versions that don't worry about those bothersome corner cases.

If we decide to go whole-hog for Toon's approach, we can replace, wholesale,
our SIN/COS/TAN and other intrinsics with similar kinds of open-coded
replacements.

So it's not that I don't respect that approach -- I do.  I just happen
to think that, in practice, the *audience* for that approach -- defined
as the audience that *knows what they're doing* -- is vanishingly small
compared to the audience for "pedestrian", straightforward, robust
IEEE 754 support.

After all, Toon represents the sort of audience I claim does not *need*
g77 to default to "gofast" mode -- I mean, they've already rewritten
their code to inline transcendentals, to avoid INTEGER*8 (so it could
run on $$$ Fujitsu iron), and so on.  To me, it's hard to understand
why it's so painful for that audience to specify `-mno-ieee' or some
such thing, but apparently it is.

Also, I keep seeing the claim that, effectively, nobody should be using
floating-point (at least in Fortran) unless they're solving one of a
particular set of problems that have been Declared Properly Solved Using
Limited Range (DPSULR) by people like Toon.

What I *don't* know is whether that claim is true.  I certainly see plenty
of people who, like Toon, seem to know a lot more than I do about how
Fortran is actually used in the field, who disagree that DPSULR problems
are the only ones worth using floating-point for, and who claim the
*other* problems rely on the full range and support of IEEE 754 being
available.

It is also non-trivial for me to just say "well, I'll pick *my* defaults
for g77, let someone else pick whatever *they* want", because, without
the support of much of the GCC team (Toon being on the Steering Committee,
but not me!), it'll be difficult to get people to help support things
like 80-bit spills and -mieee working correctly.

Similarly, if we *don't* thoroughly discuss and document the issues,
and instead Toon and others simply capitulate to my feelings, the
result could be that g77 becomes useless to most of its audience because
it no longer defaults to what they see as "gofast" mode, or doesn't
properly implement it (as is apparently the case with -mno-ieee being
the default, vis-a-vis issues I raised earlier).

In other words, we barely have enough resources to support one incarnation
of g77.  It's not clear we'd have nearly enough to support two or three.

And, my having to support the conflicting requirements of multiple camps
of Fortran users has been painful enough to deal with over the years
(e.g. backslashes in character constants).  This is yet another very
difficult schism, amplified by the fact that the very users Toon handwaves
as unworthy of g77 (since think they need full IEEE 754 support) are
the ones *most* likely to submit bug reports before they do full
investigations of the problem.  So if we choose Toon's camp, we get
(as we've been getting, only worse as g77 improves and, hopefully, becomes
more popular) bazillions of not-quite-right bug reports; if we choose
overall robustness, we get hardly any by comparison, since g77 Just Works,
and the Toon-like experts already know how to use compiler options to get
speed *and* how to investigate code generation to find bugs, before
complaining about them.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> In a message dated 7/12/99 12:20:06 PM PST, amylaar@cygnus.co.uk writes:
> 
> > What are we going to do about alignment?
> >  Make keeping 128-bit-alignment the default, and hope that soon crt0.o
> >  and libraries will provide / keep this alignment too on platforms where
> >  we care about performance?
> >  
> >  
> >  Will this design also allow to use the alignment bytes of the 
> 10-byte-values
> >  to put smaller spills there?
> 
> 
> I'm still seeing occasional 50% performance loss due to mis-aligned 64-bit 
> spills (on targets where the stack is only 32-bit aligned).  So I would 
> request again that this project not go ahead without solving this alignment 
> problem, and taking Craig up on the suggestion that it may be possible for 
> the compiler to avoid widening the spills where the data have already been 
> rounded to 32 or 64 bits.

That does not work quite right, since the 16-bit exponent means that
values that ought to be represented as denormals or underflow to zero
are still represented with 53 significant bits.

Assuming you use gcc to compile your entire program, we can solve the
aligment problem by aligning the stack in main.

Re: Bug with g77 and -mieee on Alpha Linux

[Edward Jason Riedy]

And Toon Moene writes:
 - 
 - Yep.  Still a lot of useful work is done on computers predating the
 - IEEE-754 standard (IBM 370's, VAXen and Cray PVP's).

A shrinking amount of useful work.  Almost always with older code.
Or at least that's my UCB/LBL-influenced view.  It's _very_ easy
to get time on the vector machines up the hill.  Getting full use
of the T3E (with an IEEE-happy compiler, on the Alpha), on the 
other hand...

 - Sure, when run on different architectures, they will give different
 - results.  The crucial question, however, is:  Are these differences
 - interesting ?

Yup.  When you're looking for the smallest eigenvalues, and they're
clustered together, the differences are _very_ interesting.  This
is linked to the waterpipe remark: earthquake simulations.  The
smallest eigenvalues are the ones of interest (not recalling why
at the moment, too tired).  While _your_ code may not care about
the nth place, these codes do.  Trying to rely on data that's
only as good as the input data deserves (which is a favorite
phrase of Dr. Kahan's) can be very difficult when the best bounds
on what the input deserves are still dodgy.

 - Of course there are differences - but they're all meteorologically
 - irrelevant.

That's nice.  You're right, you shouldn't be relying on your own
problem domain.  There are others where it very much matters.

 - Heck - before the 60's, [...]

Before the 60's, the idea of solving systems of equations with more 
than 6 unknowns was silly.  The fact that von Neumann's error
analysis was wrong was not widely known, according to Dr. Parlett.
And the SVD didn't even exist (iirc).

 - Of course I rather wish we could act that fast with g77 too, but you
 - should realize that:

I only used NAG's compiler as an example because I could quickly
provide a URL with further information if anyone wanted it.
There's actually an interesting tidbit in the man page on the Alpha
dealing with IEEE support.

[on using extended precision]
 - But how do you reliably code that ?  Going to assembler ?  There is no
 - way to Standardly specify IEEE-754 extended precision arithmetic, yet.

Who said anything about using IEEE-754 extended precision in hardware?
The reference implementation of the XBLAS (not my project, but one
of my advisor's) is using a software implementation.  That system
uses IEEE double (and won't work on x86s without forcing the CPU to
round to double).  It's also in C, for various reasons.

 - > Yes, it could conceivably be possible to code around
 - > a lack of IEEE-754.  The algorithms involved are complicated enough,
 - > thank you.
 - 
 - Well, OK; that's a choice you want to make.  I think it's rather obvious
 - that I don't like to live that dangerously.

Pick your choices:

	* Live dangerously by relying on a standard that allows you
	  to use, test, and debug a single version of your code, or

	* Live dangerously by littering your code with a thousand
	  tests for various platforms, many of which are inaccessible
	  to you, and try to test any problems by waving a dead
	  chicken over the code and chanting ``please don't be too 
	  wrong'' a thousand times.

These algorithms are _not_ simple.  Go read Inderjit Dhillon's
thesis on the new eigen routines in LAPACK.  Then tell me if you'd
even consider trusting an implementation that cannot be tested or 
verified with any sense of completeness.  Good thing he didn't
try to use g77 on an Alpha, or he'd still be working on it.

 - It's a pity, but I agree that one should use a tool that does the job at
 - hand.  If g77 doesn't cut it, that's fine with me.

*boggle*

I thought the idea was to improve it.

Craig Burley is very correct.  The relevant issue at hand should not be
whether or not j. random code to be compiled is numerically correct.  
It should be whether or not the compiler is even known to produce
correct code on all platforms where it is `supported.'  Since no one
even knows the definition of correct for g77...

Jason

Re: Bug with g77 and -mieee on Alpha Linux

[Mike Stump]

Guys, guys...  In the end, the gcc folks get to pick what front end
they want, and they should be willing to respect the wishes of the
frontend folks to a large degree...  The frontend folks need to agree
among themselves what the `right' behavior is.  If they cannot agree,
usually there is a natural pecking order that can resolve the issue.
If people feel strongly, they can walk as a vote of no confidence.  I
would hope that even if they do this, that they can find a way to
offer up the deaults that they care about, try it out on the community
for a year or so with a release of their own package, see if users
flock to their compiler or the other compiler...

In the end, it is about great software that users like and respect.

Personally I don't have much of an opinion if gofast or becorrect
should be the default.

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 5:28:38 PM PST, amylaar@cygnus.co.uk writes:

> Assuming you use gcc to compile your entire program, we can solve the
>  aligment problem by aligning the stack in main.
Well, yes, if you still think those other OS's are about to die.  But I 
thought I saw Jeff Law recommend against -malign-double the other day, 
without making an exception for any OS.  Now, I personally don't care whether 
g++ ever supports alignment, so I don't intend to start an argument on that. 
I'd hate to see g77 forced into a situation where it depends on alignment for 
adequate single precision performance, with little chance that Windoze will 
support that.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>Richard Henderson wrote:
>
>Of course, I *still* think that a program generating denormals (and
>getting underflow to zero) is wrong:  All of a sudden there's a number
>in the algorithm that's treated as "normal" (while it is _de_normal) -
>and now you can't divide by it.

Perhaps, but, as I pointed out earlier, almost *any* program can generate
denormals if it is compiled to randomly spill 80-bit values as 32-bit
or 64-bit values and tries to compare them to (nearly) identically
computed results to see if they're approximately equal.

In other words, *I* don't see how you can reconcile your statements that
programs generating denormals are wrong *and* that programs must expect
that comparing mathematically identical results can produce values
that are *smaller* (in magnitude) than can be represented by the declared
types of the operands (as is the case for allowing arbitrary truncating
of values computed by the processor to more precision).

That doesn't mean you *can't* reconcile it, of course -- I just don't
see how, and would love for you to explain it (to all of us), especially
so we can put it into the g77 docs.  (The only explanation I can think
of is "you can trust that, if the processor [compiler + libs + machine]
uses excess precision to compute intermediate results, it will never
crash using the resulting denormals as operands in subsequent operations",
which seems rather strained, but not entirely implausible.)

For now, it seems, for the Alpha anyway, denormals are never "generated"
by default (modulo packaging bugs, like libm exp(), or whatever), so
that's not a problem on *that* architecture.

Perhaps we should classify *inputs* of underflows mapping to denormals,
rather than zero (or whatever *other* popular systems do when they're
given the task of converting, say, "1E-500000" to floating point), as
another of these "packaging bugs"?

Now, I still stand by my statement that -mieee *should* have been made
the default from the outset, because, given all the existing code
(like glibc or libc -- apparently at least one of these uses *unsigned
integer* arithmetic to convert between decimal and floating-point, so
it isn't *naturally* going to benefit from hardware that maps underflows
to zero rather than to denormals), it would have ported *much* more
smoothly to Alpha had -mieee been the default.

But, I'm beginning to see how it is *possible*, and relatively
*feasible*, to make having -mno-ieee the default actually *work*.

Problem is, we appear to not be at that point yet, nor do we seem to
have a complete understanding of what it takes to get there.

Further, AFAIK, we have no test suite for exactly this sort of
*partial* IEEE 754 support.  Again, that's an argument for making
-mieee the default, but *also* for, having made -mno-ieee the default,
doing the hard work to make it actually *work* -- writing a test suite,
for example, one that parallels whatever tests we have for IEEE-754
conformance but is constrained to the natural full-speed range of 21[01]64
chips.

In the meantime....

I'm beginning to think we should just tell people that gcc/g77 support
for the Alpha is in "alpha testing".  IMO, based on what I'm seeing
here, it ain't ready for prime time, though, as usual, the experts
should be able to get wonderful performance out of it.

So for 2.95 we say "Alpha support is in alpha testing, especially as
regards floating-point conformance".  That addresses the COMPLEX
(LAPACK) issue as well.

Then we decide what to do for 3.0 (or 2.96), one of:

  -  Make -mieee the default, take the performance hit, but save
     ourselves some amount of time and trouble.  Leave it to experts
     to use -mno-ieee when they want that last N%, and to tell us,
     item by item, what they need changed over time to get it working,
     but, in the meantime, most everything just *works* by default.

  -  Leave -mno-ieee as the default, figure out what it *really* takes
     to make it work (across libraries, debuggers, etc.), and make that
     happen.

>You won't believe this, but I just opened Press et al's "Numerical
>Recipes" at random:

*I* believe it, because I believe the Universe conspires to provide
"coincidences" that, if we're sufficiently alert, offer us opportunities
to avoid making mistakes.  (A sort of weirdly transformed version of
Murphy's Law: "Anything that might go wrong will be warned about
sufficiently for those who are paying attention."  ;-)

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:
> 
> On Mon, Jul 12, 1999 at 07:01:10AM -0000, craig@jcb-sc.com wrote:
> >    Let's calculate accuracy well out of proportion with what anybody
> >    actually needs. They don't need it, but it doesn't matter. We'll do it
> >    that way and then there won't be any argument.
> >
> > Imagine that.  If g77/gcc delivered to -mieee and 80-bit spills from
> > the outset, we'd have avoided all *sorts* of arguments...
> 
> I spent a good bit of the afternoon reading through several of Kahan's
> papers.  Despite my earlier position, I'm mostly convinced we should do
> both -mieee and 80-bit spills by default, despite what other compilers
> for the systems in question do by default.

I think it's *either* -mieee *or* 80-bit spills.

Let me explain why.

If we turn to the example given by Kahan in the interview in Doctor
Dobbs Journal ( http://www.ddj.com/articles/1997/9711/9711a/9711a.htm -
thanks for the link, Andy), we see the following:

<QUOTE>
float x, y, z;

int j;
...
x = y + z; 
if (x >= j) replace (x);
y = x;

...

As far as we can tell, when he turns on optimization,
the value of x is computed in a register
with extra width. This happens on the Intel machine because the
registers have extra width.
It also happens on some others. The value of x that he computes is not,
in fact, a float. It's in a register that's wider.
The machine stores x somewhere, and in the course of doing that,
converts it to a float. But the value used in the comparison is the 
value in the register. That x is wider. The condition that the register 
x be greater than or equal to j doesn't guarantee that x, when stored
in y, will be less than j.
Sometimes y=j, and that should never be. My student counted on what 
compiler writers call "referential transparency" and was disappointed
because the compiler writer saved some time in optimization by not 
reloading the register from the stored value.
</QUOTE>

Note the remark: "The condition that the register x be greater than or
equal to j doesn't guarantee that x, when stored in y, will be less than
j."

Now if 80-bit spills were in effect, the code *as shown* might work in
gcc, because x, y, and z are all in registers.  However, Kahan
particularly remarks that x was stored in y.  If y is a memory location,
you'll get the effect that he and his student observed.

You will get this effect when y is not a simple float, but a float in a
(sufficiently large) struct, which will ensure that `y = x;' _will_ mean
a store in memory.

The error *this particular compiler* made was that the comparison x>=j
should have been performed with the *same* floating point value as was
later stored in y, hence a 32-bit value.

Because gcc almost certainly will not be going to do this by mere chance
if we enable 80-bit spills (and almost no compiler will do this, because
it is very hard to determine what values to store in memory
intermediately), there is only one way to *approach* IEEE-754
conformance with gcc on the ix86 target:  Use a OS-dependent routine to
set the rounding mode to 64 bits.  If you do that, there's no reason to
perform 80-bit spills, although you need to spill floats as doubles.

This still has two limitations w.r.t. to true IEEE conformance:

1. It only works correctly for doubles, because floats still get
   extended to 64 bits and are therefore not equal to their stored
   value.

2. [This one is from hearsay, on comp.arch]  One can only set the
   rounding mode on the ix86, not the range.  That means that the
   register values still will not have the same non-finite behaviour
   as stored values.

HTH,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >Richard Henderson wrote:

No, you can't hold Richard responsible for what I wrote ;-)

> >Of course, I *still* think that a program generating denormals (and
> >getting underflow to zero) is wrong:  All of a sudden there's a number
> >in the algorithm that's treated as "normal" (while it is _de_normal) -
> >and now you can't divide by it.
> 
> Perhaps, but, as I pointed out earlier, almost *any* program can generate
> denormals if it is compiled to randomly spill 80-bit values as 32-bit
> or 64-bit values and tries to compare them to (nearly) identically
> computed results to see if they're approximately equal.

I'm pretty sure that ours don't - in spite of the fact that they perform
somewhere between 10^10 and 10^11 floating point operations per run.

Why ?  Because in the problem domain we're operating in, floating point
numbers have values that are far from FLT_MAX and FLT_MIN.  In our case
that is because if you express the physical quantities in the earthly
atmosphere in SI units, you *get* values that are far from these
extremes.

My point is that every physicist engaging in numerical analysis will
choose his/her units to get this effect.  This is not surprising, as
people like to talk about the major phenomena they study using small
numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
measure distance.

The (automatic) side effect of this "change of units" is that the
numerical algorithms used to describe those phenomena will stay out of
the complexities of having to deal with denormals *unless they are
unstable or operating outside their domain of validity*.

Of course one could always set up a scenario where, using the fact that
you cannot know whether a floating point value is 80-, 64- or 32-bits
wide, you can generate small differences (which should have been zero)
that, suitably multiplied which each other, will generate a denormal
(especially if you start close to FLT_MIN).  My point is that you have
to cleverly set this up, because it will not show up in normal practice.

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < 378A7B90.7DCF39C9@moene.indiv.nluug.nl >you write:
  > > Cygnus wrote a pass based on the lcm routines to find computationally
  > > optimal locations for FP mode switching on the sh.  That same code can be
  > > used to do mode switching for any system which needed it, including stuff
  > > like mmx/fp mode swtiching on ia32.
  > 
  > Well, if this can be done reliably, I am convinced that support for IEEE
  > conformance (minus the range question, which should be documented quite
  > clearly) is possible.
It can be done reliably.  The only major issue with that code as it stands is
it introduces another hook like MACHINE_DEPENDENT_REORG.  We don't want another
hook like that one.  It's the most abused hook in the compiler.

The bulk of the code is designed to be a generic optimization of mode switching
(the precise nature of the mode switch isn't particularly important).

jeff

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>See "man ieee", and #include <machine/fpu.h>. When the program starts
>up, make a call to ieee_set_fp_control(IEEE_MAP_UMZ); , which sets the
>fp control to cause underflow to map to zero.

Ah, well, it's good people can do that if they want, though the arguments
against -mieee being the default included "because it catches bugs in code",
which, presumably, silently mapping underflows to zero won't.

My bigger problems with this whole line of argument are that, first, users
who *see* the IEEE format involved *expect* the full range, and, further,
that's what "everyone" supports on their hardware, so people code to that,
often without *thinking* they are.  The penalty for denormals should not
be program crashes, by default, since IEEE isn't (normally) implemented
that way.

(I assume -- incorrectly? -- that the FLT_MIN and related macros have the
correct values vs. for when -mieee is in effect.)

Second, when I think of how this issue intersects with Toon's *other* opinion,
that 80-bit spills are a waste of time that good programmers know how to
code around, I'm not sure I see *how* they can code around *both* defaults
portably.

To wit: how can a programmer be sure his code will *never* generate a
denormal (and, e.g., compare that denormal to some other value), if the
compiler can, as Toon says it must to "run fast" on machines like the IA32
class (and m68k?), compute *some* results to 80-bit precision and *others*
to 64-bit (or 32-bit) precision on a totally random basis?

Maybe no machine *today* has that combination of 80-bit computations
and a "fast" IEEE mode crashing on denormals (which Toon supports as the
default).  Though it's not hard to imagine Merced or McKinley might.

But, in theory at least, I can't see how a programmer can write code
that does something like

  IF (R - S .LT. .00001)

to test a tolerance *without* risking that R - S evaluates to a denormal,
because, while *mathematically* identical, and *computed* using identical
operations on identical operands, the *compiler* decided to compute R to
80 bits and S to only 32 or 64.

It's not like I feel like researching how to write floating-point correctly
at this point, given all the other stuff I have to do.

I just doubt it can be done, and *especially* that any coherent document
on *how* to do it is as freely available as g77, gcc, etc.

So maybe I'm wrong, but I throw this out in case other people, more
experienced with FP programming, can tell me so, or agree that, indeed,
writing portable code -- for machines that default in all the ways Toon
(and many others; he's just someone I *know* I can trust to care about
Fortran and related issues) wants -- is basically impossible, or at least
so hard to do right that a vanishingly small percentage of programmers
who, *today*, are writing code *intended* for such portability, code
that is being, or will be, *deployed*, are doing so *correctly*.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> I wrote:

> >If people want to get tighter control, there are two ways:

> >1. Try to influence X3J3 to impose them.

> >2. Use another language.

> In which case, you won't mind one whit if -mieee and 80-bit spills are
> made the defaults, since X3J3 doesn't impose *performance* constraints?

> Well, it's about *time* you conceded that.

> ;-/

Hmmmm, I hope I nowhere said that the Standard imposes constraints
*against* using full IEEE-754 conformance and 80-bit spills.

We (and I hold both of us responsible for that) should have discussed
this *as a quality-of-implementation-issue*.

I do not personally have a *performance* issue with 80-bit spills,
because I do not view ia32 type machines as being particularly
"performant".  (I'm still waiting for the "My Other Computer Is A 21264"
sticker Compaq employees promised me at LinuxExpo to put on my PII 300
Mhz laptop).

If we (collectively) decide that -mieee should be the default on
Alpha's, I know how to specify -mno-ieee in my makefiles :-)

What really bothers me is that this discussion is denying 40 years of
research in numerical analysis.  If the issue would just be:  Well, we
have to support 80-bit spills because the highly successful Whiz-Bang
6.0 Destructively Voidable Fortran compiler by Analog Equipment
Corporation does it, it would be fine with me.

But I cannot seriously be asked to just give up 20 years of experience
in numerical physics.

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >The evidence on these lists prove that people are still forced to write
> >programs without getting a decent course in numerical analysis.

> Again, as I pointed out before, *I* have taken no "decent course
> in numerical analysis".

Nope, and it is not necessary for you to do so.

[ Unless you want to take part in discussions like this, otherwise, 
  you should just perform useful work on g77: ]

> Why therefore am I pretending I can do useful work on g77, since its
> raison d'etre is to compile code doing numerical work?

I tried to express that to write useful numerical code one has to have
knowledge of numerical analysis.

It is *not* sufficient to paper over mistakes made by people who didn't
read their stuff just to get "the right answer".

What a Fortran compiler should do is (IMHO) compile the source to the
object code, using the latitude given by the Standard (i.e. SQRT: the
result has a value equal to a processor dependent approximation to the
square root of the argument).

If people want to get tighter control, there are two ways:

1. Try to influence X3J3 to impose them.

2. Use another language.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

PS: Just to prevent mistakes: I do not view Press et al's "Numerical
Recipes" the be-all end-all of numerical analysis.

It's just that some of the issues that came up on this list were
adequately answered by quoting passages from this book.

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Stephen L Moshier wrote:

> > Note that we are talking about correctly implementing IEEE
> > conformance.  The fact that I personally do not need it is orthogonal
> > to that issue.  (but I'm fully to blame for unduly pushing my own
> > needs, for which I apologize).

> But that is what you should do!  If there were an important need for
> "correctly implementing IEEE conformance" then someone would have put
> it in long ago.  We implement the features that we need, and we ignore
> the ones we don't.

If I were you, that would be true.

However (I almost wrote:  Unfortunately ...), I am a member of the
Steering Committee.  As Craig correctly pointed out, that means a
broader responsibility:  I can not just go for my own preferences, but
should create an atmosphere where contributions from both "sides" of the
discussion on floating point arithmetic are felt to be welcomed.

I clearly failed in this - hence my apology.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>> Perhaps, but, as I pointed out earlier, almost *any* program can generate
>> denormals if it is compiled to randomly spill 80-bit values as 32-bit
>> or 64-bit values and tries to compare them to (nearly) identically
>> computed results to see if they're approximately equal.
>
>I'm pretty sure that ours don't - in spite of the fact that they perform
>somewhere between 10^10 and 10^11 floating point operations per run.

But *yours* isn't the only code on the planet, correct?

I mean, c'mon, you *constantly* write about how people *shouldn't* (or
"don't", despite substantial evidence on these lists to the contrary)
write floating-point code, as if it applied to the entire planet's
past, present, *and* future use of Fortran.

And now you're saying "but *we* don't write code that way"?

Again, I don't know where *anyone* can find freely available information
on how they *are* supposed to write floating-point code.  I believe
such information exists as it pertains to *other* compilers, which
behave in a more predictable fashion (e.g. we've had reports that at
least some do 80-bit spills on IA32), but not as it pertains to g77.

>Why ?  Because in the problem domain we're operating in, floating point
>numbers have values that are far from FLT_MAX and FLT_MIN.  In our case
>that is because if you express the physical quantities in the earthly
>atmosphere in SI units, you *get* values that are far from these
>extremes.

How nice...for *you*.

>My point is that every physicist engaging in numerical analysis will
>choose his/her units to get this effect.  This is not surprising, as
>people like to talk about the major phenomena they study using small
>numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
>measure distance.
>
>The (automatic) side effect of this "change of units" is that the
>numerical algorithms used to describe those phenomena will stay out of
>the complexities of having to deal with denormals *unless they are
>unstable or operating outside their domain of validity*.
>
>Of course one could always set up a scenario where, using the fact that
>you cannot know whether a floating point value is 80-, 64- or 32-bits
>wide, you can generate small differences (which should have been zero)
>that, suitably multiplied which each other, will generate a denormal
>(especially if you start close to FLT_MIN).  My point is that you have
>to cleverly set this up, because it will not show up in normal practice.

I am rapidly coming to the conclusion that all of my efforts on g77
are, according to your logic, legitimately applicable to only about
50 truly expert programmers (in floating-point arithmetic) worldwide.

That might explain the stunning lack of breadth in funding I've received,
especially as this "beta" version of g77 has matured.

Needless to say, I'm *also* rapidly losing interest in investing much
more of my time making g77 even *more* attractive to programmers who,
according to you, will be writing incorrect programs, especially since
they seem to have no reasonable way to find out how to write *correct*
ones, and since *I* am nowhere near having sufficient knowledge to
either document this *or* implement and maintain a compiler (which includes
responding to bug reports) that is intended for use in this domain.

So I believe I'm approaching a choice:

  -  Mandate that g77 shall, as of the rewrite (0.6), default to precise
     (e.g. 80-bit) spills of intermediate computations, to full-range support
     of the underlying floating-point *type* (-mieee), etc., even if that
     means effectively forking gcc or writing a new compiler from scratch
     (neither of which I'm likely to undertake myself as part of doing
     the rewrite, of course).  By "mandate" I mean "refuse to work on,
     maintain, or respond to bug reports concerning any product that
     incorporates the 0.6 front end but doesn't meet my standards".  Not
     quite the DJ Bernstein approach (for qmail anyway), but one that's
     entirely reasonable, I believe.

  -  Leave working on g77 to those who actually understand how to write
     code (and therefore maintain compilers) that carefully navigates the
     murky waters left by *not* choosing defaults based on
     my understanding of what it takes to achieve overall robustness.

That means, it is quite likely that the only way I'm likely to believe I
can truly contribute effectively to g77 *and its user base* (beyond a tiny
handful of experts, anyway) is if I can convince the GCC steering committee
to mandate 80-bit spills, -mieee as a default, etc.  Which seems unlikely,
though, at least, while working on *that*, I can proceed with the rewrite,
then effectively abandon its incarnation as a gcc product when failure
to convince becomes clear, and offer it (since it is GPL'ed software) along
with my (limited) services to the community to put into a new compiler
project that meets the requirements I believe the *larger* number-crunching
community has, or which, at least, I can *explain* without tying myself
into knots.  (Whether such an offer is taken up, I could not control,
of course.)

The choice is rather stark simply because, the more I read about your
definition of what constitutes proper floating-point programming, the
*more* confused and, frankly, "frightened" I get, regarding the viability
of actually successfully understanding it or getting other people to
understand it sufficiently to write robust code to your standard.

Whereas, throughout my life, the more I've read about stuff like designing
compilers, the *less* confused and worried I've gotten regarding the
viability of *that*.  Which certainly strongly suggests where I should
devote my efforts (e.g. continue compiler development, avoid Fortran
or other loosely-specified languages used for floating-point work).

For example, ANSI FORTRAN 77 says "The value of a [variable] does not change
until the [variable] [is changed in a way clearly visible in the source
code]".

Now, putting aside any language-lawyer hat I might otherwise be willing to
wear ("but can a value be an *approximation*, the *precise* value of which
is undefined at any given point during execution?"), it seems to me that
means, quite clearly to any *typical* reader of the standard, that

  Y = 1.
  Z = 3.
  X = Y / Z
  IF (X .LT. X) PRINT *, 'FAIL'
  END

must *never*, under *any* circumstances, print "FAIL", because X can have
only *one* value, and the .LT. operator cannot reasonably be interpreted
(again, even if the standard can be *argued* to allow it) to compare
*approximations* (two *different* ones) of the *one* value in a variable
being compared to itself.

But, according to your interpretation (from last December, IIRC), the
compiler may, at any time and in any way of its choosing, choose to
use excess precision to compute results of computations *and* randomly
chop those results down to less precision, so the IF statement could
be interpreted as if it read:

  IF ((X - roundoff) .LT. X) ...

Which is, in fact, something gcc can be easily convinced to do, with
perhaps only a bit more coding, these days.  (The changes necessary to
provoke this probably involve straight assignment.  Again, from a
language-lawyer perspective, you could perhaps argue that straight
assignment is permitted to assign yet another *approximation* -- but I'm
not even going to look that up in the standard, because, even if *allowed*,
it's too poor a quality of implementation in my view.  Of course, I'm
referring here to an assignment to the same type, since it makes sense
that conversion, especially to a less-precise type, involves another
approximation.)

I find that increasingly unacceptable, because I can't explain to programmers
how to avoid it, especially without creating the sorts of denormals (very-
low-magnitude values) you say code should never create.

Now, your *new* justification appears to be that no floating-point
programs worth their salt ever compute values anywhere *near* the
boundaries of denormals, Infs (which I guess crash on Alphas compiled
using the defaults), etc., so this can "never happen".

Problems with *that* logic include:

  -  The fact that we don't, anywhere I can see, document that programmers
     must stay away from those limits (despite widespread advertising of
     the "supported range and precision" of the underlying types by
     hardware manufacturers), which, to encourage portability, must
     necessarily represent the *intersection* of all pertinent limits on
     all machines.

  -  The fact that we don't offer any option (let's call it `-fcheck-toon'
     ;-) programmers can use to more cleanly catch any computations their
     code does resulting in these "inappropriate" values, leaving it to
     the underlying implementation to crash, burn, or simply compute
     incorrect results due (at least partly) to the defaults we've chosen
     for how to compile code.  And since we have no real spec on those
     limits, `-fcheck-toon' seems to me almost as hard to implement as, say,
     `-fcheck-whether-my-code-does-what-I-want'.

  -  The fact that there's no clear, concise, industry-wide spec on
     programming to *these* limits.  (There is one on programming to
     IEEE 754 limits: it's called IEEE 754.  But we appear to be
     inventing our own standard, so we can't just tell people to use
     *that* standard, even if they limit their use of g77/gcc to
     machines that implement it as a native, or nearly native, type.)

  -  The fact that we are unlikely to ever be able to offer high-speed
     computation of an extended type that offers denotations of out-of-
     range values ("too small", "too large", etc.) for code that wants to
     quickly compute intermediate results that it might never use in
     final calculations, but can't effectively filter out, or which it
     might want to explicitly test for at run time.  (IEEE 754, of course,
     offers this -- via concepts like single and double precision,
     denormals, Infs, and NaNs.)

  -  The fact that, because we (the whole gcc project, driven by your
     view of what constitutes proper floating-point programming, which
     surely must pertain to other languages as well as Fortran) are marching
     to our own drummer, we have to modify all libraries that might be
     "downstream" of our product (the libm exp()) example to assure that
     they "behave" according to our "specification" (e.g. when computing
     results in an underflow, silently replace with zero, but if -fcheck-toon
     is specified, crash instead).  So we can't just grab a bunch of
     well-written, highly optimized IEEE-754-conforming code to implement
     underlying routines -- we have to, in effect, grow our own, to meet
     the performance needs of the types we're creating.

  -  The fact that, AFAICT, we "advertise" values like FLT_MIN and FLT_MAX
     which are, to some extent, outright *lies* in terms of what we
     actually support.  At least, FLT_MIN, anyway -- you cannot reliably
     compute down to near FLT_MIN without the compiler randomly computing
     values much smaller for you and then deciding to crash on them (again,
     this is based on *your* claims of what is permissible for the compiler
     over the past year or so).  But I don't see you, or those who appear
     to agree with you, proposing new values for FLT_MIN and FLT_MAX, or
     proposing values for new macros called something like FLT_TOON_MIN
     and FLT_TOON_MAX.  So programmers have to somehow just "know" what
     this magic range is, but are not able to actually look it up
     anywhere convenient.

Further, you appear to not particularly care how a program behaves, compiled
by default, when it computes values you consider out of range.

It was formerly suggested (by you or someone else) that *crashing* when
computing denormals was okay, because that meant the program almost
certainly had a bug.  But now we discover we likely *won't* crash,
but rather silently generate a zero, on a denormal, even though, on
most *other* targets on which we'll generate the correct (IEEE) result.

Yet that (*not* crashing) *now* seems to be cause for *celebration* among
those who claim denormals "never happen anyway" in correct code -- so
*catching* bugs seems to have not been an issue in the first place, as
whether the code crashed or computed incorrect results (admittedly from
inputs declared, by you, to be incorrect) turns out to not have mattered
at all.

So, it seems to me likely that, right now, on Alphas (and again I can't
test this), there are discontinuities in important functions, such as
reading in a floating-point number expressed as decimal.  E.g. on my
Pentium II, if I do the equivalent of "READ (UNIT='1E-5000', *) R",
the value that ends up in R is 0., with no crash.  Apparently, according
to you, it's okay if that also happens on the Alpha, even if changing
the input value to 1E-40 causes a crash (since it generates a denormal),
and even though changing it to 1E-5 produces the correct result (no crash).

But I consider that sort of discontinuity (zero -> crash -> correct value)
to be *wholly* unacceptable, whether it occurs in the inputs to the
text-to-FP converter, or in the inputs to exp(), or whatever.

Now, maybe all that's needed is for these few discontinuities to be
*fixed*, but, from your "camp", so to speak, I see little interest
in doing so, *or* in specifying exactly what the proper behaviors
*should* be so the *rest* of us can know how to fix them.  (In particular,
as far as I can tell, the specification is "anything goes, so it can be
as fast as the hardware can possibly run", which clearly allows for
discontinuous behavior across the domain -- except, of course, in the
at-best-fuzzily-specified region *you* claim is the only one programs
should be employing anyway.  And there's no point in "fixing" such
discontinuities if the "standard" is to go as fast as possible, completely
ignoring the possibility of ill-conditioned input resulting in
discontinuous behaviors.)

So, I'm stumped as to how to what path to take.  I want to work only
on *robustly designed* products, which include taking into account
how *typical* users will use it in practice, and assuring they'll not
get misleading or incorrect results.

Now, I'm not particularly interested in working on something like Java,
which offers the (IMO false) promise of "write once, run anywhere", even
though I'm sure I'd find their discussions of issues revolving around
IEEE 754, performance, and consistent behavior quite valuable in many
ways.

But, I'd like to work on a product that *at least* respects the underlying
floating-point types of the machine and *properly* implements them.  That
means respecting all the expertise that not only went into *designing* those
types but has since gone into *using* them.  If those underlying types are
IEEE 754, so be it.  It also means implementing consistent behavior across
all the intrinsics, conversions, approximations, etc.  It also means doing
80-bit spills on IA32, since that's what the *designer* of the hardware
intended (and, again, it's what the Fortran standard *clearly*, if not
*legalistically*, requires).  (It certainly means, at least, 80-bit spills
of program variables; I believe it might also mean 80-bit spills of
intermediate results.  Of course, there's the argument that what is
*really* means is always computing 32-bit or 64-bit intermediate results,
leaving IA32 performance out in the cold.)

This is all considered "quality of implementation" in the Fortran community,
I'm happy to concede, but my point is, if I can't (and I know I don't)
thoroughly understand the floating-point model the compiler offers, then
I should *not* be working on it, certainly not in a significant role
(anything approaching the role I serve now vis-a-vis g77).

And, right now, I have *serious* doubts that the floating-point model
g77/gcc offers is *anywhere* near understood by *anybody*, including
yourself.  You probably don't realize that that "IF (X .LT. X)" example
above, put into a loop that nevertheless computes the *same* pertinent
values, could result in *different* paths taken via the IF, especially if the
loop contains any other code before *or* after that sample block of code.
But it could, due to loop unrolling or other sorts of code-duplication
optimizations.

(Though you probably *do* realize that adding another identical IF to
the original sample program could legitimately result in different
paths taken in each IF even in a straight-through execution of that code.)

Certainly I don't think most of the *current* g77 audience understands
those issues, and I do not have what it takes to explain them to that
audience.

My guess is, assuming we continue on this path, the *future* g77 audience
*will* understand these issues thoroughly.

But not because we've *explained* these issues to them.

It will be because the future g77 audience (and the audience for gcc's
floating-point capabilities) will consist of maybe 50 people worldwide,
or some number so small it comes *nowhere* near justifying all the effort
we're putting into this aspect of the product.

IMO, an audience that small should either spend a few hundred thousand
dollars for a high-end compiler to be installed on all their workstations,
or use assembly, because that's what the economics of their situation
dictates.

Or, they could just use -mno-ieee, -fno-correct-spills, etc., assuming
we picked *robust* defaults for g77 (and gcc).

So, Toon, what do you suggest?  Do you insist I continue to work on a
product in which I have increasingly *less* confidence, and which is
going in a direction with which I strongly disagree (much of this
"direction" being in the realm of "revelations" about how it's actually
working, and being told not to do anything about it), resulting in my
(hopefully-still-at-least-marginally-good) name being associated with
something which I wouldn't recommend anyone actually *use*?

Or, should I work on something else, something in which I *do* have
confidence can become something which I believe is worthy of
recommendation to others to use?

(These questions are as open to others as they are to Toon, and they
should be considered as questions *other* people working on g77/gcc
might be asking as well.  E.g. maybe Toon would be unwilling to work on
g77 if it didn't have the defaults *he* wants -- after all, convincing
floating-point experts to compile with -mno-ieee might seem, to him
anyway, quite difficult.)

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 7:15:09 PM PST, law@cygnus.com writes:

> We'd be interested in a testcase for this.  To the best of my knowledge we
>  should be aligning spill slots relative to the stack frame.
Yes, I believe that individual compilations align spill slots reliably, but 
it does no good when the OS is one which doesn't believe in stack alignment 
(NT etc.).  The Fortran compilers I've seen for Windoze require all data to 
be put in COMMON if 64- or 128-bit alignments are required, including g77.  
The Livermore Fortran Kernel at ftp://members.aol.com/n8tm/lloops.shar.gz 
(lfk537g.f version) shows this behavior; Kernels 8 and 9 have spills of loop 
invariant data which are aligned in static storage.  Under i686-pc-linux-gnu 
the spills are aligned, but in i586-pc-cygwin it's a matter of luck.  Only 
those 2 kernels with 64-bit spills perform significantly different between 
linux and Windoze.  The speed difference is nearly a factor of 2 in those 
kernels in the PPro, even though most of the data are aligned in COMMON.  
With the 32-bit spills which g77 now uses for single precision, there is no 
performance difference between Windows and linux.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Stephen L Moshier]

Due to a charming DEC assembler bug, the mieee switches did  not
work well enough to be usable on alpha-osf up to osf-4.0 or so.
It would be a disaster to make ieee the default on those systems.

If you have a working assembler now, the ieee modes should be tested.
At the time, it was so hard to make anything at all run that it
would be  a miracle if it really worked.

Sat Mar 16 16:20:43 1996  David Mosberger-Tang  <davidm@azstarnet.com>

    * alpha.md (trap): New attribute.
    Modify patterns for all floating-point trap generating
    instructions.
    * alpha.h (CPP_SPEC): Added -mieee and -mieee-with-inexact.
    (alpha_trap_precision, alpha_fp_rounding_mode, alpha_fp_trap_mode):
    New enum types.

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 06:04:04PM -0600, Donn Terry wrote:
> Rounding mode only affects STORE operations.

Toon was imprecise.  We were not speaking of rounding
(bits 10:11 of fpcr) but rather precision (bits 8:9 of fpcr).

And in any case, rounding _does_ affect every operation.
Just that in 64-bit precision mode, the rounding happens
for the XFmode value of the register, rather than for the
SFmode value of a variable.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>Some elder statesmen of Numerical Analysis really suffer from "All the 
>world is a Cray"-syndrome and never really accepted IEEE semantics.

Unfortunately, they never bothered specifying the FP semantics *they*
wanted from the rest of the industry and publishing that spec for me
(and other gcc developers) to work from.

That's what this really boils down to, I'm afraid: the anti-IEEE people
refuse to accept the notion that they should just put up with the
moderate slowdown they think IEEE mandates, but they don't offer any
*specification* for what it is they *really* want.

At least with IEEE 754, people like myself (who Toon now claims should
not even participate in these discussions!) can implement compilers to a
*spec*, write tests to a *spec*, even run industry-standard tests to
that same *spec*.  All without being experts in numerical analysis or
reading Numerical Recipes -- which gives them more time to be experts
in compiler development and read the Dragon Book.

In other words, I am now prohibited (by Toon) from discussing even whether
my *own* compiler implements floating-point arithmetic *correctly*, because
I'm not a numerical-analysis expert, because there's no *spec* for how it's
supposed to work, and, therefore, I must rely totally on the opinions of
others.

Whereas, when I *thought* g77 was supposed to faithfully work to *at least*
the (sometimes-less-than-ideal) "virtual" spec provided by the underlying
hardware, I *could* discuss whether g77 met that spec, because that spec
is usually formulated in language that doesn't require numerical-analysis
expertise to understand.  (E.g. long ago, when I read IEEE 754, I found
it pretty easy to understand.  Heck, I was helping hardware developers
debug and diagnose floating-point hardware for a mini-supercomputer back
then.  All without numerical-analysis expertise.  I left the company not
long after the VP of Engineering told me there was no way the company
would fix already-deployed systems that I discovered miscompared 64-bit
values that differed starting in bit 32 -- once I learned that the people
at the top didn't care to ensure the machine could be part of an overall,
robust system, I didn't hang around long, despite being paid far more
[salary] than I'm being paid now to work on g77 [nada].)

Now, in the last six months, we've learned (all from Toon, mind you, one
of *the* most respective voices in the OSS number-crunching community, and
for good reason), that:

  1. Even if Fortran says a computation evaluates to a single approximation,
     a good compiler ignores that (as mere "advice") if evaluating multiple
     approximations leads to faster code.  After all, good code is
     (somehow -- I have yet to see any clear, correct recipes as to how to
     make this happen) insensitive to that behavior.

  2. No good compiler bothers special-casing values in the IEEE 754 denorm
     ranges.  That's because no good code computes values in those ranges.
     Therefore it's okay if values in those ranges cause undefined results,
     from crashing to returning really *huge* numbers instead to whatever.

  3. No good compiler should bother worrying about merely *tiny* numbers that
     are "close" to IEEE 754 denorms, such that comparing mathematically
     identical values that are computationally different due to #1 might
     produce denorms.

Particularly of note is that rule #3 sprang into existence, to my view,
only after I'd pointed out the apparent contradiction between rules 1 and
2, which apparently were formulated months, or years, ago.  I.e. rule #3
didn't exist until sometime yesterday.  If anyone believes otherwise,
please email a URL showing where it was publicized, I'd love to see what
I'm missing!

I'm no longer interested in trying to write a compiler to a non-existent
specification, so I've pretty much decided to stop bothering, and go find
something else to do -- which will almost certainly involve software work
only to the extent that the project is run by someone who actually
understands total quality engineering for large-scale projects, which
appears to *not* be the case for GCC (or GNU or even Linux).

I mean, Y2K is a stupid problem made with exactly the same mind-set that
is being employed here: "nobody should ever deploy this system beyond 1999,
that is just *way* too far in the future" is, from a total-quality
standpoint, nearly indiscernible from "nobody should ever compute a value
smaller than 1E-15".  The concept of "degrade gracefully", designed into
IEEE 754 (for better or for worse), doesn't exist for people with this
mind-set -- they prefer "degrade randomly, as long as it still runs fast"
or something like that, and are under the delusion that their approach
works for large-scale systems (which includes basically any project that
depends on a compiler as large and complex as gcc/g77, IMO) developed by
a large number of people with varying sorts and levels of expertise.

Seems to me I have better things to do with my time, experience, and
expertise than put a nice sheen (good docs, rewritten g77 front end,
whatever) on a project where the people in charge can't wait to add the
*next* Y2K-like bug.  However nicely I pave the road, and decorate it with
flowers, it'll still be a road to hell, and it doesn't help much that I
can claim my particular *segment* of it doesn't actually get you there all
by itself.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Edward Jason Riedy wrote:

> And Toon Moene writes:

>  - What I *do* object to is people trying to mimic that in Fortran, hoping
>  - that the Fortran Standard conformant evaluation of expressions and
>  - IEEE-754 requirements don't clash.

> You are quite right; that's the real problem.  Many people bemoan it.

Yep.  Still a lot of useful work is done on computers predating the
IEEE-754 standard (IBM 370's, VAXen and Cray PVP's).

Of these, Crays have an amazingly lax attitude towards floating point
arithmetic.

Why then is it possible to do useful numerical physics on these machines
?  Because most code can be made robust against small differences in
floating point arithmetic.

Sure, when run on different architectures, they will give different
results.  The crucial question, however, is:  Are these differences
interesting ?

I know it is dangerous to keep using our own application as an example
(I am glad you and others pipe in with different examples), but we
occasionally run our Numerical Weather Prediction code in double
precision (using compilers that support -r8 -i8 "promotion" flags).

Of course there are differences - but they're all meteorologically
irrelevant.

Heck - before the 60's, planes were designed using slide rules; that's 3
digits of precision.  The reason that worked is that aircraft designers
at that time used completely different algorithms to compute the
relevant physical phenomena around body and wings.

> However, there is a growing body of Fortran code (amazing statement,
> even without the following quantifier) that thinks Fortran supports
> IEEE-754 in most ways.  And most of the compilers I can see (just
> looked at the T3E (MIPSpro) docs, NAG's new f95 compiler's man
> page, etc.) seem to be following that.

I do not know about the MIPSpro compiler, but NAG's F95 compiler is
essentially written by one man: Malcolm Cohen.  At the latest X3J3
meeting (7-11 June, in Oxford) he pointed out that he had already
implemented most of the Chapter 15 requirements of F2K (which describe
floating point exception handling and the support of IEEE-754 features).

Of course I rather wish we could act that fast with g77 too, but you
should realize that:

1. Malcolm has been on the Committee for a very long time, so he
   knows what's coming.

2. NAG's compiler is protected from the idiosyncracies of existing
   hardware by being a Fortran 95 to C compiler.

> I'm not going to pretend that I know the differences between the
> Fortran Standard and IEEE-754.  I don't want to know the differences.
> (Just found out that F90 and F95 don't even agree on how to print
> "-0.0".  Yech.)

This was changed in Fortran 95 to support hardware that could discern
+0.0 and -0.0 (i.e. to enhance support for IEEE-754).

> Fortran seems to be moving in the IEEE direction, so I suppose the
> real question should be whether or not IEEE floating point support
> by default would _break_ functional codes.  I don't see how, but
> of course someone at Berkeley is going to be supporting IEEE
> arithmetic.  ;)

Well, you can rest assured that IEEE-754 support as formulated in the
upcoming F2K standard won't break existing code.  It's all of the form

      IF (IEEE_SUPPORTS_SOME_FEATURE()) THEN
         ... use IEEE-754 feature ...
      ELSE
         ... just plod along as in the old days ...
      ENDIF

> --other--

> BTW, in some FP experts' opinions (I don't qualify as an expert;
> I'm relaying what's been said by actual experts around here) there's
> no such thing as fast code that produces incorrect results.  It's
> incorrect code.

The problematic term here is "incorrect".  See my note above on
"irrelevant differences"; in short, if you can measure wind speed to 0.1
m/s accuracy, the fact that two model runs on two different
architectures differ in the fifth decimal is not relevant.

> And much of the code around here does _NOT_ fall into Toon's category.
> The FE codes very much have to deal with the nastier parts of the FP
> spectrum.  Some of the solutions are algorithmic (ignoring the
> vibrational modes of waterpipes)

Yep - just as we ignore sound waves in the atmosphere.  You model what
you want to describe; nothing more, nothing less.

, and others are numeric (using
> extended precision implemented on IEEE arithmetic during iterative
> refinement).

But how do you reliably code that ?  Going to assembler ?  There is no
way to Standardly specify IEEE-754 extended precision arithmetic, yet.

> Yes, it could conceivably be possible to code around
> a lack of IEEE-754.  The algorithms involved are complicated enough,
> thank you.

Well, OK; that's a choice you want to make.  I think it's rather obvious
that I don't like to live that dangerously.

> However, you won't hear a peep out of most of the folks who write
> them.  They still think g77 is only f2c, and they all bought tuned,
> vendor compilers.  Many don't even know g77 still exists, or think
> it died off (don't know it's now distributed with gcc).

Hmmm, that sounds more like a problem with our PR department :-)

> g77's market around here right now is very, very, very small,
> consisting of folks who want a nice set of tools that lets them mix
> C, C++, and Fortran nicely, typically on clusters of Linux boxes.
> To be fair, gcc doesn't deal with the biggest systems in my immediate
> vicinity (Crays of various flavors), and won't link to vendor SMP-
> capable BLAS on others (licensed libraries on Suns).  And other codes
> we want to play with (MUMPS, etc.) are in F90.  So my arguments can
> probably be dismissed; I'm not going to be using g77 for real work
> anytime soon.

It's a pity, but I agree that one should use a tool that does the job at
hand.  If g77 doesn't cut it, that's fine with me.

Thanks for sharing your insights.

[ I hope I dealt with topics raised in other e-mails too - if not,
  please take it up with me privately ]

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>BTW, in some FP experts' opinions (I don't qualify as an expert;
>I'm relaying what's been said by actual experts around here) there's 
>no such thing as fast code that produces incorrect results.  It's
>incorrect code.  

That's what Toon has been saying, isn't it?

>And much of the code around here does _NOT_ fall into Toon's category.  
>The FE codes very much have to deal with the nastier parts of the FP 
>spectrum.  Some of the solutions are algorithmic (ignoring the 
>vibrational modes of waterpipes), and others are numeric (using 
>extended precision implemented on IEEE arithmetic during iterative 
>refinement).  Yes, it could conceivably be possible to code around
>a lack of IEEE-754.  The algorithms involved are complicated enough,
>thank you.

Ah, this is where Toon and I *definitely* part company.

In Toon's world-view, all programmers need to be experts on all the
ways the compilers they use can make their lives miserable.  Being
experts on the standards isn't even enough (certainly not for Fortran,
since basically nothing worthwhile is guaranteed by the present standards),
and it is positively *stupid* to assume reasonable straightforward
quality-of-implementation decisions made by compilers.

In *my* world-view, I'd rather programmers be experts on their *problem
domain*, and the compiler implement their codes in the most straightforward
way possible, leaving optimization to people who *are* experts in
optimization.

Working in Fortran certainly de-virgin-izes my stance, just as it does
Toon's.  That is, Fortran is hardly the right language for programmers
to express their problems without regard for speed issues; and it
certainly is the wrong language for programmers to express bare-to-the-
metal optimizations.

But at least, with my world-view, things get *more* predictable the
less you worry about how the underlying components work.

With Toon's world-view, things get *less* predictable.  That is, you have
to constantly *increase* your expertise on how off-the-shelf components
work (in terms of how they don't do things as you'd "expect", whatever
that might mean) as you decide to use more off-the-shelf components.

(Which perhaps partly explains why shops like Toon's avoid using even
25-year-old off-the-shelf components, like SIN().)

Again, I perfectly agree with Toon's *view* as it pertains to certain
*domains*.

Clearly, I disagree pretty strongly that his view applies to the entire
world, or even the majority, of potential users of g77 (or g90 or g95
or g2k).

>So my arguments can 
>probably be dismissed; I'm not going to be using g77 for real work 
>anytime soon.

It's my suspicion that that statement is, and will ever remain, true
for *many* potential users of g77, as long as it chooses defaults that
favor unpredictability to achieve speed, and especially as long as it
doesn't offer fully-tested, fully-blown versions of *options* that
provide the predictability those users insist upon, as you yourself do.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >I do not personally have a *performance* issue with 80-bit spills,
> >because I do not view ia32 type machines as being particularly
> >"performant".

> IMO that statement *seriously* contradicts your statements regarding
> my 80-bit-spill proposal back in December.  In fact, I don't recall
> any reason *other* than performance that you (and others, Dave Love and
> Richard Henderson, IIRC) offered to explain your outright rejection of
> my proposal.

This is probably my fault - I should have separated the two issues more
clearly.  The point I really wanted to make is that in current, Standard
Fortran, there is no way to deal with the fact that on some hardware,
floating point numbers in registers have different precision (and range)
then when stored in memory.  Therefore, IMHO, the "most" correct way of
dealing with, say, DOUBLE PRECISION calculations is to store *all*
intermediate values of computations.  Obviously, this would be very
costly - so we settle for the less "correct" solution of ignoring all
issues with intermediate extra precision.  Not having 80-bit spills is
just one of the ways in which we are ignoring these issues.  We could
spill floating point registers in 80 (or rather 128) bit chunks, but
that would still not solve this problem completely.

> >If we (collectively) decide that -mieee should be the default on
> >Alpha's, I know how to specify -mno-ieee in my makefiles :-)
> 
> As with 80-bit spills, that decision has been made, and I consider
> it irrevocable within the GCC project at this point when it comes to
> evaluating my future involvement.  Even if that decision was changed, which
> I still think it might be, someday, it has taken *far* too much effort
> to get people (who *should* be more competent at these sorts of decisions,
> especially given the roles the play in the project) to see the light.
> 
> >What really bothers me is that this discussion is denying 40 years of
> >research in numerical analysis.
> 
> No, it isn't, and what really bothers *me* is people like you chiming
> in on discussions that pertain to software quality and usability issues
> (which you appear to have little understanding of), claiming they're
> *really* about numerics, and then turning around and telling *me* I
> shouldn't be participating in the discussions because I'm not an expert
> in numerics!
> 
> Here's the pertinent part of my *original* post on the topic, which is
> what *you* started responding to:

  ...

> Notice I said *nothing* about *numerical analysis* issues, I was referring
> *entirely* to all the special tweaks and contortions *users* must go through
> to get the IEEE-standard-conforming behavior *they* want, and have it
> actually *work*, for *them*.

Yes, my apologies - I was beating a straw man (this goes for the rest of
your e-mail, too).

Sorry,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:
> 
> On Mon, Jul 12, 1999 at 11:57:55PM +0200, Toon Moene wrote:
> > I think it's *either* -mieee *or* 80-bit spills.

> Eh?  One is proposed for Alpha, the other for IA-32.

Yes, but it still might not be totally transparent to everyone that *on
the ia32* you have to specify *either* -mieee *or* 80-bit spills.

> > This still has two limitations w.r.t. to true IEEE conformance:
> >
> > 1. It only works correctly for doubles, because floats still get
> >    extended to 64 bits and are therefore not equal to their stored
> >    value.
> 
> No, you can set the rounding mode to 24, 53, or 64 bits.

OK, but the point is you have to set it to *one* value for the complete
compilation (unless you want to call this routine after every
calculation to set the default right for the next one).

> > 2. [This one is from hearsay, on comp.arch]  One can only set the
> >    rounding mode on the ix86, not the range.  That means that the
> >    register values still will not have the same non-finite behaviour
> >    as stored values.
> 
> True.
> 
> r~

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/10/99 7:08:22 AM Pacific Daylight Time, craig@jcb-sc.com 
writes:

>   2. No good compiler bothers special-casing values in the IEEE 754 denorm
>       ranges.  That's because no good code computes values in those ranges.
>       Therefore it's okay if values in those ranges cause undefined results,
>       from crashing to returning really *huge* numbers instead to whatever.
>  
>    3. No good compiler should bother worrying about merely *tiny* numbers 
> that
>       are "close" to IEEE 754 denorms, such that comparing mathematically
>       identical values that are computationally different due to #1 might
>       produce denorms.

I'd be happy to resurrect paranoia or some similar code if you can specify 
it, in whichever version you like, and use that as the standard of whether 
g77 is dealing adequately with these issues.  It's been quite a few versions 
since I've run it, but my recollection is that g77 is as good as or better 
than other compilers.  I've put a lot of work into making it portable so that 
it gives the reports it was intended to give, regardless of whether you 
specify -ffloat-store and the like, including working on a bunch of non-IEEE 
architectures which have bittent the dust.  I believe it is possible to fix 
all the places where its behavior depended on issues such as 64-bit vs 80-bit 
spills.  When those issues matter, the writer of the code must know about it, 
and should fix things so there is no ambiguity.  When fixed up properly, 
paranoia will tell which compiler options give consistent extra precision, 
and which give inconsistent extra precision, avoiding cases which push harder 
than necessary on that issue.

No doubt there are certain targets and option selections which could present 
problems, but I would think it should be the business of g77 to let sanely 
implemented hardware systems do their job as they were designed to do.  One 
obvious point where the Fortran standard gets in the way of this is in the 
definition of NINT() and ANINT(), where most applications of which I am aware 
would be happy to do it the way the hardware is designed to do, almost 
invariably in accordance with IEEE754/854 nowadays.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>The evidence on these lists prove that people are still forced to write
>programs without getting a decent course in numerical analysis.

Again, as I pointed out before, *I* have taken no "decent course
in numerical analysis".

Why therefore am I pretending I can do useful work on g77, since its
raison d'etre is to compile code doing numerical work?

>Why do you think I quote "Numerical Recipes" ?  This stuff is so well
>known that it is equivalent to pointing out an error in gcc wrt to the
>Dragon Book.

I do not possess a copy of "Numerical Recipes", nor have I ever read it.

Why am I working on g77?

(For that matter, I do not possess a copy of the Dragon Book.  Why am
I working on gcc, or doing *any* compiler work?)

I guess the problem is, I read the *standards*, like ANSI FORTRAN 77
and ISO C, and draw conclusions, including the one I've drawn, that
failing to spill 80-bit values to 80-bit temporaries is a bug.

IMO, that fact *cannot* be contradicted by anything "Numerical Recipes"
says, since such a book can only go so far as to advise people on how
to work around bugs in products like g77 and gcc...unless you're claiming
the Fortran and C languages are *based* on the "standards" that make
up the text of "Numerical Recipes", which would be news to me.

Put another way: a 1920s-era book on just how to maintain the automobiles
of the day should not be cited as evidence against the need to improve
automobiles, e.g. by replacing the crank with an electric starter.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >> Perhaps, but, as I pointed out earlier, almost *any* program can generate
> >> denormals if it is compiled to randomly spill 80-bit values as 32-bit
> >> or 64-bit values and tries to compare them to (nearly) identically
> >> computed results to see if they're approximately equal.

> >I'm pretty sure that ours don't - in spite of the fact that they perform
> >somewhere between 10^10 and 10^11 floating point operations per run.

> But *yours* isn't the only code on the planet, correct?

No, but our code runs 7 x 24 hours in 7 operational meteorological sites
in Europe - if I knew how, I would immediately enable
"trap-on-denormals", like we would run with -fbounds-check [too large a
performance dip for operational use] on, because it would point us at
problems in our algorithms.

> I mean, c'mon, you *constantly* write about how people *shouldn't* (or
> "don't", despite substantial evidence on these lists to the contrary)
> write floating-point code, as if it applied to the entire planet's
> past, present, *and* future use of Fortran.

The evidence on these lists prove that people are still forced to write
programs without getting a decent course in numerical analysis.

Why do you think I quote "Numerical Recipes" ?  This stuff is so well
known that it is equivalent to pointing out an error in gcc wrt to the
Dragon Book.

Why do you think I put a $200 / hour premium on this ?  If people are
forced to deal with badly written software in such large IT firms like
Siemens, they'd better be prepared to pay up, or shut up.

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Edward Jason Riedy wrote:

> And Toon Moene writes:

>  - My point is that every physicist engaging in numerical analysis will
>  - choose his/her units to get this effect.  This is not surprising, as
>  - people like to talk about the major phenomena they study using small
>  - numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
>  - measure distance.
> 
> Please go to http://www.cise.ufl.edu/~davis/sparse/ and support
> the claim that all of these matrices (most from real problems,
> many from industry) never run into floating-point problems and
> always live happily in your world of `just-right' numbers when
> being factored or when used in an iterative solver.
> 
> You can't, because they don't.  Now, they can give useful solutions
> using less precision in most places and more in a few, key places
> (see papers by Demmel and Li on parallelizing SuperLU), but that's a
> different story.

It will take time to review this.

> Likewise, from a new eigenvalue routine in LAPACK, a large algorithmic
> improvement:
> *  Note 3 : ZSTEGR works only on machines which follow ieee-754
> *  floating-point standard in their handling of infinities and NaNs.
> *  Normal execution of ZSTEGR may create NaNs and infinities and hence
> *  may abort due to a floating point exception in environments which
> *  do not conform to the ieee standard.

I do not have any problems with library routines using IEEE-754
conformant floating point arithmetic to its extreme.

What I *do* object to is people trying to mimic that in Fortran, hoping
that the Fortran Standard conformant evaluation of expressions and
IEEE-754 requirements don't clash.

There's a reason the F2K standard is trying to accommodate IEEE-754
compliant floating point arithmetic.

> Mr. Burley's defaults (-mieee and appropriate-width spills) are the
> only sane ones possible for a general-purpose compiler to be used by
> anyone.  People shouldn't have to read every LAWN and ask every FP
> expert to determine what their system's particular quirks are.

They do not need to as long as they do not try to walk the bleeding
edge.

Our floating point intensive code works on several variaties of floating
point hardware *because* we do not try to walk on the bleeding edge.

> BTW, the Alpha-denorm non-implementation caused major problems
> with testing and using ScaLAPACK on a multi-architecture PVM
> machine, anecdotally from the authors.  The old argument about
> denorms is covered in
> http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.html .
> Interesting reading.

I will read that.  Thanks for the pointer.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:

> On Thu, Jul 08, 1999 at 03:58:45PM -0000, craig@jcb-sc.com wrote:

> > Ah, well, it's good people can do that if they want, though the arguments
> > against -mieee being the default included "because it catches bugs in code",
> > which, presumably, silently mapping underflows to zero won't.

> Actually, most of the bugs are where you _start_ with denormals,
> typically because of uninitialized data.  UNZ doesn't stop trapping
> when the inputs are non-normal.

[ Thanks for reminding me that underflow always maps to zero on Alpha -
  keep forgetting that ]

Of course, I *still* think that a program generating denormals (and
getting underflow to zero) is wrong:  All of a sudden there's a number
in the algorithm that's treated as "normal" (while it is _de_normal) -
and now you can't divide by it.

I cannot glean inside Martin's nonlinear equations, but the following
description:

<QUOTE>
Perhaps, but while solving highly nonlinear sets of equations, like in
analog circuit simulators nearly anything can appear. Nobody knows,
where newton will bring you. You can decide to make smaller newton 
corrections only  a f t e r  you have the bad ones and thus you have to 
deal with these bad numbers somehow.
</QUOTE>

to *me* sounds like a grave mis-use of Newton-Raphson root finding.

You won't believe this, but I just opened Press et al's "Numerical
Recipes" at random:

Chapter 9  -  Root Finding and Nonlinear Sets of Equations

9.6 Newton-Raphson Method for Nonlinear Systems of Equations

We make an extreme, but wholly defensible, statement:  There are _no_
good, general methods for solving systems of more than one nonlinear
equation.  Furthermore, it is not hard to see why (very likely) there
_never_will_be_ any good, general methods: ...

Martin, proceed from there (pay special attention to 9.7 Globally
Convergent Methods for Nonlinear Systems of Equations).

Hope this helps,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

PS: This is getting of-topic; for more help, the rate is $200 / hour :-)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Joern Rennecke wrote:

> No, we may compute intermediate values in a higher precision, as long as
> we make sure that a value that is assigned to a variable remains consistently
> the same value when it is used more than once.

For referential transparency this is sufficient.  However, if gcc
supports IEEE conformance for backends A, B, and C, and only A will
generate higher precision intermediate results, that would mean that an
IEEE conformant program could (potentially) generate different results
on A from those on B and C.

This is the same question as whether you can use an multiply-add
instruction that doesn't correctly round down the intermediate result.

This doesn't sound acceptable to me (from an IEEE-conformance
standpoint).

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Edward Jason Riedy]

And Toon Moene writes:
 - 
 - What I *do* object to is people trying to mimic that in Fortran, hoping
 - that the Fortran Standard conformant evaluation of expressions and
 - IEEE-754 requirements don't clash.

You are quite right; that's the real problem.  Many people bemoan it.

However, there is a growing body of Fortran code (amazing statement,
even without the following quantifier) that thinks Fortran supports 
IEEE-754 in most ways.  And most of the compilers I can see (just
looked at the T3E (MIPSpro) docs, NAG's new f95 compiler's man
page, etc.) seem to be following that.

I'm not going to pretend that I know the differences between the 
Fortran Standard and IEEE-754.  I don't want to know the differences.  
(Just found out that F90 and F95 don't even agree on how to print
"-0.0".  Yech.)

Fortran seems to be moving in the IEEE direction, so I suppose the
real question should be whether or not IEEE floating point support
by default would _break_ functional codes.  I don't see how, but
of course someone at Berkeley is going to be supporting IEEE 
arithmetic.  ;)

 - I will read that.  Thanks for the pointer.

And if anyone else is interested, Dr. Kahan does have some other 
interesting FP texts under http://www.cs.berkeley.edu/~wkahan/ ,
including a rant on Java's arithmetic.  (No, I don't want to think
about gcj's IEEE-754 non-support.  ;) )  Dr. Fateman has some
vague lingustic notes from FP98 at .../~fateman/fp98/, but the
only part that may be of interest is in the korenD directory,
and it's not on the denorm topic.

--other--

BTW, in some FP experts' opinions (I don't qualify as an expert;
I'm relaying what's been said by actual experts around here) there's 
no such thing as fast code that produces incorrect results.  It's
incorrect code.  

And much of the code around here does _NOT_ fall into Toon's category.  
The FE codes very much have to deal with the nastier parts of the FP 
spectrum.  Some of the solutions are algorithmic (ignoring the 
vibrational modes of waterpipes), and others are numeric (using 
extended precision implemented on IEEE arithmetic during iterative 
refinement).  Yes, it could conceivably be possible to code around
a lack of IEEE-754.  The algorithms involved are complicated enough,
thank you.

However, you won't hear a peep out of most of the folks who write 
them.  They still think g77 is only f2c, and they all bought tuned, 
vendor compilers.  Many don't even know g77 still exists, or think
it died off (don't know it's now distributed with gcc).

g77's market around here right now is very, very, very small, 
consisting of folks who want a nice set of tools that lets them mix 
C, C++, and Fortran nicely, typically on clusters of Linux boxes.  
To be fair, gcc doesn't deal with the biggest systems in my immediate 
vicinity (Crays of various flavors), and won't link to vendor SMP-
capable BLAS on others (licensed libraries on Suns).  And other codes 
we want to play with (MUMPS, etc.) are in F90.  So my arguments can 
probably be dismissed; I'm not going to be using g77 for real work 
anytime soon.

Jason

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> I spent a good bit of the afternoon reading through several of Kahan's
> papers.  Despite my earlier position, I'm mostly convinced we should do
> both -mieee and 80-bit spills by default, despite what other compilers
> for the systems in question do by default.

What are we going to do about alignment?
Make keeping 128-bit-alignment the default, and hope that soon crt0.o
and libraries will provide / keep this alignment too on platforms where
we care about performance?

> The spill widening can actually be done in such a way that it helps
> alpha ev4 wrt spilling QImode.  So I want that capability for other
> reasons.

Will this design also allow to use the alignment bytes of the 10-byte-values
to put smaller spills there?

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < 378A8274.D7309E6A@verinet.com >you write:
  > I pushed on this hard enough that I hacked up gdb to print out
  > the x86 FP registers in both hex and float decimal, along with intepreted
  > values of the control registers and a TOS marker.  It helped a lot.
  > (This is for an older gdb, and I haven't converted it to a patch, but
  > I'll be happy to ship the file to someone to convert it to a newer
  > gdb; the changes should be easily portable.)
I would strongly recommend you get this stuff to the gdb folks.  They're
in the process of trying to beef up gdb's handling of the floating point
unit.  This is precisely the kind of thing that it would be great to have
in gdb.

jeff

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>Someone else had a pointer to Prof. Kahan's home page (he is one of the
>designers of IEEE-754 arithmetic).  There is an another article there,
>"How Java's Floating-Point Hurts Everyone Everywhere",
> http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf , which talks much more
>about the pitfalls of floating point than simply bashing Java.  Kahan and
>his co-author were trying to convince Sun to abandon some bad decisions
>they'd made, and ended up writing one of the best papers on numerical
>analysis that I've ever seen.
>
>People writing numerical programs (or fortran compilers) should read this
>paper. 

A pity Kahan doesn't value standards conformance sufficiently to put
it up as a Web page (.html -- I tried it, "not found").  That means I
can't read it, nor can a friend of mine who is blind, among other things.

;-/

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Thu, Jul 08, 1999 at 03:58:45PM -0000, craig@jcb-sc.com wrote:
> >See "man ieee", and #include <machine/fpu.h>. When the program starts
> >up, make a call to ieee_set_fp_control(IEEE_MAP_UMZ); , which sets the
> >fp control to cause underflow to map to zero.

Underflow to zero always ocurrs when software completion (ie -mieee)
is not requested.  This is still useful for when you need -mieee to
handle Inf and NaN, but don't care about denormals.

> Ah, well, it's good people can do that if they want, though the arguments
> against -mieee being the default included "because it catches bugs in code",
> which, presumably, silently mapping underflows to zero won't.

Actually, most of the bugs are where you _start_ with denormals,
typically because of uninitialized data.  UNZ doesn't stop trapping
when the inputs are non-normal.

> Maybe no machine *today* has that combination of 80-bit computations
> and a "fast" IEEE mode crashing on denormals (which Toon supports as the
> default).  Though it's not hard to imagine Merced or McKinley might.

The user mode documentation for IA-64 is public.  One can see that
it supports an 80 bit mode, but it also has instruction forms that
do proper rounding to single and double precision.  Intermediate
results therefore will be properly rounded for the mode.



r~

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>I do not personally have a *performance* issue with 80-bit spills,
>because I do not view ia32 type machines as being particularly
>"performant".

IMO that statement *seriously* contradicts your statements regarding
my 80-bit-spill proposal back in December.  In fact, I don't recall
any reason *other* than performance that you (and others, Dave Love and
Richard Henderson, IIRC) offered to explain your outright rejection of
my proposal.

>If we (collectively) decide that -mieee should be the default on
>Alpha's, I know how to specify -mno-ieee in my makefiles :-)

As with 80-bit spills, that decision has been made, and I consider
it irrevocable within the GCC project at this point when it comes to
evaluating my future involvement.  Even if that decision was changed, which
I still think it might be, someday, it has taken *far* too much effort
to get people (who *should* be more competent at these sorts of decisions,
especially given the roles the play in the project) to see the light.

>What really bothers me is that this discussion is denying 40 years of
>research in numerical analysis.

No, it isn't, and what really bothers *me* is people like you chiming
in on discussions that pertain to software quality and usability issues
(which you appear to have little understanding of), claiming they're
*really* about numerics, and then turning around and telling *me* I
shouldn't be participating in the discussions because I'm not an expert
in numerics!

Here's the pertinent part of my *original* post on the topic, which is
what *you* started responding to:

>There are two things that might be working, together or separately, to
>give you trouble:
>
>  1.  You're supplying your own main(), but not running the libg2c
>      initialization code libg2c's main() normally runs.  See the
>      g77 docs for more info.  (Though in your example you wouldn't
>      need to call the routine to set up command-line info, you might
>      in the product code from which you derived the example.  And,
>      in both examples, setting up the signal stuff might be necessary.)
>
>  2.  You're using libg2c to print the FP value, but libg2c *itself*
>      has (presumably) not been compiled with -mieee.
>
>      I had assumed (erroneously, as I recently discovered) that gcc
>      automatically used multilibs to produce -mieee-compiled libraries
>      for Alpha targets, because, without that, libraries can crash (and
>      libg2c apparently *will* crash doing, e.g., output of floating-point
>      values that require -mieee).
>
>  3.  You're not supplying `-mieee' when linking the object files, so
>      even if there *was* a libg2c.a built with -mieee available (and
>      properly configured, whatever that means), you'd pick up the default
>      one and, if #2 is correct, crash anyway.
>
>My own opinion is that this sort of thing is insanely painful to deal with,
                        ^^^^^^^^^^^^^^^^^^
>and that the default should have been -mieee all along (both for gcc and
>for Digital's compilers, but I assume the latter have all-along offered
>easier ways to get the -mieee behavior), with an option (-mno-ieee) being
>needed to get -mno-ieee performance.
>
>But the philosophy of most software designers these days seems to be
>"make it fast, then offer the user a variety of tweaks to make it work
>more slowly".  (The same philosophy is why gcc won't ever default to
>80-bit spills of 80-bit FP registers on the x86, I'm afraid.)

Notice I said *nothing* about *numerical analysis* issues, I was referring
*entirely* to all the special tweaks and contortions *users* must go through
to get the IEEE-standard-conforming behavior *they* want, and have it
actually *work*, for *them*.  (Not to mention the credible evidence that
even if they are happy with -mno-ieee, the *quality* of that default is
pretty poor, e.g. as regards libm exp(), formatted/list-directed I/O,
or whatever.  Not surprising, since we haven't really been able to *test*
that stuff nearly as well as if it more thoroughly implemented IEEE 754.)

Now, you could argue we should never have even offered that (IEEE-conforming)
behavior as an *option*, but that's not what you did.  You started talking
about how preferring industry-standard-conforming behavior *as a default*
was wrong from a *numerical* perspective.  You even hand-waved the fact
that there were behaviors *entirely within the -mno-ieee framework* that
most people (including, I believe, Dave Love) recognized as buggy, so
intent were you on punishing (in my view) anyone for merely *considering*
processing a number below a certain magnitude (within some number of bits
of fraction of a denormal).

The *reality*, Toon, is that choosing -mno-ieee as a default is probably
the biggest reason we provide *no working FP environment* in gcc or
g77 on Alpha machines.  Because -mno-ieee wasn't properly designed or
implemented, and gets inadequate testing (in that there's presumably
tons of IEEE-conformance-testing code that we can't use), it doesn't offer
the sort of consistent environment usually associated with high-quality
implementations.  And -mieee, which *could* have enjoyed the benefits
of all that IEEE-conformance testing, didn't, because it wasn't offered
as the default, and we (collectively) decided -- thanks to opinions like
yours holding sway -- that it wasn't worth bothering much about, so *it*
doesn't really work, either.

(As I pointed out, at least with Digital's compilers, they actually
made sure *both* modes *worked*, and both (presumably) generate faster
code than g77.  But, then, they also *charged* people for the darned
things!  Not surprising, given the last of the three quotes I use near
the end of this email.)

It is the complete and total lack of responsibility you and, to a lesser
extent, other GCC Steering Committee members, exhibit towards even
*attempts* by people like myself to strongly encourage paying attention to
proper design and engineering, quality, and usability issues, that
has led to my decision to no longer work on g77 beyond the 2.95 release.

That plus the fact that probably 99% of *my* writings on this topic
have focused on these very issues, and, instead of responding to them
at a level of competence as befits someone who has taken on the task
of running such an important project, you *continue* to attack on
irrelevant issues like numerical analysis.

*My* analysis of this situation wouldn't change *one whit* if the issue
hadn't involved FP stuff at all, but had instead involved something like,
say, dynamic memory allocation.

For example, if the question was (to pick an actual example, beyond which
I actually don't know the details of the rest of this scenario, so it should
be treated as hypothetical)

  "Should we specify that we handle malloc(0) by crashing, by returning
   NULL, by returning a specific non-null pointer every time, or by
   returning a *unique* (until later freed) pointer every time?"

and if it was *clear* that, while one billion LOC existed that would run
a bit faster if any of the first three options were chosen, *ten*
billion LOC, *including that first one billion*, would run just fine
if the *last* option was chosen, then I'd say it was crystal-clear that,
if the project wanted to DTRT with regards to quality, one of two things
would have to be done:

  -  Choose the fourth option, swallow the performance hit, perhaps
     offer an option for users of the one billion LOC to get the faster
     behavior, but in the meantime get the benefit of all that (10B LOC)
     testing.

  -  Choose one of the other options *and* undertake rewrites of at least,
     say, 50% of the 9B LOC that won't run without modification
     under the option chosen, so at least *some* reasonable amount of
     testing occurs, to make this choice at least *approximate* the
     robustness of the fourth option.

     *And* (note how this is still part of the second choice), *definitely*
     offer the fourth option to users who want to get the other 9B
     LOC up and running without modification.

     *And* (still!) ensure that *all* that code actually gets testing to
     at least 90% of the degree to which it would have anyway had
     the *former* choice (fourth option as default) been made.

     (That latter part is harder to estimate for an OSS project,
     when anyone can do testing without "signing up".  The wrong response
     to that reality is to "punt".  The right response is to *overestimate*
     how much testing of the chosen default *would* have taken place,
     *underestimate* how much testing of it as an option *will* take
     place, and do whatever is necessary to make up the difference.)

Notice how there's simply *no need* to resort to expertise in, say,
garbage collection, heap management, and so on.  Not that those issues
aren't important -- they *underly* the decision-making process -- but,
they're entirely subordinate to that process *given the information
already at hand*.

Also note how *vastly* more expensive that latter choice appears, to
anyone truly willing to *consider* it, than the former choice.  It makes
things like a 10% hit in run-time performance seem truly puny, especially
given that users who actually *experience* that hit can at least *try* the
option they might have been given to get the faster model.

And, as it amounts to a sort of "equation", it illustrates that people
who choose performance over working correctly for a much larger superset
of existing code *necessarily* are choosing *lower quality* for the
product overall, when it comes to a project like GCC.

The exact same situation applies here (to a new compiler project, for
example).

The issue is not correct numerics per se, so no numerical analysis issues
really come up, beyond verifying these basic facts: that, if IEEE 754
conformance was chosen, a *superset* of existing, usable code would
have been available for testing, when compared to choosing some ad-hoc
degree of conformance (say, something vaguely Cray-like).

So your repeated bragging about your expertise in numerical analysis
is about as relevant to *this* discussion as a VLSI logic designer
yelling about how everyone should listen to *his* opinions regarding
whether GCC should be designed to favor efficient code generation for
RISC, for CISC, for VLIW, or for EPIC, leaving the other choices to
have to resort to more complicated machine descriptions.

He could yell all day about his incredible abilities designing logic
boards, but, unless the issue is whether existing CPU chips actually
*work* (and therefore whether GCC is worth porting to them in the first
place), all those abilities are entirely subordinate to how a compiler
like GCC (designed to be portable to a wide variety of architectures)
should be *internally* architected.  *That* decision needs to be made
with regard to *software* design, engineering, and overall quality
and usability issues -- *not* logic-design issues.

(Note that this reasoning does not always apply.  For example, someone
consider issues of how to run a software project to, say, do a Mars
mission must *necessarily* take into account how much effort will be
needed to needed to code around g77's unstable numerics, because they're
very likely to want to use off-the-shelf and/or newly written Fortran
code in the work.  In other words, our unwillingness to provide a stable
numerical environment makes g77 a *special problem* in what *should* be
a high-level evaluation of how to undertake the project, when g77 *should*
just be an issue of choice of compiler based on the usual cost factors.
The upshot is that they might easily decide to avoid g77, and hence not
fund improvements that'd make it *better*, because they can't afford what
they believe it'll take to work around unstable numerics.  It won't matter
one whit if you show up and tell them unstable numerics are ultimately
*their* problem, not g77's -- *they already know that*, they're just
more aware of risk and tolerance issues regarding testability, usability,
graceful degradation, and standards-conformance issues than you care to
bother about.  More than likely, the mere fact that g77 exposes issues
like this to such a degree that it must be considered at such a high level
will get it dismissed out-of-hand, as is surely the case due to its not
supporting, e.g., intrinsics in PARAMETER, another thing very painful to
reliably work around.  But at least nobody -- except for, long ago, maybe
one or two people who don't run important GNU projects -- is yelling at me
for suggesting g77 should support intrinsics in PARAMETER!!)

>If the issue would just be:  Well, we
>have to support 80-bit spills because the highly successful Whiz-Bang
>6.0 Destructively Voidable Fortran compiler by Analog Equipment
>Corporation does it, it would be fine with me.

But that's what *one* issue *was*, back in December: Tim Prince pointed
out that at least one vendor compiler (the only one he tried), lf95
IIRC, *did* do 80-bit spills.  It was the only datapoint I recall we
got, and an inadequate one at that, regarding the *performance* hit
we might take (inadequate because there were too many variables).

>But I cannot seriously be asked to just give up 20 years of experience
>in numerical physics.

Nobody *ever* asked you to do that.  Instead, what you (and others who
have been making top-level GNU decisions) have been doing is *forcing*
people to ignore (in my case) 25 years of experience in making large-scale
systems actually work over the long haul, when used by *typical* users
in a *typical* environment.  (I'm not saying I'm 100% successful in doing
that.  But I've made learning from my mistakes, *especially* in these
areas, one of my highest priorities since about 25 years ago, and I'm
usually -- with perhaps the exception of g77 -- able to point to substantial
and credible evidence that I'm much better at it than my "competition",
e.g. consultants or programmers who worked in those areas in the past.
And, boy, did my experience at Cadence show me just how important having
a coworker who *strongly* understands these issues is -- my coworker
there made life *vastly* easier for me on that project, despite others
who didn't "get" my approach at all, than if he'd been a "vanilla" guy
who merely wanted to "do the job", "make it fast", and move on.  Yes,
we sweated the "make it fast" part in a *very* serious way -- turned out
we, especially my coworker, were able to solve that while enjoying the
benefits of the quality work we'd done, and especially that he'd done
before I got there, up front.  I have *nobody* like that to work with
on g77.)

The upshot is that those of us who *do* know better than to make
short-sighted decisions like these have to nevertheless live with
them being made, and, at least in *my* case, we're the ones who get
to deal with the nasty issues that result -- in that people are,
legitimately, claiming it is *g77*, "Craig Burley's compiler", that
has unstable/unpredictable numerics, not gcc, and not Toon Moene's
lack of awareness of quality-engineering issues.

Or, as so *many* of the industry's experts have been saying for *decades*,
if anybody (especially in the GNU project) is paying attention:

  "Premature optimization is the root of all evil."

  "Make it work; *then* make it fast."

  "You can have it work right, work fast, or be inexpensive -- choose *two*."

I can't recall the quotes exactly, or who originally said them, but that
*last* one is particularly telling -- we've certain made GNU *cheap*
(it's free), so every time we choose *fast*, we *necessarily* choose
to not make it work right.  The lack of sufficient resources -- especially
people willing to do wide testing using comparatively obscure (and
often poorly documented) options like -mieee, not to mention non-existing
options like the IA32 ones that provide 80-bit spills or that enforce
strict-to-type evaluation -- assures this.

While I'm sorry to say the above is an accurate statement of the state
of g77, and probably many other GNU projects as well, I'm happy to finally
have reached the point of saying "My work on g77 is done, I want to
work only on *quality-first* software from now on", and to have a clear
historical document (in the form of these email archives) to point to when
people ask why I didn't try and "reform from within".

And, I now see more clearly than ever that, based on your own descriptions
of what constitutes a quality implementation, g77 *already* perfectly meets
the needs of the only audience you seem to care about...so no rewrite,
or new features, is needed.  (Most importantly, I now see clearly something
I've been concerned about since I started architecting the new front end:
the rewrite *I* have in mind will *not* be tolerated by people who don't
care about, or agree with, my views regarding these issues, since it
is *hugely* influenced by them.  The result would be, if I tried to do the
rewrite, people constantly arguing with me trying to convince me to
change the architecture, the design, and/or the implementation, rather
than spending their time, and mine, testing it.)

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>craig@jcb-sc.com wrote:
>
>> >The evidence on these lists prove that people are still forced to write
>> >programs without getting a decent course in numerical analysis.
>
>> Again, as I pointed out before, *I* have taken no "decent course
>> in numerical analysis".
>
>Nope, and it is not necessary for you to do so.
>
>[ Unless you want to take part in discussions like this, otherwise, 
>  you should just perform useful work on g77: ]

*My* original point was that choosing -mieee as a default would have
resulted in gcc/g77 support for the Alpha being, *today*, much further
along in terms of support for at least the industry-standard model of
floating-point.

Instead, we have inadequate, and probably overly buggy, support for -mieee
(no libraries compiled that way out of the box, and we sure aren't doing
enough testing of code with -mieee in effect to have much confidence that
simply changing the default wouldn't lead to all sorts of regressions,
ICEs, and so on), and buggy support (to say the least) for the default,
-mno-ieee.

In the meantime, you haven't answered my question as to why I should
be working on compilers, having taken no decent (well, no university-level)
course on compilers and having not read, well, I can't think of *one*
text on compiler design!

>I tried to express that to write useful numerical code one has to have
>knowledge of numerical analysis.

Fine.  Is it really your contention that not a single person who has
*ever* learned numerical analysis would use a wider range of IEEE 754
than you claim is all that needs to be properly supported?  (A range
you have yet to actually *specify*, I believe.)

>It is *not* sufficient to paper over mistakes made by people who didn't
>read their stuff just to get "the right answer".

So you consider implementing IEEE 754 properly to be "papering over
mistakes made by people who didn't read their stuff"??

>What a Fortran compiler should do is (IMHO) compile the source to the
>object code, using the latitude given by the Standard (i.e. SQRT: the
>result has a value equal to a processor dependent approximation to the
>square root of the argument).

In other words, it's okay if SQRT(16.0) => -1.5.  (Yes, folks, it's
true: the Fortran standard does *not* disallow that result.  I'm not
kidding.  Why Toon is suddenly resorting to the Fortran standard to
define what valid numerical behavior is, I don't know; frankly, in
comp.lang.fortran circles, it'd be a sign of incipient insanity, or
at least a indication that the person realizes he's losing the
argument.  But maybe that's no longer true; I gave up USENET about
six months ago.)

Sorry, Toon, but I've been working on Fortran compilers, and reading
comp.lang.fortran, long enough to have more of a clue than *that*.

Further, you have essentially argued that g77 need not even return *a*
value equal to that approximation, but may return *multiple* values equal
to that approximation, despite the very language you quoted.

That's what spilling 80-bit registers assigned to variables to 64-bit
or 32-bit temporaries *does* -- returns an 80-bit result and a 32-bit
result (say), allowing the processor to randomly choose which to use at
any given time, even when the result is assigned directly to a variable,
even if that variable (as assigned) is read only *once* in the user's
source code.

Getting back to the quality-of-implementation issue: I have always said
that it's ultimately okay for a Fortran processor to start WW3 if the
inputs don't conform, according to the standard.

But I *strongly* disagree with the idea that a *quality* implementation
starts WW3 only if the input happens to be between, say, 1E-60 and 1E-30,
or whatever the range of IEEE 754 single-precision denormals happens
to be.

That sort of implementation makes basic, solid, engineering practices like
testing and validation *much* less useful, because the behavior of
the underlying system changes wildly in the presence of out-of-range
values -- never mind whether the *code* running on top of it has any
actual *bugs* in it.

>If people want to get tighter control, there are two ways:
>
>1. Try to influence X3J3 to impose them.
>
>2. Use another language.

In which case, you won't mind one whit if -mieee and 80-bit spills are
made the defaults, since X3J3 doesn't impose *performance* constraints?

Well, it's about *time* you conceded that.

;-/

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Stephen L Moshier]

On Tue, 13 Jul 1999, Sylvain Pion wrote:
> Then just stop developping egcs, because all the features we need are
> already implemented...

Just so.  GCC is not a new program.  It is an old program.  Consider
that IEEE has been in effect for some 14 years.  During all those
years there have been any number of hot-shots, gurus, and hangers-on
who could have implemented the features.  They have not done so.
People have their priorities.


> The related features I miss are:
> - ability to stop constant propagation when you need the evaluation to be done
>   at runtime because of different rounding/precision.

  C9X treats this question, so you will probably get your wish.


> - an equivalent to -fforce-mem for intermediate values, where needed (after
>   arithmetic operations on ia32).

  If you are talking about the fix for extra-precise registers, I think it
may have been dropped from C9X.  It has already been done, however,
at Next.  The Next work would provide at least a focus of discussion,
for those wanting to work on the problem.

  -fforce-mem, at double precision, would tend to violate IEEE by creating
double-rounding errors.

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Joern Rennecke wrote:

> No, we may compute intermediate values in a higher precision, as long as
> we make sure that a value that is assigned to a variable remains consistently
> the same value when it is used more than once.

For referential transparency this is sufficient.  However, if gcc
supports IEEE conformance for backends A, B, and C, and only A will
generate higher precision intermediate results, that would mean that an
IEEE conformant program could (potentially) generate different results
on A from those on B and C.

This is the same question as whether you can use an multiply-add
instruction that doesn't correctly round down the intermediate result.

This doesn't sound acceptable to me (from an IEEE-conformance
standpoint).

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> This is a good point.  The reason I keep harping on the range question
> is that I *hope* that documenting it makes it possible for people to
> work around it.

It's pretty similar to trapping on denormalized values.  Except that
you can silently get incorrect results instead.

> If such a mode of working is not possible, we have no choice but to
> revert to storing *all* intermediate values.

No, we may compute intermediate values in a higher precision, as long as
we make sure that a value that is assigned to a variable remains consistently
the same value when it is used more than once.

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Joern Rennecke wrote:

> > > And in any case, rounding _does_ affect every operation.
> > > Just that in 64-bit precision mode, the rounding happens
> > > for the XFmode value of the register, rather than for the
> > > SFmode value of a variable.

> > As long as it's the same as that what happens when storing the SFmode
> > value in memory, we're safe (well, OK, modulo the range question).

> Not addressing the exponent range question means that the code still
> remains incorrect, only that you see the failures for a smaller range
> of values.  I.e. it remains buggy, but becomes harder to debug.

This is a good point.  The reason I keep harping on the range question
is that I *hope* that documenting it makes it possible for people to
work around it.

If such a mode of working is not possible, we have no choice but to
revert to storing *all* intermediate values.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> > And in any case, rounding _does_ affect every operation.
> > Just that in 64-bit precision mode, the rounding happens
> > for the XFmode value of the register, rather than for the
> > SFmode value of a variable.
> 
> As long as it's the same as that what happens when storing the SFmode
> value in memory, we're safe (well, OK, modulo the range question).

Not addressing the exponent range question means that the code still
remains incorrect, only that you see the failures for a smaller range
of values.  I.e. it remains buggy, but becomes harder to debug.

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:

> On Mon, Jul 12, 1999 at 06:04:04PM -0600, Donn Terry wrote:

> > Rounding mode only affects STORE operations.

> Toon was imprecise.  We were not speaking of rounding
> (bits 10:11 of fpcr) but rather precision (bits 8:9 of fpcr).

Not surprising - I'm operating without the document defining ia32
floating point arithmetic.  Richard, do you have an URL to it ?  I
couldn't find it on Intel's Web site.  Oops, I just realized - it's
probably in our Readings section ...

PS: Rounding mode is something different in IEEE-754 then what I was
referring to.  It doesn't tell at what bit to round, but which method of
rounding you want:  round to nearest, round to zero, round up or round
down.

> And in any case, rounding _does_ affect every operation.
> Just that in 64-bit precision mode, the rounding happens
> for the XFmode value of the register, rather than for the
> SFmode value of a variable.

As long as it's the same as that what happens when storing the SFmode
value in memory, we're safe (well, OK, modulo the range question).

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Stephen L Moshier wrote:

> > Note that we are talking about correctly implementing IEEE
> > conformance.  The fact that I personally do not need it is orthogonal
> > to that issue.  (but I'm fully to blame for unduly pushing my own
> > needs, for which I apologize).

> But that is what you should do!  If there were an important need for
> "correctly implementing IEEE conformance" then someone would have put
> it in long ago.  We implement the features that we need, and we ignore
> the ones we don't.

If I were you, that would be true.

However (I almost wrote:  Unfortunately ...), I am a member of the
Steering Committee.  As Craig correctly pointed out, that means a
broader responsibility:  I can not just go for my own preferences, but
should create an atmosphere where contributions from both "sides" of the
discussion on floating point arithmetic are felt to be welcomed.

I clearly failed in this - hence my apology.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Sylvain Pion]

On Tue, Jul 13, 1999 at 08:54:46AM -0400, Stephen L Moshier wrote:
> But that is what you should do!  If there were an important need for
> "correctly implementing IEEE conformance" then someone would have put
> it in long ago.  We implement the features that we need, and we ignore
> the ones we don't.

Then just stop developping egcs, because all the features we need are
already implemented...

I personnaly would like to have better (correct and fast) IEEE support from
GCC.  I have even started to code some small optimizations specifically for
my IEEE dependent code, but I'm not a compiler guru, so...

The related features I miss are:
- ability to stop constant propagation when you need the evaluation to be done
  at runtime because of different rounding/precision.
- an equivalent to -fforce-mem for intermediate values, where needed (after
  arithmetic operations on ia32).

Of course I can put the workarounds (volatile) in my own code, but the
compiler has more informations, and could perhaps optimize better if it
was aware of the real problem.

-- 
Sylvain

Re: Bug with g77 and -mieee on Alpha Linux

[Stephen L Moshier]

> Note that we are talking about correctly implementing IEEE
> conformance.  The fact that I personally do not need it is orthogonal
> to that issue.  (but I'm fully to blame for unduly pushing my own
> needs, for which I apologize).

But that is what you should do!  If there were an important need for
"correctly implementing IEEE conformance" then someone would have put
it in long ago.  We implement the features that we need, and we ignore
the ones we don't.

Re: Bug with g77 and -mieee on Alpha Linux

[Stephen L Moshier]

Due to a charming DEC assembler bug, the mieee switches did  not
work well enough to be usable on alpha-osf up to osf-4.0 or so.
It would be a disaster to make ieee the default on those systems.

If you have a working assembler now, the ieee modes should be tested.
At the time, it was so hard to make anything at all run that it
would be  a miracle if it really worked.

Sat Mar 16 16:20:43 1996  David Mosberger-Tang  <davidm@azstarnet.com>

    * alpha.md (trap): New attribute.
    Modify patterns for all floating-point trap generating
    instructions.
    * alpha.h (CPP_SPEC): Added -mieee and -mieee-with-inexact.
    (alpha_trap_precision, alpha_fp_rounding_mode, alpha_fp_trap_mode):
    New enum types.

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 7:15:09 PM PST, law@cygnus.com writes:

> We'd be interested in a testcase for this.  To the best of my knowledge we
>  should be aligning spill slots relative to the stack frame.
Yes, I believe that individual compilations align spill slots reliably, but 
it does no good when the OS is one which doesn't believe in stack alignment 
(NT etc.).  The Fortran compilers I've seen for Windoze require all data to 
be put in COMMON if 64- or 128-bit alignments are required, including g77.  
The Livermore Fortran Kernel at ftp://members.aol.com/n8tm/lloops.shar.gz 
(lfk537g.f version) shows this behavior; Kernels 8 and 9 have spills of loop 
invariant data which are aligned in static storage.  Under i686-pc-linux-gnu 
the spills are aligned, but in i586-pc-cygwin it's a matter of luck.  Only 
those 2 kernels with 64-bit spills perform significantly different between 
linux and Windoze.  The speed difference is nearly a factor of 2 in those 
kernels in the PPro, even though most of the data are aligned in COMMON.  
With the 32-bit spills which g77 now uses for single precision, there is no 
performance difference between Windows and linux.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < 378A8274.D7309E6A@verinet.com >you write:
  > I pushed on this hard enough that I hacked up gdb to print out
  > the x86 FP registers in both hex and float decimal, along with intepreted
  > values of the control registers and a TOS marker.  It helped a lot.
  > (This is for an older gdb, and I haven't converted it to a patch, but
  > I'll be happy to ship the file to someone to convert it to a newer
  > gdb; the changes should be easily portable.)
I would strongly recommend you get this stuff to the gdb folks.  They're
in the process of trying to beef up gdb's handling of the floating point
unit.  This is precisely the kind of thing that it would be great to have
in gdb.

jeff

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < b9c6a72c.24bbede7@aol.com >you write:
  > I'm still seeing occasional 50% performance loss due to mis-aligned 64-bit 
  > spills (on targets where the stack is only 32-bit aligned).  So I would 
  > request again that this project not go ahead without solving this alignment
We'd be interested in a testcase for this.  To the best of my knowledge we
should be aligning spill slots relative to the stack frame.

jeff

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 5:28:38 PM PST, amylaar@cygnus.co.uk writes:

> Assuming you use gcc to compile your entire program, we can solve the
>  aligment problem by aligning the stack in main.
Well, yes, if you still think those other OS's are about to die.  But I 
thought I saw Jeff Law recommend against -malign-double the other day, 
without making an exception for any OS.  Now, I personally don't care whether 
g++ ever supports alignment, so I don't intend to start an argument on that. 
I'd hate to see g77 forced into a situation where it depends on alignment for 
adequate single precision performance, with little chance that Windoze will 
support that.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> In a message dated 7/12/99 12:20:06 PM PST, amylaar@cygnus.co.uk writes:
> 
> > What are we going to do about alignment?
> >  Make keeping 128-bit-alignment the default, and hope that soon crt0.o
> >  and libraries will provide / keep this alignment too on platforms where
> >  we care about performance?
> >  
> >  
> >  Will this design also allow to use the alignment bytes of the 
> 10-byte-values
> >  to put smaller spills there?
> 
> 
> I'm still seeing occasional 50% performance loss due to mis-aligned 64-bit 
> spills (on targets where the stack is only 32-bit aligned).  So I would 
> request again that this project not go ahead without solving this alignment 
> problem, and taking Craig up on the suggestion that it may be possible for 
> the compiler to avoid widening the spills where the data have already been 
> rounded to 32 or 64 bits.

That does not work quite right, since the 16-bit exponent means that
values that ought to be represented as denormals or underflow to zero
are still represented with 53 significant bits.

Assuming you use gcc to compile your entire program, we can solve the
aligment problem by aligning the stack in main.

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/12/99 12:20:06 PM PST, amylaar@cygnus.co.uk writes:

> What are we going to do about alignment?
>  Make keeping 128-bit-alignment the default, and hope that soon crt0.o
>  and libraries will provide / keep this alignment too on platforms where
>  we care about performance?
>  
>  
>  Will this design also allow to use the alignment bytes of the 
10-byte-values
>  to put smaller spills there?


I'm still seeing occasional 50% performance loss due to mis-aligned 64-bit 
spills (on targets where the stack is only 32-bit aligned).  So I would 
request again that this project not go ahead without solving this alignment 
problem, and taking Craig up on the suggestion that it may be possible for 
the compiler to avoid widening the spills where the data have already been 
rounded to 32 or 64 bits.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 06:04:04PM -0600, Donn Terry wrote:
> Rounding mode only affects STORE operations.

Toon was imprecise.  We were not speaking of rounding
(bits 10:11 of fpcr) but rather precision (bits 8:9 of fpcr).

And in any case, rounding _does_ affect every operation.
Just that in 64-bit precision mode, the rounding happens
for the XFmode value of the register, rather than for the
SFmode value of a variable.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < 378A7B90.7DCF39C9@moene.indiv.nluug.nl >you write:
  > > Cygnus wrote a pass based on the lcm routines to find computationally
  > > optimal locations for FP mode switching on the sh.  That same code can be
  > > used to do mode switching for any system which needed it, including stuff
  > > like mmx/fp mode swtiching on ia32.
  > 
  > Well, if this can be done reliably, I am convinced that support for IEEE
  > conformance (minus the range question, which should be documented quite
  > clearly) is possible.
It can be done reliably.  The only major issue with that code as it stands is
it introduces another hook like MACHINE_DEPENDENT_REORG.  We don't want another
hook like that one.  It's the most abused hook in the compiler.

The bulk of the code is designed to be a generic optimization of mode switching
(the precise nature of the mode switch isn't particularly important).

jeff

Re: Bug with g77 and -mieee on Alpha Linux

[Donn Terry]

> 2. [This one is from hearsay, on comp.arch]  One can only set the
>    rounding mode on the ix86, not the range.  That means that the
>    register values still will not have the same non-finite behaviour
>    as stored values.

Based on my imperfect understanding of what you're saying, and on
my potentially imperfect undrstanding of the FP unit, I may be saying
something you already know, but also I may be adding something,
so I'll take the chance...  (I also came into this late.)

Rounding mode only affects STORE operations.
Thus, setting the rounding mode to 53 bits has no effect on the content
of a register (even after a subsequent arithmetic operation).  The only
way I was able to find that caused a round down from 64-bit to 53 bit
mantissa was a store-load sequence.  (One would hope that there's
a way to fix this in the newer versions, but this was a PII architecture
book I was looking at.)

I spent a lot of time looking at the FP registers very carefully, and
couldn't make them round to the right precision any way *I* could
find that didn't involve memory.   Thus, I believe the problem is worse than
just spills; computations exist which, depending on how many registers the
compiler chooses
to use (and this can be expression order sensitive, of course), the
results could differ (potentially substantially) depending on optimization
level and exact code generation strategies.  (Similar differences would
not be visible on a pure 64bit IEEE machine such as most of the RISCs.)

I pushed on this hard enough that I hacked up gdb to print out
the x86 FP registers in both hex and float decimal, along with intepreted
values of the control registers and a TOS marker.  It helped a lot.
(This is for an older gdb, and I haven't converted it to a patch, but
I'll be happy to ship the file to someone to convert it to a newer
gdb; the changes should be easily portable.)

Donn

P.S.  Thinking about it, there may be a way to get that rounding by
extracting exponents and stuffing mantissas, but I don't really want to
think about that, and it's not clear that it would be any faster than a
store-load, allowing for cache.

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Jeffrey A Law wrote:
> 
>   In message < 378A77AA.A234987F@moene.indiv.nluug.nl >you write:
>   > OK, but the point is you have to set it to *one* value for the complete
>   > compilation (unless you want to call this routine after every
>   > calculation to set the default right for the next one).

> Some targets already do this (sh for example).  Is it painful?  Yes.  But
> is the cost coming down?  Yes.

Note that to the IEEE fans cost (as expressed in slowness of
computation) is irrelevant.  We should play this by their rules.

> Cygnus wrote a pass based on the lcm routines to find computationally
> optimal locations for FP mode switching on the sh.  That same code can be
> used to do mode switching for any system which needed it, including stuff
> like mmx/fp mode swtiching on ia32.

Well, if this can be done reliably, I am convinced that support for IEEE
conformance (minus the range question, which should be documented quite
clearly) is possible.

Thanks for your input (Richard and Jeff),

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Jeffrey A Law]

  In message < 378A77AA.A234987F@moene.indiv.nluug.nl >you write:
  > OK, but the point is you have to set it to *one* value for the complete
  > compilation (unless you want to call this routine after every
  > calculation to set the default right for the next one).
Some targets already do this (sh for example).  Is it painful?  Yes.  But
is the cost coming down?  Yes.

Cygnus wrote a pass based on the lcm routines to find computationally 
optimal locations for FP mode switching on the sh.  That same code can be
used to do mode switching for any system which needed it, including stuff
like mmx/fp mode swtiching on ia32.


jeff

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:
> 
> On Mon, Jul 12, 1999 at 11:57:55PM +0200, Toon Moene wrote:
> > I think it's *either* -mieee *or* 80-bit spills.

> Eh?  One is proposed for Alpha, the other for IA-32.

Yes, but it still might not be totally transparent to everyone that *on
the ia32* you have to specify *either* -mieee *or* 80-bit spills.

> > This still has two limitations w.r.t. to true IEEE conformance:
> >
> > 1. It only works correctly for doubles, because floats still get
> >    extended to 64 bits and are therefore not equal to their stored
> >    value.
> 
> No, you can set the rounding mode to 24, 53, or 64 bits.

OK, but the point is you have to set it to *one* value for the complete
compilation (unless you want to call this routine after every
calculation to set the default right for the next one).

> > 2. [This one is from hearsay, on comp.arch]  One can only set the
> >    rounding mode on the ix86, not the range.  That means that the
> >    register values still will not have the same non-finite behaviour
> >    as stored values.
> 
> True.
> 
> r~

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 11:57:55PM +0200, Toon Moene wrote:
> I think it's *either* -mieee *or* 80-bit spills.

Eh?  One is proposed for Alpha, the other for IA-32.

> This still has two limitations w.r.t. to true IEEE conformance:
> 
> 1. It only works correctly for doubles, because floats still get
>    extended to 64 bits and are therefore not equal to their stored
>    value.

No, you can set the rounding mode to 24, 53, or 64 bits.

> 2. [This one is from hearsay, on comp.arch]  One can only set the
>    rounding mode on the ix86, not the range.  That means that the
>    register values still will not have the same non-finite behaviour
>    as stored values.

True.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:
> 
> On Mon, Jul 12, 1999 at 07:01:10AM -0000, craig@jcb-sc.com wrote:
> >    Let's calculate accuracy well out of proportion with what anybody
> >    actually needs. They don't need it, but it doesn't matter. We'll do it
> >    that way and then there won't be any argument.
> >
> > Imagine that.  If g77/gcc delivered to -mieee and 80-bit spills from
> > the outset, we'd have avoided all *sorts* of arguments...
> 
> I spent a good bit of the afternoon reading through several of Kahan's
> papers.  Despite my earlier position, I'm mostly convinced we should do
> both -mieee and 80-bit spills by default, despite what other compilers
> for the systems in question do by default.

I think it's *either* -mieee *or* 80-bit spills.

Let me explain why.

If we turn to the example given by Kahan in the interview in Doctor
Dobbs Journal ( http://www.ddj.com/articles/1997/9711/9711a/9711a.htm -
thanks for the link, Andy), we see the following:

<QUOTE>
float x, y, z;

int j;
...
x = y + z; 
if (x >= j) replace (x);
y = x;

...

As far as we can tell, when he turns on optimization,
the value of x is computed in a register
with extra width. This happens on the Intel machine because the
registers have extra width.
It also happens on some others. The value of x that he computes is not,
in fact, a float. It's in a register that's wider.
The machine stores x somewhere, and in the course of doing that,
converts it to a float. But the value used in the comparison is the 
value in the register. That x is wider. The condition that the register 
x be greater than or equal to j doesn't guarantee that x, when stored
in y, will be less than j.
Sometimes y=j, and that should never be. My student counted on what 
compiler writers call "referential transparency" and was disappointed
because the compiler writer saved some time in optimization by not 
reloading the register from the stored value.
</QUOTE>

Note the remark: "The condition that the register x be greater than or
equal to j doesn't guarantee that x, when stored in y, will be less than
j."

Now if 80-bit spills were in effect, the code *as shown* might work in
gcc, because x, y, and z are all in registers.  However, Kahan
particularly remarks that x was stored in y.  If y is a memory location,
you'll get the effect that he and his student observed.

You will get this effect when y is not a simple float, but a float in a
(sufficiently large) struct, which will ensure that `y = x;' _will_ mean
a store in memory.

The error *this particular compiler* made was that the comparison x>=j
should have been performed with the *same* floating point value as was
later stored in y, hence a 32-bit value.

Because gcc almost certainly will not be going to do this by mere chance
if we enable 80-bit spills (and almost no compiler will do this, because
it is very hard to determine what values to store in memory
intermediately), there is only one way to *approach* IEEE-754
conformance with gcc on the ix86 target:  Use a OS-dependent routine to
set the rounding mode to 64 bits.  If you do that, there's no reason to
perform 80-bit spills, although you need to spill floats as doubles.

This still has two limitations w.r.t. to true IEEE conformance:

1. It only works correctly for doubles, because floats still get
   extended to 64 bits and are therefore not equal to their stored
   value.

2. [This one is from hearsay, on comp.arch]  One can only set the
   rounding mode on the ix86, not the range.  That means that the
   register values still will not have the same non-finite behaviour
   as stored values.

HTH,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 09:17:04PM +0100, Joern Rennecke wrote:
> Make keeping 128-bit-alignment the default, and hope that soon crt0.o
> and libraries will provide / keep this alignment too on platforms where
> we care about performance?

Yes.

> > The spill widening can actually be done in such a way that it helps
> > alpha ev4 wrt spilling QImode.  So I want that capability for other
> > reasons.
> 
> Will this design also allow to use the alignment bytes of the 10-byte-values
> to put smaller spills there?

I think that's rather orthogonal.  The biggest part of the reload work
for the target to be able to request that QImode pseudos be spilled in
SImode.  Or SFmode pseudos be spilled in XFmode.

The fact that XFmode slots leave 6 bytes of extra alignment storage to
be use for other spills is simply another problem.  Not that it wouldn't
be good to do that too.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Joern Rennecke]

> I spent a good bit of the afternoon reading through several of Kahan's
> papers.  Despite my earlier position, I'm mostly convinced we should do
> both -mieee and 80-bit spills by default, despite what other compilers
> for the systems in question do by default.

What are we going to do about alignment?
Make keeping 128-bit-alignment the default, and hope that soon crt0.o
and libraries will provide / keep this alignment too on platforms where
we care about performance?

> The spill widening can actually be done in such a way that it helps
> alpha ev4 wrt spilling QImode.  So I want that capability for other
> reasons.

Will this design also allow to use the alignment bytes of the 10-byte-values
to put smaller spills there?

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Mon, Jul 12, 1999 at 07:01:10AM -0000, craig@jcb-sc.com wrote:
>    Let's calculate accuracy well out of proportion with what anybody
>    actually needs. They don't need it, but it doesn't matter. We'll do it
>    that way and then there won't be any argument.
> 
> Imagine that.  If g77/gcc delivered to -mieee and 80-bit spills from
> the outset, we'd have avoided all *sorts* of arguments...

I spent a good bit of the afternoon reading through several of Kahan's
papers.  Despite my earlier position, I'm mostly convinced we should do
both -mieee and 80-bit spills by default, despite what other compilers
for the systems in question do by default.

It's the sort of change that would require Large Announcements, so that
someone rebuilding their favourite mpeg widget doesn't scream.  Or at
least not as loudly.

The quality of code generated by -mieee definitely needs attention.
Even given the constraints it must follow, the generated code is abysmal.

The spill widening can actually be done in such a way that it helps
alpha ev4 wrt spilling QImode.  So I want that capability for other
reasons.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>Someone else had a pointer to Prof. Kahan's home page (he is one of the
>designers of IEEE-754 arithmetic).  There is an another article there,
>"How Java's Floating-Point Hurts Everyone Everywhere",
> http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf , which talks much more
>about the pitfalls of floating point than simply bashing Java.  Kahan and
>his co-author were trying to convince Sun to abandon some bad decisions
>they'd made, and ended up writing one of the best papers on numerical
>analysis that I've ever seen.
>
>People writing numerical programs (or fortran compilers) should read this
>paper. 

A pity Kahan doesn't value standards conformance sufficiently to put
it up as a Web page (.html -- I tried it, "not found").  That means I
can't read it, nor can a friend of mine who is blind, among other things.

;-/

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>DDJ also has a neat interview with Kahan at
> http://www.ddj.com/articles/1997/9711/9711a/9711a.htm

This, on the other hand, already has a priceless quote from WK that
pertains to the discussion at hand:

   Let's calculate accuracy well out of proportion with what anybody
   actually needs. They don't need it, but it doesn't matter. We'll do it
   that way and then there won't be any argument.

Imagine that.  If g77/gcc delivered to -mieee and 80-bit spills from
the outset, we'd have avoided all *sorts* of arguments, other than
discussions as to how to improve performance (and offer options to
users who insist they don't need certain assurances, like we already
do via -ffast-math).

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Andy Vaught]

Someone else had a pointer to Prof. Kahan's home page (he is one of the
designers of IEEE-754 arithmetic).  There is an another article there,
"How Java's Floating-Point Hurts Everyone Everywhere",
http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf , which talks much more
about the pitfalls of floating point than simply bashing Java.  Kahan and
his co-author were trying to convince Sun to abandon some bad decisions
they'd made, and ended up writing one of the best papers on numerical
analysis that I've ever seen.

People writing numerical programs (or fortran compilers) should read this
paper. 

DDJ also has a neat interview with Kahan at
http://www.ddj.com/articles/1997/9711/9711a/9711a.htm

        Andy

-----------------                        XOLD(K,IC,I)=
Andy Vaught               ....        DO ITERS=1, 10  XOLD(K,IC,I)
andy@maxwell.la.asu.edu   |  |   /CALLMSOLVE(A,B,X,I,ITERS,TOL)+(RANNYU(0)
Arizona State University  ======|WRITE(6,'(I5,2X,F12.6)')ITERS,TOL -HALF)
Tempe, Arizona USA        OOOOOO \ENDDORETURN PARAMETER(ZERO=1.D0)*TENTH*DELTA

Re: Bug with g77 and -mieee on Alpha Linux

[Dave Love]

>>>>> "Tim" == N8TM  <N8TM@aol.com> writes:

 Tim> I'd be happy to resurrect paranoia or some similar code if you
 Tim> can specify it, in whichever version you like, and use that as
 Tim> the standard of whether g77 is dealing adequately with these
 Tim> issues.  It's been quite a few versions since I've run it, 

I've run it a bit recently.  As reported on egcs-bugs, it's fine with
-mieee on current OSF1.  In default mode, there's one flaw, as with
the DEC compiler.

[While I'm on, it looks as though I don't actually agree with what
Toon says, contrary to what I said on egcs-bugs, but I haven't time to
plough through all this stuff right now and I'm not really sure what
his position is.  There are clearly uses for subnormals
(e.g. apparently in complex division) and I have done barrier
penetration calculations involving arbitrarily small dimensionless
numbers, not that that's a big deal or really the same issue as
numerical analysis robustness.  I did see things fall over in minor
ways when IEEE machines appeared.]

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>The point I really wanted to make is that in current, Standard
>Fortran, there is no way to deal with the fact that on some hardware,
>floating point numbers in registers have different precision (and range)
>then when stored in memory.  Therefore, IMHO, the "most" correct way of
>dealing with, say, DOUBLE PRECISION calculations is to store *all*
>intermediate values of computations.  Obviously, this would be very
>costly - so we settle for the less "correct" solution of ignoring all
>issues with intermediate extra precision.  Not having 80-bit spills is
>just one of the ways in which we are ignoring these issues.  We could
>spill floating point registers in 80 (or rather 128) bit chunks, but
>that would still not solve this problem completely.

Yes, well, standard Fortran doesn't imply much of anything useful about
precision and range.

So it is indeed permissible to use 80-bit precision for one computation,
64-bit for another, 32-bit for another, and 3-bit for yet another,
even if all computations are DOUBLE PRECISION in the code!

However, having *computed* an approximation of something (like the
result of evaluating an expression), that *one* approximation is the
only one available from that computation and/or the variable in which
it is stored, by my reading of the standard.

So while not forcing all DOUBLE PRECISION computations to 64 instead of
80 bits, we violate a principle that users accustomed to strict IEEE 754
(or even classic strict FP) machines obeyed, which is "never compute
using more precision that declared".

That's (considered) okay, because the result is much faster performance
on some hardware, and, besides, the Fortran standard does not disallow it.
(Further, it's fairly easy to code around, assuming the *next* principle
is obeyed.)

However, having computed *an* approximation to 80 bits, exposed it in
one code-visible way, then spilled it to 64 bits and afterwards used that
*distinct*, separate approximation in *another* code-visible way, we
violate the vastly more encompassing principle that a given computation
produces exactly *one* (code-visible) result.  (The result may indeed
represent an approximation of a value within some interval -- but, like
quantum mechanics, once *observed*, in the sense of being visible to the
code in any way, that observation is *fixed*, especially by assigning
it to a variable.  That variable cannot, by this principle, subsequently
appear to have *multiple* values.)

Now, this principle I *believe* to be mandated by the Fortran standard.

But, even if it isn't, it is a principle a vastly larger audience accepts
as a "given" than the one that accepts the former principle concerning
never using excess precision at all.

Further, the performance penalty of holding to the latter principle is,
AFAIK, never higher, and sometimes vastly *lower*, on all machines than
the penalty of holding to the former principle.

Now, this is all "water under the bridge", of course.

But it illustrates an aspect of what I have been talking about all along.

The attitudes, among so many who make design decisions affecting the
rest of us, that "nobody should care about low-level precision issues",
"speed is always more important than numeric stability", and so on,
make even mere *verification* that a product is doing the right things
vastly more difficult.  (Further, and this is my greater concern about
the 80-bit-spill issue, these attitudes do *not* defend against what
I believe *will* be future desired optimizations that will make code
*supposedly* resistant to truncated spills start failing.  At which point
those who *like* truncating spills will split into camps -- those who stick
to the "party line" that the *code* must be wrong, and various others who
claim that, to various degrees, the *compiler* is wrong to have made *that*
"extreme" a transformation to the code.)

In particular, those attitudes say, to *both* principles (of expectations
users have regarding FP behavior), "we don't care one whit about you,
we'll break you anytime if it gets us a few more percentage points of
performance on the codes *we* care about".

And now we're paying a very significant penalty for those attitudes.

Which cannot, and *will* (likely) not, be changed merely by changing
any of the particular short-sighted decisions under discussion (not
doing 80-bit spills, not making -mieee the default, to *summarize* --
see my earlier emails for more explanation).

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >I do not personally have a *performance* issue with 80-bit spills,
> >because I do not view ia32 type machines as being particularly
> >"performant".

> IMO that statement *seriously* contradicts your statements regarding
> my 80-bit-spill proposal back in December.  In fact, I don't recall
> any reason *other* than performance that you (and others, Dave Love and
> Richard Henderson, IIRC) offered to explain your outright rejection of
> my proposal.

This is probably my fault - I should have separated the two issues more
clearly.  The point I really wanted to make is that in current, Standard
Fortran, there is no way to deal with the fact that on some hardware,
floating point numbers in registers have different precision (and range)
then when stored in memory.  Therefore, IMHO, the "most" correct way of
dealing with, say, DOUBLE PRECISION calculations is to store *all*
intermediate values of computations.  Obviously, this would be very
costly - so we settle for the less "correct" solution of ignoring all
issues with intermediate extra precision.  Not having 80-bit spills is
just one of the ways in which we are ignoring these issues.  We could
spill floating point registers in 80 (or rather 128) bit chunks, but
that would still not solve this problem completely.

> >If we (collectively) decide that -mieee should be the default on
> >Alpha's, I know how to specify -mno-ieee in my makefiles :-)
> 
> As with 80-bit spills, that decision has been made, and I consider
> it irrevocable within the GCC project at this point when it comes to
> evaluating my future involvement.  Even if that decision was changed, which
> I still think it might be, someday, it has taken *far* too much effort
> to get people (who *should* be more competent at these sorts of decisions,
> especially given the roles the play in the project) to see the light.
> 
> >What really bothers me is that this discussion is denying 40 years of
> >research in numerical analysis.
> 
> No, it isn't, and what really bothers *me* is people like you chiming
> in on discussions that pertain to software quality and usability issues
> (which you appear to have little understanding of), claiming they're
> *really* about numerics, and then turning around and telling *me* I
> shouldn't be participating in the discussions because I'm not an expert
> in numerics!
> 
> Here's the pertinent part of my *original* post on the topic, which is
> what *you* started responding to:

  ...

> Notice I said *nothing* about *numerical analysis* issues, I was referring
> *entirely* to all the special tweaks and contortions *users* must go through
> to get the IEEE-standard-conforming behavior *they* want, and have it
> actually *work*, for *them*.

Yes, my apologies - I was beating a straw man (this goes for the rest of
your e-mail, too).

Sorry,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Edward Jason Riedy wrote:
> 
> And Toon Moene writes:

>  - Sure, when run on different architectures, they will give different
>  - results.  The crucial question, however, is:  Are these differences
>  - interesting ?
> 
> Yup.  When you're looking for the smallest eigenvalues, and they're
> clustered together, the differences are _very_ interesting.  This
> is linked to the waterpipe remark: earthquake simulations.  The
> smallest eigenvalues are the ones of interest (not recalling why
> at the moment, too tired).  While _your_ code may not care about
> the nth place, these codes do.  Trying to rely on data that's
> only as good as the input data deserves (which is a favorite
> phrase of Dr. Kahan's) can be very difficult when the best bounds
> on what the input deserves are still dodgy.

Well, if you have an error analysis that supports that (and I am sure
you have) then I am all with you.

>  - Of course there are differences - but they're all meteorologically
>  - irrelevant.
> 
> That's nice.  You're right, you shouldn't be relying on your own
> problem domain.  There are others where it very much matters.

Touche'.

>  - Heck - before the 60's, [...]
> 
> Before the 60's, the idea of solving systems of equations with more
> than 6 unknowns was silly.  The fact that von Neumann's error
> analysis was wrong was not widely known, according to Dr. Parlett.
> And the SVD didn't even exist (iirc).

Yep, after I wrote that (and reread it) it dawned on me that you can
reverse the remark I made:  Getting better defined floating point
arithmetic permits different algorithms to be used.

> [on using extended precision]
>  - But how do you reliably code that ?  Going to assembler ?  There is no
>  - way to Standardly specify IEEE-754 extended precision arithmetic, yet.
> 
> Who said anything about using IEEE-754 extended precision in hardware?
> The reference implementation of the XBLAS (not my project, but one
> of my advisor's) is using a software implementation.  That system
> uses IEEE double (and won't work on x86s without forcing the CPU to
> round to double).  It's also in C, for various reasons.

Sorry, I was jumping to conclusions, here - and down here:

>  - > Yes, it could conceivably be possible to code around
>  - > a lack of IEEE-754.  The algorithms involved are complicated enough,
>  - > thank you.
>  -
>  - Well, OK; that's a choice you want to make.  I think it's rather obvious
>  - that I don't like to live that dangerously.
> 
> Pick your choices:
> 
>         * Live dangerously by relying on a standard that allows you
>           to use, test, and debug a single version of your code, or
> 
>         * Live dangerously by littering your code with a thousand
>           tests for various platforms, many of which are inaccessible
>           to you, and try to test any problems by waving a dead
>           chicken over the code and chanting ``please don't be too
>           wrong'' a thousand times.
> 
> These algorithms are _not_ simple.  Go read Inderjit Dhillon's
> thesis on the new eigen routines in LAPACK.  Then tell me if you'd
> even consider trusting an implementation that cannot be tested or
> verified with any sense of completeness.  Good thing he didn't
> try to use g77 on an Alpha, or he'd still be working on it.

Well, as long as it is expressible in Fortran, we have no choice *but*
to support it.  That means that if it doesn't work with the current
compiler, we should regard that as a bug.

>  - It's a pity, but I agree that one should use a tool that does the job at
>  - hand.  If g77 doesn't cut it, that's fine with me.
> 
> *boggle*
> 
> I thought the idea was to improve it.

Hmmm, yes, that's what I mean - I didn't express myself very clearly: 
If g77 doesn't yet do what you need, you should use another compiler for
the time being (and that's fine with me).

> Craig Burley is very correct.  The relevant issue at hand should not be
> whether or not j. random code to be compiled is numerically correct.
> It should be whether or not the compiler is even known to produce
> correct code on all platforms where it is `supported.'  Since no one
> even knows the definition of correct for g77...

Granted, but we often get bug reports where the code itself makes
assumptions that are simply not valid - e.g. the infamous "root-finder"
that compares the final value and its latest approximation for equality.

But you are right:  If there is code for which there is a single correct
answer (or interval in which the answer should lie) then g77-compiled
code should provide that answer (an answer within that interval).

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Edward Jason Riedy]

And Toon Moene writes:
 - 
 - Yep.  Still a lot of useful work is done on computers predating the
 - IEEE-754 standard (IBM 370's, VAXen and Cray PVP's).

A shrinking amount of useful work.  Almost always with older code.
Or at least that's my UCB/LBL-influenced view.  It's _very_ easy
to get time on the vector machines up the hill.  Getting full use
of the T3E (with an IEEE-happy compiler, on the Alpha), on the 
other hand...

 - Sure, when run on different architectures, they will give different
 - results.  The crucial question, however, is:  Are these differences
 - interesting ?

Yup.  When you're looking for the smallest eigenvalues, and they're
clustered together, the differences are _very_ interesting.  This
is linked to the waterpipe remark: earthquake simulations.  The
smallest eigenvalues are the ones of interest (not recalling why
at the moment, too tired).  While _your_ code may not care about
the nth place, these codes do.  Trying to rely on data that's
only as good as the input data deserves (which is a favorite
phrase of Dr. Kahan's) can be very difficult when the best bounds
on what the input deserves are still dodgy.

 - Of course there are differences - but they're all meteorologically
 - irrelevant.

That's nice.  You're right, you shouldn't be relying on your own
problem domain.  There are others where it very much matters.

 - Heck - before the 60's, [...]

Before the 60's, the idea of solving systems of equations with more 
than 6 unknowns was silly.  The fact that von Neumann's error
analysis was wrong was not widely known, according to Dr. Parlett.
And the SVD didn't even exist (iirc).

 - Of course I rather wish we could act that fast with g77 too, but you
 - should realize that:

I only used NAG's compiler as an example because I could quickly
provide a URL with further information if anyone wanted it.
There's actually an interesting tidbit in the man page on the Alpha
dealing with IEEE support.

[on using extended precision]
 - But how do you reliably code that ?  Going to assembler ?  There is no
 - way to Standardly specify IEEE-754 extended precision arithmetic, yet.

Who said anything about using IEEE-754 extended precision in hardware?
The reference implementation of the XBLAS (not my project, but one
of my advisor's) is using a software implementation.  That system
uses IEEE double (and won't work on x86s without forcing the CPU to
round to double).  It's also in C, for various reasons.

 - > Yes, it could conceivably be possible to code around
 - > a lack of IEEE-754.  The algorithms involved are complicated enough,
 - > thank you.
 - 
 - Well, OK; that's a choice you want to make.  I think it's rather obvious
 - that I don't like to live that dangerously.

Pick your choices:

	* Live dangerously by relying on a standard that allows you
	  to use, test, and debug a single version of your code, or

	* Live dangerously by littering your code with a thousand
	  tests for various platforms, many of which are inaccessible
	  to you, and try to test any problems by waving a dead
	  chicken over the code and chanting ``please don't be too 
	  wrong'' a thousand times.

These algorithms are _not_ simple.  Go read Inderjit Dhillon's
thesis on the new eigen routines in LAPACK.  Then tell me if you'd
even consider trusting an implementation that cannot be tested or 
verified with any sense of completeness.  Good thing he didn't
try to use g77 on an Alpha, or he'd still be working on it.

 - It's a pity, but I agree that one should use a tool that does the job at
 - hand.  If g77 doesn't cut it, that's fine with me.

*boggle*

I thought the idea was to improve it.

Craig Burley is very correct.  The relevant issue at hand should not be
whether or not j. random code to be compiled is numerically correct.  
It should be whether or not the compiler is even known to produce
correct code on all platforms where it is `supported.'  Since no one
even knows the definition of correct for g77...

Jason

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>I do not personally have a *performance* issue with 80-bit spills,
>because I do not view ia32 type machines as being particularly
>"performant".

IMO that statement *seriously* contradicts your statements regarding
my 80-bit-spill proposal back in December.  In fact, I don't recall
any reason *other* than performance that you (and others, Dave Love and
Richard Henderson, IIRC) offered to explain your outright rejection of
my proposal.

>If we (collectively) decide that -mieee should be the default on
>Alpha's, I know how to specify -mno-ieee in my makefiles :-)

As with 80-bit spills, that decision has been made, and I consider
it irrevocable within the GCC project at this point when it comes to
evaluating my future involvement.  Even if that decision was changed, which
I still think it might be, someday, it has taken *far* too much effort
to get people (who *should* be more competent at these sorts of decisions,
especially given the roles the play in the project) to see the light.

>What really bothers me is that this discussion is denying 40 years of
>research in numerical analysis.

No, it isn't, and what really bothers *me* is people like you chiming
in on discussions that pertain to software quality and usability issues
(which you appear to have little understanding of), claiming they're
*really* about numerics, and then turning around and telling *me* I
shouldn't be participating in the discussions because I'm not an expert
in numerics!

Here's the pertinent part of my *original* post on the topic, which is
what *you* started responding to:

>There are two things that might be working, together or separately, to
>give you trouble:
>
>  1.  You're supplying your own main(), but not running the libg2c
>      initialization code libg2c's main() normally runs.  See the
>      g77 docs for more info.  (Though in your example you wouldn't
>      need to call the routine to set up command-line info, you might
>      in the product code from which you derived the example.  And,
>      in both examples, setting up the signal stuff might be necessary.)
>
>  2.  You're using libg2c to print the FP value, but libg2c *itself*
>      has (presumably) not been compiled with -mieee.
>
>      I had assumed (erroneously, as I recently discovered) that gcc
>      automatically used multilibs to produce -mieee-compiled libraries
>      for Alpha targets, because, without that, libraries can crash (and
>      libg2c apparently *will* crash doing, e.g., output of floating-point
>      values that require -mieee).
>
>  3.  You're not supplying `-mieee' when linking the object files, so
>      even if there *was* a libg2c.a built with -mieee available (and
>      properly configured, whatever that means), you'd pick up the default
>      one and, if #2 is correct, crash anyway.
>
>My own opinion is that this sort of thing is insanely painful to deal with,
                        ^^^^^^^^^^^^^^^^^^
>and that the default should have been -mieee all along (both for gcc and
>for Digital's compilers, but I assume the latter have all-along offered
>easier ways to get the -mieee behavior), with an option (-mno-ieee) being
>needed to get -mno-ieee performance.
>
>But the philosophy of most software designers these days seems to be
>"make it fast, then offer the user a variety of tweaks to make it work
>more slowly".  (The same philosophy is why gcc won't ever default to
>80-bit spills of 80-bit FP registers on the x86, I'm afraid.)

Notice I said *nothing* about *numerical analysis* issues, I was referring
*entirely* to all the special tweaks and contortions *users* must go through
to get the IEEE-standard-conforming behavior *they* want, and have it
actually *work*, for *them*.  (Not to mention the credible evidence that
even if they are happy with -mno-ieee, the *quality* of that default is
pretty poor, e.g. as regards libm exp(), formatted/list-directed I/O,
or whatever.  Not surprising, since we haven't really been able to *test*
that stuff nearly as well as if it more thoroughly implemented IEEE 754.)

Now, you could argue we should never have even offered that (IEEE-conforming)
behavior as an *option*, but that's not what you did.  You started talking
about how preferring industry-standard-conforming behavior *as a default*
was wrong from a *numerical* perspective.  You even hand-waved the fact
that there were behaviors *entirely within the -mno-ieee framework* that
most people (including, I believe, Dave Love) recognized as buggy, so
intent were you on punishing (in my view) anyone for merely *considering*
processing a number below a certain magnitude (within some number of bits
of fraction of a denormal).

The *reality*, Toon, is that choosing -mno-ieee as a default is probably
the biggest reason we provide *no working FP environment* in gcc or
g77 on Alpha machines.  Because -mno-ieee wasn't properly designed or
implemented, and gets inadequate testing (in that there's presumably
tons of IEEE-conformance-testing code that we can't use), it doesn't offer
the sort of consistent environment usually associated with high-quality
implementations.  And -mieee, which *could* have enjoyed the benefits
of all that IEEE-conformance testing, didn't, because it wasn't offered
as the default, and we (collectively) decided -- thanks to opinions like
yours holding sway -- that it wasn't worth bothering much about, so *it*
doesn't really work, either.

(As I pointed out, at least with Digital's compilers, they actually
made sure *both* modes *worked*, and both (presumably) generate faster
code than g77.  But, then, they also *charged* people for the darned
things!  Not surprising, given the last of the three quotes I use near
the end of this email.)

It is the complete and total lack of responsibility you and, to a lesser
extent, other GCC Steering Committee members, exhibit towards even
*attempts* by people like myself to strongly encourage paying attention to
proper design and engineering, quality, and usability issues, that
has led to my decision to no longer work on g77 beyond the 2.95 release.

That plus the fact that probably 99% of *my* writings on this topic
have focused on these very issues, and, instead of responding to them
at a level of competence as befits someone who has taken on the task
of running such an important project, you *continue* to attack on
irrelevant issues like numerical analysis.

*My* analysis of this situation wouldn't change *one whit* if the issue
hadn't involved FP stuff at all, but had instead involved something like,
say, dynamic memory allocation.

For example, if the question was (to pick an actual example, beyond which
I actually don't know the details of the rest of this scenario, so it should
be treated as hypothetical)

  "Should we specify that we handle malloc(0) by crashing, by returning
   NULL, by returning a specific non-null pointer every time, or by
   returning a *unique* (until later freed) pointer every time?"

and if it was *clear* that, while one billion LOC existed that would run
a bit faster if any of the first three options were chosen, *ten*
billion LOC, *including that first one billion*, would run just fine
if the *last* option was chosen, then I'd say it was crystal-clear that,
if the project wanted to DTRT with regards to quality, one of two things
would have to be done:

  -  Choose the fourth option, swallow the performance hit, perhaps
     offer an option for users of the one billion LOC to get the faster
     behavior, but in the meantime get the benefit of all that (10B LOC)
     testing.

  -  Choose one of the other options *and* undertake rewrites of at least,
     say, 50% of the 9B LOC that won't run without modification
     under the option chosen, so at least *some* reasonable amount of
     testing occurs, to make this choice at least *approximate* the
     robustness of the fourth option.

     *And* (note how this is still part of the second choice), *definitely*
     offer the fourth option to users who want to get the other 9B
     LOC up and running without modification.

     *And* (still!) ensure that *all* that code actually gets testing to
     at least 90% of the degree to which it would have anyway had
     the *former* choice (fourth option as default) been made.

     (That latter part is harder to estimate for an OSS project,
     when anyone can do testing without "signing up".  The wrong response
     to that reality is to "punt".  The right response is to *overestimate*
     how much testing of the chosen default *would* have taken place,
     *underestimate* how much testing of it as an option *will* take
     place, and do whatever is necessary to make up the difference.)

Notice how there's simply *no need* to resort to expertise in, say,
garbage collection, heap management, and so on.  Not that those issues
aren't important -- they *underly* the decision-making process -- but,
they're entirely subordinate to that process *given the information
already at hand*.

Also note how *vastly* more expensive that latter choice appears, to
anyone truly willing to *consider* it, than the former choice.  It makes
things like a 10% hit in run-time performance seem truly puny, especially
given that users who actually *experience* that hit can at least *try* the
option they might have been given to get the faster model.

And, as it amounts to a sort of "equation", it illustrates that people
who choose performance over working correctly for a much larger superset
of existing code *necessarily* are choosing *lower quality* for the
product overall, when it comes to a project like GCC.

The exact same situation applies here (to a new compiler project, for
example).

The issue is not correct numerics per se, so no numerical analysis issues
really come up, beyond verifying these basic facts: that, if IEEE 754
conformance was chosen, a *superset* of existing, usable code would
have been available for testing, when compared to choosing some ad-hoc
degree of conformance (say, something vaguely Cray-like).

So your repeated bragging about your expertise in numerical analysis
is about as relevant to *this* discussion as a VLSI logic designer
yelling about how everyone should listen to *his* opinions regarding
whether GCC should be designed to favor efficient code generation for
RISC, for CISC, for VLIW, or for EPIC, leaving the other choices to
have to resort to more complicated machine descriptions.

He could yell all day about his incredible abilities designing logic
boards, but, unless the issue is whether existing CPU chips actually
*work* (and therefore whether GCC is worth porting to them in the first
place), all those abilities are entirely subordinate to how a compiler
like GCC (designed to be portable to a wide variety of architectures)
should be *internally* architected.  *That* decision needs to be made
with regard to *software* design, engineering, and overall quality
and usability issues -- *not* logic-design issues.

(Note that this reasoning does not always apply.  For example, someone
consider issues of how to run a software project to, say, do a Mars
mission must *necessarily* take into account how much effort will be
needed to needed to code around g77's unstable numerics, because they're
very likely to want to use off-the-shelf and/or newly written Fortran
code in the work.  In other words, our unwillingness to provide a stable
numerical environment makes g77 a *special problem* in what *should* be
a high-level evaluation of how to undertake the project, when g77 *should*
just be an issue of choice of compiler based on the usual cost factors.
The upshot is that they might easily decide to avoid g77, and hence not
fund improvements that'd make it *better*, because they can't afford what
they believe it'll take to work around unstable numerics.  It won't matter
one whit if you show up and tell them unstable numerics are ultimately
*their* problem, not g77's -- *they already know that*, they're just
more aware of risk and tolerance issues regarding testability, usability,
graceful degradation, and standards-conformance issues than you care to
bother about.  More than likely, the mere fact that g77 exposes issues
like this to such a degree that it must be considered at such a high level
will get it dismissed out-of-hand, as is surely the case due to its not
supporting, e.g., intrinsics in PARAMETER, another thing very painful to
reliably work around.  But at least nobody -- except for, long ago, maybe
one or two people who don't run important GNU projects -- is yelling at me
for suggesting g77 should support intrinsics in PARAMETER!!)

>If the issue would just be:  Well, we
>have to support 80-bit spills because the highly successful Whiz-Bang
>6.0 Destructively Voidable Fortran compiler by Analog Equipment
>Corporation does it, it would be fine with me.

But that's what *one* issue *was*, back in December: Tim Prince pointed
out that at least one vendor compiler (the only one he tried), lf95
IIRC, *did* do 80-bit spills.  It was the only datapoint I recall we
got, and an inadequate one at that, regarding the *performance* hit
we might take (inadequate because there were too many variables).

>But I cannot seriously be asked to just give up 20 years of experience
>in numerical physics.

Nobody *ever* asked you to do that.  Instead, what you (and others who
have been making top-level GNU decisions) have been doing is *forcing*
people to ignore (in my case) 25 years of experience in making large-scale
systems actually work over the long haul, when used by *typical* users
in a *typical* environment.  (I'm not saying I'm 100% successful in doing
that.  But I've made learning from my mistakes, *especially* in these
areas, one of my highest priorities since about 25 years ago, and I'm
usually -- with perhaps the exception of g77 -- able to point to substantial
and credible evidence that I'm much better at it than my "competition",
e.g. consultants or programmers who worked in those areas in the past.
And, boy, did my experience at Cadence show me just how important having
a coworker who *strongly* understands these issues is -- my coworker
there made life *vastly* easier for me on that project, despite others
who didn't "get" my approach at all, than if he'd been a "vanilla" guy
who merely wanted to "do the job", "make it fast", and move on.  Yes,
we sweated the "make it fast" part in a *very* serious way -- turned out
we, especially my coworker, were able to solve that while enjoying the
benefits of the quality work we'd done, and especially that he'd done
before I got there, up front.  I have *nobody* like that to work with
on g77.)

The upshot is that those of us who *do* know better than to make
short-sighted decisions like these have to nevertheless live with
them being made, and, at least in *my* case, we're the ones who get
to deal with the nasty issues that result -- in that people are,
legitimately, claiming it is *g77*, "Craig Burley's compiler", that
has unstable/unpredictable numerics, not gcc, and not Toon Moene's
lack of awareness of quality-engineering issues.

Or, as so *many* of the industry's experts have been saying for *decades*,
if anybody (especially in the GNU project) is paying attention:

  "Premature optimization is the root of all evil."

  "Make it work; *then* make it fast."

  "You can have it work right, work fast, or be inexpensive -- choose *two*."

I can't recall the quotes exactly, or who originally said them, but that
*last* one is particularly telling -- we've certain made GNU *cheap*
(it's free), so every time we choose *fast*, we *necessarily* choose
to not make it work right.  The lack of sufficient resources -- especially
people willing to do wide testing using comparatively obscure (and
often poorly documented) options like -mieee, not to mention non-existing
options like the IA32 ones that provide 80-bit spills or that enforce
strict-to-type evaluation -- assures this.

While I'm sorry to say the above is an accurate statement of the state
of g77, and probably many other GNU projects as well, I'm happy to finally
have reached the point of saying "My work on g77 is done, I want to
work only on *quality-first* software from now on", and to have a clear
historical document (in the form of these email archives) to point to when
people ask why I didn't try and "reform from within".

And, I now see more clearly than ever that, based on your own descriptions
of what constitutes a quality implementation, g77 *already* perfectly meets
the needs of the only audience you seem to care about...so no rewrite,
or new features, is needed.  (Most importantly, I now see clearly something
I've been concerned about since I started architecting the new front end:
the rewrite *I* have in mind will *not* be tolerated by people who don't
care about, or agree with, my views regarding these issues, since it
is *hugely* influenced by them.  The result would be, if I tried to do the
rewrite, people constantly arguing with me trying to convince me to
change the architecture, the design, and/or the implementation, rather
than spending their time, and mine, testing it.)

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/10/99 7:08:22 AM Pacific Daylight Time, craig@jcb-sc.com 
writes:

>   2. No good compiler bothers special-casing values in the IEEE 754 denorm
>       ranges.  That's because no good code computes values in those ranges.
>       Therefore it's okay if values in those ranges cause undefined results,
>       from crashing to returning really *huge* numbers instead to whatever.
>  
>    3. No good compiler should bother worrying about merely *tiny* numbers 
> that
>       are "close" to IEEE 754 denorms, such that comparing mathematically
>       identical values that are computationally different due to #1 might
>       produce denorms.

I'd be happy to resurrect paranoia or some similar code if you can specify 
it, in whichever version you like, and use that as the standard of whether 
g77 is dealing adequately with these issues.  It's been quite a few versions 
since I've run it, but my recollection is that g77 is as good as or better 
than other compilers.  I've put a lot of work into making it portable so that 
it gives the reports it was intended to give, regardless of whether you 
specify -ffloat-store and the like, including working on a bunch of non-IEEE 
architectures which have bittent the dust.  I believe it is possible to fix 
all the places where its behavior depended on issues such as 64-bit vs 80-bit 
spills.  When those issues matter, the writer of the code must know about it, 
and should fix things so there is no ambiguity.  When fixed up properly, 
paranoia will tell which compiler options give consistent extra precision, 
and which give inconsistent extra precision, avoiding cases which push harder 
than necessary on that issue.

No doubt there are certain targets and option selections which could present 
problems, but I would think it should be the business of g77 to let sanely 
implemented hardware systems do their job as they were designed to do.  One 
obvious point where the Fortran standard gets in the way of this is in the 
definition of NINT() and ANINT(), where most applications of which I am aware 
would be happy to do it the way the hardware is designed to do, almost 
invariably in accordance with IEEE754/854 nowadays.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> I wrote:

> >If people want to get tighter control, there are two ways:

> >1. Try to influence X3J3 to impose them.

> >2. Use another language.

> In which case, you won't mind one whit if -mieee and 80-bit spills are
> made the defaults, since X3J3 doesn't impose *performance* constraints?

> Well, it's about *time* you conceded that.

> ;-/

Hmmmm, I hope I nowhere said that the Standard imposes constraints
*against* using full IEEE-754 conformance and 80-bit spills.

We (and I hold both of us responsible for that) should have discussed
this *as a quality-of-implementation-issue*.

I do not personally have a *performance* issue with 80-bit spills,
because I do not view ia32 type machines as being particularly
"performant".  (I'm still waiting for the "My Other Computer Is A 21264"
sticker Compaq employees promised me at LinuxExpo to put on my PII 300
Mhz laptop).

If we (collectively) decide that -mieee should be the default on
Alpha's, I know how to specify -mno-ieee in my makefiles :-)

What really bothers me is that this discussion is denying 40 years of
research in numerical analysis.  If the issue would just be:  Well, we
have to support 80-bit spills because the highly successful Whiz-Bang
6.0 Destructively Voidable Fortran compiler by Analog Equipment
Corporation does it, it would be fine with me.

But I cannot seriously be asked to just give up 20 years of experience
in numerical physics.

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>Some elder statesmen of Numerical Analysis really suffer from "All the 
>world is a Cray"-syndrome and never really accepted IEEE semantics.

Unfortunately, they never bothered specifying the FP semantics *they*
wanted from the rest of the industry and publishing that spec for me
(and other gcc developers) to work from.

That's what this really boils down to, I'm afraid: the anti-IEEE people
refuse to accept the notion that they should just put up with the
moderate slowdown they think IEEE mandates, but they don't offer any
*specification* for what it is they *really* want.

At least with IEEE 754, people like myself (who Toon now claims should
not even participate in these discussions!) can implement compilers to a
*spec*, write tests to a *spec*, even run industry-standard tests to
that same *spec*.  All without being experts in numerical analysis or
reading Numerical Recipes -- which gives them more time to be experts
in compiler development and read the Dragon Book.

In other words, I am now prohibited (by Toon) from discussing even whether
my *own* compiler implements floating-point arithmetic *correctly*, because
I'm not a numerical-analysis expert, because there's no *spec* for how it's
supposed to work, and, therefore, I must rely totally on the opinions of
others.

Whereas, when I *thought* g77 was supposed to faithfully work to *at least*
the (sometimes-less-than-ideal) "virtual" spec provided by the underlying
hardware, I *could* discuss whether g77 met that spec, because that spec
is usually formulated in language that doesn't require numerical-analysis
expertise to understand.  (E.g. long ago, when I read IEEE 754, I found
it pretty easy to understand.  Heck, I was helping hardware developers
debug and diagnose floating-point hardware for a mini-supercomputer back
then.  All without numerical-analysis expertise.  I left the company not
long after the VP of Engineering told me there was no way the company
would fix already-deployed systems that I discovered miscompared 64-bit
values that differed starting in bit 32 -- once I learned that the people
at the top didn't care to ensure the machine could be part of an overall,
robust system, I didn't hang around long, despite being paid far more
[salary] than I'm being paid now to work on g77 [nada].)

Now, in the last six months, we've learned (all from Toon, mind you, one
of *the* most respective voices in the OSS number-crunching community, and
for good reason), that:

  1. Even if Fortran says a computation evaluates to a single approximation,
     a good compiler ignores that (as mere "advice") if evaluating multiple
     approximations leads to faster code.  After all, good code is
     (somehow -- I have yet to see any clear, correct recipes as to how to
     make this happen) insensitive to that behavior.

  2. No good compiler bothers special-casing values in the IEEE 754 denorm
     ranges.  That's because no good code computes values in those ranges.
     Therefore it's okay if values in those ranges cause undefined results,
     from crashing to returning really *huge* numbers instead to whatever.

  3. No good compiler should bother worrying about merely *tiny* numbers that
     are "close" to IEEE 754 denorms, such that comparing mathematically
     identical values that are computationally different due to #1 might
     produce denorms.

Particularly of note is that rule #3 sprang into existence, to my view,
only after I'd pointed out the apparent contradiction between rules 1 and
2, which apparently were formulated months, or years, ago.  I.e. rule #3
didn't exist until sometime yesterday.  If anyone believes otherwise,
please email a URL showing where it was publicized, I'd love to see what
I'm missing!

I'm no longer interested in trying to write a compiler to a non-existent
specification, so I've pretty much decided to stop bothering, and go find
something else to do -- which will almost certainly involve software work
only to the extent that the project is run by someone who actually
understands total quality engineering for large-scale projects, which
appears to *not* be the case for GCC (or GNU or even Linux).

I mean, Y2K is a stupid problem made with exactly the same mind-set that
is being employed here: "nobody should ever deploy this system beyond 1999,
that is just *way* too far in the future" is, from a total-quality
standpoint, nearly indiscernible from "nobody should ever compute a value
smaller than 1E-15".  The concept of "degrade gracefully", designed into
IEEE 754 (for better or for worse), doesn't exist for people with this
mind-set -- they prefer "degrade randomly, as long as it still runs fast"
or something like that, and are under the delusion that their approach
works for large-scale systems (which includes basically any project that
depends on a compiler as large and complex as gcc/g77, IMO) developed by
a large number of people with varying sorts and levels of expertise.

Seems to me I have better things to do with my time, experience, and
expertise than put a nice sheen (good docs, rewritten g77 front end,
whatever) on a project where the people in charge can't wait to add the
*next* Y2K-like bug.  However nicely I pave the road, and decorate it with
flowers, it'll still be a road to hell, and it doesn't help much that I
can claim my particular *segment* of it doesn't actually get you there all
by itself.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Lieven Marchand]

craig@jcb-sc.com writes:
 > So it's not that I don't respect that approach -- I do.  I just happen
 > to think that, in practice, the *audience* for that approach -- defined
 > as the audience that *knows what they're doing* -- is vanishingly small
 > compared to the audience for "pedestrian", straightforward, robust
 > IEEE 754 support.

Even the audience that *knows what they're doing* needs a robust
implementation first and foremost. I'm no longer really part of that
audience but I have a degree in Numeral Mathematics and before you put 
a program in production you want to have an implementation that
catches NaNs, deNorms, out of bound references etc to write and debug
your program. Tuning a production version with faster versions of
functions, inlining etc. is only feasible with a large regression set
of known correct results to catch errors.

 > After all, Toon represents the sort of audience I claim does not *need*
 > g77 to default to "gofast" mode -- I mean, they've already rewritten
 > their code to inline transcendentals, to avoid INTEGER*8 (so it could
 > run on $$$ Fujitsu iron), and so on.  To me, it's hard to understand
 > why it's so painful for that audience to specify `-mno-ieee' or some
 > such thing, but apparently it is.
 > 

Ditto.

 > Also, I keep seeing the claim that, effectively, nobody should be using
 > floating-point (at least in Fortran) unless they're solving one of a
 > particular set of problems that have been Declared Properly Solved Using
 > Limited Range (DPSULR) by people like Toon.

Some elder statesmen of Numerical Analysis really suffer from "All the 
world is a Cray"-syndrome and never really accepted IEEE semantics.

-- 
Lieven Marchand <mal@bewoner.dma.be>
If there are aliens, they play Go. -- Lasker

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

[ Oh, I see I forgot one important issue - Emil, I hope you don't mind
  if I discuss this private e-mail on the list: ]

Emil Hallin wrote:
> 
> Toon Moene wrote:
> 
> > Of course one could always set up a scenario where, using the fact that
> > you cannot know whether a floating point value is 80-, 64- or 32-bits
> > wide, you can generate small differences (which should have been zero)
> > that, suitably multiplied which each other, will generate a denormal
> > (especially if you start close to FLT_MIN).  My point is that you have
> > to cleverly set this up, because it will not show up in normal practice.
> 
> I still think the egcs compiler should do the right thing and spill 80 bit fp
> values when it has to. (IBM thought noone would ever use more than 640K "in
> normal practice" either and look what that did to us!).
> 
> I am a  physicist so I appreciate your arguments; my programs are not likely to
> break on this issue, but I can't *guarantee* that they won't break, and I
> dislike having to rely on results that might be different depending on whether
> or not I clicked on the mouse at the wrong time, forcing a context switch and
> register spills at an inopportune moment.

Ah, ho, keep it right there !

The saving of floating point registers on interrupt is an OS-issue.  The
compiler cannot do anything about that.

[I hope and actually am quite confident that Linux actually saves the
 complete 8x80 bit contents of the ia32 floating point stack]

This, indeed, is not the sort of impredictability one would like to have
(even if your algorithms are robust against small differences in
floating point arithmetic, having two subsequent runs with the same
input differ would be rather scary).

The discussion is whether spills to the stack of floating point
registers *by the compiler* should be done in full 80-bit accuracy, even
if the relevant variable was only declared as REAL (32 bits) or DOUBLE
PRECISION (64 bits).  This is not done now, and that might mean that two
runs with the same code and same input, but a different version of the
compiler, could give you different results.

OTOH, a new version of the compiler could also decide to evaluate your
expressions differently (within the limits required by the Standard, of
course) and causing output to differ that way.

Hope this helps,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Edward Jason Riedy wrote:

> And Toon Moene writes:

>  - What I *do* object to is people trying to mimic that in Fortran, hoping
>  - that the Fortran Standard conformant evaluation of expressions and
>  - IEEE-754 requirements don't clash.

> You are quite right; that's the real problem.  Many people bemoan it.

Yep.  Still a lot of useful work is done on computers predating the
IEEE-754 standard (IBM 370's, VAXen and Cray PVP's).

Of these, Crays have an amazingly lax attitude towards floating point
arithmetic.

Why then is it possible to do useful numerical physics on these machines
?  Because most code can be made robust against small differences in
floating point arithmetic.

Sure, when run on different architectures, they will give different
results.  The crucial question, however, is:  Are these differences
interesting ?

I know it is dangerous to keep using our own application as an example
(I am glad you and others pipe in with different examples), but we
occasionally run our Numerical Weather Prediction code in double
precision (using compilers that support -r8 -i8 "promotion" flags).

Of course there are differences - but they're all meteorologically
irrelevant.

Heck - before the 60's, planes were designed using slide rules; that's 3
digits of precision.  The reason that worked is that aircraft designers
at that time used completely different algorithms to compute the
relevant physical phenomena around body and wings.

> However, there is a growing body of Fortran code (amazing statement,
> even without the following quantifier) that thinks Fortran supports
> IEEE-754 in most ways.  And most of the compilers I can see (just
> looked at the T3E (MIPSpro) docs, NAG's new f95 compiler's man
> page, etc.) seem to be following that.

I do not know about the MIPSpro compiler, but NAG's F95 compiler is
essentially written by one man: Malcolm Cohen.  At the latest X3J3
meeting (7-11 June, in Oxford) he pointed out that he had already
implemented most of the Chapter 15 requirements of F2K (which describe
floating point exception handling and the support of IEEE-754 features).

Of course I rather wish we could act that fast with g77 too, but you
should realize that:

1. Malcolm has been on the Committee for a very long time, so he
   knows what's coming.

2. NAG's compiler is protected from the idiosyncracies of existing
   hardware by being a Fortran 95 to C compiler.

> I'm not going to pretend that I know the differences between the
> Fortran Standard and IEEE-754.  I don't want to know the differences.
> (Just found out that F90 and F95 don't even agree on how to print
> "-0.0".  Yech.)

This was changed in Fortran 95 to support hardware that could discern
+0.0 and -0.0 (i.e. to enhance support for IEEE-754).

> Fortran seems to be moving in the IEEE direction, so I suppose the
> real question should be whether or not IEEE floating point support
> by default would _break_ functional codes.  I don't see how, but
> of course someone at Berkeley is going to be supporting IEEE
> arithmetic.  ;)

Well, you can rest assured that IEEE-754 support as formulated in the
upcoming F2K standard won't break existing code.  It's all of the form

      IF (IEEE_SUPPORTS_SOME_FEATURE()) THEN
         ... use IEEE-754 feature ...
      ELSE
         ... just plod along as in the old days ...
      ENDIF

> --other--

> BTW, in some FP experts' opinions (I don't qualify as an expert;
> I'm relaying what's been said by actual experts around here) there's
> no such thing as fast code that produces incorrect results.  It's
> incorrect code.

The problematic term here is "incorrect".  See my note above on
"irrelevant differences"; in short, if you can measure wind speed to 0.1
m/s accuracy, the fact that two model runs on two different
architectures differ in the fifth decimal is not relevant.

> And much of the code around here does _NOT_ fall into Toon's category.
> The FE codes very much have to deal with the nastier parts of the FP
> spectrum.  Some of the solutions are algorithmic (ignoring the
> vibrational modes of waterpipes)

Yep - just as we ignore sound waves in the atmosphere.  You model what
you want to describe; nothing more, nothing less.

, and others are numeric (using
> extended precision implemented on IEEE arithmetic during iterative
> refinement).

But how do you reliably code that ?  Going to assembler ?  There is no
way to Standardly specify IEEE-754 extended precision arithmetic, yet.

> Yes, it could conceivably be possible to code around
> a lack of IEEE-754.  The algorithms involved are complicated enough,
> thank you.

Well, OK; that's a choice you want to make.  I think it's rather obvious
that I don't like to live that dangerously.

> However, you won't hear a peep out of most of the folks who write
> them.  They still think g77 is only f2c, and they all bought tuned,
> vendor compilers.  Many don't even know g77 still exists, or think
> it died off (don't know it's now distributed with gcc).

Hmmm, that sounds more like a problem with our PR department :-)

> g77's market around here right now is very, very, very small,
> consisting of folks who want a nice set of tools that lets them mix
> C, C++, and Fortran nicely, typically on clusters of Linux boxes.
> To be fair, gcc doesn't deal with the biggest systems in my immediate
> vicinity (Crays of various flavors), and won't link to vendor SMP-
> capable BLAS on others (licensed libraries on Suns).  And other codes
> we want to play with (MUMPS, etc.) are in F90.  So my arguments can
> probably be dismissed; I'm not going to be using g77 for real work
> anytime soon.

It's a pity, but I agree that one should use a tool that does the job at
hand.  If g77 doesn't cut it, that's fine with me.

Thanks for sharing your insights.

[ I hope I dealt with topics raised in other e-mails too - if not,
  please take it up with me privately ]

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>If a code is written portably it shouldn't fail due to numerical differences 
>outside the precision of the declarations.

So you really think it's okay for any arbitrary reference to a variable
X to evaluate, at any given time, to X +/- a sufficiently small random
number?

That's what you're essentially saying -- that even IF (X .LT. X) should
potentially evaluate to .TRUE. on a random basis, which means even that
statement is not "written portably".  Further, you're saying that two
IF (X .LT. 0.) in a row should evaluate differently, if the difference
is due to bits 64-79 (so to speak) or, for single precision, 32-79, so
*that* statement is not "written portably".  For programs that *contain*
those statements to be "written portably", they must somehow cope with
the fact that the statements themselves may behave randomly.

Yet that contradicts any reasonable reading of ANSI FORTRAN 77, as far
as I can tell, which mandates that a *single* approximation is assigned to
a variable.  (It uses the singular, not plural, form of the pertinent
nouns.)

>I've noted also the correctness of Moshier's 
>suggestion that many spills are of loop-invariant quantities which are read 
>from static storage and thus a wider spill cannot contribute to accuracy.  
>It's a shame to have these spills hurt performance, and probably impractical 
>to distinguish them from those where there really are some guard bits to save.

Not impractical at all, though perhaps non-trivial for gcc as current
designed.  A compiler *necessarily* knows from where a value originally
came (static storage, or computed on-the-fly, e.g. in a register).
Whether it tracks that, I don't know.  One thing for sure: if the static
variable is *defined* with a new value initially computed as an 80-bit
result, then if that 80-bit result gets *any* visibility (i.e. if it is
not immediately truncated to the declared size of the static variable),
then, yes, any spill of that 80-bit result, *including committing the
value back to static storage for use by subsequent procedures*, must
preserve the entire 80 bits, to conform to ANSI FORTRAN 77, by my
(current) reading.

But, I didn't intend to re-propose 80-bit spills -- I withdrew my proposal
concerning that last December, and I'm sticking with my decision.  AFAIC,
"we" decided (overriding my concerns, despite the lack of any real
performance analysis showing that my proposal would be as bad as claimed if
implemented) to ignore FORTRAN standards compatibility in favor of the
ad-hoc consensus (which, *months* later, has yet to be clearly documented)
regarding how to write floating-point code for g77.

And defaulting to -mno-ieee, I've *also* already "swallowed" in terms
of what that does to the quality-of-implementation issue at a *design*
level.

What we're now beginning to discover is just how much that decision
adversely affected the quality-of-implementation at the *implementation*
and *packaging* levels -- to wit, random crashes depending on what
narrow range of numerics is involved, and confused users who don't
understand that they shouldn't go anywhere near a Fortran compiler
(much less write Fortran code).

And there's basically *nothing* I can do about this, even as g77 author
and de facto "project leader", since to effectively mandate what I believe
is the robust, properly-engineered design (which would include the
concept of *tolerances*, i.e. have the compiler support IEEE 754 in as
clean a way as possible so not only does Toon-style code work, but so
does independent spec-based testing, etc.), I'd have to either convince
the GCC team to switch everything over (which I will no longer attempt to
do) or re-do all their work in the form of a fork or new compiler (which
probably isn't worth attempting).

The issue now is, am I going to continue working on a compiler that an
extremely small percentage of the *Fortran programmer* population can
use effectively, given the decisions *already made*, and which I concede
will *not* be changed, concerning the use of truncating spills, defaulting
to -mno-ieee (without bothering to first architect, design, implement, and
document the underlying types throughout the infrastructure), and other
decisions that have been, and surely will be, made that are as wrongheaded
and short-sighted (as well as contradictory with each other) as I consider
these to be?

I expect I'll have made a decision by the end of this week, if not sooner.

After all, I've been *seriously* thinking about this issue since this
past December, and now regret that I didn't insist that we make robustness
of code generation and predictability of numerical results the #1 topic
of the Fortran BOF at LinuxExpo, since, in my view, that's the most
important decision vis-a-vis g77's future than anything else we discussed.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[N8TM]

In a message dated 7/9/99 8:12:31 PM PST, craig@jcb-sc.com writes:

> >I tried to express that to write useful numerical code one has to have
>  >knowledge of numerical analysis.

 One of the reasons we continue to use Fortran is that it's easier to fix 
code written without attention to such details in Fortran than in other 
languages.  To put it another way, Fortran is the only language where people 
who barely get by with their numerical skills can "write useful numerical 
code."  Many times those people are the only experts in their specialty 
available to put a needed procedure into practice.

>>spilling 80-bit registers assigned to variables to 64-bit
or 32-bit temporaries *does* -- returns an 80-bit result and a 32-bit
result (say), allowing the processor to randomly choose which to use at
any given time

If a code is written portably it shouldn't fail due to numerical differences 
outside the precision of the declarations.  g77 is remarkably reliable as it 
is, but it hasn't solved the problem of alignments on several important 
targets.  Acknowledged experts on egcs have advised against -malign-double, 
although I would consider it a bug if that option caused g77 to fail.  At the 
risk of being overly repetitive, I would be happy with 64-bit spills of 
single precision and 80-bit spills of double if the alignment performance 
problem could be solved.  I've noted also the correctness of Moshier's 
suggestion that many spills are of loop-invariant quantities which are read 
from static storage and thus a wider spill cannot contribute to accuracy.  
It's a shame to have these spills hurt performance, and probably impractical 
to distinguish them from those where there really are some guard bits to save.

Tim
tprince@computer.org

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>craig@jcb-sc.com wrote:
>
>> >The evidence on these lists prove that people are still forced to write
>> >programs without getting a decent course in numerical analysis.
>
>> Again, as I pointed out before, *I* have taken no "decent course
>> in numerical analysis".
>
>Nope, and it is not necessary for you to do so.
>
>[ Unless you want to take part in discussions like this, otherwise, 
>  you should just perform useful work on g77: ]

*My* original point was that choosing -mieee as a default would have
resulted in gcc/g77 support for the Alpha being, *today*, much further
along in terms of support for at least the industry-standard model of
floating-point.

Instead, we have inadequate, and probably overly buggy, support for -mieee
(no libraries compiled that way out of the box, and we sure aren't doing
enough testing of code with -mieee in effect to have much confidence that
simply changing the default wouldn't lead to all sorts of regressions,
ICEs, and so on), and buggy support (to say the least) for the default,
-mno-ieee.

In the meantime, you haven't answered my question as to why I should
be working on compilers, having taken no decent (well, no university-level)
course on compilers and having not read, well, I can't think of *one*
text on compiler design!

>I tried to express that to write useful numerical code one has to have
>knowledge of numerical analysis.

Fine.  Is it really your contention that not a single person who has
*ever* learned numerical analysis would use a wider range of IEEE 754
than you claim is all that needs to be properly supported?  (A range
you have yet to actually *specify*, I believe.)

>It is *not* sufficient to paper over mistakes made by people who didn't
>read their stuff just to get "the right answer".

So you consider implementing IEEE 754 properly to be "papering over
mistakes made by people who didn't read their stuff"??

>What a Fortran compiler should do is (IMHO) compile the source to the
>object code, using the latitude given by the Standard (i.e. SQRT: the
>result has a value equal to a processor dependent approximation to the
>square root of the argument).

In other words, it's okay if SQRT(16.0) => -1.5.  (Yes, folks, it's
true: the Fortran standard does *not* disallow that result.  I'm not
kidding.  Why Toon is suddenly resorting to the Fortran standard to
define what valid numerical behavior is, I don't know; frankly, in
comp.lang.fortran circles, it'd be a sign of incipient insanity, or
at least a indication that the person realizes he's losing the
argument.  But maybe that's no longer true; I gave up USENET about
six months ago.)

Sorry, Toon, but I've been working on Fortran compilers, and reading
comp.lang.fortran, long enough to have more of a clue than *that*.

Further, you have essentially argued that g77 need not even return *a*
value equal to that approximation, but may return *multiple* values equal
to that approximation, despite the very language you quoted.

That's what spilling 80-bit registers assigned to variables to 64-bit
or 32-bit temporaries *does* -- returns an 80-bit result and a 32-bit
result (say), allowing the processor to randomly choose which to use at
any given time, even when the result is assigned directly to a variable,
even if that variable (as assigned) is read only *once* in the user's
source code.

Getting back to the quality-of-implementation issue: I have always said
that it's ultimately okay for a Fortran processor to start WW3 if the
inputs don't conform, according to the standard.

But I *strongly* disagree with the idea that a *quality* implementation
starts WW3 only if the input happens to be between, say, 1E-60 and 1E-30,
or whatever the range of IEEE 754 single-precision denormals happens
to be.

That sort of implementation makes basic, solid, engineering practices like
testing and validation *much* less useful, because the behavior of
the underlying system changes wildly in the presence of out-of-range
values -- never mind whether the *code* running on top of it has any
actual *bugs* in it.

>If people want to get tighter control, there are two ways:
>
>1. Try to influence X3J3 to impose them.
>
>2. Use another language.

In which case, you won't mind one whit if -mieee and 80-bit spills are
made the defaults, since X3J3 doesn't impose *performance* constraints?

Well, it's about *time* you conceded that.

;-/

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>BTW, in some FP experts' opinions (I don't qualify as an expert;
>I'm relaying what's been said by actual experts around here) there's 
>no such thing as fast code that produces incorrect results.  It's
>incorrect code.  

That's what Toon has been saying, isn't it?

>And much of the code around here does _NOT_ fall into Toon's category.  
>The FE codes very much have to deal with the nastier parts of the FP 
>spectrum.  Some of the solutions are algorithmic (ignoring the 
>vibrational modes of waterpipes), and others are numeric (using 
>extended precision implemented on IEEE arithmetic during iterative 
>refinement).  Yes, it could conceivably be possible to code around
>a lack of IEEE-754.  The algorithms involved are complicated enough,
>thank you.

Ah, this is where Toon and I *definitely* part company.

In Toon's world-view, all programmers need to be experts on all the
ways the compilers they use can make their lives miserable.  Being
experts on the standards isn't even enough (certainly not for Fortran,
since basically nothing worthwhile is guaranteed by the present standards),
and it is positively *stupid* to assume reasonable straightforward
quality-of-implementation decisions made by compilers.

In *my* world-view, I'd rather programmers be experts on their *problem
domain*, and the compiler implement their codes in the most straightforward
way possible, leaving optimization to people who *are* experts in
optimization.

Working in Fortran certainly de-virgin-izes my stance, just as it does
Toon's.  That is, Fortran is hardly the right language for programmers
to express their problems without regard for speed issues; and it
certainly is the wrong language for programmers to express bare-to-the-
metal optimizations.

But at least, with my world-view, things get *more* predictable the
less you worry about how the underlying components work.

With Toon's world-view, things get *less* predictable.  That is, you have
to constantly *increase* your expertise on how off-the-shelf components
work (in terms of how they don't do things as you'd "expect", whatever
that might mean) as you decide to use more off-the-shelf components.

(Which perhaps partly explains why shops like Toon's avoid using even
25-year-old off-the-shelf components, like SIN().)

Again, I perfectly agree with Toon's *view* as it pertains to certain
*domains*.

Clearly, I disagree pretty strongly that his view applies to the entire
world, or even the majority, of potential users of g77 (or g90 or g95
or g2k).

>So my arguments can 
>probably be dismissed; I'm not going to be using g77 for real work 
>anytime soon.

It's my suspicion that that statement is, and will ever remain, true
for *many* potential users of g77, as long as it chooses defaults that
favor unpredictability to achieve speed, and especially as long as it
doesn't offer fully-tested, fully-blown versions of *options* that
provide the predictability those users insist upon, as you yourself do.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Edward Jason Riedy]

And Toon Moene writes:
 - 
 - What I *do* object to is people trying to mimic that in Fortran, hoping
 - that the Fortran Standard conformant evaluation of expressions and
 - IEEE-754 requirements don't clash.

You are quite right; that's the real problem.  Many people bemoan it.

However, there is a growing body of Fortran code (amazing statement,
even without the following quantifier) that thinks Fortran supports 
IEEE-754 in most ways.  And most of the compilers I can see (just
looked at the T3E (MIPSpro) docs, NAG's new f95 compiler's man
page, etc.) seem to be following that.

I'm not going to pretend that I know the differences between the 
Fortran Standard and IEEE-754.  I don't want to know the differences.  
(Just found out that F90 and F95 don't even agree on how to print
"-0.0".  Yech.)

Fortran seems to be moving in the IEEE direction, so I suppose the
real question should be whether or not IEEE floating point support
by default would _break_ functional codes.  I don't see how, but
of course someone at Berkeley is going to be supporting IEEE 
arithmetic.  ;)

 - I will read that.  Thanks for the pointer.

And if anyone else is interested, Dr. Kahan does have some other 
interesting FP texts under http://www.cs.berkeley.edu/~wkahan/ ,
including a rant on Java's arithmetic.  (No, I don't want to think
about gcj's IEEE-754 non-support.  ;) )  Dr. Fateman has some
vague lingustic notes from FP98 at .../~fateman/fp98/, but the
only part that may be of interest is in the korenD directory,
and it's not on the denorm topic.

--other--

BTW, in some FP experts' opinions (I don't qualify as an expert;
I'm relaying what's been said by actual experts around here) there's 
no such thing as fast code that produces incorrect results.  It's
incorrect code.  

And much of the code around here does _NOT_ fall into Toon's category.  
The FE codes very much have to deal with the nastier parts of the FP 
spectrum.  Some of the solutions are algorithmic (ignoring the 
vibrational modes of waterpipes), and others are numeric (using 
extended precision implemented on IEEE arithmetic during iterative 
refinement).  Yes, it could conceivably be possible to code around
a lack of IEEE-754.  The algorithms involved are complicated enough,
thank you.

However, you won't hear a peep out of most of the folks who write 
them.  They still think g77 is only f2c, and they all bought tuned, 
vendor compilers.  Many don't even know g77 still exists, or think
it died off (don't know it's now distributed with gcc).

g77's market around here right now is very, very, very small, 
consisting of folks who want a nice set of tools that lets them mix 
C, C++, and Fortran nicely, typically on clusters of Linux boxes.  
To be fair, gcc doesn't deal with the biggest systems in my immediate 
vicinity (Crays of various flavors), and won't link to vendor SMP-
capable BLAS on others (licensed libraries on Suns).  And other codes 
we want to play with (MUMPS, etc.) are in F90.  So my arguments can 
probably be dismissed; I'm not going to be using g77 for real work 
anytime soon.

Jason

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >The evidence on these lists prove that people are still forced to write
> >programs without getting a decent course in numerical analysis.

> Again, as I pointed out before, *I* have taken no "decent course
> in numerical analysis".

Nope, and it is not necessary for you to do so.

[ Unless you want to take part in discussions like this, otherwise, 
  you should just perform useful work on g77: ]

> Why therefore am I pretending I can do useful work on g77, since its
> raison d'etre is to compile code doing numerical work?

I tried to express that to write useful numerical code one has to have
knowledge of numerical analysis.

It is *not* sufficient to paper over mistakes made by people who didn't
read their stuff just to get "the right answer".

What a Fortran compiler should do is (IMHO) compile the source to the
object code, using the latitude given by the Standard (i.e. SQRT: the
result has a value equal to a processor dependent approximation to the
square root of the argument).

If people want to get tighter control, there are two ways:

1. Try to influence X3J3 to impose them.

2. Use another language.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

PS: Just to prevent mistakes: I do not view Press et al's "Numerical
Recipes" the be-all end-all of numerical analysis.

It's just that some of the issues that came up on this list were
adequately answered by quoting passages from this book.

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>In the end, it is about great software that users like and respect.

Great software doesn't crash and burn on input that is wrong in highly
predictable ways.  g77 *does*, in effect, do that.

>Personally I don't have much of an opinion if gofast or becorrect
>should be the default.

If only that were the choice.  In fact, the choice here is between bewrong
and beabitslowerthantheoreticallypossible.  After all, even Toon agrees
that too many people are writing Fortran code without knowing what they're
doing.  That number is, IMO, vastly higher if we choose Toon's model than
if we choose defaults consistent with what the industry, itself, has
proposed, debated, and standardized upon (and which at least *some*,
if not all, other Fortran compiler have agreed upon as well).

The wonderful thing about choosing Toon's approach is that, we can then
go rewrite all the intrinsics (such as the transcendentals) to use faster
methods that don't bother to compute correct results for inputs beyond
the "accepted" range.  After all, that's what the code Toon works on *does*
-- it doesn't bother calling SIN, COS, etc., it employs faster in-line
versions that don't worry about those bothersome corner cases.

If we decide to go whole-hog for Toon's approach, we can replace, wholesale,
our SIN/COS/TAN and other intrinsics with similar kinds of open-coded
replacements.

So it's not that I don't respect that approach -- I do.  I just happen
to think that, in practice, the *audience* for that approach -- defined
as the audience that *knows what they're doing* -- is vanishingly small
compared to the audience for "pedestrian", straightforward, robust
IEEE 754 support.

After all, Toon represents the sort of audience I claim does not *need*
g77 to default to "gofast" mode -- I mean, they've already rewritten
their code to inline transcendentals, to avoid INTEGER*8 (so it could
run on $$$ Fujitsu iron), and so on.  To me, it's hard to understand
why it's so painful for that audience to specify `-mno-ieee' or some
such thing, but apparently it is.

Also, I keep seeing the claim that, effectively, nobody should be using
floating-point (at least in Fortran) unless they're solving one of a
particular set of problems that have been Declared Properly Solved Using
Limited Range (DPSULR) by people like Toon.

What I *don't* know is whether that claim is true.  I certainly see plenty
of people who, like Toon, seem to know a lot more than I do about how
Fortran is actually used in the field, who disagree that DPSULR problems
are the only ones worth using floating-point for, and who claim the
*other* problems rely on the full range and support of IEEE 754 being
available.

It is also non-trivial for me to just say "well, I'll pick *my* defaults
for g77, let someone else pick whatever *they* want", because, without
the support of much of the GCC team (Toon being on the Steering Committee,
but not me!), it'll be difficult to get people to help support things
like 80-bit spills and -mieee working correctly.

Similarly, if we *don't* thoroughly discuss and document the issues,
and instead Toon and others simply capitulate to my feelings, the
result could be that g77 becomes useless to most of its audience because
it no longer defaults to what they see as "gofast" mode, or doesn't
properly implement it (as is apparently the case with -mno-ieee being
the default, vis-a-vis issues I raised earlier).

In other words, we barely have enough resources to support one incarnation
of g77.  It's not clear we'd have nearly enough to support two or three.

And, my having to support the conflicting requirements of multiple camps
of Fortran users has been painful enough to deal with over the years
(e.g. backslashes in character constants).  This is yet another very
difficult schism, amplified by the fact that the very users Toon handwaves
as unworthy of g77 (since think they need full IEEE 754 support) are
the ones *most* likely to submit bug reports before they do full
investigations of the problem.  So if we choose Toon's camp, we get
(as we've been getting, only worse as g77 improves and, hopefully, becomes
more popular) bazillions of not-quite-right bug reports; if we choose
overall robustness, we get hardly any by comparison, since g77 Just Works,
and the Toon-like experts already know how to use compiler options to get
speed *and* how to investigate code generation to find bugs, before
complaining about them.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>The evidence on these lists prove that people are still forced to write
>programs without getting a decent course in numerical analysis.

Again, as I pointed out before, *I* have taken no "decent course
in numerical analysis".

Why therefore am I pretending I can do useful work on g77, since its
raison d'etre is to compile code doing numerical work?

>Why do you think I quote "Numerical Recipes" ?  This stuff is so well
>known that it is equivalent to pointing out an error in gcc wrt to the
>Dragon Book.

I do not possess a copy of "Numerical Recipes", nor have I ever read it.

Why am I working on g77?

(For that matter, I do not possess a copy of the Dragon Book.  Why am
I working on gcc, or doing *any* compiler work?)

I guess the problem is, I read the *standards*, like ANSI FORTRAN 77
and ISO C, and draw conclusions, including the one I've drawn, that
failing to spill 80-bit values to 80-bit temporaries is a bug.

IMO, that fact *cannot* be contradicted by anything "Numerical Recipes"
says, since such a book can only go so far as to advise people on how
to work around bugs in products like g77 and gcc...unless you're claiming
the Fortran and C languages are *based* on the "standards" that make
up the text of "Numerical Recipes", which would be news to me.

Put another way: a 1920s-era book on just how to maintain the automobiles
of the day should not be cited as evidence against the need to improve
automobiles, e.g. by replacing the crank with an electric starter.

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Mike Stump wrote:

> Guys, guys...  In the end, the gcc folks get to pick what front end
> they want, and they should be willing to respect the wishes of the
> frontend folks to a large degree...  The frontend folks need to agree
> among themselves what the `right' behavior is. \

Good point.

We're working on it.

:-)

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Edward Jason Riedy wrote:

> And Toon Moene writes:

>  - My point is that every physicist engaging in numerical analysis will
>  - choose his/her units to get this effect.  This is not surprising, as
>  - people like to talk about the major phenomena they study using small
>  - numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
>  - measure distance.
> 
> Please go to http://www.cise.ufl.edu/~davis/sparse/ and support
> the claim that all of these matrices (most from real problems,
> many from industry) never run into floating-point problems and
> always live happily in your world of `just-right' numbers when
> being factored or when used in an iterative solver.
> 
> You can't, because they don't.  Now, they can give useful solutions
> using less precision in most places and more in a few, key places
> (see papers by Demmel and Li on parallelizing SuperLU), but that's a
> different story.

It will take time to review this.

> Likewise, from a new eigenvalue routine in LAPACK, a large algorithmic
> improvement:
> *  Note 3 : ZSTEGR works only on machines which follow ieee-754
> *  floating-point standard in their handling of infinities and NaNs.
> *  Normal execution of ZSTEGR may create NaNs and infinities and hence
> *  may abort due to a floating point exception in environments which
> *  do not conform to the ieee standard.

I do not have any problems with library routines using IEEE-754
conformant floating point arithmetic to its extreme.

What I *do* object to is people trying to mimic that in Fortran, hoping
that the Fortran Standard conformant evaluation of expressions and
IEEE-754 requirements don't clash.

There's a reason the F2K standard is trying to accommodate IEEE-754
compliant floating point arithmetic.

> Mr. Burley's defaults (-mieee and appropriate-width spills) are the
> only sane ones possible for a general-purpose compiler to be used by
> anyone.  People shouldn't have to read every LAWN and ask every FP
> expert to determine what their system's particular quirks are.

They do not need to as long as they do not try to walk the bleeding
edge.

Our floating point intensive code works on several variaties of floating
point hardware *because* we do not try to walk on the bleeding edge.

> BTW, the Alpha-denorm non-implementation caused major problems
> with testing and using ScaLAPACK on a multi-architecture PVM
> machine, anecdotally from the authors.  The old argument about
> denorms is covered in
> http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.html .
> Interesting reading.

I will read that.  Thanks for the pointer.

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[Mike Stump]

Guys, guys...  In the end, the gcc folks get to pick what front end
they want, and they should be willing to respect the wishes of the
frontend folks to a large degree...  The frontend folks need to agree
among themselves what the `right' behavior is.  If they cannot agree,
usually there is a natural pecking order that can resolve the issue.
If people feel strongly, they can walk as a vote of no confidence.  I
would hope that even if they do this, that they can find a way to
offer up the deaults that they care about, try it out on the community
for a year or so with a release of their own package, see if users
flock to their compiler or the other compiler...

In the end, it is about great software that users like and respect.

Personally I don't have much of an opinion if gofast or becorrect
should be the default.

Re: Bug with g77 and -mieee on Alpha Linux

[Edward Jason Riedy]

And Toon Moene writes:
 - 
 - My point is that every physicist engaging in numerical analysis will
 - choose his/her units to get this effect.  This is not surprising, as
 - people like to talk about the major phenomena they study using small
 - numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
 - measure distance.

Please go to http://www.cise.ufl.edu/~davis/sparse/ and support 
the claim that all of these matrices (most from real problems, 
many from industry) never run into floating-point problems and 
always live happily in your world of `just-right' numbers when 
being factored or when used in an iterative solver.

You can't, because they don't.  Now, they can give useful solutions
using less precision in most places and more in a few, key places 
(see papers by Demmel and Li on parallelizing SuperLU), but that's a 
different story.  

Likewise, from a new eigenvalue routine in LAPACK, a large algorithmic 
improvement:
*  Note 3 : ZSTEGR works only on machines which follow ieee-754
*  floating-point standard in their handling of infinities and NaNs.
*  Normal execution of ZSTEGR may create NaNs and infinities and hence
*  may abort due to a floating point exception in environments which
*  do not conform to the ieee standard.

[No, this doesn't directly apply to denorms, but it does apply to
numbers not in your nice range, even when they were before calling
this routine.]

Mr. Burley's defaults (-mieee and appropriate-width spills) are the
only sane ones possible for a general-purpose compiler to be used by
anyone.  People shouldn't have to read every LAWN and ask every FP
expert to determine what their system's particular quirks are.

BTW, the Alpha-denorm non-implementation caused major problems 
with testing and using ScaLAPACK on a multi-architecture PVM 
machine, anecdotally from the authors.  The old argument about 
denorms is covered in 
http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.html .
Interesting reading.  And IA-64 seems to get things right in spec, 
although generating the `best' division code sequence is going to 
be fun.  Intel's compiler currently only uses one sequence, according 
to a recent presentation.  The support for funky loop unrolling has 
some applications in getting boundary FP exceptions and tests right 
without acrobatics, too.

Jason

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >> Perhaps, but, as I pointed out earlier, almost *any* program can generate
> >> denormals if it is compiled to randomly spill 80-bit values as 32-bit
> >> or 64-bit values and tries to compare them to (nearly) identically
> >> computed results to see if they're approximately equal.

> >I'm pretty sure that ours don't - in spite of the fact that they perform
> >somewhere between 10^10 and 10^11 floating point operations per run.

> But *yours* isn't the only code on the planet, correct?

No, but our code runs 7 x 24 hours in 7 operational meteorological sites
in Europe - if I knew how, I would immediately enable
"trap-on-denormals", like we would run with -fbounds-check [too large a
performance dip for operational use] on, because it would point us at
problems in our algorithms.

> I mean, c'mon, you *constantly* write about how people *shouldn't* (or
> "don't", despite substantial evidence on these lists to the contrary)
> write floating-point code, as if it applied to the entire planet's
> past, present, *and* future use of Fortran.

The evidence on these lists prove that people are still forced to write
programs without getting a decent course in numerical analysis.

Why do you think I quote "Numerical Recipes" ?  This stuff is so well
known that it is equivalent to pointing out an error in gcc wrt to the
Dragon Book.

Why do you think I put a $200 / hour premium on this ?  If people are
forced to deal with badly written software in such large IT firms like
Siemens, they'd better be prepared to pay up, or shut up.

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>> Perhaps, but, as I pointed out earlier, almost *any* program can generate
>> denormals if it is compiled to randomly spill 80-bit values as 32-bit
>> or 64-bit values and tries to compare them to (nearly) identically
>> computed results to see if they're approximately equal.
>
>I'm pretty sure that ours don't - in spite of the fact that they perform
>somewhere between 10^10 and 10^11 floating point operations per run.

But *yours* isn't the only code on the planet, correct?

I mean, c'mon, you *constantly* write about how people *shouldn't* (or
"don't", despite substantial evidence on these lists to the contrary)
write floating-point code, as if it applied to the entire planet's
past, present, *and* future use of Fortran.

And now you're saying "but *we* don't write code that way"?

Again, I don't know where *anyone* can find freely available information
on how they *are* supposed to write floating-point code.  I believe
such information exists as it pertains to *other* compilers, which
behave in a more predictable fashion (e.g. we've had reports that at
least some do 80-bit spills on IA32), but not as it pertains to g77.

>Why ?  Because in the problem domain we're operating in, floating point
>numbers have values that are far from FLT_MAX and FLT_MIN.  In our case
>that is because if you express the physical quantities in the earthly
>atmosphere in SI units, you *get* values that are far from these
>extremes.

How nice...for *you*.

>My point is that every physicist engaging in numerical analysis will
>choose his/her units to get this effect.  This is not surprising, as
>people like to talk about the major phenomena they study using small
>numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
>measure distance.
>
>The (automatic) side effect of this "change of units" is that the
>numerical algorithms used to describe those phenomena will stay out of
>the complexities of having to deal with denormals *unless they are
>unstable or operating outside their domain of validity*.
>
>Of course one could always set up a scenario where, using the fact that
>you cannot know whether a floating point value is 80-, 64- or 32-bits
>wide, you can generate small differences (which should have been zero)
>that, suitably multiplied which each other, will generate a denormal
>(especially if you start close to FLT_MIN).  My point is that you have
>to cleverly set this up, because it will not show up in normal practice.

I am rapidly coming to the conclusion that all of my efforts on g77
are, according to your logic, legitimately applicable to only about
50 truly expert programmers (in floating-point arithmetic) worldwide.

That might explain the stunning lack of breadth in funding I've received,
especially as this "beta" version of g77 has matured.

Needless to say, I'm *also* rapidly losing interest in investing much
more of my time making g77 even *more* attractive to programmers who,
according to you, will be writing incorrect programs, especially since
they seem to have no reasonable way to find out how to write *correct*
ones, and since *I* am nowhere near having sufficient knowledge to
either document this *or* implement and maintain a compiler (which includes
responding to bug reports) that is intended for use in this domain.

So I believe I'm approaching a choice:

  -  Mandate that g77 shall, as of the rewrite (0.6), default to precise
     (e.g. 80-bit) spills of intermediate computations, to full-range support
     of the underlying floating-point *type* (-mieee), etc., even if that
     means effectively forking gcc or writing a new compiler from scratch
     (neither of which I'm likely to undertake myself as part of doing
     the rewrite, of course).  By "mandate" I mean "refuse to work on,
     maintain, or respond to bug reports concerning any product that
     incorporates the 0.6 front end but doesn't meet my standards".  Not
     quite the DJ Bernstein approach (for qmail anyway), but one that's
     entirely reasonable, I believe.

  -  Leave working on g77 to those who actually understand how to write
     code (and therefore maintain compilers) that carefully navigates the
     murky waters left by *not* choosing defaults based on
     my understanding of what it takes to achieve overall robustness.

That means, it is quite likely that the only way I'm likely to believe I
can truly contribute effectively to g77 *and its user base* (beyond a tiny
handful of experts, anyway) is if I can convince the GCC steering committee
to mandate 80-bit spills, -mieee as a default, etc.  Which seems unlikely,
though, at least, while working on *that*, I can proceed with the rewrite,
then effectively abandon its incarnation as a gcc product when failure
to convince becomes clear, and offer it (since it is GPL'ed software) along
with my (limited) services to the community to put into a new compiler
project that meets the requirements I believe the *larger* number-crunching
community has, or which, at least, I can *explain* without tying myself
into knots.  (Whether such an offer is taken up, I could not control,
of course.)

The choice is rather stark simply because, the more I read about your
definition of what constitutes proper floating-point programming, the
*more* confused and, frankly, "frightened" I get, regarding the viability
of actually successfully understanding it or getting other people to
understand it sufficiently to write robust code to your standard.

Whereas, throughout my life, the more I've read about stuff like designing
compilers, the *less* confused and worried I've gotten regarding the
viability of *that*.  Which certainly strongly suggests where I should
devote my efforts (e.g. continue compiler development, avoid Fortran
or other loosely-specified languages used for floating-point work).

For example, ANSI FORTRAN 77 says "The value of a [variable] does not change
until the [variable] [is changed in a way clearly visible in the source
code]".

Now, putting aside any language-lawyer hat I might otherwise be willing to
wear ("but can a value be an *approximation*, the *precise* value of which
is undefined at any given point during execution?"), it seems to me that
means, quite clearly to any *typical* reader of the standard, that

  Y = 1.
  Z = 3.
  X = Y / Z
  IF (X .LT. X) PRINT *, 'FAIL'
  END

must *never*, under *any* circumstances, print "FAIL", because X can have
only *one* value, and the .LT. operator cannot reasonably be interpreted
(again, even if the standard can be *argued* to allow it) to compare
*approximations* (two *different* ones) of the *one* value in a variable
being compared to itself.

But, according to your interpretation (from last December, IIRC), the
compiler may, at any time and in any way of its choosing, choose to
use excess precision to compute results of computations *and* randomly
chop those results down to less precision, so the IF statement could
be interpreted as if it read:

  IF ((X - roundoff) .LT. X) ...

Which is, in fact, something gcc can be easily convinced to do, with
perhaps only a bit more coding, these days.  (The changes necessary to
provoke this probably involve straight assignment.  Again, from a
language-lawyer perspective, you could perhaps argue that straight
assignment is permitted to assign yet another *approximation* -- but I'm
not even going to look that up in the standard, because, even if *allowed*,
it's too poor a quality of implementation in my view.  Of course, I'm
referring here to an assignment to the same type, since it makes sense
that conversion, especially to a less-precise type, involves another
approximation.)

I find that increasingly unacceptable, because I can't explain to programmers
how to avoid it, especially without creating the sorts of denormals (very-
low-magnitude values) you say code should never create.

Now, your *new* justification appears to be that no floating-point
programs worth their salt ever compute values anywhere *near* the
boundaries of denormals, Infs (which I guess crash on Alphas compiled
using the defaults), etc., so this can "never happen".

Problems with *that* logic include:

  -  The fact that we don't, anywhere I can see, document that programmers
     must stay away from those limits (despite widespread advertising of
     the "supported range and precision" of the underlying types by
     hardware manufacturers), which, to encourage portability, must
     necessarily represent the *intersection* of all pertinent limits on
     all machines.

  -  The fact that we don't offer any option (let's call it `-fcheck-toon'
     ;-) programmers can use to more cleanly catch any computations their
     code does resulting in these "inappropriate" values, leaving it to
     the underlying implementation to crash, burn, or simply compute
     incorrect results due (at least partly) to the defaults we've chosen
     for how to compile code.  And since we have no real spec on those
     limits, `-fcheck-toon' seems to me almost as hard to implement as, say,
     `-fcheck-whether-my-code-does-what-I-want'.

  -  The fact that there's no clear, concise, industry-wide spec on
     programming to *these* limits.  (There is one on programming to
     IEEE 754 limits: it's called IEEE 754.  But we appear to be
     inventing our own standard, so we can't just tell people to use
     *that* standard, even if they limit their use of g77/gcc to
     machines that implement it as a native, or nearly native, type.)

  -  The fact that we are unlikely to ever be able to offer high-speed
     computation of an extended type that offers denotations of out-of-
     range values ("too small", "too large", etc.) for code that wants to
     quickly compute intermediate results that it might never use in
     final calculations, but can't effectively filter out, or which it
     might want to explicitly test for at run time.  (IEEE 754, of course,
     offers this -- via concepts like single and double precision,
     denormals, Infs, and NaNs.)

  -  The fact that, because we (the whole gcc project, driven by your
     view of what constitutes proper floating-point programming, which
     surely must pertain to other languages as well as Fortran) are marching
     to our own drummer, we have to modify all libraries that might be
     "downstream" of our product (the libm exp()) example to assure that
     they "behave" according to our "specification" (e.g. when computing
     results in an underflow, silently replace with zero, but if -fcheck-toon
     is specified, crash instead).  So we can't just grab a bunch of
     well-written, highly optimized IEEE-754-conforming code to implement
     underlying routines -- we have to, in effect, grow our own, to meet
     the performance needs of the types we're creating.

  -  The fact that, AFAICT, we "advertise" values like FLT_MIN and FLT_MAX
     which are, to some extent, outright *lies* in terms of what we
     actually support.  At least, FLT_MIN, anyway -- you cannot reliably
     compute down to near FLT_MIN without the compiler randomly computing
     values much smaller for you and then deciding to crash on them (again,
     this is based on *your* claims of what is permissible for the compiler
     over the past year or so).  But I don't see you, or those who appear
     to agree with you, proposing new values for FLT_MIN and FLT_MAX, or
     proposing values for new macros called something like FLT_TOON_MIN
     and FLT_TOON_MAX.  So programmers have to somehow just "know" what
     this magic range is, but are not able to actually look it up
     anywhere convenient.

Further, you appear to not particularly care how a program behaves, compiled
by default, when it computes values you consider out of range.

It was formerly suggested (by you or someone else) that *crashing* when
computing denormals was okay, because that meant the program almost
certainly had a bug.  But now we discover we likely *won't* crash,
but rather silently generate a zero, on a denormal, even though, on
most *other* targets on which we'll generate the correct (IEEE) result.

Yet that (*not* crashing) *now* seems to be cause for *celebration* among
those who claim denormals "never happen anyway" in correct code -- so
*catching* bugs seems to have not been an issue in the first place, as
whether the code crashed or computed incorrect results (admittedly from
inputs declared, by you, to be incorrect) turns out to not have mattered
at all.

So, it seems to me likely that, right now, on Alphas (and again I can't
test this), there are discontinuities in important functions, such as
reading in a floating-point number expressed as decimal.  E.g. on my
Pentium II, if I do the equivalent of "READ (UNIT='1E-5000', *) R",
the value that ends up in R is 0., with no crash.  Apparently, according
to you, it's okay if that also happens on the Alpha, even if changing
the input value to 1E-40 causes a crash (since it generates a denormal),
and even though changing it to 1E-5 produces the correct result (no crash).

But I consider that sort of discontinuity (zero -> crash -> correct value)
to be *wholly* unacceptable, whether it occurs in the inputs to the
text-to-FP converter, or in the inputs to exp(), or whatever.

Now, maybe all that's needed is for these few discontinuities to be
*fixed*, but, from your "camp", so to speak, I see little interest
in doing so, *or* in specifying exactly what the proper behaviors
*should* be so the *rest* of us can know how to fix them.  (In particular,
as far as I can tell, the specification is "anything goes, so it can be
as fast as the hardware can possibly run", which clearly allows for
discontinuous behavior across the domain -- except, of course, in the
at-best-fuzzily-specified region *you* claim is the only one programs
should be employing anyway.  And there's no point in "fixing" such
discontinuities if the "standard" is to go as fast as possible, completely
ignoring the possibility of ill-conditioned input resulting in
discontinuous behaviors.)

So, I'm stumped as to how to what path to take.  I want to work only
on *robustly designed* products, which include taking into account
how *typical* users will use it in practice, and assuring they'll not
get misleading or incorrect results.

Now, I'm not particularly interested in working on something like Java,
which offers the (IMO false) promise of "write once, run anywhere", even
though I'm sure I'd find their discussions of issues revolving around
IEEE 754, performance, and consistent behavior quite valuable in many
ways.

But, I'd like to work on a product that *at least* respects the underlying
floating-point types of the machine and *properly* implements them.  That
means respecting all the expertise that not only went into *designing* those
types but has since gone into *using* them.  If those underlying types are
IEEE 754, so be it.  It also means implementing consistent behavior across
all the intrinsics, conversions, approximations, etc.  It also means doing
80-bit spills on IA32, since that's what the *designer* of the hardware
intended (and, again, it's what the Fortran standard *clearly*, if not
*legalistically*, requires).  (It certainly means, at least, 80-bit spills
of program variables; I believe it might also mean 80-bit spills of
intermediate results.  Of course, there's the argument that what is
*really* means is always computing 32-bit or 64-bit intermediate results,
leaving IA32 performance out in the cold.)

This is all considered "quality of implementation" in the Fortran community,
I'm happy to concede, but my point is, if I can't (and I know I don't)
thoroughly understand the floating-point model the compiler offers, then
I should *not* be working on it, certainly not in a significant role
(anything approaching the role I serve now vis-a-vis g77).

And, right now, I have *serious* doubts that the floating-point model
g77/gcc offers is *anywhere* near understood by *anybody*, including
yourself.  You probably don't realize that that "IF (X .LT. X)" example
above, put into a loop that nevertheless computes the *same* pertinent
values, could result in *different* paths taken via the IF, especially if the
loop contains any other code before *or* after that sample block of code.
But it could, due to loop unrolling or other sorts of code-duplication
optimizations.

(Though you probably *do* realize that adding another identical IF to
the original sample program could legitimately result in different
paths taken in each IF even in a straight-through execution of that code.)

Certainly I don't think most of the *current* g77 audience understands
those issues, and I do not have what it takes to explain them to that
audience.

My guess is, assuming we continue on this path, the *future* g77 audience
*will* understand these issues thoroughly.

But not because we've *explained* these issues to them.

It will be because the future g77 audience (and the audience for gcc's
floating-point capabilities) will consist of maybe 50 people worldwide,
or some number so small it comes *nowhere* near justifying all the effort
we're putting into this aspect of the product.

IMO, an audience that small should either spend a few hundred thousand
dollars for a high-end compiler to be installed on all their workstations,
or use assembly, because that's what the economics of their situation
dictates.

Or, they could just use -mno-ieee, -fno-correct-spills, etc., assuming
we picked *robust* defaults for g77 (and gcc).

So, Toon, what do you suggest?  Do you insist I continue to work on a
product in which I have increasingly *less* confidence, and which is
going in a direction with which I strongly disagree (much of this
"direction" being in the realm of "revelations" about how it's actually
working, and being told not to do anything about it), resulting in my
(hopefully-still-at-least-marginally-good) name being associated with
something which I wouldn't recommend anyone actually *use*?

Or, should I work on something else, something in which I *do* have
confidence can become something which I believe is worthy of
recommendation to others to use?

(These questions are as open to others as they are to Toon, and they
should be considered as questions *other* people working on g77/gcc
might be asking as well.  E.g. maybe Toon would be unwilling to work on
g77 if it didn't have the defaults *he* wants -- after all, convincing
floating-point experts to compile with -mno-ieee might seem, to him
anyway, quite difficult.)

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

craig@jcb-sc.com wrote:

> >Richard Henderson wrote:

No, you can't hold Richard responsible for what I wrote ;-)

> >Of course, I *still* think that a program generating denormals (and
> >getting underflow to zero) is wrong:  All of a sudden there's a number
> >in the algorithm that's treated as "normal" (while it is _de_normal) -
> >and now you can't divide by it.
> 
> Perhaps, but, as I pointed out earlier, almost *any* program can generate
> denormals if it is compiled to randomly spill 80-bit values as 32-bit
> or 64-bit values and tries to compare them to (nearly) identically
> computed results to see if they're approximately equal.

I'm pretty sure that ours don't - in spite of the fact that they perform
somewhere between 10^10 and 10^11 floating point operations per run.

Why ?  Because in the problem domain we're operating in, floating point
numbers have values that are far from FLT_MAX and FLT_MIN.  In our case
that is because if you express the physical quantities in the earthly
atmosphere in SI units, you *get* values that are far from these
extremes.

My point is that every physicist engaging in numerical analysis will
choose his/her units to get this effect.  This is not surprising, as
people like to talk about the major phenomena they study using small
numbers.  So a cosmologist uses Mega- or Giga-parsecs, not meters, to
measure distance.

The (automatic) side effect of this "change of units" is that the
numerical algorithms used to describe those phenomena will stay out of
the complexities of having to deal with denormals *unless they are
unstable or operating outside their domain of validity*.

Of course one could always set up a scenario where, using the fact that
you cannot know whether a floating point value is 80-, 64- or 32-bits
wide, you can generate small differences (which should have been zero)
that, suitably multiplied which each other, will generate a denormal
(especially if you start close to FLT_MIN).  My point is that you have
to cleverly set this up, because it will not show up in normal practice.

Cheers,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>Richard Henderson wrote:
>
>Of course, I *still* think that a program generating denormals (and
>getting underflow to zero) is wrong:  All of a sudden there's a number
>in the algorithm that's treated as "normal" (while it is _de_normal) -
>and now you can't divide by it.

Perhaps, but, as I pointed out earlier, almost *any* program can generate
denormals if it is compiled to randomly spill 80-bit values as 32-bit
or 64-bit values and tries to compare them to (nearly) identically
computed results to see if they're approximately equal.

In other words, *I* don't see how you can reconcile your statements that
programs generating denormals are wrong *and* that programs must expect
that comparing mathematically identical results can produce values
that are *smaller* (in magnitude) than can be represented by the declared
types of the operands (as is the case for allowing arbitrary truncating
of values computed by the processor to more precision).

That doesn't mean you *can't* reconcile it, of course -- I just don't
see how, and would love for you to explain it (to all of us), especially
so we can put it into the g77 docs.  (The only explanation I can think
of is "you can trust that, if the processor [compiler + libs + machine]
uses excess precision to compute intermediate results, it will never
crash using the resulting denormals as operands in subsequent operations",
which seems rather strained, but not entirely implausible.)

For now, it seems, for the Alpha anyway, denormals are never "generated"
by default (modulo packaging bugs, like libm exp(), or whatever), so
that's not a problem on *that* architecture.

Perhaps we should classify *inputs* of underflows mapping to denormals,
rather than zero (or whatever *other* popular systems do when they're
given the task of converting, say, "1E-500000" to floating point), as
another of these "packaging bugs"?

Now, I still stand by my statement that -mieee *should* have been made
the default from the outset, because, given all the existing code
(like glibc or libc -- apparently at least one of these uses *unsigned
integer* arithmetic to convert between decimal and floating-point, so
it isn't *naturally* going to benefit from hardware that maps underflows
to zero rather than to denormals), it would have ported *much* more
smoothly to Alpha had -mieee been the default.

But, I'm beginning to see how it is *possible*, and relatively
*feasible*, to make having -mno-ieee the default actually *work*.

Problem is, we appear to not be at that point yet, nor do we seem to
have a complete understanding of what it takes to get there.

Further, AFAIK, we have no test suite for exactly this sort of
*partial* IEEE 754 support.  Again, that's an argument for making
-mieee the default, but *also* for, having made -mno-ieee the default,
doing the hard work to make it actually *work* -- writing a test suite,
for example, one that parallels whatever tests we have for IEEE-754
conformance but is constrained to the natural full-speed range of 21[01]64
chips.

In the meantime....

I'm beginning to think we should just tell people that gcc/g77 support
for the Alpha is in "alpha testing".  IMO, based on what I'm seeing
here, it ain't ready for prime time, though, as usual, the experts
should be able to get wonderful performance out of it.

So for 2.95 we say "Alpha support is in alpha testing, especially as
regards floating-point conformance".  That addresses the COMPLEX
(LAPACK) issue as well.

Then we decide what to do for 3.0 (or 2.96), one of:

  -  Make -mieee the default, take the performance hit, but save
     ourselves some amount of time and trouble.  Leave it to experts
     to use -mno-ieee when they want that last N%, and to tell us,
     item by item, what they need changed over time to get it working,
     but, in the meantime, most everything just *works* by default.

  -  Leave -mno-ieee as the default, figure out what it *really* takes
     to make it work (across libraries, debuggers, etc.), and make that
     happen.

>You won't believe this, but I just opened Press et al's "Numerical
>Recipes" at random:

*I* believe it, because I believe the Universe conspires to provide
"coincidences" that, if we're sufficiently alert, offer us opportunities
to avoid making mistakes.  (A sort of weirdly transformed version of
Murphy's Law: "Anything that might go wrong will be warned about
sufficiently for those who are paying attention."  ;-)

        tq vm, (burley)

Re: Bug with g77 and -mieee on Alpha Linux

[Toon Moene]

Richard Henderson wrote:

> On Thu, Jul 08, 1999 at 03:58:45PM -0000, craig@jcb-sc.com wrote:

> > Ah, well, it's good people can do that if they want, though the arguments
> > against -mieee being the default included "because it catches bugs in code",
> > which, presumably, silently mapping underflows to zero won't.

> Actually, most of the bugs are where you _start_ with denormals,
> typically because of uninitialized data.  UNZ doesn't stop trapping
> when the inputs are non-normal.

[ Thanks for reminding me that underflow always maps to zero on Alpha -
  keep forgetting that ]

Of course, I *still* think that a program generating denormals (and
getting underflow to zero) is wrong:  All of a sudden there's a number
in the algorithm that's treated as "normal" (while it is _de_normal) -
and now you can't divide by it.

I cannot glean inside Martin's nonlinear equations, but the following
description:

<QUOTE>
Perhaps, but while solving highly nonlinear sets of equations, like in
analog circuit simulators nearly anything can appear. Nobody knows,
where newton will bring you. You can decide to make smaller newton 
corrections only  a f t e r  you have the bad ones and thus you have to 
deal with these bad numbers somehow.
</QUOTE>

to *me* sounds like a grave mis-use of Newton-Raphson root finding.

You won't believe this, but I just opened Press et al's "Numerical
Recipes" at random:

Chapter 9  -  Root Finding and Nonlinear Sets of Equations

9.6 Newton-Raphson Method for Nonlinear Systems of Equations

We make an extreme, but wholly defensible, statement:  There are _no_
good, general methods for solving systems of more than one nonlinear
equation.  Furthermore, it is not hard to see why (very likely) there
_never_will_be_ any good, general methods: ...

Martin, proceed from there (pay special attention to 9.7 Globally
Convergent Methods for Nonlinear Systems of Equations).

Hope this helps,

-- 
Toon Moene (toon@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
GNU Fortran: http://world.std.com/~burley/g77.html

PS: This is getting of-topic; for more help, the rate is $200 / hour :-)

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Thu, Jul 08, 1999 at 04:08:07PM -0400, Richard Hadsell wrote:
> The problem I ran into immediately (with C++) was a call to a math
> function (e.g., exp()) that returns a denormal.  libm must have been
> compiled with -mieee, or its equivalent.  When my code is compiled the
> default way, without -mieee, the denormal eventually causes a
> floating-point exception that crashes my program.

Yes, that is a known problem on Linux.  Digital's math libraries
don't do this.


r~

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Hadsell]

Richard Henderson wrote:
> 
> Underflow to zero always ocurrs when software completion (ie -mieee)
> is not requested.  This is still useful for when you need -mieee to
> handle Inf and NaN, but don't care about denormals.

The problem I ran into immediately (with C++) was a call to a math
function (e.g., exp()) that returns a denormal.  libm must have been
compiled with -mieee, or its equivalent.  When my code is compiled the
default way, without -mieee, the denormal eventually causes a
floating-point exception that crashes my program.  Yes, it forced me to
discover and start using -mieee, but the default compilation linked with
the standard libm does not execute gracefully.

Contributors to this list like to blame the programmer for incorrect
code.  I think someone should reconsider the interaction between the
compiled code and the libm functions.  If that means that functions like
exp() should return 0 instead of denormals, it's fine with me.  If
setting a hardware register to force denormals to 0 will have the same
effect, that's also fine.  

Just provide a reasonable way to use the compiler and the math functions
in a reliable way.  At the moment, that's not the default.

-- 
Dick Hadsell			914-381-8400 x5446 Fax: 914-381-9790
Reply-to:			hadsell@blueskystudios.com
Blue Sky Studios                http://www.blueskystudios.com
1 South Road, Harrison, NY 10528

Re: Bug with g77 and -mieee on Alpha Linux

[Richard Henderson]

On Thu, Jul 08, 1999 at 03:58:45PM -0000, craig@jcb-sc.com wrote:
> >See "man ieee", and #include <machine/fpu.h>. When the program starts
> >up, make a call to ieee_set_fp_control(IEEE_MAP_UMZ); , which sets the
> >fp control to cause underflow to map to zero.

Underflow to zero always ocurrs when software completion (ie -mieee)
is not requested.  This is still useful for when you need -mieee to
handle Inf and NaN, but don't care about denormals.

> Ah, well, it's good people can do that if they want, though the arguments
> against -mieee being the default included "because it catches bugs in code",
> which, presumably, silently mapping underflows to zero won't.

Actually, most of the bugs are where you _start_ with denormals,
typically because of uninitialized data.  UNZ doesn't stop trapping
when the inputs are non-normal.

> Maybe no machine *today* has that combination of 80-bit computations
> and a "fast" IEEE mode crashing on denormals (which Toon supports as the
> default).  Though it's not hard to imagine Merced or McKinley might.

The user mode documentation for IA-64 is public.  One can see that
it supports an 80 bit mode, but it also has instruction forms that
do proper rounding to single and double precision.  Intermediate
results therefore will be properly rounded for the mode.



r~

Re: Bug with g77 and -mieee on Alpha Linux

[craig]

>See "man ieee", and #include <machine/fpu.h>. When the program starts
>up, make a call to ieee_set_fp_control(IEEE_MAP_UMZ); , which sets the
>fp control to cause underflow to map to zero.

Ah, well, it's good people can do that if they want, though the arguments
against -mieee being the default included "because it catches bugs in code",
which, presumably, silently mapping underflows to zero won't.

My bigger problems with this whole line of argument are that, first, users
who *see* the IEEE format involved *expect* the full range, and, further,
that's what "everyone" supports on their hardware, so people code to that,
often without *thinking* they are.  The penalty for denormals should not
be program crashes, by default, since IEEE isn't (normally) implemented
that way.

(I assume -- incorrectly? -- that the FLT_MIN and related macros have the
correct values vs. for when -mieee is in effect.)

Second, when I think of how this issue intersects with Toon's *other* opinion,
that 80-bit spills are a waste of time that good programmers know how to
code around, I'm not sure I see *how* they can code around *both* defaults
portably.

To wit: how can a programmer be sure his code will *never* generate a
denormal (and, e.g., compare that denormal to some other value), if the
compiler can, as Toon says it must to "run fast" on machines like the IA32
class (and m68k?), compute *some* results to 80-bit precision and *others*
to 64-bit (or 32-bit) precision on a totally random basis?

Maybe no machine *today* has that combination of 80-bit computations
and a "fast" IEEE mode crashing on denormals (which Toon supports as the
default).  Though it's not hard to imagine Merced or McKinley might.

But, in theory at least, I can't see how a programmer can write code
that does something like

  IF (R - S .LT. .00001)

to test a tolerance *without* risking that R - S evaluates to a denormal,
because, while *mathematically* identical, and *computed* using identical
operations on identical operands, the *compiler* decided to compute R to
80 bits and S to only 32 or 64.

It's not like I feel like researching how to write floating-point correctly
at this point, given all the other stuff I have to do.

I just doubt it can be done, and *especially* that any coherent document
on *how* to do it is as freely available as g77, gcc, etc.

So maybe I'm wrong, but I throw this out in case other people, more
experienced with FP programming, can tell me so, or agree that, indeed,
writing portable code -- for machines that default in all the ways Toon
(and many others; he's just someone I *know* I can trust to care about
Fortran and related issues) wants -- is basically impossible, or at least
so hard to do right that a vanishingly small percentage of programmers
who, *today*, are writing code *intended* for such portability, code
that is being, or will be, *deployed*, are doing so *correctly*.

        tq vm, (burley)