Re: What is acceptable for -ffast-math? (Was: associative law incombine)

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Wolfgang Bangerth]

Tim Hollebeek wrote:
> Rewriting the programmers program in such a way that he gets
> significantly less accuracy than he could reasonably expect is just
> silly.

Of course it is, but that is not the point that is being discussed: nobody
proposed doing these transformations at -O2. If you don't want these
transformations, then don't switch on -fsomething. But if, for some
certainly valid reason you don't want to use these transformations, why
shouldn't other people have valid reasons to request them?

Regards
  Wolfgang


-------------------------------------------------------------------------
Wolfgang Bangerth          email: wolfgang.bangerth@iwr.uni-heidelberg.de
                             www: http://gaia.iwr.uni-heidelberg.de/~wolf

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Gabriel Dos_Reis]

The transformations in question aren't documented.  So, those having
"valid reasons to request them" don't even know what there are
requesting with -fdubious-math.

Right now, we're discussing the ability of the compiler to transform
something like a/b/c into a/(b*c).  Some argue on the ground of speed
considerations, that the programmer would expect the compiler to do
that. But then, if they would have written that form in the first 
place.

The issue isn't about "freedom vs. no-freedom".

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Wolfgang Bangerth]

> The transformations in question aren't documented.  So, those having
> "valid reasons to request them" don't even know what there are
> requesting with -fdubious-math.

Well, but then we're talking about a/b/c -> a/(b*c)  "and the like", as
  a*c+b*c -> (a+b)*c
  a*x/a   -> x
and probably a whole lot of other things that are equally "dubious".


> But then, if they would have written that form in the first place.

But, as I said, there might be cases where you can't write it in an
optimized way in the first place, such as in
  inline double f() { return a/b; }
  inline double g() { return c;   }
  x = f()/g();
plus inlining. You can't just rewrite this into a form that does the
optimized computation without the help of the compiler. However, I'd bet
that for the lower parts of the compiler the resulting stream of commands
look just the same as if it were  x=a/b/c  right away.

Regards
  Wolfgang

-------------------------------------------------------------------------
Wolfgang Bangerth          email: wolfgang.bangerth@iwr.uni-heidelberg.de
                             www: http://gaia.iwr.uni-heidelberg.de/~wolf

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Gabriel Dos_Reis]

| and probably a whole lot of other things that are equally "dubious".
| 
| 
| > But then, if they would have written that form in the first place.
| 
| But, as I said, there might be cases where you can't write it in an
| optimized way in the first place, such as in
|   inline double f() { return a/b; }
|   inline double g() { return c;   }
|   x = f()/g();
| plus inlining. 

But then is 0.0 is acceptable for 0.125 for speed?

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Theodore Papadopoulo]

Gabriel.Dos_Reis@sophia.inria.fr said:
> | and probably a whole lot of other things that are equally "dubious".
> |  |  | > But then, if they would have written that form in the first
> place. |  | But, as I said, there might be cases where you can't write
> it in an | optimized way in the first place, such as in |   inline
> double f() { return a/b; } |   inline double g() { return c;   } |   x
> = f()/g(); | plus inlining. 

> But then is 0.0 is acceptable for 0.125 for speed? 

Yes, if you know by advance that this cannot happen because of your 
input data accuracy. Then you will get the speed advantage without 
the loss of accuracy since it cannot happen.


--------------------------------------------------------------------
Theodore Papadopoulo
Email: Theodore.Papadopoulo@sophia.inria.fr Tel: (33) 04 92 38 76 01
 --------------------------------------------------------------------


-- 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Exmh version 2.2 06/23/2000

iD8DBQE7aEn/IzTj8qrxOU4RAnxLAJ9fzLZKK+35iIN5MAtX6sb+OAs55QCfTbiJ
jUnXpCa+JAuS9mmYBTBb+/g=
=lUNo
-----END PGP SIGNATURE-----

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Gabriel Dos_Reis]

| Gabriel.Dos_Reis@sophia.inria.fr said:
| > | and probably a whole lot of other things that are equally "dubious".
| > |  |  | > But then, if they would have written that form in the first
| > place. |  | But, as I said, there might be cases where you can't write
| > it in an | optimized way in the first place, such as in |   inline
| > double f() { return a/b; } |   inline double g() { return c;   } |   x
| > = f()/g(); | plus inlining. 
| 
| > But then is 0.0 is acceptable for 0.125 for speed? 
| 
| Yes, if you know by advance that this cannot happen because of your 
| input data accuracy. Then you will get the speed advantage without 
| the loss of accuracy since it cannot happen.

Data for the supposed speed in such case are still missing.

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Nick Ing-Simmons]

Gabriel Dos_reis <gdosreis@sophia.inria.fr> writes:
>| Gabriel.Dos_Reis@sophia.inria.fr said:
>| > | and probably a whole lot of other things that are equally "dubious".
>| > |  |  | > But then, if they would have written that form in the first
>| > place. |  | But, as I said, there might be cases where you can't write
>| > it in an | optimized way in the first place, such as in |   inline
>| > double f() { return a/b; } |   inline double g() { return c;   } |   x
>| > = f()/g(); | plus inlining. 
>| 
>| > But then is 0.0 is acceptable for 0.125 for speed? 
>| 
>| Yes, if you know by advance that this cannot happen because of your 
>| input data accuracy. Then you will get the speed advantage without 
>| the loss of accuracy since it cannot happen.
>
>Data for the supposed speed in such case are still missing.

   Divide is very slow compared to multiply in modern hardware.
   r = a/(b*c) has one divide r = a/b/c has two.
   Divide also seldom pipelines as well.

In real-time image processing (say) failing to compute the frame in 20mS
is catastrophic, but having a pixel black (0.0) rather 
than 0.125 white (Dark gray) is of minimal impact.


>
>-- Gaby
-- 
Nick Ing-Simmons
http://www.ni-s.u-net.com/

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Per Abrahamsen]

Gabriel Dos_Reis <gdosreis@sophia.inria.fr> writes:

> But then is 0.0 is acceptable for 0.125 for speed?

Sure, whether the NO3 leaching is 0.0 g/ha/y or 0.125 g/ha/y doesn't
matter, both numbers are far below what is measurable.

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Gabriel Dos Reis]

Theodore Papadopoulo <Theodore.Papadopoulo@sophia.inria.fr> writes:

| [1  <text/plain; us-ascii (7bit)>]
| 
| 
| gdr@codesourcery.com said:
| > No, I do mean a transformation which does not dramatically transmute
| > the computations.  
| 
| That's not a definition.

Do you have one?

Changing 0.125 to 0.0 is a dramatic change.

| Dramatically, can mean very different things 
| depending on the people and on the algorithms. That's basically the 
| point that I'm trying to make. In many applications and algorithm
| the optimization a/b/c to a/(b*c) is meaningful and relevant.

In the case I showed, I was tring to compute a unit-length vector by
first scaling in order to avoid overflow (the /c part).

I'm not dismissing the fact that there are algorithms which would
benefit from optimizations.  As I said earlier, I'm a fervent
proponent of optimizations.  Not transformations which change the
computation in a very unexpected ways. 

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On 1 Aug 2001, Gabriel Dos Reis wrote:
> |
> | That's not a definition.
>
> Do you have one?
>
> Changing 0.125 to 0.0 is a dramatic change.

Not actually necessarily.

Remember: in order to get that difference you had to be operating with
values in the 2**1024 range.

Compared to those values, the difference between 0.0 and 0.125 can often
be considered truly insignificant.

What "often" means depends on what you're doing with the values, of
course.

			Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

<<The above rules, btw, only apply for exact IEEE math, and on a number of
machines you _will_ see zero for the first one.
>>

Hello! it was you who restricted the discussion to the x86. If we are allowed
to go to other machines, we have lots more examples up our sleeves :-)

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Wed, 1 Aug 2001 dewar@gnat.com wrote:
>
> <<The above rules, btw, only apply for exact IEEE math, and on a number of
> machines you _will_ see zero for the first one.
> >>
>
> Hello! it was you who restricted the discussion to the x86. If we are allowed
> to go to other machines, we have lots more examples up our sleeves :-)

Fair enough. But note how I didn't claim that you cannot get a difference
in the ULP. I _do_ claim that it is not "wrong" unless you specifically
ask for IEEE math - by virtue of pointing to lots of other machines that
do the same (including, apparently, the most modern architecture on the
planet. I don't have one, I can't test).

			Linus

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Gabriel Dos_Reis]

| On Wed, 1 Aug 2001 dewar@gnat.com wrote:
| >
| > <<The above rules, btw, only apply for exact IEEE math, and on a number of
| > machines you _will_ see zero for the first one.
| > >>
| >
| > Hello! it was you who restricted the discussion to the x86. If we are allowed
| > to go to other machines, we have lots more examples up our sleeves :-)
| 
| Fair enough. But note how I didn't claim that you cannot get a difference
| in the ULP.

0.125 vs. 0.0 isn't a difference in the ULP.

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Gabriel Dos_Reis]

| <<We're talking here of transformations which we do know able to
| drastically change the results.
| >>
| 
| Words like drastically are not particularly helpful in this discussion,
| plain "change" would be just fine.

Do you think changing 0.125 to 0.0 is not drastic?  I used
"drastically" to stress on the magnitude or error -- no other
conotation was implied. 

| THe issue is whether the "change" is worth the speed up.

I do fully understand that.

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Wed, 1 Aug 2001, Gabriel Dos_Reis wrote:
>
> Do you think changing 0.125 to 0.0 is not drastic?

Ehh. It's 0.125 to 0.125.

Try it yourself.

That's not a "drastic difference", not if you actually do what I
explained. If you don't have access to a x86 machine, maybe you can ask
somebody else to test the following appended for you.

Surprise, surprise, they come out exactly the same. Even down to the bit
representation (in case you care).

Not that I'm arguing that they will _always_ be the same down to the ULP,
but I _am_ arguing that at least on x86 it's actually a safe optimization.
You should disable it for -mieee.

		Linus

----
#include <math.h>

void test(double, double, double);

main()
{
        test(0.5*DBL_MAX, 2*sqrt(DBL_MAX), 2*sqrt(DBL_MAX));
}

void test(double a, double b, double c)
{
        printf("%f\n%f\n", a/b/c, a/(b*c));
}

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Gabriel Dos_Reis]

| On Wed, 1 Aug 2001, Gabriel Dos_Reis wrote:
| >
| > Do you think changing 0.125 to 0.0 is not drastic?
| 
| Ehh. It's 0.125 to 0.125.
| 
| Try it yourself.

I ran an actual program on sparc-v8/solaris-2.7.  That machine is
known to operate with a 64-bits for double -- I specically mentionned
that.  Just try it yourself. 

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

a/2.0/2.0 will never yield zero if a is non-zero
a/4.0 may indeed yield zero if a is the smallest number

Indeed if one saw a/2.0/2.0 that's pretty peculiar, and it would not be
unreasonable to guess that the properly of not yielding zero was being
exploited :-)

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Wed, 1 Aug 2001 dewar@gnat.com wrote:
>
> a/2.0/2.0 will never yield zero if a is non-zero
> a/4.0 may indeed yield zero if a is the smallest number

Didn't you already agree that FTZ was acceptable even for a default mode,
much less -ffast-math?

The above rules, btw, only apply for exact IEEE math, and on a number of
machines you _will_ see zero for the first one.

So I don't think your example is a valid argument for not using the
optimization - even without -ffast-math.

			Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Tim Hollebeek]

> While I agree that by default the compiler must follow "good rules", I 
> think that always disallowing optimisations which may lose some bits of 
> accuracy (when specified with a option switch of course) just because 
> it may hurt someone, somewhere, sometime is a little too strong.
> Full safety is IMHO impossible with floating point... Only the 
> programmer knows the accuracy of its input data (Linus called that 
> noise) and the accuracy he expects on its result.

Exactly ... and it isn't at all rare that the programmer knows exactly
how much noise is in the input data (little or none) and the how much
accuracy he expects in the result (as much as possible!).

Rewriting the programmers program in such a way that he gets
significantly less accuracy than he could reasonably expect is just
silly.

Noise is a red herring.  There is no reason why computation should
introduce more innaccuracy than they need to, especially in a ad hoc
way that is beyond the programmer's control.

-Tim

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Wed, 1 Aug 2001, Tim Hollebeek wrote:
>
> Rewriting the programmers program in such a way that he gets
> significantly less accuracy than he could reasonably expect is just
> silly.

Ehh.. "significantly less accuracy"?

Now, show me a case where a/b/c -> a/(b*c) is significantly less accurate?
Let's confine this to x86 - it is the most common case by far, and hey,
especially when it comes to games etc I doubt it matters what HP-PA does,
for example. We can easily make the optimizations be enabled on a
per-architecture basis.

The one obvious case is (b*c) overflows and turns into Inf, at which point
the end result will be +-0 with the optimization. Now, in order for the
original to _not_ have been close to zero, we're talking about 'a' being
at the very limits of the number space. And remember that on x86 register
arithmetic - regardless of precision setting - the exponent is 15 bits. So
in order to get the overflow, AND to get a original value that wasn't zero
(ie to get an error), 'a' would have had to be on the order of

	2**(16383-64)

which won't even _fit_ in a double anyway (the -64 is to take even
denormals into account, but others have said that flush-to-zero would have
been acceptable anyway by default). Did I mis-do my calculations.

(Repeat argument for underflow).

So the overflow case actually doesn't exist on x86 if we keep the values
in registers or flush to stack in extended real format (and if we do NOT
flush to stack in extended real format, x86 has other problems like the
fact that the results depend on register flushes etc).

Do we have other problems with the conversion? Sure, it might be 1ULP off
even in the non-overflow case, but as we've seen others do that
optimization by default, never mind even with -ffast-math.

So tell me, what makes the optimization so dangerous? It doesn't seem to
do anything that we don't already do.

Did I miss some noteworthy case?

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Gabriel Dos_Reis]

| Now, show me a case where a/b/c -> a/(b*c) is significantly less accurate?

On a 64-bit double processor just take 

	b = c = 2 * sqrt(DBL_MAX);
	a = 0.5 * DBL_MAX

a/(b*c) is 0.0 whereas a/b/c yields 0.125.

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Wed, 1 Aug 2001, Gabriel Dos_Reis wrote:
>
> | Now, show me a case where a/b/c -> a/(b*c) is significantly less accurate?
>
> On a 64-bit double processor just take

Oooh.. Quoting me out-of-context.

Please bother to read the next line in my email and not snip it off.

Your arguments are getting weaker and weaker, and dishonest to boot.

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Gabriel Dos_Reis]

| On Wed, 1 Aug 2001, Gabriel Dos_Reis wrote:
| >
| > | Now, show me a case where a/b/c -> a/(b*c) is significantly less accurate?
| >
| > On a 64-bit double processor just take
| 
| Oooh.. Quoting me out-of-context.

You requested an example. I gave one.

Then for your the purpose of your rhetorics, you said 

   Let's confine this to x86 - it is the most common case by far, and hey,
   especially when it comes to games etc I doubt it matters what HP-PA does,
   for example. We can easily make the optimizations be enabled on a
   per-architecture basis.

I gave a pretty much fair example to illustrate the point that the
transformation does affect the result on 64-bit double processors
example -- they are quite common.

Be honest and admit that you're getting to the point of requesting a
feature for a very specific target where you know the transformation
is safe.  Which is pretty much different from the previous "it won't
matter" stance.

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

I still see *no* quantitative data showing that the transformation most
discussed here (associative redistribution) is effective in improving
performance.

Linus seems to be getting a little emotional in this discussion but swearing
does not replace data.

My viewpoint, which I will repeat briefly is that we have to look at each
proposed transformation individually, and determine, by quantitative
measurement (many of these measurements are in the literature) how great
the gain is, and then the gain has to be traded off against the effect of
the transformation.

This seems such a reasonable position that I really have a heck of a time
understanding why Linus is so incensed by it.

Note that the requirement that there be a significant gain is a gate. If a
given transformation cannot be shown to be an effective contributor to
performance, then we don't need to discuss the effect of the transformation.
I suspect that in practice, many of the proposed transformations simply won't
get past this gate, and that thus a lot of hot air is expended uselessly 
arguing about the second stage.

In the case that a given transformation *is* effective, then we can discuss.

Let's take an example, which is multiplication by a reciprocal. This of
course does give "wrong" results. But division is often quite slow. On the
other hand, I can't imagine floating-point programmers, including those
doing game stuff, not knowing this, and thus writing the multiplication
in the first place. 

<<Oh, round-to-zero is definitely acceptable in the world of "who cares
about IEEE, we want fast math, and we'll use fixed arithmetic if the FP
code is too slow".
>>

That's probably a bad idea, on most modern machines, full IEEE floating-point
is faster than integer arithmetic (particularly in the case of multiplication)

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Tue, 31 Jul 2001 dewar@gnat.com wrote:
>
> I still see *no* quantitative data showing that the transformation most
> discussed here (associative redistribution) is effective in improving
> performance.

Actually, the transformation that was discussed here was really

	a/b/c -> a/(b*c)

and I don't know where the associative redistribution thing really came
from. Probably just an example of another thing like it..

> Linus seems to be getting a little emotional in this discussion but swearing
> does not replace data.

Hey, I called people silly, not <censored>. You must have a very low
tolerance ;)

And the replacement-with-reciprocal kind of thing can certainly make a
difference. I'm too lazy to dig out my old quake3 source CD to try to dig
up what I ended up doing there, though.

> This seems such a reasonable position that I really have a heck of a time
> understanding why Linus is so incensed by it.

I'm incensed by people who think that the current -ffast-math already does
something that is positively EVIL - even though the current setup with
inline replacements of "fsin" etc has been there for years.

> Let's take an example, which is multiplication by a reciprocal. This of
> course does give "wrong" results. But division is often quite slow. On the
> other hand, I can't imagine floating-point programmers, including those
> doing game stuff, not knowing this, and thus writing the multiplication
> in the first place.

Actually, in the case of quake3, I distinctly remember how the x86 version
actually used division on purpose, because the x86 code was hand-tuned
assembler and they had counted cycles (for a Pentium machine) where the
division overlapped the other arithmetic.

The x86 hand-tuned stuff also took care to fit inside the FP stack etc -
they had started off with a C version and compiled that, and then once
they got as far as they could they just extracted the asm and did the rest
by hand.

It was also the much more obvious algorithm to do a division.

On alpha (and hey, this was before the 21264), the same did not hold true,
and it was faster to do a reciprocal. But it's been too long.

> <<Oh, round-to-zero is definitely acceptable in the world of "who cares
> about IEEE, we want fast math, and we'll use fixed arithmetic if the FP
> code is too slow".
> >>
>
> That's probably a bad idea, on most modern machines, full IEEE floating-point
> is faster than integer arithmetic (particularly in the case of multiplication)

Ehh.. You seem to think that "hardware == fast".

Denormal handling on x86 is done in microcode as a internal trap in the
pipeline. Round-to-zero is noticeably faster - even when the _code_ is the
same. Which is why Intel has special mode flags - where it internally
rounds to zero instead of making a denormal.

I'm not kidding. Look at the "Denormals are zero flag", bit 6 in the
MXCSR.

And I quote (all typos are probably mine):

 "The denormals-are-zero mode is not compatible with IEEE Standard 754.
  The denormals-are-zeros mode is provided to improve processor
  performance for applications such as streaming media processing, where
  rounding a denormal to zero does not appreciably affect the quality of
  the procesed data"

What I'm saying is that a buge company like Intel chose to do this
optimization IN HARDWARE because they thought that (a) it was meaningful
and (b) they thought the problem warranted it.

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

> The _only_ person who can judge whether it's wrong to do so it the person
> who wrote the program. He may be happy with the faster code. He's almost
> certainly happy if he used "-ffast-math".

Well it sure would be nice to here from some of these mythical numerical
programmers (I don't care if they are writing games or nuclear reactor codes)
who would be happier, so far we haven't heard this! And in my experience,
even quite inexperienced floating-point numerical programmers are very
disturbed when optimization changes the results of their programs.

> Your arguments about "numerical computation" are just silly, as you don't
> seem to realize that there are tons of problems where your theoretical
> issues are nothing more than noise.

If you think the arguments are silly, then I really fear you lack the full
context for this discussion, a discussion that has, as you should know raged
for well over thirty years.

> Go back and look at why it was added. Right - exactly so that people could
> get _fast_ math, when they do not care about the inexactness that isn't
> even an issue for most people.

Sure -ffast-math is precisely intended to allow transformations that would
not otherwise be allowed (let's not call them optimizations, that's just
too controversial a word in the context of this argument).

The question is what is the boundary of allowable transformations. No one
agrees that there should be no boundaries (even you don't like the change
results to zero, though note that abandoning denormals has exactly this
effect, and might be considered acceptable).

So, what is the boundary, can one for instance forget about denormals and
flush to zero to save a bit of time, can one truncate instead of round,
can one ignore negative zeroes, or infinity semantics, can one ignore
intermediate overflow (note: nearly all the discussed transformations are
implicitly saying yes to this last question).

I have not seen anyone writing from the point of view of serious numerical
coding saying "sure, go ahead and transform a*b + a*c, I write that all
the time and would be happy if the compiler would speed it up in the
obvious manner". So there is real doubt as to whether this particular
transformation (which can introduce overflow where none existed before)
is one that we can agree to include.

It seems to me that the criteria for including a transformation are

  a) it is well understood and clear
  b) it really makes a difference in performance
  c) it does not introduce surprises

Unfortunately, these are often in conflict. For instance, in C, we usually
degrade the precision of the x86 floating-point unit to minimize surprises,
even though operating in full precision would be more efficient in some
cases. Should -ffast-math allow full precision operation? I would think so,
since it definitely improves performance, and reduces surprises.

Similarly, I would go for ignoring accuracy in negative zero handling, since
that so clearly meets criteria a) and c). 

But really it would be very valuable to have quantitative data that show that
a particular optimization is worth while. I will say again that the particular
case that has been discussed, which is associative law redistribution, has not
carried that burden for me AT ALL. It is a dubious transformation, violating
condition c) above, and you can see that it is dubious, given the reaction of
many, not just me, on this list. Furthermore, promoting something like this
gives an impression to serious numerical analysts that "those GCC guys" simply
don't understand the requirements of floating-point arithmetic, and that's a
cost one does not want to freely pay unless you are sure you are getting 
something in return.

We can't look to the standard here, because the question is one of taste. 
Even in Ada, in relaxed mode (the equivalent of -ffast-math, but acknowledged
and defined somewhat in the stadard), the standard leaves these kind of
decisions to the implementor. However, I cannot imagine any Ada implementor
being as cavalier as you would suggest.

By the way, I said I would be shocked to find a Fortran compiler that did
associative redistribution in the absence of parens. I am somewhat surprised
that no one stepped forward with a counter-example, but I suspect in fact that
there may not be any shocking Fortran implementations around.

It is an old argument, the one that says that fpt is approximate, so why bother
to be persnickety about it. Seymour Cray always tool this viewpoint, and it
did not bother him that 81.0/3.0 did not give exactly 27.0 on the CDC 6000
class machines.

But the advent, and essentially universal adoption, of IEEE arithmetic, really
signifies that Seymour's viewpoint (and yours in this discussion) have lost the
argument and that indeed we want floating-point to be well behaved even though
we know many programmers won't know what this means. Our experience is that
even these inexperienced programmers gain from a better floating-point model,
and that, surprise! the cost in hardware to give such a model turns out to be
minimal.

Now compiler writers always get carried away with the supposed importance
of miscellaneous optimizations, and every compiler writer has the experience
of working hard on some optimization, only to find that it has little or
no effect.

If indeed we could show that associative redistribution had a large effect
on important programs (be they games, SPEC tests, or nuclear reactor codes),
then we would have a different situation from the hardware folks. But I suspect
that careful measurement would show that we are in the same boat, and that the
cost of providing decent floating-point semantics is too small for it to be
worth worrying about very much.


Robert

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Tue, 31 Jul 2001 dewar@gnat.com wrote:
>
> Well it sure would be nice to here from some of these mythical numerical
> programmers (I don't care if they are writing games or nuclear reactor codes)
> who would be happier, so far we haven't heard this! And in my experience,
> even quite inexperienced floating-point numerical programmers are very
> disturbed when optimization changes the results of their programs.

I used -ffast-math myself, when I worked on the quake3 port to Linux (it's
been five years, how time flies).

It didn't make much difference at that point, because the x86 had
hand-written assembly, and gcc for the alpha didn't do much (anything?)
with -ffast-math.

But I tried _everything_. The main FP work I did on that thing on the
alpha improved the framerate by about 50% on alpha - FP was _that_
critical for it.  Most of it was by looking at gcc output and trying to
re-organize the C code to make it be better (because gcc didn't do much on
its own).

And yes, it was exactly things like multiplying by reciprocals.

> > Your arguments about "numerical computation" are just silly, as you don't
> > seem to realize that there are tons of problems where your theoretical
> > issues are nothing more than noise.
>
> If you think the arguments are silly, then I really fear you lack the full
> context for this discussion, a discussion that has, as you should know raged
> for well over thirty years.

Most of the kind of "numercial" work that you seem to be talking about has
probably rather little to do with FP performance. Most of the traditional
heavy FP code tends to be _much_ more about cache layout and good memory
access patterns.

I'm personally aquainted with tryign to make a game engine go fast, where
the memory effects are fewer, and the FP itself is the bottleneck.

> Sure -ffast-math is precisely intended to allow transformations that would
> not otherwise be allowed (let's not call them optimizations, that's just
> too controversial a word in the context of this argument).

Why not call them optimizations? They are. The only thing we change is the
boundary of valid ranges.

> The question is what is the boundary of allowable transformations. No one
> agrees that there should be no boundaries (even you don't like the change
> results to zero, though note that abandoning denormals has exactly this
> effect, and might be considered acceptable).

Oh, round-to-zero is definitely acceptable in the world of "who cares
about IEEE, we want fast math, and we'll use fixed arithmetic if the FP
code is too slow".

In fact, it is _so_ acceptable that CPU designers design for it. Look at
MMX2, and wonder why they have a RTZ mode? Because it makes the _hardware_
go faster.

That should tell you something. Big companies that have billion-dollar
fabs spend time optimizing their chips that take several years to design
for _games_. Not for IEEE traditional Fortran-kind math.

But apparently some gcc developers don't think that is even a worthy
market, because you just want to do fluid dynamics.

> So, what is the boundary, can one for instance forget about denormals and
> flush to zero to save a bit of time, can one truncate instead of round,
> can one ignore negative zeroes, or infinity semantics, can one ignore
> intermediate overflow (note: nearly all the discussed transformations are
> implicitly saying yes to this last question).

Do them all by default with -ffast-math.

Then, you can have specific flags for people who want just _one_
optimization. I doubt you'll find many users who do that, but maybe I'm
wrong. Giving people the choice is always a good idea.

> I have not seen anyone writing from the point of view of serious numerical
> coding saying [ .. ]

There you go again. What the hell do you call "serious numerical coding"?

Take a look at the computer game market today. It's a lot more serious
than most matematicians puttering around in their labs, let me tell you.
That's a BIG industry.

Also note that _nobody_ in your kind of "serious numerical coding"
community would ever worry about "-ffast-math" in the first place. Why the
hell would they, when 99% of the time it doesn't make any difference at
all. The people you apparently consider serious are a lot more interested
in fast communication (so that they can solve the thing in parallell) and
incredible memory bandwidth.

I doubt you'll find many of your "serious numerical coding" people who
would even _notice_ the raw FP throughput. Look at SpecFP - CPU's are fast
enough, it spends most of its time waiting on memory.

> Should -ffast-math allow full precision operation? I would think so,
> since it definitely improves performance, and reduces surprises.

Ehh.. gcc right now allows full precision operation BY DEFAULT. No
-ffast-math required.

Same goes for negative zero as far as I remember - on some HP-PA stuff at
least. Simply because it was too painful to get "right" on their earlier
hardware.

In short, what you seem to argue that -ffast-math should means are all
things gcc _already_ does, with no

> By the way, I said I would be shocked to find a Fortran compiler that did
> associative redistribution in the absence of parens. I am somewhat surprised
> that no one stepped forward with a counter-example, but I suspect in fact that
> there may not be any shocking Fortran implementations around.

I would suspect that very few people have Fortran compilers around and
bother to check it.

> It is an old argument, the one that says that fpt is approximate, so why bother
> to be persnickety about it. Seymour Cray always tool this viewpoint, and it
> did not bother him that 81.0/3.0 did not give exactly 27.0 on the CDC 6000
> class machines.

..and he was universally respected for making the fastest machines around.

What you forget to mention is that these days it's so _cheap_ to get
IEEE, that from a hardware standpoint pretty much everybody includes it
anyway.

But they then often throw it away because it ends up having expensive
run-time issues (alpha with exception handling and proper denormals, Intel
with special RTZ modes etc).

Why? Because in many areas Seymour Cray is _still_ right. The thing that
killed off non-IEEE was not that he was wrong, but the fact that _some_
people do need IEEE "exact FP". Not everybody. Not even the majority. But
because some people do need it, you need to support it. Which is why
everybody does, today.

But do you see the difference between

  "We have to support it because a portion of the user base has to have
   it, and if we don't have it we won't be able to sell to any of that
   user base"

and

  "Everybody must use it, because anything else is wrong"

Eh?

Do you see that Seymours approach didn't fail because he was always wrong?
It failed because he was _sometimes_ wrong.

And you know what? He was right enough of the time to have built up an
empire for a while. That's something not everybody can say about
themselves. And that is something that you should respect.

			Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Joern Rennecke]

> That said, I don't mind about -fdubious-math as far as it is not the
> default, nor turned on at any level of optimization.

For a user, it's hard to guess or remember user that -fdubious-math
and -ffast-math are related.  How about -ffast-and-loose-math ?

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Tue, 31 Jul 2001, Joern Rennecke wrote:
>
> > That said, I don't mind about -fdubious-math as far as it is not the
> > default, nor turned on at any level of optimization.
>
> For a user, it's hard to guess or remember user that -fdubious-math
> and -ffast-math are related.  How about -ffast-and-loose-math ?

What are you guys smoking?

The _definition_ of "-ffast-math" is "fast and loose". It says so. Try
this:

	gcc -v --help 2>&1 | grep fast-math

Don't introduce new flags and names.

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Gabriel Dos Reis]

dewar@gnat.com writes:

| <<The C/C++ Standard, for example, say that a/b/c is evaluated as
| (a/b)/c. Someone proposed that it should be evaluated as a/(b*c)
| -- which of course is incorrect -- when optimizing.
| >>
| 
| Well now, let's be a little careful here ....
| 
| When the C standard says a/b/c is evaluated as (a/b)/c, it means that the
| result is *as if* it were computed that way, and a compiler is of course
| free to compute a/(b*c) if the result is still correct.

Completely agreed.

[...]

| So the "which of course is incorrect" is a bit too glib here.

What I intended to mean is that the compiler has, first, to prove that
such transformmation yields the same result.  The only argument I've
seen so far is that, GCC should make the program run fast...

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On 31 Jul 2001, Gabriel Dos Reis wrote:
>
> What I intended to mean is that the compiler has, first, to prove that
> such transformmation yields the same result.

Ehh..

You're _wrong_.

A lot of compiler optimizations yield different results. The question is
whether it is acceptable or not.

It _clearly_ is acceptable to do when the user has explicitly asked for
"-ffast-math". The whole meaning of the flag is to say "I want it fast,
even if you might lose some range or precision".

It's only a matter of defining the semantics.

There are other optimizations like that. Signed integer overflow is
undefined, so you can optimize integer operations in ways that do not give
the same results for all integers.

Alias analysis was added several years ago, even though it was _clearly_ a
"illegal optimization" from a traditional C standpoint. That one broke
tons of real programs. Again, it was a question of re-defining the
semantics, and allowing people to take advantage of them.

Already, the _default_ behaviour for gcc at least on x86 with floating
point has very little to do with IEEE. It's been discussed several times
how to fix it - and very little real work on it has been done for the last
several years. Why? Because it really doesn't matter for most people, and
the current work-arounds are too much of a performance hit.

So gcc _already_ has "better fast than correct to the last ULP in IEEE".
Live with it. Even by default. Your argument that it cannot do the same
even when the user asks for it is completely ludicrous.

Your continual insistence that it is "wrong" only shows that you have
built a cage around your brain.

Have you even _looked_ at what the gcc definition for "fast-math" are?
Look again:

  {"fast-math", &flag_fast_math, 1,
   "Improve FP speed by violating ANSI & IEEE rules" },

and realize that even something simple as "double" doesn't historcially
have an exact meaning in C (ie you can _not_ assume specific ranges or
specific IEEE rounding in traditional C - it was left up to the
implementor to do as best he could with what hardware was available).

That means that traditionally, you don't even know what overflows or not,
or what precision you get. It's historically "undefined". Which means that
the semantics are not clear.

Now, this is obviously why a lot of computational people disliked,
distrusted and didn't use C historically, preferring to use Fortran which
had more defined rules. And which is, of course, why C99 integrated the
math extensions that some people have been working on for a loong time.

But you have to realize that this is (a) fairly recent and (b) not
everybody needs or wants it.

And you should also realize that YOU are not the grand poobah that gets to
tell people what semantics they want.

[ Neither am I, of course, but I'm arguing for choice and freedom, while
  you're arguing for small-mindedness everywhere. ]

			Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Alexandre Oliva]

On Jul 30, 2001, Joern Rennecke <amylaar@redhat.com> wrote:

>> But we could use new tokens, say (( )) and ((( ))) to mark expressions
>> the preprocessor and the compiler shouldn't muck up with.  It would

> It's hard to distinguish an opening (( from two opening (s that
> are closes separately, e.g.:

> a*((b+c)*d)

Indeed.  That's why I suggested to only mark such duplicate
parentheses as optimization barriers, at the end of my message.  We'd
still see two different tokens, but if the opening and closing
parentheses of an expression of the form '(' expression ')' are both
marked as optimization barriers, we generate the expression tree
inside an optimization barrier.  The lexer/preprocessor would have to
mark all '('s but the first in a sequence, and all ')'s but the last
in a sequence, as candidate markers of optimization barriers.

> OTOH, you could re-use (abuse) statement expressions for this purpose:

> ({a+b;})*c

Yeah, but this won't compile with compilers other than GCC.  OTOH, it
might already do what we want in older releases of GCC.  Does it?

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Mark Mitchell]

>> OTOH, you could re-use (abuse) statement expressions for this purpose:
>
>> ({a+b;})*c

Goodness.  I have been away from email for a few days, and now I'm back,
but I'm considering going away from email again and not coming back. :-)
:-)

If the worst problem in GCC is that you cannot specify optimization
barriers in the midst of expressions then we are in very good shape
indeed!

Let's not go down this path.  In my opinion, we really do not want
to introduce this kind of thing.  There is much more win simply in
fixing bugs and limitations in the current optimizers.

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Alexandre Oliva]

On Jul 30, 2001, dewar@gnat.com wrote:

> <<I am serious.  The preprocessor should protect individual parentheses
> from becoming consecutive by introducing whitespace.  We should tell:
>>> 

> Sometimes I must say that people seem to work hard to dream up the most
> infeasible and ugly syntactical ideas that one could possibly imagine :-)

:-)

Would you prefer

__extension__ __attribute__ ((optimize=0)) (a*b) +
__extension__ __attribute__ ((optimize=0)) (a*c)

? :-)

I'd much rather write ((a*b)) + ((a*c)), which will even compile with
other compilers, and might be picked up in a future revision of the C
and C++ Standards.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Joseph S. Myers]

On 30 Jul 2001, Alexandre Oliva wrote:

> Would you prefer
> 
> __extension__ __attribute__ ((optimize=0)) (a*b) +
> __extension__ __attribute__ ((optimize=0)) (a*c)
> 
> ? :-)
> 
> I'd much rather write ((a*b)) + ((a*c)), which will even compile with
> other compilers, and might be picked up in a future revision of the C
> and C++ Standards.

I'd rather you use #pragma STDC FP_CONTRACT OFF - while this seems to be
intended to disable optimisations that give more accurate results, it
should probably disable optimisations here that may give less accurate
results as well.

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

>>Well, I won't be so rash as to say that it should do that *by default*,

But that of course is the issue under discussoin :-)

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Mon, 30 Jul 2001 dewar@gnat.com wrote:

> >>Well, I won't be so rash as to say that it should do that *by default*,
>
> But that of course is the issue under discussoin :-)

No it is not.

See the subject line.

Doing it by default was just one possible suggestion.

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[mike stump]

> Date: Sun, 29 Jul 2001 17:03:16 -0400 (EDT)
> From: Stephen L Moshier <moshier@mediaone.net>

> Further, GCC is supposed to be standards compliant by default.

Really?

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Stephen L Moshier]

> > Further, GCC is supposed to be standards compliant by default.
> 
> Really?

Well, then I'd like to understand the actual policy better.  You have
GNU extensions but otherwise seem to include what's in the language
definition.  The apparent conflicts arise when you say the
GNU extensions have to be turned off in order for the compiler
to be strictly compliant with the standard.

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

<<It's sad that C99 doesn't seem to allow the Fortran kind of "you go and
make the best you can of this expression unless I tell you otherwise".
>>

But if you look at real Fortran codes, they are full of parentheses to stop
the compiler from messing around, and in fact most Fortran compilers do only
very limited mucking even when no parens are present. I would be shocked
to find a FOrtraqn compiler that changed

a*c + b*c

into

(a+b)*c

a totally diffrerent expression!

I would have written the second expression if I had wanted that effect.

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Mon, 30 Jul 2001 dewar@gnat.com wrote:
>
> But if you look at real Fortran codes, they are full of parentheses to stop
> the compiler from messing around, and in fact most Fortran compilers do only
> very limited mucking even when no parens are present. I would be shocked
> to find a FOrtraqn compiler that changed
>
> a*c + b*c
>
> into
>
> (a+b)*c
>
> a totally diffrerent expression!

You'd be shocked?

What if I told you that it's actually legal? Go check it out.

Note that the mark of a language that generates fast code is that it both
(a) allows the compiler the maximum possible flexibility to do the best
job it can, and (b) allows the programmer to specify what the limits are.

Fortran, while being a pretty horrible language for anything else, does
exactly this for FP math. It doesn't do it for much anything else, of
course.

Now, the problem here is that what is permissible in Fortran is not
permissible in C, but I think we should _allow_ it to happen.

Note that I'm not really advocating breaking standards, like so many
people seem to think. If you actually go back and read my emails instead
of just reacting to them by gut feel, you'll notice that what I really
suggested was to make it _easy_ for the non-math programmer to tell the
compiler "do the best you can with this".

I argue _against_ a lot of magic compiler flags that are version-
dependent and hard to use, and I argue _for_ "-ffast-math" being very
strong.

I also argue that since we already have more standards than we can shake a
stick at (traditional, c89, gnu89, c99, etc), and C traditionally has not
been very strict about what optimizations are permissible, the argument of
"conforms to C99 by default" is not a very strong one. But I also noted
that it _is_ a very valid argument, although not as absolute as some
people seem to think it is (exactly because we do have multiple standards,
and gcc already defaults to a standard of its own rather than anything
else).

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[Gabriel Dos Reis]

Fergus Henderson <fjh@cs.mu.oz.au> writes:

| On 29-Jul-2001, Stephen L Moshier <moshier@mediaone.net> wrote:
| > 
| > Further, GCC is supposed to be standards compliant by default.
| 
| ?
| 
| I think it would be nice if GCC was standards compliant by default,
| but that is certainly not the status quo.

Agreed.  But that should be the ideal we should strive for.
And also, I don't think that is an argument for not computing
correctly, even if we're asked to compute fast. 

-- Gaby

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On 30 Jul 2001, Gabriel Dos Reis wrote:

> Fergus Henderson <fjh@cs.mu.oz.au> writes:
> |
> | I think it would be nice if GCC was standards compliant by default,
> | but that is certainly not the status quo.
>
> Agreed.  But that should be the ideal we should strive for.
> And also, I don't think that is an argument for not computing
> correctly, even if we're asked to compute fast.

Note that "a/b/c" _is_ the same as "a/(b*c)" when seen as a mathematical
expression.

So it's certainly very very debatable whether you can say "not computing
correctly" for it - the user has not actually shown a strong preference
for the double division, and it is entirely possible that the double
division has come about as a result of macro expansion and/or inline
function expansion.

Who are you to say that the user wouldn't like the code to be run faster?
I don't think this was well-defined in C before C99, for example.

Note that with the current -ffast-math on x86, for example, gcc will
already inline things like "sin()" etc using the x87 instruction directly.
Despite the fact that that instruction is well-defined only for the range
-2**63 to +2**63.

And now there are people that say that something simple as "a/b/c" should
not be optimized to "a/(b*c)" even with -ffast-math.

What planet are you people from?

We are already in the situation that -ffast-math gives a value for "sin()"
that is not even _close_ to the range of [-1,1] under various
circumstances. And you're complaining about the fact that b*c can
overflow? We've already effectively limited our argument range to +-2**63,
and there's no way a*b can overflow, guys.

So even if you don't want to make it an optimization by default with the
C99 argument (hey, I don't really disagree), there is absolutely _no_
valid argument for not doing it with -ffast-math.

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law in combine)

[dewar]

<<Further, GCC is supposed to be standards compliant by default.  You
seem to be advocating that GCC should, without warning, generate
programs that violate the standard and are mathematically wrong.
>>

and that violate the expectations of programmers who *do* know what they
are doing!

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Sun, 29 Jul 2001 dewar@gnat.com wrote:
>
> <<Further, GCC is supposed to be standards compliant by default.  You
> seem to be advocating that GCC should, without warning, generate
> programs that violate the standard and are mathematically wrong.
> >>
>
> and that violate the expectations of programmers who *do* know what they
> are doing!

Note that most of the people I know who _do_ know what they are doing also
want to have the option o ftelling the compiler when they don't care, and
want to allow the compiler to do "illegal" things if that allows the
compiler to generate better code.

It's sad that C99 doesn't seem to allow the Fortran kind of "you go and
make the best you can of this expression unless I tell you otherwise".
Which means that we'd _have_ to be non-standard to allow the optimization
at all. Which obviously means a flag (one way or the other).

Rather than have a good way to do it on a case-by-case basis, which is
probably what the people who _really_ care would prefer.

In C, you can't reasonably use the Fortran "parenthesis are significant"
approach - the way the C pre-processor works you often have lots of extra
parenthesis for that reason. However, C does have the well-established
sequence point rule, so it would make sense to have that kind of
extension.

And hey, if you want to make it be an extra flag, just make it a _simple_
one. Like "-ffast-math". Don't make people have to know 100 esoteric flags
that are compiler-release-specific. That's completely useless.

		Linus

Re: * Re: What is acceptable for -ffast-math? (Was: associative lawin combine)

[Linus Torvalds]

On Sun, 29 Jul 2001, Stephen L Moshier wrote:
>
> > Why not make this easier for the user:
> > - default to non-IEEE, but "reasonably safe".
>
> The default GCC behavior is not to allow any optimization
> that could change the value of an expression.  That rule is easy
> for users to understand and surely it is a good rule to keep.
>
> Floating point arithmetic, IEEE or otherwise, does not obey the
> associative or distributive laws.

This is YOUR opinion.

But most people who write floating point code do not think that way. They
consider FP to be "mathematically exact" (which _does_ obey associative
and distributive laws), and they write their programs that way.

And that is exactly my point: to the people who do not care about the
exact details of what it means from a mathematical standpoint, using
associative and distributive rules is _fine_. Their algorithms do not
care, simply because the person writing the code doesn't even _know_ about
the issues of limited precision and range of FP code.

See?

What I'm saying is that the compiler cannot fix "thinking errors" of the
programmer. Trying to fix them is self-defeating and ultimately stupid.
I'm claiming that 99%+ of all programmers do not know or care about the
fact that

	(a+b)*c

is not necessarily the same as

	a*c+b*c

when we're talking about floating point.

My _second_ point is that the ones that DO KNOW and DO CARE are also aware
enough that _they_ can be counted on to add a "-mieee" or other flags.

They are also knowledgeable enough that they know what "associative law"
means, which is already a step up from most programmers, I suspect. So
they can understand why the compiler has done an optimization, and can (if
it is documented) be instructed to use the appropriate switch for not
doing that optimization.

So: put the onus of understanding different FP optimizations on the people
who CAN understand them. Don't make the average person who wants to do his
own small project have to worry about it. By the time the person
understands about the range of FP arithmetic enough to care, he will also
have the capability to do something about it.

		Linus

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Stephen L Moshier]

> > Floating point arithmetic, IEEE or otherwise, does not obey the
> > associative or distributive laws.
> 
> This is YOUR opinion.

That is not just an opinion, it is simple arithmetic.

Further, GCC is supposed to be standards compliant by default.  You
seem to be advocating that GCC should, without warning, generate
programs that violate the standard and are mathematically wrong.
The C99 standard says the following, among other things, about floating
point transformations:


       [#14] EXAMPLE 5 Rearrangement for floating-point expressions
       is  often  restricted because of limitations in precision as
       well as range.  The implementation  cannot  generally  apply
       the   mathematical   associative   rules   for  addition  or
       multiplication,  nor  the  distributive  rule,  because   of
       roundoff   error,  even  in  the  absence  of  overflow  and
       underflow.   Likewise,  implementations   cannot   generally
       replace decimal constants in order to rearrange expressions.
       In  the  following  fragment,  rearrangements  suggested  by
       mathematical rules for real numbers are often not valid (see
       F.8).

               double x, y, z;
               /* ... */
               x = (x * y) * z;  // not equivalent to x *= y * z;
               z = (x - y) + y ; // not equivalent to z = x;
               z = x + x * y;    // not equivalent to z = x * (1.0 +
               y);
               y = x / 5.0;      // not equivalent to y = x * 0.2;

Re: What is acceptable for -ffast-math? (Was: associative law incombine)

[Linus Torvalds]

On Sun, 29 Jul 2001, Stephen L Moshier wrote:
>
> > > Floating point arithmetic, IEEE or otherwise, does not obey the
> > > associative or distributive laws.
> >
> > This is YOUR opinion.
>
> That is not just an opinion, it is simple arithmetic.

No, it's actually a very complex issue of representing classical
arithmetic with finite values. It's not "simple arithmetic" at all. In the
simple arithmetic, operations _are_ associative and distriubutive.

And what I'm really claiming is that until gcc is a full Artifical
Intelligence system, it cannot add information to a program.

In 99% of all cases, the information of the subtleties of floating point
arithmetic simply was never there in the program gcc tries to compile.
Why? Becuase the programmer didn't even _intend_ any of the subtleties, or
even knew about them.

Gcc trying to enforce them doesn't add any value.

> Further, GCC is supposed to be standards compliant by default.

That is a separate argument, and I don't think you'r enecessarily wrong on
that front. On the other hand, people are already used to having command
line switches for strict standards adherence, so I don't think that this
argument is necessarily a very strong one, more a matter of opinion.

Especially true since the standard you quote is a fairly recent one (has
it been finalized now?)

		Linus