Re: -ffast-math and floating point reordering

Re: -ffast-math and floating point reordering

[Chris Lattner]

>> Yeah.  We sort of discussed adding additional PLUS_EXPR operands and/or
>> attributes at last year's summit.  But I think that was the extent of
>> it.  IIRC, there was beer involved, so I doubt anybody was taking notes.

> Hmm, this is usually the point where we get the LLVM people tell us what
> they are doing here...

Okay, first, you must understand that LLVM is under vastly different
constraints than GCC is.  It makes little sense to have a -ffast-math mode
in LLVM.  What would happen when you link two .o files with different
settings?  At best the flag would just be cleared.

That said, we don't have an implemented solution to this either at
present.  My plan is to add a new "fpadd" or "strictadd" instruction or
something similar that provides strict FP semantics.  The existing LLVM
"add" instruction on fp would then be relaxed to allow allow the
equivalent of -ffast-math operations.  This allows different translation
units to be linked together without losing a notion of which operations
are strict and which ones are not.

This also allows individual translation units to mix different semantics
as well.  For example, Java has strictfp, and fortran has the parenthesis
issue.  In a fortran front-end that preserved the appropriate information,
this shows how the code generation would differ:

 x = a + b + c

   %t1 = add double %a, %b
   %x = add double %t1, %c

 x = (a + b) + c

   %t = fpadd double %a, %b
   %x = add double %t, %c

 x = a + (b + c)

   %t = fpadd double %b, %c
   %x = add double %t, %a

That's the idea.  Note again, that this has not been implemented, largely
because no one has taken it up (it should be entirely straight-forward).
The other advantage of doing this is that we would be able to change code
like this:

  // X + 0 --> X
  if (I.getOpcode() == Instruction::Add &&
     !I.getType()->isFloatingPoint() &&    // -0 + +0 = +0, so it's not a noop
      RHS == Constant::getNullValue(I.getType()))
    return ReplaceInstUsesWith(I, LHS);

With code that looks like this:

  // X + 0 --> X
  if (I.getOpcode() == Instruction::Add &&
      RHS == Constant::getNullValue(I.getType()))
    return ReplaceInstUsesWith(I, LHS);

... leading to cleaner and easier to maintain code.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

Re: -ffast-math and floating point reordering

[Chris Lattner]

On Fri, 26 Mar 2004, Chris Lattner wrote:
> Okay, first, you must understand that LLVM is under vastly different
> constraints than GCC is.  It makes little sense to have a -ffast-math mode
> in LLVM.  What would happen when you link two .o files with different
> settings?  At best the flag would just be cleared.

Oh, one other thing that I should have pointed out: when LLVM has support
for this, the C front-end will turn floating point adds into the strictadd
by default.  If -ffast-math is presented to the C front-end, the C
front-end will generate the non-strict adds instead.  This allows us to
use the *information* from the -ffast-math flag, without having to
actually preserve it in the LLVM files.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

Re: -ffast-math and floating point reordering

[Dale Johannesen]

On Mar 26, 2004, at 4:41 PM, Robert Dewar wrote:
> Dale Johannesen wrote:
>> One point I haven't seen yet:
>> A couple of strict-constructionists have asserted that people
>> who want reassociation done in a nonstandard way can easily rewrite 
>> their
>> code to achieve this.  This is not really true, because such 
>> expressions
>> often arise as a result of other optimizations, in a way that isn't 
>> obvious
>> from the source.  Offhand, inlining, unrolling, and constant/copy 
>> propagation
>> can all do it, and there are probably more.
>
> Well in the absence of the (very annoying) extra precision phenomenon,
> none of these optimizations should have observable effects.

Untrue.  If two of a+b+c are constant, for example, you want to add the
constants at compile time (assuming your requirements for accuracy
are satisfied).  This is exactly the sort of thing that's exposed by 
other
optimiziations.

Re: -ffast-math and floating point reordering

[Robert Dewar]

Dale Johannesen wrote:
> One point I haven't seen yet:
> A couple of strict-constructionists have asserted that people
> who want reassociation done in a nonstandard way can easily rewrite their
> code to achieve this.  This is not really true, because such expressions
> often arise as a result of other optimizations, in a way that isn't obvious
> from the source.  Offhand, inlining, unrolling, and constant/copy 
> propagation
> can all do it, and there are probably more.

Well in the absence of the (very annoying) extra precision phenomenon,
none of these optimizations should have observable effects.

Re: -ffast-math and floating point reordering

[Richard Henderson]

On Fri, Mar 26, 2004 at 10:50:57AM -0800, Joe Buck wrote:
> But if that idea is used, then the C and C++ front ends would
> have to put a PLUS_EXPR around *every* expression.

No, what was meant was that one of the flag bits on the PLUS_EXPR
would allow or disallow the reassociation.  Likely allow, since
that would be safest compatibility-wise with exating front-ends.


r~

Re: -ffast-math and floating point reordering

[Joseph S. Myers]

On Fri, 26 Mar 2004, Joe Buck wrote:

> One possible representation in GIMPLE would be a flag indicating that a
> temporary can be, well, I'll call it "refactored" (feel free to suggest a
> better word).  That is, given GIMPLE code like

Likewise one to indicate an expression can be contracted (evaluated
atomically without intermediate rounding), to allow multiply-adds etc.
with the FP_CONTRACT pragma (which allows expressions to be contracted
while not allowing such optimizations across separate statements).

-- 
Joseph S. Myers
jsm@polyomino.org.uk

Re: -ffast-math and floating point reordering

[Joe Buck]

On Fri, Mar 26, 2004 at 11:37:42AM -0700, law@redhat.com wrote:
> In message <4064721A.6060905@tat.physik.uni-tuebingen.de>, Richard Guenther wri
> tes:
>  >> Yeah.  We sort of discussed adding additional PLUS_EXPR operands and/or
>  >> attributes at last year's summit.  But I think that was the extent of
>  >> it.  IIRC, there was beer involved, so I doubt anybody was taking notes.
>  >
>  >Hmm, this is usually the point where we get the LLVM people tell us what 
>  >they are doing here...  I consider not being able to do something about 
>  >this in tree-ssa a major defect, too.
> Well, given that the front-ends don't even provide this information at this
> time, I don't consider it a defect.  I consider it a long term enhancement
> we need to add first to the front-ends, then to the SSA bits.

A design decision has to be made about how the information should be
provided; you can't add it "first to the front-ends, then to the SSA bits"
because first you have to say how it's to be represented.

Someone mentioned PLUS_EXPR; this seems to be a reference to a feature
that was temporarily considered for the C89 standard: it was proposed to
keep the K&R rule of wild rearrangement and use unary plus to say "don't
rearrange".  But if that idea is used, then the C and C++ front ends would
have to put a PLUS_EXPR around *every* expression.

Since Fortran is the only language that seems to permit re-association at
all, it seems that the default should be that the middle and back ends
don't do re-association unless the frond end communicates that it is OK.

Re: -ffast-math and floating point reordering

[Laurent GUERBY]

On Fri, 2004-03-26 at 17:21, law@redhat.com wrote:
> The question then becomes whether or not -ffast-math ought to turn that
> flag on.  I'm neither a significant user or expert in FP arithmetic.  So
> I've got no strong opinions here.

Given our documentation which promises nothing about results, it's clear
that it can go in. 

Laurent

<<
@item -ffast-math
@opindex ffast-math
Sets @option{-fno-math-errno}, @option{-funsafe-math-optimizations}, @*
@option{-fno-trapping-math}, @option{-ffinite-math-only},
@option{-fno-rounding-math} and @option{-fno-signaling-nans}.

This option causes the preprocessor macro @code{__FAST_MATH__} to be
defined.

This option should never be turned on by any @option{-O} option since
it can result in incorrect output for programs which depend on
an exact implementation of IEEE or ISO rules/specifications for
math functions.
...
@item -funsafe-math-optimizations
@opindex funsafe-math-optimizations
Allow optimizations for floating-point arithmetic that (a) assume
that arguments and results are valid and (b) may violate IEEE or
ANSI standards.  When used at link-time, it may include libraries
or startup files that change the default FPU control word or other
similar optimizations.


>>

Re: -ffast-math and floating point reordering

[law]

In message <4064721A.6060905@tat.physik.uni-tuebingen.de>, Richard Guenther wri
tes:
 >> Yeah.  We sort of discussed adding additional PLUS_EXPR operands and/or
 >> attributes at last year's summit.  But I think that was the extent of
 >> it.  IIRC, there was beer involved, so I doubt anybody was taking notes.
 >
 >Hmm, this is usually the point where we get the LLVM people tell us what 
 >they are doing here...  I consider not being able to do something about 
 >this in tree-ssa a major defect, too.
Well, given that the front-ends don't even provide this information at this
time, I don't consider it a defect.  I consider it a long term enhancement
we need to add first to the front-ends, then to the SSA bits.

jeff

Re: -ffast-math and floating point reordering

[Joe Buck]

On Fri, Mar 26, 2004 at 01:22:14PM -0500, Robert Dewar wrote:
> Joe Buck wrote:
> > On Thu, Mar 25, 2004 at 01:03:39AM -0500, Robert Dewar wrote:
> > What is "model interval"?
> 
> Brief answer (long answer is too long). When the result of an
> fpt operation is exactly representable, the model interval is
> the set consisting of this one number.
> 
> When the result is not exactly representable, then the model
> interval is the range of results between the two representable
> values surrounding the true real result, including the end
> points.
> 
> Generally any value within the model interval can be returned,
> either end point, or, using temporary excess precision, some
> value in between.
> 
> An optimization like reassociation is permitted only if the
> result is within the model interval of the original result
> as defined by the canonical semantics.

Thanks for the explanation.  This appears to be just a hair looser than
C99 with __STDC_IEC_599__, because with IEEE floating point the rounding
direction is specified, and this suggests that Ada allows any operation to
round in either direction.

However, I doubt that there are any associative-operator rearrangements
that are permitted by this rule in Ada but not permitted in C (given that
the as-if rule permits any transformation that does not change the
result).

Re: -ffast-math and floating point reordering

[Robert Dewar]

Joe Buck wrote:
> On Thu, Mar 25, 2004 at 01:03:39AM -0500, Robert Dewar wrote:
> What is "model interval"?

Brief answer (long answer is too long). When the result of an
fpt operation is exactly representable, the model interval is
the set consisting of this one number.

When the result is not exactly representable, then the model
interval is the range of results between the two representable
values surrounding the true real result, including the end
points.

Generally any value within the model interval can be returned,
either end point, or, using temporary excess precision, some
value in between.

An optimization like reassociation is permitted only if the
result is within the model interval of the original result
as defined by the canonical semantics.

This is quite a strict criterion. For example in Ada 83, it
forbod the optimization of x**4 to x**2**2, because there
are cases where the latter is less accurate and lies outside
the model interval of x*x*x*x, which is the canonical
semantic result. This particular effect was unintended (it
is a bit of a surprise to some that x**2**2 which has one
less multiplication, can be less accurate :-) So in Ada 95
reassociation is explicitly permitted in computing x**j
for integer exponent j.

Re: -ffast-math and floating point reordering

[Fariborz Jahanian]

On Mar 26, 2004, at 10:10 AM, Richard Guenther wrote:

> Diego Novillo wrote:
>> On Fri, 2004-03-26 at 12:42, Joe Buck wrote:
>>> One possible representation in GIMPLE would be a flag indicating 
>>> that a
>>> temporary can be, well, I'll call it "refactored" (feel free to 
>>> suggest a
>>> better word).  That is, given GIMPLE code like
>>>
>>> 	t1 = a + b;
>>> 	t2 = t1 + c;
>>> 	e = t2 + d;
>>>
>>> if the original Fortran input was
>>>
>>>   	e = a + b + c + d
>>>
>>> then t1 and t2 would be tagged as refactorable, while if the original
>>> input were
>>>
>>> 	d = ((a + b) + c) + d
>>>
>>> then t1 and t2 would not be so tagged.  In the first case, we would
>>> probably want to produce
>>>
>>> 	t1 = a + b;
>>> 	t2 = c + d;
>>> 	e = t1 + t2;
>>>
>>> since the first two additions can now be performed in parallel.  But
>>> this transformation is not legal in the second case.
>>>
>> Yeah.  We sort of discussed adding additional PLUS_EXPR operands 
>> and/or
>> attributes at last year's summit.  But I think that was the extent of
>> it.  IIRC, there was beer involved, so I doubt anybody was taking 
>> notes.
>
> Hmm, this is usually the point where we get the LLVM people tell us 
> what they are doing here...  I consider not being able to do something 
> about this in tree-ssa a major defect, too.

Yes. Also 'generic' must include an PAREN_EXPR, or ORD_EXPR that 
languages can take advantage of
and rest of the compiler honor.

- Fariborz

>
> Richard.

Re: -ffast-math and floating point reordering

[Richard Guenther]

Diego Novillo wrote:
> On Fri, 2004-03-26 at 12:42, Joe Buck wrote:
> 
> 
>>One possible representation in GIMPLE would be a flag indicating that a
>>temporary can be, well, I'll call it "refactored" (feel free to suggest a
>>better word).  That is, given GIMPLE code like
>>
>>	t1 = a + b;
>>	t2 = t1 + c;
>>	e = t2 + d;
>>
>>if the original Fortran input was
>>
>>   	e = a + b + c + d
>>
>>then t1 and t2 would be tagged as refactorable, while if the original
>>input were
>>
>>	d = ((a + b) + c) + d
>>
>>then t1 and t2 would not be so tagged.  In the first case, we would
>>probably want to produce
>>
>>	t1 = a + b;
>>	t2 = c + d;
>>	e = t1 + t2;
>>
>>since the first two additions can now be performed in parallel.  But
>>this transformation is not legal in the second case.
>>
> 
> Yeah.  We sort of discussed adding additional PLUS_EXPR operands and/or
> attributes at last year's summit.  But I think that was the extent of
> it.  IIRC, there was beer involved, so I doubt anybody was taking notes.

Hmm, this is usually the point where we get the LLVM people tell us what 
they are doing here...  I consider not being able to do something about 
this in tree-ssa a major defect, too.

Richard.

Re: -ffast-math and floating point reordering

[Diego Novillo]

On Fri, 2004-03-26 at 12:42, Joe Buck wrote:

> One possible representation in GIMPLE would be a flag indicating that a
> temporary can be, well, I'll call it "refactored" (feel free to suggest a
> better word).  That is, given GIMPLE code like
> 
> 	t1 = a + b;
> 	t2 = t1 + c;
> 	e = t2 + d;
> 
> if the original Fortran input was
> 
>    	e = a + b + c + d
> 
> then t1 and t2 would be tagged as refactorable, while if the original
> input were
> 
> 	d = ((a + b) + c) + d
> 
> then t1 and t2 would not be so tagged.  In the first case, we would
> probably want to produce
> 
> 	t1 = a + b;
> 	t2 = c + d;
> 	e = t1 + t2;
> 
> since the first two additions can now be performed in parallel.  But
> this transformation is not legal in the second case.
> 
Yeah.  We sort of discussed adding additional PLUS_EXPR operands and/or
attributes at last year's summit.  But I think that was the extent of
it.  IIRC, there was beer involved, so I doubt anybody was taking notes.


Diego.

Re: -ffast-math and floating point reordering

[Joe Buck]

On Fri, Mar 26, 2004 at 09:21:26AM -0700, law@redhat.com wrote:
> In message <20040324170719.A12420@synopsys.com>, Joe Buck writes:
>  >If -ffast-math is not specified, we should follow the language standards.
>  >First question: can tree-ssa distinguish, for Fortran, whether parentheses
>  >were present and so whether reordering is allowed?
> No.  The front-ends don't pass enough information around in the tree nodes
> to know when such changes are safe/unsafe.  Even if the information existed
> none of the optimizers would know how to use it (at least initially).

This should be considered a design bug, as it forces GCC to be inferior
to competing Fortran compilers.

One possible representation in GIMPLE would be a flag indicating that a
temporary can be, well, I'll call it "refactored" (feel free to suggest a
better word).  That is, given GIMPLE code like

	t1 = a + b;
	t2 = t1 + c;
	e = t2 + d;

if the original Fortran input was

   	e = a + b + c + d

then t1 and t2 would be tagged as refactorable, while if the original
input were

	d = ((a + b) + c) + d

then t1 and t2 would not be so tagged.  In the first case, we would
probably want to produce

	t1 = a + b;
	t2 = c + d;
	e = t1 + t2;

since the first two additions can now be performed in parallel.  But
this transformation is not legal in the second case.

For temporaries marked as refactorable, optimizers are allowed to replace
use of the temporary by the expression that computes it, and do
transformations on that expression, even if the tranformation doesn't
guarantee that roundoffs and overflows will be the same.  If the temporary
is not marked as refactorable, this is not permitted.

With such a rule, the language-independent middle end can implement
Fortran or C semantics correctly, and it would be possible to relax the
rules in a controlled manner as well:. the user could ask the C front
end to implement the Fortran rules (respect parentheses but otherwise
allow rearrangement), or the K&R rules (all rearrangements that are
correct for real numbers are allowed), but by default the compiler would
follow the standard.

Re: -ffast-math and floating point reordering

[Robert Dewar]

> No.  The front-ends don't pass enough information around in the tree nodes
> to know when such changes are safe/unsafe.  Even if the information existed
> none of the optimizers would know how to use it (at least initially).

It's a significant goof for a Fortran compiler not to remember whether
parens are used. GNAT remembers not only if parens were used, but how
many levels, but that's required due to obscure requirements in
conformance rules, and of course it is not passed on to the back end
since (a) that's not necessary, it does not affect semantics in Ada
and in any case (b) there is no way to do that.

> First I'd like to come out in favor of having a flag to turn on FP
> reassociation.

Note that there are two levels here. One is about what the Fortran
standard allows in the absence of parens, which is rearrangement
within an expression. The more extensive permission is to rearrange
over a series of separately computed expressions.

Re: -ffast-math and floating point reordering

[law]

In message <20040324170719.A12420@synopsys.com>, Joe Buck writes:
 >If -ffast-math is not specified, we should follow the language standards.
 >First question: can tree-ssa distinguish, for Fortran, whether parentheses
 >were present and so whether reordering is allowed?
No.  The front-ends don't pass enough information around in the tree nodes
to know when such changes are safe/unsafe.  Even if the information existed
none of the optimizers would know how to use it (at least initially).


 >If -ffast-math is specified, what should we do?  One option is to use the
 >K&R rule and free-associate at will.  I feel uncomfortable with that
 >because of the major loss in accuracy that can result.  If, however, we
 >implement support for the Fortran rules, one option would be to relax
 >order of evaluation when there are no parentheses (like Fortran), another
 >would be to just leave the order the same.
First I'd like to come out in favor of having a flag to turn on FP
reassociation.

The question then becomes whether or not -ffast-math ought to turn that
flag on.  I'm neither a significant user or expert in FP arithmetic.  So
I've got no strong opinions here.


jeff

Re: -ffast-math and floating point reordering

[Bradley Lucier]

In

http://gcc.gnu.org/ml/gcc/2004-03/msg01492.html

Geert Bosch writes

> This is a good point, and it also serves as an illustration
> of weaknesses in the "just add another command-line option" philosophy.
> For example, in writing various elementary floating-point functions, it
> definitely is necessary to prevent unsafe transformations in order to
> attain accuracy requirements for the entire domain of the function.
>
>
> On the other hand, it is also desirable in many cases to inline these
> functions, especially when one of the operands is a constant. How can
> we specify that this code that relies on strict semantics will not
> be affected by the command-line options.
>
>
> A similar issue exists the other way around. There may be library
> functions where the various transformations are known to be safe,
> and which may benefit from them. Think of a graphics routines for
> coordinate transformations for example.
>
>
> One solution seems to allow having tree-level operations for addition,
> multiplications etc. that specify wether they are
>   - associative
>   - communicative
>   - potentially trapping
>   - supporting non-finite value semantics
>   - supporting non-default rounding
>
>
> This way front ends can set these attributes depending on language
> rules and/or constructs in the source specifying certain behavior.

Another, perhaps equivalent approach, would be to have lexically scoped 
"compilation environments" for code trees.  These compilation 
environments would be set up at the point where the tree is defined; if 
a function is inlined, it would be inlined with its lexically-defined 
compilation environment intact.  The tree is then transformed using the 
rules of the associated compilation environment.

This is done now in the Gambit-C Scheme compiler.  If a function is 
declared to be compiled without runtime error checking, then this 
property is carried wherever this function is inlined.  This may be 
clearer when compiler options are declared using inline pragmas rather 
than with compiler switches attached to entire files.

Brad

Re: -ffast-math and floating point reordering

[Geert Bosch]

On Mar 25, 2004, at 12:58, Dale Johannesen wrote:
> One point I haven't seen yet:
> A couple of strict-constructionists have asserted that people
> who want reassociation done in a nonstandard way can easily rewrite 
> their
> code to achieve this.  This is not really true, because such 
> expressions
> often arise as a result of other optimizations, in a way that isn't 
> obvious
> from the source.  Offhand, inlining, unrolling, and constant/copy 
> propagation
> can all do it, and there are probably more.
>

This is a good point, and it also serves as an illustration
of weaknesses in the "just add another command-line option" philosophy.
For example, in writing various elementary floating-point functions, it
definitely is necessary to prevent unsafe transformations in order to
attain accuracy requirements for the entire domain of the function.

On the other hand, it is also desirable in many cases to inline these
functions, especially when one of the operands is a constant. How can
we specify that this code that relies on strict semantics will not
be affected by the command-line options.

A similar issue exists the other way around. There may be library
functions where the various transformations are known to be safe,
and which may benefit from them. Think of a graphics routines for
coordinate transformations for example.

One solution seems to allow having tree-level operations for addition,
multiplications etc. that specify wether they are
   - associative
   - communicative
   - potentially trapping
   - supporting non-finite value semantics
   - supporting non-default rounding

This way front ends can set these attributes depending on language
rules and/or constructs in the source specifying certain behavior.

   -Geert

Re: -ffast-math and floating point reordering

[Dale Johannesen]

One point I haven't seen yet:
A couple of strict-constructionists have asserted that people
who want reassociation done in a nonstandard way can easily rewrite 
their
code to achieve this.  This is not really true, because such expressions
often arise as a result of other optimizations, in a way that isn't 
obvious
from the source.  Offhand, inlining, unrolling, and constant/copy 
propagation
can all do it, and there are probably more.

RE: -ffast-math and floating point reordering

[Dave Korn]

> -----Original Message-----
> From: gcc-owner On Behalf Of Joe Buck
> Sent: 25 March 2004 17:10

> I've generally found that the fastest 
> way to get the right answer on the net is to confidently 
> assert the answer you believe to be right; those who know 
> will immediately correct you, while if you just ask, often no 
> answers arrive.  All it requires is a willingness to look bad 
> on occasion.


  This so much has the ring of a deep truth about it that it deserves formal
recognition.  Hell, I've been using the technique for years myself -
sometimes even deliberately!  So unless anyone has a better idea, I dub this
observation "Buck's law of Internet information extraction".



    cheers, 
      DaveK
-- 
Can't think of a witty .sigline today....

Re: -ffast-math and floating point reordering

[Joe Buck]

On Thu, Mar 25, 2004 at 01:03:39AM -0500, Robert Dewar wrote:
> Joe Buck wrote:
> 
> > Ada:	the 1983 LRM seems to say that the rule is the same as for C++:
> > 
> > 	"... for a sequence of operators of the same precedence level, the
> > 	operators are associated in textual order from left to right;
> > 	parentheses can be used to impose specific associations"
> 
> First, that is the wrong document, describing a language that has been
> obsolete for a decade, please reference the Ada 95 document :-)

Thanks the correction; I've generally found that the fastest way to get
the right answer on the net is to confidently assert the answer you
believe to be right; those who know will immediately correct you, while if
you just ask, often no answers arrive.  All it requires is a willingness
to look bad on occasion.

> Secondly, this statement is talking about the evaluation structure. The
> rule about reassociation is else where (and is as I stated earlier, you
> are allowed to reassociate the the result is in the same model interval)

What is "model interval"?

Re: -ffast-math and floating point reordering

[Paul Koning]

>>>>> "Joe" == Joe Buck <Joe.Buck@synopsys.COM> writes:

 Joe> ...If -ffast-math is specified, what should we do?  One option is
 Joe> to use the K&R rule and free-associate at will.  I feel
 Joe> uncomfortable with that because of the major loss in accuracy
 Joe> that can result.  If, however, we implement support for the
 Joe> Fortran rules, one option would be to relax order of evaluation
 Joe> when there are no parentheses (like Fortran), another would be
 Joe> to just leave the order the same.

I believe there are also cases where the programmer has good reason to
believe that such transformations should be allowed because the
accuracy properties are appropriate for the application.  It would be
good to be able to say so.  Perhaps the switch (-ffree-association?)
should be part of -ffast-math, perhaps not.  But it would give
non-Fortran programmers a way to get to the optimization opportunities
that Fortran has.

     paul

Re: -ffast-math and floating point reordering

[Robert Dewar]

Ian Lance Taylor wrote:

> It may not have been completely clear from my e-mail message, but
> -funsafe-math-optimizations is an existing gcc option.  I didn't make
> up that name.

I understand that :-)
> 
> (If you want to send in a patch to change it to
> -funsafe-math-transformations, that would be fine with me.)

Might be a bit controversial :-)
> 
> Ian

Re: -ffast-math and floating point reordering

[Ben Elliston]

Ian Lance Taylor <ian@wasabisystems.com> writes:

> > I think -funsafe-math-transformations
> > would be a better name that was less misleading
> 
> It may not have been completely clear from my e-mail message, but
> -funsafe-math-optimizations is an existing gcc option.  I didn't make
> up that name.
> 
> (If you want to send in a patch to change it to
> -funsafe-math-transformations, that would be fine with me.)

The nice thing about doing that is that it's one less option that has
conflicting spellings in different locales.

Ben

Re: -ffast-math and floating point reordering

[Ian Lance Taylor]

Robert Dewar <dewar@gnat.com> writes:

> > -funsafe-math-optimizations
> 
> I think -funsafe-math-transformations
> 
> would be a better name that was less misleading

It may not have been completely clear from my e-mail message, but
-funsafe-math-optimizations is an existing gcc option.  I didn't make
up that name.

(If you want to send in a patch to change it to
-funsafe-math-transformations, that would be fine with me.)

Ian

Re: -ffast-math and floating point reordering

[Robert Dewar]

> -funsafe-math-optimizations 

I think -funsafe-math-transformations

would be a better name that was less misleading

Re: -ffast-math and floating point reordering

[Robert Dewar]

Joe Buck wrote:

> Ada:	the 1983 LRM seems to say that the rule is the same as for C++:
> 
> 	"... for a sequence of operators of the same precedence level, the
> 	operators are associated in textual order from left to right;
> 	parentheses can be used to impose specific associations"

First, that is the wrong document, describing a language that has been
obsolete for a decade, please reference the Ada 95 document :-)

Secondly, this statement is talking about the evaluation structure. The
rule about reassociation is else where (and is as I stated earlier, you
are allowed to reassociate the the result is in the same model interval)

Re: -ffast-math and floating point reordering

[Ian Lance Taylor]

Joe Buck <Joe.Buck@synopsys.COM> writes:

> If -ffast-math is specified, what should we do?  One option is to use the
> K&R rule and free-associate at will.  I feel uncomfortable with that
> because of the major loss in accuracy that can result.  If, however, we
> implement support for the Fortran rules, one option would be to relax
> order of evaluation when there are no parentheses (like Fortran), another
> would be to just leave the order the same.

-ffast-math is documented in terms of turning on a set of other
options.  So another issue is: if we add support for permitting the
addition of a+b+c in any order, should we put that under
-funsafe-math-optimizations (currently implied for -ffast-math), or
should we put it under another option (-fassociative-math?).  If we
choose the latter alternative, should that new option be set by
-ffast-math, or only by explicit use?

My vote, at least right now, would be to add -fassociative-math to
mean that gcc is permitted to reassociate math expressions at will,
regardless of the presence of parentheses.  And -ffast-math should
imply -fassociative-math.  That is, with -ffast-math, (a+b)+c may be
evaluated as a+(b+c).

My rationale is that people who are paying close attention to their
floating point arithmetic should generally not use -ffast-math.  But
adding another option for this will permit people to pay close
attention to floating point arithmetic to use -ffast-math
-fno-associative-math.  In general I'm not in favor of giving people
too many knobs.  But I think this one is OK because it can be clearly
expressed in terms of its effect on the code that people write (as
opposed to many of the knobs which are expressed in terms of gcc's
internal optimizations).

People who do not pay close attention to floating point arithmetic,
and who use -ffast-math, are saying that they want the fastest
possible execution, and that their floating point code is written
accordingly.  I believe that implies that reassociating floating point
operations is acceptable when it will produce faster code.

I don't think this type of reassociation fits under
-funsafe-math-options, because it doesn't really fit the description
of that option in the documentation.  That option is described in
terms of assuming that values are in domain and results are in range.
That is different from associativity.

Ian

-ffast-math and floating point reordering

[Joe Buck]

OK, let's start a thread under the right title.

The question under consideration is how, if at all, the -ffast-math flag
should effect the evaluation of a+b+c or (a+b)+c .  It turns out that
there was a lot of confusion as to what the rules are, for various
languages.

Summary (credit to Joseph Myers, Gaby Dos Reis, and Toon Moene): for the
case where a, b, and c are really expressions, these expressions can be
evaluated in any order.  At that point:

Fortran:for a+b+c we can add in any order.  For (a+b)+c parentheses must
        be respected.

C++:	not only are parentheses respected, but a+b+c is (a+b)+c

Ada:	the 1983 LRM seems to say that the rule is the same as for C++:

	"... for a sequence of operators of the same precedence level, the
	operators are associated in textual order from left to right;
	parentheses can be used to impose specific associations"

	( http://archive.adaic.com/standards/83lrm/html/lrm-04-05.html#4.5 )

C:	if IEEE conformant behavior is promised (by defining
	__STDC_IEC_559__), it's like C++, but "the implementation can state
	that the accuracy of floating-point operations is unknown" if
	this symbol is not defined.

K&R C:	compilers could reorder FP operations at will; the user had to
        use explicit temporaries to defeat this.

Java:   not only do operators of equal precedence associate from left to
	right, but so does order of evaluation.

If -ffast-math is not specified, we should follow the language standards.
First question: can tree-ssa distinguish, for Fortran, whether parentheses
were present and so whether reordering is allowed?

If -ffast-math is specified, what should we do?  One option is to use the
K&R rule and free-associate at will.  I feel uncomfortable with that
because of the major loss in accuracy that can result.  If, however, we
implement support for the Fortran rules, one option would be to relax
order of evaluation when there are no parentheses (like Fortran), another
would be to just leave the order the same.