Parentheses not honored when using FMA

Parentheses not honored when using FMA

[Marcin Krotkiewski]

Hello, all,

I have doubts about asm generated for the following code that performs 
orientation test for a point and a segment:

double orient_test_2d(const double m[2], const double a[2], const double 
b[2])
{
   double am1 = a[0]-m[0];
   double bm1 = b[1]-m[1];
   double am2 = a[1]-m[1];
   double bm2 = b[0]-m[0];

   return ((am1)*(bm1)) - ((am2)*(bm2));
}

In the return statement the operands are all in parentheses. gcc 
optimizes the statement and introduces a FMA instruction. I think this 
is wrong because FMA causes the subtraction and multiplication to be 
effectively executed at the same time, while the source specifies that 
the multiplications should be performed before the subtraction.

This is the assembly generated by 'gcc-5.1.0 -O3 -mfma test.c -c -S'

         .cfi_startproc
         vmovsd  8(%rdi), %xmm0
         vmovsd  8(%rsi), %xmm2
         vmovsd  (%rdi), %xmm1
         vsubsd  %xmm0, %xmm2, %xmm3
         vmovsd  (%rdx), %xmm2
         vsubsd  %xmm1, %xmm2, %xmm2
         vmulsd  %xmm2, %xmm3, %xmm2
         vmovsd  (%rsi), %xmm3
         vsubsd  %xmm1, %xmm3, %xmm3
         vmovsd  8(%rdx), %xmm1
         vsubsd  %xmm0, %xmm1, %xmm1
         vfmsub132sd     %xmm1, %xmm2, %xmm3
         vmovapd %xmm3, %xmm0
         ret
         .cfi_endproc

I believe that in order to honor the parentheses gcc should NOT generate 
the FMA, but stick to the programmed operation order. If I wanted a FMA, 
I would not use parentheses around the multiplications. Current behavior 
of gcc results in inconsistent behavior in my code due to floating point 
arithmetic errors.

As a side note, the Intel compiler does exactly the same thing.

Can this be considered a bug, or is this the expected behavior? If this 
is ok, how can I assure in a portable way the effect that I need?

Thanks!

Marcin Krotkiewski

Re: Parentheses not honored when using FMA

[Marc Glisse]

On Wed, 12 Aug 2015, Marcin Krotkiewski wrote:

> Hello, all,
>
> I have doubts about asm generated for the following code that performs 
> orientation test for a point and a segment:
>
> double orient_test_2d(const double m[2], const double a[2], const double 
> b[2])
> {
>  double am1 = a[0]-m[0];
>  double bm1 = b[1]-m[1];
>  double am2 = a[1]-m[1];
>  double bm2 = b[0]-m[0];
>
>  return ((am1)*(bm1)) - ((am2)*(bm2));
> }
>
> In the return statement the operands are all in parentheses. gcc optimizes

Parentheses don't have the meaning you believe they have in C. All those 
in your return statement are useless.

> the statement and introduces a FMA instruction. I think this is wrong because 
> FMA causes the subtraction and multiplication to be effectively executed at 
> the same time, while the source specifies that the multiplications should be 
> performed before the subtraction.

-ffp-contract=off

-- 
Marc Glisse

Re: Parentheses not honored when using FMA

[Tim Prince]

On 8/12/2015 7:03 AM, Marc Glisse wrote:
> On Wed, 12 Aug 2015, Marcin Krotkiewski wrote:
>
>> Hello, all,
>>
>> I have doubts about asm generated for the following code that
>> performs orientation test for a point and a segment:
>>
>> double orient_test_2d(const double m[2], const double a[2], const
>> double b[2])
>> {
>>  double am1 = a[0]-m[0];
>>  double bm1 = b[1]-m[1];
>>  double am2 = a[1]-m[1];
>>  double bm2 = b[0]-m[0];
>>
>>  return ((am1)*(bm1)) - ((am2)*(bm2));
>> }
>>
>> In the return statement the operands are all in parentheses. gcc
>> optimizes
>
> Parentheses don't have the meaning you believe they have in C. All
> those in your return statement are useless.
No, this is a -std=gnu99 peculiarity which resembles K&R treatment.  I
know some people do mean K&R when they say C.
>
>> the statement and introduces a FMA instruction. I think this is wrong
>> because FMA causes the subtraction and multiplication to be
>> effectively executed at the same time, while the source specifies
>> that the multiplications should be performed before the subtraction.
>
> -ffp-contract=off
>
Why not gcc -std=c99 .... ?  It appears to answer the original question. 

Speaking of compilers requiring options to achieve standard behavior:
icl -Qprotect-parens or icl -fp:strict  will suppress the fma on this
example.

Neither gcc nor icc are capable of always optimizing use (or not) of
fma.  We've done a lot of benchmarking of -mno-fma vs. (default)
-mfma.   icc is improving in this respect. 
g++ inner_product is slow with fma.  Altering the template seems
hopeless, as it doesn't optimize without -ffast-math (which discards
this aspect of -std=c99).

Why did gcc reject -fprotect-parens?  This is one of the reasons I find
I must split my gcc compiled source code into files which are to be
built with or without -ffast-math .

Going the other way, to cases where fma is preferable, I find it
annoying that gfortran translates
s(i)*aa(i)+aa(i)
the same as
(s(i)+1.)*aa(i)
and requires (what I don't consider obvious)
(s(i)*aa(i))+aa(i)
to implement by fma.  gcc doesn't try this non-optimization.  Fortran
rules do encourage the replacement
a*b + c*b => b*(a+c)  (avoiding a questionable use of fma)
but I dislike this variant of that optimization.

-- 
Tim Prince

Re: Parentheses not honored when using FMA

[Tim Prince]

On 8/12/2015 7:03 AM, Marc Glisse wrote:
> On Wed, 12 Aug 2015, Marcin Krotkiewski wrote:
>
>> Hello, all,
>>
>> I have doubts about asm generated for the following code that
>> performs orientation test for a point and a segment:
>>
>> double orient_test_2d(const double m[2], const double a[2], const
>> double b[2])
>> {
>>  double am1 = a[0]-m[0];
>>  double bm1 = b[1]-m[1];
>>  double am2 = a[1]-m[1];
>>  double bm2 = b[0]-m[0];
>>
>>  return ((am1)*(bm1)) - ((am2)*(bm2));
>> }
>>
>> In the return statement the operands are all in parentheses. gcc
>> optimizes
>
>
> Parentheses don't have the meaning you believe they have in C. All
> those in your return statement are useless.
No, this is a -std=gnu99 peculiarity which resembles K&R treatment.  I
know some people do mean K&R when they say C.
>
>> the statement and introduces a FMA instruction. I think this is wrong
>> because FMA causes the subtraction and multiplication to be
>> effectively executed at the same time, while the source specifies
>> that the multiplications should be performed before the subtraction.
>
> -ffp-contract=off
>
Why not gcc -std=c99 .... ?  It appears to answer the original question.

Speaking of compilers requiring options to achieve standard behavior:
icl -Qprotect-parens or icl -fp:strict  will suppress the fma on this
example.

Neither gcc nor icc are capable of always optimizing use (or not) of
fma.  We've done a lot of benchmarking of -mno-fma vs. (default)
-mfma.   icc is improving in this respect.
g++ inner_product is slow with fma.  Altering the template seems
hopeless, as it doesn't optimize without -ffast-math (which discards
this aspect of -std=c99).

Why did gcc reject -fprotect-parens?  This is one of the reasons I find
I must split my gcc compiled source code into files which are to be
built with or without -ffast-math .

Going the other way, to cases where fma is preferable, I find it
annoying that gfortran translates
s(i)*aa(i)+aa(i)
the same as
(s(i)+1.)*aa(i)
and requires (what I don't consider obvious)
(s(i)*aa(i))+aa(i)
to implement by fma.  gcc doesn't try this non-optimization.  Fortran
rules do encourage the replacement
a*b + c*b => b*(a+c)  (avoiding a questionable use of fma)
but I dislike this variant of that optimization.

-- 
Tim Prince

Fwd: Re: Parentheses not honored when using FMA

[Tim Prince]

	

	

	

	

	



On 8/12/2015 7:03 AM, Marc Glisse wrote:
> On Wed, 12 Aug 2015, Marcin Krotkiewski wrote:
>
>> Hello, all,
>>
>> I have doubts about asm generated for the following code that
>> performs orientation test for a point and a segment:
>>
>> double orient_test_2d(const double m[2], const double a[2], const
>> double b[2])
>> {
>>  double am1 = a[0]-m[0];
>>  double bm1 = b[1]-m[1];
>>  double am2 = a[1]-m[1];
>>  double bm2 = b[0]-m[0];
>>
>>  return ((am1)*(bm1)) - ((am2)*(bm2));
>> }
>>
>> In the return statement the operands are all in parentheses. gcc
>> optimizes
>
> Parentheses don't have the meaning you believe they have in C. All
> those in your return statement are useless.
No, this is a -std=gnu99 peculiarity which resembles K&R treatment.  I
know some people do mean K&R when they say C.
>
>> the statement and introduces a FMA instruction. I think this is wrong
>> because FMA causes the subtraction and multiplication to be
>> effectively executed at the same time, while the source specifies
>> that the multiplications should be performed before the subtraction.
>
> -ffp-contract=off
>
Why not gcc -std=c99 .... ?  It appears to answer the original question. 

Speaking of compilers requiring options to achieve standard behavior:
icl -Qprotect-parens or icl -fp:strict  will suppress the fma on this
example.

Neither gcc nor icc are capable of always optimizing use (or not) of
fma.  We've done a lot of benchmarking of -mno-fma vs. (default)
-mfma.   icc is improving in this respect. 
g++ inner_product is slow with fma.  Altering the template seems
hopeless, as it doesn't optimize without -ffast-math (which discards
this aspect of -std=c99).

Why did gcc reject -fprotect-parens?  This is one of the reasons I find
I must split my gcc compiled source code into files which are to be
built with or without -ffast-math .

Going the other way, to cases where fma is preferable, I find it
annoying that gfortran translates
s(i)*aa(i)+aa(i)
the same as
(s(i)+1.)*aa(i)
and requires (what I don't consider obvious)
(s(i)*aa(i))+aa(i)
to implement by fma.  gcc doesn't try this non-optimization.  Fortran
rules do encourage the replacement
a*b + c*b => b*(a+c)  (avoiding a questionable use of fma)
but I dislike this variant of that optimization.

-- 
Tim Prince

Re: Fwd: Re: Parentheses not honored when using FMA

[Tim Prince]

On 8/12/2015 10:03 AM, Tim Prince wrote:
> On 8/12/2015 7:03 AM, Marc Glisse wrote:
>> On Wed, 12 Aug 2015, Marcin Krotkiewski wrote:
>>
>>> Hello, all,
>>>
>>> I have doubts about asm generated for the following code that
>>> performs orientation test for a point and a segment:
>>>
>>> double orient_test_2d(const double m[2], const double a[2], const
>>> double b[2])
>>> {
>>>  double am1 = a[0]-m[0];
>>>  double bm1 = b[1]-m[1];
>>>  double am2 = a[1]-m[1];
>>>  double bm2 = b[0]-m[0];
>>>
>>>  return ((am1)*(bm1)) - ((am2)*(bm2));
>>> }
>>>
>>> In the return statement the operands are all in parentheses. gcc
>>> optimizes
>> Parentheses don't have the meaning you believe they have in C. All
>> those in your return statement are useless.
> No, this is a -std=gnu99 peculiarity which resembles K&R treatment.  I
> know some people do mean K&R when they say C.
>>> the statement and introduces a FMA instruction. I think this is wrong
>>> because FMA causes the subtraction and multiplication to be
>>> effectively executed at the same time, while the source specifies
>>> that the multiplications should be performed before the subtraction.
>> -ffp-contract=off
>>
> Why not gcc -std=c99 .... ?  It appears to answer the original question. 
>
> Speaking of compilers requiring options to achieve standard behavior:
> icl -Qprotect-parens or icl -fp:strict  will suppress the fma on this
> example.
>
> Neither gcc nor icc are capable of always optimizing use (or not) of
> fma.  We've done a lot of benchmarking of -mno-fma vs. (default)
> -mfma.   icc is improving in this respect. 
> g++ inner_product is slow with fma.  Altering the template seems
> hopeless, as it doesn't optimize without -ffast-math (which discards
> this aspect of -std=c99).
>
> Why did gcc reject -fprotect-parens?  This is one of the reasons I find
> I must split my gcc compiled source code into files which are to be
> built with or without -ffast-math .
>
> Going the other way, to cases where fma is preferable, I find it
> annoying that gfortran translates
> s(i)*aa(i)+aa(i)
> the same as
> (s(i)+1.)*aa(i)
> and requires (what I don't consider obvious)
> (s(i)*aa(i))+aa(i)
> to implement by fma.  gcc doesn't try this non-optimization.  Fortran
> rules do encourage the replacement
> a*b + c*b => b*(a+c)  (avoiding a questionable use of fma)
> but I dislike this variant of that optimization.
The current Microsoft compiler makes the +1f replacement at /fp:fast. 
Without :fast, it doesn't use fma.

Re: Parentheses not honored when using FMA

[Vincent Lefevre]

On 2015-08-12 08:40:47 -0400, Tim Prince wrote:
> On 8/12/2015 7:03 AM, Marc Glisse wrote:
> > On Wed, 12 Aug 2015, Marcin Krotkiewski wrote:
> >
> >> Hello, all,
> >>
> >> I have doubts about asm generated for the following code that
> >> performs orientation test for a point and a segment:
> >>
> >> double orient_test_2d(const double m[2], const double a[2], const
> >> double b[2])
> >> {
> >>  double am1 = a[0]-m[0];
> >>  double bm1 = b[1]-m[1];
> >>  double am2 = a[1]-m[1];
> >>  double bm2 = b[0]-m[0];
> >>
> >>  return ((am1)*(bm1)) - ((am2)*(bm2));
> >> }
> >>
> >> In the return statement the operands are all in parentheses. gcc
> >> optimizes
> >
> > Parentheses don't have the meaning you believe they have in C. All
> > those in your return statement are useless.
> No, this is a -std=gnu99 peculiarity which resembles K&R treatment.  I
> know some people do mean K&R when they say C.

ISO C. The parentheses are useless here. I mean, according to the
ISO C99 standard, they don't change the meaning of the expression.
AFAIK, the C99 standard just says:

  A parenthesized expression is a primary expression. Its type and
  value are identical to those of the unparenthesized expression.

Ditto for the current ISO C11 standard.

> >> the statement and introduces a FMA instruction. I think this is wrong
> >> because FMA causes the subtraction and multiplication to be
> >> effectively executed at the same time, while the source specifies
> >> that the multiplications should be performed before the subtraction.
> >
> > -ffp-contract=off
> >
> Why not gcc -std=c99 .... ?  It appears to answer the original question. 

Probably because with recent GCC versions, -ffp-contract=on is now
an alias for -ffp-contract=off in ISO C mode, due to the fact that
the FP_CONTRACT pragma is not implemented by GCC. This has been
changed (fixed) by:

2013-11-06 16:52:47 (rev 204460, jsm28)

  c-family:
        * c-opts.c (c_common_post_options): Set -ffp-contract=off in C
        standards modes.
        * c-cppbuiltin.c (cpp_iec_559_value): Consider -ffp-contract=fast
        to mean lack of IEEE 754 support.

  testsuite:
        * gcc.dg/torture/c99-contract-1.c: New test.

gcc/c-family/c-opts.c now contains:

  /* ISO C restricts floating-point expression contraction to within
     source-language expressions (-ffp-contract=on, currently an alias
                                                    ^^^^^^^^^
     for -ffp-contract=off).  */
  if (flag_iso
      && !c_dialect_cxx ()
      && (global_options_set.x_flag_fp_contract_mode
          == (enum fp_contract_mode) 0)
      && flag_unsafe_math_optimizations == 0)
    flag_fp_contract_mode = FP_CONTRACT_OFF;

See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37845

But this may change once the FP_CONTRACT pragma is implemented in GCC.
So, for future GCC versions and other compilers, one *also* needs:

#pragma STDC FP_CONTRACT OFF

An advantage of the pragma is that one may forbid FP contraction
only on some parts of the program.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)