Should -ffp-contract=off the default on GCC?

Should -ffp-contract=off the default on GCC?

[Qing Zhao]

Hi,

Recently, we discovered some floating point precision diffs when using GCC8 to build our 
application on arm64: After some investigation, it turns out that this is due to the 
-ffp-contract=fast option that is on by default. Therefore, we have to explicitly add 
-ffp-contract=off and do a full-rebuild.  

GCC by default turns -ffp-contract=fast on.
https://gcc.gnu.org/onlinedocs/gcc-8.5.0/gcc/Optimize-Options.html#Optimize-Options
https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Optimize-Options.html#Optimize-Options

"
-ffp-contract=style
-ffp-contract=off disables floating-point expression contraction. -ffp-contract=fast enables
floating-point expression contraction such as forming of fused multiply-add operations if 
the target has native support for them. -ffp-contract=on enables floating-point expression
contraction if allowed by the language standard. This is currently not implemented and 
treated equal to -ffp-contract=off.

The default is -ffp-contract=fast.
"

This can be shown by a small example for arm64 with gcc8.5 in https://godbolt.org/z/MxYfnG8TE. 
Only when adding -std=c89 explicitly, this transformaton is off.

another exmaple also shows that Clang and MSVC only allow this transformation when speifiying
ffast-math and fp:fast:  https://godbolt.org/z/o54bYfPbP 

When searching online, we found that there were similar discussions recently on the exact same issue:
https://github.com/dotnet/runtime/issues/64604
https://github.com/dotnet/runtime/issues/64591

a summary of these discussions is:

1. "fmadd" is a fused operation and will return a different result for many inputs;  
2. therefore, -ffp-contract=fast is not a safe optimization to be on by default;
3. Clang and MSVC only allow this when specifying ffast-math and fp:fast since this is not an
  IEEE754 compliant optimization;
4. The reasons why GCC turns on this option by default are:
  A. GNU C language spec allows such transformation. 
  B. this did not expose real problem for most X86/X64 apps previously since FMA instructions
     didn't exist until 2013 when the FMA3 instruction set was added, and also these instructions
     were not always available.. 
5. Arm64 has fused multiply-add instructions as "baseline" and are always available. therefore
  -ffp-contract=fast exposed more serious problems on Arm64 platforms.  

our major question:

Should GCC turn off -ffp-contract=fast by default since it's not IEEE754 compliant and more
  modern processors have the FMA instructions available by default?

Thanks.

Qing

Re: Should -ffp-contract=off the default on GCC?

[Andrew Pinski]

On Thu, Mar 16, 2023 at 9:25 AM Qing Zhao via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> Hi,
>
> Recently, we discovered some floating point precision diffs when using GCC8 to build our
> application on arm64: After some investigation, it turns out that this is due to the
> -ffp-contract=fast option that is on by default. Therefore, we have to explicitly add
> -ffp-contract=off and do a full-rebuild.
>
> GCC by default turns -ffp-contract=fast on.
> https://gcc.gnu.org/onlinedocs/gcc-8.5.0/gcc/Optimize-Options.html#Optimize-Options
> https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Optimize-Options.html#Optimize-Options
>
> "
> -ffp-contract=style
> -ffp-contract=off disables floating-point expression contraction. -ffp-contract=fast enables
> floating-point expression contraction such as forming of fused multiply-add operations if
> the target has native support for them. -ffp-contract=on enables floating-point expression
> contraction if allowed by the language standard. This is currently not implemented and
> treated equal to -ffp-contract=off.
>
> The default is -ffp-contract=fast.
> "
>
> This can be shown by a small example for arm64 with gcc8.5 in https://godbolt.org/z/MxYfnG8TE.
> Only when adding -std=c89 explicitly, this transformaton is off.
>
> another exmaple also shows that Clang and MSVC only allow this transformation when speifiying
> ffast-math and fp:fast:  https://godbolt.org/z/o54bYfPbP
>
> When searching online, we found that there were similar discussions recently on the exact same issue:
> https://github.com/dotnet/runtime/issues/64604
> https://github.com/dotnet/runtime/issues/64591
>
> a summary of these discussions is:
>
> 1. "fmadd" is a fused operation and will return a different result for many inputs;
> 2. therefore, -ffp-contract=fast is not a safe optimization to be on by default;
> 3. Clang and MSVC only allow this when specifying ffast-math and fp:fast since this is not an
>   IEEE754 compliant optimization;
> 4. The reasons why GCC turns on this option by default are:
>   A. GNU C language spec allows such transformation.
>   B. this did not expose real problem for most X86/X64 apps previously since FMA instructions
>      didn't exist until 2013 when the FMA3 instruction set was added, and also these instructions
>      were not always available..
> 5. Arm64 has fused multiply-add instructions as "baseline" and are always available. therefore
>   -ffp-contract=fast exposed more serious problems on Arm64 platforms.

This summary ignores x87 and even ignores PowerPC in GCC having FMA
even before clang/LLVM was around.

>
> our major question:
>
> Should GCC turn off -ffp-contract=fast by default since it's not IEEE754 compliant and more
>   modern processors have the FMA instructions available by default?


NO. We have this debate every few years and such.

Thanks,
Andrew Pinski

>
> Thanks.
>
> Qing

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 16, 2023, at 12:31 PM, Andrew Pinski <pinskia@gmail.com> wrote:
> 
> On Thu, Mar 16, 2023 at 9:25 AM Qing Zhao via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>> 
>> Hi,
>> 
>> Recently, we discovered some floating point precision diffs when using GCC8 to build our
>> application on arm64: After some investigation, it turns out that this is due to the
>> -ffp-contract=fast option that is on by default. Therefore, we have to explicitly add
>> -ffp-contract=off and do a full-rebuild.
>> 
>> GCC by default turns -ffp-contract=fast on.
>> https://gcc.gnu.org/onlinedocs/gcc-8.5.0/gcc/Optimize-Options.html#Optimize-Options
>> https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Optimize-Options.html#Optimize-Options
>> 
>> "
>> -ffp-contract=style
>> -ffp-contract=off disables floating-point expression contraction. -ffp-contract=fast enables
>> floating-point expression contraction such as forming of fused multiply-add operations if
>> the target has native support for them. -ffp-contract=on enables floating-point expression
>> contraction if allowed by the language standard. This is currently not implemented and
>> treated equal to -ffp-contract=off.
>> 
>> The default is -ffp-contract=fast.
>> "
>> 
>> This can be shown by a small example for arm64 with gcc8.5 in https://godbolt.org/z/MxYfnG8TE.
>> Only when adding -std=c89 explicitly, this transformaton is off.
>> 
>> another exmaple also shows that Clang and MSVC only allow this transformation when speifiying
>> ffast-math and fp:fast:  https://godbolt.org/z/o54bYfPbP
>> 
>> When searching online, we found that there were similar discussions recently on the exact same issue:
>> https://github.com/dotnet/runtime/issues/64604
>> https://github.com/dotnet/runtime/issues/64591
>> 
>> a summary of these discussions is:
>> 
>> 1. "fmadd" is a fused operation and will return a different result for many inputs;
>> 2. therefore, -ffp-contract=fast is not a safe optimization to be on by default;
>> 3. Clang and MSVC only allow this when specifying ffast-math and fp:fast since this is not an
>>  IEEE754 compliant optimization;
>> 4. The reasons why GCC turns on this option by default are:
>>  A. GNU C language spec allows such transformation.
>>  B. this did not expose real problem for most X86/X64 apps previously since FMA instructions
>>     didn't exist until 2013 when the FMA3 instruction set was added, and also these instructions
>>     were not always available..
>> 5. Arm64 has fused multiply-add instructions as "baseline" and are always available. therefore
>>  -ffp-contract=fast exposed more serious problems on Arm64 platforms.
> 
> This summary ignores x87 and even ignores PowerPC in GCC having FMA
> even before clang/LLVM was around.

Okay. 
> 
>> 
>> our major question:
>> 
>> Should GCC turn off -ffp-contract=fast by default since it's not IEEE754 compliant and more
>>  modern processors have the FMA instructions available by default?
> 
> 
> NO. We have this debate every few years and such.

So, what’s the major reason we keep the default  that is not IEEE754 compliant from the beginning? 

thanks.

Qing
> 
> Thanks,
> Andrew Pinski
> 
>> 
>> Thanks.
>> 
>> Qing

Re: Should -ffp-contract=off the default on GCC?

[Jakub Jelinek]

On Thu, Mar 16, 2023 at 04:38:41PM +0000, Qing Zhao via Gcc-patches wrote:
> > NO. We have this debate every few years and such.
> 
> So, what’s the major reason we keep the default  that is not IEEE754 compliant from the beginning? 

It is compliant.  fusedMultiplyAdd is a standard IEEE 754 operation, and
C explicitly allows contractions.

	Jakub

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 16, 2023, at 12:53 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> 
> On Thu, Mar 16, 2023 at 04:38:41PM +0000, Qing Zhao via Gcc-patches wrote:
>>> NO. We have this debate every few years and such.
>> 
>> So, what’s the major reason we keep the default  that is not IEEE754 compliant from the beginning? 
> 
> It is compliant.  fusedMultiplyAdd is a standard IEEE 754 operation, and
> C explicitly allows contractions.

Okay, thanks for the info.

Yes, looks like that GCC’s default behavior is correct. -:)

Qing
> 
> 	Jakub
> 

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

Hi, 


> On Mar 16, 2023, at 12:53 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> 
> On Thu, Mar 16, 2023 at 04:38:41PM +0000, Qing Zhao via Gcc-patches wrote:
>>> NO. We have this debate every few years and such.
>> 
>> So, what’s the major reason we keep the default  that is not IEEE754 compliant from the beginning? 
> 
> It is compliant.  fusedMultiplyAdd is a standard IEEE 754 operation,

After checking the IEEE754, IEEE standard for Floating-Point Arithmetic, fusedMultiplyAdd is clearly documented:

"
fusedMultiplyAdd: The operation fusedMultiplyAdd(x, y, z) computes (x×y)+z as if with unbounded
range and precision, rounding only once to the destination format.
"

> and
> C explicitly allows contractions.

However, I failed to locate where in C standard the fusedMultiplyAdd is allowed.

I got the draft version of C89, C99 and C11 from the following links:
        • C89 – Draft version in ANSI text format: (https://web.archive.org/web/20161223125339/http://flash-gordon.me.uk/ansi.c.txt)
	• C89 – Draft version as HTML document: (http://port70.net/~nsz/c/c89/c89-draft.html)
	• C99 – Draft version (N1256) as HTML document: (http://port70.net/~nsz/c/c99/n1256.html)
	• C11 – Draft version (N1570) as HTML document: (http://port70.net/~nsz/c/c11/n1570.html)

In both C99 and C11, I Only found the following section:

http://port70.net/%7Ensz/c/c99/n1256.html#7.12.13:

====
7.12.13 Floating multiply-add

7.12.13.1 The fma functions

Synopsis

1

         #include <math.h>

         double fma(double x, double y, double z);
         float fmaf(float x, float y, float z);
         long double fmal(long double x, long double y,
              long double z);

Description

2 The fma functions compute (x y) + z, rounded as one ternary operation: they compute the value (as if) to infinite precision and round once to the result format, according to the current rounding mode. A range error may occur.

Returns

3 The fma functions return (x y) + z, rounded as one ternary operation.
====

My question: is the above section the place in C standard “explicitly allows contractions”? If not, where it is in C standard?

Another question to ARM port maintainers is:

Is fmadd instruction provided by Aarch64 architecture IEEE754 compliant? 

Thanks a lot for your help.

Qing


> 
> 	Jakub
> 

Re: Should -ffp-contract=off the default on GCC?

[Jakub Jelinek]

On Mon, Mar 20, 2023 at 10:05:57PM +0000, Qing Zhao via Gcc-patches wrote:
> My question: is the above section the place in C standard “explicitly allows contractions”? If not, where it is in C standard?

http://port70.net/%7Ensz/c/c99/n1256.html#6.5p8
http://port70.net/%7Ensz/c/c99/n1256.html#note78
http://port70.net/%7Ensz/c/c99/n1256.html#F.6

	Jakub

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

Thanks a lot for the info.

Qing

> On Mar 20, 2023, at 6:25 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> 
> On Mon, Mar 20, 2023 at 10:05:57PM +0000, Qing Zhao via Gcc-patches wrote:
>> My question: is the above section the place in C standard “explicitly allows contractions”? If not, where it is in C standard?
> 
> http://port70.net/%7Ensz/c/c99/n1256.html#6.5p8
> http://port70.net/%7Ensz/c/c99/n1256.html#note78
> http://port70.net/%7Ensz/c/c99/n1256.html#F.6
> 
> 	Jakub
> 

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 20, 2023, at 6:25 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> 
> On Mon, Mar 20, 2023 at 10:05:57PM +0000, Qing Zhao via Gcc-patches wrote:
>> My question: is the above section the place in C standard “explicitly allows contractions”? If not, where it is in C standard?
> 
> http://port70.net/%7Ensz/c/c99/n1256.html#6.5p8
> http://port70.net/%7Ensz/c/c99/n1256.html#note78

78) This license is specifically intended to allow implementations to exploit fast machine instructions that combine multiple C operators. As contractions potentially undermine predictability, and can even decrease accuracy for containing expressions, their use needs to be well-defined and clearly documented.

Looks like that the C99 standard clearly warned that the fp-contract might “undermine predictability”, “can even decrease accuracy” at the same time to allow it.  Should we also provide such warning in our gcc documentation for -fp-contract (currently, there is no such warning):

https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Optimize-Options
"
-ffp-contract=style
-ffp-contract=off disables floating-point expression contraction. -ffp-contract=fast enables floating-point expression contraction such as forming of fused multiply-add operations if the target has native support for them. -ffp-contract=on enables floating-point expression contraction if allowed by the language standard. This is currently not implemented and treated equal to -ffp-contract=off.

The default is -ffp-contract=fast.”

Most of the compiler users are not familiar with language standards, or no access to language standards. Without clearly documenting such warnings along with the option explicitly, the users have not way to know such potential impact. They will be confused with the result they see and raise the same questions to GCC community again and again. 

thanks.

Qing


> http://port70.net/%7Ensz/c/c99/n1256.html#F.6
> 
> 	Jakub
> 

Re: Should -ffp-contract=off the default on GCC?

[Paul Koning]

> On Mar 21, 2023, at 11:01 AM, Qing Zhao via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> ...
> Most of the compiler users are not familiar with language standards, or no access to language standards. Without clearly documenting such warnings along with the option explicitly, the users have not way to know such potential impact.

With modern highly optimized languages, not knowing the standard is going to get you in trouble.  There was a wonderful paper from MIT a few years ago describing all the many ways C can bite you if you don't know the rules.

	paul

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 21, 2023, at 12:56 PM, Paul Koning <paulkoning@comcast.net> wrote:
> 
> 
> 
>> On Mar 21, 2023, at 11:01 AM, Qing Zhao via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>> 
>> ...
>> Most of the compiler users are not familiar with language standards, or no access to language standards. Without clearly documenting such warnings along with the option explicitly, the users have not way to know such potential impact.
> 
> With modern highly optimized languages, not knowing the standard is going to get you in trouble.  There was a wonderful paper from MIT a few years ago describing all the many ways C can bite you if you don't know the rules.

Yes, it’s better to know the details of languages standard. -:)
However, I don’t think that this is a realistic expectation to the compiler users:  to know all the details of a language standard.

Qing
> 
> 	paul
> 

Re: Should -ffp-contract=off the default on GCC?

[Jeff Law]

On 3/21/23 11:00, Qing Zhao via Gcc-patches wrote:
> 
> 
>> On Mar 21, 2023, at 12:56 PM, Paul Koning <paulkoning@comcast.net> wrote:
>>
>>
>>
>>> On Mar 21, 2023, at 11:01 AM, Qing Zhao via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>>>
>>> ...
>>> Most of the compiler users are not familiar with language standards, or no access to language standards. Without clearly documenting such warnings along with the option explicitly, the users have not way to know such potential impact.
>>
>> With modern highly optimized languages, not knowing the standard is going to get you in trouble.  There was a wonderful paper from MIT a few years ago describing all the many ways C can bite you if you don't know the rules.
> 
> Yes, it’s better to know the details of languages standard. -:)
> However, I don’t think that this is a realistic expectation to the compiler users:  to know all the details of a language standard.
Umm, they really do need to know that stuff.

If the developer fails to understand the language standard, then they're 
likely going to write code that is ultimately undefined or doesn't 
behave in they expect.  How is the compiler supposed to guess what the 
developer originally intended?  How should the compiler handle the case 
when two developers have different understandings of how a particular 
piece of code should work?  In the end it's the language standard that 
defines how all this stuff should work.

Failure to understand the language is a common problem and we do try to 
emit various diagnostics to help developers avoid writing non-conformant 
code.  But ultimately if a developer fails to understand the language 
standard, then they're going to be surprised by the behavior of their code.

Jeff

Re: Should -ffp-contract=off the default on GCC?

[Paul Koning]

> On Mar 21, 2023, at 1:59 PM, Jeff Law via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> 
> 
> On 3/21/23 11:00, Qing Zhao via Gcc-patches wrote:
>>> On Mar 21, 2023, at 12:56 PM, Paul Koning <paulkoning@comcast.net> wrote:
>>> 
>>> 
>>> 
>>>> On Mar 21, 2023, at 11:01 AM, Qing Zhao via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>>>> 
>>>> ...
>>>> Most of the compiler users are not familiar with language standards, or no access to language standards. Without clearly documenting such warnings along with the option explicitly, the users have not way to know such potential impact.
>>> 
>>> With modern highly optimized languages, not knowing the standard is going to get you in trouble.  There was a wonderful paper from MIT a few years ago describing all the many ways C can bite you if you don't know the rules.
>> Yes, it’s better to know the details of languages standard. -:)
>> However, I don’t think that this is a realistic expectation to the compiler users:  to know all the details of a language standard.
> Umm, they really do need to know that stuff.
> 
> If the developer fails to understand the language standard, then they're likely going to write code that is ultimately undefined or doesn't behave in they expect.  How is the compiler supposed to guess what the developer originally intended?  How should the compiler handle the case when two developers have different understandings of how a particular piece of code should work?  In the end it's the language standard that defines how all this stuff should work.
> 
> Failure to understand the language is a common problem and we do try to emit various diagnostics to help developers avoid writing non-conformant code.  But ultimately if a developer fails to understand the language standard, then they're going to be surprised by the behavior of their code.

Conversely, of course, the problem is that C and other languages have evolved to the point that you have to be a language lawyer to write valid code.  In other words, a substantial fraction of programmers are by definition writing unreliable code.  This is not a good situation, and it may be part of the reason why modern software has such a high rate of defects.

	paul

Re: Should -ffp-contract=off the default on GCC?

[Alexander Monakov]

[-- Attachment #1: Type: text/plain, Size: 2111 bytes --]


On Tue, 21 Mar 2023, Jeff Law via Gcc-patches wrote:

> On 3/21/23 11:00, Qing Zhao via Gcc-patches wrote:
> > 
> >> On Mar 21, 2023, at 12:56 PM, Paul Koning <paulkoning@comcast.net> wrote:
> >>
> >>> On Mar 21, 2023, at 11:01 AM, Qing Zhao via Gcc-patches
> >>> <gcc-patches@gcc.gnu.org> wrote:
> >>>
> >>> ...
> >>> Most of the compiler users are not familiar with language standards, or no
> >>> access to language standards. Without clearly documenting such warnings
> >>> along with the option explicitly, the users have not way to know such
> >>> potential impact.
> >>
> >> With modern highly optimized languages, not knowing the standard is going
> >> to get you in trouble.  There was a wonderful paper from MIT a few years
> >> ago describing all the many ways C can bite you if you don't know the
> >> rules.
> > 
> > Yes, it’s better to know the details of languages standard. -:)
> > However, I don’t think that this is a realistic expectation to the compiler
> > users:  to know all the details of a language standard.
> Umm, they really do need to know that stuff.
> 
> If the developer fails to understand the language standard, then they're
> likely going to write code that is ultimately undefined or doesn't behave in
> they expect.  How is the compiler supposed to guess what the developer
> originally intended?  How should the compiler handle the case when two
> developers have different understandings of how a particular piece of code
> should work?  In the end it's the language standard that defines how all this
> stuff should work.
> 
> Failure to understand the language is a common problem and we do try to emit
> various diagnostics to help developers avoid writing non-conformant code.  But
> ultimately if a developer fails to understand the language standard, then
> they're going to be surprised by the behavior of their code.

W h a t.

This subthread concerns documenting the option better ("Without clearly
documenting such warnings ...").

Are you arguing against adding a brief notice to the documentation blurb for
the -ffp-contract= option?

Perplexed,
Alexander

Re: Should -ffp-contract=off the default on GCC?

[Jeff Law]

On 3/21/23 12:12, Alexander Monakov wrote:
>>> Yes, it’s better to know the details of languages standard. -:)
>>> However, I don’t think that this is a realistic expectation to the compiler
>>> users:  to know all the details of a language standard.
>> Umm, they really do need to know that stuff.
>>
>> If the developer fails to understand the language standard, then they're
>> likely going to write code that is ultimately undefined or doesn't behave in
>> they expect.  How is the compiler supposed to guess what the developer
>> originally intended?  How should the compiler handle the case when two
>> developers have different understandings of how a particular piece of code
>> should work?  In the end it's the language standard that defines how all this
>> stuff should work.
>>
>> Failure to understand the language is a common problem and we do try to emit
>> various diagnostics to help developers avoid writing non-conformant code.  But
>> ultimately if a developer fails to understand the language standard, then
>> they're going to be surprised by the behavior of their code.
> 
> W h a t.
> 
> This subthread concerns documenting the option better ("Without clearly
> documenting such warnings ...").
> 
> Are you arguing against adding a brief notice to the documentation blurb for
> the -ffp-contract= option?
I was merely chiming in on Qing's statement that it is not realistic to 
expect users to know the details of the language standard.



Jeff

Re: Should -ffp-contract=off the default on GCC?

[Toon Moene]

On 3/21/23 19:03, Paul Koning via Gcc-patches wrote:

>> Failure to understand the language is a common problem and we do try to emit various diagnostics to help developers avoid writing non-conformant code.  But ultimately if a developer fails to understand the language standard, then they're going to be surprised by the behavior of their code.
> 
> Conversely, of course, the problem is that C and other languages have evolved to the point that you have to be a language lawyer to write valid code.  In other words, a substantial fraction of programmers are by definition writing unreliable code.  This is not a good situation, and it may be part of the reason why modern software has such a high rate of defects.

Fortran compilers that I use regularly (I mean, aside from gfortran) 
have already given up on this battle, at least as far as floating point 
issues are concerned.

So many people want to have "repeatable floating point computations" 
that if someone writes:

READ*, X, Y, Z
PRINT*, X + Y + Z
END

they will get (if they know the compiler option that guarantees this - 
but they will) the following code:

READ*, X, Y, Z
PRINT*, (X + Y) + Z
END

even though there's no way in hell the Fortran Language Standard (any of 
them) guarantees this.

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 21, 2023, at 1:59 PM, Jeff Law via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> 
> 
> On 3/21/23 11:00, Qing Zhao via Gcc-patches wrote:
>>> On Mar 21, 2023, at 12:56 PM, Paul Koning <paulkoning@comcast.net> wrote:
>>> 
>>> 
>>> 
>>>> On Mar 21, 2023, at 11:01 AM, Qing Zhao via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>>>> 
>>>> ...
>>>> Most of the compiler users are not familiar with language standards, or no access to language standards. Without clearly documenting such warnings along with the option explicitly, the users have not way to know such potential impact.
>>> 
>>> With modern highly optimized languages, not knowing the standard is going to get you in trouble.  There was a wonderful paper from MIT a few years ago describing all the many ways C can bite you if you don't know the rules.
>> Yes, it’s better to know the details of languages standard. -:)
>> However, I don’t think that this is a realistic expectation to the compiler users:  to know all the details of a language standard.
> Umm, they really do need to know that stuff.
> 
> If the developer fails to understand the language standard, then they're likely going to write code that is ultimately undefined or doesn't behave in they expect.  How is the compiler supposed to guess what the developer originally intended?  How should the compiler handle the case when two developers have different understandings of how a particular piece of code should work?  In the end it's the language standard that defines how all this stuff should work.
Theoretically, yes, I agree with you.

But in reality, many programmers don’t know all the details of the language standard and writing problematic code, that’s one of the reasons the compiler issues different warnings to the users in order to avoid those issues. 

In addition to this, Standards have been changed from time to time.

The code previously worked now has some issue since we added some new stuff into standard, and the compiler added some new transformation based on this new stuff. Should the compiler issue some warnings to warn the users about such change? Then the user will go to the new standard to get more info? 

Qing

> 
> Failure to understand the language is a common problem and we do try to emit various diagnostics to help developers avoid writing non-conformant code.  But ultimately if a developer fails to understand the language standard, then they're going to be surprised by the behavior of their code.
> 
> Jeff

Re: Should -ffp-contract=off the default on GCC?

[Jakub Jelinek]

On Tue, Mar 21, 2023 at 07:01:36PM +0000, Qing Zhao via Gcc-patches wrote:
> In addition to this, Standards have been changed from time to time.

So, the user needs to know the standard they are compiling for.

Anyway, talking again about contractions, it isn't anything new in the
standard, C99 had those too.

	Jakub

Re: Should -ffp-contract=off the default on GCC?

[Jeff Law]

On 3/21/23 13:01, Qing Zhao wrote:

> 
> The code previously worked now has some issue since we added some new stuff into standard, and the compiler added some new transformation based on this new stuff. Should the compiler issue some warnings to warn the users about such change? Then the user will go to the new standard to get more info?
It is not at all uncommon to have old code no longer work due to a 
compiler update.

This can happen if the compiler changes the default version of the 
language it's supporting (say c99 to c11) or fixes a missed-error bug 
(more common in C++) or the optimizers just get smarter and code which 
was always buggy, but worked by accident no longer works.

When we can reasonably give diagnostics, we try to.  Improvements in 
this space are appreciated :-)

Jeff

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 21, 2023, at 3:51 PM, Jeff Law <jeffreyalaw@gmail.com> wrote:
> 
> 
> 
> On 3/21/23 13:01, Qing Zhao wrote:
> 
>> The code previously worked now has some issue since we added some new stuff into standard, and the compiler added some new transformation based on this new stuff. Should the compiler issue some warnings to warn the users about such change? Then the user will go to the new standard to get more info?
> It is not at all uncommon to have old code no longer work due to a compiler update.
> 
> This can happen if the compiler changes the default version of the language it's supporting (say c99 to c11) or fixes a missed-error bug (more common in C++) or the optimizers just get smarter and code which was always buggy, but worked by accident no longer works.
> 
> When we can reasonably give diagnostics, we try to.  Improvements in this space are appreciated :-)

For this specific case,  the C standard (C99 and later) explicitly mentioned that the fp-contract might “undermine predictability”, “can even decrease accuracy” at the same time to allow it.

http://port70.net/%7Ensz/c/c99/n1256.html#note78

78) This license is specifically intended to allow implementations to exploit fast machine instructions that combine multiple C operators. As contractions potentially undermine predictability, and can even decrease accuracy for containing expressions, their use needs to be well-defined and clearly documented.

In GCC, the fp-contract is on by default, but the documentation of it doesn’t mention such warning at all. I think it will be helpful to add explicit warning in the -ffp-contract documentation. 

If you agree on this, I can come up with a patch to add such warning for -ffp-contract.

Qing

> 
> Jeff
> 

Re: Should -ffp-contract=off the default on GCC?

[Jeff Law]

On 3/21/23 13:12, Jakub Jelinek wrote:
> On Tue, Mar 21, 2023 at 07:01:36PM +0000, Qing Zhao via Gcc-patches wrote:
>> In addition to this, Standards have been changed from time to time.
> 
> So, the user needs to know the standard they are compiling for.
> 
> Anyway, talking again about contractions, it isn't anything new in the
> standard, C99 had those too.
And I think Qing is asking if adding a warning in the manual about how 
fp contractions can cause unpredictability in FP results is appropriate 
in the fp-contract section.

I think that would be a fine addition to the manual.

jeff

Re: Should -ffp-contract=off the default on GCC?

[Jakub Jelinek]

On Tue, Mar 21, 2023 at 05:28:52PM -0600, Jeff Law via Gcc-patches wrote:
> On 3/21/23 13:12, Jakub Jelinek wrote:
> > On Tue, Mar 21, 2023 at 07:01:36PM +0000, Qing Zhao via Gcc-patches wrote:
> > > In addition to this, Standards have been changed from time to time.
> > 
> > So, the user needs to know the standard they are compiling for.
> > 
> > Anyway, talking again about contractions, it isn't anything new in the
> > standard, C99 had those too.
> And I think Qing is asking if adding a warning in the manual about how fp
> contractions can cause unpredictability in FP results is appropriate in the
> fp-contract section.
> 
> I think that would be a fine addition to the manual.

Depending on how it is worded, probably yes.

	Jakub

Re: Should -ffp-contract=off the default on GCC?

[Richard Biener]

On Tue, Mar 21, 2023 at 7:18 PM Jeff Law via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
>
>
> On 3/21/23 12:12, Alexander Monakov wrote:
> >>> Yes, it’s better to know the details of languages standard. -:)
> >>> However, I don’t think that this is a realistic expectation to the compiler
> >>> users:  to know all the details of a language standard.
> >> Umm, they really do need to know that stuff.
> >>
> >> If the developer fails to understand the language standard, then they're
> >> likely going to write code that is ultimately undefined or doesn't behave in
> >> they expect.  How is the compiler supposed to guess what the developer
> >> originally intended?  How should the compiler handle the case when two
> >> developers have different understandings of how a particular piece of code
> >> should work?  In the end it's the language standard that defines how all this
> >> stuff should work.
> >>
> >> Failure to understand the language is a common problem and we do try to emit
> >> various diagnostics to help developers avoid writing non-conformant code.  But
> >> ultimately if a developer fails to understand the language standard, then
> >> they're going to be surprised by the behavior of their code.
> >
> > W h a t.
> >
> > This subthread concerns documenting the option better ("Without clearly
> > documenting such warnings ...").
> >
> > Are you arguing against adding a brief notice to the documentation blurb for
> > the -ffp-contract= option?
> I was merely chiming in on Qing's statement that it is not realistic to
> expect users to know the details of the language standard.

I think it's even less realistic to expect users to know the details of
floating-point math.  So I doubt any such sentence will be helpful
besides spreading some FUD?

>
>
> Jeff

Re: Should -ffp-contract=off the default on GCC?

[Alexander Monakov]

On Wed, 22 Mar 2023, Richard Biener wrote:

> I think it's even less realistic to expect users to know the details of
> floating-point math.  So I doubt any such sentence will be helpful
> besides spreading some FUD?

I think it's closer to "fundamental notions" rather than "details". For
users who bother to read the GCC manual there's a decent chance it wouldn't
be for naught.

For documentation, I was thinking

  Together with -fexcess-precision=standard, -ffp-contract=off
  is necessary to ensure that rounding of intermediate results to precision
  implied by the source code and the FLT_EVAL_METHOD macro is not
  omitted by the compiler.

Alexander

Re: Should -ffp-contract=off the default on GCC?

[Alexander Monakov]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]


On Mon, 20 Mar 2023, Jakub Jelinek via Gcc-patches wrote:

> On Mon, Mar 20, 2023 at 10:05:57PM +0000, Qing Zhao via Gcc-patches wrote:
> > My question: is the above section the place in C standard “explicitly allows contractions”? If not, where it is in C standard?
> 
> http://port70.net/%7Ensz/c/c99/n1256.html#6.5p8
> http://port70.net/%7Ensz/c/c99/n1256.html#note78
> http://port70.net/%7Ensz/c/c99/n1256.html#F.6

C only allows contractions within expressions, not across statements (i.e.
either -ffp-contract=on or -ffp-contract=off would be compliant, but not
our default -ffp-contract=fast).

Unrestricted contraction across statements together with other optimizations
gives rise to difficult-to-debug issues such as PR 106902.

Alexander

Re: Should -ffp-contract=off the default on GCC?

[Richard Biener]

On Wed, Mar 22, 2023 at 1:26 PM Alexander Monakov <amonakov@ispras.ru> wrote:
>
>
> On Wed, 22 Mar 2023, Richard Biener wrote:
>
> > I think it's even less realistic to expect users to know the details of
> > floating-point math.  So I doubt any such sentence will be helpful
> > besides spreading some FUD?
>
> I think it's closer to "fundamental notions" rather than "details". For
> users who bother to read the GCC manual there's a decent chance it wouldn't
> be for naught.
>
> For documentation, I was thinking
>
>   Together with -fexcess-precision=standard, -ffp-contract=off
>   is necessary to ensure that rounding of intermediate results to precision
>   implied by the source code and the FLT_EVAL_METHOD macro is not
>   omitted by the compiler.

that sounds good to me

> Alexander

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 22, 2023, at 8:33 AM, Alexander Monakov <amonakov@ispras.ru> wrote:
> 
> 
> On Mon, 20 Mar 2023, Jakub Jelinek via Gcc-patches wrote:
> 
>> On Mon, Mar 20, 2023 at 10:05:57PM +0000, Qing Zhao via Gcc-patches wrote:
>>> My question: is the above section the place in C standard “explicitly allows contractions”? If not, where it is in C standard?
>> 
>> http://port70.net/%7Ensz/c/c99/n1256.html#6.5p8
>> http://port70.net/%7Ensz/c/c99/n1256.html#note78
>> http://port70.net/%7Ensz/c/c99/n1256.html#F.6
> 
> C only allows contractions within expressions, not across statements (i.e.
> either -ffp-contract=on or -ffp-contract=off would be compliant, but not
> our default -ffp-contract=fast).

Oh, thanks for the info.

Just read the documentation of -fp-contract=style again: -:)

"
-ffp-contract=style
-ffp-contract=off disables floating-point expression contraction. -ffp-contract=fast enables floating-point expression contraction such as forming of fused multiply-add operations if the target has native support for them. -ffp-contract=on enables floating-point expression contraction if allowed by the language standard. This is currently not implemented and treated equal to -ffp-contract=off.

The default is -ffp-contract=fast.”

I was a little confused about the difference between -ffp-contract=fast and -ffp-contract=on previously,  now I understand.  

So, looks like that it's -ffp-contract=on that is compliant with C standard, but not -ffp-contract=fast?  (However, my understanding from the above doc and also from the GCC source code is, currently, -ffp-contract=on is not implemented and is equal to -fp-contract=off).

Therefore, the default value of -fp-contract=fast is NOT compliant with the language standard? 

Do I miss anything here?

> 
> Unrestricted contraction across statements together with other optimizations
> gives rise to difficult-to-debug issues such as PR 106902.

Just read this bug’s comments, yes, I agree.

Qing
> 
> Alexander

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 22, 2023, at 9:57 AM, Richard Biener via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> On Wed, Mar 22, 2023 at 1:26 PM Alexander Monakov <amonakov@ispras.ru> wrote:
>> 
>> 
>> On Wed, 22 Mar 2023, Richard Biener wrote:
>> 
>>> I think it's even less realistic to expect users to know the details of
>>> floating-point math.  So I doubt any such sentence will be helpful
>>> besides spreading some FUD?
>> 
>> I think it's closer to "fundamental notions" rather than "details". For
>> users who bother to read the GCC manual there's a decent chance it wouldn't
>> be for naught.
>> 
>> For documentation, I was thinking
>> 
>>  Together with -fexcess-precision=standard, -ffp-contract=off
>>  is necessary to ensure that rounding of intermediate results to precision
>>  implied by the source code and the FLT_EVAL_METHOD macro is not
>>  omitted by the compiler.
> 
> that sounds good to me

Shall we add such clarification to our Gcc13 doc? That should be helpful if we keep the currently default.

Qing
> 
>> Alexander

Re: Should -ffp-contract=off the default on GCC?

[Fangrui Song]

On Wed, Mar 22, 2023 at 8:52 AM Qing Zhao via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
>
>
> > On Mar 22, 2023, at 9:57 AM, Richard Biener via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> >
> > On Wed, Mar 22, 2023 at 1:26 PM Alexander Monakov <amonakov@ispras.ru> wrote:
> >>
> >>
> >> On Wed, 22 Mar 2023, Richard Biener wrote:
> >>
> >>> I think it's even less realistic to expect users to know the details of
> >>> floating-point math.  So I doubt any such sentence will be helpful
> >>> besides spreading some FUD?
> >>
> >> I think it's closer to "fundamental notions" rather than "details". For
> >> users who bother to read the GCC manual there's a decent chance it wouldn't
> >> be for naught.
> >>
> >> For documentation, I was thinking
> >>
> >>  Together with -fexcess-precision=standard, -ffp-contract=off
> >>  is necessary to ensure that rounding of intermediate results to precision
> >>  implied by the source code and the FLT_EVAL_METHOD macro is not
> >>  omitted by the compiler.
> >
> > that sounds good to me
>
> Shall we add such clarification to our Gcc13 doc? That should be helpful if we keep the currently default.
>
> Qing
> >
> >> Alexander
>

While updating the documentation, consider adding information that
#pragma STDC FP_CONTRACT OFF is ignored with -ffp-contract=fast.

This surprising behavior motivated Clang to add
-Xclang=-ffp-contract=fast-honor-pragmas
(https://discourse.llvm.org/t/fp-contract-fast-and-pragmas/58529).



-- 
宋方睿

Re: Should -ffp-contract=off the default on GCC?

[Andrew Pinski]

On Fri, Mar 24, 2023 at 1:14 AM Fangrui Song via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> On Wed, Mar 22, 2023 at 8:52 AM Qing Zhao via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
> >
> >
> >
> > > On Mar 22, 2023, at 9:57 AM, Richard Biener via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> > >
> > > On Wed, Mar 22, 2023 at 1:26 PM Alexander Monakov <amonakov@ispras.ru> wrote:
> > >>
> > >>
> > >> On Wed, 22 Mar 2023, Richard Biener wrote:
> > >>
> > >>> I think it's even less realistic to expect users to know the details of
> > >>> floating-point math.  So I doubt any such sentence will be helpful
> > >>> besides spreading some FUD?
> > >>
> > >> I think it's closer to "fundamental notions" rather than "details". For
> > >> users who bother to read the GCC manual there's a decent chance it wouldn't
> > >> be for naught.
> > >>
> > >> For documentation, I was thinking
> > >>
> > >>  Together with -fexcess-precision=standard, -ffp-contract=off
> > >>  is necessary to ensure that rounding of intermediate results to precision
> > >>  implied by the source code and the FLT_EVAL_METHOD macro is not
> > >>  omitted by the compiler.
> > >
> > > that sounds good to me
> >
> > Shall we add such clarification to our Gcc13 doc? That should be helpful if we keep the currently default.
> >
> > Qing
> > >
> > >> Alexander
> >
>
> While updating the documentation, consider adding information that
> #pragma STDC FP_CONTRACT OFF is ignored with -ffp-contract=fast.
>
> This surprising behavior motivated Clang to add
> -Xclang=-ffp-contract=fast-honor-pragmas
> (https://discourse.llvm.org/t/fp-contract-fast-and-pragmas/58529).

`#pragma STDC FP_CONTRACT OFF` is not even implemented yet in GCC.
Rather we should document that :).
It does not matter what clang does here really since GCC does not even
implement the pragma.

Thanks,
Andrew Pinski


>
>
>
> --
> 宋方睿

Re: Should -ffp-contract=off the default on GCC?

[Qing Zhao]

> On Mar 24, 2023, at 3:42 PM, Andrew Pinski <pinskia@gmail.com> wrote:
> 
> On Fri, Mar 24, 2023 at 1:14 AM Fangrui Song via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>> 
>> On Wed, Mar 22, 2023 at 8:52 AM Qing Zhao via Gcc-patches
>> <gcc-patches@gcc.gnu.org> wrote:
>>> 
>>> 
>>> 
>>>> On Mar 22, 2023, at 9:57 AM, Richard Biener via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>>>> 
>>>> On Wed, Mar 22, 2023 at 1:26 PM Alexander Monakov <amonakov@ispras.ru> wrote:
>>>>> 
>>>>> 
>>>>> On Wed, 22 Mar 2023, Richard Biener wrote:
>>>>> 
>>>>>> I think it's even less realistic to expect users to know the details of
>>>>>> floating-point math.  So I doubt any such sentence will be helpful
>>>>>> besides spreading some FUD?
>>>>> 
>>>>> I think it's closer to "fundamental notions" rather than "details". For
>>>>> users who bother to read the GCC manual there's a decent chance it wouldn't
>>>>> be for naught.
>>>>> 
>>>>> For documentation, I was thinking
>>>>> 
>>>>> Together with -fexcess-precision=standard, -ffp-contract=off
>>>>> is necessary to ensure that rounding of intermediate results to precision
>>>>> implied by the source code and the FLT_EVAL_METHOD macro is not
>>>>> omitted by the compiler.
>>>> 
>>>> that sounds good to me
>>> 
>>> Shall we add such clarification to our Gcc13 doc? That should be helpful if we keep the currently default.
>>> 
>>> Qing
>>>> 
>>>>> Alexander
>>> 
>> 
>> While updating the documentation, consider adding information that
>> #pragma STDC FP_CONTRACT OFF is ignored with -ffp-contract=fast.
>> 
>> This surprising behavior motivated Clang to add
>> -Xclang=-ffp-contract=fast-honor-pragmas
>> (https://discourse.llvm.org/t/fp-contract-fast-and-pragmas/58529).
> 
> `#pragma STDC FP_CONTRACT OFF` is not even implemented yet in GCC.
> Rather we should document that :).

Do we have any plan to implement this pragma?
Also, do we have any plan to implement -ffp-contract=on? 

I am very curious on why -ffp-contract=on has not been implemented for so long time?

There are some documentation on Floating Point implementation of GCC on the pragma STDC FP_CONTRACT
https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Floating-point-implementation.html#Floating-point-implementation

"
	• Whether and how floating expressions are contracted when not disallowed by the FP_CONTRACT pragma (C99 and C11 6.5).
Expressions are currently only contracted if -ffp-contract=fast, -funsafe-math-optimizations or -ffast-math are used. This is subject to change.

	• The default state for the FP_CONTRACT pragma (C99 and C11 7.12.2).
This pragma is not implemented. Expressions are currently only contracted if -ffp-contract=fast, -funsafe-math-optimizations or -ffast-math are used. This is subject to change.
“

So, looks like that we have documented this. But When the user read the documentation for —ffp-contract option, there is no such information. We might add a link in the documentation of -ffp-contract option to here to make it clear.

Qing




> It does not matter what clang does here really since GCC does not even
> implement the pragma.
> 
> Thanks,
> Andrew Pinski
> 
> 
>> 
>> 
>> 
>> --
>> 宋方睿

Re: Should -ffp-contract=off the default on GCC?

[Zeson]

Any update on this thread discussion? And the thread was straying to the document of option and user-friendly stuff.
So does the default value of -ffp-contract=fast obey the C/C++ language standard? But why does clang not obey? Or is it just compiler implement-dependent which is not specified by standard?


Regards,
Zeson