Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[leon zadorin]

Under certain conditions GCC is able to perform compile-time folding
of floating point expressions. This is great.

Sometimes, however, the resulting (or indeed intermediate) expressions
may evaluate to a non-finite value (e.g. NaN, Inf) -- at compile-time.

During a traditional run-time execution of floating point arithmetic
there are various ways to detect the case of values going into
something like an overflow (individual guarding of each
possibly-violating expression via 'isfinite()'; or via reading the FPU
status register, which hardware itself may flag appropriately; or via
the 'fetestexcept' et al wrappers).

I was wondering if there was a way to ask GCC to optionally emit some
kind of a diagnostic (e.g. a warning of some kind) when it does
floating point calculations at compile-time and when such compile-time
calculations yield in a non-finite number.

For example in an oversimplified case like this:

float const x(::std::numeric_limits<float>::max());
float const y(x * 2);

It would be nice for GCC to emit a warning if it ended up setting 'y'
to 'Inf'... or similar.

Testing for such compile-time results during runtime would always be
less efficient (and at times, when not using individual
expression-guarding but rather FPU register reading, not possible...)
Forcing to always individually-guard may also be extremely defficient
(e.g. if 99% of cases are not overflowing then, statistically, it may
not be the best way of doing this). Expecting for the compile-time
'Inf', or similar, to be propagated eventually to some
runtime-detectable result is also not a solution as comparative (e.g.
x < y) math will not work either and not be detected (as 'trigger' for
y to overflow had happened at compile-time).

Best regards
Leon.

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[Vincent Lefevre]

On 2011-11-02 15:31:42 +1100, leon zadorin wrote:
> Under certain conditions GCC is able to perform compile-time folding
> of floating point expressions. This is great.
> 
> Sometimes, however, the resulting (or indeed intermediate) expressions
> may evaluate to a non-finite value (e.g. NaN, Inf) -- at compile-time.
> 
> During a traditional run-time execution of floating point arithmetic
> there are various ways to detect the case of values going into
> something like an overflow (individual guarding of each
> possibly-violating expression via 'isfinite()'; or via reading the FPU
> status register, which hardware itself may flag appropriately; or via
> the 'fetestexcept' et al wrappers).
> 
> I was wondering if there was a way to ask GCC to optionally emit some
> kind of a diagnostic (e.g. a warning of some kind) when it does
> floating point calculations at compile-time and when such compile-time
> calculations yield in a non-finite number.

In ISO C99, I don't think the compiler is allowed to replace a
floating expression with an underflow or overflow by its result
at compile time. 6.6#4 of ISO C99 says: "Each constant expression
shall evaluate to a constant that is in the range of representable
values for its type." and an underflow or overflow means that the
result is not in the range of representable values, so that the
rules for constant expressions must not be applied.

Now, I don't know what C++ allows (in your case) and I haven't
tested how GCC behaves (there may be a bug...).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[Andrew Haley]

On 11/02/2011 09:01 AM, Vincent Lefevre wrote:
> In ISO C99, I don't think the compiler is allowed to replace a
> floating expression with an underflow or overflow by its result
> at compile time.

But surely the "as if" rule applies here: if no conforming program
can tell the difference, a transformation is allowed.

Andrew.

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[leon zadorin]

On 11/2/11, Andrew Haley <aph@redhat.com> wrote:
> On 11/02/2011 09:01 AM, Vincent Lefevre wrote:
>> In ISO C99, I don't think the compiler is allowed to replace a
>> floating expression with an underflow or overflow by its result
>> at compile time.
>
> But surely the "as if" rule applies here: if no conforming program
> can tell the difference, a transformation is allowed.
>
> Andrew.
>

I dont think that this is the point i am making... Gcc allows certain
optimization options eg -Ofast (4.6) where -ffast-math (even prior to
4.6), and some other, unsafe wrt pedantic standard definitions,
optimization options become available (including, i think
floating-point constant folding opportunities).

I am very happy with such options by the way so the question was not
about standard compliance (hence my original statement about the said
compile-time calculations being done under certain conditions, not all
the time and having nothing to do with c++03/11 standard
specifications).

I am saying that having such, great, otimization options available it
would be super-helpful if there was a way to have a diagnostic/warning
about compile-time calculations producing non-finite floating point
values (whether via lib mpfr or otherwise).

Regards Leon Zadorin.

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[Vincent Lefevre]

On 2011-11-02 09:37:23 +0000, Andrew Haley wrote:
> On 11/02/2011 09:01 AM, Vincent Lefevre wrote:
> > In ISO C99, I don't think the compiler is allowed to replace a
> > floating expression with an underflow or overflow by its result
> > at compile time.
> 
> But surely the "as if" rule applies here: if no conforming program
> can tell the difference, a transformation is allowed.

Why couldn't conforming programs tell the difference???

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[Vincent Lefevre]

On 2011-11-03 00:36:20 +1100, leon zadorin wrote:
> I dont think that this is the point i am making... Gcc allows certain
> optimization options eg -Ofast (4.6) where -ffast-math (even prior to
> 4.6), and some other, unsafe wrt pedantic standard definitions,
> optimization options become available (including, i think
> floating-point constant folding opportunities).
> 
> I am very happy with such options by the way so the question was not
> about standard compliance (hence my original statement about the said
> compile-time calculations being done under certain conditions, not all
> the time and having nothing to do with c++03/11 standard
> specifications).
> 
> I am saying that having such, great, otimization options available it
> would be super-helpful if there was a way to have a diagnostic/warning
> about compile-time calculations producing non-finite floating point
> values (whether via lib mpfr or otherwise).

I'd say that this wouldn't be much useful, because similar problems
can also occur at run time, and if you use options like -ffast-math,
you won't be able to tell whether your program works correctly...
unless you can prove that no overflows (etc.) can occur. But if you
can do that, you no longer need GCC to do this for you for the
particular case of compile-time calculations.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[leon zadorin]

On 11/3/11, Vincent Lefevre <vincent+gcc@vinc17.org> wrote:
> On 2011-11-03 00:36:20 +1100, leon zadorin wrote:
>> I dont think that this is the point i am making... Gcc allows certain
>> optimization options eg -Ofast (4.6) where -ffast-math (even prior to
>> 4.6), and some other, unsafe wrt pedantic standard definitions,
>> optimization options become available (including, i think
>> floating-point constant folding opportunities).
>>
>> I am very happy with such options by the way so the question was not
>> about standard compliance (hence my original statement about the said
>> compile-time calculations being done under certain conditions, not all
>> the time and having nothing to do with c++03/11 standard
>> specifications).
>>
>> I am saying that having such, great, otimization options available it
>> would be super-helpful if there was a way to have a diagnostic/warning
>> about compile-time calculations producing non-finite floating point
>> values (whether via lib mpfr or otherwise).
>
> I'd say that this wouldn't be much useful, because similar problems
> can also occur at run time, and if you use options like -ffast-math,
> you won't be able to tell whether your program works correctly...
> unless you can prove that no overflows (etc.) can occur. But if you
> can do that, you no longer need GCC to do this for you for the
> particular case of compile-time calculations.

But as I had mentioned in my original post -- there is a way to detect
this at runtime, even if compiler optimizes based on presumptions of
finite math etc implied by -ffast-math.

If, as a statistical outlier, the two variables (whose values are only
known at run time) are say multiplied together and an overflow occurs
- the fpu hardware (for example) sets the fpu status registers to
indicate the overflow... which is what I was referring to in my
original post: "via reading the FPU status register, which hardware
itself may flag appropriately; or via the 'fetestexcept' et al
wrappers". One can even do the run time verification via an external
link to 'isfinite' function (compiled from a differently-optimized
translation unit without the -ffinite-math et al) - ie individually
guarding various expressions. The only thing being that 'fetestexcept'
is much more efficient in some cases (as once again I think i had
mentioned in my original post).

in other words - just as there is a run time after-the-fact way to
observe/detect unexpected non-finite values which is very useful in
some petformance-oriented deployment scenarios-- it would also be good
to do so for the compile-time stage also (ESP. when it is the compile
state which produces such values).

regards leon zadorin.

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[Vincent Lefevre]

On 2011-11-04 01:28:00 +1100, leon zadorin wrote:
> On 11/3/11, Vincent Lefevre <vincent+gcc@vinc17.org> wrote:
> > I'd say that this wouldn't be much useful, because similar problems
> > can also occur at run time, and if you use options like -ffast-math,
> > you won't be able to tell whether your program works correctly...
> > unless you can prove that no overflows (etc.) can occur. But if you
> > can do that, you no longer need GCC to do this for you for the
> > particular case of compile-time calculations.
> 
> But as I had mentioned in my original post -- there is a way to detect
> this at runtime, even if compiler optimizes based on presumptions of
> finite math etc implied by -ffast-math.

Hmm... Yes, this should be possible. But I'm not sure this is easy.
The reason is that the rounding mode used at compile time may be
different than the rounding mode used at run time, so that there
may be an overflow when an expression is evaluated at compile time
(in which case the compile-time evaluation is discarded, AFAIK[*])
while there will be no overflows at run time. In such a case, a
warning could be regarded as a bug.

[*] From Kaveh R. GHAZI in the gcc mailing-list on 2008-01-03:

"So I tried that, but mpfr_gamma on that value sets the global underflow
flag.  If overflow/underflow are set by mpfr, GCC will intentionally
bypass folding.  So this particular case is not something I can use."

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

Re: Compile-time floating-point expressions and subsequent detection of possible overflows etc -- during the compile-time stage.

[leon zadorin]

On 11/4/11, Vincent Lefevre <vincent+gcc@vinc17.org> wrote:
> On 2011-11-04 01:28:00 +1100, leon zadorin wrote:
>> On 11/3/11, Vincent Lefevre <vincent+gcc@vinc17.org> wrote:
>> > I'd say that this wouldn't be much useful, because similar problems
>> > can also occur at run time, and if you use options like -ffast-math,
>> > you won't be able to tell whether your program works correctly...
>> > unless you can prove that no overflows (etc.) can occur. But if you
>> > can do that, you no longer need GCC to do this for you for the
>> > particular case of compile-time calculations.
>>
>> But as I had mentioned in my original post -- there is a way to detect
>> this at runtime, even if compiler optimizes based on presumptions of
>> finite math etc implied by -ffast-math.
>
> Hmm... Yes, this should be possible. But I'm not sure this is easy.
> The reason is that the rounding mode used at compile time may be
> different than the rounding mode used at run time, so that there
> may be an overflow when an expression is evaluated at compile time
> (in which case the compile-time evaluation is discarded, AFAIK[*])
> while there will be no overflows at run time. In such a case, a
> warning could be regarded as a bug.
>
> [*] From Kaveh R. GHAZI in the gcc mailing-list on 2008-01-03:
>
> "So I tried that, but mpfr_gamma on that value sets the global underflow
> flag.  If overflow/underflow are set by mpfr, GCC will intentionally
> bypass folding.  So this particular case is not something I can use."

Aha, now this is very interesting :-) :-)
With very aggressive optimization options (-ffast-math et al) gcc
4.6.1 (from my memory - I'm typing away from coding box), the
following code:

float x(::std::numeric_limits<float>::max());
float y(x * 2);

appeared to produce an Inf at compile time (eg the runtime detection
of fpu status was not sensing overflow, but indeed the value of y when
printed or examined otherwise was Inf)... When, however, an extern
function, taking address of x in its signature yet doing nothing (in
implementation in a different translation unit compiled without -FL to
but with -fno-lto etc), was called between the above 2 lines, the
run-time fpu status was correctly showing an overflow after y
evaluation...

So... I am happy to have it either way:

a) compiler does do it at compile time (differences wrt run-time
calculations are not an issue as those will always be there in pretty
much all of the "unsafe optimization options" for the floating point
math - like associative grouping, rounding modes, hardware fpu
register architecture which may not be ieee standard anyway and have a
different accumulating precision loss when temps are being stored in
registers, etc...);

... but it should emit a diagnostic (if one is uncomfortable calling
it a warning, the just call it a verbosity output, just like the
-ftree-vectorizer-verbose meaning).

b) compiler does not do it at compile-time if its calculated
evaluation yields non-finite value (although things like Nan are
obtainable vis div by zero legitimately.. I think... but I digress);

... but may be it does not do this currently if things like
-ffinite-math are being set?

Personally I would suggest, ever so slightly, the former option -
verbosity output. The compiler, from the info you had quoted, does
already sense the non-finite states of its evaluations, and if it does
decide to go ahead with writing the value at compile-time (eg may
because the optimization options infer this), then might as well print
a message about it... and if the semantics of a warning are too
drastic, then just call it a verbosity-related output :-)

Of course, I must say - I am way out of my depth here wrt compiler
internals... I know nothing... I shall retest the aforementioned code
as soon as I get a chance...

best regards leon zadorin.

> --
> Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
>