code optimizations and numerical research

code optimizations and numerical research

[Peter Jay Salzman]

Hi all,

I'm a physicist doing research on quantum gravity.  Although I'm good at
writing programs to solve all kinds of non-linear PDEs, ODEs, and integral
equations, I'm not a computer scientist and not savvy with a lot of lingo
and warnings in the gcc man/info pages.  I'd like to ask for help with code
speed optimization options in gcc.

The most effective optimizations for the code I'm currently writing is:

   -O3 -funroll-loops -march=athlon-xp -ffast-math

That last option, -ffast-math, is what I'd like to ask about.  According to
the man page:

   can result in incorrect output for programs which depend on an exact
   implementation of IEEE or ISO rules/specifications for math functions.

and according to the GCC Complete Reference:

   Certain calculations are made faster by violating some of the ISO and
   IEEE rules.  For example, with this option set it is assumed that no
   negative values are passed to sqrt() and that all floating point values
   are valid.


First and foremost, my code must generate correct output --- this is science
that's helping to lay groundwork for a future of theory of quantum gravity,
not a Lucas Arts SCUMM emulator.  :)

So what happens when the assumptions mentioned above are NOT true?  For
example, in this code:

   int main(void)
   {
      double d = sqrt(-1);
      printf("%f\n", d);

      return 0;
   }

the behavior appears to be the same whether it's compiled with -ffast-math
or not: it simply prints "nan".

I've Googled and Googled, but everything I've found on GCC code
optimizations (like "-fno-signaling-nans") appears to simply quote the man
and info pages.  I'm not finding a "dummy's" guide to picking code
optimization.  Even the GCC Complete Reference book is very skimpy on the
details of code optimization.

The man page claims that "-ffast-math" may produce wrong results for
programs that depend on "an exact implementation of IEEE or ISO
rules/specifications for math functions."

What exactly does this vague sentence mean?

In my code, I do use errno, but it's for my own personal "die" function,
like when the program attempts to open a non-existent parameter file.  I can
easily write errno out of my code.

I also enable and catch certain floating point exceptions which appear to be
disabled by default(!) like

       * FE_DIVBYZERO   division by zero
       * FE_UNDERFLOW   result not representable due to underflow
       * FE_OVERFLOW    result not representable due to overflow
       * FE_INVALID     invalid operation

Unlike errno, which is expendable, I would definitely like to be able to
catch these FPE's.  There's a lot of powers of 10^-34 and 10^-11 and 10^-31
in my code, and even with the equation is scaled, I really do need the
program to come to a grinding halt when anything becomes inf or nan or with
underflows/overflows.

Lastly, after some experimentation, I found that it's actually the
combination of "-fno-math-errno -funsafe-math-optimizations" (both enabled
by -ffast-math) that really makes my code fly.  I'm talking about a factor
of almost 400%!!!

But, here again, the documentation describes what
-funssafe-math-optimizations does (violate IEEE and ANSI standards, assumes
values are correct, optimizes the operation of the hardware FPU in
non-standard ways) but doesn't tell me what I really care about: under what
situations (that an educated layman like I will understand) will this option
generate incorrect output?

The number of optimization options is dizzying.  Any help would be greatly
appreciated!

Thanks!!!
Pete

-- 
Every theory is killed sooner or later, but if the theory has good in it,
that good is embodied and continued in the next theory. -- Albert Einstein

GPG Fingerprint: B9F1 6CF3 47C4 7CD8 D33E  70A9 A3B9 1945 67EA 951D

Re: code optimizations and numerical research

[Ian Lance Taylor]

p@dirac.org (Peter Jay Salzman) writes:

> The man page claims that "-ffast-math" may produce wrong results for
> programs that depend on "an exact implementation of IEEE or ISO
> rules/specifications for math functions."
> 
> What exactly does this vague sentence mean?

Read "What Every Computer Scientist Should Know about Floating-Point
Arithmetic:"
    http://www.validlab.com/goldberg/paper.pdf

Unfortunately it's rather hard to understand what these optimizations
do without becoming something of an expert.

As a general rule of thumb, if you do not know the details of IEEE
floating point format, and you are not calculating your precise error
intervals and writing code which relies on them to, e.g., converge--in
other words, if you are just writing code that happens to use floating
point--then you will probably do just fine with -ffast-math.

Ian

Re: code optimizations and numerical research

[Peter Jay Salzman]

On Mon 16 May 05, 11:04 AM, Ian Lance Taylor <ian@airs.com> said:
> p@dirac.org (Peter Jay Salzman) writes:
> 
> > The man page claims that "-ffast-math" may produce wrong results for
> > programs that depend on "an exact implementation of IEEE or ISO
> > rules/specifications for math functions."
> > 
> > What exactly does this vague sentence mean?
> 
> Read "What Every Computer Scientist Should Know about Floating-Point
> Arithmetic:" http://www.validlab.com/goldberg/paper.pdf
 
Not exactly light reading.  But it does look like something I really need to
read.  I've printed this behemoth out.

> Unfortunately it's rather hard to understand what these optimizations do
> without becoming something of an expert.
 
I was afraid of that.

> As a general rule of thumb, if you do not know the details of IEEE
> floating point format, and you are not calculating your precise error
> intervals and writing code which relies on them to, e.g., converge--in
> other words, if you are just writing code that happens to use floating
> point--then you will probably do just fine with -ffast-math.

Just to make sure --- so you're saying that even though I'm enabling SIGFPE
to catch exceptions:


   void fpe_trap_enable(void)
   {
      /* Enable FPE's, by default all FPE's will not raise a signal when 
       * they happen... see fenv.h for magic constants. 
       *
       * FE_INEXACT     inexact result - don't do this!  It always occurs!
       * FE_DIVBYZERO   division by zero
       * FE_UNDERFLOW   result not representable due to underflow
       * FE_OVERFLOW    result not representable due to overflow
       * FE_INVALID     invalid operation
       */

     feenableexcept( FE_DIVBYZERO | FE_UNDERFLOW | FE_OVERFLOW | FE_INVALID);
   }



	{  /* Setup a signal handler for SIGFPE */
		struct sigaction action;
  
		memset(&action, 0, sizeof(action));
		action.sa_sigaction = fpe_callback;    /* which callback function  */
		sigemptyset(&action.sa_mask);          /* other signals to block   */
		action.sa_flags = SA_SIGINFO;          /* give details to callback */
  
		if (sigaction(SIGFPE, &action, 0))
			die("Failed to register signal handler.");
	}


all the talk about -ffast-math making the compiler assume that args and
results of math operations are correct and finite has no bearing on either
my code (which certainly doesn't do what you described above) or my enabling
and catching FPE exceptions?

Thanks very much!
Pete

-- 
Every theory is killed sooner or later, but if the theory has good in it,
that good is embodied and continued in the next theory. -- Albert Einstein

GPG Fingerprint: B9F1 6CF3 47C4 7CD8 D33E  70A9 A3B9 1945 67EA 951D

Re: code optimizations and numerical research

[Ian Lance Taylor]

p@dirac.org (Peter Jay Salzman) writes:

> Just to make sure --- so you're saying that even though I'm enabling SIGFPE
> to catch exceptions:

...

> all the talk about -ffast-math making the compiler assume that args and
> results of math operations are correct and finite has no bearing on either
> my code (which certainly doesn't do what you described above) or my enabling
> and catching FPE exceptions?

Obviously it's hard to be completely sure whether your code is going
to be OK.  But -ffast-math won't make SIGFPE signals go away.  It just
means, more or less, that the compiler will assume they won't happen.
If you don't try to resume execution of the failing operation after
receiving a SIGFPE error--if you just give an error or something--then
you should be fine.

In fact, enabling the SIGFPE signal as you do will probably make you
safer when using -ffast-math.  For example, -ffast-math may not handle
floating point infinities correctly, but you've asked for a signal
when an infinity is generated anyhow, so your code will presumably
never see one.

Ian

Re: code optimizations and numerical research

[Brian Gough]

Peter Jay Salzman writes:
 > The man page claims that "-ffast-math" may produce wrong results for
 > programs that depend on "an exact implementation of IEEE or ISO
 > rules/specifications for math functions."
 > 
 > What exactly does this vague sentence mean?

The main problem is that Infs and NaNs are manipulated as finite
numbers in optimisations and comparisons with -ffast-math.  e.g.  you
can find that x-x==0 for x=Inf and other similar oddities.  Some
programs use non-standard idioms like if(x!=x) for checking if x is
NaN.

-- 
Brian Gough
(GNU Scientific Library Maintainer)

p.s. don't forget to profile your program.

Network Theory Ltd,
Publishing "An Introduction to GCC" --- http://www.network-theory.co.uk/gcc/