Int cast / floating point isuses

Int cast / floating point isuses

[Joe Hughes]

I've been doing some work with floating point numbers and stumbled across
some peculiar results and I'm not sure where to go from here.

The code below generates this output on RedHat 2.1 ix86 using g++ 2.96,
3.2 & 3.41
original -79.937384
int cast: -79937383
lrint: -79937384
trunc: -79937384

Using g++ 3.2 on MacOSX, and g++ 2.96 on Solaris 2.7 and g++ 3.2 on 
Solaris 2.8 as well as .net studio on windows xp all return -79937384 
for int cast.

I'm stumped as to what is causing this error to occur for int cast,
especially when lrint and trunc seem to work properly.

Any insight anyone has would be much appreciated.

#include <stdio.h>
#include <math.h>

int main(int argc, char *argv[])
{
     int iTmp = 0;
     double dTmp =-79.937384;

     printf("original %.6f\n",dTmp);

     iTmp = (int)(dTmp * 1000000.0);
     printf("int cast: %d\n",iTmp);

     iTmp = llrint(dTmp * 1000000.0);
     printf("lrint: %d\n",iTmp);

     iTmp = trunc(dTmp * 1000000.0);
     printf("trunc: %d\n\n",iTmp);
}//end int main(int argc, char *argv[])

Joe

Re: Int cast / floating point isuses

[Brian Gough]

"Joe Hughes" <jhughesjr@comcast.net> writes:

> I'm stumped as to what is causing this error to occur for int cast,
> especially when lrint and trunc seem to work properly.
> 
> Any insight anyone has would be much appreciated.

The default with gcc on x86 is to use 80-bit extended precision
registers, compared with 64-bit on the other architecture.  It gives a
differently rounded result in this case, since the underlying binary
representation is on the borderline between -79.937384... and
-79.9373839999...

Use the rint type functions to round to the nearest integer reliably.

-- 
Brian Gough

Network Theory Ltd,
Publishing "An Introduction to GCC" --- http://www.network-theory.co.uk/

Re: Int cast / floating point isuses

[joe]

Thanks for the info Brian.

If I was to round when compiling with gcc on x86 instead of using an int 
cast in situations where my application is running on different 
platforms and or compiled with different compilers they would still be 
truncating numbers such as 2.7777777777777 to create 2.777777 while the 
gcc numbers will become 2.777778. I will no longer have an error with 
the numbers that were exact to begin with but I will now have rounded 
numbers instead of truncated numbers.

Which brings me to ask the question, if trunc is truncating the number 
accurately then why doesn't int cast and is there a way to replicate 
this 64 bit behavior the other architectures use when using int cast on x86?

I guess in my situation, even if the 80 bit extended precision registers 
are supposed to be more accurate/precise overall, the accuracy/precision 
is not as important to me than having a number print out the same 
regardless of the platform it is running on, which really is my main goal.

Cheers,
Joe

Brian Gough wrote on 7/24/2004, 7:55 AM:

 > Use the rint type functions to round to the nearest integer reliably.

Re: Int cast / floating point isuses

[Brian Gough]

joe <jhughesjr@comcast.net> writes:

> I guess in my situation, even if the 80 bit extended precision registers 
> are supposed to be more accurate/precise overall, the accuracy/precision 
> is not as important to me than having a number print out the same 
> regardless of the platform it is running on, which really is my main goal.

Yes, getting the standard IEEE behavior for comparing results across
platforms is really useful.  On x86 systems it is possible to put the
processor into the standard IEEE double precision mode by setting the
floating point control word.

For GNU/Linux the header file /usr/include/fpu_control.h has macros
_FPU_GETCW and _FPU_SETCW for changing the settings.  These macros are
internal (I haven't seen a documented interface to them) but you can
see how they work there anyway -- it is one line of inline assembly.

If you look in the IEEE chapter of the GNU Scientific Library
Reference Manual (http://www.gnu.org/software/gsl/) there is some
discussion about this and a function which allows you to set all the
parameters of the fpu mode from an enviroment variable.

-- 
Brian Gough

Network Theory Ltd,
Publishing "An Introduction to GCC" --- http://www.network-theory.co.uk/

Re: Int cast / floating point isuses

[Joe Hughes]

Thanks Brian this does indeed seem to be the root cause of my int cast 
issues.

The following sample code returns the proper results now.


#include <stdio.h>
#include <fpu_control.h>

double latLng[] = 
{40.496538,-79.937384,40.495530,-79.938170,40.495740,-79.940830,40.490650,-79.951690,40.480650,-79.965050,40.457950,-79.986940,40.452730,-79.996260,40.445500,-79.990520,40.429040,-79.999250,40.413790,-80.010820,40.411350,-80.005050,40.412160,-79.998210,40.414170,-79.996780,40.413860,-79.987550,40.496538,-79.937384,40.496280,-79.936740,40.496280,-79.936740,40.496050,-79.936890,40.495730,-79.937120,40.495280,-79.937510,40.495280,-79.937510,40.495530,-79.938170,40.495680,-79.938520,40.495930,-79.939060,40.495990,-79.939390,40.496000,-79.939650,40.495990,-79.939880,40.495900,-79.940320,40.495900,-79.940320,40.495890,-79.940360,40.495890,-79.940400,40.495860,-79.940660,40.495740,-79.940830,40.494570,-79.942290,40.494310,-79.942870,40.492740,-79.948960,40.492390,-79.949690,40.491790,-79.950530,40.491360,-79.950970,40.490650,-79.951690,40.489460,-79.953230,40.486970,-79.957710,40.485620,-79.959780,40.484710,-79.960840,40.482900,-79.962530,40.480650,-79.965050,40.480000,-79.965720,40.479610,-79.966100,40.479080,-79.966680,40.478000,-79.968120,40.477130,-79.968890,40.475780,-79.970030,40.474530,-79.971190,40.474120,-79.971500,40.496500,-79.937399,40.496538,-79.937384,40.414560,-80.009280,40.414210,-80.009600,40.414030,-80.009700,40.413800,-80.009820,40.413680,-80.009860,40.413680,-80.009860,40.413570,-80.010040,40.413500,-80.010230,40.413540,-80.010500,40.413790,-80.010820,40.413970,-80.010820,40.414120,-80.010700,40.414210,-80.010360,40.414090,-80.009830,40.414030,-80.009700,40.413670,-80.009160,40.412720,-80.007700,40.412520,-80.007460,40.412090,-80.007040,40.411110,-80.006100,40.411110,-80.006100,40.411350,-80.005050,40.412050,-80.001740,40.411760,-79.999480,40.411820,-79.998930,40.412160,-79.998210,40.413210,-79.997020,40.413530,-79.996810,40.413730,-79.996770,40.414170,-79.996780,40.414310,-79.992720,40.414320,-79.991850,40.414350,-79.990300,40.414410,-79.988860,40.414430,-79.988330,40.414430,-79.988240,40.414430,-79.987720,40.414430,-79.987720,40.414290,-79.987580,40.413860,-79.987550,40.413900,-79.987599,40.413860,-79.987550};

int main(int argc, char *argv[])
{
    int iLat,iLng;

    //change FPU to double precision so results
    //are consistent across platforms
    fpu_control_t cw;
    _FPU_GETCW(cw);
    cw &= ~_FPU_EXTENDED;
    cw |= _FPU_DOUBLE;
    _FPU_SETCW(cw);


    for(int iCount=0;iCount<99;iCount+=2)
    {
       printf("original %.6f,%.6f\n",latLng[iCount],latLng[iCount+1]);

       iLat = (int)(latLng[iCount]  * 1000000.0);
       iLng = (int)(latLng[iCount+1] * 1000000.0);

       printf("int cast: %d,%d\n",iLat,iLng);
    }//end for(int iCount=0;iCount<99;iCount++)
}//end int main(int argc, char *argv[])


Cheers,
Joe

Brian Gough wrote on 7/25/2004, 5:56 AM:

 > Yes, getting the standard IEEE behavior for comparing results across
 > platforms is really useful.  On x86 systems it is possible to put the
 > processor into the standard IEEE double precision mode by setting the
 > floating point control word.
 >
 > For GNU/Linux the header file /usr/include/fpu_control.h has macros
 > _FPU_GETCW and _FPU_SETCW for changing the settings.  These macros are
 > internal (I haven't seen a documented interface to them) but you can
 > see how they work there anyway -- it is one line of inline assembly.
 >
 > If you look in the IEEE chapter of the GNU Scientific Library
 > Reference Manual (http://www.gnu.org/software/gsl/) there is some
 > discussion about this and a function which allows you to set all the
 > parameters of the fpu mode from an enviroment variable.