linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Hillel (Sabba) Markowitz]

The following test program give incorrect results in a cast from double to int 
when compiled without an optimization option but correct results when compiled 
with an optimization option.

int main(int argc, char **argv)
{
  int i = 128000;
  double f = 0.0075;
  double g;
  unsigned int u;

  u = (unsigned int) (i*f);
  printf("unsigned int result of i*f: %u\n", u);

  g = i*f;
  u = (unsigned int)g;
  printf("double result: %lf\n", g);
  printf("unsigned int result: %u\n", u);

  return(0);
}


%gcc testfloat.c -o testfloat
%./testfloat

unsigned int result of i*f: 959
double result: 960
unsigned int result: 960

%gcc testfloat.c -o testfloat -O
%./testfloat

unsigned int result of i*f: 960
double result: 960
unsigned int result: 960


This error occurs only on the linux i-86 (redhat 7.2 or 7.3).  However, the 
results come out correctly on Solaris, DEC (4.0 and 5.1) and linux alpha 
(redhat 7.2).

Does anyone have an idea for a fix or workaround?  Is this an artifact of the 
Assembler or the floating point co-processor?  Is this something to do with 
gcc?

Please e-mail me with the answer as well as the mailing list.

Thanks,

-- 
Said the fox to the fish, "Join me ashore".
 The fish are the Jews, Torah is our water

Hillel (Sabba) Markowitz - sabbahem@bcpl.net

Re: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Martin Dickopp]

On Wed, Sep 18, 2002 at 10:06:51AM -0400, Hillel (Sabba) Markowitz wrote:
> The following test program give incorrect results in a cast from
> double to int when compiled without an optimization option but
> correct results when compiled with an optimization option.
> 
> int main(int argc, char **argv)
> {
>   int i = 128000;
>   double f = 0.0075;
>   double g;
>   unsigned int u;
> 
>   u = (unsigned int) (i*f);
>   printf("unsigned int result of i*f: %u\n", u);
> 
>   g = i*f;
>   u = (unsigned int)g;
>   printf("double result: %lf\n", g);
>   printf("unsigned int result: %u\n", u);
> 
>   return(0);
> }
> 
> 
> %gcc testfloat.c -o testfloat
> %./testfloat
> 
> unsigned int result of i*f: 959
> double result: 960
> unsigned int result: 960

You incorrectly assume that calculations involving floating-point
numbers are infinitely precise; in reality, rounding is inevitable.
The details of the rounding may depend, amongst other things, on the
processor type and optimization level.

Solution:

  u = (unsigned int) (i*f + 0.5);

This rounds i*f to the nearest unsigned integer, instead of always
rounding down. Therefore, you obtain your expected result even if
i*f is slightly less than 960.0.

Martin

Re: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Gokhan Kisacikoglu]

use the floor (or ceil) function with the double to int conversions
first, it will at least round down (or up) to the nearest integer for
sure...

HTH,
Gokhan


Martin Dickopp wrote:
> 
> On Wed, Sep 18, 2002 at 10:06:51AM -0400, Hillel (Sabba) Markowitz wrote:
> > The following test program give incorrect results in a cast from
> > double to int when compiled without an optimization option but
> > correct results when compiled with an optimization option.
> >
> > int main(int argc, char **argv)
> > {
> >   int i = 128000;
> >   double f = 0.0075;
> >   double g;
> >   unsigned int u;
> >
> >   u = (unsigned int) (i*f);
> >   printf("unsigned int result of i*f: %u\n", u);
> >
> >   g = i*f;
> >   u = (unsigned int)g;
> >   printf("double result: %lf\n", g);
> >   printf("unsigned int result: %u\n", u);
> >
> >   return(0);
> > }
> >
> >
> > %gcc testfloat.c -o testfloat
> > %./testfloat
> >
> > unsigned int result of i*f: 959
> > double result: 960
> > unsigned int result: 960
> 
> You incorrectly assume that calculations involving floating-point
> numbers are infinitely precise; in reality, rounding is inevitable.
> The details of the rounding may depend, amongst other things, on the
> processor type and optimization level.
> 
> Solution:
> 
>   u = (unsigned int) (i*f + 0.5);
> 
> This rounds i*f to the nearest unsigned integer, instead of always
> rounding down. Therefore, you obtain your expected result even if
> i*f is slightly less than 960.0.
> 
> Martin

-- 
Gökhan Kisacikoglu    			     CFX
Senior Technical Director		     10950 W Washington Blvd
kisa@centropolisfx.com   310.204.7300 x263   Culver City, CA, 90232
Key: Gö(GIrl)-khan Kis(cirCUS)-a-(Art)-cik(loGIC)-og(thOUGH)-lu(fLU)

Re: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Martin Dickopp]

On Wed, Sep 18, 2002 at 02:58:54PM -0500, Buddy Lott wrote:
> While I agree, that processor type would affect the precision of the
> answer, I would think the compiler should consistent when
> optimizing/not optimizing code. I would think it should never change
> the result of the code.

The observed behavior is a result of a peculiarity of the x86
processor: floating-point numbers have higher precision in registers
than in memory. Therefore, the result depends on what calculations
are done in registers, which is affected by optimization.

Gcc has an option -ffloat-store, which causes floating-point numbers
always to be stored in memory. With this option, the program gives
the same result with every optimization level.

Martin

RE: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Buddy Lott]

While I agree, that processor type would affect the precision of the
answer, I would think the compiler should consistent when optimizing/not
optimizing code. I would think it should never change the result of the
code.

In either case, I always stand by the rule that specifically defining
what you want is always better the relying on a default that might
change.

On the other hand....

To be a little more picky about this question, I would ask in what
domain this problem is in. That would define what the "proper" handling
of the rounding is.

I seem to remember that in the science arena, there are specific rules
as to when and how you round.

Bear in mind also, that rounding too early can cuases the margin of
error to increase as the value is used.

Take for instance (this is contrived):

float i = .5;
int x = 3;
int z= 5;
int n;

n = (int) ((x * i) + .5) // result is 2 if I round here
n = n*5; // result is 10

n = (int)((x*i*z) + .5) // result is 7.5



> -----Original Message-----
> From: gcc-help-owner@gcc.gnu.org [mailto:gcc-help-owner@gcc.gnu.org]
On
> Behalf Of Martin Dickopp
> Sent: Wednesday, September 18, 2002 1:56 PM
> To: Hillel (Sabba) Markowitz
> Cc: GCC Help
> Subject: Re: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)
> 
> On Wed, Sep 18, 2002 at 02:39:32PM -0400, Hillel (Sabba) Markowitz
wrote:
> > >===== Original Message From Martin Dickopp <firefly-mail@gmx.net>
=====
> > >  u = (unsigned int) (i*f + 0.5);
> >
> > Actually, I think it would have to be u = (unsigned
> int)(floor(i*f+0.5));
> > just to make sure that the default round is not to the nearest
integer.
> 
> No, that is not necessary. To quote the ISO C Standard (Section
6.3.1.4):
> 
> | (1) When a finite value of real floating type is converted to an
> | integer type other than _Bool, the fractional part is discarded
> | (i.e., the value is truncated toward zero). If the value of the
> | integral part cannot be represented by the integer type, the
> | behavior is undefined.
> 
> > The question was not that I assume that the rounding was infinitely
> precise,
> > but that the rounding from the same floating point value was
> inconsistent only
> > on one platform.  Additionally, the rounding from u = (unsigned int)
> (i*f);
> > was not the same as the rounding from g = (i*f); u = (unsigned
int)g;
> >
> > Additionally, the -O option in the compilation caused the rounding
error
> to
> > disappear, something that does not seem reasonable.
> 
> Yes, the imprecisions that appear in floating-point calculations can
> depend, amongst other things, on the processor type and optimization
> level. Even if you do the same calculation twice within the same
> program, there is no guarantee that the results will be exactly
> identical.
> 
> Martin

Re: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Martin Dickopp]

On Wed, Sep 18, 2002 at 02:39:32PM -0400, Hillel (Sabba) Markowitz wrote:
> >===== Original Message From Martin Dickopp <firefly-mail@gmx.net> =====
> >  u = (unsigned int) (i*f + 0.5);
> 
> Actually, I think it would have to be u = (unsigned int)(floor(i*f+0.5));
> just to make sure that the default round is not to the nearest integer.

No, that is not necessary. To quote the ISO C Standard (Section 6.3.1.4):

| (1) When a finite value of real floating type is converted to an
| integer type other than _Bool, the fractional part is discarded
| (i.e., the value is truncated toward zero). If the value of the
| integral part cannot be represented by the integer type, the
| behavior is undefined.

> The question was not that I assume that the rounding was infinitely precise, 
> but that the rounding from the same floating point value was inconsistent only 
> on one platform.  Additionally, the rounding from u = (unsigned int) (i*f); 
> was not the same as the rounding from g = (i*f); u = (unsigned int)g;
> 
> Additionally, the -O option in the compilation caused the rounding error to 
> disappear, something that does not seem reasonable.

Yes, the imprecisions that appear in floating-point calculations can
depend, amongst other things, on the processor type and optimization
level. Even if you do the same calculation twice within the same
program, there is no guarantee that the results will be exactly
identical.

Martin

RE: linux i-86 problem with rounding (gcc-3.1.1 & 3.2)

[Hillel (Sabba) Markowitz]

>===== Original Message From Martin Dickopp <firefly-mail@gmx.net> =====

>> unsigned int result of i*f: 959
>> double result: 960
>> unsigned int result: 960
>
>You incorrectly assume that calculations involving floating-point
>numbers are infinitely precise; in reality, rounding is inevitable.
>The details of the rounding may depend, amongst other things, on the
>processor type and optimization level.
>
>Solution:
>
>  u = (unsigned int) (i*f + 0.5);

Actually, I think it would have to be u = (unsigned int)(floor(i*f+0.5));
just to make sure that the default round is not to the nearest integer.

>
>This rounds i*f to the nearest unsigned integer, instead of always
>rounding down. Therefore, you obtain your expected result even if
>i*f is slightly less than 960.0.

The question was not that I assume that the rounding was infinitely precise, 
but that the rounding from the same floating point value was inconsistent only 
on one platform.  Additionally, the rounding from u = (unsigned int) (i*f); 
was not the same as the rounding from g = (i*f); u = (unsigned int)g;

Additionally, the -O option in the compilation caused the rounding error to 
disappear, something that does not seem reasonable.

Had this been a problem of always rounding down, then the inconsistency would 
not have appeared.

In the actual program being maintained, there appear to be too many locations 
in the code to find every placy where the rounding function could be applied.

-- 
Said the fox to the fish, "Join me ashore".
 The fish are the Jews, Torah is our water

Hillel (Sabba) Markowitz - sabbahem@bcpl.net