surprising precision

surprising precision

[Jonathan]

Please have a look at my query posted at the following 3 forums.
It has not received any explanations yet.
On the one hand it appears not to be a problem but on the other hand  
could be a spectacular bug.

http://forums.debian.net/viewtopic.php?f=8&t=49627

http://www.linux.com/community/forums?func=view&catid=17&id=4589

http://www.linuxformat.com/forums/viewtopic.php?t=11688


Thank you.
Jonathan

Re: surprising precision

[Andrew Haley]

On 03/03/10 20:29, Jonathan wrote:
> Please have a look at my query posted at the following 3 forums.
> It has not received any explanations yet.
> On the one hand it appears not to be a problem but on the other hand
> could be a spectacular bug.
> 
> http://forums.debian.net/viewtopic.php?f=8&t=49627
> 
> http://www.linux.com/community/forums?func=view&catid=17&id=4589
> 
> http://www.linuxformat.com/forums/viewtopic.php?t=11688

This does not surprise me at all.  These are just powers of 2.
I don't know why you find it surprising.  If you could explain
why you find it surprising, perhaps we could help.  Do you
understand floating-point arithmetic?

Andrew.

Re: surprising precision

[me22]

On 3 March 2010 15:29, Jonathan <jsealy@cwgsy.net> wrote:
> It has not received any explanations yet.
> On the one hand it appears not to be a problem but on the other hand could
> be a spectacular bug.
>

It's probably just you not understanding what floating point numbers are.

Go read http://en.wikipedia.org/wiki/IEEE_754-2008

Re: surprising precision

[John S. Fine]

Part of your confusion is over the difference between "precision" and 
"accuracy".

A floating point format has a FINITE number of specific values that can 
be stored with infinite accuracy.  You happen to have chosen values that 
are in that finite set.

You can set the precision of your output to whatever (finite value) you 
like.  It may be much more than the accuracy of your number.  It may be 
much less than the accuracy of your number (especially if the accuracy 
of your number is infinite).

Even if the accuracy of a stored floating point number is infinite, it 
may be possible for the algorithm that converts it from binary to 
decimal to introduce some inaccuracy.  I'm not sure of those details.  
Your results seem to indicate that translation is done surprisingly well.

To help you understand, instead of outputting just d each time, output 
both d and d+1.  At some point d will still be 100% accurate but d+1 
will not be accurate, in fact it will be exactly equal to d.

Jonathan wrote:
> Please have a look at my query posted at the following 3 forums.
> It has not received any explanations yet.
> On the one hand it appears not to be a problem but on the other hand 
> could be a spectacular bug.
>
> http://forums.debian.net/viewtopic.php?f=8&t=49627
>
> http://www.linux.com/community/forums?func=view&catid=17&id=4589
>
> http://www.linuxformat.com/forums/viewtopic.php?t=11688
>

Re: surprising precision

[Bob Plantz]

On 3/3/2010 12:51 PM, me22 wrote:
> On 3 March 2010 15:29, Jonathan<jsealy@cwgsy.net>  wrote:
>    
>> It has not received any explanations yet.
>> On the one hand it appears not to be a problem but on the other hand could
>> be a spectacular bug.
>>
>>      
> It's probably just you not understanding what floating point numbers are.
>
> Go read http://en.wikipedia.org/wiki/IEEE_754-2008
>    
Also, be careful about what you call "accurate." Try the following C program:
#include<stdio.h>

int main(void)
{
     int anInt = 19088743;
     float aFloat = 19088.743;

     printf("The integer is %i and the float is %f\n", anInt, aFloat);

     return 0;
}

It prints

      The integer is 19088743 and the float is 19088.742188

which doesn't even round off to 19088.743. The real question with floating point is: How many bits are required for the significand?

--Bob

Re: surprising precision

[Bob Plantz]

By the way, it may interest you to know that half the floating point values (in IEEE 754 format) are in the range -2.0 to +2.0, not inclusive.

--Bob


On 3/3/2010 12:29 PM, Jonathan wrote:
> Please have a look at my query posted at the following 3 forums.
> It has not received any explanations yet.
> On the one hand it appears not to be a problem but on the other hand 
> could be a spectacular bug.
>
> http://forums.debian.net/viewtopic.php?f=8&t=49627
>
> http://www.linux.com/community/forums?func=view&catid=17&id=4589
>
> http://www.linuxformat.com/forums/viewtopic.php?t=11688
>
>
> Thank you.
> Jonathan

Re: surprising precision

[Steve Teale]

On Wed, 2010-03-03 at 16:52 -0500, John S. Fine wrote: 
> Part of your confusion is over the difference between "precision" and 
> "accuracy".
> 
> A floating point format has a FINITE number of specific values that can 
> be stored with infinite accuracy.  You happen to have chosen values that 
> are in that finite set.
> 
> You can set the precision of your output to whatever (finite value) you 
> like.  It may be much more than the accuracy of your number.  It may be 
> much less than the accuracy of your number (especially if the accuracy 
> of your number is infinite).
> 
> Even if the accuracy of a stored floating point number is infinite, it 
> may be possible for the algorithm that converts it from binary to 
> decimal to introduce some inaccuracy.  I'm not sure of those details.  
> Your results seem to indicate that translation is done surprisingly well.
> 
> To help you understand, instead of outputting just d each time, output 
> both d and d+1.  At some point d will still be 100% accurate but d+1 
> will not be accurate, in fact it will be exactly equal to d.

John,

Congratulations on a helpful answer. Yours was the first one that did
not display the almost conventional GCC guru arrogance.

GCC would have a lot more participants/helpers if it were not for this
unwritten convention of rudeness in the community.

To be fair, this isn't just a GCC thing. I've worked in various shops
where the software paid the wages, but if a new employee asked for help
he/she would get a similar response.

I understand the impatience of the gurus. If they tried to answer every
naive question they'd never get any work done

But, if you don't have anything helpful to say, then maybe it's better
to bite your tongue! The person you are responding to may be an idiot,
but he/she is not the only recipient of your message.

Thanks
Steve

Re: surprising precision

[Andrew Haley]

On 03/04/2010 03:53 PM, Steve Teale wrote:
> On Wed, 2010-03-03 at 16:52 -0500, John S. Fine wrote: 
>> Part of your confusion is over the difference between "precision" and 
>> "accuracy".
>>
>> A floating point format has a FINITE number of specific values that can 
>> be stored with infinite accuracy.  You happen to have chosen values that 
>> are in that finite set.
>>
>> You can set the precision of your output to whatever (finite value) you 
>> like.  It may be much more than the accuracy of your number.  It may be 
>> much less than the accuracy of your number (especially if the accuracy 
>> of your number is infinite).
>>
>> Even if the accuracy of a stored floating point number is infinite, it 
>> may be possible for the algorithm that converts it from binary to 
>> decimal to introduce some inaccuracy.  I'm not sure of those details.  
>> Your results seem to indicate that translation is done surprisingly well.
>>
>> To help you understand, instead of outputting just d each time, output 
>> both d and d+1.  At some point d will still be 100% accurate but d+1 
>> will not be accurate, in fact it will be exactly equal to d.
> 
> John,
> 
> Congratulations on a helpful answer. Yours was the first one that did
> not display the almost conventional GCC guru arrogance.
> 
> GCC would have a lot more participants/helpers if it were not for this
> unwritten convention of rudeness in the community.
> 
> To be fair, this isn't just a GCC thing. I've worked in various shops
> where the software paid the wages, but if a new employee asked for help
> he/she would get a similar response.
> 
> I understand the impatience of the gurus. If they tried to answer every
> naive question they'd never get any work done
> 
> But, if you don't have anything helpful to say, then maybe it's better
> to bite your tongue! The person you are responding to may be an idiot,
> but he/she is not the only recipient of your message.

I strongly disagree.  The main problem may have been that the OP
didn't understand floating-point arithmetic, so the advice to find out
about that was appropriate.  Let's see what the OP says.

Andrew.

Re: surprising precision

[Bob Plantz]

Andrew Haley wrote:
> On 03/04/2010 03:53 PM, Steve Teale wrote:
>> On Wed, 2010-03-03 at 16:52 -0500, John S. Fine wrote: 
>>> Part of your confusion is over the difference between "precision" and 
>>> "accuracy".
>>>
>>> A floating point format has a FINITE number of specific values that can 
>>> be stored with infinite accuracy.  You happen to have chosen values that 
>>> are in that finite set.
>>>
>>> You can set the precision of your output to whatever (finite value) you 
>>> like.  It may be much more than the accuracy of your number.  It may be 
>>> much less than the accuracy of your number (especially if the accuracy 
>>> of your number is infinite).
>>>
>>> Even if the accuracy of a stored floating point number is infinite, it 
>>> may be possible for the algorithm that converts it from binary to 
>>> decimal to introduce some inaccuracy.  I'm not sure of those details.  
>>> Your results seem to indicate that translation is done surprisingly well.
>>>
>>> To help you understand, instead of outputting just d each time, output 
>>> both d and d+1.  At some point d will still be 100% accurate but d+1 
>>> will not be accurate, in fact it will be exactly equal to d.
>> John,
>>
>> Congratulations on a helpful answer. Yours was the first one that did
>> not display the almost conventional GCC guru arrogance.
>>
>> GCC would have a lot more participants/helpers if it were not for this
>> unwritten convention of rudeness in the community.
>>
>> To be fair, this isn't just a GCC thing. I've worked in various shops
>> where the software paid the wages, but if a new employee asked for help
>> he/she would get a similar response.
>>
>> I understand the impatience of the gurus. If they tried to answer every
>> naive question they'd never get any work done
>>
>> But, if you don't have anything helpful to say, then maybe it's better
>> to bite your tongue! The person you are responding to may be an idiot,
>> but he/she is not the only recipient of your message.
> 
> I strongly disagree.  The main problem may have been that the OP
> didn't understand floating-point arithmetic, so the advice to find out
> about that was appropriate.  Let's see what the OP says.
> 
> Andrew.

I agree with Andrew here, but I admit that at least my responses 
probably did not adequately answer the OP's question. My mistake is that 
I did not read his code carefully enough.

So I copied his code and tried it. I believe his real question is 
related to his not understanding that the function that displays a value 
is different from the storage of that value.

For example, let us say that I want you to write the number 555...5 
where there are N 5s in the number. Now if you and I agree that there 
are only 5s in this result, then I only have to tell you what N is. 
Using this protocol, I can specify a very, very large number with only 8 
bits. That is, I can specify a number up to 255 5s in only 8 bits.

Okay, floating point storage works a little like this in that a value is 
expressed as some fraction times 2 raised to a power, say P. In binary, 
2 raised to the Pth power is 1 followed by P zeroes.

OP, your program prints 2 raised to the 127th power. In binary this is 1 
followed by 127 zeroes. Now I can tell you then number of zeroes, up 
through 127, in only 7 bits. But when I convert 1 followed by 127 zeroes 
to decimal and write it out, that looks like it takes a LOT more than 7 
bits to express that number.

The key here is that using on these 7 bits, the next adjacent number is 
2 raised to the 126th power, a LONG distance from 2 raised to the 127th 
power.

So I think there are two things the OP should learn about:

1. How floating point numbers are stored.

2. That the conversion from a binary storage format to a decimal display 
on the screen is a separate function. Similarly, conversion from decimal 
input on the keyboard to an internal binary storage format is also a 
separate function.

In the spirit of full disclosure, I taught this material at the 
university level for over two decades and have written a textbook that 
covers both topics at the assembly language level. My textbook remains 
unpublished due to a perceived lack of market by commercial publishers, 
but it is still being used at the university where I taught. BTW, it 
uses the x86-64 architecture.

--Bob

Re: surprising precision

[Dean Anderson]

John's response doesn't seem to be rude or arrogant to me.

John makes essentially two helpful points:

1. that one should focus on the difference between "precision" and
"accuracy", with some explanation.

2. suggestion of an experiment that can help understand the difference.

I fail to see how this is rude. It is indeed helpful. Pretty much every
other responder gave essentially the same information (that the poster
must learn floating point), but others were even more terse and direct:
"<poster> doesn't understand floating point"

Anyway, I'll add my own reference to floating point: This months (Feb
2010) Circuit Cellar magazine has an article "Floating Point for DSP" on
implementing fast 18 bit floating point in hardware, and I found it very
illuminating for the mechanics of floating point, even if you never plan
to implement floating point in hardware or software.

Years ago, ~1988, I wrote a fixed point code to speed up my MIT computer
graphics class homework assignments.* I should try to find that again.
It also sheds some light on number representation problems.

[* ok, this was a bit more than the problem sets called for, but I took
the class with Terry Donahue, the author of Xtank; there may have been
some personal competition. Terry also wrote a fixed point code. I
believe we both got A's. ;-)]

		--Dean

On Thu, 4 Mar 2010, Steve Teale wrote:

> On Wed, 2010-03-03 at 16:52 -0500, John S. Fine wrote: 
> > Part of your confusion is over the difference between "precision" and 
> > "accuracy".
> > 
> > A floating point format has a FINITE number of specific values that can 
> > be stored with infinite accuracy.  You happen to have chosen values that 
> > are in that finite set.
> > 
> > You can set the precision of your output to whatever (finite value) you 
> > like.  It may be much more than the accuracy of your number.  It may be 
> > much less than the accuracy of your number (especially if the accuracy 
> > of your number is infinite).
> > 
> > Even if the accuracy of a stored floating point number is infinite, it 
> > may be possible for the algorithm that converts it from binary to 
> > decimal to introduce some inaccuracy.  I'm not sure of those details.  
> > Your results seem to indicate that translation is done surprisingly well.
> > 
> > To help you understand, instead of outputting just d each time, output 
> > both d and d+1.  At some point d will still be 100% accurate but d+1 
> > will not be accurate, in fact it will be exactly equal to d.
> 
> John,
> 
> Congratulations on a helpful answer. Yours was the first one that did
> not display the almost conventional GCC guru arrogance.
> 
> GCC would have a lot more participants/helpers if it were not for this
> unwritten convention of rudeness in the community.
> 
> To be fair, this isn't just a GCC thing. I've worked in various shops
> where the software paid the wages, but if a new employee asked for help
> he/she would get a similar response.
> 
> I understand the impatience of the gurus. If they tried to answer every
> naive question they'd never get any work done
> 
> But, if you don't have anything helpful to say, then maybe it's better
> to bite your tongue! The person you are responding to may be an idiot,
> but he/she is not the only recipient of your message.
> 
> Thanks
> Steve
> 
> 
> 
> 

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 256 5494

Re: surprising precision

[Toon Moene]

Bob Plantz wrote:

> By the way, it may interest you to know that half the floating point 
> values (in IEEE 754 format) are in the range -2.0 to +2.0, not inclusive.

By far the best question in the programming course of the University I 
attended (years before IEEE 754):

On <this machine> integers are represented as sign magnitude 32 bit numbers.

Floating point numbers on that machine are specified according to the 
following 32 bit format <follows description of format>.

Question:

Are there

A. More floating point numbers than integer numbers.
B. More integer numbers than floating point numbers.
C. An equal number of both.

[ The only better question in a course I ever encountered was during an
   introductory in Climate Research at my Institute (KNMI):

   1. Define the average temperature on Earth.
   2. Describe how you would set up a measurement campaign to show
      that the average is changing.

]

A hint not given in that course: "The operative word is 'measurement'".

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
At home: http://moene.org/~toon/
Progress of GNU Fortran: http://gcc.gnu.org/gcc-4.5/changes.html

Re: surprising precision

[Ian Lance Taylor]

Steve Teale <steve.teale@britseyeview.com> writes:

> Congratulations on a helpful answer. Yours was the first one that did
> not display the almost conventional GCC guru arrogance.
>
> GCC would have a lot more participants/helpers if it were not for this
> unwritten convention of rudeness in the community.
>
> To be fair, this isn't just a GCC thing. I've worked in various shops
> where the software paid the wages, but if a new employee asked for help
> he/she would get a similar response.
>
> I understand the impatience of the gurus. If they tried to answer every
> naive question they'd never get any work done
>
> But, if you don't have anything helpful to say, then maybe it's better
> to bite your tongue! The person you are responding to may be an idiot,
> but he/she is not the only recipient of your message.

Are you talking about gcc-help here?  It seems to me that people on
gcc-help genuinely try to be helpful.  There is no percentage for
anybody in replying on gcc-help with an unhelpful answer.

When people don't know how to program, I do honestly think the most
helpful answer is to tell them, politely, to learn how to program.
It's gcc-help, not intro-programming.

I am generally concerned that the gcc community be seen as helpful
rather than forbidding, which is why I spend time on gcc-help.  So I
would like to hear more about this topic.

Ian

Re: surprising precision

[John S. Fine]

Steve Teale wrote:
> Congratulations on a helpful answer.
Thanks for the compliment.  Despite my knowing much less about GCC than 
the experts who usually answer here, I think I sometimes have a better 
insight into what is confusing the OP.  Then I try to provide a helpful 
answer.  I think I did that this time.

>  Yours was the first one that did
> not display the almost conventional GCC guru arrogance.
But I have to disagree with you there.  The GCC gurus typically 
answering in this mailing list, including in this thread, do a great 
job.  I'm very impressed with the accuracy and helpfulness of answers in 
this mailing list.  I usually learn something.

I didn't initially answer your post, because I didn't want to dilute the 
discussion about precision with a lot of meta discussion about GCC 
gurus.  But now that it is already diluted by other replies, I didn't 
want to leave the impression that I agree with you about that 
"arrogance" of the GCC gurus.

Re: surprising precision

[Steve Teale]

On Fri, 2010-03-05 at 11:57 -0500, John S. Fine wrote:
> Steve Teale wrote:
> > Congratulations on a helpful answer.
> Thanks for the compliment.  Despite my knowing much less about GCC than 
> the experts who usually answer here, I think I sometimes have a better 
> insight into what is confusing the OP.  Then I try to provide a helpful 
> answer.  I think I did that this time.
> 
> >  Yours was the first one that did
> > not display the almost conventional GCC guru arrogance.
> But I have to disagree with you there.  The GCC gurus typically 
> answering in this mailing list, including in this thread, do a great 
> job.  I'm very impressed with the accuracy and helpfulness of answers in 
> this mailing list.  I usually learn something.
> 
> I didn't initially answer your post, because I didn't want to dilute the 
> discussion about precision with a lot of meta discussion about GCC 
> gurus.  But now that it is already diluted by other replies, I didn't 
> want to leave the impression that I agree with you about that 
> "arrogance" of the GCC gurus.

OK, so I overstated it, but the remark got some attention, and I do
believe there is substance behind it. It's probably just a cultural
thing.

Thanks anyway
Steve.