From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sam Lantinga To: egcs@egcs.cygnus.com Cc: highlander@lokigames.com, slouken@devolution.com, dvenckus@activision.com Subject: Re: Rounding errors using doubles? Date: Thu, 18 Mar 1999 16:45:00 -0000 Message-id: X-SW-Source: 1999-03/msg00633.html > On Thu, Mar 18, 1999 at 03:41:32PM -0800, Sam Lantinga wrote: > > On a Pentium II, the following code gives an incorrect result when using > > doubles, but a correct result when using floats: > I think this is normal. You should probably learn why at > http://www.linuxsupportline.com/~billm/ . I have to thank everyone here on this list for your responsiveness. The problem was caused by unexpected values in code using doubles that was ported from Windows. Apparently, by default, Windows uses 53 bits of precision for it's double operations. Linux uses 64 bits. Since the code relies on the exact behavior of doubles quite extensively, we are setting the precision of the FPU to 53 bits using the fldcw instruction. :) BTW, does anyone know what precision is used on the PPC and Alpha architectures? Thanks for your help! -Sam Lantinga (slouken@devolution.com) Lead Programmer, Loki Entertainment Software -- Author of Simple DirectMedia Layer - http://www.devolution.com/~slouken/SDL/ -- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sam Lantinga To: egcs@egcs.cygnus.com Cc: highlander@lokigames.com, slouken@devolution.com, dvenckus@activision.com Subject: Re: Rounding errors using doubles? Date: Wed, 31 Mar 1999 23:46:00 -0000 Message-ID: X-SW-Source: 1999-03n/msg00638.html Message-ID: <19990331234600.bT0JmvfpKd74d1LL8DfuMMNCa8z8Y0M-iqkpwvWOpk8@z> > On Thu, Mar 18, 1999 at 03:41:32PM -0800, Sam Lantinga wrote: > > On a Pentium II, the following code gives an incorrect result when using > > doubles, but a correct result when using floats: > I think this is normal. You should probably learn why at > http://www.linuxsupportline.com/~billm/ . I have to thank everyone here on this list for your responsiveness. The problem was caused by unexpected values in code using doubles that was ported from Windows. Apparently, by default, Windows uses 53 bits of precision for it's double operations. Linux uses 64 bits. Since the code relies on the exact behavior of doubles quite extensively, we are setting the precision of the FPU to 53 bits using the fldcw instruction. :) BTW, does anyone know what precision is used on the PPC and Alpha architectures? Thanks for your help! -Sam Lantinga (slouken@devolution.com) Lead Programmer, Loki Entertainment Software -- Author of Simple DirectMedia Layer - http://www.devolution.com/~slouken/SDL/ --