public inbox for libc-ports@sourceware.org
 help / color / mirror / Atom feed
* Results of cos(pi/2) tests are dependent on error rounding pi/2 to a type.
@ 2013-04-11 12:59 Carlos O'Donell
  2013-04-11 17:44 ` Andreas Schwab
  0 siblings, 1 reply; 3+ messages in thread
From: Carlos O'Donell @ 2013-04-11 12:59 UTC (permalink / raw)
  To: GNU C Library, libc-ports

Community,

I've recently fixed the cos(pi/2) tests to expect the correct
result from the rounding of pi/2. The tests previously expected
cos(M_PI_2l) to return 0.0, but this is naive. The actual answer
is a non-zero value almost equal to the error in rounding pi/2
to the floating point representation.

The first derivitive of cos(pi/2) is ~1, therefore any error
in pi/2 has a direct and linear effect on the result of the answer
either causing it to be slightly more than 0 or slightly less than
zero. The exact answer to cos(pi/2) is highly dependent on the type
used.

I've tried to take into account all of the types, but in the event
that I've made a mistake please come to me and I'll help fix it.

Cheers,
Carlos.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2013-04-12  2:22 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-04-11 12:59 Results of cos(pi/2) tests are dependent on error rounding pi/2 to a type Carlos O'Donell
2013-04-11 17:44 ` Andreas Schwab
2013-04-12  2:22   ` Carlos O'Donell

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).