Accuracy of Mathematical Functions

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Dear all,

we have updated our comparison:

https://members.loria.fr/PZimmermann/papers/accuracy.pdf

This update is for GNU libc 2.40:

* it includes the new functions exp10m1, exp2m1, log10p1, log2p1
  added in 2.40 (for all formats)
* apart from that, no main changes with respect to 2.39

Paul

Re: Accuracy of Mathematical Functions

[Joseph Myers]

On Tue, 6 Aug 2024, Paul Zimmermann wrote:

>        Dear all,
> 
> we have updated our comparison:
> 
> https://members.loria.fr/PZimmermann/papers/accuracy.pdf
> 
> This update is for GNU libc 2.40:
> 
> * it includes the new functions exp10m1, exp2m1, log10p1, log2p1
>   added in 2.40 (for all formats)

The table for single precision only has exp10m1, not the other three 
functions.

Also, the tables with inputs giving largest known errors don't seem 
consistent about whether they include functions not implemented in the 
relevant libm implementations.  For example, "Table 14: Double extended 
precision: OpenLibm and Musl." has empty lines for j0, j1, y0 and y1, but 
various such tables do not have blank lines for the four C23 functions 
newly added to glibc.  My expectation would be that either all the tables 
include all the functions (even when the corresponding lines are blank), 
or the blank lines for unimplemented functions are consistently omitted 
across all the tables.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: Accuracy of Mathematical Functions

[Paul Zimmermann]

       Dear Joseph,

> > we have updated our comparison:
> > 
> > https://members.loria.fr/PZimmermann/papers/accuracy.pdf
> > 
> > This update is for GNU libc 2.40:
> > 
> > * it includes the new functions exp10m1, exp2m1, log10p1, log2p1
> >   added in 2.40 (for all formats)
> 
> The table for single precision only has exp10m1, not the other three 
> functions.

sorry, I have fixed that in a new version (same link).

> Also, the tables with inputs giving largest known errors don't seem 
> consistent about whether they include functions not implemented in the 
> relevant libm implementations.  For example, "Table 14: Double extended 
> precision: OpenLibm and Musl." has empty lines for j0, j1, y0 and y1, but 
> various such tables do not have blank lines for the four C23 functions 
> newly added to glibc.  My expectation would be that either all the tables 
> include all the functions (even when the corresponding lines are blank), 
> or the blank lines for unimplemented functions are consistently omitted 
> across all the tables.

good catch. I have removed empty lines.

Paul

Re: Accuracy of Mathematical Functions

[Joseph Myers]

On Wed, 7 Aug 2024, Paul Zimmermann wrote:

>        Dear Joseph,
> 
> > > we have updated our comparison:
> > > 
> > > https://members.loria.fr/PZimmermann/papers/accuracy.pdf
> > > 
> > > This update is for GNU libc 2.40:
> > > 
> > > * it includes the new functions exp10m1, exp2m1, log10p1, log2p1
> > >   added in 2.40 (for all formats)
> > 
> > The table for single precision only has exp10m1, not the other three 
> > functions.
> 
> sorry, I have fixed that in a new version (same link).

Thanks.  Some further comments on the document (some of which I may have 
made before):

* The bug referenced as a reason for not testing other rounding modes was 
fixed long ago, results in other rounding modes would be interesting as 
well (they would of course make the document longer), and worst-case 
inputs for other rounding modes would be worth adding to the glibc tests.

* I don't think "dummy implementations" is a useful description for 
implementations that do actually implement the function (within a few 
ulp).  There are many possible ways to implement a function, including 
building on other functions, and those ones are optimized for getting the 
APIs into glibc for all supported formats rather than for speed or 
accuracy (and replacing by faster, more accurate implementations for 
particular formats is welcome).

* As previously mentioned, I still think it would be of interest to see 
such data for complex functions (a function with one complex argument and 
a complex result is effectively two bivariate real functions.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: Accuracy of Mathematical Functions

[Paul Zimmermann]

       Dear Joseph,

> Thanks.  Some further comments on the document (some of which I may have 
> made before):
> 
> * The bug referenced as a reason for not testing other rounding modes was 
> fixed long ago, results in other rounding modes would be interesting as 
> well (they would of course make the document longer), and worst-case 
> inputs for other rounding modes would be worth adding to the glibc tests.

in the next update we'll point out that these disastrous results with
other rounding modes were in the past. Surely results with other rounding modes
would be interesting, volunteers to implement that are welcome!

I did a quick test with exp and GNU libc 2.40:

    -0x1.49f33ad2c1c58p+9,   /* GNU libc 0.510359 */
    -0x1.9df9aecc00001p-29,  /* GNU libc RNDZ 1.24399 */
    0x1.50f5a68c91c18p-1,    /* GNU libc RNDU 1.01597 */
    -0x1.9df9aecc00001p-29,  /* GNU libc RNDD 1.24399 */

> * I don't think "dummy implementations" is a useful description for 
> implementations that do actually implement the function (within a few 
> ulp).  There are many possible ways to implement a function, including 
> building on other functions, and those ones are optimized for getting the 
> APIs into glibc for all supported formats rather than for speed or 
> accuracy (and replacing by faster, more accurate implementations for 
> particular formats is welcome).

agreed, maybe "trivial" would be more accurate. By the way, if one wants
to replace say exp10m1f by a more accurate version, what are the steps to
follow?

> * As previously mentioned, I still think it would be of interest to see 
> such data for complex functions (a function with one complex argument and 
> a complex result is effectively two bivariate real functions.

we also got similar feedback for complex functions. Indeed I believe one might
have surprises, even for basic operations since there is no standard. But
again, time is limited, and volunteers are welcome! (This might be a different
document, testing less libraries, and updated less often.)

Paul

Re: Accuracy of Mathematical Functions

[Adhemerval Zanella Netto]

On 09/08/24 05:16, Paul Zimmermann wrote:
>        Dear Joseph,
> 
>> Thanks.  Some further comments on the document (some of which I may have 
>> made before):
>>
>> * The bug referenced as a reason for not testing other rounding modes was 
>> fixed long ago, results in other rounding modes would be interesting as 
>> well (they would of course make the document longer), and worst-case 
>> inputs for other rounding modes would be worth adding to the glibc tests.
> 
> in the next update we'll point out that these disastrous results with
> other rounding modes were in the past. Surely results with other rounding modes
> would be interesting, volunteers to implement that are welcome!
> 
> I did a quick test with exp and GNU libc 2.40:
> 
>     -0x1.49f33ad2c1c58p+9,   /* GNU libc 0.510359 */
>     -0x1.9df9aecc00001p-29,  /* GNU libc RNDZ 1.24399 */
>     0x1.50f5a68c91c18p-1,    /* GNU libc RNDU 1.01597 */
>     -0x1.9df9aecc00001p-29,  /* GNU libc RNDD 1.24399 */
> 
>> * I don't think "dummy implementations" is a useful description for 
>> implementations that do actually implement the function (within a few 
>> ulp).  There are many possible ways to implement a function, including 
>> building on other functions, and those ones are optimized for getting the 
>> APIs into glibc for all supported formats rather than for speed or 
>> accuracy (and replacing by faster, more accurate implementations for 
>> particular formats is welcome).
> 
> agreed, maybe "trivial" would be more accurate. By the way, if one wants
> to replace say exp10m1f by a more accurate version, what are the steps to
> follow?

The exp10m1 should be easier because there is no arch-specific implementation.
I think it would be easier as add a sysdeps/ieee754/flt-32/s_exp10m1.c
one with the same semantics as the template one.  I can help you out if need.

> 
>> * As previously mentioned, I still think it would be of interest to see 
>> such data for complex functions (a function with one complex argument and 
>> a complex result is effectively two bivariate real functions.
> 
> we also got similar feedback for complex functions. Indeed I believe one might
> have surprises, even for basic operations since there is no standard. But
> again, time is limited, and volunteers are welcome! (This might be a different
> document, testing less libraries, and updated less often.)
> 
> Paul

Re: Accuracy of Mathematical Functions

[Joseph Myers]

On Fri, 9 Aug 2024, Adhemerval Zanella Netto wrote:

> The exp10m1 should be easier because there is no arch-specific implementation.
> I think it would be easier as add a sysdeps/ieee754/flt-32/s_exp10m1.c
> one with the same semantics as the template one.  I can help you out if need.

And also add a benchmark to make sure the new version is (hopefully) 
faster than the generic one, as well as more accurate.

-- 
Joseph S. Myers
josmyers@redhat.com

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Dear all,

we have updated our comparison:

https://members.loria.fr/PZimmermann/papers/accuracy.pdf

This is a new version with updated versions of the different libraries, new corner
cases found, and the ARM Performance Library is now included.

Paul Zimmermann

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Dear all,

we have updated our comparison with the glibc 2.38 release:

https://members.loria.fr/PZimmermann/papers/accuracy.pdf

Not much improvement for glibc with respect to 2.37.

However there is a lot of new stuff in this new version:

* the Microsoft math library is now included (it was clearly missing)
* we also added the FreeBSD math library
* and we added some new C23 functions (acospi, cospi, ...) which are available
  in the Intel and FreeBSD math libraries (but not yet in glibc)

Paul Zimmermann

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Hi,

we have updated our comparison after the glibc 2.37 release:

https://members.loria.fr/PZimmermann/papers/accuracy.pdf

Not much improvement for glibc with respect to 2.36.

The main change is that the LLVM library now implements more
correctly-rounded routines in single precision.

Paul Zimmermann

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Hi,

we have updated our comparison after the glibc 2.36 release:

https://members.loria.fr/PZimmermann/papers/accuracy.pdf

Not much improvement for glibc with respect to 2.35.

The main change is that the LLVM library now implements a few
correctly-rounded routines in single precision (log, log10, log2,
hypot), and hypot in double precision.

Paul Zimmermann

Re: Accuracy of Mathematical Functions

[Joseph Myers]

On Sat, 12 Feb 2022, Paul Zimmermann wrote:

> > 1. I don't know if any of those libraries include any of the new functions 
> > C23 adds from TS 18661-4 (beyond exp10), but if they do, it might be worth 
> > adding them to the comparison.  (My remarks from 
> > <https://sourceware.org/pipermail/libc-alpha/2022-January/135006.html> 
> > apply regarding adding them to glibc - when I get time I hope to add them, 
> > as with other new C23 features, if no-one else has done them by then.)
> 
> is http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1946.pdf the correct
> reference for the list of these functions? Most of them are available in
> MPFR, when they will be available in glibc we might indeed add them.

I think N2401 was the version used for integration into C23.  I'd advise 
using the current C23 working draft (currently N2731, but hopefully N2912 
will become available soon).

Note that the reduction functions from TS 18661-4 weren't integrated into 
C23 (but would also have been much more awkward to handle in this accuracy 
testing); instead there will be a revision of TS 18661-4 (and -5) based on 
C23 to include those features not integrated.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Accuracy of Mathematical Functions

[Paul Zimmermann]

       Dear Joseph,

> Date: Fri, 11 Feb 2022 18:23:58 +0000
> From: Joseph Myers <joseph@codesourcery.com>
> 
> On Fri, 11 Feb 2022, Paul Zimmermann wrote:
> 
> >        Hi,
> > 
> > with the release of GNU libc 2.35, we have published a new version of our
> > comparison of the accuracy of mathematical libraries:
> > 
> > https://members.loria.fr/PZimmermann/papers/accuracy.pdf
> > 
> > For the GNU libc, with respect to 2.34, we observed an improvement in j0f,
> > tgamma, hypot, hypotl and hypotf128.
> > 
> > With respect to the previous update, we compare two new libraries: LLVM libc
> > and ROCm.
> 
> Thanks for the update.  A few remarks on things it might be interesting to 
> add to the analysis:
> 
> 1. I don't know if any of those libraries include any of the new functions 
> C23 adds from TS 18661-4 (beyond exp10), but if they do, it might be worth 
> adding them to the comparison.  (My remarks from 
> <https://sourceware.org/pipermail/libc-alpha/2022-January/135006.html> 
> apply regarding adding them to glibc - when I get time I hope to add them, 
> as with other new C23 features, if no-one else has done them by then.)

is http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1946.pdf the correct
reference for the list of these functions? Most of them are available in
MPFR, when they will be available in glibc we might indeed add them.

> 2. I don't know if any of those libraries have IEEE binary16 functions (C 
> _Float16), but again, if any do, they might be worth testing (in that 
> case, exhaustive testing should be possible for functions of two 
> arguments, not just functions of a single argument).
>
> 3. As previously remarked, FreeBSD libm is another implementation that 
> might be worth testing (though that would require running the tests on a 
> FreeBSD system).
> 
> 4. As previously remarked, it would be interesting to see similar data for 
> complex functions (real and imaginary parts of the result of a function 
> with one complex argument being essentially the same as the case of a 
> function with a real result and two real arguments).

for points 2-4, I agree this would be interesting, but we have limited
manpower. If anyone wants to help us, she/he is welcome!

Best regards,
Paul

Re: Accuracy of Mathematical Functions

[Joseph Myers]

On Fri, 11 Feb 2022, Paul Zimmermann wrote:

>        Hi,
> 
> with the release of GNU libc 2.35, we have published a new version of our
> comparison of the accuracy of mathematical libraries:
> 
> https://members.loria.fr/PZimmermann/papers/accuracy.pdf
> 
> For the GNU libc, with respect to 2.34, we observed an improvement in j0f,
> tgamma, hypot, hypotl and hypotf128.
> 
> With respect to the previous update, we compare two new libraries: LLVM libc
> and ROCm.

Thanks for the update.  A few remarks on things it might be interesting to 
add to the analysis:

1. I don't know if any of those libraries include any of the new functions 
C23 adds from TS 18661-4 (beyond exp10), but if they do, it might be worth 
adding them to the comparison.  (My remarks from 
<https://sourceware.org/pipermail/libc-alpha/2022-January/135006.html> 
apply regarding adding them to glibc - when I get time I hope to add them, 
as with other new C23 features, if no-one else has done them by then.)

2. I don't know if any of those libraries have IEEE binary16 functions (C 
_Float16), but again, if any do, they might be worth testing (in that 
case, exhaustive testing should be possible for functions of two 
arguments, not just functions of a single argument).

3. As previously remarked, FreeBSD libm is another implementation that 
might be worth testing (though that would require running the tests on a 
FreeBSD system).

4. As previously remarked, it would be interesting to see similar data for 
complex functions (real and imaginary parts of the result of a function 
with one complex argument being essentially the same as the case of a 
function with a real result and two real arguments).

-- 
Joseph S. Myers
joseph@codesourcery.com

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Hi,

with the release of GNU libc 2.35, we have published a new version of our
comparison of the accuracy of mathematical libraries:

https://members.loria.fr/PZimmermann/papers/accuracy.pdf

For the GNU libc, with respect to 2.34, we observed an improvement in j0f,
tgamma, hypot, hypotl and hypotf128.

With respect to the previous update, we compare two new libraries: LLVM libc
and ROCm.

Paul Zimmermann

Accuracy of Mathematical Functions

[Paul Zimmermann]

       Hi,

a new update of "Accuracy of Mathematical Functions in Single, Double,
Extended Double and Quadruple Precision" is available:

https://members.loria.fr/PZimmermann/papers/#accuracy

This update is with GNU libc 2.34. One improvement in GNU libc is that
the maximal error for the Bessel functions in single precision (except j0)
has decreased from millions of ulps to a few ulps.

This update considers two additional libraries: the Apple math library and
the CUDA library (thanks to Vincenzo Innocente).

Best regards,
Paul Zimmermann

accuracy of mathematical functions

[Paul Zimmermann]

       Hi,

I have updated my comparison with the newly released 2.33 version.
No big difference with the previous version, except that I included
the "long double" format (aka ldbl-96), as suggested by Joseph Myers,
for which the Intel Math library is uniformly better than all other libraries.

https://members.loria.fr/PZimmermann/papers/#accuracy

Paul