Re: Fw: issue with tgamma

[Paul Zimmermann <Paul.Zimmermann@inria.fr>]

       Hi Craig,

you are perfectly right. I'll have to adapt our testing framework.
However, does Newlib yield a domain or range error? I cannot test,
since I have to provide a fake errno to be able to compile my program
with Newlib's libm.a.

Paul

> From: C Howland <cc1964t@gmail.com>
> Date: Fri, 5 Jan 2024 12:34:43 -0500
> 
> 
> [1:text/plain Show]
> 
> 
> [2:text/html Hide Save:noname (2kB)]
> 
>  From: Paul Zimmermann <Paul.Zimmermann@inria.fr>
>  Sent: Friday, January 5, 2024 5:49 AM
>  To: newlib@sourceware.org <newlib@sourceware.org>
>  Subject: issue with tgamma 
>   
>         Hi,
> 
>  I discovered the following issue with tgamma: for
>  x=-0x1.13a5ccd87c9bbp+1008
>  it yields +Inf instead of NaN:
> 
>  Checking tgamma with newlib-4.4.0.20231231
>  tgamma 0 -1 -0x1.13a5ccd87c9bbp+1008 [-nan] [inf] inf inf
>  libm gives inf
>  mpfr gives -nan
> 
>  This issue was present in earlier versions of Newlib, but we only found
>  it
>  recently since we improved our testing framework.
> 
>  Paul
> 
>      This strictly is OK, as the standards allow either a range or domain
> error for negative integers, and domain errors may return an
> implementation-defined value.   It most likely has been this way since the
> 1993 date in the source file, which includes
>  *   5. Special Cases
>  *              lgamma(2+s) ~ s*(1-Euler) for tiny s
>  *              lgamma(1)=lgamma(2)=0
>  *              lgamma(x) ~ -log(x) for tiny x
>  *              lgamma(0) = lgamma(inf) = inf
>  *              lgamma(-integer) = +-inf
> (The tgamma function ends up using the lgamma function.)
>      POSIX does have an implied preference of sorts for the NaN return, so
> changing it could possibly be considered a small improvement even if not
> necessary--although it is strange that they define the result for argument 0
> differently, calling for INF there, seems inconsistent.
>      (It's somewhat interesting that the MPFR return value you show is -NaN.
>  I suppose they chose that to help reflect it being due to a negative
> argument.)
>                          Craig