issue with tgamma

issue with tgamma

[Paul Zimmermann]

       Hi,

I discovered the following issue with tgamma: for x=-0x1.13a5ccd87c9bbp+1008
it yields +Inf instead of NaN:

Checking tgamma with newlib-4.4.0.20231231
tgamma 0 -1 -0x1.13a5ccd87c9bbp+1008 [-nan] [inf] inf inf
libm gives inf
mpfr gives -nan

This issue was present in earlier versions of Newlib, but we only found it
recently since we improved our testing framework.

Paul

Re: Fw: issue with tgamma

[C Howland]

[-- Attachment #1: Type: text/plain, Size: 1626 bytes --]

> *From:* Paul Zimmermann <Paul.Zimmermann@inria.fr>
> *Sent:* Friday, January 5, 2024 5:49 AM
> *To:* newlib@sourceware.org <newlib@sourceware.org>
> *Subject:* issue with tgamma
>
>        Hi,
>
> I discovered the following issue with tgamma: for
> x=-0x1.13a5ccd87c9bbp+1008
> it yields +Inf instead of NaN:
>
> Checking tgamma with newlib-4.4.0.20231231
> tgamma 0 -1 -0x1.13a5ccd87c9bbp+1008 [-nan] [inf] inf inf
> libm gives inf
> mpfr gives -nan
>
> This issue was present in earlier versions of Newlib, but we only found it
> recently since we improved our testing framework.
>
> Paul
>
>      This strictly is OK, as the standards allow either a range or domain
error for negative integers, and domain errors may return an
implementation-defined value.   It most likely has been this way since the
1993 date in the source file, which includes
 *   5. Special Cases
 *              lgamma(2+s) ~ s*(1-Euler) for tiny s
 *              lgamma(1)=lgamma(2)=0
 *              lgamma(x) ~ -log(x) for tiny x
 *              lgamma(0) = lgamma(inf) = inf
 *              lgamma(-integer) = +-inf
(The tgamma function ends up using the lgamma function.)
     POSIX does have an implied preference of sorts for the NaN return, so
changing it could possibly be considered a small improvement even if not
necessary--although it is strange that they define the result for argument
0 differently, calling for INF there, seems inconsistent.
     (It's somewhat interesting that the MPFR return value you show is
-NaN.  I suppose they chose that to help reflect it being due to a negative
argument.)
                         Craig

Re: Fw: issue with tgamma

[Paul Zimmermann]

       Hi Craig,

you are perfectly right. I'll have to adapt our testing framework.
However, does Newlib yield a domain or range error? I cannot test,
since I have to provide a fake errno to be able to compile my program
with Newlib's libm.a.

Paul

> From: C Howland <cc1964t@gmail.com>
> Date: Fri, 5 Jan 2024 12:34:43 -0500
> 
> 
> [1:text/plain Show]
> 
> 
> [2:text/html Hide Save:noname (2kB)]
> 
>  From: Paul Zimmermann <Paul.Zimmermann@inria.fr>
>  Sent: Friday, January 5, 2024 5:49 AM
>  To: newlib@sourceware.org <newlib@sourceware.org>
>  Subject: issue with tgamma 
>   
>         Hi,
> 
>  I discovered the following issue with tgamma: for
>  x=-0x1.13a5ccd87c9bbp+1008
>  it yields +Inf instead of NaN:
> 
>  Checking tgamma with newlib-4.4.0.20231231
>  tgamma 0 -1 -0x1.13a5ccd87c9bbp+1008 [-nan] [inf] inf inf
>  libm gives inf
>  mpfr gives -nan
> 
>  This issue was present in earlier versions of Newlib, but we only found
>  it
>  recently since we improved our testing framework.
> 
>  Paul
> 
>      This strictly is OK, as the standards allow either a range or domain
> error for negative integers, and domain errors may return an
> implementation-defined value.   It most likely has been this way since the
> 1993 date in the source file, which includes
>  *   5. Special Cases
>  *              lgamma(2+s) ~ s*(1-Euler) for tiny s
>  *              lgamma(1)=lgamma(2)=0
>  *              lgamma(x) ~ -log(x) for tiny x
>  *              lgamma(0) = lgamma(inf) = inf
>  *              lgamma(-integer) = +-inf
> (The tgamma function ends up using the lgamma function.)
>      POSIX does have an implied preference of sorts for the NaN return, so
> changing it could possibly be considered a small improvement even if not
> necessary--although it is strange that they define the result for argument 0
> differently, calling for INF there, seems inconsistent.
>      (It's somewhat interesting that the MPFR return value you show is -NaN.
>  I suppose they chose that to help reflect it being due to a negative
> argument.)
>                          Craig