[Bug libgcc/97635] New: binary-to-DFP conversions incorrect

[Bug libgcc/97635] New: binary-to-DFP conversions incorrect

[jsm28 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97635

            Bug ID: 97635
           Summary: binary-to-DFP conversions incorrect
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libgcc
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jsm28 at gcc dot gnu.org
  Target Milestone: ---

On targets using DPD decimal floating point (tested for powerpc64le-linux-gnu,
presumably applies on s390 as well), the libgcc conversions from binary
floating point to decimal floating point, using sprintf, are not correctly
rounded.  IEEE 754 requires such conversions to be correctly rounded (using the
decimal rounding mode, not the binary rounding mode), with corresponding
exceptions (and the lowest possible decimal quantum exponent if the result is
inexact, the quantum exponent as close to 0 as possible if the result is
exact).

sprintf is used with BFP_FMT, where BFP_FMT is set to use a precision depending
on the source format.  However, that precision is the solution to a different
problem: what precision is sufficient for (to-nearest) conversion from that
binary format to decimal and back to produce the same binary value (i.e. the
precisions are the values of FLT_DECIMAL_DIG, DBL_DECIMAL_DIG, etc.).  The
problem here isn't converting to decimal and back, it's converting correctly to
a given decimal format, where exactly one value in that format is a correct
result, and other values that round back to the same binary value are
incorrect.

For example, the test below, which fails for powerpc64le-linux-gnu, does a
conversion from float to _Decimal64.  Unsurprisingly, a 9-digit decimal value
from sprintf is not sufficient for a correctly rounded 16-digit _Decimal64
value.  Using the precision of the *target* type, not the source type, would
mostly work if sprintf is correctly rounding, except that the result needs to
be rounded according to the decimal rounding mode and sprintf for a binary
input rounds according to the binary rounding mode (and the case of "to
nearest, ties away from 0" doesn't have a corresponding binary mode, even if
you restrict attention to the IEEE decimal modes and ignore the other ones
supported on Power hardware), and also there would be problems with double
rounding and missing underflow exceptions when the result is in the subnormal
range for the decimal format (there could be problems with decimal quantum
exponents as well).

extern void abort (void);
extern void exit (int);

float f = 0.1f;

_Decimal64 d = 0.1000000014901161DD;
_Decimal64 dd = (_Decimal64) 0.1f;

int
main (void)
{
  /* Compile-time conversion is correct.  */
  if (dd != d)
    abort ();
  /* Run-time conversion is incorrect.  */
  if ((_Decimal64) f != d)
    abort ();
  exit (0);
}

[Bug libgcc/97635] binary-to-DFP conversions incorrect

[jsm28 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97635

--- Comment #1 from Joseph S. Myers <jsm28 at gcc dot gnu.org> ---
With some analysis of the worst cases (for every possible pair of source and
target exponents) using continued fractions, it would also be possible to
determine a single precision value that is always correct for any given pair of
formats, so that computing a string to that precision will produce a correct
result and exceptions regardless of the binary and decimal rounding modes,
without any special handling of subnormals being needed.  Heuristically, that
would be roughly the precision of the decimal format that's the target of the
conversion, plus log to base 10 of the number of different values in the source
format.