[Bug math/30220] New: String to double returns incorrectly rounded value for hexadecimal subnormal

[Bug math/30220] New: String to double returns incorrectly rounded value for hexadecimal subnormal

[michaelrj at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30220

            Bug ID: 30220
           Summary: String to double returns incorrectly rounded value for
                    hexadecimal subnormal
           Product: glibc
           Version: unspecified
            Status: UNCONFIRMED
          Severity: minor
          Priority: P2
         Component: math
          Assignee: unassigned at sourceware dot org
          Reporter: michaelrj at google dot com
  Target Milestone: ---

When evaluating the string "0x30000002222225p-1077" with atof or strtod in the
NEAREST rounding mode, the returned value is 0x0.6000000444444p-1022 when it
should be 0x0.6000000444445p-1022.

The easiest way to demonstrate this is by running the following line of code:
printf("%a\n%a\n", 0x30000002222225p-1077, atof("0x30000002222225p-1077"));

I discovered this bug with version 2.36, but it also occurs with a fresh
checkout (latest commit at time of writing is
e78c4c49831d0ca6253ff5ce5b956cdc4189c8a9).

When building glibc I used configure with the option
"--prefix=$HOME/glibc-install/", my triple is x86_64-linux-gnu
(host/build/target are the same), and I'm using gcc version 12.2.0.

I hope this is enough information to replicate my issue, thank you in advance.
--Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug math/30220] String to double returns incorrectly rounded value for hexadecimal subnormal

[michaelrj at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30220

--- Comment #1 from Michael Jones <michaelrj at google dot com> ---
I've done a bit more testing and it seems that the issue is that when shifting
away bits for a (hexadecimal) subnormal result, only the lowest bit is checked.
If two bits will be shifted away, and only the second lowest bit is set, that
bit won't be detected and it may be rounded down incorrectly. Here's an updated
test case:

printf("%a\n%a\n\n%a\n%a\n",    
  0x0.7fffffffffffeap-1022, 
  atof("0x0.7fffffffffffeap-1022"),
  0x0.7fffffffffffe9p-1022, 
  atof("0x0.7fffffffffffe9p-1022")
  );

This gives the following result:

"
0x0.7ffffffffffffp-1022
0x0.7fffffffffffep-1022

0x0.7ffffffffffffp-1022
0x0.7ffffffffffffp-1022
"

As you can see, atof is mistakenly rounding the case ending in "ea" down
despite it being higher than the case ending in "e9".

Hope this helps.
--Michael

-- 
You are receiving this mail because:
You are on the CC list for the bug.