[Bug libc/13246] New: RFE: strtold: do not include uninitialized bytes when converting "NaN"

[Bug libc/13246] New: RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jim at meyering dot net]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

             Bug #: 13246
           Summary: RFE: strtold: do not include uninitialized bytes when
                    converting "NaN"
           Product: glibc
           Version: 2.14
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper.fsp@gmail.com
        ReportedBy: jim@meyering.net
    Classification: Unclassified


strtold ("nan", NULL) appears to return a result that depends on the prior
state of the stack.  That non-determinism caused GNU sort to infloop, but we've
worked around this glibc-specific issue: http://debbugs.gnu.org/9612

Here's a little program to demonstrate the problem.
Note how the results differ with compilation options:

  #include <stdlib.h>
  #include <stdio.h>

  static char *
  fmt_nan (long double x)
  {
    unsigned int i;
    static char buf[33];
    unsigned char const *p = (unsigned char const *) &x;
    for (i = 0; i < sizeof x; i++)
      sprintf (buf + 2*i, "%02x", *p++);
    return buf;
  }

  int
  main ()
  {
    const char *q = "nan";
    long double x = strtold (q, NULL);
    printf ("%s\n", fmt_nan (x));

    x = 0;
    x = strtold (q, NULL);
    printf ("%s\n", fmt_nan (x));

    return 0;
  }

  $ gcc -O0 -Wall -Wextra -W /t/strtold-bogosity.c && ./a.out
  00000000000000c0ff7f400000000000
  00000000000000c0ff7f000000000000
  $ gcc -O1 -Wall -Wextra -W /t/strtold-bogosity.c && ./a.out
  00000000000000c0ff7f000000000000
  00000000000000c0ff7f000000000000

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID

--- Comment #1 from Andreas Schwab <schwab@linux-m68k.org> 2011-10-05 14:52:26 UTC ---
That's a compiler bug.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

Rich Felker <bugdal at aerifal dot cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |bugdal at aerifal dot cx
         Resolution|INVALID                     |

--- Comment #2 from Rich Felker <bugdal at aerifal dot cx> 2011-10-05 22:09:50 UTC ---
Please explain how this is a compiler bug. This should not be able to happen
unless glibc's implementation of strtold is invoking undefined behavior.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID

--- Comment #3 from Andreas Schwab <schwab@linux-m68k.org> 2011-10-06 07:56:35 UTC ---
glibc does not use any undefined behaviour, thus it is a compiler bug.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jakub at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

Jakub Jelinek <jakub at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at redhat dot com

--- Comment #4 from Jakub Jelinek <jakub at redhat dot com> 2011-10-06 08:15:00 UTC ---
I think it is neither.  On x86_64/i686, the hw long double is 80 bit, but
sizeof (long double) is 16 (on x86_64) resp. 12 (on i686).  If you store a long
double from a hw register into memory, it only stores there the 80 bits, the
remaining bits are left unmodified, it is padding that is ignored, thus e.g. if
it is stored
onto the stack, whatever bits were in the padding before are kept there.
It is a bug to look for something meaningful among the padding bits.
That is similar to looking at padding in structures, undefined behavior in C.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jim at meyering dot net]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #5 from jim at meyering dot net <jim at meyering dot net> 2011-10-06 08:32:36 UTC ---
Hi Jakub,

Thanks for commenting.
I realize that this is not officially a bug.  That's why I wrote "RFE: ...".
However, is there some other way to avoid valgrind "warnings" about those stack
bytes being used uninitialized?

Alternatively, can you recommend a portable way to compare the valid bits of
two NaN values?

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jakub at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #6 from Jakub Jelinek <jakub at redhat dot com> 2011-10-06 08:45:17 UTC ---
First of all, it is impossible to do anything about this on the glibc side.
Just look at the generated code, it is:
  call strtold@plt
  fstpt  (%rsp)          # resp. fstpt (%esp)
For x86_64/i686 it would be sufficient if you used memset to clear the whole
var, then store the value into it after making sure the compiler doesn't
optimize it away.  Perhaps something like:
  union U { long double l; unsigned char c[sizeof (long double); };
  volatile union U u;
  size_t i;
  for (i = 0; i < sizeof (long double); i++)
    u.c[i] = 0;
  u.l = strtold (...);
would do the job portably?

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jim at meyering dot net]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #7 from jim at meyering dot net <jim at meyering dot net> 2011-10-06 09:30:49 UTC ---
Thanks for the suggestion.
The fix I applied to sort.c was to use a bare memset, since that happened to
solve the problem at least with gcc-4.7.x.  Your approach is better if an
optimizer makes the false assumption that the bytes returned by strtold are
always defined, in which case it might eliminate my seemingly redundant memset,
and we'd have to report the optimizer bug.

However, do you really think strtold should continue to return undefined bytes?
That implies that any application that must sort (stably) user-supplied
floating point numbers including NaNs must jump through these hoops.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #8 from Andreas Schwab <schwab@linux-m68k.org> 2011-10-06 09:40:57 UTC ---
The return value of strtold does not contain any undefined bytes.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jim at meyering dot net]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #9 from jim at meyering dot net <jim at meyering dot net> 2011-10-06 09:47:47 UTC ---
"uninitialized" bytes.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[jakub at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #10 from Jakub Jelinek <jakub at redhat dot com> 2011-10-06 09:56:33 UTC ---
Andreas is right, the value is returned in %st(0), which is an 80-bit hw
register.
So, as I said earlier, there is really nothing wrong on the strtold side.
The problem is when the compiler stores that %st(0) into memory.  And you
really don't want to penalize all long double using code that on each memory
store
instead of using fstp (%rsp) there is fstp %(rsp); movw $0, 10(%rsp); movl $0,
12(%rsp);

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=13246

--- Comment #11 from Rich Felker <bugdal at aerifal dot cx> 2011-10-06 12:39:06 UTC ---
OK, it's the test program and GNU sort that are wrong (reading padding bits).
Unfortunately I don't see a portable way to "compare" NaNs like this. But if
they're being read from strings, why not just compare the string form when the
value is NaN?

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/13246] RFE: strtold: do not include uninitialized bytes when converting "NaN"

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=13246

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.