Re: [Bug libc/15854] New: strtod should avoid calling strlen

["Ondřej Bílka" <neleai@seznam.cz>]

On Tue, Aug 20, 2013 at 02:12:32AM +0000, emogenet at gmail dot com wrote:
> http://sourceware.org/bugzilla/show_bug.cgi?id=15854
> 
>             Bug ID: 15854
>            Summary: strtod should avoid calling strlen
>            Product: glibc
>            Version: 2.18
>             Status: NEW
>           Severity: enhancement
>           Priority: P2
>          Component: libc
>           Assignee: unassigned at sourceware dot org
>           Reporter: emogenet at gmail dot com
>                 CC: drepper.fsp at gmail dot com
> 
> Problem : glibc's strtod seem to systematically call strlen on its input.
> 
> To the layman that I am, there doesn't seem to be any legitimate reason why it
> should: it seems that strtod should simply consume its input one char at a time
> until it reaches a char that marks the end of a valid FP number ASCII rep. and
> should therefore work on a non-zero terminated buffer, as long said buffer ends
> with a char that terminates the parsing.
>
This is not that big problem, strtod only uses strlen in following context

  decimal = _NL_CURRENT (LC_NUMERIC, DECIMAL_POINT); // which is "."
  decimal_len = strlen (decimal); // which is 1

 
> This internal call to strlen makes it essentially impossible to call strtod
> on a no zero terminated buffer, and there seems to be no other way to otherwise
> access the non-trivial code that converts an ASCII buffer to a FP number.
> 
> This makes it in particular painful to call strtod on a very large mmap'd
> buffer of ASCII floats : strlen will plow through the entire file for every
> call to strtod, making things highly inefficient (it is also not guaranteed
> not to crash).
>
Do you have testcase to demonstrate quadratic behavior? It is possible
that end is determined by other ineffective means.
 
> To work around this shortcoming, one ends up having to figure out the end of
> the FP ASCII string, "by hand", copy the result to a zero terminated buffer,
> and then call strtod on that.
> 
> This is both inefficient and clunky.
> 
> See this article for a good description of the issue:
> 
> http://www.ryanjuckett.com/programming/c-cplusplus/25-optimizing-atof-and-strtod
> 
> Here's another instance of the problem:
> 
> http://stackoverflow.com/questions/2033845/any-one-know-how-to-convert-a-huge-char-array-to-float-very-huge-array-perform
> 
Not relevant for us as these are windows problems.