[Bug libfortran/36857] New: Non-English locale breaks gfortran float formatting ("printf is broken")

[Bug libfortran/36857] New: Non-English locale breaks gfortran float formatting ("printf is broken")

[pav at iki dot fi]

GFortran float printing breaks when a non-English locale is selected. Instead,
"Internal error: printf is broken" appears. The failing part is a sanity check
in libgfortran, where it was forgotten that in some non-English locales,
sprintf will format floats using a comma instead of a period as the decimal
separator.

Tested gcc version 4:4.2.3-1ubuntu6 as bundled on Ubuntu Hardy, but probably
present also in trunk.

The following code triggers this bug:

bug.f90:
--------
program main
    CALL badlocale()
    WRITE(*,'(G2.4)') 1.2345
end program main

bug.c:
------
#include <locale.h>
void badlocale_() { setlocale(LC_ALL, "fi_FI.UTF-8"); }

The compiled program to fails with internal error:

    gcc -c -o bug.o bug.c; gfortran -o bug bug.f90 bug.o  -lc ; ./bug
    At line 3 of file bug.f90
    Internal Error: printf is broken

If the code executed correctly, the expected output would be

    1,235

To my understanding the failing code appears to be in
libgfortran/io/write_float.def:output_float (around line 126):

    /* Check the given string has punctuation in the correct places.  */
    if (d != 0 && (buffer[2] != '.' || buffer[ndigits + 2] != 'e'))
        internal_error (&dtp->common, "printf is broken");

Before execution arrives here, the buffer is filled by sprintf in write_float,
and as said, in some locales the correct formatting of a float will contain a
comma instead of a period.


-- 
           Summary: Non-English locale breaks gfortran float formatting
                    ("printf is broken")
           Product: gcc
           Version: 4.2.3
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: pav at iki dot fi
 GCC build triplet: i486-linux-gnu
  GCC host triplet: i486-linux-gnu
GCC target triplet: i486-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[burnus at gcc dot gnu dot org]

------- Comment #1 from burnus at gcc dot gnu dot org  2008-07-16 20:45 -------
> program main
>     CALL badlocale()
>     WRITE(*,'(G2.4)') 1.2345
> end program main
> 
> If the code executed correctly, the expected output would be
>     1,235

No, the expected output would be "1.235" according to the Fortran standard. You
need to use, e.g.,
      WRITE(*, '(G2.4)',decimal='comma') 1.2345
to get the "1,235".  (decimal="comma" and "dc" are supported since gfortran 4.4
and part for the Fortran 2003 standard.)

However, I agree that gfortran should not print an internal error as setting
the locale might happen in mixed-language programming. The question is only how
to enforce a LC_ALL = "C" for libgfortran's printf calls without causing other
problems such has performance hits.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[jvdelisle at gcc dot gnu dot org]

------- Comment #2 from jvdelisle at gcc dot gnu dot org  2008-07-19 17:04 -------
A simple fix would be to add a check for ',' or '.'.  I can put a simple check
in init options or main.c to set it at run time.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
                   |dot org                     |org
             Status|UNCONFIRMED                 |ASSIGNED
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-07-19 17:04:56
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[kargl at gcc dot gnu dot org]

------- Comment #3 from kargl at gcc dot gnu dot org  2008-07-19 18:28 -------
This is untested, but you get the idea.


Index: gcc/fortran/f95-lang.c
===================================================================
--- gcc/fortran/f95-lang.c      (revision 137549)
+++ gcc/fortran/f95-lang.c      (working copy)
@@ -50,6 +50,10 @@ along with GCC; see the file COPYING3.  
 #include "trans-types.h"
 #include "trans-const.h"

+#ifndef _LOCALE_H_
+#include <locale.h>
+#endif
+
 /* Language-dependent contents of an identifier.  */

 struct lang_identifier
@@ -250,6 +254,9 @@ gfc_be_parse_file (int set_yydebug ATTRI
 static bool
 gfc_init (void)
 {
+
+  setlocale(LC_ALL, "POSIX");
+
   if (!gfc_cpp_enabled ())
     {
       linemap_add (line_table, LC_ENTER, false, gfc_source_file, 1);
Index: libgfortran/runtime/main.c
===================================================================
--- libgfortran/runtime/main.c  (revision 137549)
+++ libgfortran/runtime/main.c  (working copy)
@@ -28,6 +28,7 @@ the Free Software Foundation, 51 Frankli
 Boston, MA 02110-1301, USA.  */

 #include "libgfortran.h"
+#include <locale.h>
 #include <stdlib.h>
 #include <string.h>
 #include <limits.h>
@@ -144,6 +145,8 @@ full_exe_path (void)
 static void __attribute__((constructor))
 init (void)
 {
+  setlocale(LC_ALL, "POSIX");
+
   /* Figure out the machine endianness.  */
   determine_endianness ();



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[pav at iki dot fi]

------- Comment #4 from pav at iki dot fi  2008-07-19 18:58 -------
Does setting the locale to POSIX in the above patch fix the issue if the
program using the runtime for example changes the locale just before calling a
Fortran routine that does some printing?

Also, does the locale change affect other parts of the program calling the
gfortran runtime? If the program is threaded? If yes, just changing the locale
may cause some unexpected problems for the caller.

The reason I noticed this bug was that a user of the Scipy library (a
scientific library, for Python, that includes some routines written in Fortran)
was experiencing a mysterious "printf is broken" error when writing a program
using the library. In this case at least, locale was probably changed after
libgfortran was loaded, and it would be unexpected if calling a Fortran routine
changed the locale.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[kargl at gcc dot gnu dot org]

------- Comment #5 from kargl at gcc dot gnu dot org  2008-07-19 20:23 -------
(In reply to comment #4)
> Does setting the locale to POSIX in the above patch fix the issue if the
> program using the runtime for example changes the locale just before calling a
> Fortran routine that does some printing?

The patch sets locale to POSIX for both the gfortran frontend and the
runtime library.  POSIX defines the LC_NUMERIC decimal glyph to be
a period.

> Also, does the locale change affect other parts of the program calling the
> gfortran runtime?

If the other parts are written in Fortran, then no affect.  In a mixed
language environment see below.

> If the program is threaded? If yes, just changing the locale
> may cause some unexpected problems for the caller.

Should have no effect on threaded apps. 

> 
> The reason I noticed this bug was that a user of the Scipy library (a
> scientific library, for Python, that includes some routines written in Fortran)
> was experiencing a mysterious "printf is broken" error when writing a program
> using the library. In this case at least, locale was probably changed after
> libgfortran was loaded, and it would be unexpected if calling a Fortran routine
> changed the locale.
> 

The Fortran standard is silent with respect to locale.  The patch I propose
has gfortran enforce POSIX locale.  It is the programmers responsibility to
handle it in a mixed language environment.  The pseudocode is

   getlocale()              /* Get current locale */
   call_gfortran_routine()  /* gfc enforces POSIX */
   setlocale()              /* Restore the locale from getlocale */



Of course, this is IMHO.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[jvdelisle at gcc dot gnu dot org]

------- Comment #6 from jvdelisle at gcc dot gnu dot org  2008-07-19 23:14 -------
I should clarify something here.  At this time, gfortran does not have its own
dtoa() function.  We are using snprintf (or sprintf) to write to a char buffer.
 This is done to extract the digits.  We then process this string in the buffer
to form the necessary formatted output.

The internal error is a sanity check only to assure that the decimal point came
out in the expected position at buffer[2].  It is useful for debugging as an
indicator that the printf function is working.  This is particularly useful
since on some platforms it has been found to be broken or not behaving as
expected.

That decimal point is not actually ever output by gfortran.  Separate code in
the library emits the "point" or "comma" per the standard according to the
settings of the decimal= specifier.

What I propose to do is modify the sanity check to accept a comma or a point.

In the future, if we ever come up with a better dtoa() or do some other
enhancements in this part of the library, this sanity code is likely to go
away.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[jvdelisle at gcc dot gnu dot org]

------- Comment #7 from jvdelisle at gcc dot gnu dot org  2008-07-19 23:36 -------
The following fixes the reporters test case.  You will notice that even with
the locale set for a comma that the resulting formatted output from the test
case has a '.'

Index: write_float.def
===================================================================
--- write_float.def     (revision 137760)
+++ write_float.def     (working copy)
@@ -123,7 +123,8 @@ output_float (st_parameter_dt *dtp, cons
    */

   /* Check the given string has punctuation in the correct places.  */
-  if (d != 0 && (buffer[2] != '.' || buffer[ndigits + 2] != 'e'))
+  if (d != 0 && ((buffer[2] != '.' && buffer[2] != ',')
+                || buffer[ndigits + 2] != 'e'))
       internal_error (&dtp->common, "printf is broken");

   /* Read the exponent back in.  */


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[dannysmith at users dot sourceforge dot net]

------- Comment #8 from dannysmith at users dot sourceforge dot net  2008-07-20 09:58 -------
(In reply to comment #3)
> This is untested, but you get the idea.
> 
> +  setlocale(LC_ALL, "POSIX");

 mingw's  setlocale doesn't believe in  POSIX ; it does however go to the ISO
C89 church, and has "setlocale(LC_ALL, "C");" in its liturgy.
Danny


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[kargl at gcc dot gnu dot org]

------- Comment #9 from kargl at gcc dot gnu dot org  2008-07-20 17:44 -------
(In reply to comment #8)
> (In reply to comment #3)
> > This is untested, but you get the idea.
> > 
> > +  setlocale(LC_ALL, "POSIX");
> 
>  mingw's  setlocale doesn't believe in  POSIX ; it does however go to the ISO
> C89 church, and has "setlocale(LC_ALL, "C");" in its liturgy.
> Danny
> 

File a bug report with the mingw project, because this list is
concerned with bugs in GCC.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[jvdelisle at gcc dot gnu dot org]

------- Comment #10 from jvdelisle at gcc dot gnu dot org  2008-07-21 04:45 -------
Subject: Bug 36857

Author: jvdelisle
Date: Mon Jul 21 04:44:10 2008
New Revision: 138021

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=138021
Log:
2008-07-20  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR fortran/36857
        * io/write_float.def: Comment out locale dependent code and fix general
        comments.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/write_float.def


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857

[Bug libfortran/36857] Non-English locale breaks gfortran float formatting ("printf is broken")

[jvdelisle at gcc dot gnu dot org]

------- Comment #11 from jvdelisle at gcc dot gnu dot org  2008-07-21 04:56 -------
Fixed on trunk


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |4.4.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36857