Re: java/2313: Java SimpleDateFormat crash with non US locales (french...)

Re: java/2313: Java SimpleDateFormat crash with non US locales (french...)

[tromey]

Synopsis: Java SimpleDateFormat crash with non US locales (french...)

Responsible-Changed-From-To: unassigned->tromey
Responsible-Changed-By: tromey
Responsible-Changed-When: Wed Dec 26 15:56:55 2001
Responsible-Changed-Why:
    I'm handling this.
State-Changed-From-To: open->closed
State-Changed-By: tromey
State-Changed-When: Wed Dec 26 15:56:55 2001
State-Changed-Why:
    I was unable to reproduce this with the current trunk cvs gcc.
    So, I assume that the bug was fixed at some point.
    The fix will appear in gcc 3.1.
    If you try the trunk and still see the bug, feel free to send
    email and I'll reopen the PR.
    Thanks.

http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=2313

Re: java/2313: Java SimpleDateFormat crash with non US locales (french...)

[Tom Tromey]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1983 bytes --]

The following reply was made to PR java/2313; it has been noted by GNATS.

From: Tom Tromey <tromey@redhat.com>
To: Bryce McKinlay <bryce@albatross.co.nz>
Cc: diam@ensta.fr, gcc-gnats@gcc.gnu.org, java-patches@gcc.gnu.org
Subject: Re: java/2313: Java SimpleDateFormat crash with non US locales   (french...)
Date: 19 Mar 2001 09:42:35 -0700

 >>>>> "Bryce" == Bryce McKinlay <bryce@albatross.co.nz> writes:
 
 Bryce>     System.out.println ("Liberté, égalité, fraternité !");
 
 Bryce> works fine in the default mode, but with "--encoding=UTF-8" it
 Bryce> produces incorrect output.
 
 That's because the input file isn't actually in UTF-8, but it also
 doesn't contain an incorrect (by our rules -- see below) UTF-8
 sequence that would let us see it as erroneous.
 
 The `é' is 0xe9.  This is a valid start byte for a 2-byte UTF-8
 sequence.  That is why the following character is also removed.
 
 We ought to be noticing that the subsequent bytes in the sequence are
 invalid.  That is what Unicode specifies, and there probably isn't a
 good reason to allow incorrectly encoded characters.  However the code
 wasn't originally written this way and I never updated it to do this.
 I'll submit a PR.
 
 Bryce> Unfortunately, I know very little about character
 Bryce> encoding. Maybe Tom can suggest a fix or workaround. Perhaps
 Bryce> its possible to do something to convert the file to a UTF-8
 Bryce> encoding before trying to compile it?
 
 One fix would be to tell gcj the real encoding of the file:
 
     gcj --encoding=8859_1 ...
 
 This works for me.  However, note that the encoding names are
 system-dependent :-(.  Ideally we'd have a table of aliases mapping
 the Java-specified names to the system-dependent ones.
 
 Another fix would be to use the `iconv' or `recode' programs to
 convert the file into UTF-8 before compiling.  This is a pain to do,
 but might be the only recourse on systems with a losing (or no)
 iconv() implementation.
 
 Tom

Re: java/2313: Java SimpleDateFormat crash with non US locales (french...)

[Bryce McKinlay]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

The following reply was made to PR java/2313; it has been noted by GNATS.

From: Bryce McKinlay <bryce@albatross.co.nz>
To: diam@ensta.fr
Cc: gcc-gnats@gcc.gnu.org, java-patches@gcc.gnu.org
Subject: Re: java/2313: Java SimpleDateFormat crash with non US locales 
 (french...)
Date: Mon, 19 Mar 2001 10:30:41 +1200

 diam@ensta.fr wrote:
 
 > >Fix:
 > Thanks to Bryce McKinlay, I added the 2 following files
 > LocaleData_fr.java and LocaleData_fr_FR.java near
 > the existing files LocaleData_en.java and LocaleData_en_US.java
 > and added these two file in:
 >       src/cvs/gcc/libjava/Makefile.am
 >       src/cvs/gcc/libjava/Makefile.in
 
 Cool, thanks. I'll look at checking those in shortly.
 
 > But these LocaleData_fr.java files doen't accept french
 > accents (this is another problem, gcc should ignore accents
 > in source file)
 
 This seems to be a side-effect of compiling libjava with "--encoding=UTF-8". This simple example:
 
 public class Hello
 {
   public static void main ( String []arguments)
   {
     System.out.println ("Liberté, égalité, fraternité !");
   }
 }
 
 works fine in the default mode, but with "--encoding=UTF-8" it produces incorrect output.
 
 $ gcj Hello.java --main=Hello
 $ ./a.out
 Liberté, égalité, fraternité !
 $ gcj --encoding=UTF-8 Hello.java --main=Hello
 $ ./a.out
 Libert          lit     fraternit
 
 Unfortunately, I know very little about character encoding. Maybe Tom can suggest a fix or workaround. Perhaps its possible to do something to convert the file to a UTF-8 encoding before trying to compile it?
 
 regards
 
   [ bryce ]
 
 

java/2313: Java SimpleDateFormat crash with non US locales (french...)

[diam]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 10898 bytes --]

>Number:         2313
>Category:       java
>Synopsis:       Java SimpleDateFormat crash with non US locales (french...)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Mar 18 02:26:01 PST 2001
>Closed-Date:
>Last-Modified:
>Originator:     Maurice Diamantini
>Release:        gcc version 3.0 20010305 (prerelease)
>Organization:
>Environment:
Macintosh linuxppc 1999 - kernel 2.2.17 with french locale
and intel x86 linux readhat 6.x with french locale
>Description:
The following code compile but crash at runtime on french systems:
    // file Test.java
    import java.util.*;
    import java.text.SimpleDateFormat;
    public class Test  {    
        public static void main(String[] args) {
            // this work
            Date date = new Date();
            System.out.println ("getTime=" + date.getTime());
            // this doesn't work
            System.out.println ("date=" + date.toString());
            return;
        }
    }

gcj -O0 -g   --main=Test -o test Test.java

------------------------------------------------------------
It work (without recompilation) if I enter before run (tcsh):

    setenv LC_ALL en_US.iso-8859-1

    > locale
    LANG=POSIX
    LC_CTYPE="en_US.iso-8859-1"
    LC_NUMERIC="en_US.iso-8859-1"
    LC_TIME="en_US.iso-8859-1"
    LC_COLLATE="en_US.iso-8859-1"
    LC_MONETARY="en_US.iso-8859-1"
    LC_MESSAGES="en_US.iso-8859-1"
    LC_ALL=en_US.iso-8859-1

     ./test
    getTime=984775144183
    date=Fri Mar 16 21:39:04 GMT-01:00 2001
    Début à : 2001-03-16 21:39:04.0183


------------------------------------------------------------
But this code crashes at runtime whith my normal french 
configuration :

    setenv LC_ALL fr_FR.iso-8859-1

    > locale
    LANG=POSIX
    LC_CTYPE="fr_FR.iso-8859-1"
    LC_NUMERIC="fr_FR.iso-8859-1"
    LC_TIME="fr_FR.iso-8859-1"
    LC_COLLATE="fr_FR.iso-8859-1"
    LC_MONETARY="fr_FR.iso-8859-1"
    LC_MESSAGES="fr_FR.iso-8859-1"
    LC_ALL=fr_FR.iso-8859-1

./test 
getTime=984775319636
(I change "/usr/local/pack/gcc-3.0-cvs-20010306/powerpc-linux"
to "//" for readability)

Exception in thread "main" java.lang.NullPointerException
   at 0x0fd0dfb0: java.lang.Throwable.Throwable() (//lib/libgcj.so.1)
   at 0x0fcf9780: java.lang.Exception.Exception() (//lib/libgcj.so.1)
   at 0x0fcfffb8: java.lang.RuntimeException.RuntimeException() 
        (//lib/libgcj.so.1)
   at 0x0fcff408: java.lang.NullPointerException.NullPointerException() 
        (//lib/libgcj.so.1)
   at 0x0fcb6824: _Jv_ThrowNullPointerException (//lib/libgcj.so.1)
   at 0x0fdf364c: java.text.DateFormatSymbols.formatsForKey
        (java.util.ResourceBundle, java.lang.String) (//lib/libgcj.so.1)
   at 0x0fdf335c: java.text.DateFormatSymbols.DateFormatSymbols(java.util.Locale) 
        (//lib/libgcj.so.1)
   at 0x0fd8d388: java.util.TimeZone.getDisplayName(boolean, int, java.util.Locale) 
        (//lib/libgcj.so.1)
   at 0x0fd8d224: java.util.TimeZone.getDisplayName(boolean, int) 
        (//lib/libgcj.so.1)
   at 0x0fe03d8c: java.text.SimpleDateFormat.format(java.util.Date, 
        java.lang.StringBuffer, java.text.FieldPosition) (//lib/libgcj.so.1)
   at 0x0fdf1db8: java.text.DateFormat.format(java.util.Date) (//lib/libgcj.so.1)
   at 0x0fd7582c: java.util.Date.toString() (//lib/libgcj.so.1)
   at 0x10000f70: Test.main(java.lang.String[]) 
        (/home/diam/projets/jorlab/benchs_java/testbug/Test.java:15)
   at 0x0fcd9374: gnu.gcj.runtime.FirstThread.run() (//lib/libgcj.so.1)
   at 0x0fce7b08: java.lang.Thread.run_(java.lang.Object) (//lib/libgcj.so.1)
   at 0x0fe98e2c: _Jv_ThreadSetPriority(_Jv_Thread_t, int) (//lib/libgcj.so.1)
   at 0x0fa1f910: GC_start_routine (//lib/libgcjgc.so.1)
   at 0x0f9e3adc: pthread_detach (/lib/libpthread.so.0)
   at 0x0f91d9fc: __clone (/lib/libc.so.6)
>How-To-Repeat:
gcj -O0 -g   --main=Test -o test Test.java
Then run (without recompilation)
either with:
    setenv LC_ALL fr_FR.iso-8859-1
or with:
    setenv LC_ALL en_US.iso-8859-1
>Fix:
Thanks to Bryce McKinlay, I added the 2 following files
LocaleData_fr.java and LocaleData_fr_FR.java near
the existing files LocaleData_en.java and LocaleData_en_US.java
and added these two file in:
      src/cvs/gcc/libjava/Makefile.am
      src/cvs/gcc/libjava/Makefile.in

But these LocaleData_fr.java files doen't accept french
accents (this is another problem, gcc should ignore accents
in source file) 

###################################################
// Generic french locale data for java.text.

/* Copyright (C) 1999, 2001  Free Software Foundation

   This file is part of libgcj.

This software is copyrighted work licensed under the terms of the
Libgcj License.  Please consult the file "LIBGCJ_LICENSE" for
details.  */

package gnu.gcj.text;

import java.util.ListResourceBundle;

/**
 * @author Tom Tromey <tromey@cygnus.com> 
 * @author Maurice Diamantini <diam@ensta.fr>  (Kill this line ?)
 * @date Mars, 11, 2001
 */

public final class LocaleData_fr extends ListResourceBundle
{
  // These are for DateFormatSymbols.
  static final String[] ampmsDefault = {"AM", "PM" };
  static final String[] erasDefault = {"BC", "AD" };
  static final String localPatternCharsDefault = "GyMdkHmsSEDFwWahKz";
  static final String[] monthsDefault = {
    "Janvier", "Fevrier", "Mars", "Avril", "Mai", "Juin",
    "Juillet", "Aout", "Septembre", "Octobre", "Novembre", "Decembre", ""
  };
  static final String[] shortMonthsDefault = {
    "Jan", "Fev", "Mar", "Avr", "Mai", "Jun",
    "Jul", "Aou", "Sep", "Oct", "Nov", "Dec", ""
  };
  static final String[] shortWeekdaysDefault = {
    "", "Dim", "Lun", "Mar", "Mer", "Jeu", "Ven", "Sam"
  };
  static final String[] weekdaysDefault = {
    "", "Dimanche", "Lundi", "Mardi",
    "Mercredi", "Jeudi", "Vendredi", "Samedi"
  };

  private static final Object[][] contents =
  {
    // These are for DecimalFormatSymbols.
    { "decimalSeparator", "," },
    { "digit", "#" },
    { "exponential", "E" },
    { "groupingSeparator", " " },
    { "infinity", "\u221e" },
    { "minusSign", "-" },
    { "monetarySeparator", " " },
    { "NaN", "\ufffd" },
    { "patternSeparator", ";" },        // A VERIFIER !!
    { "percent", "%" },
    { "perMill", "\u2030" },
    { "zeroDigit", "0" },

    // These are for NumberFormat.
    { "numberFormat", "#,##0.###" },
    { "percentFormat", "#,##0%" }, 

    // These are for DateFormatSymbols.
    { "ampm", ampmsDefault },
    { "eras", erasDefault },
    { "datePatternChars", localPatternCharsDefault },
    { "months", monthsDefault },
    { "shortMonths", shortMonthsDefault },
    { "shortWeekdays", shortWeekdaysDefault },
    { "weekdays", weekdaysDefault },

    // These are for DateFormat.
    { "shortDateFormat", "dd/MM/yy" },
    { "mediumDateFormat", "d-MMM-yy" },
    { "longDateFormat", "d MMMM yyyy" },
    { "fullDateFormat", "EEEE d MMMM yyyy" },
    { "shortTimeFormat", "H:mm" },
    { "mediumTimeFormat", "H:mm:ss" },
    { "longTimeFormat", "H:mm:ss z" },
    { "fullTimeFormat", "H:mm:ss,S 'heures' z" },

    // For RuleBasedCollator.
    // FIXME: this is nowhere near complete.
    // In particular we must mark accents as ignorable,
    // and probably other things as well.
    // 
    // DOESN'T WORK WITH ACCENTS !!  
    // as in: {"collatorRule", "< 0 < 1 .... < e, é, è, ê, ë, E, É, È, Ê, Ë < ... }
    // 
    { "collatorRule", "< 0 < 1 < 2 < 3 < 4 < 5 < 6 < 7 < 8 < 9 < a,A < b,B < c,C < d,D < e,E < f,F < g,G < h,H < i,I, < j,J < k,K < l,L < m,M < n,N < o,O < p,P < q,Q < r,R < s,S < t,T < u,U < v,V < w,W < x,X < y,Y < z,Z" }
    
  };

  protected Object[][] getContents ()
    {
      return contents;
    }
}

#################################################

// US English locale data for java.text.

/* Copyright (C) 1999-2001  Free Software Foundation

   This file is part of libgcj.

This software is copyrighted work licensed under the terms of the
Libgcj License.  Please consult the file "LIBGCJ_LICENSE" for
details.  */

package gnu.gcj.text;

import java.util.ListResourceBundle;

/**
 * @author Maurice Diamantini <diam@ensta.fr>
 * @date Mars, 11, 2001
 */

public final class LocaleData_fr_FR extends ListResourceBundle
{
  // These are for DateFormatSymbols.
  static String[][] zoneStringsDefault = {
    { "GMT", "Greenwich Mean Time", "GMT",
      /**/   "Greenwich Mean Time", "GMT", "GMT" },
    { "PST", "Pacific Standard Time", "PST",
      /**/   "Pacific Daylight Time", "PDT", "San Francisco" },
    { "MST", "Mountain Standard Time", "MST",
      /**/   "Mountain Daylight Time", "MDT", "Denver" },
    { "PNT", "Mountain Standard Time", "MST",
      /**/   "Mountain Standard Time", "MST", "Phoenix" },
    { "CST", "Central Standard Time", "CST",
      /**/   "Central Daylight Time", "CDT", "Chicago" },
    { "EST", "Eastern Standard Time", "EST",
      /**/   "Eastern Daylight Time", "EDT", "Boston" },
    { "IET", "Eastern Standard Time", "EST",
      /**/   "Eastern Standard Time", "EST", "Indianapolis" },
    { "PRT", "Atlantic Standard Time", "AST",
      /**/   "Atlantic Daylight Time", "ADT", "Halifax" },
    { "CNT", "Newfoundland Standard Time", "NST",
      /**/   "Newfoundland Daylight Time", "NDT", "St. Johns" },
    { "ECT", "Central European Standard Time", "CET",
      /**/   "Central European Daylight Time", "CEST", "Paris" },
    { "CTT", "China Standard Time", "CST",
      /**/   "China Standard Time", "CST", "Shanghai" },
    { "JST", "Japan Standard Time", "JST",
      /**/   "Japan Standard Time", "JST", "Tokyo" },
    { "HST", "Hawaii Standard Time", "HST",
      /**/   "Hawaii Standard Time", "HST", "Honolulu" },
    { "AST", "Alaska Standard Time", "AKST",
      /**/   "Alaska Daylight Time", "AKDT", "Anchorage" }
  };

  private static final Object[][] contents =
  {
    // These are for DecimalFormatSymbols.
    { "currency", "FF" },
    { "intlCurrencySymbol", "FF" },  // Should be soon "euro"s (end 2001) ?

    // These are for NumberFormat.
    // { "currencyFormat", "$#,##0.00;($#,##0.00)" },  // from en_US
    { "currencyFormat", "#,##0.00FF" },  // for fr_FR, but not shure it's complete!)

    // These are for DateFormatSymbols.
    { "zoneStrings", zoneStringsDefault },

    // These are for DateFormat.
    { "shortDateFormat", "dd/MM/yy" },
    { "mediumDateFormat", "d-MMM-yy" },
    { "longDateFormat", "d MMMM yyyy" },
    { "fullDateFormat", "EEEE d MMMM yyyy" },
    { "shortTimeFormat", "H:mm" },
    { "mediumTimeFormat", "H:mm:ss" },
    { "longTimeFormat", "H:mm:ss z" },
    { "fullTimeFormat", "H:mm:ss,S 'heures' z" },
  };

  protected Object[][] getContents ()
    {
      return contents;
    }
}

###################################################



>Release-Note:
>Audit-Trail:
>Unformatted: