locales test has non ascii chars..

locales test has non ascii chars..

[Thomas Zander]

[-- Attachment #1.1: Type: text/plain, Size: 478 bytes --]

Hi,

I'm having trouble getting the localeTest class to compile correctly since 
it has non-ascii chars.  And lots of them :}

I suggest to filter the java file and replace the non ascii values with 
their \u00xx values instead.

See the attached patch and the class to generate this change.  I'm worried 
about the code on lines 410-414 which contains lots of non-standard chars. 
Maybe a typo?

Anyway; can you pick this up?  Or can I commit?
-- 
Thomas Zander

[-- Attachment #1.2: Convert.java --]
[-- Type: text/x-java, Size: 1516 bytes --]

import java.util.*;
import java.io.*;

public class Convert {
    public static void main(String[] args) throws IOException {
        if(args.length == 0) {
            System.out.println("Provide filesnames on commandline");
            return;
        }
        for(int i=0; i < args.length; i++) {
            convert(args[i]);
        }
    }

    private static void convert(String filename) throws IOException {
        File one = new File(filename);
        if(! one.exists())
            return;
        File two = new File(filename +"~");
        int count=1;
        while(two.exists()) {
            two = new File(filename + String.valueOf(count++) +"~");
        }
        Reader reader = new InputStreamReader(new FileInputStream(one), "ISO-8859-1");
        Writer writer = new FileWriter(two);
        try {
            while(true) {
                int character = reader.read();
                if(character == -1)
                    break;
                if(character < 128)
                    writer.write((char) character);
                else {
                    writer.write("\\u");
                    String val = Integer.toString((int) ((char) character), 16);
                    for(int i=4; i > val.length(); i--)
                        writer.write("0");
                    writer.write(val);
                }
            }
        } catch(IOException e) {
            two.delete();
        } finally {
            reader.close();
            writer.close();
        }
    }
}

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: locales_charset.diff --]
[-- Type: text/x-diff; charset="us-ascii"; name="locales_charset.diff", Size: 7492 bytes --]

Index: gnu/testlet/locales/LocaleTest.java
===================================================================
RCS file: /cvs/mauve/mauve/gnu/testlet/locales/LocaleTest.java,v
retrieving revision 1.1
diff -u -3 -p -r1.1 LocaleTest.java
--- gnu/testlet/locales/LocaleTest.java	26 Dec 2004 17:02:16 -0000	1.1
+++ gnu/testlet/locales/LocaleTest.java	30 Dec 2004 12:04:30 -0000
@@ -271,7 +271,7 @@ public class LocaleTest
 		new ExpectedValues("de", "DE", "", "de_DE", "deu", "DEU",
 			           "Deutsch", "Deutschland", "", "Deutsch (Deutschland)",
 				   "EUR", 2, "\u20ac"),
-		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. März 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
+		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. M\u00e4rz 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100,123", "10.000.000,123"),
 		new ExpectedNumberValues("1.000,00 \u20ac", "1.000.000,00 \u20ac", "100,00 \u20ac", "100,12 \u20ac", "10.000.000,12 \u20ac"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100", "10.000.000"),
@@ -280,9 +280,9 @@ public class LocaleTest
     // Locale: Belgium
     checkLocale(h, new Locale("fr", "BE"),
 		new ExpectedValues("fr", "BE", "", "fr_BE", "fra", "BEL",
-				   "Französisch", "Belgien", "", "Französisch (Belgien)",
+				   "Franz\u00f6sisch", "Belgien", "", "Franz\u00f6sisch (Belgien)",
 				   "EUR", 2, "\u20ac"),
-		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. März 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
+		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. M\u00e4rz 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100,123", "10.000.000,123"),
 		new ExpectedNumberValues("1.000,00 \u20ac", "1.000.000,00 \u20ac", "100,00 \u20ac", "100,12 \u20ac", "10.000.000,12 \u20ac"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100", "10.000.000"),
@@ -295,7 +295,7 @@ public class LocaleTest
 		new ExpectedValues("el", "GR", "", "el_GR", "ell", "GRC",
 				   "Griechisch", "Griechenland", "", "Griechisch (Griechenland)",
 				   "EUR", 2, "\u20ac"),
-		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. März 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
+		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. M\u00e4rz 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100,123", "10.000.000,123"),
 		new ExpectedNumberValues("1.000,00 \u20ac", "1.000.000,00 \u20ac", "100,00 \u20ac", "100,12 \u20ac", "10.000.000,12 \u20ac"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100", "10.000.000"),
@@ -307,7 +307,7 @@ public class LocaleTest
 		new ExpectedValues("en", "IE", "", "en_IE", "eng", "IRL",
 				   "Englisch", "Irland", "", "Englisch (Irland)",
 				   "EUR", 2, "\u20ac"),
-		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. März 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
+		new ExpectedDateValues("18.03.1974", "18.03.74", "18.03.1974", "18. M\u00e4rz 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100,123", "10.000.000,123"),
 		new ExpectedNumberValues("1.000,00 \u20ac", "1.000.000,00 \u20ac", "100,00 \u20ac", "100,12 \u20ac", "10.000.000,12 \u20ac"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100", "10.000.000"),
@@ -316,7 +316,7 @@ public class LocaleTest
     // Locale: France
     checkLocale(h, new Locale("fr", "FR"),
 		new ExpectedValues("fr", "FR", "", "fr_FR", "fra", "FRA",
-				   "Französisch", "Frankreich", "", "Französisch (Frankreich)",
+				   "Franz\u00f6sisch", "Frankreich", "", "Franz\u00f6sisch (Frankreich)",
 				   "EUR", 2, "\u20ac"),
 		null,
 		null,
@@ -360,7 +360,7 @@ public class LocaleTest
     // Locale: The Netherlands
     checkLocale(h, new Locale("nl", "NL"),
 		new ExpectedValues("nl", "NL", "", "nl_NL", "nld", "NLD",
-				   "Niederländisch", "Niederlande", "", "Niederländisch (Niederlande)",
+				   "Niederl\u00e4ndisch", "Niederlande", "", "Niederl\u00e4ndisch (Niederlande)",
 				   "EUR", 2, "\u20ac"),
 		new ExpectedDateValues("18-mrt-1974", "18-3-74", "18-mrt-1974", "18 maart 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100,123", "10.000.000,123"),
@@ -371,7 +371,7 @@ public class LocaleTest
     // Locale: Luxemborg
     checkLocale(h, new Locale("fr", "LU"),
 		new ExpectedValues("fr", "LU", "", "fr_LU", "fra", "LUX",
-				   "Französisch", "Luxemburg", "", "Französisch (Luxemburg)",
+				   "Franz\u00f6sisch", "Luxemburg", "", "Franz\u00f6sisch (Luxemburg)",
 				   "EUR", 2, "\u20ac"),
 		null,
 		null,
@@ -382,7 +382,7 @@ public class LocaleTest
     // Locale: United Kingdom
     checkLocale(h, Locale.UK,
 		new ExpectedValues("en", "GB", "", "en_GB", "eng", "GBR",
-				   "Englisch", "Vereinigtes Königreich", "", "Englisch (Vereinigtes Königreich)",
+				   "Englisch", "Vereinigtes K\u00f6nigreich", "", "Englisch (Vereinigtes K\u00f6nigreich)",
 				   "GBP", 2, "GBP"),
 		new ExpectedDateValues("18-Mar-1974", "18/03/74", "18-Mar-1974", "18 March 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1,000", "1,000,000", "100", "100.123", "10,000,000.123"),
@@ -407,15 +407,15 @@ public class LocaleTest
 				   "Finnisch", "Finnland", "", "Finnisch (Finnland)",
 				   "EUR", 2, "EUR"),
 		new ExpectedDateValues("18.3.1974", "18.3.1974", "18.3.1974", "18. maaliskuuta 1974", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
-		new ExpectedNumberValues("1 000", "1 000 000", "100", "100,123", "10 000 000,123"),
-		new ExpectedNumberValues("1 000,00 \u20ac", "1 000 000,00 \u20ac", "100,00 \u20ac", "100,12 \u20ac", "10 000 000,12 \u20ac"),
-		new ExpectedNumberValues("1 000", "1 000 000", "100", "100", "10 000 000"),
-		new ExpectedNumberValues("1 000", "1 000 000", "100", "100,123", "10 000 000,123"),
-		new ExpectedNumberValues("100 000%", "100 000 000%", "10 000%", "10 012%", "1 000 000 012%"));
+		new ExpectedNumberValues("1\u00a0000", "1\u00a0000\u00a0000", "100", "100,123", "10\u00a0000\u00a0000,123"),
+		new ExpectedNumberValues("1\u00a0000,00 \u20ac", "1\u00a0000\u00a0000,00 \u20ac", "100,00 \u20ac", "100,12 \u20ac", "10\u00a0000\u00a0000,12 \u20ac"),
+		new ExpectedNumberValues("1\u00a0000", "1\u00a0000\u00a0000", "100", "100", "10\u00a0000\u00a0000"),
+		new ExpectedNumberValues("1\u00a0000", "1\u00a0000\u00a0000", "100", "100,123", "10\u00a0000\u00a0000,123"),
+		new ExpectedNumberValues("100\u00a0000%", "100\u00a0000\u00a0000%", "10\u00a0000%", "10\u00a0012%", "1\u00a0000\u00a0000\u00a0012%"));
     // Locale: Turkey
     checkLocale(h, new Locale("tr", "TR"),
 		new ExpectedValues("tr", "TR", "", "tr_TR", "tur", "TUR",
-				   "Türkisch", "Türkei", "", "Türkisch (Türkei)",
+				   "T\u00fcrkisch", "T\u00fcrkei", "", "T\u00fcrkisch (T\u00fcrkei)",
 				   "TRL", 2, "TRL"),
 		new ExpectedDateValues("18.Mar.1974", "18.03.1974", "18.Mar.1974", "18 Mart 1974 Pazartesi", "17:20:30", "17:20", "17:20:30", "17:20:30 CET"),
 		new ExpectedNumberValues("1.000", "1.000.000", "100", "100,123", "10.000.000,123"),

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: locales test has non ascii chars..

[Michael Koch]

Am Donnerstag, 30. Dezember 2004 13:07 schrieb Thomas Zander:
> Hi,
>
> I'm having trouble getting the localeTest class to compile
> correctly since it has non-ascii chars.  And lots of them :}

Well, mostly german umlauts. What do you expect from me ? ;-)

> I suggest to filter the java file and replace the non ascii values
> with their \u00xx values instead.
>
> See the attached patch and the class to generate this change.  I'm
> worried about the code on lines 410-414 which contains lots of
> non-standard chars. Maybe a typo?

No, this is no typo, just a workaround.

> Anyway; can you pick this up?  Or can I commit?

Please commit. Thanks for testing.

I think we need to fix some of the testcases. JDK seem to return wrong 
values. Or at least different ones than CLDR expects.


Michael
-- 
Homepage: http://www.worldforge.org/

Re: locales test has non ascii chars..

[Thomas Zander]

[-- Attachment #1.1: Type: text/plain, Size: 387 bytes --]

On Thursday 30 December 2004 13:18, you wrote:
> Please commit. Thanks for testing.
Done.

> I think we need to fix some of the testcases. JDK seem to return wrong
> values. Or at least different ones than CLDR expects.

Attached is a nice output from the (fixed) mauve-report sources from David 
Gilbert.
This is run on the SunsJDK 1.5

Hope that helps.
-- 
Thomas Zander

[-- Attachment #1.2: LocaleTest.html.gz --]
[-- Type: application/x-gzip, Size: 2827 bytes --]

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: locales test has non ascii chars..

[Michael Koch]

Am Donnerstag, 30. Dezember 2004 14:23 schrieb Thomas Zander:
> On Thursday 30 December 2004 13:18, you wrote:
> > Please commit. Thanks for testing.
>
> Done.
>
> > I think we need to fix some of the testcases. JDK seem to return
> > wrong values. Or at least different ones than CLDR expects.
>
> Attached is a nice output from the (fixed) mauve-report sources
> from David Gilbert.
> This is run on the SunsJDK 1.5

Can you write a patch to LocaleTest.java that would fix all FAILs for 
1.5 and send it to me personally ? I know some stuff is wrong in JDK.
I just wanna see what is wrong in our testcase.


Michael
-- 
Homepage: http://www.worldforge.org/

Re: locales test has non ascii chars..

[Thomas Zander]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

On Thursday 30 December 2004 14:53, Michael Koch wrote:
> Am Donnerstag, 30. Dezember 2004 14:23 schrieb Thomas Zander:
> > > I think we need to fix some of the testcases. JDK seem to return
> > > wrong values. Or at least different ones than CLDR expects.
> >
> > Attached is a nice output from the (fixed) mauve-report sources
> > from David Gilbert.
> > This is run on the SunsJDK 1.5
>
> Can you write a patch to LocaleTest.java that would fix all FAILs for
> 1.5 and send it to me personally ? I know some stuff is wrong in JDK.
> I just wanna see what is wrong in our testcase.

Hmm, ok. Maybe I'll get to that next week.
The html file I send with my last email allows anyone to do it, though.  So 
you don't have to wait for me.

-- 
Thomas Zander

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: locales test has non ascii chars..

[Michael Koch]

Am Donnerstag, 30. Dezember 2004 15:17 schrieb Thomas Zander:
> On Thursday 30 December 2004 14:53, Michael Koch wrote:
> > Am Donnerstag, 30. Dezember 2004 14:23 schrieb Thomas Zander:
> > > > I think we need to fix some of the testcases. JDK seem to
> > > > return wrong values. Or at least different ones than CLDR
> > > > expects.
> > >
> > > Attached is a nice output from the (fixed) mauve-report sources
> > > from David Gilbert.
> > > This is run on the SunsJDK 1.5
> >
> > Can you write a patch to LocaleTest.java that would fix all FAILs
> > for 1.5 and send it to me personally ? I know some stuff is wrong
> > in JDK. I just wanna see what is wrong in our testcase.
>
> Hmm, ok. Maybe I'll get to that next week.
> The html file I send with my last email allows anyone to do it,
> though.  So you don't have to wait for me.

I'm not sure but I think the table in the html file is wrong. AFAIK 
the expected and got columns are exchanged.


Michael
-- 
Homepage: http://www.worldforge.org/

Re: locales test has non ascii chars..

[Thomas Zander]

[-- Attachment #1.1: Type: text/plain, Size: 355 bytes --]

On Thursday 30 December 2004 15:37, Michael Koch wrote:
> I'm not sure but I think the table in the html file is wrong. AFAIK
> the expected and got columns are exchanged.

You are right; the runner indeed 'printed' this incorrectly; bug was due to 
a classic example of putting code in the wrong place ;)

See new attachment.
-- 
Thomas Zander

[-- Attachment #1.2: LocaleTest.html.gz --]
[-- Type: application/x-gzip, Size: 2841 bytes --]

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]