Analysis of Mauve failures - The final chapter

[Mark Wielaard <mark@klomp.org>]

Hi,

Here is the last overview of gcj Mauve issues.
I promise that this will be the last big list and to do
some real work (tm) to solve some of the issues myself and not
just hope that others will take the lists and solve it for me.
Although I did trick some people into submitting fixes, so part
of the plan actually worked :)

Today we will look at the mauve-libgcj file which contains tests
that we don't even try to run and the libjava-mauve/xfails file
which contains tests we do run and which we expect to fail.

muave-libgcj currently contains the following:
> # These 2 are tests that fail with JDBC2.0 but the tags don't seem to
> # have the right effect.
> !java.sql.Connection.TestJdbc10
> !java.sql.DatabaseMetaData.TestJdbc10
Should investigate why the mauve configury does not work properly, but
they are harmless since they fail because we implement JDBC2.0 things
(extra methods in some abstract classes) that make these tests fail.

> # Cannot be compiled
> !java.text.ACIAttribute
We don't implement the innerclass
java.text.AttributedCharacterIterator.Attribute.

> # The following tests seem to (sometimes) hang or crash the testsuite
> !java.io.ObjectInputOutput
The gnu.testlet.java.io.ObjectInputOutput.OutputTest testlet crashes on
the Test$HairyGraph. This seems an infinite recursion since the
backtrace in gdb gives hundreds of:

#199 0x402823e1 in
java.io.ObjectOutputStream.writeObject(java.lang.Object) (
    this=0x81b9f00, obj=0x81c2b58)
    at ../../../gcc/libjava/java/io/ObjectOutputStream.java:352
#200 0x40283e7c in
java.io.ObjectOutputStream.writeFields(java.lang.Object,
java.io.ObjectStreamField[], boolean) (this=0x81b9f00, obj=0x81c2b70, 
    fields=0x81a1fa0)
    at ../../../gcc/libjava/java/io/ObjectOutputStream.java:1173

> !java.lang.reflect.Array.newInstance
Ugh, not fun. Running by hand also hangs, but turning on the -debug or
-verbose flag makes it run... When not giving any flags it only prints
Needed to allocate blacklisted block at 0x824b000
The test actually tries to force a OutOfMemoryError exception which
might explain this. But the Object.clone() test also seems to do this
and that one just works.

> !java.util.ResourceBundle.getBundle
Don't know why that was in there. It seems to run fine, although it
gives some failures which should be investigated:
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: with locale of
Canada (number 4)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: with locale of
Canada (number 5)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: with locale of
France (number 4)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 2)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 5)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 6)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 7)
7 of 23 tests failed

> !java.util.zip.GZIPInputStream.basic
Also seems to just work. Both tests seems to succeed.

O Duh. Enabling one or both of the last two tests seem to crash or hang
the testsuite again when run in full. Curious. Keep disabled for now.

> !java.net.DatagramSocket.DatagramSocketTest2
Our DatagramSocket.receive() blocks till a packet is received even when
the buffer of the DatagramPacket is zero. Which seems correct given the
spec. Wrong test?


The libjava.mauve/xfail file currently contains the following:

> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error: test_shortbyteValue failed - 5 (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_shortbyteValue failed - 5 (number 1)
Yeah! These are now XPASS.

> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Four Byte Range Error (0) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Four Byte Range Error (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Five Bytes (0) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Five Bytes (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Six Bytes (0) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Six Bytes (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Orphan Continuation (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Orphan Continuation (2) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Four Byte Range Error (2) (number 1)
The test says:Note that JDK 1.1 and JDK 1.2 don't currently pass these tests;
there are known problems in their UTF-8 encoding support at this time.
This probably also true for libgcj.

> FAIL: gnu.testlet.java.io.ObjectStreamClass.Test: getSerialVersionUID (number 7)
Now passes! 

> FAIL: gnu.testlet.java.text.DateFormatSymbols.Test: patterns (number 2)
> FAIL: gnu.testlet.java.text.SimpleDateFormat.Test: equals() (number 1)
> FAIL: gnu.testlet.java.text.SimpleDateFormat.Test: parse() strict (number 1)
> FAIL: gnu.testlet.java.text.SimpleDateFormat.getAndSet2DigitYearStart: get2DigitYearStart() initial (number 1)
Seem to be known issues with our java.text support.

> FAIL: gnu.testlet.java.net.URLConnection.URLConnectionTest: Error in test_Basics  - 2  should not have raised  Throwable here  (number 1)
> FAIL: gnu.testlet.java.net.URL.URLTest: openStream (number 1)
> FAIL: gnu.testlet.java.net.URL.URLTest: sameFile (number 2)
> FAIL: gnu.testlet.java.net.URL.URLTest:  Error in test_toString  - 5  exception should not be thrown here (number 1)
> FAIL: gnu.testlet.java.net.URL.URLTest: new URL(protocol, host, file) (number 26)
> FAIL: gnu.testlet.java.net.URL.URLTest: new URL(protocol, host, file) (number 54)
We need to merge URL and URLConnection with Classpath and check again.

> FAIL: gnu.testlet.java.net.ServerSocket.ServerSocketTest: Error : test_params failed - 5getInetAddress did not return proper values  (number 1)
> FAIL: gnu.testlet.java.net.Socket.SocketTest: Error : test_BasicServer failed - 11 exception was thrown :Illegal seek (number 1)
> FAIL: gnu.testlet.java.net.MulticastSocket.MulticastSocketTest: joinGroup() twice. (number 1)
No time to investigate. Keep them in for now.

Actions:
Character.unicode should probably also be added to the mauve-libgcj ignore list
since it generates a lot of wrong failures.
We seem to agree that some tests are bogus or wrong in Mauve. I will make
patches for those tests and submit them to the mauve mailinglist.
I will try to add all tests that FAIL now and for which we know we will not
solve the issue for 3.1 to the xfail list (and submit bug reports).
Most issues can probably not be solved for 3.1 since that release is in about
10 days, but we can try. The goal is to have zero FAILs (but a couple of XFAILs)
for 3.1.

Cheers,

Mark