Analysis of Mauve failures - The final chapter

Analysis of Mauve failures - The final chapter

[Mark Wielaard]

Hi,

Here is the last overview of gcj Mauve issues.
I promise that this will be the last big list and to do
some real work (tm) to solve some of the issues myself and not
just hope that others will take the lists and solve it for me.
Although I did trick some people into submitting fixes, so part
of the plan actually worked :)

Today we will look at the mauve-libgcj file which contains tests
that we don't even try to run and the libjava-mauve/xfails file
which contains tests we do run and which we expect to fail.

muave-libgcj currently contains the following:
> # These 2 are tests that fail with JDBC2.0 but the tags don't seem to
> # have the right effect.
> !java.sql.Connection.TestJdbc10
> !java.sql.DatabaseMetaData.TestJdbc10
Should investigate why the mauve configury does not work properly, but
they are harmless since they fail because we implement JDBC2.0 things
(extra methods in some abstract classes) that make these tests fail.

> # Cannot be compiled
> !java.text.ACIAttribute
We don't implement the innerclass
java.text.AttributedCharacterIterator.Attribute.

> # The following tests seem to (sometimes) hang or crash the testsuite
> !java.io.ObjectInputOutput
The gnu.testlet.java.io.ObjectInputOutput.OutputTest testlet crashes on
the Test$HairyGraph. This seems an infinite recursion since the
backtrace in gdb gives hundreds of:

#199 0x402823e1 in
java.io.ObjectOutputStream.writeObject(java.lang.Object) (
    this=0x81b9f00, obj=0x81c2b58)
    at ../../../gcc/libjava/java/io/ObjectOutputStream.java:352
#200 0x40283e7c in
java.io.ObjectOutputStream.writeFields(java.lang.Object,
java.io.ObjectStreamField[], boolean) (this=0x81b9f00, obj=0x81c2b70, 
    fields=0x81a1fa0)
    at ../../../gcc/libjava/java/io/ObjectOutputStream.java:1173

> !java.lang.reflect.Array.newInstance
Ugh, not fun. Running by hand also hangs, but turning on the -debug or
-verbose flag makes it run... When not giving any flags it only prints
Needed to allocate blacklisted block at 0x824b000
The test actually tries to force a OutOfMemoryError exception which
might explain this. But the Object.clone() test also seems to do this
and that one just works.

> !java.util.ResourceBundle.getBundle
Don't know why that was in there. It seems to run fine, although it
gives some failures which should be investigated:
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: with locale of
Canada (number 4)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: with locale of
Canada (number 5)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: with locale of
France (number 4)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 2)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 5)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 6)
FAIL: gnu.testlet.java.util.ResourceBundle.getBundle: book sample
(number 7)
7 of 23 tests failed

> !java.util.zip.GZIPInputStream.basic
Also seems to just work. Both tests seems to succeed.

O Duh. Enabling one or both of the last two tests seem to crash or hang
the testsuite again when run in full. Curious. Keep disabled for now.

> !java.net.DatagramSocket.DatagramSocketTest2
Our DatagramSocket.receive() blocks till a packet is received even when
the buffer of the DatagramPacket is zero. Which seems correct given the
spec. Wrong test?


The libjava.mauve/xfail file currently contains the following:

> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error: test_shortbyteValue failed - 5 (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_shortbyteValue failed - 5 (number 1)
Yeah! These are now XPASS.

> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Four Byte Range Error (0) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Four Byte Range Error (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Five Bytes (0) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Five Bytes (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Six Bytes (0) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Six Bytes (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Orphan Continuation (1) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Orphan Continuation (2) (number 1)
> FAIL: gnu.testlet.java.io.Utf8Encoding.mojo: Four Byte Range Error (2) (number 1)
The test says:Note that JDK 1.1 and JDK 1.2 don't currently pass these tests;
there are known problems in their UTF-8 encoding support at this time.
This probably also true for libgcj.

> FAIL: gnu.testlet.java.io.ObjectStreamClass.Test: getSerialVersionUID (number 7)
Now passes! 

> FAIL: gnu.testlet.java.text.DateFormatSymbols.Test: patterns (number 2)
> FAIL: gnu.testlet.java.text.SimpleDateFormat.Test: equals() (number 1)
> FAIL: gnu.testlet.java.text.SimpleDateFormat.Test: parse() strict (number 1)
> FAIL: gnu.testlet.java.text.SimpleDateFormat.getAndSet2DigitYearStart: get2DigitYearStart() initial (number 1)
Seem to be known issues with our java.text support.

> FAIL: gnu.testlet.java.net.URLConnection.URLConnectionTest: Error in test_Basics  - 2  should not have raised  Throwable here  (number 1)
> FAIL: gnu.testlet.java.net.URL.URLTest: openStream (number 1)
> FAIL: gnu.testlet.java.net.URL.URLTest: sameFile (number 2)
> FAIL: gnu.testlet.java.net.URL.URLTest:  Error in test_toString  - 5  exception should not be thrown here (number 1)
> FAIL: gnu.testlet.java.net.URL.URLTest: new URL(protocol, host, file) (number 26)
> FAIL: gnu.testlet.java.net.URL.URLTest: new URL(protocol, host, file) (number 54)
We need to merge URL and URLConnection with Classpath and check again.

> FAIL: gnu.testlet.java.net.ServerSocket.ServerSocketTest: Error : test_params failed - 5getInetAddress did not return proper values  (number 1)
> FAIL: gnu.testlet.java.net.Socket.SocketTest: Error : test_BasicServer failed - 11 exception was thrown :Illegal seek (number 1)
> FAIL: gnu.testlet.java.net.MulticastSocket.MulticastSocketTest: joinGroup() twice. (number 1)
No time to investigate. Keep them in for now.

Actions:
Character.unicode should probably also be added to the mauve-libgcj ignore list
since it generates a lot of wrong failures.
We seem to agree that some tests are bogus or wrong in Mauve. I will make
patches for those tests and submit them to the mauve mailinglist.
I will try to add all tests that FAIL now and for which we know we will not
solve the issue for 3.1 to the xfail list (and submit bug reports).
Most issues can probably not be solved for 3.1 since that release is in about
10 days, but we can try. The goal is to have zero FAILs (but a couple of XFAILs)
for 3.1.

Cheers,

Mark

Mauve on Solaris 2.8

[Andrew Haley]

It's been ages since I used Mauve.  I get this on 3.1 branch:

touch classes.stamp
make[2]: Leaving directory `/export/scratch1/aph/gcc-3_1-branch/build/sparc-sun-solaris2.8/libjava/testsuite/mauve-build'
ERROR: (DejaGnu) proc "bytecompile_file /export/scratch1/aph/gcc-3_1-branch/gcc/libjava/testsuite/libjava.mauve/DejaGNUTestHarness.java /export/scratch1/aph/gcc-3_1-branch/build/sparc-sun-solaris2.8/libjava/testsuite/mauve-build /export/scratch1/aph/mauve:/export/scratch1/aph/gcc-3_1-branch/build/sparc-sun-solaris2.8/libjava/testsuite/mauve-build" does not exist.
The error code is NONE
The info on the error is:
no files matched glob pattern "/export/scratch1/aph/gcc-3_1-branch/gcc/libjava/testsuite/libjava.mauve/libjava.mauve*"
    while executing
"glob /export/scratch1/aph/gcc-3_1-branch/gcc/libjava/testsuite/libjava.mauve/libjava.mauve*"
    invoked from within
"catch "glob ${path}/${pattern}" tmp"

                === libjava.mauve Summary ===

I've no idea what might be causing this.

I set MAUVEDIR to point to the top of the Mauve sourcedir, then do a
"make check" in build/libjava.  Am I missing something?

Andrew.

Re: Mauve on Solaris 2.8

[Mark Wielaard]

Hi,

On Thu, 2002-04-04 at 18:38, Andrew Haley wrote:
> It's been ages since I used Mauve.  I get this on 3.1 branch:
> 
> touch classes.stamp
> make[2]: Leaving directory `/export/scratch1/aph/gcc-3_1-branch/build/sparc-sun-solaris2.8/libjava/testsuite/mauve-build'
> ERROR: (DejaGnu) proc "bytecompile_file /export/scratch1/aph/gcc-3_1-branch/gcc/libjava/testsuite/libjava.mauve/DejaGNUTestHarness.java /export/scratch1/aph/gcc-3_1-branch/build/sparc-sun-solaris2.8/libjava/testsuite/mauve-build /export/scratch1/aph/mauve:/export/scratch1/aph/gcc-3_1-branch/build/sparc-sun-solaris2.8/libjava/testsuite/mauve-build" does not exist.
> The error code is NONE
> The info on the error is:
> no files matched glob pattern "/export/scratch1/aph/gcc-3_1-branch/gcc/libjava/testsuite/libjava.mauve/libjava.mauve*"
>     while executing
> "glob /export/scratch1/aph/gcc-3_1-branch/gcc/libjava/testsuite/libjava.mauve/libjava.mauve*"
>     invoked from within
> "catch "glob ${path}/${pattern}" tmp"
> 
>                 === libjava.mauve Summary ===
> 
> I've no idea what might be causing this.
> 
> I set MAUVEDIR to point to the top of the Mauve sourcedir, then do a
> "make check" in build/libjava.  Am I missing something?
> 
I remember that I had to do the aclocal; autoheader; automake; autoconf
in the toplevel mauvedir since since the configure from CVS didn't work
out of the box.

Then I do a make check-target-libgcj in my object dir (that runs all the
libgcj tests not just mauve). It should create a
${platform}/libjava/testsuite/mauve-build dir that you seem to have.

I am not sure what goes wrong.

Cheers,

Mark

Re: Mauve on Solaris 2.8

[Andrew Haley]

Mark Wielaard writes:
 > > 
 > I remember that I had to do the aclocal; autoheader; automake; autoconf
 > in the toplevel mauvedir since since the configure from CVS didn't work
 > out of the box.

Ah.  Maybe we should fix this.

Andrew.

Re: Mauve on Solaris 2.8

[Andrew Haley]

Mark Wielaard writes:
 > > 
 > I remember that I had to do the aclocal; autoheader; automake; autoconf
 > in the toplevel mauvedir since since the configure from CVS didn't work
 > out of the box.

Can you tell me what versions of these tools you used?  I don't seem
to be able to find a combination that works.

Andrew.

Re: Mauve on Solaris 2.8

[Andrew Haley]

Andrew Haley writes:
 > Mark Wielaard writes:
 >  > > 
 >  > I remember that I had to do the aclocal; autoheader; automake; autoconf
 >  > in the toplevel mauvedir since since the configure from CVS didn't work
 >  > out of the box.
 > 
 > Can you tell me what versions of these tools you used?  I don't seem
 > to be able to find a combination that works.

Ah, OK.  I found automake-gcj-1.4.tar.gz

Andrew.

Re: Mauve on Solaris 2.8

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

Hi,

On Thu, 2002-04-04 at 20:57, Andrew Haley wrote:
>  > Can you tell me what versions of these tools you used?  I don't seem
>  > to be able to find a combination that works.
> 
> Ah, OK.  I found automake-gcj-1.4.tar.gz

And does it work? I remember complaining that Classpath used automake
1.5 but Mauve needed 1.4. If I remember correctly Brian Jones then fixed
mauve to be useable with automake 1.5. I don't remember exactly from
when my version is.

But I just tried with a fresh CVS checkout. And doing:

mark@elsschot:~/src/gcc-3.2/obj$ export MAUVEDIR=/tmp/mauve
mark@elsschot:~/src/gcc-3.2/obj$ rm -rf \
	i686-pc-linux-gnu/libjava/testsuite/mauve-build
mark@elsschot:~/src/gcc-3.2/obj$ make check-target-libjava

Seems to work out of the box on my machine now.
So no aclocal; ; autoheader; automake; autoconf dance needed.
(make/configure output attached.)

At the moment the number of failures is a bit high (~200) and some of
these are actually faults in the testsuite or tests that produce lots of
failures because we have not yet implemented something. So running it is
not very productive at the moment.
I am working on getting this number down a bit, hopefully to zero FAIL
when we release 3.1 (with xfails for things we now are broken in libgcj
at the moment) so we can really use it regularly again when we start
work on 3.2.

Cheers,

Mark

[-- Attachment #2: mauve.out --]
[-- Type: text/plain, Size: 2618 bytes --]

Running /home/mark/src/gcc-3.2/gcc/libjava/testsuite/libjava.mauve/mauve.exp ...checking for a BSD compatible install... /bin/sh /home/mark/src/gcc-3.2/gcc/install-sh -c
checking whether build environment is sane... yes
checking for mawk... mawk
checking whether make sets ${MAKE}... yes
checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking for Windows and DOS and OS/2 style pathnames... no
checking for a BSD compatible install... /bin/sh /home/mark/src/gcc-3.2/gcc/install-sh -c
configure: creating ./config.status
config.status: creating Makefile
config.status: creating gnu/testlet/config.java
make[4]: Entering directory `/opt/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/mauve-build'
Makefile:351: choices: No such file or directory
ok=no; \
if test -f .save-keys && test -f choices && test "`cat .save-keys`" = "libgcj"; then \
  ok=yes; \
fi; \
here=`/bin/pwd`; \
if test "$ok" = no; then \
  echo "libgcj" > .save-keys; \
  cd /tmp/mauve && /bin/sh choose $here libgcj; \
fi
find: com: No such file or directory
make[4]: Leaving directory `/opt/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/mauve-build'
make[4]: Entering directory `/opt/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/mauve-build'
ok=no; \
if test -f .save-keys && test -f choices && test "`cat .save-keys`" = "libgcj"; then \
  ok=yes; \
fi; \
here=`/bin/pwd`; \
if test "$ok" = no; then \
  echo "libgcj" > .save-keys; \
  cd /tmp/mauve && /bin/sh choose $here libgcj; \
fi
/home/mark/src/gcc-3.2/obj/gcc/gcj -B/home/mark/src/gcc-3.2/obj/gcc/ --encoding=UTF-8 -B/home/mark/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/../ -I/home/mark/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/../libgcj.zip -C -g gnu/testlet/config.java
here=`/bin/pwd`; cd /tmp/mauve; \
  CLASSPATH=$CLASSPATH:$here:`/bin/pwd` /home/mark/src/gcc-3.2/obj/gcc/gcj -B/home/mark/src/gcc-3.2/obj/gcc/ --encoding=UTF-8 -B/home/mark/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/../ -I/home/mark/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/../libgcj.zip -C -g -d $here \
[... long line of classes ...]
touch classes.stamp
make[4]: Leaving directory `/opt/src/gcc-3.2/obj/i686-pc-linux-gnu/libjava/testsuite/mauve-build'
FAIL: gnu.testlet.java.io.BufferedByteOutputStream.interrupt: single-byte writes (number 3)
FAIL: gnu.testlet.java.io.BufferedByteOutputStream.interrupt: single-byte writes (number 4)
FAIL: gnu.testlet.java.io.File.jdk11: getCanonicalPath () java.io.IOException: No such file or directory (number 1)
[... etc ...]