public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug testsuite/30119]  New: libjava testsuite output is erratic and unhelpful
@ 2006-12-08 11:40 amylaar at gcc dot gnu dot org
  2006-12-08 11:46 ` [Bug testsuite/30119] " aph at gcc dot gnu dot org
                   ` (9 more replies)
  0 siblings, 10 replies; 11+ messages in thread
From: amylaar at gcc dot gnu dot org @ 2006-12-08 11:40 UTC (permalink / raw)
  To: gcc-bugs

I have tested the patch for PR rtl-optimization/29858 in revision 119261
on gcc01 (i686-pc-linux-gnu)
Compared to a pristine build of revision 119055, these are the
additional failures:

> FAIL: gcc.dg/visibility-11.c scan-assembler memcpy@PLT
14a16,17
> FAIL: gcc.dg/vect/vect-pow-1.c scan-tree-dump pattern recognized
> FAIL: gcc.dg/vect/vect-pow-2.c scan-tree-dump pattern recognized
107a111,112
> FAIL: PR18699 execution - gij test
> FAIL: PR18699 execution - gij test

compared to a pristine build of 119261 (the base version), this are the
aditional failures:

110a111,115
> FAIL: PR18699 execution - gij test
> FAIL: PR18699 execution - gij test
> FAIL: SyncTest execution - gij test
> FAIL: SyncTest execution - gij test
> FAIL: SyncTest execution - gij test

With failures in the gcc core or c++ / libstdc++ testsuite, reproducing
failures is very straight forward : you cut & paste the appropriate line(s)
from the log file in order, and can thus ovserve the failure interactively,
and then use gdb and/or debugging dumps to further investigate.

The log file shows this about the PR18699 failure:
PASS: PR18699 -O3 output - source compiled test
byte compile: /home/amylaar/bld/2006-11-27-29858/i686/gcc/gcj
-B/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/
-B/home/amylaar/bld/2006-11-27-29858/i686/gcc/ --encoding=UTF-8 -C
-I/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite/../libgcj-4.3.0.jar
-g
/home/amylaar/bld/2006-11-27-29858/srcw/libjava/testsuite/libjava.lang/PR18699.java
-d /home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite
2>@ stdout
PASS: PR18699 byte compilation
PR18699PR18699 set_ld_library_path_env_vars:
ld_library_path=.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc
Setting LD_LIBRARY_PATH to
.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:/home/amylaar/bld/2006-11-27-29858/i686/./bfd/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./prev-bfd/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./opcodes/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./prev-opcodes/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libstdc++-v3/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libmudflap/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libssp/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libgomp/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./gcc:/home/amylaar/bld/2006-11-27-29858/i686/./prev-gcc
FAIL: PR18699 execution - gij test

When I cut & past the line above 'PASS: PR18699 byte compilation', a file named
'@' is created, which contains:

gcj: stdout: No such file or directory

The only README file in the entire libjava testsuite is
testsuite/libjava.verify/README.verify .

The web documentation on testing http://gcc.gnu.org/install/test.html only
has the basic meaning of PASS/ FAIL etc for the benefit of a person who
installs the library without modifying any pieces of the GNU compiler
collection.

I should not be required to reverse-engineer the libjava testsuite before
I can interpret/debug test results for my patches to the gcc core.
There should be easy-to-follow documentation how I can get from
the debugging log to reproducing the failure.

Moreover, considering that there is no regession against the baseline for any
other part of the GNU compiler collection, and the recent track record of
libjava testing, it seems highly likely that the regressions are actually
testsuite failures.


-- 
           Summary: libjava testsuite output is erratic and unhelpful
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: testsuite
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: amylaar at gcc dot gnu dot org
OtherBugsDependingO 29842,29858
             nThis:


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
@ 2006-12-08 11:46 ` aph at gcc dot gnu dot org
  2006-12-08 14:59 ` amylaar at gcc dot gnu dot org
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: aph at gcc dot gnu dot org @ 2006-12-08 11:46 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #1 from aph at gcc dot gnu dot org  2006-12-08 11:46 -------
It's not necessary to do any I/O redirection when byte compiling: just execute
the command itself without the "2>" etc.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
  2006-12-08 11:46 ` [Bug testsuite/30119] " aph at gcc dot gnu dot org
@ 2006-12-08 14:59 ` amylaar at gcc dot gnu dot org
  2006-12-08 15:21 ` aph at gcc dot gnu dot org
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: amylaar at gcc dot gnu dot org @ 2006-12-08 14:59 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #2 from amylaar at gcc dot gnu dot org  2006-12-08 14:59 -------
(In reply to comment #1)
> It's not necessary to do any I/O redirection when byte compiling: just execute
> the command itself without the "2>" etc.
> 

All right, that gives me a file 'PR18699.class'  But how is this actually
executed?
The next two lines of the log file seem to be only about setting environment
variables, and the very next line has the 'FAIL' message.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
  2006-12-08 11:46 ` [Bug testsuite/30119] " aph at gcc dot gnu dot org
  2006-12-08 14:59 ` amylaar at gcc dot gnu dot org
@ 2006-12-08 15:21 ` aph at gcc dot gnu dot org
  2006-12-08 15:46 ` amylaar at gcc dot gnu dot org
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: aph at gcc dot gnu dot org @ 2006-12-08 15:21 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #3 from aph at gcc dot gnu dot org  2006-12-08 15:20 -------
Run it with the libjava/gij PR18699.class


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (2 preceding siblings ...)
  2006-12-08 15:21 ` aph at gcc dot gnu dot org
@ 2006-12-08 15:46 ` amylaar at gcc dot gnu dot org
  2006-12-08 16:45 ` aph at gcc dot gnu dot org
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: amylaar at gcc dot gnu dot org @ 2006-12-08 15:46 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #4 from amylaar at gcc dot gnu dot org  2006-12-08 15:46 -------
(In reply to comment #3)
> Run it with the libjava/gij PR18699.class

ls -l PR18699.class
-rw-r--r--  1 amylaar users 1688 2006-12-08 15:52 PR18699.class
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij PR18699.class
Exception in thread "main" java.lang.NoClassDefFoundError: PR18699.class
   at gnu.java.lang.MainThread.run(MainThread.java:102)
Caused by: java.lang.ClassNotFoundException: PR18699.class not found in
gnu.gcj.runtime.SystemClassLoader{urls=[file:./],
parent=gnu.gcj.runtime.ExtensionClassLoader{urls=[], parent=null}}
   at java.net.URLClassLoader.findClass(URLClassLoader.java:1080)
   at gnu.gcj.runtime.SystemClassLoader.findClass(natSystemClassLoader.cc:27)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:294)
   at gnu.java.lang.MainThread.run(MainThread.java:98)

gcc01 does not have jcf-dump, but when I scp the file to my local machine,
I see after the constant table:

Access flags: 0x20 super
This class: 2=PR18699, super: 4=java.util.Observable
Interfaces (count: 2):
- Implements: 6=java.lang.Runnable
- Implements: 8=java.util.Observer

So is this a bug in gij not to find the class in PR18699.class?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (3 preceding siblings ...)
  2006-12-08 15:46 ` amylaar at gcc dot gnu dot org
@ 2006-12-08 16:45 ` aph at gcc dot gnu dot org
  2006-12-13 13:25 ` amylaar at gcc dot gnu dot org
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: aph at gcc dot gnu dot org @ 2006-12-08 16:45 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #5 from aph at gcc dot gnu dot org  2006-12-08 16:45 -------
Set the classpath

gij -classpath <path to  PR18699.class>:<path to libgcj.jar>  PR18699


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (4 preceding siblings ...)
  2006-12-08 16:45 ` aph at gcc dot gnu dot org
@ 2006-12-13 13:25 ` amylaar at gcc dot gnu dot org
  2006-12-13 13:43 ` aph at gcc dot gnu dot org
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: amylaar at gcc dot gnu dot org @ 2006-12-13 13:25 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #6 from amylaar at gcc dot gnu dot org  2006-12-13 13:25 -------
(In reply to comment #5)
> Set the classpath
> 
> gij -classpath <path to  PR18699.class>:<path to libgcj.jar>  PR18699

Unfortunately, that still doesn't work:

amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath PR18699.class:../libgcj-4.3.0.jar PR18699.class 
Exception in thread "main" java.lang.NoClassDefFoundError: PR18699.class
   at gnu.java.lang.MainThread.run(MainThread.java:102)
Caused by: java.lang.ClassNotFoundException: PR18699.class not found in
gnu.gcj.runtime.SystemClassLoader{urls=[file:PR18699.class,file:../libgcj-4.3.0.jar],
parent=gnu.gcj.runtime.ExtensionClassLoader{urls=[], parent=null}}
   at java.net.URLClassLoader.findClass(URLClassLoader.java:1080)
   at gnu.gcj.runtime.SystemClassLoader.findClass(natSystemClassLoader.cc:27)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:294)
   at gnu.java.lang.MainThread.run(MainThread.java:98)
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
ls -l PR18699.class
-rw-r--r--  1 amylaar users 1688 2006-12-13 14:11 PR18699.class
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
ls -l ../libgcj-4.3.0.jar
-rw-r--r--  1 amylaar users 8547070 2006-12-05 00:07 ../libgcj-4.3.0.jar


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (5 preceding siblings ...)
  2006-12-13 13:25 ` amylaar at gcc dot gnu dot org
@ 2006-12-13 13:43 ` aph at gcc dot gnu dot org
  2006-12-13 19:47 ` amylaar at gcc dot gnu dot org
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: aph at gcc dot gnu dot org @ 2006-12-13 13:43 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #7 from aph at gcc dot gnu dot org  2006-12-13 13:42 -------
You wrote

../gij -classpath PR18699.class:../libgcj-4.3.0.jar PR18699.class

But I wrote:

../gij -classpath .:../libgcj-4.3.0.jar PR18699


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (6 preceding siblings ...)
  2006-12-13 13:43 ` aph at gcc dot gnu dot org
@ 2006-12-13 19:47 ` amylaar at gcc dot gnu dot org
  2006-12-15 11:21 ` aph at gcc dot gnu dot org
  2006-12-19 12:57 ` amylaar at gcc dot gnu dot org
  9 siblings, 0 replies; 11+ messages in thread
From: amylaar at gcc dot gnu dot org @ 2006-12-13 19:47 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #8 from amylaar at gcc dot gnu dot org  2006-12-13 19:47 -------
(In reply to comment #7)

Sorry, I misunderstood you.  Now the test works - most of the time:

amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
/home/amylaar/bld/2006-11-27-29858/i686/gcc/gcj
-B/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/
-B/home/amylaar/bld/2006-11-27-29858/i686/gcc/ --encoding=UTF-8 -C
-I/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite/../libgcj-4.3.0.jar
-g
/home/amylaar/bld/2006-11-27-29858/srcw/libjava/testsuite/libjava.lang/PR18699.java
-d /home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
export
ld_library_path=.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
export
LD_LIBRARY_PATH=.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:/home/amylaar/bld/2006-11-27-29858/i686/./bfd/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./prev-bfd/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./opcodes/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./prev-opcodes/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libstdc++-v3/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libmudflap/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libssp/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libgomp/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./gcc:/home/amylaar/bld/2006-11-27-29858/i686/./prev-gcc
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
Exception in thread "Thread-2" java.lang.NullPointerException
   at PR18699.update(PR18699.java:34)
   at java.util.Observable.notifyObservers(Observable.java:171)
   at PR18699.run(PR18699.java:26)
   at java.lang.Thread.run(Thread.java:682)
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
echo $?
0
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
echo $?
0
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
echo $?
0
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
echo $?
0

I have no idea how to make it repeat the null pointer exception, except trying
very often.  Here is another one:

amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
../gij -classpath .:../libgcj-4.3.0.jar PR18699
Exception in thread "Thread-1" java.lang.NullPointerException
   at PR18699.update(PR18699.java:34)
   at java.util.Observable.notifyObservers(Observable.java:171)
   at PR18699.run(PR18699.java:26)
   at java.lang.Thread.run(Thread.java:682)
amylaar@gcc01:~/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite$
free
             total       used       free     shared    buffers     cached
Mem:       1036088     969192      66896          0     448352     147260
-/+ buffers/cache:     373580     662508
Swap:       755012          0     755012





-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (7 preceding siblings ...)
  2006-12-13 19:47 ` amylaar at gcc dot gnu dot org
@ 2006-12-15 11:21 ` aph at gcc dot gnu dot org
  2006-12-19 12:57 ` amylaar at gcc dot gnu dot org
  9 siblings, 0 replies; 11+ messages in thread
From: aph at gcc dot gnu dot org @ 2006-12-15 11:21 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #9 from aph at gcc dot gnu dot org  2006-12-15 11:21 -------
OK, so it looks like this is not a testsuite problem: the problem really is
with the program being tested.

The next thing to do is run the program with GDB and find out where the SEGV
occurs.


-- 

aph at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at redhat dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
  2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
                   ` (8 preceding siblings ...)
  2006-12-15 11:21 ` aph at gcc dot gnu dot org
@ 2006-12-19 12:57 ` amylaar at gcc dot gnu dot org
  9 siblings, 0 replies; 11+ messages in thread
From: amylaar at gcc dot gnu dot org @ 2006-12-19 12:57 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #10 from amylaar at gcc dot gnu dot org  2006-12-19 12:57 -------
(In reply to comment #9)
> OK, so it looks like this is not a testsuite problem: the problem really is
> with the program being tested.

It is also a testsuite problem when the tests are randonly passing or failing;
such tests are not suitable as such in an automated testsuite.
I suppose you could write a wrapper to run them several times to get a
reasonable high confidence that the test generally passes - if the test does
not take too long - and maybe the probabilities can also be enhanced by
forking specifically designed other programs to perturb the load, but
unless/ until such infrastructure is put in place, these randomly failing
tests make it effectively impossible to pass a regression test.

I've redone to tests on my baseline where the test passed during the automated
test, and the results are again random:

amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$
/home/amylaar/bld/2006-11-27/i686/gcc/gcj
-B/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/
-B/home/amylaar/bld/2006-11-27/i686/gcc/ --encoding=UTF-8 -C
-I/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite/../libgcj-4.3.0.jar
-g
/home/amylaar/bld/2006-11-27/srcw/libjava/testsuite/libjava.lang/PR18699.java
-d /home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ export
ld_library_path=.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ export
LD_LIBRARY_PATH=.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:/home/amylaar/bld/2006-11-27/i686/./bfd/.libs:/home/amylaar/bld/2006-11-27/i686/./prev-bfd/.libs:/home/amylaar/bld/2006-11-27/i686/./opcodes/.libs:/home/amylaar/bld/2006-11-27/i686/./prev-opcodes/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libstdc++-v3/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libmudflap/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libssp/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libgomp/.libs:/home/amylaar/bld/2006-11-27/i686/./gcc:/home/amylaar/bld/2006-11-27/i686/./prev-gcc

amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
Exception in thread "Thread-2" java.lang.NullPointerException
   at PR18699.update(PR18699.java:34)
   at java.util.Observable.notifyObservers(Observable.java:171)
   at PR18699.run(PR18699.java:26)
   at java.lang.Thread.run(Thread.java:682)
1
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
Exception in thread "Thread-1" ^[[Ajava.lang.NullPointerException
   at PR18699.update(PR18699.java:34)
   at java.util.Observable.notifyObservers(Observable.java:171)
   at PR18699.run(PR18699.java:26)
   at java.lang.Thread.run(Thread.java:682)
1
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
Exception in thread "Thread-2" ^[[A
java.lang.NullPointerException
   at PR18699.update(PR18699.java:34)
   at java.util.Observable.notifyObservers(Observable.java:171)
   at PR18699.run(PR18699.java:26)
   at java.lang.Thread.run(Thread.java:682)
1
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119


^ permalink raw reply	[flat|nested] 11+ messages in thread
end of thread, other threads:[~2006-12-19 12:57 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-12-08 11:40 [Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful amylaar at gcc dot gnu dot org
2006-12-08 11:46 ` [Bug testsuite/30119] " aph at gcc dot gnu dot org
2006-12-08 14:59 ` amylaar at gcc dot gnu dot org
2006-12-08 15:21 ` aph at gcc dot gnu dot org
2006-12-08 15:46 ` amylaar at gcc dot gnu dot org
2006-12-08 16:45 ` aph at gcc dot gnu dot org
2006-12-13 13:25 ` amylaar at gcc dot gnu dot org
2006-12-13 13:43 ` aph at gcc dot gnu dot org
2006-12-13 19:47 ` amylaar at gcc dot gnu dot org
2006-12-15 11:21 ` aph at gcc dot gnu dot org
2006-12-19 12:57 ` amylaar at gcc dot gnu dot org

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).