Harness tweak

Harness tweak

[Anthony Balkissoon]

[-- Attachment #1: Type: text/plain, Size: 963 bytes --]

This patch adds some reporting functionality that Tom Fitzsimmons asked
for.  It reports the total number of calls to check() that failed, so
that a developer can know if his/her patch to Classpath improved the
results at all, even if that patch didn't result in the entire test
passing.

2006-06-19  Anthony Balkissoon  <abalkiss@redhat.com>

	* Harness.java:
	(total_check_fails): New field.
	(main): Print a summary even if only one test failed.  Include the 
	total number of failing check() calls in the summary.
	(runTest): When a test fails, increment total_check_fails by the 
	number of checks in that test that failed, which is passed back from
	the RunnerProcess.
	* RunnerProcess.java:
	(main): If a test fails to load, report the failure and count 0 total
	check() fails.
	(runAndReport): When a test fails, report the number of calls to
	check() that failed.
	(done): If the test failed, return the number of calls to check() 
	that failed.

--Tony

[-- Attachment #2: ReportingTweak.diff --]
[-- Type: text/x-patch, Size: 3223 bytes --]

Index: Harness.java
===================================================================
RCS file: /cvs/mauve/mauve/Harness.java,v
retrieving revision 1.10
diff -u -r1.10 Harness.java
--- Harness.java        19 Jun 2006 17:49:36 -0000      1.10
+++ Harness.java        19 Jun 2006 18:20:45 -0000
@@ -106,0 +106,0 @@

   // The total number of failing tests (not harness.check() calls)
   private static int total_test_fails = 0;
+
+  // The total number of harness.check() calls that fail
+  private static int total_check_fails = 0;

   // All the tests that were specified on the command line rather than
   // through standard input or an input file
@@ -167,1 +170,1 @@
     runAllTests();

     // If more than one test was run, print a summary.
-    if (total_tests > 1)
+    if (total_tests > 0)
       System.out.println("\nTEST RESULTS:\n" + total_test_fails + " of "
-                         + total_tests + " tests failed.");
-    else if (total_tests == 0)
+                         + total_tests + " tests failed.  " + total_check_fails
+                         + " total calls to harness.check() failed.");
+    else
       {
         // If no tests were run, try to help the user out by suggesting what
         // the problem might have been.
@@ -201,10 +205,6 @@
                                 "no tests? \n");
           }
       }
-    else if (total_test_fails == 0 && !showPasses && !verbose)
-      // Finally, if a single test was run and the passing result wasn't
-      // displayed, display it for the user.
-      System.out.println ("TEST RESULT: pass");
     harness.finalize();
     System.exit(total_test_fails > 0 ? 1 : 0);
   }
@@ -805,8 +805,10 @@
                       temp = 0;
                       break;
                     }
-                  else if (outputFromTest.endsWith("fail"))
+                  else if (outputFromTest.indexOf("fail-") != -1)
                     {
+                      total_check_fails +=
+                        Integer.parseInt(outputFromTest.substring(19));
                       temp = 1;
                       break;
                     }
Index: RunnerProcess.java
===================================================================
RCS file: /cvs/mauve/mauve/RunnerProcess.java,v
retrieving revision 1.6
diff -u -r1.6 RunnerProcess.java
--- RunnerProcess.java  19 Jun 2006 17:49:36 -0000      1.6
+++ RunnerProcess.java  19 Jun 2006 18:20:45 -0000
@@ -218,7 +218,7 @@
           else
             System.out.println("FAIL: " + stripPrefix(testname)
                                  + ": failed to load");
-          System.out.println("RunnerProcess:fail");
+          System.out.println("RunnerProcess:fail-0");
         }
       }
   }
@@ -371,5 +371,5 @@
     if (temp == 0)
       System.out.println ("RunnerProcess:pass");
     else
-      System.out.println("RunnerProcess:fail");
+      System.out.println("RunnerProcess:fail-" + temp);
   }

   private final String getDescription(StackTraceElement[] st)
@@ -784,5 +784,5 @@
     if (xfailures > 0)
       System.out.println(xfailures + " of " + total
                          + " tests expectedly failed");
-    return failures > 0 ? 1 : 0;
+    return failures;
   }

   /**