From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 25991 invoked by alias); 19 Jun 2006 18:27:37 -0000 Received: (qmail 25981 invoked by uid 22791); 19 Jun 2006 18:27:36 -0000 X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (66.187.233.31) by sourceware.org (qpsmtpd/0.31) with ESMTP; Mon, 19 Jun 2006 18:27:31 +0000 Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.12.11.20060308/8.12.11) with ESMTP id k5JIRTPa007547 for ; Mon, 19 Jun 2006 14:27:29 -0400 Received: from pobox.toronto.redhat.com (pobox.toronto.redhat.com [172.16.14.4]) by int-mx1.corp.redhat.com (8.12.11.20060308/8.12.11) with ESMTP id k5JIRThe007221 for ; Mon, 19 Jun 2006 14:27:29 -0400 Received: from tony.toronto.redhat.com (tony.toronto.redhat.com [172.16.14.158]) by pobox.toronto.redhat.com (8.12.8/8.12.8) with ESMTP id k5JIRTve031809 for ; Mon, 19 Jun 2006 14:27:29 -0400 Subject: Harness tweak From: Anthony Balkissoon To: mauve-patches@sources.redhat.com Content-Type: multipart/mixed; boundary="=-Si6TRxMrz3ucAf8hXsUd" Date: Mon, 19 Jun 2006 18:27:00 -0000 Message-Id: <1150741648.25174.59.camel@tony.toronto.redhat.com> Mime-Version: 1.0 X-Mailer: Evolution 2.6.0 (2.6.0-1) X-IsSubscribed: yes Mailing-List: contact mauve-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: mauve-patches-owner@sourceware.org X-SW-Source: 2006/txt/msg00436.txt.bz2 --=-Si6TRxMrz3ucAf8hXsUd Content-Type: text/plain Content-Transfer-Encoding: 7bit Content-length: 963 This patch adds some reporting functionality that Tom Fitzsimmons asked for. It reports the total number of calls to check() that failed, so that a developer can know if his/her patch to Classpath improved the results at all, even if that patch didn't result in the entire test passing. 2006-06-19 Anthony Balkissoon * Harness.java: (total_check_fails): New field. (main): Print a summary even if only one test failed. Include the total number of failing check() calls in the summary. (runTest): When a test fails, increment total_check_fails by the number of checks in that test that failed, which is passed back from the RunnerProcess. * RunnerProcess.java: (main): If a test fails to load, report the failure and count 0 total check() fails. (runAndReport): When a test fails, report the number of calls to check() that failed. (done): If the test failed, return the number of calls to check() that failed. --Tony --=-Si6TRxMrz3ucAf8hXsUd Content-Disposition: attachment; filename=ReportingTweak.diff Content-Type: text/x-patch; name=ReportingTweak.diff; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-length: 3223 Index: Harness.java =================================================================== RCS file: /cvs/mauve/mauve/Harness.java,v retrieving revision 1.10 diff -u -r1.10 Harness.java --- Harness.java 19 Jun 2006 17:49:36 -0000 1.10 +++ Harness.java 19 Jun 2006 18:20:45 -0000 @@ -106,0 +106,0 @@ // The total number of failing tests (not harness.check() calls) private static int total_test_fails = 0; + + // The total number of harness.check() calls that fail + private static int total_check_fails = 0; // All the tests that were specified on the command line rather than // through standard input or an input file @@ -167,1 +170,1 @@ runAllTests(); // If more than one test was run, print a summary. - if (total_tests > 1) + if (total_tests > 0) System.out.println("\nTEST RESULTS:\n" + total_test_fails + " of " - + total_tests + " tests failed."); - else if (total_tests == 0) + + total_tests + " tests failed. " + total_check_fails + + " total calls to harness.check() failed."); + else { // If no tests were run, try to help the user out by suggesting what // the problem might have been. @@ -201,10 +205,6 @@ "no tests? \n"); } } - else if (total_test_fails == 0 && !showPasses && !verbose) - // Finally, if a single test was run and the passing result wasn't - // displayed, display it for the user. - System.out.println ("TEST RESULT: pass"); harness.finalize(); System.exit(total_test_fails > 0 ? 1 : 0); } @@ -805,8 +805,10 @@ temp = 0; break; } - else if (outputFromTest.endsWith("fail")) + else if (outputFromTest.indexOf("fail-") != -1) { + total_check_fails += + Integer.parseInt(outputFromTest.substring(19)); temp = 1; break; } Index: RunnerProcess.java =================================================================== RCS file: /cvs/mauve/mauve/RunnerProcess.java,v retrieving revision 1.6 diff -u -r1.6 RunnerProcess.java --- RunnerProcess.java 19 Jun 2006 17:49:36 -0000 1.6 +++ RunnerProcess.java 19 Jun 2006 18:20:45 -0000 @@ -218,7 +218,7 @@ else System.out.println("FAIL: " + stripPrefix(testname) + ": failed to load"); - System.out.println("RunnerProcess:fail"); + System.out.println("RunnerProcess:fail-0"); } } } @@ -371,5 +371,5 @@ if (temp == 0) System.out.println ("RunnerProcess:pass"); else - System.out.println("RunnerProcess:fail"); + System.out.println("RunnerProcess:fail-" + temp); } private final String getDescription(StackTraceElement[] st) @@ -784,5 +784,5 @@ if (xfailures > 0) System.out.println(xfailures + " of " + total + " tests expectedly failed"); - return failures > 0 ? 1 : 0; + return failures; } /** --=-Si6TRxMrz3ucAf8hXsUd--