From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <sourceware-bugzilla@sourceware.org>
Received: by sourceware.org (Postfix, from userid 48)
	id 4C0523858D35; Thu, 10 Nov 2022 10:57:07 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4C0523858D35
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org;
	s=default; t=1668077827;
	bh=Ztinwdz0clprZYYI6kzMaueZhdNOvFI2dluWEOuasJI=;
	h=From:To:Subject:Date:From;
	b=lb1t5TWhdBxc6120qRxkpfO0aSSRIGg4XZ6PtEXI49GdzHoKk6h0YblpvpCvMAz7G
	 iceYFofxCZ6viHJTiFNw+R3oqP+xi/Lgejg50X0RwwbhzmYJhWS5wLrj6dMqENd/1G
	 N7CNw+pMFS79wleW1dxvxGE6hQOFRqzVFLFPPHSI=
From: "vries at gcc dot gnu.org" <sourceware-bugzilla@sourceware.org>
To: gdb-prs@sourceware.org
Subject: [Bug testsuite/29772] New: [gdb/testsuite] Detect duplicates for
 "FAIL: foo" and "FAIL: foo (timeout)"
Date: Thu, 10 Nov 2022 10:57:06 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gdb
X-Bugzilla-Component: testsuite
X-Bugzilla-Version: HEAD
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: normal
X-Bugzilla-Who: vries at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Resolution: 
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: unassigned at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags: 
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status
 bug_severity priority component assigned_to reporter target_milestone
Message-ID: <bug-29772-4717@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
List-Id: <gdb-prs.sourceware.org>

https://sourceware.org/bugzilla/show_bug.cgi?id=3D29772

            Bug ID: 29772
           Summary: [gdb/testsuite] Detect duplicates for "FAIL: foo" and
                    "FAIL: foo (timeout)"
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: testsuite
          Assignee: unassigned at sourceware dot org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

I noticed here ( https://sourceware.org/gdb/wiki/GDBTestcaseCookbook ):
...
Do not use "tail parentheses" on test messages

When you write a test, do not put text between parentheses preceded by a sp=
ace
at the end of the text message. For example, this is incorrect:

gdb_test "some_test" "expected output" "I am a test message (123)"
...

I then noticed that when checking for duplicates, we do detect:
...
FAIL: gdb.testsuite/duplicate.exp: foo
FAIL: gdb.testsuite/duplicate.exp: foo
...
but not:
...
FAIL: gdb.testsuite/duplicate.exp: foo
FAIL: gdb.testsuite/duplicate.exp: foo (timeout)
...

I wrote a patch to fix this:
...
diff --git a/gdb/testsuite/lib/check-test-names.exp
b/gdb/testsuite/lib/check-test
-names.exp
index 752981c7888..fac93259464 100644
--- a/gdb/testsuite/lib/check-test-names.exp
+++ b/gdb/testsuite/lib/check-test-names.exp
@@ -64,6 +64,9 @@ namespace eval ::CheckTestNames {
     proc _check_duplicates { message } {
        variable all_test_names

+       # Remove the "extra information" part.
+       set message [regsub { \(.*\)$} $message ""]
+
        # Initialise a count, or increment the count for this test name.
        if {![info exists all_test_names($message)]} {
            set all_test_names($message) 0
...

I ran into 6453 duplicates in 134 test-cases:
...
$ grep DUPLICATE: gdb.sum | wc -l
6453
$ grep DUPLICATE: gdb.sum | awk '{print $2}' | sort -u | wc -l
134
...

For instance:
...
DUPLICATE: gdb.arch/amd64-osabi.exp: arch=3Di386:x86-64: print sizeof (long
double)
DUPLICATE: gdb.arch/amd64-osabi.exp: arch=3Di386:x64-32: print sizeof (long
double)
DUPLICATE: gdb.arch/amd64-osabi.exp: arch=3Di386: print sizeof (long double)
...

It may be worthwhile to override all pass / fail procs to do some automated
test name substitution:
...
-PASS: gdb.arch/amd64-osabi.exp: arch=3Di386:x86-64: print sizeof (long dou=
ble)
+PASS: gdb.arch/amd64-osabi.exp: arch=3Di386:x86-64: print sizeof _(long do=
uble)_
...
and create extra versions fail_with_info to do:
...
-fail "$testname (timeout)"
+fail_with_info $testname "timeout"
...

--=20
You are receiving this mail because:
You are on the CC list for the bug.=