From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <sourceware-bugzilla@sourceware.org>
Received: by sourceware.org (Postfix, from userid 48)
	id 2ACD8385AC29; Sat, 12 Nov 2022 10:42:10 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2ACD8385AC29
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org;
	s=default; t=1668249730;
	bh=lTRg+vDCaePrFTVS8uzVL+3DnXdPW+uF8GdhkbOAb4Q=;
	h=From:To:Subject:Date:From;
	b=UsMm6Ft/+0bB3Vr07f3LzHXNlg8uzwITTGwQDbGOQCtVp3tUMt5s4b8d5eL61h4qE
	 fqDyhJYYTb9ymoYSpuNMmauSEL6tOe6rX8elgtEX8AvyYyG7KzxZywa7u+UO+l5fV3
	 ByYSWCJ7dH/oMYIQbUPdlBPHtBRs/n678ErUCLnM=
From: "vries at gcc dot gnu.org" <sourceware-bugzilla@sourceware.org>
To: gdb-prs@sourceware.org
Subject: [Bug testsuite/29778] New: [gdb/testsuite] Revise untested usage
Date: Sat, 12 Nov 2022 10:42:09 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gdb
X-Bugzilla-Component: testsuite
X-Bugzilla-Version: HEAD
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: vries at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Resolution: 
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: unassigned at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags: 
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status
 bug_severity priority component assigned_to reporter target_milestone
Message-ID: <bug-29778-4717@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
List-Id: <gdb-prs.sourceware.org>

https://sourceware.org/bugzilla/show_bug.cgi?id=3D29778

            Bug ID: 29778
           Summary: [gdb/testsuite] Revise untested usage
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: testsuite
          Assignee: unassigned at sourceware dot org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

The definitions/explanations for UNTESTED in the dejagnu manual are:
...
A test case is not yet complete, and in particular cannot yet produce a PAS=
S or
FAIL. You can also use this outcome in dummy =E2=80=9Ctests=E2=80=9D that n=
ote explicitly the
absence of a real test case for a particular property.
...
and:
...
untested
prints a message for an test case that isn=E2=80=99t run for some technical=
 reason.=20
...
and:
...
untested
prints a message for an test case that isn=E2=80=99t run for some reason.
...
and:
...
UNTESTED
A test was not run. This is a placeholder used when there is no real test c=
ase
yet.
...
and:
...
For example, you might use this in a dummy test whose only role is to record
that a test does not yet exist for some feature.
...

I don't think the definition is very clear, but as I understand it the valid
uses of UNTESTED are:
...
gdb.base/features.exp: PASS: feature a
gdb.base/features.exp: UNTESTED: feature b
...
or:
...
gdb.base/feature-b: UNTESTED: <feature b in more detail>
...

In the gdb testsuite there's an interpretation of UNTESTED as "not having
progressed sufficiently in executing an otherwise complete test-case to
generate a PASS or FAIL", which explains why it's used for say failure to
compile.

I suspect this usage is wrong, but again given the vague definition I'm not
sure.

Anyway, perhaps given the vague definition, I think we should eliminate the
usage of untested, by redefining it to:
...
proc untested { message } {
  error "untested: $message"
}
...
and fixing the errors.

Note that out of a total of:
...
$ egrep -c UNTESTED: gdb.sum=20
35
...
I have:
...
$ egrep -c UNTESTED:.*(not supported|unsupported) gdb.sum=20
18
...
which probably should use unsupported instead of untested.

--=20
You are receiving this mail because:
You are on the CC list for the bug.=