* [RFC][gdb/testsuite] Add gdbfail/envfail aliases for kfail/xfail
@ 2020-04-30 6:43 Tom de Vries
2020-06-20 15:46 ` Pedro Alves
0 siblings, 1 reply; 2+ messages in thread
From: Tom de Vries @ 2020-04-30 6:43 UTC (permalink / raw)
To: gdb-patches; +Cc: Tom Tromey
Hi,
The difference between an xfail and a kfail (in the context of gdb testing) is
that:
- a kfail marks a problem in gdb
- an xfail marks a problem in the environment (compiler, libc, kernel)
However, this difference is not immediately obvious from the contrast between
"known" and "expected".
Add aliases gdbfail for kfail, and envfail for xfail, that are more intuitive
to use.
Tested on x86_64-linux.
Any comments?
Thanks,
- Tom
[gdb/testsuite] Add gdbfail/envfail aliases for kfail/xfail
gdb/testsuite/ChangeLog:
2020-04-30 Tom de Vries <tdevries@suse.de>
* lib/gdb.exp (gdbfail, envfail, setup_gdbfail, setup_envfail): New
proc.
* gdb.ada/packed_tagged.exp: Fix incorrectly marked kfails, by using
envfail.
---
gdb/testsuite/gdb.ada/packed_tagged.exp | 12 ++++++------
gdb/testsuite/lib/gdb.exp | 26 ++++++++++++++++++++++++++
2 files changed, 32 insertions(+), 6 deletions(-)
diff --git a/gdb/testsuite/gdb.ada/packed_tagged.exp b/gdb/testsuite/gdb.ada/packed_tagged.exp
index d6ee8454c5a..1e965cffda1 100644
--- a/gdb/testsuite/gdb.ada/packed_tagged.exp
+++ b/gdb/testsuite/gdb.ada/packed_tagged.exp
@@ -38,16 +38,16 @@ foreach_with_prefix scenario {all minimal} {
set pass_re \
"= \\(exists => true, value => 10\\)"
# There is a compiler bug that causes this output.
- set kfail_re \
+ set envfail_re \
"= \\(exists => true\\)"
gdb_test_multiple "print x" "" {
-re -wrap $pass_re {
pass $gdb_test_name
}
- -re -wrap $kfail_re {
+ -re -wrap $envfail_re {
if {$scenario == "minimal"} {
- setup_kfail "gnat compiler bug" *-*-*
+ setup_envfail "gnat compiler bug" *-*-*
}
fail $gdb_test_name
}
@@ -63,7 +63,7 @@ foreach_with_prefix scenario {all minimal} {
" end case;" \
"end record" ]
# There is a compiler bug that causes this output.
- set kfail_re \
+ set envfail_re \
[multi_line "type = record" \
" exists: (boolean|range false \\.\\. true);" \
" case \\? is" \
@@ -77,9 +77,9 @@ foreach_with_prefix scenario {all minimal} {
-re -wrap $pass_re {
pass $gdb_test_name
}
- -re -wrap $kfail_re {
+ -re -wrap $envfail_re {
if {$scenario == "minimal"} {
- setup_kfail "gnat compiler bug" *-*-*
+ setup_envfail "gnat compiler bug" *-*-*
}
fail $gdb_test_name
}
diff --git a/gdb/testsuite/lib/gdb.exp b/gdb/testsuite/lib/gdb.exp
index b72ce0cda7f..00efe280a43 100644
--- a/gdb/testsuite/lib/gdb.exp
+++ b/gdb/testsuite/lib/gdb.exp
@@ -7124,5 +7124,31 @@ proc debug_types { } {
return 0
}
+# Alias for kfail, to make the distinction between kfail and xfail clearer.
+
+proc gdbfail { args } {
+ kfail {*}$args
+}
+
+# Alias for xfail, to make the distinction between kfail and xfail clearer.
+
+proc envfail { args } {
+ xfail {*}$args
+}
+
+# Alias for setup_kfail, to make the distinction between kfail and xfail
+# clearer.
+
+proc setup_gdbfail { args } {
+ setup_kfail {*}$args
+}
+
+# Alias for setup_xfail, to make the distinction between kfail and xfail
+# clearer.
+
+proc setup_envfail { args } {
+ setup_xfail {*}$args
+}
+
# Always load compatibility stuff.
load_lib future.exp
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [RFC][gdb/testsuite] Add gdbfail/envfail aliases for kfail/xfail
2020-04-30 6:43 [RFC][gdb/testsuite] Add gdbfail/envfail aliases for kfail/xfail Tom de Vries
@ 2020-06-20 15:46 ` Pedro Alves
0 siblings, 0 replies; 2+ messages in thread
From: Pedro Alves @ 2020-06-20 15:46 UTC (permalink / raw)
To: Tom de Vries, gdb-patches; +Cc: Tom Tromey
On 4/30/20 7:43 AM, Tom de Vries wrote:
> Hi,
>
> The difference between an xfail and a kfail (in the context of gdb testing) is
> that:
> - a kfail marks a problem in gdb
> - an xfail marks a problem in the environment (compiler, libc, kernel)
>
> However, this difference is not immediately obvious from the contrast between
> "known" and "expected".
I memorized the the X in xfail as "external", not "expected".
So I always think of them as "known" vs "external", which makes the
difference much more obvious, IMO.
I propose that the documentation references the "external" mnemonic to help.
KFAIL and XFAIL are documented in testsuite/README:
XFAIL
Short for "expected failure", this indicates a known problem with the
environment. This could include limitations of the operating system,
compiler version, and other components.
Maybe some new entry on KFAIL/XFAIL at:
https://sourceware.org/gdb/wiki/GDBTestcaseCookbook
would help too.
>
> Add aliases gdbfail for kfail, and envfail for xfail, that are more intuitive
> to use.
>
> Tested on x86_64-linux.
>
> Any comments?
I think this isn't a good idea:
- We still end up with XFAIL / KFAIL in result logs anyway. People
still need to know the difference.
- It adds different ways to do the same thing. I.e., more things
to learn, when you still need to know about the old way anyway.
Thanks,
Pedro Alves
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-06-20 15:46 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-30 6:43 [RFC][gdb/testsuite] Add gdbfail/envfail aliases for kfail/xfail Tom de Vries
2020-06-20 15:46 ` Pedro Alves
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).