* [PATCH] testsuite, jit: Stabilize error output.
@ 2024-01-16 11:12 Iain Sandoe
2024-01-22 8:24 ` Iain Sandoe
2024-01-24 18:46 ` David Malcolm
0 siblings, 2 replies; 3+ messages in thread
From: Iain Sandoe @ 2024-01-16 11:12 UTC (permalink / raw)
To: dmalcolm, gcc-patches
Tested on x86_64, i686 Darwin, x86_64 Linux,
OK for trunk? When?
thanks
Iain
--- 8< ---
Currently when a test fails, we print out a lot of information,
this includes items that are not stable between invocations (e.g.
the PID for the executable). That makes automated comparisons
between test runs flag any persistent fails as new ones each time
which is not usually what is wanted.
This patch amends the error output to drop the variable portion
of the message and retain items that should only change if the
failure mode changes.
gcc/testsuite/ChangeLog:
* jit.dg/jit.exp: Filter error output to remove per-run
variable content.
Signed-off-by: Iain Sandoe <iain@sandoe.co.uk>
---
gcc/testsuite/jit.dg/jit.exp | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp
index 286cfa8192a..893ff5f6dd0 100644
--- a/gcc/testsuite/jit.dg/jit.exp
+++ b/gcc/testsuite/jit.dg/jit.exp
@@ -94,25 +94,34 @@ proc parse_valgrind_logfile {name logfile} {
# unexpected exits.
proc verify_exit_status { executable wres } {
- lassign $wres pid spawnid os_error_flag value
+ set extra [lassign $wres pid spawnid os_error_flag value]
verbose "pid: $pid" 3
verbose "spawnid: $spawnid" 3
verbose "os_error_flag: $os_error_flag" 3
verbose "value: $value" 3
# Detect segfaults etc:
- if { [llength $wres] > 4 } {
- if { [lindex $wres 4] == "CHILDKILLED" } {
- fail "$executable killed: $wres"
+ set len [llength $extra]
+ if { $len >= 1 } {
+ if { [lindex $extra 0] == "CHILDKILLED" } {
+ set reason "Unknown Reason"
+ set detail "No Details"
+ if { $len >= 2 } {
+ set reason [lindex $extra 1]
+ if { $len >= 3 } {
+ set detail [lindex $extra 2]
+ }
+ }
+ fail "$executable killed: $reason $detail"
return
}
}
if { $os_error_flag != 0 } {
- fail "$executable: OS error: $wres"
+ fail "$executable: OS error: $os_error_flag $extra"
return
}
if { $value != 0 } {
- fail "$executable: non-zero exit code: $wres"
+ fail "$executable: non-zero exit code: $value $extra"
return
}
pass "$executable exited cleanly"
--
2.39.2 (Apple Git-143)
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] testsuite, jit: Stabilize error output.
2024-01-16 11:12 [PATCH] testsuite, jit: Stabilize error output Iain Sandoe
@ 2024-01-22 8:24 ` Iain Sandoe
2024-01-24 18:46 ` David Malcolm
1 sibling, 0 replies; 3+ messages in thread
From: Iain Sandoe @ 2024-01-22 8:24 UTC (permalink / raw)
To: dmalcolm; +Cc: GCC Patches
gentle ping,
with the increasing use of CI, it seems an idea to tackle this sooner rather than later.
thanks
Iain
> On 16 Jan 2024, at 11:12, Iain Sandoe <iains.gcc@gmail.com> wrote:
>
> Tested on x86_64, i686 Darwin, x86_64 Linux,
> OK for trunk? When?
> thanks
> Iain
>
> --- 8< ---
>
> Currently when a test fails, we print out a lot of information,
> this includes items that are not stable between invocations (e.g.
> the PID for the executable). That makes automated comparisons
> between test runs flag any persistent fails as new ones each time
> which is not usually what is wanted.
>
> This patch amends the error output to drop the variable portion
> of the message and retain items that should only change if the
> failure mode changes.
>
> gcc/testsuite/ChangeLog:
>
> * jit.dg/jit.exp: Filter error output to remove per-run
> variable content.
>
> Signed-off-by: Iain Sandoe <iain@sandoe.co.uk>
> ---
> gcc/testsuite/jit.dg/jit.exp | 21 +++++++++++++++------
> 1 file changed, 15 insertions(+), 6 deletions(-)
>
> diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp
> index 286cfa8192a..893ff5f6dd0 100644
> --- a/gcc/testsuite/jit.dg/jit.exp
> +++ b/gcc/testsuite/jit.dg/jit.exp
> @@ -94,25 +94,34 @@ proc parse_valgrind_logfile {name logfile} {
> # unexpected exits.
>
> proc verify_exit_status { executable wres } {
> - lassign $wres pid spawnid os_error_flag value
> + set extra [lassign $wres pid spawnid os_error_flag value]
> verbose "pid: $pid" 3
> verbose "spawnid: $spawnid" 3
> verbose "os_error_flag: $os_error_flag" 3
> verbose "value: $value" 3
>
> # Detect segfaults etc:
> - if { [llength $wres] > 4 } {
> - if { [lindex $wres 4] == "CHILDKILLED" } {
> - fail "$executable killed: $wres"
> + set len [llength $extra]
> + if { $len >= 1 } {
> + if { [lindex $extra 0] == "CHILDKILLED" } {
> + set reason "Unknown Reason"
> + set detail "No Details"
> + if { $len >= 2 } {
> + set reason [lindex $extra 1]
> + if { $len >= 3 } {
> + set detail [lindex $extra 2]
> + }
> + }
> + fail "$executable killed: $reason $detail"
> return
> }
> }
> if { $os_error_flag != 0 } {
> - fail "$executable: OS error: $wres"
> + fail "$executable: OS error: $os_error_flag $extra"
> return
> }
> if { $value != 0 } {
> - fail "$executable: non-zero exit code: $wres"
> + fail "$executable: non-zero exit code: $value $extra"
> return
> }
> pass "$executable exited cleanly"
> --
> 2.39.2 (Apple Git-143)
>
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] testsuite, jit: Stabilize error output.
2024-01-16 11:12 [PATCH] testsuite, jit: Stabilize error output Iain Sandoe
2024-01-22 8:24 ` Iain Sandoe
@ 2024-01-24 18:46 ` David Malcolm
1 sibling, 0 replies; 3+ messages in thread
From: David Malcolm @ 2024-01-24 18:46 UTC (permalink / raw)
To: iain, gcc-patches
On Tue, 2024-01-16 at 11:12 +0000, Iain Sandoe wrote:
> Tested on x86_64, i686 Darwin, x86_64 Linux,
> OK for trunk? When?
> thanks
> Iain
Thanks; looks good to me for trunk.
Given that the scope is just the jit testsuite and that you've tested
it on 3 configurations (and presumably made use of this when debugging
the other issue), I think this is OK to go in now.
Dave
>
> --- 8< ---
>
> Currently when a test fails, we print out a lot of information,
> this includes items that are not stable between invocations (e.g.
> the PID for the executable). That makes automated comparisons
> between test runs flag any persistent fails as new ones each time
> which is not usually what is wanted.
>
> This patch amends the error output to drop the variable portion
> of the message and retain items that should only change if the
> failure mode changes.
>
> gcc/testsuite/ChangeLog:
>
> * jit.dg/jit.exp: Filter error output to remove per-run
> variable content.
>
> Signed-off-by: Iain Sandoe <iain@sandoe.co.uk>
> ---
> gcc/testsuite/jit.dg/jit.exp | 21 +++++++++++++++------
> 1 file changed, 15 insertions(+), 6 deletions(-)
>
> diff --git a/gcc/testsuite/jit.dg/jit.exp
> b/gcc/testsuite/jit.dg/jit.exp
> index 286cfa8192a..893ff5f6dd0 100644
> --- a/gcc/testsuite/jit.dg/jit.exp
> +++ b/gcc/testsuite/jit.dg/jit.exp
> @@ -94,25 +94,34 @@ proc parse_valgrind_logfile {name logfile} {
> # unexpected exits.
>
> proc verify_exit_status { executable wres } {
> - lassign $wres pid spawnid os_error_flag value
> + set extra [lassign $wres pid spawnid os_error_flag value]
> verbose "pid: $pid" 3
> verbose "spawnid: $spawnid" 3
> verbose "os_error_flag: $os_error_flag" 3
> verbose "value: $value" 3
>
> # Detect segfaults etc:
> - if { [llength $wres] > 4 } {
> - if { [lindex $wres 4] == "CHILDKILLED" } {
> - fail "$executable killed: $wres"
> + set len [llength $extra]
> + if { $len >= 1 } {
> + if { [lindex $extra 0] == "CHILDKILLED" } {
> + set reason "Unknown Reason"
> + set detail "No Details"
> + if { $len >= 2 } {
> + set reason [lindex $extra 1]
> + if { $len >= 3 } {
> + set detail [lindex $extra 2]
> + }
> + }
> + fail "$executable killed: $reason $detail"
> return
> }
> }
> if { $os_error_flag != 0 } {
> - fail "$executable: OS error: $wres"
> + fail "$executable: OS error: $os_error_flag $extra"
> return
> }
> if { $value != 0 } {
> - fail "$executable: non-zero exit code: $wres"
> + fail "$executable: non-zero exit code: $value $extra"
> return
> }
> pass "$executable exited cleanly"
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-01-24 18:46 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-01-16 11:12 [PATCH] testsuite, jit: Stabilize error output Iain Sandoe
2024-01-22 8:24 ` Iain Sandoe
2024-01-24 18:46 ` David Malcolm
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).