public inbox for
 help / color / mirror / Atom feed
From: Keith Seitz <>
To: Serhei Makarov <>, Bunsen <>
Subject: Re: Unusual test_outcome_map keys
Date: Wed, 11 Aug 2021 10:03:08 -0700	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On 8/10/21 2:15 PM, Serhei Makarov wrote:
> On Fri, Aug 6, 2021, at 3:35 PM, Keith Seitz via Bunsen wrote:
>> Hi,
>> While working on fixing up the gdb.sum parser, I've encountered
>> `test_outcome_map', defined in common/
>> This is defined:
>> test_outcome_map = {'PASS':'PASS', 'XPASS':'XPASS', 'IPASS':'IPASS',
>>                     'FAIL':'FAIL', 'KFAIL':'KFAIL', 'XFAIL':'XFAIL',
>>                     'ERROR: tcl error sourcing':'ERROR',
>>                     'UNRESOLVED':'UNRESOLVED'}
>> Can you fill me in on these two unusual keys?
> Sorry for the delay, I had to dig up a bit of backstory on this.
> I thought I saw IPASS in some test results, but I wasn't able to dig up
> any indication of where, so I may be wrong. It is certainly true that KPASS should be
> included in addition to KFAIL. I took the liberty of patching accordingly:

Thank you!

> I also changed 'ERROR: tcl error sourcing':'ERROR' to 'ERROR':'ERROR'
> although I'm less sure of this part. There may be some SystemTap testcases
> where the program under observation prints 'ERROR: <stuff>'
> and that shouldn't be treated as an error outcome for the testcase.
> But that could be a SystemTap-specific nit and I would solve it by using
> the method you suggest below to edit test_outcome_map for the SystemTap
> parser only.

GDB has a similar problem (or maybe it is GDB of which you're thinking?)
in gdb.guile tests. GDB itself outputs "ERROR:". :-(

>> I'm not a huge fan of modifying global data like this, but since
>> Bunsen doesn't run over multiple tools at the same time, this shouldn't
>> be an issue.
> No objections to either method for extending the table.
> In my patch, I opted to just add 'DUPLICATE' to test_outcome_map
> since it's extremely unlikely to interfere with other testsuites.

Great, thank you!


      reply	other threads:[~2021-08-11 17:03 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-06 19:35 Keith Seitz
2021-08-10 21:15 ` Serhei Makarov
2021-08-11 17:03   ` Keith Seitz [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).