Re: Unusual test_outcome_map keys

["Serhei Makarov" <me@serhei.io>]

On Fri, Aug 6, 2021, at 3:35 PM, Keith Seitz via Bunsen wrote:
> Hi,
> 
> While working on fixing up the gdb.sum parser, I've encountered
> `test_outcome_map', defined in common/parse_dejagnu.py.
> 
> This is defined:
> 
> test_outcome_map = {'PASS':'PASS', 'XPASS':'XPASS', 'IPASS':'IPASS',
>                     'FAIL':'FAIL', 'KFAIL':'KFAIL', 'XFAIL':'XFAIL',
>                     'ERROR: tcl error sourcing':'ERROR',
>                     'UNTESTED':'UNTESTED', 'UNSUPPORTED':'UNSUPPORTED',
>                     'UNRESOLVED':'UNRESOLVED'}
> 
> Can you fill me in on these two unusual keys?
Sorry for the delay, I had to dig up a bit of backstory on this.
I thought I saw IPASS in some test results, but I wasn't able to dig up
any indication of where, so I may be wrong. It is certainly true that KPASS should be
included in addition to KFAIL. I took the liberty of patching accordingly:
https://sourceware.org/git/?p=bunsen.git;a=commit;h=763c2037ea61eafdf61df09ef8516278dd1ad12f

I also changed 'ERROR: tcl error sourcing':'ERROR' to 'ERROR':'ERROR'
although I'm less sure of this part. There may be some SystemTap testcases
where the program under observation prints 'ERROR: <stuff>'
and that shouldn't be treated as an error outcome for the testcase.
But that could be a SystemTap-specific nit and I would solve it by using
the method you suggest below to edit test_outcome_map for the SystemTap
parser only.

> What's the preferred method for adding members to this map?
> 
> In GDB, we have a special "DUPLICATE" outcome, and right now my plan
> is to simply add definitions for this (and any other additions) to our
> own gdb.parse_dejagnu_sum function, e.g.,
> 
>     test_outcome_map['DUPLICATE'] = 'DUPLICATE'
> 
> I'm not a huge fan of modifying global data like this, but since
> Bunsen doesn't run over multiple tools at the same time, this shouldn't
> be an issue.
No objections to either method for extending the table.
In my patch, I opted to just add 'DUPLICATE' to test_outcome_map
since it's extremely unlikely to interfere with other testsuites.

-- 
All the best,
    Serhei
    http://serhei.io