Re: Unusual test_outcome_map keys

[Keith Seitz <keiths@redhat.com>]

On 8/10/21 2:15 PM, Serhei Makarov wrote:
> On Fri, Aug 6, 2021, at 3:35 PM, Keith Seitz via Bunsen wrote:
>> Hi,
>>
>> While working on fixing up the gdb.sum parser, I've encountered
>> `test_outcome_map', defined in common/parse_dejagnu.py.
>>
>> This is defined:
>>
>> test_outcome_map = {'PASS':'PASS', 'XPASS':'XPASS', 'IPASS':'IPASS',
>>                     'FAIL':'FAIL', 'KFAIL':'KFAIL', 'XFAIL':'XFAIL',
>>                     'ERROR: tcl error sourcing':'ERROR',
>>                     'UNTESTED':'UNTESTED', 'UNSUPPORTED':'UNSUPPORTED',
>>                     'UNRESOLVED':'UNRESOLVED'}
>>
>> Can you fill me in on these two unusual keys?
> Sorry for the delay, I had to dig up a bit of backstory on this.
> I thought I saw IPASS in some test results, but I wasn't able to dig up
> any indication of where, so I may be wrong. It is certainly true that KPASS should be
> included in addition to KFAIL. I took the liberty of patching accordingly:
> https://sourceware.org/git/?p=bunsen.git;a=commit;h=763c2037ea61eafdf61df09ef8516278dd1ad12f

Thank you!

> I also changed 'ERROR: tcl error sourcing':'ERROR' to 'ERROR':'ERROR'
> although I'm less sure of this part. There may be some SystemTap testcases
> where the program under observation prints 'ERROR: <stuff>'
> and that shouldn't be treated as an error outcome for the testcase.
> But that could be a SystemTap-specific nit and I would solve it by using
> the method you suggest below to edit test_outcome_map for the SystemTap
> parser only.

GDB has a similar problem (or maybe it is GDB of which you're thinking?)
in gdb.guile tests. GDB itself outputs "ERROR:". :-(

>> I'm not a huge fan of modifying global data like this, but since
>> Bunsen doesn't run over multiple tools at the same time, this shouldn't
>> be an issue.
> No objections to either method for extending the table.
> In my patch, I opted to just add 'DUPLICATE' to test_outcome_map
> since it's extremely unlikely to interfere with other testsuites.

Great, thank you!

Keith