Re: [PATCH] gdb/testsuite: modernize gdb.base/maint.exp

[Andrew Burgess <aburgess@redhat.com>]

Keith Seitz via Gdb-patches <gdb-patches@sourceware.org> writes:

> On 6/24/22 08:22, Pedro Alves wrote:
>> On 2022-06-21 16:52, Andrew Burgess via Gdb-patches wrote:
>>>> +			    pass "$test_name 1"
>>> You could use:
>>>
>>>      pass "$gdb_test_name, pattern 1"
>>>
>>> here, and similar, with ', pattern 2' for the next 'pass' call.
>>>
>> 
>> How about
>> 
>>     pass "$gdb_test_name (pattern 1)"
>> 
>>     pass "$gdb_test_name (pattern 2)"
>> 
>> ?
>> 
>> The idea being that the text in the trail parens is not considered part of the
>> test name, so when comparing gdb.sum files and matching test names, that parens part
>> should be discarded.  Whether this test passes with pattern 1 or 2 should make
>> no difference IIUC, thus I think it should not be part of the (part that counts
>> as real) test name.
>> 
> I think it no surprise that I disdain this " (whatever)" idiom in test names. There
> is also a long-standing guideline in the wiki against this:
>
> https://sourceware.org/gdb/wiki/GDBTestcaseCookbook#Do_not_use_.22tail_parentheses.22_on_test_messages

Rather than being against what Pedro said, that wiki page aligns exactly
with Pedro's advice, to quote:

  "This is because our our regression analysis tools make the assumption
  that the text inserted between parentheses is extra information rather
  than part of the test name."

Thus:

  pass "$gdb_test_name (pattern 1)"
  pass "$gdb_test_name (pattern 2)"

Would, for the purpose of analysis, both be considered as:

  pass "$gdb_test_name"
  pass "$gdb_test_name"

So, if today I hit 'pattern 1' and tomorrow hit 'pattern 2' then
compared these results, I would see no changes, and all is good.

I did originally consider suggesting that Bruno make this change, but in
the end didn't because I would normally only use this for situations
where the output is known unstable, maybe something like this imaginary
situation:

  pass "$gdb_test_name (thread exited early)"
  
  pass "$gdb_test_name (thread exited late)"

where it will depend on kernel scheduling which pass you actually hit.

In contrast, I think the case Bruno is addressing depends on platform
specific information, so I wouldn't expect GDB to switch outputs between
separate test runs, and if it did, I think I'd want to know about it.

So, for me, I'd not go with the parenthesis, but I wouldn't block the
patch if they were used.

Thanks,
Andrew

>
> Has there been some unannounced, sekrit change to this policy?
>
> How is, e.g., "- pattern 1" any less desirable/informative than "(pattern 1)"?
>
> Keith