Re: [PATCH] gdb/testsuite: Improve testing of GDB's completion functions

[Tom de Vries <tdevries@suse.de>]

On 7/25/23 17:40, Bruno Larsen wrote:
> On 15/07/2023 14:13, Tom de Vries wrote:
>> On 2/22/23 10:11, Bruno Larsen via Gdb-patches wrote:
>>> When looking at some failures of gdb.linespec/cp-completion-aliases.exp,
>>> I noticed that when a completion test will fail, it always fails with a
>>> timeout.  This is because most completion tests use gdb_test_multiple
>>> and only add a check for the correct output.  This commit adds new
>>> options for both, tab and command completion.
>>>
>>> For command completion, the new option will check if the prompt was
>>> printed, and fail in this case. This is enough to know that the test has
>>> failed because the check comes after the PASS path. For tab completion,
>>> we have to check if GDB outputted more than just the input line, because
>>> sometimes GDB would have printed a partial line before finishing with
>>> the correct completion.
>>
>> This causes quite a few regressions with check-read1.
>>
>> For instance:
>> ...
>> (gdb) break baz(int, FAIL: gdb.cp/cpcompletion.exp: tab complete 
>> "break baz(int"
>> double) Quit^M
>> (gdb)
>> ...
> Hi! Sorry for taking so long to respond. I'd appreciate some help in 
> solving, if you have the time.
>>
>> Thanks,
>> - Tom
>>
>>> ---
>>>   gdb/testsuite/lib/completion-support.exp | 16 ++++++++++++++++
>>>   1 file changed, 16 insertions(+)
>>>
>>> diff --git a/gdb/testsuite/lib/completion-support.exp 
>>> b/gdb/testsuite/lib/completion-support.exp
>>> index bf9c5ad352c..275f8874f15 100644
>>> --- a/gdb/testsuite/lib/completion-support.exp
>>> +++ b/gdb/testsuite/lib/completion-support.exp
>>> @@ -94,6 +94,9 @@ proc test_gdb_complete_tab_none { line } {
>>>       -re "^$line_re$completion::bell_re$" {
>>>           pass "$test"
>>>       }
>>> +    -re "$line_re\[^ \]+ $" {
>>> +        fail "$test"
>>> +    }
>>>       }
>>>         clear_input_line $test
>>> @@ -108,11 +111,15 @@ proc test_gdb_complete_tab_unique { input_line 
>>> complete_line_re append_char_re }
>>>         set test "tab complete \"$input_line\""
>>>       send_gdb "$input_line\t"
>>> +    set partial_complete [string_to_regexp $input_line]
>>>       set res 1
>>>       gdb_test_multiple "" "$test" {
>>>       -re "^$complete_line_re$append_char_re$" {
>>>           pass "$test"
>>>       }
>>> +    -re "$partial_complete\[^ \]+ $" {
>>> +        fail "$test"
>>> +    }
> 
> This is the specific change that causes the failures. The thinking 
> behind it was that if we receive more characters, but not the whole 
> complete_line, we got a failure. Something like this could detect if we 
> have a unique - but wrong - suggestion or multiple options. This way it 
> doesn't have to go to timeout every time, because it was making clang 
> testing take too long.
> 
> Is there any other way to detect if GDB is done with the suggestion? Or 
> can we detect that read1 is being used, so this gets special cased?
> 

The purpose of read1 is to reliably exercise FAILs in the test-suite, 
that are otherwise only occasionally occurring (see also "Race 
detection" in gdb/testsuite/README).

It's typically a test-case problem where it passes or fails depending on 
how fast the input arrives.

When read1 finds such a FAIL, we want to fix it because we want 
deterministic results.

So, I'd say the relevant question is: did the change make the related 
test-cases racy, and does special casing try to hide the race?

Thanks,
- Tom