Re: [PATCH] gdb/testsuite: Improve testing of GDB's completion functions

[Guinevere Larsen <blarsen@redhat.com>]

On 15/08/2023 07:45, Tom de Vries wrote:
> On 7/25/23 17:40, Bruno Larsen wrote:
>> On 15/07/2023 14:13, Tom de Vries wrote:
>>> On 2/22/23 10:11, Bruno Larsen via Gdb-patches wrote:
>>>> When looking at some failures of 
>>>> gdb.linespec/cp-completion-aliases.exp,
>>>> I noticed that when a completion test will fail, it always fails 
>>>> with a
>>>> timeout.  This is because most completion tests use gdb_test_multiple
>>>> and only add a check for the correct output.  This commit adds new
>>>> options for both, tab and command completion.
>>>>
>>>> For command completion, the new option will check if the prompt was
>>>> printed, and fail in this case. This is enough to know that the 
>>>> test has
>>>> failed because the check comes after the PASS path. For tab 
>>>> completion,
>>>> we have to check if GDB outputted more than just the input line, 
>>>> because
>>>> sometimes GDB would have printed a partial line before finishing with
>>>> the correct completion.
>>>
>>> This causes quite a few regressions with check-read1.
>>>
>>> For instance:
>>> ...
>>> (gdb) break baz(int, FAIL: gdb.cp/cpcompletion.exp: tab complete 
>>> "break baz(int"
>>> double) Quit^M
>>> (gdb)
>>> ...
>> Hi! Sorry for taking so long to respond. I'd appreciate some help in 
>> solving, if you have the time.
>>>
>>> Thanks,
>>> - Tom
>>>
>>>> ---
>>>>   gdb/testsuite/lib/completion-support.exp | 16 ++++++++++++++++
>>>>   1 file changed, 16 insertions(+)
>>>>
>>>> diff --git a/gdb/testsuite/lib/completion-support.exp 
>>>> b/gdb/testsuite/lib/completion-support.exp
>>>> index bf9c5ad352c..275f8874f15 100644
>>>> --- a/gdb/testsuite/lib/completion-support.exp
>>>> +++ b/gdb/testsuite/lib/completion-support.exp
>>>> @@ -94,6 +94,9 @@ proc test_gdb_complete_tab_none { line } {
>>>>       -re "^$line_re$completion::bell_re$" {
>>>>           pass "$test"
>>>>       }
>>>> +    -re "$line_re\[^ \]+ $" {
>>>> +        fail "$test"
>>>> +    }
>>>>       }
>>>>         clear_input_line $test
>>>> @@ -108,11 +111,15 @@ proc test_gdb_complete_tab_unique { 
>>>> input_line complete_line_re append_char_re }
>>>>         set test "tab complete \"$input_line\""
>>>>       send_gdb "$input_line\t"
>>>> +    set partial_complete [string_to_regexp $input_line]
>>>>       set res 1
>>>>       gdb_test_multiple "" "$test" {
>>>>       -re "^$complete_line_re$append_char_re$" {
>>>>           pass "$test"
>>>>       }
>>>> +    -re "$partial_complete\[^ \]+ $" {
>>>> +        fail "$test"
>>>> +    }
>>
>> This is the specific change that causes the failures. The thinking 
>> behind it was that if we receive more characters, but not the whole 
>> complete_line, we got a failure. Something like this could detect if 
>> we have a unique - but wrong - suggestion or multiple options. This 
>> way it doesn't have to go to timeout every time, because it was 
>> making clang testing take too long.
>>
>> Is there any other way to detect if GDB is done with the suggestion? 
>> Or can we detect that read1 is being used, so this gets special cased?
>>
>
> The purpose of read1 is to reliably exercise FAILs in the test-suite, 
> that are otherwise only occasionally occurring (see also "Race 
> detection" in gdb/testsuite/README).
>
> It's typically a test-case problem where it passes or fails depending 
> on how fast the input arrives.
>
> When read1 finds such a FAIL, we want to fix it because we want 
> deterministic results.
>
> So, I'd say the relevant question is: did the change make the related 
> test-cases racy, and does special casing try to hide the race?
>
Yeah, I spoke to Andrew off-list and he explained this to me. The test 
itself wasn't racy on a light machine, but could be if it was under 
heavy load or if the expected output was too big. I have sent a v2 that 
fixes this without special casing: 
https://sourceware.org/pipermail/gdb-patches/2023-August/201361.html

-- 
Cheers,
Guinevere Larsen
She/Her/Hers