[PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Tom de Vries]

With test-case gdb.threads/attach-many-short-lived-threads.exp, I run into:
...
(gdb) attach 7773^M
Attaching to program: attach-many-short-lived-threads, process 7773^M
Cannot attach to lwp 7776: Operation not permitted (1)^M
(gdb) PASS: $exp: iter 1: attach
info threads^M
No threads.^M
(gdb) PASS: $exp: iter 1: no new threads
set breakpoint always-inserted on^M
(gdb) PASS: $exp: iter 1: set breakpoint always-inserted on
break break_fn^M
Breakpoint 1 at 0x400b4d: file attach-many-short-lived-threads.c, line 57.^M
(gdb) PASS: $exp: iter 1: break break_fn
continue^M
The program is not being run.^M
(gdb) FAIL: $exp: iter 1: break at break_fn: 1 \
  (the program is no longer running)
...

There's some code in the test-case dealing with a similar warning:
...
  -re "warning: Cannot attach to lwp $decimal: Operation not permitted" {
...

But since commit c6f7f9c80c3 ("Bail out of "attach" if a thread cannot be
traced"), the warning has been changed into an error.

Fix the FAIL by updating the test-case to expect an error instead of a
warning.

Tested on x86_64-linux.
---
 .../gdb.threads/attach-many-short-lived-threads.exp         | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp b/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp
index 3fdadf9827f..6bee0ad2e2c 100644
--- a/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp
+++ b/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp
@@ -74,7 +74,7 @@ proc test {} {
 		    # Seen when "set debug libthread_db" is on.
 		    exp_continue
 		}
-		-re "warning: Cannot attach to lwp $decimal: Operation not permitted" {
+		-re "Cannot attach to lwp $decimal: Operation not permitted" {
 		    # On Linux, PTRACE_ATTACH sometimes fails with
 		    # EPERM, even though /proc/PID/status indicates
 		    # the thread is running.
@@ -96,6 +96,10 @@ proc test {} {
 		}
 	    }
 
+	    if { $eperm } {
+		continue
+	    }
+
 	    # Sleep a bit and try updating the thread list.  We should
 	    # know about all threads already at this point.  If we see
 	    # "New Thread" or similar being output, then "attach" is

base-commit: f1870e2fadd400ff288da388497d50fdf7a5f2d5
-- 
2.35.3

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Tom Tromey]

>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:

Tom> But since commit c6f7f9c80c3 ("Bail out of "attach" if a thread cannot be
Tom> traced"), the warning has been changed into an error.

Tom> Fix the FAIL by updating the test-case to expect an error instead of a
Tom> warning.

Thanks, this looks good to me.
Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Carl Love]

Tom:

On 1/19/24 08:27, Tom Tromey wrote:
>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
> 
> Tom> But since commit c6f7f9c80c3 ("Bail out of "attach" if a thread cannot be
> Tom> traced"), the warning has been changed into an error.
> 
> Tom> Fix the FAIL by updating the test-case to expect an error instead of a
> Tom> warning.
> 
> Thanks, this looks good to me.
> Approved-By: Tom Tromey <tom@tromey.com>
> 
> Tom

The patch fixes the issues on my Power 9 box running "Ubuntu 22.04.3 LTS.
  
However, the Power 10 box running Fedora Linux 38 (Server Edition) still reports a lot of errors. I see the errors in the daily build on the machine.  I can reproduce the failures when running the test by itself with:

make check RUNTESTFLAGS='GDB=/home/carll/bin/gdb gdb.threads/attach-many-short-lived-threads.exp '

Here is some of the output from gdb.log:

The program is not being run.
(gdb) file /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads
Reading symbols from /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads...
(gdb) builtin_spawn /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads
attach 3385832
Attaching to program: /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads, process 3385832
Cannot attach to lwp 3513884: Operation not permitted (1)
(gdb) XFAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 1: attach (EPERM)
attach 3385832
Attaching to program: /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads, process 3385832
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: attach (timeout)
info threads
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: no new threads (timeout)
set breakpoint always-inserted on
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: set breakpoint always-inserted on (timeout)
break break_fn
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break break_fn (timeout)
continue
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 1 (timeout)
continue
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 2 (timeout)
continue
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 3 (timeout)
print again = 1
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: reset timer in the inferior (timeout)
print seconds_left
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: print seconds_left (timeout)
detach
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: detach (timeout)
set breakpoint always-inserted off
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: set breakpoint always-inserted off (timeout)
delete breakpoints
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: delete all breakpoints in delete_breakpoints (timeout)
ERROR: breakpoints not deleted
attach 3385832
UNRESOLVED: gdb.threads/attach-many-short-lived-threads.exp: iter 3: attach (timeout)
info threads
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: no new threads (timeout)
set breakpoint always-inserted on
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: set breakpoint always-inserted on (timeout)
break break_fn
FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: break break_fn (timeout)
continue

etc.

closing exp10
waiting for exp10
testcase /home/carll/GDB/build-current/gdb/testsuite/../../../binutils-gdb-current/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp completed in 1910 seconds

                 === gdb Summary ===

# of expected passes            1
# of unexpected failures        98
# of expected failures          1
# of unresolved testcases       8



Not sure if the failures are specific to the Distro/OS or not.  But I will try testing on a Power 10 system with other distros and let you know.  Please let me know if you have any ideas on how to fix these errors on Power 10.

                         Carl

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Carl Love]

Tom:

On 1/24/24 10:51, Carl Love wrote:
> Tom:
> 
> On 1/19/24 08:27, Tom Tromey wrote:
>>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
>>
>> Tom> But since commit c6f7f9c80c3 ("Bail out of "attach" if a thread cannot be
>> Tom> traced"), the warning has been changed into an error.
>>
>> Tom> Fix the FAIL by updating the test-case to expect an error instead of a
>> Tom> warning.
>>
>> Thanks, this looks good to me.
>> Approved-By: Tom Tromey <tom@tromey.com>
>>
>> Tom
> 
> The patch fixes the issues on my Power 9 box running "Ubuntu 22.04.3 LTS.
> 
> However, the Power 10 box running Fedora Linux 38 (Server Edition) still reports a lot of errors. I see the errors in the daily build on the machine.  I can reproduce the failures when running the test by itself with:
> 
> make check RUNTESTFLAGS='GDB=/home/carll/bin/gdb gdb.threads/attach-many-short-lived-threads.exp '
> 
> Here is some of the output from gdb.log:
> 
> The program is not being run.
> (gdb) file /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads
> Reading symbols from /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads...
> (gdb) builtin_spawn /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads
> attach 3385832
> Attaching to program: /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads, process 3385832
> Cannot attach to lwp 3513884: Operation not permitted (1)
> (gdb) XFAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 1: attach (EPERM)
> attach 3385832
> Attaching to program: /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads, process 3385832
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: attach (timeout)
> info threads
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: no new threads (timeout)
> set breakpoint always-inserted on
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: set breakpoint always-inserted on (timeout)
> break break_fn
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break break_fn (timeout)
> continue
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 1 (timeout)
> continue
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 2 (timeout)
> continue
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 3 (timeout)
> print again = 1
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: reset timer in the inferior (timeout)
> print seconds_left
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: print seconds_left (timeout)
> detach
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: detach (timeout)
> set breakpoint always-inserted off
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: set breakpoint always-inserted off (timeout)
> delete breakpoints
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: delete all breakpoints in delete_breakpoints (timeout)
> ERROR: breakpoints not deleted
> attach 3385832
> UNRESOLVED: gdb.threads/attach-many-short-lived-threads.exp: iter 3: attach (timeout)
> info threads
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: no new threads (timeout)
> set breakpoint always-inserted on
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: set breakpoint always-inserted on (timeout)
> break break_fn
> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: break break_fn (timeout)
> continue
> 
> etc.
> 
> closing exp10
> waiting for exp10
> testcase /home/carll/GDB/build-current/gdb/testsuite/../../../binutils-gdb-current/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp completed in 1910 seconds
> 
>                  === gdb Summary ===
> 
> # of expected passes            1
> # of unexpected failures        98
> # of expected failures          1
> # of unresolved testcases       8
> 
> 
> 
> Not sure if the failures are specific to the Distro/OS or not.  But I will try testing on a Power 10 system with other distros and let you know.  Please let me know if you have any ideas on how to fix these errors on Power 10.

I tried running the test on various Power 10 systems with varying results:

Ubuntu 22.04.3 LTS

                 === gdb Summary ===

# of expected passes            34
# of expected failures          7

---------------------------
Red Hat Enterprise Linux 9.3 (Plow)

# of unsupported tests          1    Seems really odd, will have to look into this more

--------------------------
SUSE Linux Enterprise Server 15 SP5
                 === gdb Summary ===

# of expected passes            109

--------------------------
Fedora Linux 38 (Server Edition)

                 === gdb Summary ===   I get inconsistent results

# of expected passes            56
# of unexpected failures        32
# of expected failures          2
# of unresolved testcases       2

                 === gdb Summary ===

# of expected passes            109    I have run this test multiple times and had failures.
                                        Now it is running without any errors???  Not sure what
                                        has changed.

I will keep an eye on this to see if things change again.

                  Carl

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Carl Love]

Tom:

The test continues to be problematic on Power 10.  The number of failures seem to vary
from run to run.  I do see the number of failures on Power 9 vary, but most of the time the test passes.

                         Carl 

On 1/24/24 12:47, Carl Love wrote:
> Tom:
> 
> On 1/24/24 10:51, Carl Love wrote:
>> Tom:
>>
>> On 1/19/24 08:27, Tom Tromey wrote:
>>>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
>>>
>>> Tom> But since commit c6f7f9c80c3 ("Bail out of "attach" if a thread cannot be
>>> Tom> traced"), the warning has been changed into an error.
>>>
>>> Tom> Fix the FAIL by updating the test-case to expect an error instead of a
>>> Tom> warning.
>>>
>>> Thanks, this looks good to me.
>>> Approved-By: Tom Tromey <tom@tromey.com>
>>>
>>> Tom
>>
>> The patch fixes the issues on my Power 9 box running "Ubuntu 22.04.3 LTS.
>>
>> However, the Power 10 box running Fedora Linux 38 (Server Edition) still reports a lot of errors. I see the errors in the daily build on the machine.  I can reproduce the failures when running the test by itself with:
>>
>> make check RUNTESTFLAGS='GDB=/home/carll/bin/gdb gdb.threads/attach-many-short-lived-threads.exp '
>>
>> Here is some of the output from gdb.log:
>>
>> The program is not being run.
>> (gdb) file /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads
>> Reading symbols from /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads...
>> (gdb) builtin_spawn /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads
>> attach 3385832
>> Attaching to program: /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads, process 3385832
>> Cannot attach to lwp 3513884: Operation not permitted (1)
>> (gdb) XFAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 1: attach (EPERM)
>> attach 3385832
>> Attaching to program: /home/carll/GDB/build-current/gdb/testsuite/outputs/gdb.threads/attach-many-short-lived-threads/attach-many-short-lived-threads, process 3385832
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: attach (timeout)
>> info threads
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: no new threads (timeout)
>> set breakpoint always-inserted on
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: set breakpoint always-inserted on (timeout)
>> break break_fn
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break break_fn (timeout)
>> continue
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 1 (timeout)
>> continue
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 2 (timeout)
>> continue
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: break at break_fn: 3 (timeout)
>> print again = 1
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: reset timer in the inferior (timeout)
>> print seconds_left
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: print seconds_left (timeout)
>> detach
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: detach (timeout)
>> set breakpoint always-inserted off
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: set breakpoint always-inserted off (timeout)
>> delete breakpoints
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 2: delete all breakpoints in delete_breakpoints (timeout)
>> ERROR: breakpoints not deleted
>> attach 3385832
>> UNRESOLVED: gdb.threads/attach-many-short-lived-threads.exp: iter 3: attach (timeout)
>> info threads
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: no new threads (timeout)
>> set breakpoint always-inserted on
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: set breakpoint always-inserted on (timeout)
>> break break_fn
>> FAIL: gdb.threads/attach-many-short-lived-threads.exp: iter 3: break break_fn (timeout)
>> continue
>>
>> etc.
>>
>> closing exp10
>> waiting for exp10
>> testcase /home/carll/GDB/build-current/gdb/testsuite/../../../binutils-gdb-current/gdb/testsuite/gdb.threads/attach-many-short-lived-threads.exp completed in 1910 seconds
>>
>>                  === gdb Summary ===
>>
>> # of expected passes            1
>> # of unexpected failures        98
>> # of expected failures          1
>> # of unresolved testcases       8
>>
>>
>>
>> Not sure if the failures are specific to the Distro/OS or not.  But I will try testing on a Power 10 system with other distros and let you know.  Please let me know if you have any ideas on how to fix these errors on Power 10.
> 
> I tried running the test on various Power 10 systems with varying results:
> 
> Ubuntu 22.04.3 LTS
> 
>                 === gdb Summary ===
> 
> # of expected passes            34
> # of expected failures          7
> 
> ---------------------------
> Red Hat Enterprise Linux 9.3 (Plow)
> 
> # of unsupported tests          1    Seems really odd, will have to look into this more
> 
> --------------------------
> SUSE Linux Enterprise Server 15 SP5
>                 === gdb Summary ===
> 
> # of expected passes            109
> 
> --------------------------
> Fedora Linux 38 (Server Edition)
> 
>                 === gdb Summary ===   I get inconsistent results
> 
> # of expected passes            56
> # of unexpected failures        32
> # of expected failures          2
> # of unresolved testcases       2
> 
>                 === gdb Summary ===
> 
> # of expected passes            109    I have run this test multiple times and had failures.
>                                        Now it is running without any errors???  Not sure what
>                                        has changed.
> 
> I will keep an eye on this to see if things change again.
> 
>                  Carl
> 
> 
> 
> 
> 
> 
> 
> 

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Tom Tromey]

>>>>> "Carl" == Carl Love <cel@linux.ibm.com> writes:

Carl> The test continues to be problematic on Power 10.  The number of
Carl> failures seem to vary from run to run.  I do see the number of
Carl> failures on Power 9 vary, but most of the time the test passes.

Would you mind filing a bug and attaching a gdb.log from a failing run?
You can CC and/or assign it to me.

Tom

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Carl Love]

Tom:

On 1/29/24 09:04, Tom Tromey wrote:
>>>>>> "Carl" == Carl Love <cel@linux.ibm.com> writes:
> 
> Carl> The test continues to be problematic on Power 10.  The number of
> Carl> failures seem to vary from run to run.  I do see the number of
> Carl> failures on Power 9 vary, but most of the time the test passes.
> 
> Would you mind filing a bug and attaching a gdb.log from a failing run?
> You can CC and/or assign it to me.
> 

I made a couple of runs on my Power 10 box.  

I created a new bugzilla and I attached both log files to the bugzilla:

https://sourceware.org/bugzilla/show_bug.cgi?id=31312

The hope is that will help show the variability in the runs.  

I was not able to set you and the assignee.

Please let me know what I can do to help resolve the issues with the test on Power.  Thanks.

                      Carl 

Re: [PATCH] [gdb/testsuite] Update xfail in gdb.threads/attach-many-short-lived-threads.exp

[Carl Love]

Tom:

So I have spent some more time playing with this test.  I have played with adjusting the timeout with no clear improvement in the reliability of the test.

I see that once we hit the EPERM failure, the test skips the tests for the rest of that iteration and then tries to attach again to the same testpid on the next iteration of the test loop.  On Power 10, this results in a timeout on the attach and all the rest of the gdb commands timeout for all subsequent test loop iterations.  So depending on which iteration the error occurs on you get a different number of failures from run to run.

So, it seems like we need handle the EPERM error differently from what is currently being done.  I can think of
three possible ways to handle the error, as I put into the latest update of the issue.  If you have any other ideas on how we could handle the EPERM failures please let me know.  My third suggestion does seem to fix the test but I have to say I am not entirely happy with that fix.

https://sourceware.org/bugzilla/show_bug.cgi?id=31312

Please add your thoughts on my solution and any additional solutions to the bugzilla.  Thanks for your help on this issue.

                        Carl