* [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time
@ 2018-08-11 0:34 agentzh at gmail dot com
2018-08-11 0:39 ` [Bug dyninst/23513] " fche at redhat dot com
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: agentzh at gmail dot com @ 2018-08-11 0:34 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
Bug ID: 23513
Summary: Dyninst runtime kills the target process when
attaching to the target process for a second time
Product: systemtap
Version: unspecified
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: dyninst
Assignee: systemtap at sourceware dot org
Reporter: agentzh at gmail dot com
Target Milestone: ---
Created attachment 11178
--> https://sourceware.org/bugzilla/attachment.cgi?id=11178&action=edit
stap-report output on the same box
Consider the following simple forever-running C program:
```C
#include <unistd.h>
int main(void) {
while (1) {
usleep (1000);
}
return 0;
}
```
Compile it and generate `./a.out`:
```
gcc -g -Wall a.c
```
Then start it in one terminal (let's call it terminal A):
```
$ ./a.out
```
It hangs as expected.
Then try running the stap dyninst command like below in another terminal (let's
say, terminal B):
```
$ /opt/stap/bin/stap -e 'probe process("/lib64/libc.so.6").function("usleep") {
println("hi") exit() }' -x `pgrep -f './a.out'` --dyninst
hi
```
It works as expected. And in terminal A, the `./a.out` program is still
running. So far so good.
But now as soon as we run the `stap` command above again, we got the following
error on terminal B:
```
$ /opt/stap/bin/stap -e 'probe process("/lib64/libc.so.6").function("usleep") {
println("hi") exit() }' -x `pgrep -f './a.out'` --dyninst
--FATAL-- #68: Dyninst was unable to attach to the specified process
--FATAL-- #68: BPatch.C[1047]: no process 40758 defined in procsByPid
stapdyn: ERROR: Couldn't attach to the target process
WARNING: /opt/stap/bin/stapdyn exited with status: 1
Pass 5: run failed. [man error::pass5]
```
And in terminal A, the `./a.out` process is already killed too:
```
agentzh@work2 ~ $ ./a.out
Killed
```
This can also be reproduced by a `probe begin` one-liner:
```
agentzh@work2 ~ $ /opt/stap/bin/stap -e 'probe begin { println("hi") exit() }'
-x `pgrep -f './a.out'` --dyninst
hi
agentzh@work2 ~ $ /opt/stap/bin/stap -e 'probe begin { println("hi") exit() }'
-x `pgrep -f './a.out'` --dyninst
--FATAL-- #68: Dyninst was unable to attach to the specified process
--FATAL-- #68: BPatch.C[1047]: no process 40816 defined in procsByPid
stapdyn: ERROR: Couldn't attach to the target process
WARNING: /opt/stap/bin/stapdyn exited with status: 1
Pass 5: run failed. [man error::pass5]
```
I'm using the 3.3 release tag of systemtap on Fedora 27 x86_64:
```
$ /opt/stap/bin/stap --version
Systemtap translator/driver (version 3.3/0.173, commit
release-3.3-0-g48867d1cface)
Copyright (C) 2005-2018 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.18 ... 4.18-rc0
enabled features: BOOST_STRING_REF DYNINST BPF NLS NSS
```
Also attached the stap-report output for the same box.
Am I doing anything wrong here?
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug dyninst/23513] Dyninst runtime kills the target process when attaching to the target process for a second time
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
@ 2018-08-11 0:39 ` fche at redhat dot com
2018-08-11 0:46 ` agentzh at gmail dot com
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: fche at redhat dot com @ 2018-08-11 0:39 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
Frank Ch. Eigler <fche at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |fche at redhat dot com
--- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
The second dyninst-based probe's failure is expected: there can be only one
concurrent "mutator" for a given target process. It is not right that the
process should be killed though, that sounds like a bug.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug dyninst/23513] Dyninst runtime kills the target process when attaching to the target process for a second time
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
2018-08-11 0:39 ` [Bug dyninst/23513] " fche at redhat dot com
@ 2018-08-11 0:46 ` agentzh at gmail dot com
2019-04-05 19:07 ` scox at redhat dot com
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: agentzh at gmail dot com @ 2018-08-11 0:46 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
--- Comment #2 from agentzh <agentzh at gmail dot com> ---
But the first `stap` command run already exits before the second one starts
(please note the `exit()` function call in the probe handler). So there is
still only *one* concurrent mutator here.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug dyninst/23513] Dyninst runtime kills the target process when attaching to the target process for a second time
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
2018-08-11 0:39 ` [Bug dyninst/23513] " fche at redhat dot com
2018-08-11 0:46 ` agentzh at gmail dot com
@ 2019-04-05 19:07 ` scox at redhat dot com
2019-04-06 1:01 ` scox at redhat dot com
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: scox at redhat dot com @ 2019-04-05 19:07 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
Stan Cox <scox at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |scox at redhat dot com
--- Comment #3 from Stan Cox <scox at redhat dot com> ---
I notice if the first dyninst run is via stap:
% stap -e 'probe process("/lib64/libc.so.6").function("usleep") { println("hi")
exit() }' -x `pgrep -f tst.x` --dyninst -k -vv
and if subsequent runs were directly via stapdyn then it worked okay:
% stapdyn -v -x 8955 /tmp/staprclqlD/stap_8970.so
stapdyn: found target "/usr/lib64/libc-2.28.so" in pid 8955, inserting 1 probes
hi
% stapdyn -v -x 8955 /tmp/staprclqlD/stap_8970.so
stapdyn: found target "/usr/lib64/libc-2.28.so" in pid 8955, inserting 1 probes
hi
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug dyninst/23513] Dyninst runtime kills the target process when attaching to the target process for a second time
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
` (2 preceding siblings ...)
2019-04-05 19:07 ` scox at redhat dot com
@ 2019-04-06 1:01 ` scox at redhat dot com
2019-04-10 12:42 ` scox at redhat dot com
2019-12-11 16:25 ` avi@cloudius-systems.com
5 siblings, 0 replies; 7+ messages in thread
From: scox at redhat dot com @ 2019-04-06 1:01 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
--- Comment #4 from Stan Cox <scox at redhat dot com> ---
Running with DYNINST_DEBUG_STARTUP=1
it seems the problem is dyninst fails to parse the stap so (not sure why) and
intentionally terminates the process.
[140306531820032]image.C[1461]: opening file
/tmp/stapSmTcdi/stap_2ecef162cf9d3c769f9c52398beab57b_1423.so
[140306531820032]image.C[1226]: created image
[140306531820032]image.C[1229]: processing shared object
[140306531820032]image.C[1247]: error in processing, deleting image and
returning
[140306531820032]mapped_object.C[157]: failed to parseImage
[140306531820032]dynProcess.C[618]: failed to create mapped object for library
/tmp/stapSmTcdi/stap_2ecef162cf9d3c769f9c52398beab57b_1423.so
[140306531820032]dynProcess.C[361]: bootstrap failed while creating mapped
objects
Failed to bootstrap process 9255: terminating...
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug dyninst/23513] Dyninst runtime kills the target process when attaching to the target process for a second time
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
` (3 preceding siblings ...)
2019-04-06 1:01 ` scox at redhat dot com
@ 2019-04-10 12:42 ` scox at redhat dot com
2019-12-11 16:25 ` avi@cloudius-systems.com
5 siblings, 0 replies; 7+ messages in thread
From: scox at redhat dot com @ 2019-04-10 12:42 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
Stan Cox <scox at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |ASSIGNED
Last reconfirmed| |2019-04-10
Ever confirmed|0 |1
--- Comment #5 from Stan Cox <scox at redhat dot com> ---
The problem seems intermittant but I do see it and the culprit is that the stap
shared object is not always detached when stapdyn completes. (gdb session
after failing stapdyn)
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00007f3bd7f64670 0x00007f3bd80afc3f Yes (*) /lib64/libc.so.6
0x00007f3bd8139110 0x00007f3bd81582b4 Yes (*) /lib64/ld-linux-x86-64.so.2
0x00007f3bd6eb2320 0x00007f3bd6eb4845 Yes (*)
/usr/lib64/dyninst/libdyninstAPI_RT.so
0x00007f3bd6eab270 0x00007f3bd6eac039 Yes (*) /lib64/libdl.so.2
No
/tmp/stapUaA2KQ/stap_4e52f2023ce6edaabf143f6d5214b684_1361.so
0x00007f3bd8124710 0x00007f3bd8127a80 Yes (*) /lib64/librt.so.1
0x00007f3bd6e60b50 0x00007f3bd6e6f025 Yes (*) /lib64/libpthread.so.0
(*): Shared library is missing debugging information.
So whenever stapdyn subsequently runs, dyninst tries to load that nonexistant
library: /tmp/stapUaA2KQ/stap_4e52f2023ce6edaabf143f6d5214b684_1361.so
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug dyninst/23513] Dyninst runtime kills the target process when attaching to the target process for a second time
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
` (4 preceding siblings ...)
2019-04-10 12:42 ` scox at redhat dot com
@ 2019-12-11 16:25 ` avi@cloudius-systems.com
5 siblings, 0 replies; 7+ messages in thread
From: avi@cloudius-systems.com @ 2019-12-11 16:25 UTC (permalink / raw)
To: systemtap
https://sourceware.org/bugzilla/show_bug.cgi?id=23513
Avi Kivity <avi@cloudius-systems.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |avi@cloudius-systems.com
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-12-11 16:25 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-11 0:34 [Bug dyninst/23513] New: Dyninst runtime kills the target process when attaching to the target process for a second time agentzh at gmail dot com
2018-08-11 0:39 ` [Bug dyninst/23513] " fche at redhat dot com
2018-08-11 0:46 ` agentzh at gmail dot com
2019-04-05 19:07 ` scox at redhat dot com
2019-04-06 1:01 ` scox at redhat dot com
2019-04-10 12:42 ` scox at redhat dot com
2019-12-11 16:25 ` avi@cloudius-systems.com
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).