python tracing

python tracing

[corpaul]

Was anyone here successful on python tracing?

I patched Python 2.7 using the patches on http://bugs.python.org/issue13405
Now when I try to run
http://sourceware.org/systemtap/wiki/PythonMarkers?action=AttachFile&do=view&target=systemtap-example.stp
, I get the error:

semantic error: while resolving probe point: identifier 'python' at
stap-python.stp:11:7
        source: probe python.function.entry
                      ^

semantic error: probe point mismatch  (alternatives: begin begin(number) end
end(number) error error(number) kernel kprobe module(string) netfilter never
perf process process(number) process(string) procfs procfs(string) stap
staprun timer): identifier 'python' at :11:7
        source: probe python.function.entry
                      ^

semantic error: while resolving probe point: identifier 'python' at :16:7
        source: probe python.function.return
                      ^

semantic error: probe point mismatch  (alternatives: begin begin(number) end
end(number) error error(number) kernel kprobe module(string) netfilter never
perf process process(number) process(string) procfs procfs(string) stap
staprun timer): identifier 'python' at :16:7
        source: probe python.function.return
                      ^

Pass 2: analysis failed.  [man error::pass2]

Does this mean that Python was not patched correctly? or do i need to change
something in Systemtap?

Thanks!

--CP



--
View this message in context: http://sourceware-org.1504.n7.nabble.com/python-tracing-tp227721.html
Sent from the Sourceware - systemtap mailing list archive at Nabble.com.

Re: python tracing

[David Smith]

On 04/08/2013 08:11 AM, corpaul wrote:
> Was anyone here successful on python tracing?
> 
> I patched Python 2.7 using the patches on http://bugs.python.org/issue13405
> Now when I try to run
> http://sourceware.org/systemtap/wiki/PythonMarkers?action=AttachFile&do=view&target=systemtap-example.stp
> , I get the error:
> 
> semantic error: while resolving probe point: identifier 'python' at
> stap-python.stp:11:7
>         source: probe python.function.entry

I haven't done traced python with systemtap, but from the error it looks
like there is supposed to be a systemtap python tapset file (probably
named something like 'python.stp') that is supposed to be added to
systemtap.

Did that exist somewhere in those patches?

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

RE: python tracing

[Cor-paul Bezemer - EWI]

You are right, the tapset was missing.
I used this one: http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/root/usr/share/systemtap/tapset/libpython2.7-64.stp

but am getting the error 
 ./stap test_python.stp --runtime=dyninst -c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
semantic error: while resolving probe point: identifier 'process' at /home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
        source: probe python.function.return = process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__return")
                                               ^

semantic error: no match
semantic error: while resolving probe point: identifier 'python' at test_python.stp:1:7
        source: probe python.function.return 
                      ^

Pass 2: analysis failed.  [man error::pass2]

--CP
________________________________________
From: David Smith [dsmith@redhat.com]
Sent: Monday, April 08, 2013 4:14 PM
To: Cor-paul Bezemer - EWI
Cc: systemtap@sourceware.org
Subject: Re: python tracing

On 04/08/2013 08:11 AM, corpaul wrote:
> Was anyone here successful on python tracing?
>
> I patched Python 2.7 using the patches on http://bugs.python.org/issue13405
> Now when I try to run
> http://sourceware.org/systemtap/wiki/PythonMarkers?action=AttachFile&do=view&target=systemtap-example.stp
> , I get the error:
>
> semantic error: while resolving probe point: identifier 'python' at
> stap-python.stp:11:7
>         source: probe python.function.entry

I haven't done traced python with systemtap, but from the error it looks
like there is supposed to be a systemtap python tapset file (probably
named something like 'python.stp') that is supposed to be added to
systemtap.

Did that exist somewhere in those patches?

--
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: python tracing

[David Smith]

On 04/08/2013 09:41 AM, Cor-paul Bezemer - EWI wrote:
> You are right, the tapset was missing.
> I used this one: http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/root/usr/share/systemtap/tapset/libpython2.7-64.stp
> 
> but am getting the error 
>  ./stap test_python.stp --runtime=dyninst -c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
> semantic error: while resolving probe point: identifier 'process' at /home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
>         source: probe python.function.return = process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__return")
>                                                ^
> 
> semantic error: no match
> semantic error: while resolving probe point: identifier 'python' at test_python.stp:1:7
>         source: probe python.function.return 
>                       ^
> 
> Pass 2: analysis failed.  [man error::pass2]

OK, we're getting a bit further now.

Can you try the following command? This should show us what tracepoints
stap thinks are present in your python executable.

# stap -L
'process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("*")'

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

RE: python tracing

[Cor-paul Bezemer - EWI]

I tried and noticed that the library path is incorrect (nonexistent). How can I found it which library python is using?

Should I have a python process running when executing this command?
Or is it trying to hook to the 'default' python executable? (my patched Python is in a different directory)

--CP


________________________________________
From: David Smith [dsmith@redhat.com]
Sent: Monday, April 08, 2013 4:59 PM
To: Cor-paul Bezemer - EWI
Cc: systemtap@sourceware.org
Subject: Re: python tracing

On 04/08/2013 09:41 AM, Cor-paul Bezemer - EWI wrote:
> You are right, the tapset was missing.
> I used this one: http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/root/usr/share/systemtap/tapset/libpython2.7-64.stp
>
> but am getting the error
>  ./stap test_python.stp --runtime=dyninst -c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
> semantic error: while resolving probe point: identifier 'process' at /home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
>         source: probe python.function.return = process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__return")
>                                                ^
>
> semantic error: no match
> semantic error: while resolving probe point: identifier 'python' at test_python.stp:1:7
>         source: probe python.function.return
>                       ^
>
> Pass 2: analysis failed.  [man error::pass2]

OK, we're getting a bit further now.

Can you try the following command? This should show us what tracepoints
stap thinks are present in your python executable.

# stap -L
'process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("*")'

--
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: python tracing

[David Smith]

On 04/09/2013 03:05 AM, Cor-paul Bezemer - EWI wrote:
> I tried and noticed that the library path is incorrect (nonexistent). How can I found it which library python is using?
> 
> Should I have a python process running when executing this command?
> Or is it trying to hook to the 'default' python executable? (my patched Python is in a different directory

Ah. The command I gave you (and the tapset itself) are trying to probe
the system python, not your private python.

You'll need to modify the tapset to point to your private version of
python (instead of the system python). You don't have to have a python
process running when executing the 'stap -L' command.

So, modify that stap command to look something like the following, where
'PRIV_PYTHON' is the path to your private python executable (probably
somewhere down in ~/Play/python2.7-2.7.4-rc1 if I'm reading your email
correctly) and 'PRIV_PYTHON_LIB' is the path to the main python shared
library.

# stap -L 'process("PRIV_PYTHON").library("PRIV_PYTHON_LIB").mark("*")'


> 
> --CP
> 
> 
> ________________________________________
> From: David Smith [dsmith@redhat.com]
> Sent: Monday, April 08, 2013 4:59 PM
> To: Cor-paul Bezemer - EWI
> Cc: systemtap@sourceware.org
> Subject: Re: python tracing
> 
> On 04/08/2013 09:41 AM, Cor-paul Bezemer - EWI wrote:
>> You are right, the tapset was missing.
>> I used this one: http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/root/usr/share/systemtap/tapset/libpython2.7-64.stp
>>
>> but am getting the error
>>  ./stap test_python.stp --runtime=dyninst -c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
>> semantic error: while resolving probe point: identifier 'process' at /home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
>>         source: probe python.function.return = process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__return")
>>                                                ^
>>
>> semantic error: no match
>> semantic error: while resolving probe point: identifier 'python' at test_python.stp:1:7
>>         source: probe python.function.return
>>                       ^
>>
>> Pass 2: analysis failed.  [man error::pass2]
> 
> OK, we're getting a bit further now.
> 
> Can you try the following command? This should show us what tracepoints
> stap thinks are present in your python executable.
> 
> # stap -L
> 'process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("*")'
> 
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)
> 


-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

RE: python tracing

[Cor-paul Bezemer - EWI]

OK, I have tried this (eventually I updated my path). Still, the list returned by the command is empty :(

--CP
________________________________________
From: David Smith [dsmith@redhat.com]
Sent: Tuesday, April 09, 2013 3:24 PM
To: Cor-paul Bezemer - EWI
Cc: systemtap@sourceware.org
Subject: Re: python tracing

On 04/09/2013 03:05 AM, Cor-paul Bezemer - EWI wrote:
> I tried and noticed that the library path is incorrect (nonexistent). How can I found it which library python is using?
>
> Should I have a python process running when executing this command?
> Or is it trying to hook to the 'default' python executable? (my patched Python is in a different directory

Ah. The command I gave you (and the tapset itself) are trying to probe
the system python, not your private python.

You'll need to modify the tapset to point to your private version of
python (instead of the system python). You don't have to have a python
process running when executing the 'stap -L' command.

So, modify that stap command to look something like the following, where
'PRIV_PYTHON' is the path to your private python executable (probably
somewhere down in ~/Play/python2.7-2.7.4-rc1 if I'm reading your email
correctly) and 'PRIV_PYTHON_LIB' is the path to the main python shared
library.

# stap -L 'process("PRIV_PYTHON").library("PRIV_PYTHON_LIB").mark("*")'


>
> --CP
>
>
> ________________________________________
> From: David Smith [dsmith@redhat.com]
> Sent: Monday, April 08, 2013 4:59 PM
> To: Cor-paul Bezemer - EWI
> Cc: systemtap@sourceware.org
> Subject: Re: python tracing
>
> On 04/08/2013 09:41 AM, Cor-paul Bezemer - EWI wrote:
>> You are right, the tapset was missing.
>> I used this one: http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/root/usr/share/systemtap/tapset/libpython2.7-64.stp
>>
>> but am getting the error
>>  ./stap test_python.stp --runtime=dyninst -c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
>> semantic error: while resolving probe point: identifier 'process' at /home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
>>         source: probe python.function.return = process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__return")
>>                                                ^
>>
>> semantic error: no match
>> semantic error: while resolving probe point: identifier 'python' at test_python.stp:1:7
>>         source: probe python.function.return
>>                       ^
>>
>> Pass 2: analysis failed.  [man error::pass2]
>
> OK, we're getting a bit further now.
>
> Can you try the following command? This should show us what tracepoints
> stap thinks are present in your python executable.
>
> # stap -L
> 'process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("*")'
>
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)
>


--
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: python tracing

[David Smith]

Let's try a couple of more things.

First, let's look for the special section that using markers create. Try
this, substituting PRIV_PYTHON_LIB as before:

# eu-readelf -S PRIV_PYTHON_LIB | grep stapsdt

On libc on my system, that reports:

# eu-readelf -S /usr/lib64/libc-2.17.so | fgrep stapsdt
[16] .stapsdt.base        PROGBITS     00000030f5780be0 00180be0
00000001  0 A      0   0  1
[35] .note.stapsdt        NOTE         0000000000000000 001ba8f0
00000294  0        0   0  4

You should see similar sections in your private python library. If you
see those sections, try the following, substituting PRIV_PYTHON_LIB as
before:

# stap -L 'process("PRIV_PYTHON_LIB").mark("*")'

On 04/09/2013 08:32 AM, Cor-paul Bezemer - EWI wrote:
> OK, I have tried this (eventually I updated my path). Still, the list returned by the command is empty :(
> 
> --CP
> ________________________________________
> From: David Smith [dsmith@redhat.com]
> Sent: Tuesday, April 09, 2013 3:24 PM
> To: Cor-paul Bezemer - EWI
> Cc: systemtap@sourceware.org
> Subject: Re: python tracing
> 
> On 04/09/2013 03:05 AM, Cor-paul Bezemer - EWI wrote:
>> I tried and noticed that the library path is incorrect (nonexistent). How can I found it which library python is using?
>>
>> Should I have a python process running when executing this command?
>> Or is it trying to hook to the 'default' python executable? (my patched Python is in a different directory
> 
> Ah. The command I gave you (and the tapset itself) are trying to probe
> the system python, not your private python.
> 
> You'll need to modify the tapset to point to your private version of
> python (instead of the system python). You don't have to have a python
> process running when executing the 'stap -L' command.
> 
> So, modify that stap command to look something like the following, where
> 'PRIV_PYTHON' is the path to your private python executable (probably
> somewhere down in ~/Play/python2.7-2.7.4-rc1 if I'm reading your email
> correctly) and 'PRIV_PYTHON_LIB' is the path to the main python shared
> library.
> 
> # stap -L 'process("PRIV_PYTHON").library("PRIV_PYTHON_LIB").mark("*")'
> 
> 
>>
>> --CP
>>
>>
>> ________________________________________
>> From: David Smith [dsmith@redhat.com]
>> Sent: Monday, April 08, 2013 4:59 PM
>> To: Cor-paul Bezemer - EWI
>> Cc: systemtap@sourceware.org
>> Subject: Re: python tracing
>>
>> On 04/08/2013 09:41 AM, Cor-paul Bezemer - EWI wrote:
>>> You are right, the tapset was missing.
>>> I used this one: http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/root/usr/share/systemtap/tapset/libpython2.7-64.stp
>>>
>>> but am getting the error
>>>  ./stap test_python.stp --runtime=dyninst -c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
>>> semantic error: while resolving probe point: identifier 'process' at /home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
>>>         source: probe python.function.return = process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__return")
>>>                                                ^
>>>
>>> semantic error: no match
>>> semantic error: while resolving probe point: identifier 'python' at test_python.stp:1:7
>>>         source: probe python.function.return
>>>                       ^
>>>
>>> Pass 2: analysis failed.  [man error::pass2]
>>
>> OK, we're getting a bit further now.
>>
>> Can you try the following command? This should show us what tracepoints
>> stap thinks are present in your python executable.
>>
>> # stap -L
>> 'process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("*")'
>>
>> --
>> David Smith
>> dsmith@redhat.com
>> Red Hat
>> http://www.redhat.com
>> 256.217.0141 (direct)
>> 256.837.0057 (fax)
>>
> 
> 
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)
> 


-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

RE: python tracing

[Cor-Paul Bezemer]

Also empty :/
Looks like the patch didn't work?



--CP

-----Original Message-----
From: David Smith [mailto:dsmith@redhat.com] 
Sent: dinsdag 9 april 2013 17:16
To: Cor-paul Bezemer - EWI
Cc: systemtap@sourceware.org
Subject: Re: python tracing

Let's try a couple of more things.

First, let's look for the special section that using markers create. Try
this, substituting PRIV_PYTHON_LIB as before:

# eu-readelf -S PRIV_PYTHON_LIB | grep stapsdt

On libc on my system, that reports:

# eu-readelf -S /usr/lib64/libc-2.17.so | fgrep stapsdt
[16] .stapsdt.base        PROGBITS     00000030f5780be0 00180be0
00000001  0 A      0   0  1
[35] .note.stapsdt        NOTE         0000000000000000 001ba8f0
00000294  0        0   0  4

You should see similar sections in your private python library. If you see
those sections, try the following, substituting PRIV_PYTHON_LIB as
before:

# stap -L 'process("PRIV_PYTHON_LIB").mark("*")'

On 04/09/2013 08:32 AM, Cor-paul Bezemer - EWI wrote:
> OK, I have tried this (eventually I updated my path). Still, the list 
> returned by the command is empty :(
> 
> --CP
> ________________________________________
> From: David Smith [dsmith@redhat.com]
> Sent: Tuesday, April 09, 2013 3:24 PM
> To: Cor-paul Bezemer - EWI
> Cc: systemtap@sourceware.org
> Subject: Re: python tracing
> 
> On 04/09/2013 03:05 AM, Cor-paul Bezemer - EWI wrote:
>> I tried and noticed that the library path is incorrect (nonexistent). How
can I found it which library python is using?
>>
>> Should I have a python process running when executing this command?
>> Or is it trying to hook to the 'default' python executable? (my 
>> patched Python is in a different directory
> 
> Ah. The command I gave you (and the tapset itself) are trying to probe 
> the system python, not your private python.
> 
> You'll need to modify the tapset to point to your private version of 
> python (instead of the system python). You don't have to have a python 
> process running when executing the 'stap -L' command.
> 
> So, modify that stap command to look something like the following, 
> where 'PRIV_PYTHON' is the path to your private python executable 
> (probably somewhere down in ~/Play/python2.7-2.7.4-rc1 if I'm reading 
> your email
> correctly) and 'PRIV_PYTHON_LIB' is the path to the main python shared 
> library.
> 
> # stap -L 'process("PRIV_PYTHON").library("PRIV_PYTHON_LIB").mark("*")'
> 
> 
>>
>> --CP
>>
>>
>> ________________________________________
>> From: David Smith [dsmith@redhat.com]
>> Sent: Monday, April 08, 2013 4:59 PM
>> To: Cor-paul Bezemer - EWI
>> Cc: systemtap@sourceware.org
>> Subject: Re: python tracing
>>
>> On 04/08/2013 09:41 AM, Cor-paul Bezemer - EWI wrote:
>>> You are right, the tapset was missing.
>>> I used this one: 
>>> http://packaging-farm.dachary.org/packaging-farm/fedora/x86_64/f14/r
>>> oot/usr/share/systemtap/tapset/libpython2.7-64.stp
>>>
>>> but am getting the error
>>>  ./stap test_python.stp --runtime=dyninst
-c'~/Play/python/python2.7-2.7.4~rc1/build-shared/python test.py'
>>> semantic error: while resolving probe point: identifier 'process' at
/home/corpaul/stap/share/systemtap/tapset/python.stp:12:32
>>>         source: probe python.function.return =
process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("function__
return")
>>>                                                ^
>>>
>>> semantic error: no match
>>> semantic error: while resolving probe point: identifier 'python' at
test_python.stp:1:7
>>>         source: probe python.function.return
>>>                       ^
>>>
>>> Pass 2: analysis failed.  [man error::pass2]
>>
>> OK, we're getting a bit further now.
>>
>> Can you try the following command? This should show us what 
>> tracepoints stap thinks are present in your python executable.
>>
>> # stap -L
>> 'process("python").library("/usr/lib64/libpython2.7.so.1.0").mark("*")'
>>
>> --
>> David Smith
>> dsmith@redhat.com
>> Red Hat
>> http://www.redhat.com
>> 256.217.0141 (direct)
>> 256.837.0057 (fax)
>>
> 
> 
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)
> 


--
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: python tracing

[David Smith]

On 04/09/2013 10:51 AM, Cor-Paul Bezemer wrote:
> Also empty :/
> Looks like the patch didn't work?

It appears so. I'll ask a basic question. After applying the patches to
python, did you use the '--with-dtrace' configure argument?

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: python tracing

[Stan Cox]

We test a variation of this in the upstream systemtap testsuite 
(testsuite/systemtap.apps/python.exp)  What the test does is: fetch 
upstream python sources, apply either one of python2.patch or 
python3.patch, build python, run test using a scenario similar to the 
following:

stap celsius-var.stp -c "python -sS celsius.py 30" '*' python.log
stap celsius-var.stp -c "python3 -sS celsius.py 30" '*' python.log

Re: python tracing

[corpaul]

I was finally able to get this working by using the following patched
version:

hg clone http://hg.jcea.es/cpython-2011
cd cpython-2011
hg checkout dtrace-issue13405_2.7
./configure --with-dtrace
cp Modules/Setup.dist Modules/Setup
make -j_Number_of_cores_in_the_machine_

and the right library path can be found by running on the built python:
ldd python

We're working on building a .deb file for the python version as it now
misses some linking with libraries.
Figured I'd post this here in case it might be of use to anyone.




--
View this message in context: http://sourceware-org.1504.n7.nabble.com/python-tracing-tp227721p229460.html
Sent from the Sourceware - systemtap mailing list archive at Nabble.com.