* No output from trace file after a few hours @ 2017-06-15 7:10 Daniel Doron 2017-06-15 7:13 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 7:10 UTC (permalink / raw) To: systemtap Hi, I built my module to probe a few system calls and functions. Nothing fancy, no big work in it, just prints some data. Then I wrote a simple python script to read data directly off the /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as sudo) I run stap in Flight recorder mode. After a few hours (varies from 8 to 20) I stop getting any output from the module... Any idea how to check why that is? dmesg does not report anything unusual... Thanks. Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:10 No output from trace file after a few hours Daniel Doron @ 2017-06-15 7:13 ` Arkady 2017-06-15 7:33 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 7:13 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap Please send the stap script On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > Hi, > > I built my module to probe a few system calls and functions. Nothing > fancy, no big work in it, just prints some data. Then I wrote a simple > python script to read data directly off the > /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as > sudo) > I run stap in Flight recorder mode. > After a few hours (varies from 8 to 20) I stop getting any output from > the module... > Any idea how to check why that is? > dmesg does not report anything unusual... > > Thanks. > Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:13 ` Arkady @ 2017-06-15 7:33 ` Arkady 2017-06-15 7:34 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 7:33 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap Do you have the kernel version pinned? On Thu, Jun 15, 2017 at 10:12 AM, Arkady <arkady.miasnikov@gmail.com> wrote: > Please send the stap script > > On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron > <danielmeirdoron@gmail.com> wrote: >> Hi, >> >> I built my module to probe a few system calls and functions. Nothing >> fancy, no big work in it, just prints some data. Then I wrote a simple >> python script to read data directly off the >> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as >> sudo) >> I run stap in Flight recorder mode. >> After a few hours (varies from 8 to 20) I stop getting any output from >> the module... >> Any idea how to check why that is? >> dmesg does not report anything unusual... >> >> Thanks. >> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:33 ` Arkady @ 2017-06-15 7:34 ` Daniel Doron 2017-06-15 7:37 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 7:34 UTC (permalink / raw) To: Arkady; +Cc: systemtap Sorry, I do not know what you mean by that... On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> wrote: > Do you have the kernel version pinned? > > On Thu, Jun 15, 2017 at 10:12 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >> Please send the stap script >> >> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >> <danielmeirdoron@gmail.com> wrote: >>> Hi, >>> >>> I built my module to probe a few system calls and functions. Nothing >>> fancy, no big work in it, just prints some data. Then I wrote a simple >>> python script to read data directly off the >>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as >>> sudo) >>> I run stap in Flight recorder mode. >>> After a few hours (varies from 8 to 20) I stop getting any output from >>> the module... >>> Any idea how to check why that is? >>> dmesg does not report anything unusual... >>> >>> Thanks. >>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:34 ` Daniel Doron @ 2017-06-15 7:37 ` Daniel Doron 2017-06-15 7:37 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 7:37 UTC (permalink / raw) To: Arkady; +Cc: systemtap if you mean that the kernel version does not get upgraded, then yes. On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > Sorry, I do not know what you mean by that... > > On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >> Do you have the kernel version pinned? >> >> On Thu, Jun 15, 2017 at 10:12 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>> Please send the stap script >>> >>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>> <danielmeirdoron@gmail.com> wrote: >>>> Hi, >>>> >>>> I built my module to probe a few system calls and functions. Nothing >>>> fancy, no big work in it, just prints some data. Then I wrote a simple >>>> python script to read data directly off the >>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as >>>> sudo) >>>> I run stap in Flight recorder mode. >>>> After a few hours (varies from 8 to 20) I stop getting any output from >>>> the module... >>>> Any idea how to check why that is? >>>> dmesg does not report anything unusual... >>>> >>>> Thanks. >>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:37 ` Daniel Doron @ 2017-06-15 7:37 ` Daniel Doron 2017-06-15 7:38 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 7:37 UTC (permalink / raw) To: Arkady; +Cc: systemtap BTW kernel version: 2.6.32-431.el6.x86_64 On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > if you mean that the kernel version does not get upgraded, then yes. > > On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron > <danielmeirdoron@gmail.com> wrote: >> Sorry, I do not know what you mean by that... >> >> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>> Do you have the kernel version pinned? >>> >>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>> Please send the stap script >>>> >>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>> <danielmeirdoron@gmail.com> wrote: >>>>> Hi, >>>>> >>>>> I built my module to probe a few system calls and functions. Nothing >>>>> fancy, no big work in it, just prints some data. Then I wrote a simple >>>>> python script to read data directly off the >>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as >>>>> sudo) >>>>> I run stap in Flight recorder mode. >>>>> After a few hours (varies from 8 to 20) I stop getting any output from >>>>> the module... >>>>> Any idea how to check why that is? >>>>> dmesg does not report anything unusual... >>>>> >>>>> Thanks. >>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:37 ` Daniel Doron @ 2017-06-15 7:38 ` Daniel Doron 2017-06-15 8:18 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 7:38 UTC (permalink / raw) To: Arkady; +Cc: systemtap Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) Copyright (C) 2005-2015 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > BTW kernel version: 2.6.32-431.el6.x86_64 > > On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron > <danielmeirdoron@gmail.com> wrote: >> if you mean that the kernel version does not get upgraded, then yes. >> >> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >> <danielmeirdoron@gmail.com> wrote: >>> Sorry, I do not know what you mean by that... >>> >>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>> Do you have the kernel version pinned? >>>> >>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>> Please send the stap script >>>>> >>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>> <danielmeirdoron@gmail.com> wrote: >>>>>> Hi, >>>>>> >>>>>> I built my module to probe a few system calls and functions. Nothing >>>>>> fancy, no big work in it, just prints some data. Then I wrote a simple >>>>>> python script to read data directly off the >>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as >>>>>> sudo) >>>>>> I run stap in Flight recorder mode. >>>>>> After a few hours (varies from 8 to 20) I stop getting any output from >>>>>> the module... >>>>>> Any idea how to check why that is? >>>>>> dmesg does not report anything unusual... >>>>>> >>>>>> Thanks. >>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 7:38 ` Daniel Doron @ 2017-06-15 8:18 ` Arkady [not found] ` <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com> 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 8:18 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap How long time it runs for you? On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) > Copyright (C) 2005-2015 Red Hat, Inc. and others > This is free software; see the source for copying conditions. > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP > > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron > <danielmeirdoron@gmail.com> wrote: >> BTW kernel version: 2.6.32-431.el6.x86_64 >> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >> <danielmeirdoron@gmail.com> wrote: >>> if you mean that the kernel version does not get upgraded, then yes. >>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>> <danielmeirdoron@gmail.com> wrote: >>>> Sorry, I do not know what you mean by that... >>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>> Do you have the kernel version pinned? >>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>>> Please send the stap script >>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I built my module to probe a few system calls and functions. Nothing >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a simple >>>>>>> python script to read data directly off the >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course running as >>>>>>> sudo) >>>>>>> I run stap in Flight recorder mode. >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output from >>>>>>> the module... >>>>>>> Any idea how to check why that is? >>>>>>> dmesg does not report anything unusual... >>>>>>> >>>>>>> Thanks. >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
[parent not found: <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com>]
* Re: No output from trace file after a few hours [not found] ` <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com> @ 2017-06-15 8:26 ` Arkady 2017-06-15 8:59 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 8:26 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap Do you do something like sudo stap -F -o ./test.log -S 1,2 test.stp ? On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > On that specific kernel about 8-10 hours. > On my machine with latest kernel, been running for 24 hours. > I need to check other distros. > > On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >> >> How long time it runs for you? >> >> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >> <danielmeirdoron@gmail.com> wrote: >> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >> > Copyright (C) 2005-2015 Red Hat, Inc. and others >> > This is free software; see the source for copying conditions. >> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >> > >> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >> > <danielmeirdoron@gmail.com> wrote: >> >> BTW kernel version: 2.6.32-431.el6.x86_64 >> >> >> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >> >> <danielmeirdoron@gmail.com> wrote: >> >>> if you mean that the kernel version does not get upgraded, then yes. >> >>> >> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >> >>> <danielmeirdoron@gmail.com> wrote: >> >>>> Sorry, I do not know what you mean by that... >> >>>> >> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >> >>>> wrote: >> >>>>> Do you have the kernel version pinned? >> >>>>> >> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >> >>>>> <arkady.miasnikov@gmail.com> wrote: >> >>>>>> Please send the stap script >> >>>>>> >> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >> >>>>>> <danielmeirdoron@gmail.com> wrote: >> >>>>>>> Hi, >> >>>>>>> >> >>>>>>> I built my module to probe a few system calls and functions. >> >>>>>>> Nothing >> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >> >>>>>>> simple >> >>>>>>> python script to read data directly off the >> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >> >>>>>>> running as >> >>>>>>> sudo) >> >>>>>>> I run stap in Flight recorder mode. >> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >> >>>>>>> from >> >>>>>>> the module... >> >>>>>>> Any idea how to check why that is? >> >>>>>>> dmesg does not report anything unusual... >> >>>>>>> >> >>>>>>> Thanks. >> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 8:26 ` Arkady @ 2017-06-15 8:59 ` Daniel Doron 2017-06-15 10:54 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 8:59 UTC (permalink / raw) To: Arkady; +Cc: systemtap On that specific kernel about 8-10 hours. On my machine with latest kernel, been running for 24 hours. I need to check other distros. sudo stap -v -F -m mymodule test.stp On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: > Do you do something like > > sudo stap -F -o ./test.log -S 1,2 test.stp ? > > On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron > <danielmeirdoron@gmail.com> wrote: >> On that specific kernel about 8-10 hours. >> On my machine with latest kernel, been running for 24 hours. >> I need to check other distros. >> >> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>> >>> How long time it runs for you? >>> >>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>> <danielmeirdoron@gmail.com> wrote: >>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>> > This is free software; see the source for copying conditions. >>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>> > >>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>> > <danielmeirdoron@gmail.com> wrote: >>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>> >> >>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>> >> <danielmeirdoron@gmail.com> wrote: >>> >>> if you mean that the kernel version does not get upgraded, then yes. >>> >>> >>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>> >>> <danielmeirdoron@gmail.com> wrote: >>> >>>> Sorry, I do not know what you mean by that... >>> >>>> >>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>> >>>> wrote: >>> >>>>> Do you have the kernel version pinned? >>> >>>>> >>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>> >>>>>> Please send the stap script >>> >>>>>> >>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>> >>>>>>> Hi, >>> >>>>>>> >>> >>>>>>> I built my module to probe a few system calls and functions. >>> >>>>>>> Nothing >>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>> >>>>>>> simple >>> >>>>>>> python script to read data directly off the >>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>> >>>>>>> running as >>> >>>>>>> sudo) >>> >>>>>>> I run stap in Flight recorder mode. >>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>> >>>>>>> from >>> >>>>>>> the module... >>> >>>>>>> Any idea how to check why that is? >>> >>>>>>> dmesg does not report anything unusual... >>> >>>>>>> >>> >>>>>>> Thanks. >>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 8:59 ` Daniel Doron @ 2017-06-15 10:54 ` Arkady 2017-06-15 11:04 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 10:54 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap Interesting. I have reproduced the problem by running two tight bash loops simultaneously sudo stap -F -o ./test.log -S 1,2 test.stp while [ 1 ];do ls > /dev/null;done & while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en "\033[0;0H";done; Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > On that specific kernel about 8-10 hours. > On my machine with latest kernel, been running for 24 hours. > I need to check other distros. > > sudo stap -v -F -m mymodule test.stp > > On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >> Do you do something like >> >> sudo stap -F -o ./test.log -S 1,2 test.stp ? >> >> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >> <danielmeirdoron@gmail.com> wrote: >>> On that specific kernel about 8-10 hours. >>> On my machine with latest kernel, been running for 24 hours. >>> I need to check other distros. >>> >>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>> >>>> How long time it runs for you? >>>> >>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>> <danielmeirdoron@gmail.com> wrote: >>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>> > This is free software; see the source for copying conditions. >>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>> > >>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>> > <danielmeirdoron@gmail.com> wrote: >>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>> >> >>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>> >> <danielmeirdoron@gmail.com> wrote: >>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>> >>> >>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>> >>> <danielmeirdoron@gmail.com> wrote: >>>> >>>> Sorry, I do not know what you mean by that... >>>> >>>> >>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>> >>>> wrote: >>>> >>>>> Do you have the kernel version pinned? >>>> >>>>> >>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>> >>>>>> Please send the stap script >>>> >>>>>> >>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>> >>>>>>> Hi, >>>> >>>>>>> >>>> >>>>>>> I built my module to probe a few system calls and functions. >>>> >>>>>>> Nothing >>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>> >>>>>>> simple >>>> >>>>>>> python script to read data directly off the >>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>> >>>>>>> running as >>>> >>>>>>> sudo) >>>> >>>>>>> I run stap in Flight recorder mode. >>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>> >>>>>>> from >>>> >>>>>>> the module... >>>> >>>>>>> Any idea how to check why that is? >>>> >>>>>>> dmesg does not report anything unusual... >>>> >>>>>>> >>>> >>>>>>> Thanks. >>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 10:54 ` Arkady @ 2017-06-15 11:04 ` Arkady 2017-06-15 11:05 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 11:04 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap STAP simply quits in a couple of seconds sudo stap -F -o ./test.log -S 1,2 test.stp 94089 ps -ef | grep -v "grep" | grep 94089 On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote: > Interesting. I have reproduced the problem by running two tight bash > loops simultaneously > > sudo stap -F -o ./test.log -S 1,2 test.stp > while [ 1 ];do ls > /dev/null;done & > while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en > "\033[0;0H";done; > > Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC > 2016 x86_64 x86_64 x86_64 GNU/Linux > > > > On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron > <danielmeirdoron@gmail.com> wrote: >> On that specific kernel about 8-10 hours. >> On my machine with latest kernel, been running for 24 hours. >> I need to check other distros. >> >> sudo stap -v -F -m mymodule test.stp >> >> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>> Do you do something like >>> >>> sudo stap -F -o ./test.log -S 1,2 test.stp ? >>> >>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >>> <danielmeirdoron@gmail.com> wrote: >>>> On that specific kernel about 8-10 hours. >>>> On my machine with latest kernel, been running for 24 hours. >>>> I need to check other distros. >>>> >>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>>> >>>>> How long time it runs for you? >>>>> >>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>>> <danielmeirdoron@gmail.com> wrote: >>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>>> > This is free software; see the source for copying conditions. >>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>>> > >>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>>> > <danielmeirdoron@gmail.com> wrote: >>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>>> >> >>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>>> >> <danielmeirdoron@gmail.com> wrote: >>>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>>> >>> >>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>>> >>> <danielmeirdoron@gmail.com> wrote: >>>>> >>>> Sorry, I do not know what you mean by that... >>>>> >>>> >>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>>> >>>> wrote: >>>>> >>>>> Do you have the kernel version pinned? >>>>> >>>>> >>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>>> >>>>>> Please send the stap script >>>>> >>>>>> >>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>> >>>>>>> Hi, >>>>> >>>>>>> >>>>> >>>>>>> I built my module to probe a few system calls and functions. >>>>> >>>>>>> Nothing >>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>>> >>>>>>> simple >>>>> >>>>>>> python script to read data directly off the >>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>>> >>>>>>> running as >>>>> >>>>>>> sudo) >>>>> >>>>>>> I run stap in Flight recorder mode. >>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>>> >>>>>>> from >>>>> >>>>>>> the module... >>>>> >>>>>>> Any idea how to check why that is? >>>>> >>>>>>> dmesg does not report anything unusual... >>>>> >>>>>>> >>>>> >>>>>>> Thanks. >>>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 11:04 ` Arkady @ 2017-06-15 11:05 ` Arkady 2017-06-15 11:13 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 11:05 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap This works fine sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F -o ./test.log -S 1,2 test.stp On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote: > STAP simply quits in a couple of seconds > > sudo stap -F -o ./test.log -S 1,2 test.stp > 94089 > ps -ef | grep -v "grep" | grep 94089 > > On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >> Interesting. I have reproduced the problem by running two tight bash >> loops simultaneously >> >> sudo stap -F -o ./test.log -S 1,2 test.stp >> while [ 1 ];do ls > /dev/null;done & >> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en >> "\033[0;0H";done; >> >> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC >> 2016 x86_64 x86_64 x86_64 GNU/Linux >> >> >> >> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron >> <danielmeirdoron@gmail.com> wrote: >>> On that specific kernel about 8-10 hours. >>> On my machine with latest kernel, been running for 24 hours. >>> I need to check other distros. >>> >>> sudo stap -v -F -m mymodule test.stp >>> >>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>> Do you do something like >>>> >>>> sudo stap -F -o ./test.log -S 1,2 test.stp ? >>>> >>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >>>> <danielmeirdoron@gmail.com> wrote: >>>>> On that specific kernel about 8-10 hours. >>>>> On my machine with latest kernel, been running for 24 hours. >>>>> I need to check other distros. >>>>> >>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>>>> >>>>>> How long time it runs for you? >>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>>>> > This is free software; see the source for copying conditions. >>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>>>> > >>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>>>> > <danielmeirdoron@gmail.com> wrote: >>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>>>> >> >>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>>>> >> <danielmeirdoron@gmail.com> wrote: >>>>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>>>> >>> >>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>>>> >>> <danielmeirdoron@gmail.com> wrote: >>>>>> >>>> Sorry, I do not know what you mean by that... >>>>>> >>>> >>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>>>> >>>> wrote: >>>>>> >>>>> Do you have the kernel version pinned? >>>>>> >>>>> >>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>>>> >>>>>> Please send the stap script >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>> >>>>>>> >>>>>> >>>>>>> I built my module to probe a few system calls and functions. >>>>>> >>>>>>> Nothing >>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>>>> >>>>>>> simple >>>>>> >>>>>>> python script to read data directly off the >>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>>>> >>>>>>> running as >>>>>> >>>>>>> sudo) >>>>>> >>>>>>> I run stap in Flight recorder mode. >>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>>>> >>>>>>> from >>>>>> >>>>>>> the module... >>>>>> >>>>>>> Any idea how to check why that is? >>>>>> >>>>>>> dmesg does not report anything unusual... >>>>>> >>>>>>> >>>>>> >>>>>>> Thanks. >>>>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 11:05 ` Arkady @ 2017-06-15 11:13 ` Daniel Doron 2017-06-15 11:32 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Daniel Doron @ 2017-06-15 11:13 UTC (permalink / raw) To: Arkady; +Cc: systemtap I will try that. BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote: > This works fine > > sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F > -o ./test.log -S 1,2 test.stp > > On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >> STAP simply quits in a couple of seconds >> >> sudo stap -F -o ./test.log -S 1,2 test.stp >> 94089 >> ps -ef | grep -v "grep" | grep 94089 >> >> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>> Interesting. I have reproduced the problem by running two tight bash >>> loops simultaneously >>> >>> sudo stap -F -o ./test.log -S 1,2 test.stp >>> while [ 1 ];do ls > /dev/null;done & >>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en >>> "\033[0;0H";done; >>> >>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC >>> 2016 x86_64 x86_64 x86_64 GNU/Linux >>> >>> >>> >>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron >>> <danielmeirdoron@gmail.com> wrote: >>>> On that specific kernel about 8-10 hours. >>>> On my machine with latest kernel, been running for 24 hours. >>>> I need to check other distros. >>>> >>>> sudo stap -v -F -m mymodule test.stp >>>> >>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>> Do you do something like >>>>> >>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ? >>>>> >>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >>>>> <danielmeirdoron@gmail.com> wrote: >>>>>> On that specific kernel about 8-10 hours. >>>>>> On my machine with latest kernel, been running for 24 hours. >>>>>> I need to check other distros. >>>>>> >>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>>>>> >>>>>>> How long time it runs for you? >>>>>>> >>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>>>>> > This is free software; see the source for copying conditions. >>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>>>>> > >>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>>>>> > <danielmeirdoron@gmail.com> wrote: >>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>>>>> >> >>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>>>>> >> <danielmeirdoron@gmail.com> wrote: >>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>>>>> >>> >>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>>>>> >>> <danielmeirdoron@gmail.com> wrote: >>>>>>> >>>> Sorry, I do not know what you mean by that... >>>>>>> >>>> >>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>>>>> >>>> wrote: >>>>>>> >>>>> Do you have the kernel version pinned? >>>>>>> >>>>> >>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>>>>> >>>>>> Please send the stap script >>>>>>> >>>>>> >>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> >>>>>>> I built my module to probe a few system calls and functions. >>>>>>> >>>>>>> Nothing >>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>>>>> >>>>>>> simple >>>>>>> >>>>>>> python script to read data directly off the >>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>>>>> >>>>>>> running as >>>>>>> >>>>>>> sudo) >>>>>>> >>>>>>> I run stap in Flight recorder mode. >>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>>>>> >>>>>>> from >>>>>>> >>>>>>> the module... >>>>>>> >>>>>>> Any idea how to check why that is? >>>>>>> >>>>>>> dmesg does not report anything unusual... >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 11:13 ` Daniel Doron @ 2017-06-15 11:32 ` Arkady 2017-06-15 11:59 ` Arkady 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 11:32 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap One possible reason is a system update which caused lot of probes to trigger. On Thu, Jun 15, 2017 at 2:13 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote: > I will try that. > BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH > > On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >> This works fine >> >> sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F >> -o ./test.log -S 1,2 test.stp >> >> On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>> STAP simply quits in a couple of seconds >>> >>> sudo stap -F -o ./test.log -S 1,2 test.stp >>> 94089 >>> ps -ef | grep -v "grep" | grep 94089 >>> >>> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>> Interesting. I have reproduced the problem by running two tight bash >>>> loops simultaneously >>>> >>>> sudo stap -F -o ./test.log -S 1,2 test.stp >>>> while [ 1 ];do ls > /dev/null;done & >>>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en >>>> "\033[0;0H";done; >>>> >>>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC >>>> 2016 x86_64 x86_64 x86_64 GNU/Linux >>>> >>>> >>>> >>>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron >>>> <danielmeirdoron@gmail.com> wrote: >>>>> On that specific kernel about 8-10 hours. >>>>> On my machine with latest kernel, been running for 24 hours. >>>>> I need to check other distros. >>>>> >>>>> sudo stap -v -F -m mymodule test.stp >>>>> >>>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>>> Do you do something like >>>>>> >>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ? >>>>>> >>>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>> On that specific kernel about 8-10 hours. >>>>>>> On my machine with latest kernel, been running for 24 hours. >>>>>>> I need to check other distros. >>>>>>> >>>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>>>>>> >>>>>>>> How long time it runs for you? >>>>>>>> >>>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>>>>>> > This is free software; see the source for copying conditions. >>>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>>>>>> > >>>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>>>>>> > <danielmeirdoron@gmail.com> wrote: >>>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>>>>>> >> >>>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>>>>>> >> <danielmeirdoron@gmail.com> wrote: >>>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>>>>>> >>> >>>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>>>>>> >>> <danielmeirdoron@gmail.com> wrote: >>>>>>>> >>>> Sorry, I do not know what you mean by that... >>>>>>>> >>>> >>>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>>>>>> >>>> wrote: >>>>>>>> >>>>> Do you have the kernel version pinned? >>>>>>>> >>>>> >>>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>>>>>> >>>>>> Please send the stap script >>>>>>>> >>>>>> >>>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>> >>>>>>> Hi, >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> I built my module to probe a few system calls and functions. >>>>>>>> >>>>>>> Nothing >>>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>>>>>> >>>>>>> simple >>>>>>>> >>>>>>> python script to read data directly off the >>>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>>>>>> >>>>>>> running as >>>>>>>> >>>>>>> sudo) >>>>>>>> >>>>>>> I run stap in Flight recorder mode. >>>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>>>>>> >>>>>>> from >>>>>>>> >>>>>>> the module... >>>>>>>> >>>>>>> Any idea how to check why that is? >>>>>>>> >>>>>>> dmesg does not report anything unusual... >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> Thanks. >>>>>>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 11:32 ` Arkady @ 2017-06-15 11:59 ` Arkady 2017-06-15 12:01 ` Daniel Doron 0 siblings, 1 reply; 17+ messages in thread From: Arkady @ 2017-06-15 11:59 UTC (permalink / raw) To: Daniel Doron; +Cc: systemtap you can also try --suppress-time-limits On Thu, Jun 15, 2017 at 2:32 PM, Arkady <arkady.miasnikov@gmail.com> wrote: > One possible reason is a system update which caused lot of probes to trigger. > > On Thu, Jun 15, 2017 at 2:13 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote: >> I will try that. >> BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH >> >> On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>> This works fine >>> >>> sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F >>> -o ./test.log -S 1,2 test.stp >>> >>> On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>> STAP simply quits in a couple of seconds >>>> >>>> sudo stap -F -o ./test.log -S 1,2 test.stp >>>> 94089 >>>> ps -ef | grep -v "grep" | grep 94089 >>>> >>>> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>> Interesting. I have reproduced the problem by running two tight bash >>>>> loops simultaneously >>>>> >>>>> sudo stap -F -o ./test.log -S 1,2 test.stp >>>>> while [ 1 ];do ls > /dev/null;done & >>>>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en >>>>> "\033[0;0H";done; >>>>> >>>>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC >>>>> 2016 x86_64 x86_64 x86_64 GNU/Linux >>>>> >>>>> >>>>> >>>>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron >>>>> <danielmeirdoron@gmail.com> wrote: >>>>>> On that specific kernel about 8-10 hours. >>>>>> On my machine with latest kernel, been running for 24 hours. >>>>>> I need to check other distros. >>>>>> >>>>>> sudo stap -v -F -m mymodule test.stp >>>>>> >>>>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>>>> Do you do something like >>>>>>> >>>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ? >>>>>>> >>>>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >>>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>> On that specific kernel about 8-10 hours. >>>>>>>> On my machine with latest kernel, been running for 24 hours. >>>>>>>> I need to check other distros. >>>>>>>> >>>>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>>>>>>> >>>>>>>>> How long time it runs for you? >>>>>>>>> >>>>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>>>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>>>>>>> > This is free software; see the source for copying conditions. >>>>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>>>>>>> > >>>>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>>>>>>> > <danielmeirdoron@gmail.com> wrote: >>>>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>>>>>>> >> >>>>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>>>>>>> >> <danielmeirdoron@gmail.com> wrote: >>>>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>>>>>>> >>> >>>>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>>>>>>> >>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>> >>>> Sorry, I do not know what you mean by that... >>>>>>>>> >>>> >>>>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>>>>>>> >>>> wrote: >>>>>>>>> >>>>> Do you have the kernel version pinned? >>>>>>>>> >>>>> >>>>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>>>>>>> >>>>>> Please send the stap script >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>> >>>>>>> Hi, >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> I built my module to probe a few system calls and functions. >>>>>>>>> >>>>>>> Nothing >>>>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>>>>>>> >>>>>>> simple >>>>>>>>> >>>>>>> python script to read data directly off the >>>>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>>>>>>> >>>>>>> running as >>>>>>>>> >>>>>>> sudo) >>>>>>>>> >>>>>>> I run stap in Flight recorder mode. >>>>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>>>>>>> >>>>>>> from >>>>>>>>> >>>>>>> the module... >>>>>>>>> >>>>>>> Any idea how to check why that is? >>>>>>>>> >>>>>>> dmesg does not report anything unusual... >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> Thanks. >>>>>>>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: No output from trace file after a few hours 2017-06-15 11:59 ` Arkady @ 2017-06-15 12:01 ` Daniel Doron 0 siblings, 0 replies; 17+ messages in thread From: Daniel Doron @ 2017-06-15 12:01 UTC (permalink / raw) To: Arkady; +Cc: systemtap ok On Thu, Jun 15, 2017 at 2:59 PM, Arkady <arkady.miasnikov@gmail.com> wrote: > you can also try --suppress-time-limits > > On Thu, Jun 15, 2017 at 2:32 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >> One possible reason is a system update which caused lot of probes to trigger. >> >> On Thu, Jun 15, 2017 at 2:13 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote: >>> I will try that. >>> BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH >>> >>> On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>> This works fine >>>> >>>> sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F >>>> -o ./test.log -S 1,2 test.stp >>>> >>>> On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>> STAP simply quits in a couple of seconds >>>>> >>>>> sudo stap -F -o ./test.log -S 1,2 test.stp >>>>> 94089 >>>>> ps -ef | grep -v "grep" | grep 94089 >>>>> >>>>> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>>> Interesting. I have reproduced the problem by running two tight bash >>>>>> loops simultaneously >>>>>> >>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp >>>>>> while [ 1 ];do ls > /dev/null;done & >>>>>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en >>>>>> "\033[0;0H";done; >>>>>> >>>>>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC >>>>>> 2016 x86_64 x86_64 x86_64 GNU/Linux >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>> On that specific kernel about 8-10 hours. >>>>>>> On my machine with latest kernel, been running for 24 hours. >>>>>>> I need to check other distros. >>>>>>> >>>>>>> sudo stap -v -F -m mymodule test.stp >>>>>>> >>>>>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote: >>>>>>>> Do you do something like >>>>>>>> >>>>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ? >>>>>>>> >>>>>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron >>>>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>> On that specific kernel about 8-10 hours. >>>>>>>>> On my machine with latest kernel, been running for 24 hours. >>>>>>>>> I need to check other distros. >>>>>>>>> >>>>>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> How long time it runs for you? >>>>>>>>>> >>>>>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron >>>>>>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6) >>>>>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others >>>>>>>>>> > This is free software; see the source for copying conditions. >>>>>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP >>>>>>>>>> > >>>>>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron >>>>>>>>>> > <danielmeirdoron@gmail.com> wrote: >>>>>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64 >>>>>>>>>> >> >>>>>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron >>>>>>>>>> >> <danielmeirdoron@gmail.com> wrote: >>>>>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes. >>>>>>>>>> >>> >>>>>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron >>>>>>>>>> >>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>>> >>>> Sorry, I do not know what you mean by that... >>>>>>>>>> >>>> >>>>>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com> >>>>>>>>>> >>>> wrote: >>>>>>>>>> >>>>> Do you have the kernel version pinned? >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady >>>>>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote: >>>>>>>>>> >>>>>> Please send the stap script >>>>>>>>>> >>>>>> >>>>>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron >>>>>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote: >>>>>>>>>> >>>>>>> Hi, >>>>>>>>>> >>>>>>> >>>>>>>>>> >>>>>>> I built my module to probe a few system calls and functions. >>>>>>>>>> >>>>>>> Nothing >>>>>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a >>>>>>>>>> >>>>>>> simple >>>>>>>>>> >>>>>>> python script to read data directly off the >>>>>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course >>>>>>>>>> >>>>>>> running as >>>>>>>>>> >>>>>>> sudo) >>>>>>>>>> >>>>>>> I run stap in Flight recorder mode. >>>>>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output >>>>>>>>>> >>>>>>> from >>>>>>>>>> >>>>>>> the module... >>>>>>>>>> >>>>>>> Any idea how to check why that is? >>>>>>>>>> >>>>>>> dmesg does not report anything unusual... >>>>>>>>>> >>>>>>> >>>>>>>>>> >>>>>>> Thanks. >>>>>>>>>> >>>>>>> Daniel. ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2017-06-15 12:01 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-15 7:10 No output from trace file after a few hours Daniel Doron
2017-06-15 7:13 ` Arkady
2017-06-15 7:33 ` Arkady
2017-06-15 7:34 ` Daniel Doron
2017-06-15 7:37 ` Daniel Doron
2017-06-15 7:37 ` Daniel Doron
2017-06-15 7:38 ` Daniel Doron
2017-06-15 8:18 ` Arkady
[not found] ` <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com>
2017-06-15 8:26 ` Arkady
2017-06-15 8:59 ` Daniel Doron
2017-06-15 10:54 ` Arkady
2017-06-15 11:04 ` Arkady
2017-06-15 11:05 ` Arkady
2017-06-15 11:13 ` Daniel Doron
2017-06-15 11:32 ` Arkady
2017-06-15 11:59 ` Arkady
2017-06-15 12:01 ` Daniel Doron
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).