* DRAFT 2.1 release notes
[not found] <377658972.5114035.1359560272799.JavaMail.root@redhat.com>
@ 2013-01-30 15:38 ` Serguei Makarov
2013-02-06 17:27 ` Serguei Makarov
0 siblings, 1 reply; 8+ messages in thread
From: Serguei Makarov @ 2013-01-30 15:38 UTC (permalink / raw)
To: systemtap; +Cc: Frank Ch. Eigler
These are the current in-progress draft SystemTap 2.1 release notes.
Feedback would be appreciated.
Looking these over, it seems that more NEWS items are needed for
various work, in particular for progress on the dyninst backend.
====
The systemtap team announces release 2.1!
perf event probe improvements, dyninst backend improvements,
error:: and warning:: man pages, experimental regex support,
... TODO ...
= Where to get it
http://sourceware.org/systemtap/ - our project page
http://sourceware.org/systemtap/ftp/releases/systemtap-2.1.tar.gz
http://koji.fedoraproject.org/koji/packageinfo?packageID=615
git tag release-2.1 (commit ...)
There have been over ... commits since the last release.
There have been over ... bugs/features fixed since the last release.
= How to build it
See the README and NEWS files at
http://sourceware.org/git/?p=systemtap.git;a=tree
Further information at http://sourceware.org/systemtap/wiki/
= Systemtap frontend (stap) changes
- EMACS and VIM editor modes for systemtap source files are included / updated.
- The translator now eliminates duplicate tapset files between its
preferred directory (as configured during the build with --prefix=/
or specified with the -I /path option), and files it may find under
$XDG_DATA_DIRS. This should eliminate a class of conflicts between
parallel system- and hand-built systemtap installations.
- The translator accepts a --suppress-time-limits option, which defeats
time-related constraints, to allows probe handlers to run for indefinite
periods. It requires the guru mode (-g) flag to work. Add the earlier
--suppress-handler-errors flag for a gung-ho "just-keep-going" attitude.
- Some error messages and warnings now refer to additional information
that is found in man pages. These are generally named
error::FOO or warning::BAR (in the 7stap man page section)
and may be read via
% man error::FOO
% man warning::BAR
= Systemtap script language changes
- Perf event probes may now be read on demand. The counter probe is
defined using the counter-name part:
probe perf.type(0).config(0).counter("NAME"). The counter is
read in a user space probe using @perf("NAME"), e.g.
process("PROCESS").statement("func@file") {stat <<< @perf("NAME")}
- Perf event probes may now be bound to a specific task using the
process-name part: probe perf.type(0).config(0).process("NAME") { }
If the probed process name is not specified, then it is inferred
from the -c CMD argument.
- The foreach looping construct can now sort aggregate arrays by the user's
choice of aggregating function. Previously, @count was implied. e.g.:
foreach ([x,y] in array @sum +) { println(@sum(array[x,y])) }
- Proof of concept support for regular expression matching has been added:
if ("aqqqqqb" =~ "q*b") { ... }
if ("abc" !~ "q*b") { ... }
The eventual aim is to support roughly the same functionality as
the POSIX Extended Regular Expressions implemented by glibc.
Currently missing features include extraction of the matched string
and subexpressions, and named character classes ([:alpha:], [:digit:], &c).
= Systemtap runtime changes
- The dyninst backend's runtime has been improved to allow much more
concurrency when probing multithreaded processes.
= Systemtap tapset changes
- New tapsets:
choose_defined.stpm add @choose_defined() macro
- Added/changed tapsets for dyninst:
ctime.stp ctime() returns "<invalid time>" when an error occurs
context.stp add support for print_regs()
- Changed tapsets:
nfs_proc.stp account for separate nfsv2/3/4 kernel modules
context.stp add ppfunc() to parse the function name from pp()
...
= Systemtap sample scripts
- Tracing examples were updated to use ppfunc() instead of probefunc().
- New samples:
perf.stp use perf.counter to access performance counters
threadstacks.stp override default new-pthread stack sizes
ltrace.stp uprobes-based ltrace demo
block.stp stapgames Breakout demo using input/output devices
eater.stp stapgames Snake demo using input/output devices
lifegame.stp stapgames Game of Life demo using input/output devices
pingpong.stp stapgames Pong demo using input/output devices
- Changed samples:
deviceseeks.stp switch to using ioblock_trace.request probe point
... TODO regex example script ...
= Examples of tested kernel versions
... TODO ...
= Known issues with this release
... TODO ...
... TODO particularly the regex PRs ...
= Contributors for this release
Alexander Lochmann, Daniel Tschan*, David Smith, Frank Ch. Eigler,
Josh Stone, Jóhann B. Guðmundsson*, Lukas Berk, Mark Wielaard,
Masami Hiramatsu, Nathan Scott, Rüdiger Sonderfeld*, Serguei Makarov,
Stan Cox, Tom Tromey, Tomoki Sekiyama*, William Cohen
Special thanks to new contributors, marked with '*' above.
= Bugs fixed for this release <http://sourceware.org/PR#####>
1887 write emacs/vim editor mode for script language
6697 need more access_ok() checks in runtime/tapsets
11576 per-process perf events
12022 Allow foreach to order iterations for aggregates on @min @max and @avg operators
12231 reading perfctr events
12443 XDG_DATA_DIRS conflict between preinstalled and hand-built stap
12642 utrace: taskfinder misses events when main thread does not go through at least one quiesce
13022 probe module("*").function("*") does not work on arm
14178 Add a pure-userspace runtime mode
14245 Support debugfs mounted 0700
14390 separate process.**.return tests and mark UNSUPPORTED
14555 tolerate /proc/sys/kernel/kptr_restrict=2
14571 Make stapdyn multithreaded
14659 task_finder2 breaks ptrace apps
14701 stapdyn needs timer probe support
14707 Support stapdyn -x PID
14709 Support process.function.return in stapdyn
14712 VM_EXECUTABLE went to heaven
14714 stap flag for bigger --suppress-handler-errors hammer
14731 regression in buildok/thirty.stp
14735 stapdyn needs timing ('-t') support
14742 more complete dwz / DW_TAG_partial_unit support
14774 process.statement("*@file:*") probes don't match all functions
14781 kmem_cache conflict leads to panic & badness
14794 autoconf auto-enable dyninst if possible
14795 add stapgames
14804 Kernel panic when running hello world stap script
14820 hang when running semok/nodwf01.stp
14842 Old "-kelf" and "--ignore-dwarf" command line options should be removed
14883 let probefunc() not fail - or fail more informatively, on non-uprobes kernels
14900 collision between tapset locals and user globals: contexts
14929 staperrors man page
14949 cannot use systemtap on linux 3.7 intel x86_64
14957 foreach_aggrsort.exp test hangs on i686 rhel5 system
14997 DW_OP_rot mistranslation
15031 stapdyn needs a _stp_print_binary() runtime function
15059 session.print_error called frequently from listing_mode
15074 pthread_stacks.exp problems
... TODO additional closed PRs ...
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-01-30 15:38 ` DRAFT 2.1 release notes Serguei Makarov
@ 2013-02-06 17:27 ` Serguei Makarov
2013-02-06 21:43 ` Timo Juhani Lindfors
0 siblings, 1 reply; 8+ messages in thread
From: Serguei Makarov @ 2013-02-06 17:27 UTC (permalink / raw)
To: systemtap; +Cc: Frank Ch. Eigler
Newer draft of the release notes. Would particularly appreciate
someone looking over the 'Known Issues' section to see if something
should be added / taken away.
====
The systemtap team announces release 2.1!
perf event probe improvements, dyninst backend improvements,
error:: and warning:: man pages, bundled emacs/vim editor modes,
customizable aggregate array sorting, optional time limit suppression,
experimental regex support
= Where to get it
http://sourceware.org/systemtap/ - our project page
http://sourceware.org/systemtap/ftp/releases/systemtap-2.1.tar.gz
http://koji.fedoraproject.org/koji/packageinfo?packageID=615
git tag release-2.1 (commit ...)
There have been over ... commits since the last release.
There have been over ... bugs/features fixed since the last release.
= How to build it
See the README and NEWS files at
http://sourceware.org/git/?p=systemtap.git;a=tree
Further information at http://sourceware.org/systemtap/wiki/
= Systemtap frontend (stap) changes
- EMACS and VIM editor modes for systemtap source files are included / updated.
- The translator now eliminates duplicate tapset files between its
preferred directory (as configured during the build with --prefix=/
or specified with the -I /path option), and files it may find under
$XDG_DATA_DIRS. This should eliminate a class of conflicts between
parallel system- and hand-built systemtap installations.
- The translator accepts a --suppress-time-limits option, which defeats
time-related constraints, to allows probe handlers to run for indefinite
periods. It requires the guru mode (-g) flag to work. Add the earlier
--suppress-handler-errors flag for a gung-ho "just-keep-going" attitude.
- Some error messages and warnings now refer to additional information
that is found in man pages. These are generally named
error::FOO or warning::BAR (in the 7stap man page section)
and may be read via
% man error::FOO
% man warning::BAR
= Systemtap script language changes
- Perf event probes may now be read on demand. The counter probe is
defined using the counter-name part:
probe perf.type(0).config(0).counter("NAME"). The counter is
read in a user space probe using @perf("NAME"), e.g.
process("PROCESS").statement("func@file") {stat <<< @perf("NAME")}
- Perf event probes may now be bound to a specific task using the
process-name part: probe perf.type(0).config(0).process("NAME") { }
If the probed process name is not specified, then it is inferred
from the -c CMD argument.
- The foreach looping construct can now sort aggregate arrays by the user's
choice of aggregating function. Previously, @count was implied. e.g.:
foreach ([x,y] in array @sum +) { println(@sum(array[x,y])) }
- Proof of concept support for regular expression matching has been added:
if ("aqqqqqb" =~ "q*b") { ... }
if ("abc" !~ "q*b") { ... }
The eventual aim is to support roughly the same functionality as
the POSIX Extended Regular Expressions implemented by glibc.
Currently missing features include extraction of the matched string
and subexpressions, and named character classes ([:alpha:], [:digit:], &c).
Special thanks go to the re2c project, whose public domain code this
functionality has been based on. For more info on re2c, see:
http://sourceforge.net/projects/re2c/
= Systemtap runtime changes
- The dyninst backend's runtime has been improved to allow much more
concurrency when probing multithreaded processes.
= Systemtap tapset changes
- New tapsets:
choose_defined.stpm add @choose_defined() macro
- Added/changed tapsets for dyninst:
ctime.stp ctime() returns "<invalid time>" when an error occurs
context.stp add support for print_regs()
- Changed tapsets:
nfs_proc.stp account for separate nfsv2/3/4 kernel modules
context.stp add ppfunc() to parse the function name from pp()
netfilter.stp fix error in decoding of ar_tha and ar_tpa fields
- The folowing tapset variables are deprecated in release 2.1 and will
be removed in release 2.2:
- The 'send2queue' variable in the 'signal.send' probe.
- The 'oldset_addr' and 'regs' variables in the 'signal.handle' probe.
- The following tapset probes are deprecated in release 2.1 and will
be removed in release 2.2:
- signal.send.return
- signal.handle.return
= Systemtap sample scripts
- Tracing examples were updated to use ppfunc() instead of probefunc().
- New samples:
perf.stp use perf.counter to access performance counters
regex.stp report opened files matching a regex <TODOXXX>
threadstacks.stp override default new-pthread stack sizes
mutex_contention.stp pthread mutex contention analysis
ltrace.stp uprobes-based ltrace demo
block.stp stapgames Breakout demo using input/output devices
eater.stp stapgames Snake demo using input/output devices
lifegame.stp stapgames Game of Life demo using input/output devices
pingpong.stp stapgames Pong demo using input/output devices
- Changed samples:
deviceseeks.stp switch to using ioblock_trace.request probe point
= Examples of tested kernel versions
3.8.0 (x86_64)
3.7.0 (x86_64, i686)
3.6.2 (x86_64)
3.4.0 (armv7l)
2.6.32 (x86_64, i686, ppc64, s390x)
2.6.18 (x86_64, i686, ia64)
2.6.9 (i686, ppc64)
= Known issues with this release
- The regular expression support is still experimental and incomplete:
+ Character class support is somewhat broken, and named character classes are missing. (PR15064)
+ Support for extracting matched strings and substrings is missing. (PR15065)
- Some kernel crashes continue to be reported when a script probes
broad kernel function wildcards. (PR2725)
- Some systems mount debugfs with mode 0700, which prevents non-root
users from connecting to stap modules. To work around this issue,
try "chmod 0755 /sys/kernel/debug" as root. (PR14245)
- The dyninst backend is still a work in progress. See dyninst/README
and the systemtap/dyninst Bugzilla component
(http://tinyurl.com/stapdyn-PR-list) if you want all the gory
details about the state of the feature.
= Contributors for this release
Alexander Lochmann, Daniel Tschan*, David Smith, Frank Ch. Eigler,
Josh Stone, Jóhann B. Guðmundsson*, Lukas Berk, Mark Wielaard,
Masami Hiramatsu, Nathan Scott, Rüdiger Sonderfeld*, Serguei Makarov,
Stan Cox, Tom Tromey, Tomoki Sekiyama*, William Cohen
Special thanks to new contributors, marked with '*' above.
= Bugs fixed for this release <http://sourceware.org/PR#####>
1887 write emacs/vim editor mode for script language
6697 need more access_ok() checks in runtime/tapsets
11576 per-process perf events
11955 signal tapsets don't match
12022 Allow foreach to order iterations for aggregates on @min @max and @avg operators
12231 reading perfctr events
12443 XDG_DATA_DIRS conflict between preinstalled and hand-built stap
12642 utrace: taskfinder misses events when main thread does not go through at least one quiesce
13022 probe module("*").function("*") does not work on arm
14178 Add a pure-userspace runtime mode
14245 Support debugfs mounted 0700
14390 separate process.**.return tests and mark UNSUPPORTED
14555 tolerate /proc/sys/kernel/kptr_restrict=2
14571 Make stapdyn multithreaded
14659 task_finder2 breaks ptrace apps
14701 stapdyn needs timer probe support
14707 Support stapdyn -x PID
14709 Support process.function.return in stapdyn
14712 VM_EXECUTABLE went to heaven
14714 stap flag for bigger --suppress-handler-errors hammer
14731 regression in buildok/thirty.stp
14735 stapdyn needs timing ('-t') support
14742 more complete dwz / DW_TAG_partial_unit support
14774 process.statement("*@file:*") probes don't match all functions
14781 kmem_cache conflict leads to panic & badness
14794 autoconf auto-enable dyninst if possible
14795 add stapgames
14804 Kernel panic when running hello world stap script
14820 hang when running semok/nodwf01.stp
14842 Old "-kelf" and "--ignore-dwarf" command line options should be removed
14883 let probefunc() not fail - or fail more informatively, on non-uprobes kernels
14900 collision between tapset locals and user globals: contexts
14929 staperrors man page
14949 cannot use systemtap on linux 3.7 intel x86_64
14957 foreach_aggrsort.exp test hangs on i686 rhel5 system
14997 DW_OP_rot mistranslation
15031 stapdyn needs a _stp_print_binary() runtime function
15059 session.print_error called frequently from listing_mode
15074 pthread_stacks.exp problems
15080 The kernel function 'signal_wake_up' is gone
15087 netfilter.arp tapset probes duplicate source/target addresses
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-02-06 17:27 ` Serguei Makarov
@ 2013-02-06 21:43 ` Timo Juhani Lindfors
2013-02-06 21:55 ` Josh Stone
0 siblings, 1 reply; 8+ messages in thread
From: Timo Juhani Lindfors @ 2013-02-06 21:43 UTC (permalink / raw)
To: Serguei Makarov; +Cc: systemtap, Frank Ch. Eigler
Serguei Makarov <smakarov@redhat.com> writes:
> Newer draft of the release notes. Would particularly appreciate
> someone looking over the 'Known Issues' section to see if something
> should be added / taken away.
do you have some planned release date for 2.1? I'm terribly busy but I
did manage to find time to test it against the debian linux 3.7 packages
(currently in debian experimental). The testsuite output is at
http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.sum
but I didn't have much time to compare it against earlier
testsuites. Note that I did the tests under xen.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-02-06 21:43 ` Timo Juhani Lindfors
@ 2013-02-06 21:55 ` Josh Stone
2013-02-08 7:47 ` Timo Juhani Lindfors
0 siblings, 1 reply; 8+ messages in thread
From: Josh Stone @ 2013-02-06 21:55 UTC (permalink / raw)
To: Timo Juhani Lindfors; +Cc: Serguei Makarov, systemtap, Frank Ch. Eigler
On 02/06/2013 01:42 PM, Timo Juhani Lindfors wrote:
> Serguei Makarov <smakarov@redhat.com> writes:
>> Newer draft of the release notes. Would particularly appreciate
>> someone looking over the 'Known Issues' section to see if something
>> should be added / taken away.
>
> do you have some planned release date for 2.1? I'm terribly busy but I
> did manage to find time to test it against the debian linux 3.7 packages
> (currently in debian experimental). The testsuite output is at
>
> http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.sum
>
> but I didn't have much time to compare it against earlier
> testsuites. Note that I did the tests under xen.
We're hoping to release very soon, perhaps in the next few days.
Your results look very bad - even the first add.exp fails! Can you also
share the full systemtap.log to see what happened there?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-02-06 21:55 ` Josh Stone
@ 2013-02-08 7:47 ` Timo Juhani Lindfors
2013-02-08 17:59 ` Frank Ch. Eigler
2013-02-08 18:34 ` Josh Stone
0 siblings, 2 replies; 8+ messages in thread
From: Timo Juhani Lindfors @ 2013-02-08 7:47 UTC (permalink / raw)
To: Josh Stone; +Cc: Serguei Makarov, systemtap, Frank Ch. Eigler
Josh Stone <jistone@redhat.com> writes:
> Your results look very bad - even the first add.exp fails! Can you also
> share the full systemtap.log to see what happened there?
Yeah sorry about that, I forgot to add setuid bit for staprun and
thought make installcheck would use sudo automatically bit turns out it
only uses it for some tests and not all. I reran the testsuite after
adding setuid, the results are here:
http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.sum
http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.log
=== systemtap Summary ===
# of expected passes 927
# of unexpected failures 598
# of unexpected successes 3
# of expected failures 275
# of known failures 7
# of untested testcases 267
# of unsupported tests 2
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-02-08 7:47 ` Timo Juhani Lindfors
@ 2013-02-08 17:59 ` Frank Ch. Eigler
2013-02-08 18:34 ` Josh Stone
1 sibling, 0 replies; 8+ messages in thread
From: Frank Ch. Eigler @ 2013-02-08 17:59 UTC (permalink / raw)
To: Timo Juhani Lindfors; +Cc: Josh Stone, Serguei Makarov, systemtap
Timo Juhani Lindfors <timo.lindfors@iki.fi> writes:
> [...]
> Yeah sorry about that, I forgot to add setuid bit for staprun
> [...]
> http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.sum
> http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.log
> [...]
> # of expected passes 927
> # of unexpected failures 598
Those are still far worse then they should be; the .log file shows
similar permission problems. Try running, ** as root **
# make installcheck
- FChE
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-02-08 7:47 ` Timo Juhani Lindfors
2013-02-08 17:59 ` Frank Ch. Eigler
@ 2013-02-08 18:34 ` Josh Stone
2013-02-08 19:58 ` Timo Juhani Lindfors
1 sibling, 1 reply; 8+ messages in thread
From: Josh Stone @ 2013-02-08 18:34 UTC (permalink / raw)
To: Timo Juhani Lindfors; +Cc: Serguei Makarov, systemtap, Frank Ch. Eigler
On 02/07/2013 11:47 PM, Timo Juhani Lindfors wrote:
> Josh Stone <jistone@redhat.com> writes:
>> Your results look very bad - even the first add.exp fails! Can you also
>> share the full systemtap.log to see what happened there?
>
> Yeah sorry about that, I forgot to add setuid bit for staprun and
> thought make installcheck would use sudo automatically bit turns out it
> only uses it for some tests and not all. I reran the testsuite after
> adding setuid, the results are here:
You still have this bit in the log:
> ERROR: The effective user ID of staprun must be set to the root user.
> Check permissions on staprun and ensure it is a setuid root program.
Just setting the setuid bit makes the process run as the owner of the
file, so it must also be owned by root to be able to load modules.
You can also run the entire installcheck as root, like Frank suggests,
but if you have staprun now as setuid non-root, it will effectively be
defeated by *dropping* permission.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: DRAFT 2.1 release notes
2013-02-08 18:34 ` Josh Stone
@ 2013-02-08 19:58 ` Timo Juhani Lindfors
0 siblings, 0 replies; 8+ messages in thread
From: Timo Juhani Lindfors @ 2013-02-08 19:58 UTC (permalink / raw)
To: Josh Stone; +Cc: Serguei Makarov, systemtap, Frank Ch. Eigler
Josh Stone <jistone@redhat.com> writes:
> You can also run the entire installcheck as root, like Frank suggests,
> but if you have staprun now as setuid non-root, it will effectively be
> defeated by *dropping* permission.
Yeah sorry for wasting your time. After sudo make installcheck the
results look bit better:
=== systemtap Summary ===
# of expected passes 1627
# of unexpected failures 86
# of unexpected successes 3
# of expected failures 278
# of known failures 7
# of untested testcases 276
# of unsupported tests 2
http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.sum
http://lindi.iki.fi/lindi/systemtap/testsuite_43e1259c0fea19e3348d2adbfe5a65e9abde30ca.log
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2013-02-08 19:58 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <377658972.5114035.1359560272799.JavaMail.root@redhat.com>
2013-01-30 15:38 ` DRAFT 2.1 release notes Serguei Makarov
2013-02-06 17:27 ` Serguei Makarov
2013-02-06 21:43 ` Timo Juhani Lindfors
2013-02-06 21:55 ` Josh Stone
2013-02-08 7:47 ` Timo Juhani Lindfors
2013-02-08 17:59 ` Frank Ch. Eigler
2013-02-08 18:34 ` Josh Stone
2013-02-08 19:58 ` Timo Juhani Lindfors
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).