The systemtap team announces release 1.3.

FIXME - better summary?

  integrated compile server client, automatic structure printing, user
  space stack traces, interesting new sample scripts, kernel versions
  2.6.9 through 2.6.35-rc4

= Where to get it

  http://sourceware.org/systemtap/ - our project page
  http://sourceware.org/systemtap/ftp/releases/systemtap-1.3.tar.gz
  http://koji.fedoraproject.org/koji/packageinfo?packageID=615
  git tag release-1.3 (commit FIXME)

= How to build it

  See the README and NEWS files at
  http://sourceware.org/git/?p=systemtap.git;a=tree
  Further information at http://sourceware.org/systemtap/wiki/

= Systemtap frontend (stap) changes

FIXME - anything else?

- A new integrated compile server client is now available as part of stap.

  o 'stap --server ...' is equivalent to 'stap-client ...'
  o 'stap --server-status' is equivalent to 'stap-find-servers'
  o 'stap --server-status=online' is equivalent to 'stap-find-servers --all'
  o stap-client and its related tools will soon be deprecated.
  o the nss-devel and avahi-devel packages are required for building stap with
    the integrated client (checked during configuration).
  o nss and avahi are required to run the integrated client.

- The new "--ldd" option automatically adds any additional shared
  libraries needed by probed or -d-listed userspace binaries to the -d
  list, to enable symbolic backtracing through them.  Similarly, the
  new "--all-modules" option automatically adds any currently loaded
  kernel modules (listed in /proc/modules) to the -d list.

- By default the systemtap-runtime RPM builds now include a shared
  library, staplog.so, that allows crash to extract systemtap data from
  a vmcore image.

- Backward compatibility flags (--compatible=VERSION, and matching
  script preprocessing predicate %( systemtap_v CMP "version" %)
  and a deprecation policy are being introduced, in case future
  tapset/language changes break valid scripts.

- The <sys/sdt.h> user-space markers no longer default to an implicit
  MARKER_NAME_ENABLED() semaphore check for each marker.  To check for
  enabled markers use a .d declaration file, then:
     if (MARKER_NAME_ENABLED()) MARKER_NAME()

- Hyphenated <sys/sdt.h> marker names such as process(...).mark("foo-bar")
  are now accepted in scripts.  They are mapped to the double-underscore
  form ("foo__bar").

- Prototype support for debuginfo-less <sys/sdt.h> user-space markers
  is included for some architectures (i686 and x86-64 initially).
  Compile your sdt.h-instrumented program with -DSTAP_SDT_V2 to
  activate this mode.  This implementation also supports preserving
  the "provider" name associated with a marker:
    probe process("foo").provider("bar").mark("baz") to match
    STAP_PROBE<n>(bar, baz <...>)

= Systemtap script language changes

FIXME - anything else?

- Iterating with "foreach" can now explicitly save the value for the loop.
     foreach(v = [i,j] in array)
       printf("array[%d,%s] = %d\n", i, j, v /* array[i,j] */)

- A new operator @entry is available for automatically saving an expression
  at entry time for use in a .return probe.
   probe foo.return { println(get_cycles() - @entry(get_cycles())) }

- Probe $target variables and @cast() can now use a suffix to print complex
  data types as strings.  Use a single '$' for a shallow view, or '$$' for a
  deeper view that includes nested types.  For example, with fs_struct:
   $fs$ : "{.users=%i, .lock={...}, .umask=%i,
            .in_exec=%i, .root={...}, .pwd={...}}"
   $fs$$ : "{.users=%i, .lock={.raw_lock={.lock=%u}}, .umask=%i, .in_exec=%i,
             .root={.mnt=%p, .dentry=%p}, .pwd={.mnt=%p, .dentry=%p}}"

- Embedded-C may be used within expressions as values, when in guru mode:
     num = %{ LINUX_VERSION_CODE %}               // int64_t
     name = %{ /* string */ THIS_MODULE->name %}  // const char*
     printf ("%s %x\n", name, num)
  The usual /* pure */, /* unprivileged */, and /* guru */ markers may be used 
  as with embedded-C functions.

- Probe wildcards can now use '**' to cross the '.' separator.
     $ stap -l 'sys**open'
     syscall.mq_open
     syscall.open

= Systemtap tapset changes

FIXME - anything else?

- It is now possible in some situations to use print_ubacktrace() to
  get a user space stack trace from a kernel probe point. e.g. for
  user backtraces when there is a pagefault:
  $ stap -d /bin/sort --ldd -e 'probe vm.pagefault {
      if (pid() == target()) {
         printf("pagefault @0x%x\n", address); print_ubacktrace();
      } }' -c /bin/sort
  [...]
  pagefault @0x7fea0595fa70
   0x000000384f07f958 : __GI_strcmp+0x12b8/0x1440 [libc-2.12.so]
   0x000000384f02824e : __gconv_lookup_cache+0xee/0x5a0 [libc-2.12.so]
   0x000000384f021092 : __gconv_find_transform+0x92/0x2cf [libc-2.12.so]
   0x000000384f094896 : __wcsmbs_load_conv+0x106/0x2b0 [libc-2.12.so]
   0x000000384f08bd90 : mbrtowc+0x1b0/0x1c0 [libc-2.12.so]
   0x0000000000404199 : ismbblank+0x39/0x90 [sort]
   0x0000000000404a4f : inittables_mb+0xef/0x290 [sort]
   0x0000000000406934 : main+0x174/0x2510 [sort]
   0x000000384f01ec5d : __libc_start_main+0xfd/0x1d0 [libc-2.12.so]
   0x0000000000402509 : _start+0x29/0x2c [sort]
  [...]

- New tapset functions to get a string representation of a stack trace:
  sprint_[u]backtrace() and sprint_[u]stack().

- New tapset function to get the module (shared library) name for a
  user space address umodname:string(long). The module name will now
  also be in the output of usymdata() and in backtrace addresses even
  when they were not given with -d at the command line.

- A new family of set_kernel_* functions make it easier for gurus to write
  new values at arbitrary memory addresses.

= New script examples

- profiling/errno.stp      Show which processes and system calls
                           return errors most frequently
- io/deviceseeks.stp       Histograms of seek behavior for each device
- process/noptrace.stp     Disable ptrace(2) from hierarchies of processes
- general/sizeof.stp       Print the size of a C type
- profiling/linetimes.stp  Show time spent on each line if a function
- process/cycle_thief.stp  Track IRQs and other processes stealing
                           cycles from a task

= Contributors for this release

  Adrien Kunysz Charley Wang, Dave Brolley, David Smith, Don Domingo,
  Frank Ch. Eigler, Josh Stone, Lars R. Damerow, Lukas Berk, Mark
  Wielaard, Przemysław Pawełczyk, Roland McGrath, Srikar Dronamraju,
  Stan Cox, Tom Callaway, Tony Jones, Wenji Huang, William Cohen

= Examples of tested kernel versions

  2.6.9 (el4/i686)
  2.6.18 (el5/ia64/i686/x86_64)
  2.6.32 (el6/i686/x86_64)
  2.6.32 (f12/i686/x86_64)
  2.6.33 (f13/i686/x86_64)
  2.6.35-rc4 (rawhide/x86_64)

= Known issues with this release

FIXME any other issues?

- When using the systemtap client and server udp port 5353 must be
  open in your firewall in order for the clent to find servers using
  avahi-browse.

- The stap-server may not be sufficiently robust to operate on an open
  untrusted network.

- Some kernel crashes continue to be reported when a script probes
  broad kernel function wildcards.

= Bugs fixed for this release

   1155  inline function parameters
   2049  revise $target->subfield syntax
   2168  incomplete translation of histogram bucket foreach
   2228  benchmark suite
   3672  Support formatted dump of struct $pointers
   3833  nfs_proc.stp and LKET/nfs_proc.stp tapset files broken
   4396  script to dump hung task's kernel stack
   4529  tapset coverage option
   4978  need better printf() documentation
   5151  rpc-all-probes.stp fails on 2.6.23-rc9 kernel
   5621  RFE: support for a general buffer kread()
   5949  instruction/block tracing for userspace
   6582  register() needs pt_regs check
   6731  improve -l (listing mode) to list probe/alias variables
   6739  global alias cache too slow
   6819  stapprobes(5) - syntactically (in)valid probepoints
   6826  shorthand or variable for probed executable's path
   6898  marker $$args / $$parms
   6902  optimize use of elfutils offline module search
   7030  signal tapset may be referring to inline functions.
   7070  registration error (rc -22), module probes, missing reloc info
   9958  Provide a way for C tapset functions to assert guru mode
   9998  Update tapset/i686/registers.stp per pt_regs change
  10025  wildcards lose when embedded in conditional %(%) expressions
  10054  improve $expr->foo->bar->zoo error messages
  10065  Create modules for Hardware Breakpoint
  10115  Regression with inet_sock @cast from 0.9 to 0.9.7 on 2.6.9 kernel
  10228  Add more vma-tracking for user space symbol/backtraces
  10245  Allow "-" in static marker names
  10327  follow symbol aliases for debuginfo search
  10393  sdt.exp utrace tests fail on 2.6.31 rawhide kernel
  10599  Always print module/map names for kernel/executable addresses
  10665  backtrace() is too slow
  10830  new pp() variant for source-level probe point name
  10913  bad/misleading translator error message
  10943  Support generic @entry saving in .return probes
  10978  Getting the size of a data type
  11095  Support "provider" for mark() probes
  11222  ABRT interface
  11266  var_expanding_visitor barfs on non-assignment lvalues
  11340  directly-accessed tracepoint args are surviving target_symbols
  11343  backward compatibility flags/conditionals
  11399  Allow ** in matching probe points
  11425  probefunc() broken when used with kprobe.function
  11427  systemtap 1.2 does not compile on sparc/sparc64
  11443  Add a "list unprivileged" option to stap
  11446  Server installed by systemtap-server rpm should be locally trusted
  11455  use XDG_DATA_DIRS for tapset path searching
  11473  Support optional "VALUE =" in foreach loops
  11474  iotop example is very slow in pass-2
  11480  bad kfail for rpc-all-probes.stp test
  11498  don't have a cow with oversize debug_frame
  11508  automate -d option for shared library searching
  11514  nd_syscall.exp takes 10 times longer than syscall.exp testcase
  11553  need better message for return probe on inline function
  11556  @cast() with array indexing on pointers
  11566  embedded-C expressions
  11573  uprobe_get_pc, sayonara
  11575  TRYLOCKDELAY too short
  11590  optimized pre/postfix operators throw away initial assignment
  11592  sdt.h should have a 'disable-semaphores' override option
  11598  @defined() should allow the address-of operator "&"
  11599  Support data pointer in tracepoint callbacks
  11609  -d ALL_LOADED_MODULES option
  11617  Gluing string literals doesn't work with token substitution
  11641  FAIL: fib (19 1) backtrace from uretprobe fails
  11649  workarounds for incomplete kernel include/linux/events/*.h decls
  11661  recvmmsg missing from syscall tapset
  11663  switch to embedded-c-expressions for kernel-marker tapset functions
  11664  try-catch squashes normal exits from next/return
  11665  umodname tapset function
  11672  utrace_report_syscall_exit crash
  11678  stap -d /bin/prog --ldd doesn't pick up ld.so
  11680  Systemtap build broken on PPC64
  11684  eliminate duplication between language reference guide and tapset docs
  11690  Semaphores are broken under STAP_SDT_V2
  11698  stap fails to figure out powerpc function aliases
  11706  unautoconfiscation of most stap*.in man pages
  11707  on ppc, all nd_syscall probes fail
  11708  SDT on i686 is register-poor
  11710  on s390x, all nd_syscall probes fail
  11713  listing of available $variables should include $
  11719  on ppc, the cmd_parse.exp and preprocessor.exp test fail
  11721  the usymbols.exp testcase shouldn't touch '__user' memory directly
  11722  on ppc, usymbols.exp should be probing syscall.rt_sigaction32
  11723  transok/buildko.stp fails incorrectly
  11725  on RHEL5 ppc, systemtap.printf/bin6.exp fails
  11726  on ppc, systemtap.printf/memory1.exp fails with compile errors
  11752  on ppc, the signal.handle_signal tapset alias needs to be updated
  11761  stap -L 'kernel.mark("*")' only lists $arg1..$argN variables
  11762  the systemtap.base/flightrec*.exp tests randomly fail
  11771  on ppc, the syscall.pipe tapset alias fails when accessing pipe0/pipe1
  11778  sprint_ustack & sprint_stack 

FIXME also list CVEs? RHBZs?

= Test results on various systems

- After running "sudo make installcheck" from the test suite, on a
  suitably equipped machine (kernel debugging data and other stuff
  installed), you should see 1000-1100 passes and a small handful of
  failures.  Those enjoying utrace in their kernels should see several
  hundred more passes.