systemtap release 4.5

systemtap release 4.5

[Serhei Makarov]

The SystemTap team announces release 4.5

Enhancements to this release include updated Java probing support,
floating-point variable access via $context variables, $context
variable support for enum values, bpf uconversions tapset, significant
concurrency/locking improvements for stable operation on large busy
servers.

= Where to get it

  https://sourceware.org/systemtap/ - our project page
  https://sourceware.org/systemtap/ftp/releases/
  https://koji.fedoraproject.org/koji/packageinfo?packageID=615
  git tag release-4.5 (commit 0eba8a46bc99c66e5dd274a9a4d661fe08ac4b8a)

  There have been over 155 commits since the last release.
  There have been 23 bugs fixed / features added since the last release.

= SystemTap frontend (stap) changes

- Enum values can now be accessed as $context variables (PR25346).

- Executables for which stap has execute but not read permissions
  (--x--x--x) can now be probed with build-id probes and debuginfod.
  This allows probing some setuid programs (PR27251).

- Enabled the -c option to work with the bpf backend (PR25177).

- Floating point variables may now be accessed directly as normal
  $context variables.  32-bit floats are automatically widened to
  doubles. (PR13838)

- On startup, stap explains the [man FOO] diagnostic syntax.  But we bet
  those reading this note already knew it!

= SystemTap backend changes

- Numerous fixes and improvements to concurrency control in the uprobes
  and transport subsystems, to more reliably handle large busy machines.

- Java probing support has been updated to work with the latest versions of the
  JVM and byteman (PR27739).

- Added VMA-tracking support to the stapdyn backend.

- The kernel runtime now uses procfs as the default transport mechanism.

- stap-server better handles NSS dbm->sqlite database transitions.

- A wider variety of SecureBoot MOK keys are now recognized for
  more reliable triggering of stap-server module signing.  Document
  some additional lockdown/secureboot administrative escape options.
  (PR26665)

= SystemTap tapset changes

- An initial version of the bpf/uconversions.stp tapset
  provides tapset functions such as user_long_error()
  to access values in userspace.

= SystemTap sample scripts

- All 190+ examples can be found at https://sourceware.org/systemtap/examples/

- The floatingpoint.stp sample script has been updated to use new
   floating-point variable access syntax.

- New sample scripts:

  cve-2011-4127.stp
  Historical emergency security band-aid script for example purposes only

= Examples of tested kernel versions 

  2.6.32 (RHEL6 i686, x86_64)
  3.10.0 (RHEL7 x86_64)
  4.15.0 (Ubuntu 18.04 x86_64)
  4.18.0 (RHEL8 x86_64, aarch64, ppc64le, s390x)
  5.3.8  (Fedora 30 i686)
  5.9.0-rc7   (Fedora rawhide x86_64)
  5.10.0-rc1  (Fedora rawhide x86_64)
  5.11.11 (Fedora 33 x86_64)
  5.11.15 (Fedora 34 x86_64)
  5.11.16 (Fedora 32 x86_64)
  5.12.0-rc2 (Fedora rawhide x86_64)

= Known issues with this release

- Some of the testcases for the BPF backend have uncovered an issue
  in the kernel's BPF runtime that can deadlock the entire kernel.
  The problem has been reported upstream. It was first seen during
  testing on kernel 5.11-rc6 but may occur intermittently in earlier
  kernel versions.
  More info at https://bugzilla.redhat.com/show_bug.cgi?id=1938312

- There are known issues on kernel 5.10+ after adapting to set_fs()
  removal, with some memory accesses that previously returned valid data
  instead returning -EFAULT (see PR26811).

- An sdt probe cannot parse a parameter that uses a segment register.
  (PR13429)

- The presence of a line such as
      *CFLAGS += $(call cc-option, -fno-var-tracking-assignments)
  in older linux kernel Makefile unnecessarily reduces debuginfo quality,
  consider removing that line if you build kernels.  Linux 5.10+ fixes this.

= Contributors for this release

Aaron Merey, Alice Zhang, Cosmin Tanislav*, Craig Ringer, Ding Hui*,
Frank Ch. Eigler, Guillaume Morin*, Jamie Bainbridge*, Kamil Dudka*,
Martin Cermak, Serhei Makarov, Stan Cox, Sultan Alsawaf, Sven Wegener*,
Tom Stellard*, William Cohen, Yichun Zhang (agentzh)

Special thanks to new contributors, marked with '*' above.

= Bugs fixed for this release <https://sourceware.org/PR#####>

15387 Add stapdyn functions to get/set errno
25177 stap --bpf -c does not work as specified
25346 support enum context variables
26844 off-by-one error in the output of sprint_ubacktrace()
26958 kernel crash when staprun ko which build id not matched
27001 4.4 runtime/transport/transport.c: security_locked_down can be undeclared depending the kernel config
27031 bpf: 'missing conversion specifier' when printing @var()
27044 lock loop on some conditional probes
27067 stapusr regression in staprun
27152 Missing tracepoints for for stap --bpf -L 'kernel.trace("*")'
27168 configure.ac contains non-posix test operator
27185 Reduce code duplication in conversions stress tests
27186 Document CONTEXT variable in embedded C docs
27251 build-id based probes should support unreadable executables via debuginfod
27273 kernel 5.11 porting
27274 stap: staptree.h:1349: virtual update_visitor::~update_visitor(): Assertion `values.empty()' failed.
27378 stap-server should support cert databases without nss dbm:
27392 build fails with --enable-htmldocs
27553 broken usym* lookups for some executables
27658 Swallowed spawn error (due to "Cannot allocate memory") causes unrelated problems later in build process (pass2)
27739 port java probing support to modern jvm/byteman
27785 The ioctl_handler.stp example causes stap to segmentation fault
27802 recent commit breaks --libexecdir 

systemtap release 4.6

[Serhei Makarov]

The SystemTap team announces release 4.6

support for 64-bit RISC-V architecture, liveness analysis for guru-mode write operations,
bpf syscall_any and abort() tapsets, bpf foreach iteration of multi-key arrays,
return of inter-cpu output ordering

= Where to get it

  https://sourceware.org/systemtap/ - our project page
  https://sourceware.org/systemtap/ftp/releases/
  https://koji.fedoraproject.org/koji/packageinfo?packageID=615
  git tag release-4.6 (commit 2e9f2f6967e44ce2bf8f34932b5bdd738ece2161)

  There have been over 149 commits since the last release.
  There have been 25 bugs fixed / features added since the last release.

= SystemTap frontend (stap) changes

- stap-prep now tries to download the main kernel debuginfo file from
  a debuginfod server, if configured.

= SystemTap backend changes

- SystemTap has added support for the 64-bit RISC-V architecture.

- SystemTap now uses DynInst to perform a liveness analysis on
  target variables and warn when a guru-mode modification to a variable
  will have no effect. The liveness analysis is currently done on
  x86_64, PowerPC, and AArch64.

- The kernel-user relayfs transport again sorts messages into a total
  time order across CPUs.  High output-volume scripts may need a
  larger "-s BUF" parameter to reliably transfer.  "-b" bulk mode
  is also available again as an alternative.
  
- The bpf backend now supports foreach iteration in multi-key associative arrays.

= SystemTap tapset changes

- Updated syscall_any tapset mapping to include newer syscalls.

- syscall_any tapset can be used by the bpf backend.

- abort() tapset can be used by the bpf backend.

= SystemTap sample scripts

- All 190+ examples can be found at https://sourceware.org/systemtap/examples/

- New sample scripts:

   io/traceaio.stp
   Trace calls to io_submit, logging iocbs and iovecs.

- The stopwatch.stp sample script is now confirmed to work with the BPF backend:

= Examples of tested kernel versions 

  2.6.32 (RHEL6 i686, x86_64)
  3.10.0 (RHEL7 x86_64)
  4.15.0 (Ubuntu 18.04 x86_64)
  4.18.0 (RHEL8 x86_64, aarch64, ppc64le, s390x)
  5.14.14 (Fedora 35 x86_64)
  5.14.16 (Fedora 34 x86_64)
  5.15.0-rc7  (Fedora rawhide x86_64)

= Known issues with this release

- There are known issues on kernel 5.10+ after adapting to set_fs()
  removal, with some memory accesses that previously returned valid data
  instead returning -EFAULT (see PR26811).

- An sdt probe cannot parse a parameter that uses a segment register.
  (PR13429)

- The presence of a line such as
      *CFLAGS += $(call cc-option, -fno-var-tracking-assignments)
  in older linux kernel Makefile unnecessarily reduces debuginfo quality,
  consider removing that line if you build kernels.  Linux 5.10+ fixes this.

= Contributors for this release

Aaron Merey, Alexandra Hájková*, Di Chen*, Du Zhe*, Frank Ch. Eigler
Junlong Li*, Nir Soffer*, Serhei Makarov, Sevan Janiyan*, Stan Cox
Sultan Alsawaf, Timm Bäder*, William Cohen

Special thanks to new contributors, marked with '*' above.

= Bugs fixed for this release <https://sourceware.org/PR#####>

6562     $SYSTEMTAP_DEBUGINFO_PATH does not work
15724     stapdyn looking for libdyninstAPI_RT.a
26839     Systemtap build failures with clang
27820     abort() tapset not implemented in the bpf mode
27829     support for floating point values passed through sdt.h markers
27864     loc2stap.cxx assertion failure on loc_unavailable type location, rawhide
27881     failed to extend vma mapped entry when the address is adjacent
27903     handle f33 glibc $$parms
27932     List Python as a prerequisite in README
27933     Use of unitialized functioncall synthetic field in
27934     failure to attach statement
27940     The /* pc=0x... */ is no longer printed by "stap -v -L 'kernel.function("*")'
27942     testsuite/systemtap.base/perf.sh drop bashism
27984     stap skipping partially-inlined instance, but it is not inline function actually
28070     extend vma end address to the different module
28079     adapt to kernel 5.14 task_struct.__state change
28084     autoconf-x86-uniregs.c compile failled with -Werror cause STAPCONF_X86_UNIREGS missing
28140     kernel panic on tracepoint activation in stap module
28184     task_fd_lookup failed on linux 5.11
28244     linux objtool imposes symbol length limits on generated function names
28384     finish nfs_proc tapset port 4.3 string server_ip
28443     Provide syscall_any tapset for bpf
28449     loss of cross-cpu output ordering
28544     procfs_bpf.exp regression due to string handling error 
28557     module kprobe insertion on modern kernels