[Bug runtime/22582] New: SystemTap 3.2 stap script compilation error opt

[Bug runtime/22582] New: SystemTap 3.2 stap script compilation error opt

[mysecondaccountabc at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

            Bug ID: 22582
           Summary: SystemTap 3.2 stap script compilation error opt
           Product: systemtap
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: runtime
          Assignee: systemtap at sourceware dot org
          Reporter: mysecondaccountabc at gmail dot com
  Target Milestone: ---

I'm having the following compilation error compiling a stap script in ARM with
SystemTap 3.2:

$ sudo stap -v -g -p4 -DSTP_NO_OVERLOAD systemtap-strace.stp -m stap_trace 
Compiling systemtap module ...
./systemtap-strace.stp -> ./stap_trace.ko
Pass 1: parsed user script and 452 library scripts using
35092virt/26404res/2260shr/24360data kb, in 4950usr/190sys/5187real ms.
Pass 2: analyzed script: 734 probes, 230 functions, 104 embeds, 63 globals
using 68040virt/60300res/3180shr/57308data kb, in
2348300usr/36720sys/2415777real ms.
Pass 3: translated to C into "/tmp/stapBqLAih/stap_trace_src.c" using
68040virt/60492res/3412shr/57308data kb, in 121940usr/440sys/124281real ms.
/tmp/stapBqLAih/stap_trace_src.c:198942:28: error: field ‘ops’ has incomplete
type
/tmp/stapBqLAih/stap_trace_src.c: In function ‘systemtap_module_init’:
/tmp/stapBqLAih/stap_trace_src.c:199900:9: error: implicit declaration of
function ‘stap_utrace_detach_ops’ [-Werror=implicit-function-declaration]
cc1: all warnings being treated as errors
make[1]: *** [/tmp/stapBqLAih/stap_trace_src.o] Error 1
make: *** [_module_/tmp/stapBqLAih] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_trace.ko" in 143290usr/13730sys/166637real ms.
Pass 4: compilation failed.  [man error::pass4]


- The script is the same that is running in the a x86 machine using the same
SystemTap version.
- Systemtap seems to be working with simple scripts:

$ sudo stap -ve 'probe begin { log("hello world") exit () }'
[sudo] password for jd: 
Pass 1: parsed user script and 452 library scripts using
35216virt/26348res/2244shr/24484data kb, in 5990usr/570sys/6758real ms.
Pass 2: analyzed script: 1 probe, 2 functions, 0 embeds, 0 globals using
35612virt/27116res/2600shr/24880data kb, in 90usr/0sys/92real ms.
Pass 3: translated to C into
"/tmp/stapUh7ouq/stap_3bc557db95fcecfd9b772ad47121f60e_1080_src.c" using
35612virt/27440res/2892shr/24880data kb, in 20usr/0sys/18real ms.
Pass 4: compiled C into "stap_3bc557db95fcecfd9b772ad47121f60e_1080.ko" in
123230usr/12460sys/139093real ms.
Pass 5: starting run.
hello world
Pass 5: run completed in 100usr/300sys/746real ms.

Versions:
- Custom Kernel 3.2.93-versatile
  $ uname -a
  Linux alex-14f9b262f2 3.2.93-versatile-gcm #1 Wed Dec 6 11:56:12 EST 2017
armv5tejl GNU/Linux
- Gcc 4.7.2 (Yes, I'm aware about the C++11 standard support needed, but please
have a look to https://sourceware.org/bugzilla/show_bug.cgi?id=22572).
- SystemTap 3.2/0.152, was compiled from the oficial release sources, just
patching staptree.h due to the C++11 issue).

--- staptree.h_ori      2017-12-10 23:23:25.872864000 -0500
+++ staptree.h  2017-12-10 23:23:36.332864000 -0500
@@ -518,10 +518,10 @@
     unsigned base;
     unsigned width;
     unsigned precision;
-    unsigned flags : 8;
-    width_type widthtype : 8;
-    precision_type prectype : 8;
-    conversion_type type : 8;
+    unsigned flags;
+    width_type widthtype;
+    precision_type prectype;
+    conversion_type type;
     interned_string literal_string;
     bool is_empty() const
     {

Any clue?

I could attach a full (-vv) output and the temporary files (-k) if it is
needed.

Cheers,
Gus

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

David Smith <dsmith at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2017-12-11
                 CC|                            |dsmith at redhat dot com
     Ever confirmed|0                           |1

--- Comment #1 from David Smith <dsmith at redhat dot com> ---
(In reply to Gustavo Moreira from comment #0)
> I'm having the following compilation error compiling a stap script in ARM
> with SystemTap 3.2:
> 
> $ sudo stap -v -g -p4 -DSTP_NO_OVERLOAD systemtap-strace.stp -m stap_trace 
> Compiling systemtap module ...
> ./systemtap-strace.stp -> ./stap_trace.ko
> Pass 1: parsed user script and 452 library scripts using
> 35092virt/26404res/2260shr/24360data kb, in 4950usr/190sys/5187real ms.
> Pass 2: analyzed script: 734 probes, 230 functions, 104 embeds, 63 globals
> using 68040virt/60300res/3180shr/57308data kb, in
> 2348300usr/36720sys/2415777real ms.
> Pass 3: translated to C into "/tmp/stapBqLAih/stap_trace_src.c" using
> 68040virt/60492res/3412shr/57308data kb, in 121940usr/440sys/124281real ms.
> /tmp/stapBqLAih/stap_trace_src.c:198942:28: error: field ‘ops’ has
> incomplete type
> /tmp/stapBqLAih/stap_trace_src.c: In function ‘systemtap_module_init’:
> /tmp/stapBqLAih/stap_trace_src.c:199900:9: error: implicit declaration of
> function ‘stap_utrace_detach_ops’ [-Werror=implicit-function-declaration]
> cc1: all warnings being treated as errors

Interesting. The task_finder functionality (part of the user-space tracing),
needs for several kernel tracepoints to exist. If they don't exist, you should
have either gotten a compile error or they should have semi-gracefully just
refused to work.

Is CONFIG_TRACEPOINTS set in your custom kernel?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[mysecondaccountabc at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

--- Comment #2 from Gustavo Moreira <mysecondaccountabc at gmail dot com> ---
Created attachment 10680
  --> https://sourceware.org/bugzilla/attachment.cgi?id=10680&action=edit
stap temporary files

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[mysecondaccountabc at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

--- Comment #3 from Gustavo Moreira <mysecondaccountabc at gmail dot com> ---
(In reply to David Smith from comment #1)
> (In reply to Gustavo Moreira from comment #0)
> > I'm having the following compilation error compiling a stap script in ARM
> > with SystemTap 3.2:
> > 
> > $ sudo stap -v -g -p4 -DSTP_NO_OVERLOAD systemtap-strace.stp -m stap_trace 
> > Compiling systemtap module ...
> > ./systemtap-strace.stp -> ./stap_trace.ko
> > Pass 1: parsed user script and 452 library scripts using
> > 35092virt/26404res/2260shr/24360data kb, in 4950usr/190sys/5187real ms.
> > Pass 2: analyzed script: 734 probes, 230 functions, 104 embeds, 63 globals
> > using 68040virt/60300res/3180shr/57308data kb, in
> > 2348300usr/36720sys/2415777real ms.
> > Pass 3: translated to C into "/tmp/stapBqLAih/stap_trace_src.c" using
> > 68040virt/60492res/3412shr/57308data kb, in 121940usr/440sys/124281real ms.
> > /tmp/stapBqLAih/stap_trace_src.c:198942:28: error: field ‘ops’ has
> > incomplete type
> > /tmp/stapBqLAih/stap_trace_src.c: In function ‘systemtap_module_init’:
> > /tmp/stapBqLAih/stap_trace_src.c:199900:9: error: implicit declaration of
> > function ‘stap_utrace_detach_ops’ [-Werror=implicit-function-declaration]
> > cc1: all warnings being treated as errors
> 
> Interesting. The task_finder functionality (part of the user-space tracing),
> needs for several kernel tracepoints to exist. If they don't exist, you
> should have either gotten a compile error or they should have
> semi-gracefully just refused to work.
> 
> Is CONFIG_TRACEPOINTS set in your custom kernel?

Hi David, 

Yes it is

# grep CONFIG_TRACEPOINTS ~/linux-3.2.93/.config
CONFIG_TRACEPOINTS=y

I've just attached the stap temporary directory with the generated source
files.

Cheers,
Gus

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[mysecondaccountabc at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

--- Comment #4 from Gustavo Moreira <mysecondaccountabc at gmail dot com> ---
Created attachment 10681
  --> https://sourceware.org/bugzilla/attachment.cgi?id=10681&action=edit
stap verbose output

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

--- Comment #5 from David Smith <dsmith at redhat dot com> ---
I took a look at your attachments. Your
stapconf_1ddada75a347accc585e3d1a06438f1e_713.h file doesn't contain
STAPCONF_UTRACE_VIA_TRACEPOINTS. This means that your kernel doesn't support
the 5 tracepoints systemtap needs to do userspace tracing. They are:

sched_process_fork, sched_process_exit, sched_process_exec, sys_enter, and
sys_exit

You can check for these tracepoints via the following:

===
# stap -l 'kernel.trace("sched_process_*")'
kernel.trace("sched:sched_process_exec")
kernel.trace("sched:sched_process_exit")
kernel.trace("sched:sched_process_fork")
kernel.trace("sched:sched_process_free")
kernel.trace("sched:sched_process_wait")
# stap -l 'kernel.trace("sys_*")'
kernel.trace("raw_syscalls:sys_enter")
kernel.trace("raw_syscalls:sys_exit")
===

If the sys_enter/sys_exit tracepoints are missing, that would mean your kernel
doesn't have CONFIG_HAVE_SYSCALL_TRACEPOINTS.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[mysecondaccountabc at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

--- Comment #6 from Gustavo Moreira <mysecondaccountabc at gmail dot com> ---
(In reply to David Smith from comment #5)
> I took a look at your attachments. Your
> stapconf_1ddada75a347accc585e3d1a06438f1e_713.h file doesn't contain
> STAPCONF_UTRACE_VIA_TRACEPOINTS. This means that your kernel doesn't support
> the 5 tracepoints systemtap needs to do userspace tracing. They are:
> 
> sched_process_fork, sched_process_exit, sched_process_exec, sys_enter, and
> sys_exit
> 
> You can check for these tracepoints via the following:
> 
> ===
> # stap -l 'kernel.trace("sched_process_*")'
> kernel.trace("sched:sched_process_exec")
> kernel.trace("sched:sched_process_exit")
> kernel.trace("sched:sched_process_fork")
> kernel.trace("sched:sched_process_free")
> kernel.trace("sched:sched_process_wait")
> # stap -l 'kernel.trace("sys_*")'
> kernel.trace("raw_syscalls:sys_enter")
> kernel.trace("raw_syscalls:sys_exit")
> ===
> 
> If the sys_enter/sys_exit tracepoints are missing, that would mean your
> kernel doesn't have CONFIG_HAVE_SYSCALL_TRACEPOINTS.


Yeah, it seems the syscall tracepoints are disabled.

# stap -l 'kernel.trace("sched_process_*")'
kernel.trace("sched:sched_process_exit")
kernel.trace("sched:sched_process_fork")
kernel.trace("sched:sched_process_free")
kernel.trace("sched:sched_process_wait")

# stap -l 'kernel.trace("sys_*")'


And unfortunately it seems this kernel version doesn't support syscall
tracepoints for ARM.

~/linux-3.2.93# grep -r HAVE_SYSCALL_TRACEPOINTS arch/*
arch/powerpc/Kconfig:   select HAVE_SYSCALL_TRACEPOINTS
arch/s390/Kconfig:      select HAVE_SYSCALL_TRACEPOINTS
arch/sh/Kconfig:        select HAVE_SYSCALL_TRACEPOINTS
arch/sparc/Kconfig:     select HAVE_SYSCALL_TRACEPOINTS
arch/x86/Kconfig:       select HAVE_SYSCALL_TRACEPOINTS


Wonderful :S

So, let's start again. I will install everything from scratch using a fresh
Debian Jessie with kernel 3.16.0-4 and GCC 4.9.2. Should those be enough?

As per the kernel config features, what exactly does SystemTap need? That's a
bit confusing because different pages say different things. For instance, in
the official wiki for using systemtap with self-built kernels
[https://sourceware.org/systemtap/wiki/SystemTapWithSelfBuiltKernel], it
doesn't mention neither CONFIG_TRACEPOINTS nor CONFIG_HAVE_SYSCALL_TRACEPOINTS.
I know that would depend on what SystemTap features I will use but, let's say,
to have full support. Where can I find the most updated documentation about it?

I've gathered this list from different sources, should this be enough? Do you
recommend to add or remove something?

CONFIG_KPROBES=y
CONFIG_KPROBES_SANITY_TEST=n
CONFIG_KPROBE_EVENT=y
CONFIG_NET_DCCPPROBE=m
CONFIG_NET_SCTPPROBE=m
CONFIG_NET_TCPPROBE=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_REDUCED=n
CONFIG_ARM_KPROBES_TEST=n
CONFIG_TRACEPOINTS=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y


Cheers,
Gus

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

--- Comment #7 from David Smith <dsmith at redhat dot com> ---
(In reply to Gustavo Moreira from comment #6)

> So, let's start again. I will install everything from scratch using a fresh
> Debian Jessie with kernel 3.16.0-4 and GCC 4.9.2. Should those be enough?

Unfortunately, I have no idea. You'll just have to try them.

> As per the kernel config features, what exactly does SystemTap need? That's
> a bit confusing because different pages say different things. For instance,
> in the official wiki for using systemtap with self-built kernels
> [https://sourceware.org/systemtap/wiki/SystemTapWithSelfBuiltKernel], it
> doesn't mention neither CONFIG_TRACEPOINTS nor
> CONFIG_HAVE_SYSCALL_TRACEPOINTS. I know that would depend on what SystemTap
> features I will use but, let's say, to have full support. Where can I find
> the most updated documentation about it?
> 
> I've gathered this list from different sources, should this be enough? Do
> you recommend to add or remove something?
> 
> CONFIG_KPROBES=y
> CONFIG_KPROBES_SANITY_TEST=n
> CONFIG_KPROBE_EVENT=y
> CONFIG_NET_DCCPPROBE=m
> CONFIG_NET_SCTPPROBE=m
> CONFIG_NET_TCPPROBE=y
> CONFIG_DEBUG_INFO=y
> CONFIG_DEBUG_INFO_REDUCED=n
> CONFIG_ARM_KPROBES_TEST=n
> CONFIG_TRACEPOINTS=y
> CONFIG_HAVE_SYSCALL_TRACEPOINTS=y

Those wiki pages were created by someone like you who was motivated to figure
out exactly what systemtap needed. I normally don't use self-built kernels, but
standard Fedora/RHEL distro kernels. I'm sure at the time that wiki page was
written that was a definitive list. But, things have changed (both on the
kernel side and the systemtap side).

As far as kernel config lines goes, it does depend on what kind of probing you
want to do. That list above looks fairly reasonable for kernel probing
(CONFIG_KPROBES being the main one). The wiki page lists several you are
missing, like: CONFIG_RELAY and CONFIG_DEBUG_FS.

Past that, it depends on what kinds of probes you want to use. CONFIG_UPROBES
is needed for userspace probing. CONFIG_NETFILTER is needed for netfilter.*
probes. CONFIG_TRACEPOINTS is needed for kernel.trace probes.
CONFIG_HAVE_HW_BREAKPOINT is needed for kernel.data probes.
CONFIG_HAVE_PERF_EVENTS and CONFIG_PERF_EVENTS are needed for perf.* probes.
CONFIG_PROC_FS is needed for procfs probes.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22582] SystemTap 3.2 stap script compilation error opt

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22582

William Cohen <wcohen at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WONTFIX
                 CC|                            |wcohen at redhat dot com
             Status|WAITING                     |RESOLVED

--- Comment #8 from William Cohen <wcohen at redhat dot com> ---
It appears to be an issue with the locally built kernel configuration rather
systemtap.  On the recent version of 32-bit raspberry pi has the
sched_process_* and sys_* tracepoints:

$ uname -a
Linux raspberrypi 6.1.21+ #1642 Mon Apr  3 17:19:14 BST 2023 armv6l GNU/Linux
$ sudo /home/wcohen/bin/perf list |grep sched_process*
  sched:sched_process_exec                           [Tracepoint event]
  sched:sched_process_exit                           [Tracepoint event]
  sched:sched_process_fork                           [Tracepoint event]
  sched:sched_process_free                           [Tracepoint event]
  sched:sched_process_hang                           [Tracepoint event]
  sched:sched_process_wait                           [Tracepoint event]
$ sudo /home/wcohen/bin/perf list |grep sys_
  raw_syscalls:sys_enter                             [Tracepoint event]
  raw_syscalls:sys_exit                              [Tracepoint event]

There hasn't been any response on this bug for over six year.  Going to close
this.

-- 
You are receiving this mail because:
You are the assignee for the bug.