public inbox for systemtap@sourceware.org
 help / color / mirror / Atom feed
* [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format
@ 2021-02-25  2:13 lyeeer at outlook dot com
  2021-03-02 20:14 ` [Bug runtime/27465] " fche at redhat dot com
                   ` (8 more replies)
  0 siblings, 9 replies; 10+ messages in thread
From: lyeeer at outlook dot com @ 2021-02-25  2:13 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

            Bug ID: 27465
           Summary: ERROR: Couldn't insert module&Invalid module format
           Product: systemtap
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: runtime
          Assignee: systemtap at sourceware dot org
          Reporter: lyeeer at outlook dot com
  Target Milestone: ---

Created attachment 13262
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13262&action=edit
Execute script, used to get system calls of docker during execution

Hello, When I use the systemTap4.4 version to test on an Ubuntu system with a
kernel version of 4.4.0-21, the following problems occur.

ly@ubuntu:~/Desktop/LiCShield-master/LiCShield-master$ sudo stap -g
/home/ly/Desktop/LiCShield-master/LiCShield-master/tracing_tool/systemtap_scripts/tracer.stp
-DSTP_NO_OVERLOAD -DMAXSTRINGLEN=4096 -v -c /usr/bin/docker -d -o
/home/ly/Desktop/LiCShield-master/LiCShield-master/training_session/python_build 

Pass 1: parsed user script and 480 library scripts using
124352virt/93704res/5424shr/88676data kb, in 340usr/100sys/434real ms. WARNING:
confusing usage, consider @entry($file) in .return probe: identifier '$file' at
/home/ly/Desktop/LiCShield-master/LiCShield-master/tracing_tool/systemtap_scripts/tracer.stp:380:43
source: path_arg[tid()] = get_path_wrapper(NULL, $file, %{ FILE %}) ^ WARNING:
confusing usage, consider @entry($new_dir->mnt) in .return probe: identifier
'$new_dir' at :391:30 source: old_path = get_path_wrapper($new_dir->mnt,
$old_dentry, %{ DENTRY %}) ^ WARNING: confusing usage, consider
@entry($old_dentry) in .return probe: identifier '$old_dentry' at :391:45
source: old_path = get_path_wrapper($new_dir->mnt, $old_dentry, %{ DENTRY %}) ^
WARNING: confusing usage, consider @entry($new_dir->mnt) in .return probe:
identifier '$new_dir' at :392:29 source: path = get_path_wrapper($new_dir->mnt,
$new_dentry, %{ DENTRY %}) ^ WARNING: confusing usage, consider
@entry($new_dentry) in .return probe: identifier '$new_dentry' at :392:44
source: path = get_path_wrapper($new_dir->mnt, $new_dentry, %{ DENTRY %}) ^
WARNING: confusing usage, consider @entry($old_dir->mnt) in .return probe:
identifier '$old_dir' at :406:30 source: old_path =
get_path_wrapper($old_dir->mnt, $old_dentry, %{ DENTRY %}) ^ WARNING: confusing
usage, consider @entry($old_dentry) in .return probe: identifier '$old_dentry'
at :406:45 source: old_path = get_path_wrapper($old_dir->mnt, $old_dentry, %{
DENTRY %}) ^ WARNING: confusing usage, consider @entry($new_dir->mnt) in
.return probe: identifier '$new_dir' at :407:29 source: path =
get_path_wrapper($new_dir->mnt, $new_dentry, %{ DENTRY %}) ^ WARNING: confusing
usage, consider @entry($new_dentry) in .return probe: identifier '$new_dentry'
at :407:44 source: path = get_path_wrapper($new_dir->mnt, $new_dentry, %{
DENTRY %}) ^ WARNING: confusing usage, consider @entry($old_dentry) in .return
probe: identifier '$old_dentry' at :411:19 source:
if(is_dentry_dir($old_dentry)) ^ WARNING: confusing usage, consider
@entry($path) in .return probe: identifier '$path' at :563:50 source:
mount_target[tid] = get_path_wrapper(NULL, $path, %{ PATH %}) ^ WARNING:
confusing usage, consider @entry($path) in .return probe: identifier '$path' at
:567:47 source: mount_source[tid] = get_path_wrapper(NULL, $path, %{ PATH %}) ^
Pass 2: analyzed script: 34 probes, 33 functions, 100 embeds, 8 globals using
176924virt/147968res/6892shr/141248data kb, in 1170usr/860sys/2376real ms.
WARNING: unused command line option $1/@1 
Pass 3: using cached
/home/ly/.systemtap/cache/80/stap_80a92b209efe4091f63377f31abb085e_93148.c 
Pass 4: using cached
/home/ly/.systemtap/cache/80/stap_80a92b209efe4091f63377f31abb085e_93148.ko 
Pass 5: starting run. ERROR: Couldn't insert module
'/tmp/stapaon1yo/stap_80a92b209efe4091f63377f31abb085e_93148.ko': Invalid
module format WARNING: /usr/local/bin/staprun exited with status: 1 
Pass 5: run completed in 0usr/0sys/2real ms. Pass 5: run failed. [man
error::pass5]

Then, I searched for a related answer(Bug 24720) that said it was a problem
with guru mode. But the C language code is embedded in my script, so I need to
turn on the guru mode. If it is not turned on, an error will occur

ly@ubuntu:~/Desktop/LiCShield-master/LiCShield-master$ sudo stap
/home/ly/Desktop/LiCShield-master/LiCShield-master/tracing_tool/systemtap_scripts/tracer.stp
-v -c /usr/bin/docker -d -o
/home/ly/Desktop/LiCShield-master/LiCShield-master/training_session/python_build
parse error: embedded code in unprivileged script; need stap -g saw:
embedded-code at
/home/ly/Desktop/LiCShield-master/LiCShield-master/tracing_tool/systemtap_scripts/tracer.stp:1:1
source: ...

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
@ 2021-03-02 20:14 ` fche at redhat dot com
  2022-02-04 18:21 ` v.mayatskih at gmail dot com
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: fche at redhat dot com @ 2021-03-02 20:14 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2021-03-02
             Status|UNCONFIRMED                 |WAITING
     Ever confirmed|0                           |1
                 CC|                            |fche at redhat dot com

--- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
dmesg might give further information about why the kernel rejected loading the
systemtap module.   Can you try it again, and report what the kernel says? 
Also,  increasing the pass-5 verbosity to a higher level may give us more tips.

# stap --vp 00004 ....

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
  2021-03-02 20:14 ` [Bug runtime/27465] " fche at redhat dot com
@ 2022-02-04 18:21 ` v.mayatskih at gmail dot com
  2022-02-04 18:23 ` fche at redhat dot com
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: v.mayatskih at gmail dot com @ 2022-02-04 18:21 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

Vitaly Mayatskikh <v.mayatskih at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |v.mayatskih at gmail dot com

--- Comment #2 from Vitaly Mayatskikh <v.mayatskih at gmail dot com> ---
Created attachment 13957
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13957&action=edit
Module init trace

Looks like it fails to resolve a symbol:

    74 insmod(34216):   ->find_symbol fsa=0xffffa1dc610c7d10
    76 insmod(34216):    ->find_exported_symbol_in_section
syms=0xffffffffb2c288c0 owner=0x0 data=0xffffa1dc610c7d10
...
   103 insmod(34216):     ->cmp_name name=0xffffffffb2fcd7ee
sym=0xffffffffb310b754
   104 insmod(34216):     <-cmp_name return=0x1
   105 insmod(34216):     ->cmp_name name=0xffffffffb2fcd7ee
sym=0xffffffffb310b760
   106 insmod(34216):     <-cmp_name return=0x0
   107 insmod(34216):    <-find_exported_symbol_in_section return=0x1
   108 insmod(34216):   <-find_symbol return=0x1
   118 insmod(34216):  <-check_version return=0x0
   122 insmod(34216): <-load_module return=0xfffffffffffffff8

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
  2021-03-02 20:14 ` [Bug runtime/27465] " fche at redhat dot com
  2022-02-04 18:21 ` v.mayatskih at gmail dot com
@ 2022-02-04 18:23 ` fche at redhat dot com
  2022-02-04 18:53 ` v.mayatskih at gmail dot com
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: fche at redhat dot com @ 2022-02-04 18:23 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

--- Comment #3 from Frank Ch. Eigler <fche at redhat dot com> ---
It'd be helpful to see the last relevant lines of the output of dmesg.  Module
loading related diagnostics are often printed there, to the kernel log.

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
                   ` (2 preceding siblings ...)
  2022-02-04 18:23 ` fche at redhat dot com
@ 2022-02-04 18:53 ` v.mayatskih at gmail dot com
  2022-02-04 19:22 ` v.mayatskih at gmail dot com
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: v.mayatskih at gmail dot com @ 2022-02-04 18:53 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

--- Comment #4 from Vitaly Mayatskikh <v.mayatskih at gmail dot com> ---
Nothing in dmesg. After enabling pr_debug for kernel/modules.c I see only that:

[ 4686.786467] finit_module: fd=3, uargs=0000000066cecbb4, flags=0

Something is broken w.r.t. how Ubuntu's kernel is configured:

$ uname -v
#31~20.04.1-Ubuntu SMP Wed Jan 19 14:08:10 UTC 2022

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
                   ` (3 preceding siblings ...)
  2022-02-04 18:53 ` v.mayatskih at gmail dot com
@ 2022-02-04 19:22 ` v.mayatskih at gmail dot com
  2022-02-04 19:30 ` v.mayatskih at gmail dot com
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: v.mayatskih at gmail dot com @ 2022-02-04 19:22 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

--- Comment #5 from Vitaly Mayatskikh <v.mayatskih at gmail dot com> ---
check_version fails for module_layout:


   204 insmod(43808):   ->check_version info={.name="stap_30878",
.mod=0xffffa1dc617d1740, .hdr=0xffffa1dc617c1000, .len=142104,
.sechdrs=0xffffa1dc617e3098, .secstrings="", .strtab="", .symoffs=0,
.stroffs=0, .init_typeoffs=0, .core_typeoffs=0, .debug=0x0, .num_debug=0,
.sig_ok=0, .mod_kallsyms_init_off=0, .index={.sym=39, .str=40, .mod=34,
.vers=0, .info=21, .pcpu=32}} symname="module_layout" crc=-1290631832 mod=ERROR
   207 insmod(43808):   <-check_version return=0x0

This is for non-guru case (does not fail):

   311 insmod(43893):   ->check_version info={.name="stap_11738",
.mod=0xffffa1dc617d1c00, .hdr=0xffffa1dc617c1000, .len=139264,
.sechdrs=0xffffa1dc617e2540, .secstrings="", .strtab="", .symoffs=0,
.stroffs=0, .init_typeoffs=0, .core_typeoffs=0, .debug=0x0, .num_debug=0,
.sig_ok=0, .mod_kallsyms_init_off=0, .index={.sym=40, .str=41, .mod=35,
.vers=24, .info=21, .pcpu=33}} symname="module_layout" crc=-1290631832
mod=ERROR
   319 insmod(43893):   <-check_version return=0x1

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
                   ` (4 preceding siblings ...)
  2022-02-04 19:22 ` v.mayatskih at gmail dot com
@ 2022-02-04 19:30 ` v.mayatskih at gmail dot com
  2022-02-04 19:40 ` v.mayatskih at gmail dot com
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: v.mayatskih at gmail dot com @ 2022-02-04 19:30 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

--- Comment #6 from Vitaly Mayatskikh <v.mayatskih at gmail dot com> ---
check_version fails for module_layout:


   204 insmod(43808):   ->check_version info={.name="stap_30878",
.mod=0xffffa1dc617d1740, .hdr=0xffffa1dc617c1000, .len=142104,
.sechdrs=0xffffa1dc617e3098, .secstrings="", .strtab="", .symoffs=0,
.stroffs=0, .init_typeoffs=0, .core_typeoffs=0, .debug=0x0, .num_debug=0,
.sig_ok=0, .mod_kallsyms_init_off=0, .index={.sym=39, .str=40, .mod=34,
.vers=0, .info=21, .pcpu=32}} symname="module_layout" crc=-1290631832 mod=ERROR
   207 insmod(43808):   <-check_version return=0x0

This is for non-guru case (does not fail):

   311 insmod(43893):   ->check_version info={.name="stap_11738",
.mod=0xffffa1dc617d1c00, .hdr=0xffffa1dc617c1000, .len=139264,
.sechdrs=0xffffa1dc617e2540, .secstrings="", .strtab="", .symoffs=0,
.stroffs=0, .init_typeoffs=0, .core_typeoffs=0, .debug=0x0, .num_debug=0,
.sig_ok=0, .mod_kallsyms_init_off=0, .index={.sym=40, .str=41, .mod=35,
.vers=24, .info=21, .pcpu=33}} symname="module_layout" crc=-1290631832
mod=ERROR
   319 insmod(43893):   <-check_version return=0x1

____versions symbol is missing in the "broken" module:

$ sudo readelf -a /tmp/stapzTtc4M/stap_11738.ko | grep version
  [24] __versions        PROGBITS         0000000000000000  0000cc80
    27: 0000000000000076    35 OBJECT  LOCAL  DEFAULT   21
__UNIQUE_ID_srcversion127
    29: 0000000000000000  8832 OBJECT  LOCAL  DEFAULT   24 ____versions

$ sudo readelf -a /tmp/stapykqAs1/stap_30878.ko | grep version
    26: 00000000000000a3    35 OBJECT  LOCAL  DEFAULT   21
__UNIQUE_ID_srcversion127

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
                   ` (5 preceding siblings ...)
  2022-02-04 19:30 ` v.mayatskih at gmail dot com
@ 2022-02-04 19:40 ` v.mayatskih at gmail dot com
  2022-02-09 23:57 ` fche at redhat dot com
  2024-02-21 13:07 ` fche at redhat dot com
  8 siblings, 0 replies; 10+ messages in thread
From: v.mayatskih at gmail dot com @ 2022-02-04 19:40 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

--- Comment #7 from Vitaly Mayatskikh <v.mayatskih at gmail dot com> ---
check_version fails for module_layout:


   204 insmod(43808):   ->check_version info={.name="stap_30878",
.mod=0xffffa1dc617d1740, .hdr=0xffffa1dc617c1000, .len=142104,
.sechdrs=0xffffa1dc617e3098, .secstrings="", .strtab="", .symoffs=0,
.stroffs=0, .init_typeoffs=0, .core_typeoffs=0, .debug=0x0, .num_debug=0,
.sig_ok=0, .mod_kallsyms_init_off=0, .index={.sym=39, .str=40, .mod=34,
.vers=0, .info=21, .pcpu=32}} symname="module_layout" crc=-1290631832 mod=ERROR
   207 insmod(43808):   <-check_version return=0x0

This is for non-guru case (does not fail):

   311 insmod(43893):   ->check_version info={.name="stap_11738",
.mod=0xffffa1dc617d1c00, .hdr=0xffffa1dc617c1000, .len=139264,
.sechdrs=0xffffa1dc617e2540, .secstrings="", .strtab="", .symoffs=0,
.stroffs=0, .init_typeoffs=0, .core_typeoffs=0, .debug=0x0, .num_debug=0,
.sig_ok=0, .mod_kallsyms_init_off=0, .index={.sym=40, .str=41, .mod=35,
.vers=24, .info=21, .pcpu=33}} symname="module_layout" crc=-1290631832
mod=ERROR
   319 insmod(43893):   <-check_version return=0x1

____versions symbol is missing in the "broken" module:

$ sudo readelf -a /tmp/stapzTtc4M/stap_11738.ko | grep version
  [24] __versions        PROGBITS         0000000000000000  0000cc80
    27: 0000000000000076    35 OBJECT  LOCAL  DEFAULT   21
__UNIQUE_ID_srcversion127
    29: 0000000000000000  8832 OBJECT  LOCAL  DEFAULT   24 ____versions

$ sudo readelf -a /tmp/stapykqAs1/stap_30878.ko | grep version
    26: 00000000000000a3    35 OBJECT  LOCAL  DEFAULT   21
__UNIQUE_ID_srcversion127

Adding -B CONFIG_MODVERSIONS=y solved it for me. And this is where it gets
messed up:

buildrun.cxx:
  // PR10280: suppress symbol versioning to restrict to exact kernel version
  //
  // XXX (PR24720): this was reported to cause problems on a PPC machine,
  // and I've spotted kernel commits (43e24e82f35) which vary the vermagic
  // string for PPC. Still, a standard PPC configuration tested fine, so
  // I'm cautiously re-enabling this.
  if (s.guru_mode)
    make_cmd.push_back("CONFIG_MODVERSIONS=");
  // Note: can re-enable from command line with "-B CONFIG_MODVERSIONS=y".

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
                   ` (6 preceding siblings ...)
  2022-02-04 19:40 ` v.mayatskih at gmail dot com
@ 2022-02-09 23:57 ` fche at redhat dot com
  2024-02-21 13:07 ` fche at redhat dot com
  8 siblings, 0 replies; 10+ messages in thread
From: fche at redhat dot com @ 2022-02-09 23:57 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

--- Comment #8 from Frank Ch. Eigler <fche at redhat dot com> ---
modversion / module_layout checks tend to fail mainly if the system compiler
used to build the kernel is different enough from the one used to build
systemtap's kernel modules.  There are some gcc version-level features that
vary the struct-module and other data types in linux, so modversions machinery
can catch that.

I suggest checking compiler versions too.

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Bug runtime/27465] ERROR: Couldn't insert module&Invalid module format
  2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
                   ` (7 preceding siblings ...)
  2022-02-09 23:57 ` fche at redhat dot com
@ 2024-02-21 13:07 ` fche at redhat dot com
  8 siblings, 0 replies; 10+ messages in thread
From: fche at redhat dot com @ 2024-02-21 13:07 UTC (permalink / raw)
  To: systemtap

https://sourceware.org/bugzilla/show_bug.cgi?id=27465

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |OBSOLETE
             Status|WAITING                     |RESOLVED

--- Comment #9 from Frank Ch. Eigler <fche at redhat dot com> ---
no recent investigation progress

-- 
You are receiving this mail because:
You are the assignee for the bug.

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2024-02-21 13:07 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-25  2:13 [Bug runtime/27465] New: ERROR: Couldn't insert module&Invalid module format lyeeer at outlook dot com
2021-03-02 20:14 ` [Bug runtime/27465] " fche at redhat dot com
2022-02-04 18:21 ` v.mayatskih at gmail dot com
2022-02-04 18:23 ` fche at redhat dot com
2022-02-04 18:53 ` v.mayatskih at gmail dot com
2022-02-04 19:22 ` v.mayatskih at gmail dot com
2022-02-04 19:30 ` v.mayatskih at gmail dot com
2022-02-04 19:40 ` v.mayatskih at gmail dot com
2022-02-09 23:57 ` fche at redhat dot com
2024-02-21 13:07 ` fche at redhat dot com

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).