[Bug translator/24224] New: tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[Bug translator/24224] New: tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

            Bug ID: 24224
           Summary: tapsets.cxx Assert kernel relocation invariants fails
                    for some kernel.function("*@*.c:*") probes
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: translator
          Assignee: systemtap at sourceware dot org
          Reporter: wcohen at redhat dot com
  Target Milestone: ---

On Fedora rawhide (kernel-5.0.0-0.rc6.git1.1.fc30.x86_64) a couple of the
systemtap examples whythefail.stp and varwatch.stp tests runs probe
kernel.statement("kernel_function"@file.c:*").  The scripts fail to run due to
tapsets.cxx throwing the error:

inconsistent relocation address

Below is the parts of the systemtap.log with the failures:

PRETEST PWD=/usr/share/systemtap/testsuite
meta taglines '' tag 'test_support' value ''
TEST PWD=/usr/share/systemtap/testsuite/systemtap.examples/general
meta taglines 'test_check: stap -wp4 varwatch.stp
'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars'' tag 'test_check' value
'stap -wp4 varwatch.stp 'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars''
attempting command stap -wp4 varwatch.stp
'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars'
OUT semantic error: inconsistent relocation address: keyword at
varwatch.stp:5:1
        source: probe $1 {
                ^

Pass 2: analysis failed.  [man error::pass2]
Number of similar error messages suppressed: 14.
Rerun with -v to see them.
child process exited abnormally
RC 1
FAIL: systemtap.examples/general/varwatch build


PRETEST PWD=/usr/share/systemtap/testsuite
meta taglines '' tag 'test_support' value ''
TEST PWD=/usr/share/systemtap/testsuite/systemtap.examples/general
meta taglines 'test_check: stap -p4 whythefail.stp kernel do_sys_open '$return
< 0'' tag 'test_check' value 'stap -p4 whythefail.stp kernel do_sys_open
'$return < 0''
attempting command stap -p4 whythefail.stp kernel do_sys_open '$return < 0'
OUT semantic error: inconsistent relocation address: keyword at
whythefail.stp:37:1
        source: probe $1 . statement(@2 "@*:*") {
                ^

Pass 2: analysis failed.  [man error::pass2]
Number of similar error messages suppressed: 14.
Rerun with -v to see them.
child process exited abnormally
RC 1
FAIL: systemtap.examples/general/whythefail build


The linetime example is also using a 'kernel.statement("do_nanosleep@*:*")',
but that does not trigger the error and the test passes.  It looks like
something is going wrong in the assert tests for the code below (some
diagnostic prints showed that the section was "_stext" rather than "" for a run
of varwatch.stp:


      // Assert kernel relocation invariants
      if (section == "" && dwfl_addr != addr) // addr should be absolute
        throw SEMANTIC_ERROR (_("missing relocation basis"), tok);
      if (section != "" && dwfl_addr == addr) // addr should be an offset
        throw SEMANTIC_ERROR (_("inconsistent relocation address"), tok);
    }

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

--- Comment #1 from William Cohen <wcohen at redhat dot com> ---
Created attachment 11609
  --> https://sourceware.org/bugzilla/attachment.cgi?id=11609&action=edit
Diagnostic to see what values are used for the assert tests

To get a better idea of what is happening have this patch printing out
information each time those asserts are run:

For nanosleep function everything fine:

[wcohen@cervelo systemtap]$ ../install/bin/stap -wp4
testsuite/systemtap.examples/general/varwatch.stp
'kernel.statement("do_nanosleep@*:*")' '$$vars'
section=_stext dwfl_addr=ffffffff819434f0 addr=9434f0
section=_stext dwfl_addr=ffffffff81943517 addr=943517
section=_stext dwfl_addr=ffffffff8194352c addr=94352c
section=_stext dwfl_addr=ffffffff81943559 addr=943559
section=_stext dwfl_addr=ffffffff81943561 addr=943561
section=_stext dwfl_addr=ffffffff81943568 addr=943568
section=_stext dwfl_addr=ffffffff819435a7 addr=9435a7
section=_stext dwfl_addr=ffffffff819435c4 addr=9435c4
section=_stext dwfl_addr=ffffffff819435d6 addr=9435d6
section=_stext dwfl_addr=ffffffff81943665 addr=943665
section=_stext dwfl_addr=ffffffff819435e1 addr=9435e1
section=_stext dwfl_addr=ffffffff819435ed addr=9435ed
section=_stext dwfl_addr=ffffffff81943601 addr=943601
section=_stext dwfl_addr=ffffffff8194363f addr=94363f
section=_stext dwfl_addr=ffffffff8194364f addr=94364f
section=_stext dwfl_addr=ffffffff81943669 addr=943669
section=_stext dwfl_addr=ffffffff81943608 addr=943608
section=_stext dwfl_addr=ffffffff810b8360 addr=b8360
section=_stext dwfl_addr=ffffffff810b7030 addr=b7030
section=_stext dwfl_addr=ffffffff810b7cf5 addr=b7cf5
section=_stext dwfl_addr=ffffffff810b8ce6 addr=b8ce6
/home/wcohen/.systemtap/cache/71/stap_7166ef1db3ec33859804979a9640e95d_22392.ko


For do_sys_open function see a bunch of addr that match up to the start of the
function ffffffff812b0000

$ ../install/bin/stap -wp4 testsuite/systemtap.examples/general/varwatch.stp
'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars'
section=_stext dwfl_addr=ffffffff812b0000 addr=2b0000
section=_stext dwfl_addr=ffffffff812b0027 addr=2b0027
section=_stext dwfl_addr=ffffffff812b0103 addr=2b0103
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
semantic error: inconsistent relocation address: keyword at
testsuite/systemtap.examples/general/varwatch.stp:5:1
        source: probe $1 {
                ^

section=_stext dwfl_addr=ffffffff812b01f6 addr=2b01f6
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b00e6 addr=2b00e6
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b010b addr=2b010b
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0113 addr=2b0113
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b011c addr=2b011c
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0177 addr=2b0177
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0189 addr=2b0189
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0191 addr=2b0191
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0193 addr=2b0193
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b019c addr=2b019c
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b01e7 addr=2b01e7
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0120 addr=2b0120
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b0128 addr=2b0128
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff812b012b addr=2b012b
section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000
section=_stext dwfl_addr=ffffffff810b8360 addr=b8360
section=_stext dwfl_addr=ffffffff810b7030 addr=b7030
section=_stext dwfl_addr=ffffffff810b7cf5 addr=b7cf5
section=_stext dwfl_addr=ffffffff810b8ce6 addr=b8ce6
Pass 2: analysis failed.  [man error::pass2]
Number of similar error messages suppressed: 14.
Rerun with -v to see them.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

--- Comment #2 from William Cohen <wcohen at redhat dot com> ---
A simpler reproducer that works both on f28 ppc64le and f29 x86_64:

$ uname -a
Linux rh-power-vm07.fit.vutbr.cz 4.20.8-100.fc28.ppc64le #1 SMP Wed Feb 13
12:45:56 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux
$ sudo stap -p4 -e 'probe netdev.change_mac { exit() }'
semantic error: inconsistent relocation address: identifier 'netdev' at
<input>:1:7
        source: probe netdev.change_mac { exit() }
                      ^

Pass 2: analysis failed.  [man error::pass2]


$ uname -a
Linux cervelo 4.20.8-200.fc29.x86_64 #1 SMP Wed Feb 13 13:08:05 UTC 2019 x86_64
x86_64 x86_64 GNU/Linux
$ stap -p4 -e 'probe netdev.change_mac { exit() }'
semantic error: inconsistent relocation address: identifier 'netdev' at
<input>:1:7
        source: probe netdev.change_mac { exit() }
                      ^

Pass 2: analysis failed.  [man error::pass2]

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

--- Comment #3 from William Cohen <wcohen at redhat dot com> ---
Created attachment 11646
  --> https://sourceware.org/bugzilla/attachment.cgi?id=11646&action=edit
Results of reproducer.

Have a scaled down reproducer:

 ../install/bin/stap -vvvv -p4 -e 'probe kernel.function("dev_set_mac_address")
{ printf("%x\n", $sa); exit() }' >& pr24224_issue.out

These are the results.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

--- Comment #4 from William Cohen <wcohen at redhat dot com> ---
The problem is tied to accessing target variables.  According to the linux
source code in net/core/dev.c there are 2 target variables:

int dev_set_mac_address(struct net_device *dev, struct sockaddr *sa)
{
...

stap listing lists the two of them:

$ stap -L 'kernel.function("dev_set_mac_address")'
kernel.function("dev_set_mac_address@net/core/dev.c:7715") $dev:struct
net_device* $sa:struct sockaddr*

When going through the $dev and $sa from the source code $dev okay, but $sa has
a problem:

[wcohen@cervelo pr24224]$ ../install/bin/stap -p4  -e 'probe
kernel.function("dev_set_mac_address") { printf("%x\n", $dev); exit() }'
/home/wcohen/.systemtap/cache/cd/stap_cdef9e8a2d4b9d5a0e4252d87f28b7c2_1881.ko
[wcohen@cervelo pr24224]$ ../install/bin/stap -p4  -e 'probe
kernel.function("dev_set_mac_address") { printf("%x\n", $sa); exit() }'
semantic error: inconsistent relocation address: keyword at <input>:1:1
        source: probe kernel.function("dev_set_mac_address") { printf("%x\n",
$sa); exit() }
                ^

Pass 2: analysis failed.  [man error::pass2]

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx Assert kernel relocation invariants fails for some kernel.function("*@*.c:*") probes

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

--- Comment #5 from William Cohen <wcohen at redhat dot com> ---
Mark Wielaard suspects the problem is due to some code in tapsets.cxx.  From
IRC log:


<mjw> I think it is this:
<mjw>                     q.results.push_back(new dwarf_derived_probe
(funcname, filename,
<mjw>                                                                 
i->decl_line, module,
<mjw>                                                                  section,
i->entrypc,
<mjw>                                                                 
i->entrypc, q,
<mjw>                                                                 
&i->die));
<mjw> The second i->entrypc should be "relative"
<wcohen> mjw, where is that?
<mjw> tapsets.cxx:5492

Those changes from the following git commit:

commit 68bd23fd0cc5efa4fccad68e0df0d0c4cb774322
Author: Jafeer Uddin <juddin@redhat.com>
Date:   Mon Dec 10 15:11:26 2018 -0500

    PR16596: add support for DW_OP_GNU_entry_value in location expressions

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx kernel relocation assert fails when attempting to access some function parameters

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

William Cohen <wcohen at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|tapsets.cxx Assert kernel   |tapsets.cxx kernel
                   |relocation invariants fails |relocation assert fails
                   |for some                    |when attempting to access
                   |kernel.function("*@*.c:*")  |some function parameters
                   |probes                      |

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug translator/24224] tapsets.cxx kernel relocation assert fails when attempting to access some function parameters

[wcohen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=24224

William Cohen <wcohen at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #6 from William Cohen <wcohen at redhat dot com> ---
Looking through test and examples that previously had the problem for fedora 29
on aarch64, powerpc, and x86. This changes look to address the problem.

For example on x86_64 the reproducer now works:

$  ../install/bin/stap -p4  -e 'probe kernel.function("dev_set_mac_address") {
printf("%x\n", $sa); exit() }'
/home/wcohen/.systemtap/cache/fc/stap_fce458c8ae35b2d2f10f9f7cc5c53508_2255.ko

-- 
You are receiving this mail because:
You are the assignee for the bug.