registration error (rc -84)

registration error (rc -84)

[Daniel Doron]

Hi,

I get this error:
WARNING: probe kernel.function("sock_sendmsg@net/socket.c:623")
(address 0xffffffff81555e40) registration error (rc -84)

from the below info, it looks like the probe point address is wrong
but I can't figure out why...


==== INFO ====
Machine:
Linux Centos7ESX 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05
UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
CentOS Linux release 7.3.1611 (Core)


Systemtap translator/driver (version 3.1/0.166/0.152, non-git sources)
Copyright (C) 2005-2017 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.18 ... 4.10-rc8
enabled features: PYTHON3 LIBXML2 NLS
Created temporary directory "/tmp/staprlOfZj"
Session arch: x86_64 release: 3.10.0-514.26.2.el7.x86_64
Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/.config",
containing 3800 tuples
Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/Module.symvers",
containing 7183 vmlinux exports
Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/System.map",
containing 34509 symbols


$> cat /boot/System.map-3.10.0-514.26.2.el7.x86_64 | grep sock_sendmsg
ffffffff81555f40 T sock_sendmsg

$> cat /lib/modules/3.10.0-514.26.2.el7.x86_64/build/System.map | grep
sock_sendmsg
ffffffff81555f40 T sock_sendmsg


$> cat /proc/kallsyms | grep sock_sendmsg
ffffffff81555f40 T sock_sendmsg
====================

Re: registration error (rc -84)

[David Smith]

Interesting. This certainly should work. Do your kernel, kernel-devel,
and kernel-debuginfo RPMs versions match *exactly*? Show us the output
of the following:

# rpm -qa | fgrep kernel | sort

On Mon, Oct 16, 2017 at 8:47 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> Hi,
>
> I get this error:
> WARNING: probe kernel.function("sock_sendmsg@net/socket.c:623")
> (address 0xffffffff81555e40) registration error (rc -84)
>
> from the below info, it looks like the probe point address is wrong
> but I can't figure out why...
>
>
> ==== INFO ====
> Machine:
> Linux Centos7ESX 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05
> UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
> CentOS Linux release 7.3.1611 (Core)
>
>
> Systemtap translator/driver (version 3.1/0.166/0.152, non-git sources)
> Copyright (C) 2005-2017 Red Hat, Inc. and others
> This is free software; see the source for copying conditions.
> tested kernel versions: 2.6.18 ... 4.10-rc8
> enabled features: PYTHON3 LIBXML2 NLS
> Created temporary directory "/tmp/staprlOfZj"
> Session arch: x86_64 release: 3.10.0-514.26.2.el7.x86_64
> Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/.config",
> containing 3800 tuples
> Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/Module.symvers",
> containing 7183 vmlinux exports
> Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/System.map",
> containing 34509 symbols
>
>
> $> cat /boot/System.map-3.10.0-514.26.2.el7.x86_64 | grep sock_sendmsg
> ffffffff81555f40 T sock_sendmsg
>
> $> cat /lib/modules/3.10.0-514.26.2.el7.x86_64/build/System.map | grep
> sock_sendmsg
> ffffffff81555f40 T sock_sendmsg
>
>
> $> cat /proc/kallsyms | grep sock_sendmsg
> ffffffff81555f40 T sock_sendmsg
> ====================



-- 
David Smith
Principal Software Engineer
Red Hat

Re: registration error (rc -84)

[Frank Ch. Eigler]

David Smith <dsmith@redhat.com> writes:

> Interesting. This certainly should work. Do your kernel, kernel-devel,
> and kernel-debuginfo RPMs versions match *exactly*? Show us the output
> of the following:
>
> # rpm -qa | fgrep kernel | sort

See also   # stap-report

But still, if there were a mismatch, so stap had to pull in symbols from
kallsyms rather than debuginfo e.g., stap's buildid-enforcement should
preclude an actual misaddress type of problem.

- FChE

Re: registration error (rc -84)

[Daniel Doron]

Hi David,

Here is the output of rpm -qa | fgrep kernel | sort

abrt-addon-kerneloops-2.1.11-45.el7.centos.x86_64
kernel-3.10.0-327.36.3.el7.x86_64
kernel-3.10.0-327.el7.x86_64
kernel-3.10.0-514.26.2.el7.x86_64
kernel-debuginfo-3.10.0-514.26.2.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-514.26.2.el7.x86_64
kernel-devel-3.10.0-514.26.2.el7.x86_64
kernel-headers-3.10.0-693.2.2.el7.x86_64
kernel-tools-3.10.0-514.26.2.el7.x86_64
kernel-tools-libs-3.10.0-514.26.2.el7.x86_64

On Tue, Oct 17, 2017 at 7:52 PM, David Smith <dsmith@redhat.com> wrote:
> Interesting. This certainly should work. Do your kernel, kernel-devel,
> and kernel-debuginfo RPMs versions match *exactly*? Show us the output
> of the following:
>
> # rpm -qa | fgrep kernel | sort
>
> On Mon, Oct 16, 2017 at 8:47 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>> Hi,
>>
>> I get this error:
>> WARNING: probe kernel.function("sock_sendmsg@net/socket.c:623")
>> (address 0xffffffff81555e40) registration error (rc -84)
>>
>> from the below info, it looks like the probe point address is wrong
>> but I can't figure out why...
>>
>>
>> ==== INFO ====
>> Machine:
>> Linux Centos7ESX 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05
>> UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>> CentOS Linux release 7.3.1611 (Core)
>>
>>
>> Systemtap translator/driver (version 3.1/0.166/0.152, non-git sources)
>> Copyright (C) 2005-2017 Red Hat, Inc. and others
>> This is free software; see the source for copying conditions.
>> tested kernel versions: 2.6.18 ... 4.10-rc8
>> enabled features: PYTHON3 LIBXML2 NLS
>> Created temporary directory "/tmp/staprlOfZj"
>> Session arch: x86_64 release: 3.10.0-514.26.2.el7.x86_64
>> Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/.config",
>> containing 3800 tuples
>> Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/Module.symvers",
>> containing 7183 vmlinux exports
>> Parsed kernel "/lib/modules/3.10.0-514.26.2.el7.x86_64/build/System.map",
>> containing 34509 symbols
>>
>>
>> $> cat /boot/System.map-3.10.0-514.26.2.el7.x86_64 | grep sock_sendmsg
>> ffffffff81555f40 T sock_sendmsg
>>
>> $> cat /lib/modules/3.10.0-514.26.2.el7.x86_64/build/System.map | grep
>> sock_sendmsg
>> ffffffff81555f40 T sock_sendmsg
>>
>>
>> $> cat /proc/kallsyms | grep sock_sendmsg
>> ffffffff81555f40 T sock_sendmsg
>> ====================
>
>
>
> --
> David Smith
> Principal Software Engineer
> Red Hat

Re: registration error (rc -84)

[Daniel Doron]

Hi Frank,

I run stap-report and the only thing that seems out of place to me is
the Build ID diff between the kernel and debug-kernel.

== (eu-readelf -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 || readelf
-n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64) | grep Build ==
    Build ID: 62b54b1aac6eeaa4cd469439011f8a97558400e0
== (eu-readelf -n
/usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux ||
readelf -n /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux)
| grep Build ==
    Build ID: 9812ec6054800f5f9ee0e65ad9c780c8273200c3

On Wed, Oct 18, 2017 at 12:27 AM, Frank Ch. Eigler <fche@redhat.com> wrote:
> David Smith <dsmith@redhat.com> writes:
>
>> Interesting. This certainly should work. Do your kernel, kernel-devel,
>> and kernel-debuginfo RPMs versions match *exactly*? Show us the output
>> of the following:
>>
>> # rpm -qa | fgrep kernel | sort
>
> See also   # stap-report
>
> But still, if there were a mismatch, so stap had to pull in symbols from
> kallsyms rather than debuginfo e.g., stap's buildid-enforcement should
> preclude an actual misaddress type of problem.
>
> - FChE

Re: registration error (rc -84)

[David Smith]

On Wed, Oct 18, 2017 at 2:28 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> Hi Frank,
>
> I run stap-report and the only thing that seems out of place to me is
> the Build ID diff between the kernel and debug-kernel.
>
> == (eu-readelf -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 || readelf
> -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64) | grep Build ==
>     Build ID: 62b54b1aac6eeaa4cd469439011f8a97558400e0
> == (eu-readelf -n
> /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux ||
> readelf -n /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux)
> | grep Build ==
>     Build ID: 9812ec6054800f5f9ee0e65ad9c780c8273200c3

That's bad. I'd suggest something odd is going on. On my RHEL7 system,
the buildids match.

Try "rpm -V" on kernel and kernel-debuginfo. One or the other will have changed.

Re: registration error (rc -84)

[Daniel Doron]

Ok, I'll check it out.

On Wed, 18 Oct 2017 at 21:11 David Smith <dsmith@redhat.com> wrote:

> On Wed, Oct 18, 2017 at 2:28 AM, Daniel Doron <danielmeirdoron@gmail.com>
> wrote:
> > Hi Frank,
> >
> > I run stap-report and the only thing that seems out of place to me is
> > the Build ID diff between the kernel and debug-kernel.
> >
> > == (eu-readelf -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 || readelf
> > -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64) | grep Build ==
> >     Build ID: 62b54b1aac6eeaa4cd469439011f8a97558400e0
> > == (eu-readelf -n
> > /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux ||
> > readelf -n /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux)
> > | grep Build ==
> >     Build ID: 9812ec6054800f5f9ee0e65ad9c780c8273200c3
>
> That's bad. I'd suggest something odd is going on. On my RHEL7 system,
> the buildids match.
>
> Try "rpm -V" on kernel and kernel-debuginfo. One or the other will have
> changed.
>

Re: registration error (rc -84)

[Frank Ch. Eigler]

Daniel Doron <danielmeirdoron@gmail.com> writes:

> I run stap-report and the only thing that seems out of place to me is
> the Build ID diff between the kernel and debug-kernel.
>
> == (eu-readelf -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 || readelf
> -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64) | grep Build ==
>     Build ID: 62b54b1aac6eeaa4cd469439011f8a97558400e0
> == (eu-readelf -n
> /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux ||
> readelf -n /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux)
> | grep Build ==
>     Build ID: 9812ec6054800f5f9ee0e65ad9c780c8273200c3

I recall seeing something like this when someone used a funky community
rhel7 respin that reused the exact rhel7 n-v-r, and tried to use someone
else's debuginfo.

Nevertheless, stap does verify buildid matches at startup in many cases.
I'm very curious why that wouldn't have worked.  Would you mind
rerunning your job with this extra option:

# stap -DDEBUG_SYMBOLS=1 ...other options...


- FChE

Re: registration error (rc -84)

[Daniel Doron]

"rpm -V" on kernel and kernel-debuginfo returned no errors

On Wed, Oct 18, 2017 at 9:11 PM, David Smith <dsmith@redhat.com> wrote:
> On Wed, Oct 18, 2017 at 2:28 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>> Hi Frank,
>>
>> I run stap-report and the only thing that seems out of place to me is
>> the Build ID diff between the kernel and debug-kernel.
>>
>> == (eu-readelf -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 || readelf
>> -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64) | grep Build ==
>>     Build ID: 62b54b1aac6eeaa4cd469439011f8a97558400e0
>> == (eu-readelf -n
>> /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux ||
>> readelf -n /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux)
>> | grep Build ==
>>     Build ID: 9812ec6054800f5f9ee0e65ad9c780c8273200c3
>
> That's bad. I'd suggest something odd is going on. On my RHEL7 system,
> the buildids match.
>
> Try "rpm -V" on kernel and kernel-debuginfo. One or the other will have changed.

Re: registration error (rc -84)

[Daniel Doron]

Hi Frank,

Are these the symbols you wanted? :
_stp_do_relocation:74: found kernel _stext load address: 0xffffffff810002b8
_stp_kmodule_update_address:1065: module kernel section _stext address
0xffffffff810002b8
_stp_kmodule_update_address:1052: module kernel special section
.note.gnu.build-id address 0x2
_stp_module_check:732: build-id validation [kernel]
_stp_kmodule_relocate:38: kernel, _stext, 1fe688
_stp_kmodule_relocate:57: address=ffffffff811fe940
_stp_kmodule_relocate:38: kernel, _stext, 595af8
_stp_kmodule_relocate:57: address=ffffffff81595db0
_stp_kmodule_relocate:38: kernel, _stext, 557338
_stp_kmodule_relocate:57: address=ffffffff815575f0
_stp_kmodule_relocate:38: kernel, _stext, 556978
_stp_kmodule_relocate:57: address=ffffffff81556c30
_stp_kmodule_relocate:38: kernel, _stext, 554d18
_stp_kmodule_relocate:57: address=ffffffff81554fd0
_stp_kmodule_relocate:38: kernel, _stext, fdfb8
_stp_kmodule_relocate:57: address=ffffffff810fe270
_stp_kmodule_relocate:38: kernel, _stext, fdfb8
_stp_kmodule_relocate:57: address=ffffffff810fe270
_stp_kmodule_relocate:38: kernel, _stext, 100de8
_stp_kmodule_relocate:57: address=ffffffff811010a0
_stp_kmodule_relocate:38: kernel, _stext, 100de8
_stp_kmodule_relocate:57: address=ffffffff811010a0
_stp_kmodule_relocate:38: kernel, _stext, 100cd8
_stp_kmodule_relocate:57: address=ffffffff81100f90
_stp_kmodule_relocate:38: kernel, _stext, 100cd8
_stp_kmodule_relocate:57: address=ffffffff81100f90
_stp_kmodule_relocate:38: kernel, _stext, 2202c8
_stp_kmodule_relocate:57: address=ffffffff81220580
_stp_kmodule_relocate:38: kernel, _stext, 20e7a8
_stp_kmodule_relocate:57: address=ffffffff8120ea60
_stp_kmodule_relocate:38: kernel, _stext, 2069d8
_stp_kmodule_relocate:57: address=ffffffff81206c90
_stp_kmodule_relocate:38: kernel, _stext, 8c618
_stp_kmodule_relocate:57: address=ffffffff8108c8d0
_stp_kmodule_relocate:38: kernel, _stext, 8c6d8
_stp_kmodule_relocate:57: address=ffffffff8108c990
_stp_kmodule_relocate:38: kernel, _stext, 555b88
_stp_kmodule_relocate:57: address=ffffffff81555e40
_stp_kmodule_relocate:38: kernel, _stext, 555b88
_stp_kmodule_relocate:57: address=ffffffff81555e40
_stp_kmodule_relocate:38: kernel, _stext, 555e88
_stp_kmodule_relocate:57: address=ffffffff81556140
_stp_kmodule_relocate:38: kernel, _stext, 257368
_stp_kmodule_relocate:57: address=ffffffff81257620
_stp_kmodule_relocate:38: kernel, _stext, 221418
_stp_kmodule_relocate:57: address=ffffffff812216d0
WARNING: probe kernel.function("sock_sendmsg@net/socket.c:623")
(address 0xffffffff81555e40) registration error (rc -84)
_stp_kmodule_relocate:38: kernel, _stext, 21ff18
_stp_kmodule_relocate:57: address=ffffffff812201d0
_stp_kmodule_relocate:38: kernel, _stext, 1fe688
_stp_kmodule_relocate:57: address=ffffffff811fe940

-Daniel

On Thu, Oct 19, 2017 at 1:16 AM, Frank Ch. Eigler <fche@redhat.com> wrote:
> Daniel Doron <danielmeirdoron@gmail.com> writes:
>
>> I run stap-report and the only thing that seems out of place to me is
>> the Build ID diff between the kernel and debug-kernel.
>>
>> == (eu-readelf -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64 || readelf
>> -n /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64) | grep Build ==
>>     Build ID: 62b54b1aac6eeaa4cd469439011f8a97558400e0
>> == (eu-readelf -n
>> /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux ||
>> readelf -n /usr/lib/debug/lib/modules/3.10.0-514.26.2.el7.x86_64/vmlinux)
>> | grep Build ==
>>     Build ID: 9812ec6054800f5f9ee0e65ad9c780c8273200c3
>
> I recall seeing something like this when someone used a funky community
> rhel7 respin that reused the exact rhel7 n-v-r, and tried to use someone
> else's debuginfo.
>
> Nevertheless, stap does verify buildid matches at startup in many cases.
> I'm very curious why that wouldn't have worked.  Would you mind
> rerunning your job with this extra option:
>
> # stap -DDEBUG_SYMBOLS=1 ...other options...
>
>
> - FChE

Re: registration error (rc -84)

[David Smith]

On Sun, Oct 22, 2017 at 4:21 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> "rpm -V" on kernel and kernel-debuginfo returned no errors

Hmm, OK. Let's see if Frank's idea about a kernel and kernel-debuginfo
from 2 different sources might be true. Can you do a "rpm -qi" on
kernel-`uname -r`, kernel-devel-`uname -r`, kernel-and
debuginfo-`uname -r` and see if their "Source RPM", "Build Date",
"Build Host", "Packager" and "Vendor" fields match up on all 3 rpms?

Re: registration error (rc -84)

[Daniel Doron]

See below requested info. The Build Host in kernel-debuginfo defers
from the others...


[root@Centos7ESX ~]# rpm -qi kernel-`uname -r`
Name        : kernel
Version     : 3.10.0
Release     : 514.26.2.el7
Architecture: x86_64
Install Date: Sun 06 Aug 2017 04:27:40 PM IDT
Group       : System Environment/Kernel
Size        : 154831215
License     : GPLv2
Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:04 PM IDT, Key ID
24c6a8a7f4a80eb5
Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
Build Host  : kbuilder.dev.centos.org
Relocations : (not relocatable)
Packager    : CentOS BuildSystem <http://bugs.centos.org>
Vendor      : CentOS
URL         : http://www.kernel.org/
Summary     : The Linux kernel
Description :
The kernel package contains the Linux kernel (vmlinuz), the core of any
Linux operating system.  The kernel handles the basic functions
of the operating system: memory allocation, process allocation, device
input and output, etc.


[root@Centos7ESX ~]# rpm -qi kernel-devel-`uname -r`
Name        : kernel-devel
Version     : 3.10.0
Release     : 514.26.2.el7
Architecture: x86_64
Install Date: Mon 16 Oct 2017 02:41:23 PM IDT
Group       : System Environment/Kernel
Size        : 35974293
License     : GPLv2
Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:30 PM IDT, Key ID
24c6a8a7f4a80eb5
Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
Build Host  : kbuilder.dev.centos.org
Relocations : (not relocatable)
Packager    : CentOS BuildSystem <http://bugs.centos.org>
Vendor      : CentOS
URL         : http://www.kernel.org/
Summary     : Development package for building kernel modules to match
the kernel
Description :
This package provides kernel headers and makefiles sufficient to build modules
against the kernel package.

[root@Centos7ESX ~]# rpm -qi kernel-debuginfo-`uname -r`
Name        : kernel-debuginfo
Version     : 3.10.0
Release     : 514.26.2.el7
Architecture: x86_64
Install Date: Mon 16 Oct 2017 02:41:10 PM IDT
Group       : Development/Debug
Size        : 1510144856
License     : GPLv2
Signature   : (none)
Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
Build Host  : c1bm.rdu2.centos.org
Relocations : (not relocatable)
Packager    : CentOS BuildSystem <http://bugs.centos.org>
Vendor      : CentOS
URL         : http://www.kernel.org/
Summary     : Debug information for package kernel
Description :
This package provides debug information for package kernel.
This is required to use SystemTap with kernel-3.10.0-514.26.2.el7.x86_64.

Re: registration error (rc -84)

[Daniel Doron]

[root@Centos7ESX ~]# rpm -qi kernel-debuginfo-common-x86_64
Name        : kernel-debuginfo-common-x86_64
Version     : 3.10.0
Release     : 514.26.2.el7
Architecture: x86_64
Install Date: Mon 16 Oct 2017 02:28:50 PM IDT
Group       : Development/Debug
Size        : 256213448
License     : GPLv2
Signature   : (none)
Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
Build Host  : c1bm.rdu2.centos.org
Relocations : (not relocatable)
Packager    : CentOS BuildSystem <http://bugs.centos.org>
Vendor      : CentOS
URL         : http://www.kernel.org/
Summary     : Kernel source files used by kernel-debuginfo packages
Description :
This package is required by kernel-debuginfo subpackages.
It provides the kernel source files common to all builds.

On Mon, Oct 23, 2017 at 4:55 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> See below requested info. The Build Host in kernel-debuginfo defers
> from the others...
>
>
> [root@Centos7ESX ~]# rpm -qi kernel-`uname -r`
> Name        : kernel
> Version     : 3.10.0
> Release     : 514.26.2.el7
> Architecture: x86_64
> Install Date: Sun 06 Aug 2017 04:27:40 PM IDT
> Group       : System Environment/Kernel
> Size        : 154831215
> License     : GPLv2
> Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:04 PM IDT, Key ID
> 24c6a8a7f4a80eb5
> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
> Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
> Build Host  : kbuilder.dev.centos.org
> Relocations : (not relocatable)
> Packager    : CentOS BuildSystem <http://bugs.centos.org>
> Vendor      : CentOS
> URL         : http://www.kernel.org/
> Summary     : The Linux kernel
> Description :
> The kernel package contains the Linux kernel (vmlinuz), the core of any
> Linux operating system.  The kernel handles the basic functions
> of the operating system: memory allocation, process allocation, device
> input and output, etc.
>
>
> [root@Centos7ESX ~]# rpm -qi kernel-devel-`uname -r`
> Name        : kernel-devel
> Version     : 3.10.0
> Release     : 514.26.2.el7
> Architecture: x86_64
> Install Date: Mon 16 Oct 2017 02:41:23 PM IDT
> Group       : System Environment/Kernel
> Size        : 35974293
> License     : GPLv2
> Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:30 PM IDT, Key ID
> 24c6a8a7f4a80eb5
> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
> Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
> Build Host  : kbuilder.dev.centos.org
> Relocations : (not relocatable)
> Packager    : CentOS BuildSystem <http://bugs.centos.org>
> Vendor      : CentOS
> URL         : http://www.kernel.org/
> Summary     : Development package for building kernel modules to match
> the kernel
> Description :
> This package provides kernel headers and makefiles sufficient to build modules
> against the kernel package.
>
> [root@Centos7ESX ~]# rpm -qi kernel-debuginfo-`uname -r`
> Name        : kernel-debuginfo
> Version     : 3.10.0
> Release     : 514.26.2.el7
> Architecture: x86_64
> Install Date: Mon 16 Oct 2017 02:41:10 PM IDT
> Group       : Development/Debug
> Size        : 1510144856
> License     : GPLv2
> Signature   : (none)
> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
> Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
> Build Host  : c1bm.rdu2.centos.org
> Relocations : (not relocatable)
> Packager    : CentOS BuildSystem <http://bugs.centos.org>
> Vendor      : CentOS
> URL         : http://www.kernel.org/
> Summary     : Debug information for package kernel
> Description :
> This package provides debug information for package kernel.
> This is required to use SystemTap with kernel-3.10.0-514.26.2.el7.x86_64.

Re: registration error (rc -84)

[David Smith]

Notice the kernel and kernel-devel packages have a build date of 'Tue
04 Jul 2017 06:54:56 PM IDT', while the 2 debuginfo packages have a
build date of 'Tue 04 Jul 2017 05:08:10 PM IDT'. So, the debuginfo
packages were somehow built almost 2 hours earlier than the binary
packages. Also note that the "Build Host" fields are also different.
These 2 sets of RPMs came from different builds, even though the NVRs
are the same.

This certainly appears to be your problem. I'd guess that the CentOS
folks rebuilt that kernel SRPM but didn't bother updating the
debuginfo.

On Mon, Oct 23, 2017 at 8:57 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> [root@Centos7ESX ~]# rpm -qi kernel-debuginfo-common-x86_64
> Name        : kernel-debuginfo-common-x86_64
> Version     : 3.10.0
> Release     : 514.26.2.el7
> Architecture: x86_64
> Install Date: Mon 16 Oct 2017 02:28:50 PM IDT
> Group       : Development/Debug
> Size        : 256213448
> License     : GPLv2
> Signature   : (none)
> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
> Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
> Build Host  : c1bm.rdu2.centos.org
> Relocations : (not relocatable)
> Packager    : CentOS BuildSystem <http://bugs.centos.org>
> Vendor      : CentOS
> URL         : http://www.kernel.org/
> Summary     : Kernel source files used by kernel-debuginfo packages
> Description :
> This package is required by kernel-debuginfo subpackages.
> It provides the kernel source files common to all builds.
>
> On Mon, Oct 23, 2017 at 4:55 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>> See below requested info. The Build Host in kernel-debuginfo defers
>> from the others...
>>
>>
>> [root@Centos7ESX ~]# rpm -qi kernel-`uname -r`
>> Name        : kernel
>> Version     : 3.10.0
>> Release     : 514.26.2.el7
>> Architecture: x86_64
>> Install Date: Sun 06 Aug 2017 04:27:40 PM IDT
>> Group       : System Environment/Kernel
>> Size        : 154831215
>> License     : GPLv2
>> Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:04 PM IDT, Key ID
>> 24c6a8a7f4a80eb5
>> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
>> Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
>> Build Host  : kbuilder.dev.centos.org
>> Relocations : (not relocatable)
>> Packager    : CentOS BuildSystem <http://bugs.centos.org>
>> Vendor      : CentOS
>> URL         : http://www.kernel.org/
>> Summary     : The Linux kernel
>> Description :
>> The kernel package contains the Linux kernel (vmlinuz), the core of any
>> Linux operating system.  The kernel handles the basic functions
>> of the operating system: memory allocation, process allocation, device
>> input and output, etc.
>>
>>
>> [root@Centos7ESX ~]# rpm -qi kernel-devel-`uname -r`
>> Name        : kernel-devel
>> Version     : 3.10.0
>> Release     : 514.26.2.el7
>> Architecture: x86_64
>> Install Date: Mon 16 Oct 2017 02:41:23 PM IDT
>> Group       : System Environment/Kernel
>> Size        : 35974293
>> License     : GPLv2
>> Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:30 PM IDT, Key ID
>> 24c6a8a7f4a80eb5
>> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
>> Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
>> Build Host  : kbuilder.dev.centos.org
>> Relocations : (not relocatable)
>> Packager    : CentOS BuildSystem <http://bugs.centos.org>
>> Vendor      : CentOS
>> URL         : http://www.kernel.org/
>> Summary     : Development package for building kernel modules to match
>> the kernel
>> Description :
>> This package provides kernel headers and makefiles sufficient to build modules
>> against the kernel package.
>>
>> [root@Centos7ESX ~]# rpm -qi kernel-debuginfo-`uname -r`
>> Name        : kernel-debuginfo
>> Version     : 3.10.0
>> Release     : 514.26.2.el7
>> Architecture: x86_64
>> Install Date: Mon 16 Oct 2017 02:41:10 PM IDT
>> Group       : Development/Debug
>> Size        : 1510144856
>> License     : GPLv2
>> Signature   : (none)
>> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
>> Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
>> Build Host  : c1bm.rdu2.centos.org
>> Relocations : (not relocatable)
>> Packager    : CentOS BuildSystem <http://bugs.centos.org>
>> Vendor      : CentOS
>> URL         : http://www.kernel.org/
>> Summary     : Debug information for package kernel
>> Description :
>> This package provides debug information for package kernel.
>> This is required to use SystemTap with kernel-3.10.0-514.26.2.el7.x86_64.



-- 
David Smith
Principal Software Engineer
Red Hat

Re: registration error (rc -84)

[Daniel Doron]

ok. that solved the problem.

I remove kernel-debuginfo and kernel-debuginfo-common (previously
pulled via wget from
https://buildlogs.centos.org/c7.1611.u/kernel/20170704132018/3.10.0-514.26.2.el7.x86_64/kernel-debuginfo-common-x86_64-$(uname
-r).rpm

https://buildlogs.centos.org/c7.1611.u/kernel/20170704132018/3.10.0-514.26.2.el7.x86_64/kernel-debuginfo-$(uname
-r).rpm
)

I took them this time directly from:
http://debuginfo.centos.org/7/x86_64/kernel-debuginfo-common-x86_64-$(uname
-r).rpm

http://debuginfo.centos.org/7/x86_64/kernel-debuginfo-$(uname -r).rpm


now there are no errors....It seems sometimes with CentOS that the
required packages for systemtap are not always in one place orr the
same one...

-Daniel


On Mon, Oct 23, 2017 at 4:57 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> [root@Centos7ESX ~]# rpm -qi kernel-debuginfo-common-x86_64
> Name        : kernel-debuginfo-common-x86_64
> Version     : 3.10.0
> Release     : 514.26.2.el7
> Architecture: x86_64
> Install Date: Mon 16 Oct 2017 02:28:50 PM IDT
> Group       : Development/Debug
> Size        : 256213448
> License     : GPLv2
> Signature   : (none)
> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
> Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
> Build Host  : c1bm.rdu2.centos.org
> Relocations : (not relocatable)
> Packager    : CentOS BuildSystem <http://bugs.centos.org>
> Vendor      : CentOS
> URL         : http://www.kernel.org/
> Summary     : Kernel source files used by kernel-debuginfo packages
> Description :
> This package is required by kernel-debuginfo subpackages.
> It provides the kernel source files common to all builds.
>
> On Mon, Oct 23, 2017 at 4:55 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>> See below requested info. The Build Host in kernel-debuginfo defers
>> from the others...
>>
>>
>> [root@Centos7ESX ~]# rpm -qi kernel-`uname -r`
>> Name        : kernel
>> Version     : 3.10.0
>> Release     : 514.26.2.el7
>> Architecture: x86_64
>> Install Date: Sun 06 Aug 2017 04:27:40 PM IDT
>> Group       : System Environment/Kernel
>> Size        : 154831215
>> License     : GPLv2
>> Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:04 PM IDT, Key ID
>> 24c6a8a7f4a80eb5
>> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
>> Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
>> Build Host  : kbuilder.dev.centos.org
>> Relocations : (not relocatable)
>> Packager    : CentOS BuildSystem <http://bugs.centos.org>
>> Vendor      : CentOS
>> URL         : http://www.kernel.org/
>> Summary     : The Linux kernel
>> Description :
>> The kernel package contains the Linux kernel (vmlinuz), the core of any
>> Linux operating system.  The kernel handles the basic functions
>> of the operating system: memory allocation, process allocation, device
>> input and output, etc.
>>
>>
>> [root@Centos7ESX ~]# rpm -qi kernel-devel-`uname -r`
>> Name        : kernel-devel
>> Version     : 3.10.0
>> Release     : 514.26.2.el7
>> Architecture: x86_64
>> Install Date: Mon 16 Oct 2017 02:41:23 PM IDT
>> Group       : System Environment/Kernel
>> Size        : 35974293
>> License     : GPLv2
>> Signature   : RSA/SHA256, Tue 04 Jul 2017 08:04:30 PM IDT, Key ID
>> 24c6a8a7f4a80eb5
>> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
>> Build Date  : Tue 04 Jul 2017 06:54:56 PM IDT
>> Build Host  : kbuilder.dev.centos.org
>> Relocations : (not relocatable)
>> Packager    : CentOS BuildSystem <http://bugs.centos.org>
>> Vendor      : CentOS
>> URL         : http://www.kernel.org/
>> Summary     : Development package for building kernel modules to match
>> the kernel
>> Description :
>> This package provides kernel headers and makefiles sufficient to build modules
>> against the kernel package.
>>
>> [root@Centos7ESX ~]# rpm -qi kernel-debuginfo-`uname -r`
>> Name        : kernel-debuginfo
>> Version     : 3.10.0
>> Release     : 514.26.2.el7
>> Architecture: x86_64
>> Install Date: Mon 16 Oct 2017 02:41:10 PM IDT
>> Group       : Development/Debug
>> Size        : 1510144856
>> License     : GPLv2
>> Signature   : (none)
>> Source RPM  : kernel-3.10.0-514.26.2.el7.src.rpm
>> Build Date  : Tue 04 Jul 2017 05:08:10 PM IDT
>> Build Host  : c1bm.rdu2.centos.org
>> Relocations : (not relocatable)
>> Packager    : CentOS BuildSystem <http://bugs.centos.org>
>> Vendor      : CentOS
>> URL         : http://www.kernel.org/
>> Summary     : Debug information for package kernel
>> Description :
>> This package provides debug information for package kernel.
>> This is required to use SystemTap with kernel-3.10.0-514.26.2.el7.x86_64.

Re: registration error (rc -84)

[Frank Ch. Eigler]

Hi -

> ok. that solved the problem.
> 
> I remove [mismatching] kernel-debuginfo and kernel-debuginfo-common [...]
> now there are no errors....It seems sometimes with CentOS that the
> required packages for systemtap are not always in one place orr the
> same one...

This tells me that our buildid verification logic is incomplete; that
the three-way vmlinuz - debuginfo-vmlinux - running-kernel match needs
to be checked, not just two of the three.

- FChE