From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id AD67E3858C98; Mon, 11 Mar 2024 13:54:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org AD67E3858C98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1710165288; bh=oyxqfvbZB3hS00s+LfooE1N+WTV77nV8GDsKwodD3sM=; h=From:To:Subject:Date:From; b=P5zBnexuuMy+CKOtyvKYLIrwbtuSzsJNJG3zrT3WS3Me+EMLR8aJE387aT48sg2TS P1vcpDtjg43cPN2OzI5vGG9PZzDzrJXjsCz3K0YKDznVqiAOLYTvrqQAbP0hQnOomK P8qZNAD85dcU9OUoYKcDL1e/EMZaGmg9HuZXuqhw= From: "wcohen at redhat dot com" To: systemtap@sourceware.org Subject: [Bug runtime/31472] New: systemtap.base/tracepoints.stp causes reboot with RHEL9 linux 5.14.0-428.el9.x86_64 Date: Mon, 11 Mar 2024 13:54:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: systemtap X-Bugzilla-Component: runtime X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: wcohen at redhat dot com X-Bugzilla-Status: NEW X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: systemtap at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 List-Id: https://sourceware.org/bugzilla/show_bug.cgi?id=3D31472 Bug ID: 31472 Summary: systemtap.base/tracepoints.stp causes reboot with RHEL9 linux 5.14.0-428.el9.x86_64 Product: systemtap Version: unspecified Status: NEW Severity: normal Priority: P2 Component: runtime Assignee: systemtap at sourceware dot org Reporter: wcohen at redhat dot com Target Milestone: --- When running "make installcheck" on x86_64 RHEL9 with a very recently built kernel, linux 5.14.0-428.el9.x86_64, the machine will reboot. This can be demonstrated with the following steps: $ cd systemtap_write/systemtap/ $ ../install/bin/stap -p4 -k -mtracepoints -v testsuite/systemtap.base/tracepoints.stp Pass 1: parsed user script and 503 library scripts using 204124virt/175760res/15488shr/160016data kb, in 290usr/60sys/461real ms. Pass 2: analyzed script: 2174 probes, 1 function, 0 embeds, 1 global using 233300virt/208684res/18960shr/189192data kb, in 81560usr/10520sys/94518real= ms. Pass 3: translated to C into "/tmp/stapoffzQT/tracepoints_src.c" using 233300virt/208684res/18960shr/189192data kb, in 190usr/20sys/226real ms. tracepoints.ko Pass 4: compiled C into "tracepoints.ko" in 177710usr/9790sys/190275real ms. Keeping temporary directory "/tmp/stapoffzQT" $ cp -a /tmp/stapoffzQT . $ ../install/bin/staprun tracepoints.ko [ 6315.911726] jump_label: Fatal kernel bug, unexpected op at __access_process_vm_.constprop.0+0x3d/0x1f0 [tracepoints] [0000000071a5b3f1] (e9 eb 00 00 00 !=3D 0f 1f 44 00 00)) size:5 type:1 [ 6315.911834] ------------[ cut here ]------------ [ 6315.911836] kernel BUG at arch/x86/kernel/jump_label.c:73! [ 6315.911844] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI [ 6315.912830] CPU: 0 PID: 13174 Comm: stapio Kdump: loaded Tainted: G = W OE ------- --- 5.14.0-428.el9.x86_64 #1 [ 6315.912830] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-1.fc39 04/01/2014 [ 6315.912830] RIP: 0010:__jump_label_patch.cold+0x24/0x26 [ 6315.912830] Code: ba e9 6a ff 40 ff 49 c7 c7 c8 95 83 bb 41 56 45 89 e1 = 4d 89 f8 4c 89 e9 4c 89 ea 4c 89 ee 48 c7 c7 58 a8 30 ba e8 ad d2 00 00 <0f> 0= b 53 48 c7 c6 58 b2 30 ba 48 89 fb 48 8d bf c0 00 00 00 e8 9a [ 6315.912830] RSP: 0018:ffffa7ba03657b50 EFLAGS: 00010246 [ 6315.912830] RAX: 00000000000000ad RBX: 00000000000000eb RCX: 0000000000000000 [ 6315.912830] RDX: 0000000000000000 RSI: ffff897b3bc20840 RDI: ffff897b3bc20840 [ 6315.912830] RBP: ffffa7ba03657b88 R08: 80000000ffff8388 R09: 0000000000ffff0a [ 6315.912830] R10: 0000000000000000 R11: 000000000000000f R12: 0000000000000005 [ 6315.912830] R13: ffffffffc0f7339d R14: 0000000000000001 R15: ffffffffb9e0a26a [ 6315.912830] FS: 00007f02e5506000(0000) GS:ffff897b3bc00000(0000) knlGS:0000000000000000 [ 6315.912830] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6315.912830] CR2: 00007f0ab50f6030 CR3: 00000000011b2005 CR4: 0000000000370ef0 [ 6315.912830] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6315.912830] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 6315.912830] Call Trace: [ 6315.912830] [ 6315.912830] ? show_trace_log_lvl+0x1c4/0x2df [ 6315.912830] ? show_trace_log_lvl+0x1c4/0x2df [ 6315.912830] ? arch_jump_label_transform_queue+0x34/0x80 [ 6315.912830] ? __die_body.cold+0x8/0xd [ 6315.912830] ? die+0x2b/0x50 [ 6315.912830] ? do_trap+0xce/0x120 [ 6315.912830] ? __jump_label_patch.cold+0x24/0x26 [ 6315.912830] ? do_error_trap+0x65/0x80 [ 6315.912830] ? __jump_label_patch.cold+0x24/0x26 [ 6315.912830] ? exc_invalid_op+0x4e/0x70 [ 6315.912830] ? __jump_label_patch.cold+0x24/0x26 [ 6315.912830] ? asm_exc_invalid_op+0x16/0x20 [ 6315.912830] ? __access_process_vm_.constprop.0+0x3d/0x1f0 [tracepoints] [ 6315.912830] ? __jump_label_patch.cold+0x24/0x26 [ 6315.912830] arch_jump_label_transform_queue+0x34/0x80 [ 6315.912830] __jump_label_update+0x6a/0x120 [ 6315.912830] jump_label_update+0x98/0xf0 [ 6315.912830] static_key_enable_cpuslocked+0x5f/0x90 [ 6315.912830] static_key_enable+0x16/0x20 [ 6315.912830] tracepoint_add_func+0x2e5/0x3f0 [ 6315.912830] ? unregister_tracepoint_probe_2100+0x30/0x30 [tracepoints] [ 6315.912830] tracepoint_probe_register+0x58/0x90 [ 6315.912830] ? unregister_tracepoint_probe_2100+0x30/0x30 [tracepoints] [ 6315.912830] ? unregister_tracepoint_probe_2100+0x30/0x30 [tracepoints] [ 6315.912830] stp_tracepoint_probe_register+0x127/0x1a0 [tracepoints] [ 6315.912830] ? systemtap_module_init.cold+0xea/0x190 [tracepoints] [ 6315.912830] ? _stp_ctl_write_cmd+0x2b4/0x440 [tracepoints] [ 6315.912830] ? proc_reg_write+0x53/0xa0 [ 6315.912830] ? vfs_write+0xe4/0x410 [ 6315.912830] ? __fget_light+0x9f/0x130 [ 6315.912830] ? ksys_write+0x5f/0xe0 [ 6315.912830] ? do_syscall_64+0x59/0x90 [ 6315.912830] ? sigprocmask+0xb4/0xe0 [ 6315.912830] ? sigprocmask+0xb4/0xe0 [ 6315.912830] ? syscall_exit_work+0x103/0x130 [ 6315.912830] ? syscall_exit_to_user_mode+0x22/0x40 [ 6315.912830] ? do_syscall_64+0x69/0x90 [ 6315.912830] ? exc_page_fault+0x62/0x150 [ 6315.912830] ? entry_SYSCALL_64_after_hwframe+0x72/0xdc [ 6315.912830] [ 6315.912830] Modules linked in: tracepoints(OE) uinput snd_seq_dummy snd_hrtimer nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 rfkill ip_set nf_tables nfnetlink qrtr sunrpc intel_rapl_msr intel_rapl_common intel_pmc_core intel_vsec pmt_telemetry pmt_class kvm_intel snd_hda_codec_generic ledtrig_audio snd_hda_intel kvm snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm irqbypass rapl snd_ti= mer iTCO_wdt snd iTCO_vendor_support i2c_i801 pcspkr lpc_ich virtio_balloon i2c_smbus soundcore joydev xfs libcrc32c qxl drm_ttm_helper ttm drm_kms_hel= per syscopyarea sysfillrect sysimgblt fb_sys_fops ahci libahci drm libata virtio_net crct10dif_pclmul crc32_pclmul crc32c_intel virtio_scsi virtio_blk ghash_clmulni_intel virtio_console net_failover failover serio_raw dm_mirror dm_region_hash dm_log dm_mod fuse This does not happen with the RHEL9 linux 5.14.0-427.el9.x86_64 as seen bel= ow: $ cd systemtap_write/systemtap/ $ uname -r 5.14.0-427.el9.x86_64 $ ../install/bin/stap -p4 -k -mtracepointsx -v testsuite/systemtap.base/tracepoints.stp Pass 1: parsed user script and 503 library scripts using 204116virt/175744res/15488shr/160008data kb, in 250usr/30sys/290real ms. Pass 2: analyzed script: 22 probes, 1 function, 0 embeds, 1 global using 209860virt/185180res/18828shr/165752data kb, in 40usr/70sys/115real ms. Pass 3: translated to C into "/tmp/stapAKS5lu/tracepointsx_src.c" using 209860virt/185180res/18828shr/165752data kb, in 10usr/0sys/10real ms. tracepointsx.ko Pass 4: compiled C into "tracepointsx.ko" in 16100usr/2020sys/18336real ms. Keeping temporary directory "/tmp/stapAKS5lu" $ cp -a /tmp/stapAKS5lu $ ../install/bin/staprun tracepointsx.ko tracepoints OK $ Looking at what is creating the error message: [ 6315.911726] jump_label: Fatal kernel bug, unexpected op at __access_process_vm_.constprop.0+0x3d/0x1f0 [tracepoints] [0000000071a5b3f1] (e9 eb 00 00 00 !=3D 0f 1f 44 00 00)) size:5 type:1 It appears to be coming from: https://elixir.bootlin.com/linux/v5.14/source/arch/x86/kernel/jump_label.c#= L71 Next step is to look to see which specific tracepoint(s) is/are triggering this. I have the temp stap build files save so can look over those. --=20 You are receiving this mail because: You are the assignee for the bug.=