* eh_frame table too big
@ 2013-02-18 23:38 Dave Cameron
2013-02-19 2:09 ` Josh Stone
0 siblings, 1 reply; 4+ messages in thread
From: Dave Cameron @ 2013-02-18 23:38 UTC (permalink / raw)
To: systemtap
I'm trying to run a systemtap script on Firefox, but I get the
following warning when I invoke systemtap like:
stap firefox.stap -d /lib64/libpthread-2.15.so \
-d /usr/lib64/firefox/libmozsqlite3.so \
-d /usr/lib64/firefox/libxul.so -d /usr/lib64/firefox/firefox \
-x `pidof firefox`
WARNING: skipping module /usr/lib64/firefox/libxul.so eh_frame table (too big: 6812836 > 6291456
-
Also after this probe compiled and loaded, I got a kpanic,
which I was unable to capture, but I will try to reproduce.
How can I bump this limit?
Regards,
Dave Cameron
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: eh_frame table too big
2013-02-18 23:38 eh_frame table too big Dave Cameron
@ 2013-02-19 2:09 ` Josh Stone
2013-02-19 2:57 ` Dave Cameron
2013-02-19 3:07 ` Dave Cameron
0 siblings, 2 replies; 4+ messages in thread
From: Josh Stone @ 2013-02-19 2:09 UTC (permalink / raw)
To: Dave Cameron; +Cc: systemtap
[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]
On 02/18/2013 03:38 PM, Dave Cameron wrote:
> I'm trying to run a systemtap script on Firefox, but I get the
> following warning when I invoke systemtap like:
>
> stap firefox.stap -d /lib64/libpthread-2.15.so \
> -d /usr/lib64/firefox/libmozsqlite3.so \
> -d /usr/lib64/firefox/libxul.so -d /usr/lib64/firefox/firefox \
> -x `pidof firefox`
>
> WARNING: skipping module /usr/lib64/firefox/libxul.so eh_frame table (too big: 6812836 > 6291456
>
> -
>
> Also after this probe compiled and loaded, I got a kpanic,
> which I was unable to capture, but I will try to reproduce.
We definitely need to address that! I think I found a place where the
unwind table is not completely skipped, possibly leading to bad memory
references. I'd appreciate if you could try the attached patch (while
the max is still set too low for you).
> How can I bump this limit?
This is MAX_UNWIND_TABLE_SIZE in translate.cxx, currently set to 6MB.
We have bumped this before for libxul.so, and can do so again.
Josh
[-- Attachment #2: 0001-Don-t-output-anything-at-all-for-skipped-unwind-tabl.patch --]
[-- Type: text/x-patch, Size: 7493 bytes --]
From dba320e01beda47a117cefafebc4558369fcbd0e Mon Sep 17 00:00:00 2001
From: Josh Stone <jistone@redhat.com>
Date: Mon, 18 Feb 2013 17:55:46 -0800
Subject: [PATCH] Don't output anything at all for skipped unwind tables
* translate.cxx (dump_unwindsym_cxt_table): New common function to
output an unwind table. It checks against MAX_UNWIND_TABLE_SIZE and
nulls out those that are skipped, so they're truly skipped everywhere.
(dump_unwindsym_cxt): Use dump_unwindsym_cxt_table for all four unwind
tables: debug_frame, debug_frame_hdr, eh_frame, and eh_frame_hdr.
---
translate.cxx | 144 ++++++++++++++++++++--------------------------------------
1 file changed, 50 insertions(+), 94 deletions(-)
diff --git a/translate.cxx b/translate.cxx
index 898a52d..102e7c3 100644
--- a/translate.cxx
+++ b/translate.cxx
@@ -5900,6 +5900,47 @@ dump_unwind_tables (Dwfl_Module *m,
return DWARF_CB_OK;
}
+static void
+dump_unwindsym_cxt_table(systemtap_session& session, ostream& output,
+ const string& modname, unsigned modindex,
+ const string& secname, unsigned secindex,
+ const string& table, void*& data, size_t& len)
+{
+ if (data == NULL || len == 0)
+ return;
+
+ if (len > MAX_UNWIND_TABLE_SIZE)
+ {
+ if (secname.empty())
+ session.print_warning (_F("skipping module %s %s table (too big: %zi > %zi)",
+ modname.c_str(), table.c_str(),
+ len, (size_t)MAX_UNWIND_TABLE_SIZE));
+ else
+ session.print_warning (_F("skipping module %s, section %s %s table (too big: %zi > %zi)",
+ modname.c_str(), secname.c_str(), table.c_str(),
+ len, (size_t)MAX_UNWIND_TABLE_SIZE));
+ data = NULL;
+ len = 0;
+ return;
+ }
+
+ output << "#if defined(STP_USE_DWARF_UNWINDER) && defined(STP_NEED_UNWIND_DATA)\n";
+ output << "static uint8_t _stp_module_" << modindex << "_" << table;
+ if (!secname.empty())
+ output << "_" << secindex;
+ output << "[] = \n";
+ output << " {";
+ for (size_t i = 0; i < len; i++)
+ {
+ int h = ((uint8_t *)data)[i];
+ output << h << ","; // decimal is less wordy than hex
+ if ((i + 1) % 16 == 0)
+ output << "\n" << " ";
+ }
+ output << "};\n";
+ output << "#endif /* STP_USE_DWARF_UNWINDER && STP_NEED_UNWIND_DATA */\n";
+}
+
static int
dump_unwindsym_cxt (Dwfl_Module *m,
unwindsym_dump_context *c,
@@ -5919,76 +5960,15 @@ dump_unwindsym_cxt (Dwfl_Module *m,
Dwarf_Addr eh_addr = c->eh_addr;
Dwarf_Addr eh_frame_hdr_addr = c->eh_frame_hdr_addr;
- if (debug_frame != NULL && debug_len > 0)
- {
- c->output << "#if defined(STP_USE_DWARF_UNWINDER) && defined(STP_NEED_UNWIND_DATA)\n";
- c->output << "static uint8_t _stp_module_" << stpmod_idx
- << "_debug_frame[] = \n";
- c->output << " {";
- if (debug_len > MAX_UNWIND_TABLE_SIZE)
- {
- c->session.print_warning ("skipping module " + modname + " debug_frame unwind table (too big: " +
- lex_cast(debug_len) + " > " + lex_cast(MAX_UNWIND_TABLE_SIZE) + ")");
- }
- else
- for (size_t i = 0; i < debug_len; i++)
- {
- int h = ((uint8_t *)debug_frame)[i];
- c->output << h << ","; // decimal is less wordy than hex
- if ((i + 1) % 16 == 0)
- c->output << "\n" << " ";
- }
- c->output << "};\n";
- c->output << "#endif /* STP_USE_DWARF_UNWINDER && STP_NEED_UNWIND_DATA */\n";
- }
+ dump_unwindsym_cxt_table(c->session, c->output, modname, stpmod_idx, "", 0,
+ "debug_frame", debug_frame, debug_len);
- if (eh_frame != NULL && eh_len > 0)
- {
- c->output << "#if defined(STP_USE_DWARF_UNWINDER) && defined(STP_NEED_UNWIND_DATA)\n";
- c->output << "static uint8_t _stp_module_" << stpmod_idx
- << "_eh_frame[] = \n";
- c->output << " {";
- if (eh_len > MAX_UNWIND_TABLE_SIZE)
- {
- c->session.print_warning ("skipping module " + modname + " eh_frame table (too big: " +
- lex_cast(eh_len) + " > " + lex_cast(MAX_UNWIND_TABLE_SIZE) + ")");
- }
- else
- for (size_t i = 0; i < eh_len; i++)
- {
- int h = ((uint8_t *)eh_frame)[i];
- c->output << h << ","; // decimal is less wordy than hex
- if ((i + 1) % 16 == 0)
- c->output << "\n" << " ";
- }
- c->output << "};\n";
- c->output << "#endif /* STP_USE_DWARF_UNWINDER && STP_NEED_UNWIND_DATA */\n";
- }
+ dump_unwindsym_cxt_table(c->session, c->output, modname, stpmod_idx, "", 0,
+ "eh_frame", eh_frame, eh_len);
+
+ dump_unwindsym_cxt_table(c->session, c->output, modname, stpmod_idx, "", 0,
+ "eh_frame_hdr", eh_frame_hdr, eh_frame_hdr_len);
- if (eh_frame_hdr != NULL && eh_frame_hdr_len > 0)
- {
- c->output << "#if defined(STP_USE_DWARF_UNWINDER) && defined(STP_NEED_UNWIND_DATA)\n";
- c->output << "static uint8_t _stp_module_" << stpmod_idx
- << "_eh_frame_hdr[] = \n";
- c->output << " {";
- if (eh_frame_hdr_len > MAX_UNWIND_TABLE_SIZE)
- {
- c->session.print_warning (_F("skipping module %s eh_frame_hdr table (too big: %s > %s)",
- modname.c_str(), lex_cast(eh_frame_hdr_len).c_str(),
- lex_cast(MAX_UNWIND_TABLE_SIZE).c_str()));
- }
- else
- for (size_t i = 0; i < eh_frame_hdr_len; i++)
- {
- int h = ((uint8_t *)eh_frame_hdr)[i];
- c->output << h << ","; // decimal is less wordy than hex
- if ((i + 1) % 16 == 0)
- c->output << "\n" << " ";
- }
- c->output << "};\n";
- c->output << "#endif /* STP_USE_DWARF_UNWINDER && STP_NEED_UNWIND_DATA */\n";
- }
-
if (c->session.need_unwind && debug_frame == NULL && eh_frame == NULL)
{
// There would be only a small benefit to warning. A user
@@ -6035,32 +6015,8 @@ dump_unwindsym_cxt (Dwfl_Module *m,
if (secname == ".dynamic" || secname == ".absolute"
|| secname == ".text" || secname == "_stext")
{
- if (debug_frame_hdr != NULL && debug_frame_hdr_len > 0)
- {
- c->output << "#if defined(STP_USE_DWARF_UNWINDER)"
- << " && defined(STP_NEED_UNWIND_DATA)\n";
- c->output << "static uint8_t _stp_module_" << stpmod_idx
- << "_debug_frame_hdr_" << secidx << "[] = \n";
- c->output << " {";
- if (debug_frame_hdr_len > MAX_UNWIND_TABLE_SIZE)
- {
- c->session.print_warning (_F("skipping module %s, section %s debug_frame_hdr"
- " table (too big: %s > %s)", modname.c_str(),
- secname.c_str(), lex_cast(debug_frame_hdr_len).c_str(),
- lex_cast(MAX_UNWIND_TABLE_SIZE).c_str()));
- }
- else
- for (size_t i = 0; i < debug_frame_hdr_len; i++)
- {
- int h = ((uint8_t *)debug_frame_hdr)[i];
- c->output << h << ","; // decimal is less wordy than hex
- if ((i + 1) % 16 == 0)
- c->output << "\n" << " ";
- }
- c->output << "};\n";
- c->output << "#endif /* STP_USE_DWARF_UNWINDER"
- << " && STP_NEED_UNWIND_DATA */\n";
- }
+ dump_unwindsym_cxt_table(c->session, c->output, modname, stpmod_idx, secname, secidx,
+ "debug_frame_hdr", debug_frame_hdr, debug_frame_hdr_len);
}
}
--
1.8.1.2
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: eh_frame table too big
2013-02-19 2:09 ` Josh Stone
@ 2013-02-19 2:57 ` Dave Cameron
2013-02-19 3:07 ` Dave Cameron
1 sibling, 0 replies; 4+ messages in thread
From: Dave Cameron @ 2013-02-19 2:57 UTC (permalink / raw)
To: Josh Stone; +Cc: systemtap
On Mon, 18 Feb 2013 18:09:27 -0800
Josh Stone <jistone@redhat.com> wrote:
> On 02/18/2013 03:38 PM, Dave Cameron wrote:
> > I'm trying to run a systemtap script on Firefox, but I get the
> > following warning when I invoke systemtap like:
> >
> > stap firefox.stap -d /lib64/libpthread-2.15.so \
> > -d /usr/lib64/firefox/libmozsqlite3.so \
> > -d /usr/lib64/firefox/libxul.so
> > -d /usr/lib64/firefox/firefox \ -x `pidof firefox`
> >
> > WARNING: skipping module /usr/lib64/firefox/libxul.so eh_frame
> > table (too big: 6812836 > 6291456
> >
> > -
> >
> > Also after this probe compiled and loaded, I got a kpanic,
> > which I was unable to capture, but I will try to reproduce.
>
> We definitely need to address that! I think I found a place where the
> unwind table is not completely skipped, possibly leading to bad memory
> references. I'd appreciate if you could try the attached patch (while
> the max is still set too low for you).
>
> > How can I bump this limit?
>
> This is MAX_UNWIND_TABLE_SIZE in translate.cxx, currently set to 6MB.
> We have bumped this before for libxul.so, and can do so again.
>
> Josh
>
I will try the patch now, here is the calltrace from the dead kernels
log:
[ 141.385017] stap_2220df8d9f449547f614e7e1db1bbf53_1263: systemtap: 2.1/0.155, base: ffffffffa04be000, memory: 22604data/49text/8ctx/2058net/25alloc kb, probes: 1
[ 143.132015] BUG: unable to handle kernel paging request at ffffffffa1f61430
[ 143.132122] IP: [<ffffffffa04c3fd0>] is_fde+0x10/0x72 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.132245] PGD 1c0f067 PUD 1c13063 PMD 0
[ 143.132317] Oops: 0000 [#1] PREEMPT SMP
[ 143.132388] Modules linked in: stap_2220df8d9f449547f614e7e1db1bbf53_1263(O) iwldvm mac80211 btusb bluetooth iwlwifi snd_hda_codec_conexant snd_hda_intel coretemp joydev snd_hda_codec kvm_intel thinkpad_acpi cfg80211 snd_hwdep snd_pcm uhci_hcd ehci_pci ehci_hcd r852 sdhci_pci sdhci sm_common e1000e pcmcia kvm snd_timer usbcore tpm_tis tpm iTCO_wdt iTCO_vendor_support nand psmouse mmc_core firewire_ohci tpm_bios mtd firewire_core snd mei r592 lpc_ich sg rfkill yenta_socket i2c_i801 pcmcia_rsrc pcmcia_core nand_ids nand_ecc acpi_cpufreq mperf nvram battery memstick soundcore usb_common snd_page_alloc mfd_core wmi serio_raw thermal evdev ac processor microcode pcspkr
[ 143.133513] CPU 1
[ 143.133548] Pid: 1236, comm: Timer Tainted: G O 3.8.0-rc6-tink #13 LENOVO 6473TBM/6473TBM
[ 143.133655] RIP: 0010:[<ffffffffa04c3fd0>] [<ffffffffa04c3fd0>] is_fde+0x10/0x72 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.133798] RSP: 0018:ffff8801bbc89bb8 EFLAGS: 00010086
[ 143.133865] RAX: 0000000002706900 RBX: ffff88012906a3b8 RCX: 0000000000000001
[ 143.133949] RDX: 000000000067f4a4 RSI: ffffffffa1adbc40 RDI: ffffffffa1f61430
[ 143.134031] RBP: ffff8801bbc89bd8 R08: 0000000000000001 R09: 0000000000000001
[ 143.134114] R10: 000000000000003b R11: 0000000000000000 R12: ffffffffa1469590
[ 143.134197] R13: ffffffffa1f61430 R14: ffffffffa12ba750 R15: 0000000000000004
[ 143.134281] FS: 00007f5fe3bfe700(0000) GS:ffff8801bbc80000(0000) knlGS:0000000000000000
[ 143.134378] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 143.134446] CR2: ffffffffa1f61430 CR3: 0000000128c1f000 CR4: 00000000000007e0
[ 143.134529] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 143.134612] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 143.134696] Process Timer (pid: 1236, threadinfo ffff8801b351a000, task ffff8801b3789860)
[ 143.134789] Stack:
[ 143.134819] ffff88012906a3b8 ffffffffa1469590 ffffffffa1f61430 ffffffffa12ba750
[ 143.134932] ffff8801bbc89d18 ffffffffa04c5d96 ffffe8ffffc800b6 ffffe8ffffc800b6
[ 143.135043] 000000000000006a 000000000000006a ffff8801bbc89ca8 ffffffffa04c383c
[ 143.135142] Call Trace:
[ 143.135142] <#DB>
[ 143.135142]
[ 143.135142] [<ffffffffa04c5d96>] unwind_frame+0x430/0xf87 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] [<ffffffffa04c383c>] ? _stp_vsnprintf+0x6f9/0x8d9 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] [<ffffffffa04c6a0d>] unwind+0x120/0x13d [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] [<ffffffffa04c6c25>] _stp_stack_unwind_one_user+0xaa/0x104 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] [<ffffffffa04c6dc8>] probe_3784+0x149/0x22f [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] [<ffffffff81187ac9>] ? sys_write+0x1/0x7e
[ 143.135142] [<ffffffff81187ac8>] ? sys_read+0x7e/0x7e
[ 143.135142] [<ffffffffa04c7f40>] enter_kprobe_probe+0x12b/0x293 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] [<ffffffff81664d1d>] kprobe_exceptions_notify+0x131/0x446
[ 143.135142] [<ffffffff81187ac9>] ? sys_write+0x1/0x7e
[ 143.135142] [<ffffffff81665be3>] notifier_call_chain+0x37/0x63
[ 143.135142] [<ffffffff81665c4c>] __atomic_notifier_call_chain+0x3d/0x4f
[ 143.135142] [<ffffffff81665c72>] atomic_notifier_call_chain+0x14/0x16
[ 143.135142] [<ffffffff81665ca2>] notify_die+0x2e/0x30
[ 143.135142] [<ffffffff816635b9>] do_int3+0x47/0xd8
[ 143.135142] [<ffffffff81662ddb>] int3+0x2b/0x40
[ 143.135142] [<ffffffff81187ac8>] ? sys_read+0x7e/0x7e
[ 143.135142] <<EOE>>
[ 143.135142]
[ 143.135142] [<ffffffff816692af>] ? tracesys+0xe1/0xe6
[ 143.135142] Code: e8 4c 89 45 f0 4c 89 4d f8 c7 45 b0 08 00 00 00 48 89 45 c0 e8 6b fe ff ff c9 c3 55 48 89 e5 41 56 41 55 41 54 53 66 66 66 66 90 <44> 8b 37 48 89 fb 49 89 f4 41 89 d5 45 85 f6 74 09 45 89 f6 41
[ 143.135142] RIP [<ffffffffa04c3fd0>] is_fde+0x10/0x72 [stap_2220df8d9f449547f614e7e1db1bbf53_1263]
[ 143.135142] RSP <ffff8801bbc89bb8>
[ 143.135142] CR2: ffffffffa1f61430
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: eh_frame table too big
2013-02-19 2:09 ` Josh Stone
2013-02-19 2:57 ` Dave Cameron
@ 2013-02-19 3:07 ` Dave Cameron
1 sibling, 0 replies; 4+ messages in thread
From: Dave Cameron @ 2013-02-19 3:07 UTC (permalink / raw)
To: systemtap
On Mon, 18 Feb 2013 18:09:27 -0800
Josh Stone <jistone@redhat.com> wrote:
> On 02/18/2013 03:38 PM, Dave Cameron wrote:
> > I'm trying to run a systemtap script on Firefox, but I get the
> > following warning when I invoke systemtap like:
> >
> > stap firefox.stap -d /lib64/libpthread-2.15.so \
> > -d /usr/lib64/firefox/libmozsqlite3.so \
> > -d /usr/lib64/firefox/libxul.so
> > -d /usr/lib64/firefox/firefox \ -x `pidof firefox`
> >
> > WARNING: skipping module /usr/lib64/firefox/libxul.so eh_frame
> > table (too big: 6812836 > 6291456
> >
> > -
> >
> > Also after this probe compiled and loaded, I got a kpanic,
> > which I was unable to capture, but I will try to reproduce.
>
> We definitely need to address that! I think I found a place where the
> unwind table is not completely skipped, possibly leading to bad memory
> references. I'd appreciate if you could try the attached patch (while
> the max is still set too low for you).
>
> > How can I bump this limit?
>
> This is MAX_UNWIND_TABLE_SIZE in translate.cxx, currently set to 6MB.
> We have bumped this before for libxul.so, and can do so again.
>
> Josh
>
Your patch fixed it.
Thanks,
Dave
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-02-19 3:07 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-02-18 23:38 eh_frame table too big Dave Cameron
2013-02-19 2:09 ` Josh Stone
2013-02-19 2:57 ` Dave Cameron
2013-02-19 3:07 ` Dave Cameron
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).