* [Bug runtime/15805] maps should not be allocated with k*alloc
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
@ 2013-07-30 13:55 ` fche at redhat dot com
2013-07-30 14:09 ` jkaluza at redhat dot com
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: fche at redhat dot com @ 2013-07-30 13:55 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
--- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
Created attachment 7128
--> http://sourceware.org/bugzilla/attachment.cgi?id=7128&action=edit
oom message
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug runtime/15805] New: maps should not be allocated with k*alloc
@ 2013-07-30 13:55 fche at redhat dot com
2013-07-30 13:55 ` [Bug runtime/15805] " fche at redhat dot com
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: fche at redhat dot com @ 2013-07-30 13:55 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
Bug ID: 15805
Summary: maps should not be allocated with k*alloc
Product: systemtap
Version: unspecified
Status: NEW
Severity: normal
Priority: P2
Component: runtime
Assignee: systemtap at sourceware dot org
Reporter: fche at redhat dot com
We should (re)turn(?) to using vmalloc to get the potentially
multi-megabyte-sized chunks of memory used for runtime/map objects.
It's too easy for kernel free memory to be fragmented, and kmalloc
to fail, even for moderate sized objects.
The attached OOM was seen in the wild for a 3.9.4-200.fc18.x86_64
kernel, running a script with
global foo
probe ... { foo[pid(),tid()]=$number }
i.e., only about a hundred kilobytes' worth.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug runtime/15805] maps should not be allocated with k*alloc
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
2013-07-30 13:55 ` [Bug runtime/15805] " fche at redhat dot com
@ 2013-07-30 14:09 ` jkaluza at redhat dot com
2013-08-07 20:29 ` dsmith at redhat dot com
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: jkaluza at redhat dot com @ 2013-07-30 14:09 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
Jan Kaluza <jkaluza at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jkaluza at redhat dot com
--- Comment #2 from Jan Kaluza <jkaluza at redhat dot com> ---
I'm the original reproducer of this bug and I presume I'll be able to reproduce
it for some time if you want some tests or information.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug runtime/15805] maps should not be allocated with k*alloc
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
2013-07-30 13:55 ` [Bug runtime/15805] " fche at redhat dot com
2013-07-30 14:09 ` jkaluza at redhat dot com
@ 2013-08-07 20:29 ` dsmith at redhat dot com
2013-08-13 16:29 ` dsmith at redhat dot com
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: dsmith at redhat dot com @ 2013-08-07 20:29 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
David Smith <dsmith at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |dsmith at redhat dot com
--- Comment #3 from David Smith <dsmith at redhat dot com> ---
Created attachment 7135
--> http://sourceware.org/bugzilla/attachment.cgi?id=7135&action=edit
potential patch
Here's a patch that changes map allocation to use vmalloc instead of kmalloc.
Tested on rhel5 (ia64, x86_64) and rawhide (x86_64).
Note a couple of things:
1) There isn't a exported rawhide kernel vmalloc variant that supports
allocating on a particular node and lets you set flags. I chose to use the
variant that lets you allocate on a particular node.
2) On rhel5 ia64 (2.6.18-348.6.1.el5debug), I'm seeing some bad/odd behavior.
With the patch applied, you can use -DUSE_KMALLOC to use kmalloc for allocation
instead.
====
# time stap -DUSE_KMALLOC --disable-cache
../src/testsuite/systemtap.maps/ix_clear.stp
foo[1] = 1 1
ERROR: empty aggregate near operator '@min' at
../src/testsuite/systemtap.maps/ix_clear.stp:14:45
WARNING: Number of errors: 1, skipped probes: 0
WARNING: /usr/local/bin/staprun exited with status: 1
Pass 5: run failed. [man error::pass5]
real 0m20.055s
user 0m16.192s
sys 0m2.734s
====
So, using kmalloc(), it takes around 20 seconds to run that test (which is
supposed to fail).
Using vmalloc():
====
time stap --disable-cache ../src/testsuite/systemtap.maps/ix_clear.stp
foo[1] = 1 1
ERROR: empty aggregate near operator '@min' at
../src/testsuite/systemtap.maps/ix_clear.stp:14:45
WARNING: Number of errors: 1, skipped probes: 0
WARNING: /usr/local/bin/staprun exited with status: 1
Pass 5: run failed. [man error::pass5]
real 11m34.415s
user 0m17.374s
sys 11m15.986s
====
With vmalloc, it takes over 11 minutes.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug runtime/15805] maps should not be allocated with k*alloc
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
` (2 preceding siblings ...)
2013-08-07 20:29 ` dsmith at redhat dot com
@ 2013-08-13 16:29 ` dsmith at redhat dot com
2013-08-13 20:02 ` dsmith at redhat dot com
2013-08-16 16:01 ` dsmith at redhat dot com
5 siblings, 0 replies; 7+ messages in thread
From: dsmith at redhat dot com @ 2013-08-13 16:29 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
--- Comment #4 from David Smith <dsmith at redhat dot com> ---
Here's some additional information. After some more debugging, I've found out
the systemtap.maps/ix_clear.stp testcase ends up making one call to
_stp_pmap_new() with the following parameters: max_entries: 2048, wrap: 0,
node_size 72. This ends up with 67618 calls to _stp_map_vzalloc().
That's right, over 67000 calls to _stp_map_vzalloc() from 1 call to
_stp_pmap_new().
Looks like if we switch to vmalloc, we need to figure out how to allocate in
bigger chunks.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug runtime/15805] maps should not be allocated with k*alloc
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
` (3 preceding siblings ...)
2013-08-13 16:29 ` dsmith at redhat dot com
@ 2013-08-13 20:02 ` dsmith at redhat dot com
2013-08-16 16:01 ` dsmith at redhat dot com
5 siblings, 0 replies; 7+ messages in thread
From: dsmith at redhat dot com @ 2013-08-13 20:02 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
David Smith <dsmith at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #7135|0 |1
is obsolete| |
--- Comment #5 from David Smith <dsmith at redhat dot com> ---
Created attachment 7143
--> http://sourceware.org/bugzilla/attachment.cgi?id=7143&action=edit
v2 patch
Here's a new version of the patch that takes allocations from over 67000 to 67.
With this new patch, here are some timed runs:
====
# time stap -DUSE_KMALLOC --disable-cache
../src/testsuite/systemtap.maps/ix_clear.stp
foo[1] = 1 1
ERROR: empty aggregate near operator '@min' at
../src/testsuite/systemtap.maps/ix_clear.stp:14:45
WARNING: Number of errors: 1, skipped probes: 0
WARNING: /usr/local/bin/staprun exited with status: 1
Pass 5: run failed. [man error::pass5]
real 0m20.810s
user 0m17.327s
sys 0m2.425s
# time stap --disable-cache ../src/testsuite/systemtap.maps/ix_clear.stp
foo[1] = 1 1
ERROR: empty aggregate near operator '@min' at
../src/testsuite/systemtap.maps/ix_clear.stp:14:45
WARNING: Number of errors: 1, skipped probes: 0
WARNING: /usr/local/bin/staprun exited with status: 1
Pass 5: run failed. [man error::pass5]
real 0m20.704s
user 0m17.352s
sys 0m2.306s
====
With this version of the patch, the new kmalloc and vmalloc versions of the
code times are very similar. With this patch, all of the systemtap.maps/*.exp
tests pass. I haven't done a full testsuite run.
Note that allocations could be lowered further by using ideas from the dyninst
runtime, which only makes 1 allocation per map.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug runtime/15805] maps should not be allocated with k*alloc
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
` (4 preceding siblings ...)
2013-08-13 20:02 ` dsmith at redhat dot com
@ 2013-08-16 16:01 ` dsmith at redhat dot com
5 siblings, 0 replies; 7+ messages in thread
From: dsmith at redhat dot com @ 2013-08-16 16:01 UTC (permalink / raw)
To: systemtap
http://sourceware.org/bugzilla/show_bug.cgi?id=15805
David Smith <dsmith at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #6 from David Smith <dsmith at redhat dot com> ---
Fixed in commit 3f873e5.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2013-08-16 16:01 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-30 13:55 [Bug runtime/15805] New: maps should not be allocated with k*alloc fche at redhat dot com
2013-07-30 13:55 ` [Bug runtime/15805] " fche at redhat dot com
2013-07-30 14:09 ` jkaluza at redhat dot com
2013-08-07 20:29 ` dsmith at redhat dot com
2013-08-13 16:29 ` dsmith at redhat dot com
2013-08-13 20:02 ` dsmith at redhat dot com
2013-08-16 16:01 ` dsmith at redhat dot com
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).