* [Bug kprobes/10839] kretprobes use excessive memory on x86_64 rawhide
2009-10-24 2:30 [Bug kprobes/10839] New: kretprobes use excessive memory on x86_64 rawhide jistone at redhat dot com
@ 2009-10-25 8:01 ` fche at redhat dot com
2009-10-26 23:07 ` jistone at redhat dot com
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: fche at redhat dot com @ 2009-10-25 8:01 UTC (permalink / raw)
To: systemtap
------- Additional Comments From fche at redhat dot com 2009-10-25 08:01 -------
Might this be accounted for by a change to NR_CPUs leading
to an excessive default kretprobe .maxactive? Try setting
a low -DKRETACTIVE=nnn parameter.
--
http://sourceware.org/bugzilla/show_bug.cgi?id=10839
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug kprobes/10839] kretprobes use excessive memory on x86_64 rawhide
2009-10-24 2:30 [Bug kprobes/10839] New: kretprobes use excessive memory on x86_64 rawhide jistone at redhat dot com
2009-10-25 8:01 ` [Bug kprobes/10839] " fche at redhat dot com
@ 2009-10-26 23:07 ` jistone at redhat dot com
2009-10-26 23:14 ` jistone at redhat dot com
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: jistone at redhat dot com @ 2009-10-26 23:07 UTC (permalink / raw)
To: systemtap
------- Additional Comments From jistone at redhat dot com 2009-10-26 23:07 -------
(In reply to comment #1)
> Might this be accounted for by a change to NR_CPUs leading
> to an excessive default kretprobe .maxactive? Try setting
> a low -DKRETACTIVE=nnn parameter.
Ah -- indeed, on x86_64 NR_CPUS = 512, where on F11 it was only 64. We default
to 6*NR_CPUS, but that still doesn't account for it:
(2802 probes)*(6*512 maxactive)*(40b kretprobe_instance) = ~330MB
A different measurement:
# stap -e 'probe kernel.function("*@fs/*").return { next }' -DKRETACTIVE=100 -c free
total used free shared buffers cached
Mem: 458900 398628 60272 0 7488 148036
-/+ buffers/cache: 243104 215796
Swap: 522104 1260 520844
# stap -e 'probe kernel.function("*@fs/*").return { next }' -DKRETACTIVE=200 -c free
total used free shared buffers cached
Mem: 458900 435944 22956 0 7488 148012
-/+ buffers/cache: 280444 178456
Swap: 522104 1260 520844
That's 37MB for a change of 100 maxactive, which comes to about 136 bytes each.
--
http://sourceware.org/bugzilla/show_bug.cgi?id=10839
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug kprobes/10839] kretprobes use excessive memory on x86_64 rawhide
2009-10-24 2:30 [Bug kprobes/10839] New: kretprobes use excessive memory on x86_64 rawhide jistone at redhat dot com
2009-10-25 8:01 ` [Bug kprobes/10839] " fche at redhat dot com
2009-10-26 23:07 ` jistone at redhat dot com
@ 2009-10-26 23:14 ` jistone at redhat dot com
2009-10-29 17:02 ` [Bug translator/10839] KRETACTIVE should have a smaller default value jistone at redhat dot com
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: jistone at redhat dot com @ 2009-10-26 23:14 UTC (permalink / raw)
To: systemtap
------- Additional Comments From jistone at redhat dot com 2009-10-26 23:14 -------
(In reply to comment #2)
> That's 37MB for a change of 100 maxactive, which comes to about 136 bytes each.
This waste is because kretprobes allocates each instance with a separate
kmalloc, which usually allocates more than actually requested.
global waste
probe kernel.trace("kmalloc") {
waste <<< ($bytes_alloc - $bytes_req)
}
probe timer.s(1) {
if (@count(waste))
printdln(" ", @count(waste), @sum(waste), @avg(waste))
delete waste
}
During KRETACTIVE=100, I get:
271 21949 80
281281 27015907 96
141 13280 94
During KRETACTIVE=200, I get:
25 2664 106
561746 53935824 96
127 12264 96
--
http://sourceware.org/bugzilla/show_bug.cgi?id=10839
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug translator/10839] KRETACTIVE should have a smaller default value
2009-10-24 2:30 [Bug kprobes/10839] New: kretprobes use excessive memory on x86_64 rawhide jistone at redhat dot com
` (2 preceding siblings ...)
2009-10-26 23:14 ` jistone at redhat dot com
@ 2009-10-29 17:02 ` jistone at redhat dot com
2009-10-30 12:12 ` fche at redhat dot com
2009-10-30 22:50 ` jistone at redhat dot com
5 siblings, 0 replies; 7+ messages in thread
From: jistone at redhat dot com @ 2009-10-29 17:02 UTC (permalink / raw)
To: systemtap
------- Additional Comments From jistone at redhat dot com 2009-10-29 17:01 -------
I've submitted bug #10869 to track the kretprobe_instance waste. Within
systemtap itself, we should find a more sane default for KRETACTIVE, probably
based on the number of cpus that are actually online instead of NR_CPUS.
--
What |Removed |Added
----------------------------------------------------------------------------
Component|kprobes |translator
Summary|kretprobes use excessive |KRETACTIVE should have a
|memory on x86_64 rawhide |smaller default value
http://sourceware.org/bugzilla/show_bug.cgi?id=10839
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug translator/10839] KRETACTIVE should have a smaller default value
2009-10-24 2:30 [Bug kprobes/10839] New: kretprobes use excessive memory on x86_64 rawhide jistone at redhat dot com
` (3 preceding siblings ...)
2009-10-29 17:02 ` [Bug translator/10839] KRETACTIVE should have a smaller default value jistone at redhat dot com
@ 2009-10-30 12:12 ` fche at redhat dot com
2009-10-30 22:50 ` jistone at redhat dot com
5 siblings, 0 replies; 7+ messages in thread
From: fche at redhat dot com @ 2009-10-30 12:12 UTC (permalink / raw)
To: systemtap
------- Additional Comments From fche at redhat dot com 2009-10-30 12:12 -------
commit 1ee6b5f
--
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |WAITING
http://sourceware.org/bugzilla/show_bug.cgi?id=10839
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug translator/10839] KRETACTIVE should have a smaller default value
2009-10-24 2:30 [Bug kprobes/10839] New: kretprobes use excessive memory on x86_64 rawhide jistone at redhat dot com
` (4 preceding siblings ...)
2009-10-30 12:12 ` fche at redhat dot com
@ 2009-10-30 22:50 ` jistone at redhat dot com
5 siblings, 0 replies; 7+ messages in thread
From: jistone at redhat dot com @ 2009-10-30 22:50 UTC (permalink / raw)
To: systemtap
------- Additional Comments From jistone at redhat dot com 2009-10-30 22:49 -------
(In reply to comment #5)
> commit 1ee6b5f
That gives me 96 on F12-x86_64, which is manageable. Thanks!
--
What |Removed |Added
----------------------------------------------------------------------------
Status|WAITING |RESOLVED
Resolution| |FIXED
http://sourceware.org/bugzilla/show_bug.cgi?id=10839
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
^ permalink raw reply [flat|nested] 7+ messages in thread