[Bug kprobes/10869] New: kretprobes waste a lot of memory on kretprobe_instances

[Bug kprobes/10869] New: kretprobes waste a lot of memory on kretprobe_instances

[jistone at redhat dot com]

As noted in bug #10839 comment #3, using kmalloc for each individual
kretprobe_instance wastes a lot of kernel memory.  In my measurement, each 40
byte kmalloc request actually reserves 136 bytes.  Having a large number of
kretprobes with large maxactive values makes this waste add up quickly.

-- 
           Summary: kretprobes waste a lot of memory on kretprobe_instances
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: kprobes
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: jistone at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=10869

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/10869] kretprobes waste a lot of memory on kretprobe_instances

[leitao at linux dot vnet dot ibm dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|systemtap at sources dot    |leitao at linux dot vnet dot
                   |redhat dot com              |ibm dot com
             Status|NEW                         |ASSIGNED


http://sourceware.org/bugzilla/show_bug.cgi?id=10869

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/10869] kretprobes waste a lot of memory on kretprobe_instances

[leitao at linux dot vnet dot ibm dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|leitao at linux dot vnet dot|systemtap at sources dot
                   |ibm dot com                 |redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=10869

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/10869] kretprobes waste a lot of memory on kretprobe_instances

[jkenisto at us dot ibm dot com]

------- Additional Comments From jkenisto at us dot ibm dot com  2009-10-29 17:27 -------
Some historical perspective: We originally allocated all the instances for a
particular kretprobe in a block. kmallocing the instances individually solved a
problem we encountered when the kretprobe was unregistered while some instances
were still active.  We have to turn loose of the kretprobe object when the user
unregisters it.

One alternative is to treat the instances as a pool separate from their
kretprobe -- in which case each instance would probably need a pointer to its
pool as well as its rp pointer.  (The rp pointer is promised by the API.)

It might also be good to explore the (probably heretical) idea of allocating the
instances (GFP_ATOMIC, since we can't block) only as needed.

I also worked on a prototype where a group of kretprobes shared a common pool --
the idea being that of all the functions you're probing, only a limited set
would be active at any one time.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10869

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/10869] kretprobes waste a lot of memory on kretprobe_instances

[ananth at in dot ibm dot com]

------- Additional Comments From ananth at in dot ibm dot com  2009-10-30 07:29 -------
Created an attachment (id=4344)
 --> (http://sourceware.org/bugzilla/attachment.cgi?id=4344&action=view)
Be smarter in allocating kretprobe instances

All along, kretprobes would allocate NR_CPU kretprobe_instance structures. Use
a more logical num_possible_cpus() instead.

If this patch is acceptable, I'll post it upstream.

Jim, Masami?

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10869

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/10869] kretprobes waste a lot of memory on kretprobe_instances

[ananth at in dot ibm dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mhiramat at redhat dot com,
                   |                            |jkenisto at us dot ibm dot
                   |                            |com
         AssignedTo|systemtap at sources dot    |ananth at in dot ibm dot com
                   |redhat dot com              |


http://sourceware.org/bugzilla/show_bug.cgi?id=10869

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

Re: [Bug kprobes/10869] kretprobes waste a lot of memory on kretprobe_instances

[Jim Keniston]

Quoting ananth at in dot ibm dot com <sourceware-bugzilla@sourceware.org>:

>
> ------- Additional Comments From ananth at in dot ibm dot com    
> 2009-10-30 07:29 -------
> Created an attachment (id=4344)
>  --> (http://sourceware.org/bugzilla/attachment.cgi?id=4344&action=view)
> Be smarter in allocating kretprobe instances
>
> All along, kretprobes would allocate NR_CPU kretprobe_instance   
> structures. Use
> a more logical num_possible_cpus() instead.
>
> If this patch is acceptable, I'll post it upstream.
>
> Jim, Masami?
>
> --
>
>
> http://sourceware.org/bugzilla/show_bug.cgi?id=10869
>
> ------- You are receiving this mail because: -------
> You are the assignee for the bug, or are watching the assignee.
>

Looks good to me.

While you're in that code... the default maxactive for  
non-preemptable, non-SMP systems is (still) 1.  That's always felt  
pretty uncomfortable.  But that's a different discussion, and  
presumably a different patch.