Default number of active kretprobes

Default number of active kretprobes

[William Cohen]

Martin noticed that some of the examples were dropping kretprobes on
his laptop running a uniprocessor kernel. It appears the maximum
number of return events allowed at a time is too small. Example works
on smp kernel, but run out of return probes on uniprocessor
kernels. Did some investigation earlier this week. The struct
kretprobe has maxactive field to track number of entries available.

struct kretprobe {
         struct kprobe kp;
         kretprobe_handler_t handler;
         int maxactive;
         int nmissed;
         struct hlist_head free_instances;
         struct hlist_head used_instances;
};

kprobe register_kretprobe() has the following questionable code. Why
handle preemptable kernels specially? Why not just have max(10, 2 
*NR_CPUS) in all cases?

#ifdef CONFIG_PREEMPT
                 rp->maxactive = max(10, 2 * NR_CPUS);
#else
                 rp->maxactive = NR_CPUS;
#endif

in include/linux/threads.h

#ifdef CONFIG_SMP
#define NR_CPUS         CONFIG_NR_CPUS
#else
#define NR_CPUS         1
#endif

CONFIG_NR_CPUS

./arch/i386/defconfig:CONFIG_NR_CPUS=8

Re: Default number of active kretprobes

[Jim Keniston]

On Thu, 2006-03-23 at 07:07, William Cohen wrote:
> Martin noticed that some of the examples were dropping kretprobes on
> his laptop running a uniprocessor kernel. It appears the maximum
> number of return events allowed at a time is too small. Example works
> on smp kernel, but run out of return probes on uniprocessor
> kernels. Did some investigation earlier this week. The struct
> kretprobe has maxactive field to track number of entries available.
> 
...
> 
> kprobe register_kretprobe() has the following questionable code. Why
> handle preemptable kernels specially? Why not just have max(10, 2 
> *NR_CPUS) in all cases?

Kprobes is working as documented in Documentation/kprobes.txt.

If the function isn't recursive and doesn't yield the CPU (which is the
case for lots of kernel functions), then there should never be more than
NR_CPUS instances active.

But if preemption is enabled, the chances of the function yielding the
CPU (involuntarily) are greater, so we bump up maxactive (somewhat
arbitrarily).

The intention was that the user will specify a bigger (or smaller) value
for maxactive if the default is not appropriate.  This is perhaps
inconvenient for SystemTap, which wants to do everything possible for
the user.  SystemTap could choose a larger "default" and/or allow the
user to explicitly specify a value for maxactive.  This is the subject
of bugzilla #1802.

Jim

> 
> #ifdef CONFIG_PREEMPT
>                  rp->maxactive = max(10, 2 * NR_CPUS);
> #else
>                  rp->maxactive = NR_CPUS;
> #endif
> 
> in include/linux/threads.h
> 
> #ifdef CONFIG_SMP
> #define NR_CPUS         CONFIG_NR_CPUS
> #else
> #define NR_CPUS         1
> #endif
> 
> CONFIG_NR_CPUS
> 
> ./arch/i386/defconfig:CONFIG_NR_CPUS=8

Re: Default number of active kretprobes

[Jim Keniston]

By the way, regarding the problem of underestimating maxactive, we
briefly considered the idea of having a system-wide pool of
kretprobe_instances to draw on in case any particular kretprobe runs
short.  But that adds a little more complexity to the implementation and
to the user interface.  (Presumably, the pool size would be a
configuration option.)

It shouldn't be too tough to prototype that -- probably just adjust
get_free_rp_inst() and recycle_rp_inst().

Jim

Re: Default number of active kretprobes

[Jim Keniston]

On Thu, 2006-03-23 at 16:42, Jim Keniston wrote:
> By the way, regarding the problem of underestimating maxactive, we
> briefly considered the idea of having a system-wide pool of
> kretprobe_instances to draw on in case any particular kretprobe runs
> short.  But that adds a little more complexity to the implementation and
> to the user interface.  (Presumably, the pool size would be a
> configuration option.)
> 
> It shouldn't be too tough to prototype that -- probably just adjust
> get_free_rp_inst() and recycle_rp_inst().
> 
> Jim
> 

Yet another possibility is per-module pools of spare
kretprobe_instances.  E.g., for each script, SystemTap could generate
code to call (say) pool = alloc_kretprobe_spares_pool(nspares) during
module init, set each kretprobe's rp.spares_pool = pool before calling
register_kretprobe(), and call free_kretprobe_spares_pool(pool) during
module exit.

Of course, allowing pools to come and go is more complicated than having
an "eternal" global pool, but I think we could handle freeing of pools
the same way we handle unregistering of kretprobes.

I figure if I keep speculating about this, somebody will come along and
point out how we don't need kretprobe_instances at all. :-}

Jim

Re: Default number of active kretprobes

[Frank Ch. Eigler]

jkenisto wrote:

> [...]  Yet another possibility is per-module pools of spare
> kretprobe_instances.  E.g., for each script, SystemTap could
> generate code to call (say) pool =
> alloc_kretprobe_spares_pool(nspares) [...]

It would not be a problem to do this in the translator.  However, it's
also not a big deal to set maxactive to a "large" number like
NR_CPU*100 (each instance being <32 bytes, individually kmalloc'd).

- FChE