* RE: Fwd: Any way to find the network usage by a process?
@ 2006-10-06 1:08 Stone, Joshua I
0 siblings, 0 replies; 8+ messages in thread
From: Stone, Joshua I @ 2006-10-06 1:08 UTC (permalink / raw)
To: Frank Ch. Eigler, Mike Mason; +Cc: systemtap
On Thursday, October 05, 2006 5:00 PM, Frank Ch. Eigler wrote:
> Hi -
>
> On Thu, Oct 05, 2006 at 04:28:10PM -0700, Mike Mason wrote:
>> [...]
>> ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4
>> WARNING: Number of errors: 1, skipped probes: 0
>> Apparently using @sum on empty aggregates isn't allowed. I expected
>> 0's to be returned.
>
> As a judgement call, to be consistent with other extractors like @avg,
> the @sum etc. of an empty set was deemed to be undefined.
FWIW, Python allows you to take the sum of an empty list, even though
other equivalent extractors fail:
>>> sum([])
0
>>> max([])
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: min() or max() arg is an empty sequence
I'm just looking for precedence in other languages...
>> The only way to avoid the error is use @sum only if @count >
>> 0, which makes the printf too complex in my opinion.
>
> Maybe so. It's worth considering some syntactic sugar to express a
> undefined=>0 intent.
One possibility is an optional second parameter to the extractor that
gives a default value. We could require this to be a constant (usually
0), or maybe even allow a numeric expression.
a <<< 1; del a // prepare an empty aggregate
printf("%d\n", @sum(a)) // runtime error
printf("%d\n", @sum(a, 0)) // prints 0
printf("%d\n", @sum(a, x)) // prints value of x
This works for min, max, and avg as well.
Or, instead of a default, it might be less confusing to consider the
second parameter as an initializer -- so when 'a' isn't empty, @sum(a,x)
== x + @sum(a). Then min and max would effectively get an upper and
lower bound, respectively. I'm not sure if avg works well this way
though.
Josh
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
2006-10-06 1:10 ` Mike Mason
@ 2006-10-06 15:30 ` Frank Ch. Eigler
0 siblings, 0 replies; 8+ messages in thread
From: Frank Ch. Eigler @ 2006-10-06 15:30 UTC (permalink / raw)
To: Mike Mason; +Cc: systemtap
Mike Mason <mmlnx@us.ibm.com> writes:
> [...]
> >> The only way to avoid the error is use @sum only if @count > 0,
> >> which makes the printf too complex in my opinion.
> [...]
Actually, another way is to add a "<<< 0" dummy value to the
stats arrays in a new "probe begin", then to display @count()-1
and @sum() etc.
- FChE
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
2006-10-06 0:00 ` Frank Ch. Eigler
@ 2006-10-06 1:10 ` Mike Mason
2006-10-06 15:30 ` Frank Ch. Eigler
0 siblings, 1 reply; 8+ messages in thread
From: Mike Mason @ 2006-10-06 1:10 UTC (permalink / raw)
To: Frank Ch. Eigler; +Cc: systemtap
Frank Ch. Eigler wrote:
> Hi -
>
> On Thu, Oct 05, 2006 at 04:28:10PM -0700, Mike Mason wrote:
>> [...]
>> ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4
>> WARNING: Number of errors: 1, skipped probes: 0
>> Apparently using @sum on empty aggregates isn't allowed. I expected 0's to
>> be returned.
>
> As a judgement call, to be consistent with other extractors like @avg,
> the @sum etc. of an empty set was deemed to be undefined.
>
>> The only way to avoid the error is use @sum only if @count >
>> 0, which makes the printf too complex in my opinion.
>
> Maybe so. It's worth considering some syntactic sugar to express a
> undefined=>0 intent.
Actually, the following works fine and doesn't complicate it too much:
foreach ([pid, dev] in ifpid-) {
n_xmit = @count(ifxmit[pid, dev])
n_recv = @count(ifrecv[pid, dev])
printf("%5d %5d %-7s %7d %7d %7d %7d %-15s\n",
pid, user[pid], dev, n_xmit, n_recv,
n_xmit ? @sum(ifxmit[pid, dev])/1024 : 0,
n_recv ? @sum(ifrecv[pid, dev])/1024 : 0,
execname[pid])
}
I'll update the wiki page. Thanks for the suggestions.
- Mike
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
2006-10-05 23:28 ` Mike Mason
@ 2006-10-06 0:00 ` Frank Ch. Eigler
2006-10-06 1:10 ` Mike Mason
0 siblings, 1 reply; 8+ messages in thread
From: Frank Ch. Eigler @ 2006-10-06 0:00 UTC (permalink / raw)
To: Mike Mason; +Cc: systemtap
Hi -
On Thu, Oct 05, 2006 at 04:28:10PM -0700, Mike Mason wrote:
> [...]
> ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4
> WARNING: Number of errors: 1, skipped probes: 0
> Apparently using @sum on empty aggregates isn't allowed. I expected 0's to
> be returned.
As a judgement call, to be consistent with other extractors like @avg,
the @sum etc. of an empty set was deemed to be undefined.
> The only way to avoid the error is use @sum only if @count >
> 0, which makes the printf too complex in my opinion.
Maybe so. It's worth considering some syntactic sugar to express a
undefined=>0 intent.
> >The way that the script tracks pid-to-uid and pid-to-execname mappings
> >is not bad, though if that part were moved to new probes on fork or
> >exec, it would allow the network-related probes to run concurrently on
> >an SMP without fighting over locks.
>
> But that would only catch processes created after the script starts,
> correct?
That's true.
- FChE
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
2006-10-05 21:22 ` Frank Ch. Eigler
@ 2006-10-05 23:28 ` Mike Mason
2006-10-06 0:00 ` Frank Ch. Eigler
0 siblings, 1 reply; 8+ messages in thread
From: Mike Mason @ 2006-10-05 23:28 UTC (permalink / raw)
To: Frank Ch. Eigler; +Cc: jrs, Irfan Habib, Linux kernel, SystemTAP
Frank Ch. Eigler wrote:
> Mike Mason <mmlnx@us.ibm.com> writes:
>
>> Here's a variation of Jose's script that uses the networking tapset
>> and prints top-like output for transmits and receives. [...]
>
> Thanks for posting it to the systemtap wiki.
>
> Some minor style suggestions follow:
>
>> [...]
>> ifxmit_p[pid(), dev_name] ++
>> ifxmit_b[pid(), dev_name] += length
>
> These could be collapsed into a single statistics-aggregate array:
> # ifxmit[pid(), dev_name] <<< length
> Then the printing routine would use @count(ifxmit[...]) and @sum(ifxmit[...])
> to extract the two values. Same of course for ifrecv.
I tried that and got the following output:
PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND
0 0 eth0 9 10 486 672 swapper
ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4
WARNING: Number of errors: 1, skipped probes: 0
Apparently using @sum on empty aggregates isn't allowed. I expected 0's to
be returned. The only way to avoid the error is use @sum only if @count >
0, which makes the printf too complex in my opinion.
>
>> execname[pid()] = execname()
>> user[pid()] = uid()
>> ifdevs[pid(), dev_name] = dev_name
>
> Calling pid() so many times is worse than calling it once and caching
> the result in a local variable ("p = pid()").
Agreed. I'll change that.
>
> The way that the script tracks pid-to-uid and pid-to-execname mappings
> is not bad, though if that part were moved to new probes on fork or
> exec, it would allow the network-related probes to run concurrently on
> an SMP without fighting over locks.
But that would only catch processes created after the script starts, correct?
- Mike
>
>
> - FChE
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
2006-10-04 20:54 ` Mike Mason
@ 2006-10-05 21:22 ` Frank Ch. Eigler
2006-10-05 23:28 ` Mike Mason
0 siblings, 1 reply; 8+ messages in thread
From: Frank Ch. Eigler @ 2006-10-05 21:22 UTC (permalink / raw)
To: Mike Mason; +Cc: jrs, Irfan Habib, Linux kernel, SystemTAP
Mike Mason <mmlnx@us.ibm.com> writes:
> Here's a variation of Jose's script that uses the networking tapset
> and prints top-like output for transmits and receives. [...]
Thanks for posting it to the systemtap wiki.
Some minor style suggestions follow:
> [...]
> ifxmit_p[pid(), dev_name] ++
> ifxmit_b[pid(), dev_name] += length
These could be collapsed into a single statistics-aggregate array:
# ifxmit[pid(), dev_name] <<< length
Then the printing routine would use @count(ifxmit[...]) and @sum(ifxmit[...])
to extract the two values. Same of course for ifrecv.
> execname[pid()] = execname()
> user[pid()] = uid()
> ifdevs[pid(), dev_name] = dev_name
Calling pid() so many times is worse than calling it once and caching
the result in a local variable ("p = pid()").
The way that the script tracks pid-to-uid and pid-to-execname mappings
is not bad, though if that part were moved to new probes on fork or
exec, it would allow the network-related probes to run concurrently on
an SMP without fighting over locks.
- FChE
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
2006-10-03 15:49 ` Jose R. Santos
@ 2006-10-04 20:54 ` Mike Mason
2006-10-05 21:22 ` Frank Ch. Eigler
0 siblings, 1 reply; 8+ messages in thread
From: Mike Mason @ 2006-10-04 20:54 UTC (permalink / raw)
To: jrs; +Cc: Irfan Habib, Linux kernel, SystemTAP
Here's a variation of Jose's script that uses the networking tapset and
prints top-like output for transmits and receives. Much of the activity
shows up under pid 0, which Jose's script filtered out. This obviously
doesn't reflect the actual process generating the traffic.
The networking tapset currently probes netif_receive_skb() for receives and
dev_queue_xmit() for transmits. Can anyone suggest better probe points to
get transmits and receives by pid?
- Mike
Sample output:
PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND
0 0 eth0 493 880 32 1238 swapper
3078 0 eth0 26 2 28 2 Xvnc
13957 0 eth0 1 2 0 2 lspci
3148 0 eth0 1 2 0 2 nautilus
5058 0 eth0 1 1 0 1 firefox-bin
12277 0 eth0 1 0 0 0 sshd
nettop.stp script:
global ifxmit_p, ifrecv_p, ifxmit_b, ifrecv_b, ifdevs, ifpid, execname, user
probe netdev.transmit
{
execname[pid()] = execname()
user[pid()] = uid()
ifdevs[pid(), dev_name] = dev_name
ifxmit_p[pid(), dev_name] ++
ifxmit_b[pid(), dev_name] += length
ifpid[pid(), dev_name] ++
}
probe netdev.receive
{
execname[pid()] = execname()
user[pid()] = uid()
ifdevs[pid(), dev_name] = dev_name
ifrecv_p[pid(), dev_name] ++
ifrecv_b[pid(), dev_name] += length
ifpid[pid(), dev_name] ++
}
function print_activity()
{
printf("%5s %5s %-7s %7s %7s %7s %7s %-15s\n",
"PID", "UID", "DEV", "XMIT_PK", "RECV_PK", "XMIT_KB",
"RECV_KB", "COMMAND")
foreach ([pid,dev] in ifpid-) {
printf("%5d %5d %-7s %7d %7d %7d %7d %-15s\n",
pid, user[pid], dev,
ifxmit_p[pid, dev], ifrecv_p[pid, dev],
ifxmit_b[pid, dev]/1024,
ifrecv_b[pid, dev]/1024,
execname[pid])
}
print("\n")
delete execname
delete user
delete ifdevs
delete ifxmit_p
delete ifrecv_p
delete ifxmit_b
delete ifrecv_b
delete ifpid
}
probe timer.ms(5000)
{
print_activity()
}
Jose R. Santos wrote:
> Irfan Habib wrote:
>> Hi,
>>
>> Is there any method either kernel or user level which tells me which
>> process is generating how much traffic from a machine. For example if
>> some process is flooding the network, then I would like to know which
>> process (PID ideally), is generating the most traffic.
>>
>>
>
> A while ago I did a SystemTap script to solve a problem similar to
> this. It's been siting in my system for a while collecting dust and you
> currently don't need the embedded C code since the networking.stp tapset
> has all this script needs(and more), but I should point you in the right
> direction.
>
> This worked a couple of months ago but it is currently untested. Hope
> it helps.
>
> -JRS
>
>
> global ifstats, ifdevs, execname
>
> %{
> #include<linux/skbuff.h>
> #include<linux/netdevice.h>
> %}
>
> probe kernel.function("dev_queue_xmit")
> {
> execname[pid()] = execname()
> name=skb_to_name($skb)
> ifdevs[name] = name
> ifstats[pid(),name] <<< 1
> }
>
> function skb_to_name:string (skbuff:long)
> %{
> struct sk_buff *skbuff = (struct sk_buff *)((long)THIS->skbuff);
> struct net_device *netdev = skbuff->dev;
> sprintf (THIS->__retvalue, "%s" , netdev->name);
> %}
>
> probe timer.ms(5000)
> {
> exit()
> }
>
> probe end {
> foreach( pid in execname) {
> if (pid == 0) continue
> printf("%15s[%5d] ->\t", execname[pid],pid)
> foreach( ifname in ifdevs) {
> printf("[%s:%7d] \t", ifname, @count(ifstats[pid,
> ifname]))
> }
> print("\n")
> }
> print("\n")
> }
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process?
[not found] ` <3420082f0610030114o4c6998en907bccce81d28c59@mail.gmail.com>
@ 2006-10-03 15:49 ` Jose R. Santos
2006-10-04 20:54 ` Mike Mason
0 siblings, 1 reply; 8+ messages in thread
From: Jose R. Santos @ 2006-10-03 15:49 UTC (permalink / raw)
To: Irfan Habib; +Cc: Linux kernel, SystemTAP
Irfan Habib wrote:
> Hi,
>
> Is there any method either kernel or user level which tells me which
> process is generating how much traffic from a machine. For example if
> some process is flooding the network, then I would like to know which
> process (PID ideally), is generating the most traffic.
>
>
A while ago I did a SystemTap script to solve a problem similar to
this. It's been siting in my system for a while collecting dust and you
currently don't need the embedded C code since the networking.stp tapset
has all this script needs(and more), but I should point you in the right
direction.
This worked a couple of months ago but it is currently untested. Hope
it helps.
-JRS
global ifstats, ifdevs, execname
%{
#include<linux/skbuff.h>
#include<linux/netdevice.h>
%}
probe kernel.function("dev_queue_xmit")
{
execname[pid()] = execname()
name=skb_to_name($skb)
ifdevs[name] = name
ifstats[pid(),name] <<< 1
}
function skb_to_name:string (skbuff:long)
%{
struct sk_buff *skbuff = (struct sk_buff *)((long)THIS->skbuff);
struct net_device *netdev = skbuff->dev;
sprintf (THIS->__retvalue, "%s" , netdev->name);
%}
probe timer.ms(5000)
{
exit()
}
probe end {
foreach( pid in execname) {
if (pid == 0) continue
printf("%15s[%5d] ->\t", execname[pid],pid)
foreach( ifname in ifdevs) {
printf("[%s:%7d] \t", ifname,
@count(ifstats[pid, ifname]))
}
print("\n")
}
print("\n")
}
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2006-10-06 15:30 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-10-06 1:08 Fwd: Any way to find the network usage by a process? Stone, Joshua I
[not found] <3420082f0610030114o5b44b8ak7797483e02002614@mail.gmail.com>
[not found] ` <3420082f0610030114o4c6998en907bccce81d28c59@mail.gmail.com>
2006-10-03 15:49 ` Jose R. Santos
2006-10-04 20:54 ` Mike Mason
2006-10-05 21:22 ` Frank Ch. Eigler
2006-10-05 23:28 ` Mike Mason
2006-10-06 0:00 ` Frank Ch. Eigler
2006-10-06 1:10 ` Mike Mason
2006-10-06 15:30 ` Frank Ch. Eigler
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).