* Re: Fwd: Any way to find the network usage by a process? [not found] ` <3420082f0610030114o4c6998en907bccce81d28c59@mail.gmail.com> @ 2006-10-03 15:49 ` Jose R. Santos 2006-10-04 20:54 ` Mike Mason 0 siblings, 1 reply; 8+ messages in thread From: Jose R. Santos @ 2006-10-03 15:49 UTC (permalink / raw) To: Irfan Habib; +Cc: Linux kernel, SystemTAP Irfan Habib wrote: > Hi, > > Is there any method either kernel or user level which tells me which > process is generating how much traffic from a machine. For example if > some process is flooding the network, then I would like to know which > process (PID ideally), is generating the most traffic. > > A while ago I did a SystemTap script to solve a problem similar to this. It's been siting in my system for a while collecting dust and you currently don't need the embedded C code since the networking.stp tapset has all this script needs(and more), but I should point you in the right direction. This worked a couple of months ago but it is currently untested. Hope it helps. -JRS global ifstats, ifdevs, execname %{ #include<linux/skbuff.h> #include<linux/netdevice.h> %} probe kernel.function("dev_queue_xmit") { execname[pid()] = execname() name=skb_to_name($skb) ifdevs[name] = name ifstats[pid(),name] <<< 1 } function skb_to_name:string (skbuff:long) %{ struct sk_buff *skbuff = (struct sk_buff *)((long)THIS->skbuff); struct net_device *netdev = skbuff->dev; sprintf (THIS->__retvalue, "%s" , netdev->name); %} probe timer.ms(5000) { exit() } probe end { foreach( pid in execname) { if (pid == 0) continue printf("%15s[%5d] ->\t", execname[pid],pid) foreach( ifname in ifdevs) { printf("[%s:%7d] \t", ifname, @count(ifstats[pid, ifname])) } print("\n") } print("\n") } ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process? 2006-10-03 15:49 ` Fwd: Any way to find the network usage by a process? Jose R. Santos @ 2006-10-04 20:54 ` Mike Mason 2006-10-05 21:22 ` Frank Ch. Eigler 0 siblings, 1 reply; 8+ messages in thread From: Mike Mason @ 2006-10-04 20:54 UTC (permalink / raw) To: jrs; +Cc: Irfan Habib, Linux kernel, SystemTAP Here's a variation of Jose's script that uses the networking tapset and prints top-like output for transmits and receives. Much of the activity shows up under pid 0, which Jose's script filtered out. This obviously doesn't reflect the actual process generating the traffic. The networking tapset currently probes netif_receive_skb() for receives and dev_queue_xmit() for transmits. Can anyone suggest better probe points to get transmits and receives by pid? - Mike Sample output: PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND 0 0 eth0 493 880 32 1238 swapper 3078 0 eth0 26 2 28 2 Xvnc 13957 0 eth0 1 2 0 2 lspci 3148 0 eth0 1 2 0 2 nautilus 5058 0 eth0 1 1 0 1 firefox-bin 12277 0 eth0 1 0 0 0 sshd nettop.stp script: global ifxmit_p, ifrecv_p, ifxmit_b, ifrecv_b, ifdevs, ifpid, execname, user probe netdev.transmit { execname[pid()] = execname() user[pid()] = uid() ifdevs[pid(), dev_name] = dev_name ifxmit_p[pid(), dev_name] ++ ifxmit_b[pid(), dev_name] += length ifpid[pid(), dev_name] ++ } probe netdev.receive { execname[pid()] = execname() user[pid()] = uid() ifdevs[pid(), dev_name] = dev_name ifrecv_p[pid(), dev_name] ++ ifrecv_b[pid(), dev_name] += length ifpid[pid(), dev_name] ++ } function print_activity() { printf("%5s %5s %-7s %7s %7s %7s %7s %-15s\n", "PID", "UID", "DEV", "XMIT_PK", "RECV_PK", "XMIT_KB", "RECV_KB", "COMMAND") foreach ([pid,dev] in ifpid-) { printf("%5d %5d %-7s %7d %7d %7d %7d %-15s\n", pid, user[pid], dev, ifxmit_p[pid, dev], ifrecv_p[pid, dev], ifxmit_b[pid, dev]/1024, ifrecv_b[pid, dev]/1024, execname[pid]) } print("\n") delete execname delete user delete ifdevs delete ifxmit_p delete ifrecv_p delete ifxmit_b delete ifrecv_b delete ifpid } probe timer.ms(5000) { print_activity() } Jose R. Santos wrote: > Irfan Habib wrote: >> Hi, >> >> Is there any method either kernel or user level which tells me which >> process is generating how much traffic from a machine. For example if >> some process is flooding the network, then I would like to know which >> process (PID ideally), is generating the most traffic. >> >> > > A while ago I did a SystemTap script to solve a problem similar to > this. It's been siting in my system for a while collecting dust and you > currently don't need the embedded C code since the networking.stp tapset > has all this script needs(and more), but I should point you in the right > direction. > > This worked a couple of months ago but it is currently untested. Hope > it helps. > > -JRS > > > global ifstats, ifdevs, execname > > %{ > #include<linux/skbuff.h> > #include<linux/netdevice.h> > %} > > probe kernel.function("dev_queue_xmit") > { > execname[pid()] = execname() > name=skb_to_name($skb) > ifdevs[name] = name > ifstats[pid(),name] <<< 1 > } > > function skb_to_name:string (skbuff:long) > %{ > struct sk_buff *skbuff = (struct sk_buff *)((long)THIS->skbuff); > struct net_device *netdev = skbuff->dev; > sprintf (THIS->__retvalue, "%s" , netdev->name); > %} > > probe timer.ms(5000) > { > exit() > } > > probe end { > foreach( pid in execname) { > if (pid == 0) continue > printf("%15s[%5d] ->\t", execname[pid],pid) > foreach( ifname in ifdevs) { > printf("[%s:%7d] \t", ifname, @count(ifstats[pid, > ifname])) > } > print("\n") > } > print("\n") > } > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process? 2006-10-04 20:54 ` Mike Mason @ 2006-10-05 21:22 ` Frank Ch. Eigler 2006-10-05 23:28 ` Mike Mason 0 siblings, 1 reply; 8+ messages in thread From: Frank Ch. Eigler @ 2006-10-05 21:22 UTC (permalink / raw) To: Mike Mason; +Cc: jrs, Irfan Habib, Linux kernel, SystemTAP Mike Mason <mmlnx@us.ibm.com> writes: > Here's a variation of Jose's script that uses the networking tapset > and prints top-like output for transmits and receives. [...] Thanks for posting it to the systemtap wiki. Some minor style suggestions follow: > [...] > ifxmit_p[pid(), dev_name] ++ > ifxmit_b[pid(), dev_name] += length These could be collapsed into a single statistics-aggregate array: # ifxmit[pid(), dev_name] <<< length Then the printing routine would use @count(ifxmit[...]) and @sum(ifxmit[...]) to extract the two values. Same of course for ifrecv. > execname[pid()] = execname() > user[pid()] = uid() > ifdevs[pid(), dev_name] = dev_name Calling pid() so many times is worse than calling it once and caching the result in a local variable ("p = pid()"). The way that the script tracks pid-to-uid and pid-to-execname mappings is not bad, though if that part were moved to new probes on fork or exec, it would allow the network-related probes to run concurrently on an SMP without fighting over locks. - FChE ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process? 2006-10-05 21:22 ` Frank Ch. Eigler @ 2006-10-05 23:28 ` Mike Mason 2006-10-06 0:00 ` Frank Ch. Eigler 0 siblings, 1 reply; 8+ messages in thread From: Mike Mason @ 2006-10-05 23:28 UTC (permalink / raw) To: Frank Ch. Eigler; +Cc: jrs, Irfan Habib, Linux kernel, SystemTAP Frank Ch. Eigler wrote: > Mike Mason <mmlnx@us.ibm.com> writes: > >> Here's a variation of Jose's script that uses the networking tapset >> and prints top-like output for transmits and receives. [...] > > Thanks for posting it to the systemtap wiki. > > Some minor style suggestions follow: > >> [...] >> ifxmit_p[pid(), dev_name] ++ >> ifxmit_b[pid(), dev_name] += length > > These could be collapsed into a single statistics-aggregate array: > # ifxmit[pid(), dev_name] <<< length > Then the printing routine would use @count(ifxmit[...]) and @sum(ifxmit[...]) > to extract the two values. Same of course for ifrecv. I tried that and got the following output: PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND 0 0 eth0 9 10 486 672 swapper ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4 WARNING: Number of errors: 1, skipped probes: 0 Apparently using @sum on empty aggregates isn't allowed. I expected 0's to be returned. The only way to avoid the error is use @sum only if @count > 0, which makes the printf too complex in my opinion. > >> execname[pid()] = execname() >> user[pid()] = uid() >> ifdevs[pid(), dev_name] = dev_name > > Calling pid() so many times is worse than calling it once and caching > the result in a local variable ("p = pid()"). Agreed. I'll change that. > > The way that the script tracks pid-to-uid and pid-to-execname mappings > is not bad, though if that part were moved to new probes on fork or > exec, it would allow the network-related probes to run concurrently on > an SMP without fighting over locks. But that would only catch processes created after the script starts, correct? - Mike > > > - FChE ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process? 2006-10-05 23:28 ` Mike Mason @ 2006-10-06 0:00 ` Frank Ch. Eigler 2006-10-06 1:10 ` Mike Mason 0 siblings, 1 reply; 8+ messages in thread From: Frank Ch. Eigler @ 2006-10-06 0:00 UTC (permalink / raw) To: Mike Mason; +Cc: systemtap Hi - On Thu, Oct 05, 2006 at 04:28:10PM -0700, Mike Mason wrote: > [...] > ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4 > WARNING: Number of errors: 1, skipped probes: 0 > Apparently using @sum on empty aggregates isn't allowed. I expected 0's to > be returned. As a judgement call, to be consistent with other extractors like @avg, the @sum etc. of an empty set was deemed to be undefined. > The only way to avoid the error is use @sum only if @count > > 0, which makes the printf too complex in my opinion. Maybe so. It's worth considering some syntactic sugar to express a undefined=>0 intent. > >The way that the script tracks pid-to-uid and pid-to-execname mappings > >is not bad, though if that part were moved to new probes on fork or > >exec, it would allow the network-related probes to run concurrently on > >an SMP without fighting over locks. > > But that would only catch processes created after the script starts, > correct? That's true. - FChE ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process? 2006-10-06 0:00 ` Frank Ch. Eigler @ 2006-10-06 1:10 ` Mike Mason 2006-10-06 15:30 ` Frank Ch. Eigler 0 siblings, 1 reply; 8+ messages in thread From: Mike Mason @ 2006-10-06 1:10 UTC (permalink / raw) To: Frank Ch. Eigler; +Cc: systemtap Frank Ch. Eigler wrote: > Hi - > > On Thu, Oct 05, 2006 at 04:28:10PM -0700, Mike Mason wrote: >> [...] >> ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4 >> WARNING: Number of errors: 1, skipped probes: 0 >> Apparently using @sum on empty aggregates isn't allowed. I expected 0's to >> be returned. > > As a judgement call, to be consistent with other extractors like @avg, > the @sum etc. of an empty set was deemed to be undefined. > >> The only way to avoid the error is use @sum only if @count > >> 0, which makes the printf too complex in my opinion. > > Maybe so. It's worth considering some syntactic sugar to express a > undefined=>0 intent. Actually, the following works fine and doesn't complicate it too much: foreach ([pid, dev] in ifpid-) { n_xmit = @count(ifxmit[pid, dev]) n_recv = @count(ifrecv[pid, dev]) printf("%5d %5d %-7s %7d %7d %7d %7d %-15s\n", pid, user[pid], dev, n_xmit, n_recv, n_xmit ? @sum(ifxmit[pid, dev])/1024 : 0, n_recv ? @sum(ifrecv[pid, dev])/1024 : 0, execname[pid]) } I'll update the wiki page. Thanks for the suggestions. - Mike ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Any way to find the network usage by a process? 2006-10-06 1:10 ` Mike Mason @ 2006-10-06 15:30 ` Frank Ch. Eigler 0 siblings, 0 replies; 8+ messages in thread From: Frank Ch. Eigler @ 2006-10-06 15:30 UTC (permalink / raw) To: Mike Mason; +Cc: systemtap Mike Mason <mmlnx@us.ibm.com> writes: > [...] > >> The only way to avoid the error is use @sum only if @count > 0, > >> which makes the printf too complex in my opinion. > [...] Actually, another way is to add a "<<< 0" dummy value to the stats arrays in a new "probe begin", then to display @count()-1 and @sum() etc. - FChE ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: Fwd: Any way to find the network usage by a process? @ 2006-10-06 1:08 Stone, Joshua I 0 siblings, 0 replies; 8+ messages in thread From: Stone, Joshua I @ 2006-10-06 1:08 UTC (permalink / raw) To: Frank Ch. Eigler, Mike Mason; +Cc: systemtap On Thursday, October 05, 2006 5:00 PM, Frank Ch. Eigler wrote: > Hi - > > On Thu, Oct 05, 2006 at 04:28:10PM -0700, Mike Mason wrote: >> [...] >> ERROR: empty aggregate near identifier 'execname' at nettop.stp:35:4 >> WARNING: Number of errors: 1, skipped probes: 0 >> Apparently using @sum on empty aggregates isn't allowed. I expected >> 0's to be returned. > > As a judgement call, to be consistent with other extractors like @avg, > the @sum etc. of an empty set was deemed to be undefined. FWIW, Python allows you to take the sum of an empty list, even though other equivalent extractors fail: >>> sum([]) 0 >>> max([]) Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: min() or max() arg is an empty sequence I'm just looking for precedence in other languages... >> The only way to avoid the error is use @sum only if @count > >> 0, which makes the printf too complex in my opinion. > > Maybe so. It's worth considering some syntactic sugar to express a > undefined=>0 intent. One possibility is an optional second parameter to the extractor that gives a default value. We could require this to be a constant (usually 0), or maybe even allow a numeric expression. a <<< 1; del a // prepare an empty aggregate printf("%d\n", @sum(a)) // runtime error printf("%d\n", @sum(a, 0)) // prints 0 printf("%d\n", @sum(a, x)) // prints value of x This works for min, max, and avg as well. Or, instead of a default, it might be less confusing to consider the second parameter as an initializer -- so when 'a' isn't empty, @sum(a,x) == x + @sum(a). Then min and max would effectively get an upper and lower bound, respectively. I'm not sure if avg works well this way though. Josh ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2006-10-06 15:30 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <3420082f0610030114o5b44b8ak7797483e02002614@mail.gmail.com> [not found] ` <3420082f0610030114o4c6998en907bccce81d28c59@mail.gmail.com> 2006-10-03 15:49 ` Fwd: Any way to find the network usage by a process? Jose R. Santos 2006-10-04 20:54 ` Mike Mason 2006-10-05 21:22 ` Frank Ch. Eigler 2006-10-05 23:28 ` Mike Mason 2006-10-06 0:00 ` Frank Ch. Eigler 2006-10-06 1:10 ` Mike Mason 2006-10-06 15:30 ` Frank Ch. Eigler 2006-10-06 1:08 Stone, Joshua I
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).