From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 67888 invoked by alias); 11 Dec 2015 05:11:40 -0000 Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org Received: (qmail 67870 invoked by uid 89); 11 Dec 2015 05:11:39 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=0.9 required=5.0 tests=AWL,BAYES_40,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-qg0-f51.google.com Received: from mail-qg0-f51.google.com (HELO mail-qg0-f51.google.com) (209.85.192.51) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Fri, 11 Dec 2015 05:11:37 +0000 Received: by qgea14 with SMTP id a14so184901053qge.0 for ; Thu, 10 Dec 2015 21:11:35 -0800 (PST) X-Received: by 10.55.77.17 with SMTP id a17mr20353547qkb.40.1449810695192; Thu, 10 Dec 2015 21:11:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.55.39.204 with HTTP; Thu, 10 Dec 2015 21:11:15 -0800 (PST) In-Reply-To: <5669B7DF.80807@redhat.com> References: <5669B7DF.80807@redhat.com> From: Shane Wayne Date: Fri, 11 Dec 2015 05:11:00 -0000 Message-ID: Subject: Re: Questions on network and block device related IRQ handling To: William Cohen Cc: systemtap@sourceware.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2015-q4/txt/msg00266.txt.bz2 Thanks William, your interpretation really helps me a lot, after viewing all the refs you mentioned apart from this I also have done some search on NAPI, this term makes me confused: 1, in the ref:https://web.archive.org/web/20111030030517/http://www.linuxfo= undation.org/collaborate/workgroups/networking/networkoverview it says that: "This interrupt handler will be called when a frame is receiv= ed" In this quote, does the term "interrupt handler" means the irq_handler.entr= y? 2, If that is true, does that mean each frame receive would trigger an individual irq_handler(It seems this is non-NAPI condition)? 3, Besides NAPI, I also found Rnx and Tnx mode of network card, does that mean the same thing? 4, in the ref:https://web.archive.org/web/20111030030517/http://www.linuxf= oundation.org/collaborate/workgroups/networking/networkoverview it also says that:"The same interrupt handler will be called when transmission of a frame is finished and under other conditions. (depends on the NIC; sometimes, the interrupt handler will be called when there is some error)." Does that mean netdev.hard_transmit would trigger the same irq_handler used in netdev.receive? But I can't profile any irq even soft irq, Is that a "NIC depended" problem? I am using virtio network device. 2015-12-11 1:35 GMT+08:00 William Cohen : > On 12/09/2015 10:13 PM, Shane Wayne wrote: >> Hi all: >> I've recently doing some research on network and block device related >> IRQ handling, so I use some database and network cache application as >> the testbed. >> I probed the following probes below: >> netdev receive >> netdev transmit >> netdev hard_transmit >> ioblock request >> ioblock end >> >> irq_handler.entry >> irq_handler,exit >> softirq.entry >> softirq.exit >> >> After the profiling, I notice that, when netdev receive is triggered. >> It always appears within a softirq, and the softirq has a irq_handler >> in front of it. It mean the sequence is like: >> >> irq_handler.entry >> irq_handler.exit >> softirq.entry >> one or several netdev receive >> softirq.exit >> >> And I found that, the irq could happened in any process, even in my >> application's context, but in most condition, it was found in swapper >> process. >> >> And when it comes to netdev transmit, it seems that this action do not >> relate with irq? The sequence just happens in the application context. >> >> netdev transmit begin >> netdev hard_transmit begin >> netdev hard_transmit end >> netdev transmit end >> >> And it doesn't use irq? >> >> So here is my question: >> 1, it seems that irq_handler from network device just be responsible >> to notify the package arrival, and the actual handling was done by the >> following soft irq? > > Yes, that is correct. The irq handlers try to minimize the time they run= . The irq task mainly schedule some task to do the majority of the work in= the softirq. > > http://www.makelinux.net/ldd3/chp-10-sect-4 >> >> 2, the irq handling seems could happen in any process, and do not stop >> current context, but when the server is not heavily loaded, the irq >> prefer to happen in swapper process? > > irqs can happen any time they are not masked. There are sections of kern= el code that disable them to make sure that certain operation are atomic. > >> >> 3, Or just because the swapper take more cpu time when the server was >> not heavily loaded? >> >> 4, does the netdev send or ioblock request really do not need irq's assi= stant? > > The netdev and ioblock works needs to be scheduled. The irq are indicati= ng that the device needs feeding/attentions. The alternative would be to d= o polling which would be probably be less efficient. However, the NAPI net= ork code does going in to a polling mode to reduce the number of interrupts > (https://en.wikipedia.org/wiki/New_API). > >> >> 5, if this is true, how could we determine when the netdev transmit >> finish it's job? is the time of netdev hard_transmit end correct? > > Actually it can be hard to determine precisely when the transmit has fini= shed sending the packet. The packet is given to the device, but it could t= ake some time for the device to complete sending it on the wire. One could= look and when the packet buffer is freed for an upper limit on the time it= took to send. > > > -Will --=20 ------------------------------------------------------------ ZHANG Xiao The Institute of Computer Software Engineering and Theory in Xi'an Jiaotong University Master-and-Doctoral Program Student ACM Student Member CCF Student Member IEEE Computer Society Sister Society Associate Member Xi'an Jiaotong University Bachelor of Engineering in Computer Science and Technology The Chinese University of Hong Kong International Asia Study Program in CS/CE/EE/IE