From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-x112b.google.com (mail-yw1-x112b.google.com [IPv6:2607:f8b0:4864:20::112b]) by sourceware.org (Postfix) with ESMTPS id D5D883858D38 for ; Tue, 13 Jun 2023 16:39:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D5D883858D38 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=dneg.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=dneg.com Received: by mail-yw1-x112b.google.com with SMTP id 00721157ae682-561b7729a12so98714137b3.1 for ; Tue, 13 Jun 2023 09:39:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dneg.com; s=google; t=1686674385; x=1689266385; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=RkzgWCLzeiQVSSaQEpV032hiKAaws40WDtJ5JOLkD5w=; b=f5oumE4waDx6YrxY94oaDKwAawXIz6cneCEgogCrCSlH8X8m7E5ac1Tn3BmFAiMc0D 1BVtPijsoo7bxGsUMEwCI9Y9vIvp0QKhTcuYtn7z8hNcb/zA7iitbxpPX7B/VFJAMDvj 2BEQA63yWFNw/eSmtras0yhDOhftgCyC6LfHNRGUexpWeULlI7HdZGJmDbYKUBSZlFh5 ymIG8Ehis52igXz2dQ3N5tNdgLObxavwgCj0I7enOXTwhPufCWC8j6eRdW6sYUwaNxck bhSthtLo5c8XRKqIm7cM6ZXVcTuSh4Qfq3WJu0Cy7wkb8ODU8Xl9Wtgb4elVUuBZv7yG XR5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686674385; x=1689266385; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RkzgWCLzeiQVSSaQEpV032hiKAaws40WDtJ5JOLkD5w=; b=DWkraamoqmImEHzq6rdrFIkn9MYf9w8BdYUsJv9CpLywC71/ndm5Q0EqnNBwywpbrS YeVVjz7iYJZF1tqZNAt+3JFkzdYjhl+ucs8nNbZz/QEf2E9hEWN3N50B0MpsYurwncUK qNyIJMrZ7rRFzD8KKh/zz3FEnpFlH87dgkoGQyXC3cx9bb3OmWQzg+I2xpRJ6phUq0TX NGd4CphGHVMECrdPMJRNmSnloJy8ESoEVWY+Psu9AVNG0zSiXCQHr7bb5rGMZeQCgXqt cZ9XpzwQTiOowOKKVW0l7eDXhiSQmgIdBePPAG+cvNi3kPhB0SnhAhUIsFz7jHqotdUr 2VSg== X-Gm-Message-State: AC+VfDxlg/x8NmUPojKtArzzXFmkIXXizLRcoupIVx6nFdaZezNU6wxd HFzWG+SLYn1ithSN7XcA0lOuFtvlw+/iq+VD3nwrX9YVmHY3FE9XCqs= X-Google-Smtp-Source: ACHHUZ4sTyog3MEoFdlbcosbdZuD/nMfuC2E4iof7OcADBFb4tNbWq9L9y+bUqyabgLXeVPS84U9P3YoB4LF47dMxyM= X-Received: by 2002:a25:90e:0:b0:ba9:b304:107b with SMTP id 14-20020a25090e000000b00ba9b304107bmr2119423ybj.27.1686674384969; Tue, 13 Jun 2023 09:39:44 -0700 (PDT) MIME-Version: 1.0 References: <4893d129-6569-4318-cf3b-7821bef98441@redhat.com> In-Reply-To: <4893d129-6569-4318-cf3b-7821bef98441@redhat.com> From: Daire Byrne Date: Tue, 13 Jun 2023 17:39:08 +0100 Message-ID: Subject: Re: vfs.add_to_page_cache not working anymore? To: William Cohen Cc: "systemtap@sourceware.org" Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-0.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, 13 Jun 2023 at 16:22, William Cohen wrote:> > Switching to systemtap-4.9 is probably not going to change the results > in this case as there are no changes in tapset/linux/vfs.stp between > 4.8 and 4.9. Good to know, I can skip trying to compile that then... > Unfortunately, the kernels changes over time and some functions probed > by the tapset change over time or the way they are used by other parts > of the kernel changes. The vfs.add_to_page cache in the vfs.stp has > three possible functions it probes: add_to_page_cache_locked, > add_to_page_cache_lru, and add_to_page_cache. The first two functions > were added due to kernel commit f00654007fe1c15. Did some git commit > archeology and only add_to_page_cache_lru is in the kernel due to > kernel git commit 2bb876b58d593d7f2522ec0f41f20a74fde76822. > > The following URL show where add_to_page_cache_lru is used in 6.2.16 > kernels nfs and can provide some method of seeing how the nfs related > functions get called: > > https://elixir.bootlin.com/linux/v6.2.16/A/ident/add_to_page_cache_lru Thanks for the feedback and pointers, that helps me understand where the changes came from at least. It was still working on my last production kernel - v5.16. So if I recall, I used vfs.add_to_page cache because at the time it was the only (or easiest) way to work out total reads for mmap files from an NFS filesystem. I also would have thought it should work for any filesystem not just NFS - but I don't get any hits at all for an entire busy system. > As far as specifically what has changed to cause vfs.add_to_page_cache > not to trigger for NFS operations I am not sure. For the 6.2 kernel > it might be good to get a backtrace of the triggering of it and then > use that information to see what has changed in the functions on the > backtrace. > > stap -ldd -e 'probe vfs.add_to_page_cache { print_backtrace(); printf("Works.\n"); exit() }' I just get the error "Cannot specify a script with -l/-L/--dump-* switches" using systemtap v4.8. Thanks for the response. It sounds like I need to find a different way to work out total NFS reads for each filename path in modern kernels. Daire BTW, this is the code I had for tracking per process and file path read IO: probe nfs.fop.open { pid = pid() filename = sprintf("%s", d_path(&$filp->f_path)) if (filename =~ "/hosts/.*/user_data") { files[pid, ino] = filename if ( !([pid, ino] in procinfo)) procinfo[pid, ino] = sprintf("%s", proc()) } } probe vfs.add_to_page_cache { pid = pid() if ([pid, ino] in files ) { readpage[pid, ino] += 4096 files_store[pid, ino] = sprintf("%s", files[pid, ino]) } }