From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18744 invoked by alias); 6 Apr 2010 21:00:05 -0000 Received: (qmail 17612 invoked by uid 48); 6 Apr 2010 20:59:41 -0000 Date: Tue, 06 Apr 2010 21:00:00 -0000 Message-ID: <20100406205941.17609.qmail@sourceware.org> From: "tglek at mozilla dot com" To: glibc-bugs@sources.redhat.com In-Reply-To: <20100325143324.11431.fche@redhat.com> References: <20100325143324.11431.fche@redhat.com> Reply-To: sourceware-bugzilla@sourceware.org Subject: [Bug libc/11431] consider madvise for ld.so X-Bugzilla-Reason: CC Mailing-List: contact glibc-bugs-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: glibc-bugs-owner@sourceware.org X-SW-Source: 2010-04/txt/msg00053.txt.bz2 ------- Additional Comments From tglek at mozilla dot com 2010-04-06 20:59 ------- (In reply to comment #2) > Can you suggest some statistics we can gather in order to inform > heuristics within ld.so to trigger a madvise(MADV_WILLNEED) > (which itself is a heuristic for the kernel)? > I don't have an effective general heuristic. The problem is worst for programs that use a large chunk of large libraries. Without modifying the compile-time linker, the best heuristic I can come up with is to only WILLNEED executable segments > 4 mb and data segments > 1mb. If we could add a compile-time ld flag to flag dependent libraries with WILLNEED, that would be ideal for Firefox. That would take the guessing out of ld.so and push it onto application devs who are likely know exactly how the libraries are meant to be used. Additionally libc could use an api to reset the madvise flags to completely erase any concerns about keeping too many pages cached. One can already change the madvise hints by parsing /proc//maps, but there is no way set madvise flags early on. On the other hand, everyday developers should not be expected to understand pagefaulting behavior of their apps, so the benefit would be limited. Another sure-fire solution is to do something like the current prelink cronjob. Except in this case the program would iterate /proc//maps figure out which libraries are paging in multi-megabyte chunks and record that similar to the prelink cache. My concern is that I have yet to see any concrete evidence that madvise hints actually hurt in low memory conditions. I started a thread on LKML asking about madvise behavior in low memory conditions. I am still waiting for an indepth reply, but it seems that the kernel throttles caching behavior under memory pressure. -- http://sourceware.org/bugzilla/show_bug.cgi?id=11431 ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.