From: Richard Guenther <richard.guenther@gmail.com>
To: Jakub Jelinek <jakub@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>,
gcc-patches@gcc.gnu.org, Andi Kleen <ak@linux.intel.com>
Subject: Re: [PATCH 1/5] Use MADV_DONTNEED for freeing in garbage collector
Date: Mon, 10 Oct 2011 11:21:00 -0000 [thread overview]
Message-ID: <CAFiYyc3SxbiU9mrgy9KrTA8YGcY9AqMR9GPi+2VEZzf-B+HF=g@mail.gmail.com> (raw)
In-Reply-To: <20111010104530.GX19412@tyan-ft48-01.lab.bos.redhat.com>
On Mon, Oct 10, 2011 at 12:45 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Mon, Oct 10, 2011 at 12:25:15PM +0200, Richard Guenther wrote:
>> On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
>> > From: Andi Kleen <ak@linux.intel.com>
>> >
>> > Use the Linux MADV_DONTNEED call to unmap free pages in the garbage
>> > collector.Then keep the unmapped pages in the free list. This avoid
>> > excessive memory fragmentation on large LTO bulds, which can lead
>> > to gcc bumping into the Linux vm_max_map limit per process.
>> >
>> > Based on a idea from Jakub.
>>
>> Shouldn't we prefer still "mapped" pages when allocating? Thus, keep
>> the freepages list "sorted"?
>
> I don't see why. MADV_DONTNEED isn't perfect, what it does (on Linux)
> is that it zaps the whole page range, which essentially brings it into
> the exact same state as immediately after mmap. Any touch of the
> pages will result in a zeroed page being inserted into the page tables.
Which means we save the zeroing when allocating non-MADV_DONTNEEDed
pages first. And will be eventually able to unmap zapped pages.
> 4 years ago there was a MADV_FREE proposal which behaved much better
> (page was removed from page tables only when the kernel actually needed
> them for something else, if the page wasn't needed and has been accessed
> again by the application, it would still contain the old content (which
> the app couldn't rely on, it could as well be cleared), but it would be much
> cheaper in that case. With MADV_FREE it would be actually preferrable
> to pick the MADV_FREEd pages over picking up freshly munmapped but not yet
> touched pages.
>
>> With the new params to call release_pages less, how does this
>> interact with using MADV_DONTNEED? The only reason to delay
>> MADV_DONTNEED is to avoid splitting huge-pages? Which would
>
> Not just that. MADV_DONTNEED needs to flush the dirty pages from the page
> tables and when they are touched again, they need to be cleared (or
> pre-cleared pages inserted). So, while MADV_DONTNEED is less expensive than
> munmap + mmap, it is still not free.
But it's free at madvise time. munmap is "synchronous" at least (well,
when file-backed).
>> > 2011-10-08 Andi Kleen <ak@linux.intel.com>
>
> Two space in between name and <.
>> >
>> > PR other/50636
>> > * config.in, configure: Regenerate.
>
> Please write each file on a separate line, and better below
> * configure.ac line because of which they have been regenerated.
>
>> >
>> > + /* Unmapped page? */
>> > + bool unmapped;
>> > +
>
> Not sure if unmapped is the best name of the flag here, because
> it hasn't been unmapped, it just has been madvised. Under unmap
> most people would imagine munmap I'd say.
>
> Jakub
>
next prev parent reply other threads:[~2011-10-10 11:11 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-10-09 19:56 Improve ggc-page fragmentation Andi Kleen
2011-10-09 19:56 ` [PATCH 2/5] Increase the GGC quite size to 2MB Andi Kleen
2011-10-10 10:21 ` Richard Guenther
2011-10-10 13:58 ` Jan Hubicka
2011-10-09 19:56 ` [PATCH 3/5] On a Linux kernel ask explicitely for a huge page in ggc Andi Kleen
2011-10-10 10:17 ` Richard Guenther
2011-10-10 10:34 ` Jakub Jelinek
2011-10-10 14:04 ` Andi Kleen
2011-10-10 13:54 ` Andi Kleen
2011-10-09 19:56 ` [PATCH 5/5] Add error checking to lto_section_read Andi Kleen
2011-10-10 10:25 ` Richard Guenther
2011-10-09 20:09 ` [PATCH 4/5] Add a freeing threshold for the garbage collector Andi Kleen
2011-10-10 10:29 ` Richard Guenther
2011-10-10 14:06 ` Andi Kleen
2011-10-10 14:59 ` Richard Guenther
2011-10-09 22:18 ` [PATCH 1/5] Use MADV_DONTNEED for freeing in " Andi Kleen
2011-10-10 10:33 ` Richard Guenther
2011-10-10 10:48 ` Jakub Jelinek
2011-10-10 11:21 ` Richard Guenther [this message]
2011-10-10 11:58 ` Jakub Jelinek
2011-10-10 14:23 ` Andi Kleen
2011-10-10 15:05 ` Richard Guenther
2011-10-10 18:09 ` Andi Kleen
2011-10-16 9:04 ` Andi Kleen
2011-10-16 11:29 ` Richard Guenther
2011-10-16 19:38 ` Andi Kleen
2011-10-17 8:41 ` Richard Guenther
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFiYyc3SxbiU9mrgy9KrTA8YGcY9AqMR9GPi+2VEZzf-B+HF=g@mail.gmail.com' \
--to=richard.guenther@gmail.com \
--cc=ak@linux.intel.com \
--cc=andi@firstfloor.org \
--cc=gcc-patches@gcc.gnu.org \
--cc=jakub@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).