From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 29496 invoked by alias); 23 Oct 2011 10:25:01 -0000 Received: (qmail 29476 invoked by uid 22791); 23 Oct 2011 10:25:01 -0000 X-SWARE-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW X-Spam-Check-By: sourceware.org Received: from mail-gy0-f175.google.com (HELO mail-gy0-f175.google.com) (209.85.160.175) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Sun, 23 Oct 2011 10:24:46 +0000 Received: by gyc15 with SMTP id 15so5663515gyc.20 for ; Sun, 23 Oct 2011 03:24:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.12.69 with SMTP id w5mr2429750obb.31.1319365486114; Sun, 23 Oct 2011 03:24:46 -0700 (PDT) Received: by 10.182.12.202 with HTTP; Sun, 23 Oct 2011 03:24:46 -0700 (PDT) In-Reply-To: References: <1319176370-26071-1-git-send-email-andi@firstfloor.org> <1319176370-26071-3-git-send-email-andi@firstfloor.org> <20111021183058.GC22535@one.firstfloor.org> Date: Sun, 23 Oct 2011 14:33:00 -0000 Message-ID: Subject: Re: [PATCH 2/3] Free large chunks in ggc From: Richard Guenther To: Andi Kleen Cc: gcc-patches@gcc.gnu.org, Andi Kleen Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2011-10/txt/msg02095.txt.bz2 On Sun, Oct 23, 2011 at 12:23 PM, Richard Guenther wrote: > On Fri, Oct 21, 2011 at 8:30 PM, Andi Kleen wrote: >>> > diff --git a/gcc/ggc-page.c b/gcc/ggc-page.c >>> > index ba88e3f..eb0eeef 100644 >>> > --- a/gcc/ggc-page.c >>> > +++ b/gcc/ggc-page.c >>> > @@ -972,6 +972,54 @@ release_pages (void) >>> > =A0 page_entry *p, *start_p; >>> > =A0 char *start; >>> > =A0 size_t len; >>> > + =A0size_t mapped_len; >>> > + =A0page_entry *next, *prev, *newprev; >>> > + =A0size_t free_unit =3D PARAM_VALUE (GGC_FREE_UNIT) * G.pagesize; >>> > + >>> > + =A0/* First free larger continuous areas to the OS. >>> > + =A0 =A0 This allows other allocators to grab these areas if needed. >>> > + =A0 =A0 This is only done on larger chunks to avoid fragmentation. >>> > + =A0 =A0 This does not always work because the free_pages list is on= ly >>> > + =A0 =A0 sorted over a single GC cycle. */ >>> >>> But release_pages is only called from ggc_collect, or what do you >> >> If there was a spike in GC usage and we end up with lots of free >> space in the free list afterward we free it back on the next GC cycle. >> Then if there's a malloc or other allocator later it can grab >> the address space we freed. >> >> That was done to address your earlier concern. >> >> This will only happen on ggc_collect of course. >> >> So one difference from before the madvise patch is that different >> generations of free pages can accumulate in the freelist. Before madvise >> the freelist would never contain more than one generation. >> Normally it's sorted by address due to the way GC works, but there's no >> attempt to keep the sort order over multiple generations. >> >> The "free in batch" heuristic requires sorting, so it will only >> work if all the pages are freed in a single gc cycle. >> >> I considered sorting, but it seemed to be too slow. >> >> I can expand the comment on that. > > Ah, now I see ... but that's of course bad - I expect large regions to be > free only after multiple collections. =A0Can you measure what sorting wou= ld > make for a difference? I wonder if the free list that falls out of a single collection is sorted (considering also ggc_free) - if it is, building a new one at each collecti= on and then merging the two sorted lists should be reasonably fast. >> >>> mean with the above? =A0Would the hitrate using the quire size increase >>> if we change how we allocate from the freelist or is it real fragmentat= ion >>> that causes it? >> >> Not sure really about the hitrate. I haven't measured it. If hitrate >> was a concern the free list should be probably split into an array. >> I'm sure there are lots of other tunings that could be done on the GC, >> but probably not by me for now :) > > Heh. =A0Yeah, I suppose the freelist could be changed into a list of > allocation groups with free pages and a bitmap. > > Richard. > >>> >>> I'm a bit hesitant to approve the new param, I'd be ok if we just hard-= code >>> quire-size / 2. >> >> Ok replacing it with a hardcoded value. >> >> -Andi >> >