From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 23733 invoked by alias); 21 Oct 2011 09:55:46 -0000 Received: (qmail 23722 invoked by uid 22791); 21 Oct 2011 09:55:45 -0000 X-SWARE-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW X-Spam-Check-By: sourceware.org Received: from mail-gx0-f175.google.com (HELO mail-gx0-f175.google.com) (209.85.161.175) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Fri, 21 Oct 2011 09:55:30 +0000 Received: by ggnj1 with SMTP id j1so2475466ggn.20 for ; Fri, 21 Oct 2011 02:55:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.13.6 with SMTP id d6mr965107obc.11.1319190929352; Fri, 21 Oct 2011 02:55:29 -0700 (PDT) Received: by 10.182.12.202 with HTTP; Fri, 21 Oct 2011 02:55:29 -0700 (PDT) In-Reply-To: <1319176370-26071-3-git-send-email-andi@firstfloor.org> References: <1319176370-26071-1-git-send-email-andi@firstfloor.org> <1319176370-26071-3-git-send-email-andi@firstfloor.org> Date: Fri, 21 Oct 2011 10:39:00 -0000 Message-ID: Subject: Re: [PATCH 2/3] Free large chunks in ggc From: Richard Guenther To: Andi Kleen Cc: gcc-patches@gcc.gnu.org, Andi Kleen Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2011-10/txt/msg01947.txt.bz2 On Fri, Oct 21, 2011 at 7:52 AM, Andi Kleen wrote: > From: Andi Kleen > > This implements the freeing back of large chunks in the ggc madvise path > Richard Guenther asked for. =A0This way on systems with limited > address space malloc() and other allocators still have > a chance to get back at some of the memory ggc freed. The > fragmented pages are still just given back, but the address space > stays allocated. > > I tried freeing only aligned 2MB areas to optimize for 2MB huge > pages, but the hit rate was quite low, so I switched to 1MB+ > unaligned areas. The target size is a param now. > > Passed bootstrap and testing on x86_64-linux > > gcc/: > 2011-10-18 =A0Andi Kleen =A0 > > =A0 =A0 =A0 =A0* ggc-page (release_pages): First free large continuous > =A0 =A0 =A0 =A0chunks in the madvise path. > =A0 =A0 =A0 =A0* params.def (GGC_FREE_UNIT): Add. > =A0 =A0 =A0 =A0* doc/invoke.texi (ggc-free-unit): Add. > --- > =A0gcc/doc/invoke.texi | =A0 =A05 +++++ > =A0gcc/ggc-page.c =A0 =A0 =A0| =A0 48 +++++++++++++++++++++++++++++++++++= +++++++++++++ > =A0gcc/params.def =A0 =A0 =A0| =A0 =A05 +++++ > =A03 files changed, 58 insertions(+), 0 deletions(-) > > diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi > index 4f55dbc..e622552 100644 > --- a/gcc/doc/invoke.texi > +++ b/gcc/doc/invoke.texi > @@ -8858,6 +8858,11 @@ very large effectively disables garbage collection= . =A0Setting this > =A0parameter and @option{ggc-min-expand} to zero causes a full collection > =A0to occur at every opportunity. > > +@item =A0ggc-free-unit > + > +Continuous areas in OS pages to free back to OS immediately. Default is = 256 > +pages, which is 1MB with 4K pages. > + > =A0@item max-reload-search-insns > =A0The maximum number of instruction reload should look backward for equi= valent > =A0register. =A0Increasing values mean more aggressive optimization, maki= ng the > diff --git a/gcc/ggc-page.c b/gcc/ggc-page.c > index ba88e3f..eb0eeef 100644 > --- a/gcc/ggc-page.c > +++ b/gcc/ggc-page.c > @@ -972,6 +972,54 @@ release_pages (void) > =A0 page_entry *p, *start_p; > =A0 char *start; > =A0 size_t len; > + =A0size_t mapped_len; > + =A0page_entry *next, *prev, *newprev; > + =A0size_t free_unit =3D PARAM_VALUE (GGC_FREE_UNIT) * G.pagesize; > + > + =A0/* First free larger continuous areas to the OS. > + =A0 =A0 This allows other allocators to grab these areas if needed. > + =A0 =A0 This is only done on larger chunks to avoid fragmentation. > + =A0 =A0 This does not always work because the free_pages list is only > + =A0 =A0 sorted over a single GC cycle. */ But release_pages is only called from ggc_collect, or what do you mean with the above? Would the hitrate using the quire size increase if we change how we allocate from the freelist or is it real fragmentation that causes it? I'm a bit hesitant to approve the new param, I'd be ok if we just hard-code quire-size / 2. Richard. > + > + =A0p =3D G.free_pages; > + =A0prev =3D NULL; > + =A0while (p) > + =A0 =A0{ > + =A0 =A0 =A0start =3D p->page; > + =A0 =A0 =A0start_p =3D p; > + =A0 =A0 =A0len =3D 0; > + =A0 =A0 =A0mapped_len =3D 0; > + =A0 =A0 =A0newprev =3D prev; > + =A0 =A0 =A0while (p && p->page =3D=3D start + len) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0len +=3D p->bytes; > + =A0 =A0 =A0 =A0 if (!p->discarded) > + =A0 =A0 =A0 =A0 =A0 =A0 mapped_len +=3D p->bytes; > + =A0 =A0 =A0 =A0 newprev =3D p; > + =A0 =A0 =A0 =A0 =A0p =3D p->next; > + =A0 =A0 =A0 =A0} > + =A0 =A0 =A0if (len >=3D free_unit) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0while (start_p !=3D p) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0next =3D start_p->next; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0free (start_p); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0start_p =3D next; > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0munmap (start, len); > + =A0 =A0 =A0 =A0 if (prev) > + =A0 =A0 =A0 =A0 =A0 prev->next =3D p; > + =A0 =A0 =A0 =A0 =A0else > + =A0 =A0 =A0 =A0 =A0 =A0G.free_pages =3D p; > + =A0 =A0 =A0 =A0 =A0G.bytes_mapped -=3D mapped_len; > + =A0 =A0 =A0 =A0 continue; > + =A0 =A0 =A0 =A0} > + =A0 =A0 =A0prev =3D newprev; > + =A0 } > + > + =A0/* Now give back the fragmented pages to the OS, but keep the address > + =A0 =A0 space to reuse it next time. */ > > =A0 for (p =3D G.free_pages; p; ) > =A0 =A0 { > diff --git a/gcc/params.def b/gcc/params.def > index 5e49c48..edbf0de 100644 > --- a/gcc/params.def > +++ b/gcc/params.def > @@ -561,6 +561,11 @@ DEFPARAM(GGC_MIN_HEAPSIZE, > =A0#undef GGC_MIN_EXPAND_DEFAULT > =A0#undef GGC_MIN_HEAPSIZE_DEFAULT > > +DEFPARAM(GGC_FREE_UNIT, > + =A0 =A0 =A0 =A0"ggc-free-unit", > + =A0 =A0 =A0 =A0"Continuous areas in OS pages to free back immediately", > + =A0 =A0 =A0 =A0256, 0, 0) > + > =A0DEFPARAM(PARAM_MAX_RELOAD_SEARCH_INSNS, > =A0 =A0 =A0 =A0 "max-reload-search-insns", > =A0 =A0 =A0 =A0 "The maximum number of instructions to search backward wh= en looking for equivalent reload", > -- > 1.7.5.4 > >