public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
From: Jan Hubicka <hubicka@ucw.cz>
To: andi-gcc at firstfloor dot org <gcc-bugzilla@gcc.gnu.org>
Cc: gcc-bugs@gcc.gnu.org
Subject: Re: [Bug other/50636] New: GC in large LTO builds cause excessive	fragmentation in memory map
Date: Thu, 06 Oct 2011 19:20:00 -0000	[thread overview]
Message-ID: <20111006192024.GA22885@atrey.karlin.mff.cuni.cz> (raw)
In-Reply-To: <bug-50636-4@http.gcc.gnu.org/bugzilla/>

> When doing a very large LTO build I fail with "out of virtual memory"
> 
> Some investigation showed the problem was not actually running out of 
> memory, but gcc excessively fragmenting its memory map. The Linux kernel
> has a default limit of 64k mappings per process and the fragmentation 
> exceeded that. This lead to gc mmap allocations failing and other problems.
> 
> A workaround is to increase /proc/sys/vm/max_map_count
> 
> Looking at /proc/$(pidof lto1)/maps I see there are lots of 1-3 page holes
> between other anonymousmemory. I think that's caused by ggc-pages free_pages()
> function freeing too early and in too small chunks
> (and perhaps LTO garbage collecting too much?)

In gcc-2.95 times ggc-page was probably not written with 8GB of memory use in
mind :) Perhaps ggc-page should simply increase the chunks as memory grows?
(i.e. release to system only when 20% of memory is unused & it exceeds some minimal
value)

Honza


  reply	other threads:[~2011-10-06 19:20 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-10-06 18:21 andi-gcc at firstfloor dot org
2011-10-06 19:20 ` Jan Hubicka [this message]
2011-10-06 19:21 ` [Bug other/50636] " hubicka at ucw dot cz
2011-10-06 21:29 ` jakub at gcc dot gnu.org
2011-10-06 21:32 ` andi-gcc at firstfloor dot org
2011-10-06 21:42 ` jakub at gcc dot gnu.org
2011-10-06 21:47 ` andi-gcc at firstfloor dot org
2011-10-07  5:49 ` andi-gcc at firstfloor dot org
2011-10-07  5:51 ` andi-gcc at firstfloor dot org
2011-10-07  8:25 ` rguenth at gcc dot gnu.org
2011-10-07 10:01 ` jakub at gcc dot gnu.org
2011-10-07 14:45 ` andi-gcc at firstfloor dot org
2011-10-08 16:48 ` andi-gcc at firstfloor dot org
2011-10-08 19:55 ` andi-gcc at firstfloor dot org
2011-10-08 20:14 ` jakub at gcc dot gnu.org
2011-10-08 21:10 ` andi-gcc at firstfloor dot org
2011-10-17 14:44 ` ak at gcc dot gnu.org
2011-10-20  0:11 ` andi-gcc at firstfloor dot org

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20111006192024.GA22885@atrey.karlin.mff.cuni.cz \
    --to=hubicka@ucw.cz \
    --cc=gcc-bugs@gcc.gnu.org \
    --cc=gcc-bugzilla@gcc.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).