From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 11334 invoked by alias); 12 May 2011 16:03:56 -0000 Received: (qmail 11325 invoked by uid 22791); 12 May 2011 16:03:55 -0000 X-SWARE-Spam-Status: No, hits=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 X-Spam-Check-By: sourceware.org Received: from localhost (HELO gcc.gnu.org) (127.0.0.1) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 12 May 2011 16:03:02 +0000 From: "hubicka at ucw dot cz" To: gcc-bugs@gcc.gnu.org Subject: [Bug lto/48978] excessive hash table allocation for large lto build X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: gcc X-Bugzilla-Component: lto X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: hubicka at ucw dot cz X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Priority: P3 X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/ Auto-Submitted: auto-generated Content-Type: text/plain; charset="UTF-8" MIME-Version: 1.0 Date: Thu, 12 May 2011 16:04:00 -0000 Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-bugs-owner@gcc.gnu.org X-SW-Source: 2011-05/txt/msg01035.txt.bz2 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48978 --- Comment #1 from Jan Hubicka 2011-05-12 15:29:42 UTC --- > I tried a large LTO build with gcc version 4.7.0 20110511 (experimental) (GCC) > on a 18GB machine. It ended with > > lto1: out of memory allocating 8589934312 bytes after a total of 6827421696 > bytes > > Since a 7+GB single malloc seems somewhat excessive I put a break point > on xmalloc_failed. It looks like the hash tables are growing > too aggressively? I think the problem was introduced by Richi's last commit: 2011-05-12 Richard Guenther * gimple.c (gtc_visit): Compare TREE_ADDRESSABLE, handle NULLPTR_TYPE similar to VOID_TYPE. Defer type-leader lookup until after simple checks. (gimple_types_compatible_p): Likewise. (iterative_hash_gimple_type): Always hash pointer targets and function return and argument types. (iterative_hash_canonical_type): Do not hash TYPE_QUALS, hash TYPE_ALIGN. Do not hash TYPE_MIN/MAX_VALUE. (gimple_canonical_types_compatible_p): Compare TREE_ADDRESSABLE, handle NULLPTR_TYPE similar to VOID_TYPE. Handle non-aggregates completely in the simple compare section. (gimple_register_canonical_type): Query the cache again after registering. So reverting this patch might get you past the problem. Ysterday I was able to build your testcase with 3GB of ram, today it got out of memory, too. The hashtable is not expanding too irrationally, but it saves O(n^2) information. I would like to see it die for sure ;) Honza