From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 8459 invoked by alias); 12 May 2011 15:29:40 -0000 Received: (qmail 8433 invoked by uid 22791); 12 May 2011 15:29:39 -0000 X-SWARE-Spam-Status: No, hits=-1.8 required=5.0 tests=AWL,BAYES_00,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from ksp.mff.cuni.cz (HELO atrey.karlin.mff.cuni.cz) (195.113.26.206) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 12 May 2011 15:29:05 +0000 Received: by atrey.karlin.mff.cuni.cz (Postfix, from userid 4018) id 7429F130B4D; Thu, 12 May 2011 17:29:04 +0200 (CEST) Date: Thu, 12 May 2011 15:35:00 -0000 From: Jan Hubicka To: andi-gcc at firstfloor dot org Cc: gcc-bugs@gcc.gnu.org Subject: Re: [Bug lto/48978] New: excessive hash table allocation for large lto build Message-ID: <20110512152904.GA31449@atrey.karlin.mff.cuni.cz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) X-IsSubscribed: yes Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-bugs-owner@gcc.gnu.org X-SW-Source: 2011-05/txt/msg01033.txt.bz2 > I tried a large LTO build with gcc version 4.7.0 20110511 (experimental) (GCC) > on a 18GB machine. It ended with > > lto1: out of memory allocating 8589934312 bytes after a total of 6827421696 > bytes > > Since a 7+GB single malloc seems somewhat excessive I put a break point > on xmalloc_failed. It looks like the hash tables are growing > too aggressively? I think the problem was introduced by Richi's last commit: 2011-05-12 Richard Guenther * gimple.c (gtc_visit): Compare TREE_ADDRESSABLE, handle NULLPTR_TYPE similar to VOID_TYPE. Defer type-leader lookup until after simple checks. (gimple_types_compatible_p): Likewise. (iterative_hash_gimple_type): Always hash pointer targets and function return and argument types. (iterative_hash_canonical_type): Do not hash TYPE_QUALS, hash TYPE_ALIGN. Do not hash TYPE_MIN/MAX_VALUE. (gimple_canonical_types_compatible_p): Compare TREE_ADDRESSABLE, handle NULLPTR_TYPE similar to VOID_TYPE. Handle non-aggregates completely in the simple compare section. (gimple_register_canonical_type): Query the cache again after registering. So reverting this patch might get you past the problem. Ysterday I was able to build your testcase with 3GB of ram, today it got out of memory, too. The hashtable is not expanding too irrationally, but it saves O(n^2) information. I would like to see it die for sure ;) Honza