From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 6274 invoked by alias); 9 Jan 2013 22:26:40 -0000 Received: (qmail 6263 invoked by uid 22791); 9 Jan 2013 22:26:39 -0000 X-SWARE-Spam-Status: No, hits=-4.9 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,KHOP_RCVD_TRUST,RCVD_IN_DNSWL_LOW,RCVD_IN_HOSTKARMA_YE,RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from mail-qc0-f179.google.com (HELO mail-qc0-f179.google.com) (209.85.216.179) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 09 Jan 2013 22:26:32 +0000 Received: by mail-qc0-f179.google.com with SMTP id b14so1908823qcs.38 for ; Wed, 09 Jan 2013 14:26:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type :x-gm-message-state; bh=EsZ+Gd0xVEqlZerbM/TEmDhimGzw+xgkR7AUKKyVAWM=; b=J3gVQkyo2FeNKVQUkcK61KN6KAWgq0XjHzoaHhIPPz8hdR3dNylj/MocOhPuxZgg3K CUrRZvdhY0KKqfxpl4UmAPjM4qJWBb4MUZ2yc1C/bWUt2tLDJQdxLC5lKOrBOqt+LAz+ akpPI39KlVxou2c/xcltcQ5PxdUP8zGxq8MCAm3UxasAi7jvnMhm3LYCjKJmFQyvEAYA fI+C54Aitpuj7CgbYciSwETaOyeQhvIiszZXBtzvsfeCMjaNqUXVbrnOL619z5nwWNUo MVuiRXuaQsoKzuJGAGXibp0mOWzrnEFuv/ShdwzMwvKzuwuRzI81Qs1c1gHmjGZeWyd7 YNeA== Received: by 10.229.106.131 with SMTP id x3mr12616324qco.50.1357770391917; Wed, 09 Jan 2013 14:26:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.132.1 with HTTP; Wed, 9 Jan 2013 14:26:11 -0800 (PST) From: Roland McGrath Date: Wed, 09 Jan 2013 22:26:00 -0000 Message-ID: Subject: [PATCH] gas: fix pathological memory use with many macros To: binutils@sourceware.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQnJwCOBGWgpcojibmiJ2zHHF2vfUL1quk++Hb9ITH7syf0nZC+sm7u31HJ4+Sn2AKrgbOUSv9+u7p0C34KGjgtTnjwZwm+lxY2k4LsQHOVR9j0bVHbQKYF8uVTZqvDFtKHyxm13bzbyV3ZKiNhMTZ12ITdTCMaQMv8jv6pwpCvK22umsSsYcgXjvxZZ1N/0BGTXnW9+ Mailing-List: contact binutils-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: binutils-owner@sourceware.org X-SW-Source: 2013-01/txt/msg00125.txt.bz2 In a case that defines ~2800 macros and has no actual instructions at all, gas uses a ridiculous amount of memory (well over a gigabyte). I tracked this down to the use of hash_new for a new hash table of parameter names for each and every macro. That gets a hash table with 65537 buckets when the typical range of number of entries is probably 0-3. Even with --reduce-memory-overheads, the memory usage is still pretty excessive (at 4051 buckets). Given that the typical number of macro parameters is so small, from a performance perspective it's probably a loss to be using a hash table at all. For memory usage, it might be a significant improvement just to use a single obstack for each macro_entry and all the allocation it points to. But this is a very simple change and it takes the memory usage in my case from pathological to uninteresting, so that is enough for now. There is no discernible difference in memory usage (as measured by massif) for table sizes up to a few hundred, because of the obstack chunk size. When setting debug_memory so the chunk size is small, there is a measurable but uninterestingly small difference between 7 and (prime) sizes up to 31 or so. But 7 seems like a reasonable upper bound on the likely number of parameters to a macro. Ok for trunk and 2.23? Thanks, Roland gas/ 2013-01-09 Roland McGrath * hash.c (hash_new_sized): Make it global. * hash.h: Declare it. * macro.c (define_macro): Use hash_new_sized instead of hash_new, pass a small size. --- a/gas/hash.c +++ b/gas/hash.c @@ -1,6 +1,6 @@ /* hash.c -- gas hash table code Copyright 1987, 1990, 1991, 1992, 1993, 1994, 1995, 1996, 1998, 1999, - 2000, 2001, 2002, 2003, 2005, 2007, 2008, 2009, 2011 + 2000, 2001, 2002, 2003, 2005, 2007, 2008, 2009, 2011, 2013 Free Software Foundation, Inc. This file is part of GAS, the GNU Assembler. @@ -83,7 +83,7 @@ set_gas_hash_table_size (unsigned long size) /* Create a hash table. This return a control block. */ -static struct hash_control * +struct hash_control * hash_new_sized (unsigned long size) { unsigned long alloc; --- a/gas/hash.h +++ b/gas/hash.h @@ -1,5 +1,5 @@ /* hash.h -- header file for gas hash table routines - Copyright 1987, 1992, 1993, 1995, 1999, 2003, 2005, 2007, 2008 + Copyright 1987, 1992, 1993, 1995, 1999, 2003, 2005, 2007, 2008, 2013 Free Software Foundation, Inc. This file is part of GAS, the GNU Assembler. @@ -31,6 +31,7 @@ void set_gas_hash_table_size (unsigned long); /* Create a hash table. This return a control block. */ extern struct hash_control *hash_new (void); +extern struct hash_control *hash_new_sized (unsigned long); /* Delete a hash table, freeing all allocated memory. */ --- a/gas/macro.c +++ b/gas/macro.c @@ -1,6 +1,6 @@ /* macro.c - macro support for gas Copyright 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, - 2004, 2005, 2006, 2007, 2008, 2011, 2012 Free Software Foundation, Inc. + 2004, 2005, 2006, 2007, 2008, 2011, 2012, 2013 Free Software Foundation, Inc. Written by Steve and Judy Chamberlain of Cygnus Support, sac@cygnus.com @@ -642,7 +642,7 @@ define_macro (size_t idx, sb *in, sb *label, macro->formal_count = 0; macro->formals = 0; - macro->formal_hash = hash_new (); + macro->formal_hash = hash_new_sized (7); idx = sb_skip_white (idx, in); if (! buffer_and_nest ("MACRO", "ENDM", ¯o->sub, get_line))