[PATCH] gas: fix pathological memory use with many macros

[PATCH] gas: fix pathological memory use with many macros

[Roland McGrath]

In a case that defines ~2800 macros and has no actual instructions at all,
gas uses a ridiculous amount of memory (well over a gigabyte).  I tracked
this down to the use of hash_new for a new hash table of parameter names
for each and every macro.  That gets a hash table with 65537 buckets when
the typical range of number of entries is probably 0-3.  Even with
--reduce-memory-overheads, the memory usage is still pretty excessive
(at 4051 buckets).

Given that the typical number of macro parameters is so small, from a
performance perspective it's probably a loss to be using a hash table at
all.  For memory usage, it might be a significant improvement just to use
a single obstack for each macro_entry and all the allocation it points to.

But this is a very simple change and it takes the memory usage in my
case from pathological to uninteresting, so that is enough for now.
There is no discernible difference in memory usage (as measured by
massif) for table sizes up to a few hundred, because of the obstack
chunk size.  When setting debug_memory so the chunk size is small, there
is a measurable but uninterestingly small difference between 7 and
(prime) sizes up to 31 or so.  But 7 seems like a reasonable upper bound
on the likely number of parameters to a macro.

Ok for trunk and 2.23?


Thanks,
Roland


gas/
2013-01-09  Roland McGrath  <mcgrathr@google.com>

	* hash.c (hash_new_sized): Make it global.
	* hash.h: Declare it.
	* macro.c (define_macro): Use hash_new_sized instead of hash_new,
	pass a small size.

--- a/gas/hash.c
+++ b/gas/hash.c
@@ -1,6 +1,6 @@
 /* hash.c -- gas hash table code
    Copyright 1987, 1990, 1991, 1992, 1993, 1994, 1995, 1996, 1998, 1999,
-   2000, 2001, 2002, 2003, 2005, 2007, 2008, 2009, 2011
+   2000, 2001, 2002, 2003, 2005, 2007, 2008, 2009, 2011, 2013
    Free Software Foundation, Inc.

    This file is part of GAS, the GNU Assembler.
@@ -83,7 +83,7 @@ set_gas_hash_table_size (unsigned long size)

 /* Create a hash table.  This return a control block.  */

-static struct hash_control *
+struct hash_control *
 hash_new_sized (unsigned long size)
 {
   unsigned long alloc;
--- a/gas/hash.h
+++ b/gas/hash.h
@@ -1,5 +1,5 @@
 /* hash.h -- header file for gas hash table routines
-   Copyright 1987, 1992, 1993, 1995, 1999, 2003, 2005, 2007, 2008
+   Copyright 1987, 1992, 1993, 1995, 1999, 2003, 2005, 2007, 2008, 2013
    Free Software Foundation, Inc.

    This file is part of GAS, the GNU Assembler.
@@ -31,6 +31,7 @@ void set_gas_hash_table_size (unsigned long);
 /* Create a hash table.  This return a control block.  */

 extern struct hash_control *hash_new (void);
+extern struct hash_control *hash_new_sized (unsigned long);

 /* Delete a hash table, freeing all allocated memory.  */

--- a/gas/macro.c
+++ b/gas/macro.c
@@ -1,6 +1,6 @@
 /* macro.c - macro support for gas
    Copyright 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
-   2004, 2005, 2006, 2007, 2008, 2011, 2012 Free Software Foundation, Inc.
+   2004, 2005, 2006, 2007, 2008, 2011, 2012, 2013 Free Software
Foundation, Inc.

    Written by Steve and Judy Chamberlain of Cygnus Support,
       sac@cygnus.com
@@ -642,7 +642,7 @@ define_macro (size_t idx, sb *in, sb *label,

   macro->formal_count = 0;
   macro->formals = 0;
-  macro->formal_hash = hash_new ();
+  macro->formal_hash = hash_new_sized (7);

   idx = sb_skip_white (idx, in);
   if (! buffer_and_nest ("MACRO", "ENDM", &macro->sub, get_line))

Re: [PATCH] gas: fix pathological memory use with many macros

[Alan Modra]

On Wed, Jan 09, 2013 at 02:26:11PM -0800, Roland McGrath wrote:
> 	* hash.c (hash_new_sized): Make it global.
> 	* hash.h: Declare it.
> 	* macro.c (define_macro): Use hash_new_sized instead of hash_new,
> 	pass a small size.

OK.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] gas: fix pathological memory use with many macros

[Roland McGrath]

Committed.


Thanks,
Roland