purify experiments

purify experiments

[Joe Buck]

Hi,

(all experiments are with sparc-sun-solaris2.5.1)

I just did a purify test on a subset of the previous example tdeque.cxx;
the only change is that I reduced the main program to just

int main() {
    deque_test(make_pair(1,2), make_pair(2,3), make_pair(3,5));
}

so the number of templates built gets divided by three (this case grows
to 113M instead of 492M with the bootstrapped egcs).  Because Purify
seems to have problems with 2.8.0/egcs object code, and because I wanted
to see easy-to-understand stack traces, I built a new cc1plus using
gcc 2.7.2.1 with the -g option.  I verified that it has the same problem
(that cc1plus -O tdeque.ii gets huge).

First, uninitialized memory reads (UMR's).  There's a probably-harmless
UMR caused by doing qsort on structs that don't have all fields set.
There's a more worrying one, though, in unify (file pt.c, line 3191):


           case TEMPLATE_TYPE_PARM:
    =>       (*nsubsts)++;
             idx = TEMPLATE_TYPE_IDX (parm);
             if (strict && (TYPE_READONLY (arg) < TYPE_READONLY (parm)
                            || TYPE_VOLATILE (arg) < TYPE_VOLATILE (parm)))

Otherwise no UMR's.

I was afraid that I wouldn't see leaks because of gcc's habit of doing
allocation through obstacks: an obstack could have a huge amount of junk
that would disappear at the end of the program.  However, there are
significant leaks, though not enough to explain the whole problem.

Over 6 megabytes are leaked, and 15 additional megabytes are "potentially
leaked" (meaning that there is something that looks like a pointer into
the middle of the object but not the beginning) from one source: memory
allocated at line 1810 of find_exception_handler_labels:

        /* We call xmalloc here instead of alloca; we did the latter in the past,
           but found that it can sometimes end up being asked to allocate space
           for more than 1 million labels.  */
   =>   labels = (rtx *) xmalloc ((max_labelno - min_labelno) * sizeof (rtx));
        bzero ((char *) labels, (max_labelno - min_labelno) * sizeof (rtx));
      
        /* Arrange for labels to be indexed directly by CODE_LABEL_NUMBER.  */

However, I've verified that exceptions are not the main problem.  If
I go back to the original example (using the bootstrapped egcs gcc this
time),

gcc -c -O -fno-exceptions tdeque.cxx

needs 395M, while

gcc -c -O tdeque.cxx

needs 492M.

So it seems that the problem must be related to too much being saved in
obstacks.  Are all the generated template functions being saved, as for inline
functions?

Re: purify experiments

[Jason Merrill]

>>>>> Jason Merrill <jason@cygnus.com> writes:

> OK, never mind again.  I fixed the obstacks that weren't being freed, and
> that saved an entire 16 bytes out of 13 megs.

I fixed it better, and it now saves about a meg (in both egcs and devo).

> Which is the other interesting point: devo gcc (from the internal Cygnus
> sources) takes 13 megs to compile your smaller testcase, while egcs takes
> 41 megs (both with -O -fno-exceptions).  The C++ frontend is identical,
> so it must be a backend problem.

Each of the inline maybepermanent_obstacks seems to take up a lot more
space with egcs:

inline obstacks: 199 obstacks, 21738336 bytes

vs.

inline obstacks: 199 obstacks, 3430016 bytes

That's quite a difference.  Hmm....

Jason

Re: purify experiments

[Jason Merrill]

OK, never mind again.  I fixed the obstacks that weren't being freed, and
that saved an entire 16 bytes out of 13 megs.  Which is the other
interesting point: devo gcc (from the internal Cygnus sources) takes 13
megs to compile your smaller testcase, while egcs takes 41 megs (both with
-O -fno-exceptions).  The C++ frontend is identical, so it must be a
backend problem.

Jason

Re: purify experiments

[Jim Wilson]

	My bad.  I've put this fix into EGCS & gcc2:

	1997-09-30  Brendan Kehoe  <brendan@lisa.cygnus.com>

	* except.c (find_exception_handler_labels): Free LABELS when we're
	done.

This patch is no good, as you are passing a value to free that did not
come from malloc.  Since this is interferring with my work, I have installed
this patch to fix the problem.  You are welcome to fix the problem differently
if you want.

Tue Sep 30 16:07:58 1997  Jim Wilson  <wilson@cygnus.com>

	* except.c (find_exception_handler_labels): Correct argument to free.

Index: except.c
===================================================================
RCS file: /cvs/cvsfiles/devo/gcc/except.c,v
retrieving revision 1.25
diff -p -r1.25 except.c
*** except.c	1997/09/30 18:29:46	1.25
--- except.c	1997/09/30 23:02:50
*************** find_exception_handler_labels ()
*** 1848,1854 ****
  	}
      }
  
!   free (labels);
  }
  
  /* Perform sanity checking on the exception_handler_labels list.
--- 1848,1854 ----
  	}
      }
  
!   free (labels + min_labelno);
  }
  
  /* Perform sanity checking on the exception_handler_labels list.

Re: purify experiments

[Jason Merrill]

>>>>> Jason Merrill <jason@cygnus.com> writes:

>>>>> Joe Buck <jbuck@synopsys.com> writes:

>> Also, even if we *do* need to save everything, any clues as to why the
>> representation is so large?

> No, not offhand.  I'm sure it could be tightened up, but I don't have the
> time to analyze it right now.

Never mind, I just realized where the problem is.  We aren't freeing the
maybepermanent_obstack for recursively instantiated templates.  I'll try to
fix it today.

Jason

Re: purify experiments

[Jason Merrill]

>>>>> Joe Buck <jbuck@synopsys.com> writes:

> Wasn't this clarified by the committee and the addition of the "export"
> keyword?  That is, shouldn't a template be always treated as a static
> function, unless export is given?

No.  Template instantiations still have weak linkage.

> Also, even if we *do* need to save everything, any clues as to why the
> representation is so large?

No, not offhand.  I'm sure it could be tightened up, but I don't have the
time to analyze it right now.

Jason

Re: purify experiments

[Brendan Kehoe]

> 15 additional megabytes are "potentially leaked" (meaning that there is
> something that looks like a pointer into the middle of the object but not
> the beginning) from one source: memory allocated at line 1810 of
> find_exception_handler_labels:
> 
>         /* We call xmalloc here instead of alloca; we did the latter in the past,
>            but found that it can sometimes end up being asked to allocate space
>            for more than 1 million labels.  */
>    =>   labels = (rtx *) xmalloc ((max_labelno - min_labelno) * sizeof (rtx));
>         bzero ((char *) labels, (max_labelno - min_labelno) * sizeof (rtx));
>       
>         /* Arrange for labels to be indexed directly by CODE_LABEL_NUMBER.  */

My bad.  I've put this fix into EGCS & gcc2:

1997-09-30  Brendan Kehoe  <brendan@lisa.cygnus.com>

	* except.c (find_exception_handler_labels): Free LABELS when we're
	done.

Index: except.c
diff -u -p -r1.24 except.c
--- except.c	1997/09/18 23:31:01	1.24
+++ except.c	1997/09/30 17:38:17
@@ -1847,6 +1847,8 @@ find_exception_handler_labels ()
 	    warning ("mismatched EH region %d", NOTE_BLOCK_NUMBER (insn));
 	}
     }
+
+  free (labels);
 }
 
 /* Perform sanity checking on the exception_handler_labels list.

-- 
Brendan Kehoe                                               brendan@cygnus.com
Cygnus Solutions, Sunnyvale, CA                                +1 408 542 9600

Web page: http://www.zen.org/~brendan/

Re: purify experiments

[David Edelsohn]

>>>>> Joe Buck writes:

>> > So it seems that the problem must be related to too much being saved in
>> > obstacks.  Are all the generated template functions being saved, as for
>> > inline functions?
>> 
>> Yes, and for the same reason: we don't know how they should be handled
>> until end-of-file.

Joe> Also, even if we *do* need to save everything, any clues as to why the
Joe> representation is so large?  It's on the order of 1000 times as large
Joe> as simply writing out the .ii file three times (for the three different
Joe> expansions).

	I know that we do not want to have multiple passes over the source
and do not want temporary files for intermediate passes, but it surely
sounds like an incorrect trade-off between space and time/computation is
being made.  The template functions can be regenerated when needed, so
maybe they should be saved in a more compact, high-level form -- more like
the original source.

David

Re: purify experiments

[Joe Buck]

> > So it seems that the problem must be related to too much being saved in
> > obstacks.  Are all the generated template functions being saved, as for
> > inline functions?
> 
> Yes, and for the same reason: we don't know how they should be handled
> until end-of-file.

Wasn't this clarified by the committee and the addition of the "export"
keyword?  That is, shouldn't a template be always treated as a static
function, unless export is given?  Then it should only be necessary to
save a template function if it is an inline function or a candidate for
inlining.

Or is there something I'm missing?

Also, even if we *do* need to save everything, any clues as to why the
representation is so large?  It's on the order of 1000 times as large
as simply writing out the .ii file three times (for the three different
expansions).

Re: purify experiments

[Jason Merrill]

>>>>> Joe Buck <jbuck@synopsys.com> writes:

> So it seems that the problem must be related to too much being saved in
> obstacks.  Are all the generated template functions being saved, as for
> inline functions?

Yes, and for the same reason: we don't know how they should be handled
until end-of-file.

Jason