Re: Patch: FYI: split up parser build

Re: Patch: FYI: split up parser build

[Andrew Haley]

Tom Tromey writes:
 > Gerald> Do you see any other idea to reduce memory consumption that leads to
 > Gerald> the failure building gnu/javax/swing/text/html/parser/HTML_401F.lo?
 > 
 > This class hasn't changed in a while.  So if this worked ok for you
 > after the big gcj-eclipse merge, and then more recently started
 > failing, then I would say that this is a space regression elsewhere
 > in GCC.

I don't think that such a space regression is surprising.  The .class
front-end generates trees that are more verbose than those of the old
.java front-end.  There are possibilities for improvement here.

Andrew.

Re: Patch: FYI: split up parser build

[Tom Tromey]

>>>>> "Andrew" == Andrew Haley <aph@redhat.com> writes:

Andrew> I don't think that such a space regression is surprising.  The .class
Andrew> front-end generates trees that are more verbose than those of the old
Andrew> .java front-end.  There are possibilities for improvement here.

I took a brief look at this today.  I compiled the .class file with
svn head and with the compiler in FC6, and I compiled the .java file
with the compiler in FC6.  I used '-ftime-report fmem-report -g -O2'.

Total memory use:

svn head .class    120037 kB
FC6 .class         181363 kB
FC6 .java           74173 kB

So, the compiler is actually improving :).  But, yeah, the .class
front end could use some love.

Tom

Re: Patch: FYI: split up parser build

[Tom Tromey]

Gerald> Do you see any other idea to reduce memory consumption that leads to
Gerald> the failure building gnu/javax/swing/text/html/parser/HTML_401F.lo?

This class hasn't changed in a while.  So if this worked ok for you
after the big gcj-eclipse merge, and then more recently started
failing, then I would say that this is a space regression elsewhere in
GCC.

Failing that I suppose we could split the class up somehow.

Tom

Re: Patch: FYI: split up parser build

[Gerald Pfeifer]

On Fri, 2 Feb 2007, Tom Tromey wrote:
> Feel like trying it with exp.complexity removed?  :)  That would be
> above and beyond.

This doesn't save us sufficient amounts of memory, I'm afraid.  I now
worked around the increased memory consumption by means of the system
configuration trick I posted over the weekend, and if there are no
further complaints about this it's probably not of too high a priority
(though we may not have hit users yet, and developers usually have nice
machines ;-).

Do you see any other idea to reduce memory consumption that leads to
the failure building gnu/javax/swing/text/html/parser/HTML_401F.lo?

Gerald

Re: Patch: FYI: split up parser build

[Tom Tromey]

>>>>> "Gerald" == Gerald Pfeifer <gerald@pfeifer.com> writes:

Gerald> I tried twice, because I was not completely sure about the
Gerald> environment the first time, so this took a bit, but sadly both
Gerald> builds failed the same way:

That is unfortunate :(

Feel like trying it with exp.complexity removed?  :)  That would be
above and beyond.

Tom

Re: Patch: FYI: split up parser build

[Gerald Pfeifer]

Hi Tom,

On Wed, 31 Jan 2007, Tom Tromey wrote:
> I didn't try to measure whether this helps compile-time memory use.
> Gerald, if you could try it, I'd appreciate it.

I tried twice, because I was not completely sure about the environment
the first time, so this took a bit, but sadly both builds failed the same
way:

/files/pfeifer/OBJ-0202-0121/gcc/gcj -B/files/pfeifer/OBJ-0202-0121/i386-unknown-freebsd5.4/libjava/ -B/files/pfeifer/OBJ-0202-0121/gcc/ -ffloat-store -fomit-frame-pointer -fclasspath= -fbootclasspath=/sw/test/GCC/trunk/libjava/classpath/lib --encoding=UTF-8 -Wno-deprecated -fbootstrap-classes -g -O2 -c -fsource-filename=/files/pfeifer/OBJ-0202-0121/i386-unknown-freebsd5.4/libjava/classpath/lib/classes -MT gnu/javax/swing/text/html/parser/HTML_401F.lo -MD -MP -MF gnu/javax/swing/text/html/parser/HTML_401F.deps @gnu/javax/swing/text/html/parser/HTML_401F.list -fPIC -o gnu/javax/swing/text/html/parser/.libs/HTML_401F.o
jc1: out of memory allocating 4072 bytes after a total of 536275496 bytes
gmake[3]: *** [gnu/javax/swing/text/html/parser/HTML_401F.lo] Error 1
gmake[3]: Leaving directory `/files/pfeifer/OBJ-0202-0121/i386-unknown-freebsd5.4/libjava'
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory `/files/pfeifer/OBJ-0202-0121/i386-unknown-freebsd5.4/libjava'
gmake[1]: *** [all-target-libjava] Error 2

Gerald

Patch: FYI: split up parser build

[Tom Tromey]

I'm checking this in.

This splits the html parser build so that it is not compiled as one
big package.  Rather, we compile the .class files coming from a single
.java file as a group.

I didn't try to measure whether this helps compile-time memory use.
Gerald, if you could try it, I'd appreciate it.


Tom

Index: ChangeLog
from  Tom Tromey  <tromey@redhat.com>
	* scripts.am, Makefile.in: Rebuilt.
	* scripts/makemake.tcl (gnu/javax/swing/text/html/parser): Build
	as 'ordinary'.
	(emit_ordinary_rule): New proc.

Index: scripts/makemake.tcl
===================================================================
--- scripts/makemake.tcl	(revision 121362)
+++ scripts/makemake.tcl	(working copy)
@@ -65,6 +65,14 @@
 set package_map(gnu/CORBA) bc
 set package_map(gnu/javax/rmi) bc
 
+# parser/HTML_401F.class is really big, and there have been complaints
+# about this package requiring too much memory to build.  So, we
+# compile it as separate objects.  But, we're careful to compile the
+# sub-packages as packages.
+set package_map(gnu/javax/swing/text/html/parser) ordinary
+set package_map(gnu/javax/swing/text/html/parser/models) package
+set package_map(gnu/javax/swing/text/html/parser/support) package
+
 # More special cases.  These end up in their own library.
 # Note that if we BC-compile AWT we must update these as well.
 set package_map(gnu/gcj/xlib) package
@@ -297,6 +305,32 @@
   }
 }
 
+# Emit a rule to build a package full of 'ordinary' files, that is,
+# one .o for each .java.
+proc emit_ordinary_rule {package} {
+  global name_map package_files
+
+  foreach file $name_map($package) {
+    # Strip off the '.java'.
+    set root [file rootname $file]
+
+    # Look for all included .class files.  Assumes that we don't have
+    # multiple top-level classes in a .java file.
+    set lname $root.list
+    set dname $root.deps
+
+    puts "$lname: classpath/$file"
+    puts "\t@\$(mkinstalldirs) \$(dir \$@)"
+    puts "\techo \$(srcdir)/classpath/lib/${root}*.class> $lname"
+    puts ""
+    puts "-include $dname"
+    puts ""
+    puts ""
+
+    lappend package_files $lname
+  }
+}
+
 # Emit a package-like rule for a platform-specific Process
 # implementation.
 proc emit_process_package_rule {platform} {
@@ -419,7 +453,7 @@
   if {$package_map($package) == "bc"} {
     emit_bc_rule $package
   } elseif {$package_map($package) == "ordinary"} {
-    # Nothing in particular to do here.
+    emit_ordinary_rule $package
   } elseif {$package_map($package) == "package"} {
     emit_package_rule $package
   } else {