[Bug java/41802] New: When attempting to compile pdftk-1.41 my machine comes to a grind

[Bug java/41802] New: When attempting to compile pdftk-1.41 my machine comes to a grind

[tink dot lq at gmail dot com]

make -C ../java_libs
make[1]: Entering directory `/home/p679492/src/pdftk-1.41/java_libs'
make -C "/home/p679492/src/pdftk-1.41/java_libs/com/lowagie/text";
make[2]: Entering directory
`/home/p679492/src/pdftk-1.41/java_libs/com/lowagie/text'
gcj -O2 -w --encoding=UTF-8
--classpath="/home/p679492/src/pdftk-1.41/java_libs" -c Anchor.java -o Anchor.o
GC Warning: Repeated allocation of very large block (appr. size 2097152000):
        May lead to memory leak and poor performance.
gcj: Internal error: Killed (program ecj1)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[2]: *** [Anchor.o] Error 1
make[2]: Leaving directory
`/home/p679492/src/pdftk-1.41/java_libs/com/lowagie/text'
make[1]: *** [itext] Error 2
make[1]: Leaving directory `/home/p679492/src/pdftk-1.41/java_libs'
make: *** [java_libs] Error 2



This bumps swap usage up to ~ 3G on a machine w/ 4GB of RAM ... the box becomes
unresponsive until ecj1 dies.


-- 
           Summary: When attempting to compile pdftk-1.41 my machine comes
                    to a grind
           Product: gcc
           Version: 4.3.3
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: java
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: tink dot lq at gmail dot com
 GCC build triplet: Configured with: ../gcc-4.3.3/configure --prefix=/usr --
                    libdir=/
  GCC host triplet: Reading specs from /usr/lib64/gcc/x86_64-slackware-
                    linux/4.3.3/s
GCC target triplet: x86_64-slackware-linux


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[lucabon at interfree dot it]

------- Comment #1 from lucabon at interfree dot it  2010-02-06 21:08 -------
I confirm the bug is present in gcj 4.3.3 and 4.4.3. Only x86-64 port is
affected (x86-32 works fine); gcj 4.2.4 works fine also for 64bit, so it seems
a regression.


-- 

lucabon at interfree dot it changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lucabon at interfree dot it


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[rguenth at gcc dot gnu dot org]

------- Comment #2 from rguenth at gcc dot gnu dot org  2010-02-06 21:29 -------
ecj1 is really the Eclipse frontend which we just inherit, the GCC java
frontend
(which just handles bytecode) is called jc1.

This bug also misses a testcase to reproduce the problem.

You can a more recent ecj version, like that which is downloaded by
the contrib/download_ecj.

Well - not really a GCC bug.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[lucabon at interfree dot it]

------- Comment #3 from lucabon at interfree dot it  2010-02-10 19:56 -------
(In reply to comment #2)
> ecj1 is really the Eclipse frontend which we just inherit, the GCC java
> frontend (which just handles bytecode) is called jc1.

Why starting from gcc 4.3 jc1 handles only bytecode and no more java source?

> This bug also misses a testcase to reproduce the problem.

You can reproduce the problem only into x86_64 platform:
1. Download pdftk http://www.pdfhacks.com/pdftk/pdftk-1.41.tar.bz2
2. cd pdftk-1.41/java_libs/com/lowagie/text
3. gcj -O2 -w --classpath="../../../" -c Anchor.java -o Anchor.o
4. ecj1 allocates about 4.2GB of memory
5. If you have enough memory (I have 2GB RAM + 3GB swap), compilation ends
successfully, even if after many time due to swapping.

ecj1 on x86_32 (same gcc version and build options) or jc1 on x86_64 (gcc
4.2.4) require only about 85 MB of memory.

> You can a more recent ecj version, like that which is downloaded by
> the contrib/download_ecj.

ecj-4.5.jar (actually the latest ecj) does not resolve the problem. 

> Well - not really a GCC bug.

Ok, but we have no other options to compile java sources with gcc 4.3 (only
downgrade to 4.2)... 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[rguenth at gcc dot gnu dot org]

------- Comment #4 from rguenth at gcc dot gnu dot org  2010-02-10 20:46 -------
(In reply to comment #3)
> (In reply to comment #2)
> > ecj1 is really the Eclipse frontend which we just inherit, the GCC java
> > frontend (which just handles bytecode) is called jc1.
> 
> Why starting from gcc 4.3 jc1 handles only bytecode and no more java source?
> 
> > This bug also misses a testcase to reproduce the problem.
> 
> You can reproduce the problem only into x86_64 platform:
> 1. Download pdftk http://www.pdfhacks.com/pdftk/pdftk-1.41.tar.bz2
> 2. cd pdftk-1.41/java_libs/com/lowagie/text
> 3. gcj -O2 -w --classpath="../../../" -c Anchor.java -o Anchor.o
> 4. ecj1 allocates about 4.2GB of memory
> 5. If you have enough memory (I have 2GB RAM + 3GB swap), compilation ends
> successfully, even if after many time due to swapping.
> 
> ecj1 on x86_32 (same gcc version and build options) or jc1 on x86_64 (gcc
> 4.2.4) require only about 85 MB of memory.
> 
> > You can a more recent ecj version, like that which is downloaded by
> > the contrib/download_ecj.
> 
> ecj-4.5.jar (actually the latest ecj) does not resolve the problem. 
> 
> > Well - not really a GCC bug.
> 
> Ok, but we have no other options to compile java sources with gcc 4.3 (only
> downgrade to 4.2)... 

Use another Java-to-bytecode compiler and feed gcc with bytecode.

Richard.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[rguenth at gcc dot gnu dot org]

------- Comment #5 from rguenth at gcc dot gnu dot org  2010-02-10 20:47 -------
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > ecj1 is really the Eclipse frontend which we just inherit, the GCC java
> > > frontend (which just handles bytecode) is called jc1.
> > 
> > Why starting from gcc 4.3 jc1 handles only bytecode and no more java source?
> > 
> > > This bug also misses a testcase to reproduce the problem.
> > 
> > You can reproduce the problem only into x86_64 platform:
> > 1. Download pdftk http://www.pdfhacks.com/pdftk/pdftk-1.41.tar.bz2
> > 2. cd pdftk-1.41/java_libs/com/lowagie/text
> > 3. gcj -O2 -w --classpath="../../../" -c Anchor.java -o Anchor.o
> > 4. ecj1 allocates about 4.2GB of memory
> > 5. If you have enough memory (I have 2GB RAM + 3GB swap), compilation ends
> > successfully, even if after many time due to swapping.
> > 
> > ecj1 on x86_32 (same gcc version and build options) or jc1 on x86_64 (gcc
> > 4.2.4) require only about 85 MB of memory.
> > 
> > > You can a more recent ecj version, like that which is downloaded by
> > > the contrib/download_ecj.
> > 
> > ecj-4.5.jar (actually the latest ecj) does not resolve the problem. 
> > 
> > > Well - not really a GCC bug.
> > 
> > Ok, but we have no other options to compile java sources with gcc 4.3 (only
> > downgrade to 4.2)... 
> 
> Use another Java-to-bytecode compiler and feed gcc with bytecode.

Btw, the symptoms you are seeing hint at not working garbage collection.
You might want to report this to your operating system vendor.

Richard.

> Richard.
> 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[howarth at nitro dot med dot uc dot edu]

------- Comment #6 from howarth at nitro dot med dot uc dot edu  2010-02-10 21:38 -------
Even if you fix this issue, I don't believe you will be able to compile
pdftk.cc with gcc 4.3 or later...

http://gcc.gnu.org/ml/java/2008-03/msg00028.html

due to the code incorrectly mixing c++ and java exceptions which are no longer
allowed under FSF gcc.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[lucabon at interfree dot it]

------- Comment #7 from lucabon at interfree dot it  2010-02-11 15:38 -------
(In reply to comment #5)
> (In reply to comment #4)
> > (In reply to comment #3)
> > 
> > Use another Java-to-bytecode compiler and feed gcc with bytecode.

I'd like to use only free software... There is another free (GPL-compatible)
java-to-bytecode compiler?

> Btw, the symptoms you are seeing hint at not working garbage collection.
> You might want to report this to your operating system vendor.

Sorry, but I think the garbage collector is from gcc (libgcj), not from
operating system.
I try to substitute ecj1 with a shell script:

#!/usr/bin/sh
gij -cp /usr/share/java/ecj.jar org.eclipse.jdt.internal.compiler.batch.GCCMain
"$@"

And the simptom are the same, so I think the problem is inside gcc java
interpreter. If I substitute "gij" with Sun "java", the compilation ends with
"only" about 400MB memory used.
Go deeply, gcj call ecj1 with the following arguments:
ecj1 Anchor.java -w -fbootclasspath=../../../:/usr/share/java/libgcj-4.3.3.jar
-fencoding=utf8 -fsource=1.5 -ftarget=1.5 -fzip-dependency /tmp/ccAgHLvl.zip
-fzip-target /tmp/cccrxtPw.jar
If I remove the last argument, class files are generated in the file system and
ecj1/gij uses "only" about 130MB of memory. If I re-add the -fzip-target
parameter with .class files into the file system, compilation is really fast.

So, I think the problem is in gcc 64bit java interpreter when it stores the
.class files in memory instead of file system.

(In reply to comment #6)
> Even if you fix this issue, I don't believe you will be able to compile
> pdftk.cc with gcc 4.3 or later...
> 
> http://gcc.gnu.org/ml/java/2008-03/msg00028.html
> 
> due to the code incorrectly mixing c++ and java exceptions which are no longer
> allowed under FSF gcc.

pdftk will finally compile correctly (I have the executable), but it takes many
hours to compile... I have only to apply a little patch to remove local
java.security.MessageDigest.

Luca


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[pinskia at gcc dot gnu dot org]

------- Comment #8 from pinskia at gcc dot gnu dot org  2010-02-11 18:10 -------
>There is another free (GPL-compatible) java-to-bytecode compiler?

Sun's OpenJDK (which includes javac) is GPLv2 IIRC.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[lucabon at interfree dot it]

------- Comment #9 from lucabon at interfree dot it  2010-02-11 21:14 -------
I fully compiled ecj1 with:
  gcj --main=org.eclipse.jdt.internal.compiler.batch.GCCMain -o ecj1 ecj.jar
and now it works fine (and very quickly!)

So, the problem seems to be in 64bit gij (java bytecode interpreter), used to
run ecj.jar (java-to-bytecode compiler).

How could I help to debug gij?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[howarth at nitro dot med dot uc dot edu]

------- Comment #10 from howarth at nitro dot med dot uc dot edu  2010-02-12 14:40 -------
Does slackware provide packaging files to show how they build their gcc?
Normally all you need to do is download
ftp://sourceware.org/pub/java/ecj-latest.jar and place it as ecj.jar in the top
of the gcc source tree for the build to properly utilize it. Does your stock
gcj compile simple java code like...

public class testme { 
  public static void main(String args[]){ 
    System.out.println("Hello"); 
  } 
} 

with just 'gcj --main=testme -O testme.java'? You shouldn't need to pass more
than that for gcj to work if it is using ecj.jar.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[lucabon at interfree dot it]

------- Comment #11 from lucabon at interfree dot it  2010-02-12 17:18 -------
(In reply to comment #10)
> Does slackware provide packaging files to show how they build their gcc?

Yes, sure. It is a "standard" build, with ecj.jar placed in the top source
tree, and it is exactly the same build script between 32 and 64 bits (always
x86 architecture), but only 64 bits has the problem. I could try another x86_64
distribution, but I think the problem is into gcc java interpreter...

> Does your stock gcj compile simple java code like...
> with just 'gcj --main=testme -O testme.java'? 

Yes, the problem seems to be only if ecj.jar (run by ecj1/gij interpreter) has
to compile many .java source files and in the same time it has to keep in
memory many bytecode .class file (option -fzip-target), like in pdftk.
If you run ecj.jar by Sun java interpreter or if you fully compile ecj.jar with
gcj to build a complete ecj1 (not only a wrapper for gij), pdftk compilation
works fine.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[lucabon at interfree dot it]

------- Comment #12 from lucabon at interfree dot it  2010-02-13 10:28 -------
I upgraded all gcc components to 4.4.3 (before I upgraded only java parts) and
it works fine. I downgraded to 4.3.3 and put only libgcc_s.so.1 from gcc 4.4.3
and it works fine. So, the problem was in libgcc_s and it was fixed in gcc 4.4.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[rguenth at gcc dot gnu dot org]

------- Comment #13 from rguenth at gcc dot gnu dot org  2010-02-13 10:58 -------
(In reply to comment #12)
> I upgraded all gcc components to 4.4.3 (before I upgraded only java parts) and
> it works fine. I downgraded to 4.3.3 and put only libgcc_s.so.1 from gcc 4.4.3
> and it works fine. So, the problem was in libgcc_s and it was fixed in gcc 4.4.

Exactly as I thought.  There is/was an unwinding problem with made garbage
collection break, there is a glibc fix available or an unwinder fix in gcc 4.4.
Your distributor didn't pick up the glibc fix appearantly.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |WORKSFORME


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[mikpe at it dot uu dot se]

------- Comment #14 from mikpe at it dot uu dot se  2010-02-13 11:52 -------
(In reply to comment #13)
> Exactly as I thought.  There is/was an unwinding problem with made garbage
> collection break, there is a glibc fix available or an unwinder fix in gcc 4.4.
> Your distributor didn't pick up the glibc fix appearantly.

Do you have the PR numbers or commit dates for the glibc and unwinder fixes?


-- 

mikpe at it dot uu dot se changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mikpe at it dot uu dot se


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802

[Bug java/41802] When attempting to compile pdftk-1.41 my machine comes to a grind

[rguenth at gcc dot gnu dot org]

------- Comment #15 from rguenth at gcc dot gnu dot org  2010-02-13 13:31 -------
(In reply to comment #14)
> (In reply to comment #13)
> > Exactly as I thought.  There is/was an unwinding problem with made garbage
> > collection break, there is a glibc fix available or an unwinder fix in gcc 4.4.
> > Your distributor didn't pick up the glibc fix appearantly.
> 
> Do you have the PR numbers or commit dates for the glibc and unwinder fixes?

I think this was fixed in the unwinder by

2009-04-04  Jakub Jelinek  <jakub@redhat.com>

        * unwind-dw2.h (_Unwind_FrameState): Add REG_UNDEFINED enum value.
        * unwind-dw2.c (execute_cfa_program): Set how to REG_UNDEFINED
        instead of REG_UNSAVED for DW_CFA_undefined.
        (uw_update_context_1): Handle REG_UNDEFINED the same as REG_UNSAVED.
        (uw_update_context): If RA column is REG_UNDEFINED, mark it as
        outermost frame.

and the bug can be avoided in glibc by

Wed Jun 25 13:30:42 CEST 2008 - rguenther@suse.de
- Add patch to fix unwinding through clone for x86_64 and i386.
  [bnc#290807, bnc#403464]

bnc#403464 contains useful links (that's bugzilla.novell.com), one
reference to PR36568 which contains a glibc patch.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41802