GCJ .jar to .so with native method

GCJ .jar to .so with native method

[Joe Hoffert]

Hi.

I'm running into some problems converting a Java library (i.e., .jar
file) into a native shared library (i.e., .so file) and then using that
library with g++. In particular, I have a native method in the .jar file
that I can't figure out how to define to the linker's satisfaction. I'm
using GCC 4.1.2 on Linux.

If I modify the gcjh-generated header file and define the native method
inline (e.g., virtual void receive(JArray< jbyte > *, jint) {} ) the
linker still complains about an undefined reference for the native
method:

.//libricochet.so: undefined reference to `void
multishot::examples::RicochetApp::receive(JArray<char>*, int)'
collect2: ld returned 1 exit status

If I define the native method outside of the header file (e.g., in a C++
source file like so:
void multishot::examples::RicochetApp::receive(JArray< jbyte > *, jint)
{} ) then the linker complains about a hidden symbol:

/usr/bin/ld: RicochetApp: hidden symbol `void
multishot::examples::RicochetApp::receive(JArray<char>*, int)'
in .obj/RicochetAppMain.o is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output

Any ideas as to what I need to do to rectify my problem would be greatly
appreciated.

Thanks!

-Joe

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:

 > I'm running into some problems converting a Java library (i.e., .jar
 > file) into a native shared library (i.e., .so file) and then using that
 > library with g++. In particular, I have a native method in the .jar file
 > that I can't figure out how to define to the linker's satisfaction. I'm
 > using GCC 4.1.2 on Linux.
 > 
 > If I modify the gcjh-generated header file and define the native method
 > inline (e.g., virtual void receive(JArray< jbyte > *, jint) {} ) the
 > linker still complains about an undefined reference for the native
 > method:
 > 
 > .//libricochet.so: undefined reference to `void
 > multishot::examples::RicochetApp::receive(JArray<char>*, int)'
 > collect2: ld returned 1 exit status
 > 
 > If I define the native method outside of the header file (e.g., in a C++
 > source file like so:
 > void multishot::examples::RicochetApp::receive(JArray< jbyte > *, jint)
 > {} ) then the linker complains about a hidden symbol:
 > 
 > /usr/bin/ld: RicochetApp: hidden symbol `void
 > multishot::examples::RicochetApp::receive(JArray<char>*, int)'
 > in .obj/RicochetAppMain.o is referenced by DSO
 > /usr/bin/ld: final link failed: Nonrepresentable section on output
 > 
 > Any ideas as to what I need to do to rectify my problem would be greatly
 > appreciated.

This seems odd; we link C++ and Java code all the time.

Is it possible for you to reduce your problem to a small test case?
Then we can see what's going wrong.  I think you're linking
incorrectly, but at the moment that's just a wild guess.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Joe Hoffert]

Hi, Andrew.

On Tue, 2007-12-04 at 13:27 +0000, Andrew Haley wrote: 
> Joe Hoffert writes:
> 
>  > I'm running into some problems converting a Java library (i.e., .jar
>  > file) into a native shared library (i.e., .so file) and then using that
>  > library with g++. In particular, I have a native method in the .jar file
>  > that I can't figure out how to define to the linker's satisfaction. I'm
>  > using GCC 4.1.2 on Linux.
>  > 
>  > If I modify the gcjh-generated header file and define the native method
>  > inline (e.g., virtual void receive(JArray< jbyte > *, jint) {} ) the
>  > linker still complains about an undefined reference for the native
>  > method:
>  > 
>  > .//libricochet.so: undefined reference to `void
>  > multishot::examples::RicochetApp::receive(JArray<char>*, int)'
>  > collect2: ld returned 1 exit status
>  > 
>  > If I define the native method outside of the header file (e.g., in a C++
>  > source file like so:
>  > void multishot::examples::RicochetApp::receive(JArray< jbyte > *, jint)
>  > {} ) then the linker complains about a hidden symbol:
>  > 
>  > /usr/bin/ld: RicochetApp: hidden symbol `void
>  > multishot::examples::RicochetApp::receive(JArray<char>*, int)'
>  > in .obj/RicochetAppMain.o is referenced by DSO
>  > /usr/bin/ld: final link failed: Nonrepresentable section on output
>  > 
>  > Any ideas as to what I need to do to rectify my problem would be greatly
>  > appreciated.
> 
> This seems odd; we link C++ and Java code all the time.

Do you have a small example with C++ calling into Java (using the
invocation API) and then Java calling back to C++? I'm trying to
integrate a transport protocol written in Java with middleware written
in C++ so I need to call the protocol to send bytes and have the
protocol call C++ when it receives bytes.

> Is it possible for you to reduce your problem to a small test case?

I'll try to get a test case with a small .jar file. The test case I have
right now is fairly small although the .jar file isn't (i.e., 208031
bytes).

> Then we can see what's going wrong.  I think you're linking
> incorrectly, but at the moment that's just a wild guess.

Here is the gcj command I'm using to generate the .so library:

gcj -shared -fPIC -Wl,-Bsymbolic
~/Ricochet/jdk1.5_baseline/package/lib/ricochet.jar -o libricochet.so

Another possible wrinkle is that the .jar file was generated with JDK
1.5 rather than with gcj. (The Java source code uses generics quite a
bit which the gcj compiler doesn't like - at least for 4.1.2.) I'm
assuming byte codes are byte codes so this shouldn't be a problem but
perhaps this is naive on my part.

BTW, is there a later version of gch that might be able to handle the
generics in the Java source code?

Thanks for your help!

-Joe

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:
 > Hi, Andrew.
 > 
 > On Tue, 2007-12-04 at 13:27 +0000, Andrew Haley wrote: 
 > > Joe Hoffert writes:
 > > 
 > >  > I'm running into some problems converting a Java library (i.e., .jar
 > >  > file) into a native shared library (i.e., .so file) and then using that
 > >  > library with g++. In particular, I have a native method in the .jar file
 > >  > that I can't figure out how to define to the linker's satisfaction. I'm
 > >  > using GCC 4.1.2 on Linux.
 > >  > 
 > >  > If I modify the gcjh-generated header file and define the native method
 > >  > inline (e.g., virtual void receive(JArray< jbyte > *, jint) {} ) the
 > >  > linker still complains about an undefined reference for the native
 > >  > method:
 > >  > 
 > >  > .//libricochet.so: undefined reference to `void
 > >  > multishot::examples::RicochetApp::receive(JArray<char>*, int)'
 > >  > collect2: ld returned 1 exit status
 > >  > 
 > >  > If I define the native method outside of the header file (e.g., in a C++
 > >  > source file like so:
 > >  > void multishot::examples::RicochetApp::receive(JArray< jbyte > *, jint)
 > >  > {} ) then the linker complains about a hidden symbol:
 > >  > 
 > >  > /usr/bin/ld: RicochetApp: hidden symbol `void
 > >  > multishot::examples::RicochetApp::receive(JArray<char>*, int)'
 > >  > in .obj/RicochetAppMain.o is referenced by DSO
 > >  > /usr/bin/ld: final link failed: Nonrepresentable section on output
 > >  > 
 > >  > Any ideas as to what I need to do to rectify my problem would be greatly
 > >  > appreciated.
 > > 
 > > This seems odd; we link C++ and Java code all the time.
 > 
 > Do you have a small example with C++ calling into Java (using the
 > invocation API) and then Java calling back to C++?

No.  I have a really big one, libgcj itself.

 > I'm trying to integrate a transport protocol written in Java with
 > middleware written in C++ so I need to call the protocol to send
 > bytes and have the protocol call C++ when it receives bytes.
 > 
 > > Is it possible for you to reduce your problem to a small test case?
 > 
 > I'll try to get a test case with a small .jar file. The test case I have
 > right now is fairly small although the .jar file isn't (i.e., 208031
 > bytes).

That is plenty small enough.

 > > Then we can see what's going wrong.  I think you're linking
 > > incorrectly, but at the moment that's just a wild guess.
 > 
 > Here is the gcj command I'm using to generate the .so library:
 > 
 > gcj -shared -fPIC -Wl,-Bsymbolic
 > ~/Ricochet/jdk1.5_baseline/package/lib/ricochet.jar -o libricochet.so

I thought you were linking the jar with some native code.  If so, the
native code must appear on that line also.

 > Another possible wrinkle is that the .jar file was generated with JDK
 > 1.5 rather than with gcj. (The Java source code uses generics quite a
 > bit which the gcj compiler doesn't like - at least for 4.1.2.) I'm
 > assuming byte codes are byte codes so this shouldn't be a problem but
 > perhaps this is naive on my part.

It might be OK, but later versions of gcj (that, sadly are still not
yet officially released!) handle Java 1.5 much better.  

 > BTW, is there a later version of gch that might be able to handle the
 > generics in the Java source code?

It's quite hard to handle generics in gcjh, but for the purposes we
use native methods for it works well enough.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:
 > Hi, Andrew.
 > 
 > On Tue, 2007-12-04 at 15:16 +0000, Andrew Haley wrote:
 > 
 > OK. I've attached a zip file with everything that should be needed. The
 > gcj-commands file simply contains the gcj and gcjh commands I used to
 > generate the .so library and the .h header files.

Works for me:

zorro:tmp $  gcjh -classpath ./ricochet.jar multishot.examples.RicochetApp
zorro:tmp $  gcj -shared -fPIC -Wl,-Bsymbolic ./ricochet.jar -o libricochet.so
zorro:tmp $ gcj RicochetAppMain.cpp -L. -lricochet -fPIC  -lstdc++
cc1plus: warning: command line option "-fbootclasspath=./:/usr/share/java/libgcj-4.1.2.jar" is valid for Java but not for C++
zorro:tmp $  LD_LIBRARY_PATH=. ./a.out 
Usage: RicochetApp GMS_id FDS_id startPortNumber [propertyString]

You really need to call JvCreateJavaVM() first.

If this doesn't give you the same results, the problem is likely to be
that you need a new version of gcj.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Joe Hoffert]

Hi, Andrew.

On Tue, 2007-12-04 at 15:59 +0000, Andrew Haley wrote: 
> Joe Hoffert writes:
>  > Hi, Andrew.
>  > 
>  > On Tue, 2007-12-04 at 15:16 +0000, Andrew Haley wrote:
>  > 
>  > OK. I've attached a zip file with everything that should be needed. The
>  > gcj-commands file simply contains the gcj and gcjh commands I used to
>  > generate the .so library and the .h header files.
> 
> Works for me:
> 
> zorro:tmp $  gcjh -classpath ./ricochet.jar multishot.examples.RicochetApp
> zorro:tmp $  gcj -shared -fPIC -Wl,-Bsymbolic ./ricochet.jar -o libricochet.so
> zorro:tmp $ gcj RicochetAppMain.cpp -L. -lricochet -fPIC  -lstdc++
> cc1plus: warning: command line option "-fbootclasspath=./:/usr/share/java/libgcj-4.1.2.jar" is valid for Java but not for C++

Ah! This is the missing piece for me. I was linking with g++. I was
assuming since I had a .so library and .h headers I could treat the
transformed .jar file like a C++ library. It never crossed my mind to
compile the C++ file with gcj and link everything with gcj.

I assume I will need to use gcj rather than g++ for all my C++ files. Is
this correct?

> zorro:tmp $  LD_LIBRARY_PATH=. ./a.out 
> Usage: RicochetApp GMS_id FDS_id startPortNumber [propertyString]
> 
> You really need to call JvCreateJavaVM() first.

In some other code I have I call JNI_CreateJavaVM. Is this different
from JvCreateJavaVM and, if so, do you know how it differs? The
prototype for JvCreateJavaVM doesn't exist in the jni.h file that I'm
using.

Adding this call raises some questions for me. Does this call add
overhead (and/or footprint)? I was (seemingly naively) assuming that
once the .jar file was converted to a .so file I was done with needing
any Java support. This doesn't seem to be the case. Why is the JVM still
needed?

Are there any other calls I should add?

> If this doesn't give you the same results, the problem is likely to be
> that you need a new version of gcj.

Thanks very much! You've been a tremendous help. I have struggled with
this C++/Java integration for a while.

-Joe

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:
 > Hi, Andrew.
 > 
 > On Tue, 2007-12-04 at 15:59 +0000, Andrew Haley wrote: 
 > > Joe Hoffert writes:
 > >  > Hi, Andrew.
 > >  > 
 > >  > On Tue, 2007-12-04 at 15:16 +0000, Andrew Haley wrote:
 > >  > 
 > >  > OK. I've attached a zip file with everything that should be needed. The
 > >  > gcj-commands file simply contains the gcj and gcjh commands I used to
 > >  > generate the .so library and the .h header files.
 > > 
 > > Works for me:
 > > 
 > > zorro:tmp $  gcjh -classpath ./ricochet.jar multishot.examples.RicochetApp
 > > zorro:tmp $  gcj -shared -fPIC -Wl,-Bsymbolic ./ricochet.jar -o libricochet.so
 > > zorro:tmp $ gcj RicochetAppMain.cpp -L. -lricochet -fPIC  -lstdc++
 > > cc1plus: warning: command line option "-fbootclasspath=./:/usr/share/java/libgcj-4.1.2.jar" is valid for Java but not for C++
 > 
 > Ah! This is the missing piece for me. I was linking with g++. I was
 > assuming since I had a .so library and .h headers I could treat the
 > transformed .jar file like a C++ library. It never crossed my mind to
 > compile the C++ file with gcj and link everything with gcj.
 > 
 > I assume I will need to use gcj rather than g++ for all my C++ files. Is
 > this correct?

I don't think it matters.  I could have done this instead:

zorro:tmp $ g++ RicochetAppMain.cpp -L. -lricochet -fPIC  -lgcj

 > > zorro:tmp $  LD_LIBRARY_PATH=. ./a.out 
 > > Usage: RicochetApp GMS_id FDS_id startPortNumber [propertyString]
 > > 
 > > You really need to call JvCreateJavaVM() first.
 > 
 > In some other code I have I call JNI_CreateJavaVM. Is this
 > different from JvCreateJavaVM and, if so, do you know how it
 > differs? The prototype for JvCreateJavaVM doesn't exist in the
 > jni.h file that I'm using.

No, but it is included.  You must call JvCreateJavaVM() as the very
first call into libgcj.  This is *really* important.

 > Adding this call raises some questions for me. Does this call add
 > overhead (and/or footprint)? I was (seemingly naively) assuming
 > that once the .jar file was converted to a .so file I was done with
 > needing any Java support. This doesn't seem to be the case. Why is
 > the JVM still needed?

There's a ton of stuff: threads, garbage collection, and of course the
entire Java runtime library.

 > Are there any other calls I should add?

I can't think of any.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Joe Hoffert]

Hi, Andrew.

You're right that using gcj or g++ to link doesn't matter. The difference
with what I was doing before was the compiler/linker options.

One more question. Is there any advantage to converting a .jar library
to a .so library and using it as opposed to simply using the
original .jar file with the Java invocation API and the JNI to call back
to C++ from Java? I thought I would get a performance increase with
using the .so library but perhaps there are no substantive differences
between the two approaches.

Thanks again for all your help!

-Joe

On Tue, 2007-12-04 at 16:42 +0000, Andrew Haley wrote:
> Joe Hoffert writes:
>  > Hi, Andrew.
>  > 
>  > On Tue, 2007-12-04 at 15:59 +0000, Andrew Haley wrote: 
>  > > Joe Hoffert writes:
>  > >  > Hi, Andrew.
>  > >  > 
>  > >  > On Tue, 2007-12-04 at 15:16 +0000, Andrew Haley wrote:
>  > >  > 
>  > >  > OK. I've attached a zip file with everything that should be needed. The
>  > >  > gcj-commands file simply contains the gcj and gcjh commands I used to
>  > >  > generate the .so library and the .h header files.
>  > > 
>  > > Works for me:
>  > > 
>  > > zorro:tmp $  gcjh -classpath ./ricochet.jar multishot.examples.RicochetApp
>  > > zorro:tmp $  gcj -shared -fPIC -Wl,-Bsymbolic ./ricochet.jar -o libricochet.so
>  > > zorro:tmp $ gcj RicochetAppMain.cpp -L. -lricochet -fPIC  -lstdc++
>  > > cc1plus: warning: command line option "-fbootclasspath=./:/usr/share/java/libgcj-4.1.2.jar" is valid for Java but not for C++
>  > 
>  > Ah! This is the missing piece for me. I was linking with g++. I was
>  > assuming since I had a .so library and .h headers I could treat the
>  > transformed .jar file like a C++ library. It never crossed my mind to
>  > compile the C++ file with gcj and link everything with gcj.
>  > 
>  > I assume I will need to use gcj rather than g++ for all my C++ files. Is
>  > this correct?
> 
> I don't think it matters.  I could have done this instead:
> 
> zorro:tmp $ g++ RicochetAppMain.cpp -L. -lricochet -fPIC  -lgcj
> zorro:tmp $  LD_LIBRARY_PATH=. ./a.out 
> 
>  > > Usage: RicochetApp GMS_id FDS_id startPortNumber [propertyString]
>  > > 
>  > > You really need to call JvCreateJavaVM() first.
>  > 
>  > In some other code I have I call JNI_CreateJavaVM. Is this
>  > different from JvCreateJavaVM and, if so, do you know how it
>  > differs? The prototype for JvCreateJavaVM doesn't exist in the
>  > jni.h file that I'm using.
> 
> No, but it is included.  You must call JvCreateJavaVM() as the very
> first call into libgcj.  This is *really* important.
> 
>  > Adding this call raises some questions for me. Does this call add
>  > overhead (and/or footprint)? I was (seemingly naively) assuming
>  > that once the .jar file was converted to a .so file I was done with
>  > needing any Java support. This doesn't seem to be the case. Why is
>  > the JVM still needed?
> 
> There's a ton of stuff: threads, garbage collection, and of course the
> entire Java runtime library.
> 
>  > Are there any other calls I should add?
> 
> I can't think of any.
> 
> Andrew.
> 

Re: GCJ .jar to .so with native method

[David Daney]

Joe Hoffert wrote:

> One more question. Is there any advantage to converting a .jar library
> to a .so library and using it as opposed to simply using the
> original .jar file with the Java invocation API and the JNI to call back
> to C++ from Java? I thought I would get a performance increase with
> using the .so library but perhaps there are no substantive differences
> between the two approaches.
> 

It depends on how much work the code in the .jar is doing.  Compiling to 
a .so will generally produce faster code, but the memory overhead of a 
.so may make using the interpreter on the .jar a better option.

If you really care, you could try both ways.

David Daney

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:

 > You're right that using gcj or g++ to link doesn't matter. The
 > difference with what I was doing before was the compiler/linker
 > options.

Can you let us know what in our instructions didn't give you the
information you needed?  It might help us improve our documentation.

 > One more question. Is there any advantage to converting a .jar
 > library to a .so library and using it as opposed to simply using
 > the original .jar file with the Java invocation API and the JNI to
 > call back to C++ from Java? I thought I would get a performance
 > increase with using the .so library but perhaps there are no
 > substantive differences between the two approaches.

You will probably get a performance increase, yes.  In particular,
calling native code from compiled code is very quick.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Joe Hoffert]

Hi, Andrew.

On Tue, 2007-12-04 at 18:13 +0000, Andrew Haley wrote:
> Joe Hoffert writes:
> 
>  > You're right that using gcj or g++ to link doesn't matter. The
>  > difference with what I was doing before was the compiler/linker
>  > options.
> 
> Can you let us know what in our instructions didn't give you the
> information you needed?  It might help us improve our documentation.

I didn't see anything in gcj.pdf that talks about conflicting compiler
or linker options. The options that gave me trouble are the
-fvisibility=hidden and -fvisibility-inlines-hidden. When I take these
out the program compiles and links. I think I'll be fine without these
options.

Now the program is aborting with the first "Java" call. I'm simply
calling a method specified by the gcjh generated header file. The code
is aborting from initializeClass:

(gdb) where
#0  0x00110402 in __kernel_vsyscall ()
#1  0x0084bfa0 in raise () from /lib/libc.so.6
#2  0x0084d8b1 in abort () from /lib/libc.so.6
#3  0x02da0d38 in _Jv_Throw () from /usr/lib/libgcj.so.8rh
#4  0x02ddc888 in java::lang::Class::initializeClass ()
   from /usr/lib/libgcj.so.8rh
#5  0x02dddd36 in _Jv_InitClass () from /usr/lib/libgcj.so.8rh
#6  0x00145a44 in
multishot.examples.RicochetApp.initialize(java.lang.String,
java.lang.String, int, java.lang.String)multishot.examples.RicochetApp
()
   from /home/jhoffert/Ricochet/trimmed/c
++-java/linux-jdk1.5/libricochet.so
#7  0x08048ec2 in main (argc=4, argv=0xbfca39e4) at
RicochetAppMain.cpp:95


I also saw in the documentation (i.e., chapter 12, section 12.7 of
gcj.pdf) that the C++ compiler doesn't invoke extra code to make sure a
class is initialized. The method I'm calling is static. I added code to
try and initialize the class:

  JvInitClass(&multishot::examples::RicochetApp::class$); // this is
line 87

 but the program aborts in that call:

(gdb) where
#0  0x02da2610 in _Jv_Linker::verify_class ()
from /usr/lib/libgcj.so.8rh
#1  0x02da4762 in _Jv_Linker::wait_for_state ()
from /usr/lib/libgcj.so.8rh
#2  0x02ddc6b4 in java::lang::Class::initializeClass ()
   from /usr/lib/libgcj.so.8rh
#3  0x02dddd36 in _Jv_InitClass () from /usr/lib/libgcj.so.8rh
#4  0x0804908d in JvInitClass (cls=0x804a690)
    at /usr/lib/gcc/i386-redhat-linux/4.1.2/../../../../include/c
++/4.1.2/gcj/cni.h:30
#5  0x08048f7e in main (argc=4, argv=0xbfe54394) at
RicochetAppMain.cpp:87


Is there I'm not doing when calling JvCreateJavaVM? I called this
initially with an empty JvVMInitArgs argument and the program aborted so
I added the -D option for the .jar file. This seems redundant to me but
I wasn't sure what else to try. This argument is what I pass when I'm
using the invocation API.

I appreciate your time but don't want to waste it. Please point me to
documentation, examples, or tutorials if you don't have time to keep
helping me.

Thanks!

-Joe

>  > One more question. Is there any advantage to converting a .jar
>  > library to a .so library and using it as opposed to simply using
>  > the original .jar file with the Java invocation API and the JNI to
>  > call back to C++ from Java? I thought I would get a performance
>  > increase with using the .so library but perhaps there are no
>  > substantive differences between the two approaches.
> 
> You will probably get a performance increase, yes.  In particular,
> calling native code from compiled code is very quick.
> 
> Andrew.
> 

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:
 > Hi, Andrew.
 > 
 > On Tue, 2007-12-04 at 18:13 +0000, Andrew Haley wrote:
 > > Joe Hoffert writes:
 > > 
 > >  > You're right that using gcj or g++ to link doesn't matter. The
 > >  > difference with what I was doing before was the compiler/linker
 > >  > options.
 > > 
 > > Can you let us know what in our instructions didn't give you the
 > > information you needed?  It might help us improve our documentation.
 > 
 > I didn't see anything in gcj.pdf that talks about conflicting compiler
 > or linker options. The options that gave me trouble are the
 > -fvisibility=hidden and -fvisibility-inlines-hidden. When I take these
 > out the program compiles and links. I think I'll be fine without these
 > options.

Ah, OK.  I'm not sure we could fix that, even in principle: you may
well be the first person ever to experiment with changing symbol
visibility, and I would have had to try it to discover what would
happen.  OK, I could have guessed that it wouldn't work!

 > Now the program is aborting with the first "Java" call. I'm simply
 > calling a method specified by the gcjh generated header file. The code
 > is aborting from initializeClass:
 > 
 > (gdb) where
 > #0  0x00110402 in __kernel_vsyscall ()
 > #1  0x0084bfa0 in raise () from /lib/libc.so.6
 > #2  0x0084d8b1 in abort () from /lib/libc.so.6
 > #3  0x02da0d38 in _Jv_Throw () from /usr/lib/libgcj.so.8rh
 > #4  0x02ddc888 in java::lang::Class::initializeClass ()
 >    from /usr/lib/libgcj.so.8rh
 > #5  0x02dddd36 in _Jv_InitClass () from /usr/lib/libgcj.so.8rh
 > #6  0x00145a44 in
 > multishot.examples.RicochetApp.initialize(java.lang.String,
 > java.lang.String, int, java.lang.String)multishot.examples.RicochetApp
 > ()
 >    from /home/jhoffert/Ricochet/trimmed/c
 > ++-java/linux-jdk1.5/libricochet.so
 > #7  0x08048ec2 in main (argc=4, argv=0xbfca39e4) at
 > RicochetAppMain.cpp:95

You really did call JvCreateJavaVM() first, didn't you?

 > Is there I'm not doing when calling JvCreateJavaVM? I called this
 > initially with an empty JvVMInitArgs argument and the program
 > aborted so I added the -D option for the .jar file. This seems
 > redundant to me but I wasn't sure what else to try. This argument
 > is what I pass when I'm using the invocation API.
 > 
 > I appreciate your time but don't want to waste it. Please point me to
 > documentation, examples, or tutorials if you don't have time to keep
 > helping me.

You have the documentation.  :-)

When asking a question like this, you are far more likely to get an
answer if you submit a runnable test case.  That way I can see what's
going wrong.  Right now, I have no idea.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Andrew Haley]

jhoffert@dre.vanderbilt.edu writes:
 > 
 > > When asking a question like this, you are far more likely to get an
 > > answer if you submit a runnable test case.  That way I can see what's
 > > going wrong.  Right now, I have no idea.
 > 
 > I've attached a zip file with my code. The example is about as small as
 > what I sent you previously. (In fact, I made only some minor changes from
 > what I sent you earlier, e.g., hard coding command line options). The
 > commands that I used for building are in the file gcj-commands.
 > 
 > I think I'm doing everything I'm supposed to be doing but the call to
 > JvInitClass(&multishot::examples::RicochetApp::class$) seems to be
 > failing. I'm not sure why.
 > 
 > I do get the following message when I link with gcj (rather than g++):
 > 
 > cc1plus: warning: command line option
 > "-fbootclasspath=./:/usr/share/java/libgcj-4.1.2.jar" is valid for Java
 > but not for C++
 > 
 > I assuming this message isn't important but I may be mistaken.
 
No, it isn't important.

The problem occurs when your program calls
java.lang.System.loadLibrary(java.lang.String)void with the arg
"ricochet-jni".  The library isn't there, so System.loadLibrary fails.
I don't know why it doesn't print a stack trace at that point: it
should.  I'll investigate.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[Andrew Haley]

Andrew Haley writes:
 
 > 
 > The problem occurs when your program calls
 > java.lang.System.loadLibrary(java.lang.String)void with the arg
 > "ricochet-jni".  The library isn't there, so System.loadLibrary fails.
 > I don't know why it doesn't print a stack trace at that point: it
 > should.  I'll investigate.

Ha!  Stupid me!  

Yes, I do know why your stack trace isn't being printed: you need to
wrap your entire main in a handler, like this:

  try
    {

      ...

      ra->run ();
    }
  catch (::java::lang::Throwable *ex)
    {
      ex->printStackTrace ();
    }

You also need

      JvAttachCurrentThread (NULL, NULL);

That's in the docs, too.

You'll then get

zorro:tmp $ LD_LIBRARY_PATH=.:/home/aph/gcc/trunk/install/lib64/:. ./RicochetApp
Status from JvCreateJavaVM call is 0
Before call to JvInitClass
java.lang.UnsatisfiedLinkError: libricochet-jni: libricochet-jni.so: cannot open shared object file: No such file or directory
   at java.lang.Runtime._load(natRuntime.cc:188)
   at java.lang.Runtime.loadLibrary(Runtime.java:655)
   at java.lang.System.loadLibrary(System.java:608)
   at multishot.examples.RicochetApp.<clinit>(libricochet.so)
   at java.lang.Class.initializeClass(natClass.cc:780)

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

Re: GCJ .jar to .so with native method

[jhoffert]

Thanks, Andrew! This helps greatly!

> Andrew Haley writes:
>
>  >
>  > The problem occurs when your program calls
>  > java.lang.System.loadLibrary(java.lang.String)void with the arg
>  > "ricochet-jni".  The library isn't there, so System.loadLibrary fails.
>  > I don't know why it doesn't print a stack trace at that point: it
>  > should.  I'll investigate.
>
> Ha!  Stupid me!
>
> Yes, I do know why your stack trace isn't being printed: you need to
> wrap your entire main in a handler, like this:
>
>   try
>     {
>
>       ...
>
>       ra->run ();
>     }
>   catch (::java::lang::Throwable *ex)
>     {
>       ex->printStackTrace ();
>     }
>
> You also need
>
>       JvAttachCurrentThread (NULL, NULL);
>
> That's in the docs, too.
>
> You'll then get
>
> zorro:tmp $ LD_LIBRARY_PATH=.:/home/aph/gcc/trunk/install/lib64/:.
> ./RicochetApp
> Status from JvCreateJavaVM call is 0
> Before call to JvInitClass
> java.lang.UnsatisfiedLinkError: libricochet-jni: libricochet-jni.so:
> cannot open shared object file: No such file or directory
>    at java.lang.Runtime._load(natRuntime.cc:188)
>    at java.lang.Runtime.loadLibrary(Runtime.java:655)
>    at java.lang.System.loadLibrary(System.java:608)
>    at multishot.examples.RicochetApp.<clinit>(libricochet.so)
>    at java.lang.Class.initializeClass(natClass.cc:780)
>
> Andrew.
>
> --
> Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor,
> Berkshire, SL4 1TE, UK
> Registered in England and Wales No. 3798903
>

Re: GCJ .jar to .so with native method

[Joe Hoffert]

Hi, Andrew.

You had helped me very much last month trying to integrate Java and C++
using gcj/gcjh. I've run into problems (both using the gcj/gcjh .so
approach and the Java Invocation API approach). I can present details
about the specific problems but I'm wondering if you knew of problems in
general with generating a Java library using JDK 1.5 and then using
that .jar file with g++/gcj 4.1.2.

I needed to use both JDK and gcj since the Java source code uses
generics which gcj won't compile and I'm using g++ for my C++ compiler.
I had problems loading and calling the Java library using JDK 1.6 and g
++/gcj and then switched to compiling with JDK 1.5. This got me to my
present situation of segmentation errors showing up.

I'm suspecting the mixing of JDK and g++/gcj as a problem area because
of the problems I saw using JDK 1.6 and also because of the following. I
have C++ code that uses the Java Invocation API to load the Java library
and make calls into it. The Java library also has a native method that
calls back into C (and then C++).

However, I get strange segmentation errors from the JVM with the C++
code that is called back from the Java library. I have the C++ code
running in its own thread (spawned after the native method is invoked)
with the stack trace from the JVM crash showing all C/C++ code. I am
puzzled why the JVM would crash with a call stack like this. That's why
I'm suspecting a larger problem of JDK and g++/gcj not working well
together. Do you have experience with these types of problems?

I can provide more detail if that's helpful but I want to see if someone
else knows of problems mixing JDK 1.5 and g++/gcj 4.1.2 before I spend
any more time looking into this. Do you know of problems mixing JDK and
gcj?

Thank you very much for all your help!

-Joe

Re: GCJ .jar to .so with native method

[Andrew Haley]

Joe Hoffert writes:

 > You had helped me very much last month trying to integrate Java and C++
 > using gcj/gcjh. I've run into problems (both using the gcj/gcjh .so
 > approach and the Java Invocation API approach). I can present details
 > about the specific problems but I'm wondering if you knew of problems in
 > general with generating a Java library using JDK 1.5 and then using
 > that .jar file with g++/gcj 4.1.2.
 > 
 > I needed to use both JDK and gcj since the Java source code uses
 > generics which gcj won't compile and I'm using g++ for my C++ compiler.

Ouch.  GCC trunk is much better at this now.

 > I had problems loading and calling the Java library using JDK 1.6 and g
 > ++/gcj and then switched to compiling with JDK 1.5. This got me to my
 > present situation of segmentation errors showing up.

OK.

 > I'm suspecting the mixing of JDK and g++/gcj as a problem area because
 > of the problems I saw using JDK 1.6 and also because of the following. I
 > have C++ code that uses the Java Invocation API to load the Java library
 > and make calls into it. The Java library also has a native method that
 > calls back into C (and then C++).

 > However, I get strange segmentation errors from the JVM with the
 > C++ code that is called back from the Java library. I have the C++
 > code running in its own thread (spawned after the native method is
 > invoked) with the stack trace from the JVM crash showing all C/C++
 > code. I am puzzled why the JVM would crash with a call stack like
 > this. That's why I'm suspecting a larger problem of JDK and g++/gcj
 > not working well together. Do you have experience with these types
 > of problems?

Yeah.  The question is, who created the thread?  If it was created by
C++ then the JVM won't know about it and the gc won't know about it.
This is a Bad Thing, and will lead to tears.

See the manual:

 -- Function: java::lang::Thread* JvAttachCurrentThread (jstring NAME,
          java::lang::ThreadGroup* GROUP)
     Registers an existing thread with the Java runtime.  This must be
     called once from each thread, before that thread makes any other
     Java or CNI calls. It must be called after `JvCreateJavaVM'.  NAME
     specifies a name for the thread. It may be `NULL', in which case a
     name will be generated.  GROUP is the ThreadGroup in which this
     thread will be a member. If it is `NULL', the thread will be a
     member of the main thread group.  The return value is the Java
     `Thread' object that represents the thread.  It is safe to call
     `JvAttachCurrentThread()' more than once from the same thread. If
     the thread is already attached, the call is ignored and the current
     thread object is returned.

 -- Function: jint JvDetachCurrentThread ()
     Unregisters a thread from the Java runtime. This should be called
     by threads that were attached using `JvAttachCurrentThread()',
     after they have finished making calls to Java code. This ensures
     that any resources associated with the thread become eligible for
     garbage collection.  This function returns `0' upon success, or
     `-1' if the current thread is not attached.

What you are doing sounds basically OK, but mixing Java API versions
probably isn't a good idea.  If I were you I'd grab a prerelease gcj
snapshot and use that.  This should give you much better compatibility
with code written for the Java 1.5 API. 

 > I can provide more detail if that's helpful but I want to see if
 > someone else knows of problems mixing JDK 1.5 and g++/gcj 4.1.2
 > before I spend any more time looking into this. Do you know of
 > problems mixing JDK and gcj?

Usually it "just works", but you are trying to do something pretty
complex with this mixture of languages and versions.  I'm surprised it
even links: there are a lot of methods in the Java 1.5 API that are
missing in the version of gcj you're using.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903