CNI and interface methods

CNI and interface methods

[Stephen Kell]

Hi,

I'm using CNI for the first time and have hit a query. What exactly is 
and isn't supposed to work when invoking interface methods through 
*object* references (from C++, so I mean pointers), and passing these 
references around?

The CNI chapter in the gcj manual says a lot about interface references, 
but not about the case of using object references where that object's 
class implements one or more interfaces. One thing that would appear to 
be worth mentioning is that clearly, passing them to methods wanting an 
interface reference won't compile without some sort of cast, since the 
gcjh-generated headers don't encode the subtyping relationship between 
interfaces and their implementing classes. I'm guessing a simple static 
cast isn't the right thing to do though, because I'm getting segfaults 
when I try.

Specifically, I'm CNI-calling from C++ to Java code which, in turn, 
makes a call through an interface reference that I previously passed in 
to the Java object's constructor (i.e. I passed in an *object* 
reference, again from the C++ code, which required a cast as I just 
described). I get a segfault during the calling sequence of the 
interface call made by the Java code. The fault occurs just before the 
call to _Jv_LookupInterfaceMethodIdx.

(This is all on a fresh build of vanilla gcc 4.3.3, on i686 Linux, by 
the way.)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7f5e8f0 (LWP 26704)]
0x080828ad in cakeJavaParser.toplevel()cakeJavaParser$toplevel_return () 
at cakeJavaParser.java:145
145             retval.start = input.LT(1);
Current language:  auto; currently java
(gdb) bt
#0  0x080828ad in 
cakeJavaParser.toplevel()cakeJavaParser$toplevel_return ()
    at cakeJavaParser.java:145
#1  0x0805a4e2 in cake::request::process (this=@bffd0c18) at cake.cpp:33
#2  0x0805a7da in main (argc=2, argv=@bffd0cd4) at main.cpp:52

(Possibly significant: in earlier test runs, with a previous build of my 
code, I was getting the segfault *during* the call to 
_Jv_LookupInterfaceMethodIdx, with iface->ioffsets being null when it 
probably shouldn't. I can't reproduce that now for some reason.)

The extent of my CNI code is

	JvCreateJavaVM(NULL);
	JvAttachCurrentThread(NULL, NULL);
	JvInitClass(&java::lang::System::class$);

and shortly afterwards

	org::antlr::runtime::ANTLRInputStream *stream = new org::antlr::runtime::ANTLRInputStream(in_file);
	cakeJavaLexer *lexer = new cakeJavaLexer((org::antlr::runtime::CharStream*) stream);
    org::antlr::runtime::CommonTokenStream *tokenStream = new org::antlr::runtime::CommonTokenStream((org::antlr::runtime::TokenSource*) lexer);
    cakeJavaParser *parser = new cakeJavaParser((org::antlr::runtime::TokenStream*) tokenStream);
    parser->toplevel();
	
where the last statement triggers the segfault. The toplevel() method is 
Java code; it immediately calls a method on lexer (the object I passed 
earlier) and it's this call whose dispatch fails.

Should this code work?  I thought about class initialization, but having 
read the docs, it wouldn't appear that I need to add any JvInitClass 
calls manually in this case. 

There's a tarball at 
<http://www.cl.cam.ac.uk/~srk31/private/cake-snapshot-20090402.tar.bz2> 
which should be sufficient to reproduce the above (just do 'make test'), 
unless I've forgotten to include something. Let me know if there's any 
more details that would be useful.

Thanks for reading! Any answers or ideas appreciated,

Stephen.

Re: CNI and interface methods

[Andrew Haley]

Stephen Kell wrote:

> I'm using CNI for the first time and have hit a query. What exactly is 
> and isn't supposed to work when invoking interface methods through 
> *object* references (from C++, so I mean pointers), and passing these 
> references around?
> 
> The CNI chapter in the gcj manual says a lot about interface references, 
> but not about the case of using object references where that object's 
> class implements one or more interfaces. One thing that would appear to 
> be worth mentioning is that clearly, passing them to methods wanting an 
> interface reference won't compile without some sort of cast, since the 
> gcjh-generated headers don't encode the subtyping relationship between 
> interfaces and their implementing classes. I'm guessing a simple static 
> cast isn't the right thing to do though, because I'm getting segfaults 
> when I try.

Right.  To put it simply, we have never got around to implementing Java
interface calls in CNI.  It wouldn't be enormously difficult, but it
needs some way to tell the C++ compiler about interface calls.  We could
have implemented Java interfaces via C++ multiple inheritance but we
didn't, so we would need to invent some sort of notation to express the
idea of an interface to the C++ compiler.

> Specifically, I'm CNI-calling from C++ to Java code which, in turn, 
> makes a call through an interface reference that I previously passed in 
> to the Java object's constructor (i.e. I passed in an *object* 
> reference, again from the C++ code, which required a cast as I just 
> described). I get a segfault during the calling sequence of the 
> interface call made by the Java code. The fault occurs just before the 
> call to _Jv_LookupInterfaceMethodIdx.
> 
> (This is all on a fresh build of vanilla gcc 4.3.3, on i686 Linux, by 
> the way.)
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xb7f5e8f0 (LWP 26704)]
> 0x080828ad in cakeJavaParser.toplevel()cakeJavaParser$toplevel_return () 
> at cakeJavaParser.java:145
> 145             retval.start = input.LT(1);
> Current language:  auto; currently java
> (gdb) bt
> #0  0x080828ad in 
> cakeJavaParser.toplevel()cakeJavaParser$toplevel_return ()
>     at cakeJavaParser.java:145
> #1  0x0805a4e2 in cake::request::process (this=@bffd0c18) at cake.cpp:33
> #2  0x0805a7da in main (argc=2, argv=@bffd0cd4) at main.cpp:52
> 
> (Possibly significant: in earlier test runs, with a previous build of my 
> code, I was getting the segfault *during* the call to 
> _Jv_LookupInterfaceMethodIdx, with iface->ioffsets being null when it 
> probably shouldn't. I can't reproduce that now for some reason.)
> 
> The extent of my CNI code is
> 
> 	JvCreateJavaVM(NULL);
> 	JvAttachCurrentThread(NULL, NULL);
> 	JvInitClass(&java::lang::System::class$);
> 
> and shortly afterwards
> 
> 	org::antlr::runtime::ANTLRInputStream *stream = new org::antlr::runtime::ANTLRInputStream(in_file);
> 	cakeJavaLexer *lexer = new cakeJavaLexer((org::antlr::runtime::CharStream*) stream);
>     org::antlr::runtime::CommonTokenStream *tokenStream = new org::antlr::runtime::CommonTokenStream((org::antlr::runtime::TokenSource*) lexer);
>     cakeJavaParser *parser = new cakeJavaParser((org::antlr::runtime::TokenStream*) tokenStream);
>     parser->toplevel();
> 	
> where the last statement triggers the segfault. The toplevel() method is 
> Java code; it immediately calls a method on lexer (the object I passed 
> earlier) and it's this call whose dispatch fails.
> 
> Should this code work?  I thought about class initialization, but having 
> read the docs, it wouldn't appear that I need to add any JvInitClass 
> calls manually in this case. 
> 
> There's a tarball at 
> <http://www.cl.cam.ac.uk/~srk31/private/cake-snapshot-20090402.tar.bz2> 
> which should be sufficient to reproduce the above (just do 'make test'), 
> unless I've forgotten to include something. Let me know if there's any 
> more details that would be useful.
> 
> Thanks for reading! Any answers or ideas appreciated,

It's because you're using -Bsymbolic but not -fpic.  -Bsymbolic is dangerous,
and any use of it without PIC means instant death.  So, compile everything
PIC, even the executable, and you'll be fine.  That worked for me on your
example.

Andrew.

Re: CNI and interface methods

[Bryce McKinlay]

On Fri, Apr 3, 2009 at 9:37 AM, Andrew Haley <aph@redhat.com> wrote:
> Stephen Kell wrote:
>
>> I'm using CNI for the first time and have hit a query. What exactly is
>> and isn't supposed to work when invoking interface methods through
>> *object* references (from C++, so I mean pointers), and passing these
>> references around?
>>
>> The CNI chapter in the gcj manual says a lot about interface references,
>> but not about the case of using object references where that object's
>> class implements one or more interfaces. One thing that would appear to
>> be worth mentioning is that clearly, passing them to methods wanting an
>> interface reference won't compile without some sort of cast, since the
>> gcjh-generated headers don't encode the subtyping relationship between
>> interfaces and their implementing classes. I'm guessing a simple static
>> cast isn't the right thing to do though, because I'm getting segfaults
>> when I try.
>
> Right.  To put it simply, we have never got around to implementing Java
> interface calls in CNI.  It wouldn't be enormously difficult, but it
> needs some way to tell the C++ compiler about interface calls.  We could
> have implemented Java interfaces via C++ multiple inheritance but we
> didn't, so we would need to invent some sort of notation to express the
> idea of an interface to the C++ compiler.

Actually we did implement a rather limited form of interface calls in
CNI. Interfaces types are described in CNI headers with __attribute__
((java_interface)), and the C++ compiler knows how to call a method on
a type declared as such.

What is missing in the C++ compiler (and the CNI headers) is knowledge
of interface inheritance, so you have to manually cast interface
references if the method you want to call was declared in a
super-interface.

This limitation is described here:

http://gcc.gnu.org/onlinedocs/gcj/Interfaces.html

Bryce

Re: CNI and interface methods

[Andrew Haley]

Bryce McKinlay wrote:
> On Fri, Apr 3, 2009 at 9:37 AM, Andrew Haley <aph@redhat.com> wrote:
> 
>> Stephen Kell wrote:
>>
>>> I'm using CNI for the first time and have hit a query. What exactly is
>>> and isn't supposed to work when invoking interface methods through
>>> *object* references (from C++, so I mean pointers), and passing these
>>> references around?
>>>
>>> The CNI chapter in the gcj manual says a lot about interface references,
>>> but not about the case of using object references where that object's
>>> class implements one or more interfaces. One thing that would appear to
>>> be worth mentioning is that clearly, passing them to methods wanting an
>>> interface reference won't compile without some sort of cast, since the
>>> gcjh-generated headers don't encode the subtyping relationship between
>>> interfaces and their implementing classes. I'm guessing a simple static
>>> cast isn't the right thing to do though, because I'm getting segfaults
>>> when I try.
>
>> Right.  To put it simply, we have never got around to implementing Java
>> interface calls in CNI.  It wouldn't be enormously difficult, but it
>> needs some way to tell the C++ compiler about interface calls.  We could
>> have implemented Java interfaces via C++ multiple inheritance but we
>> didn't, so we would need to invent some sort of notation to express the
>> idea of an interface to the C++ compiler.
> 
> Actually we did implement a rather limited form of interface calls in CNI.
> Interfaces types are described in CNI headers with __attribute__
> ((java_interface)), and the C++ compiler knows how to call a method on a
> type declared as such.
> 
> What is missing in the C++ compiler (and the CNI headers) is knowledge of
> interface inheritance, so you have to manually cast interface references if
> the method you want to call was declared in a super-interface.
> 
> This limitation is described here:
> 
> http://gcc.gnu.org/onlinedocs/gcj/Interfaces.html

Ah yes, thanks for the clarification.

Andrew.

Re: CNI and interface methods

[Stephen Kell]

On Fri, 03 Apr 2009 09:37:40 +0100, Andrew Haley wrote:
> It's because you're using -Bsymbolic but not -fpic.  -Bsymbolic is
> dangerous, and any use of it without PIC means instant death.  So,
> compile everything PIC, even the executable, and you'll be fine.
> That worked for me on your example.

Thanks for this. I thought I already was compiling everything -fpic,
but maybe I missed something. Was there a particular make rule that was
in need of -fpic, that you altered?  I still can't find it.

(Actually I was using -fPIC, but I've tried again with -fpic instead, in
case it made a difference... still getting the same segfault though.)

Stephen

Re: CNI and interface methods

[Andrew Haley]

Stephen Kell wrote:
> On Fri, 03 Apr 2009 09:37:40 +0100, Andrew Haley wrote:
>> It's because you're using -Bsymbolic but not -fpic.  -Bsymbolic is
>> dangerous, and any use of it without PIC means instant death.  So,
>> compile everything PIC, even the executable, and you'll be fine.
>> That worked for me on your example.
> 
> Thanks for this. I thought I already was compiling everything -fpic,
> but maybe I missed something. Was there a particular make rule that was
> in need of -fpic, that you altered?  I still can't find it.
> 
> (Actually I was using -fPIC, but I've tried again with -fpic instead, in
> case it made a difference... still getting the same segfault though.)

Here are my changes.  I also got rid of antlr-runtime which didn't seem
necessary.

Andrew.


--- Makefile~   2009-04-02 19:31:29.000000000 +0100
+++ Makefile    2009-04-03 09:31:06.000000000 +0100
@@ -16,13 +16,13 @@
 default: cake

 %.o: %.cpp
-       g++ -c -o "$@" ${CXXFLAGS} "$<"
+       g++ -c -fpic -o "$@" ${CXXFLAGS} "$<"

 main.o: main.cpp parser-recursive Makefile antlr-headers
-       g++ -c -o "$@" ${CXXFLAGS} "$<"
+       g++ -c -fpic -o "$@" ${CXXFLAGS} "$<"

-cake: main.o $(OBJS) antlr-runtime-3.1.jar.so stringtemplate-3.2.jar.so antlr-3.1.2.jar.so junit-3.8.2.jar.so
-       gcj $(GCJFLAGS) -Wl,-R. -o "$@" $(OBJS) "$<" -lstdc++ parser/cakeJava{Parser,Lexer}*.o antlr-runtime-3.1.jar.so stringtemplate-3.2.jar.so antlr-3.1.2.jar.so junit-3.8.2.jar.so
+cake: main.o $(OBJS) stringtemplate-3.2.jar.so antlr-3.1.2.jar.so junit-3.8.2.jar.so
+       gcj $(GCJFLAGS) -Wl,-R. -o "$@" $(OBJS) "$<" -lstdc++ parser/cakeJava{Parser,Lexer}*.o stringtemplate-3.2.jar.so antlr-3.1.2.jar.so junit-3.8.2.jar.so

 # FIXME: to build, change the following .jar paths

@@ -41,14 +41,14 @@
                org.antlr.runtime.IntStream

 %.jar.so: %.jar
-       gcj $(GCJFLAGS) -shared -fPIC -Wl,-Bsymbolic "$<" -o "$@"
+       gcj $(GCJFLAGS) -shared -g -fPIC -Wl,-Bsymbolic "$<" -o "$@"

 #java-headers:
 #      gcjh -d gcjh_generated -classpath .:$(CLASSPATH) java.io.File java.io.FileInputStream

 .PHONY: parser-recursive
 parser-recursive:
-       $(MAKE) -C parser
+       $(MAKE) -C parser CLASSPATH=$(CLASSPATH)

 .PHONY: clean
 clean:
--- parser/Makefile~    2009-04-02 19:32:24.000000000 +0100
+++ parser/Makefile     2009-04-03 09:31:06.000000000 +0100
@@ -33,14 +33,14 @@
        $(ANTLR) "$<"

 %.class: %.java
-       gcj -MD -C -Wno-unused "$<"
+       gcj -g -MD -C -Wno-unused "$<"

 %.o: %.class
-       gcj $(JFLAGS) -MD -Wno-unused -c -o "$@" "$<"
+       gcj $(JFLAGS) -MD -Wno-unused -c -o "$@" "$<" -g -fpic
 # HACK: also make nested classes' object files, because we can't enumerate these statically for Makefile purposes
        for classfile in "$*"\$$*.class; do \
                if [[ $$classfile != "$*"\$$"*".class ]]; then \
-                       gcj -o $$( echo "$$classfile" | sed 's/class$$/o/') -MD -Wno-unused -c "$$classfile"; \
+                       gcj -o $$( echo "$$classfile" | sed 's/class$$/o/') -MD -Wno-unused -fpic -c "$$classfile"; \
                fi; \
        done

Re: CNI and interface methods

[Stephen Kell]

> > (Actually I was using -fPIC, but I've tried again with -fpic
> > instead, in case it made a difference... still getting the same
> > segfault though.)
> 
> Here are my changes.  I also got rid of antlr-runtime which didn't
> seem necessary.

Thanks again!  At first I thought I'd made identical fixes already, and
the segfaults were still happening. But getting rid of antlr-runtime is
the key -- turns out there are some symbols in common between
antlr-runtime and antlr, and I'm guessing that some version skew
between those was causing the problem. Phew!

Stephen

Re: CNI and interface methods

[Stephen Kell]

> Actually we did implement a rather limited form of interface calls in
> CNI. Interfaces types are described in CNI headers with __attribute__
> ((java_interface)), and the C++ compiler knows how to call a method
> on a type declared as such.
> 
> What is missing in the C++ compiler (and the CNI headers) is
> knowledge of interface inheritance, so you have to manually cast
> interface references if the method you want to call was declared in a
> super-interface.
> 
> This limitation is described here:
> 
> http://gcc.gnu.org/onlinedocs/gcj/Interfaces.html

Thanks for this. I'm fine with that limitation, but there seems to be a
second thing missing too (missing in the same sense, i.e. that it
requires casts that ideally wouldn't be there). The question was: from
the C++ side, given a pointer p to some object implementing interface
J, is it safe to pass that pointer to a Java method who CNI prototype
looks like, for example,

void foo(J *arg);

or not? Clearly from C++ we can't do

foo(p);

because it won't type-check; but we can do the following.

foo((J*) p);

It now appears the answer is "yes, this is okay" (whereas I'd been
worried that maybe some multiple-inheritance-style pointer adjustment
was not being done and was causing the segfaults I was seeing). 

The CNI docs should probably say that these casts are fine and
indeed required. I'll gladly submit a small patch to the docs if you
agree (and let me know where's best for me to send it).

Stephen

Re: CNI and interface methods

[David Daney]

Stephen Kell wrote:

> (whereas I'd been
> worried that maybe some multiple-inheritance-style pointer adjustment
> was not being done and was causing the segfaults I was seeing). 
> 

GCJ doesn't use C++ style multiple inheritance to implement interfaces. 
  The initialization of the interface dispatch tables is done at runtime 
by some somewhat trick code.

Actually I am surprised that you can call interface methods from CNI. 
You learn something new everyday.

David Daney

Re: CNI and interface methods

[Andrew Haley]

Stephen Kell wrote:
>> Actually we did implement a rather limited form of interface calls in
>> CNI. Interfaces types are described in CNI headers with __attribute__
>> ((java_interface)), and the C++ compiler knows how to call a method
>> on a type declared as such.
>>
>> What is missing in the C++ compiler (and the CNI headers) is
>> knowledge of interface inheritance, so you have to manually cast
>> interface references if the method you want to call was declared in a
>> super-interface.
>>
>> This limitation is described here:
>>
>> http://gcc.gnu.org/onlinedocs/gcj/Interfaces.html
> 
> Thanks for this. I'm fine with that limitation, but there seems to be a
> second thing missing too (missing in the same sense, i.e. that it
> requires casts that ideally wouldn't be there). The question was: from
> the C++ side, given a pointer p to some object implementing interface
> J, is it safe to pass that pointer to a Java method who CNI prototype
> looks like, for example,
> 
> void foo(J *arg);
> 
> or not? Clearly from C++ we can't do
> 
> foo(p);
> 
> because it won't type-check; but we can do the following.
> 
> foo((J*) p);
> 
> It now appears the answer is "yes, this is okay" (whereas I'd been
> worried that maybe some multiple-inheritance-style pointer adjustment
> was not being done and was causing the segfaults I was seeing). 
> 
> The CNI docs should probably say that these casts are fine and
> indeed required. I'll gladly submit a small patch to the docs if you
> agree (and let me know where's best for me to send it).

java-patches is the best place to send it.  In this case, an example is
worth a thousand words.  If you could write a reasonably small one to
insert in your description, that would be great.

Please don't write much, or we'd need to ask you for a copyright
assignment.

Andrew.

Re: CNI and interface methods

[Tom Tromey]

>>>>> "Oskar" == Oskar Liljeblad <osk@hem.passagen.se> writes:

Oskar> I read in the CNI documentation that it is not possible to call
Oskar> methods using interface references. Do I have an old egcs/gcj
Oskar> (2.95.2)? If not, any ideas when this will this be possible?

This hasn't changed -- there is no way to directly make a call from
C++ via an interface reference.  As far as I know, nobody is working
on this, so I don't know when it will be possible.

Oskar> Are there any tricks to use meanwhile? I'm experimenting with
Oskar> Gtk bindings using CNI, and using interfaces for signal
Oskar> listeners is (usually) necessary.

Yeah, I like the signal/interface approach too.  Are you generating
the bindings automatically from gtk.def?

Anyway, you can use _Jv_LookupInterfaceMethod to look up the method
via an interface, and then call it directly (you get back a raw
pointer which you can treat as a function pointer; be sure to pass the
object reference as the first argument).  If you do this, though, be
warned that this interface isn't really exported, and we might change
it randomly.

Tom

CNI and interface methods

[Oskar Liljeblad]

I read in the CNI documentation that it is not possible
to call methods using interface references. Do I have an
old egcs/gcj (2.95.2)? If not, any ideas when this will
this be possible?

Are there any tricks to use meanwhile? I'm experimenting
with Gtk bindings using CNI, and using interfaces for signal
listeners is (usually) necessary.

Thanks

Oskar Liljeblad (osk@hem.passagen.se)