Re: Receiver class does not define or inherit an implementation of the resolved method

[spellcard199 <spellcard199@protonmail.com>]

> Looking at Java source files feels like a weird thing to do.

> The problem is that Jooby does _Java_ source code analysis (not
> reflection). It literally parses your handler source code to figure
> out what the return type actually is (since the Object return type
> masks it). It only does this though if there's not a method it can
> introspect without source code analysis.

Today I tried to read the Joobly code backward from the point where
the error happens and in the code path that affects us it reads our
"apply" method's bytecode using the asm library.

I understand what you are trying to do is to avoid this code path
altogether, but I think there is a less bad approach to the problem
than having 2 methods with the same name and parameter types.

The reason why asm fails to read the "apply" method defined from Kawa
is it searches our object's class, which implements Route.Handler, in
the wrong ClassLoader.

Let's follow where the ClassLoader asm receives comes from.

the Jooby object gets its own classLoader
https://github.com/jooby-project/jooby/blob/f47eda4500bc4b76b23d24d4d77aa2ab3cc19e95/jooby/src/main/java/io/jooby/Jooby.java#L129

the Jooby object passes classLoader to a RouterImpl object
https://github.com/jooby-project/jooby/blob/f47eda4500bc4b76b23d24d4d77aa2ab3cc19e95/jooby/src/main/java/io/jooby/Jooby.java#L131

the RouterImpl object passes classLoader to a ClassSource object,
which is basically a wrapper for storing the ClassLoader and calling
classLoader.getResourceAsStream(...) when needed.
https://github.com/jooby-project/jooby/blob/f47eda4500bc4b76b23d24d4d77aa2ab3cc19e95/jooby/src/main/java/io/jooby/internal/RouterImpl.java#L538

the ClassSource object is passed to a RouteAnalyzer object
https://github.com/jooby-project/jooby/blob/f47eda4500bc4b76b23d24d4d77aa2ab3cc19e95/jooby/src/main/java/io/jooby/internal/RouterImpl.java#L539

the RouteAnalyzer object:
1. Gets the java.lang.reflect.Method corresponding to our
RouteHandler's "apply" method.
https://github.com/jooby-project/jooby/blob/f47eda4500bc4b76b23d24d4d77aa2ab3cc19e95/jooby/src/main/java/io/jooby/internal/RouteAnalyzer.java#L36
2. Gets "apply"'s class calling method.getDeclaringClass() on the
Method it just got
3. Searches for the class it just got in classLoader: and here the
error raises. This classLoader comes from
Joobly.getClass().getClassloader(), but our class is in Kawa's
classloader.
https://github.com/jooby-project/jooby/blob/f47eda4500bc4b76b23d24d4d77aa2ab3cc19e95/jooby/src/main/java/io/jooby/internal/RouteAnalyzer.java#L44

Maybe RouteAnalyzer.returnType should use
method.getDeclaringClass().getClassLoader() as "source" (where
"source" is a ClassLoader) instead of this.classLoader? To confirm
this I tried to...:
1. clone the Jooby repo
2. Replace line 44 in RouteAnalyzer:
- from:

ClassReader reader = new ClassReader(source.byteCode(method.getDeclaringClass()));

- to:

ClassSource classSourceFix = new ClassSource(cl);
ClassReader reader = new ClassReader(classSourceFix.byteCode(method.getDeclaringClass()));

3. Add Kawa in the dependencies for tests in pom.xml
4. Add a new test that starts a remote Kawa repl
5. Run new test and connect to the remote Kawa repl
5. Evaluate a minimal jooby example
... and it worked, so this a possible starting point for a
solution. But maybe that could create new errors in some other
scenarios? And BTW, in the next release of Jooby they are removing
asm: https://github.com/jooby-project/jooby/issues/1712

Here's a dirty hack for a working minimal example in Kawa that doesn't
require patching Joobly (you can copy and paste it directly into the
kawa/main.scm in your repo, replacing its current contents):

#+BEGIN_SRC scheme

(define rh ::<io.jooby.Route$Handler>
(object (io.jooby.Route$Handler)
;; ((apply) ::<java.lang.String> ("empy":toString))
((apply (ctx ::<io.jooby.Context>)) ::<java.lang.Object>
throws: (java.lang.Exception)
(java.lang.String "Hello!"))))

(define app ::<io.jooby.Jooby> (io.jooby.Jooby))
;; Dirty hack to replace app.router with another one that uses Kawa's
;; classloader instead of app.getClass().getClassLoader().
(define routerField ::<java.lang.reflect.Field>
(invoke (app:getClass) 'getDeclaredField "router"))
(routerField:setAccessible #t)
(routerField:set app (io.jooby.internal.RouterImpl
(invoke (rh:getClass) 'getClassLoader)))
(routerField:setAccessible #f)

;; app.route adds a Route$Handler to app.router.routes
(app:route "GET" "/greeting" rh)

;; Start server
(define server ::<io.jooby.netty.Netty> (io.jooby.netty.Netty))
(server:setOptions ((io.jooby.ServerOptions):setPort 8080))
(server:start app)
(server:join)

#+END_SRC

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, September 7th, 2021 at 9:35 PM, Phil Eaton <phil@eatonphil.com> wrote:

>> Since it is a Java library and,
> as I supposed, Java does not allow to have 2 methods with the same
> name and parameter types [1][2], if I were in you I would first try to
> translate your example to plain Java without Jooby's annotations (I
> don't know how to do it). Maybe asking the people working on Jooby may
> be an option?
>
> The problem is that Jooby does _Java_ source code analysis (not reflection). It literally parses your handler source code to figure out what the return type actually is (since the Object return type masks it). It only does this though if there's not a method it can introspect without source code analysis.
>
> I spent a while digging around Jooby's APIs and this was the only way I could shoehorn non-Java code into it.
>
> Since I put decent amount of effort into the ABCL code (needed to submit a patch to the ABCL team) I am reluctant to give up now in Kawa without finding a way to hack this up.
>
> Furthermore the problem is that almost every hip Java API is designed like this. They are very unfriendly to function-oriented paradigms.
>
> Now that I think of it though maybe using Kawa's class annotation support will help me fight less with Jooby since it does want you to annotate classes (which I was ignoring because I was trying to use classes as little as possible).
>
> On Tue, Sep 7, 2021 at 1:58 PM spellcard199 <spellcard199@protonmail.com> wrote:
>
>> I cloned your repo, cd'ed into abcl, run mvn install and then make. I
>> still had to copy-paste your main.lisp in the repl but it
>> worked. After commenting out each of the 2 apply methods I could see
>> what problem you have.
>>
>> If I understand correctly, basically what you are trying to do is
>> replicate at a language level what Jooby already does in pure
>> Java.
>>
>> My opinion as a not-experienced programmer that has never used Jooby
>> is that there may be some other way to use the Jooby api. Why
>> re-solving with a language-specific solution a problem that somewhere
>> in Jooby has already been implemented? Since it is a Java library and,
>> as I supposed, Java does not allow to have 2 methods with the same
>> name and parameter types [1][2], if I were in you I would first try to
>> translate your example to plain Java without Jooby's annotations (I
>> don't know how to do it). Maybe asking the people working on Jooby may
>> be an option?
>>
>>> I guess that's not the place where the method name is generated for
>>> bytecode compilation. If anyone could point me at where that happens
>>> that would help.
>>
>> Just out of curiosity... I've not tested it, but I think it happens in
>> gnu.expr.LambdaExp.addMethodFor, around line 1144.
>>
>>> Thanks!
>>
>> np, but I wasn't very helpful =/.
>>
>> [1] https://stackoverflow.com/questions/2439782/overload-with-different-return-type-in-java
>> [2] https://stackoverflow.com/questions/5561436/can-two-java-methods-have-same-name-with-different-return-types
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Tuesday, September 7th, 2021 at 3:02 PM, Phil Eaton <phil@eatonphil.com> wrote:
>>
>>> Good question! I don't know how to get the ABCL-emitted bytecode otherwise I would have just posted that.
>>>
>>> But I can describe to you the behavior that makes me say it emits two functions with the same name.
>>>
>>> Here is my ABCL code: https://github.com/eatonphil/jvm-lisp-examples/blob/main/abcl/main.lisp#L22.
>>>
>>> In this code I define two methods of the same name. The "fake" method comes first. This is found when the jooby library does reflection! But it is not the one called by Jooby. The second method that returns an Object (the one that actually implements the interface) is the one that is called.
>>>
>>> Separately, I was messing around in Kawa trying to understand where this $X suffix is generated on duplicate methods. I did find fixParamNames in gnu/bytecode/Scope.java:110. It has a note saying that $X suffix was required because of _Android_/dex. I commented this method out and recompiled and ran my example but it still generated the $X suffix. So I guess that's not the place where the method name is generated for bytecode compilation. If anyone could point me at where that happens that would help.
>>>
>>> Thanks!
>>>
>>> On Tue, Sep 7, 2021 at 6:59 AM spellcard199 <spellcard199@protonmail.com> wrote:
>>>
>>>> Hello.
>>>>
>>>>> For what it's worth this kind of redefinition of the same method
>>>> worked for me in ABCL lisp. So I know it's definitely possible to
>>>> express in Java.
>>>>
>>>> I know nothing about ABCL so I trust you on the fact ABCL handles
>>>> this differently from Kawa, but I don't think in Java you do it.
>>>>
>>>> If I try to write the following class in plain Java...
>>>>
>>>> public class Main {
>>>> public static void main(String[] args) {
>>>> Main main = new Main();
>>>> System.out.println(main.apply("x"));
>>>> }
>>>> public java.lang.CharSequence apply(String s) {
>>>> return s.concat(s);
>>>> }
>>>> public java.lang.Object apply(String s) {
>>>> return s.concat(s).concat(s);
>>>> }
>>>> }
>>>>
>>>> ... it gives a compile time error:
>>>>
>>>> apply(String) is already defined in 'Main'
>>>>
>>>> And the same thing happens when the methods are static.
>>>>
>>>> I suppose in plain Java you can't have more than one method with both
>>>> the same:
>>>> - name
>>>> - input types
>>>>
>>>> I remember being confused by this the first time I saw it, but in
>>>> hindsight it makes sense: if there were 2 methods with the same name
>>>> and input types, how could Java know which one should be called when
>>>> applied to arguments?
>>>>
>>>> So my question is: does ABCL really let you have both methods at the
>>>> same time or just the lastly defined one?
>>>>
>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>>
>>>> On Tuesday, August 31st, 2021 at 1:18 AM, Phil Eaton <phil@eatonphil.com> wrote:
>>>>
>>>>> I spoke too soon. The issue is that I truly do need both of these methods
>>>>>
>>>>> to be called `apply` not `apply` and `apply$1`.
>>>>>
>>>>> For what it's worth this kind of redefinition of the same method worked for
>>>>>
>>>>> me in ABCL lisp. So I know it's definitely possible to express in Java.
>>>>>
>>>>> On Mon, Aug 30, 2021 at 7:09 PM Phil Eaton phil@eatonphil.com wrote:
>>>>>
>>>>> > Aha! It gave my second apply method a different suffix. By reordering
>>>>> >
>>>>> > these two methods the whole thing somehow works.
>>>>> >
>>>>> > $ javap main\$0.class
>>>>> >
>>>>> > Compiled from "main.scm"
>>>>> >
>>>>> > public class main$0 implements io.jooby.Route$Handler {
>>>>> >
>>>>> > main$frame this$0;
>>>>> >
>>>>> > public java.lang.CharSequence apply(io.jooby.Context);
>>>>> >
>>>>> > public java.lang.Object apply$1(io.jooby.Context);
>>>>> >
>>>>> > public main$0(main$frame);
>>>>> >
>>>>> > }
>>>>> >
>>>>> > It's weird but I'll take it.
>>>>> >
>>>>> > On Mon, Aug 30, 2021 at 12:40 PM Per Bothner per@bothner.com wrote:
>>>>> >
>>>>> > > On 8/29/21 12:03 PM, Phil Eaton wrote:> Still new to Kawa. I'm trying to
>>>>> > >
>>>>> > > implement an interface (
>>>>> > >
>>>>> > > > io.jooby.Route$Handler
>>>>> > > >
>>>>> > > > <
>>>>> > > >
>>>>> > > > https://github.com/jooby-project/jooby/blob/2.x/jooby/src/main/java/io/jooby/Route.java#L247
>>>>> > > >
>>>>> > > > ).
>>>>> > > >
>>>>> > > > It only has a single non-default method, apply.
>>>>> > > >
>>>>> > > > Here's what I've got
>>>>> > > >
>>>>> > > > (define (route app method path handler)
>>>>> > > >
>>>>> > > > (let ((handler (object (io.jooby.Route$Handler)
>>>>> > > >
>>>>> > > > #| This method exists just to stop Jooby from
>>>>> > > >
>>>>> > > > trying to introspect Java code that doesn't exist because this isn't
>>>>> > > >
>>>>> > > > written in Java. |#
>>>>> > > >
>>>>> > > > ((apply (ctx ::io.jooby.Context)) ::string
>>>>> > > >
>>>>> > > > #!null)
>>>>> > > >
>>>>> > > > ((apply (ctx ::io.jooby.Context))
>>>>> > > >
>>>>> > > > ::java.lang.Object
>>>>> > > >
>>>>> > > > (handler ctx)))))
>>>>> > > >
>>>>> > > > (app:route method path handler)))
>>>>> > > >
>>>>> > > > But when this gets exercised, I get:
>>>>> > > >
>>>>> > > > [worker-1-3] ERROR io.jooby.Jooby - GET /hello-world 500 Server Error
>>>>> > > >
>>>>> > > > java.lang.AbstractMethodError: Receiver class main$0 does not define or
>>>>> > > >
>>>>> > > > inherit an implementation of the resolved method 'abstract
>>>>> > > >
>>>>> > > > java.lang.Object
>>>>> > > >
>>>>> > > > apply(io.jooby.Context)' of interface io.jooby.Route$Handler.
>>>>> > >
>>>>> > > I don't see anything obviously wrong. One thing to try is instead of an
>>>>> > >
>>>>> > > anonymous class (with object) use a named class (with
>>>>> > >
>>>>> > > define-simple-class).
>>>>> > >
>>>>> > > The anonymous class is more convenient of course there is some extra
>>>>> > >
>>>>> > > "magic"
>>>>> > >
>>>>> > > (such as invisible fields) that might complicate things.
>>>>> > >
>>>>> > > > Also on a tangent, I was excited about the lambda shorthand for single
>>>>> > > >
>>>>> > > > method objects. Like I said this interface only has a single non-default
>>>>> > > >
>>>>> > > > method: apply. But I tried just calling `(app:route method handler)`
>>>>> > > >
>>>>> > > > without wrapping it in the io.jooby.Route$Handler object but it still
>>>>> > > >
>>>>> > > > failed. I guess it couldn't figure out this one method.
>>>>> > >
>>>>> > > Kawa has to be able to figure out at compile time that a specific
>>>>> > >
>>>>> > > class/interface
>>>>> > >
>>>>> > > is required before it can convert the lambda to an object. (As far as I
>>>>> > >
>>>>> > > can
>>>>> > >
>>>>> > > remember, doing this conversion at run-time isn't implemented, and would
>>>>> > >
>>>>> > > be
>>>>> > >
>>>>> > > fairly complicated.) So you may need to add some more type-specifiers.
>>>>> > >
>>>>> > > I suggest using javap to look at the generated classes, to see what is
>>>>> > >
>>>>> > > going on.
>>>>> > > ---------------------------------------------------------------------------------
>>>>> > >
>>>>> > > --Per Bothner
>>>>> > >
>>>>> > >
>>>>> > > per@bothner.com http://per.bothner.com/