Porting SLAYER to Android

Porting SLAYER to Android

[Panicz Maciej Godek]

Hi!

A few years back I have been working on a simple framework for developing
3d and multimedia applications in Scheme. The project is somewhat
documented here

http://puszcza.gnu.org.ua/software/slayer/

and its API documentation is available here:

http://puszcza.gnu.org.ua/software/slayer/manual/1.1.0/

It uses Guile 2.0 under the hood, and it implements a JavaScript-like event
model. When SLAYER runs, it reads a Scheme file, which sets up the
callbacks for input and drawing, and then the inner loop simply calls these
callbacks (and garbage collector from time to time):

https://bitbucket.org/panicz/slayer/src/default/src/slayer.c#lines-219

The example project (image browser) should give the impression how to use
it:

http://puszcza.gnu.org.ua/software/slayer/manual/1.1.0/Getting-started.html#Getting-started

Now, I've been thinking about porting this project to the Android platform.
But I know very little about Java, and even less about Android.

A while ago, I managed to build some project on my PC using Android Studio,
but I have since lost the computer.

Fortunately, I recently ran into a project which allows to build some
Android apps on Android, using the Termux terminal:

https://github.com/SDRausty/buildAPKs

After some digging I've found that this is the essential script for
building APKs:

https://github.com/SDRausty/buildAPKs/blob/master/scripts/bash/build/build.one.bash

I also managed to successfully build and run the latest Kawa in Termux,
with Java acquired from this script:
https://github.com/MasterDevX/Termux-Java

Now I would like to use this setup to re-create the SLAYER API on Android,
and I've been considering Kawa for this purpose. But I'm not sure how to do
it, or wherher it's even possible, but the rough idea that I have now is,
that I would like to include the sources of Kawa in my project and use it
to process the Scheme files when the SLAYER app is loaded. The most similar
project that I know is LÖVE, which has been ported to Android (using native
functions):

https://bitbucket.org/MartinFelis/love-android-sdl2/src

So the questions are:

- is there some class that could be used to represent the Kawa closures in
Java that could be stored in an array and invoked from Java?
- if so, is there some way to compile Scheme expressions to the instances
of that class? I mean something like

Closure square = kawa.compile("(lambda (x) (* x x))", environment);
square.invoke(2);

but I don't expect the first argument to be a string, but rather some
representation of AST.

In the documentation, I have found this:
https://www.gnu.org/software/kawa/Evaluating-Scheme-expressions-from-Java.html

but the question is whether the Scheme.eval method compiles the expression
(if it's, say, a function) or only interprets it?

- given some text file, is there any way to obtain some representations of
the subsequent s-expressions from that file?

Does the above idea have any chance to work, or is there some other way you
would recommend?

Re: Porting SLAYER to Android

[Per Bothner]

On 1/19/20 1:43 AM, Panicz Maciej Godek wrote:

> Now, I've been thinking about porting this project to the Android platform.
> But I know very little about Java, and even less about Android.

Welcome!

I did port Kawa to Android years ago, but I generally don't do much if any
Android programming.  I know very little about the state of Android programming,
but I believe there are people on this list who know more.

> - is there some class that could be used to represent the Kawa closures in
> Java that could be stored in an array and invoked from Java?

If you eval Scheme lambda (as  a string or an s-expression) the result is a Java object -
specifically an instance of the class gnu.mapping.Procedure.  You can definitely
store that in an array and invoke it from Java.

> In the documentation, I have found this:
> https://www.gnu.org/software/kawa/Evaluating-Scheme-expressions-from-Java.html
> 
> but the question is whether the Scheme.eval method compiles the expression
> (if it's, say, a function) or only interprets it?

The terms "compiling" and "interpreting" are vague and have various gray areas.
Kawa on normal platforms (i.e. not Android/Dalvik) doesn't really distinguish:
When a Scheme form it evaluated (as in the eval function/method or a REPL),
it is parsed, compiled to a bytecode class, and then a method in that class
is called.  Then the bytecode is thrown away.  However, if the eval result
is a function (or an object like a list that contains a function), then
of course the function is preserved as long as there is a reference to it,
which means that the bytecode is preserved and reused each time the function is called.

So yes, the evaluating a function compiles it to bytecode only once, and
reuses the bytecode each time it is called.

On Android the situation is different. It used its own non-standard bytecode and
virtual machine "Dalvik". Generating "Dalvik" bytecode is not supported by Kawa,
so instead Kawa just generates its internal "AST" data type (the Expression
class) and then evaluates it.

It should be possible to generate regular JVM bytercode and then convert them
as needed to Dalvik using dx.jar.  However, I haven't tried that - someone else may have.

The Dalvik VM is discontinued, but Dalvik bytecode is still used for
distribution.  I don't know what tools are used these days to generate
code on-the-fly on Android.

> - given some text file, is there any way to obtain some representations of
> the subsequent s-expressions from that file?

If you literally means the s-expressions, that is what the standard Scheme read
procedure does.  If you mean the Kawa AST, I don't think there is a official
documented way to do that, but it is of course possible and probably simple.

The --debug-print-final-expr will print out the resulting (pre-bytecode-generation)
Expression in human-readable form, but it is purely for debugging, and is not
meant to be parseable.

> Does the above idea have any chance to work, or is there some other way you
> would recommend?

Well, it's pretty vague, but it can definitely be made to work.

If performance of the Scheme code is an issue, you will need to find a
way to compile Kawa to Android internal format.

-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Porting SLAYER to Android

[Per Bothner]

One relatively-recent article that may be helpful:

https://medium.com/quick-code/hot-swapping-code-in-android-3043ccf6dd9b

-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Porting SLAYER to Android

[Adrián Medraño Calvo]

Dear Panicz Maciej Godek, Per Bothner,

> On 20. Jan 2020, at 16:12, Per Bothner <per@bothner.com> wrote:
> 
> On 1/19/20 1:43 AM, Panicz Maciej Godek wrote:
> 
>> Now, I've been thinking about porting this project to the Android platform.
>> But I know very little about Java, and even less about Android.
> 
> Welcome!
> 
> I did port Kawa to Android years ago, but I generally don't do much if any
> Android programming.  I know very little about the state of Android programming,
> but I believe there are people on this list who know more.

I successfully integrated dx.jar with Kawa, though would need more work (and guidance).  My approach was to expose some parts of KawaClassLoader so that I could implement a separate DexKawaClassLoader.

There’s also https://github.com/medranocalvo/gradle-kawa-plugin <https://github.com/medranocalvo/gradle-kawa-plugin>.  I worked on updating it for newer Android Plugin for Gradle, but never published the updates.

I have very little free time.  If I find any, I’ll clean up the changes open a pull request for further discussion, and update the gradle plugin.

Best regards,
Adrián.

> 
>> - is there some class that could be used to represent the Kawa closures in
>> Java that could be stored in an array and invoked from Java?
> 
> If you eval Scheme lambda (as  a string or an s-expression) the result is a Java object -
> specifically an instance of the class gnu.mapping.Procedure.  You can definitely
> store that in an array and invoke it from Java.
> 
>> In the documentation, I have found this:
>> https://www.gnu.org/software/kawa/Evaluating-Scheme-expressions-from-Java.html
>> but the question is whether the Scheme.eval method compiles the expression
>> (if it's, say, a function) or only interprets it?
> 
> The terms "compiling" and "interpreting" are vague and have various gray areas.
> Kawa on normal platforms (i.e. not Android/Dalvik) doesn't really distinguish:
> When a Scheme form it evaluated (as in the eval function/method or a REPL),
> it is parsed, compiled to a bytecode class, and then a method in that class
> is called.  Then the bytecode is thrown away.  However, if the eval result
> is a function (or an object like a list that contains a function), then
> of course the function is preserved as long as there is a reference to it,
> which means that the bytecode is preserved and reused each time the function is called.
> 
> So yes, the evaluating a function compiles it to bytecode only once, and
> reuses the bytecode each time it is called.
> 
> On Android the situation is different. It used its own non-standard bytecode and
> virtual machine "Dalvik". Generating "Dalvik" bytecode is not supported by Kawa,
> so instead Kawa just generates its internal "AST" data type (the Expression
> class) and then evaluates it.
> 
> It should be possible to generate regular JVM bytercode and then convert them
> as needed to Dalvik using dx.jar.  However, I haven't tried that - someone else may have.
> 
> The Dalvik VM is discontinued, but Dalvik bytecode is still used for
> distribution.  I don't know what tools are used these days to generate
> code on-the-fly on Android.
> 
>> - given some text file, is there any way to obtain some representations of
>> the subsequent s-expressions from that file?
> 
> If you literally means the s-expressions, that is what the standard Scheme read
> procedure does.  If you mean the Kawa AST, I don't think there is a official
> documented way to do that, but it is of course possible and probably simple.
> 
> The --debug-print-final-expr will print out the resulting (pre-bytecode-generation)
> Expression in human-readable form, but it is purely for debugging, and is not
> meant to be parseable.
> 
>> Does the above idea have any chance to work, or is there some other way you
>> would recommend?
> 
> Well, it's pretty vague, but it can definitely be made to work.
> 
> If performance of the Scheme code is an issue, you will need to find a
> way to compile Kawa to Android internal format.
> 
> -- 
> 	--Per Bothner
> per@bothner.com   http://per.bothner.com/