Successfully mixed java and kawa-scheme in a "full" Android app

Successfully mixed java and kawa-scheme in a "full" Android app

[Marius Kjeldahl]

If you've been following my last posts with Per helping me out, you 
probably know by now that I've been trying to get Kawa to build and run 
together with the latest releases from Google, specifically get it to 
build with a "gradle"-based project which is the latest flavour from 
Google (it's what Android Studio uses for build and project configuration).

Anyway, I can successfully report that I've successfully managed to 
integrate kawa with a java app I have in the Google Play store 
("Killermatch" for the curios). I specifically wanted to see if I could 
get it working with a "full" app to avoid any issues popping up when 
trying to integrate kawa with a "full" Android app.

Sure enough, there are some issues. One is integrating kawa builds with 
the android gradle plugin. I've made some progress, but it isn't smooth 
yet. I've attacked the gradle build issues from a couple of angles, and 
I'm not yet satisified, but suffice to say, adding gradle tasks like:

> task kawaReleaseInt(type: Exec) {
>   environment['CLASSPATH'] = "/opt/android-studio/sdk/platforms/android-15/android.jar:build/classes/release/net/kjeldahl/tournman"
>   commandLine 'kawa', '-d', 'build/kawaclasses/release', '-P', 'net.kjeldahl.tournman.', '--warn-undefined-variable', '--warn-invoke-unknown-method', '--warn-as-error', '-C', 'src/main/kawa/MyKawaActivity.scm', 'src/main/kawa/Something.scm'
> }
>
> task kawaRelease(type: Exec, dependsOn: ["kawaReleaseInt"]) {
>   commandLine "jar", "cf", "build/scmfiles.jar", "-C", "build/kawaclasses/release", "."
> }

These at least make it easier to use kawa with an app using the android 
plugin. Essentially you'll do a "./gradlew kawaRelease assembleRelease" 
(and similar for debug). While most people seem to love gradle, I'm not 
convinced yet. While it's probably great for writing a build process 
from scratch, you'll be writing thousand lines of java/groovy code to 
make it build, it certainly does not seem very great as far as modifying 
existing build processes. There are easy improvements to even my simple 
gradle tasks above (like automatically finding the source files), which 
I haven't gotten around to doing yet.

Regarding adding kawa support for gradle; My first strategy was to 
"modify" the JavaPlugin task to basically call the kawa compiler instead 
of the java compiler for all *.scm files. While that sounds easy, it's 
not. First of all, the java "plugin" basically has code and abstractions 
in lots of different source locations in Gradle, so it's not like it's 
easy to figure out what you need.

I found another plugin named "Clojuresque" which is a ClojurePlugin for 
building Clojure projects. At least that one has all it's files in one 
place, but it's still a thousand lines of relatively low level code just 
to build (despite inheriting from JavaPlugin...). I ended up using the 
clojure based one and wrote a similar KawaPlugin. When it finally 
started working, imagine how fun it was when I tried to combine my 
KawaPlugin with the Android plugin from when the Android plugin spit out 
a message saying it's not compatible with the JavaPlugin.

After that I basically gave up (correction, I looked at the Android 
plugin source and puked) and settled for simple "make-like" rules in 
gradle, like the ones I showed higher up.

When I started adding kawa into my android project, some lessons had to 
be learned which I'm sharing here in case my conclusions seem off (feel 
free to suggest how to fix them).

* Android View Construction

If you use the example from:

http://www.gnu.org/software/kawa/Android-view-construction.html

you can not use the kawart jar, but need to use kawa.jar. It seems the 
view construction bits use classes that are not included in kawart.

* Proguard issues

My app apk was roughly around 1MB before I added kawa into it. I suspect 
at least half of that are third party libraries (AdMob, Facebook SDK and 
various other SDKs I use in my app). After adding kawa, but before 
running proguard, the apk grew to 2.6MB.

To make sure everything worked, I made two kawa classes that I called 
from my java code; one class just returns a string that gets displayed 
as an android notification (Toast). The other class was the activity 
example from the "Android view construction" documentation. I put the 
kawa activity into the manifest and activated it from the java code. 
Everything worked fine before running proguard, but not after.

As mentioned, I needed to use the kawa.jar (not kawart.jar). I also 
needed to make the following additions to my proguard configuration file 
(while it may not seem much, I had to crash the app quite a few times to 
come up with that short list):

-dontwarn gnu.**
-dontwarn kawa.**

-keep class gnu.expr.Language
-keep class gnu.mapping.**
-keep class gnu.lists.LList
-keep class kawa.lang.*

It seems proguard got rid of the listed classes and the intent would 
crash as soon as I tried starting it. After modifying the proguard build 
as shown above, everything worked. And best of all, after progguard it 
only added roughly 250MB to my release apk, so with a full kawa activity 
(pulling in most of the relevant kawa dependencies I suspect), my final 
apk size after running proguard became roughly 1.26MB. I'm pretty sure 
more kawa code will grow the apk a lot slower since most of the stuff is 
pulled in already. Time and more testing will show.

Anyway, just wanted to share the fact that kawa seems to work great with 
a "full" android app, which make the prospect of writing more android 
code a lot more fun than it would otherwise be.

When I get some time I'm considering doing more testing with a separate 
app example, including comparing a "pure" java app, with a "pure" kawa 
app, and a mixed app, with a full writeup. But first I want to get more 
experience with using it in my own app.

If anybody else will or is trying to go down the same path, hopefully 
these details will make your path a little less painful, and at least 
demonstrate that there is gold at the end of the rainbow.

With the great work Per has already done, and with more examples and 
demonstrations, I believe there is a good change kawa could become a 
popular language for making or extending android apps.

Thanks,

Marius Kjeldahl

Re: Successfully mixed java and kawa-scheme in a "full" Android app

[Matthieu Vachon]

On Wed, Jul 17, 2013 at 1:38 PM, Marius Kjeldahl <marius@kjeldahl.net> wrote:
> If you've been following my last posts with Per helping me out, you probably
> know by now that I've been trying to get Kawa to build and run together with
> the latest releases from Google, specifically get it to build with a
> "gradle"-based project which is the latest flavour from Google (it's what
> Android Studio uses for build and project configuration).
>

I was following and I'm pretty happy that you finally made it :)

> These at least make it easier to use kawa with an app using the android
> plugin. Essentially you'll do a "./gradlew kawaRelease assembleRelease" (and
> similar for debug). While most people seem to love gradle, I'm not convinced
> yet. While it's probably great for writing a build process from scratch,
> you'll be writing thousand lines of java/groovy code to make it build, it
> certainly does not seem very great as far as modifying existing build
> processes. There are easy improvements to even my simple gradle tasks above
> (like automatically finding the source files), which I haven't gotten around
> to doing yet.

You're right, when you have to hack around it, it can be really a
pain. That's why I think having reusable plugins that can be used
directly by following some conventions is a must. Sad thing, Kawa
plugin for Gradle does not exist ... yet.

>
> Regarding adding kawa support for gradle; My first strategy was to "modify"
> the JavaPlugin task to basically call the kawa compiler instead of the java
> compiler for all *.scm files. While that sounds easy, it's not. First of
> all, the java "plugin" basically has code and abstractions in lots of
> different source locations in Gradle, so it's not like it's easy to figure
> out what you need.
>
> I found another plugin named "Clojuresque" which is a ClojurePlugin for
> building Clojure projects. At least that one has all it's files in one
> place, but it's still a thousand lines of relatively low level code just to
> build (despite inheriting from JavaPlugin...). I ended up using the clojure
> based one and wrote a similar KawaPlugin. When it finally started working,
> imagine how fun it was when I tried to combine my KawaPlugin with the
> Android plugin from when the Android plugin spit out a message saying it's
> not compatible with the JavaPlugin.
>
> After that I basically gave up (correction, I looked at the Android plugin
> source and puked) and settled for simple "make-like" rules in gradle, like
> the ones I showed higher up.

We are currently in the process of moving from an in-house made build
system to a Gradle one. Since we have a fairly decent Kawa code base,
compiling Kawa from Gradle was an important task. We developed a
plugin for Gradle that is compiling Kawa code. This plugin is not yet
ready for prime-time for different reasons (mainly, we use wrapper
around Kawa compiler and are still on version 1.7.90 which is pretty
old, but that is another story).

This plugin follows Gradle conventions for source locations and stuff
like that. I did not do it myself, so I'm not sure if it's based on
the JavaPlugin or not, I will tell you tomorrow.

I think it would be something possible for the company I work for to
release our Kawa plugin on github. For sure, I need to check with my
the management team for he OK. We could then check if we could work
together to integrate what you have done in a single and powerful Kawa
plugin for Gradle :).

I really don't know if my current version would play well in the
context of an Android build, I guess I would need to try to find it
out but that would be a first start that could be improved.

>
> If anybody else will or is trying to go down the same path, hopefully these
> details will make your path a little less painful, and at least demonstrate
> that there is gold at the end of the rainbow.
>

Thanks, those kind of information sharing is always welcome in my opinion.

> With the great work Per has already done, and with more examples and
> demonstrations, I believe there is a good change kawa could become a popular
> language for making or extending android apps.
>

I believe also, Kawa is a nice language and need more exposure in the
JVM language like community.

Regards,
Matt

Re: Successfully mixed java and kawa-scheme in a "full" Android app

[Marius Kjeldahl]

On 07/22/2013 08:31 AM, Matthieu Vachon wrote:
> I think it would be something possible for the company I work for to
> release our Kawa plugin on github. For sure, I need to check with my
> the management team for he OK. We could then check if we could work
> together to integrate what you have done in a single and powerful Kawa
> plugin for Gradle :).
> 
> I really don't know if my current version would play well in the
> context of an Android build, I guess I would need to try to find it
> out but that would be a first start that could be improved.

Sounds like great work. Just in case you did not pick it up; if you've
using the JavaPlugin inside your KawaPlugin in Gradle, the Android
plugin from Google will refuse to run with a message saying they are not
compatible. That's why my own KawaPlugin did not get anywhere; after
looking at the source for the JavaPlugin and Android plugin, I figured
there has got to be an easier way, and I think there is...

> I believe also, Kawa is a nice language and need more exposure in the
> JVM language like community.

I agree wholeheartedly.

FWIW, in my quest to be able to automate (and _understand_) the whole
build process involved when building Android apps with Kawa _and_ Java
mixed in, after a journey through Gradle, Maven and Ant, I'm currently
having a lot of success with Buildr. If I succeed I will share the
details here, but an executive summary of Buildr is that it's Ruby based
with a dependency system which is a lot easier to understand for people
who have ever used Makefiles. And for "simple stuff" like copying files
and similar, you typically use Ruby or even shell commands directly.

Thanks,

Marius K.