Kawa source tree organization

Kawa source tree organization

[Arvydas Silanskas]

Good evening,

I've been digging around kawa source tree a few times, and frankly it feels
abit overwhelming to get into happy-hacking of its internals. Not really
because of code itself, but because of all peripheral code sitting in one
place. Things that depend on servlets, jline, echo2, swt, domterm, etc,
don't really have significance to the implementation of the core, but they
do add mental and menial library imports overhead to get the whole thing
running.

Perhaps it could be worthwhile splitting things up? For example:

* kawa-the-compiler base project, that'd consist basically of
language-agnostic compiler what now are gnu.bytecode, gnu.expr, etc.
Ideally one could just use it to create a jvm targeting language.
* kawa-the-scheme project, which would be what implements the minimal core
scheme, such as the native syntax forms seen in kawa.lang package. Depends
on kawa-the-compiler project. Ideally one could just use it as a
self-sufficient minimal scheme library.
* various feature-projects, that depend on either just kawa-the-compiler,
or kawa-the-scheme, but not on other feature-projects.
* the main kawa project, which includes everything listed above, implements
the public static main, and handles command line arguments. Basically, the
module that compiles to what now is kawa.jar.

What do you think? In terms of actual ways of implementing this, I'd like
to see maven or gradle used, but git submodules or ant build scripts could
be used just as well. If the base idea sounds ok, I could tinker around and
show a prototype of how it could look.

Regards,
Arvydas

Re: Kawa source tree organization

[Per Bothner]

On 11/21/20 4:23 PM, Arvydas Silanskas via Kawa wrote:
> Perhaps it could be worthwhile splitting things up? For example:
> 
> * kawa-the-compiler base project, that'd consist basically of
> language-agnostic compiler what now are gnu.bytecode, gnu.expr, etc.
> Ideally one could just use it to create a jvm targeting language.
> * kawa-the-scheme project, which would be what implements the minimal core
> scheme, such as the native syntax forms seen in kawa.lang package. Depends
> on kawa-the-compiler project. Ideally one could just use it as a
> self-sufficient minimal scheme library.
> * various feature-projects, that depend on either just kawa-the-compiler,
> or kawa-the-scheme, but not on other feature-projects.
> * the main kawa project, which includes everything listed above, implements
> the public static main, and handles command line arguments. Basically, the
> module that compiles to what now is kawa.jar.
I think that would be good, but it may be a lot of work (and hard thought)
to split things up.  There are lots of cross-dependencies.  We do have one
advantage, in that Kawa has always been able to bootstrap from just Java,
without requiring an earlier version of itself (or some other version of Scheme).
This has forced some separation.

I have also tried to separate out compile-time-only clases, but that still
needs a lot of work.

I have generally preferred relatively shallow directory trees, with the
source for class X.Y as file X/Y.java with no extra nesting.
It is probably reasonable to change that, though I don't think going
as far as Maven's src/main/java/X/Y.java would be desirable.

A first draft:
   core-utils (packages such as gnu.lists gnu.math gnu.kawa.io gnu.text.
      These are used by both compiler and the runtime.)
   kawa-compiler (gnu.bytecode gnu.expr)
   scheme-language (maybe split into parts such as separate scheme-library)
   extras (probably split this up)
   commonlisp-language
   xquery-language
   jemacs-language and brl-language are probably not used by anyone
   (gnu.ecmascript can probably be removed)

A lot of gnu/kawa/functions and gnu/kawa/lis

Perhaps the first step would be to create a core-utils sub-directory
with classes that does not depend on any classes outside of it.
That may require some re-factoring and moving classes around.

Or start from the other end: classes that are definitely *not* needed
to build or run a functional Scheme.

I am not convinced that replacing make+autotools would be worthwhile.
They're clunky, but they work without installing some other major
subsystem, and it is not clear to me that any alternative is sufficiently
clearly better.  Specifically, I don't see writing build/configuration files
using XML syntax (as needed for Ant or Maven) is an improvement.  And I'm
not quite ready to embrace Gradle (or the Groovy language), though I could
be talked into it if that is the consensus and someone else does the heavy lifting.
Regardless, I suggest changing build tools is a separate and orthogonal task:
switching both directory layout and build tools at the same time is probably too much.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Kawa source tree organization

[Arvydas Silanskas]

Hmm I agree the toolchain and grouping are orthogonal changes, and I won't
do changes to the former now.

Regarding make/autotools alternatives, specifically maven and gradle, I
think they're great if for nothing else then at least for the consequences
of their mainstream use and popularity -- great IDE support and familiarity
for java programmers (which consequently mean preexisting understanding of
structure and where to look for things if project has idiomatic layout, as
well as having necessary things already installed). Aside that, their
merits are mostly dependency installment & management (for 3rd party like
jline, but also for internal use, say like kawa-compiler would have
dependency on core-utils) and taking care of invoking javac with right
classpaths. I'm also fairly sure those tools are more friendly for
developers using windows.

That said, I'm somewhat fine with the "don't fix if it's not broke"
reasoning, because I'm not entirely sure how maven or gradle would play out
with preprocessing that's being done in kawa. While fully new source file
generation (eg vector primitives like FVector) seems solvable with
annotation processor code generation, the inline preprocessing looks
trickier.

Arvydas

2020-11-22, sk, 03:30 Per Bothner <per@bothner.com> rašė:

> On 11/21/20 4:23 PM, Arvydas Silanskas via Kawa wrote:
> > Perhaps it could be worthwhile splitting things up? For example:
> >
> > * kawa-the-compiler base project, that'd consist basically of
> > language-agnostic compiler what now are gnu.bytecode, gnu.expr, etc.
> > Ideally one could just use it to create a jvm targeting language.
> > * kawa-the-scheme project, which would be what implements the minimal
> core
> > scheme, such as the native syntax forms seen in kawa.lang package.
> Depends
> > on kawa-the-compiler project. Ideally one could just use it as a
> > self-sufficient minimal scheme library.
> > * various feature-projects, that depend on either just kawa-the-compiler,
> > or kawa-the-scheme, but not on other feature-projects.
> > * the main kawa project, which includes everything listed above,
> implements
> > the public static main, and handles command line arguments. Basically,
> the
> > module that compiles to what now is kawa.jar.
> I think that would be good, but it may be a lot of work (and hard thought)
> to split things up.  There are lots of cross-dependencies.  We do have one
> advantage, in that Kawa has always been able to bootstrap from just Java,
> without requiring an earlier version of itself (or some other version of
> Scheme).
> This has forced some separation.
>
> I have also tried to separate out compile-time-only clases, but that still
> needs a lot of work.
>
> I have generally preferred relatively shallow directory trees, with the
> source for class X.Y as file X/Y.java with no extra nesting.
> It is probably reasonable to change that, though I don't think going
> as far as Maven's src/main/java/X/Y.java would be desirable.
>
> A first draft:
>    core-utils (packages such as gnu.lists gnu.math gnu.kawa.io gnu.text.
>       These are used by both compiler and the runtime.)
>    kawa-compiler (gnu.bytecode gnu.expr)
>    scheme-language (maybe split into parts such as separate scheme-library)
>    extras (probably split this up)
>    commonlisp-language
>    xquery-language
>    jemacs-language and brl-language are probably not used by anyone
>    (gnu.ecmascript can probably be removed)
>
> A lot of gnu/kawa/functions and gnu/kawa/lis
>
> Perhaps the first step would be to create a core-utils sub-directory
> with classes that does not depend on any classes outside of it.
> That may require some re-factoring and moving classes around.
>
> Or start from the other end: classes that are definitely *not* needed
> to build or run a functional Scheme.
>
> I am not convinced that replacing make+autotools would be worthwhile.
> They're clunky, but they work without installing some other major
> subsystem, and it is not clear to me that any alternative is sufficiently
> clearly better.  Specifically, I don't see writing build/configuration
> files
> using XML syntax (as needed for Ant or Maven) is an improvement.  And I'm
> not quite ready to embrace Gradle (or the Groovy language), though I could
> be talked into it if that is the consensus and someone else does the heavy
> lifting.
> Regardless, I suggest changing build tools is a separate and orthogonal
> task:
> switching both directory layout and build tools at the same time is
> probably too much.
> --
>         --Per Bothner
> per@bothner.com   http://per.bothner.com/
>