Re: Incremental compilation

["Marc Nieper-Wißkirchen" <marc@nieper-wisskirchen.de>]

Am So., 30. Dez. 2018 um 13:28 Uhr schrieb Basile Starynkevitch
<basile@starynkevitch.net>:
>
>
> On 12/30/18 12:54 PM, Marc Nieper-Wißkirchen wrote:
>
> > My use case is compiling a Scheme library. It contains macro
> > definitions whose transformation procedures have to be compiled at
> > first to expand subsequent parts of the library body. Furthermore, the
> > compiled transformation procedures of global macro defintions have to
> > appear in the final dynamic library as well.
>
> I am quite interested in your work, since I did (in the past) GCC MELT
> http://starynkevitch.net/Basile/gcc-melt/ (which now is a dead project;
> it was a Lisp dialect compiled to C or C++ for GCC extensions); I am
> working on http://github.com/bstarynk/bismon right now, and you might be
> interested in reading
> http://starynkevitch.net/Basile/bismon-chariot-doc.pdf
>
> Is your project some open-source stuff? Could you give us the URL of its
> source code?

At the moment, most parts of the project is just in my head. I have
written some code to be able to do some experiments but nothing is
finished yet. As soon as I have something presentable, I'd love to
share it with you. It will be GPL'd code.

The goal of the project is to have a reasonably fast Scheme
implementation that I can use for experiments with language extension.

> In MELT, I also had macros similar to yours. So I believe I understand
> your work quite well (since it is similar to what I did in GCC MELT,
> which was a Lisp -but not Scheme- dialect with macros). In practice, you
> still can compile every function in a separate *.so plugin.
>
> You could group several (unrelated) functions together by JIT compiling
> them at once, but that is an optimization.
>
> You could, once all macros have been compiled, regenerate a single
> "shared object" for all your library, but that is an optimization. The
> main reason to do so would be to avoid wasting virtual address space by
> having many thousands of shared objects. I am not sure it is worthwhile.

The reason why I would like to have a single shared object is for the
convenience of the user. The compiler should produce a single shared
object from a Scheme library definition which can then later be
visited/invoked in lieu of loading the Scheme source code of the
library.

> Since you are implementing a Scheme, all your functions are in fact used
> in closures (which contains a function pointer). So updating all the
> required closures (to put there a function pointer to a newer code)
> might be quite easy in practice.
>
> The potential issue is to "garbage collect" useless code. In your case,
> that would mean doing dlclose in your GC. Do you want to do that?

There will be two sources of (longer lived) compiled code: Libraries
that have been loaded and functions generated by `eval'. The former
will never be garbage collected. The latter will. `Eval' would
(internally) return closures created that should be disposable by
calling 'gcc_jit_result_release`.

> Since you can in practice have many hundred thousands of useless
> generated plugins, you might decide to never call dlclose (and accept
> the "code segment leak" that follows).
>
> Again, I really am interested by your work. If you have any additional
> references (draft reports, source code, etc...) please share them with
> us if possible.

I really should write up my concept. It isn't going to be finished
this year, however.

Best,

Marc

>
>
> Cheers
>
> --
> Basile STARYNKEVITCH   == http://starynkevitch.net/Basile
> opinions are mine only - les opinions sont seulement miennes
> Bourg La Reine, France
>


-- 
Prof. Dr. Marc Nieper-Wißkirchen

Universität Augsburg
Institut für Mathematik
Universitätsstraße 14
86159 Augsburg

Tel: 0821/598-2146
Fax: 0821/598-2090

E-Mail: marc.nieper-wisskirchen@math.uni-augsburg.de
Web: www.math.uni-augsburg.de/alg/mitarbeiter/mnieper/