Digging a bit deeper into JIT's compile internals

Digging a bit deeper into JIT's compile internals

[Joshua Saxby]

[-- Attachment #1: Type: text/plain, Size: 3174 bytes --]

After following files jit-recording.cc and jit-playback.cc, I think I've
found out where I need to patch JIT to do what I want it to do.

Looks like both compiling to file and memory ultimately rely upon
playback::context::compile() to do the bulk of their work and then override
a postprocess() method to do additional handling pertaining to their
specific task.

If I can find a way to cache or store the intermediate result generated by
compile(), I should then be able to restructure this to have the follow-up
tasks required by both different forms of compilation to be done starting
with this intermediate result.

Anyone see a problem with my approach? I'm hoping it will be possible to
reüse compilation results produced by compile() in this way without it
causing any conflicts...

Regards,
*J.S.*



*My PGP Public Key Identity*

pub   4096R/*B7A947E4* 2016-11-16 [expires: 2025-12-31]
      Key fingerprint = *E2C4 514F F0FA 52D1 896A  B1D6 3D42 BFD9 B7A9 47E4*
uid       Joshua Saxby <joshua.a.saxby+UMvLnvbsOxBHaeiCHvbdunpz@gmail.com>
uid                   Joshua Saxby (saxbophone) <joshua.a.saxby@gmail.com>
sub   4096R/0A445946 2016-11-16 [expires: 2025-12-31]




On Sat, 10 Jun 2023 at 13:00, <jit-request@gcc.gnu.org> wrote:

> Send Jit mailing list submissions to
>         jit@gcc.gnu.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://gcc.gnu.org/mailman/listinfo/jit
> or, via email, send a message with subject or body 'help' to
>         jit-request@gcc.gnu.org
>
> You can reach the person managing the list at
>         jit-owner@gcc.gnu.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Jit digest..."
> Today's Topics:
>
>    1. Modifying the jit compiler API? (Joshua Saxby)
>
>
>
> ---------- Forwarded message ----------
> From: Joshua Saxby <joshua.a.saxby@gmail.com>
> To: jit@gcc.gnu.org
> Cc:
> Bcc:
> Date: Sat, 10 Jun 2023 00:04:43 +0100
> Subject: Modifying the jit compiler API?
> Currently the JIT has two functions allowing you to compile a context
> either to memory or to file.
>
> But what if you want to compile to both? There doesn't seem to be any way
> to do this except by calling both functions separately which I believe will
> effectively be two separate compilations...
>
> Presumably, it should be possible to modify this part of the API to compile
> to some form of intermediate representation of the work that is common to
> both kinds of compilation, and then turn that into code in memory and in
> file, respectively.
>
> Anyone got any pointers for me on where in the code would be the best place
> for me to modify to dupport this? I did take a look in the implementation
> code of JIT sone weeks ago and remember seeing lots of complicated stuff
> regarding recordings and replays that looked relevant...
>
> A slighty simpler alteration I am also interested in making is allowing
> compile to file to compile to a buffer in memory as well as a file (I
> suppose it could be termed "compile to binary"). Maybe I should start with
> that...
>
>

Re: Digging a bit deeper into JIT's compile internals

[David Malcolm]

On Sat, 2023-06-10 at 17:26 +0100, Joshua Saxby via Jit wrote:

Hi Joshua

> After following files jit-recording.cc and jit-playback.cc, I think
> I've
> found out where I need to patch JIT to do what I want it to do.

Indeed.  In case you haven't seen it yet, see:
https://gcc.gnu.org/onlinedocs/jit/internals/index.html#overview-of-code-structure

You'll see there (and in jit-playback.cc) that the
gcc_jit_compile[_to_file] functions are basically running the
equivalent of cc1 in-process to generate a .s file in a tempdir, and
then calling out to subprocesses to run the assembler and linker as
needed.  FWIW I've experimented in the past with libraries for the
assembler and linker (keeping it all in-process), and got some
speedups, but it needed nontrivial patches to binutils to turn as and
ld into shared libraries.

> Looks like both compiling to file and memory ultimately rely upon
> playback::context::compile() to do the bulk of their work and then
> override
> a postprocess() method to do additional handling pertaining to their
> specific task.
> 
> If I can find a way to cache or store the intermediate result
> generated by
> compile(), I should then be able to restructure this to have the
> follow-up
> tasks required by both different forms of compilation to be done
> starting
> with this intermediate result.
> 
> Anyone see a problem with my approach? I'm hoping it will be possible
> to
> reüse compilation results produced by compile() in this way without
> it
> causing any conflicts...

I have a higher-level question, which is why do you want to compile to
both/reuse results?

I believe you ought to be able to compile a context multiple times
(provided no errors occur), so if you need both you can currently write
code like this:

  gcc_jit_context *ctxt = populate_ctxt ();
  gcc_jit_context_compile_to_file (ctxt,
                                   GCC_JIT_OUTPUT_KIND_ASSEMBLER,
                                   "foo.s");
  gcc_jit_result *result = gcc_jit_context_compile (ctxt);

albeit with the drawback that it's duplicating work.  Is it the
duplicated work that you're trying to avoid?

Do you have ideas about what the API you'd want would look like?

[...snip...]

Hope this is helpful
Dave

> > 
> > ---------- Forwarded message ----------
> > From: Joshua Saxby <joshua.a.saxby@gmail.com>
> > To: jit@gcc.gnu.org
> > Cc:
> > Bcc:
> > Date: Sat, 10 Jun 2023 00:04:43 +0100
> > Subject: Modifying the jit compiler API?
> > Currently the JIT has two functions allowing you to compile a
> > context
> > either to memory or to file.
> > 
> > But what if you want to compile to both? There doesn't seem to be
> > any way
> > to do this except by calling both functions separately which I
> > believe will
> > effectively be two separate compilations...
> > 
> > Presumably, it should be possible to modify this part of the API to
> > compile
> > to some form of intermediate representation of the work that is
> > common to
> > both kinds of compilation, and then turn that into code in memory
> > and in
> > file, respectively.
> > 
> > Anyone got any pointers for me on where in the code would be the
> > best place
> > for me to modify to dupport this? I did take a look in the
> > implementation
> > code of JIT sone weeks ago and remember seeing lots of complicated
> > stuff
> > regarding recordings and replays that looked relevant...
> > 
> > A slighty simpler alteration I am also interested in making is
> > allowing
> > compile to file to compile to a buffer in memory as well as a file
> > (I
> > suppose it could be termed "compile to binary"). Maybe I should
> > start with
> > that...
> > 
> > 
> 

Re: Digging a bit deeper into JIT's compile internals

[Joshua Saxby]

[-- Attachment #1: Type: text/plain, Size: 4481 bytes --]

Hi David,

Thanks very much for that helpful information. I think I now know what I
need to do to achieve my goal.

Essentially I will be changing the structure of the code slightly so that
.s file from the initial compilation gets reüsed when compiling to both
file and memory.

It'll probably take me quite a while to modify this, but if I am successful
I will post again on mailing list with an update.

Best Regards,
J.S.

On Sat, 10 Jun 2023, 22:47 David Malcolm, <dmalcolm@redhat.com> wrote:

> On Sat, 2023-06-10 at 17:26 +0100, Joshua Saxby via Jit wrote:
>
> Hi Joshua
>
> > After following files jit-recording.cc and jit-playback.cc, I think
> > I've
> > found out where I need to patch JIT to do what I want it to do.
>
> Indeed.  In case you haven't seen it yet, see:
>
> https://gcc.gnu.org/onlinedocs/jit/internals/index.html#overview-of-code-structure
>
> You'll see there (and in jit-playback.cc) that the
> gcc_jit_compile[_to_file] functions are basically running the
> equivalent of cc1 in-process to generate a .s file in a tempdir, and
> then calling out to subprocesses to run the assembler and linker as
> needed.  FWIW I've experimented in the past with libraries for the
> assembler and linker (keeping it all in-process), and got some
> speedups, but it needed nontrivial patches to binutils to turn as and
> ld into shared libraries.
>
> > Looks like both compiling to file and memory ultimately rely upon
> > playback::context::compile() to do the bulk of their work and then
> > override
> > a postprocess() method to do additional handling pertaining to their
> > specific task.
> >
> > If I can find a way to cache or store the intermediate result
> > generated by
> > compile(), I should then be able to restructure this to have the
> > follow-up
> > tasks required by both different forms of compilation to be done
> > starting
> > with this intermediate result.
> >
> > Anyone see a problem with my approach? I'm hoping it will be possible
> > to
> > reüse compilation results produced by compile() in this way without
> > it
> > causing any conflicts...
>
> I have a higher-level question, which is why do you want to compile to
> both/reuse results?
>
> I believe you ought to be able to compile a context multiple times
> (provided no errors occur), so if you need both you can currently write
> code like this:
>
>   gcc_jit_context *ctxt = populate_ctxt ();
>   gcc_jit_context_compile_to_file (ctxt,
>                                    GCC_JIT_OUTPUT_KIND_ASSEMBLER,
>                                    "foo.s");
>   gcc_jit_result *result = gcc_jit_context_compile (ctxt);
>
> albeit with the drawback that it's duplicating work.  Is it the
> duplicated work that you're trying to avoid?
>
> Do you have ideas about what the API you'd want would look like?
>
> [...snip...]
>
> Hope this is helpful
> Dave
>
> > >
> > > ---------- Forwarded message ----------
> > > From: Joshua Saxby <joshua.a.saxby@gmail.com>
> > > To: jit@gcc.gnu.org
> > > Cc:
> > > Bcc:
> > > Date: Sat, 10 Jun 2023 00:04:43 +0100
> > > Subject: Modifying the jit compiler API?
> > > Currently the JIT has two functions allowing you to compile a
> > > context
> > > either to memory or to file.
> > >
> > > But what if you want to compile to both? There doesn't seem to be
> > > any way
> > > to do this except by calling both functions separately which I
> > > believe will
> > > effectively be two separate compilations...
> > >
> > > Presumably, it should be possible to modify this part of the API to
> > > compile
> > > to some form of intermediate representation of the work that is
> > > common to
> > > both kinds of compilation, and then turn that into code in memory
> > > and in
> > > file, respectively.
> > >
> > > Anyone got any pointers for me on where in the code would be the
> > > best place
> > > for me to modify to dupport this? I did take a look in the
> > > implementation
> > > code of JIT sone weeks ago and remember seeing lots of complicated
> > > stuff
> > > regarding recordings and replays that looked relevant...
> > >
> > > A slighty simpler alteration I am also interested in making is
> > > allowing
> > > compile to file to compile to a buffer in memory as well as a file
> > > (I
> > > suppose it could be termed "compile to binary"). Maybe I should
> > > start with
> > > that...
> > >
> > >
> >
>
>

Rationale for compiling to memory and file

[Joshua Saxby]

> I have a higher-level question, which is why do you want to compile to
> both/reuse results?

I am in the process of designing a programming language for which I
want to use libgccjit as the backend. My language will feature
compile-time function execution and reusing the JIT for both
compile-time evaluated code and code going into the binary simplifies
the design greatly (although it is probably less efficient than an
interpreter for CTFE).

> albeit with the drawback that it's duplicating work.  Is it the
> duplicated work that you're trying to avoid?

I'm sure you're right that contexts can be compiled twice in this way,
and this is what I considered at first, but yes, this is the work
duplication I'd like to avoid, as JIT's compile times are non-trivial.

> Do you have ideas about what the API you'd want would look like?

I have some ideas, yes. The chief principle on my mind is the need to
not break the existing API compatibility, therefore retaining the
current functions that compile to memory and file in one-shot. For the
new thing I propose to build, I anticipate an API approximately as
follows:

compile_intermediate(context) -> intermediate_result
finalise_to_file(intermediate_result) -> jit_result
finalise_to_memory(intermediate_result) -> jit_result

I should stress, those are just provisional names I picked out of the
air for functions and data structures, so I have some way to
illustrate it, but I think this captures the main structure I'd aim
for.

I would be interested to know what you think if you have any thoughts on it.

Best Regards,
J.S.

On Sat, 10 Jun 2023 at 22:47, David Malcolm <dmalcolm@redhat.com> wrote:
>
> On Sat, 2023-06-10 at 17:26 +0100, Joshua Saxby via Jit wrote:
>
> Hi Joshua
>
> > After following files jit-recording.cc and jit-playback.cc, I think
> > I've
> > found out where I need to patch JIT to do what I want it to do.
>
> Indeed.  In case you haven't seen it yet, see:
> https://gcc.gnu.org/onlinedocs/jit/internals/index.html#overview-of-code-structure
>
> You'll see there (and in jit-playback.cc) that the
> gcc_jit_compile[_to_file] functions are basically running the
> equivalent of cc1 in-process to generate a .s file in a tempdir, and
> then calling out to subprocesses to run the assembler and linker as
> needed.  FWIW I've experimented in the past with libraries for the
> assembler and linker (keeping it all in-process), and got some
> speedups, but it needed nontrivial patches to binutils to turn as and
> ld into shared libraries.
>
> > Looks like both compiling to file and memory ultimately rely upon
> > playback::context::compile() to do the bulk of their work and then
> > override
> > a postprocess() method to do additional handling pertaining to their
> > specific task.
> >
> > If I can find a way to cache or store the intermediate result
> > generated by
> > compile(), I should then be able to restructure this to have the
> > follow-up
> > tasks required by both different forms of compilation to be done
> > starting
> > with this intermediate result.
> >
> > Anyone see a problem with my approach? I'm hoping it will be possible
> > to
> > reüse compilation results produced by compile() in this way without
> > it
> > causing any conflicts...
>
> I have a higher-level question, which is why do you want to compile to
> both/reuse results?
>
> I believe you ought to be able to compile a context multiple times
> (provided no errors occur), so if you need both you can currently write
> code like this:
>
>   gcc_jit_context *ctxt = populate_ctxt ();
>   gcc_jit_context_compile_to_file (ctxt,
>                                    GCC_JIT_OUTPUT_KIND_ASSEMBLER,
>                                    "foo.s");
>   gcc_jit_result *result = gcc_jit_context_compile (ctxt);
>
> albeit with the drawback that it's duplicating work.  Is it the
> duplicated work that you're trying to avoid?
>
> Do you have ideas about what the API you'd want would look like?
>
> [...snip...]
>
> Hope this is helpful
> Dave
>
> > >
> > > ---------- Forwarded message ----------
> > > From: Joshua Saxby <joshua.a.saxby@gmail.com>
> > > To: jit@gcc.gnu.org
> > > Cc:
> > > Bcc:
> > > Date: Sat, 10 Jun 2023 00:04:43 +0100
> > > Subject: Modifying the jit compiler API?
> > > Currently the JIT has two functions allowing you to compile a
> > > context
> > > either to memory or to file.
> > >
> > > But what if you want to compile to both? There doesn't seem to be
> > > any way
> > > to do this except by calling both functions separately which I
> > > believe will
> > > effectively be two separate compilations...
> > >
> > > Presumably, it should be possible to modify this part of the API to
> > > compile
> > > to some form of intermediate representation of the work that is
> > > common to
> > > both kinds of compilation, and then turn that into code in memory
> > > and in
> > > file, respectively.
> > >
> > > Anyone got any pointers for me on where in the code would be the
> > > best place
> > > for me to modify to dupport this? I did take a look in the
> > > implementation
> > > code of JIT sone weeks ago and remember seeing lots of complicated
> > > stuff
> > > regarding recordings and replays that looked relevant...
> > >
> > > A slighty simpler alteration I am also interested in making is
> > > allowing
> > > compile to file to compile to a buffer in memory as well as a file
> > > (I
> > > suppose it could be termed "compile to binary"). Maybe I should
> > > start with
> > > that...
> > >
> > >
> >
>