Re: JIT compilation speed

[David Malcolm <dmalcolm@redhat.com>]

On Tue, 2015-07-28 at 00:20 +0200, Basile Starynkevitch wrote:
> On 07/27/2015 22:29, David Malcolm wrote:
> > On Sat, 2015-07-25 at 11:28 +0100, Dibyendu Majumdar wrote:
> >> Hi
> >>
> >> Is there a timeframe for the performance improvements to the JIT
> >> compilation speed?
> > The patch kit I posted here:
> >    https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00116.html
> > gives about a 5x speedup (from about 50ms down to about 10ms on my box),
> 
> 
> Of course, I'll be happy if that patch was eventually incorporated in 
> GCC. I guess that the speedup would be that big only for JIT-ing of 
> *small* gcc_jit_context-s with few (e.g. only hundreds) of 
> gcc_jit_object-s and using -O0 or -O1.

Indeed: test-benchmark.c is what I used: it simulates compiling hundreds
of small functions one at time, by compiling and run a function
containing a simple loop (sum-of-squares from 0..N) 100 times, at each
optimization level, then calculate the average time take for
compile-and-run at each optimization level.

> I guess that for JIT-ing or 
> larger contexts with -O2, the time for generating assembler code won't 
> dominate (since it does not dominate in GCC itself), so the effect of 
> the above patch would be much smaller...

The issue that the patch kit above resolves is overhead from the .s
to .so conversion:  libgccjit invokes the driver binary in a subprocess,
which in turn invokes "as" and "ld" binaries in subprocesses.  All of
this fork/exec and related activity is overhead, and the patch kit above
eliminates it by moving everything into the original process.

> However, I don't think that GCCJIT compilation time (I mean the time 
> needed for GCCJIT to emit machine code from GCCJIT objects) should 
> matter that much.

> The point is that, by design, GCCJIT is GCC based. And GCC is known to 
> compile quite slowly but to produce fast machine code. So GCCJIT also 
> will continue to compile a bit slowly but can be configured -by passing 
> -O2 using GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL, to produce fast machine 
> code.
> 
> In that respect, GCCJIT is a bit like LLVM (the library). It won't be 
> able to produce quickly some machine code, but it will be able to 
> produce, rather slowly, efficient machine code.
> 
> If the generation time matters much more than the performance of the 
> generated JIT-ed machine code, GCCJIT is not the right tool (and 
> probably would never be, in particular because -O0 is giving code which 
> is worse than what many other JIT-ing libraries do, and the GCCJIT 
> infrastructure won't compete with them).
> 
> There are several JIT libraries. On one hand, you have libraries like 
> GNU lightning and GNU libjit and asmjit. they all are able to emit 
> *quickly* some machine code (whose runtime performance is poor, but 
> which can be quickly generated). On the other hand, you have 
> compiler-related libraries like LLVM & GCCJIT. they can emit quite 
> performant machine code, at the cost of some quite slow JIT generation.

> Ultimately I would rather hope that eventually GCCJIT would be able to 
> even do, with some even more flags à la GCC_JIT_STRING_OPTION*, 
> link-time-optimization by doing the equivalent of what GCC is doing 
> today with -flto -O3. I cannot help on that (because LTO internals are 
> unfamiliar to me) and for my interests in GCCJIT today this is not a big 
> priority.
> 
> So, for Dibyendu Majumdar: if the GCCJIT compilation time matters a lot 
> to you in RAVI and the performance of the emitted machine code matters 
> much less to you, I am afraid that GCCJIT is not the optimal library for 
> your needs in RAVI (using GNU lightning or asmjit would make JIT-ing 
> time much faster, at the expense of a produced machine code which runs 2 
> to 5 times slower than what GCCJIT can achieve with -O1).

> If several JIT libraries (GCCJIT, LLVM, libjit, asmjit, GNU lightning) 
> are co-existing, it is because they have different position in the JIT 
> spectrum. On one side (GNU lightning, asmjit, libjit, ...) you have 
> libraries able to emit very quickly some slow machine code. On the other 
> side (LLVM, GCCJIT) you have compiler-based libraries which are emitting 
> slowly some quickly running machine code. Different needs have different 
> libraries! But we cannot realistically hope for a JIT library (or a C 
> compiler) which emits quickly some fast running machine code. There is 
> no such thing as a free lunch.

It depends how you define "quickly" :)

Generally it helps if people have some kind of expectation of what is
"quick enough".  I suspect that for libgccjit it needs to be "about as
quick as LLVM when doing the same thing".

FWIW, I don't think that "different needs have different libraries" is
particularly desirable.  I think that from the point of view of client
code, a single API with multiple settings would be better: it's a pain
to have to switch APIs (various VMs will do a quick jit for code that's
somewhat "hot", and then do a more involved jit-compile for code that's
very hot; it would be nice to be able to cater to both through one API).

It may be that there is potential for further optimizing gcc's code
generation, so that we can generate code even faster at the expense of
the speed of that code.  For example, I believe LLVM has a mode for
cheap instruction selection, where for compilation speed it can ignore
some of the ISA and use a simpler set of opcodes, to avoid spending a
lot of time figuring out which is the most optimal.  Whether that's a
good choice or not for us to work on depends on what profiling
indicates.

Profile, and optimize accordingly.

The patch kit mentioned above adds some infrastructure for doing timing,
so that ought to help.

> I praise David Malcolm for his good work on GCCJIT, which provides a 
> GCC-based JIT-ing library able to emit (a bit slowly, since based upon 
> GCC) some quickly running code, thanks to and leveraging above all the 
> optimizing infrastructure of GCC. See also this (mine) explanation 
> http://programmers.stackexchange.com/a/273711/40065 about why there are 
> so few C compilers!
> 
> GCCJIT has also (eventually) a very good side: it enables people 
> experimenting new language design to profit of GCC infrastructure 
> without having to generate C code.

(nods)   Though I like to think the "(eventually)" qualifier isn't
needed: it's actually possible to use libgccjit to build a "frontend" to
GCC today, without using the frontend hooks; see e.g.:
https://gcc.gnu.org/onlinedocs/gcc-5.2.0/jit/intro/tutorial05.html

I'm biased, of course :)

Dave