JIT compilation speed

JIT compilation speed

[Dibyendu Majumdar]

Hi

Is there a timeframe for the performance improvements to the JIT
compilation speed?

Thanks and Regards
Dibyendu

Re: JIT compilation speed

[David Malcolm]

On Sat, 2015-07-25 at 11:28 +0100, Dibyendu Majumdar wrote:
> Hi
> 
> Is there a timeframe for the performance improvements to the JIT
> compilation speed?

The patch kit I posted here:
  https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00116.html
gives about a 5x speedup (from about 50ms down to about 10ms on my box),
but parts of it are very invasive, so unlikely to be backported to the
gcc 5 branch.  *If* I get it in, it'll be a gcc 6 thing (hence perhaps
April 2016 in terms of an official release).

Is the compilation speed something you're looking at for Ravi?  What
kind of times are you seeing, and what were you hoping to see?

Dave

Re: JIT compilation speed

[Basile Starynkevitch]

On 07/27/2015 22:29, David Malcolm wrote:
> On Sat, 2015-07-25 at 11:28 +0100, Dibyendu Majumdar wrote:
>> Hi
>>
>> Is there a timeframe for the performance improvements to the JIT
>> compilation speed?
> The patch kit I posted here:
>    https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00116.html
> gives about a 5x speedup (from about 50ms down to about 10ms on my box),


Of course, I'll be happy if that patch was eventually incorporated in 
GCC. I guess that the speedup would be that big only for JIT-ing of 
*small* gcc_jit_context-s with few (e.g. only hundreds) of 
gcc_jit_object-s and using -O0 or -O1. I guess that for JIT-ing or 
larger contexts with -O2, the time for generating assembler code won't 
dominate (since it does not dominate in GCC itself), so the effect of 
the above patch would be much smaller...

However, I don't think that GCCJIT compilation time (I mean the time 
needed for GCCJIT to emit machine code from GCCJIT objects) should 
matter that much.

The point is that, by design, GCCJIT is GCC based. And GCC is known to 
compile quite slowly but to produce fast machine code. So GCCJIT also 
will continue to compile a bit slowly but can be configured -by passing 
-O2 using GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL, to produce fast machine 
code.

In that respect, GCCJIT is a bit like LLVM (the library). It won't be 
able to produce quickly some machine code, but it will be able to 
produce, rather slowly, efficient machine code.

If the generation time matters much more than the performance of the 
generated JIT-ed machine code, GCCJIT is not the right tool (and 
probably would never be, in particular because -O0 is giving code which 
is worse than what many other JIT-ing libraries do, and the GCCJIT 
infrastructure won't compete with them).

There are several JIT libraries. On one hand, you have libraries like 
GNU lightning and GNU libjit and asmjit. they all are able to emit 
*quickly* some machine code (whose runtime performance is poor, but 
which can be quickly generated). On the other hand, you have 
compiler-related libraries like LLVM & GCCJIT. they can emit quite 
performant machine code, at the cost of some quite slow JIT generation.

Ultimately I would rather hope that eventually GCCJIT would be able to 
even do, with some even more flags à la GCC_JIT_STRING_OPTION*, 
link-time-optimization by doing the equivalent of what GCC is doing 
today with -flto -O3. I cannot help on that (because LTO internals are 
unfamiliar to me) and for my interests in GCCJIT today this is not a big 
priority.

So, for Dibyendu Majumdar: if the GCCJIT compilation time matters a lot 
to you in RAVI and the performance of the emitted machine code matters 
much less to you, I am afraid that GCCJIT is not the optimal library for 
your needs in RAVI (using GNU lightning or asmjit would make JIT-ing 
time much faster, at the expense of a produced machine code which runs 2 
to 5 times slower than what GCCJIT can achieve with -O1).

If several JIT libraries (GCCJIT, LLVM, libjit, asmjit, GNU lightning) 
are co-existing, it is because they have different position in the JIT 
spectrum. On one side (GNU lightning, asmjit, libjit, ...) you have 
libraries able to emit very quickly some slow machine code. On the other 
side (LLVM, GCCJIT) you have compiler-based libraries which are emitting 
slowly some quickly running machine code. Different needs have different 
libraries! But we cannot realistically hope for a JIT library (or a C 
compiler) which emits quickly some fast running machine code. There is 
no such thing as a free lunch.

I praise David Malcolm for his good work on GCCJIT, which provides a 
GCC-based JIT-ing library able to emit (a bit slowly, since based upon 
GCC) some quickly running code, thanks to and leveraging above all the 
optimizing infrastructure of GCC. See also this (mine) explanation 
http://programmers.stackexchange.com/a/273711/40065 about why there are 
so few C compilers!

GCCJIT has also (eventually) a very good side: it enables people 
experimenting new language design to profit of GCC infrastructure 
without having to generate C code.

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: JIT compilation speed

[Dibyendu Majumdar]

On 27 July 2015 at 23:20, Basile Starynkevitch <basile@starynkevitch.net> wrote:
>>> Is there a timeframe for the performance improvements to the JIT
>>> compilation speed?
>>
>> The patch kit I posted here:
>>    https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00116.html
>> gives about a 5x speedup (from about 50ms down to about 10ms on my box),
>

Ok, that seems good improvement. I don't have a specific benchmark but
I have noticed that when I enable full compilation the Lua test suite
takes 4-5 times longer to run compared to LLVM - in both cases the JIT
compilation time dominates - so improvement of 5x may put libgccjit in
the same ballpark as LLVM. That would be good enough to start with.

>
> So, for Dibyendu Majumdar: if the GCCJIT compilation time matters a lot to
> you in RAVI and the performance of the emitted machine code matters much
> less to you, I am afraid that GCCJIT is not the optimal library for your
> needs in RAVI (using GNU lightning or asmjit would make JIT-ing time much
> faster, at the expense of a produced machine code which runs 2 to 5 times
> slower than what GCCJIT can achieve with -O1).
>

Yes understand that - I think that libgccjit needs to be at least as
efficient as LLVM - that should be good enough.

Regards
Dibyendu

Re: JIT compilation speed

[David Malcolm]

On Mon, 2015-07-27 at 23:52 +0100, Dibyendu Majumdar wrote:
> On 27 July 2015 at 23:20, Basile Starynkevitch <basile@starynkevitch.net> wrote:
> >>> Is there a timeframe for the performance improvements to the JIT
> >>> compilation speed?
> >>
> >> The patch kit I posted here:
> >>    https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00116.html
> >> gives about a 5x speedup (from about 50ms down to about 10ms on my box),
> >
> 
> Ok, that seems good improvement. I don't have a specific benchmark but
> I have noticed that when I enable full compilation the Lua test suite
> takes 4-5 times longer to run compared to LLVM - in both cases the JIT
> compilation time dominates - so improvement of 5x may put libgccjit in
> the same ballpark as LLVM. That would be good enough to start with.

FWIW, the timer patch:
 https://gcc.gnu.org/ml/gcc-patches/2015-07/msg02715.html
is now in trunk (r226530).

So if you're using trunk, you should be able to:

  /* One-time initialization. */
  gcc_jit_timer *timer = gcc_jit_timer_new ();



  /* For all of your contexts, have them all use the
     same shared timer:  */
  gcc_jit_context_set_timer (ctxt, timer);



  /* On shutdown, print the accumulated timings.  */
  gcc_jit_timer_print (timer, stderr);
  gcc_jit_timer_release (timer);

to get a profile on stderr of where all that time is being spent.

[snip]


Dave

Re: JIT compilation speed

[Dibyendu Majumdar]

On 4 August 2015 at 01:18, David Malcolm <dmalcolm@redhat.com> wrote:
> FWIW, the timer patch:
>  https://gcc.gnu.org/ml/gcc-patches/2015-07/msg02715.html
> is now in trunk (r226530).
>
> So if you're using trunk, you should be able to:
>
>   /* One-time initialization. */
>   gcc_jit_timer *timer = gcc_jit_timer_new ();
>
>
>
>   /* For all of your contexts, have them all use the
>      same shared timer:  */
>   gcc_jit_context_set_timer (ctxt, timer);
>
>
>
>   /* On shutdown, print the accumulated timings.  */
>   gcc_jit_timer_print (timer, stderr);
>   gcc_jit_timer_release (timer);
>
> to get a profile on stderr of where all that time is being spent.
>

Thanks Dave. I am currently tied up with nailing down an elusive bug
in Ravi - and have not had time to look at libgccjit related work. But
I hope to get back to it soon. Will keep you posted.

Regards
Dibyendu

Re: JIT compilation speed

[Basile Starynkevitch]

On 07/28/2015 00:52, Dibyendu Majumdar wrote:
>
>> So, for Dibyendu Majumdar: if the GCCJIT compilation time matters a lot to
>> you in RAVI and the performance of the emitted machine code matters much
>> less to you, I am afraid that GCCJIT is not the optimal library for your
>> needs in RAVI (using GNU lightning or asmjit would make JIT-ing time much
>> faster, at the expense of a produced machine code which runs 2 to 5 times
>> slower than what GCCJIT can achieve with -O1).
>>
> Yes understand that - I think that libgccjit needs to be at least as
> efficient as LLVM - that should be good enough.


Given that today Clang/LLVM is significantly quicker to compile C code  
than GCC is for -O1, that won't happen very soon. GCCJIT is GCC based, 
so will continue to keep the heavy weight of GCC.

Some people within GCC are trying to make it work faster (as a C or C++ 
compiler). When that would happen (and it might take years) GCCJIT will 
also work faster. But don't hope much of that for this 2015 year (and 
probably not even for 2016).

GCCJIT is GCC based, so cannot be faster than GCC is!

Between GCCJIT & LLVM, I guess that GCCJIT -when used with -O3- will 
produce slightly faster machine code than what LLVM can give, at the 
expense of a significantly slower JIT time.

Look at GCCJIT as something GCC based, without the inconvenience of 
having to generate by yourself some C code. BTW, in my MELT experience, 
generating C code and compiling and dlopen-ing it, can be fast enough 
for some interactive usage.

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: JIT compilation speed

[David Malcolm]

On Tue, 2015-07-28 at 01:00 +0200, Basile Starynkevitch wrote:
> On 07/28/2015 00:52, Dibyendu Majumdar wrote:
> >
> >> So, for Dibyendu Majumdar: if the GCCJIT compilation time matters a lot to
> >> you in RAVI and the performance of the emitted machine code matters much
> >> less to you, I am afraid that GCCJIT is not the optimal library for your
> >> needs in RAVI (using GNU lightning or asmjit would make JIT-ing time much
> >> faster, at the expense of a produced machine code which runs 2 to 5 times
> >> slower than what GCCJIT can achieve with -O1).
> >>
> > Yes understand that - I think that libgccjit needs to be at least as
> > efficient as LLVM - that should be good enough.
> 
> 
> Given that today Clang/LLVM is significantly quicker to compile C code  
> than GCC is for -O1, that won't happen very soon. GCCJIT is GCC based, 
> so will continue to keep the heavy weight of GCC.

I don't know that -O1 in gcc vs -O1 in llvm is an "apples-to-apples"
comparison; different compilers may have a different idea of what
optimizations should be enabled at each numbered level.

> Some people within GCC are trying to make it work faster (as a C or C++ 
> compiler). When that would happen (and it might take years) GCCJIT will 
> also work faster. But don't hope much of that for this 2015 year (and 
> probably not even for 2016).
> 
> GCCJIT is GCC based, so cannot be faster than GCC is!

That said, we could tweak GCC for specific workloads.  For example, each
-O level expresses a particular combination of passes.  libgccjit could
potentially build an alternate pass_manager for a particular
gcc_jit_context, and build a specifically-tuned tree of passes,
eliminating passes that aren't wanted, and perhaps even having custom
passes.  I'm not sure how to express any of that in the API though.

> Between GCCJIT & LLVM, I guess that GCCJIT -when used with -O3- will 
> produce slightly faster machine code than what LLVM can give, at the 
> expense of a significantly slower JIT time.
> 
> Look at GCCJIT as something GCC based, without the inconvenience of 
> having to generate by yourself some C code. BTW, in my MELT experience, 
> generating C code and compiling and dlopen-ing it, can be fast enough 
> for some interactive usage.
> 
> Regards.
> 

Re: JIT compilation speed

[David Malcolm]

On Tue, 2015-07-28 at 00:20 +0200, Basile Starynkevitch wrote:
> On 07/27/2015 22:29, David Malcolm wrote:
> > On Sat, 2015-07-25 at 11:28 +0100, Dibyendu Majumdar wrote:
> >> Hi
> >>
> >> Is there a timeframe for the performance improvements to the JIT
> >> compilation speed?
> > The patch kit I posted here:
> >    https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00116.html
> > gives about a 5x speedup (from about 50ms down to about 10ms on my box),
> 
> 
> Of course, I'll be happy if that patch was eventually incorporated in 
> GCC. I guess that the speedup would be that big only for JIT-ing of 
> *small* gcc_jit_context-s with few (e.g. only hundreds) of 
> gcc_jit_object-s and using -O0 or -O1.

Indeed: test-benchmark.c is what I used: it simulates compiling hundreds
of small functions one at time, by compiling and run a function
containing a simple loop (sum-of-squares from 0..N) 100 times, at each
optimization level, then calculate the average time take for
compile-and-run at each optimization level.

> I guess that for JIT-ing or 
> larger contexts with -O2, the time for generating assembler code won't 
> dominate (since it does not dominate in GCC itself), so the effect of 
> the above patch would be much smaller...

The issue that the patch kit above resolves is overhead from the .s
to .so conversion:  libgccjit invokes the driver binary in a subprocess,
which in turn invokes "as" and "ld" binaries in subprocesses.  All of
this fork/exec and related activity is overhead, and the patch kit above
eliminates it by moving everything into the original process.

> However, I don't think that GCCJIT compilation time (I mean the time 
> needed for GCCJIT to emit machine code from GCCJIT objects) should 
> matter that much.

> The point is that, by design, GCCJIT is GCC based. And GCC is known to 
> compile quite slowly but to produce fast machine code. So GCCJIT also 
> will continue to compile a bit slowly but can be configured -by passing 
> -O2 using GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL, to produce fast machine 
> code.
> 
> In that respect, GCCJIT is a bit like LLVM (the library). It won't be 
> able to produce quickly some machine code, but it will be able to 
> produce, rather slowly, efficient machine code.
> 
> If the generation time matters much more than the performance of the 
> generated JIT-ed machine code, GCCJIT is not the right tool (and 
> probably would never be, in particular because -O0 is giving code which 
> is worse than what many other JIT-ing libraries do, and the GCCJIT 
> infrastructure won't compete with them).
> 
> There are several JIT libraries. On one hand, you have libraries like 
> GNU lightning and GNU libjit and asmjit. they all are able to emit 
> *quickly* some machine code (whose runtime performance is poor, but 
> which can be quickly generated). On the other hand, you have 
> compiler-related libraries like LLVM & GCCJIT. they can emit quite 
> performant machine code, at the cost of some quite slow JIT generation.

> Ultimately I would rather hope that eventually GCCJIT would be able to 
> even do, with some even more flags à la GCC_JIT_STRING_OPTION*, 
> link-time-optimization by doing the equivalent of what GCC is doing 
> today with -flto -O3. I cannot help on that (because LTO internals are 
> unfamiliar to me) and for my interests in GCCJIT today this is not a big 
> priority.
> 
> So, for Dibyendu Majumdar: if the GCCJIT compilation time matters a lot 
> to you in RAVI and the performance of the emitted machine code matters 
> much less to you, I am afraid that GCCJIT is not the optimal library for 
> your needs in RAVI (using GNU lightning or asmjit would make JIT-ing 
> time much faster, at the expense of a produced machine code which runs 2 
> to 5 times slower than what GCCJIT can achieve with -O1).

> If several JIT libraries (GCCJIT, LLVM, libjit, asmjit, GNU lightning) 
> are co-existing, it is because they have different position in the JIT 
> spectrum. On one side (GNU lightning, asmjit, libjit, ...) you have 
> libraries able to emit very quickly some slow machine code. On the other 
> side (LLVM, GCCJIT) you have compiler-based libraries which are emitting 
> slowly some quickly running machine code. Different needs have different 
> libraries! But we cannot realistically hope for a JIT library (or a C 
> compiler) which emits quickly some fast running machine code. There is 
> no such thing as a free lunch.

It depends how you define "quickly" :)

Generally it helps if people have some kind of expectation of what is
"quick enough".  I suspect that for libgccjit it needs to be "about as
quick as LLVM when doing the same thing".

FWIW, I don't think that "different needs have different libraries" is
particularly desirable.  I think that from the point of view of client
code, a single API with multiple settings would be better: it's a pain
to have to switch APIs (various VMs will do a quick jit for code that's
somewhat "hot", and then do a more involved jit-compile for code that's
very hot; it would be nice to be able to cater to both through one API).

It may be that there is potential for further optimizing gcc's code
generation, so that we can generate code even faster at the expense of
the speed of that code.  For example, I believe LLVM has a mode for
cheap instruction selection, where for compilation speed it can ignore
some of the ISA and use a simpler set of opcodes, to avoid spending a
lot of time figuring out which is the most optimal.  Whether that's a
good choice or not for us to work on depends on what profiling
indicates.

Profile, and optimize accordingly.

The patch kit mentioned above adds some infrastructure for doing timing,
so that ought to help.

> I praise David Malcolm for his good work on GCCJIT, which provides a 
> GCC-based JIT-ing library able to emit (a bit slowly, since based upon 
> GCC) some quickly running code, thanks to and leveraging above all the 
> optimizing infrastructure of GCC. See also this (mine) explanation 
> http://programmers.stackexchange.com/a/273711/40065 about why there are 
> so few C compilers!
> 
> GCCJIT has also (eventually) a very good side: it enables people 
> experimenting new language design to profit of GCC infrastructure 
> without having to generate C code.

(nods)   Though I like to think the "(eventually)" qualifier isn't
needed: it's actually possible to use libgccjit to build a "frontend" to
GCC today, without using the frontend hooks; see e.g.:
https://gcc.gnu.org/onlinedocs/gcc-5.2.0/jit/intro/tutorial05.html

I'm biased, of course :)

Dave