Re: error handling in GCCJIT?

Re: error handling in GCCJIT?

[David Malcolm]

On Thu, 2015-07-09 at 12:51 +0200, Basile Starynkevitch wrote:
> Hello all,
> 
> What is the design philosophy of GCCJIT w.r.t. errors? Is the calling
> application supposed to avoid as much as possible any errors,

Yes.

>  or on
> the contrary can it rely on GCCJIT error checking and not do the
> checking itself?

If an error occurs, that gcc_jit_context * effectively becomes unusable:
it attempts to gracefully continue without crashing, but you won't get
code out at the end.

The idea is that all errors need to be fixed at development time, but to
fail gracefully in production, without needing lots of run-time
error-checking in client code at each callsite, 

FWIW, this is a somewhat similar approach to how OpenGL handles errors.

Most of this error-checking occurs in gcc/jit/libgccjit.c.

> For examples:
> 
>    should the caller check that every case in a switch statement 
> is unique and non-overlapping for case ranges, 

Yes, the client code needs to ensure that those requirements hold...

> or should it leave that to GCCJIT?

If the requirements don't hold, then the call to
gcc_jit_block_end_with_switch will fail with an error, and the
gcc_jit_context * won't generate any code.

See e.g.:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/jit.dg/test-error-gcc_jit_block_end_with_switch-overlapping-ranges.c

>    should the caller check that arithmetic operations are only done on
> numeric variables I guess that gcc_jit_context_new_binary_op with
> GCC_JIT_BINARY_OP_PLUS of two pointer variables is a non-sense, as it
> is in C.

Yes, the caller should enforce this requirement...

> Or is that checked (and does gcc_jit_context_new_binary_op
> returns NULL in that case)?

It looks like gcc_jit_context_new_binary_op is missing error-handling
for that case.  That's a bug.

>    what happens (and when?) if gcc_jit_block_end_with_conditional is
>    given a non-boolean boolval?

It will fail with an error; c.f.:

  RETURN_IF_FAIL_PRINTF2 (
   is_bool (boolval), ctxt, loc,
   "%s (type: %s) is not of boolean type ",
   boolval->get_debug_string (),
   boolval->get_type ()->get_debug_string ());

> etc....
> 
> PS. Perhaps the documentation might tell more about this...

See:
https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#error-handling
and
https://gcc.gnu.org/onlinedocs/jit/internals/index.html#design-notes

Please file bugs about any missing error-checking you see, or missing
documentation.

error handling in GCCJIT?

[Basile Starynkevitch]

Hello all,

What is the design philosophy of GCCJIT w.r.t. errors? Is the calling
application supposed to avoid as much as possible any errors, or on
the contrary can it rely on GCCJIT error checking and not do the
checking itself?

For examples:

   should the caller check that every case in a switch statement 
is unique and non-overlapping for case ranges, or should it leave that to GCCJIT?

   should the caller check that arithmetic operations are only done on
numeric variables I guess that gcc_jit_context_new_binary_op with
GCC_JIT_BINARY_OP_PLUS of two pointer variables is a non-sense, as it
is in C. Or is that checked (and does gcc_jit_context_new_binary_op
returns NULL in that case)?

   what happens (and when?) if gcc_jit_block_end_with_conditional is
   given a non-boolean boolval?

etc....

PS. Perhaps the documentation might tell more about this...

Regards.
-- 
Basile Starynkevitch         http://starynkevitch.net/Basile/
France

Re: error handling in GCCJIT?

[Basile Starynkevitch]

On 07/09/2015 04:05 PM, David Malcolm wrote:
> On Thu, 2015-07-09 at 12:51 +0200, Basile Starynkevitch wrote:
>> Hello all,
>>
>> What is the design philosophy of GCCJIT w.r.t. errors? Is the calling
>> application supposed to avoid as much as possible any errors,
> Yes.
>
>>   or on
>> the contrary can it rely on GCCJIT error checking and not do the
>> checking itself?
> If an error occurs, that gcc_jit_context * effectively becomes unusable:
> it attempts to gracefully continue without crashing, but you won't get
> code out at the end.

This is OK for me, and I don't expect GCCJIT to be able to do anything 
more.
So I don't expect, in cases of errors, to get any code; but it would be 
nice to be at least sure (or be able to reasonably hope)
that the process does not crash. Actually, giving only one error message 
would be ok for me.

>
> The idea is that all errors need to be fixed at development time, but to
> fail gracefully in production, without needing lots of run-time
> error-checking in client code at each callsite,
>
> FWIW, this is a somewhat similar approach to how OpenGL handles errors.
>
> Most of this error-checking occurs in gcc/jit/libgccjit.c.
>
>> For examples:
>>
>>     should the caller check that every case in a switch statement
>> is unique and non-overlapping for case ranges,
> Yes, the client code needs to ensure that those requirements hold...
>
>> or should it leave that to GCCJIT?
> If the requirements don't hold, then the call to
> gcc_jit_block_end_with_switch will fail with an error, and the
> gcc_jit_context * won't generate any code.
>
> See e.g.:
> https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/jit.dg/test-error-gcc_jit_block_end_with_switch-overlapping-ranges.c

So an error message is certain, and GCCJIT won't crash? Then I feel I do 
not have to check that.
I don't mind that GCCJIT gives me only one error in that case.

>>     should the caller check that arithmetic operations are only done on
>> numeric variables I guess that gcc_jit_context_new_binary_op with
>> GCC_JIT_BINARY_OP_PLUS of two pointer variables is a non-sense, as it
>> is in C.
> Yes, the caller should enforce this requirement...
>
>> Or is that checked (and does gcc_jit_context_new_binary_op
>> returns NULL in that case)?
> It looks like gcc_jit_context_new_binary_op is missing error-handling
> for that case.  That's a bug.

Hope it would be corrected (or at least documented as undefined behavior).
>
>>     what happens (and when?) if gcc_jit_block_end_with_conditional is
>>     given a non-boolean boolval?
> It will fail with an error; c.f.:
>
>    RETURN_IF_FAIL_PRINTF2 (
>     is_bool (boolval), ctxt, loc,
>     "%s (type: %s) is not of boolean type ",
>     boolval->get_debug_string (),
>     boolval->get_type ()->get_debug_string ());
>
>> etc....
>>
>> PS. Perhaps the documentation might tell more about this...
> See:
> https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#error-handling
>

There is another kind of error that a JIT engine is more keen to 
encounter. Resource exhaustion.
To be more precise, first assume that we use GCCJIT with optimization 
turned on, i.e. we call gcc_jit_context_set_int_option with
GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL at least set to 1. This is in 
practice extremely likely: a JIT coder which do not care about 
optimizations would not choose GCCJIT, but some other libraries, such as 
libjit or GNU lightning, which are simpler to use.

So we are using GCCJIT with -O1 at least.

Then, since GCCJIT is GCC based, it is well known (and I encountered 
that in the beginning of my work on MELT) that experimentally the 
practical complexity of compilation time and memory is essentially 
O(n^2) - i.e. square complexity - w.r.t. the size n of the biggest 
routine. For simplicity, we can assume that we are compiling only one 
routine thru GCCJIT.

On some occasions, it is not uncommon when JIT-ing or just generating 
code to have huge routines (e.g. a function with 50K basic blocks and 
many hundred thousands of simple instructions). In such cases, it could 
happen that GCC itself could fail (e.g. with an internal malloc failure 
deep inside GCC).
What would happen then at the GCCJIT level? Do we get a recoverable 
error? Or would our GCCJIT-ing process badly crash?
If there is a recoverable error, are we sure (after having noticed that 
no code can be produced by GCCJIT) that almost all internal resources 
used by GCCJIT are released after we called gcc_jit_context_release 
(after gcc_jit_context_compile failed by returning NULL)? Or should be 
expect huge memory leaks?

Cheers.


-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***