How do i debug --no-inline or --full-tailcalls options?

How do i debug --no-inline or --full-tailcalls options?

[OKUMURA Yuki]

Hi list,

I'm currently porting my own R7RS library into Kawa 2.1 but I'm
struggling to understand Kawa's compiler behaviour.

Some portions of my library require --no-inline or --full-tailcalls
options to compile correctly but
I think these should not be required on my code, at least reading on
the reference manual.

EXAMPLE1: --full-tailcalls
(https://github.com/okuoku/yuni/issues/33)

Following program requires --full-tailcalls to run with "java
kawa.repl -r7rs prog.sps".
Otherwise, Kawa gives java.lang.VerifyError.

(import (scheme base))

(define (ssplit-parse-byte0-ORIG byte)
  (case byte
    ((NONE) (values 'NONE #f))
    (else (values 'OTHERS #f))))

Why do we need --full-tailcalls here?
Removing CASE should compile fine so I guess there's something
different around (scheme base)
exported VALUES procedure here..


EXAMPLE2: --no-inline
(https://github.com/okuoku/yuni/issues/35)

Following program behaves differently with/without --no-inline option:

(import (scheme base)
        (scheme write)
        (scheme eval))

(let ((c (eval 'cons2 (environment '(rename (scheme base) (cons cons2))))))
 (write c)(newline))

With --no-inline option, EVAL will fail to bind renamed identifier so
it will fail to execute:

$ java kawa.repl --no-inline -r7rs check.sps
unbound location: cons2

Oddly, it is fine without --no-inline option:

$ java kawa.repl -r7rs check.sps
#<procedure cons>

AFAIK, other R7RSs Gauche, Chibi-scheme, Sagittarius and Chicken behaves same as
the latter(without --no-inline) case.
I expect Kawa also behaves same; why --no-inline changes program behaviour here?
The reference manual suggests mutating procedures would affect inliner
behaviour but I believe
RENAME in R7RS should not the case.

Sorry for vague/newbie questions but I'm totally getting lost..

Re: How do i debug --no-inline or --full-tailcalls options?

[Per Bothner]

On 02/11/2016 12:51 AM, OKUMURA Yuki wrote:

> EXAMPLE1: --full-tailcalls
> (https://github.com/okuoku/yuni/issues/33)
>
> Following program requires --full-tailcalls to run with "java
> kawa.repl -r7rs prog.sps".
> Otherwise, Kawa gives java.lang.VerifyError.
>
> (import (scheme base))
>
> (define (ssplit-parse-byte0-ORIG byte)
>    (case byte
>      ((NONE) (values 'NONE #f))
>      (else (values 'OTHERS #f))))
>
> Why do we need --full-tailcalls here?
> Removing CASE should compile fine so I guess there's something
> different around (scheme base)
> exported VALUES procedure here..

It looks like a bug relating to type-inference and related
code-generation.  I'll look into it tomorrow.

> EXAMPLE2: --no-inline
> (https://github.com/okuoku/yuni/issues/35)
>
> Following program behaves differently with/without --no-inline option:
>
> (import (scheme base)
>          (scheme write)
>          (scheme eval))
>
> (let ((c (eval 'cons2 (environment '(rename (scheme base) (cons cons2))))))
>   (write c)(newline))
>
> With --no-inline option, EVAL will fail to bind renamed identifier so
> it will fail to execute:
>
> $ java kawa.repl --no-inline -r7rs check.sps
> unbound location: cons2
>
> Oddly, it is fine without --no-inline option:
>
> $ java kawa.repl -r7rs check.sps
> #<procedure cons>

I would generally avoid using --no-inline.   It is not very well specified
or as well tested.  One problem is that the type-inference machinery is tied
in with the inlining machinery.

Why do you want to use --no-inlining?  One advantage I can see is beter
stack traces (at least when --no-full-tailcalls).  It might be reasonable to
retrict --no-inlining's effect to "normal" functions.

That said, I don't see any obvious reasons --no-inline shouldnt work in
this case, and if it's not too difficult it may be worth fixing. I'll take a look tomorrow.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: How do i debug --no-inline or --full-tailcalls options?

[OKUMURA Yuki]

Ah, thanks for the comments!

2016-02-11 23:01 GMT+09:00 Per Bothner <per@bothner.com>:
- snip -
> Why do you want to use --no-inlining?  One advantage I can see is beter
> stack traces (at least when --no-full-tailcalls).  It might be reasonable to
> retrict --no-inlining's effect to "normal" functions.

Use of --no-inline is just because it (accidentally) solved another problem:
(https://github.com/okuoku/yuni/issues/34)

I don't fully understand what is happening on that issue yet;
with --full-tailcalls and without --no-inline,
Kawa seems to skip some procedures when its result is not
consumed at all. (ie. every skipped procedures were wrapper around SET!)

I prepared (somewhat) minified program which exhibits the problem:

https://gist.github.com/okuoku/d5cba6f31b781db911aa

The code prepares 2 vectors which hold some data but when
I move some setters into a R7RS library, these setters are
mysteriously "skipped" and a vector field would left uninitialized.

Without --full-tailcall nor --no-inline, it works fine:

$ java kawa.repl --r7rs check3.scm
(SET!-CALLED: 0 test)
(SET!-CALLED: 1 #<procedure testfunc>)
(SET!-CALLED: 0 #(test #<procedure testfunc>))
(SET!-CALLED: 1 (INIT1 . INIT2))
(#(test #<procedure testfunc>) ("INIT1" . "INIT2"))

but with --full-tailcall, a SET!-CALLED message is missing;

$ java kawa.repl --r7rs --full-tailcalls check3.scm
(SET!-CALLED: 0 test)
(SET!-CALLED: 1 #<procedure testfunc>)
(SET!-CALLED: 1 (INIT1 . INIT2))
("UNINITIALIZED_x" ("INIT1" . "INIT2"))

this means a line here★

(define (make-minidispatch-obj class param)
  (let ((obj (make-minitype-obj)))
   (baseset! obj 0 class) ★
   (baseset! obj 1 param)
   obj))))

was skipped.


The problem is gone in case we added --no-inline:

$ java kawa.repl --r7rs --full-tailcalls --no-inline check3.scm
(SET!-CALLED: 0 test)
(SET!-CALLED: 1 #<procedure testfunc>)
(SET!-CALLED: 0 #(test #<procedure testfunc>))
(SET!-CALLED: 1 (INIT1 . INIT2))
(#(test #<procedure testfunc>) ("INIT1" . "INIT2"))

This is why I ended up specifying --no-inline.

I suspect this behaviour is backend's problem since when I merged
explicit libraries into a module, problem would just disappear.

Re: How do i debug --no-inline or --full-tailcalls options?

[Per Bothner]

On 02/11/2016 12:51 AM, OKUMURA Yuki wrote:
> EXAMPLE1: --full-tailcalls
> (https://github.com/okuoku/yuni/issues/33)
>
> Following program requires --full-tailcalls to run with "java
> kawa.repl -r7rs prog.sps".
> Otherwise, Kawa gives java.lang.VerifyError.
>
> (import (scheme base))
>
> (define (ssplit-parse-byte0-ORIG byte)
>    (case byte
>      ((NONE) (values 'NONE #f))
>      (else (values 'OTHERS #f))))
>
> Why do we need --full-tailcalls here?
> Removing CASE should compile fine so I guess there's something
> different around (scheme base)
> exported VALUES procedure here..

I've checked in a fix for this, but I'm not satisfied with it.
Or rather: The fix is fine, but there is some needless code
generated by the compiler.  I know how to fix that - but then
that causes other problems I'm still working on.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: How do i debug --no-inline or --full-tailcalls options?

[Per Bothner]

On 02/11/2016 05:26 PM, Per Bothner wrote:
>
>
> On 02/11/2016 12:51 AM, OKUMURA Yuki wrote:
>> EXAMPLE1: --full-tailcalls
>> (https://github.com/okuoku/yuni/issues/33)
>>
>> Following program requires --full-tailcalls to run with "java
>> kawa.repl -r7rs prog.sps".
>> Otherwise, Kawa gives java.lang.VerifyError.
>>
>> (import (scheme base))
>>
>> (define (ssplit-parse-byte0-ORIG byte)
>>    (case byte
>>      ((NONE) (values 'NONE #f))
>>      (else (values 'OTHERS #f))))
>>
>> Why do we need --full-tailcalls here?
>> Removing CASE should compile fine so I guess there's something
>> different around (scheme base)
>> exported VALUES procedure here..
>
> I've checked in a fix for this, but I'm not satisfied with it.
> Or rather: The fix is fine, but there is some needless code
> generated by the compiler.  I know how to fix that - but then
> that causes other problems I'm still working on.

I now have a fix for this program that I'm satisfied with,
and it's checked in.

I don't have an understanding yet of what causes your EXAMPLE 2
to fail.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: How do i debug --no-inline or --full-tailcalls options?

[OKUMURA Yuki]

2016-02-12 16:25 GMT+09:00 Per Bothner <per@bothner.com>:
- snip -
> I now have a fix for this program that I'm satisfied with,
> and it's checked in.

Great! Now my library tests are passing without --full-tailcalls nor
--no-inline.

$ run/kawa.sh _sanity.sps
(LOADING: "_sanity.sps")
Test: 110/110 passed.

$ java kawa.repl --version
Kawa 2.1.1 (revision 8704)
Copyright (C) 2015 Per Bothner

Re: How do i debug --no-inline or --full-tailcalls options?

[Per Bothner]

On 02/11/2016 10:26 AM, OKUMURA Yuki wrote:
> I don't fully understand what is happening on that issue yet;
> with --full-tailcalls and without --no-inline,
> Kawa seems to skip some procedures when its result is not
> consumed at all. (ie. every skipped procedures were wrapper around SET!)

This was a bug in the code generation.  When you compile with --full-tailcalls,
a function call is split into two parts:
(1) The first part pushes the arguments and a reference to the function
into the per-thread CallContext.
(2) The actual body of the function is a separate method that is executed by the
"trampoline loop" in CallContext.

If the call is a tail-call, then the caller returns after doing (1), and the trampoline
loop is done by caller's caller (and so on if the caller's call is also a tail-call).

The problem when the call is *not* a tail-call.  In that case, the
caller must make sure the trampoline loop is called so we get the
function's result (or in this case side effects).  This was not happening
in your test-case.

> I suspect this behaviour is backend's problem since when I merged
> explicit libraries into a module, problem would just disappear.

The define-library affects the code generation because it affects
what functions are exported and how things are inlined.

I checked in a fix, along with a simplified version of your testcase.  Thanks!
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/