Splicing lists and vectors into argument lists

Splicing lists and vectors into argument lists

[Per Bothner]

Please try out this new experimental Kawa feature:
The form '@EXPRESSION' (which is reader syntax
for ($splice$ EXPRESSION)) can be used in a
function application.  The EXPRESSION must evaluate
to a list or vector (more generally a java.util.List)
*or* a native Java array.  The effect is that each
element becomes a separate argument.

For example if lst is a list, then
   (fun x y @lst)
is the same as:
   (apply fun x y lst)

To sum the elements of list/vector/array lst, do:
   (+ @lst)

To concate vectors vec1 and vec2 separated by a single value x, do:
   [@vec1 x @vec2]

Splice notation isn't optimized much so far. If an application contains
a splice argument then we just create an array of the elements and
invoke the applyN method.  This means there is no function-specific
optimization.  Which also means if inlined compilation behaves
differently from run-time application you get the latter behavior.
(This can happen when resolving overloaded methods, for example.)

The plan is to do some optimization, starting with array/list
constructors.  Calling varargs methods can also be optimized: If all
the splice arguments correspond to a varargs array, then the compiler
should generate code to create the varargs array and then emit a regular
'invoke' instruction.

Right now Kawa follows Java when handling varargs, in that
you can optionally in the final argument pass an explicit array
which is treated as a varargs array.  This will be deprecated
soon, and ultimately disallowed.  E.g. the following is currently
allowed:
   (define args (object[] 4 5 6))
   (java.lang.String:format "<x: %s y: %s z: %s>" args) ; DEPRECATED
       ;; as equivalent to:
   (java.lang.String:format "<x: %s y: %s z: %s>" 4 5 6)
In the future this will have to be written:
   (java.lang.String:format "<x: %s y: %s z: %s>" @args)

Note that the syntax (@TYPE ARG ...) is also used for annotations.
That usage don't conflict for a number of reasons:
- The @TYPE syntax for an annotation is only allowed in function
call position, while a slice is *not* allowed in function call position.
- The "expression" following @ for an annotation must be a class or type,
while for a slice it must be a list/vector/array.
- Annotations are only supported in restricted contexts.

The old syntax (TYPE:@ VALUE) for coercions still works,
but deprecated.  Use (->TYPE VALUE) instead.

However, there is *some* compatibility breakage, since '@' no longer starts
a symbol.  Thus for example I had to quote |Q| in syntaxutils.scm.  This
macro could presumably be replaced by colon notation.

Finally: Don't use keyword values in the elements of splice.
This will soon stop working, but that is a topic for another email.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Splicing lists and vectors into argument lists

[Helmut Eller]

On Fri, May 09 2014, Per Bothner wrote:

> The form '@EXPRESSION' (which is reader syntax
> for ($splice$ EXPRESSION))

Is there an option analog to --output-format to change the reader syntax
to something stable and standard?

Helmut

Re: Splicing lists and vectors into argument lists

[Per Bothner]

On 05/10/2014 12:26 AM, Helmut Eller wrote:
> On Fri, May 09 2014, Per Bothner wrote:
>
>> The form '@EXPRESSION' (which is reader syntax
>> for ($splice$ EXPRESSION))
>
> Is there an option analog to --output-format to change the reader syntax
> to something stable and standard?

There is a  mechanism for for that.  Kawa has --r5rs --r6rs and --r7rs
command-line options which can tweak the dialect.  Right now these
options doesn't do much - it looks like it only disables colon notation.
There is also a --pedantic option, which currently only affects the XQuery
parser, but can be extended.

This framework can be extended in various ways.

A issue is that initial '@' in a symbol is *not* allowed by R7RS or R6RS.
('@' is a <special subsequent> so is not allowed as an initial character.)
Thus disabling the '@' reader syntax for splice is not appropriate for
a --r7rs flag, which is intended for areas where Kawa conflicts with a
strict reading of R7RS, not disabling extensions.  In this case we're
conflicting with a previous Kawa extension, not any standard.  We could
consider a more generic flag like --compat=VERSION - i.e. --compat=kawa-1.10.
That might promise more than we can deliver, though it can be helpful for
occasional frustrations like this.

Another possibility: If the main problem you're having is with @ as a
single-letter symbol, I could special case a hack for that (e.g. '@'
followed by whitespace or ')' or ']'), emitting a once-per-compilation
deprecation warning.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Splicing lists and vectors into argument lists

[Helmut Eller]

On Sat, May 10 2014, Per Bothner wrote:

>> Is there an option analog to --output-format to change the reader syntax
>> to something stable and standard?
>
> There is a  mechanism for for that.  Kawa has --r5rs --r6rs and --r7rs
[...]
> A issue is that initial '@' in a symbol is *not* allowed by R7RS or
> R6RS. ('@' is a <special subsequent> so is not allowed as an initial
> character.)

Oh boy.  They let you put Unicode chars there but no @.  Oh wait, they
actually say "it is an error for the first character to have a general
category of Nd, Mc, or Me."  But @ belongs to category Po.  And again
they use the meaningless phrase "it is an error" which of course only
means that it may or may not be allowed by an implementation.

[...]

> Thus disabling the '@' reader syntax for splice is not appropriate for
> a --r7rs flag, which is intended for areas where Kawa conflicts with a
> strict reading of R7RS, not disabling extensions.  In this case we're
> conflicting with a previous Kawa extension, not any standard.  We could
> consider a more generic flag like --compat=VERSION - i.e. --compat=kawa-1.10.
> That might promise more than we can deliver, though it can be helpful for
> occasional frustrations like this.

Don't bother.  It's easier if I use an old Kawa version directly.

Helmut

Re: Splicing lists and vectors into argument lists

[Per Bothner]

On 05/10/2014 11:49 AM, Helmut Eller wrote:
> On Sat, May 10 2014, Per Bothner wrote:
>
>> A issue is that initial '@' in a symbol is *not* allowed by R7RS or
>> R6RS. ('@' is a <special subsequent> so is not allowed as an initial
>> character.)
>
> Oh boy.  They let you put Unicode chars there but no @.  Oh wait, they
> actually say "it is an error for the first character to have a general
> category of Nd, Mc, or Me."  But @ belongs to category Po.  And again
> they use the meaningless phrase "it is an error" which of course only
> means that it may or may not be allowed by an implementation.

I just noticed the r7rs errata:
http://trac.sacrideo.us/wg/wiki/R7RSSmallErrata

     7, In Section 7.1.1, the lexical rule <special initial> incorrectly omits @.

Since (in my reading) r6rs also allows an initial '@', I think it would be
correct for the --r7rs/--r6rs/--r5rs flags to disable the '@' reader
mapping from @EXPR to ($splice$ EXPR).  (r5rs doesn't allow an initial '@',
but allowing it would still be a compatible extension, so for simplify
--r5rs behaves the same.)

So I checked in a fix for this:

$ kawa --r7rs
#|kawa:1|# (define @ 3)
#|kawa:2|# (define @2 2)
#|kawa:3|# (+ @ @2)
5
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/