Kawa type-conversions

Kawa type-conversions

[Per Bothner]

[Some thoughts about Kawa type-checking and type-conversions.
I don't know when any of this will be implemented, but this is
what I'm currently thinking makes most sense - though some details
would be need to be worked out in practice.  Maybe we can think
of this as a sketch for a Kawa Enhancement Proposal.]

Right now Kawa has multiple syntaxes for type conversion:

(as TYPE VALUE)

(->TYPE VALUE)

(define VAR ::TYPE VALUE) ;; and other variants using type specifiers

These mostly allow the same set of conversions, but using 'as' is more
"lenient" than '->TYPE' in that it allows also conversions of the the
raw types are the same.  (This is made use of in converting between the
tring-cursor type and int, for example.)

We also have type tests:

(? VAR ::TYPE VALUE) ;; generalizable to patterns: (? PATTERN VALUE)

(instance? VALUE TYPE)

These aren't really consistent:

(if (? x::int 3.4) x #f) ==> 3
(instance? 3.4 int) ==> 3

At some point I'd like to experiment with a --strict-typing option,
so it's desirable to come up consistent and useful model.

So what I'm thinking we shoudl aim for:

* (as TYPE VALUE) would be short-hand for
   (let ((TMP ::TYPE VALUE)) TMP)
and so the set of allowable conversions would be the same.

* Using 'as' or a type-specifier would invoke an *implicit*
conversion.  Using (->TYPE VALUE) would be an *explicit* conversion,
and would allow more conversions.

* As a general rule implicit conversion of VALUE to TYPE should be
OK if (instance? VALUE TYPE) or if it's a loss-less conversion to
semantically more-or-less the same type.  We may need to fine-tune this.

* As a general rule (? VAR::TYPE VALUE) should succeed if
(as TYPE VALUE) doesn't throw an exception. We may need to fine-tune this.

* It seems reasonable to introduce:
   (convert TYPE VALUE)
as equivalent to:
   (->TYPE VALUE)

* As an example (->list SEQUENCE) would convert any SEQUENCE value
to a list.  It would be equivalent to (list @SEQUENCE) except it
wouldn't necessarily make a fresh copy if the SEQUENCE is a list.

   (->list #(3 4 5)) ==> (3 4 5)
   (as list #(3 4 5)) ==> ERROR
   (define x::list #(3 4 5)) ==> ERROR

* Assume we a new type sequence, optionally parameterized as
sequence[ELEMENT-TYPE].  This would match any type allowed
for splicing or the new generalized map, including strings
and native Java arrays.  Since all of these types are "instances"
of the sequence types, all of these types can be *implicitly*
converted to sequence - even if we have to allocate a wrapper
object.

In general, special types that aren't just native Java types
might require custom conversion methods - and may need different
methods for implicit and explicit conversions.

* As an incompatible change, converting float to int should (IMO) no
longer be allowed for implicit conversions:

   (->int 3.4) ==> 3
   (as int 3.4) ==> ERROR ;; incompatible change
   (define x::int 3.4) ==> ERROR ;; incompatible change

* In general static type-checking wouldn't change much from today,
except for some corner cases.  For example assigning a double to an int
using be dis-allowed if using an implicit conversion but allowed if
using an explicit conversion.

* If we implement strict type-checking, it would be only be enabled
by a flag like --strict-typing.  In that case implicit conversions
change so that the *type* of the VALUE expression needs to be a sub-type
of the required TYPE.  For example example you can't convert an
expression of type java.util.List to a required type java.util.ArrayList.
This would be ok with non-strict type-checking as long as the expression
type and the required type have a non-empty intersection.

Explicit conversion should work the same with strict and non-strict type-checking:
It would be allowed as long as the types have a non-empty intersection.

* If we implement strict type-checking, we'd also define a new 'dynamic' type,
similar to C#.  A expression that has type dynamic is allowed at compile-time
to be converted to any type; of course you might get a run-time exception.

The default type of a variable would no type-specifier should probably be
'dynamic'.  Of course the implementation type for dynamic is java.lang.Object;
the only difference is static type-checking.  This would actually allow *more*
programs to type-check without warnings than today, which may or may not
be desirable.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Kawa type-conversions

[Jamison Hope]

Hi Per,

On Mar 20, 2015, at 4:05 PM, Per Bothner <per@bothner.com> wrote:

> [Some thoughts about Kawa type-checking and type-conversions.
> I don't know when any of this will be implemented, but this is
> what I'm currently thinking makes most sense - though some details
> would be need to be worked out in practice.  Maybe we can think
> of this as a sketch for a Kawa Enhancement Proposal.]
> 
> Right now Kawa has multiple syntaxes for type conversion:
> 
> (as TYPE VALUE)
> 
> (->TYPE VALUE)
> 
> (define VAR ::TYPE VALUE) ;; and other variants using type specifiers
> 
> These mostly allow the same set of conversions, but using 'as' is more
> "lenient" than '->TYPE' in that it allows also conversions of the the
> raw types are the same.  (This is made use of in converting between the
> tring-cursor type and int, for example.)
> 
> We also have type tests:
> 
> (? VAR ::TYPE VALUE) ;; generalizable to patterns: (? PATTERN VALUE)
> 
> (instance? VALUE TYPE)
> 
> These aren't really consistent:
> 
> (if (? x::int 3.4) x #f) ==> 3
> (instance? 3.4 int) ==> 3

I think you meant "==> #f" for the second one (otherwise they seem
pretty consistent…).

> At some point I'd like to experiment with a --strict-typing option,
> so it's desirable to come up consistent and useful model.
> 
> So what I'm thinking we shoudl aim for:
> 
> * (as TYPE VALUE) would be short-hand for
>  (let ((TMP ::TYPE VALUE)) TMP)
> and so the set of allowable conversions would be the same.
> 
> * Using 'as' or a type-specifier would invoke an *implicit*
> conversion.  Using (->TYPE VALUE) would be an *explicit* conversion,
> and would allow more conversions.
> 
> * As a general rule implicit conversion of VALUE to TYPE should be
> OK if (instance? VALUE TYPE) or if it's a loss-less conversion to
> semantically more-or-less the same type.  We may need to fine-tune this.
> 
> * As a general rule (? VAR::TYPE VALUE) should succeed if
> (as TYPE VALUE) doesn't throw an exception. We may need to fine-tune this.
> 
> * It seems reasonable to introduce:
>  (convert TYPE VALUE)
> as equivalent to:
>  (->TYPE VALUE)
> 
> * As an example (->list SEQUENCE) would convert any SEQUENCE value
> to a list.  It would be equivalent to (list @SEQUENCE) except it
> wouldn't necessarily make a fresh copy if the SEQUENCE is a list.
> 
>  (->list #(3 4 5)) ==> (3 4 5)
>  (as list #(3 4 5)) ==> ERROR
>  (define x::list #(3 4 5)) ==> ERROR
> 
> * Assume we a new type sequence, optionally parameterized as
> sequence[ELEMENT-TYPE].  This would match any type allowed
> for splicing or the new generalized map, including strings
> and native Java arrays.  Since all of these types are "instances"
> of the sequence types, all of these types can be *implicitly*
> converted to sequence - even if we have to allocate a wrapper
> object.
> 
> In general, special types that aren't just native Java types
> might require custom conversion methods - and may need different
> methods for implicit and explicit conversions.
> 
> * As an incompatible change, converting float to int should (IMO) no
> longer be allowed for implicit conversions:
> 
>  (->int 3.4) ==> 3
>  (as int 3.4) ==> ERROR ;; incompatible change
>  (define x::int 3.4) ==> ERROR ;; incompatible change

Agreed, those last two just look weird.  But what about (as int 3.0)?
In other words, is it an error to try to coerce *any* floating point
number to an int, or just when it isn't already integer-valued? (in
the R6RS sense).

We already have `exact', so I don't have strong feelings either way,
as long as (as int (exact 3.0)) works.

> * In general static type-checking wouldn't change much from today,
> except for some corner cases.  For example assigning a double to an int
> using be dis-allowed if using an implicit conversion but allowed if
> using an explicit conversion.
> 
> * If we implement strict type-checking, it would be only be enabled
> by a flag like --strict-typing.  In that case implicit conversions
> change so that the *type* of the VALUE expression needs to be a sub-type
> of the required TYPE.  For example example you can't convert an
> expression of type java.util.List to a required type java.util.ArrayList.
> This would be ok with non-strict type-checking as long as the expression
> type and the required type have a non-empty intersection.
> 
> Explicit conversion should work the same with strict and non-strict type-checking:
> It would be allowed as long as the types have a non-empty intersection.
> 
> * If we implement strict type-checking, we'd also define a new 'dynamic' type,
> similar to C#.  A expression that has type dynamic is allowed at compile-time
> to be converted to any type; of course you might get a run-time exception.
> 
> The default type of a variable would no type-specifier should probably be
> 'dynamic'.  Of course the implementation type for dynamic is java.lang.Object;
> the only difference is static type-checking.  This would actually allow *more*
> programs to type-check without warnings than today, which may or may not
> be desirable.


Here's one other feature I would like to see, which I *think* fits in with
your description of (convert TYPE VALUE) and (->TYPE VALUE).

I would like a way to extend the system with custom conversion functions
for particular from/to type pairs.  This would be a big help when
interacting with different Java libraries.

For example, AWT provides a class representing a two-dimensional point
with coordinates stored as doubles, namely java.awt.geom.Point2D.Double.

Vecmath also provides a 2D point with double-valued coordinates,
javax.vecmath.Point2d.

And Kawa, of course, has complex numbers, which are 2D points in the
complex plane.

It would be super handy if I could write a couple of functions:

(define (javax.vecmath.Point2d->complex p::javax.vecmath.Point2d)
  ::complex
  (make-rectangular p:x p:y))

(define (java.awt.geom.Point2D->complex p::java.awt.geom.Point2D)
  ::complex
  (make-rectangular (p:getX) (p:getY)))

and then elsewhere, if I've got a heterogenous mix of points, just
use ->complex to turn them all into Kawa complex numbers.


So perhaps (->TYPE VALUE) and (convert TYPE VALUE) could work
something like this:

1. If VALUE is an instance of TYPE, just return it.

2. If VALUE has a "toTYPE" method, invoke that.

3. If TYPE has a one-argument constructor which works on VALUE,
   then return new TYPE(VALUE).

4. Otherwise, starting with VALUE.getClass() and working up to
   java.lang.Object, look for a function "FQCN->TYPE".  If one
   is found, then invoke it.

Implemented interfaces should probably be in that search list, too,
and I'm not sure what to do about type aliases.  I used
"xxx->complex" for the function names in that example, but maybe it
should be "xxx->gnu.math.Complex".

Does that fit with what you're thinking?

--
Jamison Hope
The PTR Group
www.theptrgroup.com

Re: Kawa type-conversions

[Per Bothner]

On 03/20/2015 05:05 PM, Jamison Hope wrote:
>
>> These aren't really consistent:
>>
>> (if (? x::int 3.4) x #f) ==> 3
>> (instance? 3.4 int) ==> 3
>
> I think you meant "==> #f" for the second one (otherwise they seem
> pretty consistentÂ…).

Right.  (instance? 3.4 int) returns #f - which is fine,
but (if (? x::int 3.4) x #f) currently returns 3 but should return #f.

>> * As an incompatible change, converting float to int should (IMO) no
>> longer be allowed for implicit conversions:
>>
>>   (->int 3.4) ==> 3
>>   (as int 3.4) ==> ERROR ;; incompatible change
>>   (define x::int 3.4) ==> ERROR ;; incompatible change
>
> Agreed, those last two just look weird.  But what about (as int 3.0)?
> In other words, is it an error to try to coerce *any* floating point
> number to an int, or just when it isn't already integer-valued? (in
> the R6RS sense).
>
> We already have `exact', so I don't have strong feelings either way,
> as long as (as int (exact 3.0)) works.

Likewise, I don't feel strongly about it, but it's easier to implement
if (as int 3.0) is an ERROR.  One important reason: If instead of a
constant you have an inexact-valued expression you want to complain
at compile-time.  So I think a reasonable specification for (as int XXX)
is that it works if and only if XXX has an exact integral type.  (It
might be reasonable to do range-checking, at run-time if needed.)

> Here's one other feature I would like to see, which I *think* fits in with
> your description of (convert TYPE VALUE) and (->TYPE VALUE).
>
> I would like a way to extend the system with custom conversion functions
> for particular from/to type pairs.  This would be a big help when
> interacting with different Java libraries.

I agree that could be useful.  Of course, if we're distinguishing between
explicit and implicit conversions we need to specify which custom conversions
are explicit and which are implicit.  You discuss an extra search mechanism
(convention) for explicit conversion, but I don't understand how
that so much more convenient than just re-defining the ->TYPE function.

Customization of *implicit* conversion seems more interesting.
Since we want static type-checking, that seems to imply that the
custom conversion functions must be visible at compile-time.

For example something like:

(define-conversion (name::T1)::T2 (convert-T1-to-T2 name))

This would get compiled to a static method:

public static T2 $convert_to$T2(T1 name) {convert_T1_to_T2(name); }

When needing to do an implicit type conversion, the compiler picks
"the most specific applicable" (in the sense of method overload resolution)
conversion method and compiles in a call to the corresponding static method.

I don't know how practical and safe this is.  It could certainly
easily be abused.

> So perhaps (->TYPE VALUE) and (convert TYPE VALUE) could work
> something like this:
>
> 1. If VALUE is an instance of TYPE, just return it.
>
> 2. If VALUE has a "toTYPE" method, invoke that.
>
> 3. If TYPE has a one-argument constructor which works on VALUE,
>     then return new TYPE(VALUE).

Actually, we want to prefer factory methods over constructors.
So perhaps:

> 3a. If TYPE has a static one-argument method which works on VALUE,
>     named 'valueOf' then return TYPE.valueOf(VALUE).  Likewise for 'make'.

>
> 4. Otherwise, starting with VALUE.getClass() and working up to
>     java.lang.Object, look for a function "FQCN->TYPE".  If one
>     is found, then invoke it.
>
> Implemented interfaces should probably be in that search list, too,
> and I'm not sure what to do about type aliases.  I used
> "xxx->complex" for the function names in that example, but maybe it
> should be "xxx->gnu.math.Complex".

-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Kawa type-conversions

[Jamison Hope]

On Mar 27, 2015, at 10:23 PM, Per Bothner <per@bothner.com> wrote:

> You discuss an extra search mechanism
> (convention) for explicit conversion, but I don't understand how
> that so much more convenient than just re-defining the ->TYPE function.

If all of the source is being compiled together, then perhaps it
isn't, except that it avoids having to explicitly group all of the
possible conversions together into a big case-lambda or whatever.
The search mechanism would make ->TYPE more akin to a CLOS generic
function, where the backing methods could be in separate modules.

Anyway, my thought was for separately-compiled libraries.  For example,
suppose I've got a textbook shape class library with a bunch of useful
functions to do things like calculate the minimum distances between pairs
of shapes (sphere/sphere, prism/cylinder, etc).  Everything is working
fine, and then the boss says that I need to support JBullet shape
primitives.  Knowing how these things go (next week it'll be ode4j or
some other package instead of JBullet), instead of making a bunch of
huge changes to the guts of my library, I change my top-level functions:


(define (calculate-distance s1::com.example.shape s2::com.example.shape)
  ::real
  ...)

becomes

(define (calculate-distance o1 o2) ::real
  (let ((s1 (->com.example.shape o1))
        (s2 (->com.example.shape o2)))
    ...))


I then compile that to a JAR and never touch it again.


To support JBullet, I then write a helper library which does the
conversion between JBullet shapes and com.example.shape, and then make
sure that JAR gets loaded any time I have to support JBullet shapes.
Likewise for ode4j and the next third-party library that comes along.


If I just redefine ->TYPE, then I end up needing 2^n different
re-definitions, for all of the possible combinations of "from" types.
In this example, I would need one that just handles JBullet shapes,
another which just handles ode4j shapes, a third which handles both,
etc.

(Again, the problem is the big generic procedure with everything all
in one place.  That means to compile/load that one function, I would
need to load the classes of all types mentioned by that function.)

Does all that make sense?

Maybe this isn't the job of ->TYPE, maybe it's some other COERCE-like
function which does runtime (not compile-time) lookup to enable the
kind of modularity I've described?

--
Jamison Hope
The PTR Group
www.theptrgroup.com