Opinions wanted: What should (.list pmacro-name 42) do?

Opinions wanted: What should (.list pmacro-name 42) do?

[Doug Evans]

Hi.

What should the output of

(define-pmacro (foo a) (add a 1))
(.list foo 42)

be?

Currently the result is (<pmacro> 42)
where <pmacro> is the pmacro object for "foo".

It's not ideal because it means that a pmacro object "escapes" 
pmacro-processing and can be seen by, for example, rtl compilation.

However, it's not that unexpected.  The user asked for a list containing 
two objects, the pmacro and 42.
This is akin to saying (list + 42) in Scheme.

We *could* have the pmacro evaluator re-examine the result and if it 
sees (<pmacro> mumble) then re-expand it.  That's what we do for symbols 
today (i.e. if the result of pmacro-expansion is a symbol that names a 
pmacro, we re-evaluate it), but that has problems, e.g.
http://sourceware.org/ml/cgen/2009-q3/msg00052.html
and I'm leaning toward removing that behavior.

It's not clear to me what The Right Thing to do is.
Opinions?

Re: Opinions wanted: What should (.list pmacro-name 42) do?

[Doug Evans]

Doug Evans wrote:
> Hi.
>
> What should the output of
>
> (define-pmacro (foo a) (add a 1))
> (.list foo 42)
>
> be?
>
> Currently the result is (<pmacro> 42)
> where <pmacro> is the pmacro object for "foo".
>
> It's not ideal because it means that a pmacro object "escapes" 
> pmacro-processing and can be seen by, for example, rtl compilation.
>
> However, it's not that unexpected.  The user asked for a list 
> containing two objects, the pmacro and 42.
> This is akin to saying (list + 42) in Scheme.
>
> We *could* have the pmacro evaluator re-examine the result and if it 
> sees (<pmacro> mumble) then re-expand it.  That's what we do for 
> symbols today (i.e. if the result of pmacro-expansion is a symbol that 
> names a pmacro, we re-evaluate it), but that has problems, e.g.
> http://sourceware.org/ml/cgen/2009-q3/msg00052.html
> and I'm leaning toward removing that behavior.
>
> It's not clear to me what The Right Thing to do is.
> Opinions?
>

[For completeness' sake ...]
An alternative is of course to flag any pmacros that "escape" as 
errors.  We need to do that anyway for cases like (.list 42 foo).

Re: Opinions wanted: What should (.list pmacro-name 42) do?

[Jim Blandy]

On Thu, Aug 20, 2009 at 10:30 AM, Doug Evans<dje@sebabeach.org> wrote:
> Hi.
>
> What should the output of
>
> (define-pmacro (foo a) (add a 1))
> (.list foo 42)
>
> be?
>
> Currently the result is (<pmacro> 42)
> where <pmacro> is the pmacro object for "foo".
>
> It's not ideal because it means that a pmacro object "escapes"
> pmacro-processing and can be seen by, for example, rtl compilation.
>
> However, it's not that unexpected.  The user asked for a list containing two
> objects, the pmacro and 42.
> This is akin to saying (list + 42) in Scheme.
>
> We *could* have the pmacro evaluator re-examine the result and if it sees
> (<pmacro> mumble) then re-expand it.  That's what we do for symbols today
> (i.e. if the result of pmacro-expansion is a symbol that names a pmacro, we
> re-evaluate it), but that has problems, e.g.
> http://sourceware.org/ml/cgen/2009-q3/msg00052.html
> and I'm leaning toward removing that behavior.

Macros should definitely be able to expand to calls to other macros
somehow; doesn't (.list 'foo 42) do this?  If it does, then I don't
see much point in allowing (.list foo 42) to do so as well.  The
source world and the value world need to interact only in
easy-to-reason-about ways, or you end up with stuff like m4, which is
just chaos.

This reminds me of the "3-d macro" problem in Scheme, where macros can
place values that can't actually be written in Scheme source code
(procedure objects, say) into the tree that the compiler eventually
sees.  I've always thought the cleanest thing was to reject such code,
as it didn't seem useful, and made a weird breach between macro
expansion time and run time.

Re: Opinions wanted: What should (.list pmacro-name 42) do?

[Doug Evans]

Jim Blandy wrote:
> On Thu, Aug 20, 2009 at 10:30 AM, Doug Evans<dje@sebabeach.org> wrote:
>   
>> Hi.
>>
>> What should the output of
>>
>> (define-pmacro (foo a) (add a 1))
>> (.list foo 42)
>>
>> be?
>>
>> Currently the result is (<pmacro> 42)
>> where <pmacro> is the pmacro object for "foo".
>>
>> It's not ideal because it means that a pmacro object "escapes"
>> pmacro-processing and can be seen by, for example, rtl compilation.
>>
>> However, it's not that unexpected.  The user asked for a list containing two
>> objects, the pmacro and 42.
>> This is akin to saying (list + 42) in Scheme.
>>
>> We *could* have the pmacro evaluator re-examine the result and if it sees
>> (<pmacro> mumble) then re-expand it.  That's what we do for symbols today
>> (i.e. if the result of pmacro-expansion is a symbol that names a pmacro, we
>> re-evaluate it), but that has problems, e.g.
>> http://sourceware.org/ml/cgen/2009-q3/msg00052.html
>> and I'm leaning toward removing that behavior.
>>     
>
> Macros should definitely be able to expand to calls to other macros
> somehow; doesn't (.list 'foo 42) do this?

Hey Jim!  Thanks for the feedback.

I'm not quite sure what you mean, alas.  [Apologies.  I can guess, but ...]

CGEN's pmacro system doesn't (currently) have quoting a la ' (or quote), 
btw.
[The Scheme reader will accept it of course, but the pmacro system won't 
(currently) do anything sensible with (quote foo).]

> If it does, then I don't
> see much point in allowing (.list foo 42) to do so as well.  The
> source world and the value world need to interact only in
> easy-to-reason-about ways, or you end up with stuff like m4, which is
> just chaos.
>
> This reminds me of the "3-d macro" problem in Scheme, where macros can
> place values that can't actually be written in Scheme source code
> (procedure objects, say) into the tree that the compiler eventually
> sees.  I've always thought the cleanest thing was to reject such code,
> as it didn't seem useful, and made a weird breach between macro
> expansion time and run time.
>   

Note that one can pass pmacros as parameters to other pmacros and invoke 
them.  But the user is required to explicitly do this, CGEN won't 
(currently) take the result of a macro and rescan it for further pmacro 
invocations (except, currently, in the case where the result of the 
pmacro expansion is a single symbol).

In Scheme the result of (list + 1 2) is (<primitive-procedure-plus> 1 
2), not 3.

For reference sake, in Scheme,
(define-macro (foo a) `(bar ,a))
(define-macro (bar a) `(baz ,a))
(foo 3) ;; expands to (bar 3) which in turn expands to (baz 3)

In CGEN that would be
(define-pmacro (foo a) (bar a))
(define-pmacro (bar a) (baz a))

Using pmacro-trace in Guile I get:

guile> (pmacro-trace '(foo 3) (unspecified-location))
Pmacro expanding: (foo 3)
Pmacro location: standard input:10:16
    Expanding: (foo 3)
          env: ()
     location: standard input:10:16
    Expanding: (bar a)
          env: ((a . 3))
     location: standard input:3:26
       result: (baz 3)
       result: (baz 3)
Pmacro result: (baz 3)
(baz 3)

And in Scheme,
(define-macro (foo a) `(list bar ,a))
(define-macro (bar a) `(baz ,a))
(foo 3) ;; ==> (#<macro! bar> 3) ;; Note that Scheme didn't re-expand 
the result and turn it into (baz 3).

In CGEN,
(define-pmacro (foo a) (.list bar a))
(define-pmacro (bar a) (baz a))
(foo 3) ;; ==> (<pmacro bar> 3)

We don't have to follow Scheme per se, but I'm worried that being 
excessively clever here will lead to issues (like in  
http://sourceware.org/ml/cgen/2009-q3/msg00052.html).
One issue is: what if the user *wants* (.list foo 42) to be (<pmacro 
foo> 42).  S/he can always explicitly re-expand the expression to turn 
it into (add 42 1), e.g., with (.eval (.list foo 42)), but there's no 
(current) way to prevent the re-expansion if it was done automagically 
(unless, for example, we add a quoting system, which I hesitate to do in 
part because it's yet another complication to the language).