RFC: variadic closures in x86/x86_64

RFC: variadic closures in x86/x86_64

[Hogan, D. (GE Power & Water)]

Hi,

I'm requesting feedback on an implementation of variadic closures in
libffi.  It currently supports x86 and x86_64.  This change allows for
FMI logging callbacks through JNA in the JFMI[1] and Ptolemy II[2]
projects.  The libffi changes are in a github branch[3].  I'm waiting
to finalize these changes before pushing the JNA branch.

It looks like it will be easy to add support for platforms such as
SPARC.  Are there any platforms that support variadic arguments but
would not support variadic closures in this manner?

I tested this on x86_64 Linux and x86 Windows.  None of the dejagnu
tests failed after this change.  I could use help testing it on other
platforms.  Thanks to Christopher Brooks from UC Berkeley for feedback
and testing on x86_64 Darwin.


Regards,
Doug

[1] http://ptolemy.eecs.berkeley.edu/java/jfmi
[2] http://ptolemy.eecs.berkeley.edu/ptolemyII
[3] https://github.com/dhogan1/libffi/tree/variadic_closures

Re: RFC: variadic closures in x86/x86_64

[Andrew Haley]

On 11/25/2013 02:45 AM, Hogan, D. (GE Power & Water) wrote:
> I'm requesting feedback on an implementation of variadic closures in
> libffi.  It currently supports x86 and x86_64.  This change allows for
> FMI logging callbacks through JNA in the JFMI[1] and Ptolemy II[2]
> projects.  The libffi changes are in a github branch[3].

Why is this necessary?  I thought that the variadic calling conventions
on x86 were the same as the non-variadic ones.

Andrew.

Re: RFC: variadic closures in x86/x86_64

[Jakub Jelinek]

On Mon, Nov 25, 2013 at 09:28:52AM +0000, Andrew Haley wrote:
> On 11/25/2013 02:45 AM, Hogan, D. (GE Power & Water) wrote:
> > I'm requesting feedback on an implementation of variadic closures in
> > libffi.  It currently supports x86 and x86_64.  This change allows for
> > FMI logging callbacks through JNA in the JFMI[1] and Ptolemy II[2]
> > projects.  The libffi changes are in a github branch[3].
> 
> Why is this necessary?  I thought that the variadic calling conventions
> on x86 were the same as the non-variadic ones.

On x86 they are, on x86_64 they are not (%rax register must contain the
number of floating point varargs arguments AFAIK).

	Jakub

Re: RFC: variadic closures in x86/x86_64

[Andrew Haley]

On 11/25/2013 09:37 AM, Jakub Jelinek wrote:
> On Mon, Nov 25, 2013 at 09:28:52AM +0000, Andrew Haley wrote:
>> On 11/25/2013 02:45 AM, Hogan, D. (GE Power & Water) wrote:
>>> I'm requesting feedback on an implementation of variadic closures in
>>> libffi.  It currently supports x86 and x86_64.  This change allows for
>>> FMI logging callbacks through JNA in the JFMI[1] and Ptolemy II[2]
>>> projects.  The libffi changes are in a github branch[3].
>>
>> Why is this necessary?  I thought that the variadic calling conventions
>> on x86 were the same as the non-variadic ones.
> 
> On x86 they are, on x86_64 they are not (%rax register must contain the
> number of floating point varargs arguments AFAIK).

Sure, but we don't need a new libffi interface to do that.  We need
only to set %rax.  And we can just do that anyway; it doesn't hurt.

I must be missing something.

Andrew.

Re: RFC: variadic closures in x86/x86_64

[Alan Modra]

On Mon, Nov 25, 2013 at 10:10:37AM +0000, Andrew Haley wrote:
> Sure, but we don't need a new libffi interface to do that.  We need
> only to set %rax.  And we can just do that anyway; it doesn't hurt.
> 
> I must be missing something.

The claim to fame looks to be the ability to call variadic functions
without describing the arguments via ffi_prep_cif_var at the point of
call.  Instead you do so in the function consuming the args.  I'm not
sure what that gains you..

The patch suffers from the same flaw as my first attempt at adding
variadic argument support for PowerPC64 ELFv2, namely that you cannot
add fields to ffi_cif without breaking ABI compatibility.
Applications linked against an older version of libffi.so will supply
the old ffi_cif to a new shared library..

Oh, and gcc still contains the first attempt since no one has yet
approved an import from libffi to cure the problem.

-- 
Alan Modra
Australia Development Lab, IBM

RE: RFC: variadic closures in x86/x86_64

[Hogan, D. (GE Power & Water)]

On Wed, Nov 27, 2013 at 00:57:23PM +1030, Alan Modra wrote:
> The claim to fame looks to be the ability to call variadic functions
> without describing the arguments via ffi_prep_cif_var at the point of
> call.  Instead you do so in the function consuming the args.  I'm not
> sure what that gains you..

This is specifically for variadic callbacks.  If it wasn't a callback,
I could use ffi_prep_cif_var.

To give a little background, FMI is a standard for (among other things)
model exchange so you can use a dynamic system model in various
modeling or simulation environments.  The model is exposed as a C shared
library.  The shared library executes callbacks (some variadic) provided
by a FMU driver.

JFMI allows you to drive a FMU from Java.  The Java FMU driver provides
a Java implementation for the C callbacks.  The nonvariadic callbacks
were already supported by JFMI through libffi and JNA.  In order to
handle variadic callbacks, libffi and JNA need to be modified so you
can access the variadic arguments inside of a callback.  This patch
adds the libffi support.

I cannot construct a ffi_prep_cif_var because the model's C shared
library is the one running a variadic callback in the driver.  I don't
know how many arguments or what types it will provide in the calls.  I
have to rely on printf style formatting flags in order to know what to
access from inside of the callback.

> The patch suffers from the same flaw as my first attempt at adding
> variadic argument support for PowerPC64 ELFv2, namely that you cannot
> add fields to ffi_cif without breaking ABI compatibility.
> Applications linked against an older version of libffi.so will supply
> the old ffi_cif to a new shared library..

That's a good point.  This should be changed to avoid breaking the ABI.

Re: RFC: variadic closures in x86/x86_64

[Philip Ashmore]

On 04/12/13 08:02, Hogan, D. (GE Power & Water) wrote:
> On Wed, Nov 27, 2013 at 00:57:23PM +1030, Alan Modra wrote:
>> The claim to fame looks to be the ability to call variadic functions
>> without describing the arguments via ffi_prep_cif_var at the point of
>> call.  Instead you do so in the function consuming the args.  I'm not
>> sure what that gains you..
> 
> This is specifically for variadic callbacks.  If it wasn't a callback,
> I could use ffi_prep_cif_var.
> 
> To give a little background, FMI is a standard for (among other things)
> model exchange so you can use a dynamic system model in various
> modeling or simulation environments.  The model is exposed as a C shared
> library.  The shared library executes callbacks (some variadic) provided
> by a FMU driver.
> 
> JFMI allows you to drive a FMU from Java.  The Java FMU driver provides
> a Java implementation for the C callbacks.  The nonvariadic callbacks
> were already supported by JFMI through libffi and JNA.  In order to
> handle variadic callbacks, libffi and JNA need to be modified so you
> can access the variadic arguments inside of a callback.  This patch
> adds the libffi support.
> 
> I cannot construct a ffi_prep_cif_var because the model's C shared
> library is the one running a variadic callback in the driver.  I don't
> know how many arguments or what types it will provide in the calls.  I
> have to rely on printf style formatting flags in order to know what to
> access from inside of the callback.
> 
>> The patch suffers from the same flaw as my first attempt at adding
>> variadic argument support for PowerPC64 ELFv2, namely that you cannot
>> add fields to ffi_cif without breaking ABI compatibility.
>> Applications linked against an older version of libffi.so will supply
>> the old ffi_cif to a new shared library..
> 
> That's a good point.  This should be changed to avoid breaking the ABI.
> 
I may be talking through my hat but if there's a format argument then
couldn't you provide a function to parse it and then prepare other cifs
on an as-needed basis? The key would be the concatenation of all the
types in the format argument.

That way you wouldn't need to modify libffi.

Regards,
Philip Ashmore

Re: RFC: variadic closures in x86/x86_64

[Andrew Haley]

On 12/04/2013 08:02 AM, Hogan, D. (GE Power & Water) wrote:
> On Wed, Nov 27, 2013 at 00:57:23PM +1030, Alan Modra wrote:
>> The claim to fame looks to be the ability to call variadic functions
>> without describing the arguments via ffi_prep_cif_var at the point of
>> call.  Instead you do so in the function consuming the args.  I'm not
>> sure what that gains you..
> 
> This is specifically for variadic callbacks.  If it wasn't a callback,
> I could use ffi_prep_cif_var.
> 
> To give a little background, FMI is a standard for (among other things)
> model exchange so you can use a dynamic system model in various
> modeling or simulation environments.  The model is exposed as a C shared
> library.  The shared library executes callbacks (some variadic) provided
> by a FMU driver.

Please, I'm finding this extremely difficult to understand.  Where is
the variadic function?  You say that a C shared library "executes" a
callback, but what does that mean?

At the point of the call, the caller knows for certain what arguments
are passed, and the type of these arguments.

> JFMI allows you to drive a FMU from Java.  The Java FMU driver provides
> a Java implementation for the C callbacks.

Right, so a C program thinks it's calling C, but in fact it's calling
Java, and a libffi closure provides the glue.

> The nonvariadic callbacks

Ah!  OK, so the function that is called as a C function is a libffi
closure, and it is called in a variadic form.

> were already supported by JFMI through libffi and JNA.  In order to
> handle variadic callbacks, libffi and JNA need to be modified so you
> can access the variadic arguments inside of a callback.  This patch
> adds the libffi support.
> 
> I cannot construct a ffi_prep_cif_var because the model's C shared
> library is the one running a variadic callback in the driver.

Again, I don't know what it means to "run a variadic callback." Does
this mean that the shared library contains this callback function, or
that it calls it?

> I don't
> know how many arguments or what types it will provide in the calls.  I
> have to rely on printf style formatting flags in order to know what to
> access from inside of the callback.

OK, so I'm guessing that the C model's shared library calls the variadic
function.

Why do you not define a variadic C function which does this:

void f1(int n, ...) {
  va_list ap;

  jintArray argsArray = (*env_p)->NewIntArray(n);
  jint *args = (*env_p)->GetIntArrayElements(argsArray, NULL);

  int argno = 0;
  va_start(ap, n);
  while (argno < n) {
	args[argno++] = va_arg(ap, int);
  }
  va_end(ap);
  (*env_p)->ReleaseIntArrayElements(argsArray, args, 0);
  (*env_p)->CallStaticVoidMethod(clsH, printargs, argsArray);
}

to call this variadic Java code:

    public static void printargs(int... args) {
        for (int i : args)
            System.out.print(i + " ");
        System.out.println();
    }

There is no reason that JNA cannot use this mechanism, is there?

Andrew.

RE: RFC: variadic closures in x86/x86_64

[Hogan, D. (GE Power & Water)]

On Wed, Dec 04, 2013 at 00:07:21AM, Andrew Haley wrote:
> Please, I'm finding this extremely difficult to understand.  Where is
> the variadic function?  You say that a C shared library "executes" a
> callback, but what does that mean?

Sorry about the confusion.  I mean the driver sends to the shared
library a struct of function pointers when it creates a model instance.
It is represented in JNA as a Structure of Callbacks.  One of the
function pointers is variadic:

typedef struct { 
    void (*logger)(fmiComponent c, fmiString instanceName,
                   fmiStatus status, fmiString category,
                   fmiString message, ...); 
    void *(*allocateMemory)(size_t nobj, size_t size); 
    void (*freeMemory) (void *obj); 
} fmiCallbackFunctions;

And this is a function inside of every shared library model:

fmiComponent instantiateModel(char* fname, fmiString instanceName,
  fmiString GUID, fmiCallbackFunctions functions, fmiBoolean loggingOn)

The model dereferences the function pointers inside of a common file,
fmuTemplate.c, which is included in every model.  Here's an example:

fmiStatus fmiSetInteger(fmiComponent c, const fmiValueReference vr[],
                        size_t nvr, const fmiInteger value[]){
  int i;
  ModelInstance* comp = (ModelInstance *)c;
...
  if (comp->loggingOn)
    comp->functions.logger(c, comp->instanceName, fmiOK, "log",
                           "fmiSetInteger: nvr = %d",nvr);
...
}

> Right, so a C program thinks it's calling C, but in fact it's calling
> Java, and a libffi closure provides the glue.

Yeah.  JNA uses a libffi closure to provide the glue.  The problem is
closures cannot currently access variadic arguments.  If you try to
write an implementation of the logger above as a Callback in JNA,
it won't be able to access anything beyond message.  This is a
limitation inherited from libffi's closure.

> Ah!  OK, so the function that is called as a C function is a libffi
> closure, and it is called in a variadic form.

Right.

> Again, I don't know what it means to "run a variadic callback." Does
> this mean that the shared library contains this callback function, or
> that it calls it?

The shared library dereferences a pointer to a variadic function.

> Why do you not define a variadic C function which does this:
>
> void f1(int n, ...) {
>   va_list ap;

In the libffi manual, there's a TODO about variadic closures.  My
interpretation of variadic closures is a closure which can access
variadic arguments without sending in the number of arguments and
types at the ffi_prep_cif or ffi_prep_cif_var time.  Once you have
a variadic closure, you should be call it any number of times with
any number of variadic arguments.

If libffi had that, JNA could create a closure and the Java
callback/function pointer could access variadic arguments in any manner
that it sees fit.  It wouldn't need an application specific C wrapper
that captures the varargs.

For instance, taking the simple testsuite/libffi.call/cls_sint.c,
what would it look like if you had variadic closures?  Below is
similar to what it would look like with this patch.

diff --git a/testsuite/libffi.call/cls_sint.c b/testsuite/libffi.call/cls_sint.c
index c7e13b7..d456be5 100644
--- a/testsuite/libffi.call/cls_sint.c
+++ b/testsuite/libffi.call/cls_sint.c
@@ -7,14 +7,21 @@
 /* { dg-do run } */
 #include "ffitest.h"
 
-static void cls_ret_sint_fn(ffi_cif* cif __UNUSED__, void* resp, void** args,
+static void cls_ret_sint_fn(ffi_cif* cif, void* resp, void** args,
 			    void* userdata __UNUSED__)
 {
+  void *vals[1];
+
   *(ffi_arg*)resp = *(signed int *)args[0];
-  printf("%d: %d\n",*(signed int *)args[0],
-	 (int)*(ffi_arg *)(resp));
+
+  /* Assuming the first variadic argument is sint. */
+  CHECK(ffi_closure_va_arg(cif, &ffi_type_sint32, &vals[0]) == FFI_OK);
+
+  printf("%d: %d %d\n",*(signed int *)args[0],
+	 (int)*(ffi_arg *)(resp), *(signed int *)vals[0]);
+
 }
-typedef signed int (*cls_ret_sint)(signed int);
+typedef signed int (*cls_ret_sint)(signed int, ...);
 
 int main (void)
 {
@@ -33,8 +40,10 @@ int main (void)
 
   CHECK(ffi_prep_closure_loc(pcl, &cif, cls_ret_sint_fn, NULL, code)  == FFI_OK);
 
-  res = (*((cls_ret_sint)code))(65534);
-  /* { dg-output "65534: 65534" } */
+  res = (*((cls_ret_sint)code))(65534, 32);
+  /* { dg-output "65534: 65534 32" } */
+  res = (*((cls_ret_sint)code))(65534, 32, 64.5); /* 64.5 is ignored but allowed */
+  /* { dg-output "65534: 65534 32" } */
   printf("res: %d\n",res);
   /* { dg-output "\nres: 65534" } */

Re: RFC: variadic closures in x86/x86_64

[Andrew Haley]

On 12/04/2013 06:28 PM, Hogan, D. (GE Power & Water) wrote:
> On Wed, Dec 04, 2013 at 00:07:21AM, Andrew Haley wrote:

> 
>> Why do you not define a variadic C function which does this:
>>
>> void f1(int n, ...) {
>>   va_list ap;
> 
> In the libffi manual, there's a TODO about variadic closures. 

That's AFAIK to do with targets that have a different calling convention
for variadic calls.

> My interpretation of variadic closures is a closure which can access
> variadic arguments without sending in the number of arguments and
> types at the ffi_prep_cif or ffi_prep_cif_var time.  Once you have a
> variadic closure, you should be call it any number of times with any
> number of variadic arguments.

Forgive me, but that's not an answer.  I have provided an example of a
mechanism that JNA could use that would not require us to change
libffi.  Couldn't you do this in JNA?  It doesn't have to be
application-specific.

Andrew.

RE: RFC: variadic closures in x86/x86_64

[Hogan, D. (GE Power & Water)]

On Wed, Dec 04, 2013 at 00:01:51PM, Andrew Haley wrote:
> > In the libffi manual, there's a TODO about variadic closures.
> 
> That's AFAIK to do with targets that have a different calling
> convention for variadic calls.

Ok, this is the source of my confusion then.  I thought the manual was
saying you wanted libffi to be able to create closures that could
iterate through variadic arguments they were called with.

> > My interpretation of variadic closures is a closure which can access
> > variadic arguments without sending in the number of arguments and
> > types at the ffi_prep_cif or ffi_prep_cif_var time.  Once you have a
> > variadic closure, you should be call it any number of times with any
> > number of variadic arguments.
> 
> Forgive me, but that's not an answer.  I have provided an example of a
> mechanism that JNA could use that would not require us to change
> libffi.  Couldn't you do this in JNA?  It doesn't have to be
> application-specific.

I'm sure it would work.  A similar suggestion is mentioned in the CFFI
documentation.  It would require a C function for every assumed
variadic processing convention.  For instance, FMI requires
#<Type><valueReference># which is not a typical printf/logger
convention. I didn't go down that path because I thought the manual
was asking for patches for a different way.

Thanks for the feedback.

Re: RFC: variadic closures in x86/x86_64

[Andrew Haley]

On 12/05/2013 12:47 AM, Hogan, D. (GE Power & Water) wrote:
> I'm sure it would work.  A similar suggestion is mentioned in the CFFI
> documentation.  It would require a C function for every assumed
> variadic processing convention.  For instance, FMI requires
> #<Type><valueReference># which is not a typical printf/logger
> convention. I didn't go down that path because I thought the manual
> was asking for patches for a different way.
> 
> Thanks for the feedback.

Hmm, I'm sorry for pushing back in this way, but I think we'd need to
make the changes to all of our twenty or so back ends.  And of course
no-one wants to do that.

Andrew.

Re: RFC: variadic closures in x86/x86_64

[Philip Ashmore]

On 05/12/13 00:54, Hogan, D. (GE Power & Water) wrote:
> Hi Philip,
> 
>> I may be talking through my hat but if there's a format argument then
>> couldn't you provide a function to parse it and then prepare other cifs
>> on an as-needed basis? The key would be the concatenation of all the
>> types in the format argument.
> 
> Do you mean inside of a closure?  I could access the assumed type
> information from the formatting string.  The problem would be with
> ffi_call().  I can't send in a value for a variadic argument since
> I don't have a pointer to it.
> 
No, I mean inside a regular function, as libffi doesn't (yet) support
variadic closures.

I don't know if it's possible to portably get pointers to variadic
arguments using va_arg, but if it is then you build up a list of
variadic arguments, synthesize a call using libffi by constructing a cif
with prep_cif_var if you don't already have an identical one, then call
the "real" function.

But then why not just call the "real" function?

I was (in my own convoluted way) trying to suggest a possible libffi
variadic closure implementation.

Any approach needs some hint as to the number and types of arguments to
unpack, like a format string.

Then it can do what it wants, like provide the arguments as a type/value
list, to the psuedo-variadic closure function to process.

The psuedo-variadic closure function could then have extra arguments - a
count and a type/value array pointer (or a pointer in libffi type-speak)
so you wouldn't even need a cif to call it.

Regards,
Philip Ashmore