RE: RFC: variadic closures in x86/x86_64

["Hogan, D. (GE Power & Water)" <D.Hogan@ge.com>]

On Wed, Dec 04, 2013 at 00:07:21AM, Andrew Haley wrote:
> Please, I'm finding this extremely difficult to understand.  Where is
> the variadic function?  You say that a C shared library "executes" a
> callback, but what does that mean?

Sorry about the confusion.  I mean the driver sends to the shared
library a struct of function pointers when it creates a model instance.
It is represented in JNA as a Structure of Callbacks.  One of the
function pointers is variadic:

typedef struct { 
    void (*logger)(fmiComponent c, fmiString instanceName,
                   fmiStatus status, fmiString category,
                   fmiString message, ...); 
    void *(*allocateMemory)(size_t nobj, size_t size); 
    void (*freeMemory) (void *obj); 
} fmiCallbackFunctions;

And this is a function inside of every shared library model:

fmiComponent instantiateModel(char* fname, fmiString instanceName,
  fmiString GUID, fmiCallbackFunctions functions, fmiBoolean loggingOn)

The model dereferences the function pointers inside of a common file,
fmuTemplate.c, which is included in every model.  Here's an example:

fmiStatus fmiSetInteger(fmiComponent c, const fmiValueReference vr[],
                        size_t nvr, const fmiInteger value[]){
  int i;
  ModelInstance* comp = (ModelInstance *)c;
...
  if (comp->loggingOn)
    comp->functions.logger(c, comp->instanceName, fmiOK, "log",
                           "fmiSetInteger: nvr = %d",nvr);
...
}

> Right, so a C program thinks it's calling C, but in fact it's calling
> Java, and a libffi closure provides the glue.

Yeah.  JNA uses a libffi closure to provide the glue.  The problem is
closures cannot currently access variadic arguments.  If you try to
write an implementation of the logger above as a Callback in JNA,
it won't be able to access anything beyond message.  This is a
limitation inherited from libffi's closure.

> Ah!  OK, so the function that is called as a C function is a libffi
> closure, and it is called in a variadic form.

Right.

> Again, I don't know what it means to "run a variadic callback." Does
> this mean that the shared library contains this callback function, or
> that it calls it?

The shared library dereferences a pointer to a variadic function.

> Why do you not define a variadic C function which does this:
>
> void f1(int n, ...) {
>   va_list ap;

In the libffi manual, there's a TODO about variadic closures.  My
interpretation of variadic closures is a closure which can access
variadic arguments without sending in the number of arguments and
types at the ffi_prep_cif or ffi_prep_cif_var time.  Once you have
a variadic closure, you should be call it any number of times with
any number of variadic arguments.

If libffi had that, JNA could create a closure and the Java
callback/function pointer could access variadic arguments in any manner
that it sees fit.  It wouldn't need an application specific C wrapper
that captures the varargs.

For instance, taking the simple testsuite/libffi.call/cls_sint.c,
what would it look like if you had variadic closures?  Below is
similar to what it would look like with this patch.

diff --git a/testsuite/libffi.call/cls_sint.c b/testsuite/libffi.call/cls_sint.c
index c7e13b7..d456be5 100644
--- a/testsuite/libffi.call/cls_sint.c
+++ b/testsuite/libffi.call/cls_sint.c
@@ -7,14 +7,21 @@
 /* { dg-do run } */
 #include "ffitest.h"
 
-static void cls_ret_sint_fn(ffi_cif* cif __UNUSED__, void* resp, void** args,
+static void cls_ret_sint_fn(ffi_cif* cif, void* resp, void** args,
 			    void* userdata __UNUSED__)
 {
+  void *vals[1];
+
   *(ffi_arg*)resp = *(signed int *)args[0];
-  printf("%d: %d\n",*(signed int *)args[0],
-	 (int)*(ffi_arg *)(resp));
+
+  /* Assuming the first variadic argument is sint. */
+  CHECK(ffi_closure_va_arg(cif, &ffi_type_sint32, &vals[0]) == FFI_OK);
+
+  printf("%d: %d %d\n",*(signed int *)args[0],
+	 (int)*(ffi_arg *)(resp), *(signed int *)vals[0]);
+
 }
-typedef signed int (*cls_ret_sint)(signed int);
+typedef signed int (*cls_ret_sint)(signed int, ...);
 
 int main (void)
 {
@@ -33,8 +40,10 @@ int main (void)
 
   CHECK(ffi_prep_closure_loc(pcl, &cif, cls_ret_sint_fn, NULL, code)  == FFI_OK);
 
-  res = (*((cls_ret_sint)code))(65534);
-  /* { dg-output "65534: 65534" } */
+  res = (*((cls_ret_sint)code))(65534, 32);
+  /* { dg-output "65534: 65534 32" } */
+  res = (*((cls_ret_sint)code))(65534, 32, 64.5); /* 64.5 is ignored but allowed */
+  /* { dg-output "65534: 65534 32" } */
   printf("res: %d\n",res);
   /* { dg-output "\nres: 65534" } */