libffi stdcall closures

libffi stdcall closures

[Timothy Wall]

I'm working on a patch for win32/x86 stdcall closures to support JNA  
(http://jna.dev.java.net), a ctypes-like library for Java.  I'm  
working off of a libffi snapshot from the gcc trunk.

I've got the closure code working now, and I need some pointers on  
adding a win32-only stdcall callback test to the testsuite (or a  
general one if stdcall is supported elsewhere).

I'm running the testsuite under cygwin, which works except for about  
4 failures unrelated to what I'm working on.

Oddly enough, some stdcall closures would work while others would not  
(seems to be tied to the caller environment as opposed to the number  
of closure arguments).   Probably has to do with whether the caller  
is maintaining a frame pointer or whether it relies entirely on the  
stack pointer.

Timothy Wall
http://abbot.sf.net

PS Please CC me explicitly; I'm not subscribed to gcc-patches.

Re: libffi stdcall closures

[Tom Tromey]

>>>>> "Timothy" == Timothy Wall <twall@users.sf.net> writes:

Timothy> I'm working on a patch for win32/x86 stdcall closures to support JNA
Timothy> (http://jna.dev.java.net), a ctypes-like library for Java.  I'm
Timothy> working off of a libffi snapshot from the gcc trunk.

Nice.  JNA looks cool.

BTW, I think I read somewhere that you have some standalone configure
changes for libffi...?  We could put those in the separate libffi
repository if that would help you.  I occasionally think it would be
nice to revive the standalone libffi -- but only if there's interest.

Timothy> I've got the closure code working now, and I need some pointers on
Timothy> adding a win32-only stdcall callback test to the testsuite (or a
Timothy> general one if stdcall is supported elsewhere).

libffi uses the dejagnu 'dg' code, so you should be able to just drop
your test cases into the test suite.  If the test is platform-specific
I think you can use the "target" clause to dg-do.  Grep for "target"
in gcc/testsuite/**/*.c to see uses.

Tom

Re: libffi stdcall closures

[Timothy Wall]

Ok, thanks, I got a test that runs only for w32.  It also uncovered a  
bug in the cygwin gcc I am using (3.4.4 cygming special); it pops  
arguments for stdcall calls unless invoked in "-mno-cygwin" mode,  
which is likely to screw up terminal output.  Now I have to figure  
out a way around a) the arg popping and b) a good way to verify the  
stack is consistent before/after the call.

I'll have to check the config changes; they're probably pretty  
minor.  I know python has a couple of libffi snapshots, so they at  
least might find a standalone libffi useful.

On Nov 8, 2007, at 12:48 PM, Tom Tromey wrote:

>>>>>> "Timothy" == Timothy Wall <twall@users.sf.net> writes:
>
> Timothy> I'm working on a patch for win32/x86 stdcall closures to  
> support JNA
> Timothy> (http://jna.dev.java.net), a ctypes-like library for  
> Java.  I'm
> Timothy> working off of a libffi snapshot from the gcc trunk.
>
> Nice.  JNA looks cool.
>
> BTW, I think I read somewhere that you have some standalone configure
> changes for libffi...?  We could put those in the separate libffi
> repository if that would help you.  I occasionally think it would be
> nice to revive the standalone libffi -- but only if there's interest.
>
> Timothy> I've got the closure code working now, and I need some  
> pointers on
> Timothy> adding a win32-only stdcall callback test to the testsuite  
> (or a
> Timothy> general one if stdcall is supported elsewhere).
>
> libffi uses the dejagnu 'dg' code, so you should be able to just drop
> your test cases into the test suite.  If the test is platform-specific
> I think you can use the "target" clause to dg-do.  Grep for "target"
> in gcc/testsuite/**/*.c to see uses.
>
> Tom

Re: libffi stdcall closures

[Timothy Wall]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

Here's the patch for stdcall closures, including a basic test.  The  
diff is against the JNA repo; I can probably do one against a gcc  
checkout if necessary.

BTW, the lack of support came up as people were trying to install  
Java callbacks into various windows API functions (services, keyboard  
hooks, etc).  Some would work fine, others would crash.  I wasn't  
aware that libffi didn't have stdcall closure support since the few  
simple tests and examples I wrote just happened to work.


[-- Attachment #2: diffs.txt --]
[-- Type: text/plain, Size: 6605 bytes --]

Index: libffi/ChangeLog
===================================================================
--- libffi/ChangeLog	(revision 243)
+++ libffi/ChangeLog	(working copy)
@@ -1,3 +1,12 @@
+2007-11-08  Timothy Wall  <twall@users.sf.net>
+
+	* testsuite/libffi.call/closure_stdcall.c: Add test for stdcall
+	closures. 
+	* src/x86/ffitarget.h: Increase size of trampoline for stdcall
+	closures. 
+	* src/x86/win32.S: Add assembly for stdcall closure.
+	* src/x86/ffi.c: Initialize stdcall closure trampoline.
+	
 2007-05-13  Release Manager
 
 	* GCC 4.2.0 released.
Index: libffi/src/x86/ffitarget.h
===================================================================
--- libffi/src/x86/ffitarget.h	(revision 243)
+++ libffi/src/x86/ffitarget.h	(working copy)
@@ -73,7 +73,11 @@
 #define FFI_TRAMPOLINE_SIZE 24
 #define FFI_NATIVE_RAW_API 0
 #else
+#ifdef X86_WIN32
+#define FFI_TRAMPOLINE_SIZE 13  
+#else
 #define FFI_TRAMPOLINE_SIZE 10
+#endif
 #define FFI_NATIVE_RAW_API 1	/* x86 has native raw api support */
 #endif
 
Index: libffi/src/x86/win32.S
===================================================================
--- libffi/src/x86/win32.S	(revision 243)
+++ libffi/src/x86/win32.S	(working copy)
@@ -258,6 +258,22 @@
 
 .ffi_call_STDCALL_end:
 
+	.globl _ffi_closure_STDCALL
+_ffi_closure_STDCALL:
+	pushl	%ebp
+	movl	%esp, %ebp
+	subl	$40, %esp
+	leal	-24(%ebp), %edx
+	movl	%edx, -12(%ebp)	/* resp */
+	leal	12(%ebp), %edx  /* account for stub return address on stack */
+	movl	%edx, 4(%esp)	/* args */
+	leal	-12(%ebp), %edx
+	movl	%edx, (%esp)	/* &resp */
+	call	_ffi_closure_SYSV_inner
+	movl	-12(%ebp), %ecx
+	jmp     .Lcls_return_result
+.ffi_closure_STDCALL_end:
+
 	.globl _ffi_closure_SYSV
 _ffi_closure_SYSV:
 	pushl	%ebp
@@ -271,6 +287,7 @@
 	movl	%edx, (%esp)	/* &resp */
 	call	_ffi_closure_SYSV_inner
 	movl	-12(%ebp), %ecx
+.Lcls_return_result:
 	cmpl	$FFI_TYPE_INT, %eax
 	je	.Lcls_retint
 	cmpl	$FFI_TYPE_FLOAT, %eax
Index: libffi/src/x86/ffi.c
===================================================================
--- libffi/src/x86/ffi.c	(revision 243)
+++ libffi/src/x86/ffi.c	(working copy)
@@ -227,6 +227,10 @@
      __attribute__ ((regparm(1)));
 void FFI_HIDDEN ffi_closure_raw_SYSV (ffi_raw_closure *)
      __attribute__ ((regparm(1)));
+#ifdef X86_WIN32
+void FFI_HIDDEN ffi_closure_STDCALL (ffi_closure *)
+     __attribute__ ((regparm(1)));
+#endif
 
 /* This function is jumped to by the trampoline */
 
@@ -302,13 +306,26 @@
 ({ unsigned char *__tramp = (unsigned char*)(TRAMP); \
    unsigned int  __fun = (unsigned int)(FUN); \
    unsigned int  __ctx = (unsigned int)(CTX); \
-   unsigned int  __dis = __fun - ((unsigned int) __tramp + FFI_TRAMPOLINE_SIZE); \
+   unsigned int  __dis = __fun - ((unsigned int) __tramp + 10); \
    *(unsigned char*) &__tramp[0] = 0xb8; \
    *(unsigned int*)  &__tramp[1] = __ctx; /* movl __ctx, %eax */ \
    *(unsigned char *)  &__tramp[5] = 0xe9; \
    *(unsigned int*)  &__tramp[6] = __dis; /* jmp __fun  */ \
  })
 
+#define FFI_INIT_TRAMPOLINE_STDCALL(TRAMP,FUN,CTX,SIZE)  \
+({ unsigned char *__tramp = (unsigned char*)(TRAMP); \
+   unsigned int  __fun = (unsigned int)(FUN); \
+   unsigned int  __ctx = (unsigned int)(CTX); \
+   unsigned int  __dis = __fun - ((unsigned int) __tramp + 10); \
+   unsigned short __size = (unsigned short)(SIZE); \
+   *(unsigned char*) &__tramp[0] = 0xb8; \
+   *(unsigned int*)  &__tramp[1] = __ctx; /* movl __ctx, %eax */ \
+   *(unsigned char *)  &__tramp[5] = 0xe8; \
+   *(unsigned int*)  &__tramp[6] = __dis; /* call __fun  */ \
+   *(unsigned char *)  &__tramp[10] = 0xc2; \
+   *(unsigned short*)  &__tramp[11] = __size; /* ret __size  */ \
+ })
 
 /* the cif must already be prep'ed */
 
@@ -318,12 +335,24 @@
 		  void (*fun)(ffi_cif*,void*,void**,void*),
 		  void *user_data)
 {
+#ifdef X86_WIN32
+  FFI_ASSERT (cif->abi == FFI_SYSV || cif->abi == FFI_STDCALL);
+#else
   FFI_ASSERT (cif->abi == FFI_SYSV);
+#endif
 
   FFI_INIT_TRAMPOLINE (&closure->tramp[0], \
 		       &ffi_closure_SYSV,  \
 		       (void*)closure);
-    
+
+#ifdef X86_WIN32
+  if (cif->abi == FFI_STDCALL) {
+    FFI_INIT_TRAMPOLINE_STDCALL (&closure->tramp[0],    \
+                                 &ffi_closure_STDCALL,  \
+                                 (void*)closure, cif->bytes);
+  }
+#endif
+
   closure->cif  = cif;
   closure->user_data = user_data;
   closure->fun  = fun;
Index: libffi/testsuite/libffi.call/closure_stdcall.c
===================================================================
--- libffi/testsuite/libffi.call/closure_stdcall.c	(revision 0)
+++ libffi/testsuite/libffi.call/closure_stdcall.c	(revision 376)
@@ -0,0 +1,72 @@
+/* Area:	closure_call (stdcall convention)
+   Purpose:	Check handling when caller expects stdcall callee 
+   Limitations:	none.
+   PR:		none.
+   Originator:	<twalljava@dev.java.net> */
+
+/* { dg-do run { target i?86-*-cygwin* i?86-*-mingw* } } */
+#include "ffitest.h"
+
+static void
+closure_test_stdcall(ffi_cif* cif __UNUSED__, void* resp, void** args,
+		 void* userdata)
+{
+  *(ffi_arg*)resp =
+    (int)*(int *)args[0] + (int)(*(int *)args[1]) 
+    + (int)(*(int *)args[2])  + (int)(*(int *)args[3]) 
+    + (int)(long)userdata;
+
+  printf("%d %d %d %d: %d\n",
+	 (int)*(int *)args[0], (int)(*(int *)args[1]),
+	 (int)(*(int *)args[2]), (int)(*(int *)args[3]),
+         (int)*(ffi_arg *)resp);
+
+}
+
+typedef int (__stdcall *closure_test_type0)(int, int, int, int);
+
+int main (void)
+{
+  ffi_cif cif;
+#ifndef USING_MMAP
+  static ffi_closure cl;
+#endif
+  ffi_closure *pcl;
+  ffi_type * cl_arg_types[17];
+  int res;
+  void* sp_pre;
+  void* sp_post;
+  char buf[1024];
+
+#ifdef USING_MMAP
+  pcl = allocate_mmap (sizeof(ffi_closure));
+#else
+  pcl = &cl;
+#endif
+
+  cl_arg_types[0] = &ffi_type_uint;
+  cl_arg_types[1] = &ffi_type_uint;
+  cl_arg_types[2] = &ffi_type_uint;
+  cl_arg_types[3] = &ffi_type_uint;
+  cl_arg_types[4] = NULL;
+
+  /* Initialize the cif */
+  CHECK(ffi_prep_cif(&cif, FFI_STDCALL, 4,
+		     &ffi_type_sint, cl_arg_types) == FFI_OK);
+
+  CHECK(ffi_prep_closure(pcl, &cif, closure_test_stdcall,
+			 (void *) 3 /* userdata */) == FFI_OK);
+
+  asm volatile (" movl %%esp,%0" : "=g" (sp_pre));
+  res = (*(closure_test_type0)pcl)(0, 1, 2, 3);
+  asm volatile (" movl %%esp,%0" : "=g" (sp_post));
+  /* { dg-output "0 1 2 3: 9" } */
+
+  printf("res: %d\n",res);
+  /* { dg-output "\nres: 9" } */
+
+  sprintf(buf, "mismatch: pre=%p vs post=%p", sp_pre, sp_post);
+  printf("stack pointer %s\n", (sp_pre == sp_post ? "match" : buf));
+  /* { dg-output "\nstack pointer match" } */
+  exit(0);
+}

Re: libffi stdcall closures

[Tom Tromey]

>>>>> "Timothy" == Timothy Wall <twalljava@dev.java.net> writes:

Timothy> Here's the patch for stdcall closures, including a basic test.  The
Timothy> diff is against the JNA repo; I can probably do one against a gcc
Timothy> checkout if necessary.

I'm not really an x86 expert, so perhaps a port maintainer could look
it over.  The test case also looked good at first blush, but Andreas
is really the person for that...

I have a couple nits to pick, but nothing major.

Timothy> BTW, the lack of support came up as people were trying to install
Timothy> Java callbacks into various windows API functions (services, keyboard
Timothy> hooks, etc).  Some would work fine, others would crash.  I wasn't
Timothy> aware that libffi didn't have stdcall closure support since the few
Timothy> simple tests and examples I wrote just happened to work.

FWIW there are other areas where you may run into problems.  The
biggest one is varargs -- libffi basically doesn't support this at
all.  I'm sure there are more obscure ones, too, involving any new ABI
additions... maybe vectors, or decimal float, or I don't know what.

Timothy> +#ifdef X86_WIN32
Timothy> +  if (cif->abi == FFI_STDCALL) {
Timothy> +    FFI_INIT_TRAMPOLINE_STDCALL (&closure->tramp[0],    \
Timothy> +                                 &ffi_closure_STDCALL,  \
Timothy> +                                 (void*)closure, cif->bytes);
Timothy> +  }
Timothy> +#endif

You don't need the backslashes here.  I see them elsewhere in the
file, but they are bogus.  Also the "{" should be on its own line.

Also, it looks to me as though there's a missing 'else' in
here... shouldn't we call either FFI_INIT_TRAMPOLINE_STDCALL or
FFI_INIT_TRAMPOLINE, but not both?

thanks,
Tom

Re: libffi stdcall closures

[Timothy Wall]

On Nov 13, 2007, at 4:08 PM, Tom Tromey wrote:

>>>>>> "Timothy" == Timothy Wall <twalljava@dev.java.net> writes:
>
> Timothy> Here's the patch for stdcall closures, including a basic  
> test.  The
> Timothy> diff is against the JNA repo; I can probably do one  
> against a gcc
> Timothy> checkout if necessary.
>
> I'm not really an x86 expert, so perhaps a port maintainer could look
> it over.  The test case also looked good at first blush, but Andreas
> is really the person for that...

Should I ping him directly?

>
> I have a couple nits to pick, but nothing major.
>
> FWIW there are other areas where you may run into problems.  The
> biggest one is varargs -- libffi basically doesn't support this at
> all.  I'm sure there are more obscure ones, too, involving any new ABI
> additions... maybe vectors, or decimal float, or I don't know what.

Do you mean varargs out or varargs in closures?  I'm not too worried  
about varargs in closures, and varargs calls through libffi seem to  
work ok in the platforms we've tried so far.


>
> Timothy> +#ifdef X86_WIN32
> Timothy> +  if (cif->abi == FFI_STDCALL) {
> Timothy> +    FFI_INIT_TRAMPOLINE_STDCALL (&closure->tramp[0],    \
> Timothy> +                                 &ffi_closure_STDCALL,  \
> Timothy> +                                 (void*)closure, cif- 
> >bytes);
> Timothy> +  }
> Timothy> +#endif
>
> You don't need the backslashes here.  I see them elsewhere in the
> file, but they are bogus.  Also the "{" should be on its own line.
>
> Also, it looks to me as though there's a missing 'else' in
> here... shouldn't we call either FFI_INIT_TRAMPOLINE_STDCALL or
> FFI_INIT_TRAMPOLINE, but not both?

That's just me trying to avoid intersecting conditional code with  
logic constructs.

Following is a better construct, dropping the somewhat useless  
ASSERTions:

>   if (cif->abi == FFI_SYSV)
>     {
>       FFI_INIT_TRAMPOLINE (&closure->tramp[0],
>                            &ffi_closure_SYSV,
>                            (void*)closure);
>     }
> #ifdef X86_WIN32
>   else if (cif->abi == FFI_STDCALL)
>     {
>       FFI_INIT_TRAMPOLINE_STDCALL (&closure->tramp[0],
>                                    &ffi_closure_STDCALL,
>                                    (void*)closure, cif->bytes);
>     }
> #endif
>   else
>     {
>       return FFI_BAD_ABI;
>     }
>

I'm looking over the JNA standalone libffi config changes and will  
post those separately.