Libffi problem on darwin (MacOSX)

Libffi problem on darwin (MacOSX)

[Ronald Oussoren]

Hi,

The libffi mailing seems to be dead, hence my post here. I'm using  
libffi to interface Python and Objective-C (http://pyobjc.sf.net). This  
is working pretty well, but I'm running into a problem that seems to be  
libffi related.

We're using libffi closures in the Objective-C method tables to forward  
calls from Objective-C to python. This is working fine for basic calls,  
but gives wrong results when structs are passed. It looks like a  
mismatch between the compiler and libffi: ffi_closure_helper_DARWIN in  
libffi_darwin.c seems to assume that structs are passed "by reference"  
(line 615), while the compiler passes small structs entirely in  
registers (see assembly code emited by the following C snippet):

struct foobar {
         float a;
         int b;
};

extern struct foobar h;
extern int i;


void callee(struct foobar a1, int a2)
{
}

void caller(void)
{
         struct foobar g = { 1.0, 2.0 };
         callee(g, 42);
}


The compiler behaviour seems to be the same as prescribed by 'Mach-O  
runtime conventions on PowerPC'  
(http://developer.apple.com/techpubs/macosx/DeveloperTools/ 
MachORuntime/2rt_powerpc_abi/index.html).

I'm using Apple's version of GCC 3.1 and a libffi from the GCC  
repository (HEAD branch). I've disabled the eh_frames in darwin*.S  
(otherwise this version of libffi won't work with Apple's GCC), but  
otherwise it's the same as in CVS.

Ronald

Re: Libffi problem on darwin (MacOSX)

[Dale Johannesen]

On Friday, May 16, 2003, at 01:55  PM, Ronald Oussoren wrote:
> We're using libffi closures in the Objective-C method tables to 
> forward calls from Objective-C to python. This is working fine for 
> basic calls, but gives wrong results when structs are passed. It looks 
> like a mismatch between the compiler and libffi: 
> ffi_closure_helper_DARWIN in libffi_darwin.c seems to assume that 
> structs are passed "by reference" (line 615), while the compiler 
> passes small structs entirely in registers (see assembly code emited 
> by the following C snippet):

I'm not familiar with libffi, but I can confirm that the compiler is 
doing the
right thing, and it's the same thing it's always done.  It looks like 
ffi_prep_args
and ffi_prep_cif_machdep correctly understand that structs are passed 
by value, so
this is just a bug in ffi_closure_helper_DARWIN.

Re: Libffi problem on darwin (MacOSX)

[Andreas Tobler]

Dale Johannesen wrote:
> On Friday, May 16, 2003, at 01:55  PM, Ronald Oussoren wrote:
> 
>> We're using libffi closures in the Objective-C method tables to 
>> forward calls from Objective-C to python. This is working fine for 
>> basic calls, but gives wrong results when structs are passed. It looks 
>> like a mismatch between the compiler and libffi: 
>> ffi_closure_helper_DARWIN in libffi_darwin.c seems to assume that 
>> structs are passed "by reference" (line 615), while the compiler 
>> passes small structs entirely in registers (see assembly code emited 
>> by the following C snippet):
> 
> 
> I'm not familiar with libffi, but I can confirm that the compiler is 
> doing the
> right thing, and it's the same thing it's always done.  It looks like 
> ffi_prep_args
> and ffi_prep_cif_machdep correctly understand that structs are passed by 
> value, so
> this is just a bug in ffi_closure_helper_DARWIN.

Hey great, people are using libffi on darwin-ppc. I guess this is born 
on my camp. I'll have a look at, as soon as  time allows here.
It is possible that I didn't finish the libffi regarding structures 
since I didn't need it for libjava work. So, you could be right.

Thank you for the report.

Andreas

Re: Libffi problem on darwin (MacOSX)

[Ronald Oussoren]

On Saturday, May 17, 2003, at 07:52 Europe/Amsterdam, Andreas Tobler 
wrote:

> Dale Johannesen wrote:
>> On Friday, May 16, 2003, at 01:55  PM, Ronald Oussoren wrote:
>>> We're using libffi closures in the Objective-C method tables to 
>>> forward calls from Objective-C to python. This is working fine for 
>>> basic calls, but gives wrong results when structs are passed. It 
>>> looks like a mismatch between the compiler and libffi: 
>>> ffi_closure_helper_DARWIN in libffi_darwin.c seems to assume that 
>>> structs are passed "by reference" (line 615), while the compiler 
>>> passes small structs entirely in registers (see assembly code emited 
>>> by the following C snippet):
>> I'm not familiar with libffi, but I can confirm that the compiler is 
>> doing the
>> right thing, and it's the same thing it's always done.  It looks like 
>> ffi_prep_args
>> and ffi_prep_cif_machdep correctly understand that structs are passed 
>> by value, so
>> this is just a bug in ffi_closure_helper_DARWIN.
>
> Hey great, people are using libffi on darwin-ppc.
And were very glad this is available on darwin-ppc, libffi is a very 
usefull library.

> I guess this is born on my camp. I'll have a look at, as soon as  time 
> allows here.
> It is possible that I didn't finish the libffi regarding structures 
> since I didn't need it for libjava work. So, you could be right.

I just noticed that this might well be stupidity on my part, I tried to 
write a minimal program that demonstrates the problem and that runs 
without problems. Our real code builds ffi_cifs and ffi_closures at 
runtime, and maybe that code is bogus.

Sorry about the inconvenience,
	Ronald

--- example.c ---
#include "ffi.h"
#include <stdio.h>

struct foobar {
	float a;
	int b;
};

void callee(struct foobar a1, int a2)
{
	printf("%g@%d %d\n", a1.a, a1.b, a2);
}

void stub(ffi_cif* cif, void* resp, void** args, void* userdata)
{	
	struct foobar a1;
	int  a2;

	a1 = *(struct foobar*)(args[0]);
	a2 = *(int*)(args[1]);

	callee(a1, a2);
}


int main(void)
{
	ffi_type* foobar_fields[3];
	ffi_type foobar_type;
	ffi_cif cif;
	ffi_closure cl;
	ffi_status rv;
	void* args[4];
	ffi_type* arg_types[3];
	
	struct foobar g = { 1.0, 2.0 };
	int fourtytwo = 42;

	foobar_type.size = 0;
	foobar_type.alignment = 0;
	foobar_type.type = FFI_TYPE_STRUCT;
	foobar_type.elements = foobar_fields;

	foobar_fields[0] = &ffi_type_float;
	foobar_fields[1] = &ffi_type_sint;
	foobar_fields[2] = NULL;

	arg_types[0] = &foobar_type;
	arg_types[1] = &ffi_type_sint;
	arg_types[2] = NULL;

	rv = ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 2, &ffi_type_void, arg_types);
	if (rv != FFI_OK) {
		printf("ffi cif: %d\n", rv);
		abort();
	}

	printf ("Direct call:");
	callee(g, fourtytwo);

	printf("FFI call:");
	args[0] = &g;
	args[1] = &fourtytwo;
	args[2] = NULL;
	ffi_call(&cif, FFI_FN(callee), NULL, args);

	printf("FFI closure call:");
	rv = ffi_prep_closure(&cl, &cif, stub, NULL);
	if (rv != FFI_OK) {
		printf("ffi cl: %d\n", rv);
		abort();
	}
	((void(*)(struct foobar, int))(&cl))(g, fourtytwo);



	return 0;
}

Re: Libffi problem on darwin (MacOSX)

[Andreas Tobler]

Ronald Oussoren wrote:

> 
> On Saturday, May 17, 2003, at 07:52 Europe/Amsterdam, Andreas Tobler wrote:

Hm, Amsterdam? I live in Zürich.... :)

>> Hey great, people are using libffi on darwin-ppc.
> 
> And were very glad this is available on darwin-ppc, libffi is a very 
> usefull library.

Fine, let me know if you need some more functionality. I implemented the 
closure stuff only for libjava. And I seem to be one of the only persons 
using that. But hopefully soon I get the libjava on darwin that far that 
everyone can use it.
> 
>> I guess this is born on my camp. I'll have a look at, as soon as  time 
>> allows here.
>> It is possible that I didn't finish the libffi regarding structures 
>> since I didn't need it for libjava work. So, you could be right.
> 
> 
> I just noticed that this might well be stupidity on my part, I tried to 
> write a minimal program that demonstrates the problem and that runs 
> without problems. Our real code builds ffi_cifs and ffi_closures at 
> runtime, and maybe that code is bogus.

Could be, but if I remember right, I didn't complete the structure 
passings. Maybe it works, but if not, then I certainly have to check 
again. Well, I'll do that anyway.

> 
> Sorry about the inconvenience,

no worry, you give me some kick to go over again :)

Thanks,

Andreas

Re: Libffi problem on darwin (MacOSX)

[Ronald Oussoren]

On Saturday, May 17, 2003, at 17:21 Europe/Amsterdam, Andreas Tobler  
wrote:

>> I just noticed that this might well be stupidity on my part, I tried  
>> to write a minimal program that demonstrates the problem and that  
>> runs without problems. Our real code builds ffi_cifs and ffi_closures  
>> at runtime, and maybe that code is bogus.
>
> Could be, but if I remember right, I didn't complete the structure  
> passings. Maybe it works, but if not, then I certainly have to check  
> again. Well, I'll do that anyway.

The following hack seems to work for the limited tests that I've done.  
Note that this is *not* based on any understanding of the PPC calling  
conventions, and therefore likely incorrect.

Ronald

***  
/Volumes/Data/Users/ronald/Python-CVS/gcc/libffi/src/powerpc/ 
ffi_darwin.c   Sat Feb 22 20:10:14 2003
--- ffi_darwin.c        Sat May 24 22:02:44 2003
***************
*** 612,619 ****
--- 612,633 ----
           case FFI_TYPE_SINT32:
           case FFI_TYPE_UINT32:
           case FFI_TYPE_POINTER:
+ //        case FFI_TYPE_STRUCT:
+         /* there are 8 gpr registers used to pass values */
+           if (ng < 8) {
+              avalue[i] = pgr;
+              ng++;
+              pgr++;
+           } else {
+              avalue[i] = pst;
+              pst++;
+           }
+           break;
+
           case FFI_TYPE_STRUCT:
           /* there are 8 gpr registers used to pass values */
+       {
+           int left = (arg_types[i]->size);
             if (ng < 8) {
                avalue[i] = pgr;
                ng++;
***************
*** 622,627 ****
--- 636,654 ----
                avalue[i] = pst;
                pst++;
             }
+
+         /* Structs are in registers, skip values */
+         left -= sizeof(long);
+         while (left > 0) {
+                 if (ng < 8) {
+                    ng++;
+                    pgr++;
+                 } else {
+                    pst++;
+                 }
+                 left -= sizeof(long);
+         }
+       }
             break;

           case FFI_TYPE_SINT64:

Re: Libffi problem on darwin (MacOSX)

[Andreas Tobler]

Ronald Oussoren wrote:
> 
> On Saturday, May 17, 2003, at 17:21 Europe/Amsterdam, Andreas Tobler  
> wrote:
> 
>>> I just noticed that this might well be stupidity on my part, I tried  
>>> to write a minimal program that demonstrates the problem and that  
>>> runs without problems. Our real code builds ffi_cifs and 
>>> ffi_closures  at runtime, and maybe that code is bogus.
>>
>>
>> Could be, but if I remember right, I didn't complete the structure  
>> passings. Maybe it works, but if not, then I certainly have to check  
>> again. Well, I'll do that anyway.
> 
> 
> The following hack seems to work for the limited tests that I've done.  
> Note that this is *not* based on any understanding of the PPC calling  
> conventions, and therefore likely incorrect.

Thanks, I'll have a look at. It'll take a while since I will test 
against libgcj which needs a longer time to build.
Do you have a dedicated testcase for?

Andreas

Re: Libffi problem on darwin (MacOSX)

[Ronald Oussoren]

On Monday, May 26, 2003, at 07:12 Europe/Amsterdam, Andreas Tobler 
wrote:

> Ronald Oussoren wrote:
>> On Saturday, May 17, 2003, at 17:21 Europe/Amsterdam, Andreas Tobler  
>> wrote:
>>>> I just noticed that this might well be stupidity on my part, I 
>>>> tried  to write a minimal program that demonstrates the problem and 
>>>> that  runs without problems. Our real code builds ffi_cifs and 
>>>> ffi_closures  at runtime, and maybe that code is bogus.
>>>
>>>
>>> Could be, but if I remember right, I didn't complete the structure  
>>> passings. Maybe it works, but if not, then I certainly have to check 
>>>  again. Well, I'll do that anyway.
>> The following hack seems to work for the limited tests that I've 
>> done.  Note that this is *not* based on any understanding of the PPC 
>> calling  conventions, and therefore likely incorrect.
>
> Thanks, I'll have a look at. It'll take a while since I will test 
> against libgcj which needs a longer time to build.
> Do you have a dedicated testcase for?

I don't have a small standalone testcase for this problem. The problem 
is triggered by a unittest in PyObjC, I can use that to check patches.

Ronald