libffi does not follow proper ABI on ia32 (?)

[u-xsnf@aetey.se]

Hello,

I am following the directions given on https://sourceware.org/libffi/
about submitting bug reports to the list.

Below you see excerpts from a letter posted to the pcc mailing list
based on a try to build libffi with pcc (the "Portable C Compiler")

This looks like libffi is not following the necessary ABI. Is there any
chance for this to be fixed, short of having to learn the internals of
libffi and submitting a patch?

Regards,
Rune

----- Forwarded message -----

Date: Sun, 21 Dec 2014 18:47:01 +0100
Subject: Re: [Pcc] libffi tests segfault with pcc (cvs 20141115 on i486)
To: pcc@lists.ludd.ltu.se

[skipped ...]

On Sat, Dec 20, 2014 at 05:40:27PM +0100, u-lxna@aetey.se wrote:
> res_dbl = ((cls_struct_12byte(*)(cls_struct_12byte, cls_struct_12byte))(code))(h_dbl, j_dbl);

> The relevant code generated by pcc looks like
>  ...
> 0x8048826 <main+430>:   call   *%eax                 <=== the call
> 0x8048828 <main+432>:   add    $0x18,%esp
> 0x804882b <main+435>:   mov    %eax,%esi
> 0x804882d <main+437>:   lea    -0x80(%ebp),%edi
> 0x8048830 <main+440>:   movsl  %ds:(%esi),%es:(%edi) <=== segfault

> (gdb) i r
> eax            0xd      13

> while with gcc (4.2.3) it is afaict
>  ...
> 0x080487ad <main+509>:  call   *0xec(%esp)           <=== the call
> 0x080487b4 <main+516>:  cld    
> 0x080487b5 <main+517>:  mov    0x34(%esp),%edi
> 0x080487b9 <main+521>:  mov    0x28(%esp),%esi
> 0x080487bd <main+525>:  mov    %ebx,%ecx
> 0x080487bf <main+527>:  rep movsl %ds:(%esi),%es:(%edi)
> 0x080487c1 <main+529>:  pop    %ebp
> 0x080487c2 <main+530>:  pop    %eax

In other words pcc is relying on the contents of %eax at the return
from the function call while gcc clearly ignores/overwrites that value.

https://en.wikipedia.org/wiki/X86_calling_conventions
says among others about "cdecl" ABI:
"
the caller allocates memory and passes a pointer to it as a hidden first
parameter; the callee populates the memory and returns the pointer
"

Which makes me wonder whether libffi conforms to the above in this case,
%eax on return is 13, which value is being used as the pointer
by pcc and crashes the program.

Gcc seems to remember where the memory area for the data to return is
and does not use the pointer "expected" to be returned by the callee.

Does this analysis look correct?

[skipped ...]

A half-hearted check seems to show that gcc actually makes the callee
return the pointer even though the caller never uses it.

[skipped ...]

So libffi might have to be fixed to provide the pointer too but the opinion
of its developers is yet to be determined.

[skipped ...]

----- End forwarded message -----