showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf@aetey.se]

I understand that under this season people are occupied more than usual
with the so called real life.

Nevertheless I would appreciate at least hints about the prospect to fix
this bug (apparently non-conformant C ABI of libffi on x86 Linux),
which presumably went unnoticed for a long time due to gcc not exercising
the full ABI.

This is crucial, a showstopper, uncovered by the Portable C Compiler
which strictly follows the ABI.

Is anybody looking at bug reports here?

Merry Christmas to everyone!

Rune

On Mon, Dec 22, 2014 at 08:35:38PM +0100, u-xsnf@aetey.se wrote:
> Hello,
> 
> I am following the directions given on https://sourceware.org/libffi/
> about submitting bug reports to the list.
> 
> Below you see excerpts from a letter posted to the pcc mailing list
> based on a try to build libffi with pcc (the "Portable C Compiler")
> 
> This looks like libffi is not following the necessary ABI. Is there any
> chance for this to be fixed, short of having to learn the internals of
> libffi and submitting a patch?
> 
> Regards,
> Rune
> 
> ----- Forwarded message -----
> 
> Date: Sun, 21 Dec 2014 18:47:01 +0100
> Subject: Re: [Pcc] libffi tests segfault with pcc (cvs 20141115 on i486)
> To: pcc@lists.ludd.ltu.se
> 
> [skipped ...]
> 
> On Sat, Dec 20, 2014 at 05:40:27PM +0100, u-lxna@aetey.se wrote:
> > res_dbl = ((cls_struct_12byte(*)(cls_struct_12byte, cls_struct_12byte))(code))(h_dbl, j_dbl);
> 
> > The relevant code generated by pcc looks like
> >  ...
> > 0x8048826 <main+430>:   call   *%eax                 <=== the call
> > 0x8048828 <main+432>:   add    $0x18,%esp
> > 0x804882b <main+435>:   mov    %eax,%esi
> > 0x804882d <main+437>:   lea    -0x80(%ebp),%edi
> > 0x8048830 <main+440>:   movsl  %ds:(%esi),%es:(%edi) <=== segfault
> 
> > (gdb) i r
> > eax            0xd      13
> 
> > while with gcc (4.2.3) it is afaict
> >  ...
> > 0x080487ad <main+509>:  call   *0xec(%esp)           <=== the call
> > 0x080487b4 <main+516>:  cld    
> > 0x080487b5 <main+517>:  mov    0x34(%esp),%edi
> > 0x080487b9 <main+521>:  mov    0x28(%esp),%esi
> > 0x080487bd <main+525>:  mov    %ebx,%ecx
> > 0x080487bf <main+527>:  rep movsl %ds:(%esi),%es:(%edi)
> > 0x080487c1 <main+529>:  pop    %ebp
> > 0x080487c2 <main+530>:  pop    %eax
> 
> In other words pcc is relying on the contents of %eax at the return
> from the function call while gcc clearly ignores/overwrites that value.
> 
> https://en.wikipedia.org/wiki/X86_calling_conventions
> says among others about "cdecl" ABI:
> "
> the caller allocates memory and passes a pointer to it as a hidden first
> parameter; the callee populates the memory and returns the pointer
> "
> 
> Which makes me wonder whether libffi conforms to the above in this case,
> %eax on return is 13, which value is being used as the pointer
> by pcc and crashes the program.
> 
> Gcc seems to remember where the memory area for the data to return is
> and does not use the pointer "expected" to be returned by the callee.
> 
> Does this analysis look correct?
> 
> [skipped ...]
> 
> A half-hearted check seems to show that gcc actually makes the callee
> return the pointer even though the caller never uses it.
> 
> [skipped ...]
> 
> So libffi might have to be fixed to provide the pointer too but the opinion
> of its developers is yet to be determined.
> 
> [skipped ...]
> 
> ----- End forwarded message -----
>