libffi does not follow proper ABI on ia32 (?)

libffi does not follow proper ABI on ia32 (?)

[u-xsnf]

Hello,

I am following the directions given on https://sourceware.org/libffi/
about submitting bug reports to the list.

Below you see excerpts from a letter posted to the pcc mailing list
based on a try to build libffi with pcc (the "Portable C Compiler")

This looks like libffi is not following the necessary ABI. Is there any
chance for this to be fixed, short of having to learn the internals of
libffi and submitting a patch?

Regards,
Rune

----- Forwarded message -----

Date: Sun, 21 Dec 2014 18:47:01 +0100
Subject: Re: [Pcc] libffi tests segfault with pcc (cvs 20141115 on i486)
To: pcc@lists.ludd.ltu.se

[skipped ...]

On Sat, Dec 20, 2014 at 05:40:27PM +0100, u-lxna@aetey.se wrote:
> res_dbl = ((cls_struct_12byte(*)(cls_struct_12byte, cls_struct_12byte))(code))(h_dbl, j_dbl);

> The relevant code generated by pcc looks like
>  ...
> 0x8048826 <main+430>:   call   *%eax                 <=== the call
> 0x8048828 <main+432>:   add    $0x18,%esp
> 0x804882b <main+435>:   mov    %eax,%esi
> 0x804882d <main+437>:   lea    -0x80(%ebp),%edi
> 0x8048830 <main+440>:   movsl  %ds:(%esi),%es:(%edi) <=== segfault

> (gdb) i r
> eax            0xd      13

> while with gcc (4.2.3) it is afaict
>  ...
> 0x080487ad <main+509>:  call   *0xec(%esp)           <=== the call
> 0x080487b4 <main+516>:  cld    
> 0x080487b5 <main+517>:  mov    0x34(%esp),%edi
> 0x080487b9 <main+521>:  mov    0x28(%esp),%esi
> 0x080487bd <main+525>:  mov    %ebx,%ecx
> 0x080487bf <main+527>:  rep movsl %ds:(%esi),%es:(%edi)
> 0x080487c1 <main+529>:  pop    %ebp
> 0x080487c2 <main+530>:  pop    %eax

In other words pcc is relying on the contents of %eax at the return
from the function call while gcc clearly ignores/overwrites that value.

https://en.wikipedia.org/wiki/X86_calling_conventions
says among others about "cdecl" ABI:
"
the caller allocates memory and passes a pointer to it as a hidden first
parameter; the callee populates the memory and returns the pointer
"

Which makes me wonder whether libffi conforms to the above in this case,
%eax on return is 13, which value is being used as the pointer
by pcc and crashes the program.

Gcc seems to remember where the memory area for the data to return is
and does not use the pointer "expected" to be returned by the callee.

Does this analysis look correct?

[skipped ...]

A half-hearted check seems to show that gcc actually makes the callee
return the pointer even though the caller never uses it.

[skipped ...]

So libffi might have to be fixed to provide the pointer too but the opinion
of its developers is yet to be determined.

[skipped ...]

----- End forwarded message -----

showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

I understand that under this season people are occupied more than usual
with the so called real life.

Nevertheless I would appreciate at least hints about the prospect to fix
this bug (apparently non-conformant C ABI of libffi on x86 Linux),
which presumably went unnoticed for a long time due to gcc not exercising
the full ABI.

This is crucial, a showstopper, uncovered by the Portable C Compiler
which strictly follows the ABI.

Is anybody looking at bug reports here?

Merry Christmas to everyone!

Rune

On Mon, Dec 22, 2014 at 08:35:38PM +0100, u-xsnf@aetey.se wrote:
> Hello,
> 
> I am following the directions given on https://sourceware.org/libffi/
> about submitting bug reports to the list.
> 
> Below you see excerpts from a letter posted to the pcc mailing list
> based on a try to build libffi with pcc (the "Portable C Compiler")
> 
> This looks like libffi is not following the necessary ABI. Is there any
> chance for this to be fixed, short of having to learn the internals of
> libffi and submitting a patch?
> 
> Regards,
> Rune
> 
> ----- Forwarded message -----
> 
> Date: Sun, 21 Dec 2014 18:47:01 +0100
> Subject: Re: [Pcc] libffi tests segfault with pcc (cvs 20141115 on i486)
> To: pcc@lists.ludd.ltu.se
> 
> [skipped ...]
> 
> On Sat, Dec 20, 2014 at 05:40:27PM +0100, u-lxna@aetey.se wrote:
> > res_dbl = ((cls_struct_12byte(*)(cls_struct_12byte, cls_struct_12byte))(code))(h_dbl, j_dbl);
> 
> > The relevant code generated by pcc looks like
> >  ...
> > 0x8048826 <main+430>:   call   *%eax                 <=== the call
> > 0x8048828 <main+432>:   add    $0x18,%esp
> > 0x804882b <main+435>:   mov    %eax,%esi
> > 0x804882d <main+437>:   lea    -0x80(%ebp),%edi
> > 0x8048830 <main+440>:   movsl  %ds:(%esi),%es:(%edi) <=== segfault
> 
> > (gdb) i r
> > eax            0xd      13
> 
> > while with gcc (4.2.3) it is afaict
> >  ...
> > 0x080487ad <main+509>:  call   *0xec(%esp)           <=== the call
> > 0x080487b4 <main+516>:  cld    
> > 0x080487b5 <main+517>:  mov    0x34(%esp),%edi
> > 0x080487b9 <main+521>:  mov    0x28(%esp),%esi
> > 0x080487bd <main+525>:  mov    %ebx,%ecx
> > 0x080487bf <main+527>:  rep movsl %ds:(%esi),%es:(%edi)
> > 0x080487c1 <main+529>:  pop    %ebp
> > 0x080487c2 <main+530>:  pop    %eax
> 
> In other words pcc is relying on the contents of %eax at the return
> from the function call while gcc clearly ignores/overwrites that value.
> 
> https://en.wikipedia.org/wiki/X86_calling_conventions
> says among others about "cdecl" ABI:
> "
> the caller allocates memory and passes a pointer to it as a hidden first
> parameter; the callee populates the memory and returns the pointer
> "
> 
> Which makes me wonder whether libffi conforms to the above in this case,
> %eax on return is 13, which value is being used as the pointer
> by pcc and crashes the program.
> 
> Gcc seems to remember where the memory area for the data to return is
> and does not use the pointer "expected" to be returned by the callee.
> 
> Does this analysis look correct?
> 
> [skipped ...]
> 
> A half-hearted check seems to show that gcc actually makes the callee
> return the pointer even though the caller never uses it.
> 
> [skipped ...]
> 
> So libffi might have to be fixed to provide the pointer too but the opinion
> of its developers is yet to be determined.
> 
> [skipped ...]
> 
> ----- End forwarded message -----
> 

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[Richard Henderson]

On 12/24/2014 05:58 AM, u-xsnf@aetey.se wrote:
> Nevertheless I would appreciate at least hints about the prospect to fix
> this bug (apparently non-conformant C ABI of libffi on x86 Linux),
> which presumably went unnoticed for a long time due to gcc not exercising
> the full ABI.

Yes, I see the problem.  Fixing it may have to wait until the new year.


r~

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

Thanks Richard,

for the reply and the confirmation!

On Wed, Dec 24, 2014 at 08:31:13AM -0800, Richard Henderson wrote:
> On 12/24/2014 05:58 AM, u-xsnf@aetey.se wrote:
> > Nevertheless I would appreciate at least hints about the prospect to fix
> > this bug (apparently non-conformant C ABI of libffi on x86 Linux),
> > which presumably went unnoticed for a long time due to gcc not exercising
> > the full ABI.
> 
> Yes, I see the problem.  Fixing it may have to wait until the new year.

This is fortunately not a very long time to come :)

Looking forward to a fix - when you find time for it.

Regards,
Rune

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[Richard Henderson]

On 12/24/2014 08:31 AM, Richard Henderson wrote:
> On 12/24/2014 05:58 AM, u-xsnf@aetey.se wrote:
>> Nevertheless I would appreciate at least hints about the prospect to fix
>> this bug (apparently non-conformant C ABI of libffi on x86 Linux),
>> which presumably went unnoticed for a long time due to gcc not exercising
>> the full ABI.
> 
> Yes, I see the problem.  Fixing it may have to wait until the new year.

Please try the branch

  git://github.com/rth7680/libffi.git pcc


r~

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

Hello Richard,

Thanks for coming back on the New Year's Eve!

On Wed, Dec 31, 2014 at 08:12:47AM -0800, Richard Henderson wrote:
> On 12/24/2014 08:31 AM, Richard Henderson wrote:
> > On 12/24/2014 05:58 AM, u-xsnf@aetey.se wrote:
> >> Nevertheless I would appreciate at least hints about the prospect to fix
> >> this bug (apparently non-conformant C ABI of libffi on x86 Linux),
> >> which presumably went unnoticed for a long time due to gcc not exercising
> >> the full ABI.
> > 
> > Yes, I see the problem.  Fixing it may have to wait until the new year.
> 
> Please try the branch
> 
>   git://github.com/rth7680/libffi.git pcc

I was able to compile it and the behaviour is different now, unfortunately
getting a segfault anyway, in a different place:

-------------------
                === libffi Summary ===

# of expected passes            870
# of unexpected failures        1265
-------------------

Below is some gdb data for the first failing test:

(gdb) run

Starting program: /[....]/i486-pc-linux-gnu/testsuite/closure_fn0.exe
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)

Program received signal SIGSEGV, Segmentation fault.
0xf7f4969b in ffi_closure_inner () from ../.libs/libffi.so.6
(gdb) bt
#0  0xf7f4969b in ffi_closure_inner () from ../.libs/libffi.so.6
#1  0xf7f49cfe in ffi_closure_i386 () from ../.libs/libffi.so.6
#2  0x080488e6 in main ()
(gdb) info reg
eax            0xfa831e74       -92070284
ecx            0x0      0
edx            0xffffd738       -10440
ebx            0xf7f4b41c       -134958052
esp            0xffffd6d0       0xffffd6d0
ebp            0xffffd700       0xffffd700
esi            0xf7ffb8b0       -134235984
edi            0x4      4
eip            0xf7f4969b       0xf7f4969b <ffi_closure_inner+43>
eflags         0x10282  [ SF IF RF ]
cs             0x23     35
ss             0x2b     43
ds             0x2b     43
es             0x2b     43
fs             0x0      0
gs             0x63     99
(gdb) x/20i ffi_closure_inner
0xf7f49670 <ffi_closure_inner>:     enter  $0x30,$0x0
0xf7f49674 <ffi_closure_inner+4>:   mov    %ebx,-0x28(%ebp)
0xf7f49677 <ffi_closure_inner+7>:   mov    %esi,-0x2c(%ebp)
0xf7f4967a <ffi_closure_inner+10>:  mov    %edi,-0x30(%ebp)
0xf7f4967d <ffi_closure_inner+13>:  call   0xf7f49682 <ffi_closure_inner+18>
0xf7f49682 <ffi_closure_inner+18>:  popl   -0x4(%ebp)
0xf7f49685 <ffi_closure_inner+21>:  addl   $0x1d9a,-0x4(%ebp)
0xf7f4968c <ffi_closure_inner+28>:  mov    0xc(%ebp),%ecx
0xf7f4968f <ffi_closure_inner+31>:  mov    0x8(%ebp),%eax
0xf7f49692 <ffi_closure_inner+34>:  mov    0x1c(%eax),%eax
0xf7f49695 <ffi_closure_inner+37>:  mov    %eax,-0x8(%ebp)
0xf7f49698 <ffi_closure_inner+40>:  mov    -0x8(%ebp),%eax
0xf7f4969b <ffi_closure_inner+43>:  mov    (%eax),%eax      <==== segfault
0xf7f4969d <ffi_closure_inner+45>:  mov    %eax,-0xc(%ebp)
0xf7f496a0 <ffi_closure_inner+48>:  mov    -0x8(%ebp),%eax
0xf7f496a3 <ffi_closure_inner+51>:  mov    0x14(%eax),%eax
0xf7f496a6 <ffi_closure_inner+54>:  mov    %eax,-0x10(%ebp)
0xf7f496a9 <ffi_closure_inner+57>:  movl   $0x0,-0x14(%ebp)
0xf7f496b0 <ffi_closure_inner+64>:  mov    0x8(%ebp),%eax
0xf7f496b3 <ffi_closure_inner+67>:  mov    %eax,-0x18(%ebp)

Looking at the source I wonder if this has to do with the reliance
on the fastcall attribute, which pcc does not support.

Hope this helps for making another iteration.

Rune

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[Richard Henderson]

On 01/02/2015 10:56 AM, u-xsnf@aetey.se wrote:
> Looking at the source I wonder if this has to do with the reliance
> on the fastcall attribute, which pcc does not support.

Ah, well, that could well be.  Since fastcall is a standard Windows
calling convention, I sort of assume every decent compiler supports it.

In which case I'm going to suggest not building libffi with pcc.
An executable built with pcc should work with a library built with
gcc or clang.


r~

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

On Fri, Jan 02, 2015 at 02:20:22PM -0800, Richard Henderson wrote:
> On 01/02/2015 10:56 AM, u-xsnf@aetey.se wrote:
> > Looking at the source I wonder if this has to do with the reliance
> > on the fastcall attribute, which pcc does not support.
> 
> Ah, well, that could well be.  Since fastcall is a standard Windows
> calling convention, I sort of assume every decent compiler supports it.
> 
> In which case I'm going to suggest not building libffi with pcc.
> An executable built with pcc should work with a library built with
> gcc or clang.

Oh that's too bad.

As long as C code is concerned it would be a burden to build and
eventually maintain either gcc or clang for the sole purpose to compile
libffi.

Is it hard to make libffi implementation compatible with cdecl?
The calling conventions is exactly what the library is competent about. 

If the internal use of fastcall is desirable in certain cases, then
a compile time choice between fastcall and cdecl would alleviate the
problem and make one-compiler-shops with pcc (or even something else
like tcc?) happy.

Rune

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

On Sat, Jan 03, 2015 at 11:14:57AM +0100, u-xsnf@aetey.se wrote:
> If the internal use of fastcall is desirable in certain cases, then
> a compile time choice between fastcall and cdecl would alleviate the
> problem and make one-compiler-shops with pcc (or even something else
> like tcc?) happy.

Actually a bad example as tcc presumably supports fastcall but 
there are (and/or will be) other compilers anyway.

Rune

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[Richard Henderson]

On 01/03/2015 02:14 AM, u-xsnf@aetey.se wrote:
> Is it hard to make libffi implementation compatible with cdecl?
> The calling conventions is exactly what the library is competent about. 

I guess it wasn't that hard.

See https://github.com/atgreen/libffi/pull/165


r~

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

Hello Richard,

On Mon, Jan 05, 2015 at 01:34:35PM -0800, Richard Henderson wrote:
> On 01/03/2015 02:14 AM, u-xsnf@aetey.se wrote:
> > Is it hard to make libffi implementation compatible with cdecl?
> > The calling conventions is exactly what the library is competent about. 
> 
> I guess it wasn't that hard.
> 
> See https://github.com/atgreen/libffi/pull/165

Thanks a lot.

I wonder which version of pcc you were testing with, the comments
about the pcc behaviour indicate that the version used was either
too old or its install was broken (pcc used to ship redundant and
in practice harmful startup files, not actually being a part of the compiler).

To avoid misunderstanding let me list the positive points:

"Apparently, PCC doesn't support the fastcall calling convention.
Nor does it issue a warning or error for the attribute that it
does not understand."

pcc complains/warns about unsupported attributes:

$ cat a.c
__attribute__((fastcall)) int fastc(int i) {
  return i;
}
$ pcc -c a.c
a.c:1: warning: unsupported attribute `fastcall'
$ pcc --version
pcc 1.2.0.DEVEL 20141228 for i486-pc-linux-gnu

This is the current version of pcc, mostly the same as the newly released
pcc-1.1.0 (2014-12-10) with the reservation that I did not test the latter.

"Note that it also, apparently, doesn't support shared libraries. Test
cases linked against the pcc-built shared library segfault inside ld.so
before reaching main."

This is quite doubtless the result of using the broken startup files
regrettably used to be included in the pcc-libs package.
Shared libraries do work, in fact this very MUA is running shared
libraries compiled with pcc.

Now to the point:

I tested to apply
 https://github.com/rth7680/libffi/commit/3fa5d70cbb18b39a5e44f1c7984dedf73446bf6c

and it now passes the majority of the tests (2119) !

306 of the still failing tests try to test FASTCALL and THISCALL which
is not expected to pass. At least some of the remaining 210 ones quite
probably depend on a bug in pcc which was independently discovered this
week and fixed ("Float complex struct return were not dealt with as it
should in the PIC case.").

Moreover, at about the same time pcc seems to have got support for fastcall (!)

(Nevertheless I would argue that avoiding hardcoded reliance on fastcall
in libffi on x86 is good, placing less constraints on the compilers to
be used with it)

I am going to make the next round with the newest pcc and see how many
test failures remain.

Thanks again,
Rune

Re: libffi vs pcc (was: showstopper bug on x86 / libffi does not follow proper ABI on ia32)

[u-xsnf]

On Tue, Jan 06, 2015 at 11:21:02AM +0100, u-xsnf@aetey.se wrote:
> 306 of the still failing tests try to test FASTCALL and THISCALL which
> is not expected to pass. At least some of the remaining 210 ones quite
> probably depend on a bug in pcc which was independently discovered this
> week and fixed

> Moreover, at about the same time pcc seems to have got support for fastcall (!)

> I am going to make the next round with the newest pcc and see how many
> test failures remain.

The fastcall support in pcc is not yet complete.

I was able to compile libffi both with HAVE_FASTCALL=0 and HAVE_FASTCALL=1
and it looks mostly working in both cases (nice!).

Yet many tests are still failing, both related and unrelated to fastcall -
I'll take this up on the pcc list.

Rune

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[Richard Henderson]

On 01/06/2015 02:21 AM, u-xsnf@aetey.se wrote:
> I wonder which version of pcc you were testing with, the comments
> about the pcc behaviour indicate that the version used was either
> too old or its install was broken (pcc used to ship redundant and
> in practice harmful startup files, not actually being a part of the compiler).

I was using

pcc 1.1.0.DEVEL 20140817 for i686-pc-linux-gnu

as shipped with fedora 21.


r~

Re: showstopper bug on x86 (Re: libffi does not follow proper ABI on ia32)

[u-xsnf]

On Tue, Jan 06, 2015 at 08:20:43AM -0800, Richard Henderson wrote:
> On 01/06/2015 02:21 AM, u-xsnf@aetey.se wrote:
> > I wonder which version of pcc you were testing with, the comments
> > about the pcc behaviour indicate that the version used was either
> > too old or its install was broken (pcc used to ship redundant and
> > in practice harmful startup files, not actually being a part of the compiler).
> 
> I was using
> 
> pcc 1.1.0.DEVEL 20140817 for i686-pc-linux-gnu
> 
> as shipped with fedora 21.

The version does not look utterly old (even though much has happened
since then) but the Fedora packaging people were apparently misled by
the then badly documented data, which results in a quite broken package.

Now this is cleaned up and somewhat documented upstream, so future pcc
packages, for Fedora or otherwise, will be hopefully better.

Rune