x86 code generation question

x86 code generation question

[Manuel Lauss]

I'm trying to analyze a segfault in 32bit glibc.  The program crashes
in glibc/dlfcn/dlvsym.c::dlvsym_implementation(), namely
at the following assignment (line 50 of head source):
 args.handle = handle;
with args being
struct dlvsym_args
{
 /* The arguments to dlvsym_doit.  */
 void *handle;
 const char *name;
 const char *version;
 void *who;

 /* The return values of dlvsym_doit.  */
 void *sym;
};

The crash is at
Dump of assembler code for function ___dlvsym:
[...]
  0xf7d4458b <+75>:    vpunpcklqdq %xmm1,%xmm0,%xmm0
=> 0xf7d4458f <+79>:    vmovdqa %xmm0,0xc(%esp)

with esp = 0xffc30310

Now the descriptions of "vmovdqa" I could find say that it generates a
#GP when the destination is not 16-byte aligned. Does this alignment
include the displacement, like above?  If yes, is this then a code
generation bug in gcc?

Thanks,
     Manuel

Re: x86 code generation question

[Florian Weimer]

* Manuel Lauss via Gcc-help:

> Now the descriptions of "vmovdqa" I could find say that it generates a
> #GP when the destination is not 16-byte aligned. Does this alignment
> include the displacement, like above?  If yes, is this then a code
> generation bug in gcc?

If your build uses vector instructions, you need to build glibc with
-mstackrealign for compatibility with legacy i386 applications.

This is still fallout from the i386 ABI break when SSE2 support with an
increase in stack alignment was introduced some twenty years ago.

Thanks,
Florian

Re: x86 code generation question

[Manuel Lauss]

Servus Florian,

On Mon, Dec 13, 2021 at 2:04 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Manuel Lauss via Gcc-help:
>
> > Now the descriptions of "vmovdqa" I could find say that it generates a
> > #GP when the destination is not 16-byte aligned. Does this alignment
> > include the displacement, like above?  If yes, is this then a code
> > generation bug in gcc?
>
> If your build uses vector instructions, you need to build glibc with
> -mstackrealign for compatibility with legacy i386 applications.

-mstackrealign breaks with an error about invalid use of "bp".  I removed
the vectorization switches and it works now.


> This is still fallout from the i386 ABI break when SSE2 support with an
> increase in stack alignment was introduced some twenty years ago.

The application is part of the Steam client, so it's not that old, but
it is still 32bit x86.

Thanks for your help!
     Manuel

Re: x86 code generation question

[Manuel Lauss]

On Mon, Dec 13, 2021 at 2:04 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Manuel Lauss via Gcc-help:
>
> > Now the descriptions of "vmovdqa" I could find say that it generates a
> > #GP when the destination is not 16-byte aligned. Does this alignment
> > include the displacement, like above?  If yes, is this then a code
> > generation bug in gcc?
>
> If your build uses vector instructions, you need to build glibc with
> -mstackrealign for compatibility with legacy i386 applications.

I read that gcc-12 enables "-ftree-vectorize" with -O2 by default, and
indeed a glibc snapshot built with gcc-12 and "-O2 -march=haswell" suffers
now from the same problem.  Should "-mstackrealign" be added to 32bit x86 build
unconditionally?

Manuel

Re: x86 code generation question

[Florian Weimer]

* Manuel Lauss:

> On Mon, Dec 13, 2021 at 2:04 PM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * Manuel Lauss via Gcc-help:
>>
>> > Now the descriptions of "vmovdqa" I could find say that it generates a
>> > #GP when the destination is not 16-byte aligned. Does this alignment
>> > include the displacement, like above?  If yes, is this then a code
>> > generation bug in gcc?
>>
>> If your build uses vector instructions, you need to build glibc with
>> -mstackrealign for compatibility with legacy i386 applications.
>
> I read that gcc-12 enables "-ftree-vectorize" with -O2 by default, and
> indeed a glibc snapshot built with gcc-12 and "-O2 -march=haswell"
> suffers now from the same problem.  Should "-mstackrealign" be added
> to 32bit x86 build unconditionally?

I think it's just something you need to know when building glibc for
legacy applications.  Most users will likely prefer full optimization
for their builds (otherwise they wouldn't build with -march=haswell).

Thanks,
Florian

Re: x86 code generation question

[Manuel Lauss]

On Tue, Dec 14, 2021 at 12:21 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Manuel Lauss:
>
> > On Mon, Dec 13, 2021 at 2:04 PM Florian Weimer <fweimer@redhat.com> wrote:
> >>
> >> * Manuel Lauss via Gcc-help:
> >>
> >> > Now the descriptions of "vmovdqa" I could find say that it generates a
> >> > #GP when the destination is not 16-byte aligned. Does this alignment
> >> > include the displacement, like above?  If yes, is this then a code
> >> > generation bug in gcc?
> >>
> >> If your build uses vector instructions, you need to build glibc with
> >> -mstackrealign for compatibility with legacy i386 applications.
> >
> > I read that gcc-12 enables "-ftree-vectorize" with -O2 by default, and
> > indeed a glibc snapshot built with gcc-12 and "-O2 -march=haswell"
> > suffers now from the same problem.  Should "-mstackrealign" be added
> > to 32bit x86 build unconditionally?
>
> I think it's just something you need to know when building glibc for
> legacy applications.  Most users will likely prefer full optimization
> for their builds (otherwise they wouldn't build with -march=haswell).

Is there an explanation of how this whole stack magic works?  I wrote
a small test for dlvsym() and its stack pointer is precisely
aligned so that the vmovdqa with displacement 0xc points to a 16-byte
aligned address.  However I don't get how the failing program gets its
misaligned stack. Does it have to be built with special compiler flags or
linker scripts?

Thanks!
      Manuel

Re: x86 code generation question

[Florian Weimer]

* Manuel Lauss:

> Is there an explanation of how this whole stack magic works?  I wrote
> a small test for dlvsym() and its stack pointer is precisely
> aligned so that the vmovdqa with displacement 0xc points to a 16-byte
> aligned address.  However I don't get how the failing program gets its
> misaligned stack. Does it have to be built with special compiler flags or
> linker scripts?

There are compiler flags to mess up stack alignment, yes.  But in
general this is caused by old binaries, built with compilers that still
used the old stack alignment logic.  Somewhat rarer is manually written
new assembler code that is just broken.

Thanks,
Florian

Re: x86 code generation question

[Stefan Ring]

On Tue, Dec 14, 2021 at 12:40 PM Manuel Lauss via Gcc-help
<gcc-help@gcc.gnu.org> wrote:
> Is there an explanation of how this whole stack magic works?  I wrote
> a small test for dlvsym() and its stack pointer is precisely
> aligned so that the vmovdqa with displacement 0xc points to a 16-byte
> aligned address.  However I don't get how the failing program gets its
> misaligned stack. Does it have to be built with special compiler flags or
> linker scripts?

No magic involved. The rule is that you are only allowed to invoke C
functions if $esp is divisible by 16 before execution of the call
instruction. $esp is forced to an aligned address at the beginning of
main, and the compiler will not generate functions that create odd
stack frames, so it will stay aligned at all times without further
runtime penalties. The only price to pay is a minuscule increase of 8
bytes in stack frame size for some functions compared to the previous
rules (I think 4-byte alignment was never allowed).