Disappeared flag: -maes on -march=ivybridge, present in -march=native

Disappeared flag: -maes on -march=ivybridge, present in -march=native

[Kevin Weidemann]

Hello everyone,

I have recently randomly discovered the fact, that building with 
`-march=ivybridge` does not necessarily produce the same output as 
`-march=native` on an Intel Core i7 3770K (Ivy Bridge).

In particular, `-march=native` sets `-maes` whilst `-march=ivybridge` 
does not.

After carefully considering the option that I'm in fact turning crazy 
(e.g. I triple-checked that `-march=native` really does set 
`-march=ivybridge` and `-mtune=ivybridge`), it seems like while this is 
the case on the GCC version that I'm actively using (9.1.0), it is not 
on a different machine (6.3.0 20170516 (Debian 6.3.0-18+deb9u1)).

Consider this method that yields a `test.s` file result:

$ touch test.cc
$ gcc -fverbose-asm -march=ivybridge test.cc -S
$ sed -i 1,/options\ enabled/d test.s

Given the 6.3.0 compiler, test.s contains `-maes`.
The 9.1.0 compiler's test.s does not contain `-maes`.

In fact, when you use the 9.1.0 compiler on the CPU mentioned before to 
create `test1.s` with `-march=native` and `test2.s` with 
`-march=ivybridge` as before, this is the only difference:

$ diff test1.s test.s
21c21
< # -m128bit-long-double -m64 -m80387 -maes -malign-stringops -mavx
---
 > # -m128bit-long-double -m64 -m80387 -malign-stringops -mavx


I have run a git bisect starting with
4b5e15daff8b54440e3fda451c318ad31e532fab as the last known good commit and
12b43fabe5fc5aa7d6217a6736409e180d13db6e as the first known bad commit [1].

If I'm understanding the commit correctly, the change that causes this 
is the removal of `PTA_AES` in

const wide_int_bitmask PTA_WESTMERE = PTA_NEHALEM | PTA_AES | PTA_PCLMUL;

as PTA_IVYBRIDGE contains PTA_SANDYBRIDGE which in turn contains 
PTA_WESTMERE.


Is this an oversight or mistake?


Best regards,

Kevin Weidemann


References:

[1] git bisect result:

71c8e4e2f720bc7155ba2da7c0ee9136a9ab3283 is the first bad commit
commit 71c8e4e2f720bc7155ba2da7c0ee9136a9ab3283
Author: hjl <hjl@138bc75d-0d04-0410-961f-82ee72b054a4>
Date:   Fri Feb 22 12:49:21 2019 +0000

     x86: (Reapply) Move AESNI generation to Skylake and Goldmont

     This is a repeat of commit r263989, which commit r264052 accidentally
     reverted.

     2019-02-22  Thiago Macieira  <thiago.macieira@intel.com>

             PR target/89444
             * config/i386/i386.h (PTA_WESTMERE): Remove PTA_AES.
             (PTA_SKYLAKE): Add PTA_AES.
             (PTA_GOLDMONT): Likewise.


     git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@269114 
138bc75d-0d04-0410-961f-82ee72b054a4

:040000 040000 2c24ae2c6c6074acd52215a3340240989b3c1f6a 
479b008acd81e383293abb23a69c9fa5da4afdc6 M      gcc
bisect run success

Re: Disappeared flag: -maes on -march=ivybridge, present in -march=native

[Marc Glisse]

On Mon, 29 Jul 2019, Kevin Weidemann wrote:

> I have recently randomly discovered the fact, that building with 
> `-march=ivybridge` does not necessarily produce the same output as 
> `-march=native` on an Intel Core i7 3770K (Ivy Bridge).

Nothing so surprising there. Not all Ivy Bridge processors are equivalent, 
and -march=ivybridge has to conservatively target those with less 
features.

> 71c8e4e2f720bc7155ba2da7c0ee9136a9ab3283 is the first bad commit
> commit 71c8e4e2f720bc7155ba2da7c0ee9136a9ab3283
> Author: hjl <hjl@138bc75d-0d04-0410-961f-82ee72b054a4>
> Date:   Fri Feb 22 12:49:21 2019 +0000
>
>     x86: (Reapply) Move AESNI generation to Skylake and Goldmont
>
>     This is a repeat of commit r263989, which commit r264052 accidentally
>     reverted.
>
>     2019-02-22  Thiago Macieira  <thiago.macieira@intel.com>
>
>             PR target/89444
>             * config/i386/i386.h (PTA_WESTMERE): Remove PTA_AES.
>             (PTA_SKYLAKE): Add PTA_AES.
>             (PTA_GOLDMONT): Likewise.

As you can see, this is very much on purpose. See 
https://gcc.gnu.org/ml/gcc-patches/2018-08/msg01940.html for the 
explanation that came with the patch.

-- 
Marc Glisse

Re: Disappeared flag: -maes on -march=ivybridge, present in -march=native

[Thiago Macieira]

On Sunday, 28 July 2019 16:16:21 PDT Kevin Weidemann wrote:
> Hello everyone,
> 
> I have recently randomly discovered the fact, that building with
> `-march=ivybridge` does not necessarily produce the same output as
> `-march=native` on an Intel Core i7 3770K (Ivy Bridge).
> 
> In particular, `-march=native` sets `-maes` whilst `-march=ivybridge`
> does not.

That is the expected behaviour today. -march=native gives you what your CPU 
has, even if it's different from what a baseline CPU architecture would 
provide. As an example, if you ran GCC inside a virtual machine that somehow 
turned some CPU features off, you'd get a fully working build for your CPU 
using all CPU features you can use.

That's different from LLVM, which attempts to find which CPU most closely 
matches yours. In that VM scenario, it would either disable some features you 
could use or enable ones you can't.

> After carefully considering the option that I'm in fact turning crazy
> (e.g. I triple-checked that `-march=native` really does set
> `-march=ivybridge` and `-mtune=ivybridge`), it seems like while this is
> the case on the GCC version that I'm actively using (9.1.0), it is not
> on a different machine (6.3.0 20170516 (Debian 6.3.0-18+deb9u1)).

True. Older versions of GCC turned AES on for -march=westmere and up, but not 
anymore. The commit you found changed that.

> Is this an oversight or mistake?

Neither, it's intentional.

Not all Westmere, Sandybridge, Ivybridge, Haswell or Broadwell machines have 
AES.[*] Therefore, if you want to target *all* Ivybridges, you should not be 
getting AES turned on by default. You should be doing runtime checking.

This specifically affected us in the Clear Linux distribution, since we build 
the entire distro with -march=westmere as a baseline. Up until earlier this 
year, with the GCC 9 upgrade, we were turning users away who had machines 
without AES, since our installer complained (still does, I think) because GCC 
had set the __AES__ macro, even though I found absolutely no software in the 
distro using AES without CPUID checks.

[*] I have yet to see a Skylake or up SKU listed in https://ark.intel.com 
without AES, but it's possible one will show up in the future.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products

Re: Disappeared flag: -maes on -march=ivybridge, present in -march=native

[Kevin Weidemann]

On Monday, 29 July 2019, Marc Glisse wrote:
>> This is a repeat of commit r263989, which commit r264052 
>> accidentally reverted.
> (...) As you can see, this is very much on purpose. See 
> https://gcc.gnu.org/ml/gcc-patches/2018-08/msg01940.html for the 
> explanation that came with the patch.

D'oh! I shouldn't have investigated this in the middle of the night,
apparently. I grepped the history for `r263989`, but the commit doesn't
contain that string, but instead
> git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@263989
(so without the `r` prefix). I didn't find the commit so I discarded the
idea of trying to find it way too quickly. Especially given the fact, 
that the repeated commit was found by the bisect and not the original 
one, I randomly decided to stop looking for it. Should've investigated 
this further after starting the day clear-headed!

On Monday, 29 July 2019 07:47 CEST Thiago Macieira wrote:
> Older versions of GCC turned AES on for -march=westmere and up, but 
> not anymore. The commit you found changed that.
Ok, gotcha! I didn't know that not all machines with those platforms
have AES.

> if you ran GCC [...] you'd get a fully working build for your CPU 
> using all CPU features you can use.
> 
> That's different from LLVM, which attempts to find which CPU most 
> closely matches yours. In that VM scenario, it would either disable 
> some features you could use or enable ones you can't.

Thanks for the interesting pointers, and both of your answers!

--
Kevin Weidemann