[Bug target/109007] New: building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[Bug target/109007] New: building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

            Bug ID: 109007
           Summary: building for POWER8 leaks into POWER9 ISA with g++
                    11.3 (cross-compiler on x86_64 host)
           Product: gcc
           Version: 11.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: bugreporter66 at gmail dot com
  Target Milestone: ---

Created attachment 54577
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=54577&action=edit
archive with the source code and screenshot to reproduce the issue

The code sample is attached with the source files and a screenshot. It's a
regression from g++ 9.x series, where it worked properly.

Building for POWER8 (-mcpu=power8) creates binary that has illegal instruction
for POWER8 ISA, as confirmed by running the code with QEMU (see screenshot).

The command line for building is the following (change to test directory
first):
make -f core_make_p64.mk build_le -j4

The command line for emulation with QEMU (spits illegal instruction):
qemu-ppc64le -cpu POWER8 core_test.p64f32Lp8

The test case is taken from https://github.com/VectorChief/QuadRay-engine,
where that test fails (master) for POWER8 target on Ubuntu MATE 22.04.2

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
It would be helpful if you could tell which instruction is considered illegal
by QEMU, found in which *.o file it is and provided preprocessed source + gcc
command line how that single compilation unit has been compiled.
See https://gcc.gnu.org/bugs.html on details for what we want and don't want.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2023-03-03

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
The easiest way to figure out what the instruction is failing is run qemu with
gdb stub turned on and connect to it with gdb and then continue and gdb should
stop on the illegal instruction.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #3 from bugreporter66 at gmail dot com ---
Thanks, I will try to find out and be more specific where exactly it leaks.
It was my first attempt at reducing the code that would fit into 1MB
attachment.

Checked g++ 10.4 today, it works as it should.
11.3 and 12.1 were tested to show the issue so far.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #4 from bugreporter66 at gmail dot com ---
I did the remote debugging as suggested by Andrew Pinski and it seems to fail
somewhere in _GLOBAL__sub_i_eh_alloc.cc

After connecting to qemu from gdb I issued step command and the program was
terminated with SIGILL (illegal instruction) single stepping until exit from
that function (which has no line number information).

I am attaching the screenshot.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #5 from bugreporter66 at gmail dot com ---
Created attachment 54582
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=54582&action=edit
Screenshot with QEMU+GDB

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
That doesn't tell anything, you need to stepi through the function to reach the
problematic instruction and disas $pc-32,$pc+32 around it or so.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #7 from bugreporter66 at gmail dot com ---
After disassembling the instruction in question is stxv at 100034b0:

0000000010003470 <_GLOBAL__sub_I_eh_alloc.cc>:
    10003470:   15 10 40 3c     lis     r2,4117
    10003474:   00 7c 42 38     addi    r2,r2,31744
    10003478:   a6 02 08 7c     mflr    r0
    1000347c:   f0 ff c1 fb     std     r30,-16(r1)
    10003480:   f8 ff e1 fb     std     r31,-8(r1)
    10003484:   10 00 01 f8     std     r0,16(r1)
    10003488:   d1 ff 21 f8     stdu    r1,-48(r1)
    1000348c:   01 00 e0 3f     lis     r31,1
    10003490:   d0 02 00 f0     xxspltib vs0,0
    10003494:   00 00 c0 3b     li      r30,0
    10003498:   00 00 00 60     nop
    1000349c:   00 1c ff 63     ori     r31,r31,7168
    100034a0:   00 00 00 60     nop
    100034a4:   f0 c7 22 39     addi    r9,r2,-14352
    100034a8:   78 fb e3 7f     mr      r3,r31
    100034ac:   28 c8 e2 fb     std     r31,-14296(r2)
    100034b0:   05 00 09 f4     stxv    vs0,0(r9)
    100034b4:   15 00 09 f4     stxv    vs0,16(r9)
    100034b8:   00 00 00 60     nop
    100034bc:   10 c8 c2 fb     std     r30,-14320(r2)
    100034c0:   29 6a 07 48     bl      10079ee8 <__libc_malloc+0x8>

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #8 from bugreporter66 at gmail dot com ---
Created attachment 54583
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=54583&action=edit
Screenshot with GDB+disassembly

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |meissner at gcc dot gnu.org,
                   |                            |segher at gcc dot gnu.org

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I believe that instruction is power9 only, on the other side
_GLOBAL__sub_I_eh_alloc.cc doesn't come from the code you were compiling, but
from libstdc++.a(eh_alloc.o).
So, it depends on how the gcc you are using has been compiled.
If it is from an Ubuntu package, you need to report it to Ubuntu.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #10 from bugreporter66 at gmail dot com ---
Yes, it seems so. They've switched to POWER9 by default in Ubuntu 22.04, so it
means that gcc itself (along with standard libraries) was compiled for POWER9
as well. It used to be POWER8 in the previous releases.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|WAITING                     |RESOLVED

--- Comment #11 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Then you can't expect it to work on power8...

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #12 from bugreporter66 at gmail dot com ---
Yep, thanks.

At least I've learned how to debug with QEMU and GDB and I think building
without -static should still produce code compatible with POWER8 for the actual
target, but not for QEMU linux-user emulation (which obviously requires it).

This link I found was helpful in getting GDB and QEMU working together:
https://mariokartwii.com/showthread.php?tid=1998&pid=9867#pid9867

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #13 from Segher Boessenkool <segher at gcc dot gnu.org> ---
(In reply to bugreporter66 from comment #10)
> Yes, it seems so. They've switched to POWER9 by default in Ubuntu 22.04, so
> it means that gcc itself (along with standard libraries) was compiled for
> POWER9 as well. It used to be POWER8 in the previous releases.

Power9 *only*, actually.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #14 from bugreporter66 at gmail dot com ---
I should be able to workaround that by emulating all LE targets on POWER9, with
a comment that building for POWER8 natively on target should work too.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #15 from Segher Boessenkool <segher at gcc dot gnu.org> ---
(In reply to bugreporter66 from comment #14)
> I should be able to workaround that by emulating all LE targets on POWER9,
> with a comment that building for POWER8 natively on target should work too.

If you want to default to Power9 but still support Power8 (on builds that use
-mcpu=power8), you need to set up appropriate multilibs.  If you want that,
please do a feature request for that?  As a new PR, not hidden inside this
one please :-)

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #16 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
(In reply to Segher Boessenkool from comment #15)
> (In reply to bugreporter66 from comment #14)
> > I should be able to workaround that by emulating all LE targets on POWER9,
> > with a comment that building for POWER8 natively on target should work too.
> 
> If you want to default to Power9 but still support Power8 (on builds that use
> -mcpu=power8), you need to set up appropriate multilibs.  If you want that,
> please do a feature request for that?  As a new PR, not hidden inside this
> one please :-)

Or just make sure the libraries are still built with -mcpu=power8 even when the
compiler defaults to something else.
That said, neither multilibs nor just making sure libraries are built with
-mcpu=power8 can help with libraries outside of gcc (unless they are built as
multilibs or with the extra flags).
I'd say though that if you want to use distro packages of a distro that
switched to power9, then just live with it and don't try to run it on power8 or
power8 emulation,
or switch to a different distro which still supports power8 (many still do), or
build gcc and all the needed libraries yourself.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #17 from Segher Boessenkool <segher at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #16)
> Or just make sure the libraries are still built with -mcpu=power8 even when
> the compiler defaults to something else.

That doesn't scale.

> That said, neither multilibs nor just making sure libraries are built with
> -mcpu=power8 can help with libraries outside of gcc (unless they are built
> as multilibs or with the extra flags).

Sure.  But this bug is about that a compiler built --with-cpu=X can not build
anything that can run on any older CPU (or does not have some feature that X
has, more generally).  This is a problem that we already do have a solution
for, but something we have disabled in the powerpc64le-linux subtarget
unfortunately.

> or build gcc and all the needed libraries yourself.

Yup.  But it should not be necessary to build a different GCC.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #18 from bugreporter66 at gmail dot com ---
I noticed that building on newer Linux distros often produces a binary that
then cannot run on older ones even with the same CPU, it requires newer
libraries at runtime (tried that with x86_64). That alone limits
backward-compatibility among Linux releases, but forward-compatibility does
work (I always build release binaries on the oldest distro available, then it
works everywhere).

So, I might be willing to let go of the idea of building for the older targets
on the up-to-date LTS release, and as a workaround I should always emulate on
the latest available CPU to future-proof for the next transitions.

[Bug target/109007] building for POWER8 leaks into POWER9 ISA with g++ 11.3 (cross-compiler on x86_64 host)

[bugreporter66 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109007

--- Comment #19 from bugreporter66 at gmail dot com ---
On the side note, my POWER8 machine can only run Ubuntu 16.04 LTS and Ubuntu
18.04 LTS, later releases simply won't boot. So it's getting outdated anyway.