[Bug target/102107] New: protocol register (r12) corrupted before a tail call

[Bug target/102107] New: protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

            Bug ID: 102107
           Summary: protocol register (r12) corrupted before a tail call
           Product: gcc
           Version: 11.2.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: pc at us dot ibm.com
  Target Milestone: ---

Created attachment 51367
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51367&action=edit
preprocessed source (large)

I've been working on an effort to improve Python performance, and hit an issue
when running with a libpython.so that was built with "-mcpu=power10". The
problem appears to be not correctly setting up (and preserving) register 12
before calling into a dynamically loaded, non-PCrel Python module in the form
of a shared object.

GDB shows the following instruction stream:
=> 0x7ffff7d25014 <do_mkvalue+1924>:    ld      r12,0(r9)
=> 0x7ffff7d25018 <do_mkvalue+1928>:    addi    r1,r1,112
r12            0x7fffe921af60      140737104686944
=> 0x7ffff7d2501c <do_mkvalue+1932>:    std     r10,0(r30)
=> 0x7ffff7d25020 <do_mkvalue+1936>:    ld      r3,8(r9)
=> 0x7ffff7d25024 <do_mkvalue+1940>:    ld      r9,0(r31)
=> 0x7ffff7d25028 <do_mkvalue+1944>:    ld      r29,-24(r1)
=> 0x7ffff7d2502c <do_mkvalue+1948>:    ld      r30,-16(r1)
=> 0x7ffff7d25030 <do_mkvalue+1952>:    mtctr   r12
=> 0x7ffff7d25034 <do_mkvalue+1956>:    lwz     r12,8(r1)
r12            0x4000              16384
=> 0x7ffff7d25038 <do_mkvalue+1960>:    addi    r9,r9,1
=> 0x7ffff7d2503c <do_mkvalue+1964>:    std     r9,0(r31)
=> 0x7ffff7d25040 <do_mkvalue+1968>:    ld      r31,-8(r1)
=> 0x7ffff7d25044 <do_mkvalue+1972>:    mtocrf  8,r12
=> 0x7ffff7d25048 <do_mkvalue+1976>:    bctr
=> 0x7fffe921af60 <return_none>:        addis   r2,r12,4  
=> 0x7fffe921af64 <return_none+4>:      addi    r2,r2,-12384
=> 0x7fffe921af68 <return_none+8>:      nop
=> 0x7fffe921af6c <return_none+12>:     ld      r3,-32728(r2)
Program received signal SIGSEGV, Segmentation fault.
0x00007fffe921af6c in _Py_INCREF (op=<optimized out>) at
../Python-3.9.6/Include/object.h:408
408         op->ob_refcnt++;

After setting r12 to the address of the caller (0x7ffff7d25014), the load at
0x7ffff7d25034 overwrites it with the CR save value just before the tail call
(bctr) at 0x7ffff7d25048, resulting in the badness when setting up and using
the TOC.

I suspect some sort of instruction scheduling issue?

I've attached a rather large pre-processed C file. It's complicated to reduce
because of functions calling other functions. I gave "creduce" a shot at it,
but it's challenging (for me, at least) to craft a script that knows what to
look for. I'll also attach the best I could get from creduce, but shield your
eyes before looking at it.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

Paul Clarke <pc at us dot ibm.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #51367|0                           |1
        is obsolete|                            |

--- Comment #1 from Paul Clarke <pc at us dot ibm.com> ---
Created attachment 51368
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51368&action=edit
preprocessed source (large)

Attach correct file. :-/

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #2 from Paul Clarke <pc at us dot ibm.com> ---
Created attachment 51369
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51369&action=edit
creduced version (tiny, but ugly)

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #3 from Paul Clarke <pc at us dot ibm.com> ---
Created attachment 51371
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51371&action=edit
preprocessed source (a bit smaller)

I was able to cut out a bit more than half of the original code. It gets more
difficult from here. If this is still "too big", I can hack at it some more.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #4 from Paul Clarke <pc at us dot ibm.com> ---
Created attachment 51372
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51372&action=edit
preprocessed source (yet a bit smaller)

I was able to remove one of the cases of the switch statement in the function
which exhibits the issue. Interestingly, removing any of the others hides the
issue.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #5 from Paul Clarke <pc at us dot ibm.com> ---
Fails with "-mcpu=power10" and "-O2" or "-O3".

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |segher at gcc dot gnu.org

--- Comment #6 from Segher Boessenkool <segher at gcc dot gnu.org> ---
I get (-mcpu=power10 -mabi=elfv2 -O2):

.L2:
        ld 9,0(30)       # 59   [c=8 l=4]  *movdi_internal64/1
        addi 1,1,112     # 124  [c=4 l=4]  *adddi3/1
        .cfi_def_cfa_offset 0
        li 3,0           # 63   [c=4 l=4]  *movdi_internal64/3
        ld 0,16(1)       # 125  [c=8 l=4]  *movdi_internal64/1
        ld 31,-8(1)      # 129  [c=8 l=4]  *movdi_internal64/1
        ld 12,0(9)       # 62   [c=8 l=4]  *movdi_internal64/1
        addi 9,9,8       # 60   [c=4 l=4]  *adddi3/1
        std 9,0(30)      # 61   [c=4 l=4]  *movdi_internal64/0
        ld 30,-16(1)     # 128  [c=8 l=4]  *movdi_internal64/1
        mtlr 0           # 127  [c=4 l=4]  *movdi_internal64/23
        .cfi_restore 65
        mtctr 12                 # 103  [c=4 l=4]  *movdi_internal64/23
        lwz 12,8(1)      # 126  [c=8 l=4]  *movsi_internal1/2
        mtcrf 8,12       # 130  [c=4 l=4]  *mtcrfsi
        .cfi_restore 72
        .cfi_restore 31
        .cfi_restore 30
        bctr             # 66   [c=4 l=4]  *sibcall_value_aixdi/1

This looks just fine.  What is wrong with your build?

[Bug target/102107] protocol register (r12) corrupted before a tail call

[wschmidt at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

Bill Schmidt <wschmidt at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |wschmidt at gcc dot gnu.org

--- Comment #7 from Bill Schmidt <wschmidt at gcc dot gnu.org> ---
Paul, what does gcc -v show?

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #8 from Paul Clarke <pc at us dot ibm.com> ---
$ /opt/at15.0/bin/gcc --version
gcc (GCC) 11.2.1 20210802 (Advance-Toolchain 15.0-0) [ebcfb7a665c2]

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #9 from Segher Boessenkool <segher at gcc dot gnu.org> ---
So that is something older from the GCC 11 branch, aha.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #10 from Segher Boessenkool <segher at gcc dot gnu.org> ---
(In reply to Segher Boessenkool from comment #9)
> So that is something older from the GCC 11 branch, aha.

And I cannot reproduce it with anything that has been on the GCC 11 branch,
either.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #11 from Paul Clarke <pc at us dot ibm.com> ---
This does produce the issue for me:
--
$ git checkout remotes/vendors/ibm/gcc-11-branch gcc-AT
$ mkdir gcc-AT-build
$ cd gcc-AT-build
$ ../gcc-AT/configure --enable-languages=c,c++ --disable-libada
--disable-libsanitizer --disable-libssp --disable-libgomp --disable-libvtv
--disable-nls --prefix=/home/pc/gcc-AT-install
$ make
$ make install
$ ~/gcc-AT-install/bin/gcc -S -O3 -mcpu=power10 -fverbose-asm r12test2.c
$ grep --before-context=15 bctr r12test2.s
        mtctr 12                 # func, func
 # r12test2.c:3030: }
        lwz 12,8(1)      #,
 # r12test2.c:3013:             ++*p_format;
        addi 9,9,1       # tmp251, *p_format_31(D),
        std 9,0(31)      # *p_format_31(D), tmp251
 # r12test2.c:3030: }
        ld 31,-8(1)      #,
        mtcrf 8,12       #,
        .cfi_restore 72
        .cfi_restore 31
        .cfi_restore 30
        .cfi_restore 28
        .cfi_restore 27
 # r12test2.c:3014:             return (*func)();
        bctr             # func
--

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-08-31
             Status|UNCONFIRMED                 |NEW

--- Comment #12 from Segher Boessenkool <segher at gcc dot gnu.org> ---
So this actually happens during pro_and_epilogue already.  Confirmed.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #13 from Segher Boessenkool <segher at gcc dot gnu.org> ---
  /* If we need to save CR, put it into r12 or r11.  Choose r12 except when
     r12 will be needed by out-of-line gpr save.  */
  cr_save_regno = ((DEFAULT_ABI == ABI_AIX || DEFAULT_ABI == ABI_ELFv2)
                   && !(strategy & (SAVE_INLINE_GPRS
                                    | SAVE_NOINLINE_GPRS_SAVES_LR))
                   ? 11 : 12);

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |segher at gcc dot gnu.org

--- Comment #14 from Segher Boessenkool <segher at gcc dot gnu.org> ---
I have a patch.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #15 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Segher Boessenkool <segher@gcc.gnu.org>:

https://gcc.gnu.org/g:2484f7a4b0f52e6ed04754be336f1fa6fde47f6b

commit r12-3353-g2484f7a4b0f52e6ed04754be336f1fa6fde47f6b
Author: Segher Boessenkool <segher@kernel.crashing.org>
Date:   Thu Sep 2 16:38:24 2021 +0000

    rs6000: Don't use r12 for CR save on ELFv2 (PR102107)

    CR is saved and/or restored on some paths where GPR12 is already live
    since it has a meaning in the calling convention in the ELFv2 ABI.

    It is not completely clear to me that we can always use r11 here, but
    it does seem save, there is checking code (to detect conflicts here),
    and it is stage 1.  So here goes.

    2021-09-03  Segher Boessenkool <segher@kernel.crashing.org>

            PR target/102107
            * config/rs6000/rs6000-logue.c (rs6000_emit_prologue): On ELFv2 use
r11
            instead of r12 for CR save, in all cases.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #16 from Segher Boessenkool <segher at gcc dot gnu.org> ---
(Hopefully) fixed on trunk, but it needs backports.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #17 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Segher Boessenkool <segher@gcc.gnu.org>:

https://gcc.gnu.org/g:86e6268cff328e27ee6f90e2afc35b6f437a25cd

commit r12-3418-g86e6268cff328e27ee6f90e2afc35b6f437a25cd
Author: Segher Boessenkool <segher@kernel.crashing.org>
Date:   Wed Sep 8 13:10:30 2021 +0000

    rs6000: Fix ELFv2 r12 use in epilogue

    We cannot use r12 here, it is already in use as the GEP (for sibling
    calls).

    2021-09-08  Segher Boessenkool  <segher@kernel.crashing.org>
            PR target/102107
            * config/rs6000/rs6000-logue.c (rs6000_emit_epilogue): For ELFv2
use
            r11 instead of r12 for restoring CR.

[Bug target/102107] protocol register (r12) corrupted before a tail call

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #18 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Segher Boessenkool
<segher@gcc.gnu.org>:

https://gcc.gnu.org/g:5210f05c6eba12561914cb0fdf7aec149b28f179

commit r11-9021-g5210f05c6eba12561914cb0fdf7aec149b28f179
Author: Segher Boessenkool <segher@kernel.crashing.org>
Date:   Thu Sep 2 16:38:24 2021 +0000

    rs6000: Don't use r12 for CR save on ELFv2 (PR102107)

    CR is saved and/or restored on some paths where GPR12 is already live
    since it has a meaning in the calling convention in the ELFv2 ABI.

    It is not completely clear to me that we can always use r11 here, but
    it does seem save, there is checking code (to detect conflicts here),
    and it is stage 1.  So here goes.

    2021-09-03  Segher Boessenkool <segher@kernel.crashing.org>

            PR target/102107
            * config/rs6000/rs6000-logue.c (rs6000_emit_prologue): On ELFv2 use
r11
            instead of r12 for CR save, in all cases.

    (cherry picked from commit 2484f7a4b0f52e6ed04754be336f1fa6fde47f6b)

[Bug target/102107] protocol register (r12) corrupted before a tail call

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #19 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Segher Boessenkool
<segher@gcc.gnu.org>:

https://gcc.gnu.org/g:da0a5628d47ee59320dac9787f3232b0911fbc14

commit r11-9022-gda0a5628d47ee59320dac9787f3232b0911fbc14
Author: Segher Boessenkool <segher@kernel.crashing.org>
Date:   Wed Sep 8 13:10:30 2021 +0000

    rs6000: Fix ELFv2 r12 use in epilogue

    We cannot use r12 here, it is already in use as the GEP (for sibling
    calls).

    2021-09-08  Segher Boessenkool  <segher@kernel.crashing.org>
            PR target/102107
            * config/rs6000/rs6000-logue.c (rs6000_emit_epilogue): For ELFv2
use
            r11 instead of r12 for restoring CR.

    (cherry picked from commit 86e6268cff328e27ee6f90e2afc35b6f437a25cd)

[Bug target/102107] protocol register (r12) corrupted before a tail call

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #20 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Segher Boessenkool
<segher@gcc.gnu.org>:

https://gcc.gnu.org/g:5a989ccd3cbc31cb0e5fecd9423fc1577f3694b0

commit r10-10143-g5a989ccd3cbc31cb0e5fecd9423fc1577f3694b0
Author: Segher Boessenkool <segher@kernel.crashing.org>
Date:   Thu Sep 2 16:38:24 2021 +0000

    rs6000: Don't use r12 for CR save on ELFv2 (PR102107)

    CR is saved and/or restored on some paths where GPR12 is already live
    since it has a meaning in the calling convention in the ELFv2 ABI.

    It is not completely clear to me that we can always use r11 here, but
    it does seem save, there is checking code (to detect conflicts here),
    and it is stage 1.  So here goes.

    2021-09-03  Segher Boessenkool <segher@kernel.crashing.org>

            PR target/102107
            * config/rs6000/rs6000-logue.c (rs6000_emit_prologue): On ELFv2 use
r11
            instead of r12 for CR save, in all cases.

    (cherry picked from commit 2484f7a4b0f52e6ed04754be336f1fa6fde47f6b)

[Bug target/102107] protocol register (r12) corrupted before a tail call

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #21 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Segher Boessenkool
<segher@gcc.gnu.org>:

https://gcc.gnu.org/g:14b7e8ace8236bd341a182dc3ef044fe98de5525

commit r10-10144-g14b7e8ace8236bd341a182dc3ef044fe98de5525
Author: Segher Boessenkool <segher@kernel.crashing.org>
Date:   Wed Sep 8 13:10:30 2021 +0000

    rs6000: Fix ELFv2 r12 use in epilogue

    We cannot use r12 here, it is already in use as the GEP (for sibling
    calls).

    2021-09-08  Segher Boessenkool  <segher@kernel.crashing.org>
            PR target/102107
            * config/rs6000/rs6000-logue.c (rs6000_emit_epilogue): For ELFv2
use
            r11 instead of r12 for restoring CR.

    (cherry picked from commit 86e6268cff328e27ee6f90e2afc35b6f437a25cd)

[Bug target/102107] protocol register (r12) corrupted before a tail call

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

--- Comment #22 from Segher Boessenkool <segher at gcc dot gnu.org> ---
Backported to 11 and 10.  Not doing GCC 9; the problem is older than that,
and the backport wouldn't be completely trivial.  Paul, please close if
everything is okay now?

[Bug target/102107] protocol register (r12) corrupted before a tail call

[pc at us dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102107

Paul Clarke <pc at us dot ibm.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #23 from Paul Clarke <pc at us dot ibm.com> ---
Tested (trunk), works for me.