[Bug target/110899] New: RFE: Attributes preserve_most and preserve_all

[Bug target/110899] New: RFE: Attributes preserve_most and preserve_all

[elver at google dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

            Bug ID: 110899
           Summary: RFE: Attributes preserve_most and preserve_all
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: elver at google dot com
  Target Milestone: ---

Clang/LLVM implements the function attributes "preserve_most" and
"preserve_all":

[1] preserve_most: "On X86-64 and AArch64 targets, this attribute changes the
calling convention of a function. The preserve_most calling convention attempts
to make the code in the caller as unintrusive as possible. This convention
behaves identically to the C calling convention on how arguments and return
values are passed, but it uses a different set of caller/callee-saved
registers. This alleviates the burden of saving and recovering a large register
set before and after the call in the caller. If the arguments are passed in
callee-saved registers, then they will be preserved by the callee across the
call. This doesn’t apply for values returned in callee-saved registers.

- On X86-64 the callee preserves all general purpose registers, except for R11.
R11 can be used as a scratch register. Floating-point registers (XMMs/YMMs) are
not preserved and need to be saved by the caller.

- On AArch64 the callee preserve all general purpose registers, except X0-X8
and X16-X18."

[2] preserve_all: "On X86-64 and AArch64 targets, this attribute changes the
calling convention of a function. The preserve_all calling convention attempts
to make the code in the caller even less intrusive than the preserve_most
calling convention. This calling convention also behaves identical to the C
calling convention on how arguments and return values are passed, but it uses a
different set of caller/callee-saved registers. This removes the burden of
saving and recovering a large register set before and after the call in the
caller. If the arguments are passed in callee-saved registers, then they will
be preserved by the callee across the call. This doesn’t apply for values
returned in callee-saved registers.

- On X86-64 the callee preserves all general purpose registers, except for R11.
R11 can be used as a scratch register. Furthermore it also preserves all
floating-point registers (XMMs/YMMs).

- On AArch64 the callee preserve all general purpose registers, except X0-X8
and X16-X18. Furthermore it also preserves lower 128 bits of V8-V31 SIMD -
floating point registers."

[1] https://clang.llvm.org/docs/AttributeReference.html#preserve-most
[2] https://clang.llvm.org/docs/AttributeReference.html#preserve-all


These attributes, esp. preserve_most, provides a convenient way to optimize the
generated code for calls to rarely taken slow paths, such as error-reporting
functions. Recently, we're looking to make use of this in the Linux kernel [3],
with potentially additional usecases being discussed.

[3] https://lkml.kernel.org/r/20230804090621.400-1-elver@google.com

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-*-* aarch64-*-*
            Version|unknown                     |14.0
                 CC|                            |matz at gcc dot gnu.org

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
The specification needs to be more detailed (obviously).  For kernel use you
are targeting GPRs only I suppose.

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[fw at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

Florian Weimer <fw at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fw at gcc dot gnu.org

--- Comment #2 from Florian Weimer <fw at gcc dot gnu.org> ---
I tried to write up something for the x86-64 psABI:

Document the ABI for __preserve_most__ function calls
<https://gitlab.com/x86-psABIs/x86-64-ABI/-/merge_requests/45>

This should match what Clang implements (but not what it documents).

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[matz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #3 from Michael Matz <matz at gcc dot gnu.org> ---
Huh, since when does clang implement this?  See also 
  https://gcc.gnu.org/pipermail/gcc-patches/2023-July/624004.html
where I asked for comments about a similar, but not same, mechanism.  I came
from
the angle of preserving a couple of SSE registers on x86-64.

For ABIs you generally want a good mix between caller- and callee-saved
registers. The x86-64 psABI didn't do that on the SSE regs for conscious, but
meanwhile irrelevant, reasons, so my approach above tried to rectify this.

The clang attributes seem to go against that general idea, they move all regs
(or all general regs) into being callee-saved (except, strangely for aarch64?).
It also makes argument registers be callee-saved, which is very unconventional.

Does the clang implementation take into account the various problematic cases
that arise when calling a normal function from a (say) preserve_all function
(hint: such call can't usually be done)?  Does exception handling,
setjmp/longjmp
and make/swapcontext interact with the clang attributes?

That said: the implementation itself (after a more detailed spec) can be
implemented in the same framework like the above patch.  (It's possible that
something more needs doing for the unusual situation that argument regs are
then
callee-saved).

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[matz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #4 from Michael Matz <matz at gcc dot gnu.org> ---
(In reply to Florian Weimer from comment #2)
> I tried to write up something for the x86-64 psABI:
> 
> Document the ABI for __preserve_most__ function calls
> <https://gitlab.com/x86-psABIs/x86-64-ABI/-/merge_requests/45>
> 
> This should match what Clang implements (but not what it documents).

Shouldn't we first discuss whether any of this, or particular aspects of it,
are
a good idea at all, before specifying behaviour that a random compiler happened
to implement?

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[fw at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #5 from Florian Weimer <fw at gcc dot gnu.org> ---
(In reply to Michael Matz from comment #3)
> For ABIs you generally want a good mix between caller- and callee-saved
> registers. The x86-64 psABI didn't do that on the SSE regs for conscious, but
> meanwhile irrelevant, reasons, so my approach above tried to rectify this.
> 
> The clang attributes seem to go against that general idea, they move all regs
> (or all general regs) into being callee-saved (except, strangely for
> aarch64?).

This is intended for functions that are called conditionally, but rarely, such
as debug logging. It's not a generally useful calling convention.

> It also makes argument registers be callee-saved, which is very
> unconventional.

Isn't this done for the this pointer in some C++ ABIs?

> Does the clang implementation take into account the various problematic
> cases that arise when calling a normal function from a (say) preserve_all
> function
> (hint: such call can't usually be done)?

How so? We need to version the __preserve_most__ attribute with the ISA level,
of course.

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hubicka at gcc dot gnu.org

--- Comment #6 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
This may be interesting option for string functions. If block is small one can
do copying without clobbering (many) registers. If block is large there is
enough time to save all registers used.

But indeed preserving even the registers holding function parameters is bit odd
IMO.

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[matz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #7 from Michael Matz <matz at gcc dot gnu.org> ---
(In reply to Florian Weimer from comment #5)
> > It also makes argument registers be callee-saved, which is very
> > unconventional.
> 
> Isn't this done for the this pointer in some C++ ABIs?

There are some, yes.  But those are then incompatible with the normal
definition of C++ calling conventions in terms of the ones for C (where 'this'
is simply
the first pointer argument).

The sense of that deviation lies in observing that 'this' usually isn't changed
by the callee.  For arbitrary arguments this doesn't hold, they belong to the
called function and so using a call-clobbered reg makes most sense.  It's
possible to do different of course, but it's very unusual.

> > Does the clang implementation take into account the various problematic
> > cases that arise when calling a normal function from a (say) preserve_all
> > function
> > (hint: such call can't usually be done)?
> 
> How so? We need to version the __preserve_most__ attribute with the ISA
> level, of course.

I don't see how that helps.  Imagine a preserve_all function foo that calls
printf.  How do you propose that 'foo' saves all parts of the SSE registers,
even those that aren't invented yet, or those that can't be touched by the
current ISA?  (printf might clobber all of these)

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[fw at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #8 from Florian Weimer <fw at gcc dot gnu.org> ---
(In reply to Michael Matz from comment #7)
> > > Does the clang implementation take into account the various problematic
> > > cases that arise when calling a normal function from a (say) preserve_all
> > > function
> > > (hint: such call can't usually be done)?
> > 
> > How so? We need to version the __preserve_most__ attribute with the ISA
> > level, of course.
> 
> I don't see how that helps.  Imagine a preserve_all function foo that calls
> printf.  How do you propose that 'foo' saves all parts of the SSE registers,
> even those that aren't invented yet, or those that can't be touched by the
> current ISA?  (printf might clobber all of these)

Vector registers are out of scope for this.

But lets look at APX. If printf is recompiled to use APX, then it will clobber
the extended register file. If we define __preserve_most__ the way we do in my
psABI proposal (i.e., *not* as everything but %r11), the extended APX registers
are still caller-saved. We will have to introduce a __preserve_most_apx__
attribute and a different psABI specification that says that the new APX
registers are callee-saved, too.

(These details are the main reason why I want this in the psABI documentation.
This stuff is out there and will be used, so let's make sure that it's somewhat
reasonable.)

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[matz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #9 from Michael Matz <matz at gcc dot gnu.org> ---
(In reply to Florian Weimer from comment #8)
> (In reply to Michael Matz from comment #7)
> > > > Does the clang implementation take into account the various problematic
> > > > cases that arise when calling a normal function from a (say) preserve_all
> > > > function
> > > > (hint: such call can't usually be done)?
> > > 
> > > How so? We need to version the __preserve_most__ attribute with the ISA
> > > level, of course.
> > 
> > I don't see how that helps.  Imagine a preserve_all function foo that calls
> > printf.  How do you propose that 'foo' saves all parts of the SSE registers,
> > even those that aren't invented yet, or those that can't be touched by the
> > current ISA?  (printf might clobber all of these)
> 
> Vector registers are out of scope for this.

Why do you say that?  From clang: "Furthermore it also preserves all
floating-point registers (XMMs/YMMs)."  (for preserve_all, but this bugreport
does
include that variant of the attribute).

> But lets look at APX. If printf is recompiled to use APX, then it will
> clobber the extended register file. If we define __preserve_most__ the way
> we do in my psABI proposal (i.e., *not* as everything but %r11), the
> extended APX registers are still caller-saved.

Right, for preserve_most _with your wording_ it works out.  preserve_all
or preserve_most with clang wording doesn't.

> (These details are the main reason why I want this in the psABI
> documentation. This stuff is out there and will be used, so let's make sure
> that it's somewhat reasonable.)

I agree with that.  I just want a little hashing-out-the-details before
putting anything in the psABI.

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[fw at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #10 from Florian Weimer <fw at gcc dot gnu.org> ---
(In reply to Michael Matz from comment #9)
> > > I don't see how that helps.  Imagine a preserve_all function foo that calls
> > > printf.  How do you propose that 'foo' saves all parts of the SSE registers,
> > > even those that aren't invented yet, or those that can't be touched by the
> > > current ISA?  (printf might clobber all of these)
> > 
> > Vector registers are out of scope for this.
> 
> Why do you say that?  From clang: "Furthermore it also preserves all
> floating-point registers (XMMs/YMMs)."  (for preserve_all, but this
> bugreport does include that variant of the attribute).

Ugh, I preferred not to look at it because it's likely that the Clang
implementation is broken (not future-proof).

> > But lets look at APX. If printf is recompiled to use APX, then it will
> > clobber the extended register file. If we define __preserve_most__ the way
> > we do in my psABI proposal (i.e., *not* as everything but %r11), the
> > extended APX registers are still caller-saved.
> 
> Right, for preserve_most _with your wording_ it works out.  preserve_all
> or preserve_most with clang wording doesn't.

In glibc, we already use a full context switch with XSAVE for the dynamic
loader trampoline. As far as I understand it, it's not future-proof. The kernel
could provide an interface that is guaranteed to work because it only enables
those parts of the register file that it can context-switch. I can probably get
the userspace-only implementation into glibc, but the kernel interface seems
unlikely. We'd also have to work out the interaction of preserve_all and
unwinding, setjmp etc.; not sure if there is a proper solution for that.

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[matz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #11 from Michael Matz <matz at gcc dot gnu.org> ---
(In reply to Florian Weimer from comment #10)
> (In reply to Michael Matz from comment #9)
> > > > I don't see how that helps.  Imagine a preserve_all function foo that calls
> > > > printf.  How do you propose that 'foo' saves all parts of the SSE registers,
> > > > even those that aren't invented yet, or those that can't be touched by the
> > > > current ISA?  (printf might clobber all of these)
> > > 
> > > Vector registers are out of scope for this.
> > 
> > Why do you say that?  From clang: "Furthermore it also preserves all
> > floating-point registers (XMMs/YMMs)."  (for preserve_all, but this
> > bugreport does include that variant of the attribute).
> 
> Ugh, I preferred not to look at it because it's likely that the Clang
> implementation is broken (not future-proof).

I see, then we need to make clear that we aren't going to do anything about
preserve_all with clangs wording, in context of this report.

FWIW, in my implementation referred to above I chose to also have two variants:
one saving/restoring only the SSE2 parts of *mm8-*mm15 (i.e. xmm8-xmm15),
and one guaranteering to not clobber anything of *mm8-*mm15.  (No guarantees
about the *mm16 upwards).  The first variant can call foreign functions,
the second variant simply is allowed to only call functions that also give
that guarantee.

(There is also the question of mask registers, the clang docu doesn't talk
about them.  And I still would like to know the reason for the seemingly
arbitrary choice to leave some regs call clobbered for aarch64).

> > > But lets look at APX. If printf is recompiled to use APX, then it will
> > > clobber the extended register file. If we define __preserve_most__ the way
> > > we do in my psABI proposal (i.e., *not* as everything but %r11), the
> > > extended APX registers are still caller-saved.
> > 
> > Right, for preserve_most _with your wording_ it works out.  preserve_all
> > or preserve_most with clang wording doesn't.
> 
> In glibc, we already use a full context switch with XSAVE for the dynamic
> loader trampoline. As far as I understand it, it's not future-proof. The
> kernel could provide an interface that is guaranteed to work because it only
> enables those parts of the register file that it can context-switch. I can
> probably get the userspace-only implementation into glibc, but the kernel
> interface seems unlikely. We'd also have to work out the interaction of
> preserve_all and unwinding, setjmp etc.; not sure if there is a proper
> solution for that.

There are a couple possibilities to implement a halfway solution for this,
via XSAVE and friends, or via runtime dispatching dependend on current CPU
(e.g. provide a generic save/restore-stuff function in libgcc).  The problem
will always be where the memory for this save/restore pattern should come from,
its size isn't constant at compile time.  That's also solvable, but it's 
becoming more and more hairy.

That's why I chose to simply disallow calling foreign functions from those
that want to give very strict guarantees.  We could do the same for
preserve_all, if we absolutely want to have it.

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[sjames at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

Sam James <sjames at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |arsen at gcc dot gnu.org,
                   |                            |sjames at gcc dot gnu.org

--- Comment #12 from Sam James <sjames at gcc dot gnu.org> ---
See
https://discourse.llvm.org/t/rfc-exposing-ghccc-calling-convention-as-preserve-none-to-clang/74233
too (via https://github.com/python/cpython/pull/113465 &
https://github.com/python/cpython/issues/113464).

[Bug target/110899] RFE: Attributes preserve_most and preserve_all

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899

--- Comment #13 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Seems to be most of this should not implemented this way but rather an IPA RA
pass (which GCC already does but not always and definitely not across IPA
boundaries).