Re: [PATCH,V1 00/14] Definition and support for SFrame unwind format

[Indu Bhagat <indu.bhagat@oracle.com>]

On 9/30/22 01:09, Jan Beulich wrote:
> On 30.09.2022 02:04, Indu Bhagat via Binutils wrote:
>> What is SFrame format and why do we need it
>> -------------------------------------------
>> SFrame format is the Simple Frame format.  It can be used to represent the
>> minimal necessary information for backtracing.  As such, it only encodes how to
>> recover the CFA (based on SP/FP) and the return address (RA) for all
>> instructions of a program.
>>
>> The format is supported on AMD64 and AARCH64 ABIs only.  The information stored
>> in the .sframe section is a subset of what .eh_frame can convey: .eh_frame can
>> convey how to resurrect all callee-saved registers, if need be; but .sframe
>> does not.
> 
> I expect the question was asked before, but since the summary here doesn't
> answer it: What about cases where the return address is stored in a
> callee-saved register? You won't be able to determine its value if that
> wasn't the leaf-most function in the active call chain and some inner
> function saved and then modified that (presumably) GPR. The same would
> likely apply if the return address was saved to some secondary stack.
> 
> Jan

Yes, this question was asked before. And thanks for asking this here.

(In the context of recovering return address) As far as x86_64 and 
aarch64 are concerned, I *believe* SFrame can represent most 
ABI-conforming cases with standard stack usage (except the #exceptions 
noted at the end of this message):
  - For x86_64, the return address is saved on stack with every call 
instruction
  - For aarch64, (from the ABI doc) all "Conforming code shall construct 
a linked list of stack-frames. Each frame shall link to the frame of its 
caller by means
of a frame record of two 64-bit values on the stack (independent of the 
data model).". So the return address is tracked as it flows from LR to 
stack and so on in the SFrame format.

So, IIUC, the case you mention will be "non ABI conforming code", 
correct ? In such a case, SFrame format cannot be used as is. It will 
need to be extended to support such cases:
  - first, encode the return register per SFrame FDE (so additional few 
bits per FDE).
  - second, recover all possible return registers (i.e., all 
callee-saved registers per FDE) via stack offsets in every FRE as 
applicable.

The second item will not fly in the SFrame format (atleast not in its 
current form); storing stack offsets explicitly like that (for a bunch 
of registers) will cause the unwind metadata to get quite large.  If 
there is a high usage of non-conforming code in applications, and SFrame 
is still desirable for those, something needs to be done.

My viewpoint here was to keep the format simple by targeting to support 
all ABI-conforming code. Is that reasonable ? (Posing this question to 
also the wider community).

Now a subset your question, I believe, also intends to ask - how do you 
plan to support architectures/ABIs where the return address can be in 
any register. The answer goes back to the above-mentioned two pieces of 
information (needed for non ABI conforming code in context of x86_64 and 
aarch64) and their high cost in SFrame format. Instead, an alternative 
strategy could be to take compiler's help to assign a "recommended 
register" for return register usage in such ABIs. The SFrame format will 
still need extensions (but these extensions are minimal) :
  - an additional field in the SFrame header to designate the 
"recommended return register" (the same register which was used at 
compile time).
  - the new ABI marker in the SFrame header.

Now if the compiler is not able to use the recommended return register 
for some functions, it will mean that SFrame-based backtracing will not 
work through those functions, hence affecting asynchronicity. I must 
admit that I haven't fully thought this one through, so not completely 
sure if such a solution work ? I think it might.

re: return address was saved to some secondary stack. I need some more 
understanding of the issue here. I was thinking an entity higher up, 
like the unwinder, will have enough context to pick the appropriate 
stack to which the stack offsets apply; because the decision as to when 
to switch to a secondary stack (or not) must be a application specific 
property ? If you have some resources to share here, it will be helpful. 
Alternatively, if usage of secondary stack is specified at an ABI level, 
can you please provide a reference so I can take a look ?

#exceptions (PS: I plan to work on these) :
  - .cfi_negate_ra_state handling in aarch64: if the code uses pointer 
signing for the return address, SFrame needs extensions to represent that.
  - .cfi_escape asm directives: these are also being skipped at this 
time. I may not be able to handle everything here, but even if we handle 
the most commonly occurring cases, I think we can get close.

Thanks