Re: [RFC patch 2/5] RISC-V: Reserve about 5K space in mcontext_t to support future ISA expansion.

[Adhemerval Zanella <adhemerval.zanella@linaro.org>]

On 30/09/2021 22:43, Vincent Chen wrote:
> On Tue, Sep 21, 2021 at 1:10 AM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * DJ Delorie via Libc-alpha:
>>
>>> Vincent Chen <vincent.chen@sifive.com> writes:
>>>> I am not familiar with the mechanism of LD_AUDIT, so I actually do not
>>>> know if this modification may have any effect on LD_AUDIT. If
>>>> possible, could you briefly introduce the issues for me? Thank you
>>>> very much.
>>>
>>> In general, when function foo() calls DSO function bar(), and bar() is
>>> in an object that needs to be loaded from disk, the loader needs to save
>>> foo()'s context, do a bunch of work, restore the context, and call
>>> bar().
>>>
>>> The LD_AUDIT feature adds a lot more "do a bunch of work" both on the
>>> foo->bar call, and on the bar->foo return, typically calling some third
>>> party functions to process the audit messages.
>>>
>>> However, if the "do a bunch of work" changes registers that aren't saved
>>> in the context, and aren't agreed on as "call clobbered" and thus
>>> changeable, problems happen.  If foo() expects a register to be
>>> preserved across the call to bar(), and the loader and audit functions
>>> don't know that and clobber it, foo() breaks.
>>
>> One point of clarification:
>>
>> The issue is with register usage for passing argument and return values.
>> It's more or less unrelated to whether registers are callee-saved or
>> caller-saved.  So you need special LD_AUDIT support as soon it's
>> possible to pass vector arguments and return values in registers (as
>> opposed to memory).
>>
>> Thanks,
>> Florian
>>
> Thank DJ Delorie and Florian very much for the detailed explanation
> and clarification. It is really helpful for me to understand this
> problem I have not noticed. Currently, I have some findings. If my
> understanding is wrong, please correct me. Thank you.
> 
> The ABI for using vector registers to pass arguments and return value
> is under discussion and close to ratification. As far as I know, the
> riscv Glibc resolver will have a similar issue to this LD_AUDIT
> problem after this new ABI is used. Hsiangkai Wang has sent a patch,
> https://sourceware.org/pipermail/libc-alpha/2021-August/129931.html,
> to deal with this issue in GLIBC resolver. This patch adds a new tag,
> STO_RISCV_VARIANT_CC, to indicate whether the function uses this new
> ABI or not. During the relocation process, if STO_RISCV_VARIANT_CC is
> set in the st_other field of the symbol being processed, the delayed
> binding mechanism will be disabled. It can avoid saving vector
> registers before entering the resolver function.
> 
> The LD_AUDIT problem is similar to this because we need to prevent the
> auditing function from clobbering the vector registers that store the
> argument to pass to the audited symbol. Therefore, I think this patch
> can resolve the LD_AUDIT issue as well.
> 

It solves the LD_AUDIT issue by not enabling it, so it won't work with
STO_RISCV_VARIANT_CC. It is essentially the same issue we have for AArch64
SVE. 

It should be a fair approach to just not support it, although it seems
that HPC tools does use it extensively for some specific usercases.  For
SVE case I tried to fix by saving/restoring the required SVE defined
registers by ABI argument passing [1], although the ABI does not really
defines it (Szabolcs thinks we should save *all* registers).

In any case, LD_AUDIT is corner case and I think we can workaround within
glibc (since we can use a different runtime trampoline to save/restore
the required state).

[1] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=c8315ccd30fcecc1b93a9bc3f073010190a86e05;hp=171fdd4bd4f337001db053721477add60d205ed8