Re: [QUERY]: Adding support for a platform

[Palmer Dabbelt <palmer@dabbelt.com>]

On Tue, 29 Nov 2022 13:40:53 PST (-0800), prabhakar.csengg@gmail.com wrote:
> Hi Palmer,
>
> On Tue, Nov 29, 2022 at 8:57 PM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>
>> On Tue, 29 Nov 2022 12:21:31 PST (-0800), Andrew Waterman wrote:
>> > On Tue, Nov 29, 2022 at 12:12 PM Jessica Clarke <jrtc27@jrtc27.com> wrote:
>> >>
>> >> On 29 Nov 2022, at 19:28, Lad, Prabhakar <prabhakar.csengg@gmail.com> wrote:
>> >> >
>> >> > Hi Palmer,
>> >> >
>> >> > Oops I should have included linux-riscv to the CC list here (doing it now)
>> >> >
>> >> > On Tue, Nov 29, 2022 at 7:20 PM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> >> >>
>> >> >> On Tue, 29 Nov 2022 11:16:29 PST (-0800), binutils@sourceware.org wrote:
>> >> >>> Hi All,
>> >> >>>
>> >> >>> If this is not the right place to ask this question please let me know.
>> >> >>>
>> >> >>> So we have a RISCV platform (Renesas RZ/Five) for which we need to
>> >> >>> adjust the TEXT_START_ADDR. So below are my queries:
>> >> >>> * What is the procedure for upstreaming?
>> >> >>> * Are patches accepted for individual platforms (for adjusting TEXT_START_ADDR)?
>> >> >>
>> >> >> Is this still related to that bug with the static binaries you'd
>> >> >> mentioned before?  I think we'd really need to figure out the root cause
>> >> >> before we can make a reasoned decision here, my guess would be that
>> >> >> changing TEXT_START_ADDR just works around the real bug.
>> >> >>
>> >> > Below is the root cause and the solution:
>> >> >
>> >> > TEXT_START_ADDR is the start of the text segment of an application.
>> >> > This is being set to 0x10000 for RISCV platforms.
>> >> >
>> >> > So when an application is compiled with the static flag the load would
>> >> > start from 0x10000 - xyz (depending on size of the application)
>> >> >
>> >> > Entry point 0x101c0
>> >> > There are 5 program headers, starting at offset 64Program Headers:
>> >> >  Type           Offset             VirtAddr           PhysAddr
>> >> >                 FileSiz            MemSiz              Flags  Align
>> >> >  LOAD           0x0000000000000000 0x0000000000010000 0x0000000000010000
>> >> >                 0x0000000000059b48 0x0000000000059b48  R E    0x1000
>> >> >  LOAD           0x0000000000059b60 0x000000000006ab60 0x000000000006ab60
>> >> >                 0x0000000000001f68 0x0000000000003528  RW     0x1000
>> >> >
>> >> > So for the above application which is compiled statically we can see
>> >> > the entry point is 0x101c0 and load 0x0000000000010000.
>> >> >
>> >> > Andes AX45MP cores have local memory ILM and DLM that are mapped in
>> >> > the region H’0_0003_0000 - H’0_0004_FFFF on the RZ/Five SoC. When the
>> >> > virtual address falls in this range the MMU doesn't trigger a page
>> >> > fault and assumes the virtual address as physical address and hence
>> >> > the application fails to run (panics somewhere).
>> >> >
>> >> > So to avoid this issue we set the TEXT_START_ADDR to 0x50000 so that
>> >> > the virtual address of any statically compiled application does not
>> >> > fall in the range of H’0_0003_0000 - H’0_0004_FFFF.
>> >> >
>> >> > Elf file type is EXEC (Executable file)
>> >> > Entry point 0x504e4
>> >> > There are 5 program headers, starting at offset 64
>> >> >
>> >> > Program Headers:
>> >> >  Type           Offset             VirtAddr           PhysAddr
>> >> >                 FileSiz            MemSiz              Flags  Align
>> >> >  LOAD           0x0000000000000000 0x0000000000050000 0x0000000000050000
>> >> >                 0x0000000000057dc8 0x0000000000057dc8  R E    0x1000
>> >> >  LOAD           0x00000000000585b8 0x00000000000a95b8 0x00000000000a95b8
>> >> >                 0x0000000000004ee0 0x00000000000064b0  RW     0x1000
>> >> >  NOTE           0x0000000000000158 0x0000000000050158 0x0000000000050158
>> >> >                 0x0000000000000044 0x0000000000000044  R      0x4
>> >> >
>> >> > So now with the fix for statically compiled applications we can see
>> >> > its offsetted and entry point is 0x504e4 and load is at
>> >> > 0x0000000000050000. So with this we are for sure the MMU will always
>> >> > trigger a page fault.
>> >>
>> >> Well that’s just a blatant violation of the spec.
>> >
>> > Right - this isn't a platform quirk; it's a violation of the ISA.
>> > Unless this behavior is engaged by a custom mode bit, that is, in
>> > which case, it should just be disabled by default and none of this
>> > would be a problem.  If the behavior is unconditionally enabled, this
>> > should be treated as an erratum.
>>
>> I don't see this as any different than the T-Head page table attributes,
>> where it was made abundantly clear that vendors self-certify their
>> implementations and thus anything goes.  It doesn't really matter if we
>> called it an errata, bug, surprise feature, vendor extension, whatever;
>> it's RISC-V so we've got to live with it.
>>
> phew that's a relief.
>
>> >> What happens if a
>> >> malicious (or buggy) userspace binary goes and accesses unmapped memory
>> >> in that range? Does it actually get access to this ILM/DLM? If so
>> >> that’s a gaping side channel.
>>
>> IIUC that's the case, but that's just from reading this post.
>>
>> There's nothing we can do to prevent vendors from building hardware with
>> side channels.  In this case I suppose we could swap out the memory
>> region on context switches, from a quick look at the datasheet these are
>> single-core so that'd be safe.  My guess is it'd be too slow for users
>> to want that on by default, but I don't have any use case for this HW so
>> I'm not sure.
>>
> Enabling the local memory (ILM/DLM) on the core is a specification
> option and is enabled on RZ/Five SoC. I'm checking with Andes if this
> can be forcefully disabled.
> The only use case would be to speed up things for some slower block
> (but said that its user application specific)

If it's possible to disable this somehow that'd be great, but after 
writing up the "maybe we can context switch this" bit I think we could 
probably get away with some sort of compatibility mode here.  
Essentially: Don't allocate out of this region by default, but if 
userspace explicitly maps this region (as happens with static binaries) 
then allow it, but track those allocations and begin swapping it on 
context switches.

That might be a lot of work and would definitely be slow, but as long as 
other VAs can map to these PAs then I think we'd be pretty much safe -- 
we'd lose read-only and execute-only permission tracking, but existing 
binaries would still run.  That would allow this TEXT_START_ADDR change 
to be just a performance thing, and we might want that as a tunable for 
distros anyway (for huge page alignment, for example).

A lot of that would depend on exactly how the hardware treats these 
special VAs, though.  Probably best if you just dig through the docs, 
see how this all works, and then propose some rough patches?