Re: RFC: ABI support for special memory area

["H.J. Lu" <hjl.tools@gmail.com>]

On Tue, Feb 28, 2017 at 8:19 AM, Carlos O'Donell <carlos@redhat.com> wrote:
> On 02/23/2017 09:59 PM, H.J. Lu wrote:
>>> Why does it run _after_ all shared objects and the executable file are loaded?
>>
>> Since __gnu_mbind_setup may call any external functions, it can only
>> be done after everything is loaded and relocated.
>
> Who defines this function?

Platform vendor with special memory support should provide such function.

> Where is it implemented?

We are working on libmbind to implement it.

> What does a typical implementation look like for MCDRAM use?

It uses NUMA, similar to memkind:

https://github.com/memkind/memkind

to bind pages to a NUMA node.

>>> Why not let the dynamic loader choose when it needs to setup the memory?
>>
>> 1. We want to be able to add support for new type memory by just
>> updating the run-time library of __gnu_mbind_setup, instead of
>> updating glibc.
>
> Which library defines it?

The default __gnu_mbind_setup is a weak function in ld.so since
ld.so can't have undefined function.  The real one is in libmbind
which overrides the default one in ld.so.

> Can two libraries define it? Does the dynamic loader run every DSO's
> version of __gnu_mbind_setup?

Only one will be used by ld.so.

>> 2. Since __gnu_mbind_setup may depend on other libraries, we
>> don't want a simple executable requires libfoo and libbar, in addition
>> to glibc, nor make libfoo and libbar part of glibc.
>
> Why can't this be run in a constructor? Is that too late?

We can use MCDRAM for dynamically allocated memory with
memkind.  We are looking for a user-friendly way to use MCDRAM
for normal data variables.

> This seems like a specialized form of constructor that is guaranteed
> to run before all other constructors?

Yes.

>>>> int
>>>> __gnu_mbind_setup (unsigned int type, void *addr, size_t length)
>>>> {
>>>>   return 0;
>>>> }
>>>>
>>>> which can be overridden by a different implementation at link-time.
>>>
>>> What if you _can't_ bind at ADDR?
>>
>> It happens on systems without special memory.  __gnu_mbind_setup
>> returns a positive value and ld.so keeps going.
>
> Isn't this a violation of what the application binary requested?

Even on systems with MCDROM, you may not exceed the limit.
The application should still run correctly.

> This is a soft-failure that that application doesn't know about.

Performance may be lower.  But it will run correctly.

> Might this become a security issue if the application expected the
> specific memory type?

__gnu_mbind_setup returns a negative value for fatal error if
security is involved and ld.so aborts on fatal error.

>>> What if the binding would work if ADD was any value?
>>>
>>
>> GNU_MBIND isn't a LOAD segment,  similar to GNU_RELRO:
>>
>> Program Headers:
>>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>>   LOAD           0x000000 0x00000000 0x00000000 0x54624 0x54624 R E 0x1000
>>   LOAD           0x054e9c 0x00055e9c 0x00055e9c 0x001b0 0x001b8 RW  0x1000
>>   DYNAMIC        0x054eac 0x00055eac 0x00055eac 0x00110 0x00110 RW  0x4
>>   NOTE           0x000114 0x00000114 0x00000114 0x00044 0x00044 R   0x4
>>   GNU_EH_FRAME   0x048eb8 0x00048eb8 0x00048eb8 0x00ff4 0x00ff4 R   0x4
>>   GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10
>>   GNU_RELRO      0x054e9c 0x00055e9c 0x00055e9c 0x00164 0x00164 R   0x1
>>
>> ADDR contains the start of a memory region within the LOAD segment.
>
> What are the constraints of GNU_MBIND then?

Each GNU_MBIND segment must be aligned at page boundary
and within one LOAD segment.

> Is it required that it covers only the SHF_GNU_MBIND marked sections which
> are part of a PT_LOAD segment?

The SHF_ALLOC bit must be set for SHF_GNU_MBIND sections.


-- 
H.J.