Re: RFC: ABI support for special memory area

["H.J. Lu" <hjl.tools@gmail.com>]

On Wed, Mar 22, 2017 at 9:18 AM, Suprateeka R Hegde
<hegdesmailbox@gmail.com> wrote:
>
>
> On 03/20/17 23:52, H.J. Lu wrote:
>> On Fri, Mar 17, 2017 at 11:12 AM, Suprateeka R Hegde
>> <hegdesmailbox@gmail.com> wrote:
>>> On Friday 17 March 2017 02:55 AM, H.J. Lu wrote:
>>>>> Since ld.so is not meant only for programs with C style linkage, what if
>>>>> the real implementation library is written in C++ and wants to export
>>>>> only mangled names (interfaces) without any "extern C" kludge? Or is
>>>>> this considered to be a standard C library call just like mmap etc.?
>>>>
>>>> Only the __gnu_mbind_setup symbol is used.  We can change the
>>>> second argument to "void *data" and make it dependent on memory
>>>> type.  But to support a new memory type, we have to update ld.so.  I'd
>>>> like to use the same ld.so binary to support any memory types even if
>>>> it means that we need to pass info to __gnu_mbind_setup which isn't
>>>> used by all memory types.
>>>
>>> Ah! Now I understand the design completely (I think). Looks like Carlos
>>> understood this quite earlier in the discussion.
>>>
>>> You are saying that the interface -
>>>
>>> int __gnu_mbind_setup (unsigned int type, void *addr, size_t length);
>>>
>>> - is fixed in ld.so and also in the real implementation library. And,
>>> the real implementation in turn calls the actual-real-implementation, as
>>> shown in your libmbind code:
>>>
>>> int
>>> __gnu_mbind_setup (unsigned int type, void *addr, size_t length)
>>> {
>>>   // in turn calls actual implementation
>>>   return vendor_specific_mbind_setup (vendor specific types);
>>> }
>>>
>>>
>>> All these while, based on the current description, I was of the
>>> impression that your design allows __gnu_mbind_setup interface itself to
>>> be overridden in the real implementation, something like:
>>>
>>> int
>>> __gnu_mbind_setup (__nvm_kmem_t *nvm_obj, void *nvm_handle)
>>> {
>>>   // actual implementation directly here in the body
>>> }
>>>
>>> So I was wondering how and hence most of my points were out-of-phase.
>>>
>>>>  The question is what the possible info needed
>>>> for all memory types is.
>>>
>>> Thats too much to predict right now. And the current interface you
>>> defined also does not seem to be generic. For instance, my NVM
>>> implementation, though not complete, needs a totally different set of
>>> arguments. So going by the current design, I will have to use
>>> __gnu_mbind_setup (unsigned int type, void *addr, size_t length) just to
>>> call my real setup, without using any of the arguments passed by ld.so.
>>>
>>> Assuming I am in sync with you now, I would say that the pseudo code I
>>> showed earlier works for you as well as for me as well as for anybody
>>> else. In other words it is more generic.
>>>
>>> With that approach, there is
>>>
>>> 1. No need to update ld.so every time for every new mem type
>>> 2. No need to know all possible info needed for all mem types
>>> 3. No need to encode all types in the API (as Carlos said)
>>>
>>> We just use pointer to implementation interface - struct
>>> __gnu_mbind_context that I showed. And we can have a default struct
>>> instantiated in ld.so and a global pointer pointing to that. And later
>>> the global pointer can be made to point to the vendor specific struct,
>>> before ld.so actually calls __gnu_mbind_setup, thereby completing a
>>> successful override (if necessary, that is when special memory types are
>>> in use).
>>>
>>> Or similar mechanisms to override default struct instantiated in ld.so.
>>> There are many well known ways to override the default struct as we all
>>> know.
>>>
>>> Personally I think this would be a better way to provide the ABI support
>>> in a generic way.
>>
>> ld.so needs to call the real __gnu_mbind_setup implementation
>> with the correct argument.
>
> Yes and with my example code, ld.so calls with correct argument always.
> And its always only one argument -- a pointer to struct. By default
> pointing  to default struct, and when overridden pointing to
> implementation specific struct.
>
>>  We can keep it ASIS and add a new
>> new one, __gnu_mbind_setup_v2, if needed.
>
> Hmm :-)
>
> This also looks good. Though whoever adds this _v2 (assuming its me
> right now :-)), gets to ensure all the herculean compatibility hooks for
> _v1 are in place. But thats OK I believe.
>
>>
>>> That said, I am OK to live with minor kludges and we can keep the design
>>> as is.
>>>
>>>>
>>>>> And you may also want to define the flow for fully archive bound static
>>>>> binaries.
>>>>
>>>> For static executable, __gnu_mbind_setup will be called on all MBIND
>>>> segments before constructors are called.  __gnu_mbind_setup in libc.a
>>>> is weak and will be overridden by the real one in libmbind.a.
>>>
>>> Lets add this also in the ABI support document.
>>>
>>
>> How about this:
>>
>> Run-time support
>>
>> int __gnu_mbind_setup_v1 (unsigned int type, void *addr, size_t length);
>>
>> It sets up special memory area of 'type' and 'length' at 'addr' where
>> 'addr' is a multiple of page size.  It returns zero for success, positive
>> value of ERRNO for non-fatal error and negative value of ERRNO for fatal
>> error.
>>
>> After all shared objects and the executable file are loaded, relocations
>> are processed, for each GNU_MBIND segment in a shared object or the
>> executable file, run-time loader calls __gnu_mbind_setup_v1 with type,
>> address and length.  If __gnu_mbind_setup_v1 must be defined in run-time
>> loader, it should be implemented as a weak function:
>>
>> int
>> __gnu_mbind_setup_v1 (unsigned int type, void *addr, size_t length)
>> {
>>   return 0;
>> }
>>
>> in run-time loader so that the GNU_MBIND run-time library isn't required
>> for normal executable nor shared object.  The real implementation of
>> __gnu_mbind_setup_v1 should be in the GNU_MBIND run-time library and
>> overridde the weak one in run-time loader.
>
> Looks good to me.
>
> --
> Supra

There is a way to support GNU_MBIND segments without the glibc changes.
Instead, dl_iterate_phdr

int dl_iterate_phdr (int (*callback) (struct dl_phdr_info *info,
                                      size_t size, void *data),
                     void *data);

is called via the .init_array section to process GNU_MBIND segments in
executable and shared objects:

static int
callback (struct dl_phdr_info *info, size_t size, void *data)
{
  Compute the load address of the current module.
  if info->dlpi_addr == the load address of the current module
    {
      check ELF program headers and process GNU_MBIND segments
      return 1;
    }

  return 0;
}

static void
call_gnu_mbind_setup (void)
{
  dl_iterate_phdr (callback, NULL);
}

static void (*init_array) (void)
 __attribute__ ((section (".init_array"), used))
 = &call_gnu_mbind_setup;


-- 
H.J.