How can I wrap ld-linux or execve into it?

How can I wrap ld-linux or execve into it?

[Farid Zakaria]

Hi,

I am looking to perform some functionality before the dynamic linker
(linux-ld/ld.so) is invoked.

My naive assessment was that I would be able to set in the PT_INTERP
section of a binary, my *static binary*, which will then execve into
the dynamic linker after doing some precanned actions.

Unfortunately, trying this has resulted in some SIGSEGV...

I came across https://github.com/Mic92/nix-ld which seems to do
something similar, but I was curious why it has to do a lot more to
achieve the same effect with a jump.

I have also been pointed to LD_AUDIT however I am also interested in
having it agnostic to libc (glibc vs. musl)

Thank you for any tips, guidance or links you can provide.
FZ

Re: How can I wrap ld-linux or execve into it?

[Farid Zakaria]

Adding some more context based on some private responses:
My goal is to explore some potential changes as part of a PhD thesis
to control dynamic linking.

I have explored modifying glibc/musl but I was exploring an execve approach to:
1. make the change libc agnostic
2. restrict my changes to a smaller codebase for the purpose of the exploration

On Mon, Jan 31, 2022 at 12:10 PM Farid Zakaria <fmzakari@ucsc.edu> wrote:
>
> Hi,
>
> I am looking to perform some functionality before the dynamic linker
> (linux-ld/ld.so) is invoked.
>
> My naive assessment was that I would be able to set in the PT_INTERP
> section of a binary, my *static binary*, which will then execve into
> the dynamic linker after doing some precanned actions.
>
> Unfortunately, trying this has resulted in some SIGSEGV...
>
> I came across https://github.com/Mic92/nix-ld which seems to do
> something similar, but I was curious why it has to do a lot more to
> achieve the same effect with a jump.
>
> I have also been pointed to LD_AUDIT however I am also interested in
> having it agnostic to libc (glibc vs. musl)
>
> Thank you for any tips, guidance or links you can provide.
> FZ

Re: How can I wrap ld-linux or execve into it?

[Adhemerval Zanella]

On 31/01/2022 17:10, Farid Zakaria via Libc-help wrote:
> Hi,
> 
> I am looking to perform some functionality before the dynamic linker
> (linux-ld/ld.so) is invoked.
> 
> My naive assessment was that I would be able to set in the PT_INTERP
> section of a binary, my *static binary*, which will then execve into
> the dynamic linker after doing some precanned actions.
> 
> Unfortunately, trying this has resulted in some SIGSEGV...
> 
> I came across https://github.com/Mic92/nix-ld which seems to do
> something similar, but I was curious why it has to do a lot more to
> achieve the same effect with a jump.
> 
> I have also been pointed to LD_AUDIT however I am also interested in
> having it agnostic to libc (glibc vs. musl)
> 
> Thank you for any tips, guidance or links you can provide.
> FZ

The issue is for static linking _dl_aux_init will setup the _dl_phdr
to the loaded binary (since it was done by the kernel) passed on auxiliary
vectors:

elf/dl-support.c:

246 void
247 _dl_aux_init (ElfW(auxv_t) *av)
248 {
[...]
269       case AT_PHDR:
270         GL(dl_phdr) = (const void *) av->a_un.a_val;
271         break;
[...]

And this is later used to setup the TCB:

csu/libc-tls.c

104 void
105 __libc_setup_tls (void)
106 {
[...]
120   /* Look through the TLS segment if there is any.  */
121   if (_dl_phdr != NULL)
122     for (phdr = _dl_phdr; phdr < &_dl_phdr[_dl_phnum]; ++phdr)
123       if (phdr->p_type == PT_TLS)
124         {
125           /* Remember the values we need.  */
126           memsz = phdr->p_memsz;
127           filesz = phdr->p_filesz;
128           initimage = (void *) phdr->p_vaddr + main_map->l_addr;
129           align = phdr->p_align;
130           if (phdr->p_align > max_align)
131             max_align = phdr->p_align;
132           break;
133         }

The problem is seice _dl_phdr is not pointing to the static programs acting
as loader, the PT_TLS is not considered and thus not initialized correctly.
That's why once __ctype_init tries to access TLS variables it triggers an
invalid memory reference:

(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x000000000045f464 in __ctype_init () at ctype-info.c:31
31        *bp = (const uint16_t *) _NL_CURRENT (LC_CTYPE, _NL_CTYPE_CLASS) + 128;


And I don't think this would be easy to support without changing a *lot* 
on static linking organization.  If you check the loader code, it avoids
to use TLS exactly to avoid this initialization issue.

I think the best option to work by checking elt/rtld.c and see if you can
hack a way link you code after its initialization. 

Re: How can I wrap ld-linux or execve into it?

[Farid Zakaria]

Thank you -- I need to parse your solution more in depth.
I will also take a look more at rtld -- however I was hoping to be
compatible with musl as well.

I was told by a colleague that execve solution may work if the static
binary has "no relocations".
How can I control that through the compilation of a program beyond
limiting myself to a single object-file?

On Tue, Feb 1, 2022 at 4:43 AM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 31/01/2022 17:10, Farid Zakaria via Libc-help wrote:
> > Hi,
> >
> > I am looking to perform some functionality before the dynamic linker
> > (linux-ld/ld.so) is invoked.
> >
> > My naive assessment was that I would be able to set in the PT_INTERP
> > section of a binary, my *static binary*, which will then execve into
> > the dynamic linker after doing some precanned actions.
> >
> > Unfortunately, trying this has resulted in some SIGSEGV...
> >
> > I came across https://github.com/Mic92/nix-ld which seems to do
> > something similar, but I was curious why it has to do a lot more to
> > achieve the same effect with a jump.
> >
> > I have also been pointed to LD_AUDIT however I am also interested in
> > having it agnostic to libc (glibc vs. musl)
> >
> > Thank you for any tips, guidance or links you can provide.
> > FZ
>
> The issue is for static linking _dl_aux_init will setup the _dl_phdr
> to the loaded binary (since it was done by the kernel) passed on auxiliary
> vectors:
>
> elf/dl-support.c:
>
> 246 void
> 247 _dl_aux_init (ElfW(auxv_t) *av)
> 248 {
> [...]
> 269       case AT_PHDR:
> 270         GL(dl_phdr) = (const void *) av->a_un.a_val;
> 271         break;
> [...]
>
> And this is later used to setup the TCB:
>
> csu/libc-tls.c
>
> 104 void
> 105 __libc_setup_tls (void)
> 106 {
> [...]
> 120   /* Look through the TLS segment if there is any.  */
> 121   if (_dl_phdr != NULL)
> 122     for (phdr = _dl_phdr; phdr < &_dl_phdr[_dl_phnum]; ++phdr)
> 123       if (phdr->p_type == PT_TLS)
> 124         {
> 125           /* Remember the values we need.  */
> 126           memsz = phdr->p_memsz;
> 127           filesz = phdr->p_filesz;
> 128           initimage = (void *) phdr->p_vaddr + main_map->l_addr;
> 129           align = phdr->p_align;
> 130           if (phdr->p_align > max_align)
> 131             max_align = phdr->p_align;
> 132           break;
> 133         }
>
> The problem is seice _dl_phdr is not pointing to the static programs acting
> as loader, the PT_TLS is not considered and thus not initialized correctly.
> That's why once __ctype_init tries to access TLS variables it triggers an
> invalid memory reference:
>
> (gdb) c
> Continuing.
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x000000000045f464 in __ctype_init () at ctype-info.c:31
> 31        *bp = (const uint16_t *) _NL_CURRENT (LC_CTYPE, _NL_CTYPE_CLASS) + 128;
>
>
> And I don't think this would be easy to support without changing a *lot*
> on static linking organization.  If you check the loader code, it avoids
> to use TLS exactly to avoid this initialization issue.
>
> I think the best option to work by checking elt/rtld.c and see if you can
> hack a way link you code after its initialization.

Re: How can I wrap ld-linux or execve into it?

[Adhemerval Zanella]

On 01/02/2022 14:14, Farid Zakaria wrote:
> Thank you -- I need to parse your solution more in depth.
> I will also take a look more at rtld -- however I was hoping to be
> compatible with musl as well.
> 
> I was told by a colleague that execve solution may work if the static
> binary has "no relocations".
> How can I control that through the compilation of a program beyond
> limiting myself to a single object-file?

Different than musl that packs everything in libc.so, glibc still has a 
contract between ld.so and libc.so, where the loader will setup multiple 
global resources where some will be shared between ld.so and libc.so.  
One of the is the thread pointer (TP), required and defined by ABI (for 
all supported architectures).  

So it should be possible to chain execution using this loader like program,
this loader-like program will need to setup the expect machine state for the 
loader's entrypoint (ElfW(Ehdr).e_entry): argc, argv, env, and auxv (you might
also check if some ABI also requires some other state I am forgetting here).
It also means that it would *very* trick to call back the loader as glibc
libc.so does for ld.so on, on for instance, symbol resolution (since TP and
other global states can no be shared).  

It really depends of what you want to perform before dynamic linker. The
"no relocations" I take it means no dynamic relocations, but this is not
suffice (as indicate by the global resource setup required by libc, such
as TP).

I think it would be easier to use a a minimal Linux support without any
libc or standard libraries (similar to a bare-metal target).  I think
you might accomplish it with newlib, but I am not sure.  

> 
> On Tue, Feb 1, 2022 at 4:43 AM Adhemerval Zanella
> <adhemerval.zanella@linaro.org> wrote:
>>
>>
>>
>> On 31/01/2022 17:10, Farid Zakaria via Libc-help wrote:
>>> Hi,
>>>
>>> I am looking to perform some functionality before the dynamic linker
>>> (linux-ld/ld.so) is invoked.
>>>
>>> My naive assessment was that I would be able to set in the PT_INTERP
>>> section of a binary, my *static binary*, which will then execve into
>>> the dynamic linker after doing some precanned actions.
>>>
>>> Unfortunately, trying this has resulted in some SIGSEGV...
>>>
>>> I came across https://github.com/Mic92/nix-ld which seems to do
>>> something similar, but I was curious why it has to do a lot more to
>>> achieve the same effect with a jump.
>>>
>>> I have also been pointed to LD_AUDIT however I am also interested in
>>> having it agnostic to libc (glibc vs. musl)
>>>
>>> Thank you for any tips, guidance or links you can provide.
>>> FZ
>>
>> The issue is for static linking _dl_aux_init will setup the _dl_phdr
>> to the loaded binary (since it was done by the kernel) passed on auxiliary
>> vectors:
>>
>> elf/dl-support.c:
>>
>> 246 void
>> 247 _dl_aux_init (ElfW(auxv_t) *av)
>> 248 {
>> [...]
>> 269       case AT_PHDR:
>> 270         GL(dl_phdr) = (const void *) av->a_un.a_val;
>> 271         break;
>> [...]
>>
>> And this is later used to setup the TCB:
>>
>> csu/libc-tls.c
>>
>> 104 void
>> 105 __libc_setup_tls (void)
>> 106 {
>> [...]
>> 120   /* Look through the TLS segment if there is any.  */
>> 121   if (_dl_phdr != NULL)
>> 122     for (phdr = _dl_phdr; phdr < &_dl_phdr[_dl_phnum]; ++phdr)
>> 123       if (phdr->p_type == PT_TLS)
>> 124         {
>> 125           /* Remember the values we need.  */
>> 126           memsz = phdr->p_memsz;
>> 127           filesz = phdr->p_filesz;
>> 128           initimage = (void *) phdr->p_vaddr + main_map->l_addr;
>> 129           align = phdr->p_align;
>> 130           if (phdr->p_align > max_align)
>> 131             max_align = phdr->p_align;
>> 132           break;
>> 133         }
>>
>> The problem is seice _dl_phdr is not pointing to the static programs acting
>> as loader, the PT_TLS is not considered and thus not initialized correctly.
>> That's why once __ctype_init tries to access TLS variables it triggers an
>> invalid memory reference:
>>
>> (gdb) c
>> Continuing.
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x000000000045f464 in __ctype_init () at ctype-info.c:31
>> 31        *bp = (const uint16_t *) _NL_CURRENT (LC_CTYPE, _NL_CTYPE_CLASS) + 128;
>>
>>
>> And I don't think this would be easy to support without changing a *lot*
>> on static linking organization.  If you check the loader code, it avoids
>> to use TLS exactly to avoid this initialization issue.
>>
>> I think the best option to work by checking elt/rtld.c and see if you can
>> hack a way link you code after its initialization.