Appending program headers to existing ELFs?

Appending program headers to existing ELFs?

[Matheus Afonso Martins Moreira]

I'd like to embed arbitrary data into existing ELF executables
in such a way that the data is automatically loaded onto memory
and is reachable by the program at runtime.

Linux passes a pointer to the program header table
of the running program via the auxiliary vector
which is how I plan on accessing the data.

With objcopy it is already possible to embed arbitrary files
to ELF files by placing them into sections:

    objcopy --add-section program.file1=file1 \
            --add-section program.file2=file2 \
            program program+files

However, those sections are unreachable when the program actually runs.
Given the address of the program header table, it is possible to obtain
a pointer to the ELF header. However, no progress can be made from
there since only the segments described by the program headers are
loaded by the kernel which excludes the sections and the data
I want to access. If I set the alloc flag on the new sections,
objcopy will even warn me about the fact they're not loaded.

So it seems to me that the solution is to add a new program header
to the ELF executable instead of a new section. How can this be done
with binutils tools?

If it's not possible to do this with objcopy, can it be done with ld?
Is it possible to add a placeholder program header table entry at link
time so that it can be adjusted later by external tools to encompass the
newly added sections without needing to relink the whole executable?

I've been experimenting with linker scripts but with no success.

Re: Appending program headers to existing ELFs?

[Nick Clifton]

Hi Matheus,

> I'd like to embed arbitrary data into existing ELF executables
> in such a way that the data is automatically loaded onto memory
> and is reachable by the program at runtime.

I have to say that this does sound suspicious.  Why can't you add
the data at link time ?


> With objcopy it is already possible to embed arbitrary files
> to ELF files by placing them into sections:
> 
>      objcopy --add-section program.file1=file1 \
>              --add-section program.file2=file2 \
>              program program+files
> 
> However, those sections are unreachable when the program actually runs.

Hmm, could you create a new symbol for the section and reference the data
that way.  eg:

   objcopy \
     --add-section foo=file1 \
     --add-symbol foo_start=foo \
     --set-section-flags foo=alloc,load \
     a.out a.extra

This assumes that you can access the symbol table at runtime.  I have
no idea if this is true for kernel images however.


> So it seems to me that the solution is to add a new program header
> to the ELF executable instead of a new section. How can this be done
> with binutils tools?

At the moment this can only be done by the linker, and it will only work
if the input files are object files or relocatable binaries.  ie if a
final link has been performed then you are not going to be able to add
new program header entries using the binutils tools.  (At least not as
they are today).


> If it's not possible to do this with objcopy, can it be done with ld?
> Is it possible to add a placeholder program header table entry at link
> time so that it can be adjusted later by external tools to encompass the
> newly added sections without needing to relink the whole executable?

Possibly.  With a linker script you can layout a PHDRS structure which
describes the entries that should go into the program header.  So in
theory you can include an extra entry for use by later tools.  I do not
know if this has ever been done in practice however, so it may be that
there are unknown bugs waiting to be discovered.

Cheers
   Nick

Re: Appending program headers to existing ELFs?

[Matheus Afonso Martins Moreira]

Hello Nick,

> Why can't you add the data at link time ?

I'd like to add a section to the executable
that users can freely modify after the binary
has been built and distributed without needing
to recompile or relink anything.

In concrete terms:

I'm building a small programming language interpreter
and would like to add support for embedding programs
into the executable. My intention is to treat the
embedded programs section as an additional load path
for libraries and modules as well as the main function.

> could you create a new symbol for the section
> and reference the data that way

That is indeed possible but as you noted
it would require access to the symbol table
at runtime and that's not guaranteed.
That information may have been stripped away
and the kernel does not load it to begin with.
I'm also not using libc so I don't have access
to dlopen and dlsym.

> At the moment this can only be done by the linker

I see. I suspected as much but hoped there would be a way.
Thanks for clarifying that.

> With a linker script you can layout a PHDRS structure

Yes. It's one of the solutions I tried.

I copied ld's internal linker script
and customized it with a PHDRS command:

    PHDRS
    {
        headers PT_PHDR FILEHDR PHDRS;
        text PT_LOAD;
        rodata PT_LOAD;
        data PT_LOAD;
        bss PT_LOAD;
        extra PT_LOAD; /* meant to be empty */
                       /* size/offset will be set later */
    }

I was not successful. I keep getting the following error:

    error: could not allocate headers

I was unable to determine the cause and fix it.

For ease of maintenance, I don't intend to mess with
the sections and segments generated by the GNU tools.
I would have liked to merely append a custom header
to the ones generated by ld. Unfortunately, merely
including a PHDRS command disables ld's program header
generation logic which sensibly maps the sections
to the headers.

A linker script directive that applies defaults
while allowing further customization would be
a great new feature.

In the meantime, I'm trying to work around the problem
by placing a statically allocated packed structure
into an isolated section so that its entire contents
can be overwritten by objcopy:

    objcopy --update-section .structure=./file
            lang lang+file

This reduces the problem to a binary file format problem
and seems like the way forward for me at the moment.
Would have been nice to be able to reuse the ELF machinery though...

Thanks for the guidance,

    Matheus

Re: Appending program headers to existing ELFs?

[Nick Clifton]

Hi Matheus,

> In concrete terms:
> 
> I'm building a small programming language interpreter
> and would like to add support for embedding programs
> into the executable. My intention is to treat the
> embedded programs section as an additional load path
> for libraries and modules as well as the main function.

Couldn't your interpreter be a library (shared or static)
and then all that a programmer would need to do is to
include your library along with their code when they perform
a link of their own ?

Cheers
   Nick

Re: Appending program headers to existing ELFs?

[Fangrui Song]

I think "error: could not allocate headers" means that you need to set
the location counter to an appropriate address.
In the internal linker script, it's something like   . =
SEGMENT_START("text-segment", 0) + SIZEOF_HEADERS;

For your original feature request, I wonder whether it should be the
responsibility of the post-link tool.
For example, patchelf supports adding a new program header. This is
done by appending the program headers to the end of the object file,
then changing e_phoff in the ELF header.
The original program headers immediately after the ELF header (added
by the linker) are then unused, but that is a small amount of waste.

On Thu, Nov 16, 2023 at 12:51 AM Nick Clifton <nickc@redhat.com> wrote:
>
> Hi Matheus,
>
> > In concrete terms:
> >
> > I'm building a small programming language interpreter
> > and would like to add support for embedding programs
> > into the executable. My intention is to treat the
> > embedded programs section as an additional load path
> > for libraries and modules as well as the main function.
>
> Couldn't your interpreter be a library (shared or static)
> and then all that a programmer would need to do is to
> include your library along with their code when they perform
> a link of their own ?
>
> Cheers
>    Nick
>
>