ELF LOAD segments creating holes in the process image on GNU/Linux

ELF LOAD segments creating holes in the process image on GNU/Linux

[Florian Weimer]

At least on x86-64 on GNU/Linux, for PIE and non-PIE binaries, LOAD
segments are not always contiguous.  The triggering conditions are not
quite clear to me.

For ET_EXEC (non-PIE) executables, I think this is a supported feature,
and the kernel and glibc dynamic loader handle this the same way,
creating holes.  However, these holes make dealing with the main
executable slightly more complicated in some cases (e.g., we can handle
contiguous mapping slightly more efficiently when looking objects based
on addresses).  Holes also make page tables less compact (although that
effect is probably minuscule).  Abstractly, I don't see a reason why the
link editor has to create such holes on the most common targets.  (I
have heard that ia64 really needed those holes.)

For ET_DYN (PIE) executables, we have a Linux vs glibc difference: The
Linux loader creates holes as well, while the glibc loader refuses to
create them and fills them with a PROT_NONE mapping from the executable.

Is this something we could fix in ld?

Thanks,
Florian

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[H.J. Lu]

On Mon, Jan 3, 2022 at 4:31 AM Florian Weimer via Binutils
<binutils@sourceware.org> wrote:
>
> At least on x86-64 on GNU/Linux, for PIE and non-PIE binaries, LOAD
> segments are not always contiguous.  The triggering conditions are not
> quite clear to me.
>
> For ET_EXEC (non-PIE) executables, I think this is a supported feature,
> and the kernel and glibc dynamic loader handle this the same way,
> creating holes.  However, these holes make dealing with the main
> executable slightly more complicated in some cases (e.g., we can handle
> contiguous mapping slightly more efficiently when looking objects based
> on addresses).  Holes also make page tables less compact (although that
> effect is probably minuscule).  Abstractly, I don't see a reason why the
> link editor has to create such holes on the most common targets.  (I
> have heard that ia64 really needed those holes.)
>
> For ET_DYN (PIE) executables, we have a Linux vs glibc difference: The
> Linux loader creates holes as well, while the glibc loader refuses to
> create them and fills them with a PROT_NONE mapping from the executable.
>
> Is this something we could fix in ld?
>

Is it created by -z separate-code?

-- 
H.J.

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[Florian Weimer]

* H. J. Lu:

> On Mon, Jan 3, 2022 at 4:31 AM Florian Weimer via Binutils
> <binutils@sourceware.org> wrote:
>>
>> At least on x86-64 on GNU/Linux, for PIE and non-PIE binaries, LOAD
>> segments are not always contiguous.  The triggering conditions are not
>> quite clear to me.
>>
>> For ET_EXEC (non-PIE) executables, I think this is a supported feature,
>> and the kernel and glibc dynamic loader handle this the same way,
>> creating holes.  However, these holes make dealing with the main
>> executable slightly more complicated in some cases (e.g., we can handle
>> contiguous mapping slightly more efficiently when looking objects based
>> on addresses).  Holes also make page tables less compact (although that
>> effect is probably minuscule).  Abstractly, I don't see a reason why the
>> link editor has to create such holes on the most common targets.  (I
>> have heard that ia64 really needed those holes.)
>>
>> For ET_DYN (PIE) executables, we have a Linux vs glibc difference: The
>> Linux loader creates holes as well, while the glibc loader refuses to
>> create them and fills them with a PROT_NONE mapping from the executable.
>>
>> Is this something we could fix in ld?
>>
>
> Is it created by -z separate-code?

I believe so because it is the default?

Thanks,
Florian

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[H.J. Lu]

On Mon, Jan 3, 2022 at 4:49 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * H. J. Lu:
>
> > On Mon, Jan 3, 2022 at 4:31 AM Florian Weimer via Binutils
> > <binutils@sourceware.org> wrote:
> >>
> >> At least on x86-64 on GNU/Linux, for PIE and non-PIE binaries, LOAD
> >> segments are not always contiguous.  The triggering conditions are not
> >> quite clear to me.
> >>
> >> For ET_EXEC (non-PIE) executables, I think this is a supported feature,
> >> and the kernel and glibc dynamic loader handle this the same way,
> >> creating holes.  However, these holes make dealing with the main
> >> executable slightly more complicated in some cases (e.g., we can handle
> >> contiguous mapping slightly more efficiently when looking objects based
> >> on addresses).  Holes also make page tables less compact (although that
> >> effect is probably minuscule).  Abstractly, I don't see a reason why the
> >> link editor has to create such holes on the most common targets.  (I
> >> have heard that ia64 really needed those holes.)
> >>
> >> For ET_DYN (PIE) executables, we have a Linux vs glibc difference: The
> >> Linux loader creates holes as well, while the glibc loader refuses to
> >> create them and fills them with a PROT_NONE mapping from the executable.
> >>
> >> Is this something we could fix in ld?
> >>
> >
> > Is it created by -z separate-code?
>
> I believe so because it is the default?
>

What do you want linker to do?

-- 
H.J.

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[Florian Weimer]

* H. J. Lu:

> On Mon, Jan 3, 2022 at 4:49 AM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * H. J. Lu:
>>
>> > On Mon, Jan 3, 2022 at 4:31 AM Florian Weimer via Binutils
>> > <binutils@sourceware.org> wrote:
>> >>
>> >> At least on x86-64 on GNU/Linux, for PIE and non-PIE binaries, LOAD
>> >> segments are not always contiguous.  The triggering conditions are not
>> >> quite clear to me.
>> >>
>> >> For ET_EXEC (non-PIE) executables, I think this is a supported feature,
>> >> and the kernel and glibc dynamic loader handle this the same way,
>> >> creating holes.  However, these holes make dealing with the main
>> >> executable slightly more complicated in some cases (e.g., we can handle
>> >> contiguous mapping slightly more efficiently when looking objects based
>> >> on addresses).  Holes also make page tables less compact (although that
>> >> effect is probably minuscule).  Abstractly, I don't see a reason why the
>> >> link editor has to create such holes on the most common targets.  (I
>> >> have heard that ia64 really needed those holes.)
>> >>
>> >> For ET_DYN (PIE) executables, we have a Linux vs glibc difference: The
>> >> Linux loader creates holes as well, while the glibc loader refuses to
>> >> create them and fills them with a PROT_NONE mapping from the executable.
>> >>
>> >> Is this something we could fix in ld?
>> >>
>> >
>> > Is it created by -z separate-code?
>>
>> I believe so because it is the default?

> What do you want linker to do?

Ideally?  p_offset, p_vaddr, p_filesz, p_memsz should all be multiples
of the page size, to closer reflect what can actually be loaded.

Then, for each LOAD segment, p_vaddr plus p_memsz should be equal to
p_vaddr of the next LOAD segment.

Thanks,
Floriam

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[H.J. Lu]

On Mon, Jan 3, 2022 at 5:14 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * H. J. Lu:
>
> > On Mon, Jan 3, 2022 at 4:49 AM Florian Weimer <fweimer@redhat.com> wrote:
> >>
> >> * H. J. Lu:
> >>
> >> > On Mon, Jan 3, 2022 at 4:31 AM Florian Weimer via Binutils
> >> > <binutils@sourceware.org> wrote:
> >> >>
> >> >> At least on x86-64 on GNU/Linux, for PIE and non-PIE binaries, LOAD
> >> >> segments are not always contiguous.  The triggering conditions are not
> >> >> quite clear to me.
> >> >>
> >> >> For ET_EXEC (non-PIE) executables, I think this is a supported feature,
> >> >> and the kernel and glibc dynamic loader handle this the same way,
> >> >> creating holes.  However, these holes make dealing with the main
> >> >> executable slightly more complicated in some cases (e.g., we can handle
> >> >> contiguous mapping slightly more efficiently when looking objects based
> >> >> on addresses).  Holes also make page tables less compact (although that
> >> >> effect is probably minuscule).  Abstractly, I don't see a reason why the
> >> >> link editor has to create such holes on the most common targets.  (I
> >> >> have heard that ia64 really needed those holes.)
> >> >>
> >> >> For ET_DYN (PIE) executables, we have a Linux vs glibc difference: The
> >> >> Linux loader creates holes as well, while the glibc loader refuses to
> >> >> create them and fills them with a PROT_NONE mapping from the executable.
> >> >>
> >> >> Is this something we could fix in ld?
> >> >>
> >> >
> >> > Is it created by -z separate-code?
> >>
> >> I believe so because it is the default?
>
> > What do you want linker to do?
>
> Ideally?  p_offset, p_vaddr, p_filesz, p_memsz should all be multiples
> of the page size, to closer reflect what can actually be loaded.
>
> Then, for each LOAD segment, p_vaddr plus p_memsz should be equal to
> p_vaddr of the next LOAD segment.

Filled with zeros?

-- 
H.J.

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[Florian Weimer]

* H. J. Lu:

>> Ideally?  p_offset, p_vaddr, p_filesz, p_memsz should all be multiples
>> of the page size, to closer reflect what can actually be loaded.
>>
>> Then, for each LOAD segment, p_vaddr plus p_memsz should be equal to
>> p_vaddr of the next LOAD segment.
>
> Filled with zeros?

No, with file contents.  We can't zero-fill at a sub-page granularity.
There shouldn't be page-sized holes, definitely not for 4K-only targets.

Thanks,
Florian

Re: ELF LOAD segments creating holes in the process image on GNU/Linux

[H.J. Lu]

On Mon, Jan 3, 2022 at 5:23 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * H. J. Lu:
>
> >> Ideally?  p_offset, p_vaddr, p_filesz, p_memsz should all be multiples
> >> of the page size, to closer reflect what can actually be loaded.
> >>
> >> Then, for each LOAD segment, p_vaddr plus p_memsz should be equal to
> >> p_vaddr of the next LOAD segment.
> >
> > Filled with zeros?
>
> No, with file contents.  We can't zero-fill at a sub-page granularity.
> There shouldn't be page-sized holes, definitely not for 4K-only targets.

Do you mean padding these segments with zeros in file?


-- 
H.J.