Re: [RFC] SHF_GNU_RETAIN ELF Section Flag

[Jozef Lawrynowicz <jozef.l@mittosystems.com>]

On Tue, Sep 15, 2020 at 10:11:30AM -0400, Carlos O'Donell wrote:
> On 9/15/20 9:29 AM, Jozef Lawrynowicz wrote:
> > On Tue, Sep 15, 2020 at 02:55:05PM +0200, Florian Weimer wrote:
> >> * Carlos O'Donell:
> >>
> >>> On 9/15/20 8:37 AM, Florian Weimer via Gnu-gabi wrote:
> >>>> * Jozef Lawrynowicz:
> >>>>
> >>>>> On Tue, Sep 15, 2020 at 02:09:22PM +0200, Florian Weimer wrote:
> >>>>>> * Jozef Lawrynowicz:
> >>>>>>
> >>>>>>> I'd like to propose a new ELF section flag, SHF_GNU_RETAIN, for addition
> >>>>>>> to the GNU gABI.
> >>>>>>>
> >>>>>>> This flag instructs the linker to "retain" the section in the output
> >>>>>>> file, even if garbage collection would remove it because it appears
> >>>>>>> unused.
> >>>>>>
> >>>>>> How does this flag interaction with libraries (.a files)?
> >>>>>
> >>>>> If a section in a library has SHF_GNU_RETAIN set, and that library gets
> >>>>> searched by the linker for some undefined symbol, then the
> >>>>> SHF_GNU_RETAIN section will also be pulled into the program, and
> >>>>> retained in the linked output file.
> >>>>
> >>>> Sorry, that's not quite what I meant.  What happens if the .o file with
> >>>> SHF_GNU_RETAIN in an .a library is not otherwise referenced and thus
> >>>> never loaded by the link editor?  How would the link editor realize that
> >>>> it is even there?
> >>>
> >>> Why would it be loaded?
> >>
> >> Hypothetically: Because the ranlib section tells the link editor to load
> >> it (so more specification updates are needed).
> >>
> > 
> > An SHF_GNU_RETAIN section would only be kept if it's containing object
> > was loaded in the first place, and the section was therefore considered for
> > garbage collection. So no, SHF_GNU_RETAIN is not intended to be used to
> > force inclusion of sections which the linker would not have otherwise
> > seen.
> > SHF_GNU_RETAIN can be thought of to essentially "turn off" garbage
> > collection for that section, rather than change the fundamental linking
> > behavior for that section or containing object.
> > 
> > Carlos' ammendment to the definition is accurate:
> > 
> >> SHF_GNU_RETAIN
> >>   When an object file containing such a marked section is included in
> >>   the link, the section should not be garbage collected by the linker,
> >>   even if it appears unused.
> > 
> >>> Why does the link editor need to detect the presence of such a file?
> >>
> >> To make SHF_GNU_RETAIN work with libraries.
> >>
> >> I think without that, the same effect can be had today with
> >> SHF_GROUP/SHT_GROUP, perhaps with an assembler-only change to implement
> >> the .retain pseudo.
> >>
> > 
> > Perhaps, but without making any further extensions, wouldn't the
> > assembler need to know of a section which will definitiley be kept in
> > the output file? Only the linker can truly know this, by looking at the
> > entry point (when there is one).
> > 
> > A new bit in GRP_MASKOS could define that sections in a group with this
> > flag must always be kept, but that seems like a more round about way of
> > using a new section flag.
> 
> Florian's point here, and let me reiterate it to see if I understood it,
> is that SHF_GROUP / SHT_GROUP is the right mechanic here because:
> 
> (a) *Something* needs the section that would otherwise have been garbage
>     collected.
> 
> (b) Expressing the "depends on" relationship could be achieved with
>     a SHF_GROUP / SHT_GROUP and a new bit GRP_DEPENDS to indicate that
>     a group of sections depend upon eachother (k-connected dependency).
> 
> Then the linker during garbage collection must either be able to discard
> all sections in the group or none of them.
> 
> The underlying idea here is that SHF_GNU_RETAIN is really an expression
> of "depended upon by something" with no further information about the
> dependee or other related dependents.

I suppose the most compelling use cases for SHF_GNU_RETAIN are when the
dependency cannot be expressed with references to ELF sections. You
can't use SHF_GROUP because there is nothing to group the section with.

Consider when it is in fact the hardware that depends on the
SHF_GNU_RETAIN section:
- Interrupt vector table
- Bootloader code
- Memory mapped registers

These use cases perhaps make more sense when paired another flag I was
going to propose, which enables the setting of a section's VMA from the
source code.

So a user could set the value of a memory mapped register directly from
the source code, without any help from the linker script.

I realize these use cases are focussed towards embedded
microcontrollers, but there are many different processors, following
different processor-specific ABIs which fall under this category.
Maybe I should just work towards getting this functionality added for
ARM, and then whatever other targets want to use it can just piggyback
off that...

Thanks,
Jozef
> 
> How would "depends on" (GRP_DEPENDS) be expressed to the developer?
> 
> They would have to put code, and data, and other things into the this
> new group to make a collection of things that depend upon eachother
> in some non-"symbol dependency" way.
> 
> In the end you have to define the collection of things that would go
> into the section group.
> 
> -- 
> Cheers,
> Carlos.
>