Re: RFC: Program Properties

["H.J. Lu" <hjl.tools@gmail.com>]

On Tue, Oct 18, 2016 at 8:16 AM, Nick Clifton <nickc@redhat.com> wrote:
> Hi Guys,
>
>   It looks like we are all working towards a common goal, which is good.
>
>   H.J. - I like your program property scheme, especially the idea of having
>   a lightweight, allocatable section that can be quickly parsed by the loader.
>
>   I assume that the NT_GNU_PROPERTY_TYPE_0 note type also serves as a version
>   indicator ?  Ie future versions of the specification would use a different
>   value to indicate newer features ?

Each property note entry, identified by note type, supports up to 4G
different properties.  I only defined NT_GNU_PROPERTY_TYPE_0.
We can add another one if we run out of available properties.

>   One thing that I would like to propose is an extension to the scheme to add
>   a second, larger note section that is not allocatable, but which instead
>   contains information to be parsed by static tools.  In particular this
>   section would contain information about the tools used to build the binary
>   and the security features enabled (or disabled).  Also this section would
>   be able to discriminate this information on a per-symbol basis if necessary,
>   so that multiple, conflicting properties can be recorded for a single file.
>
>   I have a preliminary proposal to implement this second section (see below)
>   and I would be very interested in any thoughts that you might have.
>
> Cheers
>   Nick
>
> The purpose of this non-allocatable note section is to provide a way for
> package maintainers and distributions to answer questions about the binaries
> in their distribution.  Especially security related questions.  Here is the
> current, preliminary, specification:
>
> * The information is stored in a new section in the file using the ELF
>   NOTE format.  Creator tools (compilers, assemblers etc) place the notes
>   into the binary files.  Consumer tools (none written yet, but readelf
>   and/or objdump could be enhanced for this purpose) read the notes and
>   answer questions about the binaries concerned.  Static linkers need
>   special care to handle merging of the notes.
>
> * The information is stored in a section called .gnu.build.attributes.
>   (The name can be changed - it is basically irrelevant anyway, it is
>   the new section flag (defined below) that matters).   The section has
>   the SHT_NOTE type and a new section flag set: SHF_GNU_BUILD_ATTRIBUTES.
>   (Suggested value: 0x00100000).  This ndicates the special needs when
>   merging notes (see below).
>
>   The sh_link field should be set to contain the index of symbol table
>   section.  If this field is 0 then the consumer should assume that
>   the first section of type SHT_SYMTAB in the section headers is
>   symbol table being used.

It doesn't sound like a note section, whose content should be
described by the note type.  Will a new section type work
better?

> * The specification breaks the name/description convention of ELF
>   notes to instead use a key/value/applies-to list.  (This not a
>   problem as we are only breaking a convention not a requirement of
>   the ELF NOTE specification).  The type of the note is the key.  The
>   name of the note is the value and the description field is the
>   applies-to list.
>
>   By default the description field contains the filename of the source
>   file that was used to produce the binary.  (FIXME: Absolute pathname
>   ?  Relative pathname ?  Just the filename with no path ?)  This
>   indicates that the key/value pair applies to all symbols in the
>   file.  The length of this string must *not* be multiple of 4 (with
>   the terminating NUL byte included).  If necessary the filename
>   should be padded with an extra NUL byte.  (Note - this padding byte
>   is separate from the padding bytes used to align the description
>   field to its normal boundary).
>
>   This restriction is so that a description containing symbol names
>   (see below) can be distinguished from a description containing a
>   file name.
>
>   If a key/value pair applies to just some of the symbols in a file,
>   then the description instead contains a list of 4-byte or 8-byte wide
>   numbers.  These are indices into the symbol table, (pointed to by
>   the sh_link field of the section header).  Notes:
>
>     + In unrelocated files the offset should instead be zero, with a
>       relocation present to set the actual value once the file is
>       linked.  FIXME: Unable to implement at the moment.  Instead
>       the relocation generated by the assembler evaluates to *value*
>       of the symbol not its index in the ELF symbol table section.
>       May have to change this spec if I cannot find a way around
>       this.
>
>     + The numbers are stored in the same endian format as that
>       specified in the EI_DATA field of the ELF header of the file
>       containing the note.
>
>     + The symbol table is indexed rather than the string table because
>       consumers are most likely to be interested the symbol as a
>       whole, not just its name.  (FIXME: Is this true ?)
>
>   An empty description field is a special case.  It should be treated
>   as if it had the same filename as the nearest preceding version
>   note.  (See NT_GNU_BUILD_ATTRIBUTE_VERSION below).  FIXME: This
>   assumes that a linker will preserve the order of notes when
>   linking.  Does this actually happen ?
>
>   Multiple notes of the same key can exist, providing that they have
>   different values and that their applies-to lists do not intersect.
>   (FIXME: is this restriction necessary ?  Perhaps there are times when
>   a symbol can have multiple values for the same key).
>
>   Where notes for the same key exist in both symbol index form and
>   filename form, the symbol index form takes precedence.  Any symbol
>   in the given file not explicitly indexed by one of the notes will
>   take its value from the note using the filename form.
>
>   At most one note for a given key can exist containing a filename
>   rather than symbol indices.  If this rule is broken then this
>   indicates that the file has been created by a linker that has not
>   been enhanced to support this specification.  In such cases all
>   notes containing symbol indices should be ignored.
>
>
> * When the linker merges two or more files containing these notes it
>   should ensure that the above rules are maintained, and that the
>   notes are merged appropriately.
>
>   The linker will create a new version note (see the definition of
>   NT_GNU_BUILD_ATTRIBUTE_VERSION below), with the output filename as
>   its description, and the name set to any version of this
>   specification that it chooses.  Any input version notes that match
>   this version are discarded.  Other version notes are preserved and
>   included in the output file.
>
>   When notes are merged the following rules apply:
>
>    1. If all input notes of a given type just contain filenames and
>       they all have the same value string then a single output note is
>       created with this type/value and the output filename as its
>       description.  Otherwise:
>
>    2. If rule 1 would match except for one or more symbol containing
>       notes then rule 1 is executed, but the symbol containing notes
>       are also preserved and copied to the output.  If this is a
>       relocatable link then the relocations associated with the symbol
>       indices should also be updated.  Otherwise:
>
>   3.  [This rule triggers if there are filename containing notes with
>       different value strings].  The linker chooses one of the input
>       value strings to be the default for the output and creates an
>       output note using this value.  (Presumably the linker will
>       choose the value with the most matching input files).  Input
>       notes containing filenames but with a value that does not match
>       this output value must be converted into symbol containing notes
>       listing *all* of the symbols in the input file.  Failure to do
>       this breaks the requirement that there only be one filename
>       containing output note for the given key.
>
>   If this is a final link, then relocations on the notes should of
>   course be resolved.
>
>   The linker is also able to create and insert its own notes.  Eg to
>   indicate that -z relro is enabled.
>
>   Linkers that have not been enhanced to support this proposal will
>   simply concatenate the notes.  (They may also eliminate duplicate
>   notes, although this is not guaranteed.  They may also sort the
>   notes which would break the use of empty description fields, as
>   mentioned above).  In this case the output file is likely to contain
>   multiple notes with the same key/value pair.  Consumers can detect
>   this situation by noticing that there is no
>   NT_GNU_BUILD_ATTRIBUTE_VERSION note with output file name, and hence
>   deduce that any notes containing symbol indices are broken.  (The
>   linker will not have updated the indices when merging the notes).
>   Despite only supporting a file level granularity however, these
>   notes may still prove useful.
>
>
> * Three new note types defined (so far):
>
>   Type: NT_GNU_BUILD_ATTRIBUTE_VERSION  (0x100)
>   Name: A string identifying the version of this specification
>         that is implemented in the accompanying notes.  Currently set
>         to "1.0".
>
>   Type: NT_GNU_BUILD_ATTRIBUTE_CREATOR  (0x101)
>   Name: A string identifying the tool that created the symbols and
>         their associated code eg:
>         "gcc (GCC) 6.2.1 20160916 (Red Hat 6.2.1-2)"
>         includes name, date and version.
>
>   Type: NT_GNU_BUILD_ATTRIBUTE_OPTIONS  (0x102)
>   Name: A string identifying the *significant* compile time options
>         affecting the specified symbols.  Ie those that affect ABI,
>         security, etc.
>
>         Note: selection of *significant* compile time options may be
>         subject to debate.  But the actual choice can vary over time,
>         this does not affect the current proposal.
>



-- 
H.J.