Re: RFC: Program Properties

[Suprateeka R Hegde <hegdesmailbox@gmail.com>]

H.J,

IMHO, this looks too huge to apprehend in one discussion. Would you 
consider:

1. Splitting into multiple (at least 3) logical groups based on use cases?
2. If possible, a one-liner problem statement or inefficiencies in the 
existing gABI or GNU extensions?

(Just trying to streamline and avoid a turbulent discussion)

--
Supra



On 31-Mar-2016 08:32 PM, H.J. Lu wrote:
> Here is the updated proposal.  The main changes are
>
> 1. Remove SHF_GNU_PROPERTIES and PT_GNU_PROPERTIES.
> 2. Add NT_GNU_PROPERTIES_NO_COPY_ON_PROTECTED to
> prevent copy relocation against protected data symbol.
>
>
> H.J.
> ---
> Program Properties
>
> There are cases where linker and run-time loader need more information
> about ELF objects beyond what the current gABI provides:
>
> 1. Minimum ISAs.  Executables and shared objects, which are optimized
> specifically to run on a particular processor, will not run on processors
> which don't support the same set of ISAs.  Since x86 only has EM_IAMCU,
> EM_386 and EM_X86_64 ELF machine codes, run-time loader needs additional
> information to tell if an executable or a shared object is compatible
> with available ISAs.
> 2. Stack size.  Compilers may generate binaries which require larger stack
> size than normal.  If run-time loader can query the stack size required
> by executable or shared object, it can increase stack size as needed.
> 3. Additional linking command line options embedded in relocatable object
> files:
>     a. Additional libraries should be linked in when creating executable
>        or shared object.
>        http://sourceware.org/bugzilla/show_bug.cgi?id=12485
>     b. Additional compiler command line options are needed to properly
>        link LTO objects.
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41756
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=47785
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54231
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53777
> 4. Relocatable object files compiled against header files from a specific
> version of a library must be linked with a compatible library when
> creating executable or shared object.
> 5. Copy relocation and protected visibility are fundamentally incompatible.
> On on hand, copy relocation is the part of the psABI and is used to
> access global data defined in a shared object from the executable.  It
> moves the definition of global data, which is defined in a share object,
> to the executable at run-time.  On the other hand, protected visibility
> indicates that a symbol is defined locally in the shared object at
> run-time.  Both can't be true at the same time.  The current solution
> is to make protected symbol more or less like normal symbol, which
> prevents optimizing local access to protected symbol within the shared
> object.
>
> GNU attributes
>
> GNU binutils supports build attribute and run-time platform compatibility
> data in relocatable object files.  Issues with GNU attributes:
>
> 1. Many architectures, including x86, don't support GNU attributes:
>     a. There are some overlaps between GNU attributes and program
>     properties.
>     b. GNU attributes may be extended to embed additional linking command
>     line options in relocatable object files.
> 2. On x86, linking a relocatable object full of AVX instructions doesn't
> always make the resulting executables or shared libraries to require AVX
> to run since AVX functions may be called only via GNU_IFUNC at run-time.
> Linker can't set minimum ISAs just from ISAs used by input relocatable
> objects.
> 3. There is no program segment for GNU attributes in executables and
> shared objects.
> 4. Most of attributes aren't applicable to run-time loader.
> 5. The format of GNU attributes isn't optimal for run-time loader.  A
> separate string table is used to store string attributes.
>
> gABI support for program properties
>
> To the "Special Sections" section, add:
>
>       Name                Type                 Attributes
> .note.GNU-properties    SHT_NOTE              SHF_ALLOC
>
> A .note.GNU-properties section contains a property note descriptor,
> starting with a property note descriptor header and followed by an array
> of properties.  It can be merged with other SHT_NOTE sections.  The
> property note descriptor header has the following structure:
>
> typedef struct {
>    Elf_Word namsz;
>    Elf_Word descsz;
>    Elf_Word type;
>    unsigned char name[4];
> } Elf_GNU_Notehdr;
>
> 1. namesz is 4.
> 2. descsz contains the size of the property array.
> 3. type specifies the property type:
>
> #define NT_GNU_PROPERTIES_TYPE_0   5
>
> 4. name is a null-terminated character string. It should be "GNU".
>
> Each array element represents one property with type, data size and data.
> In 64-bit objects, each element is an array of 8-byte words, whose first
> element is 4-byte type and data size, in the format of the target processor.
> In 32-bit objects, each element is an array of 4-byte words, whose first 2
> elements are 4-byte type and data size, in the format of the target
> processor.  An array element has the following structure:
>
> typedef struct {
>    Elf_Word pr_type;
>    Elf_Word pr_datasz
>    unsigned char pr_data[PR_DATASZ];
>    unsigned char pr_padding[PR_PADDING];
> } Elf_Prop;
>
> where PR_DATASZ is the data size and PR_PADDING, if necessary, aligns
> array element to 8 or 4-byte alignment (depending on whether the file
> is a 64-bit or 32-bit object).  The array elements are sorted by the
> property type.  The interpretation of property array depends on both
> ph_kind and pr_type.
>
> Types of program properties
>
> #define NT_GNU_PROPERTIES_LOPROC        0xb0000000
> #define NT_GNU_PROPERTIES_HIPROC        0xdfffffff
> #define NT_GNU_PROPERTIES_LOUSER        0xe0000000
> #define NT_GNU_PROPERTIES_HIUSER        0xffffffff
>
> Proposed properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]>                            /export/gnu/import/git/properties
> gnu-6:pts/11[293]> cat properties.txt         /export/gnu/import/git/properties
> Program Properties
>
> There are cases where linker and run-time loader need more information
> about ELF objects beyond what the current gABI provides:
>
> 1. Minimum ISAs.  Executables and shared objects, which are optimized
> specifically to run on a particular processor, will not run on processors
> which don't support the same set of ISAs.  Since x86 only has EM_IAMCU,
> EM_386 and EM_X86_64 ELF machine codes, run-time loader needs additional
> information to tell if an executable or a shared object is compatible
> with available ISAs.
> 2. Stack size.  Compilers may generate binaries which require larger stack
> size than normal.  If run-time loader can query the stack size required
> by executable or shared object, it can increase stack size as needed.
> 3. Additional linking command line options embedded in relocatable object
> files:
>     a. Additional libraries should be linked in when creating executable
>        or shared object.
>        http://sourceware.org/bugzilla/show_bug.cgi?id=12485
>     b. Additional compiler command line options are needed to properly
>        link LTO objects.
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41756
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=47785
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54231
>        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53777
> 4. Relocatable object files compiled against header files from a specific
> version of a library must be linked with a compatible library when
> creating executable or shared object.
> 5. Copy relocation and protected visibility are fundamentally incompatible.
> On on hand, copy relocation is the part of the psABI and is used to
> access global data defined in a shared object from the executable.  It
> moves the definition of global data, which is defined in a share object,
> to the executable at run-time.  On the other hand, protected visibility
> indicates that a symbol is defined locally in the shared object at
> run-time.  Both can't be true at the same time.  The current solution
> is to make protected symbol more or less like normal symbol, which
> prevents optimizing local access to protected symbol within the shared
> object.
>
> GNU attributes
>
> GNU binutils supports build attribute and run-time platform compatibility
> data in relocatable object files.  Issues with GNU attributes:
>
> 1. Many architectures, including x86, don't support GNU attributes:
>     a. There are some overlaps between GNU attributes and program
>     properties.
>     b. GNU attributes may be extended to embed additional linking command
>     line options in relocatable object files.
> 2. On x86, linking a relocatable object full of AVX instructions doesn't
> always make the resulting executables or shared libraries to require AVX
> to run since AVX functions may be called only via GNU_IFUNC at run-time.
> Linker can't set minimum ISAs just from ISAs used by input relocatable
> objects.
> 3. There is no program segment for GNU attributes in executables and
> shared objects.
> 4. Most of attributes aren't applicable to run-time loader.
> 5. The format of GNU attributes isn't optimal for run-time loader.  A
> separate string table is used to store string attributes.
>
> gABI support for program properties
>
> To the "Special Sections" section, add:
>
>       Name                Type                 Attributes
> .note.GNU-properties    SHT_NOTE              SHF_ALLOC
>
> A .note.GNU-properties section contains a property note descriptor,
> starting with a property note descriptor header and followed by an array
> of properties.  It can be merged with other SHT_NOTE sections.  The
> property note descriptor header has the following structure:
>
> typedef struct {
>    Elf_Word namsz;
>    Elf_Word descsz;
>    Elf_Word type;
>    unsigned char name[4];
> } Elf_GNU_Notehdr;
>
> 1. namesz is 4.
> 2. descsz contains the size of the property array.
> 3. type specifies the property type:
>
> #define NT_GNU_PROPERTIES_TYPE_0   5
>
> 4. name is a null-terminated character string. It should be "GNU".
>
> Each array element represents one property with type, data size and data.
> In 64-bit objects, each element is an array of 8-byte words, whose first
> element is 4-byte type and data size, in the format of the target processor.
> In 32-bit objects, each element is an array of 4-byte words, whose first 2
> elements are 4-byte type and data size, in the format of the target
> processor.  An array element has the following structure:
>
> typedef struct {
>    Elf_Word pr_type;
>    Elf_Word pr_datasz
>    unsigned char pr_data[PR_DATASZ];
>    unsigned char pr_padding[PR_PADDING];
> } Elf_Prop;
>
> where PR_DATASZ is the data size and PR_PADDING, if necessary, aligns
> array element to 8 or 4-byte alignment (depending on whether the file
> is a 64-bit or 32-bit object).  The array elements are sorted by the
> property type.  The interpretation of property array depends on both
> ph_kind and pr_type.
>
> Types of program properties
>
> #define NT_GNU_PROPERTIES_LOPROC        0xb0000000
> #define NT_GNU_PROPERTIES_HIPROC        0xdfffffff
> #define NT_GNU_PROPERTIES_LOUSER        0xe0000000
> #define NT_GNU_PROPERTIES_HIUSER        0xffffffff
>
> Proposed properties
>
> For NT_GNU_PROPERTIES_TYPE_0:
>
> #define NT_GNU_PROPERTIES_STACK_SIZE              1
>
> Integer value for minimum stack size.
>
> #define NT_GNU_PROPERTIES_REQUIRED_LIBRARY        2
>
> String of the required library, NAMESPEC:SONAME.  NAMESPEC is the string
> after the linker -l option.  SONAME is DT_SONAME of the required library.
>
> #define NT_GNU_PROPERTIES_PROVIDED_LIBRARY        3
>
> String of the provided library, NAMESPEC:SONAME.  NAMESPEC is the string
> after the linker -l option.  SONAME is DT_SONAME of the provided library.
>
> There is a conflict in linker inputs if
>
> 1. Any NT_GNU_PROPERTIES_REQUIRED_LIBRARY or
> NT_GNU_PROPERTIES_PROVIDED_LIBRARY have the same NAMESPEC, but different
> SONAMEs.
> 2. NAMESPEC in any NT_GNU_PROPERTIES_REQUIRED_LIBRARY matches an input
> libNAMESPEC.so whose DT_SONAME is different from SONAME.
>
> #define NT_GNU_PROPERTIES_GCC_OPTIONS             4
>
> GCC options must be passed to GCC when used with GCC.
>
> #define NT_GNU_PROPERTIES_NO_COPY_ON_PROTECTED    5
>
> This indicates that there should be no copy relocations against protected
> data symbols.  If a relocatable object contains this property, linker
> should treat protected data symbol as defined locally at run-time and copy
> this property to the output share object.  Run-time loader should disallow
> copy relocations against protected data symbols defined in share objects
> with NT_GNU_PROPERTIES_NO_COPY_ON_PROTECTED property.
>
> #define NT_GNU_PROPERTIES_X86_ISA_1_USED          0xc0000000
>
> The x86 instruction sets indicated by the corresponding bits are used
> in program.  But their support in the hardware is optional.
>
> #define NT_GNU_PROPERTIES_X86_ISA_1_NEEDED        0xc0000001
>
> The x86 instruction sets indicated by the corresponding bits are used
> in program and they must be supported by the hardware.  A bit set in
> NT_GNU_PROPERTIES_X86_ISA_1_NEEDED must also be set in
> NT_GNU_PROPERTIES_X86_ISA_1_USED.
>
> Integer value for the x86 instruction set support.
>
> #define NT_GNU_PROPERTIES_X86_ISA_1_486           (1U << 0)
> #define NT_GNU_PROPERTIES_X86_ISA_1_586           (1U << 1)
> #define NT_GNU_PROPERTIES_X86_ISA_1_686           (1U << 2)
> #define NT_GNU_PROPERTIES_X86_ISA_1_SSE           (1U << 3)
> #define NT_GNU_PROPERTIES_X86_ISA_1_SSE2          (1U << 4)
> #define NT_GNU_PROPERTIES_X86_ISA_1_SSE3          (1U << 5)
> #define NT_GNU_PROPERTIES_X86_ISA_1_SSSE3         (1U << 6)
> #define NT_GNU_PROPERTIES_X86_ISA_1_SSE4_1        (1U << 7)
> #define NT_GNU_PROPERTIES_X86_ISA_1_SSE4_2        (1U << 8)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX           (1U << 9)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX2          (1U << 10)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512F       (1U << 11)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512CD      (1U << 12)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512ER      (1U << 13)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512PF      (1U << 14)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512VL      (1U << 15)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512DQ      (1U << 16)
> #define NT_GNU_PROPERTIES_X86_ISA_1_AVX512BW      (1U << 17)
>