Re: ABI document

["Vivek Das Mohapatra" <vivek@collabora.com>]

[-- Attachment #1: Type: text/plain, Size: 3895 bytes --]

This patch is cumulative with the last one.

> I can't tell whether the formatting is okay, and whether we need to add
> markup (like ``) for program text.

I asked and was told it was more important to get the actual
content agreed before we worried about that (and tbh the
content is the hard part).

> Can you dig out a link to a document that describes for the format of
> the GNU_EH_FRAME segment?

Done.

> I think this should also say that the virtual address range must be
> covered by an earlier PT_LOAD segment.

Done.

>> +PT_GNU_STACK     0x6474e551
>> +  Otherwise the stack is executable (the default).
>
> The default depends on the architecture, I think.

Fixed.

> I think we have differing behavior in regards to the size of the
> segment.  glibc ignores it, other implementations may use it to set the
> stack size.

Annotated.

>> +PT_GNU_RELRO     0x6474e552
>> +
>> +  The specified segment should be made read-only once run-time linking
>> +  has completed.
>
> I think it's relocation of this object, not the entire linking
> operation.

Fixed.

> Also we should try to sketch the interaction with PT_LOAD.

Not done yet.

>> +PT_GNU_PROPERTY  0x6474e553
>> +  some action before loading and ELF file (eg AArch64 BTI or intel CET)
>
> “Intel”

Fixed.

> The requirement could be worded better.  It must only be present if
> these features are to be enabled.

Clarified.

>> +  Properties are sorted in ascending order of pr_type;
>> +
>> +  pr_data is aligned to 4 bytes in 32-bit objects and 8 bytes in 64-bit ones.
>
> What's the overall alignment of the segment?  8 bytes on 64-bit?

Clarified. Is explicitly given by the p_align of the PT_* program header.

> This also has to say where the padding is inserted: before pr_data?
> After Elf_Prop?  I think it's the latter, and that Elf_Prop is aligned
> even if the pr_data member is absent.

Clairified. Always aligned. The padding is as laid out in the struct
given, so after pr_data.

>> +  GNU_PROPERTY_STACK_SIZE  0x1
>> +
>> +  A native format & size integer specifying the minimum stack size.
>> +  The linker should pick the highest instance of this from all relocatable
>> +  objects in the link chain and ensure the stack is at least this big.
>
> So this is an Elf*_Addr?

I guess, but explicitly defined in the doc now.

>> +  GNU_PROPERTY_NO_COPY_ON_PROTECTED 0x2
>> +  This type has a PR_DATASZ of 0.
>
> “pr_datasz field”?

Fixed.

>> +DT_GNU_PRELINKED  0x6ffffdf5
>> +
> That's not good for reproducibility (but then prelink results depend

Agreed, but that seems to be how prelinking works.

>> +DT_GNU_CONFLICTSZ 0x6ffffdf6
>> +DT_GNU_LIBLISTSZ  0x6ffffdf7
> It would be nice to add a link there to the prelink documentation.
> (There's a prelink.tex file in the sources.)

Couldn't see this - perhaps I was looking in the wrong place?


>> +DT_GNU_HASH       0x6ffffef5
> Do we have a format documentation for those?

No - added a note that this may need to be reversed from the
glibc source.

>> +DT_GNU_CONFLICT   0x6ffffef8
>> +DT_GNU_LIBLIST    0x6ffffef9
>
> Maybe group this with the earlier prelink items?

Moved.

>> +Section Headers
>> +===============
>
>> +SHT_GNU_verdef             0x6ffffffd
>> +SHT_GNU_verneed            0x6ffffffe
>> +SHT_GNU_versym             0x6fffffff
> I think the canonical reference for these is:
>
>  <https://www.akkadia.org/drepper/symbol-versioning>

Documented and references to LSB with the details given.

>> +Note section descriptors (SHT_NOTE extensions)
>> +==============================================
>
>> +NT_GNU_BUILD_ID        3
>> +
>> +  descsz bytes of build-id data.
>> +  Typically presented as a hex string.
>
> But stored in binary?

Yes.

> Maybe reference the ld documentation here, and say that the actual
> computation mechanism is unspecified?

Documented as binary+opaque, and the required unique-ness properties
noted.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-diff; name=0001-Incorporate-feedback-and-add-DOCUMENTME-tags-where-m.patch, Size: 15931 bytes --]

From a93c0212f53cd069ba46261c57a9bab289b2d466 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Vivek=20Das=C2=A0Mohapatra?= <vivek@collabora.com>
Date: Wed, 2 Sep 2020 18:15:43 +0100
Subject: [PATCH] Incorporate feedback and add DOCUMENTME tags where more info
 is needed

---
 program-loading-and-dynamic-linking.txt | 262 ++++++++++++++++++++++--
 1 file changed, 243 insertions(+), 19 deletions(-)

diff --git a/program-loading-and-dynamic-linking.txt b/program-loading-and-dynamic-linking.txt
index 751eaca..f7f4284 100644
--- a/program-loading-and-dynamic-linking.txt
+++ b/program-loading-and-dynamic-linking.txt
@@ -9,9 +9,19 @@ PT_SUNW_EH_FRAME 0x6474e550
 
   Segment contains the EH_FRAME_HDR section (stack frame unwind information)
 
+  NOTE: The virtual address range referred to by PT_GNU_EH_FRAME must be
+  covered by a separate PT_LOAD header - PT_GNU_EH_FRAME on its own does
+  not trigger the mapping/loading of any data into memory.
+
   PT_SUNW_EH_FRAME is used by a non-GNU implementation for the same purpose,
   and has the same value (although this does not imply compatible contents).
 
+  The contents of the EH_FRAME_HDR are described in the LSB. As of v5.0:
+
+  Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/ehframechpt.html
+  Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/ehframechpt.html#EHFRAMEHDR
+  Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic.html#PROGHEADER
+
 PT_GNU_STACK     0x6474e551
 
   The p_flags member of this ElfW(Phdr) structure apply to the stack.
@@ -19,22 +29,34 @@ PT_GNU_STACK     0x6474e551
   If present AND p_flags DOES NOT contain PF_X (0x1) then the stack
   should _not_ be executable.
 
-  Otherwise the stack is executable (the default).
+  Otherwise the stack follows the architecture specific default for
+  executability: For example on x86 the stack is executable by default.
+
+  NOTE: Some implementations may use this header's p_size to set the stack size.
+  glibc does NOT do this: See GNU_PROPERTY_STACK_SIZE instead.
+
+  Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic.html#PROGHEADER
 
 PT_GNU_RELRO     0x6474e552
 
   The specified segment should be made read-only once run-time linking
-  has completed.
+  of this object has completed.
+
+  DOCUMENTME: Interaction with PT_LOAD here
+
+  Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic.html#PROGHEADER
 
 PT_GNU_PROPERTY  0x6474e553
 
   The Linux kernel uses this program header to locate the
-  .note.gnu.property section.
+  ".note.gnu.property section".
 
   If there is a program property that requires the kernel to perform
-  some action before loading and ELF file (eg AArch64 BTI or intel CET)
+  some action before loading and ELF file (eg AArch64 BTI or Intel CET)
   then this header MUST be present.
 
+  If no such features are to be enabled this header MUST NOT be present.
+
   The contents are laid out as follows:
 
   Field      | Length   | Contents
@@ -53,18 +75,32 @@ PT_GNU_PROPERTY  0x6474e553
     unsigned char pr_padding[PR_PADDING];
   } Elf_Prop;
 
-  Properties are sorted in ascending order of pr_type;
-
   pr_data is aligned to 4 bytes in 32-bit objects and 8 bytes in 64-bit ones.
 
+  The segment itself is aligned according to the program header's p_align
+  field.
+
+  PR_PADDING bytes are added _after_ PR_DATASZ so that each property is
+  aligned to 4 bytes (on 32 bit architectures) and to 8 bytes on 64 bit
+  architectures.
+
+  This is true even if pr_datasz is 0 (cf GNU_PROPERTY_NO_COPY_ON_PROTECTED).
+
+  Properties are sorted in ascending order of pr_type;
+
   Defined properties are:
 
   GNU_PROPERTY_STACK_SIZE  0x1
 
-  A native format & size integer specifying the minimum stack size.
+  pr_data holds a native sized (4 bytes on 32 bit architecures,
+  8 bytes on 64 bit) integer in the target processor's native format.
+
   The linker should pick the highest instance of this from all relocatable
   objects in the link chain and ensure the stack is at least this big.
 
+  There is no implication or requirement that the linker should or will
+  reduce the stack size to match this value.
+
   GNU_PROPERTY_NO_COPY_ON_PROTECTED 0x2
 
   The linker should treat protected data symbol as defined locally at
@@ -76,7 +112,8 @@ PT_GNU_PROPERTY  0x6474e553
   The run-time loader should disallow copy relocations against protected
   data symbols defined such objects.
 
-  This type has a PR_DATASZ of 0.
+  This type is expected to have a pr_datasz field of 0, and no pr_data
+  contents (only padding).
 
   GNU_PROPERTY_LOPROC  0xc0000000
   GNU_PROPERTY_HIPROC  0xdfffffff
@@ -88,8 +125,10 @@ PT_GNU_PROPERTY  0x6474e553
 
   Reserved for application specific values.
 
+  Reference: https://raw.githubusercontent.com/wiki/hjl-tools/linux-abi/linux-abi-draft.pdf
+
 There are further extensions to p_type but currently they are all
-architecture specific and will be documented in the relevant ABIs.
+architecture specific and should be documented in the relevant ABIs.
 
 Dynamic segment extensions (PT_DYNAMIC entries)
 ===============================================
@@ -137,11 +176,18 @@ DT_GNU_FLAGS_1    0x6ffffdf4
 
   This behaviour can always be explicitly overridden by the caller of dlmopen(3).
 
+  Reference: This document is canonical.
+
+Prelinking
+==========
+
 DT_GNU_PRELINKED  0x6ffffdf5
 
   The d_val field contains a time_t value giving the UTC time at which the
   object was (pre)linked.
 
+  Reference: DOCUMENTME
+
 DT_GNU_CONFLICTSZ 0x6ffffdf6
 
   Used in prelinked objects.
@@ -152,10 +198,6 @@ DT_GNU_LIBLISTSZ  0x6ffffdf7
   Used in prelinked objects.
   d_val contains the size of the library list.
 
-DT_GNU_HASH       0x6ffffef5
-
-  The d_ptr value gives the location of the GNU style symbol hash table.
-
 DT_GNU_CONFLICT   0x6ffffef8
 
   Used in prelinked objects.
@@ -175,6 +217,23 @@ DT_GNU_LIBLIST    0x6ffffef9
   This is used to check that all required prelinked libraries are still
   present, loaded, and have the correct checksums at runtime.
 
+  typedef struct {
+    ElfW(Word) l_name;          /* Name (string table index) */
+    ElfW(Word) l_time_stamp;    /* Timestamp */
+    ElfW(Word) l_checksum;      /* Checksum */
+    ElfW(Word) l_version;       /* Interface version */
+    ElfW(Word) l_flags;         /* Flags */
+  } ElfW(Lib);
+
+Hashes
+======
+
+DT_GNU_HASH       0x6ffffef5
+
+  The d_ptr value gives the location of the GNU style symbol hash table.
+
+  DOCUMENTME: find the canonical spec for this (or reverse e from glibc source)
+
 Section Headers
 ===============
 
@@ -183,11 +242,14 @@ Typically found in ElfW(Shdr).sh_type.
 
 SHT_GNU_INCREMENTAL_INPUTS 0x6fff4700
 
-  The section should be named ".gnu_incremental_inputs".
-  Currently used for incremental linking by gold.
+  Section name: ".gnu_incremental_inputs"
+
+  Currently used internally for incremental linking by gold.
 
 SHT_GNU_ATTRIBUTES         0x6ffffff5
 
+  Section name: ".gnu.attributes"
+
   GNU specific program attributes, sh_size bytes at sh_offset into the file.
 
   The first byte is the version of the attribute spec: Currently only 'A'
@@ -200,29 +262,180 @@ SHT_GNU_ATTRIBUTES         0x6ffffff5
       - a tag byte
       - a 4 byte native integer size (including the tag byte and the size itself)
         - if the tag is 2 or 3: a LEB128 encoded value stored in the remaining space
-      - DOCUMENTME: possibly some remaining attribute bytes
+      - DOCUMENTME: some attribute bytes? reverse engineer from readelf?
 
 SHT_GNU_HASH               0x6ffffff6
 
+  Section name: ".gnu.hash" (architecture specific ABI may override this)
+
+  This section contains the GNU style hash table. See DT_GNU_HASH.
+  Currently only the MIPS architecture is known to use a different name.
+
 SHT_GNU_LIBLIST            0x6ffffff7
 
+  See DT_GNU_LIBLIST.
+
+  Section name: ".gnu.liblist"
+
+  This section should refer to a SHT_STRTAB type section via its sh_link
+  field: That strtab holds the names of the libraries listed in each
+  ElfW(Lib) struct contained in this GNU_LIBLIST section.
+
+Symbol Versioning
+=================
+
+These sections implement GNU symbol versioning.
+These sections all have the SHF_ALLOC atribute.
+
+Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic.html
+Reference: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic.html#SYMVERSION
+
 SHT_GNU_verdef             0x6ffffffd
 
+  Section name: ".gnu.version_d"
+
+  This section contains symbol version definitions.
+
+  The number of entries it contains is given by the DT_VERDEFNUM entry
+  of the Dynamic Section SHT_DYNAMIC/".dynamic".
+
+  The sh_link member of this section header (see the System V ABI) points to
+  the SHT_STRTAB section that contains the strings referenced by this section.
+
+  This section contains an array of ElfW(Verdef) structures optionally
+  followed by an array of ElfW(Verdaux) structures.
+
+  ElfW(Verdef):
+
+  typedef struct {
+    ElfW(Half) vd_version; /* 0, 1 or 2. See below                        */
+    ElfW(Half) vd_flags;   /* A flag bitfield. See below                  */
+    ElfW(Half) vd_ndx;     /* Referred to by SHT_GNU_versym. See below    */
+    ElfW(Half) vd_cnt;     /* Number of associated ElfW(Verdaux) entries  */
+    ElfW(Word) vd_hash;    /* Version name hash (per ELF Hash function)   */
+    ElfW(Word) vd_aux;     /* Offset in bytes from this ElfW(Verdef)
+                              to its ElfW(Verdaux) array                  */
+    ElfW(Word) vd_next;    /* Offset in bytes frm this ElfW(Verdef)
+                              to the next ElfW(Verdef) entry.
+                              0 for last entry.                           */
+  } ElfW(Verdef);
+
+  vd_version:
+    VER_DEF_NONE    0 // No version
+    VER_DEF_CURRENT 1 // Currrent version
+
+  vd_flags:
+    VER_FLG_BASE 0x1 // [Default] Version of the whole object
+    VER_FLG_WEAK 0x2 // Weak version identifier
+
+  vd_ndx:
+    VER_NDX_LOCAL     0      // private symbol
+    VER_NDX_GLOBAL    1      // global symbol
+    VER_NDX_LORESERVE 0xff00 // Beginning of reserved entries
+    VER_NDX_ELIMINATE 0xff01 // DOCUMENTME: Symbol should be eliminated
+
+  ElfW(Verdaux):
+
+  typedef struct {
+    ElfW(Word) vda_name; // byte offset into the strtab of the version name
+    ElfW(Word) vda_next; // byte offset from this ElfW(Verdaux) to the next
+  } ElfW(Verdaux);
+
 SHT_GNU_verneed            0x6ffffffe
 
+  Section name: ".gnu.version_r"
+
+  This section contains symbol version requirements.
+
+  The number of entries it contains is given by the DT_VERNEEDNUM entry
+  of the Dynamic Section SHT_DYNAMIC/".dynamic".
+
+  The sh_link member of this section header (see the System V ABI) points to
+  the SHT_STRTAB section that contains the strings referenced by this section.
+
+  This section contains an array of ElfW(Verneed) structures optionally
+  followed by an array of ElfW(Vernaux) structures.
+
+  ElfW(Verneed):
+
+  typedef struct {
+    ElfW(Half) vn_version; /* See below                                      */
+    ElfW(Half) vn_cnt;     /* Number of associated ElfW(Vernaux) entries     */
+    ElfW(Word) vn_file;    /* Byte offset in strtab of required DSO filename */
+    ElfW(Word) vn_aux;     /* Byte offset from this ElfW(Verneed)
+                              to its ElfW(Vernaux) array                     */
+    ElfW(Word) vn_next;    /* Byte offset from this ElfW(Verneed) to the
+                              next one. 0 in the last one                    */
+  } ElfW(Verneed);
+
+  ElfW(Vernaux):
+
+  typedef struct {
+    ElfW(Word) vna_hash;  /* Dependency name hash (per ELF hash function)    */
+    ElfW(Half) vna_flags; /* Dependency flag bitfield. See below             */
+    ElfW(Half) vna_other; /* Referred to by SHT_GNU_versym, but see below    */
+    ElfW(Word) vna_name;  /* Byte offset in strtab of required (symbol) name */
+    ElfW(Word) vna_next;  /* Byte offset from this ElfW(Vernaux) to the next
+                             0 for the last entry                            */
+  } ElfW(Vernaux);
+
+  vna_flags:
+    VER_FLG_WEAK 0x2  Weak version identifier:
+                      Not fatal if this symbol+version is missing.
+
+  vna_other:
+    If bit 15 (0x8000) is set then this symbol is hidden.
+    vna_other should therefore be bitwise-anded with 0x7fff before
+    comparison with the value from SHT_GNU_versym.
+
 SHT_GNU_versym             0x6fffffff
 
+  Section name: ".gnu.version"
+
+  The versioned symbol table.
+
+  If present, this must have the same number ofentries as the
+  SHT_DYNSYM/".dynsym" section. The entries in this section are in the
+  same order as those in SHT_DYNSYM.
+
+  That is to say: Entry 2 in this table corresponds to entry 2 in
+  SHT_DYNSYM, entry 3 here to entry 3 in SHT_DYNSYM, and so on.
+
+  This section contains an array of elements of type ElfW(Half).
+
+  Each entry specifies the version defined for or required by the
+  corresponding symbol in the Dynamic Symbol Table.
+
+  Entries do not give the version directly - instead they refer to the
+  corresponding ElfW(Vernaux).vna_other or ElfW(Verdef).vd_ndx structure+member.
+
+Two values are reserved:
+
+  VER_NDX_LOCAL  0 - The symbol is private, and is not available outside this object.
+  VER_NDX_GLOBAL 1 - The symbol is globally available (ie not versioned? DOCUMENTME).
+
 Note section descriptors (SHT_NOTE extensions)
 ==============================================
 
-These Note descriptor types are GNU extensions
+These SHT_NOTE descriptor types are GNU extensions
 Found in the type field of the ELF note layout.
 
+Section name: ".note" as per standard SHT_NOTE sections.
+
+Each note entry should be aligned to 4 bytes in 32-bit objects or
+8 bytes in 64-bit objects (see below for exceptions to this).
+
+Alignment: A note parser should use p_align from the program section
+header for note alignment rather than assuming alignment based on ELF
+file class.
+
 NT_GNU_ABI_TAG         1
 
   Use to indicate kernel type and minimum kernel version:
   Section must be named ".note.ABI-tag"
 
+  Alignment: Always 4-bytes, Even on 64 bit architectures.
+
   The name field (namesz/name) contains the string "GNU".
 
   The descsz field must be at least 16,
@@ -255,7 +468,18 @@ NT_GNU_HWCAP           2
 NT_GNU_BUILD_ID        3
 
   descsz bytes of build-id data.
-  Typically presented as a hex string.
+
+  Alignment: Always 4-bytes, Even on 64 bit architectures.
+
+  Typically presented as a hex string by user-facing tools.
+  Stored as binary (ie not necessarily printable, not encoded).
+
+  The build-id is desctribed as having the following properties:
+  It is unique among the set of meaningful contents for ELF files and
+  identical when the output filewould otherwise have been identical.
+
+  The computation mechanism for the build-id is not given, and is in
+  any case opaque after compile time.
 
 NT_GNU_GOLD_VERSION    4
 
@@ -269,4 +493,4 @@ NT_GNU_PROPERTY_TYPE_0 5
   4 bytes, a native integer giving the subtype.
   4 bytes, a native integer giving the size of the entry
 
-  See PT_GNU_PROPERTY and/architecture specific ABIs for details.
+  See PT_GNU_PROPERTY and/or architecture specific ABIs for details.
-- 
2.20.1